diff options
233 files changed, 7627 insertions, 3645 deletions
diff --git a/.gitignore b/.gitignore index 8dd66f605a..9497169cde 100644 --- a/.gitignore +++ b/.gitignore @@ -142,6 +142,7 @@ JAVADOC-GENERATED /make/output.mk /make/emd2exml /make/make_emakefile +/make/install_dir_data.sh # Created by "out_build update_primary" /bootstrap/primary_compiler/ @@ -317,6 +318,7 @@ JAVADOC-GENERATED /lib/jinterface/doc/html/java /lib/jinterface/pom.xml /lib/jinterface/target +/lib/jinterface/doc/src/jdoc # kernel @@ -371,7 +373,6 @@ JAVADOC-GENERATED /system/doc/html /system/doc/xml /system/doc/top/PR.template -/system/doc/top/erlresolvelinks.js # test_server diff --git a/HOWTO/INSTALL-RASPBERRYPI3.md b/HOWTO/INSTALL-RASPBERRYPI3.md index 536d095cb4..b9cffbe0c5 100644 --- a/HOWTO/INSTALL-RASPBERRYPI3.md +++ b/HOWTO/INSTALL-RASPBERRYPI3.md @@ -4,7 +4,7 @@ ## Introduction This document describes how to build a toolchain and cross compile Erlang/OTP -to Raspberry Pi 3 on macOS High Sierra. It is recommended to consult +to Raspberry Pi 3 on macOS Mojave. It is recommended to consult [Building and Installing Erlang/OTP](https://github.com/erlang/otp/blob/master/HOWTO/INSTALL.md) and [Cross Compiling Erlang/OTP](https://github.com/erlang/otp/blob/master/HOWTO/INSTALL-CROSS.md) before attempting to follow the instructions in this guide. The whole process takes several hours and depending on the package versions different problems may arise that require additional @@ -18,9 +18,15 @@ toolchain and sysroot. #### Tested Configuration -macOS High Sierra 10.13.2<br> +macOS Mojave 10.14.3<br> Raspberry Pi Model B Rev 1.2<br> -Crosstools-NG 1.23.0_1 +Crosstools-NG 1.23.0_3 + +``` +build = x86_64-apple-darwin18.2.0 +host = x86_64-apple-darwin18.2.0 +target = armv8-rpi3-linux-gnueabihf +``` > Note: /proc/device/tree/model contains model information of your > Raspberry Pi. @@ -34,10 +40,13 @@ Crosstools-NG 1.23.0_1 $ brew install grep --default-names # needed by crosstools-ng scripts $ brew install md5sha1sum # needed by crosstools-ng populate script + (2) + + $ chmod 744 /usr/local/Cellar/crosstool-ng/1.23.0_3/lib/crosstool-ng-1.23.0/scripts/crosstool-NG.sh #### Create case-sensitive disk images - (2) + (3) Create two case-sensitive disk images using Disk Utility: @@ -47,47 +56,106 @@ Format: `Mac OS Extended (Case-sensitive, Journaled)` ``` /Volumes/xtools-build-env 15 GB -/Volumes/xtools 500 MB +/Volumes/xtools 500 MB ``` -The first image holds all source and object files while building the toolchain. The second image houses the compiled +> The first image holds all source and object files while building the toolchain. The second image houses the compiled toolchain. -## Building the Toolchain +## Building the Toolchain -#### Configure crosstool-ng +### Environment settings (4) + $ ulimit -n 1024 + +### Inspect target system + + (5) + + $ uname -a + Linux raspberrypi 4.9.35-v7+ #1014 SMP Fri Jun 30 14:47:43 BST 2017 armv7l GNU/Linux + $ ld -v + GNU ld (GNU Binutils for Raspbian) 2.25 + $ ldd --version + ldd (Debian GLIBC 2.19-18+deb8u10) 2.19 + Copyright (C) 2014 Free Software Foundation, Inc. + This is free software; see the source for copying conditions. There is NO + warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. + Written by Roland McGrath and Ulrich Drepper. + +> Use the versions available on your target system! + +> See https://wiki.osdev.org/Cross-Compiler_Successful_Builds + +### Configure crosstool-ng + + (6) + + $ cd /Volumes/xtools-build-env $ ct-ng armv8-rpi3-linux-gnueabihf $ ct-ng menuconfig -#### Modify *path* section +#### Path and misc options + +##### crosstool-NG behavior + + (7) + +``` +[*] Use obsolete features + +[*] Debug crosstool-NG +[ ] Pause between every steps +[*] Save intermediate steps +[*] gzip saved states +[*] Interactive shell on failed commands +``` + +> Should the build break at a particular build step, you can fix the problem and continue the build from where it broke. + +##### Paths + + (8) * Local tarballs directory: `/Volumes/xtools-build-env/src` * Working directory: `/Volumes/xtools-build-env/.build` * Prefix directory: `/Volumes/xtools/${CT_TARGET}` -#### Modify *Extracting* section +##### Extracting + + (9) + +`[*] Stop after extracting tarballs` -* Check option: _Stop after extracting tarballs_. +> Stop the build process right after the tarballs have been extracted. This can be handy to fix known source code problems before the actual build process is started. -> Note: The build shall stop after the tarballs have been extracted to give us time to fix source code problems. +#### Operating System -#### Enable STOP / RESTART + (10) -Edit /Volumes/xtools-build-env/.config - `CT_DEBUG_CT_SAVE_STEPS=y` +`Linux kernel version (4.9.20)` -Should the build break at a particular build step, you can fix the problem and continue the build from where it broke. +#### Binary utilities -Short summary of the most common `ct-ng` commands: + (11) -* Listing all build steps +`bintutils version (2.28)` + +#### C-library + + (12) + +`glibc version (2.19 (OBSOLETE))` + +#### Sample `ct-ng` commands: + +* List all build steps ``` - $ ct-ng list-steps + $ ct-ng list-steps Available build steps, in order: - companion_tools_for_build @@ -108,216 +176,153 @@ Short summary of the most common `ct-ng` commands: - binutils_for_target - debug - test_suite - - finish + - finish ``` -* Re-run step -``` - $ ct-ng step -``` +* Re-run step `companion_libs_for_host` -* Restart from step ``` - $ ct-ng step+ + $ ct-ng companion_libs_for_host ``` -* Run until step +* Restart from `companion_libs_for_host` + ``` - $ ct-ng +step + $ ct-ng companion_libs_for_host+ ``` -#### Fix file permissions on crosstool-NG.sh - - (5) - - $ chmod 744 /usr/local/Cellar/crosstool-ng/1.23.0_1/lib/crosstool-ng-1.23.0/scripts/crosstool-NG.sh +* Run until step `companion_libs_for_host` -#### Run build command +``` + $ ct-ng +companion_libs_for_host +``` -Build process stops just after the tarballs have been extracted. +### Build - (6) + (13) $ ct-ng build - Retrieving needed toolchain components' tarballs - [EXTRA] Retrieving 'make-4.2.1' - [EXTRA] Retrieving 'm4-1.4.18' - [EXTRA] Retrieving 'linux-4.10.8' - [EXTRA] Retrieving 'gmp-6.1.2' - [EXTRA] Retrieving 'mpfr-3.1.5' - [EXTRA] Retrieving 'isl-0.16.1' - [EXTRA] Retrieving 'mpc-1.0.3' - [EXTRA] Retrieving 'expat-2.2.0' - [EXTRA] Retrieving 'ncurses-6.0' - [EXTRA] Retrieving 'libiconv-1.15' - [EXTRA] Retrieving 'gettext-0.19.8.1' - [EXTRA] Retrieving 'binutils-2.28' - [EXTRA] Retrieving 'gcc-6.3.0' - [EXTRA] Retrieving 'glibc-2.25' - [EXTRA] Retrieving 'gdb-7.12.1' +> Build process stops just after the tarballs have been extracted. #### Fix source files - (7) - -Add macro to /Volumes/xtools-build-env/.build/src/gdb-7.12.1/gdb/doublest.c: -```C -#define min(a,b) \ - ({ typeof (a) _a = (a); \ - typeof (b) _b = (b); \ - _a < _b ? _a : _b; }) -``` - - (8) Update ulimit + (14) - $ ulimit -n 1024 + $ pushd .build/src/gettext-0.19.8.1/ + $ autoreconf + $ popd -#### Modify *extract* section +#### Update configuration - (8) + (15) $ ct-ng menuconfig - Uncheck option: _Stop after extracting tarballs_ +Uncheck option: -#### Re-run build command +`[ ] Stop after extracting tarballs` -Restarts build process from where it previously stopped. +#### Continue build - (9) + (16) $ ct-ng build -#### Fix gettext +> Restart build process from where it previously stopped. -Build will fail at step `companion_tools_for_build` but it can be fixed by running autoreconf: + (17) - (10) + $ export PATH=/Volumes/xtools/armv8-rpi3-linux-gnueabihf/bin:$PATH - $ cd .build/src/gettext-0.19.8.1/ - $ ./autoreconf - $ ct-ng companion_tools_for_build+ +### Test -#### Test the toolchain - - (11) + (18) $ cat > test.c $ int main() { printf("Hello, world!\n"); return 0; } - $ /Volumes/xtools/arm-unknown-linux-gnueabi-gcc -o test test.c - - (12) OPTIONAL - - “Render the toolchain read-only” from crosstool-NG’s “Paths and misc options” configuration page. + <CTRL+D> + $ armv8-rpi3-linux-gnueabihf-gcc -o test test.c ## Cross compiling dependencies - (13) + (19) - $ export PATH=/Volumes/xtools/armv8-rpi3-linux-gnueabihf/bin:$PATH + $ mkdir local # prefix directory + $ mkdir 3pps # OTP dependencies + $ cd 3pps -#### Cross compiling zlib +#### zlib - (14) + (20) $ wget http://zlib.net/zlib-1.2.11.tar.gz $ tar xf zlib-1.2.11.tar.gz - $ cd zlib-1.2.11 - $ CHOST=armv8-rpi3-linux-gnueabihf ./configure --prefix=/Users/<username>/git/raspberrypi/arm + $ pushd zlib-1.2.11 + $ CHOST=armv8-rpi3-linux-gnueabihf ./configure --prefix=/Volumes/xtools-build-env/local $ make $ make install + $ popd -#### Cross compiling openssl +#### openssl - (15) + (21) - $ wget http://openssl.org/source/openssl-1.1.0g.tar.gz - $ tar xf openssl-1.1.0g.tar.gz - $ cd openssl-1.1.0g - $ ./Configure linux-generic32 --prefix=/Users/<username>/git/raspberrypi/arm --openssldir=/Users/<username>/git/raspberrypi/arm/openssl --cross-compile-prefix=armv8-rpi3-linux-gnueabihf + $ wget http://openssl.org/source/openssl-1.1.1b.tar.gz + $ tar xf openssl-1.1.1b.tar.gz + $ pushd openssl-1.1.1b + $ ./Configure linux-generic32 --prefix=/Volumes/xtools-build-env/local \ + --openssldir=/Volumes/xtools-build-env/local/openssl \ + --cross-compile-prefix=armv8-rpi3-linux-gnueabihf- $ make $ make install + $ popd -#### Cross compiling ncurses - - (16) - - $ wget http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.9.tar.gz +> A compatible openssl library shall be available on the target system! +#### ncurses - (17) - -Apply patch: - -```patch ---- a/ncurses/base/MKlib_gen.sh -+++ b/ncurses/base/MKlib_gen.sh -@@ -474,11 +474,22 @@ sed -n -f $ED1 \ - -e 's/gen_$//' \ - -e 's/ / /g' >>$TMP - -+cat >$ED1 <<EOF -+s/ / /g -+s/^ // -+s/ $// -+s/P_NCURSES_BOOL/NCURSES_BOOL/g -+EOF -+ -+# A patch discussed here: -+# https://gcc.gnu.org/ml/gcc-patches/2014-06/msg02185.html -+# introduces spurious #line markers. Work around that by ignoring the system's -+# attempt to define "bool" and using our own symbol here. -+sed -e 's/bool/P_NCURSES_BOOL/g' $TMP > $ED2 -+cat $ED2 >$TMP -+ - $preprocessor $TMP 2>/dev/null \ --| sed \ -- -e 's/ / /g' \ -- -e 's/^ //' \ -- -e 's/_Bool/NCURSES_BOOL/g' \ -+| sed -f $ED1 \ - | $AWK -f $AW2 \ - | sed -f $ED3 \ - | sed \ -``` + (22) - (18) - - $ ./configure --build=x86_64-apple-darwin17.3.0 --host=armv8-rpi3-linux-gnueabihf --without-ada --without-cxx --without-cxx-binding --without-manpages --without-progs --without-tests --prefix=/usr --libdir=/lib --with-build-cc="gcc -D_GNU_SOURCE" --with-shared + $ wget http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.9.tar.gz + $ tar xf ncurses-5.9.tar.gz + $ pushd ncurses-5.9 + $ wget https://gist.githubusercontent.com/peterdmv/1068b2f9e1fec6e1330ad62ed87461ad/\ + raw/065597b63654ed6a9f28d02fdfbca844413847ad/ncurses-5.9.patch + $ patch -p0 < ncurses-5.9.patch + $ ./configure --build=x86_64-apple-darwin18.2.0 --host=armv8-rpi3-linux-gnueabihf \ + --without-ada --without-cxx --without-cxx-binding --without-manpages \ + --without-progs --without-tests --prefix=/usr --libdir=/lib \ + --with-build-cc="gcc -D_GNU_SOURCE" --with-shared $ make - $ make DESTDIR=/Users/<username>/git/raspberrypi/arm install - - (19) - -Compile ncurses test program: - - $ cd test - $ armv8-rpi3-linux-gnueabihf-gcc -o nctest ncurses.c -I${RPI_SYSROOT}/usr/include -L${RPI_SYSROOT}/lib -lncursesw + $ make DESTDIR=/Volumes/xtools-build-env/local install + $ popd ## Populating sysroot - (19) - - Edit /Volumes/xtools/armv8-rpi3-linux-gnueabihf/bin/armv8-rpi3-linux-gnueabihf-populate: + (23) - sed="gsed" + $ chmod 755 /Volumes/xtools/armv8-rpi3-linux-gnueabihf/bin + $ chmod 755 /Volumes/xtools/armv8-rpi3-linux-gnueabihf/bin/armv8-rpi3-linux-gnueabihf-populate + $ gsed -i 's/"sed"/"gsed"/g' \ + /Volumes/xtools/armv8-rpi3-linux-gnueabihf/bin/armv8-rpi3-linux-gnueabihf-populate + $ chmod 555 /Volumes/xtools/armv8-rpi3-linux-gnueabihf/bin + $ chmod 555 /Volumes/xtools/armv8-rpi3-linux-gnueabihf/bin/armv8-rpi3-linux-gnueabihf-populate - (20) + (24) - $ armv8-rpi3-linux-gnueabihf-populate -s /Users/<username>/git/raspberrypi/arm -d /Users/<username>/git/raspberrypi/sysroot - $ export RPI_SYSROOT=/Users/<username>/git/raspberrypi/sysroot + $ armv8-rpi3-linux-gnueabihf-populate -s /Volumes/xtools-build-env/local \ + -d /Volumes/xtools-build-env/sysroot + $ export RPI_SYSROOT=/Volumes/xtools-build-env/sysroot ## Cross compiling Erlang/OTP - (21) + (25) $ LC_CTYPE=C && LANG=C && ./otp_build autoconf - $ ./otp_build configure --disable-dynamic-ssl-lib --xcomp-conf=./xcomp/erl-xcomp-armv8-rpi3-linux-gnueabihf.conf + $ ./otp_build configure --xcomp-conf=./xcomp/erl-xcomp-armv8-rpi3-linux-gnueabihf.conf $ ./otp_build boot -a - $ ./otp_build release -a /Users/<username>/git/raspberrypi/erlang - $ tar czf erlang.tgz ./erlang - + $ ./otp_build release -a /Volumes/xtools-build-env/otp_22.0 diff --git a/Makefile.in b/Makefile.in index 88b3670999..49a0d9f8af 100644 --- a/Makefile.in +++ b/Makefile.in @@ -434,7 +434,7 @@ endif PATH=$(BOOT_PREFIX)"$${PATH}" \ ERL_TOP=$(ERL_TOP) $(MAKE) TESTROOT="$(RELEASE_ROOT)" DOCGEN=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen $@ ifneq ($(OTP_SMALL_BUILD),true) - echo "OTP doc built" > $(ERL_TOP)/make/otp_doc_built + test -f $(ERL_TOP)/make/otp_doc_built || echo "OTP doc built" > $(ERL_TOP)/make/otp_doc_built endif xmllint: docs @@ -450,7 +450,9 @@ else $(MAKE) -C system/doc $@ endif -mod2app: +mod2app: $(ERL_TOP)/make/$(TARGET)/mod2app.xml + +$(ERL_TOP)/make/$(TARGET)/mod2app.xml: erts/doc/src/Makefile lib/*/doc/src/Makefile PATH=$(BOOT_PREFIX)"$${PATH}" escript $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen/priv/bin/xref_mod_app.escript -topdir $(ERL_TOP) -outfile $(ERL_TOP)/make/$(TARGET)/mod2app.xml # ---------------------------------------------------------------------- @@ -488,7 +490,7 @@ else $(make_verbose)cd lib && \ ERL_TOP=$(ERL_TOP) PATH=$(BOOT_PREFIX)"$${PATH}" \ $(MAKE) opt BUILD_ALL=true - $(V_at)echo "OTP built" > $(ERL_TOP)/make/otp_built + $(V_at)test -f $(ERL_TOP)/make/otp_built || echo "OTP built" > $(ERL_TOP)/make/otp_built endif kernel: $(make_verbose)cd lib/kernel && \ @@ -767,7 +769,7 @@ tertiary_bootstrap_copy: true; \ done # copy erl_interface includes - $(V_at)for x in lib/erl_interface/include/* lib/erl_interface/include/$(TARGET)/*; do \ + $(V_at)for x in lib/erl_interface/include/*.h; do \ BN=`basename $$x`; \ TF=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_interface/include/$$BN; \ test -f $$TF && \ @@ -4,6 +4,8 @@ **OTP** is a set of Erlang libraries, which consists of the Erlang runtime system, a number of ready-to-use components mainly written in Erlang, and a set of design principles for Erlang programs. [Learn more about Erlang and OTP](http://erlang.org/doc/system_architecture_intro/sys_arch_intro.html). +[Release notes](http://erlang.org/download/otp_versions_tree.html) for all OTP versions. + [Learn how to program in Erlang](http://learnyousomeerlang.com/content). ## Examples diff --git a/configure.src b/configure.src index 889902eb96..4b748f2545 100644 --- a/configure.src +++ b/configure.src @@ -70,8 +70,10 @@ while test $# != 0; do ERL_TOP="$user_srcdir" ;; --enable-bootstrap-only) + config_arguments="$config_arguments --enable-bootstrap-only" bootstrap_only=yes;; --disable-bootstrap-only) + config_arguments="$config_arguments --disable-bootstrap-only" bootstrap_only=no;; --enable-option-checking) echo "ERROR: Cannot enable option checking" 1>&2 diff --git a/erts/autoconf/vxworks/sed.general b/erts/autoconf/vxworks/sed.general index 0e99b4dba4..d32fbdc5c0 100644 --- a/erts/autoconf/vxworks/sed.general +++ b/erts/autoconf/vxworks/sed.general @@ -103,7 +103,6 @@ s|@INSTALL_PROGRAM@|${INSTALL}| s|@INSTALL_SCRIPT@|${INSTALL}| s|@INSTALL_DATA@|${INSTALL} -m 644| s|@INSTALL_DIR@|$(INSTALL) -d| -s|@RM@|/bin/rm| s|@MKDIR@|/bin/mkdir| s|@ERLANG_OSTYPE@|vxworks| s|@vxworks_reclaim@|reclaim.h| diff --git a/erts/configure.in b/erts/configure.in index a9bd1b117a..b070ad0649 100644 --- a/erts/configure.in +++ b/erts/configure.in @@ -772,11 +772,6 @@ AC_SUBST(LIBCARBON) _search_path=/bin:/usr/bin:/usr/local/bin:$PATH -AC_PATH_PROG(RM, rm, false, $_search_path) -if test "$ac_cv_path_RM" = false; then - AC_MSG_ERROR([No 'rm' command found]) -fi - AC_PATH_PROG(MKDIR, mkdir, false, $_search_path) if test "$ac_cv_path_MKDIR" = false; then AC_MSG_ERROR([No 'mkdir' command found]) @@ -791,9 +786,9 @@ _search_path= # Remove old configuration information. -# Next line should be placed after AC_PATH_PROG(RM, ...), but before -# first output to CONN_INFO. So this is just the right place. -$RM -f "$ERL_TOP/erts/CONF_INFO" +# Next line should be before first output to CONN_INFO. So this is +# just the right place. +rm -f "$ERL_TOP/erts/CONF_INFO" dnl Check if we should/can build a sharing-preserving emulator AC_MSG_CHECKING(if we are building a sharing-preserving emulator) @@ -831,7 +826,7 @@ fi ## Delete previous failed configure results if test -f doc/CONF_INFO; then - $RM doc/CONF_INFO + rm -f doc/CONF_INFO fi AC_CHECK_PROGS(XSLTPROC, xsltproc) @@ -3085,14 +3080,14 @@ if test "$enable_dtrace_test" = "yes" ; then AC_MSG_CHECKING([for 2-stage DTrace precompilation]) AC_TRY_COMPILE([ #include "foo-dtrace.h" ], [ERLANG_DIST_PORT_BUSY_ENABLED();], - [$RM -f $DTRACE_2STEP_TEST + [rm -f $DTRACE_2STEP_TEST dtrace -G $DTRACE_CPP $DTRACE_BITS_FLAG -Iemulator/beam -o $DTRACE_2STEP_TEST -s emulator/beam/erlang_dtrace.d conftest.$OBJEXT 2>&AS_MESSAGE_LOG_FD if test -f $DTRACE_2STEP_TEST; then - $RM $DTRACE_2STEP_TEST + rm -f $DTRACE_2STEP_TEST DTRACE_ENABLED_2STEP=yes fi], []) - $RM -f foo-dtrace.h + rm -f foo-dtrace.h AS_IF([test "x$DTRACE_ENABLED_2STEP" = "xyes"], [AC_MSG_RESULT([yes])], [AC_MSG_RESULT([no])]) @@ -3190,7 +3185,7 @@ need_java="jinterface ic/java_src" # Remove all SKIP files from previous runs for a in $need_java ; do - $RM -f $ERL_TOP/lib/$a/SKIP + rm -f $ERL_TOP/lib/$a/SKIP done if test "X$with_javac" = "Xno"; then @@ -3241,7 +3236,7 @@ dnl this deliberately does not believe that 'gcc' is a C++ compiler AC_CHECK_TOOLS(CXX, [$CCC c++ g++ CC cxx cc++ cl], false) # Remove SKIP file from previous run -$RM -f $ERL_TOP/lib/orber/SKIP +rm -f $ERL_TOP/lib/orber/SKIP if test "$CXX" = false; then echo "No C++ compiler found" > $ERL_TOP/lib/orber/SKIP @@ -3397,11 +3392,12 @@ AC_CONFIG_FILES([../make/make_emakefile:../make/make_emakefile.in], dnl dnl The ones below should be moved to their respective lib dnl -dnl ../lib/ssl/c_src/$host/Makefile:../lib/ssl/c_src/Makefile.in AC_CONFIG_FILES([ ../lib/os_mon/c_src/$host/Makefile:../lib/os_mon/c_src/Makefile.in ../lib/runtime_tools/c_src/$host/Makefile:../lib/runtime_tools/c_src/Makefile.in ../lib/tools/c_src/$host/Makefile:../lib/tools/c_src/Makefile.in ]) +AC_CONFIG_FILES([../make/install_dir_data.sh:../make/install_dir_data.sh.in], [chmod +x ../make/install_dir_data.sh]) + AC_OUTPUT diff --git a/erts/doc/src/alt_dist.xml b/erts/doc/src/alt_dist.xml index e6245130fc..7c997cae20 100644 --- a/erts/doc/src/alt_dist.xml +++ b/erts/doc/src/alt_dist.xml @@ -60,7 +60,7 @@ parts of the logic in Erlang code, and you perhaps do not even need a new driver for the protocol. One example could be Erlang distribution over UDP using <c>gen_udp</c> (your - Erlang code will of course have to take care of retranspissions, + Erlang code will of course have to take care of retransmissions, etc in this example). That is, depending on what you want to do you perhaps do not need to implement a driver at all and can then skip the driver related sections below. diff --git a/erts/doc/src/erl_ext_dist.xml b/erts/doc/src/erl_ext_dist.xml index ca55c6233c..4721747097 100644 --- a/erts/doc/src/erl_ext_dist.xml +++ b/erts/doc/src/erl_ext_dist.xml @@ -367,8 +367,9 @@ </row> <tcaption>FLOAT_EXT</tcaption></table> <p> - A float is stored in string format. The format used in sprintf to - format the float is "%.20e" + A finite float (i.e. not inf, -inf or NaN) is stored in + string format. The format used in sprintf to format the + float is "%.20e" (there are more bytes allocated than necessary). To unpack the float, use sscanf with format "%lf". </p> @@ -983,7 +984,8 @@ </row> <tcaption>NEW_FLOAT_EXT</tcaption></table> <p> - A float is stored as 8 bytes in big-endian IEEE format. + A finite float (i.e. not inf, -inf or NaN) is stored as 8 bytes + in big-endian IEEE format. </p> <p> This term is used in minor version 1 of the external format. diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml index cc7452bab5..cf1994887a 100644 --- a/erts/doc/src/erl_nif.xml +++ b/erts/doc/src/erl_nif.xml @@ -3369,6 +3369,48 @@ if (retval & ERL_NIF_SELECT_STOP_CALLED) { </func> <func> + <name since="OTP @OTP-15640@"><ret>ErlNifTermType</ret> + <nametext>enif_term_type(ErlNifEnv *env, ERL_NIF_TERM term)</nametext> + </name> + <fsummary>Determine the type of a term.</fsummary> + <desc> + <p>Determines the type of the given term. The term must be an ordinary + Erlang term and not one of the special terms returned by + <seealso marker="#enif_raise_exception"> + <c>enif_raise_exception</c></seealso>, + <seealso marker="#enif_schedule_nif"> + <c>enif_schedule_nif</c></seealso>, or similar.</p> + <p>The following types are defined at the moment:</p> + <taglist> + <tag><c>ERL_NIF_TERM_TYPE_ATOM</c></tag> + <item/> + <tag><c>ERL_NIF_TERM_TYPE_BITSTRING</c></tag> + <item><p>A bitstring or binary</p></item> + <tag><c>ERL_NIF_TERM_TYPE_FLOAT</c></tag> + <item/> + <tag><c>ERL_NIF_TERM_TYPE_FUN</c></tag> + <item/> + <tag><c>ERL_NIF_TERM_TYPE_INTEGER</c></tag> + <item/> + <tag><c>ERL_NIF_TERM_TYPE_LIST</c></tag> + <item><p>A list, empty or not</p></item> + <tag><c>ERL_NIF_TERM_TYPE_MAP</c></tag> + <item/> + <tag><c>ERL_NIF_TERM_TYPE_PID</c></tag> + <item/> + <tag><c>ERL_NIF_TERM_TYPE_PORT</c></tag> + <item/> + <tag><c>ERL_NIF_TERM_TYPE_REFERENCE</c></tag> + <item/> + <tag><c>ERL_NIF_TERM_TYPE_TUPLE</c></tag> + <item/> + </taglist> + <p>Note that new types may be added in the future, so the caller must + be prepared to handle unknown types.</p> + </desc> + </func> + + <func> <name since="OTP R13B04"><ret>int</ret> <nametext>enif_thread_create(char *name,ErlNifTid *tid,void * (*func)(void *),void *args,ErlNifThreadOpts diff --git a/erts/doc/src/net.xml b/erts/doc/src/net.xml index bd85594c98..b9e2cffce9 100644 --- a/erts/doc/src/net.xml +++ b/erts/doc/src/net.xml @@ -33,6 +33,9 @@ <modulesummary>Network interface.</modulesummary> <description> <p>This module provides an API for the network interface.</p> + <note> + <p>There is currently <em>no</em> support for Windows. </p> + </note> </description> <datatypes> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index 3473a12526..bdae994d06 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -31,6 +31,222 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 10.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>When multiplying a number by itself, a word beyond the + number on the heap could be read (and ignored). This bug + was extremely unlikely to actually cause a real + problem.</p> + <p> + Own Id: OTP-15484</p> + </item> + <item> + <p> + Fix bug where doing <c>seq_trace:reset_trace()</c> while + another process was doing a garbage collection could + cause the run-time system to segfault.</p> + <p> + Own Id: OTP-15490</p> + </item> + <item> + <p> + Fix reading of ancillary data from packet oriented + sockets on old Linux kernel versions. Without this fix, + getting the data would cause the port to enter an + infinite loop.</p> + <p> + Own Id: OTP-15494</p> + </item> + <item> + <p> + Fix bug where crash dumping or doing + <c>erlang:system_info(procs)</c> while another process + was doing a garbage collection could cause the run-time + system to segfault.</p> + <p> + Own Id: OTP-15527</p> + </item> + <item> + <p> + Fix <c>erlang:system_info(kernel_poll)</c> to return + correct value. Before this fix, the call always returned + <c>false</c>.</p> + <p> + Own Id: OTP-15556</p> + </item> + <item> + <p> + Fix bug in <c>enif_make_map_from_arrays</c> that would + produce broken maps when number of keys were 32. Bug + exists since OTP 21.0.</p> + <p> + Own Id: OTP-15567</p> + </item> + <item> + <p> + Fix a bug in <c>binary:encode_unsigned</c> that may cause + a read of uninitialized memory.</p> + <p> + The bug existed since the function was added (OTP + R16B02).</p> + <p> + Own Id: OTP-15583 Aux Id: PR-2118 </p> + </item> + <item> + <p> + Fixed a bug that could cause <c>heart</c> to kill an + exiting node before it had time to flush all buffered + writes. If environment variable + <c>HEART_KILL_SIGNAL=SIGABRT</c> was set a superfluous + core dump could also be generated.</p> + <p> + Own Id: OTP-15599 Aux Id: ERIERL-298 </p> + </item> + <item> + <p> + Fix <c>enif_consume_timeslice</c> to be a no-op on dirty + scheduler and not crash debug compiled emulator.</p> + <p> + Own Id: OTP-15604</p> + </item> + <item> + <p> + Fixed macro redefinition warnings.</p> + <p> + Own Id: OTP-15629</p> + </item> + <item> + <p> + <c>to_erl</c> fixed to not garble terminal input beyond + 7-bit ASCII.</p> + <p> + Own Id: OTP-15650 Aux Id: ERL-854, PR-2161 </p> + </item> + <item> + <p> + Minor fixes for <c>make clean</c>.</p> + <p> + Own Id: OTP-15657</p> + </item> + <item> + <p> + Fixed a bug in all <c>ets:select*</c> and + <c>ets:match*</c> functions that could in some rare cases + lead to very poor performance.</p> + <p> + Own Id: OTP-15660 Aux Id: ERL-869 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add <c>erlang:system_flag(system_logger, Pid)</c> and + <c>erlang:system_info(system_logger)</c>. This + system_flag can be used to set the process that will + receive the logging messages generated by ERTS.</p> + <p> + Own Id: OTP-15375</p> + </item> + <item> + <p><c>integer_to_list/2</c> and + <c>integer_to_binary/2</c> are now implemented in C, + improving their performance.</p> + <p> + Own Id: OTP-15503 Aux Id: PR-2052 </p> + </item> + <item> + <p> + Improved <c>term_to_binary</c> to do more fair reduction + count and yielding when encoding large byte lists + (strings).</p> + <p> + Own Id: OTP-15514 Aux Id: ERL-774 </p> + </item> + <item> + <p> + Made internal port drivers more robust against + <c>erlang:port_control</c> with invalid arguments and + added documentation warnings about such abuse.</p> + <p> + Own Id: OTP-15555 Aux Id: ERIERL-231 </p> + </item> + <item> + <p> + Fix bug on NetBSD where the <c>exit_status</c> from a + port program would never be sent.</p> + <p> + Own Id: OTP-15558 Aux Id: ERL-725 </p> + </item> + <item> + <p>There is a new function <c>persistent:term(Key, + Default)</c> to allow specifying a default when looking + up a persistent term.</p> + <p> + Own Id: OTP-15576 Aux Id: ERL-843 </p> + </item> + <item> + <p>A transitory emulator option '<c>+ztma true</c>' has + been added to allow running existing BEAM code that + relies on "tuple calls" (dispatch on parameterized + modules) which has been compiled under OTP 20 or earlier. + This option will be removed in OTP 22, so such modules + should eventually be recompiled with the + <c>+tuple_calls</c> option.</p> + <p> + Own Id: OTP-15580 Aux Id: PR-2113 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.2.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixes of install/release phase in build system.</p> + <list> <item>The source tree was modified when + installing/releasing and/or applying a patch.</item> + <item>Some files were installed with wrong access + rights.</item> <item>If applying a patch (using + <c>otp_patch_apply</c>) as another user (except root) + than the user that built the source, the documentation + was not properly updated.</item> </list> + <p> + Own Id: OTP-15551</p> + </item> + <item> + <p> + Setting the <c>recbuf</c> size of an inet socket the + <c>buffer</c> is also automatically increased. Fix a bug + where the auto adjustment of inet buffer size would be + triggered even if an explicit inet buffer size had + already been set.</p> + <p> + Own Id: OTP-15651 Aux Id: ERIERL-304 </p> + </item> + <item> + <p> + Reading from UDP using active <c>true</c> or active + <c>N</c> mode has been optimized when more packets than + specified by <c>read_packets</c> are available on the + socket.</p> + <p> + Own Id: OTP-15652 Aux Id: ERIERL-304 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 10.2.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/doc/src/socket.xml b/erts/doc/src/socket.xml index caf7058b34..f6195a65b2 100644 --- a/erts/doc/src/socket.xml +++ b/erts/doc/src/socket.xml @@ -40,6 +40,11 @@ the functions, e.g. <seealso marker="#recv/3"><c>recv/3</c></seealso>, has a timeout argument. </p> + <note> + <p>There is currently <em>no</em> support for Windows. </p> + <p>Support for IPv6 has been implemented but <em>not</em> tested. </p> + <p>SCTP has only been partly implemented (and not tested). </p> + </note> </description> <datatypes> diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in index 3e9cfc990a..21351df656 100644 --- a/erts/emulator/Makefile.in +++ b/erts/emulator/Makefile.in @@ -267,7 +267,6 @@ DEXPORT = @DEXPORT@ RANLIB = @RANLIB@ STRIP = strip PERL = @PERL@ -RM = @RM@ MKDIR = @MKDIR@ USING_MINGW=@MIXED_CYGWIN_MINGW@ @@ -468,13 +467,13 @@ $(ERTS_LIB): .PHONY: clean clean: - $(RM) -f $(GENERATE) - $(RM) -rf $(TARGET)/*.c $(TARGET)/*.h $(TARGET)/*-GENERATED - $(RM) -rf $(TARGET)/*/* - $(RM) -rf obj/$(TARGET) - $(RM) -rf pcre/obj/$(TARGET) $(PCRE_GENINC) - $(RM) -rf zlib/obj/$(TARGET) - $(RM) -rf bin/$(TARGET) + $(RM) $(GENERATE) + $(RM) -r $(TARGET)/*.c $(TARGET)/*.h $(TARGET)/*-GENERATED + $(RM) -r $(TARGET)/*/* + $(RM) -r obj/$(TARGET) + $(RM) -r pcre/obj/$(TARGET) $(PCRE_GENINC) + $(RM) -r zlib/obj/$(TARGET) + $(RM) -r bin/$(TARGET) cd $(ERTS_LIB_DIR) && $(MAKE) clean .PHONY: docs diff --git a/erts/emulator/beam/arith_instrs.tab b/erts/emulator/beam/arith_instrs.tab index 574fceec5b..5f23b2c168 100644 --- a/erts/emulator/beam/arith_instrs.tab +++ b/erts/emulator/beam/arith_instrs.tab @@ -116,6 +116,17 @@ increment.execute(IncrementVal, Dst) { i_times(Fail, Op1, Op2, Dst) { Eterm op1 = $Op1; Eterm op2 = $Op2; +#ifdef HAVE_OVERFLOW_CHECK_BUILTINS + if (ERTS_LIKELY(is_both_small(op1, op2))) { + Sint a = signed_val(op1); + Sint b = signed_val(op2); + Sint res; + if (ERTS_LIKELY(!__builtin_mul_overflow(a, b, &res) && IS_SSMALL(res))) { + $Dst = make_small(res); + $NEXT0(); + } + } +#endif $OUTLINED_ARITH_2($Fail, mixed_times, BIF_stimes_2, op1, op2, $Dst); } diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index 90162a6543..04a2a83123 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -322,19 +322,19 @@ void** beam_ops; #define Arg(N) I[(N)+1] -#define GetR(pos, tr) \ +#define GetSource(raw, dst) \ do { \ - tr = Arg(pos); \ - switch (loader_tag(tr)) { \ + dst = raw; \ + switch (loader_tag(dst)) { \ case LOADER_X_REG: \ - tr = x(loader_x_reg_index(tr)); \ + dst = x(loader_x_reg_index(dst)); \ break; \ case LOADER_Y_REG: \ - ASSERT(loader_y_reg_index(tr) >= 1); \ - tr = y(loader_y_reg_index(tr)); \ + ASSERT(loader_y_reg_index(dst) >= 1); \ + dst = y(loader_y_reg_index(dst)); \ break; \ } \ - CHECK_TERM(tr); \ + CHECK_TERM(dst); \ } while (0) #define PUT_TERM_REG(term, desc) \ diff --git a/erts/emulator/beam/beam_load.c b/erts/emulator/beam/beam_load.c index f4eeb54a1b..0ad5329b2f 100644 --- a/erts/emulator/beam/beam_load.c +++ b/erts/emulator/beam/beam_load.c @@ -1425,7 +1425,7 @@ load_atom_table(LoaderState* stp, ErtsAtomEncoding enc) ap = atom_tab(atom_val(stp->atom[1])); sys_memcpy(sbuf, ap->name, ap->len); sbuf[ap->len] = '\0'; - LoadError1(stp, "module name in object code is %s", sbuf); + LoadError1(stp, "BEAM file exists but it defines a module named %s", sbuf); } return 1; @@ -3295,11 +3295,11 @@ gen_get_integer2(LoaderState* stp, GenOpArg Fail, GenOpArg Ms, GenOpArg Live, } else { op->op = genop_i_bs_get_integer_6; op->arity = 6; - op->a[0] = Fail; - op->a[1] = Live; - op->a[2].type = TAG_u; - op->a[2].val = (Unit.val << 3) | Flags.val; - op->a[3] = Ms; + op->a[0] = Ms; + op->a[1] = Fail; + op->a[2] = Live; + op->a[3].type = TAG_u; + op->a[3].val = (Unit.val << 3) | Flags.val; op->a[4] = Size; op->a[5] = Dst; op->next = NULL; @@ -3332,8 +3332,8 @@ gen_get_binary2(LoaderState* stp, GenOpArg Fail, GenOpArg Ms, GenOpArg Live, } else { op->op = genop_i_bs_get_binary_all2_5; op->arity = 5; - op->a[0] = Fail; - op->a[1] = Ms; + op->a[0] = Ms; + op->a[1] = Fail; op->a[2] = Live; op->a[3] = Unit; op->a[4] = Dst; @@ -3341,8 +3341,8 @@ gen_get_binary2(LoaderState* stp, GenOpArg Fail, GenOpArg Ms, GenOpArg Live, } else if (Size.type == TAG_i) { op->op = genop_i_bs_get_binary_imm2_6; op->arity = 6; - op->a[0] = Fail; - op->a[1] = Ms; + op->a[0] = Ms; + op->a[1] = Fail; op->a[2] = Live; op->a[3].type = TAG_u; if (!safe_mul(Size.val, Unit.val, &op->a[3].val)) { @@ -3362,8 +3362,8 @@ gen_get_binary2(LoaderState* stp, GenOpArg Fail, GenOpArg Ms, GenOpArg Live, } else { op->op = genop_i_bs_get_binary_imm2_6; op->arity = 6; - op->a[0] = Fail; - op->a[1] = Ms; + op->a[0] = Ms; + op->a[1] = Fail; op->a[2] = Live; op->a[3].type = TAG_u; if (!safe_mul(bigval, Unit.val, &op->a[3].val)) { @@ -3375,8 +3375,8 @@ gen_get_binary2(LoaderState* stp, GenOpArg Fail, GenOpArg Ms, GenOpArg Live, } else { op->op = genop_i_bs_get_binary2_6; op->arity = 6; - op->a[0] = Fail; - op->a[1] = Ms; + op->a[0] = Ms; + op->a[1] = Fail; op->a[2] = Live; op->a[3] = Size; op->a[4].type = TAG_u; @@ -3409,8 +3409,8 @@ gen_put_binary(LoaderState* stp, GenOpArg Fail,GenOpArg Size, if (Size.type == TAG_a && Size.val == am_all) { op->op = genop_i_new_bs_put_binary_all_3; op->arity = 3; - op->a[0] = Fail; - op->a[1] = Src; + op->a[0] = Src; + op->a[1] = Fail; op->a[2] = Unit; } else if (Size.type == TAG_i) { op->op = genop_i_new_bs_put_binary_imm_3; @@ -3420,10 +3420,33 @@ gen_put_binary(LoaderState* stp, GenOpArg Fail,GenOpArg Size, if (safe_mul(Size.val, Unit.val, &op->a[1].val)) { op->a[2] = Src; } else { + error: op->op = genop_badarg_1; op->arity = 1; op->a[0] = Fail; } + } else if (Size.type == TAG_q) { +#ifdef ARCH_64 + /* + * There is no way that this binary would fit in memory. + */ + goto error; +#else + Eterm big = stp->literals[Size.val].term; + Uint bigval; + Uint size; + + if (!term_to_Uint(big, &bigval) || + !safe_mul(bigval, Unit.val, &size)) { + goto error; + } + op->op = genop_i_new_bs_put_binary_imm_3; + op->arity = 3; + op->a[0] = Fail; + op->a[1].type = TAG_u; + op->a[1].val = size; + op->a[2] = Src; +#endif } else { op->op = genop_i_new_bs_put_binary_4; op->arity = 4; @@ -3448,11 +3471,8 @@ gen_put_integer(LoaderState* stp, GenOpArg Fail, GenOpArg Size, NATIVE_ENDIAN(Flags); /* Negative size must fail */ if (Size.type == TAG_i) { - op->op = genop_i_new_bs_put_integer_imm_4; - op->arity = 4; - op->a[0] = Fail; - op->a[1].type = TAG_u; - if (!safe_mul(Size.val, Unit.val, &op->a[1].val)) { + Uint size; + if (!safe_mul(Size.val, Unit.val, &size)) { error: op->op = genop_badarg_1; op->arity = 1; @@ -3460,26 +3480,31 @@ gen_put_integer(LoaderState* stp, GenOpArg Fail, GenOpArg Size, op->next = NULL; return op; } - op->a[1].val = Size.val * Unit.val; - op->a[2].type = Flags.type; - op->a[2].val = (Flags.val & 7); - op->a[3] = Src; + op->op = genop_i_new_bs_put_integer_imm_4; + op->arity = 4; + op->a[0] = Src; + op->a[1] = Fail; + op->a[2].type = TAG_u; + op->a[2].val = size; + op->a[3].type = Flags.type; + op->a[3].val = (Flags.val & 7); } else if (Size.type == TAG_q) { Eterm big = stp->literals[Size.val].term; Uint bigval; + Uint size; - if (!term_to_Uint(big, &bigval)) { + if (!term_to_Uint(big, &bigval) || + !safe_mul(bigval, Unit.val, &size)) { goto error; - } else { - op->op = genop_i_new_bs_put_integer_imm_4; - op->arity = 4; - op->a[0] = Fail; - op->a[1].type = TAG_u; - op->a[1].val = bigval * Unit.val; - op->a[2].type = Flags.type; - op->a[2].val = (Flags.val & 7); - op->a[3] = Src; } + op->op = genop_i_new_bs_put_integer_imm_4; + op->arity = 4; + op->a[0] = Src; + op->a[1] = Fail; + op->a[2].type = TAG_u; + op->a[2].val = size; + op->a[3].type = Flags.type; + op->a[3].val = (Flags.val & 7); } else { op->op = genop_i_new_bs_put_integer_4; op->arity = 4; @@ -3541,8 +3566,8 @@ gen_get_float2(LoaderState* stp, GenOpArg Fail, GenOpArg Ms, GenOpArg Live, NATIVE_ENDIAN(Flags); op->op = genop_i_bs_get_float2_6; op->arity = 6; - op->a[0] = Fail; - op->a[1] = Ms; + op->a[0] = Ms; + op->a[1] = Fail; op->a[2] = Live; op->a[3] = Size; op->a[4].type = TAG_u; @@ -3565,10 +3590,22 @@ gen_skip_bits2(LoaderState* stp, GenOpArg Fail, GenOpArg Ms, NATIVE_ENDIAN(Flags); NEW_GENOP(stp, op); if (Size.type == TAG_a && Size.val == am_all) { - op->op = genop_i_bs_skip_bits_all2_3; + /* + * This kind of skip instruction will only be found in modules + * compiled before OTP 19. From OTP 19, the compiler generates + * a test_unit instruction of a bs_skip at the end of a + * binary. + * + * It is safe to replace the skip instruction with a test_unit + * instruction, because the position will never be used again. + * If the match context itself is used again, it will be used by + * a bs_restore2 instruction which will overwrite the position + * by one of the stored positions. + */ + op->op = genop_bs_test_unit_3; op->arity = 3; op->a[0] = Fail; - op->a[1] = Ms; + op->a[1] = Ms; op->a[2] = Unit; } else if (Size.type == TAG_i) { op->op = genop_i_bs_skip_bits_imm2_3; @@ -3601,9 +3638,9 @@ gen_skip_bits2(LoaderState* stp, GenOpArg Fail, GenOpArg Ms, } else { op->op = genop_i_bs_skip_bits2_4; op->arity = 4; - op->a[0] = Fail; - op->a[1] = Ms; - op->a[2] = Size; + op->a[0] = Ms; + op->a[1] = Size; + op->a[2] = Fail; op->a[3] = Unit; } op->next = NULL; diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index 7faba35e1c..c102ddbee6 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -5165,6 +5165,12 @@ erts_schedule_bif(Process *proc, pc = i; mfa = &exp->info.mfa; } + else if (BeamIsOpCode(*i, op_call_bif_only_e)) { + /* Pointer to bif export in i+1 */ + exp = (Export *) i[1]; + pc = i; + mfa = &exp->info.mfa; + } else if (BeamIsOpCode(*i, op_apply_bif)) { /* Pointer to bif in i+1, and mfa in i-3 */ pc = c_p->cp; diff --git a/erts/emulator/beam/bif_instrs.tab b/erts/emulator/beam/bif_instrs.tab index 418bbe2b23..8e0caa38a3 100644 --- a/erts/emulator/beam/bif_instrs.tab +++ b/erts/emulator/beam/bif_instrs.tab @@ -209,8 +209,8 @@ i_length.execute(Fail, Live, Dst) { } // -// The most general BIF call. The BIF may build any amount of data -// on the heap. The result is always returned in r(0). +// Call a BIF, store the result in x(0) and transfer control to the +// next instruction. // call_bif(Exp) { ErtsBifFunc bf; @@ -219,8 +219,10 @@ call_bif(Exp) { Export *export = (Export*) $Exp; if (!((FCALLS - 1) > 0 || (FCALLS-1) > neg_o_reds)) { - /* If we have run out of reductions, we do a context - switch before calling the bif */ + /* + * If we have run out of reductions, do a context + * switch before calling the BIF. + */ c_p->arity = GET_BIF_ARITY(export); c_p->current = &export->info.mfa; goto context_switch3; @@ -257,9 +259,12 @@ call_bif(Exp) { HTOP = HEAP_TOP(c_p); FCALLS = c_p->fcalls; ERTS_DBG_CHK_REDS(c_p, FCALLS); - /* We have to update the cache if we are enabled in order - to make sure no book keeping is done after we disabled - msacc. We don't always do this as it is quite expensive. */ + + /* + * We have to update the cache if we are enabled in order + * to make sure no bookkeeping is done after we disabled + * msacc. We don't always do this as it is quite expensive. + */ if (ERTS_MSACC_IS_ENABLED_CACHED_X()) { ERTS_MSACC_UPDATE_CACHE_X(); } @@ -269,7 +274,13 @@ call_bif(Exp) { CHECK_TERM(r(0)); $NEXT0(); } else if (c_p->freason == TRAP) { - SET_CP(c_p, I+2); + /* + * Set the continuation pointer to return to next + * instruction after the trap (either by a return from + * erlang code or by nif_bif.epilogue() when the BIF + * is done). + */ + SET_CP(c_p, $NEXT_INSTRUCTION); SET_I(c_p->i); SWAPIN; Dispatch(); @@ -281,6 +292,95 @@ call_bif(Exp) { ASSERT(c_p->stop == E); I = handle_error(c_p, I, reg, &export->info.mfa); goto post_error_handling; + //| -no_next +} + +// +// Call a BIF tail-recursively, storing the result in x(0) and doing +// a return to the continuation poiner (c_p->cp). +// + +call_bif_only(Exp) { + ErtsBifFunc bf; + Eterm result; + ErlHeapFragment *live_hf_end; + Export *export = (Export*) $Exp; + + if (!((FCALLS - 1) > 0 || (FCALLS-1) > neg_o_reds)) { + /* + * If we have run out of reductions, do a context + * switch before calling the BIF. + */ + c_p->arity = GET_BIF_ARITY(export); + c_p->current = &export->info.mfa; + goto context_switch3; + } + + ERTS_MSACC_SET_BIF_STATE_CACHED_X(GET_BIF_MODULE(export), + GET_BIF_ADDRESS(export)); + + bf = GET_BIF_ADDRESS(export); + + PRE_BIF_SWAPOUT(c_p); + ERTS_DBG_CHK_REDS(c_p, FCALLS); + c_p->fcalls = FCALLS - 1; + if (FCALLS <= 0) { + save_calls(c_p, export); + } + ASSERT(!ERTS_PROC_IS_EXITING(c_p)); + ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); + live_hf_end = c_p->mbuf; + ERTS_CHK_MBUF_SZ(c_p); + result = (*bf)(c_p, reg, I); + ERTS_CHK_MBUF_SZ(c_p); + ASSERT(!ERTS_PROC_IS_EXITING(c_p) || is_non_value(result)); + ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); + ERTS_HOLE_CHECK(c_p); + ERTS_REQ_PROC_MAIN_LOCK(c_p); + if (ERTS_IS_GC_DESIRED(c_p)) { + Uint arity = GET_BIF_ARITY(export); + result = erts_gc_after_bif_call_lhf(c_p, live_hf_end, result, + reg, arity); + E = c_p->stop; + } + PROCESS_MAIN_CHK_LOCKS(c_p); + HTOP = HEAP_TOP(c_p); + FCALLS = c_p->fcalls; + ERTS_DBG_CHK_REDS(c_p, FCALLS); + + /* + * We have to update the cache if we are enabled in order + * to make sure no bookkeeping is done after we disabled + * msacc. We don't always do this as it is quite expensive. + */ + if (ERTS_MSACC_IS_ENABLED_CACHED_X()) { + ERTS_MSACC_UPDATE_CACHE_X(); + } + ERTS_MSACC_SET_STATE_CACHED_M_X(ERTS_MSACC_STATE_EMULATOR); + if (ERTS_LIKELY(is_value(result))) { + /* + * Success. Store the result and return to the caller. + */ + r(0) = result; + CHECK_TERM(r(0)); + $return(); + } else if (c_p->freason == TRAP) { + /* + * Dispatch to a trap. When the trap is done, a jump + * to the continuation pointer (c_p->cp) will be done. + */ + SET_I(c_p->i); + SWAPIN; + Dispatch(); + } + + /* + * Error handling. SWAPOUT is not needed because it was done above. + */ + ASSERT(c_p->stop == E); + I = handle_error(c_p, I, reg, &export->info.mfa); + goto post_error_handling; + //| -no_next } // @@ -313,7 +413,7 @@ send() { r(0) = result; CHECK_TERM(r(0)); } else if (c_p->freason == TRAP) { - SET_CP(c_p, I+1); + SET_CP(c_p, $NEXT_INSTRUCTION); SET_I(c_p->i); SWAPIN; Dispatch(); diff --git a/erts/emulator/beam/bs_instrs.tab b/erts/emulator/beam/bs_instrs.tab index 2dde70c2e1..652460a66d 100644 --- a/erts/emulator/beam/bs_instrs.tab +++ b/erts/emulator/beam/bs_instrs.tab @@ -21,12 +21,57 @@ %if ARCH_64 BS_SAFE_MUL(A, B, Fail, Dst) { - Uint64 res = ($A) * ($B); - if (res / $B != $A) { + Uint a = $A; + Uint b = $B; + Uint res; +#ifdef HAVE_OVERFLOW_CHECK_BUILTINS + if (__builtin_mul_overflow(a, b, &res)) { + $Fail; + } +#else + res = a * b; + if (res / b != a) { $Fail; } +#endif $Dst = res; } + +BS_GET_FIELD_SIZE(Bits, Unit, Fail, Dst) { + if (is_small($Bits)) { + Uint uint_size; + Sint signed_size = signed_val($Bits); + if (signed_size < 0) { + $Fail; + } + uint_size = (Uint) signed_size; + $BS_SAFE_MUL(uint_size, $Unit, $Fail, $Dst); + } else { + /* + * On a 64-bit architecture, the size of any binary + * that would fit in the memory fits in a small. + */ + $Fail; + } +} + +BS_GET_UNCHECKED_FIELD_SIZE(Bits, Unit, Fail, Dst) { + if (is_small($Bits)) { + Uint uint_size; + Sint signed_size = signed_val($Bits); + if (signed_size < 0) { + $Fail; + } + uint_size = (Uint) signed_size; + $Dst = uint_size * $Unit; + } else { + /* + * On a 64-bit architecture, the size of any binary + * that would fit in the memory fits in a small. + */ + $Fail; + } +} %else BS_SAFE_MUL(A, B, Fail, Dst) { Uint64 res = (Uint64)($A) * (Uint64)($B); @@ -35,7 +80,6 @@ BS_SAFE_MUL(A, B, Fail, Dst) { } $Dst = res; } -%endif BS_GET_FIELD_SIZE(Bits, Unit, Fail, Dst) { Sint signed_size; @@ -76,6 +120,7 @@ BS_GET_UNCHECKED_FIELD_SIZE(Bits, Unit, Fail, Dst) { } $Dst = uint_size * $Unit; } +%endif TEST_BIN_VHEAP(VNh, Nh, Live) { Uint need = $Nh; @@ -90,12 +135,22 @@ TEST_BIN_VHEAP(VNh, Nh, Live) { HEAP_SPACE_VERIFIED(need); } -i_bs_get_binary_all2(Fail, Ms, Live, Unit, Dst) { +i_bs_get_binary_all2 := i_bs_get_binary_all2.fetch.execute; + +i_bs_get_binary_all2.head() { + Eterm context; +} + +i_bs_get_binary_all2.fetch(Ctx) { + context = $Ctx; +} + +i_bs_get_binary_all2.execute(Fail, Live, Unit, Dst) { ErlBinMatchBuffer *_mb; Eterm _result; - $GC_TEST(0, ERL_SUB_BIN_SIZE, $Live); - _mb = ms_matchbuffer($Ms); + $GC_TEST_PRESERVE(ERL_SUB_BIN_SIZE, $Live, context); + _mb = ms_matchbuffer(context); if (((_mb->size - _mb->offset) % $Unit) == 0) { LIGHT_SWAPOUT; _result = erts_bs_get_binary_all_2(c_p, _mb); @@ -109,14 +164,23 @@ i_bs_get_binary_all2(Fail, Ms, Live, Unit, Dst) { $FAIL($Fail); } } +i_bs_get_binary2 := i_bs_get_binary2.fetch.execute; + +i_bs_get_binary2.head() { + Eterm context; +} + +i_bs_get_binary2.fetch(Ctx) { + context = $Ctx; +} -i_bs_get_binary2(Fail, Ms, Live, Sz, Flags, Dst) { +i_bs_get_binary2.execute(Fail, Live, Sz, Flags, Dst) { ErlBinMatchBuffer *_mb; Eterm _result; Uint _size; $BS_GET_FIELD_SIZE($Sz, (($Flags) >> 3), $FAIL($Fail), _size); - $GC_TEST(0, ERL_SUB_BIN_SIZE, $Live); - _mb = ms_matchbuffer($Ms); + $GC_TEST_PRESERVE(ERL_SUB_BIN_SIZE, $Live, context); + _mb = ms_matchbuffer(context); LIGHT_SWAPOUT; _result = erts_bs_get_binary_2(c_p, _size, $Flags, _mb); LIGHT_SWAPIN; @@ -129,11 +193,22 @@ i_bs_get_binary2(Fail, Ms, Live, Sz, Flags, Dst) { } } -i_bs_get_binary_imm2(Fail, Ms, Live, Sz, Flags, Dst) { +i_bs_get_binary_imm2 := i_bs_get_binary_imm2.fetch.execute; + +i_bs_get_binary_imm2.head() { + Eterm context; +} + +i_bs_get_binary_imm2.fetch(Ctx) { + context = $Ctx; +} + +i_bs_get_binary_imm2.execute(Fail, Live, Sz, Flags, Dst) { ErlBinMatchBuffer *_mb; Eterm _result; - $GC_TEST(0, heap_bin_size(ERL_ONHEAP_BIN_LIMIT), $Live); - _mb = ms_matchbuffer($Ms); + $GC_TEST_PRESERVE(heap_bin_size(ERL_ONHEAP_BIN_LIMIT), + $Live, context); + _mb = ms_matchbuffer(context); LIGHT_SWAPOUT; _result = erts_bs_get_binary_2(c_p, $Sz, $Flags, _mb); LIGHT_SWAPIN; @@ -145,8 +220,17 @@ i_bs_get_binary_imm2(Fail, Ms, Live, Sz, Flags, Dst) { $Dst = _result; } } +i_bs_get_float2 := i_bs_get_float2.fetch.execute; + +i_bs_get_float2.head() { + Eterm context; +} + +i_bs_get_float2.fetch(Ctx) { + context = $Ctx; +} -i_bs_get_float2(Fail, Ms, Live, Sz, Flags, Dst) { +i_bs_get_float2.execute(Fail, Live, Sz, Flags, Dst) { ErlBinMatchBuffer *_mb; Eterm _result; Sint _size; @@ -155,8 +239,8 @@ i_bs_get_float2(Fail, Ms, Live, Sz, Flags, Dst) { $FAIL($Fail); } _size *= (($Flags) >> 3); - $GC_TEST(0, FLOAT_SIZE_OBJECT, $Live); - _mb = ms_matchbuffer($Ms); + $GC_TEST_PRESERVE(FLOAT_SIZE_OBJECT, $Live, context); + _mb = ms_matchbuffer(context); LIGHT_SWAPOUT; _result = erts_bs_get_float_2(c_p, _size, ($Flags), _mb); LIGHT_SWAPIN; @@ -169,13 +253,24 @@ i_bs_get_float2(Fail, Ms, Live, Sz, Flags, Dst) { } } -i_bs_skip_bits2(Fail, Ms, Bits, Unit) { +i_bs_skip_bits2 := i_bs_skip_bits2.fetch.execute; + +i_bs_skip_bits2.head() { + Eterm context, bits; +} + +i_bs_skip_bits2.fetch(Ctx, Bits) { + context = $Ctx; + bits = $Bits; +} + +i_bs_skip_bits2.execute(Fail, Unit) { ErlBinMatchBuffer *_mb; size_t new_offset; Uint _size; - _mb = ms_matchbuffer($Ms); - $BS_GET_FIELD_SIZE($Bits, $Unit, $FAIL($Fail), _size); + _mb = ms_matchbuffer(context); + $BS_GET_FIELD_SIZE(bits, $Unit, $FAIL($Fail), _size); new_offset = _mb->offset + _size; if (new_offset <= _mb->size) { _mb->offset = new_offset; @@ -184,16 +279,6 @@ i_bs_skip_bits2(Fail, Ms, Bits, Unit) { } } -i_bs_skip_bits_all2(Fail, Ms, Unit) { - ErlBinMatchBuffer *_mb; - _mb = ms_matchbuffer($Ms); - if (((_mb->size - _mb->offset) % $Unit) == 0) { - _mb->offset = _mb->size; - } else { - $FAIL($Fail); - } -} - i_bs_skip_bits_imm2(Fail, Ms, Bits) { ErlBinMatchBuffer *_mb; size_t new_offset; @@ -207,15 +292,25 @@ i_bs_skip_bits_imm2(Fail, Ms, Bits) { } i_new_bs_put_binary(Fail, Sz, Flags, Src) { + Eterm sz = $Sz; Sint _size; - $BS_GET_UNCHECKED_FIELD_SIZE($Sz, (($Flags) >> 3), $BADARG($Fail), _size); + $BS_GET_UNCHECKED_FIELD_SIZE(sz, (($Flags) >> 3), $BADARG($Fail), _size); if (!erts_new_bs_put_binary(ERL_BITS_ARGS_2(($Src), _size))) { $BADARG($Fail); } } +i_new_bs_put_binary_all := i_new_bs_put_binary_all.fetch.execute; + +i_new_bs_put_binary_all.head() { + Eterm src; +} + +i_new_bs_put_binary_all.fetch(Src) { + src = $Src; +} -i_new_bs_put_binary_all(Fail, Src, Unit) { - if (!erts_new_bs_put_binary_all(ERL_BITS_ARGS_2(($Src), ($Unit)))) { +i_new_bs_put_binary_all.execute(Fail, Unit) { + if (!erts_new_bs_put_binary_all(ERL_BITS_ARGS_2(src, ($Unit)))) { $BADARG($Fail); } } @@ -227,9 +322,11 @@ i_new_bs_put_binary_imm(Fail, Sz, Src) { } i_new_bs_put_float(Fail, Sz, Flags, Src) { + Eterm sz = $Sz; + Eterm flags = $Flags; Sint _size; - $BS_GET_UNCHECKED_FIELD_SIZE($Sz, (($Flags) >> 3), $BADARG($Fail), _size); - if (!erts_new_bs_put_float(c_p, ($Src), _size, ($Flags))) { + $BS_GET_UNCHECKED_FIELD_SIZE(sz, (flags >> 3), $BADARG($Fail), _size); + if (!erts_new_bs_put_float(c_p, ($Src), _size, flags)) { $BADARG($Fail); } } @@ -241,15 +338,27 @@ i_new_bs_put_float_imm(Fail, Sz, Flags, Src) { } i_new_bs_put_integer(Fail, Sz, Flags, Src) { - Sint _size; - $BS_GET_UNCHECKED_FIELD_SIZE($Sz, (($Flags) >> 3), $BADARG($Fail), _size); - if (!erts_new_bs_put_integer(ERL_BITS_ARGS_3(($Src), _size, ($Flags)))) { - $BADARG($Fail); - } + Eterm sz = $Sz; + Eterm flags = $Flags; + Sint _size; + $BS_GET_UNCHECKED_FIELD_SIZE(sz, (flags >> 3), $BADARG($Fail), _size); + if (!erts_new_bs_put_integer(ERL_BITS_ARGS_3(($Src), _size, flags))) { + $BADARG($Fail); + } +} + +i_new_bs_put_integer_imm := i_new_bs_put_integer_imm.fetch.execute; + +i_new_bs_put_integer_imm.head() { + Eterm src; } -i_new_bs_put_integer_imm(Fail, Sz, Flags, Src) { - if (!erts_new_bs_put_integer(ERL_BITS_ARGS_3(($Src), ($Sz), ($Flags)))) { +i_new_bs_put_integer_imm.fetch(Src) { + src = $Src; +} + +i_new_bs_put_integer_imm.execute(Fail, Sz, Flags) { + if (!erts_new_bs_put_integer(ERL_BITS_ARGS_3(src, ($Sz), ($Flags)))) { $BADARG($Fail); } } @@ -809,9 +918,19 @@ bs_test_unit8(Fail, Ctx) { } } -i_bs_get_integer_8(Ctx, Fail, Dst) { +i_bs_get_integer_8 := i_bs_get_integer_8.fetch.execute; + +i_bs_get_integer_8.head() { + Eterm context; +} + +i_bs_get_integer_8.fetch(Ctx) { + context = $Ctx; +} + +i_bs_get_integer_8.execute(Fail, Dst) { Eterm _result; - ErlBinMatchBuffer* _mb = ms_matchbuffer($Ctx); + ErlBinMatchBuffer* _mb = ms_matchbuffer(context); if (_mb->size - _mb->offset < 8) { $FAIL($Fail); @@ -825,9 +944,19 @@ i_bs_get_integer_8(Ctx, Fail, Dst) { $Dst = _result; } -i_bs_get_integer_16(Ctx, Fail, Dst) { +i_bs_get_integer_16 := i_bs_get_integer_16.fetch.execute; + +i_bs_get_integer_16.head() { + Eterm context; +} + +i_bs_get_integer_16.fetch(Ctx) { + context = $Ctx; +} + +i_bs_get_integer_16.execute(Fail, Dst) { Eterm _result; - ErlBinMatchBuffer* _mb = ms_matchbuffer($Ctx); + ErlBinMatchBuffer* _mb = ms_matchbuffer(context); if (_mb->size - _mb->offset < 16) { $FAIL($Fail); @@ -842,9 +971,19 @@ i_bs_get_integer_16(Ctx, Fail, Dst) { } %if ARCH_64 -i_bs_get_integer_32(Ctx, Fail, Dst) { +i_bs_get_integer_32 := i_bs_get_integer_32.fetch.execute; + +i_bs_get_integer_32.head() { + Eterm context; +} + +i_bs_get_integer_32.fetch(Ctx) { + context = $Ctx; +} + +i_bs_get_integer_32.execute(Fail, Dst) { Uint32 _integer; - ErlBinMatchBuffer* _mb = ms_matchbuffer($Ctx); + ErlBinMatchBuffer* _mb = ms_matchbuffer(context); if (_mb->size - _mb->offset < 32) { $FAIL($Fail); @@ -894,15 +1033,23 @@ bs_get_integer.execute(Fail, Flags, Dst) { $Dst = result; } -i_bs_get_integer(Fail, Live, FlagsAndUnit, Ms, Sz, Dst) { +i_bs_get_integer := i_bs_get_integer.fetch.execute; + +i_bs_get_integer.head() { + Eterm context; +} + +i_bs_get_integer.fetch(Ctx) { + context = $Ctx; +} + +i_bs_get_integer.execute(Fail, Live, FlagsAndUnit, Sz, Dst) { Uint flags; Uint size; - Eterm ms; ErlBinMatchBuffer* mb; Eterm result; flags = $FlagsAndUnit; - ms = $Ms; $BS_GET_FIELD_SIZE($Sz, (flags >> 3), $FAIL($Fail), size); if (size >= SMALL_BITS) { Uint wordsneeded; @@ -913,15 +1060,15 @@ i_bs_get_integer(Fail, Live, FlagsAndUnit, Ms, Sz, Dst) { * Remember to re-acquire the matchbuffer after gc. */ - mb = ms_matchbuffer(ms); + mb = ms_matchbuffer(context); if (mb->size - mb->offset < size) { $FAIL($Fail); } wordsneeded = 1+WSIZE(NBYTES((Uint) size)); - $GC_TEST_PRESERVE(wordsneeded, $Live, ms); + $GC_TEST_PRESERVE(wordsneeded, $Live, context); $REFRESH_GEN_DEST(); } - mb = ms_matchbuffer(ms); + mb = ms_matchbuffer(context); LIGHT_SWAPOUT; result = erts_bs_get_integer_2(c_p, size, flags, mb); LIGHT_SWAPIN; @@ -932,9 +1079,19 @@ i_bs_get_integer(Fail, Live, FlagsAndUnit, Ms, Sz, Dst) { $Dst = result; } -i_bs_get_utf8(Ctx, Fail, Dst) { +i_bs_get_utf8 := i_bs_get_utf8.fetch.execute; + +i_bs_get_utf8.head() { + Eterm context; +} + +i_bs_get_utf8.fetch(Ctx) { + context = $Ctx; +} + +i_bs_get_utf8.execute(Fail, Dst) { Eterm result; - ErlBinMatchBuffer* mb = ms_matchbuffer($Ctx); + ErlBinMatchBuffer* mb = ms_matchbuffer(context); if (mb->size - mb->offset < 8) { $FAIL($Fail); @@ -957,8 +1114,18 @@ i_bs_get_utf8(Ctx, Fail, Dst) { $Dst = result; } -i_bs_get_utf16(Ctx, Fail, Flags, Dst) { - ErlBinMatchBuffer* mb = ms_matchbuffer($Ctx); +i_bs_get_utf16 := i_bs_get_utf16.fetch.execute; + +i_bs_get_utf16.head() { + Eterm context; +} + +i_bs_get_utf16.fetch(Ctx) { + context = $Ctx; +} + +i_bs_get_utf16.execute(Fail, Flags, Dst) { + ErlBinMatchBuffer* mb = ms_matchbuffer(context); Eterm result = erts_bs_get_utf16(mb, $Flags); if (is_non_value(result)) { @@ -1055,13 +1222,20 @@ i_bs_restore2(Src, Slot) { _ms->mb.offset = _ms->save_offset[$Slot]; } -bs_get_tail(Src, Dst, Live) { - ErlBinMatchBuffer* mb; - Uint size, offs; - ErlSubBin* sb; +bs_get_tail := bs_get_tail.fetch.execute; + +bs_get_tail.head() { Eterm context; +} +bs_get_tail.fetch(Src) { context = $Src; +} + +bs_get_tail.execute(Dst, Live) { + ErlBinMatchBuffer* mb; + Uint size, offs; + ErlSubBin* sb; ASSERT(header_is_bin_matchstate(*boxed_val(context))); @@ -1090,11 +1264,20 @@ bs_get_tail(Src, Dst, Live) { %if ARCH_64 -i_bs_start_match3_gp(Src, Live, Fail, Dst, Pos) { - Eterm context, header; - Uint position, live; +i_bs_start_match3_gp := i_bs_start_match3_gp.fetch.execute; +i_bs_start_match3_gp.head() { + Eterm context; +} + +i_bs_start_match3_gp.fetch(Src) { context = $Src; +} + +i_bs_start_match3_gp.execute(Live, Fail, Dst, Pos) { + Eterm header; + Uint position, live; + live = $Live; if (!is_boxed(context)) { @@ -1139,11 +1322,20 @@ i_bs_start_match3_gp(Src, Live, Fail, Dst, Pos) { $Pos = make_small(position); } -i_bs_start_match3(Src, Live, Fail, Dst) { - Eterm context, header; - Uint live; +i_bs_start_match3 := i_bs_start_match3.fetch.execute; + +i_bs_start_match3.head() { + Eterm context; +} +i_bs_start_match3.fetch(Src) { context = $Src; +} + +i_bs_start_match3.execute(Live, Fail, Dst) { + Eterm header; + Uint live; + live = $Live; if (!is_boxed(context)) { @@ -1213,12 +1405,19 @@ i_bs_get_position(Ctx, Dst) { # match at a position beyond 16MB. # -bs_set_position(Ctx, Pos) { +bs_set_position := bs_set_position.fetch.execute; + +bs_set_position.head() { Eterm context, position; - ErlBinMatchState *ms; +} +bs_set_position.fetch(Ctx, Pos) { context = $Ctx; position = $Pos; +} + +bs_set_position.execute() { + ErlBinMatchState *ms; ASSERT(header_is_bin_matchstate(*boxed_val(context))); ms = (ErlBinMatchState*)boxed_val(context); @@ -1231,12 +1430,19 @@ bs_set_position(Ctx, Pos) { } } -bs_get_position(Ctx, Dst, Live) { - ErlBinMatchState *ms; +bs_get_position := bs_get_position.fetch.execute; + +bs_get_position.head() { Eterm context; - Uint position; +} +bs_get_position.fetch(Ctx) { context = $Ctx; +} + +bs_get_position.execute(Dst, Live) { + ErlBinMatchState *ms; + Uint position; ASSERT(header_is_bin_matchstate(*boxed_val(context))); ms = (ErlBinMatchState*)boxed_val(context); @@ -1261,11 +1467,20 @@ bs_get_position(Ctx, Dst, Live) { } } -i_bs_start_match3(Src, Live, Fail, Dst) { - Eterm context, header; - Uint live; +i_bs_start_match3 := i_bs_start_match3.fetch.execute; + +i_bs_start_match3.head() { + Eterm context; +} +i_bs_start_match3.fetch(Src) { context = $Src; +} + +i_bs_start_match3.execute(Live, Fail, Dst) { + Eterm header; + Uint live; + live = $Live; if (!is_boxed(context)) { diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 74708b2caa..8c51bdb630 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -4689,6 +4689,16 @@ BIF_RETTYPE erts_debug_set_internal_state_2(BIF_ALIST_2) BIF_RET(am_ok); } } + else if (ERTS_IS_ATOM_STR("mbuf", BIF_ARG_1)) { + Uint sz = size_object(BIF_ARG_2); + ErlHeapFragment* frag = new_message_buffer(sz); + Eterm *hp = frag->mem; + Eterm copy = copy_struct(BIF_ARG_2, sz, &hp, &frag->off_heap); + frag->next = BIF_P->mbuf; + BIF_P->mbuf = frag; + BIF_P->mbuf_sz += sz; + BIF_RET(copy); + } } BIF_ERROR(BIF_P, BADARG); diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c index 4132a54934..0a50af4d1a 100644 --- a/erts/emulator/beam/erl_db.c +++ b/erts/emulator/beam/erl_db.c @@ -81,14 +81,6 @@ static BIF_RETTYPE db_bif_fail(Process* p, Uint freason, /* Get a key from any table structure and a tagged object */ #define TERM_GETKEY(tb, obj) db_getkey((tb)->common.keypos, (obj)) - -/* How safe are we from double-hits or missed objects -** when iterating without fixation? */ -enum DbIterSafety { - ITER_UNSAFE, /* Must fixate to be safe */ - ITER_SAFE_LOCKED, /* Safe while table is locked, not between trap calls */ - ITER_SAFE /* No need to fixate at all */ -}; # define ITERATION_SAFETY(Proc,Tab) \ ((IS_TREE_TABLE((Tab)->common.status) || IS_CATREE_TABLE((Tab)->common.status) \ || ONLY_WRITER(Proc,Tab)) ? ITER_SAFE \ @@ -195,9 +187,6 @@ static int fixed_tabs_find(DbFixation* first, DbFixation* fix) #define ERTS_RBT_WANT_DELETE #define ERTS_RBT_WANT_FOREACH #define ERTS_RBT_WANT_FOREACH_DESTROY -#ifdef DEBUG -# define ERTS_RBT_WANT_LOOKUP -#endif #define ERTS_RBT_UNDEF #include "erl_rbtree.h" @@ -2287,6 +2276,7 @@ static BIF_RETTYPE ets_select_delete_trap_1(BIF_ALIST_1) Eterm ret; Eterm *tptr; db_lock_kind_t kind = LCK_WRITE_REC; + enum DbIterSafety safety = ITER_SAFE; CHECK_TABLES(); ASSERT(is_tuple(a1)); @@ -2296,10 +2286,11 @@ static BIF_RETTYPE ets_select_delete_trap_1(BIF_ALIST_1) DB_TRAP_GET_TABLE(tb, tptr[1], DB_WRITE, kind, &ets_select_delete_continue_exp); - cret = tb->common.meth->db_select_delete_continue(p,tb,a1,&ret); + cret = tb->common.meth->db_select_delete_continue(p,tb,a1,&ret,&safety); - if(!DID_TRAP(p,ret) && ITERATION_SAFETY(p,tb) != ITER_SAFE) { - unfix_table_locked(p, tb, &kind); + if(!DID_TRAP(p,ret) && safety != ITER_SAFE) { + ASSERT(erts_refc_read(&tb->common.fix_count,1)); + unfix_table_locked(p, tb, &kind); } db_unlock(tb, kind); @@ -2337,7 +2328,8 @@ BIF_RETTYPE ets_internal_select_delete_2(BIF_ALIST_2) if (safety == ITER_UNSAFE) { local_fix_table(tb); } - cret = tb->common.meth->db_select_delete(BIF_P, tb, BIF_ARG_1, BIF_ARG_2, &ret); + cret = tb->common.meth->db_select_delete(BIF_P, tb, BIF_ARG_1, BIF_ARG_2, + &ret, safety); if (DID_TRAP(BIF_P,ret) && safety != ITER_SAFE) { fix_table_locked(BIF_P,tb); @@ -2729,7 +2721,7 @@ ets_select3(Process* p, DbTable* tb, Eterm tid, Eterm ms, Sint chunk_size) cret = tb->common.meth->db_select_chunk(p, tb, tid, ms, chunk_size, 0 /* not reversed */, - &ret); + &ret, safety); if (DID_TRAP(p,ret) && safety != ITER_SAFE) { fix_table_locked(p, tb); } @@ -2756,7 +2748,8 @@ ets_select3(Process* p, DbTable* tb, Eterm tid, Eterm ms, Sint chunk_size) } -/* We get here instead of in the real BIF when trapping */ +/* Trap here from: ets_select_1/2/3 + */ static BIF_RETTYPE ets_select_trap_1(BIF_ALIST_1) { Process *p = BIF_P; @@ -2767,6 +2760,7 @@ static BIF_RETTYPE ets_select_trap_1(BIF_ALIST_1) Eterm ret; Eterm *tptr; db_lock_kind_t kind = LCK_READ; + enum DbIterSafety safety = ITER_SAFE; CHECK_TABLES(); @@ -2776,11 +2770,13 @@ static BIF_RETTYPE ets_select_trap_1(BIF_ALIST_1) DB_TRAP_GET_TABLE(tb, tptr[1], DB_READ, kind, &ets_select_continue_exp); - cret = tb->common.meth->db_select_continue(p, tb, a1, - &ret); + cret = tb->common.meth->db_select_continue(p, tb, a1, &ret, &safety); - if (!DID_TRAP(p,ret) && ITERATION_SAFETY(p,tb) != ITER_SAFE) { - unfix_table_locked(p, tb, &kind); + if (!DID_TRAP(p,ret)) { + if (safety != ITER_SAFE) { + ASSERT(erts_refc_read(&tb->common.fix_count,1)); + unfix_table_locked(p, tb, &kind); + } } db_unlock(tb, kind); @@ -2805,8 +2801,12 @@ static BIF_RETTYPE ets_select_trap_1(BIF_ALIST_1) BIF_RETTYPE ets_select_1(BIF_ALIST_1) { return ets_select1(BIF_P, BIF_ets_select_1, BIF_ARG_1); + /* TRAP: ets_select_trap_1 */ } +/* + * Common impl for select/1, select_reverse/1, match/1 and match_object/1 + */ static BIF_RETTYPE ets_select1(Process *p, int bif_ix, Eterm arg1) { BIF_RETTYPE result; @@ -2814,7 +2814,7 @@ static BIF_RETTYPE ets_select1(Process *p, int bif_ix, Eterm arg1) int cret; Eterm ret; Eterm *tptr; - enum DbIterSafety safety; + enum DbIterSafety safety, safety_copy; CHECK_TABLES(); @@ -2839,7 +2839,8 @@ static BIF_RETTYPE ets_select1(Process *p, int bif_ix, Eterm arg1) local_fix_table(tb); } - cret = tb->common.meth->db_select_continue(p,tb, arg1, &ret); + safety_copy = safety; + cret = tb->common.meth->db_select_continue(p,tb, arg1, &ret, &safety_copy); if (DID_TRAP(p,ret) && safety != ITER_SAFE) { fix_table_locked(p, tb); @@ -2871,6 +2872,7 @@ BIF_RETTYPE ets_select_2(BIF_ALIST_2) DbTable* tb; DB_BIF_GET_TABLE(tb, DB_READ, LCK_READ, BIF_ets_select_2); return ets_select2(BIF_P, tb, BIF_ARG_1, BIF_ARG_2); + /* TRAP: ets_select_trap_1 */ } static BIF_RETTYPE @@ -2888,7 +2890,7 @@ ets_select2(Process* p, DbTable* tb, Eterm tid, Eterm ms) local_fix_table(tb); } - cret = tb->common.meth->db_select(p, tb, tid, ms, 0, &ret); + cret = tb->common.meth->db_select(p, tb, tid, ms, 0, &ret, safety); if (DID_TRAP(p,ret) && safety != ITER_SAFE) { fix_table_locked(p, tb); @@ -2926,6 +2928,7 @@ static BIF_RETTYPE ets_select_count_1(BIF_ALIST_1) Eterm ret; Eterm *tptr; db_lock_kind_t kind = LCK_READ; + enum DbIterSafety safety = ITER_SAFE; CHECK_TABLES(); @@ -2935,9 +2938,10 @@ static BIF_RETTYPE ets_select_count_1(BIF_ALIST_1) DB_TRAP_GET_TABLE(tb, tptr[1], DB_READ, kind, &ets_select_count_continue_exp); - cret = tb->common.meth->db_select_count_continue(p, tb, a1, &ret); + cret = tb->common.meth->db_select_count_continue(p, tb, a1, &ret, &safety); - if (!DID_TRAP(p,ret) && ITERATION_SAFETY(p,tb) != ITER_SAFE) { + if (!DID_TRAP(p,ret) && safety != ITER_SAFE) { + ASSERT(erts_refc_read(&tb->common.fix_count,1)); unfix_table_locked(p, tb, &kind); } db_unlock(tb, kind); @@ -2975,7 +2979,8 @@ BIF_RETTYPE ets_select_count_2(BIF_ALIST_2) if (safety == ITER_UNSAFE) { local_fix_table(tb); } - cret = tb->common.meth->db_select_count(BIF_P,tb, BIF_ARG_1, BIF_ARG_2, &ret); + cret = tb->common.meth->db_select_count(BIF_P,tb, BIF_ARG_1, BIF_ARG_2, + &ret, safety); if (DID_TRAP(BIF_P,ret) && safety != ITER_SAFE) { fix_table_locked(BIF_P, tb); @@ -3014,6 +3019,7 @@ static BIF_RETTYPE ets_select_replace_1(BIF_ALIST_1) Eterm ret; Eterm *tptr; db_lock_kind_t kind = LCK_WRITE_REC; + enum DbIterSafety safety = ITER_SAFE; CHECK_TABLES(); ASSERT(is_tuple(a1)); @@ -3023,9 +3029,10 @@ static BIF_RETTYPE ets_select_replace_1(BIF_ALIST_1) DB_TRAP_GET_TABLE(tb, tptr[1], DB_WRITE, kind, &ets_select_replace_continue_exp); - cret = tb->common.meth->db_select_replace_continue(p,tb,a1,&ret); + cret = tb->common.meth->db_select_replace_continue(p,tb,a1,&ret,&safety); - if(!DID_TRAP(p,ret) && ITERATION_SAFETY(p,tb) != ITER_SAFE) { + if(!DID_TRAP(p,ret) && safety != ITER_SAFE) { + ASSERT(erts_refc_read(&tb->common.fix_count,1)); unfix_table_locked(p, tb, &kind); } @@ -3068,7 +3075,8 @@ BIF_RETTYPE ets_select_replace_2(BIF_ALIST_2) if (safety == ITER_UNSAFE) { local_fix_table(tb); } - cret = tb->common.meth->db_select_replace(BIF_P, tb, BIF_ARG_1, BIF_ARG_2, &ret); + cret = tb->common.meth->db_select_replace(BIF_P, tb, BIF_ARG_1, BIF_ARG_2, + &ret, safety); if (DID_TRAP(BIF_P,ret) && safety != ITER_SAFE) { fix_table_locked(BIF_P,tb); @@ -3120,7 +3128,7 @@ BIF_RETTYPE ets_select_reverse_3(BIF_ALIST_3) } cret = tb->common.meth->db_select_chunk(BIF_P,tb, BIF_ARG_1, BIF_ARG_2, chunk_size, - 1 /* reversed */, &ret); + 1 /* reversed */, &ret, safety); if (DID_TRAP(BIF_P,ret) && safety != ITER_SAFE) { fix_table_locked(BIF_P, tb); } @@ -3165,7 +3173,7 @@ BIF_RETTYPE ets_select_reverse_2(BIF_ALIST_2) local_fix_table(tb); } cret = tb->common.meth->db_select(BIF_P,tb, BIF_ARG_1, BIF_ARG_2, - 1 /*reversed*/, &ret); + 1 /*reversed*/, &ret, safety); if (DID_TRAP(BIF_P,ret) && safety != ITER_SAFE) { fix_table_locked(BIF_P, tb); diff --git a/erts/emulator/beam/erl_db_catree.c b/erts/emulator/beam/erl_db_catree.c index 75ac1c4a93..0402c6b7b4 100644 --- a/erts/emulator/beam/erl_db_catree.c +++ b/erts/emulator/beam/erl_db_catree.c @@ -116,24 +116,31 @@ static int db_erase_object_catree(DbTable *tbl, Eterm object,Eterm *ret); static int db_slot_catree(Process *p, DbTable *tbl, Eterm slot_term, Eterm *ret); static int db_select_catree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, int reversed, Eterm *ret); + Eterm pattern, int reversed, Eterm *ret, + enum DbIterSafety); static int db_select_count_catree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret); + Eterm pattern, Eterm *ret, enum DbIterSafety); static int db_select_chunk_catree(Process *p, DbTable *tbl, Eterm tid, Eterm pattern, Sint chunk_size, - int reversed, Eterm *ret); + int reversed, Eterm *ret, enum DbIterSafety); static int db_select_continue_catree(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_select_count_continue_catree(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_select_delete_catree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret); + Eterm pattern, Eterm *ret, + enum DbIterSafety); static int db_select_delete_continue_catree(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_select_replace_catree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret); + Eterm pattern, Eterm *ret, + enum DbIterSafety); static int db_select_replace_continue_catree(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_take_catree(Process *, DbTable *, Eterm, Eterm *); static void db_print_catree(fmtfn_t to, void *to_arg, int show, DbTable *tbl); @@ -1843,7 +1850,8 @@ static int db_slot_catree(Process *p, DbTable *tbl, static int db_select_continue_catree(Process *p, DbTable *tbl, Eterm continuation, - Eterm *ret) + Eterm *ret, + enum DbIterSafety* safety_p) { int result; CATreeRootIterator iter; @@ -1856,7 +1864,8 @@ static int db_select_continue_catree(Process *p, } static int db_select_catree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, int reverse, Eterm *ret) + Eterm pattern, int reverse, Eterm *ret, + enum DbIterSafety safety) { int result; CATreeRootIterator iter; @@ -1871,7 +1880,8 @@ static int db_select_catree(Process *p, DbTable *tbl, Eterm tid, static int db_select_count_continue_catree(Process *p, DbTable *tbl, Eterm continuation, - Eterm *ret) + Eterm *ret, + enum DbIterSafety* safety_p) { int result; CATreeRootIterator iter; @@ -1885,7 +1895,8 @@ static int db_select_count_continue_catree(Process *p, } static int db_select_count_catree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret) + Eterm pattern, Eterm *ret, + enum DbIterSafety safety) { int result; CATreeRootIterator iter; @@ -1899,7 +1910,8 @@ static int db_select_count_catree(Process *p, DbTable *tbl, Eterm tid, static int db_select_chunk_catree(Process *p, DbTable *tbl, Eterm tid, Eterm pattern, Sint chunk_size, - int reversed, Eterm *ret) + int reversed, Eterm *ret, + enum DbIterSafety safety) { int result; CATreeRootIterator iter; @@ -1915,7 +1927,8 @@ static int db_select_chunk_catree(Process *p, DbTable *tbl, Eterm tid, static int db_select_delete_continue_catree(Process *p, DbTable *tbl, Eterm continuation, - Eterm *ret) + Eterm *ret, + enum DbIterSafety* safety_p) { DbTreeStack stack; TreeDbTerm * stack_array[STACK_NEED]; @@ -1931,7 +1944,8 @@ static int db_select_delete_continue_catree(Process *p, } static int db_select_delete_catree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret) + Eterm pattern, Eterm *ret, + enum DbIterSafety safety) { DbTreeStack stack; TreeDbTerm * stack_array[STACK_NEED]; @@ -1948,7 +1962,8 @@ static int db_select_delete_catree(Process *p, DbTable *tbl, Eterm tid, } static int db_select_replace_catree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret) + Eterm pattern, Eterm *ret, + enum DbIterSafety safety_p) { int result; CATreeRootIterator iter; @@ -1961,7 +1976,8 @@ static int db_select_replace_catree(Process *p, DbTable *tbl, Eterm tid, } static int db_select_replace_continue_catree(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret) + Eterm continuation, Eterm *ret, + enum DbIterSafety* safety_p) { int result; CATreeRootIterator iter; diff --git a/erts/emulator/beam/erl_db_hash.c b/erts/emulator/beam/erl_db_hash.c index f05a3b51c9..f225730029 100644 --- a/erts/emulator/beam/erl_db_hash.c +++ b/erts/emulator/beam/erl_db_hash.c @@ -404,26 +404,31 @@ static int db_slot_hash(Process *p, DbTable *tbl, static int db_select_chunk_hash(Process *p, DbTable *tbl, Eterm tid, Eterm pattern, Sint chunk_size, - int reverse, Eterm *ret); + int reverse, Eterm *ret, enum DbIterSafety); static int db_select_hash(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, int reverse, Eterm *ret); + Eterm pattern, int reverse, Eterm *ret, + enum DbIterSafety); static int db_select_continue_hash(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_select_count_hash(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret); + Eterm pattern, Eterm *ret, enum DbIterSafety); static int db_select_count_continue_hash(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); - + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_select_delete_hash(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret); + Eterm pattern, Eterm *ret, + enum DbIterSafety); static int db_select_delete_continue_hash(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_select_replace_hash(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret); + Eterm pattern, Eterm *ret, enum DbIterSafety); static int db_select_replace_continue_hash(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_take_hash(Process *, DbTable *, Eterm, Eterm *); static void db_print_hash(fmtfn_t to, @@ -535,7 +540,7 @@ DbTableMethod db_hash = db_select_chunk_hash, db_select_hash, db_select_delete_hash, - db_select_continue_hash, /* hmm continue_hash? */ + db_select_continue_hash, db_select_delete_continue_hash, db_select_count_hash, db_select_count_continue_hash, @@ -1154,8 +1159,9 @@ static int db_slot_hash(Process *p, DbTable *tbl, Eterm slot_term, Eterm *ret) * Match traversal callbacks */ -typedef struct match_callbacks_t_ match_callbacks_t; -struct match_callbacks_t_ + +typedef struct traverse_context_t_ traverse_context_t; +struct traverse_context_t_ { /* Called when no match is possible. * context_ptr: Pointer to context @@ -1163,7 +1169,7 @@ struct match_callbacks_t_ * * Both the direct return value and 'ret' are used as the traversal function return values. */ - int (*on_nothing_can_match)(match_callbacks_t* ctx, Eterm* ret); + int (*on_nothing_can_match)(traverse_context_t* ctx, Eterm* ret); /* Called for each match result. * context_ptr: Pointer to context @@ -1174,7 +1180,7 @@ struct match_callbacks_t_ * * Should return 1 for successful match, 0 otherwise. */ - int (*on_match_res)(match_callbacks_t* ctx, Sint slot_ix, + int (*on_match_res)(traverse_context_t* ctx, Sint slot_ix, HashDbTerm*** current_ptr_ptr, Eterm match_res); /* Called when either we've matched enough elements in this cycle or EOT was reached. @@ -1188,7 +1194,7 @@ struct match_callbacks_t_ * Both the direct return value and 'ret' are used as the traversal function return values. * If *mpp is set to NULL, it won't be deallocated (useful for trapping.) */ - int (*on_loop_ended)(match_callbacks_t* ctx, Sint slot_ix, Sint got, + int (*on_loop_ended)(traverse_context_t* ctx, Sint slot_ix, Sint got, Sint iterations_left, Binary** mpp, Eterm* ret); /* Called when it's time to trap @@ -1201,16 +1207,21 @@ struct match_callbacks_t_ * Both the direct return value and 'ret' are used as the traversal function return values. * If *mpp is set to NULL, it won't be deallocated (useful for trapping.) */ - int (*on_trap)(match_callbacks_t* ctx, Sint slot_ix, Sint got, Binary** mpp, + int (*on_trap)(traverse_context_t* ctx, Sint slot_ix, Sint got, Binary** mpp, Eterm* ret); + Process* p; + DbTableHash* tb; + Eterm tid; + Eterm* prev_continuation_tptr; + enum DbIterSafety safety; }; /* * Begin hash table match traversal */ -static int match_traverse(Process* p, DbTableHash* tb, +static int match_traverse(traverse_context_t* ctx, Eterm pattern, extra_match_validator_t extra_match_validator, /* Optional */ Sint chunk_size, /* If 0, no chunking */ @@ -1218,9 +1229,9 @@ static int match_traverse(Process* p, DbTableHash* tb, Eterm** hpp, /* Heap */ int lock_for_write, /* Set to 1 if we're going to delete or modify existing terms */ - match_callbacks_t* ctx, Eterm* ret) { + DbTableHash* tb = ctx->tb; Sint slot_ix; /* Slot index */ HashDbTerm** current_ptr; /* Refers to either the bucket pointer or * the 'next' pointer in the previous term @@ -1287,7 +1298,7 @@ static int match_traverse(Process* p, DbTableHash* tb, for(;;) { if (*current_ptr != NULL) { if (!is_pseudo_deleted(*current_ptr)) { - match_res = db_match_dbterm(&tb->common, p, mpi.mp, + match_res = db_match_dbterm(&tb->common, ctx->p, mpi.mp, &(*current_ptr)->dbterm, hpp, 2); saved_current = *current_ptr; if (ctx->on_match_res(ctx, slot_ix, ¤t_ptr, match_res)) { @@ -1323,11 +1334,7 @@ static int match_traverse(Process* p, DbTableHash* tb, unlock_hash_function(lck); break; } - if (iterations_left <= 0 || MBUF(p)) { - /* - * We have either reached our limit, or just created some heap fragments. - * Since many heap fragments will make the GC slower, trap and GC now. - */ + if (iterations_left <= 0) { unlock_hash_function(lck); ret_value = ctx->on_trap(ctx, slot_ix, got, &mpi.mp, ret); goto done; @@ -1356,7 +1363,7 @@ done: /* * Continue hash table match traversal */ -static int match_traverse_continue(Process* p, DbTableHash* tb, +static int match_traverse_continue(traverse_context_t* ctx, Sint chunk_size, /* If 0, no chunking */ Sint iterations_left, /* Nr. of iterations left */ Eterm** hpp, /* Heap */ @@ -1365,9 +1372,9 @@ static int match_traverse_continue(Process* p, DbTableHash* tb, Binary** mpp, /* Existing match program */ int lock_for_write, /* Set to 1 if we're going to delete or modify existing terms */ - match_callbacks_t* ctx, Eterm* ret) { + DbTableHash* tb = ctx->tb; HashDbTerm** current_ptr; /* Refers to either the bucket pointer or * the 'next' pointer in the previous term */ @@ -1410,7 +1417,7 @@ static int match_traverse_continue(Process* p, DbTableHash* tb, for(;;) { if (*current_ptr != NULL) { if (!is_pseudo_deleted(*current_ptr)) { - match_res = db_match_dbterm(&tb->common, p, *mpp, + match_res = db_match_dbterm(&tb->common, ctx->p, *mpp, &(*current_ptr)->dbterm, hpp, 2); saved_current = *current_ptr; if (ctx->on_match_res(ctx, slot_ix, ¤t_ptr, match_res)) { @@ -1433,11 +1440,7 @@ static int match_traverse_continue(Process* p, DbTableHash* tb, unlock_hash_function(lck); break; } - if (iterations_left <= 0 || MBUF(p)) { - /* - * We have either reached our limit, or just created some heap fragments. - * Since many heap fragments will make the GC slower, trap and GC now. - */ + if (iterations_left <= 0) { unlock_hash_function(lck); ret_value = ctx->on_trap(ctx, slot_ix, got, mpp, ret); goto done; @@ -1464,52 +1467,50 @@ done: */ static ERTS_INLINE int on_simple_trap(Export* trap_function, - Process* p, - DbTableHash* tb, - Eterm tid, - Eterm* prev_continuation_tptr, - Sint slot_ix, - Sint got, - Binary** mpp, - Eterm* ret) + traverse_context_t* ctx, + Sint slot_ix, + Sint got, + Binary** mpp, + Eterm* ret) { Eterm* hp; Eterm egot; Eterm mpb; Eterm continuation; - int is_first_trap = (prev_continuation_tptr == NULL); + int is_first_trap = (ctx->prev_continuation_tptr == NULL); size_t base_halloc_sz = (is_first_trap ? ERTS_MAGIC_REF_THING_SIZE : 0); - BUMP_ALL_REDS(p); + BUMP_ALL_REDS(ctx->p); if (IS_USMALL(0, got)) { - hp = HAllocX(p, base_halloc_sz + 5, ERTS_MAGIC_REF_THING_SIZE); + hp = HAllocX(ctx->p, base_halloc_sz + 6, ERTS_MAGIC_REF_THING_SIZE); egot = make_small(got); } else { - hp = HAllocX(p, base_halloc_sz + BIG_UINT_HEAP_SIZE + 5, + hp = HAllocX(ctx->p, base_halloc_sz + BIG_UINT_HEAP_SIZE + 6, ERTS_MAGIC_REF_THING_SIZE); egot = uint_to_big(got, hp); hp += BIG_UINT_HEAP_SIZE; } if (is_first_trap) { - if (is_atom(tid)) - tid = erts_db_make_tid(p, &tb->common); - mpb = erts_db_make_match_prog_ref(p, *mpp, &hp); + if (is_atom(ctx->tid)) + ctx->tid = erts_db_make_tid(ctx->p, &ctx->tb->common); + mpb = erts_db_make_match_prog_ref(ctx->p, *mpp, &hp); *mpp = NULL; /* otherwise the caller will destroy it */ } else { - ASSERT(!is_atom(tid)); - mpb = prev_continuation_tptr[3]; + ASSERT(!is_atom(ctx->tid)); + mpb = ctx->prev_continuation_tptr[3]; } - continuation = TUPLE4( + continuation = TUPLE5( hp, - tid, + ctx->tid, make_small(slot_ix), mpb, - egot); - ERTS_BIF_PREP_TRAP1(*ret, trap_function, p, continuation); + egot, + make_small(ctx->safety)); + ERTS_BIF_PREP_TRAP1(*ret, trap_function, ctx->p, continuation); return DB_ERROR_NONE; } @@ -1518,17 +1519,18 @@ static ERTS_INLINE int unpack_simple_continuation(Eterm continuation, Eterm* tid_ptr, Sint* slot_ix_p, Binary** mpp, - Sint* got_p) + Sint* got_p, + enum DbIterSafety* safety_p) { Eterm* tptr; ASSERT(is_tuple(continuation)); tptr = tuple_val(continuation); - if (arityval(*tptr) != 4) + if (*tptr != make_arityval(5)) return 1; - if (! is_small(tptr[2]) || !(is_big(tptr[4]) || is_small(tptr[4]))) { + if (!is_small(tptr[2]) || !(is_big(tptr[4]) || is_small(tptr[4])) + || !is_small(tptr[5])) return 1; - } *tptr_ptr = tptr; *tid_ptr = tptr[1]; @@ -1540,6 +1542,7 @@ static ERTS_INLINE int unpack_simple_continuation(Eterm continuation, else { *got_p = unsigned_val(tptr[4]); } + *safety_p = signed_val(tptr[5]); return 0; } @@ -1553,24 +1556,20 @@ static ERTS_INLINE int unpack_simple_continuation(Eterm continuation, #define MAX_SELECT_CHUNK_ITERATIONS 1000 typedef struct { - match_callbacks_t base; - Process* p; - DbTableHash* tb; - Eterm tid; + traverse_context_t base; Eterm* hp; Sint chunk_size; Eterm match_list; - Eterm* prev_continuation_tptr; } select_chunk_context_t; -static int select_chunk_on_nothing_can_match(match_callbacks_t* ctx_base, Eterm* ret) +static int select_chunk_on_nothing_can_match(traverse_context_t* ctx_base, Eterm* ret) { select_chunk_context_t* ctx = (select_chunk_context_t*) ctx_base; *ret = (ctx->chunk_size > 0 ? am_EOT : NIL); return DB_ERROR_NONE; } -static int select_chunk_on_match_res(match_callbacks_t* ctx_base, Sint slot_ix, +static int select_chunk_on_match_res(traverse_context_t* ctx_base, Sint slot_ix, HashDbTerm*** current_ptr_ptr, Eterm match_res) { @@ -1582,7 +1581,7 @@ static int select_chunk_on_match_res(match_callbacks_t* ctx_base, Sint slot_ix, return 0; } -static int select_chunk_on_loop_ended(match_callbacks_t* ctx_base, +static int select_chunk_on_loop_ended(traverse_context_t* ctx_base, Sint slot_ix, Sint got, Sint iterations_left, Binary** mpp, Eterm* ret) @@ -1598,7 +1597,7 @@ static int select_chunk_on_loop_ended(match_callbacks_t* ctx_base, } else { ASSERT(iterations_left < MAX_SELECT_CHUNK_ITERATIONS); - BUMP_REDS(ctx->p, MAX_SELECT_CHUNK_ITERATIONS - iterations_left); + BUMP_REDS(ctx->base.p, MAX_SELECT_CHUNK_ITERATIONS - iterations_left); if (ctx->chunk_size) { Eterm continuation; Eterm rest = NIL; @@ -1617,14 +1616,14 @@ static int select_chunk_on_loop_ended(match_callbacks_t* ctx_base, been in 'user space' */ } if (rest != NIL || slot_ix >= 0) { /* Need more calls */ - Eterm tid = ctx->tid; - ctx->hp = HAllocX(ctx->p, + Eterm tid = ctx->base.tid; + ctx->hp = HAllocX(ctx->base.p, 3 + 7 + ERTS_MAGIC_REF_THING_SIZE, ERTS_MAGIC_REF_THING_SIZE); - mpb = erts_db_make_match_prog_ref(ctx->p, *mpp, &ctx->hp); + mpb = erts_db_make_match_prog_ref(ctx->base.p, *mpp, &ctx->hp); if (is_atom(tid)) - tid = erts_db_make_tid(ctx->p, - &ctx->tb->common); + tid = erts_db_make_tid(ctx->base.p, + &ctx->base.tb->common); continuation = TUPLE6( ctx->hp, tid, @@ -1639,7 +1638,7 @@ static int select_chunk_on_loop_ended(match_callbacks_t* ctx_base, } else { /* All data is exhausted */ if (ctx->match_list != NIL) { /* No more data to search but still a result to return to the caller */ - ctx->hp = HAlloc(ctx->p, 3); + ctx->hp = HAlloc(ctx->base.p, 3); *ret = TUPLE2(ctx->hp, ctx->match_list, am_EOT); return DB_ERROR_NONE; } else { /* Reached the end of the ttable with no data to return */ @@ -1653,7 +1652,7 @@ static int select_chunk_on_loop_ended(match_callbacks_t* ctx_base, } } -static int select_chunk_on_trap(match_callbacks_t* ctx_base, +static int select_chunk_on_trap(traverse_context_t* ctx_base, Sint slot_ix, Sint got, Binary** mpp, Eterm* ret) { @@ -1662,74 +1661,77 @@ static int select_chunk_on_trap(match_callbacks_t* ctx_base, Eterm continuation; Eterm* hp; - BUMP_ALL_REDS(ctx->p); + BUMP_ALL_REDS(ctx->base.p); - if (ctx->prev_continuation_tptr == NULL) { - Eterm tid = ctx->tid; + if (ctx->base.prev_continuation_tptr == NULL) { + Eterm tid = ctx->base.tid; /* First time we're trapping */ - hp = HAllocX(ctx->p, 7 + ERTS_MAGIC_REF_THING_SIZE, + hp = HAllocX(ctx->base.p, 8 + ERTS_MAGIC_REF_THING_SIZE, ERTS_MAGIC_REF_THING_SIZE); if (is_atom(tid)) - tid = erts_db_make_tid(ctx->p, &ctx->tb->common); - mpb = erts_db_make_match_prog_ref(ctx->p, *mpp, &hp); - continuation = TUPLE6( + tid = erts_db_make_tid(ctx->base.p, &ctx->base.tb->common); + mpb = erts_db_make_match_prog_ref(ctx->base.p, *mpp, &hp); + continuation = TUPLE7( hp, tid, make_small(slot_ix), make_small(ctx->chunk_size), mpb, ctx->match_list, - make_small(got)); + make_small(got), + make_small(ctx->base.safety)); *mpp = NULL; /* otherwise the caller will destroy it */ } else { /* Not the first time we're trapping; reuse continuation terms */ - hp = HAlloc(ctx->p, 7); - continuation = TUPLE6( + hp = HAlloc(ctx->base.p, 8); + continuation = TUPLE7( hp, - ctx->prev_continuation_tptr[1], + ctx->base.prev_continuation_tptr[1], make_small(slot_ix), - ctx->prev_continuation_tptr[3], - ctx->prev_continuation_tptr[4], + ctx->base.prev_continuation_tptr[3], + ctx->base.prev_continuation_tptr[4], ctx->match_list, - make_small(got)); + make_small(got), + make_small(ctx->base.safety)); } - ERTS_BIF_PREP_TRAP1(*ret, &ets_select_continue_exp, ctx->p, + ERTS_BIF_PREP_TRAP1(*ret, &ets_select_continue_exp, ctx->base.p, continuation); return DB_ERROR_NONE; } static int db_select_hash(Process *p, DbTable *tbl, Eterm tid, Eterm pattern, - int reverse, Eterm *ret) + int reverse, Eterm *ret, enum DbIterSafety safety) { - return db_select_chunk_hash(p, tbl, tid, pattern, 0, reverse, ret); + return db_select_chunk_hash(p, tbl, tid, pattern, 0, reverse, ret, safety); } static int db_select_chunk_hash(Process *p, DbTable *tbl, Eterm tid, Eterm pattern, Sint chunk_size, - int reverse, Eterm *ret) + int reverse, Eterm *ret, enum DbIterSafety safety) { select_chunk_context_t ctx; ctx.base.on_nothing_can_match = select_chunk_on_nothing_can_match; ctx.base.on_match_res = select_chunk_on_match_res; ctx.base.on_loop_ended = select_chunk_on_loop_ended; - ctx.base.on_trap = select_chunk_on_trap, - ctx.p = p; - ctx.tb = &tbl->hash; - ctx.tid = tid; + ctx.base.on_trap = select_chunk_on_trap; + ctx.base.p = p; + ctx.base.tb = &tbl->hash; + ctx.base.tid = tid; + ctx.base.prev_continuation_tptr = NULL; + ctx.base.safety = safety; ctx.hp = NULL; ctx.chunk_size = chunk_size; ctx.match_list = NIL; - ctx.prev_continuation_tptr = NULL; return match_traverse( - ctx.p, ctx.tb, + &ctx.base, pattern, NULL, ctx.chunk_size, MAX_SELECT_CHUNK_ITERATIONS, &ctx.hp, 0, - &ctx.base, ret); + ret); } /* @@ -1739,7 +1741,7 @@ static int db_select_chunk_hash(Process *p, DbTable *tbl, Eterm tid, */ static -int select_chunk_continue_on_loop_ended(match_callbacks_t* ctx_base, +int select_chunk_continue_on_loop_ended(traverse_context_t* ctx_base, Sint slot_ix, Sint got, Sint iterations_left, Binary** mpp, Eterm* ret) @@ -1750,14 +1752,14 @@ int select_chunk_continue_on_loop_ended(match_callbacks_t* ctx_base, Eterm* hp; ASSERT(iterations_left <= MAX_SELECT_CHUNK_ITERATIONS); - BUMP_REDS(ctx->p, MAX_SELECT_CHUNK_ITERATIONS - iterations_left); + BUMP_REDS(ctx->base.p, MAX_SELECT_CHUNK_ITERATIONS - iterations_left); if (ctx->chunk_size) { Sint rest_size = 0; if (got > ctx->chunk_size) { /* Cannot write destructively here, the list may have been in user space */ - hp = HAlloc(ctx->p, (got - ctx->chunk_size) * 2); + hp = HAlloc(ctx->base.p, (got - ctx->chunk_size) * 2); while (got-- > ctx->chunk_size) { rest = CONS(hp, CAR(list_val(ctx->match_list)), rest); hp += 2; @@ -1766,13 +1768,13 @@ int select_chunk_continue_on_loop_ended(match_callbacks_t* ctx_base, } } if (rest != NIL || slot_ix >= 0) { - hp = HAlloc(ctx->p, 3 + 7); + hp = HAlloc(ctx->base.p, 3 + 7); continuation = TUPLE6( hp, - ctx->prev_continuation_tptr[1], + ctx->base.prev_continuation_tptr[1], make_small(slot_ix), - ctx->prev_continuation_tptr[3], - ctx->prev_continuation_tptr[4], + ctx->base.prev_continuation_tptr[3], + ctx->base.prev_continuation_tptr[4], rest, make_small(rest_size)); hp += 7; @@ -1780,7 +1782,7 @@ int select_chunk_continue_on_loop_ended(match_callbacks_t* ctx_base, return DB_ERROR_NONE; } else { if (ctx->match_list != NIL) { - hp = HAlloc(ctx->p, 3); + hp = HAlloc(ctx->base.p, 3); *ret = TUPLE2(hp, ctx->match_list, am_EOT); return DB_ERROR_NONE; } else { @@ -1794,10 +1796,11 @@ int select_chunk_continue_on_loop_ended(match_callbacks_t* ctx_base, } /* - * This is called when select traps + * This is called when ets:select/1/2/3 traps + * and for ets:select/1 with user continuation term. */ static int db_select_continue_hash(Process* p, DbTable* tbl, Eterm continuation, - Eterm* ret) + Eterm* ret, enum DbIterSafety* safety_p) { select_chunk_context_t ctx; Eterm* tptr; @@ -1813,7 +1816,13 @@ static int db_select_continue_hash(Process* p, DbTable* tbl, Eterm continuation, ASSERT(is_tuple(continuation)); tptr = tuple_val(continuation); - if (arityval(*tptr) != 6) + /* + * 6-tuple is select/1 user continuation term + * 7-tuple is select trap continuation + */ + if (*tptr == make_arityval(7) && is_small(tptr[7])) + *safety_p = signed_val(tptr[7]); + else if (*tptr != make_arityval(6)) goto badparam; if (!is_small(tptr[2]) || !is_small(tptr[3]) || @@ -1837,18 +1846,19 @@ static int db_select_continue_hash(Process* p, DbTable* tbl, Eterm continuation, ctx.base.on_match_res = select_chunk_on_match_res; ctx.base.on_loop_ended = select_chunk_continue_on_loop_ended; ctx.base.on_trap = select_chunk_on_trap; - ctx.p = p; - ctx.tb = &tbl->hash; - ctx.tid = tid; + ctx.base.p = p; + ctx.base.tb = &tbl->hash; + ctx.base.tid = tid; + ctx.base.prev_continuation_tptr = tptr; + ctx.base.safety = *safety_p; ctx.hp = NULL; ctx.chunk_size = chunk_size; ctx.match_list = match_list; - ctx.prev_continuation_tptr = tptr; return match_traverse_continue( - ctx.p, ctx.tb, ctx.chunk_size, - iterations_left, &ctx.hp, slot_ix, got, &mp, 0, - &ctx.base, ret); + &ctx.base, ctx.chunk_size, + iterations_left, &ctx.hp, slot_ix, got, &mp, 0, + ret); badparam: *ret = NIL; @@ -1866,84 +1876,73 @@ badparam: #define MAX_SELECT_COUNT_ITERATIONS 1000 -typedef struct { - match_callbacks_t base; - Process* p; - DbTableHash* tb; - Eterm tid; - Eterm* prev_continuation_tptr; -} select_count_context_t; - -static int select_count_on_nothing_can_match(match_callbacks_t* ctx_base, +static int select_count_on_nothing_can_match(traverse_context_t* ctx_base, Eterm* ret) { *ret = make_small(0); return DB_ERROR_NONE; } -static int select_count_on_match_res(match_callbacks_t* ctx_base, Sint slot_ix, +static int select_count_on_match_res(traverse_context_t* ctx_base, Sint slot_ix, HashDbTerm*** current_ptr_ptr, Eterm match_res) { return (match_res == am_true); } -static int select_count_on_loop_ended(match_callbacks_t* ctx_base, +static int select_count_on_loop_ended(traverse_context_t* ctx, Sint slot_ix, Sint got, Sint iterations_left, Binary** mpp, Eterm* ret) { - select_count_context_t* ctx = (select_count_context_t*) ctx_base; ASSERT(iterations_left <= MAX_SELECT_COUNT_ITERATIONS); BUMP_REDS(ctx->p, MAX_SELECT_COUNT_ITERATIONS - iterations_left); *ret = erts_make_integer(got, ctx->p); return DB_ERROR_NONE; } -static int select_count_on_trap(match_callbacks_t* ctx_base, +static int select_count_on_trap(traverse_context_t* ctx, Sint slot_ix, Sint got, Binary** mpp, Eterm* ret) { - select_count_context_t* ctx = (select_count_context_t*) ctx_base; return on_simple_trap( - &ets_select_count_continue_exp, - ctx->p, - ctx->tb, - ctx->tid, - ctx->prev_continuation_tptr, + &ets_select_count_continue_exp, ctx, slot_ix, got, mpp, ret); } static int db_select_count_hash(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret) + Eterm pattern, Eterm *ret, + enum DbIterSafety safety) { - select_count_context_t ctx; + traverse_context_t ctx; Sint iterations_left = MAX_SELECT_COUNT_ITERATIONS; Sint chunk_size = 0; - ctx.base.on_nothing_can_match = select_count_on_nothing_can_match; - ctx.base.on_match_res = select_count_on_match_res; - ctx.base.on_loop_ended = select_count_on_loop_ended; - ctx.base.on_trap = select_count_on_trap; + ctx.on_nothing_can_match = select_count_on_nothing_can_match; + ctx.on_match_res = select_count_on_match_res; + ctx.on_loop_ended = select_count_on_loop_ended; + ctx.on_trap = select_count_on_trap; ctx.p = p; ctx.tb = &tbl->hash; ctx.tid = tid; ctx.prev_continuation_tptr = NULL; + ctx.safety = safety; return match_traverse( - ctx.p, ctx.tb, + &ctx, pattern, NULL, chunk_size, iterations_left, NULL, 0, - &ctx.base, ret); + ret); } /* * This is called when select_count traps */ static int db_select_count_continue_hash(Process* p, DbTable* tbl, - Eterm continuation, Eterm* ret) + Eterm continuation, Eterm* ret, + enum DbIterSafety* safety_p) { - select_count_context_t ctx; + traverse_context_t ctx; Eterm* tptr; Eterm tid; Binary* mp; @@ -1952,24 +1951,26 @@ static int db_select_count_continue_hash(Process* p, DbTable* tbl, Sint chunk_size = 0; *ret = NIL; - if (unpack_simple_continuation(continuation, &tptr, &tid, &slot_ix, &mp, &got)) { + if (unpack_simple_continuation(continuation, &tptr, &tid, &slot_ix, &mp, + &got, safety_p)) { *ret = NIL; return DB_ERROR_BADPARAM; } - ctx.base.on_match_res = select_count_on_match_res; - ctx.base.on_loop_ended = select_count_on_loop_ended; - ctx.base.on_trap = select_count_on_trap; + ctx.on_match_res = select_count_on_match_res; + ctx.on_loop_ended = select_count_on_loop_ended; + ctx.on_trap = select_count_on_trap; ctx.p = p; ctx.tb = &tbl->hash; ctx.tid = tid; ctx.prev_continuation_tptr = tptr; + ctx.safety = *safety_p; return match_traverse_continue( - ctx.p, ctx.tb, chunk_size, + &ctx, chunk_size, MAX_SELECT_COUNT_ITERATIONS, NULL, slot_ix, got, &mp, 0, - &ctx.base, ret); + ret); } #undef MAX_SELECT_COUNT_ITERATIONS @@ -1984,24 +1985,20 @@ static int db_select_count_continue_hash(Process* p, DbTable* tbl, #define MAX_SELECT_DELETE_ITERATIONS 1000 typedef struct { - match_callbacks_t base; - Process* p; - DbTableHash* tb; - Eterm tid; - Eterm* prev_continuation_tptr; + traverse_context_t base; erts_aint_t fixated_by_me; Uint last_pseudo_delete; HashDbTerm* free_us; } select_delete_context_t; -static int select_delete_on_nothing_can_match(match_callbacks_t* ctx_base, +static int select_delete_on_nothing_can_match(traverse_context_t* ctx_base, Eterm* ret) { *ret = make_small(0); return DB_ERROR_NONE; } -static int select_delete_on_match_res(match_callbacks_t* ctx_base, Sint slot_ix, +static int select_delete_on_match_res(traverse_context_t* ctx_base, Sint slot_ix, HashDbTerm*** current_ptr_ptr, Eterm match_res) { @@ -2011,9 +2008,9 @@ static int select_delete_on_match_res(match_callbacks_t* ctx_base, Sint slot_ix, if (match_res != am_true) return 0; - if (NFIXED(ctx->tb) > ctx->fixated_by_me) { /* fixated by others? */ + if (NFIXED(ctx->base.tb) > ctx->fixated_by_me) { /* fixated by others? */ if (slot_ix != ctx->last_pseudo_delete) { - if (!add_fixed_deletion(ctx->tb, slot_ix, ctx->fixated_by_me)) + if (!add_fixed_deletion(ctx->base.tb, slot_ix, ctx->fixated_by_me)) goto do_erase; ctx->last_pseudo_delete = slot_ix; } @@ -2026,46 +2023,43 @@ static int select_delete_on_match_res(match_callbacks_t* ctx_base, Sint slot_ix, del->next = ctx->free_us; ctx->free_us = del; } - erts_atomic_dec_nob(&ctx->tb->common.nitems); + erts_atomic_dec_nob(&ctx->base.tb->common.nitems); return 1; } -static int select_delete_on_loop_ended(match_callbacks_t* ctx_base, +static int select_delete_on_loop_ended(traverse_context_t* ctx_base, Sint slot_ix, Sint got, Sint iterations_left, Binary** mpp, Eterm* ret) { select_delete_context_t* ctx = (select_delete_context_t*) ctx_base; - free_term_list(ctx->tb, ctx->free_us); + free_term_list(ctx->base.tb, ctx->free_us); ctx->free_us = NULL; ASSERT(iterations_left <= MAX_SELECT_DELETE_ITERATIONS); - BUMP_REDS(ctx->p, MAX_SELECT_DELETE_ITERATIONS - iterations_left); + BUMP_REDS(ctx->base.p, MAX_SELECT_DELETE_ITERATIONS - iterations_left); if (got) { - try_shrink(ctx->tb); + try_shrink(ctx->base.tb); } - *ret = erts_make_integer(got, ctx->p); + *ret = erts_make_integer(got, ctx->base.p); return DB_ERROR_NONE; } -static int select_delete_on_trap(match_callbacks_t* ctx_base, +static int select_delete_on_trap(traverse_context_t* ctx_base, Sint slot_ix, Sint got, Binary** mpp, Eterm* ret) { select_delete_context_t* ctx = (select_delete_context_t*) ctx_base; - free_term_list(ctx->tb, ctx->free_us); + free_term_list(ctx->base.tb, ctx->free_us); ctx->free_us = NULL; return on_simple_trap( - &ets_select_delete_continue_exp, - ctx->p, - ctx->tb, - ctx->tid, - ctx->prev_continuation_tptr, + &ets_select_delete_continue_exp, &ctx->base, slot_ix, got, mpp, ret); } static int db_select_delete_hash(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret) + Eterm pattern, Eterm *ret, + enum DbIterSafety safety) { select_delete_context_t ctx; Sint chunk_size = 0; @@ -2074,27 +2068,29 @@ static int db_select_delete_hash(Process *p, DbTable *tbl, Eterm tid, ctx.base.on_match_res = select_delete_on_match_res; ctx.base.on_loop_ended = select_delete_on_loop_ended; ctx.base.on_trap = select_delete_on_trap; - ctx.p = p; - ctx.tb = &tbl->hash; - ctx.tid = tid; - ctx.prev_continuation_tptr = NULL; - ctx.fixated_by_me = ctx.tb->common.is_thread_safe ? 0 : 1; /* TODO: something nicer */ + ctx.base.p = p; + ctx.base.tb = &tbl->hash; + ctx.base.tid = tid; + ctx.base.prev_continuation_tptr = NULL; + ctx.base.safety = safety; + ctx.fixated_by_me = ctx.base.tb->common.is_thread_safe ? 0 : 1; ctx.last_pseudo_delete = (Uint) -1; ctx.free_us = NULL; return match_traverse( - ctx.p, ctx.tb, + &ctx.base, pattern, NULL, chunk_size, MAX_SELECT_DELETE_ITERATIONS, NULL, 1, - &ctx.base, ret); + ret); } /* * This is called when select_delete traps */ static int db_select_delete_continue_hash(Process* p, DbTable* tbl, - Eterm continuation, Eterm* ret) + Eterm continuation, Eterm* ret, + enum DbIterSafety* safety_p) { select_delete_context_t ctx; Eterm* tptr; @@ -2104,7 +2100,8 @@ static int db_select_delete_continue_hash(Process* p, DbTable* tbl, Sint slot_ix; Sint chunk_size = 0; - if (unpack_simple_continuation(continuation, &tptr, &tid, &slot_ix, &mp, &got)) { + if (unpack_simple_continuation(continuation, &tptr, &tid, &slot_ix, &mp, + &got, safety_p)) { *ret = NIL; return DB_ERROR_BADPARAM; } @@ -2112,19 +2109,20 @@ static int db_select_delete_continue_hash(Process* p, DbTable* tbl, ctx.base.on_match_res = select_delete_on_match_res; ctx.base.on_loop_ended = select_delete_on_loop_ended; ctx.base.on_trap = select_delete_on_trap; - ctx.p = p; - ctx.tb = &tbl->hash; - ctx.tid = tid; - ctx.prev_continuation_tptr = tptr; - ctx.fixated_by_me = ONLY_WRITER(p, ctx.tb) ? 0 : 1; /* TODO: something nicer */ + ctx.base.p = p; + ctx.base.tb = &tbl->hash; + ctx.base.tid = tid; + ctx.base.prev_continuation_tptr = tptr; + ctx.base.safety = *safety_p; + ctx.fixated_by_me = ONLY_WRITER(p, ctx.base.tb) ? 0 : 1; ctx.last_pseudo_delete = (Uint) -1; ctx.free_us = NULL; return match_traverse_continue( - ctx.p, ctx.tb, chunk_size, + &ctx.base, chunk_size, MAX_SELECT_DELETE_ITERATIONS, NULL, slot_ix, got, &mp, 1, - &ctx.base, ret); + ret); } #undef MAX_SELECT_DELETE_ITERATIONS @@ -2138,26 +2136,17 @@ static int db_select_delete_continue_hash(Process* p, DbTable* tbl, #define MAX_SELECT_REPLACE_ITERATIONS 1000 -typedef struct { - match_callbacks_t base; - Process* p; - DbTableHash* tb; - Eterm tid; - Eterm* prev_continuation_tptr; -} select_replace_context_t; - -static int select_replace_on_nothing_can_match(match_callbacks_t* ctx_base, +static int select_replace_on_nothing_can_match(traverse_context_t* ctx_base, Eterm* ret) { *ret = make_small(0); return DB_ERROR_NONE; } -static int select_replace_on_match_res(match_callbacks_t* ctx_base, Sint slot_ix, +static int select_replace_on_match_res(traverse_context_t* ctx, Sint slot_ix, HashDbTerm*** current_ptr_ptr, Eterm match_res) { - select_replace_context_t* ctx = (select_replace_context_t*) ctx_base; DbTableHash* tb = ctx->tb; HashDbTerm* new; HashDbTerm* next; @@ -2183,11 +2172,10 @@ static int select_replace_on_match_res(match_callbacks_t* ctx_base, Sint slot_ix return 0; } -static int select_replace_on_loop_ended(match_callbacks_t* ctx_base, Sint slot_ix, +static int select_replace_on_loop_ended(traverse_context_t* ctx, Sint slot_ix, Sint got, Sint iterations_left, Binary** mpp, Eterm* ret) { - select_replace_context_t* ctx = (select_replace_context_t*) ctx_base; ASSERT(iterations_left <= MAX_SELECT_REPLACE_ITERATIONS); /* the more objects we've replaced, the more reductions we've consumed */ BUMP_REDS(ctx->p, @@ -2197,23 +2185,20 @@ static int select_replace_on_loop_ended(match_callbacks_t* ctx_base, Sint slot_i return DB_ERROR_NONE; } -static int select_replace_on_trap(match_callbacks_t* ctx_base, +static int select_replace_on_trap(traverse_context_t* ctx, Sint slot_ix, Sint got, Binary** mpp, Eterm* ret) { - select_replace_context_t* ctx = (select_replace_context_t*) ctx_base; return on_simple_trap( - &ets_select_replace_continue_exp, - ctx->p, - ctx->tb, - ctx->tid, - ctx->prev_continuation_tptr, + &ets_select_replace_continue_exp, ctx, slot_ix, got, mpp, ret); } -static int db_select_replace_hash(Process *p, DbTable *tbl, Eterm tid, Eterm pattern, Eterm *ret) +static int db_select_replace_hash(Process *p, DbTable *tbl, Eterm tid, + Eterm pattern, Eterm *ret, + enum DbIterSafety safety) { - select_replace_context_t ctx; + traverse_context_t ctx; Sint chunk_size = 0; /* Bag implementation presented both semantic consistency and performance issues, @@ -2221,29 +2206,32 @@ static int db_select_replace_hash(Process *p, DbTable *tbl, Eterm tid, Eterm pat */ ASSERT(!(tbl->hash.common.status & DB_BAG)); - ctx.base.on_nothing_can_match = select_replace_on_nothing_can_match; - ctx.base.on_match_res = select_replace_on_match_res; - ctx.base.on_loop_ended = select_replace_on_loop_ended; - ctx.base.on_trap = select_replace_on_trap; + ctx.on_nothing_can_match = select_replace_on_nothing_can_match; + ctx.on_match_res = select_replace_on_match_res; + ctx.on_loop_ended = select_replace_on_loop_ended; + ctx.on_trap = select_replace_on_trap; ctx.p = p; ctx.tb = &tbl->hash; ctx.tid = tid; ctx.prev_continuation_tptr = NULL; + ctx.safety = safety; return match_traverse( - ctx.p, ctx.tb, + &ctx, pattern, db_match_keeps_key, chunk_size, MAX_SELECT_REPLACE_ITERATIONS, NULL, 1, - &ctx.base, ret); + ret); } /* * This is called when select_replace traps */ -static int db_select_replace_continue_hash(Process* p, DbTable* tbl, Eterm continuation, Eterm* ret) +static int db_select_replace_continue_hash(Process* p, DbTable* tbl, + Eterm continuation, Eterm* ret, + enum DbIterSafety* safety_p) { - select_replace_context_t ctx; + traverse_context_t ctx; Eterm* tptr; Eterm tid ; Binary* mp; @@ -2252,25 +2240,27 @@ static int db_select_replace_continue_hash(Process* p, DbTable* tbl, Eterm conti Sint chunk_size = 0; *ret = NIL; - if (unpack_simple_continuation(continuation, &tptr, &tid, &slot_ix, &mp, &got)) { + if (unpack_simple_continuation(continuation, &tptr, &tid, &slot_ix, &mp, + &got, safety_p)) { *ret = NIL; return DB_ERROR_BADPARAM; } /* Proceed */ - ctx.base.on_match_res = select_replace_on_match_res; - ctx.base.on_loop_ended = select_replace_on_loop_ended; - ctx.base.on_trap = select_replace_on_trap; + ctx.on_match_res = select_replace_on_match_res; + ctx.on_loop_ended = select_replace_on_loop_ended; + ctx.on_trap = select_replace_on_trap; ctx.p = p; ctx.tb = &tbl->hash; ctx.tid = tid; ctx.prev_continuation_tptr = tptr; + ctx.safety = *safety_p; return match_traverse_continue( - ctx.p, ctx.tb, chunk_size, + &ctx, chunk_size, MAX_SELECT_REPLACE_ITERATIONS, NULL, slot_ix, got, &mp, 1, - &ctx.base, ret); + ret); } diff --git a/erts/emulator/beam/erl_db_tree.c b/erts/emulator/beam/erl_db_tree.c index 02a5934a6e..f9ba04f399 100644 --- a/erts/emulator/beam/erl_db_tree.c +++ b/erts/emulator/beam/erl_db_tree.c @@ -397,24 +397,31 @@ static int db_erase_object_tree(DbTable *tbl, Eterm object,Eterm *ret); static int db_slot_tree(Process *p, DbTable *tbl, Eterm slot_term, Eterm *ret); static int db_select_tree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, int reversed, Eterm *ret); + Eterm pattern, int reversed, Eterm *ret, + enum DbIterSafety); static int db_select_count_tree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret); + Eterm pattern, Eterm *ret, enum DbIterSafety); static int db_select_chunk_tree(Process *p, DbTable *tbl, Eterm tid, Eterm pattern, Sint chunk_size, - int reversed, Eterm *ret); + int reversed, Eterm *ret, enum DbIterSafety); static int db_select_continue_tree(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_select_count_continue_tree(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_select_delete_tree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret); + Eterm pattern, Eterm *ret, + enum DbIterSafety); static int db_select_delete_continue_tree(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_select_replace_tree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret); + Eterm pattern, Eterm *ret, + enum DbIterSafety); static int db_select_replace_continue_tree(Process *p, DbTable *tbl, - Eterm continuation, Eterm *ret); + Eterm continuation, Eterm *ret, + enum DbIterSafety*); static int db_take_tree(Process *, DbTable *, Eterm, Eterm *); static void db_print_tree(fmtfn_t to, void *to_arg, int show, DbTable *tbl); @@ -1160,7 +1167,8 @@ int db_select_continue_tree_common(Process *p, static int db_select_continue_tree(Process *p, DbTable *tbl, Eterm continuation, - Eterm *ret) + Eterm *ret, + enum DbIterSafety* safety_p) { DbTableTree *tb = &tbl->tree; return db_select_continue_tree_common(p, &tb->common, @@ -1297,7 +1305,8 @@ int db_select_tree_common(Process *p, DbTable *tb, } static int db_select_tree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, int reverse, Eterm *ret) + Eterm pattern, int reverse, Eterm *ret, + enum DbIterSafety safety) { return db_select_tree_common(p, tbl, tid, pattern, reverse, ret, &tbl->tree, NULL); @@ -1408,7 +1417,8 @@ int db_select_count_continue_tree_common(Process *p, static int db_select_count_continue_tree(Process *p, DbTable *tbl, Eterm continuation, - Eterm *ret) + Eterm *ret, + enum DbIterSafety* safety_p) { DbTableTree *tb = &tbl->tree; return db_select_count_continue_tree_common(p, tbl, @@ -1527,7 +1537,8 @@ int db_select_count_tree_common(Process *p, DbTable *tb, } static int db_select_count_tree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret) + Eterm pattern, Eterm *ret, + enum DbIterSafety safety) { DbTableTree *tb = &tbl->tree; return db_select_count_tree_common(p, tbl, @@ -1704,7 +1715,7 @@ int db_select_chunk_tree_common(Process *p, DbTable *tb, static int db_select_chunk_tree(Process *p, DbTable *tbl, Eterm tid, Eterm pattern, Sint chunk_size, int reverse, - Eterm *ret) + Eterm *ret, enum DbIterSafety safety) { DbTableTree *tb = &tbl->tree; return db_select_chunk_tree_common(p, tbl, @@ -1813,7 +1824,8 @@ int db_select_delete_continue_tree_common(Process *p, static int db_select_delete_continue_tree(Process *p, DbTable *tbl, Eterm continuation, - Eterm *ret) + Eterm *ret, + enum DbIterSafety* safety_p) { DbTableTree *tb = &tbl->tree; ASSERT(!erts_atomic_read_nob(&tb->is_stack_busy)); @@ -1942,7 +1954,8 @@ int db_select_delete_tree_common(Process *p, DbTable *tbl, } static int db_select_delete_tree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret) + Eterm pattern, Eterm *ret, + enum DbIterSafety safety) { DbTableTree *tb = &tbl->tree; return db_select_delete_tree_common(p, tbl, tid, pattern, ret, @@ -2052,7 +2065,8 @@ int db_select_replace_continue_tree_common(Process *p, static int db_select_replace_continue_tree(Process *p, DbTable *tbl, Eterm continuation, - Eterm *ret) + Eterm *ret, + enum DbIterSafety* safety_p) { return db_select_replace_continue_tree_common(p, tbl, continuation, ret, &tbl->tree, NULL); @@ -2177,7 +2191,8 @@ int db_select_replace_tree_common(Process *p, DbTable *tbl, } static int db_select_replace_tree(Process *p, DbTable *tbl, Eterm tid, - Eterm pattern, Eterm *ret) + Eterm pattern, Eterm *ret, + enum DbIterSafety safety) { return db_select_replace_tree_common(p, tbl, tid, pattern, ret, &tbl->tree, NULL); @@ -3743,13 +3758,6 @@ static int doit_select(DbTableCommon *tb, TreeDbTerm *this, if (is_value(ret)) { sc->accum = CONS(hp, ret, sc->accum); } - if (MBUF(sc->p)) { - /* - * Force a trap and GC if a heap fragment was created. Many heap fragments - * make the GC slow. - */ - sc->max = 0; - } if (--(sc->max) <= 0) { return 0; } @@ -3806,13 +3814,6 @@ static int doit_select_chunk(DbTableCommon *tb, TreeDbTerm *this, ++(sc->got); sc->accum = CONS(hp, ret, sc->accum); } - if (MBUF(sc->p)) { - /* - * Force a trap and GC if a heap fragment was created. Many heap fragments - * make the GC slow. - */ - sc->max = 0; - } if (--(sc->max) <= 0 || sc->got == sc->chunk_size) { return 0; } diff --git a/erts/emulator/beam/erl_db_util.h b/erts/emulator/beam/erl_db_util.h index e1af9210ea..e3d3c0e804 100644 --- a/erts/emulator/beam/erl_db_util.h +++ b/erts/emulator/beam/erl_db_util.h @@ -101,6 +101,14 @@ typedef struct { } u; } DbUpdateHandle; +/* How safe are we from double-hits or missed objects + * when iterating without fixation? + */ +enum DbIterSafety { + ITER_UNSAFE, /* Must fixate to be safe */ + ITER_SAFE_LOCKED, /* Safe while table is locked, not between trap calls */ + ITER_SAFE /* No need to fixate at all */ +}; typedef struct db_table_method { @@ -150,44 +158,53 @@ typedef struct db_table_method Eterm pattern, Sint chunk_size, int reverse, - Eterm* ret); + Eterm* ret, + enum DbIterSafety); int (*db_select)(Process* p, DbTable* tb, /* [in out] */ Eterm tid, Eterm pattern, int reverse, - Eterm* ret); + Eterm* ret, + enum DbIterSafety); int (*db_select_delete)(Process* p, DbTable* tb, /* [in out] */ Eterm tid, Eterm pattern, - Eterm* ret); + Eterm* ret, + enum DbIterSafety); int (*db_select_continue)(Process* p, DbTable* tb, /* [in out] */ Eterm continuation, - Eterm* ret); + Eterm* ret, + enum DbIterSafety*); int (*db_select_delete_continue)(Process* p, DbTable* tb, /* [in out] */ Eterm continuation, - Eterm* ret); + Eterm* ret, + enum DbIterSafety*); int (*db_select_count)(Process* p, DbTable* tb, /* [in out] */ Eterm tid, Eterm pattern, - Eterm* ret); + Eterm* ret, + enum DbIterSafety); int (*db_select_count_continue)(Process* p, DbTable* tb, /* [in out] */ Eterm continuation, - Eterm* ret); + Eterm* ret, + enum DbIterSafety*); int (*db_select_replace)(Process* p, DbTable* tb, /* [in out] */ Eterm tid, Eterm pattern, - Eterm* ret); + Eterm* ret, + enum DbIterSafety); int (*db_select_replace_continue)(Process* p, DbTable* tb, /* [in out] */ Eterm continuation, - Eterm* ret); + Eterm* ret, + enum DbIterSafety*); int (*db_take)(Process *, DbTable *, Eterm, Eterm *); SWord (*db_delete_all_objects)(Process* p, DbTable* db, SWord reds); diff --git a/erts/emulator/beam/erl_message.c b/erts/emulator/beam/erl_message.c index 9d40754d2d..e350a20339 100644 --- a/erts/emulator/beam/erl_message.c +++ b/erts/emulator/beam/erl_message.c @@ -776,6 +776,13 @@ erts_send_message(Process* sender, #endif erts_queue_proc_message(sender, receiver, *receiver_locks, mp, message); + + if (msize > ERTS_MSG_COPY_WORDS_PER_REDUCTION) { + Uint reds = msize / ERTS_MSG_COPY_WORDS_PER_REDUCTION; + if (reds > CONTEXT_REDS) + reds = CONTEXT_REDS; + BUMP_REDS(sender, (int) reds); + } } diff --git a/erts/emulator/beam/erl_message.h b/erts/emulator/beam/erl_message.h index 4c2674394e..e5f623a370 100644 --- a/erts/emulator/beam/erl_message.h +++ b/erts/emulator/beam/erl_message.h @@ -26,6 +26,12 @@ #include "erl_proc_sig_queue.h" #undef ERTS_PROC_SIG_QUEUE_TYPE_ONLY +#ifdef DEBUG +#define ERTS_MSG_COPY_WORDS_PER_REDUCTION 4 +#else +#define ERTS_MSG_COPY_WORDS_PER_REDUCTION 64 +#endif + struct proc_bin; struct external_thing_; diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index 349d9bf13a..af1acbfc90 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -731,6 +731,23 @@ ErtsMessage* erts_create_message_from_nif_env(ErlNifEnv* msg_env) return mp; } +static ERTS_INLINE ERL_NIF_TERM make_copy(ErlNifEnv* dst_env, + ERL_NIF_TERM src_term, + Uint *cpy_szp) +{ + Uint sz; + Eterm* hp; + /* + * No preserved sharing allowed as long as literals are also preserved. + * Process independent environment can not be reached by purge. + */ + sz = size_object(src_term); + if (cpy_szp) + *cpy_szp += sz; + hp = alloc_heap(dst_env, sz); + return copy_struct(src_term, sz, &hp, &MSO(dst_env->proc)); +} + int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, ErlNifEnv* msg_env, ERL_NIF_TERM msg) { @@ -743,6 +760,7 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, Eterm from; Eterm receiver = to_pid->pid; int scheduler; + Uint copy_sz = 0; execution_state(env, &c_p, &scheduler); @@ -806,14 +824,14 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, stoken = NIL; } #endif - token = enif_make_copy(msg_env, stoken); + token = make_copy(msg_env, stoken, ©_sz); #ifdef USE_VM_PROBES if (DT_UTAG_FLAGS(c_p) & DT_UTAG_SPREADING) { if (is_immed(DT_UTAG(c_p))) utag = DT_UTAG(c_p); else - utag = enif_make_copy(msg_env, DT_UTAG(c_p)); + utag = make_copy(msg_env, DT_UTAG(c_p), ©_sz); } if (DTRACE_ENABLED(message_send)) { if (have_seqtrace(stoken)) { @@ -835,6 +853,7 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, Uint sz; INITIALIZE_LITERAL_PURGE_AREA(litarea); sz = size_object_litopt(msg, &litarea); + copy_sz += sz; if (c_p && !env->tracee) { full_flush_env(env); mp = erts_alloc_message_heap(rp, &rp_locks, sz, &hp, &ohp); @@ -867,6 +886,12 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, trace_send(c_p, receiver, msg); full_cache_env(env); } + if (c_p && scheduler > 0 && copy_sz > ERTS_MSG_COPY_WORDS_PER_REDUCTION) { + Uint reds = copy_sz / ERTS_MSG_COPY_WORDS_PER_REDUCTION; + if (reds > CONTEXT_REDS) + reds = CONTEXT_REDS; + BUMP_REDS(c_p, (int) reds); + } } else { /* This clause is taken when the nif is called in the context @@ -935,6 +960,7 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, erts_queue_message(rp, rp_locks, mp, msg, from); done: + if (c_p == rp) rp_locks &= ~ERTS_PROC_LOCK_MAIN; if (rp_locks & ~lc_locks) @@ -1047,18 +1073,9 @@ int enif_whereis_port(ErlNifEnv *env, ERL_NIF_TERM name, ErlNifPort *port) ERL_NIF_TERM enif_make_copy(ErlNifEnv* dst_env, ERL_NIF_TERM src_term) { - Uint sz; - Eterm* hp; - /* - * No preserved sharing allowed as long as literals are also preserved. - * Process independent environment cannot be reached by purge. - */ - sz = size_object(src_term); - hp = alloc_heap(dst_env, sz); - return copy_struct(src_term, sz, &hp, &MSO(dst_env->proc)); + return make_copy(dst_env, src_term, NULL); } - #ifdef DEBUG static int is_offheap(const ErlOffHeap* oh) { @@ -1158,6 +1175,47 @@ int enif_is_number(ErlNifEnv* env, ERL_NIF_TERM term) return is_number(term); } +ErlNifTermType enif_term_type(ErlNifEnv* env, ERL_NIF_TERM term) { + (void)env; + + switch (tag_val_def(term)) { + case ATOM_DEF: + return ERL_NIF_TERM_TYPE_ATOM; + case BINARY_DEF: + return ERL_NIF_TERM_TYPE_BITSTRING; + case FLOAT_DEF: + return ERL_NIF_TERM_TYPE_FLOAT; + case EXPORT_DEF: + case FUN_DEF: + return ERL_NIF_TERM_TYPE_FUN; + case BIG_DEF: + case SMALL_DEF: + return ERL_NIF_TERM_TYPE_INTEGER; + case LIST_DEF: + case NIL_DEF: + return ERL_NIF_TERM_TYPE_LIST; + case MAP_DEF: + return ERL_NIF_TERM_TYPE_MAP; + case EXTERNAL_PID_DEF: + case PID_DEF: + return ERL_NIF_TERM_TYPE_PID; + case EXTERNAL_PORT_DEF: + case PORT_DEF: + return ERL_NIF_TERM_TYPE_PORT; + case EXTERNAL_REF_DEF: + case REF_DEF: + return ERL_NIF_TERM_TYPE_REFERENCE; + case TUPLE_DEF: + return ERL_NIF_TERM_TYPE_TUPLE; + default: + /* tag_val_def() aborts on its own when passed complete garbage, but + * it's possible that the user has given us garbage that just happens + * to match something that tag_val_def() accepts but we don't, like + * binary match contexts. */ + ERTS_INTERNAL_ERROR("Invalid term passed to enif_term_type"); + } +} + static void aligned_binary_dtor(struct enif_tmp_obj_t* obj) { erts_free_aligned_binary_bytes_extra((byte*)obj, obj->allocator); diff --git a/erts/emulator/beam/erl_nif.h b/erts/emulator/beam/erl_nif.h index 3fd1a8fd4c..a599511c78 100644 --- a/erts/emulator/beam/erl_nif.h +++ b/erts/emulator/beam/erl_nif.h @@ -55,6 +55,7 @@ ** 2.14: 21.0 add enif_ioq_peek_head, enif_(mutex|cond|rwlock|thread)_name ** enif_vfprintf, enif_vsnprintf, enif_make_map_from_arrays ** 2.15: 22.0 ERL_NIF_SELECT_CANCEL, enif_select_(read|write) +** enif_term_type */ #define ERL_NIF_MAJOR_VERSION 2 #define ERL_NIF_MINOR_VERSION 15 @@ -63,7 +64,7 @@ * with ticket syntax like "erts-@OTP-12345@", or a temporary placeholder * between two @ like "erts-@MyName@", if you don't know what a ticket is. */ -#define ERL_NIF_MIN_ERTS_VERSION "erts-@OTP-15095@ (OTP-22)" +#define ERL_NIF_MIN_ERTS_VERSION "erts-@OTP-15095 OTP-15640@ (OTP-22)" /* * The emulator will refuse to load a nif-lib with a major version @@ -282,6 +283,26 @@ typedef enum { ERL_NIF_IOQ_NORMAL = 1 } ErlNifIOQueueOpts; +typedef enum { + ERL_NIF_TERM_TYPE_ATOM = 1, + ERL_NIF_TERM_TYPE_BITSTRING = 2, + ERL_NIF_TERM_TYPE_FLOAT = 3, + ERL_NIF_TERM_TYPE_FUN = 4, + ERL_NIF_TERM_TYPE_INTEGER = 5, + ERL_NIF_TERM_TYPE_LIST = 6, + ERL_NIF_TERM_TYPE_MAP = 7, + ERL_NIF_TERM_TYPE_PID = 8, + ERL_NIF_TERM_TYPE_PORT = 9, + ERL_NIF_TERM_TYPE_REFERENCE = 10, + ERL_NIF_TERM_TYPE_TUPLE = 11, + + /* This is a dummy value intended to coax the compiler into warning about + * unhandled values in a switch even if all the above values have been + * handled. We can add new entries at any time so the user must always + * have a default case. */ + ERL_NIF_TERM_TYPE__MISSING_DEFAULT_CASE__READ_THE_MANUAL = -1 +} ErlNifTermType; + /* * Return values from enif_thread_type(). Negative values * reserved for specific types of non-scheduler threads. diff --git a/erts/emulator/beam/erl_nif_api_funcs.h b/erts/emulator/beam/erl_nif_api_funcs.h index 8ab454c8dd..d57f6ec97c 100644 --- a/erts/emulator/beam/erl_nif_api_funcs.h +++ b/erts/emulator/beam/erl_nif_api_funcs.h @@ -215,6 +215,8 @@ ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM,enif_make_monitor_term,(ErlNifEnv* env, const ERL_NIF_API_FUNC_DECL(void,enif_set_pid_undefined,(ErlNifPid* pid)); ERL_NIF_API_FUNC_DECL(int,enif_is_pid_undefined,(const ErlNifPid* pid)); +ERL_NIF_API_FUNC_DECL(ErlNifTermType,enif_term_type,(ErlNifEnv* env, ERL_NIF_TERM term)); + /* ** ADD NEW ENTRIES HERE (before this comment) !!! */ @@ -401,6 +403,7 @@ ERL_NIF_API_FUNC_DECL(int,enif_is_pid_undefined,(const ErlNifPid* pid)); # define enif_make_monitor_term ERL_NIF_API_FUNC_MACRO(enif_make_monitor_term) # define enif_set_pid_undefined ERL_NIF_API_FUNC_MACRO(enif_set_pid_undefined) # define enif_is_pid_undefined ERL_NIF_API_FUNC_MACRO(enif_is_pid_undefined) +# define enif_term_type ERL_NIF_API_FUNC_MACRO(enif_term_type) /* ** ADD NEW ENTRIES HERE (before this comment) diff --git a/erts/emulator/beam/instrs.tab b/erts/emulator/beam/instrs.tab index fc88cab22f..1eb83b61f2 100644 --- a/erts/emulator/beam/instrs.tab +++ b/erts/emulator/beam/instrs.tab @@ -512,6 +512,15 @@ move_shift(Src, SD, D) { $SD = V; } +move_window2(S1, S2, D) { + Eterm xt0, xt1; + Eterm* y = &$D; + xt0 = $S1; + xt1 = $S2; + y[0] = xt0; + y[1] = xt1; +} + move_window3(S1, S2, S3, D) { Eterm xt0, xt1, xt2; Eterm* y = &$D; @@ -875,12 +884,16 @@ i_is_ne_exact_literal(Fail, Src, Literal) { } } -is_eq(Fail, X, Y) { - CMP_EQ_ACTION($X, $Y, $FAIL($Fail)); +is_eq(Fail, A, B) { + Eterm a = $A; + Eterm b = $B; + CMP_EQ_ACTION(a, b, $FAIL($Fail)); } -is_ne(Fail, X, Y) { - CMP_NE_ACTION($X, $Y, $FAIL($Fail)); +is_ne(Fail, A, B) { + Eterm a = $A; + Eterm b = $B; + CMP_NE_ACTION(a, b, $FAIL($Fail)); } is_lt(Fail, X, Y) { diff --git a/erts/emulator/beam/ops.tab b/erts/emulator/beam/ops.tab index 3cfc685336..da5364183c 100644 --- a/erts/emulator/beam/ops.tab +++ b/erts/emulator/beam/ops.tab @@ -74,23 +74,19 @@ trace_jump W return +# To ensure that a "move Src x(0)" instruction can be combined with +# the following call instruction, we need to make sure that there is +# no line/1 instruction between the move and the call. # -# To ensure that a "move Src x(0)" instruction can be combined -# with the following call instruction, we need to make sure that -# there is no line/1 instruction between the move and the call. -# -# A tail-recursive call to an external function (non-BIF) will -# never be saved on the stack, so there is no reason to keep -# the line instruction. (The compiler did not remove the line -# instruction because it cannot tell the difference between -# BIFs and ordinary Erlang functions.) -# +# A tail-recursive call to an external function (BIF or non-BIF) will +# never be saved on the stack, so there is no reason to keep the line +# instruction. move S X0=x==0 | line Loc | call_ext Ar Func => \ line Loc | move S X0 | call_ext Ar Func -move S X0=x==0 | line Loc | call_ext_last Ar Func=u$is_not_bif D => \ +move S X0=x==0 | line Loc | call_ext_last Ar Func D => \ move S X0 | call_ext_last Ar Func D -move S X0=x==0 | line Loc | call_ext_only Ar Func=u$is_not_bif => \ +move S X0=x==0 | line Loc | call_ext_only Ar Func => \ move S X0 | call_ext_only Ar Func move S X0=x==0 | line Loc | call Ar Func => \ line Loc | move S X0 | call Ar Func @@ -102,9 +98,9 @@ line I allocate t t? allocate_heap t I t? -%cold +# This instruction when a BIF is called tail-recursively when +# ther is stack frame. deallocate Q -%hot init y allocate_zero t t? @@ -215,11 +211,7 @@ set_tuple_element s S P # Get tuple element -i_get_tuple_element xy P x - -%cold -i_get_tuple_element xy P y -%hot +i_get_tuple_element xy P xy i_get_tuple_element2 x P x i_get_tuple_element2_dst x P x x @@ -285,6 +277,9 @@ move_window/6 move X1=x Y1=y | move X2=x Y2=y | move X3=x Y3=y | succ(Y1,Y2) | succ(Y2,Y3) => \ move_window X1 X2 X3 Y1 Y3 +move X1=x Y1=y | move X2=x Y2=y | succ(Y1,Y2) => \ + move_window2 X1 X2 Y1 + move_window X1=x X2=x X3=x Y1=y Y3=y | move X4=x Y4=y | succ(Y3,Y4) => \ move_window X1 X2 X3 X4 Y1 Y4 @@ -294,6 +289,7 @@ move_window X1=x X2=x X3=x X4=x Y1=y Y4=y | move X5=x Y5=y | succ(Y4,Y5) => \ move_window X1=x X2=x X3=x Y1=y Y3=y => move_window3 X1 X2 X3 Y1 move_window X1=x X2=x X3=x X4=x Y1=y Y4=y => move_window4 X1 X2 X3 X4 Y1 +move_window2 x x y move_window3 x x x y move_window4 x x x x y move_window5 x x x x x y @@ -346,11 +342,6 @@ move X1=x X2=x | move X3=x X4=x | independent_moves(X1, X2, X3, X4) => \ move2_par X1 X2 X3 X4 move2_par x x x x -# move2_par x y x y - -move X1=x Y1=y | move X2=x Y2=y => move2_par X1 Y1 X2 Y2 -move2_par x y x y - # move2_par x x x y move X1=x X2=x | move X3=x Y1=y | independent_moves(X1, X2, X3, Y1) => \ @@ -393,20 +384,14 @@ move3 x x x x x x move C=aiq X=x==1 => move_x1 C move C=aiq X=x==2 => move_x2 C +move n D=y => init D + move_x1 c move_x2 c -# The compiler almost never generates a "move Literal y(Y)" instruction, -# so let's cheat if we encounter one. -move S=n D=y => init D -move S=c D=y => move S x | move x D - -move x x -move x y -move y x -move c x +move xy xy +move c xy move n x -move y y # The following move instructions using x(0) are frequently used. @@ -500,9 +485,13 @@ is_ge f? c x is_ge f? s s %hot -is_eq f? s s +is_eq Fail=f Const=c Reg=xy => is_eq Fail Reg Const +is_eq Fail=f C1=c C2=c => move C1 x | is_eq Fail x C2 +is_eq f? S s -is_ne f? s s +is_ne Fail=f Const=c Reg=xy => is_ne Fail Reg Const +is_ne Fail=f C1=c C2=c => move C1 x | is_ne Fail x C2 +is_ne f? S s # # Putting tuples. @@ -992,10 +981,9 @@ call_ext_only u==0 u$func:os:perf_counter/0 => \ call_ext u Bif=u$is_bif => call_bif Bif -call_ext_last u Bif=u$is_bif D => call_bif Bif | deallocate_return D +call_ext_last u Bif=u$is_bif D => deallocate D | call_bif_only Bif -call_ext_only Ar=u Bif=u$is_bif => \ - allocate u Ar | call_bif Bif | deallocate_return u +call_ext_only Ar=u Bif=u$is_bif => call_bif_only Bif # # Any remaining calls are calls to Erlang functions, not BIFs. @@ -1027,6 +1015,7 @@ i_perf_counter %hot call_bif e +call_bif_only e # # Calls to non-building and guard BIFs. @@ -1035,6 +1024,9 @@ call_bif e bif0 u$bif:erlang:self/0 Dst=d => self Dst bif0 u$bif:erlang:node/0 Dst=d => node Dst +bif1 Fail=f Bif=u$bif:erlang:hd/1 Src=x Dst=x => is_nonempty_list_get_hd Fail Src Dst +bif1 Fail=f Bif=u$bif:erlang:tl/1 Src=x Dst=x => is_nonempty_list_get_tl Fail Src Dst + bif1 Fail Bif=u$bif:erlang:get/1 Src=s Dst=d => gen_get(Src, Dst) bif2 Jump=j u$bif:erlang:element/2 S1=s S2=xy Dst=d => gen_element(Jump, S1, S2, Dst) @@ -1123,19 +1115,33 @@ is_function Fail=f c => jump Fail func_info M F A => i_func_info u M F A # ================================================================ -# New bit syntax matching (R11B). +# Bit syntax matching obsoleted in OTP 22. # ================================================================ -%warm +%cold bs_start_match2 Fail=f ica X Y D => jump Fail bs_start_match2 Fail Bin X Y D => i_bs_start_match2 Bin Fail X Y D i_bs_start_match2 xy f t t d +bs_save2 Y=y Index => move Y x | bs_save2 x Index bs_save2 Reg Index => gen_bs_save(Reg, Index) -i_bs_save2 xy t +i_bs_save2 x t +bs_restore2 Y=y Index => move Y x | bs_restore2 x Index bs_restore2 Reg Index => gen_bs_restore(Reg, Index) -i_bs_restore2 xy t +i_bs_restore2 x t + +bs_context_to_binary Y=y | line L | badmatch Y => \ + move Y x | bs_context_to_binary x | line L | badmatch x +bs_context_to_binary Y=y => move Y x | bs_context_to_binary x +bs_context_to_binary x +%warm + +# ================================================================ +# New bit syntax matching (R11B). +# ================================================================ + +%warm # Matching integers bs_match_string Fail Ms Bits Val => i_bs_match_string Ms Fail Bits Val @@ -1154,7 +1160,7 @@ i_bs_get_integer_imm Ms Bits Live Fail Flags Y=y => \ i_bs_get_integer_small_imm xy W f? t x i_bs_get_integer_imm xy W t f? t x -i_bs_get_integer f? t t xy s d +i_bs_get_integer xy f? t t s d i_bs_get_integer_8 xy f? d i_bs_get_integer_16 xy f? d @@ -1166,9 +1172,9 @@ i_bs_get_integer_32 xy f? d bs_get_binary2 Fail=f Ms=xy Live=u Sz=sq Unit=u Flags=u Dst=d => \ gen_get_binary2(Fail, Ms, Live, Sz, Unit, Flags, Dst) -i_bs_get_binary_imm2 f? xy t W t d -i_bs_get_binary2 f xy t? s t d -i_bs_get_binary_all2 f? xy t t d +i_bs_get_binary_imm2 xy f? t W t d +i_bs_get_binary2 xy f t? s t d +i_bs_get_binary_all2 xy f? t t d i_bs_get_binary_all_reuse xy f? t # Fetching float from binaries. @@ -1177,7 +1183,7 @@ bs_get_float2 Fail=f Ms=xy Live=u Sz=s Unit=u Flags=u Dst=d => \ bs_get_float2 Fail=f Ms=x Live=u Sz=q Unit=u Flags=u Dst=d => jump Fail -i_bs_get_float2 f? xy t s t d +i_bs_get_float2 xy f? t s t d # Miscellanous @@ -1185,8 +1191,7 @@ bs_skip_bits2 Fail=f Ms=xy Sz=sq Unit=u Flags=u => \ gen_skip_bits2(Fail, Ms, Sz, Unit, Flags) i_bs_skip_bits_imm2 f? xy W -i_bs_skip_bits2 f? xy xy t -i_bs_skip_bits_all2 f? xy t +i_bs_skip_bits2 xy xy f? t bs_test_tail2 Fail=f Ms=xy Bits=u==0 => bs_test_zero_tail2 Fail Ms bs_test_tail2 Fail=f Ms=xy Bits=u => bs_test_tail_imm2 Fail Ms Bits @@ -1197,16 +1202,6 @@ bs_test_unit F Ms Unit=u==8 => bs_test_unit8 F Ms bs_test_unit f? xy t bs_test_unit8 f? xy -# An y register operand for bs_context_to_binary is rare, -# but can happen because of inlining. - -bs_context_to_binary Y=y | line L | badmatch Y => \ - move Y x | bs_context_to_binary x | line L | badmatch x - -bs_context_to_binary Y=y => move Y x | bs_context_to_binary x - -bs_context_to_binary x - # Gets a bitstring from the tail of a context. bs_get_tail xy d t @@ -1332,31 +1327,35 @@ i_bs_private_append j? t s S x bs_put_integer Fail=j Sz=sq Unit=u Flags=u Src=s => \ gen_put_integer(Fail, Sz, Unit, Flags, Src) -i_new_bs_put_integer j? s t s -i_new_bs_put_integer_imm j? W t s +i_new_bs_put_integer j? S t s +i_new_bs_put_integer_imm xyc j? W t # # Utf8/utf16/utf32 support. (R12B-5) # -bs_utf8_size j Src=s Dst=d => i_bs_utf8_size Src Dst - -i_bs_utf8_size s x +bs_utf8_size j Src Dst=d => i_bs_utf8_size Src Dst +bs_utf16_size j Src Dst=d => i_bs_utf16_size Src Dst -bs_utf16_size j Src=s Dst=d => i_bs_utf16_size Src Dst +bs_put_utf8 Fail u Src => i_bs_put_utf8 Fail Src -i_bs_utf16_size s x - -bs_put_utf8 Fail u Src=s => i_bs_put_utf8 Fail Src +bs_put_utf32 Fail=j Flags=u Src=s => \ + i_bs_validate_unicode Fail Src | bs_put_integer Fail i=32 u=1 Flags Src -i_bs_put_utf8 j? s +i_bs_utf8_size S x +i_bs_utf16_size S x -bs_put_utf16 j? t s +i_bs_put_utf8 j? S +bs_put_utf16 j? t S -bs_put_utf32 Fail=j Flags=u Src=s => \ - i_bs_validate_unicode Fail Src | bs_put_integer Fail i=32 u=1 Flags Src +i_bs_validate_unicode j? S -i_bs_validate_unicode j? s +# Handle unoptimized code. +i_bs_utf8_size Src=c Dst => move Src x | i_bs_utf8_size x Dst +i_bs_utf16_size Src=c Dst => move Src x | i_bs_utf16_size x Dst +i_bs_put_utf8 Fail Src=c => move Src x | i_bs_put_utf8 Fail x +bs_put_utf16 Fail Flags Src=c => move Src x | bs_put_utf16 Fail Flags x +i_bs_validate_unicode Fail Src=c => move Src x | i_bs_validate_unicode Fail x # # Storing floats into binaries. @@ -1366,7 +1365,7 @@ bs_put_float Fail Sz=q Unit Flags Val => badarg Fail bs_put_float Fail=j Sz=s Unit=u Flags=u Src=s => \ gen_put_float(Fail, Sz, Unit, Flags, Src) -i_new_bs_put_float j? s t s +i_new_bs_put_float j? S t s i_new_bs_put_float_imm j? W t s # @@ -1376,9 +1375,18 @@ i_new_bs_put_float_imm j? W t s bs_put_binary Fail=j Sz=s Unit=u Flags=u Src=s => \ gen_put_binary(Fail, Sz, Unit, Flags, Src) -i_new_bs_put_binary j? s t s -i_new_bs_put_binary_imm j? W s -i_new_bs_put_binary_all j? s t +# In unoptimized code, the binary argument could be a literal. (In optimized code, +# there would be a bs_put_string instruction.) +i_new_bs_put_binary Fail Size Unit Lit=c => \ + move Lit x | i_new_bs_put_binary Fail Size Unit x +i_new_bs_put_binary_imm Fail Size Lit=c => \ + move Lit x | i_new_bs_put_binary_imm Fail Size x +i_new_bs_put_binary_all Lit=c Fail Unit => \ + move Lit x | i_new_bs_put_binary_all x Fail Unit + +i_new_bs_put_binary j? S t S +i_new_bs_put_binary_imm j? W S +i_new_bs_put_binary_all xy j? t # # Warning: The i_bs_put_string and i_new_bs_put_string instructions diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h index a69da4d762..a6312293cc 100644 --- a/erts/emulator/beam/sys.h +++ b/erts/emulator/beam/sys.h @@ -111,6 +111,23 @@ #endif #endif +/* + * Test for clang's convenient __has_builtin feature checking macro. + */ +#ifndef __has_builtin + #define __has_builtin(x) 0 +#endif + +/* + * Define HAVE_OVERFLOW_CHECK_BUILTINS if the overflow checking arithmetic + * builtins are available. + */ +#if ERTS_AT_LEAST_GCC_VSN__(5, 1, 0) +# define HAVE_OVERFLOW_CHECK_BUILTINS 1 +#elif __has_builtin(__builtin_mul_overflow) +# define HAVE_OVERFLOW_CHECK_BUILTINS 1 +#endif + #include "erl_misc_utils.h" /* diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 78411f324c..c93966d24f 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -574,7 +574,7 @@ static int my_strncasecmp(const char *s1, const char *s2, size_t n) driver_select(port, e, mode | (on?ERL_DRV_USE:0), on) #define sock_select(d, flags, onoff) do { \ - ASSERT(!(d)->is_ignored); \ + ASSERT(!INET_IGNORED(d)); \ (d)->event_mask = (onoff) ? \ ((d)->event_mask | (flags)) : \ ((d)->event_mask & ~(flags)); \ @@ -898,6 +898,15 @@ static size_t my_strnlen(const char *s, size_t maxlen) #define INET_CMSG_RECVTCLASS (1 << 1) /* am_recvtclass, am_tclass */ #define INET_CMSG_RECVTTL (1 << 2) /* am_recvttl, am_ttl */ +/* Inet flags */ +#define INET_FLG_BUFFER_SET (1 << 0) /* am_buffer has been set by user */ +#define INET_FLG_IS_IGNORED (1 << 1) /* If a fd is ignored by the inet_drv. + This flag should be set to true when + the fd is used outside of inet_drv. */ +#define INET_FLG_IS_IGNORED_RD (1 << 2) +#define INET_FLG_IS_IGNORED_WR (1 << 3) +#define INET_FLG_IS_IGNORED_PASS (1 << 4) + /* ** End of interface constants. **--------------------------------------------------------------------------*/ @@ -943,10 +952,11 @@ static size_t my_strnlen(const char *s, size_t maxlen) #define INET_IFNAMSIZ 16 /* INET Ignore states */ -#define INET_IGNORE_NONE 0 -#define INET_IGNORE_READ (1 << 0) -#define INET_IGNORE_WRITE (1 << 1) -#define INET_IGNORE_PASSIVE (1 << 2) +#define INET_IGNORE_CLEAR(desc) ((desc)->flags & ~(INET_IGNORE_READ|INET_IGNORE_WRITE|INET_IGNORE_PASSIVE)) +#define INET_IGNORED(desc) ((desc)->flags & INET_FLG_IS_IGNORED) +#define INET_IGNORE_READ (INET_FLG_IS_IGNORED|INET_FLG_IS_IGNORED_RD) +#define INET_IGNORE_WRITE (INET_FLG_IS_IGNORED|INET_FLG_IS_IGNORED_WR) +#define INET_IGNORE_PASSIVE (INET_FLG_IS_IGNORED|INET_FLG_IS_IGNORED_PASS) /* Max length of Erlang Term Buffer (for outputting structured terms): */ #ifdef HAVE_SCTP @@ -1128,9 +1138,7 @@ typedef struct { double send_avg; /* average packet size sent */ subs_list empty_out_q_subs; /* Empty out queue subscribers */ - int is_ignored; /* if a fd is ignored by the inet_drv. - This flag should be set to true when - the fd is used outside of inet_drv. */ + int flags; #ifdef HAVE_SETNS char *netns; /* Socket network namespace name as full file path */ @@ -4551,7 +4559,7 @@ static void desc_close(inet_descriptor* desc) * We should close the fd here, but the other driver might still * be selecting on it. */ - if (!desc->is_ignored) + if (!INET_IGNORED(desc)) driver_select(desc->port,(ErlDrvEvent)(long)desc->event, ERL_DRV_USE, 0); else @@ -6312,6 +6320,7 @@ static int inet_set_opts(inet_descriptor* desc, char* ptr, int len) (long)desc->port, desc->s, ival)); if (ival < INET_MIN_BUFFER) ival = INET_MIN_BUFFER; desc->bufsz = ival; + desc->flags |= INET_FLG_BUFFER_SET; continue; case INET_LOPT_ACTIVE: @@ -6507,6 +6516,16 @@ static int inet_set_opts(inet_descriptor* desc, char* ptr, int len) case INET_OPT_RCVBUF: type = SO_RCVBUF; DEBUGF(("inet_set_opts(%ld): s=%d, SO_RCVBUF=%d\r\n", (long)desc->port, desc->s, ival)); + if (!(desc->flags & INET_FLG_BUFFER_SET)) { + /* make sure we have desc->bufsz >= SO_RCVBUF */ + if (ival > (1 << 16) && desc->stype == SOCK_DGRAM && !IS_SCTP(desc)) + /* For UDP we don't want to automatically + set the buffer size to be larger than + the theoretical max MTU */ + desc->bufsz = 1 << 16; + else if (ival > desc->bufsz) + desc->bufsz = ival; + } break; case INET_OPT_LINGER: type = SO_LINGER; if (len < 4) @@ -6752,23 +6771,17 @@ static int inet_set_opts(inet_descriptor* desc, char* ptr, int len) } DEBUGF(("inet_set_opts(%ld): s=%d returned %d\r\n", (long)desc->port, desc->s, res)); - if (type == SO_RCVBUF) { - /* make sure we have desc->bufsz >= SO_RCVBUF */ - if (ival > (1 << 16) && desc->stype == SOCK_DGRAM && !IS_SCTP(desc)) - /* For UDP we don't want to automatically - set the buffer size to be larger than - the theoretical max MTU */ - desc->bufsz = 1 << 16; - else if (ival > desc->bufsz) - desc->bufsz = ival; - } } if ( ((desc->stype == SOCK_STREAM) && IS_CONNECTED(desc)) || ((desc->stype == SOCK_DGRAM) && IS_OPEN(desc))) { - if (desc->active != old_active) + if (desc->active != old_active) { + /* Need to cancel the read_packet timer if we go from active to passive. */ + if (desc->active == INET_PASSIVE && desc->stype == SOCK_DGRAM) + driver_cancel_timer(desc->port); sock_select(desc, (FD_READ|FD_CLOSE), (desc->active>0)); + } /* XXX: UDP sockets could also trigger immediate read here NIY */ if ((desc->stype==SOCK_STREAM) && desc->active) { @@ -6910,6 +6923,7 @@ static int sctp_set_opts(inet_descriptor* desc, char* ptr, int len) if (desc->bufsz < INET_MIN_BUFFER) desc->bufsz = INET_MIN_BUFFER; + desc->flags |= INET_FLG_BUFFER_SET; res = 0; /* This does not affect the kernel buffer size */ continue; @@ -7031,6 +7045,7 @@ static int sctp_set_opts(inet_descriptor* desc, char* ptr, int len) smaller than the kernel one: */ if (desc->bufsz <= arg.ival) desc->bufsz = arg.ival; + desc->flags |= INET_FLG_BUFFER_SET; break; } case INET_OPT_SNDBUF: @@ -7041,10 +7056,6 @@ static int sctp_set_opts(inet_descriptor* desc, char* ptr, int len) arg_ptr = (char*) (&arg.ival); arg_sz = sizeof ( arg.ival); - /* Adjust the size of the user-level recv buffer, so it's not - smaller than the kernel one: */ - if (desc->bufsz <= arg.ival) - desc->bufsz = arg.ival; break; } case INET_OPT_REUSEADDR: @@ -9101,7 +9112,7 @@ static ErlDrvData inet_start(ErlDrvPort port, int size, int protocol) sys_memzero((char *)&desc->remote,sizeof(desc->remote)); - desc->is_ignored = 0; + desc->flags = 0; #ifdef HAVE_SETNS desc->netns = NULL; @@ -9503,19 +9514,19 @@ static ErlDrvSSizeT inet_ctl(inet_descriptor* desc, int cmd, char* buf, if (desc->stype != SOCK_STREAM) return ctl_error(EINVAL, rbuf, rsize); - if (*buf == 1 && !desc->is_ignored) { + if (*buf == 1 && !INET_IGNORED(desc)) { sock_select(desc, (FD_READ|FD_WRITE|FD_CLOSE|ERL_DRV_USE_NO_CALLBACK), 0); if (desc->active) - desc->is_ignored = INET_IGNORE_READ; + desc->flags |= INET_IGNORE_READ; else - desc->is_ignored = INET_IGNORE_PASSIVE; - } else if (*buf == 0 && desc->is_ignored) { + desc->flags |= INET_IGNORE_PASSIVE; + } else if (*buf == 0 && INET_IGNORED(desc)) { int flags = FD_CLOSE; - if (desc->is_ignored & INET_IGNORE_READ) + if (desc->flags & INET_IGNORE_READ) flags |= FD_READ; - if (desc->is_ignored & INET_IGNORE_WRITE) + if (desc->flags & INET_IGNORE_WRITE) flags |= FD_WRITE; - desc->is_ignored = INET_IGNORE_NONE; + desc->flags = INET_IGNORE_CLEAR(desc); if (flags != FD_CLOSE) sock_select(desc, flags, 1); } else @@ -10231,17 +10242,17 @@ static ErlDrvSSizeT tcp_inet_ctl(ErlDrvData e, unsigned int cmd, if (enq_async(INETP(desc), tbuf, TCP_REQ_RECV) < 0) return ctl_error(EALREADY, rbuf, rsize); - if (INETP(desc)->is_ignored || tcp_recv(desc, n) == 0) { + if (INET_IGNORED(INETP(desc)) || tcp_recv(desc, n) == 0) { if (timeout == 0) async_error_am(INETP(desc), am_timeout); else { if (timeout != INET_INFINITY) add_multi_timer(desc, INETP(desc)->port, 0, timeout, &tcp_inet_recv_timeout); - if (!INETP(desc)->is_ignored) + if (!INET_IGNORED(INETP(desc))) sock_select(INETP(desc),(FD_READ|FD_CLOSE),1); else - INETP(desc)->is_ignored |= INET_IGNORE_READ; + INETP(desc)->flags |= INET_IGNORE_READ; } } return ctl_reply(INET_REP_OK, tbuf, 2, rbuf, rsize); @@ -11108,7 +11119,7 @@ static int tcp_inet_input(tcp_descriptor* desc, HANDLE event) #ifdef DEBUG long port = (long) desc->inet.port; /* Used after driver_exit() */ #endif - ASSERT(!INETP(desc)->is_ignored); + ASSERT(!INET_IGNORED(INETP(desc))); DEBUGF(("tcp_inet_input(%ld) {s=%d\r\n", port, desc->inet.s)); /* XXX fprintf(stderr,"tcp_inet_input(%ld) {s=%d}\r\n",(long) desc->inet.port, desc->inet.s); */ if (desc->inet.state == INET_STATE_ACCEPTING) { @@ -11443,8 +11454,8 @@ static int tcp_sendv(tcp_descriptor* desc, ErlIOVec* ev) DEBUGF(("tcp_sendv(%ld): s=%d, about to send "LLU","LLU" bytes\r\n", (long)desc->inet.port, desc->inet.s, (llu_t)h_len, (llu_t)len)); - if (INETP(desc)->is_ignored) { - INETP(desc)->is_ignored |= INET_IGNORE_WRITE; + if (INET_IGNORED(INETP(desc))) { + INETP(desc)->flags |= INET_IGNORE_WRITE; n = 0; } else if (desc->tcp_add_flags & TCP_ADDF_DELAY_SEND) { driver_enqv(ix, ev, 0); @@ -11479,7 +11490,7 @@ static int tcp_sendv(tcp_descriptor* desc, ErlIOVec* ev) DEBUGF(("tcp_sendv(%ld): s=%d, Send failed, queuing\r\n", (long)desc->inet.port, desc->inet.s)); driver_enqv(ix, ev, n); - if (!INETP(desc)->is_ignored) + if (!INET_IGNORED(INETP(desc))) sock_select(INETP(desc),(FD_WRITE|FD_CLOSE), 1); } return 0; @@ -11548,8 +11559,8 @@ static int tcp_send(tcp_descriptor* desc, char* ptr, ErlDrvSizeT len) DEBUGF(("tcp_send(%ld): s=%d, about to send "LLU","LLU" bytes\r\n", (long)desc->inet.port, desc->inet.s, (llu_t)h_len, (llu_t)len)); - if (INETP(desc)->is_ignored) { - INETP(desc)->is_ignored |= INET_IGNORE_WRITE; + if (INET_IGNORED(INETP(desc))) { + INETP(desc)->flags |= INET_IGNORE_WRITE; n = 0; } else if (desc->tcp_add_flags & TCP_ADDF_DELAY_SEND) { sock_send(desc->inet.s, buf, 0, 0); @@ -11582,7 +11593,7 @@ static int tcp_send(tcp_descriptor* desc, char* ptr, ErlDrvSizeT len) n -= h_len; driver_enq(ix, ptr+n, len-n); } - if (!INETP(desc)->is_ignored) + if (!INET_IGNORED(INETP(desc))) sock_select(INETP(desc),(FD_WRITE|FD_CLOSE), 1); } return 0; @@ -11863,7 +11874,7 @@ static int tcp_inet_output(tcp_descriptor* desc, HANDLE event) int ret = 0; ErlDrvPort ix = desc->inet.port; - ASSERT(!INETP(desc)->is_ignored); + ASSERT(!INET_IGNORED(INETP(desc))); DEBUGF(("tcp_inet_output(%ld) {s=%d\r\n", (long)desc->inet.port, desc->inet.s)); if (desc->inet.state == INET_STATE_CONNECTING) { @@ -12538,9 +12549,12 @@ static void packet_inet_timeout(ErlDrvData e) { udp_descriptor * udesc = (udp_descriptor*) e; inet_descriptor * desc = INETP(udesc); - if (!(desc->active)) + if (!(desc->active)) { sock_select(desc, FD_READ, 0); - async_error_am (desc, am_timeout); + async_error_am (desc, am_timeout); + } else { + (void)packet_inet_input(udesc, desc->s); + } } @@ -12896,6 +12910,15 @@ static int packet_inet_input(udp_descriptor* udesc, HANDLE event) sock_select(desc, FD_READ, 1); } #endif + + /* We set a timer on the port to trigger now. + This emulates a "yield" operation as that is + what we want to do here. We do *NOT* do a deselect + as that is expensive, instead we check if the + socket it still active when the timeout triggers + and if it is not, then we just ignore the timeout */ + driver_set_timer(desc->port, 0); + return count; } diff --git a/erts/emulator/nifs/common/net_nif.c b/erts/emulator/nifs/common/net_nif.c index 9a18ed3b15..252aa3c835 100644 --- a/erts/emulator/nifs/common/net_nif.c +++ b/erts/emulator/nifs/common/net_nif.c @@ -240,35 +240,37 @@ static NetData data; /* THIS IS JUST TEMPORARY */ extern char* erl_errno_id(int error); +/* All the nif "callback" functions for the net API has + * the exact same API: + * + * nif_<funcname>(ErlNifEnv* env, + * int argc, + * const ERL_NIF_TERM argv[]); + * + * So, to simplify, use some macro magic to define those. + * + * These are the functions making up the "official" API. + */ -static ERL_NIF_TERM nif_info(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_command(ErlNifEnv* env, - int argc, +#define ENET_NIF_FUNCS \ + ENET_NIF_FUNC_DEF(info); \ + ENET_NIF_FUNC_DEF(command); \ + ENET_NIF_FUNC_DEF(gethostname); \ + ENET_NIF_FUNC_DEF(getnameinfo); \ + ENET_NIF_FUNC_DEF(getaddrinfo); \ + ENET_NIF_FUNC_DEF(if_name2index); \ + ENET_NIF_FUNC_DEF(if_index2name); \ + ENET_NIF_FUNC_DEF(if_names); + +#define ENET_NIF_FUNC_DEF(F) \ + static ERL_NIF_TERM nif_##F(ErlNifEnv* env, \ + int argc, \ const ERL_NIF_TERM argv[]); +ENET_NIF_FUNCS +#undef ENET_NIF_FUNC_DEF -static ERL_NIF_TERM nif_gethostname(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); - -static ERL_NIF_TERM nif_getnameinfo(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_getaddrinfo(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); - -static ERL_NIF_TERM nif_if_name2index(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_if_index2name(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_if_names(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); +/* And here comes the functions that does the actual work (for the most part) */ static ERL_NIF_TERM ncommand(ErlNifEnv* env, ERL_NIF_TERM cmd); static ERL_NIF_TERM ngethostname(ErlNifEnv* env); @@ -359,66 +361,42 @@ static const struct in6_addr in6addr_loopback = -/* *** String constants *** */ -static char str_address_info[] = "address_info"; -static char str_debug[] = "debug"; -static char str_idn[] = "idn"; -static char str_idna_allow_unassigned[] = "idna_allow_unassigned"; -static char str_idna_use_std3_ascii_rules[] = "idna_use_std3_ascii_rules"; -static char str_namereqd[] = "namereqd"; -static char str_name_info[] = "name_info"; -static char str_nofqdn[] = "nofqdn"; -static char str_numerichost[] = "numerichost"; -static char str_numericserv[] = "numericserv"; - -/* (special) error string constants */ -static char str_eaddrfamily[] = "eaddrfamily"; -static char str_ebadflags[] = "ebadflags"; -static char str_efail[] = "efail"; -static char str_efamily[] = "efamily"; -static char str_efault[] = "efault"; -static char str_emem[] = "emem"; -static char str_enametoolong[] = "enametoolong"; -static char str_enodata[] = "enodata"; -static char str_enoname[] = "enoname"; -static char str_enxio[] = "enxio"; -static char str_eoverflow[] = "eoverflow"; -static char str_eservice[] = "eservice"; -static char str_esocktype[] = "esocktype"; -static char str_esystem[] = "esystem"; - - -/* *** Atoms *** */ - -static ERL_NIF_TERM atom_address_info; -static ERL_NIF_TERM atom_debug; -static ERL_NIF_TERM atom_host; -static ERL_NIF_TERM atom_idn; -static ERL_NIF_TERM atom_idna_allow_unassigned; -static ERL_NIF_TERM atom_idna_use_std3_ascii_rules; -static ERL_NIF_TERM atom_namereqd; -static ERL_NIF_TERM atom_name_info; -static ERL_NIF_TERM atom_nofqdn; -static ERL_NIF_TERM atom_numerichost; -static ERL_NIF_TERM atom_numericserv; -static ERL_NIF_TERM atom_service; - - -static ERL_NIF_TERM atom_eaddrfamily; -// static ERL_NIF_TERM atom_eagain; -static ERL_NIF_TERM atom_ebadflags; -static ERL_NIF_TERM atom_efail; -static ERL_NIF_TERM atom_efamily; -static ERL_NIF_TERM atom_efault; -static ERL_NIF_TERM atom_emem; -static ERL_NIF_TERM atom_enametoolong; -static ERL_NIF_TERM atom_enodata; -static ERL_NIF_TERM atom_enoname; -static ERL_NIF_TERM atom_enxio; -static ERL_NIF_TERM atom_eoverflow; -static ERL_NIF_TERM atom_eservice; -static ERL_NIF_TERM atom_esocktype; -static ERL_NIF_TERM atom_esystem; +/* *** Local atoms *** */ + +#define LOCAL_ATOMS \ + LOCAL_ATOM_DECL(address_info); \ + LOCAL_ATOM_DECL(debug); \ + LOCAL_ATOM_DECL(host); \ + LOCAL_ATOM_DECL(idn); \ + LOCAL_ATOM_DECL(idna_allow_unassigned); \ + LOCAL_ATOM_DECL(idna_use_std3_ascii_rules); \ + LOCAL_ATOM_DECL(namereqd); \ + LOCAL_ATOM_DECL(name_info); \ + LOCAL_ATOM_DECL(nofqdn); \ + LOCAL_ATOM_DECL(numerichost); \ + LOCAL_ATOM_DECL(numericserv); \ + LOCAL_ATOM_DECL(service); + +#define LOCAL_ERROR_REASON_ATOMS \ + LOCAL_ATOM_DECL(eaddrfamily); \ + LOCAL_ATOM_DECL(ebadflags); \ + LOCAL_ATOM_DECL(efail); \ + LOCAL_ATOM_DECL(efamily); \ + LOCAL_ATOM_DECL(efault); \ + LOCAL_ATOM_DECL(emem); \ + LOCAL_ATOM_DECL(enametoolong); \ + LOCAL_ATOM_DECL(enodata); \ + LOCAL_ATOM_DECL(enoname); \ + LOCAL_ATOM_DECL(enxio); \ + LOCAL_ATOM_DECL(eoverflow); \ + LOCAL_ATOM_DECL(eservice); \ + LOCAL_ATOM_DECL(esocktype); \ + LOCAL_ATOM_DECL(esystem); + +#define LOCAL_ATOM_DECL(A) static ERL_NIF_TERM atom_##A +LOCAL_ATOMS +LOCAL_ERROR_REASON_ATOMS +#undef LOCAL_ATOM_DECL /* *** net *** */ @@ -1385,7 +1363,7 @@ ERL_NIF_TERM encode_address_infos(ErlNifEnv* env, NDBG( ("NET", "encode_address_infos -> len: %d\r\n", len) ); if (len > 0) { - ERL_NIF_TERM* array = MALLOC(len * sizeof(ERL_NIF_TERM)); // LEAK? + ERL_NIF_TERM* array = MALLOC(len * sizeof(ERL_NIF_TERM)); unsigned int i = 0; struct addrinfo* p = addrInfo; @@ -1396,6 +1374,7 @@ ERL_NIF_TERM encode_address_infos(ErlNifEnv* env, } result = MKLA(env, array, len); + FREE(array); } else { result = MKEL(env); } @@ -1651,35 +1630,10 @@ int on_load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) NDBG( ("NET", "on_load -> entry\r\n") ); #endif - /* +++ Misc atoms +++ */ - atom_address_info = MKA(env, str_address_info); - atom_debug = MKA(env, str_debug); - atom_host = MKA(env, "host"); - atom_idn = MKA(env, str_idn); - atom_idna_allow_unassigned = MKA(env, str_idna_allow_unassigned); - atom_idna_use_std3_ascii_rules = MKA(env, str_idna_use_std3_ascii_rules); - atom_namereqd = MKA(env, str_namereqd); - atom_name_info = MKA(env, str_name_info); - atom_nofqdn = MKA(env, str_nofqdn); - atom_numerichost = MKA(env, str_numerichost); - atom_numericserv = MKA(env, str_numericserv); - atom_service = MKA(env, "service"); - - /* Error codes */ - atom_eaddrfamily = MKA(env, str_eaddrfamily); - atom_ebadflags = MKA(env, str_ebadflags); - atom_efail = MKA(env, str_efail); - atom_efamily = MKA(env, str_efamily); - atom_efault = MKA(env, str_efault); - atom_emem = MKA(env, str_emem); - atom_enametoolong = MKA(env, str_enametoolong); - atom_enodata = MKA(env, str_enodata); - atom_enoname = MKA(env, str_enoname); - atom_enxio = MKA(env, str_enxio); - atom_eoverflow = MKA(env, str_eoverflow); - atom_eservice = MKA(env, str_eservice); - atom_esocktype = MKA(env, str_esocktype); - atom_esystem = MKA(env, str_esystem); +#define LOCAL_ATOM_DECL(A) atom_##A = MKA(env, #A) +LOCAL_ATOMS +LOCAL_ERROR_REASON_ATOMS +#undef LOCAL_ATOM_DECL // For storing "global" things... // data.env = enif_alloc_env(); // We should really check diff --git a/erts/emulator/nifs/common/socket_int.h b/erts/emulator/nifs/common/socket_int.h index a9e83adc21..043303a60d 100644 --- a/erts/emulator/nifs/common/socket_int.h +++ b/erts/emulator/nifs/common/socket_int.h @@ -103,209 +103,218 @@ typedef unsigned int BOOLEAN_T; /* ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ * "Global" atoms */ -extern ERL_NIF_TERM esock_atom_abort; -extern ERL_NIF_TERM esock_atom_accept; -extern ERL_NIF_TERM esock_atom_acceptconn; -extern ERL_NIF_TERM esock_atom_acceptfilter; -extern ERL_NIF_TERM esock_atom_adaption_layer; -extern ERL_NIF_TERM esock_atom_addr; -extern ERL_NIF_TERM esock_atom_addrform; -extern ERL_NIF_TERM esock_atom_add_membership; -extern ERL_NIF_TERM esock_atom_add_source_membership; -extern ERL_NIF_TERM esock_atom_any; -extern ERL_NIF_TERM esock_atom_associnfo; -extern ERL_NIF_TERM esock_atom_authhdr; -extern ERL_NIF_TERM esock_atom_auth_active_key; -extern ERL_NIF_TERM esock_atom_auth_asconf; -extern ERL_NIF_TERM esock_atom_auth_chunk; -extern ERL_NIF_TERM esock_atom_auth_delete_key; -extern ERL_NIF_TERM esock_atom_auth_key; -extern ERL_NIF_TERM esock_atom_auth_level; -extern ERL_NIF_TERM esock_atom_autoclose; -extern ERL_NIF_TERM esock_atom_bindtodevice; -extern ERL_NIF_TERM esock_atom_block_source; -extern ERL_NIF_TERM esock_atom_broadcast; -extern ERL_NIF_TERM esock_atom_busy_poll; -extern ERL_NIF_TERM esock_atom_checksum; -extern ERL_NIF_TERM esock_atom_close; -extern ERL_NIF_TERM esock_atom_connect; -extern ERL_NIF_TERM esock_atom_congestion; -extern ERL_NIF_TERM esock_atom_context; -extern ERL_NIF_TERM esock_atom_cork; -extern ERL_NIF_TERM esock_atom_credentials; -extern ERL_NIF_TERM esock_atom_ctrl; -extern ERL_NIF_TERM esock_atom_ctrunc; -extern ERL_NIF_TERM esock_atom_data; -extern ERL_NIF_TERM esock_atom_debug; -extern ERL_NIF_TERM esock_atom_default_send_params; -extern ERL_NIF_TERM esock_atom_delayed_ack_time; -extern ERL_NIF_TERM esock_atom_dgram; -extern ERL_NIF_TERM esock_atom_disable_fragments; -extern ERL_NIF_TERM esock_atom_domain; -extern ERL_NIF_TERM esock_atom_dontfrag; -extern ERL_NIF_TERM esock_atom_dontroute; -extern ERL_NIF_TERM esock_atom_drop_membership; -extern ERL_NIF_TERM esock_atom_drop_source_membership; -extern ERL_NIF_TERM esock_atom_dstopts; -extern ERL_NIF_TERM esock_atom_eor; -extern ERL_NIF_TERM esock_atom_error; -extern ERL_NIF_TERM esock_atom_errqueue; -extern ERL_NIF_TERM esock_atom_esp_network_level; -extern ERL_NIF_TERM esock_atom_esp_trans_level; -extern ERL_NIF_TERM esock_atom_events; -extern ERL_NIF_TERM esock_atom_explicit_eor; -extern ERL_NIF_TERM esock_atom_faith; -extern ERL_NIF_TERM esock_atom_false; -extern ERL_NIF_TERM esock_atom_family; -extern ERL_NIF_TERM esock_atom_flags; -extern ERL_NIF_TERM esock_atom_flowinfo; -extern ERL_NIF_TERM esock_atom_fragment_interleave; -extern ERL_NIF_TERM esock_atom_freebind; -extern ERL_NIF_TERM esock_atom_get_peer_addr_info; -extern ERL_NIF_TERM esock_atom_hdrincl; -extern ERL_NIF_TERM esock_atom_hmac_ident; -extern ERL_NIF_TERM esock_atom_hoplimit; -extern ERL_NIF_TERM esock_atom_hopopts; -extern ERL_NIF_TERM esock_atom_ifindex; -extern ERL_NIF_TERM esock_atom_inet; -extern ERL_NIF_TERM esock_atom_inet6; -extern ERL_NIF_TERM esock_atom_info; -extern ERL_NIF_TERM esock_atom_initmsg; -extern ERL_NIF_TERM esock_atom_iov; -extern ERL_NIF_TERM esock_atom_ip; -extern ERL_NIF_TERM esock_atom_ipcomp_level; -extern ERL_NIF_TERM esock_atom_ipv6; -extern ERL_NIF_TERM esock_atom_i_want_mapped_v4_addr; -extern ERL_NIF_TERM esock_atom_join_group; -extern ERL_NIF_TERM esock_atom_keepalive; -extern ERL_NIF_TERM esock_atom_keepcnt; -extern ERL_NIF_TERM esock_atom_keepidle; -extern ERL_NIF_TERM esock_atom_keepintvl; -extern ERL_NIF_TERM esock_atom_leave_group; -extern ERL_NIF_TERM esock_atom_level; -extern ERL_NIF_TERM esock_atom_linger; -extern ERL_NIF_TERM esock_atom_local; -extern ERL_NIF_TERM esock_atom_local_auth_chunks; -extern ERL_NIF_TERM esock_atom_loopback; -extern ERL_NIF_TERM esock_atom_lowdelay; -extern ERL_NIF_TERM esock_atom_mark; -extern ERL_NIF_TERM esock_atom_maxburst; -extern ERL_NIF_TERM esock_atom_maxseg; -extern ERL_NIF_TERM esock_atom_md5sig; -extern ERL_NIF_TERM esock_atom_mincost; -extern ERL_NIF_TERM esock_atom_minttl; -extern ERL_NIF_TERM esock_atom_msfilter; -extern ERL_NIF_TERM esock_atom_mtu; -extern ERL_NIF_TERM esock_atom_mtu_discover; -extern ERL_NIF_TERM esock_atom_multicast_all; -extern ERL_NIF_TERM esock_atom_multicast_hops; -extern ERL_NIF_TERM esock_atom_multicast_if; -extern ERL_NIF_TERM esock_atom_multicast_loop; -extern ERL_NIF_TERM esock_atom_multicast_ttl; -extern ERL_NIF_TERM esock_atom_nodelay; -extern ERL_NIF_TERM esock_atom_nodefrag; -extern ERL_NIF_TERM esock_atom_noopt; -extern ERL_NIF_TERM esock_atom_nopush; -extern ERL_NIF_TERM esock_atom_not_found; -extern ERL_NIF_TERM esock_atom_not_owner; -extern ERL_NIF_TERM esock_atom_ok; -extern ERL_NIF_TERM esock_atom_oob; -extern ERL_NIF_TERM esock_atom_oobinline; -extern ERL_NIF_TERM esock_atom_options; -extern ERL_NIF_TERM esock_atom_origdstaddr; -extern ERL_NIF_TERM esock_atom_partial_delivery_point; -extern ERL_NIF_TERM esock_atom_passcred; -extern ERL_NIF_TERM esock_atom_path; -extern ERL_NIF_TERM esock_atom_peekcred; -extern ERL_NIF_TERM esock_atom_peek_off; -extern ERL_NIF_TERM esock_atom_peer_addr_params; -extern ERL_NIF_TERM esock_atom_peer_auth_chunks; -extern ERL_NIF_TERM esock_atom_pktinfo; -extern ERL_NIF_TERM esock_atom_pktoptions; -extern ERL_NIF_TERM esock_atom_port; -extern ERL_NIF_TERM esock_atom_portrange; -extern ERL_NIF_TERM esock_atom_primary_addr; -extern ERL_NIF_TERM esock_atom_priority; -extern ERL_NIF_TERM esock_atom_protocol; -extern ERL_NIF_TERM esock_atom_raw; -extern ERL_NIF_TERM esock_atom_rcvbuf; -extern ERL_NIF_TERM esock_atom_rcvbufforce; -extern ERL_NIF_TERM esock_atom_rcvlowat; -extern ERL_NIF_TERM esock_atom_rcvtimeo; -extern ERL_NIF_TERM esock_atom_rdm; -extern ERL_NIF_TERM esock_atom_recv; -extern ERL_NIF_TERM esock_atom_recvdstaddr; -extern ERL_NIF_TERM esock_atom_recverr; -extern ERL_NIF_TERM esock_atom_recvfrom; -extern ERL_NIF_TERM esock_atom_recvif; -extern ERL_NIF_TERM esock_atom_recvmsg; -extern ERL_NIF_TERM esock_atom_recvopts; -extern ERL_NIF_TERM esock_atom_recvorigdstaddr; -extern ERL_NIF_TERM esock_atom_recvpktinfo; -extern ERL_NIF_TERM esock_atom_recvtclass; -extern ERL_NIF_TERM esock_atom_recvtos; -extern ERL_NIF_TERM esock_atom_recvttl; -extern ERL_NIF_TERM esock_atom_reliability; -extern ERL_NIF_TERM esock_atom_reset_streams; -extern ERL_NIF_TERM esock_atom_retopts; -extern ERL_NIF_TERM esock_atom_reuseaddr; -extern ERL_NIF_TERM esock_atom_reuseport; -extern ERL_NIF_TERM esock_atom_rights; -extern ERL_NIF_TERM esock_atom_router_alert; -extern ERL_NIF_TERM esock_atom_rthdr; -extern ERL_NIF_TERM esock_atom_rtoinfo; -extern ERL_NIF_TERM esock_atom_rxq_ovfl; -extern ERL_NIF_TERM esock_atom_scope_id; -extern ERL_NIF_TERM esock_atom_sctp; -extern ERL_NIF_TERM esock_atom_sec; -extern ERL_NIF_TERM esock_atom_select_failed; -extern ERL_NIF_TERM esock_atom_select_sent; -extern ERL_NIF_TERM esock_atom_send; -extern ERL_NIF_TERM esock_atom_sendmsg; -extern ERL_NIF_TERM esock_atom_sendsrcaddr; -extern ERL_NIF_TERM esock_atom_sendto; -extern ERL_NIF_TERM esock_atom_seqpacket; -extern ERL_NIF_TERM esock_atom_setfib; -extern ERL_NIF_TERM esock_atom_set_peer_primary_addr; -extern ERL_NIF_TERM esock_atom_sndbuf; -extern ERL_NIF_TERM esock_atom_sndbufforce; -extern ERL_NIF_TERM esock_atom_sndlowat; -extern ERL_NIF_TERM esock_atom_sndtimeo; -extern ERL_NIF_TERM esock_atom_socket; -extern ERL_NIF_TERM esock_atom_socket_tag; -extern ERL_NIF_TERM esock_atom_spec_dst; -extern ERL_NIF_TERM esock_atom_status; -extern ERL_NIF_TERM esock_atom_stream; -extern ERL_NIF_TERM esock_atom_syncnt; -extern ERL_NIF_TERM esock_atom_tclass; -extern ERL_NIF_TERM esock_atom_tcp; -extern ERL_NIF_TERM esock_atom_throughput; -extern ERL_NIF_TERM esock_atom_timestamp; -extern ERL_NIF_TERM esock_atom_tos; -extern ERL_NIF_TERM esock_atom_transparent; -extern ERL_NIF_TERM esock_atom_true; -extern ERL_NIF_TERM esock_atom_trunc; -extern ERL_NIF_TERM esock_atom_ttl; -extern ERL_NIF_TERM esock_atom_type; -extern ERL_NIF_TERM esock_atom_udp; -extern ERL_NIF_TERM esock_atom_unblock_source; -extern ERL_NIF_TERM esock_atom_undefined; -extern ERL_NIF_TERM esock_atom_unicast_hops; -extern ERL_NIF_TERM esock_atom_unknown; -extern ERL_NIF_TERM esock_atom_usec; -extern ERL_NIF_TERM esock_atom_user_timeout; -extern ERL_NIF_TERM esock_atom_use_ext_recvinfo; -extern ERL_NIF_TERM esock_atom_use_min_mtu; -extern ERL_NIF_TERM esock_atom_v6only; + +#define GLOBAL_ATOM_DEFS \ + GLOBAL_ATOM_DEF(abort); \ + GLOBAL_ATOM_DEF(accept); \ + GLOBAL_ATOM_DEF(acceptconn); \ + GLOBAL_ATOM_DEF(acceptfilter); \ + GLOBAL_ATOM_DEF(adaption_layer); \ + GLOBAL_ATOM_DEF(addr); \ + GLOBAL_ATOM_DEF(addrform); \ + GLOBAL_ATOM_DEF(add_membership); \ + GLOBAL_ATOM_DEF(add_source_membership); \ + GLOBAL_ATOM_DEF(any); \ + GLOBAL_ATOM_DEF(associnfo); \ + GLOBAL_ATOM_DEF(authhdr); \ + GLOBAL_ATOM_DEF(auth_active_key); \ + GLOBAL_ATOM_DEF(auth_asconf); \ + GLOBAL_ATOM_DEF(auth_chunk); \ + GLOBAL_ATOM_DEF(auth_delete_key); \ + GLOBAL_ATOM_DEF(auth_key); \ + GLOBAL_ATOM_DEF(auth_level); \ + GLOBAL_ATOM_DEF(autoclose); \ + GLOBAL_ATOM_DEF(bindtodevice); \ + GLOBAL_ATOM_DEF(block_source); \ + GLOBAL_ATOM_DEF(broadcast); \ + GLOBAL_ATOM_DEF(busy_poll); \ + GLOBAL_ATOM_DEF(checksum); \ + GLOBAL_ATOM_DEF(close); \ + GLOBAL_ATOM_DEF(connect); \ + GLOBAL_ATOM_DEF(congestion); \ + GLOBAL_ATOM_DEF(context); \ + GLOBAL_ATOM_DEF(cork); \ + GLOBAL_ATOM_DEF(credentials); \ + GLOBAL_ATOM_DEF(ctrl); \ + GLOBAL_ATOM_DEF(ctrunc); \ + GLOBAL_ATOM_DEF(data); \ + GLOBAL_ATOM_DEF(debug); \ + GLOBAL_ATOM_DEF(default_send_params); \ + GLOBAL_ATOM_DEF(delayed_ack_time); \ + GLOBAL_ATOM_DEF(dgram); \ + GLOBAL_ATOM_DEF(disable_fragments); \ + GLOBAL_ATOM_DEF(domain); \ + GLOBAL_ATOM_DEF(dontfrag); \ + GLOBAL_ATOM_DEF(dontroute); \ + GLOBAL_ATOM_DEF(drop_membership); \ + GLOBAL_ATOM_DEF(drop_source_membership); \ + GLOBAL_ATOM_DEF(dstopts); \ + GLOBAL_ATOM_DEF(eor); \ + GLOBAL_ATOM_DEF(error); \ + GLOBAL_ATOM_DEF(errqueue); \ + GLOBAL_ATOM_DEF(esp_network_level); \ + GLOBAL_ATOM_DEF(esp_trans_level); \ + GLOBAL_ATOM_DEF(events); \ + GLOBAL_ATOM_DEF(explicit_eor); \ + GLOBAL_ATOM_DEF(faith); \ + GLOBAL_ATOM_DEF(false); \ + GLOBAL_ATOM_DEF(family); \ + GLOBAL_ATOM_DEF(flags); \ + GLOBAL_ATOM_DEF(flowinfo); \ + GLOBAL_ATOM_DEF(fragment_interleave); \ + GLOBAL_ATOM_DEF(freebind); \ + GLOBAL_ATOM_DEF(get_peer_addr_info); \ + GLOBAL_ATOM_DEF(hdrincl); \ + GLOBAL_ATOM_DEF(hmac_ident); \ + GLOBAL_ATOM_DEF(hoplimit); \ + GLOBAL_ATOM_DEF(hopopts); \ + GLOBAL_ATOM_DEF(ifindex); \ + GLOBAL_ATOM_DEF(inet); \ + GLOBAL_ATOM_DEF(inet6); \ + GLOBAL_ATOM_DEF(info); \ + GLOBAL_ATOM_DEF(initmsg); \ + GLOBAL_ATOM_DEF(iov); \ + GLOBAL_ATOM_DEF(ip); \ + GLOBAL_ATOM_DEF(ipcomp_level); \ + GLOBAL_ATOM_DEF(ipv6); \ + GLOBAL_ATOM_DEF(i_want_mapped_v4_addr); \ + GLOBAL_ATOM_DEF(join_group); \ + GLOBAL_ATOM_DEF(keepalive); \ + GLOBAL_ATOM_DEF(keepcnt); \ + GLOBAL_ATOM_DEF(keepidle); \ + GLOBAL_ATOM_DEF(keepintvl); \ + GLOBAL_ATOM_DEF(leave_group); \ + GLOBAL_ATOM_DEF(level); \ + GLOBAL_ATOM_DEF(linger); \ + GLOBAL_ATOM_DEF(local); \ + GLOBAL_ATOM_DEF(local_auth_chunks); \ + GLOBAL_ATOM_DEF(loopback); \ + GLOBAL_ATOM_DEF(lowdelay); \ + GLOBAL_ATOM_DEF(mark); \ + GLOBAL_ATOM_DEF(maxburst); \ + GLOBAL_ATOM_DEF(maxseg); \ + GLOBAL_ATOM_DEF(md5sig); \ + GLOBAL_ATOM_DEF(mincost); \ + GLOBAL_ATOM_DEF(minttl); \ + GLOBAL_ATOM_DEF(msfilter); \ + GLOBAL_ATOM_DEF(mtu); \ + GLOBAL_ATOM_DEF(mtu_discover); \ + GLOBAL_ATOM_DEF(multicast_all); \ + GLOBAL_ATOM_DEF(multicast_hops); \ + GLOBAL_ATOM_DEF(multicast_if); \ + GLOBAL_ATOM_DEF(multicast_loop); \ + GLOBAL_ATOM_DEF(multicast_ttl); \ + GLOBAL_ATOM_DEF(nodelay); \ + GLOBAL_ATOM_DEF(nodefrag); \ + GLOBAL_ATOM_DEF(noopt); \ + GLOBAL_ATOM_DEF(nopush); \ + GLOBAL_ATOM_DEF(not_found); \ + GLOBAL_ATOM_DEF(not_owner); \ + GLOBAL_ATOM_DEF(ok); \ + GLOBAL_ATOM_DEF(oob); \ + GLOBAL_ATOM_DEF(oobinline); \ + GLOBAL_ATOM_DEF(options); \ + GLOBAL_ATOM_DEF(origdstaddr); \ + GLOBAL_ATOM_DEF(partial_delivery_point); \ + GLOBAL_ATOM_DEF(passcred); \ + GLOBAL_ATOM_DEF(path); \ + GLOBAL_ATOM_DEF(peekcred); \ + GLOBAL_ATOM_DEF(peek_off); \ + GLOBAL_ATOM_DEF(peer_addr_params); \ + GLOBAL_ATOM_DEF(peer_auth_chunks); \ + GLOBAL_ATOM_DEF(pktinfo); \ + GLOBAL_ATOM_DEF(pktoptions); \ + GLOBAL_ATOM_DEF(port); \ + GLOBAL_ATOM_DEF(portrange); \ + GLOBAL_ATOM_DEF(primary_addr); \ + GLOBAL_ATOM_DEF(priority); \ + GLOBAL_ATOM_DEF(protocol); \ + GLOBAL_ATOM_DEF(raw); \ + GLOBAL_ATOM_DEF(rcvbuf); \ + GLOBAL_ATOM_DEF(rcvbufforce); \ + GLOBAL_ATOM_DEF(rcvlowat); \ + GLOBAL_ATOM_DEF(rcvtimeo); \ + GLOBAL_ATOM_DEF(rdm); \ + GLOBAL_ATOM_DEF(recv); \ + GLOBAL_ATOM_DEF(recvdstaddr); \ + GLOBAL_ATOM_DEF(recverr); \ + GLOBAL_ATOM_DEF(recvfrom); \ + GLOBAL_ATOM_DEF(recvif); \ + GLOBAL_ATOM_DEF(recvmsg); \ + GLOBAL_ATOM_DEF(recvopts); \ + GLOBAL_ATOM_DEF(recvorigdstaddr); \ + GLOBAL_ATOM_DEF(recvpktinfo); \ + GLOBAL_ATOM_DEF(recvtclass); \ + GLOBAL_ATOM_DEF(recvtos); \ + GLOBAL_ATOM_DEF(recvttl); \ + GLOBAL_ATOM_DEF(reliability); \ + GLOBAL_ATOM_DEF(reset_streams); \ + GLOBAL_ATOM_DEF(retopts); \ + GLOBAL_ATOM_DEF(reuseaddr); \ + GLOBAL_ATOM_DEF(reuseport); \ + GLOBAL_ATOM_DEF(rights); \ + GLOBAL_ATOM_DEF(router_alert); \ + GLOBAL_ATOM_DEF(rthdr); \ + GLOBAL_ATOM_DEF(rtoinfo); \ + GLOBAL_ATOM_DEF(rxq_ovfl); \ + GLOBAL_ATOM_DEF(scope_id); \ + GLOBAL_ATOM_DEF(sctp); \ + GLOBAL_ATOM_DEF(sec); \ + GLOBAL_ATOM_DEF(select_failed); \ + GLOBAL_ATOM_DEF(select_sent); \ + GLOBAL_ATOM_DEF(send); \ + GLOBAL_ATOM_DEF(sendmsg); \ + GLOBAL_ATOM_DEF(sendsrcaddr); \ + GLOBAL_ATOM_DEF(sendto); \ + GLOBAL_ATOM_DEF(seqpacket); \ + GLOBAL_ATOM_DEF(setfib); \ + GLOBAL_ATOM_DEF(set_peer_primary_addr); \ + GLOBAL_ATOM_DEF(sndbuf); \ + GLOBAL_ATOM_DEF(sndbufforce); \ + GLOBAL_ATOM_DEF(sndlowat); \ + GLOBAL_ATOM_DEF(sndtimeo); \ + GLOBAL_ATOM_DEF(socket); \ + GLOBAL_ATOM_DEF(socket_tag); \ + GLOBAL_ATOM_DEF(spec_dst); \ + GLOBAL_ATOM_DEF(status); \ + GLOBAL_ATOM_DEF(stream); \ + GLOBAL_ATOM_DEF(syncnt); \ + GLOBAL_ATOM_DEF(tclass); \ + GLOBAL_ATOM_DEF(tcp); \ + GLOBAL_ATOM_DEF(throughput); \ + GLOBAL_ATOM_DEF(timestamp); \ + GLOBAL_ATOM_DEF(tos); \ + GLOBAL_ATOM_DEF(transparent); \ + GLOBAL_ATOM_DEF(true); \ + GLOBAL_ATOM_DEF(trunc); \ + GLOBAL_ATOM_DEF(ttl); \ + GLOBAL_ATOM_DEF(type); \ + GLOBAL_ATOM_DEF(udp); \ + GLOBAL_ATOM_DEF(unblock_source); \ + GLOBAL_ATOM_DEF(undefined); \ + GLOBAL_ATOM_DEF(unicast_hops); \ + GLOBAL_ATOM_DEF(unknown); \ + GLOBAL_ATOM_DEF(usec); \ + GLOBAL_ATOM_DEF(user_timeout); \ + GLOBAL_ATOM_DEF(use_ext_recvinfo); \ + GLOBAL_ATOM_DEF(use_min_mtu); \ + GLOBAL_ATOM_DEF(v6only); /* ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - * Error value (=reason) atoms + * Error reason atoms */ -extern ERL_NIF_TERM esock_atom_eafnosupport; -extern ERL_NIF_TERM esock_atom_eagain; -extern ERL_NIF_TERM esock_atom_einval; +#define GLOBAL_ERROR_REASON_ATOM_DEFS \ + GLOBAL_ATOM_DEF(eafnosupport); \ + GLOBAL_ATOM_DEF(eagain); \ + GLOBAL_ATOM_DEF(einval); + + +#define GLOBAL_ATOM_DEF(A) extern ERL_NIF_TERM esock_atom_##A +GLOBAL_ATOM_DEFS +GLOBAL_ERROR_REASON_ATOM_DEFS +#undef GLOBAL_ATOM_DEF /* ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ @@ -343,14 +352,13 @@ extern ERL_NIF_TERM esock_atom_einval; #define MLOCK(M) enif_mutex_lock((M)) #define MUNLOCK(M) enif_mutex_unlock((M)) -// #define MONP(S,E,D,P,M) enif_monitor_process((E), (D), (P), (M)) -// #define DEMONP(S,E,D,M) enif_demonitor_process((E), (D), (M)) #define MONP(S,E,D,P,M) esock_monitor((S), (E), (D), (P), (M)) #define DEMONP(S,E,D,M) esock_demonitor((S), (E), (D), (M)) #define MON_INIT(M) esock_monitor_init((M)) -// #define MON_COMP(M1, M2) esock_monitor_compare((M1), (M2)) +#define MON2T(E, M) enif_make_monitor_term((E), (M)) -#define COMPARE(A, B) enif_compare((A), (B)) +#define COMPARE(A, B) enif_compare((A), (B)) +#define COMPARE_PIDS(P1, P2) enif_compare_pids((P1), (P2)) #define IS_ATOM(E, TE) enif_is_atom((E), (TE)) #define IS_BIN(E, TE) enif_is_binary((E), (TE)) diff --git a/erts/emulator/nifs/common/socket_nif.c b/erts/emulator/nifs/common/socket_nif.c index d56b70e3fd..052c585032 100644 --- a/erts/emulator/nifs/common/socket_nif.c +++ b/erts/emulator/nifs/common/socket_nif.c @@ -32,7 +32,7 @@ * * esock_dbg_printf("DEMONP", "[%d] %s: %T\r\n", * descP->sock, slogan, - * my_make_monitor_term(env, &monP->mon)); + * MON2T(env, &monP->mon)); * */ @@ -392,6 +392,13 @@ static void (*esock_sctp_freepaddrs)(struct sockaddr *addrs) = NULL; #endif +#if defined(TCP_CONGESTION) || defined(SO_BINDTODEVICE) +#define USE_GETOPT_STR_OPT +#define USE_SETOPT_STR_OPT +#endif + + + /* *** Socket state defs *** */ #define SOCKET_FLAG_OPEN 0x0001 @@ -882,81 +889,81 @@ typedef struct { */ +extern char* erl_errno_id(int error); /* THIS IS JUST TEMPORARY??? */ -static ERL_NIF_TERM nif_info(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_supports(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -/* -This is a *global* debug function (enable or disable for all -operations and all sockets. -static ERL_NIF_TERM nif_debug(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -*/ -static ERL_NIF_TERM nif_open(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_bind(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_connect(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_listen(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_accept(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_send(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_sendto(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_sendmsg(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_recv(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_recvfrom(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_recvmsg(ErlNifEnv* env, - int argc, + +/* All the nif "callback" functions for the socket API has + * the exact same API: + * + * nif_<funcname>(ErlNifEnv* env, + * int argc, + * const ERL_NIF_TERM argv[]); + * + * So, to simplify, use some macro magic to define those. + * + * These are the functions making up the "official" API. + * Basically, these functions does some preliminary checks and argument + * extractions and then call the functions called 'n<funcname>', which + * does the actual work. Except for the info function. + * + * nif_info + * nif_supports + * nif_open + * nif_bind + * nif_connect + * nif_listen + * nif_accept + * nif_send + * nif_sendto + * nif_sendmsg + * nif_recv + * nif_recvfrom + * nif_recvmsg + * nif_close + * nif_shutdown + * nif_setopt + * nif_getopt + * nif_sockname + * nif_peername + * nif_finalize_connection + * nif_finalize_close + * nif_cancel + */ + +#define ESOCK_NIF_FUNCS \ + ESOCK_NIF_FUNC_DEF(info); \ + ESOCK_NIF_FUNC_DEF(supports); \ + ESOCK_NIF_FUNC_DEF(open); \ + ESOCK_NIF_FUNC_DEF(bind); \ + ESOCK_NIF_FUNC_DEF(connect); \ + ESOCK_NIF_FUNC_DEF(listen); \ + ESOCK_NIF_FUNC_DEF(accept); \ + ESOCK_NIF_FUNC_DEF(send); \ + ESOCK_NIF_FUNC_DEF(sendto); \ + ESOCK_NIF_FUNC_DEF(sendmsg); \ + ESOCK_NIF_FUNC_DEF(recv); \ + ESOCK_NIF_FUNC_DEF(recvfrom); \ + ESOCK_NIF_FUNC_DEF(recvmsg); \ + ESOCK_NIF_FUNC_DEF(close); \ + ESOCK_NIF_FUNC_DEF(shutdown); \ + ESOCK_NIF_FUNC_DEF(setopt); \ + ESOCK_NIF_FUNC_DEF(getopt); \ + ESOCK_NIF_FUNC_DEF(sockname); \ + ESOCK_NIF_FUNC_DEF(peername); \ + ESOCK_NIF_FUNC_DEF(finalize_connection); \ + ESOCK_NIF_FUNC_DEF(finalize_close); \ + ESOCK_NIF_FUNC_DEF(cancel); + +#define ESOCK_NIF_FUNC_DEF(F) \ + static ERL_NIF_TERM nif_##F(ErlNifEnv* env, \ + int argc, \ const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_close(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_shutdown(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_setopt(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_getopt(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_sockname(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_peername(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_finalize_connection(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_finalize_close(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM nif_cancel(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]); +ESOCK_NIF_FUNCS +#undef ESOCK_NIF_FUNC_DEF + +#if !defined(__WIN32__) +/* And here comes the functions that does the actual work (for the most part) */ static ERL_NIF_TERM nsupports(ErlNifEnv* env, int key); static ERL_NIF_TERM nsupports_options(ErlNifEnv* env); static ERL_NIF_TERM nsupports_options_socket(ErlNifEnv* env); @@ -984,6 +991,7 @@ static ERL_NIF_TERM nlisten(ErlNifEnv* env, int backlog); static ERL_NIF_TERM naccept(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM ref); static ERL_NIF_TERM naccept_listening(ErlNifEnv* env, SocketDescriptor* descP, @@ -1000,17 +1008,21 @@ static ERL_NIF_TERM naccept_listening_accept(ErlNifEnv* env, SocketAddress* remote); static ERL_NIF_TERM naccept_accepting(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM ref); static ERL_NIF_TERM naccept_accepting_current(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM ref); static ERL_NIF_TERM naccept_accepting_current_accept(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, SOCKET accSock, SocketAddress* remote); static ERL_NIF_TERM naccept_accepting_current_error(ErlNifEnv* env, SocketDescriptor* descP, - ERL_NIF_TERM ref, + ERL_NIF_TERM sockRef, + ERL_NIF_TERM opRef, int save_errno); static ERL_NIF_TERM naccept_accepting_other(ErlNifEnv* env, SocketDescriptor* descP, @@ -1078,28 +1090,34 @@ static ERL_NIF_TERM nsetopt(ErlNifEnv* env, int level, int eOpt, ERL_NIF_TERM eVal); + +/* Set OTP level options */ static ERL_NIF_TERM nsetopt_otp(ErlNifEnv* env, SocketDescriptor* descP, int eOpt, ERL_NIF_TERM eVal); -static ERL_NIF_TERM nsetopt_otp_debug(ErlNifEnv* env, - SocketDescriptor* descP, - ERL_NIF_TERM eVal); -static ERL_NIF_TERM nsetopt_otp_iow(ErlNifEnv* env, - SocketDescriptor* descP, - ERL_NIF_TERM eVal); -static ERL_NIF_TERM nsetopt_otp_ctrl_proc(ErlNifEnv* env, - SocketDescriptor* descP, - ERL_NIF_TERM eVal); -static ERL_NIF_TERM nsetopt_otp_rcvbuf(ErlNifEnv* env, - SocketDescriptor* descP, - ERL_NIF_TERM eVal); -static ERL_NIF_TERM nsetopt_otp_rcvctrlbuf(ErlNifEnv* env, - SocketDescriptor* descP, - ERL_NIF_TERM eVal); -static ERL_NIF_TERM nsetopt_otp_sndctrlbuf(ErlNifEnv* env, - SocketDescriptor* descP, - ERL_NIF_TERM eVal); +/* *** nsetopt_otp_debug *** + * *** nsetopt_otp_iow *** + * *** nsetopt_otp_ctrl_proc *** + * *** nsetopt_otp_rcvbuf *** + * *** nsetopt_otp_rcvctrlbuf *** + * *** nsetopt_otp_sndctrlbuf *** + */ +#define NSETOPT_OTP_FUNCS \ + NSETOPT_OTP_FUNC_DEF(debug); \ + NSETOPT_OTP_FUNC_DEF(iow); \ + NSETOPT_OTP_FUNC_DEF(ctrl_proc); \ + NSETOPT_OTP_FUNC_DEF(rcvbuf); \ + NSETOPT_OTP_FUNC_DEF(rcvctrlbuf); \ + NSETOPT_OTP_FUNC_DEF(sndctrlbuf); +#define NSETOPT_OTP_FUNC_DEF(F) \ + static ERL_NIF_TERM nsetopt_otp_##F(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + ERL_NIF_TERM eVal) +NSETOPT_OTP_FUNCS +#undef NSETOPT_OTP_FUNC_DEF + +/* Set native options */ static ERL_NIF_TERM nsetopt_native(ErlNifEnv* env, SocketDescriptor* descP, int level, @@ -1579,29 +1597,38 @@ static ERL_NIF_TERM ngetopt(ErlNifEnv* env, BOOLEAN_T isOTP, int level, ERL_NIF_TERM eOpt); + static ERL_NIF_TERM ngetopt_otp(ErlNifEnv* env, SocketDescriptor* descP, int eOpt); -static ERL_NIF_TERM ngetopt_otp_debug(ErlNifEnv* env, - SocketDescriptor* descP); -static ERL_NIF_TERM ngetopt_otp_iow(ErlNifEnv* env, - SocketDescriptor* descP); -static ERL_NIF_TERM ngetopt_otp_ctrl_proc(ErlNifEnv* env, - SocketDescriptor* descP); -static ERL_NIF_TERM ngetopt_otp_rcvbuf(ErlNifEnv* env, - SocketDescriptor* descP); -static ERL_NIF_TERM ngetopt_otp_rcvctrlbuf(ErlNifEnv* env, - SocketDescriptor* descP); -static ERL_NIF_TERM ngetopt_otp_sndctrlbuf(ErlNifEnv* env, - SocketDescriptor* descP); -static ERL_NIF_TERM ngetopt_otp_fd(ErlNifEnv* env, - SocketDescriptor* descP); -static ERL_NIF_TERM ngetopt_otp_domain(ErlNifEnv* env, - SocketDescriptor* descP); -static ERL_NIF_TERM ngetopt_otp_type(ErlNifEnv* env, - SocketDescriptor* descP); -static ERL_NIF_TERM ngetopt_otp_protocol(ErlNifEnv* env, - SocketDescriptor* descP); +/* *** ngetopt_otp_debug *** + * *** ngetopt_otp_iow *** + * *** ngetopt_otp_ctrl_proc *** + * *** ngetopt_otp_rcvbuf *** + * *** ngetopt_otp_rcvctrlbuf *** + * *** ngetopt_otp_sndctrlbuf *** + * *** ngetopt_otp_fd *** + * *** ngetopt_otp_domain *** + * *** ngetopt_otp_type *** + * *** ngetopt_otp_protocol *** + */ +#define NGETOPT_OTP_FUNCS \ + NGETOPT_OTP_FUNC_DEF(debug); \ + NGETOPT_OTP_FUNC_DEF(iow); \ + NGETOPT_OTP_FUNC_DEF(ctrl_proc); \ + NGETOPT_OTP_FUNC_DEF(rcvbuf); \ + NGETOPT_OTP_FUNC_DEF(rcvctrlbuf); \ + NGETOPT_OTP_FUNC_DEF(sndctrlbuf); \ + NGETOPT_OTP_FUNC_DEF(fd); \ + NGETOPT_OTP_FUNC_DEF(domain); \ + NGETOPT_OTP_FUNC_DEF(type); \ + NGETOPT_OTP_FUNC_DEF(protocol); +#define NGETOPT_OTP_FUNC_DEF(F) \ + static ERL_NIF_TERM ngetopt_otp_##F(ErlNifEnv* env, \ + SocketDescriptor* descP) +NGETOPT_OTP_FUNCS +#undef NGETOPT_OTP_FUNC_DEF + static ERL_NIF_TERM ngetopt_native(ErlNifEnv* env, SocketDescriptor* descP, int level, @@ -1938,31 +1965,38 @@ static ERL_NIF_TERM npeername(ErlNifEnv* env, static ERL_NIF_TERM ncancel(ErlNifEnv* env, SocketDescriptor* descP, ERL_NIF_TERM op, + ERL_NIF_TERM sockRef, ERL_NIF_TERM opRef); static ERL_NIF_TERM ncancel_connect(ErlNifEnv* env, SocketDescriptor* descP, ERL_NIF_TERM opRef); static ERL_NIF_TERM ncancel_accept(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM opRef); static ERL_NIF_TERM ncancel_accept_current(ErlNifEnv* env, - SocketDescriptor* descP); + SocketDescriptor* descP, + ERL_NIF_TERM sockRef); static ERL_NIF_TERM ncancel_accept_waiting(ErlNifEnv* env, SocketDescriptor* descP, ERL_NIF_TERM opRef); static ERL_NIF_TERM ncancel_send(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM opRef); static ERL_NIF_TERM ncancel_send_current(ErlNifEnv* env, - SocketDescriptor* descP); + SocketDescriptor* descP, + ERL_NIF_TERM sockRef); static ERL_NIF_TERM ncancel_send_waiting(ErlNifEnv* env, SocketDescriptor* descP, ERL_NIF_TERM opRef); static ERL_NIF_TERM ncancel_recv(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM opRef); static ERL_NIF_TERM ncancel_recv_current(ErlNifEnv* env, - SocketDescriptor* descP); + SocketDescriptor* descP, + ERL_NIF_TERM sockRef); static ERL_NIF_TERM ncancel_recv_waiting(ErlNifEnv* env, SocketDescriptor* descP, ERL_NIF_TERM opRef); @@ -1978,12 +2012,14 @@ static ERL_NIF_TERM ncancel_mode_select(ErlNifEnv* env, int smode, int rmode); +#if defined(USE_SETOPT_STR_OPT) static ERL_NIF_TERM nsetopt_str_opt(ErlNifEnv* env, SocketDescriptor* descP, int level, int opt, int max, ERL_NIF_TERM eVal); +#endif static ERL_NIF_TERM nsetopt_bool_opt(ErlNifEnv* env, SocketDescriptor* descP, int level, @@ -2000,11 +2036,13 @@ static ERL_NIF_TERM nsetopt_timeval_opt(ErlNifEnv* env, int opt, ERL_NIF_TERM eVal); +#if defined(USE_GETOPT_STR_OPT) static ERL_NIF_TERM ngetopt_str_opt(ErlNifEnv* env, SocketDescriptor* descP, int level, int opt, int max); +#endif static ERL_NIF_TERM ngetopt_bool_opt(ErlNifEnv* env, SocketDescriptor* descP, int level, @@ -2037,7 +2075,8 @@ static char* recv_init_current_reader(ErlNifEnv* env, SocketDescriptor* descP, ERL_NIF_TERM ref); static ERL_NIF_TERM recv_update_current_reader(ErlNifEnv* env, - SocketDescriptor* descP); + SocketDescriptor* descP, + ERL_NIF_TERM sockRef); static void recv_error_current_reader(ErlNifEnv* env, SocketDescriptor* descP, ERL_NIF_TERM sockRef, @@ -2206,6 +2245,7 @@ static BOOLEAN_T decode_native_get_opt(ErlNifEnv* env, static ERL_NIF_TERM encode_ip_tos(ErlNifEnv* env, int val); static void inform_waiting_procs(ErlNifEnv* env, + char* role, SocketDescriptor* descP, SocketRequestQueue* q, BOOLEAN_T free, @@ -2221,11 +2261,6 @@ static BOOLEAN_T verify_is_connected(SocketDescriptor* descP, int* err); static SocketDescriptor* alloc_descriptor(SOCKET sock, HANDLE event); -static int compare_pids(ErlNifEnv* env, - const ErlNifPid* pid1, - const ErlNifPid* pid2); - - static BOOLEAN_T edomain2domain(int edomain, int* domain); static BOOLEAN_T etype2type(int etype, int* type); @@ -2252,56 +2287,71 @@ static void inc_socket(int domain, int type, int protocol); static void dec_socket(int domain, int type, int protocol); -static BOOLEAN_T acceptor_search4pid(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid); -static ERL_NIF_TERM acceptor_push(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid pid, - ERL_NIF_TERM ref); -static BOOLEAN_T acceptor_pop(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid, - // ErlNifMonitor* mon, - ESockMonitor* mon, - ERL_NIF_TERM* ref); -static BOOLEAN_T acceptor_unqueue(ErlNifEnv* env, - SocketDescriptor* descP, - const ErlNifPid* pid); -static BOOLEAN_T writer_search4pid(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid); -static ERL_NIF_TERM writer_push(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid pid, - ERL_NIF_TERM ref); -static BOOLEAN_T writer_pop(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid, - // ErlNifMonitor* mon, - ESockMonitor* mon, - ERL_NIF_TERM* ref); -static BOOLEAN_T writer_unqueue(ErlNifEnv* env, - SocketDescriptor* descP, - const ErlNifPid* pid); +/* *** activate_next_acceptor *** + * *** activate_next_writer *** + * *** activate_next_reader *** + * + * All the activate-next functions for acceptor, writer and reader + * have exactly the same API, so we apply some macro magic to simplify. + * They simply operates on dufferent data structures. + * + */ -static BOOLEAN_T reader_search4pid(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid); -static ERL_NIF_TERM reader_push(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid pid, - ERL_NIF_TERM ref); -static BOOLEAN_T reader_pop(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid, - // ErlNifMonitor* mon, - ESockMonitor* mon, - ERL_NIF_TERM* ref); -static BOOLEAN_T reader_unqueue(ErlNifEnv* env, - SocketDescriptor* descP, - const ErlNifPid* pid); +#define ACTIVATE_NEXT_FUNCS_DEFS \ + ACTIVATE_NEXT_FUNC_DEF(acceptor) \ + ACTIVATE_NEXT_FUNC_DEF(writer) \ + ACTIVATE_NEXT_FUNC_DEF(reader) + +#define ACTIVATE_NEXT_FUNC_DEF(F) \ + static BOOLEAN_T activate_next_##F(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + ERL_NIF_TERM sockRef); +ACTIVATE_NEXT_FUNCS_DEFS +#undef ACTIVATE_NEXT_FUNC_DEF + +/* +static BOOLEAN_T activate_next(ErlNifEnv* env, + SocketDescriptor* descP, + SocketRequestor* reqP, + SocketRequestQueue* q, + ERL_NIF_TERM sockRef); +*/ + +/* *** acceptor_search4pid | writer_search4pid | reader_search4pid *** + * *** acceptor_push | writer_push | reader_push *** + * *** acceptor_pop | writer_pop | reader_pop *** + * *** acceptor_unqueue | writer_unqueue | reader_unqueue *** + * + * All the queue operator functions (search4pid, push, pop + * and unqueue) for acceptor, writer and reader has exactly + * the same API, so we apply some macro magic to simplify. + */ + +#define ESOCK_OPERATOR_FUNCS_DEFS \ + ESOCK_OPERATOR_FUNCS_DEF(acceptor) \ + ESOCK_OPERATOR_FUNCS_DEF(writer) \ + ESOCK_OPERATOR_FUNCS_DEF(reader) + +#define ESOCK_OPERATOR_FUNCS_DEF(O) \ + static BOOLEAN_T O##_search4pid(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + ErlNifPid* pid); \ + static ERL_NIF_TERM O##_push(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + ErlNifPid pid, \ + ERL_NIF_TERM ref); \ + static BOOLEAN_T O##_pop(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + SocketRequestor* reqP); \ + static BOOLEAN_T O##_unqueue(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + const ErlNifPid* pid); +ESOCK_OPERATOR_FUNCS_DEFS +#undef ESOCK_OPERATOR_FUNCS_DEF + +static BOOLEAN_T requestor_pop(SocketRequestQueue* q, + SocketRequestor* reqP); static BOOLEAN_T qsearch4pid(ErlNifEnv* env, SocketRequestQueue* q, @@ -2325,11 +2375,8 @@ static int esock_demonitor(const char* slogan, SocketDescriptor* descP, ESockMonitor* monP); static void esock_monitor_init(ESockMonitor* mon); -/* -static int esock_monitor_compare(const ErlNifMonitor* mon1, - const ESockMonitor* mon2); -*/ +#endif // if defined(__WIN32__) /* #if defined(HAVE_SYS_UN_H) || defined(SO_BINDTODEVICE) @@ -2346,14 +2393,20 @@ static void socket_down(ErlNifEnv* env, void* obj, const ErlNifPid* pid, const ErlNifMonitor* mon); + +#if !defined(__WIN32__) + static void socket_down_acceptor(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, const ErlNifPid* pid); static void socket_down_writer(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, const ErlNifPid* pid); static void socket_down_reader(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, const ErlNifPid* pid); static char* esock_send_close_msg(ErlNifEnv* env, @@ -2397,6 +2450,8 @@ static BOOLEAN_T extract_debug(ErlNifEnv* env, static BOOLEAN_T extract_iow(ErlNifEnv* env, ERL_NIF_TERM map); +#endif // if defined(__WIN32__) + static int on_load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info); @@ -2424,354 +2479,301 @@ static const struct in6_addr in6addr_loopback = -/* *** String constants *** */ -static char str_adaptation_layer[] = "adaptation_layer"; -static char str_address[] = "address"; -static char str_association[] = "association"; -static char str_assoc_id[] = "assoc_id"; -static char str_authentication[] = "authentication"; -// static char str_any[] = "any"; -static char str_bool[] = "bool"; -static char str_close[] = "close"; -static char str_closed[] = "closed"; -static char str_closing[] = "closing"; -static char str_cookie_life[] = "cookie_life"; -static char str_data_in[] = "data_in"; -static char str_do[] = "do"; -static char str_dont[] = "dont"; -static char str_exclude[] = "exclude"; -static char str_false[] = "false"; -static char str_global_counters[] = "global_counters"; -static char str_in4_sockaddr[] = "in4_sockaddr"; -static char str_in6_sockaddr[] = "in6_sockaddr"; -static char str_include[] = "include"; -static char str_initial[] = "initial"; -static char str_int[] = "int"; -static char str_interface[] = "interface"; -static char str_iow[] = "iow"; -static char str_local_rwnd[] = "local_rwnd"; -// static char str_loopback[] = "loopback"; -static char str_max[] = "max"; -static char str_max_attempts[] = "max_attempts"; -static char str_max_init_timeo[] = "max_init_timeo"; -static char str_max_instreams[] = "max_instreams"; -static char str_max_rxt[] = "max_rxt"; -static char str_min[] = "min"; -static char str_mode[] = "mode"; -static char str_multiaddr[] = "multiaddr"; -// static char str_nif_abort[] = "nif_abort"; -static char str_null[] = "null"; -static char str_num_dlocal[] = "num_domain_local"; -static char str_num_dinet[] = "num_domain_inet"; -static char str_num_dinet6[] = "num_domain_inet6"; -static char str_num_outstreams[] = "num_outstreams"; -static char str_num_peer_dests[] = "num_peer_dests"; -static char str_num_pip[] = "num_proto_ip"; -static char str_num_psctp[] = "num_proto_sctp"; -static char str_num_ptcp[] = "num_proto_tcp"; -static char str_num_pudp[] = "num_proto_udp"; -static char str_num_sockets[] = "num_sockets"; -static char str_num_tdgrams[] = "num_type_dgram"; -static char str_num_tseqpkgs[] = "num_type_seqpacket"; -static char str_num_tstreams[] = "num_type_stream"; -static char str_partial_delivery[] = "partial_delivery"; -static char str_peer_error[] = "peer_error"; -static char str_peer_rwnd[] = "peer_rwnd"; -static char str_probe[] = "probe"; -static char str_select[] = "select"; -static char str_sender_dry[] = "sender_dry"; -static char str_send_failure[] = "send_failure"; -static char str_shutdown[] = "shutdown"; -static char str_slist[] = "slist"; -static char str_sourceaddr[] = "sourceaddr"; -static char str_timeout[] = "timeout"; -static char str_true[] = "true"; -static char str_want[] = "want"; - /* (special) error string constants */ -static char str_eisconn[] = "eisconn"; -static char str_enotclosing[] = "enotclosing"; -static char str_enotconn[] = "enotconn"; -static char str_exalloc[] = "exalloc"; -static char str_exbadstate[] = "exbadstate"; -static char str_exbusy[] = "exbusy"; static char str_exmon[] = "exmonitor"; // failed monitor static char str_exself[] = "exself"; // failed self static char str_exsend[] = "exsend"; // failed send -/* *** "Global" Atoms *** */ -ERL_NIF_TERM esock_atom_abort; -ERL_NIF_TERM esock_atom_accept; -ERL_NIF_TERM esock_atom_acceptconn; -ERL_NIF_TERM esock_atom_acceptfilter; -ERL_NIF_TERM esock_atom_adaption_layer; -ERL_NIF_TERM esock_atom_addr; -ERL_NIF_TERM esock_atom_addrform; -ERL_NIF_TERM esock_atom_add_membership; -ERL_NIF_TERM esock_atom_add_source_membership; -ERL_NIF_TERM esock_atom_any; -ERL_NIF_TERM esock_atom_associnfo; -ERL_NIF_TERM esock_atom_authhdr; -ERL_NIF_TERM esock_atom_auth_active_key; -ERL_NIF_TERM esock_atom_auth_asconf; -ERL_NIF_TERM esock_atom_auth_chunk; -ERL_NIF_TERM esock_atom_auth_delete_key; -ERL_NIF_TERM esock_atom_auth_key; -ERL_NIF_TERM esock_atom_auth_level; -ERL_NIF_TERM esock_atom_autoclose; -ERL_NIF_TERM esock_atom_bindtodevice; -ERL_NIF_TERM esock_atom_block_source; -ERL_NIF_TERM esock_atom_broadcast; -ERL_NIF_TERM esock_atom_busy_poll; -ERL_NIF_TERM esock_atom_checksum; -ERL_NIF_TERM esock_atom_close; -ERL_NIF_TERM esock_atom_connect; -ERL_NIF_TERM esock_atom_congestion; -ERL_NIF_TERM esock_atom_context; -ERL_NIF_TERM esock_atom_cork; -ERL_NIF_TERM esock_atom_credentials; -ERL_NIF_TERM esock_atom_ctrl; -ERL_NIF_TERM esock_atom_ctrunc; -ERL_NIF_TERM esock_atom_data; -ERL_NIF_TERM esock_atom_debug; -ERL_NIF_TERM esock_atom_default_send_params; -ERL_NIF_TERM esock_atom_delayed_ack_time; -ERL_NIF_TERM esock_atom_dgram; -ERL_NIF_TERM esock_atom_disable_fragments; -ERL_NIF_TERM esock_atom_domain; -ERL_NIF_TERM esock_atom_dontfrag; -ERL_NIF_TERM esock_atom_dontroute; -ERL_NIF_TERM esock_atom_drop_membership; -ERL_NIF_TERM esock_atom_drop_source_membership; -ERL_NIF_TERM esock_atom_dstopts; -ERL_NIF_TERM esock_atom_eor; -ERL_NIF_TERM esock_atom_error; -ERL_NIF_TERM esock_atom_errqueue; -ERL_NIF_TERM esock_atom_esp_network_level; -ERL_NIF_TERM esock_atom_esp_trans_level; -ERL_NIF_TERM esock_atom_events; -ERL_NIF_TERM esock_atom_explicit_eor; -ERL_NIF_TERM esock_atom_faith; -ERL_NIF_TERM esock_atom_false; -ERL_NIF_TERM esock_atom_family; -ERL_NIF_TERM esock_atom_flags; -ERL_NIF_TERM esock_atom_flowinfo; -ERL_NIF_TERM esock_atom_fragment_interleave; -ERL_NIF_TERM esock_atom_freebind; -ERL_NIF_TERM esock_atom_get_peer_addr_info; -ERL_NIF_TERM esock_atom_hdrincl; -ERL_NIF_TERM esock_atom_hmac_ident; -ERL_NIF_TERM esock_atom_hoplimit; -ERL_NIF_TERM esock_atom_hopopts; -ERL_NIF_TERM esock_atom_ifindex; -ERL_NIF_TERM esock_atom_inet; -ERL_NIF_TERM esock_atom_inet6; -ERL_NIF_TERM esock_atom_info; -ERL_NIF_TERM esock_atom_initmsg; -ERL_NIF_TERM esock_atom_iov; -ERL_NIF_TERM esock_atom_ip; -ERL_NIF_TERM esock_atom_ipcomp_level; -ERL_NIF_TERM esock_atom_ipv6; -ERL_NIF_TERM esock_atom_i_want_mapped_v4_addr; -ERL_NIF_TERM esock_atom_join_group; -ERL_NIF_TERM esock_atom_keepalive; -ERL_NIF_TERM esock_atom_keepcnt; -ERL_NIF_TERM esock_atom_keepidle; -ERL_NIF_TERM esock_atom_keepintvl; -ERL_NIF_TERM esock_atom_leave_group; -ERL_NIF_TERM esock_atom_level; -ERL_NIF_TERM esock_atom_linger; -ERL_NIF_TERM esock_atom_local; -ERL_NIF_TERM esock_atom_local_auth_chunks; -ERL_NIF_TERM esock_atom_loopback; -ERL_NIF_TERM esock_atom_lowdelay; -ERL_NIF_TERM esock_atom_mark; -ERL_NIF_TERM esock_atom_maxburst; -ERL_NIF_TERM esock_atom_maxseg; -ERL_NIF_TERM esock_atom_md5sig; -ERL_NIF_TERM esock_atom_mincost; -ERL_NIF_TERM esock_atom_minttl; -ERL_NIF_TERM esock_atom_msfilter; -ERL_NIF_TERM esock_atom_mtu; -ERL_NIF_TERM esock_atom_mtu_discover; -ERL_NIF_TERM esock_atom_multicast_all; -ERL_NIF_TERM esock_atom_multicast_hops; -ERL_NIF_TERM esock_atom_multicast_if; -ERL_NIF_TERM esock_atom_multicast_loop; -ERL_NIF_TERM esock_atom_multicast_ttl; -ERL_NIF_TERM esock_atom_nodelay; -ERL_NIF_TERM esock_atom_nodefrag; -ERL_NIF_TERM esock_atom_noopt; -ERL_NIF_TERM esock_atom_nopush; -ERL_NIF_TERM esock_atom_not_found; -ERL_NIF_TERM esock_atom_not_owner; -ERL_NIF_TERM esock_atom_ok; -ERL_NIF_TERM esock_atom_oob; -ERL_NIF_TERM esock_atom_oobinline; -ERL_NIF_TERM esock_atom_options; -ERL_NIF_TERM esock_atom_origdstaddr; -ERL_NIF_TERM esock_atom_partial_delivery_point; -ERL_NIF_TERM esock_atom_passcred; -ERL_NIF_TERM esock_atom_path; -ERL_NIF_TERM esock_atom_peekcred; -ERL_NIF_TERM esock_atom_peek_off; -ERL_NIF_TERM esock_atom_peer_addr_params; -ERL_NIF_TERM esock_atom_peer_auth_chunks; -ERL_NIF_TERM esock_atom_pktinfo; -ERL_NIF_TERM esock_atom_pktoptions; -ERL_NIF_TERM esock_atom_port; -ERL_NIF_TERM esock_atom_portrange; -ERL_NIF_TERM esock_atom_primary_addr; -ERL_NIF_TERM esock_atom_priority; -ERL_NIF_TERM esock_atom_protocol; -ERL_NIF_TERM esock_atom_raw; -ERL_NIF_TERM esock_atom_rcvbuf; -ERL_NIF_TERM esock_atom_rcvbufforce; -ERL_NIF_TERM esock_atom_rcvlowat; -ERL_NIF_TERM esock_atom_rcvtimeo; -ERL_NIF_TERM esock_atom_rdm; -ERL_NIF_TERM esock_atom_recv; -ERL_NIF_TERM esock_atom_recvdstaddr; -ERL_NIF_TERM esock_atom_recverr; -ERL_NIF_TERM esock_atom_recvfrom; -ERL_NIF_TERM esock_atom_recvif; -ERL_NIF_TERM esock_atom_recvmsg; -ERL_NIF_TERM esock_atom_recvopts; -ERL_NIF_TERM esock_atom_recvorigdstaddr; -ERL_NIF_TERM esock_atom_recvpktinfo; -ERL_NIF_TERM esock_atom_recvtclass; -ERL_NIF_TERM esock_atom_recvtos; -ERL_NIF_TERM esock_atom_recvttl; -ERL_NIF_TERM esock_atom_reliability; -ERL_NIF_TERM esock_atom_reset_streams; -ERL_NIF_TERM esock_atom_retopts; -ERL_NIF_TERM esock_atom_reuseaddr; -ERL_NIF_TERM esock_atom_reuseport; -ERL_NIF_TERM esock_atom_rights; -ERL_NIF_TERM esock_atom_router_alert; -ERL_NIF_TERM esock_atom_rthdr; -ERL_NIF_TERM esock_atom_rtoinfo; -ERL_NIF_TERM esock_atom_rxq_ovfl; -ERL_NIF_TERM esock_atom_scope_id; -ERL_NIF_TERM esock_atom_sctp; -ERL_NIF_TERM esock_atom_sec; -ERL_NIF_TERM esock_atom_select_failed; -ERL_NIF_TERM esock_atom_select_sent; -ERL_NIF_TERM esock_atom_send; -ERL_NIF_TERM esock_atom_sendmsg; -ERL_NIF_TERM esock_atom_sendsrcaddr; -ERL_NIF_TERM esock_atom_sendto; -ERL_NIF_TERM esock_atom_seqpacket; -ERL_NIF_TERM esock_atom_setfib; -ERL_NIF_TERM esock_atom_set_peer_primary_addr; -ERL_NIF_TERM esock_atom_socket; -ERL_NIF_TERM esock_atom_socket_tag; -ERL_NIF_TERM esock_atom_sndbuf; -ERL_NIF_TERM esock_atom_sndbufforce; -ERL_NIF_TERM esock_atom_sndlowat; -ERL_NIF_TERM esock_atom_sndtimeo; -ERL_NIF_TERM esock_atom_spec_dst; -ERL_NIF_TERM esock_atom_status; -ERL_NIF_TERM esock_atom_stream; -ERL_NIF_TERM esock_atom_syncnt; -ERL_NIF_TERM esock_atom_tclass; -ERL_NIF_TERM esock_atom_tcp; -ERL_NIF_TERM esock_atom_throughput; -ERL_NIF_TERM esock_atom_timestamp; -ERL_NIF_TERM esock_atom_tos; -ERL_NIF_TERM esock_atom_transparent; -ERL_NIF_TERM esock_atom_true; -ERL_NIF_TERM esock_atom_trunc; -ERL_NIF_TERM esock_atom_ttl; -ERL_NIF_TERM esock_atom_type; -ERL_NIF_TERM esock_atom_udp; -ERL_NIF_TERM esock_atom_unblock_source; -ERL_NIF_TERM esock_atom_undefined; -ERL_NIF_TERM esock_atom_unicast_hops; -ERL_NIF_TERM esock_atom_unknown; -ERL_NIF_TERM esock_atom_usec; -ERL_NIF_TERM esock_atom_user_timeout; -ERL_NIF_TERM esock_atom_use_ext_recvinfo; -ERL_NIF_TERM esock_atom_use_min_mtu; -ERL_NIF_TERM esock_atom_v6only; - -/* *** "Global" error (=reason) atoms *** */ -ERL_NIF_TERM esock_atom_eagain; -ERL_NIF_TERM esock_atom_eafnosupport; -ERL_NIF_TERM esock_atom_einval; - -/* *** Atoms *** */ -static ERL_NIF_TERM atom_adaptation_layer; -static ERL_NIF_TERM atom_address; -static ERL_NIF_TERM atom_association; -static ERL_NIF_TERM atom_assoc_id; -static ERL_NIF_TERM atom_authentication; -static ERL_NIF_TERM atom_bool; -static ERL_NIF_TERM atom_close; -static ERL_NIF_TERM atom_closed; -static ERL_NIF_TERM atom_closing; -static ERL_NIF_TERM atom_cookie_life; -static ERL_NIF_TERM atom_data_in; -static ERL_NIF_TERM atom_do; -static ERL_NIF_TERM atom_dont; -static ERL_NIF_TERM atom_exclude; -static ERL_NIF_TERM atom_false; -static ERL_NIF_TERM atom_global_counters; -static ERL_NIF_TERM atom_in4_sockaddr; -static ERL_NIF_TERM atom_in6_sockaddr; -static ERL_NIF_TERM atom_include; -static ERL_NIF_TERM atom_initial; -static ERL_NIF_TERM atom_int; -static ERL_NIF_TERM atom_interface; -static ERL_NIF_TERM atom_iow; -static ERL_NIF_TERM atom_local_rwnd; -static ERL_NIF_TERM atom_max; -static ERL_NIF_TERM atom_max_attempts; -static ERL_NIF_TERM atom_max_init_timeo; -static ERL_NIF_TERM atom_max_instreams; -static ERL_NIF_TERM atom_max_rxt; -static ERL_NIF_TERM atom_min; -static ERL_NIF_TERM atom_mode; -static ERL_NIF_TERM atom_multiaddr; -// static ERL_NIF_TERM atom_nif_abort; -static ERL_NIF_TERM atom_null; -static ERL_NIF_TERM atom_num_dinet; -static ERL_NIF_TERM atom_num_dinet6; -static ERL_NIF_TERM atom_num_dlocal; -static ERL_NIF_TERM atom_num_outstreams; -static ERL_NIF_TERM atom_num_peer_dests; -static ERL_NIF_TERM atom_num_pip; -static ERL_NIF_TERM atom_num_psctp; -static ERL_NIF_TERM atom_num_ptcp; -static ERL_NIF_TERM atom_num_pudp; -static ERL_NIF_TERM atom_num_sockets; -static ERL_NIF_TERM atom_num_tdgrams; -static ERL_NIF_TERM atom_num_tseqpkgs; -static ERL_NIF_TERM atom_num_tstreams; -static ERL_NIF_TERM atom_partial_delivery; -static ERL_NIF_TERM atom_peer_error; -static ERL_NIF_TERM atom_peer_rwnd; -static ERL_NIF_TERM atom_probe; -static ERL_NIF_TERM atom_select; -static ERL_NIF_TERM atom_sender_dry; -static ERL_NIF_TERM atom_send_failure; -static ERL_NIF_TERM atom_shutdown; -static ERL_NIF_TERM atom_slist; -static ERL_NIF_TERM atom_sourceaddr; -static ERL_NIF_TERM atom_timeout; -static ERL_NIF_TERM atom_true; -static ERL_NIF_TERM atom_want; - -static ERL_NIF_TERM atom_eisconn; -static ERL_NIF_TERM atom_enotclosing; -static ERL_NIF_TERM atom_enotconn; -static ERL_NIF_TERM atom_exalloc; -static ERL_NIF_TERM atom_exbadstate; -static ERL_NIF_TERM atom_exbusy; -static ERL_NIF_TERM atom_exmon; -static ERL_NIF_TERM atom_exself; -static ERL_NIF_TERM atom_exsend; + +/* *** Global atoms *** */ +#define GLOBAL_ATOMS \ + GLOBAL_ATOM_DECL(abort); \ + GLOBAL_ATOM_DECL(accept); \ + GLOBAL_ATOM_DECL(acceptconn); \ + GLOBAL_ATOM_DECL(acceptfilter); \ + GLOBAL_ATOM_DECL(adaption_layer); \ + GLOBAL_ATOM_DECL(addr); \ + GLOBAL_ATOM_DECL(addrform); \ + GLOBAL_ATOM_DECL(add_membership); \ + GLOBAL_ATOM_DECL(add_source_membership); \ + GLOBAL_ATOM_DECL(any); \ + GLOBAL_ATOM_DECL(associnfo); \ + GLOBAL_ATOM_DECL(authhdr); \ + GLOBAL_ATOM_DECL(auth_active_key); \ + GLOBAL_ATOM_DECL(auth_asconf); \ + GLOBAL_ATOM_DECL(auth_chunk); \ + GLOBAL_ATOM_DECL(auth_delete_key); \ + GLOBAL_ATOM_DECL(auth_key); \ + GLOBAL_ATOM_DECL(auth_level); \ + GLOBAL_ATOM_DECL(autoclose); \ + GLOBAL_ATOM_DECL(bindtodevice); \ + GLOBAL_ATOM_DECL(block_source); \ + GLOBAL_ATOM_DECL(broadcast); \ + GLOBAL_ATOM_DECL(busy_poll); \ + GLOBAL_ATOM_DECL(checksum); \ + GLOBAL_ATOM_DECL(close); \ + GLOBAL_ATOM_DECL(connect); \ + GLOBAL_ATOM_DECL(congestion); \ + GLOBAL_ATOM_DECL(context); \ + GLOBAL_ATOM_DECL(cork); \ + GLOBAL_ATOM_DECL(credentials); \ + GLOBAL_ATOM_DECL(ctrl); \ + GLOBAL_ATOM_DECL(ctrunc); \ + GLOBAL_ATOM_DECL(data); \ + GLOBAL_ATOM_DECL(debug); \ + GLOBAL_ATOM_DECL(default_send_params); \ + GLOBAL_ATOM_DECL(delayed_ack_time); \ + GLOBAL_ATOM_DECL(dgram); \ + GLOBAL_ATOM_DECL(disable_fragments); \ + GLOBAL_ATOM_DECL(domain); \ + GLOBAL_ATOM_DECL(dontfrag); \ + GLOBAL_ATOM_DECL(dontroute); \ + GLOBAL_ATOM_DECL(drop_membership); \ + GLOBAL_ATOM_DECL(drop_source_membership); \ + GLOBAL_ATOM_DECL(dstopts); \ + GLOBAL_ATOM_DECL(eor); \ + GLOBAL_ATOM_DECL(error); \ + GLOBAL_ATOM_DECL(errqueue); \ + GLOBAL_ATOM_DECL(esp_network_level); \ + GLOBAL_ATOM_DECL(esp_trans_level); \ + GLOBAL_ATOM_DECL(events); \ + GLOBAL_ATOM_DECL(explicit_eor); \ + GLOBAL_ATOM_DECL(faith); \ + GLOBAL_ATOM_DECL(false); \ + GLOBAL_ATOM_DECL(family); \ + GLOBAL_ATOM_DECL(flags); \ + GLOBAL_ATOM_DECL(flowinfo); \ + GLOBAL_ATOM_DECL(fragment_interleave); \ + GLOBAL_ATOM_DECL(freebind); \ + GLOBAL_ATOM_DECL(get_peer_addr_info); \ + GLOBAL_ATOM_DECL(hdrincl); \ + GLOBAL_ATOM_DECL(hmac_ident); \ + GLOBAL_ATOM_DECL(hoplimit); \ + GLOBAL_ATOM_DECL(hopopts); \ + GLOBAL_ATOM_DECL(ifindex); \ + GLOBAL_ATOM_DECL(inet); \ + GLOBAL_ATOM_DECL(inet6); \ + GLOBAL_ATOM_DECL(info); \ + GLOBAL_ATOM_DECL(initmsg); \ + GLOBAL_ATOM_DECL(iov); \ + GLOBAL_ATOM_DECL(ip); \ + GLOBAL_ATOM_DECL(ipcomp_level); \ + GLOBAL_ATOM_DECL(ipv6); \ + GLOBAL_ATOM_DECL(i_want_mapped_v4_addr); \ + GLOBAL_ATOM_DECL(join_group); \ + GLOBAL_ATOM_DECL(keepalive); \ + GLOBAL_ATOM_DECL(keepcnt); \ + GLOBAL_ATOM_DECL(keepidle); \ + GLOBAL_ATOM_DECL(keepintvl); \ + GLOBAL_ATOM_DECL(leave_group); \ + GLOBAL_ATOM_DECL(level); \ + GLOBAL_ATOM_DECL(linger); \ + GLOBAL_ATOM_DECL(local); \ + GLOBAL_ATOM_DECL(local_auth_chunks); \ + GLOBAL_ATOM_DECL(loopback); \ + GLOBAL_ATOM_DECL(lowdelay); \ + GLOBAL_ATOM_DECL(mark); \ + GLOBAL_ATOM_DECL(maxburst); \ + GLOBAL_ATOM_DECL(maxseg); \ + GLOBAL_ATOM_DECL(md5sig); \ + GLOBAL_ATOM_DECL(mincost); \ + GLOBAL_ATOM_DECL(minttl); \ + GLOBAL_ATOM_DECL(msfilter); \ + GLOBAL_ATOM_DECL(mtu); \ + GLOBAL_ATOM_DECL(mtu_discover); \ + GLOBAL_ATOM_DECL(multicast_all); \ + GLOBAL_ATOM_DECL(multicast_hops); \ + GLOBAL_ATOM_DECL(multicast_if); \ + GLOBAL_ATOM_DECL(multicast_loop); \ + GLOBAL_ATOM_DECL(multicast_ttl); \ + GLOBAL_ATOM_DECL(nodelay); \ + GLOBAL_ATOM_DECL(nodefrag); \ + GLOBAL_ATOM_DECL(noopt); \ + GLOBAL_ATOM_DECL(nopush); \ + GLOBAL_ATOM_DECL(not_found); \ + GLOBAL_ATOM_DECL(not_owner); \ + GLOBAL_ATOM_DECL(ok); \ + GLOBAL_ATOM_DECL(oob); \ + GLOBAL_ATOM_DECL(oobinline); \ + GLOBAL_ATOM_DECL(options); \ + GLOBAL_ATOM_DECL(origdstaddr); \ + GLOBAL_ATOM_DECL(partial_delivery_point); \ + GLOBAL_ATOM_DECL(passcred); \ + GLOBAL_ATOM_DECL(path); \ + GLOBAL_ATOM_DECL(peekcred); \ + GLOBAL_ATOM_DECL(peek_off); \ + GLOBAL_ATOM_DECL(peer_addr_params); \ + GLOBAL_ATOM_DECL(peer_auth_chunks); \ + GLOBAL_ATOM_DECL(pktinfo); \ + GLOBAL_ATOM_DECL(pktoptions); \ + GLOBAL_ATOM_DECL(port); \ + GLOBAL_ATOM_DECL(portrange); \ + GLOBAL_ATOM_DECL(primary_addr); \ + GLOBAL_ATOM_DECL(priority); \ + GLOBAL_ATOM_DECL(protocol); \ + GLOBAL_ATOM_DECL(raw); \ + GLOBAL_ATOM_DECL(rcvbuf); \ + GLOBAL_ATOM_DECL(rcvbufforce); \ + GLOBAL_ATOM_DECL(rcvlowat); \ + GLOBAL_ATOM_DECL(rcvtimeo); \ + GLOBAL_ATOM_DECL(rdm); \ + GLOBAL_ATOM_DECL(recv); \ + GLOBAL_ATOM_DECL(recvdstaddr); \ + GLOBAL_ATOM_DECL(recverr); \ + GLOBAL_ATOM_DECL(recvfrom); \ + GLOBAL_ATOM_DECL(recvif); \ + GLOBAL_ATOM_DECL(recvmsg); \ + GLOBAL_ATOM_DECL(recvopts); \ + GLOBAL_ATOM_DECL(recvorigdstaddr); \ + GLOBAL_ATOM_DECL(recvpktinfo); \ + GLOBAL_ATOM_DECL(recvtclass); \ + GLOBAL_ATOM_DECL(recvtos); \ + GLOBAL_ATOM_DECL(recvttl); \ + GLOBAL_ATOM_DECL(reliability); \ + GLOBAL_ATOM_DECL(reset_streams); \ + GLOBAL_ATOM_DECL(retopts); \ + GLOBAL_ATOM_DECL(reuseaddr); \ + GLOBAL_ATOM_DECL(reuseport); \ + GLOBAL_ATOM_DECL(rights); \ + GLOBAL_ATOM_DECL(router_alert); \ + GLOBAL_ATOM_DECL(rthdr); \ + GLOBAL_ATOM_DECL(rtoinfo); \ + GLOBAL_ATOM_DECL(rxq_ovfl); \ + GLOBAL_ATOM_DECL(scope_id); \ + GLOBAL_ATOM_DECL(sctp); \ + GLOBAL_ATOM_DECL(sec); \ + GLOBAL_ATOM_DECL(select_failed); \ + GLOBAL_ATOM_DECL(select_sent); \ + GLOBAL_ATOM_DECL(send); \ + GLOBAL_ATOM_DECL(sendmsg); \ + GLOBAL_ATOM_DECL(sendsrcaddr); \ + GLOBAL_ATOM_DECL(sendto); \ + GLOBAL_ATOM_DECL(seqpacket); \ + GLOBAL_ATOM_DECL(setfib); \ + GLOBAL_ATOM_DECL(set_peer_primary_addr); \ + GLOBAL_ATOM_DECL(socket); \ + GLOBAL_ATOM_DECL(sndbuf); \ + GLOBAL_ATOM_DECL(sndbufforce); \ + GLOBAL_ATOM_DECL(sndlowat); \ + GLOBAL_ATOM_DECL(sndtimeo); \ + GLOBAL_ATOM_DECL(spec_dst); \ + GLOBAL_ATOM_DECL(status); \ + GLOBAL_ATOM_DECL(stream); \ + GLOBAL_ATOM_DECL(syncnt); \ + GLOBAL_ATOM_DECL(tclass); \ + GLOBAL_ATOM_DECL(tcp); \ + GLOBAL_ATOM_DECL(throughput); \ + GLOBAL_ATOM_DECL(timestamp); \ + GLOBAL_ATOM_DECL(tos); \ + GLOBAL_ATOM_DECL(transparent); \ + GLOBAL_ATOM_DECL(true); \ + GLOBAL_ATOM_DECL(trunc); \ + GLOBAL_ATOM_DECL(ttl); \ + GLOBAL_ATOM_DECL(type); \ + GLOBAL_ATOM_DECL(udp); \ + GLOBAL_ATOM_DECL(unblock_source); \ + GLOBAL_ATOM_DECL(undefined); \ + GLOBAL_ATOM_DECL(unicast_hops); \ + GLOBAL_ATOM_DECL(unknown); \ + GLOBAL_ATOM_DECL(usec); \ + GLOBAL_ATOM_DECL(user_timeout); \ + GLOBAL_ATOM_DECL(use_ext_recvinfo); \ + GLOBAL_ATOM_DECL(use_min_mtu); \ + GLOBAL_ATOM_DECL(v6only); + + +/* *** Global error reason atoms *** */ +#define GLOBAL_ERROR_REASON_ATOMS \ + GLOBAL_ATOM_DECL(eagain); \ + GLOBAL_ATOM_DECL(eafnosupport); \ + GLOBAL_ATOM_DECL(einval); + + +#define GLOBAL_ATOM_DECL(A) ERL_NIF_TERM esock_atom_##A +GLOBAL_ATOMS +GLOBAL_ERROR_REASON_ATOMS +#undef GLOBAL_ATOM_DECL +ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') + +/* *** Local atoms *** */ +#define LOCAL_ATOMS \ + LOCAL_ATOM_DECL(adaptation_layer); \ + LOCAL_ATOM_DECL(address); \ + LOCAL_ATOM_DECL(association); \ + LOCAL_ATOM_DECL(assoc_id); \ + LOCAL_ATOM_DECL(authentication); \ + LOCAL_ATOM_DECL(bool); \ + LOCAL_ATOM_DECL(close); \ + LOCAL_ATOM_DECL(closed); \ + LOCAL_ATOM_DECL(closing); \ + LOCAL_ATOM_DECL(cookie_life); \ + LOCAL_ATOM_DECL(data_in); \ + LOCAL_ATOM_DECL(do); \ + LOCAL_ATOM_DECL(dont); \ + LOCAL_ATOM_DECL(exclude); \ + LOCAL_ATOM_DECL(false); \ + LOCAL_ATOM_DECL(global_counters); \ + LOCAL_ATOM_DECL(in4_sockaddr); \ + LOCAL_ATOM_DECL(in6_sockaddr); \ + LOCAL_ATOM_DECL(include); \ + LOCAL_ATOM_DECL(initial); \ + LOCAL_ATOM_DECL(int); \ + LOCAL_ATOM_DECL(interface); \ + LOCAL_ATOM_DECL(iow); \ + LOCAL_ATOM_DECL(local_rwnd); \ + LOCAL_ATOM_DECL(max); \ + LOCAL_ATOM_DECL(max_attempts); \ + LOCAL_ATOM_DECL(max_init_timeo); \ + LOCAL_ATOM_DECL(max_instreams); \ + LOCAL_ATOM_DECL(max_rxt); \ + LOCAL_ATOM_DECL(min); \ + LOCAL_ATOM_DECL(mode); \ + LOCAL_ATOM_DECL(multiaddr); \ + LOCAL_ATOM_DECL(null); \ + LOCAL_ATOM_DECL(num_dinet); \ + LOCAL_ATOM_DECL(num_dinet6); \ + LOCAL_ATOM_DECL(num_dlocal); \ + LOCAL_ATOM_DECL(num_outstreams); \ + LOCAL_ATOM_DECL(num_peer_dests); \ + LOCAL_ATOM_DECL(num_pip); \ + LOCAL_ATOM_DECL(num_psctp); \ + LOCAL_ATOM_DECL(num_ptcp); \ + LOCAL_ATOM_DECL(num_pudp); \ + LOCAL_ATOM_DECL(num_sockets); \ + LOCAL_ATOM_DECL(num_tdgrams); \ + LOCAL_ATOM_DECL(num_tseqpkgs); \ + LOCAL_ATOM_DECL(num_tstreams); \ + LOCAL_ATOM_DECL(partial_delivery); \ + LOCAL_ATOM_DECL(peer_error); \ + LOCAL_ATOM_DECL(peer_rwnd); \ + LOCAL_ATOM_DECL(probe); \ + LOCAL_ATOM_DECL(select); \ + LOCAL_ATOM_DECL(sender_dry); \ + LOCAL_ATOM_DECL(send_failure); \ + LOCAL_ATOM_DECL(shutdown); \ + LOCAL_ATOM_DECL(slist); \ + LOCAL_ATOM_DECL(sourceaddr); \ + LOCAL_ATOM_DECL(timeout); \ + LOCAL_ATOM_DECL(true); \ + LOCAL_ATOM_DECL(want); + +/* Local error reason atoms */ +#define LOCAL_ERROR_REASON_ATOMS \ + LOCAL_ATOM_DECL(eisconn); \ + LOCAL_ATOM_DECL(enotclosing); \ + LOCAL_ATOM_DECL(enotconn); \ + LOCAL_ATOM_DECL(exalloc); \ + LOCAL_ATOM_DECL(exbadstate); \ + LOCAL_ATOM_DECL(exbusy); \ + LOCAL_ATOM_DECL(exmon); \ + LOCAL_ATOM_DECL(exself); \ + LOCAL_ATOM_DECL(exsend); + +#define LOCAL_ATOM_DECL(LA) static ERL_NIF_TERM atom_##LA +LOCAL_ATOMS +LOCAL_ERROR_REASON_ATOMS +#undef LOCAL_ATOM_DECL /* *** Sockets *** */ @@ -2887,7 +2889,7 @@ ERL_NIF_TERM nif_info(ErlNifEnv* env, * * Description: * This function is intended to answer the question: "Is X supported?" - * Currently only one key is "supported": options + * Currently three keys are "supported": options | sctp | ipv6 * That results in a list of all *known options* (known by us) and if * the platform supports (OS) it or not. * @@ -4928,14 +4930,15 @@ ERL_NIF_TERM nif_accept(ErlNifEnv* env, return enif_raise_exception(env, MKA(env, "notsup")); #else SocketDescriptor* descP; - ERL_NIF_TERM ref, res; + ERL_NIF_TERM sockRef, ref, res; SGDBG( ("SOCKET", "nif_accept -> entry with argc: %d\r\n", argc) ); /* Extract arguments and perform preliminary validation */ + sockRef = argv[0]; if ((argc != 2) || - !enif_get_resource(env, argv[0], sockets, (void**) &descP)) { + !enif_get_resource(env, sockRef, sockets, (void**) &descP)) { return enif_make_badarg(env); } ref = argv[1]; @@ -4948,7 +4951,7 @@ ERL_NIF_TERM nif_accept(ErlNifEnv* env, MLOCK(descP->accMtx); - res = naccept(env, descP, ref); + res = naccept(env, descP, sockRef, ref); MUNLOCK(descP->accMtx); @@ -4962,6 +4965,7 @@ ERL_NIF_TERM nif_accept(ErlNifEnv* env, static ERL_NIF_TERM naccept(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM ref) { ERL_NIF_TERM res; @@ -4975,7 +4979,7 @@ ERL_NIF_TERM naccept(ErlNifEnv* env, break; case SOCKET_STATE_ACCEPTING: - res = naccept_accepting(env, descP, ref); + res = naccept_accepting(env, descP, sockRef, ref); break; default: @@ -5112,6 +5116,7 @@ ERL_NIF_TERM naccept_listening_accept(ErlNifEnv* env, static ERL_NIF_TERM naccept_accepting(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM ref) { ErlNifPid caller; @@ -5128,15 +5133,12 @@ ERL_NIF_TERM naccept_accepting(ErlNifEnv* env, "\r\n Current: %T" "\r\n", caller, descP->currentAcceptor.pid) ); - - - - if (compare_pids(env, &descP->currentAcceptor.pid, &caller)) { + if (COMPARE_PIDS(&descP->currentAcceptor.pid, &caller) == 0) { SSDBG( descP, ("SOCKET", "naccept_accepting -> current acceptor\r\n") ); - res = naccept_accepting_current(env, descP, ref); + res = naccept_accepting_current(env, descP, sockRef, ref); } else { @@ -5161,7 +5163,8 @@ ERL_NIF_TERM naccept_accepting(ErlNifEnv* env, static ERL_NIF_TERM naccept_accepting_current(ErlNifEnv* env, SocketDescriptor* descP, - ERL_NIF_TERM ref) + ERL_NIF_TERM sockRef, + ERL_NIF_TERM accRef) { SocketAddress remote; unsigned int n; @@ -5182,13 +5185,15 @@ ERL_NIF_TERM naccept_accepting_current(ErlNifEnv* env, "naccept_accepting_current -> accept failed: %d\r\n", save_errno) ); - res = naccept_accepting_current_error(env, descP, ref, save_errno); + res = naccept_accepting_current_error(env, descP, sockRef, + accRef, save_errno); } else { SSDBG( descP, ("SOCKET", "naccept_accepting_current -> accepted\r\n") ); - res = naccept_accepting_current_accept(env, descP, accSock, &remote); + res = naccept_accepting_current_accept(env, descP, sockRef, + accSock, &remote); } @@ -5203,10 +5208,10 @@ ERL_NIF_TERM naccept_accepting_current(ErlNifEnv* env, static ERL_NIF_TERM naccept_accepting_current_accept(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, SOCKET accSock, SocketAddress* remote) { - int sres; ERL_NIF_TERM res; if (naccept_accepted(env, descP, accSock, @@ -5220,33 +5225,21 @@ ERL_NIF_TERM naccept_accepting_current_accept(ErlNifEnv* env, * */ - if (acceptor_pop(env, descP, - &descP->currentAcceptor.pid, - &descP->currentAcceptor.mon, - &descP->currentAcceptor.ref)) { - - /* There was another one */ + if (!activate_next_acceptor(env, descP, sockRef)) { SSDBG( descP, ("SOCKET", - "naccept_accepting_current_accept -> new (active) acceptor: " - "\r\n pid: %T" - "\r\n ref: %T" - "\r\n", - descP->currentAcceptor.pid, - descP->currentAcceptor.ref) ); + "naccept_accepting_current_accept -> " + "no more writers\r\n") ); - if ((sres = esock_select_read(env, descP->sock, descP, - &descP->currentAcceptor.pid, - descP->currentAcceptor.ref)) < 0) { - esock_warning_msg("Failed select (%d) for new acceptor " - "after current (%T) died\r\n", - sres, descP->currentAcceptor.pid); - } - } else { - descP->currentAcceptorP = NULL; - descP->state = SOCKET_STATE_LISTENING; + descP->state = SOCKET_STATE_LISTENING; + + descP->currentAcceptorP = NULL; + descP->currentAcceptor.ref = esock_atom_undefined; + enif_set_pid_undefined(&descP->currentAcceptor.pid); + esock_monitor_init(&descP->currentAcceptor.mon); } + } return res; @@ -5262,10 +5255,12 @@ ERL_NIF_TERM naccept_accepting_current_accept(ErlNifEnv* env, static ERL_NIF_TERM naccept_accepting_current_error(ErlNifEnv* env, SocketDescriptor* descP, - ERL_NIF_TERM ref, + ERL_NIF_TERM sockRef, + ERL_NIF_TERM opRef, int save_errno) { - ERL_NIF_TERM res; + SocketRequestor req; + ERL_NIF_TERM res, reason; if (save_errno == ERRNO_BLOCK) { @@ -5275,14 +5270,27 @@ ERL_NIF_TERM naccept_accepting_current_error(ErlNifEnv* env, SSDBG( descP, ("SOCKET", - "naccept_accepting_current_error -> would block: try again\r\n") ); + "naccept_accepting_current_error -> " + "would block: try again\r\n") ); - res = naccept_busy_retry(env, descP, ref, &descP->currentAcceptor.pid, + res = naccept_busy_retry(env, descP, opRef, &descP->currentAcceptor.pid, /* No state change */ descP->state); } else { - res = esock_make_error_errno(env, save_errno); + + reason = MKA(env, erl_errno_id(save_errno)); + res = esock_make_error(env, reason); + + while (acceptor_pop(env, descP, &req)) { + SSDBG( descP, + ("SOCKET", "naccept_accepting_current_error -> abort %T\r\n", + req.pid) ); + esock_send_abort_msg(env, sockRef, req.ref, reason, &req.pid); + DEMONP("naccept_accepting_current_error -> pop'ed writer", + env, descP, &req.mon); + } + } return res; @@ -7002,7 +7010,7 @@ ERL_NIF_TERM nsetopt_otp_ctrl_proc(ErlNifEnv* env, if (enif_self(env, &caller) == NULL) return esock_make_error(env, atom_exself); - if (!compare_pids(env, &descP->ctrlPid, &caller)) { + if (COMPARE_PIDS(&descP->ctrlPid, &caller) != 0) { SSDBG( descP, ("SOCKET", "nsetopt_otp_ctrl_proc -> not owner (%T)\r\n", descP->ctrlPid) ); return esock_make_error(env, esock_atom_not_owner); @@ -9491,7 +9499,6 @@ ERL_NIF_TERM nsetopt_lvl_sctp_associnfo(ErlNifEnv* env, int res; size_t sz; unsigned int tmp; - Sint32 tmpAssocId; SSDBG( descP, ("SOCKET", "nsetopt_lvl_sctp_associnfo -> entry with" @@ -9532,10 +9539,29 @@ ERL_NIF_TERM nsetopt_lvl_sctp_associnfo(ErlNifEnv* env, /* On some platforms the assoc id is typed as an unsigned integer (uint32) * So, to avoid warnings there, we always make an explicit cast... + * Also, size of types matter, so adjust for that... */ - if (!GET_INT(env, eAssocId, &tmpAssocId)) - return esock_make_error(env, esock_atom_einval); - assocParams.sasoc_assoc_id = (typeof(assocParams.sasoc_assoc_id)) tmpAssocId; + +#if (SIZEOF_INT == 4) + { + int tmpAssocId; + if (!GET_INT(env, eAssocId, &tmpAssocId)) + return esock_make_error(env, esock_atom_einval); + assocParams.sasoc_assoc_id = + (typeof(assocParams.sasoc_assoc_id)) tmpAssocId; + } +#elif (SIZEOF_LONG == 4) + { + long tmpAssocId; + if (!GET_LONG(env, eAssocId, &tmpAssocId)) + return esock_make_error(env, esock_atom_einval); + assocParams.sasoc_assoc_id = + (typeof(assocParams.sasoc_assoc_id)) tmpAssocId; + } +#else + SIZE CHECK FOR ASSOC ID FAILED +#endif + /* * We should really make sure this is ok in erlang (to ensure that @@ -9620,7 +9646,10 @@ ERL_NIF_TERM nsetopt_lvl_sctp_events(ErlNifEnv* env, ERL_NIF_TERM result; ERL_NIF_TERM eDataIn, eAssoc, eAddr, eSndFailure; ERL_NIF_TERM ePeerError, eShutdown, ePartialDelivery; - ERL_NIF_TERM eAdaptLayer, eAuth; + ERL_NIF_TERM eAdaptLayer; +#if defined(HAVE_STRUCT_SCTP_EVENT_SUBSCRIBE_SCTP_AUTHENTICATION_EVENT) + ERL_NIF_TERM eAuth; +#endif #if defined(HAVE_STRUCT_SCTP_EVENT_SUBSCRIBE_SCTP_SENDER_DRY_EVENT) ERL_NIF_TERM eSndDry; #endif @@ -9841,7 +9870,6 @@ ERL_NIF_TERM nsetopt_lvl_sctp_rtoinfo(ErlNifEnv* env, struct sctp_rtoinfo rtoInfo; int res; size_t sz; - Sint32 tmpAssocId; SSDBG( descP, ("SOCKET", "nsetopt_lvl_sctp_rtoinfo -> entry with" @@ -9876,10 +9904,31 @@ ERL_NIF_TERM nsetopt_lvl_sctp_rtoinfo(ErlNifEnv* env, /* On some platforms the assoc id is typed as an unsigned integer (uint32) * So, to avoid warnings there, we always make an explicit cast... + * Also, size of types matter, so adjust for that... */ + +#if (SIZEOF_INT == 4) + { + int tmpAssocId; + if (!GET_INT(env, eAssocId, &tmpAssocId)) + return esock_make_error(env, esock_atom_einval); + rtoInfo.srto_assoc_id = (typeof(rtoInfo.srto_assoc_id)) tmpAssocId; + } +#elif (SIZEOF_LONG == 4) + { + long tmpAssocId; + if (!GET_LONG(env, eAssocId, &tmpAssocId)) + return esock_make_error(env, esock_atom_einval); + rtoInfo.srto_assoc_id = (typeof(rtoInfo.srto_assoc_id)) tmpAssocId; + } +#else + SIZE CHECK FOR ASSOC ID FAILED +#endif + /* if (!GET_INT(env, eAssocId, &tmpAssocId)) return esock_make_error(env, esock_atom_einval); rtoInfo.srto_assoc_id = (typeof(rtoInfo.srto_assoc_id)) tmpAssocId; + */ if (!GET_UINT(env, eInitial, &rtoInfo.srto_initial)) return esock_make_error(env, esock_atom_einval); @@ -9985,6 +10034,7 @@ ERL_NIF_TERM nsetopt_int_opt(ErlNifEnv* env, /* nsetopt_str_opt - set an option that has an string value */ +#if defined(USE_SETOPT_STR_OPT) static ERL_NIF_TERM nsetopt_str_opt(ErlNifEnv* env, SocketDescriptor* descP, @@ -10013,6 +10063,7 @@ ERL_NIF_TERM nsetopt_str_opt(ErlNifEnv* env, return result; } +#endif /* nsetopt_timeval_opt - set an option that has an (timeval) bool value @@ -10503,7 +10554,7 @@ static ERL_NIF_TERM ngetopt_otp_domain(ErlNifEnv* env, SocketDescriptor* descP) { - ERL_NIF_TERM result; + ERL_NIF_TERM result, reason; int val = descP->domain; switch (val) { @@ -10524,10 +10575,8 @@ ERL_NIF_TERM ngetopt_otp_domain(ErlNifEnv* env, #endif default: - result = esock_make_error(env, - MKT2(env, - esock_atom_unknown, - MKI(env, val))); + reason = MKT2(env, esock_atom_unknown, MKI(env, val)); + result = esock_make_error(env, reason); break; } @@ -10541,7 +10590,7 @@ static ERL_NIF_TERM ngetopt_otp_type(ErlNifEnv* env, SocketDescriptor* descP) { - ERL_NIF_TERM result; + ERL_NIF_TERM result, reason; int val = descP->type; switch (val) { @@ -10567,8 +10616,8 @@ ERL_NIF_TERM ngetopt_otp_type(ErlNifEnv* env, break; default: - result = esock_make_error(env, - MKT2(env, esock_atom_unknown, MKI(env, val))); + reason = MKT2(env, esock_atom_unknown, MKI(env, val)); + result = esock_make_error(env, reason); break; } @@ -10582,7 +10631,7 @@ static ERL_NIF_TERM ngetopt_otp_protocol(ErlNifEnv* env, SocketDescriptor* descP) { - ERL_NIF_TERM result; + ERL_NIF_TERM result, reason; int val = descP->protocol; switch (val) { @@ -10605,8 +10654,8 @@ ERL_NIF_TERM ngetopt_otp_protocol(ErlNifEnv* env, #endif default: - result = esock_make_error(env, - MKT2(env, esock_atom_unknown, MKI(env, val))); + reason = MKT2(env, esock_atom_unknown, MKI(env, val)); + result = esock_make_error(env, reason); break; } @@ -11021,7 +11070,7 @@ static ERL_NIF_TERM ngetopt_lvl_sock_domain(ErlNifEnv* env, SocketDescriptor* descP) { - ERL_NIF_TERM result; + ERL_NIF_TERM result, reason; int val; SOCKOPTLEN_T valSz = sizeof(val); int res; @@ -11050,10 +11099,8 @@ ERL_NIF_TERM ngetopt_lvl_sock_domain(ErlNifEnv* env, #endif default: - result = esock_make_error(env, - MKT2(env, - esock_atom_unknown, - MKI(env, val))); + reason = MKT2(env, esock_atom_unknown, MKI(env, val)); + result = esock_make_error(env, reason); break; } } @@ -11148,7 +11195,7 @@ static ERL_NIF_TERM ngetopt_lvl_sock_protocol(ErlNifEnv* env, SocketDescriptor* descP) { - ERL_NIF_TERM result; + ERL_NIF_TERM result, reason; int val; SOCKOPTLEN_T valSz = sizeof(val); int res; @@ -11179,8 +11226,8 @@ ERL_NIF_TERM ngetopt_lvl_sock_protocol(ErlNifEnv* env, #endif default: - result = esock_make_error(env, - MKT2(env, esock_atom_unknown, MKI(env, val))); + reason = MKT2(env, esock_atom_unknown, MKI(env, val)); + result = esock_make_error(env, reason); break; } } @@ -11285,7 +11332,7 @@ static ERL_NIF_TERM ngetopt_lvl_sock_type(ErlNifEnv* env, SocketDescriptor* descP) { - ERL_NIF_TERM result; + ERL_NIF_TERM result, reason; int val; SOCKOPTLEN_T valSz = sizeof(val); int res; @@ -11314,8 +11361,8 @@ ERL_NIF_TERM ngetopt_lvl_sock_type(ErlNifEnv* env, result = esock_make_ok2(env, esock_atom_rdm); break; default: - result = esock_make_error(env, - MKT2(env, esock_atom_unknown, MKI(env, val))); + reason = MKT2(env, esock_atom_unknown, MKI(env, val)); + result = esock_make_error(env, reason); break; } } @@ -11485,7 +11532,8 @@ ERL_NIF_TERM ngetopt_lvl_ip(ErlNifEnv* env, #endif default: - SSDBG( descP, ("SOCKET", "ngetopt_lvl_ip -> unknown opt %d\r\n", eOpt) ); + SSDBG( descP, + ("SOCKET", "ngetopt_lvl_ip -> unknown opt %d\r\n", eOpt) ); result = esock_make_error(env, esock_atom_einval); break; } @@ -12910,6 +12958,7 @@ ERL_NIF_TERM ngetopt_timeval_opt(ErlNifEnv* env, * The actual size of the (read) value will be communicated * in the optSz variable. */ +#if defined(USE_GETOPT_STR_OPT) static ERL_NIF_TERM ngetopt_str_opt(ErlNifEnv* env, SocketDescriptor* descP, @@ -12948,6 +12997,7 @@ ERL_NIF_TERM ngetopt_str_opt(ErlNifEnv* env, return result; } +#endif // if defined(USE_GETOPT_STR_OPT) #endif // if !defined(__WIN32__) @@ -13120,14 +13170,15 @@ ERL_NIF_TERM nif_cancel(ErlNifEnv* env, return enif_raise_exception(env, MKA(env, "notsup")); #else SocketDescriptor* descP; - ERL_NIF_TERM op, opRef, result; + ERL_NIF_TERM op, sockRef, opRef, result; SGDBG( ("SOCKET", "nif_cancel -> entry with argc: %d\r\n", argc) ); /* Extract arguments and perform preliminary validation */ + sockRef = argv[0]; if ((argc != 3) || - !enif_get_resource(env, argv[0], sockets, (void**) &descP)) { + !enif_get_resource(env, sockRef, sockets, (void**) &descP)) { return enif_make_badarg(env); } op = argv[1]; @@ -13142,7 +13193,7 @@ ERL_NIF_TERM nif_cancel(ErlNifEnv* env, "\r\n opRef: %T" "\r\n", descP->sock, op, opRef) ); - result = ncancel(env, descP, op, opRef); + result = ncancel(env, descP, op, sockRef, opRef); SSDBG( descP, ("SOCKET", "nif_cancel -> done with result: " @@ -13159,6 +13210,7 @@ static ERL_NIF_TERM ncancel(ErlNifEnv* env, SocketDescriptor* descP, ERL_NIF_TERM op, + ERL_NIF_TERM sockRef, ERL_NIF_TERM opRef) { /* <KOLLA> @@ -13172,19 +13224,19 @@ ERL_NIF_TERM ncancel(ErlNifEnv* env, if (COMPARE(op, esock_atom_connect) == 0) { return ncancel_connect(env, descP, opRef); } else if (COMPARE(op, esock_atom_accept) == 0) { - return ncancel_accept(env, descP, opRef); + return ncancel_accept(env, descP, sockRef, opRef); } else if (COMPARE(op, esock_atom_send) == 0) { - return ncancel_send(env, descP, opRef); + return ncancel_send(env, descP, sockRef, opRef); } else if (COMPARE(op, esock_atom_sendto) == 0) { - return ncancel_send(env, descP, opRef); + return ncancel_send(env, descP, sockRef, opRef); } else if (COMPARE(op, esock_atom_sendmsg) == 0) { - return ncancel_send(env, descP, opRef); + return ncancel_send(env, descP, sockRef, opRef); } else if (COMPARE(op, esock_atom_recv) == 0) { - return ncancel_recv(env, descP, opRef); + return ncancel_recv(env, descP, sockRef, opRef); } else if (COMPARE(op, esock_atom_recvfrom) == 0) { - return ncancel_recv(env, descP, opRef); + return ncancel_recv(env, descP, sockRef, opRef); } else if (COMPARE(op, esock_atom_recvmsg) == 0) { - return ncancel_recv(env, descP, opRef); + return ncancel_recv(env, descP, sockRef, opRef); } else { return esock_make_error(env, esock_atom_einval); } @@ -13218,6 +13270,7 @@ ERL_NIF_TERM ncancel_connect(ErlNifEnv* env, static ERL_NIF_TERM ncancel_accept(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM opRef) { ERL_NIF_TERM res; @@ -13233,7 +13286,7 @@ ERL_NIF_TERM ncancel_accept(ErlNifEnv* env, if (descP->currentAcceptorP != NULL) { if (COMPARE(opRef, descP->currentAcceptor.ref) == 0) { - res = ncancel_accept_current(env, descP); + res = ncancel_accept_current(env, descP, sockRef); } else { res = ncancel_accept_waiting(env, descP, opRef); } @@ -13259,9 +13312,9 @@ ERL_NIF_TERM ncancel_accept(ErlNifEnv* env, */ static ERL_NIF_TERM ncancel_accept_current(ErlNifEnv* env, - SocketDescriptor* descP) + SocketDescriptor* descP, + ERL_NIF_TERM sockRef) { - int sres; ERL_NIF_TERM res; SSDBG( descP, ("SOCKET", "ncancel_accept_current -> entry\r\n") ); @@ -13270,36 +13323,22 @@ ERL_NIF_TERM ncancel_accept_current(ErlNifEnv* env, env, descP, &descP->currentAcceptor.mon); res = ncancel_read_select(env, descP, descP->currentAcceptor.ref); - SSDBG( descP, ("SOCKET", "ncancel_accept_current -> cancel res: %T\r\n", res) ); + SSDBG( descP, ("SOCKET", + "ncancel_accept_current -> cancel res: %T\r\n", res) ); - if (acceptor_pop(env, descP, - &descP->currentAcceptor.pid, - &descP->currentAcceptor.mon, - &descP->currentAcceptor.ref)) { - - /* There was another one */ - - SSDBG( descP, ("SOCKET", "ncancel_accept_current -> new (active) acceptor: " - "\r\n pid: %T" - "\r\n ref: %T" - "\r\n", - descP->currentAcceptor.pid, - descP->currentAcceptor.ref) ); - - if ((sres = esock_select_read(env, descP->sock, descP, - &descP->currentAcceptor.pid, - descP->currentAcceptor.ref)) < 0) { - return esock_make_error(env, - MKT2(env, - esock_atom_select_failed, - MKI(env, sres))); - } - } else { - SSDBG( descP, ("SOCKET", "ncancel_accept_current -> no more acceptors\r\n") ); - descP->currentAcceptorP = NULL; - descP->state = SOCKET_STATE_LISTENING; + if (!activate_next_acceptor(env, descP, sockRef)) { + + SSDBG( descP, + ("SOCKET", "ncancel_accept_current -> no more writers\r\n") ); + + descP->state = SOCKET_STATE_LISTENING; + + descP->currentAcceptorP = NULL; + descP->currentAcceptor.ref = esock_atom_undefined; + enif_set_pid_undefined(&descP->currentAcceptor.pid); + esock_monitor_init(&descP->currentAcceptor.mon); } - + SSDBG( descP, ("SOCKET", "ncancel_accept_current -> done with result:" "\r\n %T" "\r\n", res) ); @@ -13341,6 +13380,7 @@ ERL_NIF_TERM ncancel_accept_waiting(ErlNifEnv* env, static ERL_NIF_TERM ncancel_send(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM opRef) { ERL_NIF_TERM res; @@ -13356,7 +13396,7 @@ ERL_NIF_TERM ncancel_send(ErlNifEnv* env, if (descP->currentWriterP != NULL) { if (COMPARE(opRef, descP->currentWriter.ref) == 0) { - res = ncancel_send_current(env, descP); + res = ncancel_send_current(env, descP, sockRef); } else { res = ncancel_send_waiting(env, descP, opRef); } @@ -13383,46 +13423,29 @@ ERL_NIF_TERM ncancel_send(ErlNifEnv* env, */ static ERL_NIF_TERM ncancel_send_current(ErlNifEnv* env, - SocketDescriptor* descP) + SocketDescriptor* descP, + ERL_NIF_TERM sockRef) { - int sres; ERL_NIF_TERM res; SSDBG( descP, ("SOCKET", "ncancel_send_current -> entry\r\n") ); - DEMONP("ncancel_recv_current -> current writer", + DEMONP("ncancel_send_current -> current writer", env, descP, &descP->currentWriter.mon); res = ncancel_write_select(env, descP, descP->currentWriter.ref); - SSDBG( descP, ("SOCKET", "ncancel_send_current -> cancel res: %T\r\n", res) ); + SSDBG( descP, + ("SOCKET", "ncancel_send_current -> cancel res: %T\r\n", res) ); - if (writer_pop(env, descP, - &descP->currentWriter.pid, - &descP->currentWriter.mon, - &descP->currentWriter.ref)) { - - /* There was another one */ - - SSDBG( descP, ("SOCKET", "ncancel_send_current -> new (active) writer: " - "\r\n pid: %T" - "\r\n ref: %T" - "\r\n", - descP->currentWriter.pid, - descP->currentWriter.ref) ); - - if ((sres = esock_select_write(env, descP->sock, descP, - &descP->currentWriter.pid, - descP->currentWriter.ref)) < 0) { - return esock_make_error(env, - MKT2(env, - esock_atom_select_failed, - MKI(env, sres))); - } - } else { - SSDBG( descP, ("SOCKET", "ncancel_send_current -> no more writers\r\n") ); - descP->currentWriterP = NULL; + if (!activate_next_writer(env, descP, sockRef)) { + SSDBG( descP, + ("SOCKET", "ncancel_send_current -> no more writers\r\n") ); + descP->currentWriterP = NULL; + descP->currentWriter.ref = esock_atom_undefined; + enif_set_pid_undefined(&descP->currentWriter.pid); + esock_monitor_init(&descP->currentWriter.mon); } - + SSDBG( descP, ("SOCKET", "ncancel_send_current -> done with result:" "\r\n %T" "\r\n", res) ); @@ -13464,6 +13487,7 @@ ERL_NIF_TERM ncancel_send_waiting(ErlNifEnv* env, static ERL_NIF_TERM ncancel_recv(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, ERL_NIF_TERM opRef) { ERL_NIF_TERM res; @@ -13479,7 +13503,7 @@ ERL_NIF_TERM ncancel_recv(ErlNifEnv* env, if (descP->currentReaderP != NULL) { if (COMPARE(opRef, descP->currentReader.ref) == 0) { - res = ncancel_recv_current(env, descP); + res = ncancel_recv_current(env, descP, sockRef); } else { res = ncancel_recv_waiting(env, descP, opRef); } @@ -13505,9 +13529,9 @@ ERL_NIF_TERM ncancel_recv(ErlNifEnv* env, */ static ERL_NIF_TERM ncancel_recv_current(ErlNifEnv* env, - SocketDescriptor* descP) + SocketDescriptor* descP, + ERL_NIF_TERM sockRef) { - int sres; ERL_NIF_TERM res; SSDBG( descP, ("SOCKET", "ncancel_recv_current -> entry\r\n") ); @@ -13516,35 +13540,18 @@ ERL_NIF_TERM ncancel_recv_current(ErlNifEnv* env, env, descP, &descP->currentReader.mon); res = ncancel_read_select(env, descP, descP->currentReader.ref); - SSDBG( descP, ("SOCKET", "ncancel_recv_current -> cancel res: %T\r\n", res) ); + SSDBG( descP, + ("SOCKET", "ncancel_recv_current -> cancel res: %T\r\n", res) ); - if (reader_pop(env, descP, - &descP->currentReader.pid, - &descP->currentReader.mon, - &descP->currentReader.ref)) { - - /* There was another one */ - - SSDBG( descP, ("SOCKET", "ncancel_recv_current -> new (active) reader: " - "\r\n pid: %T" - "\r\n ref: %T" - "\r\n", - descP->currentReader.pid, - descP->currentReader.ref) ); - - if ((sres = esock_select_read(env, descP->sock, descP, - &descP->currentReader.pid, - descP->currentReader.ref)) < 0) { - return esock_make_error(env, - MKT2(env, - esock_atom_select_failed, - MKI(env, sres))); - } - } else { - SSDBG( descP, ("SOCKET", "ncancel_recv_current -> no more readers\r\n") ); - descP->currentReaderP = NULL; + if (!activate_next_reader(env, descP, sockRef)) { + SSDBG( descP, + ("SOCKET", "ncancel_recv_current -> no more readers\r\n") ); + descP->currentReaderP = NULL; + descP->currentReader.ref = esock_atom_undefined; + enif_set_pid_undefined(&descP->currentReader.pid); + esock_monitor_init(&descP->currentReader.mon); } - + SSDBG( descP, ("SOCKET", "ncancel_recv_current -> done with result:" "\r\n %T" "\r\n", res) ); @@ -13653,7 +13660,7 @@ BOOLEAN_T send_check_writer(ErlNifEnv* env, return FALSE; } - if (!compare_pids(env, &descP->currentWriter.pid, &caller)) { + if (COMPARE_PIDS(&descP->currentWriter.pid, &caller) != 0) { /* Not the "current writer", so (maybe) push onto queue */ SSDBG( descP, @@ -13727,30 +13734,11 @@ ERL_NIF_TERM send_check_result(ErlNifEnv* env, /* Ok, this write is done maybe activate the next (if any) */ - if (writer_pop(env, descP, - &descP->currentWriter.pid, - &descP->currentWriter.mon, - &descP->currentWriter.ref)) { - - /* There was another one */ - - SSDBG( descP, ("SOCKET", "send_check_result -> new (active) writer: " - "\r\n pid: %T" - "\r\n ref: %T" - "\r\n", - descP->currentWriter.pid, - descP->currentWriter.ref) ); - - if ((sres = esock_select_write(env, descP->sock, descP, - &descP->currentWriter.pid, - descP->currentWriter.ref)) < 0) { - return esock_make_error(env, - MKT2(env, - esock_atom_select_failed, - MKI(env, sres))); - } - } else { - descP->currentWriterP = NULL; + if (!activate_next_writer(env, descP, sockRef)) { + descP->currentWriterP = NULL; + descP->currentWriter.ref = esock_atom_undefined; + enif_set_pid_undefined(&descP->currentWriter.pid); + esock_monitor_init(&descP->currentWriter.mon); } return esock_atom_ok; @@ -13760,10 +13748,8 @@ ERL_NIF_TERM send_check_result(ErlNifEnv* env, /* Some kind of send failure - check what kind */ if ((saveErrno != EAGAIN) && (saveErrno != EINTR)) { - ErlNifPid pid; - // ErlNifMonitor mon; - ESockMonitor mon; - ERL_NIF_TERM ref, res; + SocketRequestor req; + ERL_NIF_TERM res, reason; /* * An actual failure - we (and everyone waiting) give up @@ -13772,20 +13758,25 @@ ERL_NIF_TERM send_check_result(ErlNifEnv* env, cnt_inc(&descP->writeFails, 1); SSDBG( descP, - ("SOCKET", "send_check_result -> error: %d\r\n", saveErrno) ); + ("SOCKET", + "send_check_result -> error: %d\r\n", saveErrno) ); - res = esock_make_error_errno(env, saveErrno); + reason = MKA(env, erl_errno_id(saveErrno)); + res = esock_make_error(env, reason); if (descP->currentWriterP != NULL) { + DEMONP("send_check_result -> current writer", env, descP, &descP->currentWriter.mon); - while (writer_pop(env, descP, &pid, &mon, &ref)) { + while (writer_pop(env, descP, &req)) { SSDBG( descP, - ("SOCKET", "send_check_result -> abort %T\r\n", pid) ); - esock_send_abort_msg(env, sockRef, ref, res, &pid); + ("SOCKET", "send_check_result -> abort %T\r\n", + req.pid) ); + esock_send_abort_msg(env, sockRef, req.ref, + reason, &req.pid); DEMONP("send_check_result -> pop'ed writer", - env, descP, &mon); + env, descP, &req.mon); } } @@ -13877,7 +13868,7 @@ BOOLEAN_T recv_check_reader(ErlNifEnv* env, return FALSE; } - if (!compare_pids(env, &descP->currentReader.pid, &caller)) { + if (COMPARE_PIDS(&descP->currentReader.pid, &caller) != 0) { ERL_NIF_TERM tmp; /* Not the "current reader", so (maybe) push onto queue */ @@ -13950,9 +13941,9 @@ char* recv_init_current_reader(ErlNifEnv* env, static ERL_NIF_TERM recv_update_current_reader(ErlNifEnv* env, - SocketDescriptor* descP) + SocketDescriptor* descP, + ERL_NIF_TERM sockRef) { - int sres; ERL_NIF_TERM res = esock_atom_ok; if (descP->currentReaderP != NULL) { @@ -13960,32 +13951,18 @@ ERL_NIF_TERM recv_update_current_reader(ErlNifEnv* env, DEMONP("recv_update_current_reader -> current reader", env, descP, &descP->currentReader.mon); - if (reader_pop(env, descP, - &descP->currentReader.pid, - &descP->currentReader.mon, - &descP->currentReader.ref)) { - - /* There was another one */ - + if (!activate_next_reader(env, descP, sockRef)) { + SSDBG( descP, - ("SOCKET", "recv_update_current_reader -> new (active) reader: " - "\r\n pid: %T" - "\r\n ref: %T" - "\r\n", - descP->currentReader.pid, - descP->currentReader.ref) ); - - if ((sres = esock_select_read(env, descP->sock, descP, - &descP->currentReader.pid, - descP->currentReader.ref)) < 0) { - res = esock_make_error(env, - MKT2(env, - esock_atom_select_failed, - MKI(env, sres))); - } - } else { - descP->currentReaderP = NULL; + ("SOCKET", + "recv_update_current_reader -> no more readers\r\n") ); + + descP->currentReaderP = NULL; + descP->currentReader.ref = esock_atom_undefined; + enif_set_pid_undefined(&descP->currentReader.pid); + esock_monitor_init(&descP->currentReader.mon); } + } return res; @@ -14006,21 +13983,20 @@ void recv_error_current_reader(ErlNifEnv* env, ERL_NIF_TERM sockRef, ERL_NIF_TERM reason) { + SocketRequestor req; + if (descP->currentReaderP != NULL) { - ErlNifPid pid; - // ErlNifMonitor mon; - ESockMonitor mon; - ERL_NIF_TERM ref; DEMONP("recv_error_current_reader -> current reader", env, descP, &descP->currentReader.mon); - while (reader_pop(env, descP, &pid, &mon, &ref)) { + while (reader_pop(env, descP, &req)) { SSDBG( descP, - ("SOCKET", "recv_error_current_reader -> abort %T\r\n", pid) ); - esock_send_abort_msg(env, sockRef, ref, reason, &pid); + ("SOCKET", "recv_error_current_reader -> abort %T\r\n", + req.pid) ); + esock_send_abort_msg(env, sockRef, req.ref, reason, &req.pid); DEMONP("recv_error_current_reader -> pop'ed reader", - env, descP, &mon); + env, descP, &req.mon); } } } @@ -14137,7 +14113,7 @@ ERL_NIF_TERM recv_check_result(ErlNifEnv* env, cnt_inc(&descP->readPkgCnt, 1); - recv_update_current_reader(env, descP); + recv_update_current_reader(env, descP, sockRef); /* This transfers "ownership" of the *allocated* binary to an * erlang term (no need for an explicit free). @@ -14182,7 +14158,7 @@ ERL_NIF_TERM recv_check_result(ErlNifEnv* env, "recv_check_result -> [%d] " "we got exactly what we could fit\r\n", toRead) ); - recv_update_current_reader(env, descP); + recv_update_current_reader(env, descP, sockRef); /* This transfers "ownership" of the *allocated* binary to an * erlang term (no need for an explicit free). @@ -14294,7 +14270,7 @@ ERL_NIF_TERM recv_check_result(ErlNifEnv* env, cnt_inc(&descP->readPkgCnt, 1); cnt_inc(&descP->readByteCnt, read); - recv_update_current_reader(env, descP); + recv_update_current_reader(env, descP, sockRef); /* This transfers "ownership" of the *allocated* binary to an * erlang term (no need for an explicit free). @@ -14479,7 +14455,7 @@ ERL_NIF_TERM recvfrom_check_result(ErlNifEnv* env, data = MKSBIN(env, data, 0, read); } - recv_update_current_reader(env, descP); + recv_update_current_reader(env, descP, sockRef); return esock_make_ok2(env, MKT2(env, eSockAddr, data)); @@ -14644,7 +14620,7 @@ ERL_NIF_TERM recvmsg_check_result(ErlNifEnv* env, "recvmsg_check_result -> " "(msghdr) encode failed: %s\r\n", xres) ); - recv_update_current_reader(env, descP); + recv_update_current_reader(env, descP, sockRef); FREE_BIN(dataBufP); FREE_BIN(ctrlBufP); @@ -14656,7 +14632,7 @@ ERL_NIF_TERM recvmsg_check_result(ErlNifEnv* env, "recvmsg_check_result -> " "(msghdr) encode ok: %T\r\n", eMsgHdr) ); - recv_update_current_reader(env, descP); + recv_update_current_reader(env, descP, sockRef); return esock_make_ok2(env, eMsgHdr); } @@ -16299,6 +16275,7 @@ ERL_NIF_TERM encode_ip_tos(ErlNifEnv* env, int val) /* *** alloc_descriptor *** + * * Allocate and perform basic initialization of a socket descriptor. * */ @@ -16370,14 +16347,16 @@ SocketDescriptor* alloc_descriptor(SOCKET sock, HANDLE event) -/* decrement counters for when a socket is closed */ +/* Decrement counters for when a socket is closed + */ static void dec_socket(int domain, int type, int protocol) { MLOCK(data.cntMtx); cnt_dec(&data.numSockets, 1); - + + /* *** Domain counter *** */ if (domain == AF_INET) cnt_dec(&data.numDomainInet, 1); #if defined(HAVE_IN6) && defined(AF_INET6) @@ -16389,6 +16368,7 @@ void dec_socket(int domain, int type, int protocol) cnt_dec(&data.numDomainInet6, 1); #endif + /* *** Type counter *** */ if (type == SOCK_STREAM) cnt_dec(&data.numTypeStreams, 1); else if (type == SOCK_DGRAM) @@ -16398,6 +16378,7 @@ void dec_socket(int domain, int type, int protocol) cnt_dec(&data.numTypeSeqPkgs, 1); #endif + /* *** Protocol counter *** */ if (protocol == IPPROTO_IP) cnt_dec(&data.numProtoIP, 1); else if (protocol == IPPROTO_TCP) @@ -16413,7 +16394,8 @@ void dec_socket(int domain, int type, int protocol) } -/* increment counters for when a socket is opened */ +/* Increment counters for when a socket is opened + */ static void inc_socket(int domain, int type, int protocol) { @@ -16421,6 +16403,7 @@ void inc_socket(int domain, int type, int protocol) cnt_inc(&data.numSockets, 1); + /* *** Domain counter *** */ if (domain == AF_INET) cnt_inc(&data.numDomainInet, 1); #if defined(HAVE_IN6) && defined(AF_INET6) @@ -16432,6 +16415,7 @@ void inc_socket(int domain, int type, int protocol) cnt_inc(&data.numDomainInet6, 1); #endif + /* *** Type counter *** */ if (type == SOCK_STREAM) cnt_inc(&data.numTypeStreams, 1); else if (type == SOCK_DGRAM) @@ -16441,6 +16425,7 @@ void inc_socket(int domain, int type, int protocol) cnt_inc(&data.numTypeSeqPkgs, 1); #endif + /* *** Protocol counter *** */ if (protocol == IPPROTO_IP) cnt_inc(&data.numProtoIP, 1); else if (protocol == IPPROTO_TCP) @@ -16456,20 +16441,6 @@ void inc_socket(int domain, int type, int protocol) } - -/* compare_pids - Test if two pids are equal - * - */ -static -int compare_pids(ErlNifEnv* env, - const ErlNifPid* pid1, - const ErlNifPid* pid2) -{ - ERL_NIF_TERM p1 = enif_make_pid(env, pid1); - ERL_NIF_TERM p2 = enif_make_pid(env, pid2); - - return enif_is_identical(p1, p2); -} #endif // if !defined(__WIN32__) @@ -17070,280 +17041,249 @@ int esock_select_cancel(ErlNifEnv* env, /* ---------------------------------------------------------------------- - * R e q u e s t Q u e u e F u n c t i o n s + * A c t i v a t e N e x t ( o p e r a t o r ) F u n c t i o n s * ---------------------------------------------------------------------- */ -/* *** acceptor search for pid *** +/* *** activate_next_acceptor *** + * *** activate_next_writer *** + * *** activate_next_reader *** * - * Search for a pid in the acceptor queue. - */ -#if !defined(__WIN32__) -static -BOOLEAN_T acceptor_search4pid(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid) -{ - return qsearch4pid(env, &descP->acceptorsQ, pid); -} - - -/* *** acceptor push *** - * - * Push an acceptor onto the acceptor queue. - * This happens when we already have atleast one current acceptor. - */ -static -ERL_NIF_TERM acceptor_push(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid pid, - ERL_NIF_TERM ref) -{ - SocketRequestQueueElement* e = MALLOC(sizeof(SocketRequestQueueElement)); - SocketRequestor* reqP = &e->data; - - reqP->pid = pid; - reqP->ref = enif_make_copy(descP->env, ref); - - if (MONP("acceptor_push -> acceptor request", - env, descP, &pid, &reqP->mon) != 0) { - FREE(reqP); - return esock_make_error(env, atom_exmon); + * This functions pops the requestors queue and then selects until it + * manages to successfully activate a requestor or the queue is empty. + * Return value indicates if a new requestor was activated or not. + */ + +#define ACTIVATE_NEXT_FUNCS \ + ACTIVATE_NEXT_FUNC_DECL(acceptor, read, currentAcceptor, acceptorsQ) \ + ACTIVATE_NEXT_FUNC_DECL(writer, write, currentWriter, writersQ) \ + ACTIVATE_NEXT_FUNC_DECL(reader, read, currentReader, readersQ) + +#define ACTIVATE_NEXT_FUNC_DECL(F, S, R, Q) \ + static \ + BOOLEAN_T activate_next_##F(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + ERL_NIF_TERM sockRef) \ + { \ + BOOLEAN_T popped, activated; \ + int sres; \ + SocketRequestor* reqP = &descP->R; \ + SocketRequestQueue* q = &descP->Q; \ + \ + popped = FALSE; \ + do { \ + \ + if (requestor_pop(q, reqP)) { \ + \ + /* There was another one */ \ + \ + SSDBG( descP, \ + ("SOCKET", \ + "activate_next_" #F " -> new (active) requestor: " \ + "\r\n pid: %T" \ + "\r\n ref: %T" \ + "\r\n", reqP->pid, reqP->ref) ); \ + \ + if ((sres = esock_select_##S(env, descP->sock, descP, \ + &reqP->pid, reqP->ref)) < 0) { \ + /* We need to inform this process, reqP->pid, */ \ + /* that we failed to select, so we don't leave */ \ + /* it hanging. */ \ + /* => send abort */ \ + \ + esock_send_abort_msg(env, sockRef, reqP->ref, \ + sres, &reqP->pid); \ + \ + } else { \ + \ + /* Success: New requestor selected */ \ + popped = TRUE; \ + activated = FALSE; \ + \ + } \ + \ + } else { \ + \ + SSDBG( descP, \ + ("SOCKET", \ + "activate_next_" #F " -> no more requestors\r\n") ); \ + \ + popped = TRUE; \ + activated = FALSE; \ + } \ + \ + } while (!popped); \ + \ + SSDBG( descP, \ + ("SOCKET", "activate_next_" #F " -> " \ + "done with %s\r\n", B2S(activated)) ); \ + \ + return activated; \ } - - qpush(&descP->acceptorsQ, e); - - // THIS IS OK => MAKES THE CALLER WAIT FOR ITS TURN - return esock_make_error(env, esock_atom_eagain); -} +ACTIVATE_NEXT_FUNCS +#undef ACTIVATE_NEXT_FUNC_DECL -/* *** acceptor pop *** - * - * Pop an acceptor from the acceptor queue. - */ -static -BOOLEAN_T acceptor_pop(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid, - // ErlNifMonitor* mon, - ESockMonitor* mon, - ERL_NIF_TERM* ref) -{ - SocketRequestQueueElement* e = qpop(&descP->acceptorsQ); - if (e != NULL) { - *pid = e->data.pid; - *mon = e->data.mon; - *ref = e->data.ref; - FREE(e); - return TRUE; - } else { - /* (acceptors) Queue was empty */ - // *pid = NULL; we have no null value for pids - // *mon = NULL; we have no null value for monitors - *ref = esock_atom_undefined; // Just in case - return FALSE; - } - -} - -/* *** acceptor unqueue *** +/* ---------------------------------------------------------------------- + * R e q u e s t o r Q u e u e F u n c t i o n s + * ---------------------------------------------------------------------- * - * Remove an acceptor from the acceptor queue. + * Since each of these functions (search4pid, push, pop and unqueue + * are virtually identical for acceptors, writers and readers, + * we make use of set of declaration macros. */ -static -BOOLEAN_T acceptor_unqueue(ErlNifEnv* env, - SocketDescriptor* descP, - const ErlNifPid* pid) -{ - return qunqueue(env, descP, "qunqueue -> waiting acceptor", - &descP->acceptorsQ, pid); -} - +#if !defined(__WIN32__) -/* *** writer search for pid *** +/* *** acceptor_search4pid *** + * *** writer_search4pid *** + * *** reader_search4pid *** + * + * Search for a pid in the requestor (acceptor, writer, or reader) queue. * - * Search for a pid in the writer queue. */ -static -BOOLEAN_T writer_search4pid(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid) -{ - return qsearch4pid(env, &descP->writersQ, pid); -} +#define REQ_SEARCH4PID_FUNCS \ + REQ_SEARCH4PID_FUNC_DECL(acceptor, acceptorsQ) \ + REQ_SEARCH4PID_FUNC_DECL(writer, writersQ) \ + REQ_SEARCH4PID_FUNC_DECL(reader, readersQ) -/* *** writer push *** - * - * Push an writer onto the writer queue. - * This happens when we already have atleast one current writer. - */ -static -ERL_NIF_TERM writer_push(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid pid, - ERL_NIF_TERM ref) -{ - SocketRequestQueueElement* e = MALLOC(sizeof(SocketRequestQueueElement)); - SocketRequestor* reqP = &e->data; - - reqP->pid = pid; - reqP->ref = enif_make_copy(descP->env, ref); - - if (MONP("writer_push -> writer request", - env, descP, &pid, &reqP->mon) != 0) { - FREE(reqP); - return esock_make_error(env, atom_exmon); +#define REQ_SEARCH4PID_FUNC_DECL(F, Q) \ + static \ + BOOLEAN_T F##_search4pid(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + ErlNifPid* pid) \ + { \ + return qsearch4pid(env, &descP->Q, pid); \ } - - qpush(&descP->writersQ, e); - - // THIS IS OK => MAKES THE CALLER WAIT FOR ITS TURN - return esock_make_error(env, esock_atom_eagain); -} +REQ_SEARCH4PID_FUNCS +#undef REQ_SEARCH4PID_FUNC_DECL -/* *** writer pop *** + +/* *** acceptor_push *** + * *** writer_push *** + * *** reader_push *** + * + * Push a requestor (acceptor, writer, or reader) onto its queue. + * This happens when we already have a current request (of its type). * - * Pop an writer from the writer queue. */ -static -BOOLEAN_T writer_pop(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid, - // ErlNifMonitor* mon, - ESockMonitor* mon, - ERL_NIF_TERM* ref) -{ - SocketRequestQueueElement* e = qpop(&descP->writersQ); - if (e != NULL) { - *pid = e->data.pid; - *mon = e->data.mon; - *ref = e->data.ref; // At this point the ref has already been copied (env) - FREE(e); - return TRUE; - } else { - /* (writers) Queue was empty */ - // *pid = NULL; we have no null value for pids - // *mon = NULL; we have no null value for monitors - *ref = esock_atom_undefined; // Just in case - return FALSE; +#define REQ_PUSH_FUNCS \ + REQ_PUSH_FUNC_DECL(acceptor, acceptorsQ) \ + REQ_PUSH_FUNC_DECL(writer, writersQ) \ + REQ_PUSH_FUNC_DECL(reader, readersQ) + +#define REQ_PUSH_FUNC_DECL(F, Q) \ + static \ + ERL_NIF_TERM F##_push(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + ErlNifPid pid, \ + ERL_NIF_TERM ref) \ + { \ + SocketRequestQueueElement* e = MALLOC(sizeof(SocketRequestQueueElement)); \ + SocketRequestor* reqP = &e->data; \ + \ + reqP->pid = pid; \ + reqP->ref = enif_make_copy(descP->env, ref); \ + \ + if (MONP("reader_push -> " #F " request", \ + env, descP, &pid, &reqP->mon) != 0) { \ + FREE(reqP); \ + return esock_make_error(env, atom_exmon); \ + } \ + \ + qpush(&descP->Q, e); \ + \ + return esock_make_error(env, esock_atom_eagain); \ } - -} +REQ_PUSH_FUNCS +#undef REQ_PUSH_FUNC_DECL -/* *** writer unqueue *** + +/* *** acceptor_pop *** + * *** writer_pop *** + * *** reader_pop *** + * + * Pop a requestor (acceptor, writer, or reader) from its queue. * - * Remove an writer from the writer queue. */ -static -BOOLEAN_T writer_unqueue(ErlNifEnv* env, - SocketDescriptor* descP, - const ErlNifPid* pid) -{ - return qunqueue(env, descP, "qunqueue -> waiting writer", - &descP->writersQ, pid); -} +#define REQ_POP_FUNCS \ + REQ_POP_FUNC_DECL(acceptor, acceptorsQ) \ + REQ_POP_FUNC_DECL(writer, writersQ) \ + REQ_POP_FUNC_DECL(reader, readersQ) + +#define REQ_POP_FUNC_DECL(F, Q) \ + static \ + BOOLEAN_T F##_pop(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + SocketRequestor* reqP) \ + { \ + return requestor_pop(&descP->Q, reqP); \ + } +REQ_POP_FUNCS +#undef REQ_POP_FUNC_DECL -/* *** reader search for pid *** +/* *** acceptor_unqueue *** + * *** writer_unqueue *** + * *** reader_unqueue *** + * + * Remove a requestor (acceptor, writer, or reader) from its queue. * - * Search for a pid in the reader queue. */ -static -BOOLEAN_T reader_search4pid(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid) -{ - return qsearch4pid(env, &descP->readersQ, pid); -} +#define REQ_UNQUEUE_FUNCS \ + REQ_UNQUEUE_FUNC_DECL(acceptor, acceptorsQ) \ + REQ_UNQUEUE_FUNC_DECL(writer, writersQ) \ + REQ_UNQUEUE_FUNC_DECL(reader, readersQ) -/* *** reader push *** - * - * Push an reader onto the raeder queue. - * This happens when we already have atleast one current reader. - */ -static -ERL_NIF_TERM reader_push(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid pid, - ERL_NIF_TERM ref) -{ - SocketRequestQueueElement* e = MALLOC(sizeof(SocketRequestQueueElement)); - SocketRequestor* reqP = &e->data; - - reqP->pid = pid; - reqP->ref = enif_make_copy(descP->env, ref); - - if (MONP("reader_push -> reader request", - env, descP, &pid, &reqP->mon) != 0) { - FREE(reqP); - return esock_make_error(env, atom_exmon); +#define REQ_UNQUEUE_FUNC_DECL(F, Q) \ + static \ + BOOLEAN_T F##_unqueue(ErlNifEnv* env, \ + SocketDescriptor* descP, \ + const ErlNifPid* pid) \ + { \ + return qunqueue(env, descP, "qunqueue -> waiting " #F, \ + &descP->Q, pid); \ } - - qpush(&descP->readersQ, e); - - // THIS IS OK => MAKES THE CALLER WAIT FOR ITS TURN - return esock_make_error(env, esock_atom_eagain); -} +REQ_UNQUEUE_FUNCS +#undef REQ_UNQUEUE_FUNC_DECL + -/* *** reader pop *** +/* *** requestor pop *** * - * Pop an writer from the reader queue. + * Pop an requestor from its queue. */ static -BOOLEAN_T reader_pop(ErlNifEnv* env, - SocketDescriptor* descP, - ErlNifPid* pid, - // ErlNifMonitor* mon, - ESockMonitor* mon, - ERL_NIF_TERM* ref) +BOOLEAN_T requestor_pop(SocketRequestQueue* q, + SocketRequestor* reqP) { - SocketRequestQueueElement* e = qpop(&descP->readersQ); + SocketRequestQueueElement* e = qpop(q); if (e != NULL) { - *pid = e->data.pid; - *mon = e->data.mon; - *ref = e->data.ref; // At this point the ref has already been copied (env) + reqP->pid = e->data.pid; + reqP->mon = e->data.mon; + reqP->ref = e->data.ref; FREE(e); return TRUE; } else { - /* (readers) Queue was empty */ - // *pid = NULL; we have no null value for pids - // *mon = NULL; we have no null value for monitors - *ref = esock_atom_undefined; // Just in case + /* (writers) Queue was empty */ + enif_set_pid_undefined(&reqP->pid); + // *reqP->mon = NULL; we have no null value for monitors + reqP->ref = esock_atom_undefined; // Just in case return FALSE; } } -/* *** reader unqueue *** - * - * Remove an reader from the reader queue. - */ -static -BOOLEAN_T reader_unqueue(ErlNifEnv* env, - SocketDescriptor* descP, - const ErlNifPid* pid) -{ - return qunqueue(env, descP, "qunqueue -> waiting reader", - &descP->readersQ, pid); -} - - - - - static BOOLEAN_T qsearch4pid(ErlNifEnv* env, SocketRequestQueue* q, @@ -17352,7 +17292,7 @@ BOOLEAN_T qsearch4pid(ErlNifEnv* env, SocketRequestQueueElement* tmp = q->first; while (tmp != NULL) { - if (compare_pids(env, &tmp->data.pid, pid)) + if (COMPARE_PIDS(&tmp->data.pid, pid) == 0) return TRUE; else tmp = tmp->nextP; @@ -17411,7 +17351,7 @@ BOOLEAN_T qunqueue(ErlNifEnv* env, /* Check if it was one of the waiting acceptor processes */ while (e != NULL) { - if (compare_pids(env, &e->data.pid, pid)) { + if (COMPARE_PIDS(&e->data.pid, pid) == 0) { /* We have a match */ @@ -17502,12 +17442,6 @@ void cnt_dec(Uint32* cnt, Uint32 dec) #if !defined(__WIN32__) static -ERL_NIF_TERM my_make_monitor_term(ErlNifEnv* env, const ErlNifMonitor* mon) -{ - return ((ERL_NIF_TERM)&mon->data) + 2; -} - -static int esock_monitor(const char* slogan, ErlNifEnv* env, SocketDescriptor* descP, @@ -17660,7 +17594,7 @@ void socket_stop(ErlNifEnv* env, void* obj, int fd, int is_direct_call) descP->readTries, descP->readWaits) ); - sockRef = enif_make_resource(env, descP), + sockRef = enif_make_resource(env, descP); descP->state = SOCKET_STATE_CLOSING; // Just in case...??? descP->isReadable = FALSE; descP->isWritable = FALSE; @@ -17692,9 +17626,7 @@ void socket_stop(ErlNifEnv* env, void* obj, int fd, int is_direct_call) env, descP, &descP->currentWriter.mon); SSDBG( descP, ("SOCKET", "socket_stop -> handle current writer\r\n") ); - if (!compare_pids(env, - &descP->closerPid, - &descP->currentWriter.pid)) { + if (COMPARE_PIDS(&descP->closerPid, &descP->currentWriter.pid) != 0) { SSDBG( descP, ("SOCKET", "socket_stop -> " "send abort message to current writer %T\r\n", descP->currentWriter.pid) ); @@ -17715,7 +17647,8 @@ void socket_stop(ErlNifEnv* env, void* obj, int fd, int is_direct_call) /* And also deal with the waiting writers (in the same way) */ SSDBG( descP, ("SOCKET", "socket_stop -> handle waiting writer(s)\r\n") ); - inform_waiting_procs(env, descP, &descP->writersQ, TRUE, atom_closed); + inform_waiting_procs(env, "writer", + descP, &descP->writersQ, TRUE, atom_closed); } @@ -17736,12 +17669,15 @@ void socket_stop(ErlNifEnv* env, void* obj, int fd, int is_direct_call) env, descP, &descP->currentReader.mon); SSDBG( descP, ("SOCKET", "socket_stop -> handle current reader\r\n") ); - if (!compare_pids(env, - &descP->closerPid, - &descP->currentReader.pid)) { + if (COMPARE_PIDS(&descP->closerPid, &descP->currentReader.pid) != 0) { SSDBG( descP, ("SOCKET", "socket_stop -> " "send abort message to current reader %T\r\n", descP->currentReader.pid) ); + /* + esock_dbg_printf("SOCKET", "socket_stop -> " + "send abort message to current reader %T\r\n", + descP->currentReader.pid); + */ if (esock_send_abort_msg(env, sockRef, descP->currentReader.ref, @@ -17759,7 +17695,8 @@ void socket_stop(ErlNifEnv* env, void* obj, int fd, int is_direct_call) /* And also deal with the waiting readers (in the same way) */ SSDBG( descP, ("SOCKET", "socket_stop -> handle waiting reader(s)\r\n") ); - inform_waiting_procs(env, descP, &descP->readersQ, TRUE, atom_closed); + inform_waiting_procs(env, "reader", + descP, &descP->readersQ, TRUE, atom_closed); } @@ -17780,12 +17717,10 @@ void socket_stop(ErlNifEnv* env, void* obj, int fd, int is_direct_call) env, descP, &descP->currentAcceptor.mon); SSDBG( descP, ("SOCKET", "socket_stop -> handle current acceptor\r\n") ); - if (!compare_pids(env, - &descP->closerPid, - &descP->currentAcceptor.pid)) { + if (COMPARE_PIDS(&descP->closerPid, &descP->currentAcceptor.pid) != 0) { SSDBG( descP, ("SOCKET", "socket_stop -> " "send abort message to current acceptor %T\r\n", - descP->currentWriter.pid) ); + descP->currentAcceptor.pid) ); if (esock_send_abort_msg(env, sockRef, descP->currentAcceptor.ref, @@ -17803,7 +17738,8 @@ void socket_stop(ErlNifEnv* env, void* obj, int fd, int is_direct_call) /* And also deal with the waiting acceptors (in the same way) */ SSDBG( descP, ("SOCKET", "socket_stop -> handle waiting acceptor(s)\r\n") ); - inform_waiting_procs(env, descP, &descP->acceptorsQ, TRUE, atom_closed); + inform_waiting_procs(env, "acceptor", + descP, &descP->acceptorsQ, TRUE, atom_closed); } @@ -17859,15 +17795,24 @@ void socket_stop(ErlNifEnv* env, void* obj, int fd, int is_direct_call) * and if the 'free' argument is TRUE, the queue will be emptied. */ #if !defined(__WIN32__) -static -void inform_waiting_procs(ErlNifEnv* env, - SocketDescriptor* descP, - SocketRequestQueue* q, - BOOLEAN_T free, - ERL_NIF_TERM reason) +static void inform_waiting_procs(ErlNifEnv* env, + char* role, + SocketDescriptor* descP, + SocketRequestQueue* q, + BOOLEAN_T free, + ERL_NIF_TERM reason) { SocketRequestQueueElement* currentP = q->first; SocketRequestQueueElement* nextP; + ERL_NIF_TERM sockRef = enif_make_resource(env, descP); + + /* + esock_dbg_printf("SOCKET", "inform_waiting_procs -> entry with: " + "\r\n role: %s" + "\r\n free: %s" + "\r\n reason: %T" + "\r\n", role, B2S(free), reason); + */ while (currentP != NULL) { @@ -17884,11 +17829,25 @@ void inform_waiting_procs(ErlNifEnv* env, ("SOCKET", "inform_waiting_procs -> abort request %T (from %T)\r\n", currentP->data.ref, currentP->data.pid) ); - ESOCK_ASSERT( (NULL == esock_send_abort_msg(env, - esock_atom_undefined, - currentP->data.ref, - reason, - ¤tP->data.pid)) ); + /* + esock_dbg_printf("SOCKET", "inform_waiting_procs -> " + "try sending abort to %s %T " + "\r\n", role, currentP->data.pid); + */ + + if (esock_send_abort_msg(env, + sockRef, + currentP->data.ref, + reason, + ¤tP->data.pid) != NULL) { + + esock_warning_msg("Failed sending abort (%T) message to " + "current %s %T\r\n", + currentP->data.ref, + role, + currentP->data.pid); + + } DEMONP("inform_waiting_procs -> current 'request'", env, descP, ¤tP->data.mon); @@ -17918,6 +17877,7 @@ void socket_down(ErlNifEnv* env, #if !defined(__WIN32__) SocketDescriptor* descP = (SocketDescriptor*) obj; int sres; + ERL_NIF_TERM sockRef; SSDBG( descP, ("SOCKET", "socket_down -> entry with" "\r\n sock: %d" @@ -17930,7 +17890,7 @@ void socket_down(ErlNifEnv* env, if (!IS_CLOSED(descP)) { - if (compare_pids(env, &descP->ctrlPid, pid)) { + if (COMPARE_PIDS(&descP->ctrlPid, pid) == 0) { /* We don't bother with the queue cleanup here - * we leave it to the stop callback function. @@ -18019,7 +17979,7 @@ void socket_down(ErlNifEnv* env, "\r\n Descriptor: %d" "\r\n Monitor: %T" "\r\n", sres, pid, descP->sock, - my_make_monitor_term(env, mon)); + MON2T(env, mon)); } } else { @@ -18032,19 +17992,21 @@ void socket_down(ErlNifEnv* env, SSDBG( descP, ("SOCKET", "socket_down -> other process term\r\n") ); + sockRef = enif_make_resource(env, descP); + MLOCK(descP->accMtx); if (descP->currentAcceptorP != NULL) - socket_down_acceptor(env, descP, pid); + socket_down_acceptor(env, descP, sockRef, pid); MUNLOCK(descP->accMtx); MLOCK(descP->writeMtx); if (descP->currentWriterP != NULL) - socket_down_writer(env, descP, pid); + socket_down_writer(env, descP, sockRef, pid); MUNLOCK(descP->writeMtx); MLOCK(descP->readMtx); if (descP->currentReaderP != NULL) - socket_down_reader(env, descP, pid); + socket_down_reader(env, descP, sockRef, pid); MUNLOCK(descP->readMtx); } @@ -18066,49 +18028,28 @@ void socket_down(ErlNifEnv* env, static void socket_down_acceptor(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, const ErlNifPid* pid) { - if (compare_pids(env, &descP->currentAcceptor.pid, pid)) { + if (COMPARE_PIDS(&descP->currentAcceptor.pid, pid) == 0) { SSDBG( descP, ("SOCKET", "socket_down_acceptor -> " - "current acceptor - try pop the queue\r\n") ); + "current acceptor - try activate next\r\n") ); - if (acceptor_pop(env, descP, - &descP->currentAcceptor.pid, - &descP->currentAcceptor.mon, - &descP->currentAcceptor.ref)) { - int res; - - /* There was another one, so we will still be in accepting state */ - - SSDBG( descP, ("SOCKET", - "socket_down_acceptor -> new (active) acceptor: " - "\r\n pid: %T" - "\r\n ref: %T" - "\r\n", - descP->currentAcceptor.pid, - descP->currentAcceptor.ref) ); - - if ((res = esock_select_read(env, descP->sock, descP, - &descP->currentAcceptor.pid, - descP->currentAcceptor.ref) < 0)) { - - esock_warning_msg("Failed select (%d) for new acceptor " - "after current (%T) died\r\n", - res, *pid); - - } - - } else { - - SSDBG( descP, ("SOCKET", - "socket_down_acceptor -> no active acceptor\r\n") ); - - descP->currentAcceptorP = NULL; - descP->state = SOCKET_STATE_LISTENING; + if (!activate_next_acceptor(env, descP, sockRef)) { + + SSDBG( descP, + ("SOCKET", "socket_down_acceptor -> no more writers\r\n") ); + + descP->state = SOCKET_STATE_LISTENING; + + descP->currentAcceptorP = NULL; + descP->currentAcceptor.ref = esock_atom_undefined; + enif_set_pid_undefined(&descP->currentAcceptor.pid); + esock_monitor_init(&descP->currentAcceptor.mon); } - + } else { /* Maybe unqueue one of the waiting acceptors */ @@ -18132,45 +18073,22 @@ void socket_down_acceptor(ErlNifEnv* env, static void socket_down_writer(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, const ErlNifPid* pid) { - if (compare_pids(env, &descP->currentWriter.pid, pid)) { + if (COMPARE_PIDS(&descP->currentWriter.pid, pid) == 0) { SSDBG( descP, ("SOCKET", "socket_down_writer -> " - "current writer - try pop the queue\r\n") ); + "current writer - try activate next\r\n") ); - if (writer_pop(env, descP, - &descP->currentWriter.pid, - &descP->currentWriter.mon, - &descP->currentWriter.ref)) { - int res; - - /* There was another one */ - - SSDBG( descP, ("SOCKET", "socket_down_writer -> new (current) writer: " - "\r\n pid: %T" - "\r\n ref: %T" - "\r\n", - descP->currentWriter.pid, - descP->currentWriter.ref) ); - - if ((res = esock_select_write(env, descP->sock, descP, - &descP->currentWriter.pid, - descP->currentWriter.ref) < 0)) { - - esock_warning_msg("Failed select (%d) for new writer " - "after current (%T) died\r\n", - res, *pid); - - } - - } else { - + if (!activate_next_writer(env, descP, sockRef)) { SSDBG( descP, ("SOCKET", "socket_down_writer -> no active writer\r\n") ); - - descP->currentWriterP = NULL; + descP->currentWriterP = NULL; + descP->currentWriter.ref = esock_atom_undefined; + enif_set_pid_undefined(&descP->currentWriter.pid); + esock_monitor_init(&descP->currentWriter.mon); } } else { @@ -18196,47 +18114,24 @@ void socket_down_writer(ErlNifEnv* env, static void socket_down_reader(ErlNifEnv* env, SocketDescriptor* descP, + ERL_NIF_TERM sockRef, const ErlNifPid* pid) { - if (compare_pids(env, &descP->currentReader.pid, pid)) { + if (COMPARE_PIDS(&descP->currentReader.pid, pid) == 0) { SSDBG( descP, ("SOCKET", "socket_down_reader -> " - "current reader - try pop the queue\r\n") ); + "current reader - try activate next\r\n") ); - if (reader_pop(env, descP, - &descP->currentReader.pid, - &descP->currentReader.mon, - &descP->currentReader.ref)) { - int res; - - /* There was another one */ - - SSDBG( descP, ("SOCKET", "socket_down_reader -> new (current) reader: " - "\r\n pid: %T" - "\r\n ref: %T" - "\r\n", - descP->currentReader.pid, - descP->currentReader.ref) ); - - if ((res = esock_select_read(env, descP->sock, descP, - &descP->currentReader.pid, - descP->currentReader.ref) < 0)) { - - esock_warning_msg("Failed select (%d) for new reader " - "after current (%T) died\r\n", - res, *pid); - - } - - } else { - - SSDBG( descP, ("SOCKET", - "socket_down_reader -> no active reader\r\n") ); - - descP->currentReaderP = NULL; + if (!activate_next_reader(env, descP, sockRef)) { + SSDBG( descP, + ("SOCKET", "ncancel_recv_current -> no more readers\r\n") ); + descP->currentReaderP = NULL; + descP->currentReader.ref = esock_atom_undefined; + enif_set_pid_undefined(&descP->currentReader.pid); + esock_monitor_init(&descP->currentReader.mon); } - + } else { /* Maybe unqueue one of the waiting reader(s) */ @@ -18358,284 +18253,18 @@ int on_load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) data.numProtoSCTP = 0; #endif - /* +++ Misc atoms +++ */ - atom_adaptation_layer = MKA(env, str_adaptation_layer); - atom_address = MKA(env, str_address); - atom_association = MKA(env, str_association); - atom_assoc_id = MKA(env, str_assoc_id); - atom_authentication = MKA(env, str_authentication); - atom_bool = MKA(env, str_bool); - atom_close = MKA(env, str_close); - atom_closed = MKA(env, str_closed); - atom_closing = MKA(env, str_closing); - atom_cookie_life = MKA(env, str_cookie_life); - atom_data_in = MKA(env, str_data_in); - atom_do = MKA(env, str_do); - atom_dont = MKA(env, str_dont); - atom_exclude = MKA(env, str_exclude); - atom_false = MKA(env, str_false); - atom_global_counters = MKA(env, str_global_counters); - atom_in4_sockaddr = MKA(env, str_in4_sockaddr); - atom_in6_sockaddr = MKA(env, str_in6_sockaddr); - atom_include = MKA(env, str_include); - atom_initial = MKA(env, str_initial); - atom_int = MKA(env, str_int); - atom_interface = MKA(env, str_interface); - atom_iow = MKA(env, str_iow); - atom_local_rwnd = MKA(env, str_local_rwnd); - atom_max = MKA(env, str_max); - atom_max_attempts = MKA(env, str_max_attempts); - atom_max_init_timeo = MKA(env, str_max_init_timeo); - atom_max_instreams = MKA(env, str_max_instreams); - atom_max_rxt = MKA(env, str_max_rxt); - atom_min = MKA(env, str_min); - atom_mode = MKA(env, str_mode); - atom_multiaddr = MKA(env, str_multiaddr); - // atom_nif_abort = MKA(env, str_nif_abort); - atom_null = MKA(env, str_null); - atom_num_dinet = MKA(env, str_num_dinet); - atom_num_dinet6 = MKA(env, str_num_dinet6); - atom_num_dlocal = MKA(env, str_num_dlocal); - atom_num_outstreams = MKA(env, str_num_outstreams); - atom_num_peer_dests = MKA(env, str_num_peer_dests); - atom_num_pip = MKA(env, str_num_pip); - atom_num_psctp = MKA(env, str_num_psctp); - atom_num_ptcp = MKA(env, str_num_ptcp); - atom_num_pudp = MKA(env, str_num_pudp); - atom_num_sockets = MKA(env, str_num_sockets); - atom_num_tdgrams = MKA(env, str_num_tdgrams); - atom_num_tseqpkgs = MKA(env, str_num_tseqpkgs); - atom_num_tstreams = MKA(env, str_num_tstreams); - atom_partial_delivery = MKA(env, str_partial_delivery); - atom_peer_rwnd = MKA(env, str_peer_rwnd); - atom_peer_error = MKA(env, str_peer_error); - atom_probe = MKA(env, str_probe); - atom_select = MKA(env, str_select); - atom_sender_dry = MKA(env, str_sender_dry); - atom_send_failure = MKA(env, str_send_failure); - atom_shutdown = MKA(env, str_shutdown); - atom_slist = MKA(env, str_slist); - atom_sourceaddr = MKA(env, str_sourceaddr); - atom_timeout = MKA(env, str_timeout); - atom_true = MKA(env, str_true); - atom_want = MKA(env, str_want); - - /* Global atom(s) */ - esock_atom_abort = MKA(env, "abort"); - esock_atom_accept = MKA(env, "accept"); - esock_atom_acceptconn = MKA(env, "acceptconn"); - esock_atom_acceptfilter = MKA(env, "acceptfilter"); - esock_atom_adaption_layer = MKA(env, "adaption_layer"); - esock_atom_addr = MKA(env, "addr"); - esock_atom_addrform = MKA(env, "addrform"); - esock_atom_add_membership = MKA(env, "add_membership"); - esock_atom_add_source_membership = MKA(env, "add_source_membership"); - esock_atom_any = MKA(env, "any"); - esock_atom_associnfo = MKA(env, "associnfo"); - esock_atom_authhdr = MKA(env, "authhdr"); - esock_atom_auth_active_key = MKA(env, "auth_active_key"); - esock_atom_auth_asconf = MKA(env, "auth_asconf"); - esock_atom_auth_chunk = MKA(env, "auth_chunk"); - esock_atom_auth_delete_key = MKA(env, "auth_delete_key"); - esock_atom_auth_key = MKA(env, "auth_key"); - esock_atom_auth_level = MKA(env, "auth_level"); - esock_atom_autoclose = MKA(env, "autoclose"); - esock_atom_bindtodevice = MKA(env, "bindtodevice"); - esock_atom_block_source = MKA(env, "block_source"); - esock_atom_broadcast = MKA(env, "broadcast"); - esock_atom_busy_poll = MKA(env, "busy_poll"); - esock_atom_checksum = MKA(env, "checksum"); - esock_atom_close = MKA(env, "close"); - esock_atom_connect = MKA(env, "connect"); - esock_atom_congestion = MKA(env, "congestion"); - esock_atom_context = MKA(env, "context"); - esock_atom_cork = MKA(env, "cork"); - esock_atom_credentials = MKA(env, "credentials"); - esock_atom_ctrl = MKA(env, "ctrl"); - esock_atom_ctrunc = MKA(env, "ctrunc"); - esock_atom_data = MKA(env, "data"); - esock_atom_debug = MKA(env, "debug"); - esock_atom_default_send_params = MKA(env, "default_send_params"); - esock_atom_delayed_ack_time = MKA(env, "delayed_ack_time"); - esock_atom_dgram = MKA(env, "dgram"); - esock_atom_disable_fragments = MKA(env, "disable_fragments"); - esock_atom_domain = MKA(env, "domain"); - esock_atom_dontfrag = MKA(env, "dontfrag"); - esock_atom_dontroute = MKA(env, "dontroute"); - esock_atom_drop_membership = MKA(env, "drop_membership"); - esock_atom_drop_source_membership = MKA(env, "drop_source_membership"); - esock_atom_dstopts = MKA(env, "dstpopts"); - esock_atom_eor = MKA(env, "eor"); - esock_atom_error = MKA(env, "error"); - esock_atom_errqueue = MKA(env, "errqueue"); - esock_atom_esp_network_level = MKA(env, "esp_network_level"); - esock_atom_esp_trans_level = MKA(env, "esp_trans_level"); - esock_atom_events = MKA(env, "events"); - esock_atom_explicit_eor = MKA(env, "explicit_eor"); - esock_atom_faith = MKA(env, "faith"); - esock_atom_false = MKA(env, "false"); - esock_atom_family = MKA(env, "family"); - esock_atom_flags = MKA(env, "flags"); - esock_atom_flowinfo = MKA(env, "flowinfo"); - esock_atom_fragment_interleave = MKA(env, "fragment_interleave"); - esock_atom_freebind = MKA(env, "freebind"); - esock_atom_get_peer_addr_info = MKA(env, "get_peer_addr_info"); - esock_atom_hdrincl = MKA(env, "hdrincl"); - esock_atom_hmac_ident = MKA(env, "hmac_ident"); - esock_atom_hoplimit = MKA(env, "hoplimit"); - esock_atom_hopopts = MKA(env, "hopopts"); - esock_atom_ifindex = MKA(env, "ifindex"); - esock_atom_inet = MKA(env, "inet"); - esock_atom_inet6 = MKA(env, "inet6"); - esock_atom_info = MKA(env, "info"); - esock_atom_initmsg = MKA(env, "initmsg"); - esock_atom_iov = MKA(env, "iov"); - esock_atom_ip = MKA(env, "ip"); - esock_atom_ipcomp_level = MKA(env, "ipcomp_level"); - esock_atom_ipv6 = MKA(env, "ipv6"); - esock_atom_i_want_mapped_v4_addr = MKA(env, "i_want_mapped_v4_addr"); - esock_atom_join_group = MKA(env, "join_group"); - esock_atom_keepalive = MKA(env, "keepalive"); - esock_atom_keepcnt = MKA(env, "keepcnt"); - esock_atom_keepidle = MKA(env, "keepidle"); - esock_atom_keepintvl = MKA(env, "keepintvl"); - esock_atom_leave_group = MKA(env, "leave_group"); - esock_atom_level = MKA(env, "level"); - esock_atom_linger = MKA(env, "linger"); - esock_atom_local = MKA(env, "local"); - esock_atom_local_auth_chunks = MKA(env, "local_auth_chunks"); - esock_atom_loopback = MKA(env, "loopback"); - esock_atom_lowdelay = MKA(env, "lowdelay"); - esock_atom_mark = MKA(env, "mark"); - esock_atom_maxburst = MKA(env, "maxburst"); - esock_atom_maxseg = MKA(env, "maxseg"); - esock_atom_md5sig = MKA(env, "md5sig"); - esock_atom_mincost = MKA(env, "mincost"); - esock_atom_minttl = MKA(env, "minttl"); - esock_atom_msfilter = MKA(env, "msfilter"); - esock_atom_mtu = MKA(env, "mtu"); - esock_atom_mtu_discover = MKA(env, "mtu_discover"); - esock_atom_multicast_all = MKA(env, "multicast_all"); - esock_atom_multicast_hops = MKA(env, "multicast_hops"); - esock_atom_multicast_if = MKA(env, "multicast_if"); - esock_atom_multicast_loop = MKA(env, "multicast_loop"); - esock_atom_multicast_ttl = MKA(env, "multicast_ttl"); - esock_atom_nodefrag = MKA(env, "nodefrag"); - esock_atom_nodelay = MKA(env, "nodelay"); - esock_atom_noopt = MKA(env, "noopt"); - esock_atom_nopush = MKA(env, "nopush"); - esock_atom_not_found = MKA(env, "not_found"); - esock_atom_not_owner = MKA(env, "not_owner"); - esock_atom_ok = MKA(env, "ok"); - esock_atom_oob = MKA(env, "oob"); - esock_atom_oobinline = MKA(env, "oobinline"); - esock_atom_options = MKA(env, "options"); - esock_atom_origdstaddr = MKA(env, "origdstaddr"); - esock_atom_partial_delivery_point = MKA(env, "partial_delivery_point"); - esock_atom_passcred = MKA(env, "passcred"); - esock_atom_path = MKA(env, "path"); - esock_atom_peekcred = MKA(env, "peekcred"); - esock_atom_peek_off = MKA(env, "peek_off"); - esock_atom_peer_addr_params = MKA(env, "peer_addr_params"); - esock_atom_peer_auth_chunks = MKA(env, "peer_auth_chunks"); - esock_atom_pktinfo = MKA(env, "pktinfo"); - esock_atom_pktoptions = MKA(env, "pktoptions"); - esock_atom_port = MKA(env, "port"); - esock_atom_portrange = MKA(env, "portrange"); - esock_atom_primary_addr = MKA(env, "primary_addr"); - esock_atom_priority = MKA(env, "priority"); - esock_atom_protocol = MKA(env, "protocol"); - esock_atom_raw = MKA(env, "raw"); - esock_atom_rcvbuf = MKA(env, "rcvbuf"); - esock_atom_rcvbufforce = MKA(env, "rcvbufforce"); - esock_atom_rcvlowat = MKA(env, "rcvlowat"); - esock_atom_rcvtimeo = MKA(env, "rcvtimeo"); - esock_atom_rdm = MKA(env, "rdm"); - esock_atom_recv = MKA(env, "recv"); - esock_atom_recvdstaddr = MKA(env, "recvdstaddr"); - esock_atom_recverr = MKA(env, "recverr"); - esock_atom_recvfrom = MKA(env, "recvfrom"); - esock_atom_recvif = MKA(env, "recvif"); - esock_atom_recvmsg = MKA(env, "recvmsg"); - esock_atom_recvopts = MKA(env, "recvopts"); - esock_atom_recvorigdstaddr = MKA(env, "recvorigdstaddr"); - esock_atom_recvpktinfo = MKA(env, "recvpktinfo"); - esock_atom_recvtclass = MKA(env, "recvtclass"); - esock_atom_recvtos = MKA(env, "recvtos"); - esock_atom_recvttl = MKA(env, "recvttl"); - esock_atom_reliability = MKA(env, "reliability"); - esock_atom_reset_streams = MKA(env, "reset_streams"); - esock_atom_retopts = MKA(env, "retopts"); - esock_atom_reuseaddr = MKA(env, "reuseaddr"); - esock_atom_reuseport = MKA(env, "reuseport"); - esock_atom_rights = MKA(env, "rights"); - esock_atom_router_alert = MKA(env, "router_alert"); - esock_atom_rthdr = MKA(env, "rthdr"); - esock_atom_rtoinfo = MKA(env, "rtoinfo"); - esock_atom_rxq_ovfl = MKA(env, "rxq_ovfl"); - esock_atom_scope_id = MKA(env, "scope_id"); - esock_atom_sctp = MKA(env, "sctp"); - esock_atom_sec = MKA(env, "sec"); - esock_atom_select_failed = MKA(env, "select_failed"); - esock_atom_select_sent = MKA(env, "select_sent"); - esock_atom_send = MKA(env, "send"); - esock_atom_sendmsg = MKA(env, "sendmsg"); - esock_atom_sendsrcaddr = MKA(env, "sendsrcaddr"); - esock_atom_sendto = MKA(env, "sendto"); - esock_atom_seqpacket = MKA(env, "seqpacket"); - esock_atom_setfib = MKA(env, "setfib"); - esock_atom_set_peer_primary_addr = MKA(env, "set_peer_primary_addr"); - esock_atom_sndbuf = MKA(env, "sndbuf"); - esock_atom_sndbufforce = MKA(env, "sndbufforce"); - esock_atom_sndlowat = MKA(env, "sndlowat"); - esock_atom_sndtimeo = MKA(env, "sndtimeo"); - esock_atom_socket = MKA(env, "socket"); - esock_atom_socket_tag = MKA(env, "$socket"); - esock_atom_spec_dst = MKA(env, "spec_dst"); - esock_atom_status = MKA(env, "status"); - esock_atom_stream = MKA(env, "stream"); - esock_atom_syncnt = MKA(env, "syncnt"); - esock_atom_tclass = MKA(env, "tclass"); - esock_atom_tcp = MKA(env, "tcp"); - esock_atom_throughput = MKA(env, "throughput"); - esock_atom_timestamp = MKA(env, "timestamp"); - esock_atom_tos = MKA(env, "tos"); - esock_atom_transparent = MKA(env, "transparent"); - esock_atom_true = MKA(env, "true"); - esock_atom_trunc = MKA(env, "trunc"); - esock_atom_ttl = MKA(env, "ttl"); - esock_atom_type = MKA(env, "type"); - esock_atom_udp = MKA(env, "udp"); - esock_atom_unblock_source = MKA(env, "unblock_source"); - esock_atom_undefined = MKA(env, "undefined"); - esock_atom_unicast_hops = MKA(env, "unicast_hops"); - esock_atom_unknown = MKA(env, "unknown"); - esock_atom_usec = MKA(env, "usec"); - esock_atom_user_timeout = MKA(env, "user_timeout"); - esock_atom_use_ext_recvinfo = MKA(env, "use_ext_recvinfo"); - esock_atom_use_min_mtu = MKA(env, "use_min_mtu"); - esock_atom_v6only = MKA(env, "v6only"); - - /* Global error codes */ - esock_atom_eafnosupport = MKA(env, ESOCK_STR_EAFNOSUPPORT); - esock_atom_eagain = MKA(env, ESOCK_STR_EAGAIN); - esock_atom_einval = MKA(env, ESOCK_STR_EINVAL); - - /* Error codes */ - atom_eisconn = MKA(env, str_eisconn); - atom_enotclosing = MKA(env, str_enotclosing); - atom_enotconn = MKA(env, str_enotconn); - atom_exalloc = MKA(env, str_exalloc); - atom_exbadstate = MKA(env, str_exbadstate); - atom_exbusy = MKA(env, str_exbusy); - atom_exmon = MKA(env, str_exmon); - atom_exself = MKA(env, str_exself); - atom_exsend = MKA(env, str_exsend); - - // For storing "global" things... - // data.env = enif_alloc_env(); // We should really check - // data.version = MKA(env, ERTS_VERSION); - // data.buildDate = MKA(env, ERTS_BUILD_DATE); + /* +++ Local atoms and error reason atoms +++ */ +#define LOCAL_ATOM_DECL(A) atom_##A = MKA(env, #A) +LOCAL_ATOMS +LOCAL_ERROR_REASON_ATOMS +#undef LOCAL_ATOM_DECL + + /* Global atom(s) and error reason atom(s) */ +#define GLOBAL_ATOM_DECL(A) esock_atom_##A = MKA(env, #A) +GLOBAL_ATOMS +GLOBAL_ERROR_REASON_ATOMS +#undef GLOBAL_ATOM_DECL + esock_atom_socket_tag = MKA(env, "$socket"); sockets = enif_open_resource_type_x(env, "sockets", diff --git a/erts/emulator/nifs/common/socket_util.c b/erts/emulator/nifs/common/socket_util.c index f6e4781977..b817ae7636 100644 --- a/erts/emulator/nifs/common/socket_util.c +++ b/erts/emulator/nifs/common/socket_util.c @@ -177,7 +177,7 @@ char* esock_decode_iov(ErlNifEnv* env, return ESOCK_STR_EINVAL; if (IS_BIN(env, elem) && GET_BIN(env, elem, &bufs[i])) { - iov[i].iov_base = bufs[i].data; + iov[i].iov_base = (caddr_t) bufs[i].data; iov[i].iov_len = bufs[i].size; sz += bufs[i].size; } else { diff --git a/erts/emulator/test/nif_SUITE.erl b/erts/emulator/test/nif_SUITE.erl index 4a0ad9c1d5..2309f844b9 100644 --- a/erts/emulator/test/nif_SUITE.erl +++ b/erts/emulator/test/nif_SUITE.erl @@ -65,7 +65,8 @@ nif_whereis/1, nif_whereis_parallel/1, nif_whereis_threaded/1, nif_whereis_proxy/1, nif_ioq/1, - pid/1 + pid/1, + nif_term_type/1 ]). -export([many_args_100/100]). @@ -105,7 +106,8 @@ all() -> nif_phash2, nif_whereis, nif_whereis_parallel, nif_whereis_threaded, nif_ioq, - pid]. + pid, + nif_term_type]. groups() -> [{G, [], api_repeaters()} || G <- api_groups()] @@ -3392,6 +3394,41 @@ pid(Config) -> {false, _} = get_local_pid_nif(undefined), ok. +nif_term_type(Config) -> + ensure_lib_loaded(Config), + + atom = term_type_nif(atom), + + bitstring = term_type_nif(<<1:1>>), + bitstring = term_type_nif(<<1:8>>), + + float = term_type_nif(0.0), + + 'fun' = term_type_nif(fun external:function/1), + 'fun' = term_type_nif(fun(A) -> A end), + 'fun' = term_type_nif(fun id/1), + + integer = term_type_nif(1 bsl 1024), %Bignum. + integer = term_type_nif(1), + + list = term_type_nif([list]), + list = term_type_nif([]), + + LargeMap = maps:from_list([{N, N} || N <- lists:seq(1, 256)]), + map = term_type_nif(LargeMap), + map = term_type_nif(#{ small => map }), + + pid = term_type_nif(self()), + + Port = open_port({spawn,echo_drv},[eof]), + port = term_type_nif(Port), + port_close(Port), + + reference = term_type_nif(make_ref()), + + tuple = term_type_nif({}), + + ok. id(I) -> I. @@ -3507,5 +3544,7 @@ set_pid_undefined_nif() -> ?nif_stub. is_pid_undefined_nif(_) -> ?nif_stub. compare_pids_nif(_, _) -> ?nif_stub. +term_type_nif(_) -> ?nif_stub. + nif_stub_error(Line) -> exit({nif_not_loaded,module,?MODULE,line,Line}). diff --git a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c index 0d5d900d31..1906384af4 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c +++ b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c @@ -3583,6 +3583,36 @@ static ERL_NIF_TERM compare_pids_nif(ErlNifEnv* env, int argc, const ERL_NIF_TER return enif_make_int(env, enif_compare_pids(&a, &b)); } +static ERL_NIF_TERM term_type_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + switch (enif_term_type(env, argv[0])) { + case ERL_NIF_TERM_TYPE_ATOM: + return enif_make_atom(env, "atom"); + case ERL_NIF_TERM_TYPE_BITSTRING: + return enif_make_atom(env, "bitstring"); + case ERL_NIF_TERM_TYPE_FLOAT: + return enif_make_atom(env, "float"); + case ERL_NIF_TERM_TYPE_FUN: + return enif_make_atom(env, "fun"); + case ERL_NIF_TERM_TYPE_INTEGER: + return enif_make_atom(env, "integer"); + case ERL_NIF_TERM_TYPE_LIST: + return enif_make_atom(env, "list"); + case ERL_NIF_TERM_TYPE_MAP: + return enif_make_atom(env, "map"); + case ERL_NIF_TERM_TYPE_PID: + return enif_make_atom(env, "pid"); + case ERL_NIF_TERM_TYPE_PORT: + return enif_make_atom(env, "port"); + case ERL_NIF_TERM_TYPE_REFERENCE: + return enif_make_atom(env, "reference"); + case ERL_NIF_TERM_TYPE_TUPLE: + return enif_make_atom(env, "tuple"); + default: + return enif_make_badarg(env); + } +} + static ErlNifFunc nif_funcs[] = { {"lib_version", 0, lib_version}, @@ -3690,7 +3720,8 @@ static ErlNifFunc nif_funcs[] = {"make_pid_nif", 1, make_pid_nif}, {"set_pid_undefined_nif", 0, set_pid_undefined_nif}, {"is_pid_undefined_nif", 1, is_pid_undefined_nif}, - {"compare_pids_nif", 2, compare_pids_nif} + {"compare_pids_nif", 2, compare_pids_nif}, + {"term_type_nif", 1, term_type_nif} }; ERL_NIF_INIT(nif_SUITE,nif_funcs,load,NULL,upgrade,unload) diff --git a/erts/emulator/test/socket_SUITE.erl b/erts/emulator/test/socket_SUITE.erl index 52d002c9b8..8a32efcd85 100644 --- a/erts/emulator/test/socket_SUITE.erl +++ b/erts/emulator/test/socket_SUITE.erl @@ -4541,7 +4541,7 @@ api_to_recv_tcp4(_Config) when is_list(_Config) -> Recv = fun(Sock, To) -> socket:recv(Sock, 0, To) end, InitState = #{domain => inet, recv => Recv, - timeout => 5000}, + timeout => 2000}, ok = api_to_receive_tcp(InitState) end). @@ -4566,7 +4566,7 @@ api_to_recv_tcp6(_Config) when is_list(_Config) -> end, InitState = #{domain => inet6, recv => Recv, - timeout => 5000}, + timeout => 2000}, ok = api_to_receive_tcp(InitState); false -> skip("ipv6 not supported") @@ -4900,7 +4900,7 @@ api_to_recvfrom_udp4(_Config) when is_list(_Config) -> Recv = fun(Sock, To) -> socket:recvfrom(Sock, 0, To) end, InitState = #{domain => inet, recv => Recv, - timeout => 5000}, + timeout => 2000}, ok = api_to_receive_udp(InitState) end). @@ -4921,7 +4921,7 @@ api_to_recvfrom_udp6(_Config) when is_list(_Config) -> Recv = fun(Sock, To) -> socket:recvfrom(Sock, 0, To) end, InitState = #{domain => inet6, recv => Recv, - timeout => 5000}, + timeout => 2000}, ok = api_to_receive_udp(InitState) end). @@ -4984,6 +4984,7 @@ api_to_receive_udp(InitState) -> %% *** Termination *** #{desc => "close socket", cmd => fun(#{sock := Sock} = _State) -> + socket:setopt(Sock, otp, debug, true), sock_close(Sock), ok end}, @@ -5015,7 +5016,7 @@ api_to_recvmsg_udp4(_Config) when is_list(_Config) -> Recv = fun(Sock, To) -> socket:recvmsg(Sock, To) end, InitState = #{domain => inet, recv => Recv, - timeout => 5000}, + timeout => 2000}, ok = api_to_receive_udp(InitState) end). @@ -5036,7 +5037,7 @@ api_to_recvmsg_udp6(_Config) when is_list(_Config) -> Recv = fun(Sock, To) -> socket:recvmsg(Sock, To) end, InitState = #{domain => inet6, recv => Recv, - timeout => 5000}, + timeout => 2000}, ok = api_to_receive_udp(InitState) end). @@ -5056,7 +5057,7 @@ api_to_recvmsg_tcp4(_Config) when is_list(_Config) -> Recv = fun(Sock, To) -> socket:recvmsg(Sock, To) end, InitState = #{domain => inet, recv => Recv, - timeout => 5000}, + timeout => 2000}, ok = api_to_receive_tcp(InitState) end). @@ -5077,7 +5078,7 @@ api_to_recvmsg_tcp6(_Config) when is_list(_Config) -> Recv = fun(Sock, To) -> socket:recvmsg(Sock, To) end, InitState = #{domain => inet6, recv => Recv, - timeout => 5000}, + timeout => 2000}, ok = api_to_receive_tcp(InitState) end). @@ -5543,6 +5544,7 @@ sc_lc_receive_response_tcp(InitState) -> end}, #{desc => "attempt recv (=> closed)", cmd => fun(#{sock := Sock, recv := Recv} = State) -> + %% ok = socket:setopt(Sock, otp, debug, true), case Recv(Sock) of {ok, _Data} -> ?SEV_EPRINT("Unexpected data received"), diff --git a/erts/emulator/test/socket_test_evaluator.erl b/erts/emulator/test/socket_test_evaluator.erl index fe6a6ff70a..c5748ac21b 100644 --- a/erts/emulator/test/socket_test_evaluator.erl +++ b/erts/emulator/test/socket_test_evaluator.erl @@ -104,8 +104,9 @@ start(Name, Seq, InitState) erlang:error({already_used, parent}); error -> InitState2 = InitState#{parent => self()}, - {Pid, MRef} = erlang:spawn_monitor( - fun() -> init(Name, Seq, InitState2) end), + Pid = erlang:spawn_link( + fun() -> init(Name, Seq, InitState2) end), + MRef = erlang:monitor(process, Pid), #ev{name = Name, pid = Pid, mref = MRef} end. @@ -149,55 +150,93 @@ loop(ID, [#{desc := Desc, Evs :: [ev()]. await_finish(Evs) -> - await_finish(Evs, []). + await_finish(Evs, [], []). -await_finish([], []) -> +await_finish([], _, []) -> ok; -await_finish([], Fails) -> +await_finish([], _OK, Fails) -> ?SEV_EPRINT("Fails: " "~n ~p", [Fails]), Fails; -await_finish(Evs, Fails) -> +await_finish(Evs, OK, Fails) -> receive %% Successfull termination of evaluator {'DOWN', _MRef, process, Pid, normal} -> - case lists:keysearch(Pid, #ev.pid, Evs) of - {value, #ev{name = Name}} -> - iprint("evaluator '~s' (~p) success", [Name, Pid]), - NewEvs = lists:keydelete(Pid, #ev.pid, Evs), - await_finish(NewEvs, Fails); - false -> - iprint("unknown process ~p died (normal)", [Pid]), - await_finish(Evs, Fails) - end; + {Evs2, OK2, Fails2} = await_finish_normal(Pid, Evs, OK, Fails), + await_finish(Evs2, OK2, Fails2); + {'EXIT', Pid, normal} -> + {Evs2, OK2, Fails2} = await_finish_normal(Pid, Evs, OK, Fails), + await_finish(Evs2, OK2, Fails2); %% The evaluator can skip the teat case: {'DOWN', _MRef, process, Pid, {skip, Reason}} -> - case lists:keysearch(Pid, #ev.pid, Evs) of - {value, #ev{name = Name}} -> - iprint("evaluator '~s' (~p) issued SKIP: " - "~n ~p", [Name, Pid, Reason]); + await_finish_skip(Pid, Reason, Evs, OK); + {'EXIT', Pid, {skip, Reason}} -> + await_finish_skip(Pid, Reason, Evs, OK); + + %% Evaluator failed + {'DOWN', _MRef, process, Pid, Reason} -> + {Evs2, OK2, Fails2} = await_finish_fail(Pid, Reason, Evs, OK, Fails), + await_finish(Evs2, OK2, Fails2); + {'EXIT', Pid, Reason} -> + {Evs2, OK2, Fails2} = await_finish_fail(Pid, Reason, Evs, OK, Fails), + await_finish(Evs2, OK2, Fails2) + end. + + +await_finish_normal(Pid, Evs, OK, Fails) -> + case lists:keysearch(Pid, #ev.pid, Evs) of + {value, #ev{name = Name}} -> + iprint("evaluator '~s' (~p) success", [Name, Pid]), + NewEvs = lists:keydelete(Pid, #ev.pid, Evs), + {NewEvs, [Pid|OK], Fails}; + false -> + case lists:member(Pid, OK) of + true -> + ok; + false -> + iprint("unknown process ~p died (normal)", [Pid]), + ok + end, + {Evs, OK, Fails} + end. + +await_finish_skip(Pid, Reason, Evs, OK) -> + case lists:keysearch(Pid, #ev.pid, Evs) of + {value, #ev{name = Name}} -> + iprint("evaluator '~s' (~p) issued SKIP: " + "~n ~p", [Name, Pid, Reason]); + false -> + case lists:member(Pid, OK) of + true -> + ok; false -> iprint("unknown process ~p issued SKIP: " "~n ~p", [Pid, Reason]) - end, - ?LIB:skip(Reason); + end + end, + ?LIB:skip(Reason). - %% Evaluator failed - {'DOWN', _MRef, process, Pid, Reason} -> - case lists:keysearch(Pid, #ev.pid, Evs) of - {value, #ev{name = Name}} -> - iprint("evaluator '~s' (~p) failed", [Name, Pid]), - NewEvs = lists:keydelete(Pid, #ev.pid, Evs), - await_finish(NewEvs, [{Pid, Reason}|Fails]); + +await_finish_fail(Pid, Reason, Evs, OK, Fails) -> + case lists:keysearch(Pid, #ev.pid, Evs) of + {value, #ev{name = Name}} -> + iprint("evaluator '~s' (~p) failed", [Name, Pid]), + NewEvs = lists:keydelete(Pid, #ev.pid, Evs), + {NewEvs, OK, [{Pid, Reason}|Fails]}; + false -> + case lists:member(Pid, OK) of + true -> + ok; false -> iprint("unknown process ~p died: " - "~n ~p", [Pid, Reason]), - await_finish(Evs, Fails) - end + "~n ~p", [Pid, Reason]) + end, + {Evs, OK, Fails} end. + %% ============================================================================ -spec announce_start(To) -> ok when @@ -471,7 +510,7 @@ await(ExpPid, Name, Announcement, Slogan, OtherPids) "~n Announcement: ~p" "~n Slogan: ~p" "~nwhen" - "~n Messages: ~p", + "~n Messages: ~p", [ExpPid, Name, Announcement, Slogan, pi(messages)]), await(ExpPid, Name, Announcement, Slogan, OtherPids) end. diff --git a/erts/emulator/test/trace_local_SUITE.erl b/erts/emulator/test/trace_local_SUITE.erl index 253d5fed23..ad802352b9 100644 --- a/erts/emulator/test/trace_local_SUITE.erl +++ b/erts/emulator/test/trace_local_SUITE.erl @@ -1181,7 +1181,9 @@ undef(X) -> ?MODULE:undef(X, X). % undef lists_reverse(A, B) -> - lists:reverse(A, B). + Res = lists:reverse(A, B), + _ = (catch abs(A)), + Res. diff --git a/erts/emulator/utils/beam_makeops b/erts/emulator/utils/beam_makeops index f73e2362bf..1625b2cc65 100755 --- a/erts/emulator/utils/beam_makeops +++ b/erts/emulator/utils/beam_makeops @@ -1488,7 +1488,7 @@ sub code_gen { $var_decls .= "Eterm $tmp;\n"; $tmp_arg_num++; push(@f, $tmp); - $prefix .= "GetR($arg_offset, $tmp);\n"; + $prefix .= "GetSource(" . arg_offset($arg_offset) . ", $tmp);\n"; $need_block = 1; last SWITCH; }; diff --git a/erts/lib_src/Makefile.in b/erts/lib_src/Makefile.in index 8e1f5b58c4..1da11c2d0a 100644 --- a/erts/lib_src/Makefile.in +++ b/erts/lib_src/Makefile.in @@ -29,7 +29,6 @@ CC=@CC@ LD=@LD@ AR=@AR@ RANLIB=@RANLIB@ -RM=@RM@ MKDIR=@MKDIR@ INSTALL=@INSTALL@ INSTALL_DIR=@INSTALL_DIR@ @@ -536,9 +535,9 @@ release_docs_spec: # .PHONY: clean clean: - $(RM) -rf ../lib/internal/$(TARGET)/* - $(RM) -rf ../lib/$(TARGET)/* - $(RM) -rf obj/$(TARGET)/* + $(RM) -r ../lib/internal/$(TARGET)/* + $(RM) -r ../lib/$(TARGET)/* + $(RM) -r obj/$(TARGET)/* # # Make dependencies diff --git a/erts/vsn.mk b/erts/vsn.mk index e4bdb1a8eb..fac608ed4e 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -18,7 +18,7 @@ # %CopyrightEnd% # -VSN = 10.2.4 +VSN = 10.3 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/Makefile b/lib/Makefile index 6605c6145c..ea2827bef0 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -58,10 +58,10 @@ else SUB_DIRECTORIES = hipe parsetools asn1/src else ifdef TERTIARY_BOOTSTRAP - SUB_DIRECTORIES = snmp sasl jinterface syntax_tools wx + SUB_DIRECTORIES = snmp sasl erl_interface jinterface syntax_tools wx else ifdef DOC_BOOTSTRAP - SUB_DIRECTORIES = xmerl edoc erl_docgen + SUB_DIRECTORIES = xmerl edoc erl_docgen public_key else # Not bootstrap build SUB_DIRECTORIES = $(ERTS_APPLICATIONS) \ $(ERLANG_APPLICATIONS) \ diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index 2f53f1c29e..c8e0722a0f 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -33,6 +33,66 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.17</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A bug caused <c>ct:encrypt_config_file/3</c> and + <c>ct:decrypt_config_file/3</c> to fail with + <c>badmatch</c> if input parameter <c>KeyOrFile</c> was + <c>{key,string()}</c>. This is now corrected.</p> + <p> + Own Id: OTP-15540</p> + </item> + <item> + <p> + The status of a test case which failed with timetrap + timeout in <c>end_per_testcase</c> could not be modified + by returning <c>{fail,Reason}</c> from a + <c>post_end_per_testcase</c> hook function. This is now + corrected.</p> + <p> + Own Id: OTP-15584 Aux Id: ERIERL-282 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + A new variant of the <c>newline</c> option to + <c>ct_telnet:cmd/3</c> and <c>ct_telnet:send/3</c> is + added, which allows to specify a string to append as + newline indicator on a command. By default, the value is + "\n", but in some cases it is required to be "\r\n", + which this option allows.</p> + <p> + A faulty regular expression given as parameter to + <c>ct_telnet:expect/2,3</c> would earlier crash and look + like an internal error in common_test. A better error + indication is now given, but the test case will still + fail.</p> + <p> + Own Id: OTP-15229 Aux Id: ERIERL-203 </p> + </item> + <item> + <p> + Since the yang RFC allows more than one top element of + config data in an <c>edit-config</c> element, + <c>ct_netconfc:edit_config/3,4,5</c> can now take a list + of XML elements.</p> + <p> + Own Id: OTP-15298</p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.16.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/common_test/test_server/ts_erl_config.erl b/lib/common_test/test_server/ts_erl_config.erl index c2852131d6..f3972bea4e 100644 --- a/lib/common_test/test_server/ts_erl_config.erl +++ b/lib/common_test/test_server/ts_erl_config.erl @@ -197,28 +197,24 @@ system_include(Root, Vars) -> " -I" ++ quote(filename:nativename(filename:join([Root, "erts", "emulator", SysDir]))). erl_interface(Vars,OsType) -> - {Incl, TargetIncl, {LibPath, MkIncl}} = + {Incl, {LibPath, MkIncl}} = case lib_dir(Vars, erl_interface) of {error, bad_name} -> throw({cannot_find_app, erl_interface}); Dir -> - BaseIncl = filename:join(Dir, "include"), - case erl_root(Vars) of - {installed, _Root} -> - {BaseIncl, - [], - {filename:join(Dir, "lib"), - filename:join([Dir, "src", "eidefs.mk"])}}; - {srctree, _Root, Target} -> - Obj = case is_debug_build() of - true -> "obj.debug"; - false -> "obj" - end, - {BaseIncl, - filename:join(BaseIncl, Target), - {filename:join([Dir, Obj, Target]), - filename:join([Dir, "src", Target, "eidefs.mk"])}} - end + {filename:join(Dir, "include"), + case erl_root(Vars) of + {installed, _Root} -> + {filename:join(Dir, "lib"), + filename:join([Dir, "src", "eidefs.mk"])}; + {srctree, _Root, Target} -> + Obj = case is_debug_build() of + true -> "obj.debug"; + false -> "obj" + end, + {filename:join([Dir, Obj, Target]), + filename:join([Dir, "src", Target, "eidefs.mk"])} + end} end, Lib = link_library("erl_interface",OsType), Lib1 = link_library("ei",OsType), @@ -264,10 +260,6 @@ erl_interface(Vars,OsType) -> {erl_interface_eilib_drv, quote(filename:join(LibPath, Lib1Drv))}, {erl_interface_threadlib, ThreadLib}, {erl_interface_include, quote(filename:nativename(Incl))}, - {erl_interface_target_include, case TargetIncl of - [] -> []; - _ -> "-I" ++ quote(filename:nativename(TargetIncl)) - end}, {erl_interface_mk_include, quote(filename:nativename(MkIncl))} | Vars]. diff --git a/lib/common_test/vsn.mk b/lib/common_test/vsn.mk index fd5d4a57aa..23eb8d9656 100644 --- a/lib/common_test/vsn.mk +++ b/lib/common_test/vsn.mk @@ -1 +1 @@ -COMMON_TEST_VSN = 1.16.1 +COMMON_TEST_VSN = 1.17 diff --git a/lib/compiler/doc/src/notes.xml b/lib/compiler/doc/src/notes.xml index 02e6203137..d45dfef8f3 100644 --- a/lib/compiler/doc/src/notes.xml +++ b/lib/compiler/doc/src/notes.xml @@ -32,6 +32,38 @@ <p>This document describes the changes made to the Compiler application.</p> +<section><title>Compiler 7.3.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>An expression such as <c>(A / B) band 16#ff</c> would + crash the compiler.</p> + <p> + Own Id: OTP-15518 Aux Id: ERL-829 </p> + </item> + <item> + <p>There could be an incorrect warning when the + <c>tuple_calls</c> option was given. The generated code + would be correct. Here is an example of code that would + trigger the warning:</p> + <p><c>(list_to_atom("prefix_" ++ + atom_to_list(suffix))):doit(X)</c>.</p> + <p> + Own Id: OTP-15552 Aux Id: ERL-838 </p> + </item> + <item> + <p>Optimize (again) Dialyzer's handling of + left-associative use of <c>andalso</c> and <c>orelse</c> + in guards.</p> + <p> + Own Id: OTP-15577 Aux Id: ERL-851, PR-2141, PR-1944 </p> + </item> + </list> + </section> + +</section> + <section><title>Compiler 7.3.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/compiler/src/beam_ssa_opt.erl b/lib/compiler/src/beam_ssa_opt.erl index bcf55f3fda..90c0d3cf16 100644 --- a/lib/compiler/src/beam_ssa_opt.erl +++ b/lib/compiler/src/beam_ssa_opt.erl @@ -911,6 +911,11 @@ ssa_opt_float({#st{ssa=Linear0,cnt=Count0}=St, FuncDb}) -> {Linear,Count} = float_opt(Linear0, Count0, Fs), {St#st{ssa=Linear,cnt=Count}, FuncDb}. +float_blk_is_in_guard(#b_blk{last=#b_br{fail=F}}, #fs{non_guards=NonGuards}) -> + not gb_sets:is_member(F, NonGuards); +float_blk_is_in_guard(#b_blk{}, #fs{}) -> + false. + float_non_guards([{L,#b_blk{is=Is}}|Bs]) -> case Is of [#b_set{op=landingpad}|_] -> @@ -920,21 +925,18 @@ float_non_guards([{L,#b_blk{is=Is}}|Bs]) -> end; float_non_guards([]) -> [?BADARG_BLOCK]. -float_opt([{L,#b_blk{last=#b_br{fail=F}}=Blk}|Bs0], - Count0, #fs{non_guards=NonGuards}=Fs) -> - case gb_sets:is_member(F, NonGuards) of +float_opt([{L,Blk}|Bs0], Count0, Fs) -> + case float_blk_is_in_guard(Blk, Fs) of true -> - %% This block is not inside a guard. - %% We can do the optimization. - float_opt_1(L, Blk, Bs0, Count0, Fs); - false -> %% This block is inside a guard. Don't do %% any floating point optimizations. {Bs,Count} = float_opt(Bs0, Count0, Fs), - {[{L,Blk}|Bs],Count} + {[{L,Blk}|Bs],Count}; + false -> + %% This block is not inside a guard. + %% We can do the optimization. + float_opt_1(L, Blk, Bs0, Count0, Fs) end; -float_opt([{L,Blk}|Bs], Count, Fs) -> - float_opt_1(L, Blk, Bs, Count, Fs); float_opt([], Count, _Fs) -> {[],Count}. @@ -1010,10 +1012,14 @@ float_conv([{L,#b_blk{is=Is0}=Blk0}|Bs0], Fail, Count0) -> float_maybe_flush(Blk0, #fs{s=cleared,fail=Fail,bs=Blocks}=Fs0, Count0) -> #b_blk{last=#b_br{bool=#b_var{},succ=Succ}=Br} = Blk0, - #b_blk{is=Is} = map_get(Succ, Blocks), + + %% If the success block starts with a floating point operation, we can + %% defer flushing to that block as long as it isn't a guard. + #b_blk{is=Is} = SuccBlk = map_get(Succ, Blocks), + SuccIsGuard = float_blk_is_in_guard(SuccBlk, Fs0), + case Is of - [#b_set{anno=#{float_op:=_}}|_] -> - %% The next operation is also a floating point operation. + [#b_set{anno=#{float_op:=_}}|_] when not SuccIsGuard -> %% No flush needed. {[],Blk0,Fs0,Count0}; _ -> diff --git a/lib/compiler/src/beam_validator.erl b/lib/compiler/src/beam_validator.erl index 296c095be2..ab8caa1a0d 100644 --- a/lib/compiler/src/beam_validator.erl +++ b/lib/compiler/src/beam_validator.erl @@ -579,7 +579,11 @@ valfun_1({get_tuple_element,Src,N,Dst}, Vst) -> Type = get_element_type(Index, Src, Vst), extract_term(Type, {bif,element}, [Index, Src], Dst, Vst); valfun_1({jump,{f,Lbl}}, Vst) -> - kill_state(branch_state(Lbl, Vst)); + branch(Lbl, Vst, + fun(SuccVst) -> + %% The next instruction is never executed. + kill_state(SuccVst) + end); valfun_1(I, Vst) -> valfun_2(I, Vst). @@ -589,14 +593,17 @@ init_try_catch_branch(Tag, Dst, Fail, Vst0) -> St = St0#st{ct=[[Fail]|Fails]}, Vst = Vst0#vst{current=St}, - complex_test(Fail, - fun(CatchVst) -> - #vst{current=#st{ys=Ys}} = CatchVst, - maps:fold(fun init_catch_handler_1/3, CatchVst, Ys) - end, - fun(SuccVst) -> - SuccVst - end, Vst). + branch(Fail, Vst, + fun(CatchVst) -> + #vst{current=#st{ys=Ys}} = CatchVst, + maps:fold(fun init_catch_handler_1/3, CatchVst, Ys) + end, + fun(SuccVst) -> + %% All potentially-throwing instructions after this + %% one will implicitly branch to the fail label; + %% see valfun_2/2 + SuccVst + end). %% Set the initial state at the try/catch label. Assume that Y registers %% contain terms or try/catch tags. @@ -607,21 +614,27 @@ init_catch_handler_1(Reg, uninitialized, Vst) -> init_catch_handler_1(_, _, Vst) -> Vst. -%% Update branched state if necessary and try next set of instructions. -valfun_2(I, #vst{current=#st{ct=[]}}=Vst) -> - valfun_3(I, Vst); valfun_2(I, #vst{current=#st{ct=[[Fail]|_]}}=Vst) when is_integer(Fail) -> - %% Update branched state. + %% We have an active try/catch tag and we can jump there from this + %% instruction, so we need to update the branched state of the try/catch + %% handler. valfun_3(I, branch_state(Fail, Vst)); +valfun_2(I, #vst{current=#st{ct=[]}}=Vst) -> + valfun_3(I, Vst); valfun_2(_, _) -> error(ambiguous_catch_try_state). %% Handle the remaining floating point instructions here. %% Floating point. -valfun_3({fconv,Src,{fr,_}=Dst}, Vst0) -> - assert_term(Src, Vst0), - Vst = update_type(fun meet/2, number, Src, Vst0), - set_freg(Dst, Vst); +valfun_3({fconv,Src,{fr,_}=Dst}, Vst) -> + assert_term(Src, Vst), + + %% An exception is raised on error, hence branching to 0. + branch(0, Vst, + fun(SuccVst0) -> + SuccVst = update_type(fun meet/2, number, Src, SuccVst0), + set_freg(Dst, SuccVst) + end); valfun_3({bif,fadd,_,[_,_]=Ss,Dst}, Vst) -> float_op(Ss, Dst, Vst); valfun_3({bif,fdiv,_,[_,_]=Ss,Dst}, Vst) -> @@ -682,67 +695,87 @@ valfun_4({call_ext_last,_,_,_}, #vst{current=#st{numy=NumY}}) -> valfun_4({make_fun2,_,_,_,Live}, Vst) -> call(make_fun, Live, Vst); %% Other BIFs -valfun_4({bif,element,{f,Fail},[Pos,Tuple],Dst}, Vst0) -> - PosType = get_term_type(Pos, Vst0), - ElementType = get_element_type(PosType, Tuple, Vst0), - InferredType = {tuple,[get_tuple_size(PosType)],#{}}, - Vst1 = branch_state(Fail, Vst0), - Vst2 = update_type(fun meet/2, InferredType, Tuple, Vst1), - Vst = update_type(fun meet/2, {integer,[]}, Pos, Vst2), - extract_term(ElementType, {bif,element}, [Pos,Tuple], Dst, Vst); +valfun_4({bif,element,{f,Fail},[Pos,Src],Dst}, Vst) -> + branch(Fail, Vst, + fun(SuccVst0) -> + PosType = get_term_type(Pos, SuccVst0), + TupleType = {tuple,[get_tuple_size(PosType)],#{}}, + + SuccVst1 = update_type(fun meet/2, TupleType, + Src, SuccVst0), + SuccVst = update_type(fun meet/2, {integer,[]}, + Pos, SuccVst1), + + ElementType = get_element_type(PosType, Src, SuccVst), + extract_term(ElementType, {bif,element}, [Pos,Src], + Dst, SuccVst) + end); valfun_4({bif,raise,{f,0},Src,_Dst}, Vst) -> validate_src(Src, Vst), kill_state(Vst); valfun_4(raw_raise=I, Vst) -> call(I, 3, Vst); -valfun_4({bif,Op,{f,Fail},[Cons]=Ss,Dst}, Vst0) - when Op =:= hd; Op =:= tl -> - validate_src(Ss, Vst0), - Vst = type_test(Fail, cons, Cons, Vst0), - Type = bif_return_type(Op, Ss, Vst), - extract_term(Type, {bif,Op}, Ss, Dst, Vst); -valfun_4({bif,Op,{f,Fail},Ss,Dst}, Vst0) -> - validate_src(Ss, Vst0), - Vst1 = branch_state(Fail, Vst0), - - %% Infer argument types. Note that we can't type_test in the general case - %% as the BIF could fail for reasons other than bad argument types. - ArgTypes = bif_arg_types(Op, Ss), - Vst = foldl(fun({Arg, T}, Vsti) -> - update_type(fun meet/2, T, Arg, Vsti) - end, Vst1, zip(Ss, ArgTypes)), - - Type = bif_return_type(Op, Ss, Vst), - extract_term(Type, {bif,Op}, Ss, Dst, Vst); +valfun_4({bif,Op,{f,Fail},[Src]=Ss,Dst}, Vst) when Op =:= hd; Op =:= tl -> + assert_term(Src, Vst), + branch(Fail, Vst, + fun(FailVst) -> + update_type(fun subtract/2, cons, Src, FailVst) + end, + fun(SuccVst0) -> + SuccVst = update_type(fun meet/2, cons, Src, SuccVst0), + extract_term(term, {bif,Op}, Ss, Dst, SuccVst) + end); +valfun_4({bif,Op,{f,Fail},Ss,Dst}, Vst) -> + validate_src(Ss, Vst), + branch(Fail, Vst, + fun(SuccVst0) -> + %% Infer argument types. Note that we can't subtract + %% types as the BIF could fail for reasons other than + %% bad argument types. + ArgTypes = bif_arg_types(Op, Ss), + SuccVst = foldl(fun({Arg, T}, V) -> + update_type(fun meet/2, T, Arg, V) + end, SuccVst0, zip(Ss, ArgTypes)), + Type = bif_return_type(Op, Ss, SuccVst), + extract_term(Type, {bif,Op}, Ss, Dst, SuccVst) + end); valfun_4({gc_bif,Op,{f,Fail},Live,Ss,Dst}, #vst{current=St0}=Vst0) -> validate_src(Ss, Vst0), verify_live(Live, Vst0), verify_y_init(Vst0), + + %% Heap allocations and X registers are killed regardless of whether we + %% fail or not, as we may fail after GC. St = kill_heap_allocation(St0), - Vst1 = Vst0#vst{current=St}, - Vst2 = branch_state(Fail, Vst1), + Vst = prune_x_regs(Live, Vst0#vst{current=St}), + + branch(Fail, Vst, + fun(SuccVst0) -> + ArgTypes = bif_arg_types(Op, Ss), + SuccVst = foldl(fun({Arg, T}, V) -> + update_type(fun meet/2, T, Arg, V) + end, SuccVst0, zip(Ss, ArgTypes)), - ArgTypes = bif_arg_types(Op, Ss), - Vst3 = foldl(fun({Arg, T}, Vsti) -> - update_type(fun meet/2, T, Arg, Vsti) - end, Vst2, zip(Ss, ArgTypes)), + Type = bif_return_type(Op, Ss, SuccVst), - Type = bif_return_type(Op, Ss, Vst3), - Vst = prune_x_regs(Live, Vst3), - extract_term(Type, {gc_bif,Op}, Ss, Dst, Vst, Vst0); + %% We're passing Vst0 as the original because the + %% registers were pruned before the branch. + extract_term(Type, {gc_bif,Op}, Ss, Dst, SuccVst, Vst0) + end); valfun_4(return, #vst{current=#st{numy=none}}=Vst) -> assert_durable_term({x,0}, Vst), kill_state(Vst); valfun_4(return, #vst{current=#st{numy=NumY}}) -> error({stack_frame,NumY}); -valfun_4({loop_rec,{f,Fail},Dst}, Vst0) -> - %% This term may not be part of the root set until - %% remove_message/0 is executed. If control transfers - %% to the loop_rec_end/1 instruction, no part of - %% this term must be stored in a Y register. - Vst1 = branch_state(Fail, Vst0), - {Ref, Vst} = new_value(term, loop_rec, [], Vst1), - mark_fragile(Dst, set_reg_vref(Ref, Dst, Vst)); +valfun_4({loop_rec,{f,Fail},Dst}, Vst) -> + %% This term may not be part of the root set until remove_message/0 is + %% executed. If control transfers to the loop_rec_end/1 instruction, no + %% part of this term must be stored in a Y register. + branch(Fail, Vst, + fun(SuccVst0) -> + {Ref, SuccVst} = new_value(term, loop_rec, [], SuccVst0), + mark_fragile(Dst, set_reg_vref(Ref, Dst, SuccVst)) + end); valfun_4({wait,_}, Vst) -> verify_y_init(Vst), kill_state(Vst); @@ -769,14 +802,14 @@ valfun_4({set_tuple_element,Src,Tuple,N}, Vst) -> Es = set_element_type({integer,I}, get_term_type(Src, Vst), Es0), override_type({tuple, Sz, Es}, Tuple, Vst); %% Match instructions. -valfun_4({select_val,Src,{f,Fail},{list,Choices}}, Vst0) -> - assert_term(Src, Vst0), +valfun_4({select_val,Src,{f,Fail},{list,Choices}}, Vst) -> + assert_term(Src, Vst), assert_choices(Choices), - select_val_branches(Fail, Src, Choices, Vst0); + validate_select_val(Fail, Choices, Src, Vst); valfun_4({select_tuple_arity,Tuple,{f,Fail},{list,Choices}}, Vst) -> assert_type(tuple, Tuple, Vst), assert_arities(Choices), - select_arity_branches(Fail, Choices, Tuple, Vst); + validate_select_tuple_arity(Fail, Choices, Tuple, Vst); %% New bit syntax matching instructions. valfun_4({test,bs_start_match3,{f,Fail},Live,[Src],Dst}, Vst) -> @@ -785,17 +818,17 @@ valfun_4({test,bs_start_match2,{f,Fail},Live,[Src,Slots],Dst}, Vst) -> validate_bs_start_match(Fail, Live, bsm_match_state(Slots), Src, Dst, Vst); valfun_4({test,bs_match_string,{f,Fail},[Ctx,_,_]}, Vst) -> bsm_validate_context(Ctx, Vst), - branch_state(Fail, Vst); + branch(Fail, Vst, fun(V) -> V end); valfun_4({test,bs_skip_bits2,{f,Fail},[Ctx,Src,_,_]}, Vst) -> bsm_validate_context(Ctx, Vst), assert_term(Src, Vst), - branch_state(Fail, Vst); + branch(Fail, Vst, fun(V) -> V end); valfun_4({test,bs_test_tail2,{f,Fail},[Ctx,_]}, Vst) -> bsm_validate_context(Ctx, Vst), - branch_state(Fail, Vst); + branch(Fail, Vst, fun(V) -> V end); valfun_4({test,bs_test_unit,{f,Fail},[Ctx,_]}, Vst) -> bsm_validate_context(Ctx, Vst), - branch_state(Fail, Vst); + branch(Fail, Vst, fun(V) -> V end); valfun_4({test,bs_skip_utf8,{f,Fail},[Ctx,Live,_]}, Vst) -> validate_bs_skip_utf(Fail, Ctx, Live, Vst); valfun_4({test,bs_skip_utf16,{f,Fail},[Ctx,Live,_]}, Vst) -> @@ -830,6 +863,10 @@ valfun_4({bs_set_position, Ctx, Pos}, Vst) -> Vst; %% Other test instructions. +valfun_4({test,has_map_fields,{f,Lbl},Src,{list,List}}, Vst) -> + assert_type(map, Src, Vst), + assert_unique_map_keys(List), + branch(Lbl, Vst, fun(V) -> V end); valfun_4({test,is_atom,{f,Lbl},[Src]}, Vst) -> type_test(Lbl, {atom,[]}, Src, Vst); valfun_4({test,is_binary,{f,Lbl},[Src]}, Vst) -> @@ -850,70 +887,68 @@ valfun_4({test,is_number,{f,Lbl},[Src]}, Vst) -> type_test(Lbl, number, Src, Vst); valfun_4({test,is_list,{f,Lbl},[Src]}, Vst) -> type_test(Lbl, list, Src, Vst); +valfun_4({test,is_map,{f,Lbl},[Src]}, Vst) -> + type_test(Lbl, map, Src, Vst); valfun_4({test,is_nil,{f,Lbl},[Src]}, Vst) -> %% is_nil is an exact check against the 'nil' value, and should not be %% treated as a simple type test. assert_term(Src, Vst), - complex_test(Lbl, - fun(FailVst) -> - update_ne_types(Src, nil, FailVst) - end, - fun(SuccVst) -> - update_eq_types(Src, nil, SuccVst) - end, Vst); -valfun_4({test,is_map,{f,Lbl},[Src]}, Vst) -> - case Src of - {Tag,_} when Tag =:= x; Tag =:= y -> - type_test(Lbl, map, Src, Vst); - {literal,Map} when is_map(Map) -> - Vst; - _ -> - assert_term(Src, Vst), - kill_state(Vst) - end; -valfun_4({test,test_arity,{f,Lbl},[Tuple,Sz]}, Vst0) when is_integer(Sz) -> - assert_type(tuple, Tuple, Vst0), - Vst = branch_state(Lbl, Vst0), - update_type(fun meet/2, {tuple,Sz,#{}}, Tuple, Vst); -valfun_4({test,is_tagged_tuple,{f,Lbl},[Src,Sz,Atom]}, Vst0) -> - assert_term(Src, Vst0), - Vst = branch_state(Lbl, Vst0), - update_type(fun meet/2, {tuple,Sz,#{ {integer,1} => Atom }}, Src, Vst); -valfun_4({test,has_map_fields,{f,Lbl},Src,{list,List}}, Vst) -> - assert_type(map, Src, Vst), - assert_unique_map_keys(List), - branch_state(Lbl, Vst); + branch(Lbl, Vst, + fun(FailVst) -> + update_ne_types(Src, nil, FailVst) + end, + fun(SuccVst) -> + update_eq_types(Src, nil, SuccVst) + end); +valfun_4({test,test_arity,{f,Lbl},[Tuple,Sz]}, Vst) when is_integer(Sz) -> + assert_type(tuple, Tuple, Vst), + Type = {tuple, Sz, #{}}, + type_test(Lbl, Type, Tuple, Vst); +valfun_4({test,is_tagged_tuple,{f,Lbl},[Src,Sz,Atom]}, Vst) -> + assert_term(Src, Vst), + Type = {tuple, Sz, #{ {integer,1} => Atom }}, + type_test(Lbl, Type, Src, Vst); valfun_4({test,is_eq_exact,{f,Lbl},[Src,Val]=Ss}, Vst) -> validate_src(Ss, Vst), - complex_test(Lbl, - fun(FailVst) -> - update_ne_types(Src, Val, FailVst) - end, - fun(SuccVst) -> - update_eq_types(Src, Val, SuccVst) - end, Vst); + branch(Lbl, Vst, + fun(FailVst) -> + update_ne_types(Src, Val, FailVst) + end, + fun(SuccVst) -> + update_eq_types(Src, Val, SuccVst) + end); valfun_4({test,is_ne_exact,{f,Lbl},[Src,Val]=Ss}, Vst) -> validate_src(Ss, Vst), - complex_test(Lbl, - fun(FailVst) -> - update_eq_types(Src, Val, FailVst) - end, - fun(SuccVst) -> - update_ne_types(Src, Val, SuccVst) - end, Vst); + branch(Lbl, Vst, + fun(FailVst) -> + update_eq_types(Src, Val, FailVst) + end, + fun(SuccVst) -> + update_ne_types(Src, Val, SuccVst) + end); valfun_4({test,_Op,{f,Lbl},Src}, Vst) -> + %% is_pid, is_reference, et cetera. validate_src(Src, Vst), - branch_state(Lbl, Vst); + branch(Lbl, Vst, fun(V) -> V end); valfun_4({bs_add,{f,Fail},[A,B,_],Dst}, Vst) -> assert_term(A, Vst), assert_term(B, Vst), - create_term({integer,[]}, bs_add, [A, B], Dst, branch_state(Fail, Vst)); + branch(Fail, Vst, + fun(SuccVst) -> + create_term({integer,[]}, bs_add, [A, B], Dst, SuccVst) + end); valfun_4({bs_utf8_size,{f,Fail},A,Dst}, Vst) -> assert_term(A, Vst), - create_term({integer,[]}, bs_utf8_size, [A], Dst, branch_state(Fail, Vst)); + branch(Fail, Vst, + fun(SuccVst) -> + create_term({integer,[]}, bs_utf8_size, [A], Dst, SuccVst) + end); valfun_4({bs_utf16_size,{f,Fail},A,Dst}, Vst) -> assert_term(A, Vst), - create_term({integer,[]}, bs_utf16_size, [A], Dst, branch_state(Fail, Vst)); + branch(Fail, Vst, + fun(SuccVst) -> + create_term({integer,[]}, bs_utf16_size, [A], Dst, SuccVst) + end); valfun_4({bs_init2,{f,Fail},Sz,Heap,Live,_,Dst}, Vst0) -> verify_live(Live, Vst0), verify_y_init(Vst0), @@ -923,10 +958,12 @@ valfun_4({bs_init2,{f,Fail},Sz,Heap,Live,_,Dst}, Vst0) -> true -> assert_term(Sz, Vst0) end, - Vst1 = heap_alloc(Heap, Vst0), - Vst2 = branch_state(Fail, Vst1), - Vst = prune_x_regs(Live, Vst2), - create_term(binary, bs_init2, [], Dst, Vst, Vst0); + Vst = heap_alloc(Heap, Vst0), + branch(Fail, Vst, + fun(SuccVst0) -> + SuccVst = prune_x_regs(Live, SuccVst0), + create_term(binary, bs_init2, [], Dst, SuccVst, SuccVst0) + end); valfun_4({bs_init_bits,{f,Fail},Sz,Heap,Live,_,Dst}, Vst0) -> verify_live(Live, Vst0), verify_y_init(Vst0), @@ -936,47 +973,71 @@ valfun_4({bs_init_bits,{f,Fail},Sz,Heap,Live,_,Dst}, Vst0) -> true -> assert_term(Sz, Vst0) end, - Vst1 = heap_alloc(Heap, Vst0), - Vst2 = branch_state(Fail, Vst1), - Vst = prune_x_regs(Live, Vst2), - create_term(binary, bs_init_bits, [], Dst, Vst); + Vst = heap_alloc(Heap, Vst0), + branch(Fail, Vst, + fun(SuccVst0) -> + SuccVst = prune_x_regs(Live, SuccVst0), + create_term(binary, bs_init_bits, [], Dst, SuccVst) + end); valfun_4({bs_append,{f,Fail},Bits,Heap,Live,_Unit,Bin,_Flags,Dst}, Vst0) -> verify_live(Live, Vst0), verify_y_init(Vst0), assert_term(Bits, Vst0), assert_term(Bin, Vst0), - Vst1 = heap_alloc(Heap, Vst0), - Vst2 = branch_state(Fail, Vst1), - Vst = prune_x_regs(Live, Vst2), - create_term(binary, bs_append, [Bin], Dst, Vst, Vst0); -valfun_4({bs_private_append,{f,Fail},Bits,_Unit,Bin,_Flags,Dst}, Vst0) -> - assert_term(Bits, Vst0), - assert_term(Bin, Vst0), - Vst = branch_state(Fail, Vst0), - create_term(binary, bs_private_append, [Bin], Dst, Vst); + Vst = heap_alloc(Heap, Vst0), + branch(Fail, Vst, + fun(SuccVst0) -> + SuccVst = prune_x_regs(Live, SuccVst0), + create_term(binary, bs_append, [Bin], Dst, SuccVst, SuccVst0) + end); +valfun_4({bs_private_append,{f,Fail},Bits,_Unit,Bin,_Flags,Dst}, Vst) -> + assert_term(Bits, Vst), + assert_term(Bin, Vst), + branch(Fail, Vst, + fun(SuccVst) -> + create_term(binary, bs_private_append, [Bin], Dst, SuccVst) + end); valfun_4({bs_put_string,Sz,_}, Vst) when is_integer(Sz) -> Vst; valfun_4({bs_put_binary,{f,Fail},Sz,_,_,Src}, Vst) -> assert_term(Sz, Vst), assert_term(Src, Vst), - branch_state(Fail, Vst); + branch(Fail, Vst, + fun(SuccVst) -> + update_type(fun meet/2, binary, Src, SuccVst) + end); valfun_4({bs_put_float,{f,Fail},Sz,_,_,Src}, Vst) -> assert_term(Sz, Vst), assert_term(Src, Vst), - branch_state(Fail, Vst); + branch(Fail, Vst, + fun(SuccVst) -> + update_type(fun meet/2, {float,[]}, Src, SuccVst) + end); valfun_4({bs_put_integer,{f,Fail},Sz,_,_,Src}, Vst) -> assert_term(Sz, Vst), assert_term(Src, Vst), - branch_state(Fail, Vst); + branch(Fail, Vst, + fun(SuccVst) -> + update_type(fun meet/2, {integer,[]}, Src, SuccVst) + end); valfun_4({bs_put_utf8,{f,Fail},_,Src}, Vst) -> assert_term(Src, Vst), - branch_state(Fail, Vst); + branch(Fail, Vst, + fun(SuccVst) -> + update_type(fun meet/2, {integer,[]}, Src, SuccVst) + end); valfun_4({bs_put_utf16,{f,Fail},_,Src}, Vst) -> assert_term(Src, Vst), - branch_state(Fail, Vst); + branch(Fail, Vst, + fun(SuccVst) -> + update_type(fun meet/2, {integer,[]}, Src, SuccVst) + end); valfun_4({bs_put_utf32,{f,Fail},_,Src}, Vst) -> assert_term(Src, Vst), - branch_state(Fail, Vst); + branch(Fail, Vst, + fun(SuccVst) -> + update_type(fun meet/2, {integer,[]}, Src, SuccVst) + end); %% Map instructions. valfun_4({put_map_assoc=Op,{f,Fail},Src,Dst,Live,{list,List}}, Vst) -> verify_put_map(Op, Fail, Src, Dst, Live, List, Vst); @@ -991,15 +1052,15 @@ verify_get_map(Fail, Src, List, Vst0) -> assert_not_literal(Src), %OTP 22. assert_type(map, Src, Vst0), - complex_test(Fail, - fun(FailVst) -> - clobber_map_vals(List, Src, FailVst) - end, - fun(SuccVst) -> - Keys = extract_map_keys(List), - assert_unique_map_keys(Keys), - extract_map_vals(List, Src, SuccVst, SuccVst) - end, Vst0). + branch(Fail, Vst0, + fun(FailVst) -> + clobber_map_vals(List, Src, FailVst) + end, + fun(SuccVst) -> + Keys = extract_map_keys(List), + assert_unique_map_keys(Keys), + extract_map_vals(List, Src, SuccVst, SuccVst) + end). %% get_map_elements may leave its destinations in an inconsistent state when %% the fail label is taken. Consider the following: @@ -1033,13 +1094,16 @@ verify_put_map(Op, Fail, Src, Dst, Live, List, Vst0) -> assert_type(map, Src, Vst0), verify_live(Live, Vst0), verify_y_init(Vst0), - [assert_term(Term, Vst0) || Term <- List], - Vst1 = heap_alloc(0, Vst0), - Vst2 = branch_state(Fail, Vst1), - Vst = prune_x_regs(Live, Vst2), - Keys = extract_map_keys(List), - assert_unique_map_keys(Keys), - create_term(map, Op, [Src], Dst, Vst, Vst0). + _ = [assert_term(Term, Vst0) || Term <- List], + Vst = heap_alloc(0, Vst0), + + branch(Fail, Vst, + fun(SuccVst0) -> + SuccVst = prune_x_regs(Live, SuccVst0), + Keys = extract_map_keys(List), + assert_unique_map_keys(Keys), + create_term(map, Op, [Src], Dst, SuccVst, SuccVst0) + end). %% %% Common code for validating bs_start_match* instructions. @@ -1052,40 +1116,60 @@ validate_bs_start_match(Fail, Live, Type, Src, Dst, Vst) -> %% #ms{} can represent either a match context or a term, so we have to mark %% the source as a term if it fails with a match context as an input. This %% hack is only needed until we get proper union types. - complex_test(Fail, - fun(FailVst) -> - case get_raw_type(Src, FailVst) of - #ms{} -> override_type(term, Src, FailVst); - _ -> FailVst - end - end, - fun(SuccVst0) -> - SuccVst1 = update_type(fun meet/2, binary, Src, SuccVst0), - SuccVst = prune_x_regs(Live, SuccVst1), - extract_term(Type, bs_start_match, [Src], Dst, - SuccVst, SuccVst0) - end, Vst). + branch(Fail, Vst, + fun(FailVst) -> + case get_movable_term_type(Src, FailVst) of + #ms{} -> override_type(term, Src, FailVst); + _ -> FailVst + end + end, + fun(SuccVst0) -> + SuccVst1 = update_type(fun meet/2, binary, + Src, SuccVst0), + SuccVst = prune_x_regs(Live, SuccVst1), + extract_term(Type, bs_start_match, [Src], Dst, + SuccVst, SuccVst0) + end). %% %% Common code for validating bs_get* instructions. %% -validate_bs_get(Op, Fail, Ctx, Live, Type, Dst, Vst0) -> - bsm_validate_context(Ctx, Vst0), - verify_live(Live, Vst0), - verify_y_init(Vst0), - Vst1 = prune_x_regs(Live, Vst0), - Vst = branch_state(Fail, Vst1), - extract_term(Type, Op, [Ctx], Dst, Vst, Vst0). +validate_bs_get(Op, Fail, Ctx, Live, Type, Dst, Vst) -> + bsm_validate_context(Ctx, Vst), + verify_live(Live, Vst), + verify_y_init(Vst), + + branch(Fail, Vst, + fun(SuccVst0) -> + SuccVst = prune_x_regs(Live, SuccVst0), + extract_term(Type, Op, [Ctx], Dst, SuccVst, SuccVst0) + end). %% %% Common code for validating bs_skip_utf* instructions. %% -validate_bs_skip_utf(Fail, Ctx, Live, Vst0) -> - bsm_validate_context(Ctx, Vst0), - verify_y_init(Vst0), - verify_live(Live, Vst0), - Vst = prune_x_regs(Live, Vst0), - branch_state(Fail, Vst). +validate_bs_skip_utf(Fail, Ctx, Live, Vst) -> + bsm_validate_context(Ctx, Vst), + verify_y_init(Vst), + verify_live(Live, Vst), + + branch(Fail, Vst, + fun(SuccVst) -> + prune_x_regs(Live, SuccVst) + end). + +%% +%% Common code for is_$type instructions. +%% +type_test(Fail, Type, Reg, Vst) -> + assert_term(Reg, Vst), + branch(Fail, Vst, + fun(FailVst) -> + update_type(fun subtract/2, Type, Reg, FailVst) + end, + fun(SuccVst) -> + update_type(fun meet/2, Type, Reg, SuccVst) + end). %% %% Special state handling for setelement/3 and set_tuple_element/3 instructions. @@ -1156,9 +1240,8 @@ verify_local_args(-1, _Lbl, _CtxIds, _Vst) -> ok; verify_local_args(X, Lbl, CtxIds, Vst) -> Reg = {x, X}, - assert_movable(Reg, Vst), assert_not_fragile(Reg, Vst), - case get_raw_type(Reg, Vst) of + case get_movable_term_type(Reg, Vst) of #ms{id=Id}=Type -> case CtxIds of #{ Id := Other } -> @@ -1347,7 +1430,7 @@ assert_arities(_) -> error(bad_tuple_arity_list). %%% float_op(Ss, Dst, Vst0) -> - [assert_freg_set(S, Vst0) || S <- Ss], + _ = [assert_freg_set(S, Vst0) || S <- Ss], assert_fls(cleared, Vst0), Vst = set_fls(cleared, Vst0), set_freg(Dst, Vst). @@ -1424,7 +1507,7 @@ bsm_validate_context(Reg, Vst) -> ok. bsm_get_context({Kind,_}=Reg, Vst) when Kind =:= x; Kind =:= y-> - case get_raw_type(Reg, Vst) of + case get_movable_term_type(Reg, Vst) of #ms{}=Ctx -> Ctx; _ -> error({no_bsm_context,Reg}) end; @@ -1459,44 +1542,46 @@ bsm_restore(Reg, SavePoint, Vst) -> _ -> error({illegal_restore,SavePoint,range}) end. -select_val_branches(Fail, Src, Choices, Vst0) -> - Vst = svb_1(Choices, Src, Vst0), - kill_state(branch_state(Fail, Vst)). - -svb_1([Val,{f,L}|T], Src, Vst0) -> - Vst = complex_test(L, - fun(BranchVst) -> - update_eq_types(Src, Val, BranchVst) - end, - fun(FailVst) -> - update_ne_types(Src, Val, FailVst) - end, Vst0), - svb_1(T, Src, Vst); -svb_1([], _, Vst) -> - Vst. - -select_arity_branches(Fail, List, Tuple, Vst0) -> - Type = get_term_type(Tuple, Vst0), - Vst = sab_1(List, Tuple, Type, Vst0), - kill_state(branch_state(Fail, Vst)). - -sab_1([Sz,{f,L}|T], Tuple, {tuple,[_],Es}=Type0, Vst0) -> - #vst{current=St0} = Vst0, - Vst1 = update_type(fun meet/2, {tuple,Sz,Es}, Tuple, Vst0), - Vst2 = branch_state(L, Vst1), - Vst = Vst2#vst{current=St0}, - - sab_1(T, Tuple, Type0, Vst); -sab_1([Sz,{f,L}|T], Tuple, {tuple,Sz,_Es}=Type, Vst0) -> - %% The type is already correct. (This test is redundant.) - Vst = branch_state(L, Vst0), - sab_1(T, Tuple, Type, Vst); -sab_1([_,{f,_}|T], Tuple, Type, Vst) -> - %% We already have an established different exact size for the tuple. - %% This label can't possibly be reached. - sab_1(T, Tuple, Type, Vst); -sab_1([], _, _, #vst{}=Vst) -> - Vst. +validate_select_val(_Fail, _Choices, _Src, #vst{current=none}=Vst) -> + %% We've already branched on all of Src's possible values, so we know we + %% can't reach the fail label or any of the remaining choices. + Vst; +validate_select_val(Fail, [Val,{f,L}|T], Src, Vst0) -> + Vst = branch(L, Vst0, + fun(BranchVst) -> + update_eq_types(Src, Val, BranchVst) + end, + fun(FailVst) -> + update_ne_types(Src, Val, FailVst) + end), + validate_select_val(Fail, T, Src, Vst); +validate_select_val(Fail, [], _, Vst) -> + branch(Fail, Vst, + fun(SuccVst) -> + %% The next instruction is never executed. + kill_state(SuccVst) + end). + +validate_select_tuple_arity(_Fail, _Choices, _Src, #vst{current=none}=Vst) -> + %% We've already branched on all of Src's possible values, so we know we + %% can't reach the fail label or any of the remaining choices. + Vst; +validate_select_tuple_arity(Fail, [Arity,{f,L}|T], Tuple, Vst0) -> + Type = {tuple, Arity, #{}}, + Vst = branch(L, Vst0, + fun(BranchVst) -> + update_type(fun meet/2, Type, Tuple, BranchVst) + end, + fun(FailVst) -> + update_type(fun subtract/2, Type, Tuple, FailVst) + end), + validate_select_tuple_arity(Fail, T, Tuple, Vst); +validate_select_tuple_arity(Fail, [], _, #vst{}=Vst) -> + branch(Fail, Vst, + fun(SuccVst) -> + %% The next instruction is never executed. + kill_state(SuccVst) + end). infer_types({Kind,_}=Reg, Vst) when Kind =:= x; Kind =:= y -> infer_types(get_reg_vref(Reg, Vst), Vst); @@ -1628,26 +1713,6 @@ resolve_args([Lit | Args], Vst) -> resolve_args([], _Vst) -> []. -%% Helper functions for tests that alter state on both the success and fail -%% branches, keeping the states from tainting each other. -complex_test(Fail, FailFun, SuccFun, Vst0) -> - #vst{current=St0} = Vst0, - Vst1 = FailFun(Vst0), - Vst2 = branch_state(Fail, Vst1), - Vst = Vst2#vst{current=St0}, - SuccFun(Vst). - -%% Helper function for simple "is_type" tests. -type_test(Fail, Type, Reg, Vst) -> - assert_term(Reg, Vst), - complex_test(Fail, - fun(FailVst) -> - update_type(fun subtract/2, Type, Reg, FailVst) - end, - fun(SuccVst) -> - update_type(fun meet/2, Type, Reg, SuccVst) - end, Vst). - %% Overrides the type of Reg. This is ugly but a necessity for certain %% destructive operations. override_type(Type, Reg, Vst) -> @@ -1655,7 +1720,7 @@ override_type(Type, Reg, Vst) -> %% This is used when linear code finds out more and more information about a %% type, so that the type gets more specialized. -update_type(Merge, Type0, #value_ref{}=Ref, Vst) -> +update_type(Merge, With, #value_ref{}=Ref, Vst) -> %% If the old type can't be merged with the new one, the type information %% is inconsistent and we know that some instructions will never be %% executed at run-time. For example: @@ -1664,21 +1729,47 @@ update_type(Merge, Type0, #value_ref{}=Ref, Vst) -> %% {test,is_tuple,Fail,[Reg]}. %% {test,test_arity,Fail,[Reg,5]}. %% - %% Note that the test_arity instruction can never be reached, so we use the - %% new type instead of 'none'. - Type = case Merge(get_raw_type(Ref, Vst), Type0) of - none -> Type0; - T -> T - end, - set_type(Type, Ref, Vst); -update_type(Merge, Type, {Kind,_}=Reg, Vst) when Kind =:= x; Kind =:= y -> - update_type(Merge, Type, get_reg_vref(Reg, Vst), Vst); -update_type(_Merge, _Type, Literal, Vst) -> + %% Note that the test_arity instruction can never be reached, so we need to + %% kill the state to avoid raising an error when we encounter it. + %% + %% Simply returning `kill_state(Vst)` is unsafe however as we might be in + %% the middle of an instruction, and altering the rest of the validator + %% (eg. prune_x_regs/2) to no-op on dead states is prone to error. + %% + %% We therefore throw a 'type_conflict' error instead, which causes + %% validation to fail unless we're in a context where such errors can be + %% handled, such as in a branch handler. + Current = get_raw_type(Ref, Vst), + case Merge(Current, With) of + none -> throw({type_conflict, Current, With}); + Type -> set_type(Type, Ref, Vst) + end; +update_type(Merge, With, {Kind,_}=Reg, Vst) when Kind =:= x; Kind =:= y -> + update_type(Merge, With, get_reg_vref(Reg, Vst), Vst); +update_type(Merge, With, Literal, Vst) -> assert_literal(Literal), - Vst. + %% Literals always retain their type, but we still need to bail on type + %% conflicts. + case Merge(Literal, With) of + none -> throw({type_conflict, Literal, With}); + _Type -> Vst + end. update_ne_types(LHS, RHS, Vst) -> - update_type(fun subtract/2, get_term_type(RHS, Vst), LHS, Vst). + %% While updating types on equality is fairly straightforward, inequality + %% is a bit trickier since all we know is that the *value* of LHS differs + %% from RHS, so we can't blindly subtract their types. + %% + %% Consider `number =/= {integer,[]}`; all we know is that LHS isn't equal + %% to some *specific integer* of unknown value, and if we were to subtract + %% {integer,[]} we would erroneously infer that the new type is {float,[]}. + %% + %% Therefore, we only subtract when we know that RHS has a specific value. + RType = get_term_type(RHS, Vst), + case is_literal(RType) of + true -> update_type(fun subtract/2, RType, LHS, Vst); + false -> Vst + end. update_eq_types(LHS, RHS, Vst0) -> Infer = infer_types(LHS, Vst0), @@ -1786,19 +1877,27 @@ assert_term(Src, Vst) -> ok. assert_movable(Src, Vst) -> - _ = get_move_term_type(Src, Vst), + _ = get_movable_term_type(Src, Vst), ok. -assert_literal(nil) -> ok; -assert_literal({atom,A}) when is_atom(A) -> ok; -assert_literal({float,F}) when is_float(F) -> ok; -assert_literal({integer,I}) when is_integer(I) -> ok; -assert_literal({literal,_L}) -> ok; -assert_literal(T) -> error({literal_required,T}). +assert_literal(Src) -> + case is_literal(Src) of + true -> ok; + false -> error({literal_required,Src}) + end. + +assert_not_literal(Src) -> + case is_literal(Src) of + true -> error({literal_not_allowed,Src}); + false -> ok + end. -assert_not_literal({x,_}) -> ok; -assert_not_literal({y,_}) -> ok; -assert_not_literal(Literal) -> error({literal_not_allowed,Literal}). +is_literal(nil) -> true; +is_literal({atom,A}) when is_atom(A) -> true; +is_literal({float,F}) when is_float(F) -> true; +is_literal({integer,I}) when is_integer(I) -> true; +is_literal({literal,_L}) -> true; +is_literal(_) -> false. %% The possible types. %% @@ -2050,6 +2149,7 @@ assert_tuple_elements(Limit, Es) -> %% Subtract Type2 from Type2. Example: %% subtract(list, nil) -> cons +subtract(Same, Same) -> none; subtract(list, nil) -> cons; subtract(list, cons) -> nil; subtract(number, {integer,[]}) -> {float,[]}; @@ -2082,11 +2182,10 @@ assert_type(Needed, Actual) -> get_element_type(Key, Src, Vst) -> get_element_type_1(Key, get_term_type(Src, Vst)). -get_element_type_1({integer,Index}=Key, {tuple,Sz,Es}) -> +get_element_type_1({integer,_}=Key, {tuple,_Sz,Es}) -> case Es of #{ Key := Type } -> Type; - #{} when Index =< Sz -> term; - #{} -> none + #{} -> term end; get_element_type_1(_Index, _Type) -> term. @@ -2103,7 +2202,7 @@ get_tuple_size({integer,Sz}) -> Sz; get_tuple_size(_) -> 0. validate_src(Ss, Vst) when is_list(Ss) -> - [assert_term(S, Vst) || S <- Ss], + _ = [assert_term(S, Vst) || S <- Ss], ok. %% get_term_type(Src, ValidatorState) -> Type @@ -2111,22 +2210,23 @@ validate_src(Ss, Vst) when is_list(Ss) -> %% a standard Erlang type (no catch/try tags or match contexts). get_term_type(Src, Vst) -> - case get_move_term_type(Src, Vst) of + case get_movable_term_type(Src, Vst) of #ms{} -> error({match_context,Src}); Type -> Type end. -%% get_move_term_type(Src, ValidatorState) -> Type +%% get_movable_term_type(Src, ValidatorState) -> Type %% Get the type of the source Src. The returned type Type will be %% a standard Erlang type (no catch/try tags). Match contexts are OK. -get_move_term_type(Src, Vst) -> +get_movable_term_type(Src, Vst) -> case get_raw_type(Src, Vst) of initialized -> error({unassigned,Src}); uninitialized -> error({uninitialized_reg,Src}); {catchtag,_} -> error({catchtag,Src}); {trytag,_} -> error({trytag,Src}); tuple_in_progress -> error({tuple_in_progress,Src}); + {literal,_}=Lit -> get_literal_type(Lit); Type -> Type end. @@ -2145,7 +2245,8 @@ get_tag_type(Src, _) -> error({invalid_tag_register,Src}). %% get_raw_type(Src, ValidatorState) -> Type -%% Return the type of a register without doing any validity checks. +%% Return the type of a register without doing any validity checks or +%% conversions. get_raw_type({x,X}=Src, #vst{current=#st{xs=Xs}}=Vst) when is_integer(X) -> check_limit(Src), case Xs of @@ -2192,10 +2293,53 @@ glt_1(T) when is_tuple(T) -> glt_1(L) -> {literal, L}. +%%% +%%% Branch tracking +%%% + +%% Forks the execution flow, with the provided funs returning the new state of +%% their respective branch; the "fail" fun returns the state where the branch +%% is taken, and the "success" fun returns the state where it's not. +%% +%% If either path is known not to be taken at runtime (eg. due to a type +%% conflict), it will simply be discarded. +-spec branch(Lbl :: label(), + Original :: #vst{}, + FailFun :: BranchFun, + SuccFun :: BranchFun) -> #vst{} when + BranchFun :: fun((#vst{}) -> #vst{}). +branch(Lbl, Vst0, FailFun, SuccFun) -> + #vst{current=St0} = Vst0, + try FailFun(Vst0) of + Vst1 -> + Vst2 = branch_state(Lbl, Vst1), + Vst = Vst2#vst{current=St0}, + try SuccFun(Vst) of + V -> V + catch + {type_conflict, _, _} -> + %% The instruction is guaranteed to fail; kill the state. + kill_state(Vst) + end + catch + {type_conflict, _, _} -> + %% This instruction is guaranteed not to fail, so we run the + %% success branch *without* catching type conflicts to avoid hiding + %% errors in the validator itself; one of the branches must + %% succeed. + SuccFun(Vst0) + end. + +%% A shorthand version of branch/4 for when the state is only altered on +%% success. +branch(Fail, Vst, SuccFun) -> + branch(Fail, Vst, fun(V) -> V end, SuccFun). + +%% Directly branches off the state. This is an "internal" operation that should +%% be used sparingly. branch_state(0, #vst{}=Vst) -> - %% If the instruction fails, the stack may be scanned - %% looking for a catch tag. Therefore the Y registers - %% must be initialized at this point. + %% If the instruction fails, the stack may be scanned looking for a catch + %% tag. Therefore the Y registers must be initialized at this point. verify_y_init(Vst), Vst; branch_state(L, #vst{current=St,branched=B,ref_ctr=Counter0}=Vst) -> diff --git a/lib/compiler/test/beam_except_SUITE.erl b/lib/compiler/test/beam_except_SUITE.erl index 9380fe06c8..8e3b373d29 100644 --- a/lib/compiler/test/beam_except_SUITE.erl +++ b/lib/compiler/test/beam_except_SUITE.erl @@ -84,9 +84,16 @@ coverage(_) -> {'EXIT',{function_clause, [{?MODULE,fc,[y],[File,{line,2}]}|_]}} = (catch fc(y)), - {'EXIT',{function_clause, - [{?MODULE,fc,[[a,b,c]],[File,{line,6}]}|_]}} = - (catch fc([a,b,c])), + case ?MODULE of + beam_except_no_opt_SUITE -> + %% There will be a different stack fram in + %% unoptimized code. + ok; + _ -> + {'EXIT',{function_clause, + [{?MODULE,fc,[[a,b,c]],[File,{line,6}]}|_]}} = + (catch fc([a,b,c])) + end, {'EXIT',{undef,[{erlang,error,[a,b,c],_}|_]}} = (catch erlang:error(a, b, c)), diff --git a/lib/compiler/test/beam_validator_SUITE.erl b/lib/compiler/test/beam_validator_SUITE.erl index 265da43f9d..8b39fce479 100644 --- a/lib/compiler/test/beam_validator_SUITE.erl +++ b/lib/compiler/test/beam_validator_SUITE.erl @@ -34,7 +34,7 @@ undef_label/1,illegal_instruction/1,failing_gc_guard_bif/1, map_field_lists/1,cover_bin_opt/1, val_dsetel/1,bad_tuples/1,bad_try_catch_nesting/1, - receive_stacked/1,aliased_types/1]). + receive_stacked/1,aliased_types/1,type_conflict/1]). -include_lib("common_test/include/ct.hrl"). @@ -63,7 +63,7 @@ groups() -> undef_label,illegal_instruction,failing_gc_guard_bif, map_field_lists,cover_bin_opt,val_dsetel, bad_tuples,bad_try_catch_nesting, - receive_stacked,aliased_types]}]. + receive_stacked,aliased_types,type_conflict]}]. init_per_suite(Config) -> test_lib:recompile(?MODULE), @@ -156,8 +156,8 @@ call_last(Config) when is_list(Config) -> merge_undefined(Config) when is_list(Config) -> Errors = do_val(merge_undefined, Config), [{{t,handle_call,2}, - {{call_ext,1,{extfunc,erlang,exit,1}}, - 10, + {{call_ext,2,{extfunc,debug,filter,2}}, + 22, {uninitialized_reg,{y,_}}}}] = Errors, ok. @@ -630,6 +630,27 @@ aliased_types_3(Bug) -> hd(List) end. + +%% ERL-867; validation proceeded after a type conflict, causing incorrect types +%% to be joined. + +-record(r, { e1 = e1, e2 = e2 }). + +type_conflict(Config) when is_list(Config) -> + {e1, e2} = type_conflict_1(#r{}), + ok. + +type_conflict_1(C) -> + Src = id(C#r.e2), + TRes = try id(Src) of + R -> R + catch + %% C:R can never match, yet it assumed that the type of 'C' was + %% an atom from here on. + C:R -> R + end, + {C#r.e1, TRes}. + %%%------------------------------------------------------------------------- transform_remove(Remove, Module) -> diff --git a/lib/compiler/test/beam_validator_SUITE_data/merge_undefined.S b/lib/compiler/test/beam_validator_SUITE_data/merge_undefined.S index 481d55045d..aa344807e4 100644 --- a/lib/compiler/test/beam_validator_SUITE_data/merge_undefined.S +++ b/lib/compiler/test/beam_validator_SUITE_data/merge_undefined.S @@ -15,8 +15,9 @@ {select_val,{x,0},{f,1},{list,[{atom,gurka},{f,3},{atom,delete},{f,4}]}}. {label,3}. {allocate_heap,2,6,2}. - %% The Y registers are not initialized here. {test,is_eq_exact,{f,5},[{x,0},{atom,ok}]}. + %% This is unreachable since {x,0} is known not to be 'ok'. We should not + %% fail with "uninitialized y registers" on erlang:exit/1 {move,{atom,nisse},{x,0}}. {call_ext,1,{extfunc,erlang,exit,1}}. {label,4}. @@ -29,6 +30,7 @@ {call_ext,2,{extfunc,io,format,2}}. {test,is_ne_exact,{f,6},[{x,0},{atom,ok}]}. {label,5}. + %% The Y registers are not initialized here. {move,{atom,logReader},{x,1}}. {move,{atom,console},{x,0}}. {call_ext,2,{extfunc,debug,filter,2}}. diff --git a/lib/compiler/test/beam_validator_SUITE_data/receive_stacked.S b/lib/compiler/test/beam_validator_SUITE_data/receive_stacked.S index 5b974119c6..a878204d16 100644 --- a/lib/compiler/test/beam_validator_SUITE_data/receive_stacked.S +++ b/lib/compiler/test/beam_validator_SUITE_data/receive_stacked.S @@ -240,7 +240,7 @@ {y,0}}. {'%',{no_bin_opt,{binary_used_in,{test,is_binary,{f,34},[{y,0}]}}, [63,{file,"receive_stacked.erl"}]}}. - {test,is_binary,{f,34},[{y,0}]}. + {test,is_eq_exact,{f,34},[{y,0},{literal,<<0,1,2,3>>}]}. remove_message. {move,{integer,42},{x,0}}. {line,[{location,"receive_stacked.erl",64}]}. @@ -283,7 +283,7 @@ {y,0}}. {'%',{no_bin_opt,{[{x,1},{y,0}],{loop_rec_end,{f,38}},not_handled}, [70,{file,"receive_stacked.erl"}]}}. - {test,is_binary,{f,39},[{x,0}]}. + {test,is_eq_exact,{f,39},[{x,0},{literal,<<0,1,2,3>>}]}. remove_message. {move,{integer,42},{x,0}}. {line,[{location,"receive_stacked.erl",71}]}. diff --git a/lib/compiler/test/float_SUITE.erl b/lib/compiler/test/float_SUITE.erl index 831e8279aa..0fa8070dc8 100644 --- a/lib/compiler/test/float_SUITE.erl +++ b/lib/compiler/test/float_SUITE.erl @@ -21,15 +21,16 @@ -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, init_per_group/2,end_per_group/2, pending/1,bif_calls/1,math_functions/1,mixed_float_and_int/1, - subtract_number_type/1]). + subtract_number_type/1,float_followed_by_guard/1]). -include_lib("common_test/include/ct.hrl"). suite() -> [{ct_hooks,[ts_install_cth]}]. -all() -> +all() -> [pending, bif_calls, math_functions, - mixed_float_and_int, subtract_number_type]. + mixed_float_and_int, subtract_number_type, + float_followed_by_guard]. groups() -> []. @@ -187,5 +188,21 @@ fact(0, P) -> P; fact(1, P) -> P; fact(N, P) -> fact(N-1, P*N). +float_followed_by_guard(Config) when is_list(Config) -> + true = ffbg_1(5, 1), + false = ffbg_1(1, 5), + ok. + +ffbg_1(A, B0) -> + %% This is a non-guard block followed by a *guard block* that starts with a + %% floating point operation, and the compiler erroneously assumed that it + %% was safe to skip fcheckerror because the next block started with a float + %% op. + B = id(B0) / 1.0, + if + A - B > 0.0 -> true; + A - B =< 0.0 -> false + end. + id(I) -> I. diff --git a/lib/compiler/vsn.mk b/lib/compiler/vsn.mk index efedb414ad..a523627384 100644 --- a/lib/compiler/vsn.mk +++ b/lib/compiler/vsn.mk @@ -1 +1 @@ -COMPILER_VSN = 7.3.1 +COMPILER_VSN = 7.3.2 diff --git a/lib/crypto/c_src/atoms.c b/lib/crypto/c_src/atoms.c index 2e417da7f4..798c26c9bb 100644 --- a/lib/crypto/c_src/atoms.c +++ b/lib/crypto/c_src/atoms.c @@ -41,6 +41,18 @@ ERL_NIF_TERM atom_not_enabled; ERL_NIF_TERM atom_not_supported; #endif +ERL_NIF_TERM atom_type; +ERL_NIF_TERM atom_size; +ERL_NIF_TERM atom_block_size; +ERL_NIF_TERM atom_key_length; +ERL_NIF_TERM atom_iv_length; +ERL_NIF_TERM atom_mode; +ERL_NIF_TERM atom_ecb_mode; +ERL_NIF_TERM atom_cbc_mode; +ERL_NIF_TERM atom_cfb_mode; +ERL_NIF_TERM atom_ofb_mode; +ERL_NIF_TERM atom_stream_cipher; + #if defined(HAVE_EC) ERL_NIF_TERM atom_prime_field; ERL_NIF_TERM atom_characteristic_two_field; @@ -140,6 +152,18 @@ int init_atoms(ErlNifEnv *env, const ERL_NIF_TERM fips_mode, const ERL_NIF_TERM atom_notsup = enif_make_atom(env,"notsup"); atom_digest = enif_make_atom(env,"digest"); + atom_type = enif_make_atom(env,"type"); + atom_size = enif_make_atom(env,"size"); + atom_block_size = enif_make_atom(env,"block_size"); + atom_key_length = enif_make_atom(env,"key_length"); + atom_iv_length = enif_make_atom(env,"iv_length"); + atom_mode = enif_make_atom(env,"mode"); + atom_ecb_mode = enif_make_atom(env,"ecb_mode"); + atom_cbc_mode = enif_make_atom(env,"cbc_mode"); + atom_cfb_mode = enif_make_atom(env,"cfb_mode"); + atom_ofb_mode = enif_make_atom(env,"ofb_mode"); + atom_stream_cipher = enif_make_atom(env,"stream_cipher"); + #if defined(HAVE_EC) atom_prime_field = enif_make_atom(env,"prime_field"); atom_characteristic_two_field = enif_make_atom(env,"characteristic_two_field"); diff --git a/lib/crypto/c_src/atoms.h b/lib/crypto/c_src/atoms.h index f15523d865..f8e9211459 100644 --- a/lib/crypto/c_src/atoms.h +++ b/lib/crypto/c_src/atoms.h @@ -45,6 +45,18 @@ extern ERL_NIF_TERM atom_not_enabled; extern ERL_NIF_TERM atom_not_supported; #endif +extern ERL_NIF_TERM atom_type; +extern ERL_NIF_TERM atom_size; +extern ERL_NIF_TERM atom_block_size; +extern ERL_NIF_TERM atom_key_length; +extern ERL_NIF_TERM atom_iv_length; +extern ERL_NIF_TERM atom_mode; +extern ERL_NIF_TERM atom_ecb_mode; +extern ERL_NIF_TERM atom_cbc_mode; +extern ERL_NIF_TERM atom_cfb_mode; +extern ERL_NIF_TERM atom_ofb_mode; +extern ERL_NIF_TERM atom_stream_cipher; + #if defined(HAVE_EC) extern ERL_NIF_TERM atom_prime_field; extern ERL_NIF_TERM atom_characteristic_two_field; diff --git a/lib/crypto/c_src/cipher.c b/lib/crypto/c_src/cipher.c index f8e44b228a..c055a62654 100644 --- a/lib/crypto/c_src/cipher.c +++ b/lib/crypto/c_src/cipher.c @@ -67,14 +67,26 @@ static struct cipher_type_t cipher_types[] = {{"aes_cfb8"}, {&EVP_aes_192_cfb8}, 24, NO_FIPS_CIPHER | AES_CFBx}, {{"aes_cfb8"}, {&EVP_aes_256_cfb8}, 32, NO_FIPS_CIPHER | AES_CFBx}, + {{"aes_128_cfb8"}, {&EVP_aes_128_cfb8}, 16, NO_FIPS_CIPHER | AES_CFBx}, + {{"aes_192_cfb8"}, {&EVP_aes_192_cfb8}, 24, NO_FIPS_CIPHER | AES_CFBx}, + {{"aes_256_cfb8"}, {&EVP_aes_256_cfb8}, 32, NO_FIPS_CIPHER | AES_CFBx}, + {{"aes_cfb128"}, {&EVP_aes_128_cfb128}, 16, NO_FIPS_CIPHER | AES_CFBx}, {{"aes_cfb128"}, {&EVP_aes_192_cfb128}, 24, NO_FIPS_CIPHER | AES_CFBx}, {{"aes_cfb128"}, {&EVP_aes_256_cfb128}, 32, NO_FIPS_CIPHER | AES_CFBx}, + {{"aes_128_cfb128"}, {&EVP_aes_128_cfb128}, 16, NO_FIPS_CIPHER | AES_CFBx}, + {{"aes_192_cfb128"}, {&EVP_aes_192_cfb128}, 24, NO_FIPS_CIPHER | AES_CFBx}, + {{"aes_256_cfb128"}, {&EVP_aes_256_cfb128}, 32, NO_FIPS_CIPHER | AES_CFBx}, + {{"aes_ecb"}, {&EVP_aes_128_ecb}, 16, ECB_BUG_0_9_8L}, {{"aes_ecb"}, {&EVP_aes_192_ecb}, 24, ECB_BUG_0_9_8L}, {{"aes_ecb"}, {&EVP_aes_256_ecb}, 32, ECB_BUG_0_9_8L}, + {{"aes_128_ecb"}, {&EVP_aes_128_ecb}, 16, ECB_BUG_0_9_8L}, + {{"aes_192_ecb"}, {&EVP_aes_192_ecb}, 24, ECB_BUG_0_9_8L}, + {{"aes_256_ecb"}, {&EVP_aes_256_ecb}, 32, ECB_BUG_0_9_8L}, + #if defined(HAVE_EVP_AES_CTR) {{"aes_128_ctr"}, {&EVP_aes_128_ctr}, 16, 0}, {{"aes_192_ctr"}, {&EVP_aes_192_ctr}, 24, 0}, @@ -207,6 +219,87 @@ int cmp_cipher_types(const void *keyp, const void *elemp) { } +ERL_NIF_TERM cipher_info_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (Type) */ + const struct cipher_type_t *cipherp; + const EVP_CIPHER *cipher; + ERL_NIF_TERM ret, ret_mode; + unsigned type; + unsigned long mode; + + if ((cipherp = get_cipher_type_no_key(argv[0])) == NULL) + return enif_make_badarg(env); + + if (FORBIDDEN_IN_FIPS(cipherp)) + return enif_raise_exception(env, atom_notsup); + if ((cipher = cipherp->cipher.p) == NULL) + return enif_raise_exception(env, atom_notsup); + + ret = enif_make_new_map(env); + + type = EVP_CIPHER_type(cipher); + enif_make_map_put(env, ret, atom_type, + type == NID_undef ? atom_undefined : enif_make_int(env, type), + &ret); + + enif_make_map_put(env, ret, atom_key_length, + enif_make_int(env, EVP_CIPHER_key_length(cipher)), &ret); + enif_make_map_put(env, ret, atom_iv_length, + enif_make_int(env, EVP_CIPHER_iv_length(cipher)), &ret); + enif_make_map_put(env, ret, atom_block_size, + enif_make_int(env, EVP_CIPHER_block_size(cipher)), &ret); + + mode = EVP_CIPHER_mode(cipher); + switch (mode) { + case EVP_CIPH_ECB_MODE: + ret_mode = atom_ecb_mode; + break; + + case EVP_CIPH_CBC_MODE: + ret_mode = atom_cbc_mode; + break; + + case EVP_CIPH_CFB_MODE: + ret_mode = atom_cfb_mode; + break; + + case EVP_CIPH_OFB_MODE: + ret_mode = atom_ofb_mode; + break; + + case EVP_CIPH_STREAM_CIPHER: + ret_mode = atom_stream_cipher; + break; + + default: + ret_mode = atom_undefined; + break; + } + + enif_make_map_put(env, ret, atom_mode, ret_mode, &ret); + + return ret; +} + +const struct cipher_type_t* get_cipher_type_no_key(ERL_NIF_TERM type) +{ + struct cipher_type_t key; + + key.type.atom = type; + + return bsearch(&key, cipher_types, num_cipher_types, sizeof(cipher_types[0]), cmp_cipher_types_no_key); +} + +int cmp_cipher_types_no_key(const void *keyp, const void *elemp) { + const struct cipher_type_t *key = keyp; + const struct cipher_type_t *elem = elemp; + + if (key->type.atom < elem->type.atom) return -1; + else if (key->type.atom > elem->type.atom) return 1; + else /* key->type.atom == elem->type.atom */ return 0; +} + + ERL_NIF_TERM cipher_types_as_list(ErlNifEnv* env) { struct cipher_type_t* p; diff --git a/lib/crypto/c_src/cipher.h b/lib/crypto/c_src/cipher.h index 6b43afea99..b0d9d324e1 100644 --- a/lib/crypto/c_src/cipher.h +++ b/lib/crypto/c_src/cipher.h @@ -61,12 +61,16 @@ struct evp_cipher_ctx { EVP_CIPHER_CTX* ctx; }; +ERL_NIF_TERM cipher_info_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); + int init_cipher_ctx(ErlNifEnv *env); void init_cipher_types(ErlNifEnv* env); +const struct cipher_type_t* get_cipher_type_no_key(ERL_NIF_TERM type); const struct cipher_type_t* get_cipher_type(ERL_NIF_TERM type, size_t key_len); int cmp_cipher_types(const void *keyp, const void *elemp); +int cmp_cipher_types_no_key(const void *keyp, const void *elemp); ERL_NIF_TERM cipher_types_as_list(ErlNifEnv* env); diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c index 06439c34b2..261590d9a5 100644 --- a/lib/crypto/c_src/crypto.c +++ b/lib/crypto/c_src/crypto.c @@ -67,6 +67,7 @@ static ErlNifFunc nif_funcs[] = { {"info_fips", 0, info_fips, 0}, {"enable_fips_mode", 1, enable_fips_mode, 0}, {"algorithms", 0, algorithms, 0}, + {"hash_info", 1, hash_info_nif, 0}, {"hash_nif", 2, hash_nif, 0}, {"hash_init_nif", 1, hash_init_nif, 0}, {"hash_update_nif", 2, hash_update_nif, 0}, @@ -78,6 +79,7 @@ static ErlNifFunc nif_funcs[] = { {"hmac_final_nif", 1, hmac_final_nif, 0}, {"hmac_final_nif", 2, hmac_final_nif, 0}, {"cmac_nif", 3, cmac_nif, 0}, + {"cipher_info_nif", 1, cipher_info_nif, 0}, {"block_crypt_nif", 5, block_crypt_nif, 0}, {"block_crypt_nif", 4, block_crypt_nif, 0}, {"aes_ige_crypt_nif", 4, aes_ige_crypt_nif, 0}, diff --git a/lib/crypto/c_src/hash.c b/lib/crypto/c_src/hash.c index 457e9d071a..0a9f64acef 100644 --- a/lib/crypto/c_src/hash.c +++ b/lib/crypto/c_src/hash.c @@ -61,6 +61,32 @@ int init_hash_ctx(ErlNifEnv* env) { #endif } +ERL_NIF_TERM hash_info_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (Type) */ + struct digest_type_t *digp = NULL; + const EVP_MD *md; + ERL_NIF_TERM ret; + + ASSERT(argc == 1); + + if ((digp = get_digest_type(argv[0])) == NULL) + return enif_make_badarg(env); + + if ((md = digp->md.p) == NULL) + return atom_notsup; + + ret = enif_make_new_map(env); + + enif_make_map_put(env, ret, atom_type, + enif_make_int(env, EVP_MD_type(md)), &ret); + enif_make_map_put(env, ret, atom_size, + enif_make_int(env, EVP_MD_size(md)), &ret); + enif_make_map_put(env, ret, atom_block_size, + enif_make_int(env, EVP_MD_block_size(md)), &ret); + + return ret; +} + ERL_NIF_TERM hash_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (Type, Data) */ struct digest_type_t *digp = NULL; diff --git a/lib/crypto/c_src/hash.h b/lib/crypto/c_src/hash.h index 8bae07f39a..92a25cedb7 100644 --- a/lib/crypto/c_src/hash.h +++ b/lib/crypto/c_src/hash.h @@ -25,6 +25,7 @@ int init_hash_ctx(ErlNifEnv *env); +ERL_NIF_TERM hash_info_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); ERL_NIF_TERM hash_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); ERL_NIF_TERM hash_init_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); ERL_NIF_TERM hash_update_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); diff --git a/lib/crypto/c_src/openssl_config.h b/lib/crypto/c_src/openssl_config.h index 45144a0c25..1c138e3bd1 100644 --- a/lib/crypto/c_src/openssl_config.h +++ b/lib/crypto/c_src/openssl_config.h @@ -203,12 +203,17 @@ #if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION_PLAIN(1,1,0) # ifndef HAS_LIBRESSL -# define HAVE_CHACHA20 # define HAVE_CHACHA20_POLY1305 # define HAVE_RSA_OAEP_MD # endif #endif +#if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION(1,1,0,'d') +# ifndef HAS_LIBRESSL +# define HAVE_CHACHA20 +# endif +#endif + // OPENSSL_VERSION_NUMBER >= 1.1.1-pre8 #if OPENSSL_VERSION_NUMBER >= (PACKED_OPENSSL_VERSION_PLAIN(1,1,1)-7) # ifndef HAS_LIBRESSL diff --git a/lib/crypto/c_src/poly1305.c b/lib/crypto/c_src/poly1305.c index db3433dce3..76579c0a29 100644 --- a/lib/crypto/c_src/poly1305.c +++ b/lib/crypto/c_src/poly1305.c @@ -85,6 +85,6 @@ ERL_NIF_TERM poly1305_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) return ret; #else - return atom_notsup; + return enif_raise_exception(env, atom_notsup); #endif } diff --git a/lib/crypto/doc/src/notes.xml b/lib/crypto/doc/src/notes.xml index 0a3f68ade2..c0b302734e 100644 --- a/lib/crypto/doc/src/notes.xml +++ b/lib/crypto/doc/src/notes.xml @@ -31,6 +31,38 @@ </header> <p>This document describes the changes made to the Crypto application.</p> +<section><title>Crypto 4.4.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixes a bug that caused <c>crypto:sign</c> and + <c>crypto:verify</c> to return the error message + <c>badarg</c> instead of <c>notsup</c> in one case. That + case was when signing or verifying with eddsa keys (that + is, ed15519 or ed448), but only when FIPS was supported + and enabled.</p> + <p> + Own Id: OTP-15634</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Added a crypto benchmark test suite.</p> + <p> + Own Id: OTP-15447</p> + </item> + </list> + </section> + +</section> + <section><title>Crypto 4.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/crypto/src/crypto.erl b/lib/crypto/src/crypto.erl index fe8390c5b8..97a4a7a3f0 100644 --- a/lib/crypto/src/crypto.erl +++ b/lib/crypto/src/crypto.erl @@ -24,6 +24,7 @@ -export([start/0, stop/0, info_lib/0, info_fips/0, supports/0, enable_fips_mode/1, version/0, bytes_to_integer/1]). +-export([cipher_info/1, hash_info/1]). -export([hash/2, hash_init/1, hash_update/2, hash_final/1]). -export([sign/4, sign/5, verify/5, verify/6]). -export([generate_key/2, generate_key/3, compute_key/4]). @@ -403,6 +404,11 @@ enable_fips_mode(_) -> ?nif_stub. -define(HASH_HASH_ALGORITHM, sha1() | sha2() | sha3() | blake2() | ripemd160 | compatibility_only_hash() ). +-spec hash_info(Type) -> map() when Type :: ?HASH_HASH_ALGORITHM. + +hash_info(Type) -> + notsup_to_error(hash_info_nif(Type)). + -spec hash(Type, Data) -> Digest when Type :: ?HASH_HASH_ALGORITHM, Data :: iodata(), Digest :: binary(). @@ -531,6 +537,12 @@ poly1305(Key, Data) -> %%% %%%================================================================ +-spec cipher_info(Type) -> map() when Type :: block_cipher_with_iv() + | aead_cipher() + | block_cipher_without_iv(). +cipher_info(Type) -> + cipher_info_nif(Type). + %%%---- Block ciphers %%%---------------------------------------------------------------- @@ -1726,6 +1738,7 @@ hash_update(State0, Data, _, MaxBytes) -> State = notsup_to_error(hash_update_nif(State0, Increment)), hash_update(State, Rest, erlang:byte_size(Rest), MaxBytes). +hash_info_nif(_Hash) -> ?nif_stub. hash_nif(_Hash, _Data) -> ?nif_stub. hash_init_nif(_Hash) -> ?nif_stub. hash_update_nif(_State, _Data) -> ?nif_stub. @@ -1770,6 +1783,8 @@ poly1305_nif(_Key, _Data) -> ?nif_stub. %% CIPHERS -------------------------------------------------------------------- +cipher_info_nif(_Type) -> ?nif_stub. + block_crypt_nif(_Type, _Key, _Ivec, _Text, _IsEncrypt) -> ?nif_stub. block_crypt_nif(_Type, _Key, _Text, _IsEncrypt) -> ?nif_stub. diff --git a/lib/crypto/test/crypto_SUITE.erl b/lib/crypto/test/crypto_SUITE.erl index ab6d88deb2..7257f4fb9f 100644 --- a/lib/crypto/test/crypto_SUITE.erl +++ b/lib/crypto/test/crypto_SUITE.erl @@ -40,7 +40,9 @@ all() -> rand_uniform, rand_threads, rand_plugin, - rand_plugin_s + rand_plugin_s, + cipher_info, + hash_info ]. groups() -> @@ -666,6 +668,23 @@ rand_plugin_s(Config) when is_list(Config) -> rand_plugin_aux(explicit_state). %%-------------------------------------------------------------------- +cipher_info() -> + [{doc, "crypto cipher_info testing"}]. +cipher_info(Config) when is_list(Config) -> + #{type := _,key_length := _,iv_length := _, + block_size := _,mode := _} = crypto:cipher_info(aes_128_cbc), + {'EXIT',_} = (catch crypto:cipher_info(not_a_cipher)), + ok. + +%%-------------------------------------------------------------------- +hash_info() -> + [{doc, "crypto hash_info testing"}]. +hash_info(Config) when is_list(Config) -> + #{type := _,size := _,block_size := _} = crypto:hash_info(sha256), + {'EXIT',_} = (catch crypto:hash_info(not_a_hash)), + ok. + +%%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- hash(_, [], []) -> diff --git a/lib/crypto/vsn.mk b/lib/crypto/vsn.mk index 6a91244715..deba17fb66 100644 --- a/lib/crypto/vsn.mk +++ b/lib/crypto/vsn.mk @@ -1 +1 @@ -CRYPTO_VSN = 4.4 +CRYPTO_VSN = 4.4.1 diff --git a/lib/debugger/doc/src/Makefile b/lib/debugger/doc/src/Makefile index 56d6085e9c..49b5a4be57 100644 --- a/lib/debugger/doc/src/Makefile +++ b/lib/debugger/doc/src/Makefile @@ -114,8 +114,7 @@ include $(ERL_TOP)/make/otp_release_targets.mk release_docs_spec: docs $(INSTALL_DIR) "$(RELSYSDIR)/doc/pdf" $(INSTALL_DATA) $(TOP_PDF_FILE) "$(RELSYSDIR)/doc/pdf" - $(INSTALL_DIR) "$(RELSYSDIR)/doc/html" - ($(CP) -rf $(HTMLDIR) "$(RELSYSDIR)/doc") + $(INSTALL_DIR_DATA) $(HTMLDIR) "$(RELSYSDIR)/doc/html" $(INSTALL_DATA) $(INFO_FILE) "$(RELSYSDIR)" $(INSTALL_DIR) "$(RELEASE_PATH)/man/man3" $(INSTALL_DATA) $(MAN3DIR)/* "$(RELEASE_PATH)/man/man3" diff --git a/lib/dialyzer/doc/src/notes.xml b/lib/dialyzer/doc/src/notes.xml index 3cf776e566..bc422c43a0 100644 --- a/lib/dialyzer/doc/src/notes.xml +++ b/lib/dialyzer/doc/src/notes.xml @@ -32,6 +32,36 @@ <p>This document describes the changes made to the Dialyzer application.</p> +<section><title>Dialyzer 3.3.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug that caused Dialyzer to crash when + analyzing a contract with a module name differing from + the analyzed module's name. The bug was introduced in + Erlang/OTP 18. </p> + <p> + Own Id: OTP-15562 Aux Id: ERL-845 </p> + </item> + <item> + <p> Fix a bug in the handling of the <c>Key</c> argument + of <c>lists:{keysearch, keyfind, keymember}</c>. </p> + <p> + Own Id: OTP-15570</p> + </item> + <item> + <p>Optimize (again) Dialyzer's handling of + left-associative use of <c>andalso</c> and <c>orelse</c> + in guards.</p> + <p> + Own Id: OTP-15577 Aux Id: ERL-851, PR-2141, PR-1944 </p> + </item> + </list> + </section> + +</section> + <section><title>Dialyzer 3.3.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/dialyzer/vsn.mk b/lib/dialyzer/vsn.mk index 98ab533a58..7221993963 100644 --- a/lib/dialyzer/vsn.mk +++ b/lib/dialyzer/vsn.mk @@ -1 +1 @@ -DIALYZER_VSN = 3.3.1 +DIALYZER_VSN = 3.3.2 diff --git a/lib/diameter/doc/src/diameter.xml b/lib/diameter/doc/src/diameter.xml index 0a0194af2d..85522c99b2 100644 --- a/lib/diameter/doc/src/diameter.xml +++ b/lib/diameter/doc/src/diameter.xml @@ -1,7 +1,9 @@ <?xml version="1.0" encoding="utf-8" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ - <!ENTITY spawn_opt + <!ENTITY spawn_opt2 '<seealso marker="erts:erlang#spawn_opt-2">erlang:spawn_opt/2</seealso>'> + <!ENTITY spawn_opt5 + '<seealso marker="erts:erlang#spawn_opt-5">erlang:spawn_opt/5</seealso>'> <!ENTITY nodes '<seealso marker="erts:erlang#nodes-0">erlang:nodes/0</seealso>'> <!ENTITY make_ref @@ -21,7 +23,7 @@ <copyright> <year>2011</year> -<year>2017</year> +<year>2019</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -1384,12 +1386,22 @@ the same peer.</p> </item> <tag> -<marker id="spawn_opt"/><c>{spawn_opt, [term()]}</c></tag> +<marker id="spawn_opt"/><c>{spawn_opt, [term()] | {M,F,A}}</c></tag> <item> <p> -Options passed to &spawn_opt; when spawning a process for an -incoming Diameter request. -Options <c>monitor</c> and <c>link</c> are ignored.</p> +An options list passed to &spawn_opt2; to spawn a handler process for an +incoming Diameter request on the local node, or an MFA that returns +the pid of a handler process.</p> + +<p> +Options <c>monitor</c> and <c>link</c> are ignored in the list-valued +case. +An MFA is applied with an additional term prepended to its argument +list, and should return either the pid of the handler process that +invokes <c>diameter_traffic:request/1</c> on the term in order to +process the request, or the atom <c>discard</c>. +The handler process need not be local, but diameter must be started on +the remote node.</p> <p> Defaults to the empty list.</p> diff --git a/lib/diameter/doc/src/diameter_transport.xml b/lib/diameter/doc/src/diameter_transport.xml index 67fd54bc56..0a8ef321c6 100644 --- a/lib/diameter/doc/src/diameter_transport.xml +++ b/lib/diameter/doc/src/diameter_transport.xml @@ -14,7 +14,8 @@ <erlref> <header> <copyright> -<year>2011</year><year>2016</year> +<year>2011</year> +<year>2019</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -174,10 +175,13 @@ its parent.</p> <taglist> -<tag><c>{diameter, {send, &message;}}</c></tag> +<tag><c>{diameter, {send, &message; | false}}</c></tag> <item> <p> -An outbound Diameter message.</p> +An outbound Diameter message. +The atom <c>false</c> can only be received when request +acknowledgements have been requests: see the <c>ack</c> message +below.</p> </item> <tag><c>{diameter, {close, Pid}}</c></tag> @@ -246,6 +250,27 @@ A <c>LocalAddr</c> list has the same semantics as one returned from &start;.</p> </item> +<tag><c>{diameter, ack}</c></tag> +<item> +<p> +Request acknowledgements of unanswered requests. +A transport process should send this once before passing incoming +Diameter messages into diameter. +As a result, every Diameter request passed into diameter with a +<c>recv</c> message (below) will be answered with a +<c>send</c> message (above), either a &message; for the transport +process to send or the atom <c>false</c> if the request has been +discarded or otherwise not answered.</p> + +<p> +This is to allow a transport process to keep count of the number +of incoming request messages that have not yet been answered or +discarded, to allow it to regulate the amount of incoming traffic. +Both diameter_tcp and diameter_sctp request acknowledgements when a +<c>message_cb</c> is configured, turning send/recv message into +callbacks that can be used to regulate traffic.</p> +</item> + <tag><c>{diameter, {recv, &message;}}</c></tag> <item> <p> diff --git a/lib/diameter/doc/src/notes.xml b/lib/diameter/doc/src/notes.xml index cc92bd99f0..5777225ae7 100644 --- a/lib/diameter/doc/src/notes.xml +++ b/lib/diameter/doc/src/notes.xml @@ -43,6 +43,41 @@ first.</p> <!-- ===================================================================== --> +<section><title>diameter 2.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix failure of incoming answer message with faulty + Experimental-Result-Code. Failure to decode the AVP + resulted in an uncaught exception, with no no + handle_answer/error callback as a consequence.</p> + <p> + Own Id: OTP-15569 Aux Id: ERIERL-302 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add spawn_opt MFA configuration to allow a callback to + spawn a handler process for an incoming Diameter request + on an an arbitrary node. Module diameter_dist provides a + route_session/2 that can be used to distribute requests + based on Session-Id, although this module is currently + only documented in the module itself and may change.</p> + <p> + Own Id: OTP-15398</p> + </item> + </list> + </section> + +</section> + <section><title>diameter 2.1.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/diameter/src/base/diameter.erl b/lib/diameter/src/base/diameter.erl index b90b794611..7f172e1fa1 100644 --- a/lib/diameter/src/base/diameter.erl +++ b/lib/diameter/src/base/diameter.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -365,7 +365,7 @@ call(SvcName, App, Message) -> | {connect_timer, 'Unsigned32'()} | {watchdog_timer, 'Unsigned32'() | {module(), atom(), list()}} | {watchdog_config, [{okay|suspect, non_neg_integer()}]} - | {spawn_opt, list()}. + | {spawn_opt, list() | mfa()}. %% Options passed to start_service/2 diff --git a/lib/diameter/src/base/diameter_callback.erl b/lib/diameter/src/base/diameter_callback.erl index d04a416bef..3bcf550cd8 100644 --- a/lib/diameter/src/base/diameter_callback.erl +++ b/lib/diameter/src/base/diameter_callback.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -70,7 +70,7 @@ -module(diameter_callback). -%% Default callbacks when no aleternate is specified. +%% Default callbacks when no alternate is specified. -export([peer_up/3, peer_down/3, pick_peer/4, diff --git a/lib/diameter/src/base/diameter_dist.erl b/lib/diameter/src/base/diameter_dist.erl new file mode 100644 index 0000000000..5c29ea95a4 --- /dev/null +++ b/lib/diameter/src/base/diameter_dist.erl @@ -0,0 +1,525 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2019. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +-module(diameter_dist). + +-behaviour(gen_server). + +%% +%% Implements callbacks that can be configured as a spawn_opt +%% transport configuration, to be able to distribute incoming Diameter +%% requests to handler processes (local or remote) in various ways. +%% + +%% spawn_opt callbacks +-export([spawn_local/2, + spawn_local/1, + route_session/2, + route_session/1]). + +%% signal availability for handling incoming requests to route_sesssion/2 +-export([attach/1, + detach/1]). + +%% consistent hashing +-export([hash/3, %% for use as default MFA in route_session/2 options map + hash/2]). %% arbitrary key/values + +-include_lib("diameter/include/diameter.hrl"). + +%% server start +-export([start_link/0]). + +%% gen_server callbacks +-export([init/1, + handle_info/2, + handle_cast/2, + handle_call/3, + code_change/3, + terminate/2]). + +-type request() :: tuple(). %% callback argument from diameter_traffic + +-define(SERVER, ?MODULE). %% server monitoring node connections + +%% Maps a node name binary to the corresponding atom. Used by +%% route_session/2 to map the optional value of a Session-Id to +%% node(). +-define(NODE_TABLE, diameter_dist_node). + +%% Maps a diameter:service_name() to a node() that has called attach/1 +%% to declare its willingness to handle incoming requests for the +%% service. Use by route_session/2 in case the optional value mapping +%% has failed. +-define(SERVICE_TABLE, diameter_dist_service). + +-define(B(A), atom_to_binary(A, utf8)). +-define(ORCOND(List), list_to_tuple(['orelse', false | List])). +-define(HASH(T), erlang:phash2(T, 16#100000000)). + +%% spawn_local/2 +%% +%% Callback that is equivalent to an options list. That is, the +%% following are equivalent when passed as options to +%% diameter:add_transport/2. +%% +%% {spawn_opt, Opts} +%% {spawn_opt, {diameter_dist, spawn_local, [Opts]}} + +-spec spawn_local(ReqT :: request(), Opts :: list()) + -> pid(). + +spawn_local(ReqT, Opts) -> + spawn_opt(diameter_traffic, request, [ReqT], Opts). + +%% spawn_local/1 + +spawn_local(ReqT) -> + spawn_local(ReqT, []). + +%% route_session/2 +%% +%% Callback that maps the Session-Id of an incoming request to a +%% handler node. +%% +%% With an options list, maps an id whose optional value is the name +%% of a connected node to the same node, to handle the case that the +%% session id has been returned from diameter:session_id/1; otherwise +%% to a node that has called diameter_dist:attach/1 using the +%% consistent hashing provided by hash/3, or to the local node() if a +%% session id could not be extracted or there are no attached nodes. A +%% handler process is spawned on the selected node using +%% erlang:spawn_opt/4. +%% +%% Different behaviour can be configured by supplying an options map +%% of the following form: +%% +%% #{search => non_neg_integer(), +%% id => [binary()], +%% default => discard | local | mfa(), +%% dispatch => list() | mfa()} +%% +%% The search member limits the number of AVPs that are examined in +%% the message (from the front), to avoid searching entire message in +%% case it's known that peers follow RFC 6733's recommendation that +%% Session-Id be placed at the head of a message. The default is to +%% search the entire message. +%% +%% The id member restricts the optional value mapping to session ids +%% whose DiamterIdentity is one of those specified. Set this to the +%% list of Diameter identities advertised by the service in question +%% (typically one) to ensure that only locally generated session ids +%% are mapped; or to the empty list to disable the mapping. +%% +%% The default member determines where to handle a message whose +%% Session-Id isn't found or whose optional value isn't mapped to the +%% name of a connected node. The atom local says the local node, an +%% MFA is invoked on Session-Id | false, the name of the diameter +%% service, and the message binary, and should return either a node() +%% or false to discard the message. Defaults to {diameter_dist, hash, []}. +%% +%% The dispatch member determines how the pid() of the request handler +%% process is retrieved. An MFA is applied to a previously selected +%% node(), and the module, function, and arguments list to apply in +%% the handler process to handle the request, the MFA being supplied +%% by diameter, and returns pid() | discard. A list is equivalent to +%% {erlang, spawn_opt, []}. Defaults to []. +%% +%% This can be used with search = 0 to route on something other than +%% Session-Id, but this is probably no simpler than just implementing +%% an own spawn_opt callback. (Except with the default dispatch possibly.) +%% +%% Note that if the peer is also implemented with OTP diameter and +%% generating session ids with diameter:session_id/1 then +%% route_session/2 can map an optional value to a local node that +%% happens to have the same name as one of the peer's nodes. This +%% could lead to an uneven distribution; for example, if the peer +%% nodes are a subset of the local nodes. In practice, it's typically +%% known if it's peers or the local node originating sessions; if the +%% former then setting id = [] disables the optional value mapping, if +%% the latter then setting default = local disables the hashing. +-spec route_session(ReqT :: request(), Opts) + -> discard + | pid() + when Opts :: pos_integer() %% aka #{search => N} + | list() %% aka #{dispatch => Opts} + | #{search => non_neg_integer(), %% limit number of examined AVPs + id => [binary()], %% restrict optional value map on DiamIdent + default => local %% handle locally + | discard + | mfa(), %% return node() | false + dispatch => list() %% spawn options + | mfa()}. %% (Node, M, F, A) -> pid() | discard + +route_session(ReqT, Opts) -> + {_, Bin} = Info = diameter_traffic:request_info(ReqT), + Sid = session_id(avps(Bin), search(Opts)), + Node = default(node_of_session_id(Sid, Opts), Sid, Opts, Info), + dispatch(Node, ReqT, dispatch(Opts)). + +%% avps/1 + +avps(<<_:20/binary, Bin/binary>>) -> + Bin; + +avps(_) -> + false. + +%% dispatch/3 + +dispatch(false, _, _) -> + discard; + +dispatch(Node, ReqT, {M,F,A}) -> + apply(M, F, [Node, diameter_traffic, request, [ReqT] | A]); + +dispatch(Node, ReqT, Opts) -> + spawn_opt(Node, diameter_traffic, request, [ReqT], Opts). + +%% route_session/1 + +route_session(ReqT) -> + route_session(ReqT, []). + +%% node_of_session_id/2 +%% +%% Return the node name encoded as optional value in a Session-Id, +%% assuming the id has been created with diameter:session_id/0. Lookup +%% the node name to ensure we don't convert arbitrary binaries to +%% atom. + +node_of_session_id([Id, _, _, Bin], #{id := Ids}) -> + lists:member(Id, Ids) andalso nodemap(Bin); + +node_of_session_id([_, _, _, Bin], _) -> + nodemap(Bin); + +node_of_session_id(_, _) -> + false. + +%% nodemap/1 + +nodemap(Bin) -> + try + ets:lookup_element(?NODE_TABLE, Bin, 2) + catch + error: badarg -> false + end. + +%% session_id/2 + +session_id(_, 0) -> %% give up + false; + +%% Session-Id = Command Code 263, V-bit = 0. +session_id(<<263:32, 0:1, _:7, Len:24, _/binary>> = Bin, _) -> + case Bin of + <<Avp:Len/binary, _/binary>> -> + <<_:8/binary, Sid/binary>> = Avp, + split(Sid); + _ -> + false + end; + +%% Jump to the next AVP. This is potentially costly for a message with +%% many AVPs and no Session-Id, which an attacker is prone to send. +%% 8.8 or RFC 6733 says that Session-Id SHOULD (but not MUST) appear +%% immediately following the Diameter Header, so there is no +%% guarantee. +session_id(<<_:40, Len:24, _/binary>> = Bin, N) -> + Pad = (4 - (Len rem 4)) rem 4, + case Bin of + <<_:Len/binary, _:Pad/binary, Rest/binary>> -> + session_id(Rest, if N == infinity -> N; true -> N-1 end); + _ -> + false + end; + +session_id(_, _) -> + false. + +%% split/1 +%% +%% Split a Session-Id at no more than three semicolons: the optional +%% value (if any) follows the third. binary:split/2 does better than +%% matching character by character, especially when the pattern is +%% compiled. + +split(Bin) -> + split(3, Bin, pattern()). + +%% split/3 + +split(0, Bin, _) -> + [Bin]; + +split(N, Bin, Pattern) -> + [H|T] = binary:split(Bin, Pattern), + [H | case T of + [] -> + T; + [Rest] -> + split(N-1, Rest, Pattern) + end]. + +%% pattern/0 +%% +%% Since this is being called in a watchdog process, compile the +%% pattern once and maintain it in the process dictionary. + +pattern() -> + case get(?MODULE) of + undefined -> + CP = binary:compile_pattern(<<$;>>), %% tuple + put(?MODULE, CP), + CP; + CP -> + CP + end. + +%% dispatch/1 + +dispatch(#{} = Opts) -> + maps:get(dispatch, Opts, []); + +dispatch(Opts) + when is_list(Opts) -> + Opts; + +dispatch(_) -> + []. + +%% search/1 +%% +%% Bound number of AVPs examined when looking for Session-Id. + +search(#{search := N}) + when is_integer(N), 0 =< N -> + N; + +search(N) + when is_integer(N), 0 =< N -> + N; + +search(_) -> + infinity. + +%% default/3 +%% +%% Choose a node when Session-Id lookup has failed. + +default(false, _, #{default := discard}, _) -> + false; + +default(false, _, #{default := local}, _) -> + node(); + +default(false, Sid, #{default := {M,F,A}}, Info) -> + {ServiceName, Bin} = Info, + apply(M, F, [Sid, ServiceName, Bin | A]); %% node() | false + +default(false, Sid, _, Info) -> %% aka {?MODULE, hash, []} + {ServiceName, Bin} = Info, + hash(Sid, ServiceName, Bin); + +default(Node, _, _, _) -> + Node. + +%% =========================================================================== + +%% hash/3 +%% +%% Consistent hashing of Session-Id to an attached node, or the local +%% node if Session-Id = false or no attached nodes. + +hash(Sid, ServiceName, _) -> + case false /= Sid andalso attached(ServiceName) of + [_|_] = Nodes -> + hash(Sid, Nodes); + _ -> + node() + end. + +%% hash/2 +%% +%% Consistent hashing on arbitrary key/values. Returns false if the +%% list is empty. + +%% No key or no values. +hash(_, []) -> + false; + +%% Not much choice. +hash(_, [Value]) -> + Value; + +%% Hash on a circle and choose the closest predecessor. +hash(Key, Values) -> + Hash = ?HASH(Key), + tl(lists:foldl(fun(V,A) -> + choose(Hash, [?HASH({Key, V}) | V], A) + end, + false, %% < list() + Values)). + +%% choose/3 + +choose(Hash, [Hash1 | _] = T, [Hash2 | _]) + when Hash1 =< Hash, Hash < Hash2 -> + T; + +choose(Hash, [Hash1 | _], [Hash2 | _] = T) + when Hash2 =< Hash, Hash < Hash1 -> + T; + +choose(_, T1, T2) -> + max(T1, T2). + +%% =========================================================================== + +%% attach/1 +%% +%% Register the local node as a handler of incoming requests for the +%% specified services when using the route_session/2 spawn_opt +%% callback. + +attach(ServiceNames) -> + abcast({attach, node(), ServiceNames}). + +%% detach/1 +%% +%% Deregister the local node as a handler of incoming requests. + +detach(ServiceNames) -> + abcast({detach, node(), ServiceNames}). + +%% abcast/1 + +abcast(T) -> + gen_server:abcast([node() | nodes()], ?SERVER, T), + ok. + +%% attached/1 + +attached(ServiceName) -> + try + ets:lookup_element(?SERVICE_TABLE, ServiceName, 2) + catch + error: badarg -> [] + end. + +%% cast/2 + +cast(Node, T) -> + gen_server:cast({?SERVER, Node}, T). + +%% attach/2 + +attach(Node, S) -> + case sets:to_list(S) of + [] -> + ok; + Services -> + cast(Node, {attach, node(), Services}) + end. + +%% =========================================================================== + +start_link() -> + gen_server:start_link({local, ?SERVER}, ?MODULE, _Args = [], _Opts = []). + +%% init/1 +%% +%% Maintain [node() | nodes()] in a table that maps from binary-valued +%% names, so we can lookup the corresponding atoms rather than convert +%% binaries that aren't necessarily node names. + +init([]) -> + ets:new(?NODE_TABLE, [set, named_table]), + ets:new(?SERVICE_TABLE, [bag, named_table]), + ok = net_kernel:monitor_nodes(true, [{node_type, all}, nodedown_reason]), + ets:insert(?NODE_TABLE, [{?B(N), N} || N <- [node() | nodes()]]), + abcast({attach, node()}), + {ok, sets:new()}. + +%% handle_call/3 + +handle_call(_, _From, S) -> + {reply, nok, S}. + +%% handle_cast/2 + +%% Remote node is asking which services the local node wants to handle. +handle_cast({attach, Node}, S) + when Node /= node() -> + attach(Node, S), + {noreply, S}; + +%% Node wants to handle incoming requests ... +handle_cast({attach, Node, ServiceNames}, S) -> + ets:insert(?SERVICE_TABLE, [{N, Node} || N <- ServiceNames]), + {noreply, case node() of + Node -> + sets:union(S, sets:from_list(ServiceNames)); + _ -> + S + end}; + +%% ... or not. +handle_cast({detach, Node, ServiceNames}, S) -> + ets:select_delete(?SERVICE_TABLE, [{{'$1', Node}, + [?ORCOND([{'==', '$1', {const, N}} + || N <- ServiceNames])], + [true]}]), + {noreply, case node() of + Node -> + sets:subtract(S, sets:from_list(ServiceNames)); + _ -> + S + end}; + +handle_cast(_, S) -> + {noreply, S}. + +%% handle_info/2 + +handle_info({nodeup, Node, _}, S) -> + ets:insert(?NODE_TABLE, {?B(Node), Node}), + cast(Node, {attach, node()}), %% ask which services remote node handles + attach(Node, S), %% say which service local node handles + {noreply, S}; + +handle_info({nodedown, Node, _}, S) -> + ets:delete(?NODE_TABLE, ?B(Node)), + ets:select_delete(?SERVICE_TABLE, [{{'_', Node}, [], [true]}]), + {noreply, S}; + +handle_info(_, S) -> + {noreply, S}. + +%% terminate/2 + +terminate(_, _) -> + ok. + +%% code_change/3 + +code_change(_OldVsn, State, _Extra) -> + {ok, State}. diff --git a/lib/diameter/src/base/diameter_misc_sup.erl b/lib/diameter/src/base/diameter_misc_sup.erl index 343688be23..fec5a41b5c 100644 --- a/lib/diameter/src/base/diameter_misc_sup.erl +++ b/lib/diameter/src/base/diameter_misc_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -35,6 +35,7 @@ diameter_stats, %% statistics counter management diameter_reg, %% service/property publishing diameter_peer, %% remote peer manager + diameter_dist, %% request distribution diameter_config]). %% configuration/restart %% start_link/0 diff --git a/lib/diameter/src/base/diameter_traffic.erl b/lib/diameter/src/base/diameter_traffic.erl index 2d3e4a2ac9..8423e30269 100644 --- a/lib/diameter/src/base/diameter_traffic.erl +++ b/lib/diameter/src/base/diameter_traffic.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -42,8 +42,12 @@ peer_up/1, peer_down/1]). +%% towards diameter_dist +-export([request_info/1]). + %% internal -export([send/1, %% send from remote node + request/1, %% process request in handler process init/1]). %% monitor process start -include_lib("diameter/include/diameter.hrl"). @@ -232,7 +236,7 @@ incr_rc(Dir, Pkt, TPid, MsgDict, AppDict, Dict0) -> -spec receive_message(pid(), Route, #diameter_packet{}, module(), RecvData) -> pid() %% request handler | boolean() %% answer, known request or not - | discard %% request discarded by MFA + | discard %% request discarded when Route :: {Handler, RequestRef, TPid} | Ack, RecvData :: {[SpawnOpt], #recvdata{}}, @@ -252,7 +256,8 @@ receive_message(TPid, Route, Pkt, Dict0, RecvData) -> recv(true, Ack, TPid, Pkt, Dict0, T) when is_boolean(Ack) -> {Opts, RecvData} = T, - spawn_request(Ack, TPid, Pkt, Dict0, RecvData, Opts); + AppT = find_app(TPid, Pkt, RecvData), + ack(Ack, TPid, spawn_request(AppT, Opts, Ack, TPid, Pkt, Dict0, RecvData)); %% ... answer to known request ... recv(false, {Pid, Ref, TPid}, _, Pkt, Dict0, _) -> @@ -274,58 +279,73 @@ recv(false, false, TPid, Pkt, _, _) -> incr(TPid, {{unknown, 0}, recv, discarded}), false. -%% spawn_request/6 +%% spawn_request/7 + +spawn_request(false, _, _, _, _, _, _) -> %% no transport + discard; -%% An MFA should return a pid() or the atom 'discard'. The latter -%% results in an acknowledgment back to the transport process when -%% appropriate, to ensure that send/recv callbacks can count -%% outstanding requests. Acknowledgement is implicit if the +%% An MFA should return the pid() of a process in which the argument +%% fun in applied, or the atom 'discard' if the fun is not applied. +%% The latter results in an acknowledgment back to the transport +%% process when appropriate, to ensure that send/recv callbacks can +%% count outstanding requests. Acknowledgement is implicit if the %% handler process dies (in a handle_request callback for example). -spawn_request(Ack, TPid, Pkt, Dict0, RecvData, {M,F,A}) -> - ReqF = fun() -> - ack(Ack, TPid, recv_request(Ack, TPid, Pkt, Dict0, RecvData)) - end, - ack(Ack, TPid, apply(M, F, [ReqF | A])); +spawn_request(AppT, {M,F,A}, Ack, TPid, Pkt, Dict0, RecvData) -> + %% Term to pass to request/1 in an appropriate process. Module + %% diameter_dist implements callbacks. + ReqT = {Pkt, AppT, Ack, TPid, Dict0, RecvData}, + apply(M, F, [ReqT | A]); %% A spawned process acks implicitly when it dies, so there's no need %% to handle 'discard'. -spawn_request(Ack, TPid, Pkt, Dict0, RecvData, Opts) -> +spawn_request(AppT, Opts, Ack, TPid, Pkt, Dict0, RecvData) -> spawn_opt(fun() -> - recv_request(Ack, TPid, Pkt, Dict0, RecvData) + recv_request(Ack, TPid, Pkt, Dict0, RecvData, AppT) end, Opts). +%% request_info/1 +%% +%% Limited request information for diameter_dist. + +request_info({Pkt, _AppT, _Ack, _TPid, _Dict0, RecvData} = _ReqT) -> + {RecvData#recvdata.service_name, Pkt#diameter_packet.bin}. + +%% request/1 +%% +%% Called from a handler process chosen by a transport spawn_opt MFA +%% to process an incoming request. + +request({Pkt, AppT, Ack, TPid, Dict0, RecvData} = _ReqT) -> + ack(Ack, TPid, recv_request(Ack, TPid, Pkt, Dict0, RecvData, AppT)). + %% ack/3 ack(Ack, TPid, RC) -> - RC == discard andalso Ack andalso (TPid ! {send, false}), + RC == discard + andalso Ack + andalso (TPid ! {send, false}), RC. %% --------------------------------------------------------------------------- -%% recv_request/5 +%% recv_request/6 %% --------------------------------------------------------------------------- -spec recv_request(Ack :: boolean(), TPid :: pid(), #diameter_packet{}, Dict0 :: module(), - #recvdata{}) + #recvdata{}, + AppT :: {#diameter_app{}, #diameter_caps{}} + | #diameter_caps{}) %% no suitable app -> ok %% answer was sent - | discard %% or not - | false. %% no transport - -recv_request(Ack, - TPid, - #diameter_packet{header = #diameter_header{application_id = Id}} - = Pkt, - Dict0, - #recvdata{peerT = PeerT, - apps = Apps, - counters = Count} - = RecvData) -> + | discard. %% or not + +recv_request(Ack, TPid, Pkt, Dict0, RecvData, AppT) -> Ack andalso (TPid ! {handler, self()}), - case diameter_service:find_incoming_app(PeerT, TPid, Id, Apps) of + case AppT of {#diameter_app{id = Aid, dictionary = AppDict} = App, Caps} -> + Count = RecvData#recvdata.counters, Count andalso incr(recv, Pkt, TPid, AppDict), DecPkt = decode(Aid, AppDict, RecvData, Pkt), Count andalso incr_error(recv, DecPkt, TPid, AppDict), @@ -349,11 +369,20 @@ recv_request(Ack, Dict0, RecvData, DecPkt, - [[]]); - false = No -> %% transport has gone down - No + [[]]) end. +%% find_app/3 +%% +%% Lookup the application of a received Diameter request on the node +%% on which it's received. + +find_app(TPid, + #diameter_packet{header = #diameter_header{application_id = Id}}, + #recvdata{peerT = PeerT, + apps = Apps}) -> + diameter_service:find_incoming_app(PeerT, TPid, Id, Apps). + %% decode/4 decode(Id, Dict, #recvdata{codec = Opts}, Pkt) -> diff --git a/lib/diameter/src/diameter.appup.src b/lib/diameter/src/diameter.appup.src index 4e6b983bac..52263633fb 100644 --- a/lib/diameter/src/diameter.appup.src +++ b/lib/diameter/src/diameter.appup.src @@ -60,7 +60,8 @@ {"2.1.3", [{restart_application, diameter}]}, %% 20.2 {"2.1.4", [{restart_application, diameter}]}, %% 20.3 {"2.1.4.1", [{restart_application, diameter}]}, %% 20.3.8.19 - {"2.1.5", [{update, diameter_peer_fsm}]} %% 21.0 + {"2.1.5", [{restart_application, diameter}]}, %% 21.0 + {"2.1.6", [{restart_application, diameter}]} %% 21.1 ], [ {"0.9", [{restart_application, diameter}]}, @@ -102,6 +103,7 @@ {"2.1.3", [{restart_application, diameter}]}, {"2.1.4", [{restart_application, diameter}]}, {"2.1.4.1", [{restart_application, diameter}]}, - {"2.1.5", [{update, diameter_peer_fsm}]} + {"2.1.5", [{restart_application, diameter}]}, + {"2.1.6", [{restart_application, diameter}]} ] }. diff --git a/lib/diameter/src/modules.mk b/lib/diameter/src/modules.mk index bb86de016a..d16292bb88 100644 --- a/lib/diameter/src/modules.mk +++ b/lib/diameter/src/modules.mk @@ -1,7 +1,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 2010-2017. All Rights Reserved. +# Copyright Ericsson AB 2010-2019. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -40,6 +40,7 @@ RT_MODULES = \ base/diameter_config \ base/diameter_config_sup \ base/diameter_codec \ + base/diameter_dist \ base/diameter_gen \ base/diameter_lib \ base/diameter_misc_sup \ diff --git a/lib/diameter/test/diameter_dist_SUITE.erl b/lib/diameter/test/diameter_dist_SUITE.erl new file mode 100644 index 0000000000..b2e4c35b9a --- /dev/null +++ b/lib/diameter/test/diameter_dist_SUITE.erl @@ -0,0 +1,332 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2019. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +%% +%% Tests of traffic between two Diameter nodes, the server being +%% spread across three Erlang nodes. +%% + +-module(diameter_dist_SUITE). + +-export([suite/0, + all/0]). + +%% testcases +-export([enslave/1, enslave/0, + ping/1, + start/1, + connect/1, + send/1, + stop/1, stop/0]). + +%% diameter callbacks +-export([peer_up/3, + peer_down/3, + pick_peer/4, + prepare_request/3, + prepare_retransmit/3, + handle_answer/4, + handle_error/4, + handle_request/3]). + +-export([call/1]). + +-include("diameter.hrl"). +-include("diameter_gen_base_rfc6733.hrl"). + +%% =========================================================================== + +-define(util, diameter_util). + +-define(CLIENT, 'CLIENT'). +-define(SERVER, 'SERVER'). +-define(REALM, "erlang.org"). +-define(DICT, diameter_gen_base_rfc6733). +-define(ADDR, {127,0,0,1}). + +%% Config for diameter:start_service/2. +-define(SERVICE(Host), + [{'Origin-Host', Host ++ [$.|?REALM]}, + {'Origin-Realm', ?REALM}, + {'Host-IP-Address', [?ADDR]}, + {'Vendor-Id', 12345}, + {'Product-Name', "OTP/diameter"}, + {'Auth-Application-Id', [?DICT:id()]}, + {'Origin-State-Id', origin()}, + {spawn_opt, {diameter_dist, route_session, [#{id => []}]}}, + {sequence, fun sequence/0}, + {string_decode, false}, + {application, [{dictionary, ?DICT}, + {module, ?MODULE}, + {request_errors, callback}, + {answer_errors, callback}]}]). + +-define(SUCCESS, 2001). +-define(BUSY, 3004). +-define(LOGOUT, ?'DIAMETER_BASE_TERMINATION-CAUSE_LOGOUT'). +-define(MOVED, ?'DIAMETER_BASE_TERMINATION-CAUSE_USER_MOVED'). +-define(TIMEOUT, ?'DIAMETER_BASE_TERMINATION-CAUSE_SESSION_TIMEOUT'). + +-define(L, atom_to_list). +-define(A, list_to_atom). + +%% The order here is significant and causes the server to listen +%% before the clients connect. The server listens on the first node, +%% and distributes requests to the other two. +-define(NODES, [{server0, ?SERVER}, + {server1, ?SERVER}, + {server2, ?SERVER}, + {client, ?CLIENT}]). + +%% Options to ct_slave:start/2. +-define(TIMEOUTS, [{T, 15000} || T <- [boot_timeout, + init_timeout, + start_timeout]]). + +%% =========================================================================== + +suite() -> + [{timetrap, {seconds, 60}}]. + +all() -> + [enslave, + ping, + start, + connect, + send, + stop]. + +%% =========================================================================== +%% start/stop testcases + +%% enslave/1 +%% +%% Start four slave nodes, three to implement a Diameter server, +%% one to implement a client. + +enslave() -> + [{timetrap, {seconds, 30*length(?NODES)}}]. + +enslave(Config) -> + Here = filename:dirname(code:which(?MODULE)), + Ebin = filename:join([Here, "..", "ebin"]), + Dirs = [Here, Ebin], + Nodes = [{N,S} || {M,S} <- ?NODES, N <- [slave(M, Dirs)]], + ?util:write_priv(Config, nodes, [{N,S} || {{N,ok},S} <- Nodes]), + [] = [{T,S} || {{_,E} = T, S} <- Nodes, E /= ok]. + +slave(Name, Dirs) -> + add_pathsa(Dirs, ct_slave:start(Name, ?TIMEOUTS)). + +add_pathsa(Dirs, {ok, Node}) -> + {Node, rpc:call(Node, code, add_pathsa, [Dirs])}; +add_pathsa(_, No) -> + {No, error}. + +%% ping/1 +%% +%% Ensure the server nodes are connected so that diameter_dist can attach. + +ping({S, Nodes}) -> + ?SERVER = S, + [N || {N,_} <- Nodes, + node() /= N, + pang <- [net_adm:ping(N)]]; + +ping(Config) -> + Nodes = lists:droplast(?util:read_priv(Config, nodes)), + [] = [{N,RC} || {N,S} <- Nodes, + RC <- [rpc:call(N, ?MODULE, ping, [{S,Nodes}])], + RC /= []]. + +%% start/1 +%% +%% Start diameter services. + +start(SvcName) + when is_atom(SvcName) -> + ok = diameter:start(), + ok = diameter:start_service(SvcName, ?SERVICE((?L(SvcName)))); + +start(Config) -> + Nodes = ?util:read_priv(Config, nodes), + [] = [{N,RC} || {N,S} <- Nodes, + RC <- [rpc:call(N, ?MODULE, start, [S])], + RC /= ok]. + +sequence() -> + sequence(sname()). + +sequence(client) -> + {0,32}; +sequence(Server) -> + "server" ++ N = ?L(Server), + {list_to_integer(N), 30}. + +origin() -> + origin(sname()). + +origin(client) -> + 99; +origin(Server) -> + "server" ++ N = ?L(Server), + list_to_integer(N). + +%% connect/1 +%% +%% Establish one connection from the client, terminated on the first +%% server node, the others handling requests. + +connect({?SERVER, Config, [{Node, _} | _]}) -> + if Node == node() -> %% server0 + ?util:write_priv(Config, lref, {Node, ?util:listen(?SERVER, tcp)}); + true -> + diameter_dist:attach([?SERVER]) + end, + ok; + +connect({?CLIENT, Config, _}) -> + ?util:connect(?CLIENT, tcp, ?util:read_priv(Config, lref)), + ok; + +connect(Config) -> + Nodes = ?util:read_priv(Config, nodes), + [] = [{N,RC} || {N,S} <- Nodes, + RC <- [rpc:call(N, ?MODULE, connect, [{S, Config, Nodes}])], + RC /= ok]. + +%% stop/1 +%% +%% Stop the slave nodes. + +stop() -> + [{timetrap, {seconds, 30*length(?NODES)}}]. + +stop(_Config) -> + [] = [{N,E} || {N,_} <- ?NODES, + {error, _, _} = E <- [ct_slave:stop(N)]]. + +%% =========================================================================== +%% traffic testcases + +%% send/1 +%% +%% Send 100 requests and ensure the node name sent as User-Name isn't +%% the node terminating transport. + +send(Config) -> + send(Config, 100, dict:new()). + +%% send/2 + +send(Config, 0, Dict) -> + [{Server0, _} | _] = ?util:read_priv(Config, nodes) , + Node = atom_to_binary(Server0, utf8), + {false, _} = {dict:is_key(Node, Dict), dict:to_list(Dict)}; + +send(Config, N, Dict) -> + #diameter_base_STA{'Result-Code' = ?SUCCESS, + 'User-Name' = [ServerNode]} + = send(Config, str(?LOGOUT)), + true = is_binary(ServerNode), + send(Config, N-1, dict:update_counter(ServerNode, 1, Dict)). + +%% =========================================================================== + +str(Cause) -> + #diameter_base_STR{'Destination-Realm' = ?REALM, + 'Auth-Application-Id' = ?DICT:id(), + 'Termination-Cause' = Cause}. + +%% send/2 + +send(Config, Req) -> + {Node, _} = lists:last(?util:read_priv(Config, nodes)), + rpc:call(Node, ?MODULE, call, [Req]). + +%% call/1 + +call(Req) -> + diameter:call(?CLIENT, ?DICT, Req, []). + +%% sname/0 + +sname() -> + ?A(hd(string:tokens(?L(node()), "@"))). + +%% =========================================================================== +%% diameter callbacks + +%% peer_up/3 + +peer_up(_SvcName, _Peer, State) -> + State. + +%% peer_down/3 + +peer_down(_SvcName, _Peer, State) -> + State. + +%% pick_peer/4 + +pick_peer([Peer], [], ?CLIENT, _State) -> + {ok, Peer}. + +%% prepare_request/3 + +prepare_request(Pkt, ?CLIENT, {_Ref, Caps}) -> + #diameter_packet{msg = Req} + = Pkt, + #diameter_caps{origin_host = {OH, _}, + origin_realm = {OR, _}} + = Caps, + {send, Req#diameter_base_STR{'Origin-Host' = OH, + 'Origin-Realm' = OR, + 'Session-Id' = diameter:session_id(OH)}}. + +%% prepare_retransmit/3 + +prepare_retransmit(_, ?CLIENT, _) -> + discard. + +%% handle_answer/5 + +handle_answer(Pkt, _Req, ?CLIENT, _Peer) -> + #diameter_packet{msg = Rec, errors = []} = Pkt, + Rec. + +%% handle_error/5 + +handle_error(Reason, _Req, ?CLIENT, _Peer) -> + {error, Reason}. + +%% handle_request/3 + +handle_request(Pkt, ?SERVER, {_, Caps}) -> + #diameter_packet{msg = #diameter_base_STR{'Session-Id' = SId}} + = Pkt, + #diameter_caps{origin_host = {OH, _}, + origin_realm = {OR, _}} + = Caps, + {reply, #diameter_base_STA{'Result-Code' = ?SUCCESS, + 'Session-Id' = SId, + 'Origin-Host' = OH, + 'Origin-Realm' = OR, + 'User-Name' = [atom_to_binary(node(), utf8)]}}. diff --git a/lib/diameter/test/diameter_distribution_SUITE.erl b/lib/diameter/test/diameter_distribution_SUITE.erl index 5146f68ff1..5fe02284ae 100644 --- a/lib/diameter/test/diameter_distribution_SUITE.erl +++ b/lib/diameter/test/diameter_distribution_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -76,6 +76,7 @@ {share_peers, peers()}, {use_shared_peers, peers()}, {restrict_connections, false}, + {spawn_opt, {diameter_dist, spawn_local, []}}, {sequence, fun sequence/0}, {application, [{dictionary, ?DICT}, {module, ?MODULE}, @@ -125,7 +126,7 @@ all() -> %% enslave/1 %% %% Start four slave nodes, one to implement a Diameter server, -%% two three to implement a client. +%% three to implement a client. enslave() -> [{timetrap, {seconds, 30*length(?NODES)}}]. @@ -331,6 +332,8 @@ prepare_request(Pkt, ?CLIENT, {_Ref, Caps}, {_, client0}) -> 'Origin-Realm' = OR, 'Session-Id' = diameter:session_id(OH)}}. +%% prepare_retransmit/4 + prepare_retransmit(Pkt, ?CLIENT, _, {_, client0}) -> #diameter_packet{msg = #diameter_base_STR{'Termination-Cause' = ?MOVED}} = Pkt, %% assert diff --git a/lib/diameter/test/diameter_pool_SUITE.erl b/lib/diameter/test/diameter_pool_SUITE.erl index 97c16940ff..a36a4fa17a 100644 --- a/lib/diameter/test/diameter_pool_SUITE.erl +++ b/lib/diameter/test/diameter_pool_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015-2017. All Rights Reserved. +%% Copyright Ericsson AB 2015-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -51,6 +51,7 @@ {'Auth-Application-Id', [0]}, %% common {'Acct-Application-Id', [3]}, %% accounting {restrict_connections, false}, + {spawn_opt, {diameter_dist, route_session, []}}, {application, [{alias, common}, {dictionary, diameter_gen_base_rfc6733}, {module, diameter_callback}]}, diff --git a/lib/diameter/test/diameter_traffic_SUITE.erl b/lib/diameter/test/diameter_traffic_SUITE.erl index 434aef01dd..47b00c25a2 100644 --- a/lib/diameter/test/diameter_traffic_SUITE.erl +++ b/lib/diameter/test/diameter_traffic_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2018. All Rights Reserved. +%% Copyright Ericsson AB 2010-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -539,8 +539,7 @@ add_transports(Config) -> ++ [{unordered, unordered()} || T == sctp], [{capabilities_cb, fun capx/2}, {pool_size, 8} - | server_apps()] - ++ [{spawn_opt, {erlang, spawn, []}} || CS]), + | server_apps()]), Cs = [?util:connect(CN, [T, {sender, CS} | client_opts(T)], LRef, diff --git a/lib/diameter/test/modules.mk b/lib/diameter/test/modules.mk index 0c73adca12..90b0a25d5f 100644 --- a/lib/diameter/test/modules.mk +++ b/lib/diameter/test/modules.mk @@ -1,7 +1,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 2010-2017. All Rights Reserved. +# Copyright Ericsson AB 2010-2019. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -31,6 +31,7 @@ MODULES = \ diameter_codec_test \ diameter_config_SUITE \ diameter_compiler_SUITE \ + diameter_dist_SUITE \ diameter_distribution_SUITE \ diameter_dpr_SUITE \ diameter_event_SUITE \ diff --git a/lib/diameter/vsn.mk b/lib/diameter/vsn.mk index 8c75c9e55e..a900e8f28e 100644 --- a/lib/diameter/vsn.mk +++ b/lib/diameter/vsn.mk @@ -17,5 +17,5 @@ # %CopyrightEnd% APPLICATION = diameter -DIAMETER_VSN = 2.1.6 +DIAMETER_VSN = 2.2 APP_VSN = $(APPLICATION)-$(DIAMETER_VSN)$(PRE_VSN) diff --git a/lib/edoc/doc/src/notes.xml b/lib/edoc/doc/src/notes.xml index e818887eb8..145856bcaa 100644 --- a/lib/edoc/doc/src/notes.xml +++ b/lib/edoc/doc/src/notes.xml @@ -32,6 +32,22 @@ <p>This document describes the changes made to the EDoc application.</p> +<section><title>Edoc 0.10</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Print a helpful message explaining that adding + <c>{preprocess, true}</c> can help if reading a source + file fails. </p> + <p> + Own Id: OTP-15605 Aux Id: ERL-841 </p> + </item> + </list> + </section> + +</section> + <section><title>Edoc 0.9.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/edoc/vsn.mk b/lib/edoc/vsn.mk index 0b3636f030..b6e1422623 100644 --- a/lib/edoc/vsn.mk +++ b/lib/edoc/vsn.mk @@ -1 +1 @@ -EDOC_VSN = 0.9.4 +EDOC_VSN = 0.10 diff --git a/lib/erl_docgen/doc/src/notes.xml b/lib/erl_docgen/doc/src/notes.xml index 97c842a324..54f0a36b27 100644 --- a/lib/erl_docgen/doc/src/notes.xml +++ b/lib/erl_docgen/doc/src/notes.xml @@ -31,7 +31,31 @@ </header> <p>This document describes the changes made to the <em>erl_docgen</em> application.</p> - <section><title>Erl_Docgen 0.8.1</title> + <section><title>Erl_Docgen 0.9</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The HTML reference documentation shows the OTP version + where modules and functions were first introduced. + Versions older than R13B04 are not shown.</p> + <p> + Own Id: OTP-15460</p> + </item> + <item> + <p> + Make html documentation of C functions with many + arguments more readable.</p> + <p> + Own Id: OTP-15637 Aux Id: PR-2160 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Docgen 0.8.1</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/erl_docgen/vsn.mk b/lib/erl_docgen/vsn.mk index 3b2f6db6a1..6321f229dd 100644 --- a/lib/erl_docgen/vsn.mk +++ b/lib/erl_docgen/vsn.mk @@ -1 +1 @@ -ERL_DOCGEN_VSN = 0.8.1 +ERL_DOCGEN_VSN = 0.9 diff --git a/lib/erl_interface/configure.in b/lib/erl_interface/configure.in index 53c2f7069c..f0e9b2eb3f 100644 --- a/lib/erl_interface/configure.in +++ b/lib/erl_interface/configure.in @@ -61,7 +61,7 @@ fi TARGET=$host AC_SUBST(TARGET) -AC_CONFIG_HEADER([src/$host/config.h:config.h.in include/$host/ei_config.h:include/ei_config.h.in]) +AC_CONFIG_HEADER([src/$host/config.h:config.h.in]) dnl ---------------------------------------------------------------------- dnl Optional features @@ -184,14 +184,6 @@ AC_TRY_COMPILE([#include <sys/types.h> AC_DEFINE(HAVE_SOCKLEN_T, [], [Define if you have the `socklen_t' type])], [AC_MSG_RESULT(no)]) -AC_MSG_CHECKING([for deprecated attribute]) -AC_TRY_COMPILE([], -[void my_function(void) __attribute__((deprecated));], -[AC_MSG_RESULT(yes) - AC_DEFINE(HAVE_DEPRECATED_ATTRIBUTE, [], [Define to 1 if you have the `deprecated' attribute])], -[AC_MSG_RESULT(no)]) - - # Checks for library functions. AC_FUNC_ALLOCA dnl AC_FUNC_FORK diff --git a/lib/erl_interface/doc/src/ei.xml b/lib/erl_interface/doc/src/ei.xml index 2bdb390644..f081ca926a 100644 --- a/lib/erl_interface/doc/src/ei.xml +++ b/lib/erl_interface/doc/src/ei.xml @@ -736,7 +736,7 @@ ei_encode_tuple_header(buf, &i, 0);</pre> </func> <func> - <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_init(void)</nametext></name> + <name since="OTP 21.3"><ret>int</ret><nametext>ei_init(void)</nametext></name> <fsummary>Initialize the ei library.</fsummary> <desc> <p>Initialize the <c>ei</c> library. This function should be called once diff --git a/lib/erl_interface/doc/src/ei_connect.xml b/lib/erl_interface/doc/src/ei_connect.xml index df40973270..795f1249b3 100644 --- a/lib/erl_interface/doc/src/ei_connect.xml +++ b/lib/erl_interface/doc/src/ei_connect.xml @@ -412,7 +412,7 @@ typedef struct { </func> <func> - <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_close_connection(int fd)</nametext></name> + <name since="OTP 21.3"><ret>int</ret><nametext>ei_close_connection(int fd)</nametext></name> <fsummary>Close a connection.</fsummary> <desc> <p>Closes a previously opened connection or listen socket.</p> @@ -472,9 +472,9 @@ fd = ei_xconnect(&ec, &addr, ALIVE); <func> <name since=""><ret>int</ret><nametext>ei_connect_init(ei_cnode* ec, const char* this_node_name, const char *cookie, short creation)</nametext></name> - <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_connect_init_ussi(ei_cnode* ec, const char* this_node_name, const char *cookie, short creation, ei_socket_callbacks *cbs, int cbs_sz, void *setup_context)</nametext></name> + <name since="OTP 21.3"><ret>int</ret><nametext>ei_connect_init_ussi(ei_cnode* ec, const char* this_node_name, const char *cookie, short creation, ei_socket_callbacks *cbs, int cbs_sz, void *setup_context)</nametext></name> <name since=""><ret>int</ret><nametext>ei_connect_xinit(ei_cnode* ec, const char *thishostname, const char *thisalivename, const char *thisnodename, Erl_IpAddr thisipaddr, const char *cookie, short creation)</nametext></name> - <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_connect_xinit_ussi(ei_cnode* ec, const char *thishostname, const char *thisalivename, const char *thisnodename, Erl_IpAddr thisipaddr, const char *cookie, short creation, ei_socket_callbacks *cbs, int cbs_sz, void *setup_context)</nametext></name> + <name since="OTP 21.3"><ret>int</ret><nametext>ei_connect_xinit_ussi(ei_cnode* ec, const char *thishostname, const char *thisalivename, const char *thisnodename, Erl_IpAddr thisipaddr, const char *cookie, short creation, ei_socket_callbacks *cbs, int cbs_sz, void *setup_context)</nametext></name> <fsummary>Initialize for a connection.</fsummary> <desc> <p>Initializes the <c>ec</c> structure, to @@ -595,8 +595,8 @@ if (ei_connect_init(&ec, "madonna", "cookie...", n++) < 0) { </func> <func> - <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_listen(ei_cnode *ec, int *port, int backlog)</nametext></name> - <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_xlisten(ei_cnode *ec, Erl_IpAddr adr, int *port, int backlog)</nametext></name> + <name since="OTP 21.3"><ret>int</ret><nametext>ei_listen(ei_cnode *ec, int *port, int backlog)</nametext></name> + <name since="OTP 21.3"><ret>int</ret><nametext>ei_xlisten(ei_cnode *ec, Erl_IpAddr adr, int *port, int backlog)</nametext></name> <fsummary>Create a listen socket.</fsummary> <desc> <p>Used by a server process to setup a listen socket which diff --git a/lib/erl_interface/doc/src/notes.xml b/lib/erl_interface/doc/src/notes.xml index 07ddd82718..b686cfbf33 100644 --- a/lib/erl_interface/doc/src/notes.xml +++ b/lib/erl_interface/doc/src/notes.xml @@ -31,6 +31,23 @@ </header> <p>This document describes the changes made to the Erl_interface application.</p> +<section><title>Erl_Interface 3.11</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Support for plugin of a <seealso + marker="ei_connect#ussi">user supplied socket + implementation</seealso> has been added.</p> + <p> + Own Id: OTP-15442 Aux Id: ERIERL-258 </p> + </item> + </list> + </section> + +</section> + <section><title>Erl_Interface 3.10.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/erl_interface/include/ei.h b/lib/erl_interface/include/ei.h index 10a6e5bf50..aa2a49098f 100644 --- a/lib/erl_interface/include/ei.h +++ b/lib/erl_interface/include/ei.h @@ -47,21 +47,26 @@ typedef LONG_PTR ssize_t; /* Sigh... */ # include <netdb.h> #endif -#ifndef EI_INCLUDED_CONFIG_H__ -#include "ei_config.h" +#ifdef __has_attribute +#if __has_attribute(deprecated) +#define EI_HAVE_DEPRECATED_ATTR__ 1 +#else +#undef EI_HAVE_DEPRECATED_ATTR__ +#endif #endif -#if defined(HAVE_DEPRECATED_ATTRIBUTE) && !defined(EI_NO_DEPR_WARN) -#define EI_HAVE_DEPRECATED_ATTR__ 1 +#ifdef EI_NO_DEPR_WARN +#undef EI_HAVE_DEPRECATED_ATTR__ +#endif + +#ifdef EI_HAVE_DEPRECATED_ATTR__ #define EI_DEPRECATED_ATTR_NAME deprecated #define EI_DEPRECATED_ATTR __attribute__((EI_DEPRECATED_ATTR_NAME)) #else -#undef EI_HAVE_DEPRECATED_ATTR__ #define EI_DEPRECATED_ATTR_NAME #define EI_DEPRECATED_ATTR #endif - /* -------------------------------------------------------------------- */ /* Defines part of API */ /* -------------------------------------------------------------------- */ diff --git a/lib/erl_interface/include/ei_config.h.in b/lib/erl_interface/include/ei_config.h.in deleted file mode 100644 index dfcf676bed..0000000000 --- a/lib/erl_interface/include/ei_config.h.in +++ /dev/null @@ -1,3 +0,0 @@ - -/* Define to 1 if you have the `deprecated' attribute */ -#undef HAVE_DEPRECATED_ATTRIBUTE diff --git a/lib/erl_interface/src/Makefile b/lib/erl_interface/src/Makefile index 800557fbfe..00c49f1622 100644 --- a/lib/erl_interface/src/Makefile +++ b/lib/erl_interface/src/Makefile @@ -27,7 +27,9 @@ include $(ERL_TOP)/make/output.mk include $(ERL_TOP)/make/target.mk debug opt shared purify quantify purecov gcov: +ifndef TERTIARY_BOOTSTRAP $(make_verbose)$(MAKE) -f $(TARGET)/Makefile TYPE=$@ +endif clean depend docs release release_docs tests release_tests check xmllint: $(make_verbose)$(MAKE) -f $(TARGET)/Makefile $@ diff --git a/lib/erl_interface/src/Makefile.in b/lib/erl_interface/src/Makefile.in index 1b19a85f1e..b2600f0fab 100644 --- a/lib/erl_interface/src/Makefile.in +++ b/lib/erl_interface/src/Makefile.in @@ -118,7 +118,7 @@ else AR_FLAGS=rcv endif -INCFLAGS = -I. -I../include -I../include/$(TARGET) -Iconnect -Iencode -Idecode -Imisc -Iepmd \ +INCFLAGS = -I. -I../include -Iconnect -Iencode -Idecode -Imisc -Iepmd \ -Iregistry -I$(TARGET) ifeq ($(USING_VC),yes) @@ -322,8 +322,7 @@ HEADERS = \ ../include/ei.h \ ../include/ei_connect.h \ ../include/eicode.h \ - ../include/erl_interface.h \ - ../include/$(TARGET)/ei_config.h + ../include/erl_interface.h EISOURCES = \ $(CONNECTSRC) \ diff --git a/lib/erl_interface/src/misc/eidef.h b/lib/erl_interface/src/misc/eidef.h index 083814c6e9..f38824d826 100644 --- a/lib/erl_interface/src/misc/eidef.h +++ b/lib/erl_interface/src/misc/eidef.h @@ -25,7 +25,6 @@ /* Common definitions used in ei user interface */ -#define EI_INCLUDED_CONFIG_H__ #include "config.h" /* Central include of config.h */ /* vxWorks.h needs to be before stddef.h */ diff --git a/lib/erl_interface/test/all_SUITE_data/Makefile.src b/lib/erl_interface/test/all_SUITE_data/Makefile.src index 57e522fd3e..4f27b097c8 100644 --- a/lib/erl_interface/test/all_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/all_SUITE_data/Makefile.src @@ -21,8 +21,8 @@ include @erl_interface_mk_include@ CC0 = @CC@ CC = .@DS@gccifier@exe@ -CC"$(CC0)" -CFLAGS0 = @CFLAGS@ -I@erl_interface_include@ @erl_interface_target_include@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ +CFLAGS0 = @CFLAGS@ -I@erl_interface_include@ +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ EI_COMMON_OBJS = runner@obj@ ei_runner@obj@ ALL_OBJS = gccifier@exe@ $(EI_COMMON_OBJS) diff --git a/lib/erl_interface/test/ei_accept_SUITE_data/Makefile.src b/lib/erl_interface/test/ei_accept_SUITE_data/Makefile.src index 76dc84221f..10ef437f8b 100644 --- a/lib/erl_interface/test/ei_accept_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/ei_accept_SUITE_data/Makefile.src @@ -27,7 +27,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/ei_runner@obj@ \ $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data EI_ACCEPT_OBJS = ei_accept_test@obj@ ei_accept_test_decl@obj@ EIACCNODE_OBJS = eiaccnode@obj@ diff --git a/lib/erl_interface/test/ei_connect_SUITE_data/Makefile.src b/lib/erl_interface/test/ei_connect_SUITE_data/Makefile.src index d1694e607d..c2d8261dd8 100644 --- a/lib/erl_interface/test/ei_connect_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/ei_connect_SUITE_data/Makefile.src @@ -28,7 +28,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/ei_runner@obj@ \ $(LIBERL) $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data EI_CONNECT_OBJS = ei_connect_test@obj@ ei_connect_test_decl@obj@ EINODE_OBJS = einode@obj@ diff --git a/lib/erl_interface/test/ei_decode_SUITE_data/Makefile.src b/lib/erl_interface/test/ei_decode_SUITE_data/Makefile.src index 76591d893c..e678914a40 100644 --- a/lib/erl_interface/test/ei_decode_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/ei_decode_SUITE_data/Makefile.src @@ -27,7 +27,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/ei_runner@obj@ \ $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data EI_DECODE_OBJS = ei_decode_test@obj@ ei_decode_test_decl@obj@ all: ei_decode_test@exe@ diff --git a/lib/erl_interface/test/ei_decode_encode_SUITE_data/Makefile.src b/lib/erl_interface/test/ei_decode_encode_SUITE_data/Makefile.src index 3f5fa4f295..853fe9ddeb 100644 --- a/lib/erl_interface/test/ei_decode_encode_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/ei_decode_encode_SUITE_data/Makefile.src @@ -27,7 +27,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/ei_runner@obj@ \ $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data EI_DECODE_ENCODE_OBJS = ei_decode_encode_test@obj@ ei_decode_encode_test_decl@obj@ all: ei_decode_encode_test@exe@ diff --git a/lib/erl_interface/test/ei_encode_SUITE_data/Makefile.src b/lib/erl_interface/test/ei_encode_SUITE_data/Makefile.src index 489382d85e..3b2cab7af4 100644 --- a/lib/erl_interface/test/ei_encode_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/ei_encode_SUITE_data/Makefile.src @@ -27,7 +27,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/ei_runner@obj@ \ $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data EI_ENCODE_OBJS = ei_encode_test@obj@ ei_encode_test_decl@obj@ all: ei_encode_test@exe@ diff --git a/lib/erl_interface/test/ei_format_SUITE_data/Makefile.src b/lib/erl_interface/test/ei_format_SUITE_data/Makefile.src index 9e5a271db6..b89dcae45a 100644 --- a/lib/erl_interface/test/ei_format_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/ei_format_SUITE_data/Makefile.src @@ -27,7 +27,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/ei_runner@obj@ \ $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data EI_FORMAT_OBJS = ei_format_test@obj@ ei_format_test_decl@obj@ all: ei_format_test@exe@ diff --git a/lib/erl_interface/test/ei_print_SUITE_data/Makefile.src b/lib/erl_interface/test/ei_print_SUITE_data/Makefile.src index 354011f1a5..150c11b99c 100644 --- a/lib/erl_interface/test/ei_print_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/ei_print_SUITE_data/Makefile.src @@ -27,7 +27,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/ei_runner@obj@ \ $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data EI_PRINT_OBJS = ei_print_test@obj@ ei_print_test_decl@obj@ all: ei_print_test@exe@ diff --git a/lib/erl_interface/test/ei_tmo_SUITE_data/Makefile.src b/lib/erl_interface/test/ei_tmo_SUITE_data/Makefile.src index 76a9c6a606..b4ee361939 100644 --- a/lib/erl_interface/test/ei_tmo_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/ei_tmo_SUITE_data/Makefile.src @@ -27,7 +27,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/ei_runner@obj@ \ $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data EI_TMO_OBJS = ei_tmo_test@obj@ ei_tmo_test_decl@obj@ all: ei_tmo_test@exe@ diff --git a/lib/erl_interface/test/erl_connect_SUITE_data/Makefile.src b/lib/erl_interface/test/erl_connect_SUITE_data/Makefile.src index 19b076794a..ff4c382c97 100644 --- a/lib/erl_interface/test/erl_connect_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/erl_connect_SUITE_data/Makefile.src @@ -28,7 +28,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/runner@obj@ \ $(LIBERL) $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data OBJS = erl_connect_test@obj@ erl_connect_test_decl@obj@ all: erl_connect_test@exe@ diff --git a/lib/erl_interface/test/erl_eterm_SUITE_data/Makefile.src b/lib/erl_interface/test/erl_eterm_SUITE_data/Makefile.src index cd4de56d7c..4b1ddf77b6 100644 --- a/lib/erl_interface/test/erl_eterm_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/erl_eterm_SUITE_data/Makefile.src @@ -28,7 +28,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/runner@obj@ \ $(LIBERL) $(LIBEI) @erl_interface_sock_libs@ @LIBS@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data ETERM_OBJS = eterm_test@obj@ eterm_test_decl@obj@ CNODE_OBJS = cnode@obj@ PRINT_OBJS = print_term@obj@ diff --git a/lib/erl_interface/test/erl_ext_SUITE_data/Makefile.src b/lib/erl_interface/test/erl_ext_SUITE_data/Makefile.src index 50b60637cd..fe8caebbd6 100644 --- a/lib/erl_interface/test/erl_ext_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/erl_ext_SUITE_data/Makefile.src @@ -28,7 +28,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/runner@obj@ \ $(LIBERL) $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data EXT_OBJS = ext_test@obj@ ext_test_decl@obj@ all: ext_test@exe@ diff --git a/lib/erl_interface/test/erl_format_SUITE_data/Makefile.src b/lib/erl_interface/test/erl_format_SUITE_data/Makefile.src index 7d51cd6007..2ba59ab651 100644 --- a/lib/erl_interface/test/erl_format_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/erl_format_SUITE_data/Makefile.src @@ -28,7 +28,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/runner@obj@ \ $(LIBERL) $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data FORMAT_OBJS = format_test@obj@ format_test_decl@obj@ all: format_test@exe@ diff --git a/lib/erl_interface/test/erl_global_SUITE_data/Makefile.src b/lib/erl_interface/test/erl_global_SUITE_data/Makefile.src index 9f2a8619ac..1c1530d1b6 100644 --- a/lib/erl_interface/test/erl_global_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/erl_global_SUITE_data/Makefile.src @@ -28,7 +28,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/runner@obj@ \ $(LIBERL) $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data OBJS = erl_global_test@obj@ erl_global_test_decl@obj@ all: erl_global_test@exe@ diff --git a/lib/erl_interface/test/erl_match_SUITE_data/Makefile.src b/lib/erl_interface/test/erl_match_SUITE_data/Makefile.src index 56ed1df203..156214a269 100644 --- a/lib/erl_interface/test/erl_match_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/erl_match_SUITE_data/Makefile.src @@ -28,7 +28,7 @@ LIBEI = @erl_interface_eilib@ LIBFLAGS = ../all_SUITE_data/runner@obj@ \ $(LIBERL) $(LIBEI) @LIBS@ @erl_interface_sock_libs@ \ @erl_interface_threadlib@ -CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +CFLAGS = @EI_CFLAGS@ $(THR_DEFS) -I@erl_interface_include@ -I../all_SUITE_data MATCH_OBJS = match_test@obj@ match_test_decl@obj@ all: match_test@exe@ diff --git a/lib/erl_interface/test/port_call_SUITE_data/Makefile.src b/lib/erl_interface/test/port_call_SUITE_data/Makefile.src index e09e0fe175..0f97ce9f70 100644 --- a/lib/erl_interface/test/port_call_SUITE_data/Makefile.src +++ b/lib/erl_interface/test/port_call_SUITE_data/Makefile.src @@ -27,7 +27,7 @@ LIBERL = @erl_interface_lib_drv@ LIBEI = @erl_interface_eilib_drv@ SHLIB_EXTRA_LDLIBS = $(LIBERL) $(LIBEI) @erl_interface_threadlib@ -SHLIB_EXTRA_CFLAGS = -I@erl_interface_include@ @erl_interface_target_include@ -I../all_SUITE_data +SHLIB_EXTRA_CFLAGS = -I@erl_interface_include@ -I../all_SUITE_data all: port_call_drv@dll@ diff --git a/lib/erl_interface/vsn.mk b/lib/erl_interface/vsn.mk index 06ef907d6c..0ed5c07bca 100644 --- a/lib/erl_interface/vsn.mk +++ b/lib/erl_interface/vsn.mk @@ -1,2 +1,2 @@ -EI_VSN = 3.10.4 +EI_VSN = 3.11 ERL_INTERFACE_VSN = $(EI_VSN) diff --git a/lib/ftp/doc/src/notes.xml b/lib/ftp/doc/src/notes.xml index 01c1f88cf1..61da079900 100644 --- a/lib/ftp/doc/src/notes.xml +++ b/lib/ftp/doc/src/notes.xml @@ -33,7 +33,23 @@ <file>notes.xml</file> </header> - <section><title>Ftp 1.0.1</title> + <section><title>Ftp 1.0.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed timing related bug that could make ftp functions + behave badly.</p> + <p> + Own Id: OTP-15659 Aux Id: ERIERL-316 </p> + </item> + </list> + </section> + +</section> + +<section><title>Ftp 1.0.1</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/ftp/src/ftp.erl b/lib/ftp/src/ftp.erl index 40f6b53fa3..18cd8c7524 100644 --- a/lib/ftp/src/ftp.erl +++ b/lib/ftp/src/ftp.erl @@ -1065,9 +1065,9 @@ handle_call({_, {open, ip_comm, Opts, {CtrlOpts, DataPassOpts, DataActOpts}}}, F end end; -handle_call({_, {open, tls_upgrade, TLSOptions}}, From, State) -> - _ = send_ctrl_message(State, mk_cmd("AUTH TLS", [])), - activate_ctrl_connection(State), +handle_call({_, {open, tls_upgrade, TLSOptions}}, From, State0) -> + _ = send_ctrl_message(State0, mk_cmd("AUTH TLS", [])), + State = activate_ctrl_connection(State0), {noreply, State#state{client = From, caller = open, tls_options = TLSOptions}}; handle_call({_, {user, User, Password}}, From, @@ -1081,17 +1081,17 @@ handle_call({_, {user, User, Password, Acc}}, From, handle_call({_, {account, Acc}}, From, State)-> handle_user_account(Acc, State#state{client = From}); -handle_call({_, pwd}, From, #state{chunk = false} = State) -> - _ = send_ctrl_message(State, mk_cmd("PWD", [])), - activate_ctrl_connection(State), +handle_call({_, pwd}, From, #state{chunk = false} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("PWD", [])), + State = activate_ctrl_connection(State0), {noreply, State#state{client = From, caller = pwd}}; handle_call({_, lpwd}, From, #state{ldir = LDir} = State) -> {reply, {ok, LDir}, State#state{client = From}}; -handle_call({_, {cd, Dir}}, From, #state{chunk = false} = State) -> - _ = send_ctrl_message(State, mk_cmd("CWD ~s", [Dir])), - activate_ctrl_connection(State), +handle_call({_, {cd, Dir}}, From, #state{chunk = false} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("CWD ~s", [Dir])), + State = activate_ctrl_connection(State0), {noreply, State#state{client = From, caller = cd}}; handle_call({_,{lcd, Dir}}, _From, #state{ldir = LDir0} = State) -> @@ -1108,41 +1108,41 @@ handle_call({_, {dir, Len, Dir}}, {_Pid, _} = From, setup_data_connection(State#state{caller = {dir, Dir, Len}, client = From}); handle_call({_, {rename, CurrFile, NewFile}}, From, - #state{chunk = false} = State) -> - _ = send_ctrl_message(State, mk_cmd("RNFR ~s", [CurrFile])), - activate_ctrl_connection(State), + #state{chunk = false} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("RNFR ~s", [CurrFile])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {rename, NewFile}, client = From}}; handle_call({_, {delete, File}}, {_Pid, _} = From, - #state{chunk = false} = State) -> - _ = send_ctrl_message(State, mk_cmd("DELE ~s", [File])), - activate_ctrl_connection(State), + #state{chunk = false} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("DELE ~s", [File])), + State = activate_ctrl_connection(State0), {noreply, State#state{client = From}}; -handle_call({_, {mkdir, Dir}}, From, #state{chunk = false} = State) -> - _ = send_ctrl_message(State, mk_cmd("MKD ~s", [Dir])), - activate_ctrl_connection(State), +handle_call({_, {mkdir, Dir}}, From, #state{chunk = false} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("MKD ~s", [Dir])), + State = activate_ctrl_connection(State0), {noreply, State#state{client = From}}; -handle_call({_,{rmdir, Dir}}, From, #state{chunk = false} = State) -> - _ = send_ctrl_message(State, mk_cmd("RMD ~s", [Dir])), - activate_ctrl_connection(State), +handle_call({_,{rmdir, Dir}}, From, #state{chunk = false} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("RMD ~s", [Dir])), + State = activate_ctrl_connection(State0), {noreply, State#state{client = From}}; -handle_call({_,{type, Type}}, From, #state{chunk = false} = State) -> +handle_call({_,{type, Type}}, From, #state{chunk = false} = State0) -> case Type of ascii -> - _ = send_ctrl_message(State, mk_cmd("TYPE A", [])), - activate_ctrl_connection(State), + _ = send_ctrl_message(State0, mk_cmd("TYPE A", [])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = type, type = ascii, client = From}}; binary -> - _ = send_ctrl_message(State, mk_cmd("TYPE I", [])), - activate_ctrl_connection(State), + _ = send_ctrl_message(State0, mk_cmd("TYPE I", [])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = type, type = binary, client = From}}; _ -> - {reply, {error, etype}, State} + {reply, {error, etype}, State0} end; handle_call({_,{recv, RemoteFile, LocalFile}}, From, @@ -1181,8 +1181,8 @@ handle_call({_, recv_chunk}, _From, #state{chunk = true, } = State0) -> %% The ftp:recv_chunk call was the last event we waited for, finnish and clean up ?DBG("recv_chunk_closing ftp:recv_chunk, last event",[]), - activate_ctrl_connection(State0), - {reply, ok, State0#state{caller = undefined, + State = activate_ctrl_connection(State0), + {reply, ok, State#state{caller = undefined, chunk = false, client = undefined}}; @@ -1238,18 +1238,18 @@ handle_call({_, {transfer_chunk, Bin}}, _, #state{chunk = true} = State) -> handle_call({_, {transfer_chunk, _}}, _, #state{chunk = false} = State) -> {reply, {error, echunk}, State}; -handle_call({_, chunk_end}, From, #state{chunk = true} = State) -> - close_data_connection(State), - activate_ctrl_connection(State), +handle_call({_, chunk_end}, From, #state{chunk = true} = State0) -> + close_data_connection(State0), + State = activate_ctrl_connection(State0), {noreply, State#state{client = From, dsock = undefined, caller = end_chunk_transfer, chunk = false}}; handle_call({_, chunk_end}, _, #state{chunk = false} = State) -> {reply, {error, echunk}, State}; -handle_call({_, {quote, Cmd}}, From, #state{chunk = false} = State) -> - _ = send_ctrl_message(State, mk_cmd(Cmd, [])), - activate_ctrl_connection(State), +handle_call({_, {quote, Cmd}}, From, #state{chunk = false} = State0) -> + _ = send_ctrl_message(State0, mk_cmd(Cmd, [])), + State = activate_ctrl_connection(State0), {noreply, State#state{client = From, caller = quote}}; handle_call({_, _Req}, _From, #state{csock = CSock} = State) @@ -1325,38 +1325,38 @@ handle_info({Trpt, Socket, Data}, #state{dsock = {Trpt,Socket}} = State0) when T Data/binary>>}}; handle_info({Cls, Socket}, #state{dsock = {Trpt,Socket}, - caller = {recv_file, Fd}} = State) + caller = {recv_file, Fd}} = State0) when {Cls,Trpt}=={tcp_closed,tcp} ; {Cls,Trpt}=={ssl_closed,ssl} -> file_close(Fd), - progress_report({transfer_size, 0}, State), - activate_ctrl_connection(State), + progress_report({transfer_size, 0}, State0), + State = activate_ctrl_connection(State0), ?DBG("Data channel close",[]), {noreply, State#state{dsock = undefined, data = <<>>}}; handle_info({Cls, Socket}, #state{dsock = {Trpt,Socket}, client = Client, - caller = recv_chunk} = State) + caller = recv_chunk} = State0) when {Cls,Trpt}=={tcp_closed,tcp} ; {Cls,Trpt}=={ssl_closed,ssl} -> ?DBG("Data channel close recv_chunk",[]), - activate_ctrl_connection(State), + State = activate_ctrl_connection(State0), {noreply, State#state{dsock = undefined, caller = #recv_chunk_closing{dconn_closed = true, client_called_us = Client =/= undefined} }}; handle_info({Cls, Socket}, #state{dsock = {Trpt,Socket}, caller = recv_bin, - data = Data} = State) + data = Data} = State0) when {Cls,Trpt}=={tcp_closed,tcp} ; {Cls,Trpt}=={ssl_closed,ssl} -> ?DBG("Data channel close",[]), - activate_ctrl_connection(State), + State = activate_ctrl_connection(State0), {noreply, State#state{dsock = undefined, data = <<>>, caller = {recv_bin, Data}}}; handle_info({Cls, Socket}, #state{dsock = {Trpt,Socket}, data = Data, - caller = {handle_dir_result, Dir}} - = State) when {Cls,Trpt}=={tcp_closed,tcp} ; {Cls,Trpt}=={ssl_closed,ssl} -> + caller = {handle_dir_result, Dir}} + = State0) when {Cls,Trpt}=={tcp_closed,tcp} ; {Cls,Trpt}=={ssl_closed,ssl} -> ?DBG("Data channel close",[]), - activate_ctrl_connection(State), + State = activate_ctrl_connection(State0), {noreply, State#state{dsock = undefined, caller = {handle_dir_result, Dir, Data}, % data = <<?CR,?LF>>}}; @@ -1377,7 +1377,7 @@ handle_info({Transport, Socket, Data}, #state{csock = {Transport, Socket}, client = From, ctrl_data = {CtrlData, AccLines, LineStatus}} - = State) -> + = State0) -> ?DBG('--ctrl ~p ----> ~s~p~n',[Socket,<<CtrlData/binary, Data/binary>>,State]), case ftp_response:parse_lines(<<CtrlData/binary, Data/binary>>, AccLines, LineStatus) of @@ -1387,21 +1387,21 @@ handle_info({Transport, Socket, Data}, #state{csock = {Transport, Socket}, case Caller of quote -> gen_server:reply(From, string:tokens(Lines, [?CR, ?LF])), - {noreply, State#state{client = undefined, - caller = undefined, - latest_ctrl_response = Lines, - ctrl_data = {NextMsgData, [], - start}}}; + {noreply, State0#state{client = undefined, + caller = undefined, + latest_ctrl_response = Lines, + ctrl_data = {NextMsgData, [], + start}}}; _ -> ?DBG(' ...handle_ctrl_result(~p,...) ctrl_data=~p~n',[CtrlResult,{NextMsgData, [], start}]), handle_ctrl_result(CtrlResult, - State#state{latest_ctrl_response = Lines, - ctrl_data = - {NextMsgData, [], start}}) + State0#state{latest_ctrl_response = Lines, + ctrl_data = + {NextMsgData, [], start}}) end; {continue, NewCtrlData} -> ?DBG(' ...Continue... ctrl_data=~p~n',[NewCtrlData]), - activate_ctrl_connection(State), + State = activate_ctrl_connection(State0), {noreply, State#state{ctrl_data = NewCtrlData}} end; @@ -1567,19 +1567,19 @@ start_link(Opts, GenServerOptions) -> %%% Help functions to handle_call and/or handle_ctrl_result %%-------------------------------------------------------------------------- %% User handling -handle_user(User, Password, Acc, State) -> - _ = send_ctrl_message(State, mk_cmd("USER ~s", [User])), - activate_ctrl_connection(State), +handle_user(User, Password, Acc, State0) -> + _ = send_ctrl_message(State0, mk_cmd("USER ~s", [User])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {handle_user, Password, Acc}}}. -handle_user_passwd(Password, Acc, State) -> - _ = send_ctrl_message(State, mk_cmd("PASS ~s", [Password])), - activate_ctrl_connection(State), +handle_user_passwd(Password, Acc, State0) -> + _ = send_ctrl_message(State0, mk_cmd("PASS ~s", [Password])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {handle_user_passwd, Acc}}}. -handle_user_account(Acc, State) -> - _ = send_ctrl_message(State, mk_cmd("ACCT ~s", [Acc])), - activate_ctrl_connection(State), +handle_user_account(Acc, State0) -> + _ = send_ctrl_message(State0, mk_cmd("ACCT ~s", [Acc])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = handle_user_account}}. @@ -1594,9 +1594,9 @@ handle_ctrl_result({tls_upgrade, _}, #state{csock = {tcp, Socket}, ?DBG('<--ctrl ssl:connect(~p, ~p)~n~p~n',[Socket,TLSOptions,State0]), case ssl:connect(Socket, TLSOptions, Timeout) of {ok, TLSSocket} -> - State = State0#state{csock = {ssl,TLSSocket}}, - _ = send_ctrl_message(State, mk_cmd("PBSZ 0", [])), - activate_ctrl_connection(State), + State1 = State0#state{csock = {ssl,TLSSocket}}, + _ = send_ctrl_message(State1, mk_cmd("PBSZ 0", [])), + State = activate_ctrl_connection(State1), {noreply, State#state{tls_upgrading_data_connection = {true, pbsz}} }; {error, _} = Error -> gen_server:reply(From, {Error, self()}), @@ -1605,9 +1605,9 @@ handle_ctrl_result({tls_upgrade, _}, #state{csock = {tcp, Socket}, tls_upgrading_data_connection = false}} end; -handle_ctrl_result({pos_compl, _}, #state{tls_upgrading_data_connection = {true, pbsz}} = State) -> - _ = send_ctrl_message(State, mk_cmd("PROT P", [])), - activate_ctrl_connection(State), +handle_ctrl_result({pos_compl, _}, #state{tls_upgrading_data_connection = {true, pbsz}} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("PROT P", [])), + State = activate_ctrl_connection(State0), {noreply, State#state{tls_upgrading_data_connection = {true, prot}}}; handle_ctrl_result({pos_compl, _}, #state{tls_upgrading_data_connection = {true, prot}, @@ -1801,10 +1801,10 @@ handle_ctrl_result({pos_compl, _}, #state{caller = {handle_dir_result, Dir, handle_ctrl_result({pos_compl, Lines}, #state{caller = {handle_dir_data, Dir, DirData}} = - State) -> + State0) -> OldDir = pwd_result(Lines), - _ = send_ctrl_message(State, mk_cmd("CWD ~s", [Dir])), - activate_ctrl_connection(State), + _ = send_ctrl_message(State0, mk_cmd("CWD ~s", [Dir])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {handle_dir_data_second_phase, OldDir, DirData}}}; handle_ctrl_result({Status, _}, @@ -1818,9 +1818,9 @@ handle_ctrl_result(S={_Status, _}, handle_ctrl_result({pos_compl, _}, #state{caller = {handle_dir_data_second_phase, OldDir, - DirData}} = State) -> - _ = send_ctrl_message(State, mk_cmd("CWD ~s", [OldDir])), - activate_ctrl_connection(State), + DirData}} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("CWD ~s", [OldDir])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {handle_dir_data_third_phase, DirData}}}; handle_ctrl_result({Status, _}, #state{caller = {handle_dir_data_second_phase, _, _}} @@ -1840,9 +1840,9 @@ handle_ctrl_result(Status={epath, _}, #state{caller = {dir,_}} = State) -> %%-------------------------------------------------------------------------- %% File renaming handle_ctrl_result({pos_interm, _}, #state{caller = {rename, NewFile}} - = State) -> - _ = send_ctrl_message(State, mk_cmd("RNTO ~s", [NewFile])), - activate_ctrl_connection(State), + = State0) -> + _ = send_ctrl_message(State0, mk_cmd("RNTO ~s", [NewFile])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = rename_second_phase}}; handle_ctrl_result({Status, _}, @@ -1916,10 +1916,10 @@ handle_ctrl_result({pos_compl, _}, #state{client = From, %% The pos_compl was the last event we waited for, finnish and clean up ?DBG("recv_chunk_closing pos_compl, last event",[]), gen_server:reply(From, ok), - activate_ctrl_connection(State0), - {noreply, State0#state{caller = undefined, - chunk = false, - client = undefined}}; + State = activate_ctrl_connection(State0), + {noreply, State#state{caller = undefined, + chunk = false, + client = undefined}}; handle_ctrl_result({pos_compl, _}, #state{caller = #recv_chunk_closing{}=R} = State0) -> @@ -2013,71 +2013,71 @@ ctrl_result_response(_, #state{client = From} = State, ErrorMsg) -> {noreply, State#state{client = undefined, caller = undefined}}. %%-------------------------------------------------------------------------- -handle_caller(#state{caller = {dir, Dir, Len}} = State) -> +handle_caller(#state{caller = {dir, Dir, Len}} = State0) -> Cmd = case Len of short -> "NLST"; long -> "LIST" end, _ = case Dir of "" -> - send_ctrl_message(State, mk_cmd(Cmd, "")); + send_ctrl_message(State0, mk_cmd(Cmd, "")); _ -> - send_ctrl_message(State, mk_cmd(Cmd ++ " ~s", [Dir])) + send_ctrl_message(State0, mk_cmd(Cmd ++ " ~s", [Dir])) end, - activate_ctrl_connection(State), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {dir, Dir}}}; -handle_caller(#state{caller = {recv_bin, RemoteFile}} = State) -> - _ = send_ctrl_message(State, mk_cmd("RETR ~s", [RemoteFile])), - activate_ctrl_connection(State), +handle_caller(#state{caller = {recv_bin, RemoteFile}} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("RETR ~s", [RemoteFile])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = recv_bin}}; handle_caller(#state{caller = {start_chunk_transfer, Cmd, RemoteFile}} = - State) -> - _ = send_ctrl_message(State, mk_cmd("~s ~s", [Cmd, RemoteFile])), - activate_ctrl_connection(State), + State0) -> + _ = send_ctrl_message(State0, mk_cmd("~s ~s", [Cmd, RemoteFile])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = start_chunk_transfer}}; -handle_caller(#state{caller = {recv_file, RemoteFile, Fd}} = State) -> - _ = send_ctrl_message(State, mk_cmd("RETR ~s", [RemoteFile])), - activate_ctrl_connection(State), +handle_caller(#state{caller = {recv_file, RemoteFile, Fd}} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("RETR ~s", [RemoteFile])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {recv_file, Fd}}}; handle_caller(#state{caller = {transfer_file, {Cmd, LocalFile, RemoteFile}}, - ldir = LocalDir, client = From} = State) -> + ldir = LocalDir, client = From} = State0) -> case file_open(filename:absname(LocalFile, LocalDir), read) of {ok, Fd} -> - _ = send_ctrl_message(State, mk_cmd("~s ~s", [Cmd, RemoteFile])), - activate_ctrl_connection(State), + _ = send_ctrl_message(State0, mk_cmd("~s ~s", [Cmd, RemoteFile])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {transfer_file, Fd}}}; {error, _} -> gen_server:reply(From, {error, epath}), - {noreply, State#state{client = undefined, caller = undefined, - dsock = undefined}} + {noreply, State0#state{client = undefined, caller = undefined, + dsock = undefined}} end; handle_caller(#state{caller = {transfer_data, {Cmd, Bin, RemoteFile}}} = - State) -> - _ = send_ctrl_message(State, mk_cmd("~s ~s", [Cmd, RemoteFile])), - activate_ctrl_connection(State), + State0) -> + _ = send_ctrl_message(State0, mk_cmd("~s ~s", [Cmd, RemoteFile])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {transfer_data, Bin}}}. %% ----------- FTP SERVER COMMUNICATION ------------------------- %% Connect to FTP server at Host (default is TCP port 21) %% in order to establish a control connection. -setup_ctrl_connection(Host, Port, Timeout, #state{sockopts_ctrl = SockOpts} = State) -> +setup_ctrl_connection(Host, Port, Timeout, #state{sockopts_ctrl = SockOpts} = State0) -> MsTime = erlang:monotonic_time(), - case connect(Host, Port, SockOpts, Timeout, State) of + case connect(Host, Port, SockOpts, Timeout, State0) of {ok, IpFam, CSock} -> - NewState = State#state{csock = {tcp, CSock}, ipfamily = IpFam}, - activate_ctrl_connection(NewState), + State1 = State0#state{csock = {tcp, CSock}, ipfamily = IpFam}, + State = activate_ctrl_connection(State1), case Timeout - millisec_passed(MsTime) of Timeout2 when (Timeout2 >= 0) -> - {ok, NewState#state{caller = open}, Timeout2}; + {ok, State#state{caller = open}, Timeout2}; _ -> %% Oups: Simulate timeout - {ok, NewState#state{caller = open}, 0} + {ok, State#state{caller = open}, 0} end; Error -> Error @@ -2087,7 +2087,7 @@ setup_data_connection(#state{mode = active, caller = Caller, csock = CSock, sockopts_data_active = SockOpts, - ftp_extension = FtpExt} = State) -> + ftp_extension = FtpExt} = State0) -> case (catch sockname(CSock)) of {ok, {{_, _, _, _, _, _, _, _} = IP0, _}} -> IP = proplists:get_value(ip, SockOpts, IP0), @@ -2098,8 +2098,8 @@ setup_data_connection(#state{mode = active, {ok, {_, Port}} = sockname({tcp,LSock}), IpAddress = inet_parse:ntoa(IP), Cmd = mk_cmd("EPRT |2|~s|~p|", [IpAddress, Port]), - _ = send_ctrl_message(State, Cmd), - activate_ctrl_connection(State), + _ = send_ctrl_message(State0, Cmd), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {setup_data_connection, {LSock, Caller}}}}; {ok, {{_,_,_,_} = IP0, _}} -> @@ -2112,37 +2112,37 @@ setup_data_connection(#state{mode = active, false -> {IP1, IP2, IP3, IP4} = IP, {Port1, Port2} = {Port div 256, Port rem 256}, - send_ctrl_message(State, + send_ctrl_message(State0, mk_cmd("PORT ~w,~w,~w,~w,~w,~w", [IP1, IP2, IP3, IP4, Port1, Port2])); true -> IpAddress = inet_parse:ntoa(IP), Cmd = mk_cmd("EPRT |1|~s|~p|", [IpAddress, Port]), - send_ctrl_message(State, Cmd) + send_ctrl_message(State0, Cmd) end, - activate_ctrl_connection(State), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {setup_data_connection, {LSock, Caller}}}} end; setup_data_connection(#state{mode = passive, ipfamily = inet6, - caller = Caller} = State) -> - _ = send_ctrl_message(State, mk_cmd("EPSV", [])), - activate_ctrl_connection(State), + caller = Caller} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("EPSV", [])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {setup_data_connection, Caller}}}; setup_data_connection(#state{mode = passive, ipfamily = inet, caller = Caller, - ftp_extension = false} = State) -> - _ = send_ctrl_message(State, mk_cmd("PASV", [])), - activate_ctrl_connection(State), + ftp_extension = false} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("PASV", [])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {setup_data_connection, Caller}}}; setup_data_connection(#state{mode = passive, ipfamily = inet, caller = Caller, - ftp_extension = true} = State) -> - _ = send_ctrl_message(State, mk_cmd("EPSV", [])), - activate_ctrl_connection(State), + ftp_extension = true} = State0) -> + _ = send_ctrl_message(State0, mk_cmd("EPSV", [])), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = {setup_data_connection, Caller}}}. connect(Host, Port, SockOpts, Timeout, #state{ipfamily = inet = IpFam}) -> @@ -2248,14 +2248,15 @@ send_message({tcp, Socket}, Message) -> send_message({ssl, Socket}, Message) -> ssl:send(Socket, Message). -activate_ctrl_connection(#state{csock = CSock, ctrl_data = {<<>>, _, _}}) -> - activate_connection(CSock); -activate_ctrl_connection(#state{csock = CSock}) -> +activate_ctrl_connection(#state{csock = CSock, ctrl_data = {<<>>, _, _}} = State) -> + activate_connection(CSock), + State; +activate_ctrl_connection(#state{csock = CSock} = State0) -> activate_connection(CSock), %% We have already received at least part of the next control message, %% that has been saved in ctrl_data, process this first. - self() ! {socket_type(CSock), unwrap_socket(CSock), <<>>}, - ok. + {noreply, State} = handle_info({socket_type(CSock), unwrap_socket(CSock), <<>>}, State0), + State. activate_data_connection(#state{dsock = DSock} = State) -> activate_connection(DSock), @@ -2290,22 +2291,22 @@ close_connection({ssl, Socket}) -> ignore_return_value( ssl:close(Socket) ). %% ------------ FILE HANDLING ---------------------------------------- send_file(#state{tls_upgrading_data_connection = {true, CTRL, _}} = State, Fd) -> {noreply, State#state{tls_upgrading_data_connection = {true, CTRL, ?MODULE, send_file, Fd}}}; -send_file(State, Fd) -> +send_file(State0, Fd) -> case file_read(Fd) of {ok, N, Bin} when N > 0 -> - send_data_message(State, Bin), - progress_report({binary, Bin}, State), - send_file(State, Fd); + send_data_message(State0, Bin), + progress_report({binary, Bin}, State0), + send_file(State0, Fd); {ok, _, _} -> file_close(Fd), - close_data_connection(State), - progress_report({transfer_size, 0}, State), - activate_ctrl_connection(State), + close_data_connection(State0), + progress_report({transfer_size, 0}, State0), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = transfer_file_second_phase, dsock = undefined}}; {error, Reason} -> - gen_server:reply(State#state.client, {error, Reason}), - {stop, normal, State#state{client = undefined}} + gen_server:reply(State0#state.client, {error, Reason}), + {stop, normal, State0#state{client = undefined}} end. file_open(File, Option) -> @@ -2347,10 +2348,10 @@ cast(GenServer, Msg) -> send_bin(#state{tls_upgrading_data_connection = {true, CTRL, _}} = State, Bin) -> State#state{tls_upgrading_data_connection = {true, CTRL, ?MODULE, send_bin, Bin}}; -send_bin(State, Bin) -> - send_data_message(State, Bin), - close_data_connection(State), - activate_ctrl_connection(State), +send_bin(State0, Bin) -> + send_data_message(State0, Bin), + close_data_connection(State0), + State = activate_ctrl_connection(State0), {noreply, State#state{caller = transfer_data_second_phase, dsock = undefined}}. diff --git a/lib/ftp/test/ftp_SUITE.erl b/lib/ftp/test/ftp_SUITE.erl index 7c87d5cbdb..0b070ee8cb 100644 --- a/lib/ftp/test/ftp_SUITE.erl +++ b/lib/ftp/test/ftp_SUITE.erl @@ -96,6 +96,7 @@ ftp_tests()-> recv_chunk, recv_chunk_twice, recv_chunk_three_times, + recv_chunk_delay, type, quote, error_elogin, @@ -669,9 +670,9 @@ recv_chunk(Config0) -> Contents = list_to_binary( lists:duplicate(1000, lists:seq(0,255)) ), Config = set_state([reset, {mkfile,File,Contents}], Config0), Pid = proplists:get_value(ftp, Config), - {{error, "ftp:recv_chunk_start/2 not called"},_} = recv_chunk(Pid, <<>>), + {error, "ftp:recv_chunk_start/2 not called"} = do_recv_chunk(Pid), ok = ftp:recv_chunk_start(Pid, id2ftp(File,Config)), - {ok, ReceivedContents, _Ncunks} = recv_chunk(Pid, <<>>), + {ok, ReceivedContents} = do_recv_chunk(Pid), find_diff(ReceivedContents, Contents). recv_chunk_twice() -> @@ -683,11 +684,11 @@ recv_chunk_twice(Config0) -> Contents2 = crypto:strong_rand_bytes(1200), Config = set_state([reset, {mkfile,File1,Contents1}, {mkfile,File2,Contents2}], Config0), Pid = proplists:get_value(ftp, Config), - {{error, "ftp:recv_chunk_start/2 not called"},_} = recv_chunk(Pid, <<>>), + {error, "ftp:recv_chunk_start/2 not called"} = do_recv_chunk(Pid), ok = ftp:recv_chunk_start(Pid, id2ftp(File1,Config)), - {ok, ReceivedContents1, _Ncunks1} = recv_chunk(Pid, <<>>), + {ok, ReceivedContents1} = do_recv_chunk(Pid), ok = ftp:recv_chunk_start(Pid, id2ftp(File2,Config)), - {ok, ReceivedContents2, _Ncunks2} = recv_chunk(Pid, <<>>), + {ok, ReceivedContents2} = do_recv_chunk(Pid), find_diff(ReceivedContents1, Contents1), find_diff(ReceivedContents2, Contents2). @@ -704,46 +705,56 @@ recv_chunk_three_times(Config0) -> Config = set_state([reset, {mkfile,File1,Contents1}, {mkfile,File2,Contents2}, {mkfile,File3,Contents3}], Config0), Pid = proplists:get_value(ftp, Config), - {{error, "ftp:recv_chunk_start/2 not called"},_} = recv_chunk(Pid, <<>>), + {error, "ftp:recv_chunk_start/2 not called"} = do_recv_chunk(Pid), + ok = ftp:recv_chunk_start(Pid, id2ftp(File3,Config)), + {ok, ReceivedContents3} = do_recv_chunk(Pid), + ok = ftp:recv_chunk_start(Pid, id2ftp(File1,Config)), - {ok, ReceivedContents1, Nchunks1} = recv_chunk(Pid, <<>>), + {ok, ReceivedContents1} = do_recv_chunk(Pid), ok = ftp:recv_chunk_start(Pid, id2ftp(File2,Config)), - {ok, ReceivedContents2, _Nchunks2} = recv_chunk(Pid, <<>>), - - ok = ftp:recv_chunk_start(Pid, id2ftp(File3,Config)), - {ok, ReceivedContents3, _Nchunks3} = recv_chunk(Pid, <<>>, 10000, 0, Nchunks1), + {ok, ReceivedContents2} = do_recv_chunk(Pid), find_diff(ReceivedContents1, Contents1), find_diff(ReceivedContents2, Contents2), find_diff(ReceivedContents3, Contents3). - +do_recv_chunk(Pid) -> + recv_chunk(Pid, <<>>). recv_chunk(Pid, Acc) -> - recv_chunk(Pid, Acc, 0, 0, undefined). - - - -%% ExpectNchunks :: integer() | undefined -recv_chunk(Pid, Acc, DelayMilliSec, N, ExpectNchunks) when N+1 < ExpectNchunks -> - %% for all I in integer(), I < undefined - recv_chunk1(Pid, Acc, DelayMilliSec, N, ExpectNchunks); - -recv_chunk(Pid, Acc, DelayMilliSec, N, ExpectNchunks) -> - %% N >= ExpectNchunks-1 - timer:sleep(DelayMilliSec), - recv_chunk1(Pid, Acc, DelayMilliSec, N, ExpectNchunks). + case ftp:recv_chunk(Pid) of + ok -> + {ok, Acc}; + {ok, Bin} -> + recv_chunk(Pid, <<Acc/binary, Bin/binary>>); + Error -> + Error + end. +recv_chunk_delay(Config0) when is_list(Config0) -> + File1 = "big_file1.txt", + Contents = list_to_binary(lists:duplicate(1000, lists:seq(0,255))), + Config = set_state([reset, {mkfile,File1,Contents}], Config0), + Pid = proplists:get_value(ftp, Config), + ok = ftp:recv_chunk_start(Pid, id2ftp(File1,Config)), + {ok, ReceivedContents} = delay_recv_chunk(Pid), + find_diff(ReceivedContents, Contents). -recv_chunk1(Pid, Acc, DelayMilliSec, N, ExpectNchunks) -> - ct:log("Call ftp:recv_chunk",[]), +delay_recv_chunk(Pid) -> + delay_recv_chunk(Pid, <<>>). +delay_recv_chunk(Pid, Acc) -> + ct:pal("Recived size ~p", [byte_size(Acc)]), case ftp:recv_chunk(Pid) of - ok -> {ok, Acc, N}; - {ok, Bin} -> recv_chunk(Pid, <<Acc/binary, Bin/binary>>, DelayMilliSec, N+1, ExpectNchunks); - Error -> {Error, N} - end. + ok -> + {ok, Acc}; + {ok, Bin} -> + ct:sleep(100), + delay_recv_chunk(Pid, <<Acc/binary, Bin/binary>>); + Error -> + Error + end. %%------------------------------------------------------------------------- type() -> diff --git a/lib/ftp/vsn.mk b/lib/ftp/vsn.mk index d5d6c45b28..9f14658099 100644 --- a/lib/ftp/vsn.mk +++ b/lib/ftp/vsn.mk @@ -19,6 +19,6 @@ # %CopyrightEnd% APPLICATION = ftp -FTP_VSN = 1.0.1 +FTP_VSN = 1.0.2 PRE_VSN = APP_VSN = "$(APPLICATION)-$(FTP_VSN)$(PRE_VSN)" diff --git a/lib/hipe/doc/src/notes.xml b/lib/hipe/doc/src/notes.xml index e9cdf42018..9a803cb9df 100644 --- a/lib/hipe/doc/src/notes.xml +++ b/lib/hipe/doc/src/notes.xml @@ -31,6 +31,21 @@ </header> <p>This document describes the changes made to HiPE.</p> +<section><title>Hipe 3.18.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug in the handling of the <c>Key</c> argument + of <c>lists:{keysearch, keyfind, keymember}</c>. </p> + <p> + Own Id: OTP-15570</p> + </item> + </list> + </section> + +</section> + <section><title>Hipe 3.18.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/hipe/vsn.mk b/lib/hipe/vsn.mk index 12d621bf01..39565d721f 100644 --- a/lib/hipe/vsn.mk +++ b/lib/hipe/vsn.mk @@ -1 +1 @@ -HIPE_VSN = 3.18.2 +HIPE_VSN = 3.18.3 diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml index 31dae6317e..91dd9cd6ed 100644 --- a/lib/inets/doc/src/notes.xml +++ b/lib/inets/doc/src/notes.xml @@ -33,7 +33,43 @@ <file>notes.xml</file> </header> - <section><title>Inets 7.0.5</title> + <section><title>Inets 7.0.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix the internal handling of the option + erl_script_timeout in httpd. When httpd was started with + explicit erl_script_timeout, the value of the option was + converted to milliseconds before storage. Subsequent + calls to httpd:info/1 returned the input value multiplied + by 1000.</p> + <p> + This change fixes the handing of erl_script_timeout by + storing the timeout in seconds and converting to + milliseconds before usage.</p> + <p> + Own Id: OTP-15669 Aux Id: ERIERL-321 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Enhance documentation</p> + <p> + Own Id: OTP-15508 Aux Id: ERL-816 </p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 7.0.5</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/inets/src/http_server/mod_esi.erl b/lib/inets/src/http_server/mod_esi.erl index 443b7ee564..f495f12f03 100644 --- a/lib/inets/src/http_server/mod_esi.erl +++ b/lib/inets/src/http_server/mod_esi.erl @@ -33,7 +33,7 @@ -include("httpd_internal.hrl"). -define(VMODULE,"ESI"). --define(DEFAULT_ERL_TIMEOUT,15000). +-define(DEFAULT_ERL_TIMEOUT,15). %%%========================================================================= @@ -174,7 +174,7 @@ store({erl_script_alias, Value}, _) -> {error, {wrong_type, {erl_script_alias, Value}}}; store({erl_script_timeout, TimeoutSec}, _) when is_integer(TimeoutSec) andalso (TimeoutSec >= 0) -> - {ok, {erl_script_timeout, TimeoutSec * 1000}}; + {ok, {erl_script_timeout, TimeoutSec}}; store({erl_script_timeout, Value}, _) -> {error, {wrong_type, {erl_script_timeout, Value}}}; store({erl_script_nocache, Value} = Conf, _) @@ -500,7 +500,7 @@ kill_esi_delivery_process(Pid) -> erl_script_timeout(Db) -> - httpd_util:lookup(Db, erl_script_timeout, ?DEFAULT_ERL_TIMEOUT). + httpd_util:lookup(Db, erl_script_timeout, ?DEFAULT_ERL_TIMEOUT * 1000). script_elements(FuncAndInput, Input) -> case input_type(FuncAndInput) of diff --git a/lib/inets/test/httpd_SUITE.erl b/lib/inets/test/httpd_SUITE.erl index 5b6740fba3..fcb9ad7905 100644 --- a/lib/inets/test/httpd_SUITE.erl +++ b/lib/inets/test/httpd_SUITE.erl @@ -78,7 +78,8 @@ all() -> {group, http_rel_path_script_alias}, {group, http_not_sup}, {group, https_not_sup}, - mime_types_format + mime_types_format, + erl_script_timeout_option ]. groups() -> @@ -1777,6 +1778,18 @@ mime_types_format(Config) when is_list(Config) -> {"hqx","application/mac-binhex40"}]} = httpd_conf:load_mime_types(MimeTypes). +erl_script_timeout_option(Config) when is_list(Config) -> + inets:start(), + {ok, Pid} = inets:start(httpd, [{erl_script_timeout, 215}, + {server_name, "test"}, + {port,0}, + {server_root, "."}, + {document_root, "."}]), + Info = httpd:info(Pid), + 215 = proplists:get_value(erl_script_timeout, Info), + inets:stop(). + + %%-------------------------------------------------------------------- %% Internal functions ----------------------------------- %%-------------------------------------------------------------------- diff --git a/lib/inets/vsn.mk b/lib/inets/vsn.mk index 921161dce1..b7ddf39ebd 100644 --- a/lib/inets/vsn.mk +++ b/lib/inets/vsn.mk @@ -19,6 +19,6 @@ # %CopyrightEnd% APPLICATION = inets -INETS_VSN = 7.0.5 +INETS_VSN = 7.0.6 PRE_VSN = APP_VSN = "$(APPLICATION)-$(INETS_VSN)$(PRE_VSN)" diff --git a/lib/jinterface/doc/src/Makefile b/lib/jinterface/doc/src/Makefile index d80bb38ec1..f5cba9d074 100644 --- a/lib/jinterface/doc/src/Makefile +++ b/lib/jinterface/doc/src/Makefile @@ -137,6 +137,7 @@ clean clean_docs: jdoc:$(JAVA_SRC_FILES) (cd ../../java_src;$(JAVADOC) -sourcepath . -d $(JAVADOC_DEST) \ -windowtitle $(JAVADOC_TITLE) $(JAVADOC_PKGS)) + touch jdoc man: @@ -152,15 +153,8 @@ include $(ERL_TOP)/make/otp_release_targets.mk release_docs_spec: docs $(INSTALL_DIR) "$(RELSYSDIR)/doc/pdf" $(INSTALL_DATA) $(TOP_PDF_FILE) "$(RELSYSDIR)/doc/pdf" - $(INSTALL_DIR) "$(RELSYSDIR)/doc/html" - $(INSTALL_DIR) "$(RELSYSDIR)/doc/html/java/$(JAVA_PKG_PATH)" $(INSTALL_DATA) $(INFO_FILE) "$(RELSYSDIR)" - ($(CP) -rf ../html "$(RELSYSDIR)/doc") - -# $(INSTALL_DATA) $(GIF_FILES) $(EXTRA_FILES) $(HTML_FILES) \ -# "$(RELSYSDIR)/doc/html" -# $(INSTALL_DATA) $(JAVA_EXTRA_FILES) "$(RELSYSDIR)/doc/html/java" -# $(INSTALL_DATA) $(TOP_HTML_FILES) "$(RELSYSDIR)/doc" + $(INSTALL_DIR_DATA) $(HTMLDIR) "$(RELSYSDIR)/doc/html" release_spec: diff --git a/lib/kernel/doc/src/application.xml b/lib/kernel/doc/src/application.xml index 5170502581..83a83ebad2 100644 --- a/lib/kernel/doc/src/application.xml +++ b/lib/kernel/doc/src/application.xml @@ -238,8 +238,8 @@ Nodes = [cp1@cave, {cp2@cave, cp3@cave}]</code> </desc> </func> <func> - <name name="set_env" arity="1" since="OTP @OTP-15642@"/> - <name name="set_env" arity="2" since="OTP @OTP-15642@"/> + <name name="set_env" arity="1" since="OTP 21.3"/> + <name name="set_env" arity="2" since="OTP 21.3"/> <fsummary>Sets the configuration parameters of multiple applications.</fsummary> <desc> <p>Sets the configuration <c><anno>Config</anno></c> for multiple diff --git a/lib/kernel/doc/src/kernel_app.xml b/lib/kernel/doc/src/kernel_app.xml index 15dbdb47dc..7f9609d5c1 100644 --- a/lib/kernel/doc/src/kernel_app.xml +++ b/lib/kernel/doc/src/kernel_app.xml @@ -369,6 +369,14 @@ MaxT = TickTime + TickTime / 4</code> performed. This option ensures that <c>global</c> is synchronized.</p> </item> + <tag><c>start_distribution = true | false</c></tag> + <item> + <p>Starts all distribution services, such as <c>rpc</c>, + <c>global</c>, and <c>net_kernel</c> if the parameter is + <c>true</c>. This parameter is to be set to <c>false</c> + for systems who want to disable all distribution functionality.</p> + <p>Defaults to <c>true</c>.</p> + </item> <tag><c>start_dist_ac = true | false</c></tag> <item> <p>Starts the <c>dist_ac</c> server if the parameter is @@ -510,11 +518,13 @@ MaxT = TickTime + TickTime / 4</code> parameters for Logger are not set.</p> <taglist> <tag><c>error_logger</c></tag> - <item>Replaced by setting the type of the default - <seealso marker="logger_std_h#type"><c>logger_std_h</c></seealso> - to the same value. Example: + <item>Replaced by setting the <seealso + marker="logger_std_h#type"><c>type</c></seealso>, and possibly + <seealso marker="logger_std_h#file"><c>file</c></seealso> and + <seealso marker="logger_std_h#modes"><c>modes</c></seealso> + parameters of the default <c>logger_std_h</c> handler. Example: <code type="none"> -erl -kernel logger '[{handler,default,logger_std_h,#{config=>#{type=>{file,"/tmp/erlang.log"}}}}]' +erl -kernel logger '[{handler,default,logger_std_h,#{config=>#{file=>"/tmp/erlang.log"}}}]' </code> </item> <tag><c>error_logger_format_depth</c></tag> diff --git a/lib/kernel/doc/src/logger.xml b/lib/kernel/doc/src/logger.xml index e6448e144e..2a060ce1f9 100644 --- a/lib/kernel/doc/src/logger.xml +++ b/lib/kernel/doc/src/logger.xml @@ -66,7 +66,7 @@ logger:error("error happened because: ~p", [Reason]). % Without macro [{kernel, [{logger, [{handler, default, logger_std_h, - #{config => #{type => {file, "path/to/file.log"}}}}]}]}]. + #{config => #{file => "path/to/file.log"}}}]}]}]. </code> <p> For more information about: @@ -1119,7 +1119,7 @@ logger:set_proxy_config(maps:merge(Old, Config)). </func> <func> - <name name="timestamp" arity="0" since="OTP @OTP-15625@"/> + <name name="timestamp" arity="0" since="OTP 21.3"/> <fsummary>Return a timestamp to insert in meta data for a log event.</fsummary> <desc> @@ -1351,5 +1351,3 @@ logger:set_proxy_config(maps:merge(Old, Config)). </p> </section> </erlref> - - diff --git a/lib/kernel/doc/src/logger_chapter.xml b/lib/kernel/doc/src/logger_chapter.xml index 03b9edcf8f..8458ffa042 100644 --- a/lib/kernel/doc/src/logger_chapter.xml +++ b/lib/kernel/doc/src/logger_chapter.xml @@ -801,7 +801,7 @@ logger:debug(#{got => connection_request, id => Id, state => State}, [{kernel, [{logger, [{handler, default, logger_std_h, % {handler, HandlerId, Module, - #{config => #{type => {file,"log/erlang.log"}}}} % Config} + #{config => #{file => "log/erlang.log"}}} % Config} ]}]}]. </code> <p>Modify the default handler to print each log event as a @@ -831,10 +831,10 @@ logger:debug(#{got => connection_request, id => Id, state => State}, [{logger, [{handler, default, logger_std_h, #{level => error, - config => #{type => {file, "log/erlang.log"}}}}, + config => #{file => "log/erlang.log"}}}, {handler, info, logger_std_h, #{level => debug, - config => #{type => {file, "log/debug.log"}}}} + config => #{file => "log/debug.log"}}} ]}]}]. </code> </section> @@ -1004,10 +1004,10 @@ ok</pre> <p>Then, add a new handler which prints to file. You can use the handler module <seealso marker="logger_std_h"><c>logger_std_h</c></seealso>, - and specify type <c>{file,File}</c>.:</p> + and configure it to log to file:</p> <pre> -4> <input>Config = #{config => #{type => {file,"./info.log"}}, level => info}.</input> -#{config => #{type => {file,"./info.log"}},level => info} +4> <input>Config = #{config => #{file => "./info.log"}, level => info}.</input> +#{config => #{file => "./info.log"},level => info} 5> <input>logger:add_handler(myhandler, logger_std_h, Config).</input> ok</pre> <p>Since <c>filter_default</c> defaults to <c>log</c>, this @@ -1246,7 +1246,7 @@ do_log(Fd, LogEvent, #{formatter := {FModule, FConfig}}) -> <p>A configuration example:</p> <code type="none"> logger:add_handler(my_standard_h, logger_std_h, - #{config => #{type => {file,"./system_info.log"}, + #{config => #{file => "./system_info.log", sync_mode_qlen => 100, drop_mode_qlen => 1000, flush_qlen => 2000}}). diff --git a/lib/kernel/doc/src/logger_std_h.xml b/lib/kernel/doc/src/logger_std_h.xml index fcd180abd6..5ed1a2f210 100644 --- a/lib/kernel/doc/src/logger_std_h.xml +++ b/lib/kernel/doc/src/logger_std_h.xml @@ -55,30 +55,105 @@ is stored in a sub map with the key <c>config</c>, and can contain the following parameters:</p> <taglist> - <tag><marker id="type"/><c>type</c></tag> + <tag><marker id="type"/><c>type = standard_io | standard_error | file</c></tag> <item> - <p>This has the value <c>standard_io</c>, <c>standard_error</c>, - <c>{file,LogFileName}</c>, or <c>{file,LogFileName,LogFileOpts}</c>.</p> - <p>If <c>LogFileOpts</c> is specified, it replaces the default - list of options used when opening the log file. The default - list is <c>[raw,append,delayed_write]</c>. One reason to do - so can be to change <c>append</c> to, for - example, <c>write</c>, ensuring that the old log is - truncated when a node is restarted. See the reference manual - for <seealso marker="file#open-2"><c>file:open/2</c></seealso> - for more information about file options.</p> + <p>Specifies the log destination.</p> + <p>The value is set when the handler is added, and it can not + be changed in runtime.</p> + <p>Defaults to <c>standard_io</c>, unless + parameter <seealso marker="#file"><c>file</c></seealso> is + given, in which case it defaults to <c>file</c>.</p> + </item> + <tag><marker id="file"/><c>file = </c><seealso marker="file#type-filename"><c>file:filename()</c></seealso></tag> + <item> + <p>This specifies the name of the log file when the handler is + of type <c>file</c>.</p> + <p>The value is set when the handler is added, and it can not + be changed in runtime.</p> + <p>Defaults to the same name as the handler identity, in the + current directory.</p> + </item> + <tag><marker id="modes"/><c>modes = [</c><seealso marker="file#type-mode"><c>file:mode()</c></seealso><c>]</c></tag> + <item> + <p>This specifies the file modes to use when opening the log + file, + see <seealso marker="file#open-2"><c>file:open/2</c></seealso>. + If <c>modes</c> are not specified, the default list used + is <c>[raw,append,delayed_write]</c>. If <c>modes</c> are + specified, the list replaces the default modes list with the + following adjustments:</p> + <list> + <item> + If <c>raw</c> is not found in the list, it is added. + </item> + <item> + If none of <c>write</c>, <c>append</c> or <c>exclusive</c> is + found in the list, <c>append</c> is added.</item> + <item>If none of <c>delayed_write</c> + or <c>{delayed_write,Size,Delay}</c> is found in the + list, <c>delayed_write</c> is added.</item> + </list> <p>Log files are always UTF-8 encoded. The encoding can not be - changed by setting the option <c>{encoding,Encoding}</c> - in <c>LogFileOpts</c>.</p> - <p>Notice that the standard handler does not have support for - circular logging. Use the disk_log handler, - <seealso marker="logger_disk_log_h"><c>logger_disk_log_h</c></seealso>, - for this.</p> + changed by setting the mode <c>{encoding,Encoding}</c>.</p> <p>The value is set when the handler is added, and it can not be changed in runtime.</p> - <p>Defaults to <c>standard_io</c>.</p> + <p>Defaults to <c>[raw,append,delayed_write]</c>.</p> + </item> + <tag><marker id="max_no_bytes"/><c>max_no_bytes = pos_integer() | infinity</c></tag> + <item> + <p>This parameter specifies if the log file should be rotated + or not. The value <c>infinity</c> means the log file will + grow indefinitely, while an integer value specifies at which + file size (bytes) the file is rotated.</p> + <p>Defaults to <c>infinity</c>.</p> + </item> + <tag><marker id="max_no_files"/><c>max_no_files = non_neg_integer()</c></tag> + <item> + <p>This parameter specifies the number of rotated log file + archives to keep. This has meaning only + if <seealso marker="#max_no_bytes"><c>max_no_bytes</c></seealso> + is set to an integer value.</p> + <p>The log archives are + named <c>FileName.0</c>, <c>FileName.1</c>, + ... <c>FileName.N</c>, where <c>FileName</c> is the name of + the current log file. <c>FileName.0</c> is the newest of the + archives. The maximum value for <c>N</c> is the value + of <c>max_no_files</c> minus 1.</p> + <p>Notice that setting this value to <c>0</c> does not turn of + rotation. It only specifies that no archives are kept.</p> + <p>Defaults to <c>0</c>.</p> + </item> + <tag><marker id="compress_on_rotate"/><c>compress_on_rotate = boolean()</c></tag> + <item> + <p>This parameter specifies if the rotated log file archives + shall be compressed or not. If set to <c>true</c>, all + archives are compressed with <c>gzip</c>, and renamed + to <c>FileName.N.gz</c></p> + <p><c>compress_on_rotate</c> has no meaning if <seealso + marker="#max_no_bytes"><c>max_no_bytes</c></seealso> has the + value <c>infinity</c>.</p> + <p>Defaults to <c>false</c>.</p> + </item> + <tag><marker id="file_check"/><c>file_check = non_neg_integer()</c></tag> + <item> + <p>When <c>logger_std_h</c> logs to a file, it reads the file + information of the log file prior to each write + operation. This is to make sure the file still exists and + has the same inode as when it was opened. This implies some + performance loss, but ensures that no log events are lost in + the case when the file has been removed or renamed by an + external actor.</p> + <p>In order to allow minimizing the performance loss, the + <c>file_check</c> parameter can be set to a positive integer + value, <c>N</c>. The handler will then skip reading the file + information prior to writing, as long as no more + than <c>N</c> milliseconds have passed since it was last + read.</p> + <p>Notice that the risk of loosing log events grows when + the <c>file_check</c> value grows.</p> + <p>Defaults to 0.</p> </item> - <tag><c>filesync_repeat_interval</c></tag> + <tag><c>filesync_repeat_interval = pos_integer() | no_repeat</c></tag> <item> <p>This value, in milliseconds, specifies how often the handler does a file sync operation to write buffered data to disk. The handler attempts @@ -97,12 +172,13 @@ standard handler and the disk_log handler, and are documented in the <seealso marker="logger_chapter#overload_protection"><c>User's Guide</c> </seealso>.</p> - <p>Notice that if changing the configuration of the handler in runtime, - the <c>type</c> parameter must not be modified.</p> + <p>Notice that if changing the configuration of the handler in + runtime, the <c>type</c>, <c>file</c>, or <c>modes</c> parameters + must not be modified.</p> <p>Example of adding a standard handler:</p> <code type="none"> logger:add_handler(my_standard_h, logger_std_h, - #{config => #{type => {file,"./system_info.log"}, + #{config => #{file => "./system_info.log", filesync_repeat_interval => 1000}}). </code> <p>To set the default handler, that starts initially with @@ -110,7 +186,7 @@ logger:add_handler(my_standard_h, logger_std_h, change the Kernel default logger configuration. Example:</p> <code type="none"> erl -kernel logger '[{handler,default,logger_std_h, - #{config => #{type => {file,"./log.log"}}}}]' + #{config => #{file => "./log.log"}}}]' </code> <p>An example of how to replace the standard handler with a disk_log handler at startup is found in the diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index 021ecfa40d..0c187eb19f 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -31,6 +31,204 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 6.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + If for example the <c>/etc/hosts</c> did not come into + existence until after the kernel application had started, + its content was never read. This bug has now been + corrected.</p> + <p> + Own Id: OTP-14702 Aux Id: PR-2066 </p> + </item> + <item> + <p> + Fix bug where doing <c>seq_trace:reset_trace()</c> while + another process was doing a garbage collection could + cause the run-time system to segfault.</p> + <p> + Own Id: OTP-15490</p> + </item> + <item> + <p> + Fix <c>erl_epmd:port_please</c> spec to include + <c>atom()</c> and <c>string()</c>.</p> + <p> + Own Id: OTP-15557 Aux Id: PR-2117 </p> + </item> + <item> + <p> + The Logger handler logger_std_h now keeps track of the + inode of its log file in order to re-open the file if the + inode changes. This may happen, for instance, if the log + file is opened and saved by an editor.</p> + <p> + Own Id: OTP-15578 Aux Id: ERL-850 </p> + </item> + <item> + <p> + When user specific file modes are given to the logger + handler <c>logger_std_h</c>, they were earlier accepted + without any control. This is now changes, so Logger will + adjust the file modes as follows:</p> + <p> + - If <c>raw</c> is not found in the list, it is + added.<br/> - If none of <c>write</c>, <c>append</c> or + <c>exclusive</c> are found in the list, <c>append</c> is + added.<br/> - If none of <c>delayed_write</c> or + <c>{delayed_write,Size,Delay}</c> are found in the list, + <c>delayed_write</c> is added.</p> + <p> + Own Id: OTP-15602</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The standard logger handler, <c>logger_std_h</c>, now has + a new internal feature for log rotation. The rotation + scheme is as follows:</p> + <p> + The log file to which the handler currently writes always + has the same name, i.e. the name which is configured for + the handler. The archived files have the same name, but + with extension ".N", where N is an integer. The newest + archive has extension ".0", and the oldest archive has + the highest number. </p> + <p> + The size at which the log file is rotated, and the number + of archive files that are kept, is specified with the + handler specific configuration parameters + <c>max_no_bytes</c> and <c>max_no_files</c> respectively. </p> + <p> + Archives can be compressed, in which case they get a + ".gz" file extension after the integer. Compression is + specified with the handler specific configuration + parameter <c>compress_on_rotate</c>.</p> + <p> + Own Id: OTP-15479</p> + </item> + <item> + <p> + The new functions <c>logger:i/0</c> and <c>logger:i/1</c> + are added. These provide the same information as + <c>logger:get_config/0</c> and other + <c>logger:get_*_config</c> functions, but the information + is more logically sorted and more readable.</p> + <p> + Own Id: OTP-15600</p> + </item> + <item> + <p> + Logger is now protected against overload due to massive + amounts of log events from the emulator or from remote + nodes.</p> + <p> + Own Id: OTP-15601</p> + </item> + <item> + <p> + Logger now uses os:system_time/1 instead of + erlang:system_time/1 to generate log event timestamps.</p> + <p> + Own Id: OTP-15625</p> + </item> + <item> + <p> + Add functions <c>application:set_env/1,2</c> and + <c>application:set_env/2</c>. These take a list of + application configuration parameters, and the behaviour + is equivalent to calling <c>application:set_env/4</c> + individually for each application/key combination, except + it is more efficient.</p> + <p> + <c>set_env/1,2</c> warns about duplicated applications or + keys. The warning is also emitted during boot, if + applications or keys are duplicated within one + configuration file, e.g. sys.config.</p> + <p> + Own Id: OTP-15642 Aux Id: PR-2164 </p> + </item> + <item> + <p> + Handler specific configuration parameters for the + standard handler <c>logger_std_h</c> are changed to be + more intuitive and more similar to the disk_log handler.</p> + <p> + Earlier there was only one parameter, <c>type</c>, which + could have the values <c>standard_io</c>, + <c>standard_error</c>, <c>{file,FileName}</c> or + <c>{file,FileName,Modes}</c>.</p> + <p> + This is now changed, so the following parameters are + allowed:</p> + <p> + <c>type = standard_io | standard_error | file</c><br/> + <c>file = file:filename()</c><br/> <c>modes = + [file:mode()]</c></p> + <p> + All parameters are optional. <c>type</c> defaults to + <c>standard_io</c>, unless a file name is given, in which + case it defaults to <c>file</c>. If <c>type</c> is set to + <c>file</c>, the file name defaults to the same as the + handler id.</p> + <p> + The potential incompatibility is that + <c>logger:get_config/0</c> and + <c>logger:get_handler_config/1</c> now returns the new + parameters, even if the configuration was set with the + old variant, e.g. <c>#{type=>{file,FileName}}</c>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15662</p> + </item> + <item> + <p> + The new configuration parameter <c>file_check</c> is + added to the Logger handler <c>logger_std_h</c>. This + parameter specifies how long (in milliseconds) the + handler may wait before checking if the log file still + exists and the inode is the same as when it was opened.</p> + <p> + The default value is 0, which means that this check is + done prior to each write operation. Setting a higher + number may improve performance, but adds the risk of + loosing log events.</p> + <p> + Own Id: OTP-15663</p> + </item> + </list> + </section> + +</section> + +<section><title>Kernel 6.2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Setting the <c>recbuf</c> size of an inet socket the + <c>buffer</c> is also automatically increased. Fix a bug + where the auto adjustment of inet buffer size would be + triggered even if an explicit inet buffer size had + already been set.</p> + <p> + Own Id: OTP-15651 Aux Id: ERIERL-304 </p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 6.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/src/kernel.app.src b/lib/kernel/src/kernel.app.src index 4b48f6cd1d..8fe6bdd1ca 100644 --- a/lib/kernel/src/kernel.app.src +++ b/lib/kernel/src/kernel.app.src @@ -147,6 +147,6 @@ {logger_sasl_compatible, false} ]}, {mod, {kernel, []}}, - {runtime_dependencies, ["erts-10.1", "stdlib-3.5", "sasl-3.0"]} + {runtime_dependencies, ["erts-10.2.5", "stdlib-3.5", "sasl-3.0"]} ] }. diff --git a/lib/kernel/src/kernel.appup.src b/lib/kernel/src/kernel.appup.src index ccf0a82ced..8fa3f5c588 100644 --- a/lib/kernel/src/kernel.appup.src +++ b/lib/kernel/src/kernel.appup.src @@ -40,7 +40,10 @@ {<<"^6\\.0\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^6\\.1$">>,[restart_new_emulator]}, {<<"^6\\.1\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, - {<<"^6\\.1\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], + {<<"^6\\.1\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, + {<<"^6\\.2$">>,[restart_new_emulator]}, + {<<"^6\\.2\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^6\\.2\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], [{<<"^5\\.3$">>,[restart_new_emulator]}, {<<"^5\\.3\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^5\\.3\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, @@ -54,4 +57,7 @@ {<<"^6\\.0\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^6\\.1$">>,[restart_new_emulator]}, {<<"^6\\.1\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, - {<<"^6\\.1\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. + {<<"^6\\.1\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, + {<<"^6\\.2$">>,[restart_new_emulator]}, + {<<"^6\\.2\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^6\\.2\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. diff --git a/lib/kernel/src/kernel.erl b/lib/kernel/src/kernel.erl index c68d04e279..111d103df2 100644 --- a/lib/kernel/src/kernel.erl +++ b/lib/kernel/src/kernel.erl @@ -145,26 +145,11 @@ init([]) -> {ok, {SupFlags, [Code, File, StdError, User, LoggerSup, Config, RefC, SafeSup]}}; _ -> - Rpc = #{id => rex, - start => {rpc, start_link, []}, - restart => permanent, - shutdown => 2000, - type => worker, - modules => [rpc]}, - - Global = #{id => global_name_server, - start => {global, start_link, []}, - restart => permanent, - shutdown => 2000, - type => worker, - modules => [global]}, - - GlGroup = #{id => global_group, - start => {global_group,start_link,[]}, - restart => permanent, - shutdown => 2000, - type => worker, - modules => [global_group]}, + DistChildren = + case application:get_env(kernel, start_distribution) of + {ok, false} -> []; + _ -> start_distribution() + end, InetDb = #{id => inet_db, start => {inet_db, start_link, []}, @@ -173,13 +158,6 @@ init([]) -> type => worker, modules => [inet_db]}, - NetSup = #{id => net_sup, - start => {erl_distribution, start_link, []}, - restart => permanent, - shutdown => infinity, - type => supervisor, - modules => [erl_distribution]}, - SigSrv = #{id => erl_signal_server, start => {gen_event, start_link, [{local, erl_signal_server}]}, restart => permanent, @@ -187,14 +165,11 @@ init([]) -> type => worker, modules => dynamic}, - DistAC = start_dist_ac(), - Timer = start_timer(), {ok, {SupFlags, - [Code, Rpc, Global, InetDb | DistAC] ++ - [NetSup, GlGroup, File, SigSrv, - StdError, User, Config, RefC, SafeSup, LoggerSup] ++ Timer}} + [Code, InetDb | DistChildren] ++ + [File, SigSrv, StdError, User, Config, RefC, SafeSup, LoggerSup] ++ Timer}} end; init(safe) -> SupFlags = #{strategy => one_for_one, @@ -213,6 +188,39 @@ init(safe) -> {ok, {SupFlags, Boot ++ DiskLog ++ Pg2}}. +start_distribution() -> + Rpc = #{id => rex, + start => {rpc, start_link, []}, + restart => permanent, + shutdown => 2000, + type => worker, + modules => [rpc]}, + + Global = #{id => global_name_server, + start => {global, start_link, []}, + restart => permanent, + shutdown => 2000, + type => worker, + modules => [global]}, + + DistAC = start_dist_ac(), + + NetSup = #{id => net_sup, + start => {erl_distribution, start_link, []}, + restart => permanent, + shutdown => infinity, + type => supervisor, + modules => [erl_distribution]}, + + GlGroup = #{id => global_group, + start => {global_group,start_link,[]}, + restart => permanent, + shutdown => 2000, + type => worker, + modules => [global_group]}, + + [Rpc, Global | DistAC] ++ [NetSup, GlGroup]. + start_dist_ac() -> Spec = [#{id => dist_ac, start => {dist_ac,start_link,[]}, diff --git a/lib/kernel/src/logger_formatter.erl b/lib/kernel/src/logger_formatter.erl index ded89bac9f..8696adbd72 100644 --- a/lib/kernel/src/logger_formatter.erl +++ b/lib/kernel/src/logger_formatter.erl @@ -64,7 +64,7 @@ format(#{level:=Level,msg:=Msg0,meta:=Meta},Config0) Config; Size0 -> Size = - case Size0 - string:length([B,A]) of + case Size0 - io_lib:chars_length([B,A]) of S when S>=0 -> S; _ -> 0 end, @@ -75,7 +75,11 @@ format(#{level:=Level,msg:=Msg0,meta:=Meta},Config0) true -> %% Trim leading and trailing whitespaces, and replace %% newlines with ", " - re:replace(string:trim(MsgStr0),",?\r?\n\s*",", ", + T = lists:reverse( + trim( + lists:reverse( + trim(MsgStr0,false)),true)), + re:replace(T,",?\r?\n\s*",", ", [{return,list},global,unicode]); _false -> MsgStr0 @@ -83,7 +87,26 @@ format(#{level:=Level,msg:=Msg0,meta:=Meta},Config0) true -> "" end, - truncate([B,MsgStr,A],maps:get(max_size,Config)). + truncate(B,MsgStr,A,maps:get(max_size,Config)). + +trim([H|T],Rev) when H==$\s; H==$\r; H==$\n -> + trim(T,Rev); +trim([H|T],false) when is_list(H) -> + case trim(H,false) of + [] -> + trim(T,false); + TrimmedH -> + [TrimmedH|T] + end; +trim([H|T],true) when is_list(H) -> + case trim(lists:reverse(H),true) of + [] -> + trim(T,true); + TrimmedH -> + [lists:reverse(TrimmedH)|T] + end; +trim(String,_) -> + String. do_format(Level,Data,[level|Format],Config) -> [to_string(level,Level,Config)|do_format(Level,Data,Format,Config)]; @@ -239,21 +262,47 @@ chardata_to_list(Chardata) -> throw(Error) end. -truncate(String,unlimited) -> - String; -truncate(String,Size) -> - Length = string:length(String), +truncate(B,Msg,A,unlimited) -> + [B,Msg,A]; +truncate(B,Msg,A,Size) -> + String = [B,Msg,A], + Length = io_lib:chars_length(String), if Length>Size -> - case lists:reverse(lists:flatten(String)) of - [$\n|_] -> - string:slice(String,0,Size-4)++"...\n"; + {Last,FlatString} = + case A of + [] -> + case Msg of + [] -> + {get_last(B),lists:flatten(B)}; + _ -> + {get_last(Msg),lists:flatten([B,Msg])} + end; + _ -> + {get_last(A),lists:flatten(String)} + end, + case Last of + $\n-> + lists:sublist(FlatString,1,Size-4)++"...\n"; _ -> - string:slice(String,0,Size-3)++"..." + lists:sublist(FlatString,1,Size-3)++"..." end; true -> String end. +get_last(L) -> + get_first(lists:reverse(L)). + +get_first([]) -> + error; +get_first([C|_]) when is_integer(C) -> + C; +get_first([L|Rest]) when is_list(L) -> + case get_last(L) of + error -> get_first(Rest); + First -> First + end. + %% SysTime is the system time in microseconds format_time(SysTime,#{time_offset:=Offset,time_designator:=Des}) when is_integer(SysTime) -> diff --git a/lib/kernel/src/logger_h_common.erl b/lib/kernel/src/logger_h_common.erl index 4b5e0a7dd0..16946ff97c 100644 --- a/lib/kernel/src/logger_h_common.erl +++ b/lib/kernel/src/logger_h_common.erl @@ -142,8 +142,9 @@ changing_config(SetOrUpdate, maps:with(?OLP_KEYS,NewHConfig0)), case logger_olp:set_opts(Olp,NewOlpOpts) of ok -> - maybe_set_repeated_filesync(Olp,OldCommonConfig, - NewCommonConfig), + logger_olp:cast(Olp, {config_changed, + NewCommonConfig, + NewHandlerConfig}), ReadOnly = maps:with(?READ_ONLY_KEYS,OldHConfig), NewHConfig = maps:merge( @@ -281,11 +282,24 @@ handle_cast(repeated_filesync, State#{handler_state => HS, last_op => sync} end, {noreply,set_repeated_filesync(State1)}; - -handle_cast({set_repeated_filesync,FSyncInt},State) -> - State1 = State#{filesync_repeat_interval=>FSyncInt}, - State2 = set_repeated_filesync(cancel_repeated_filesync(State1)), - {noreply, State2}. +handle_cast({config_changed, CommonConfig, HConfig}, + State = #{id := Name, + module := Module, + handler_state := HandlerState, + filesync_repeat_interval := OldFSyncInt}) -> + State1 = + case maps:get(filesync_repeat_interval,CommonConfig) of + OldFSyncInt -> + State; + FSyncInt -> + set_repeated_filesync( + cancel_repeated_filesync( + State#{filesync_repeat_interval=>FSyncInt})) + end, + HS = try Module:config_changed(Name, HConfig, HandlerState) + catch error:undef -> HandlerState + end, + {noreply, State1#{handler_state => HS}}. handle_info(Info, #{id := Name, module := Module, handler_state := HandlerState} = State) -> @@ -447,10 +461,3 @@ cancel_repeated_filesync(State) -> end. error_notify(Term) -> ?internal_log(error, Term). - -maybe_set_repeated_filesync(_Olp, - #{filesync_repeat_interval:=FSyncInt}, - #{filesync_repeat_interval:=FSyncInt}) -> - ok; -maybe_set_repeated_filesync(Olp,_,#{filesync_repeat_interval:=FSyncInt}) -> - logger_olp:cast(Olp,{set_repeated_filesync,FSyncInt}). diff --git a/lib/kernel/src/logger_olp.erl b/lib/kernel/src/logger_olp.erl index 009280a9c9..8365383fe2 100644 --- a/lib/kernel/src/logger_olp.erl +++ b/lib/kernel/src/logger_olp.erl @@ -515,10 +515,11 @@ check_load(State = #{id:=_Name, mode_ref := ModeRef, mode := Mode, end, State1 = ?update_other(drops,DROPS,_NewDrops,State), State2 = ?update_max_qlen(QLen,State1), - State3 = maybe_notify_mode_change(Mode1,State2), + State3 = ?update_max_mem(Mem,State2), + State4 = maybe_notify_mode_change(Mode1,State3), {Mode1, QLen, Mem, ?update_other(flushes,FLUSHES,_NewFlushes, - State3#{last_qlen => QLen})}. + State4#{last_qlen => QLen})}. limit_burst(#{burst_limit_enable := false}=State) -> {true,State}; diff --git a/lib/kernel/src/logger_olp.hrl b/lib/kernel/src/logger_olp.hrl index 9b4f5ebf27..d68b5c048d 100644 --- a/lib/kernel/src/logger_olp.hrl +++ b/lib/kernel/src/logger_olp.hrl @@ -114,12 +114,16 @@ flushes => 0, flushed => 0, drops => 0, burst_drops => 0, casts => 0, calls => 0, writes => 0, max_qlen => 0, max_time => 0, - freq => {TIME,0,0}} end). + max_mem => 0, freq => {TIME,0,0}} end). -define(update_max_qlen(QLEN, STATE), begin #{max_qlen := QLEN0} = STATE, STATE#{max_qlen => ?max(QLEN0,QLEN)} end). + -define(update_max_mem(MEM, STATE), + begin #{max_mem := MEM0} = STATE, + STATE#{max_mem => ?max(MEM0,MEM)} end). + -define(update_calls_or_casts(CALL_OR_CAST, INC, STATE), case CALL_OR_CAST of cast -> @@ -154,6 +158,7 @@ -else. % DEFAULT! -define(merge_with_stats(STATE), STATE). -define(update_max_qlen(_QLEN, STATE), STATE). + -define(update_max_mem(_MEM, STATE), STATE). -define(update_calls_or_casts(_CALL_OR_CAST, _INC, STATE), STATE). -define(update_max_time(_TIME, STATE), STATE). -define(update_other(_OTHER, _VAR, _INCVAL, STATE), STATE). diff --git a/lib/kernel/src/logger_std_h.erl b/lib/kernel/src/logger_std_h.erl index 65f5b3876e..c8f1acfca4 100644 --- a/lib/kernel/src/logger_std_h.erl +++ b/lib/kernel/src/logger_std_h.erl @@ -29,7 +29,7 @@ -export([filesync/1]). %% logger_h_common callbacks --export([init/2, check_config/4, reset_state/2, +-export([init/2, check_config/4, config_changed/3, reset_state/2, filesync/3, write/4, handle_info/3, terminate/3]). %% logger callbacks @@ -105,85 +105,169 @@ filter_config(Config) -> %%%=================================================================== %%% logger_h_common callbacks %%%=================================================================== -init(Name, #{type := Type}) -> - case open_log_file(Name, Type) of +init(Name, Config) -> + MyConfig = maps:with([type,file,modes,file_check,max_no_bytes, + max_no_files,compress_on_rotate],Config), + case file_ctrl_start(Name, MyConfig) of {ok,FileCtrlPid} -> - {ok,#{type=>Type,file_ctrl_pid=>FileCtrlPid}}; + {ok,MyConfig#{file_ctrl_pid=>FileCtrlPid}}; Error -> Error end. -check_config(_Name,set,undefined,NewHConfig) -> - check_config(maps:merge(get_default_config(),NewHConfig)); -check_config(_Name,SetOrUpdate,OldHConfig,NewHConfig0) -> - WriteOnce = maps:with([type],OldHConfig), +check_config(Name,set,undefined,NewHConfig) -> + check_h_config(merge_default_config(Name,normalize_config(NewHConfig))); +check_config(Name,SetOrUpdate,OldHConfig,NewHConfig0) -> + WriteOnce = maps:with([type,file,modes],OldHConfig), Default = case SetOrUpdate of set -> %% Do not reset write-once fields to defaults - maps:merge(get_default_config(),WriteOnce); + merge_default_config(Name,WriteOnce); update -> OldHConfig end, - NewHConfig = maps:merge(Default, NewHConfig0), + NewHConfig = maps:merge(Default, normalize_config(NewHConfig0)), %% Fail if write-once fields are changed - case maps:with([type],NewHConfig) of + case maps:with([type,file,modes],NewHConfig) of WriteOnce -> - check_config(NewHConfig); + check_h_config(NewHConfig); Other -> {error,{illegal_config_change,?MODULE,WriteOnce,Other}} end. -check_config(#{type:=Type}=HConfig) -> - case check_h_config(maps:to_list(HConfig)) of - ok when is_atom(Type) -> - {ok,HConfig#{filesync_repeat_interval=>no_repeat}}; +check_h_config(HConfig) -> + case check_h_config(maps:get(type,HConfig),maps:to_list(HConfig)) of ok -> - {ok,HConfig}; + {ok,fix_file_opts(HConfig)}; {error,{Key,Value}} -> {error,{invalid_config,?MODULE,#{Key=>Value}}} end. -check_h_config([{type,Type} | Config]) when Type == standard_io; - Type == standard_error -> - check_h_config(Config); -check_h_config([{type,{file,File}} | Config]) when is_list(File) -> - check_h_config(Config); -check_h_config([{type,{file,File,Modes}} | Config]) when is_list(File), - is_list(Modes) -> - check_h_config(Config); -check_h_config([Other | _]) -> +check_h_config(Type,[{type,Type} | Config]) when Type =:= standard_io; + Type =:= standard_error; + Type =:= file -> + check_h_config(Type,Config); +check_h_config(file,[{file,File} | Config]) when is_list(File) -> + check_h_config(file,Config); +check_h_config(file,[{modes,Modes} | Config]) when is_list(Modes) -> + check_h_config(file,Config); +check_h_config(file,[{max_no_bytes,Size} | Config]) + when (is_integer(Size) andalso Size>0) orelse Size=:=infinity -> + check_h_config(file,Config); +check_h_config(file,[{max_no_files,Num} | Config]) when is_integer(Num), Num>=0 -> + check_h_config(file,Config); +check_h_config(file,[{compress_on_rotate,Bool} | Config]) when is_boolean(Bool) -> + check_h_config(file,Config); +check_h_config(file,[{file_check,FileCheck} | Config]) + when is_integer(FileCheck), FileCheck>=0 -> + check_h_config(file,Config); +check_h_config(_Type,[Other | _]) -> {error,Other}; -check_h_config([]) -> +check_h_config(_Type,[]) -> ok. -get_default_config() -> - #{type => standard_io}. +normalize_config(#{type:={file,File}}=HConfig) -> + HConfig#{type=>file,file=>File}; +normalize_config(#{type:={file,File,Modes}}=HConfig) -> + HConfig#{type=>file,file=>File,modes=>Modes}; +normalize_config(HConfig) -> + HConfig. + +merge_default_config(Name,#{type:=Type}=HConfig) -> + merge_default_config(Name,Type,HConfig); +merge_default_config(Name,#{file:=_}=HConfig) -> + merge_default_config(Name,file,HConfig); +merge_default_config(Name,HConfig) -> + merge_default_config(Name,standard_io,HConfig). + +merge_default_config(Name,Type,HConfig) -> + maps:merge(get_default_config(Name,Type),HConfig). + +get_default_config(Name,file) -> + #{type => file, + file => atom_to_list(Name), + modes => [raw,append], + file_check => 0, + max_no_bytes => infinity, + max_no_files => 0, + compress_on_rotate => false}; +get_default_config(_Name,Type) -> + #{type => Type}. + +fix_file_opts(#{modes:=Modes}=HConfig) -> + HConfig#{modes=>fix_modes(Modes)}; +fix_file_opts(HConfig) -> + HConfig#{filesync_repeat_interval=>no_repeat}. + +fix_modes(Modes) -> + %% Ensure write|append|exclusive + Modes1 = + case [M || M <- Modes, + lists:member(M,[write,append,exclusive])] of + [] -> [append|Modes]; + _ -> Modes + end, + %% Ensure raw + Modes2 = + case lists:member(raw,Modes) of + false -> [raw|Modes1]; + true -> Modes1 + end, + %% Ensure delayed_write + case lists:partition(fun(delayed_write) -> true; + ({delayed_write,_,_}) -> true; + (_) -> false + end, Modes2) of + {[],_} -> + [delayed_write|Modes2]; + _ -> + Modes2 + end. -filesync(_Name, _Mode, #{type := Type}=State) when is_atom(Type) -> - {ok,State}; -filesync(_Name, async, #{file_ctrl_pid := FileCtrlPid} = State) -> - ok = file_ctrl_filesync_async(FileCtrlPid), - {ok,State}; -filesync(_Name, sync, #{file_ctrl_pid := FileCtrlPid} = State) -> - Result = file_ctrl_filesync_sync(FileCtrlPid), +config_changed(_Name, + #{file_check:=FileCheck, + max_no_bytes:=Size, + max_no_files:=Count, + compress_on_rotate:=Compress}, + #{file_check:=FileCheck, + max_no_bytes:=Size, + max_no_files:=Count, + compress_on_rotate:=Compress}=State) -> + State; +config_changed(_Name, + #{file_check:=FileCheck, + max_no_bytes:=Size, + max_no_files:=Count, + compress_on_rotate:=Compress}, + #{file_ctrl_pid := FileCtrlPid} = State) -> + FileCtrlPid ! {update_config,#{file_check=>FileCheck, + max_no_bytes=>Size, + max_no_files=>Count, + compress_on_rotate=>Compress}}, + State#{file_check:=FileCheck, + max_no_bytes:=Size, + max_no_files:=Count, + compress_on_rotate:=Compress}; +config_changed(_Name,_NewHConfig,State) -> + State. + +filesync(_Name, SyncAsync, #{file_ctrl_pid := FileCtrlPid} = State) -> + Result = file_ctrl_filesync(SyncAsync, FileCtrlPid), {Result,State}. -write(_Name, async, Bin, #{file_ctrl_pid:=FileCtrlPid} = State) -> - ok = file_write_async(FileCtrlPid, Bin), - {ok,State}; -write(_Name, sync, Bin, #{file_ctrl_pid:=FileCtrlPid} = State) -> - Result = file_write_sync(FileCtrlPid, Bin), +write(_Name, SyncAsync, Bin, #{file_ctrl_pid:=FileCtrlPid} = State) -> + Result = file_write(SyncAsync, FileCtrlPid, Bin), {Result,State}. reset_state(_Name, State) -> State. -handle_info(_Name, {'EXIT',Pid,Why}, #{type := FileInfo, file_ctrl_pid := Pid}) -> +handle_info(_Name, {'EXIT',Pid,Why}, #{file_ctrl_pid := Pid}=State) -> %% file_ctrl_pid died, file error, terminate handler - exit({error,{write_failed,FileInfo,Why}}); + exit({error,{write_failed,maps:with([type,file,modes],State),Why}}); handle_info(_, _, State) -> State. @@ -211,27 +295,33 @@ terminate(_Name, _Reason, #{file_ctrl_pid:=FWPid}) -> %%%----------------------------------------------------------------- %%% -open_log_file(HandlerName, FileInfo) -> - case file_ctrl_start(HandlerName, FileInfo) of - OK = {ok,_FileCtrlPid} -> OK; - Error -> Error - end. - -do_open_log_file({file,FileName}) -> - do_open_log_file({file,FileName,[raw,append,delayed_write]}); - -do_open_log_file({file,FileName,[]}) -> - do_open_log_file({file,FileName,[raw,append,delayed_write]}); - -do_open_log_file({file,FileName,Modes}) -> +open_log_file(HandlerName,#{type:=file, + file:=FileName, + modes:=Modes, + file_check:=FileCheck, + max_no_bytes:=Size, + max_no_files:=Count, + compress_on_rotate:=Compress}) -> try case filelib:ensure_dir(FileName) of ok -> case file:open(FileName, Modes) of {ok, Fd} -> {ok,#file_info{inode=INode}} = - file:read_file_info(FileName), - {ok, {Fd, INode}}; + file:read_file_info(FileName,[raw]), + UpdateModes = [append | Modes--[write,append,exclusive]], + State0 = #{handler_name=>HandlerName, + file_name=>FileName, + modes=>UpdateModes, + file_check=>FileCheck, + fd=>Fd, + inode=>INode, + last_check=>timestamp(), + synced=>false, + write_res=>ok, + sync_res=>ok}, + State = update_rotation({Size,Count,Compress},State0), + {ok,State}; Error -> Error end; @@ -242,21 +332,23 @@ do_open_log_file({file,FileName,Modes}) -> _:Reason -> {error,Reason} end. -close_log_file(Std) when Std == standard_io; Std == standard_error -> - ok; -close_log_file({Fd,_}) -> +close_log_file(#{fd:=Fd}) -> _ = file:datasync(Fd), - _ = file:close(Fd). + _ = file:close(Fd), + ok; +close_log_file(_) -> + ok. + %%%----------------------------------------------------------------- %%% File control process -file_ctrl_start(HandlerName, FileInfo) -> +file_ctrl_start(HandlerName, HConfig) -> Starter = self(), FileCtrlPid = spawn_link(fun() -> - file_ctrl_init(HandlerName, FileInfo, Starter) + file_ctrl_init(HandlerName, HConfig, Starter) end), receive {FileCtrlPid,ok} -> @@ -271,18 +363,16 @@ file_ctrl_start(HandlerName, FileInfo) -> file_ctrl_stop(Pid) -> Pid ! stop. -file_write_async(Pid, Bin) -> +file_write(async, Pid, Bin) -> Pid ! {log,Bin}, - ok. - -file_write_sync(Pid, Bin) -> + ok; +file_write(sync, Pid, Bin) -> file_ctrl_call(Pid, {log,Bin}). -file_ctrl_filesync_async(Pid) -> +file_ctrl_filesync(async, Pid) -> Pid ! filesync, - ok. - -file_ctrl_filesync_sync(Pid) -> + ok; +file_ctrl_filesync(sync, Pid) -> file_ctrl_call(Pid, filesync). file_ctrl_call(Pid, Msg) -> @@ -299,98 +389,255 @@ file_ctrl_call(Pid, Msg) -> {error,{no_response,Pid}} end. -file_ctrl_init(HandlerName, FileInfo, Starter) when is_tuple(FileInfo) -> +file_ctrl_init(HandlerName, + #{type:=file, + file:=FileName} = HConfig, + Starter) -> process_flag(message_queue_data, off_heap), - FileName = element(2, FileInfo), - case do_open_log_file(FileInfo) of - {ok,File} -> + case open_log_file(HandlerName,HConfig) of + {ok,State} -> Starter ! {self(),ok}, - file_ctrl_loop(File, FileName, false, ok, ok, HandlerName); + file_ctrl_loop(State); {error,Reason} -> Starter ! {self(),{error,{open_failed,FileName,Reason}}} end; -file_ctrl_init(HandlerName, StdDev, Starter) -> +file_ctrl_init(HandlerName, #{type:=StdDev}, Starter) -> Starter ! {self(),ok}, - file_ctrl_loop(StdDev, StdDev, false, ok, ok, HandlerName). + file_ctrl_loop(#{handler_name=>HandlerName,dev=>StdDev}). -file_ctrl_loop(File, DevName, Synced, - PrevWriteResult, PrevSyncResult, HandlerName) -> +file_ctrl_loop(State) -> receive %% asynchronous event {log,Bin} -> - File1 = ensure(File, DevName), - Result = write_to_dev(File1, Bin, DevName, - PrevWriteResult, HandlerName), - file_ctrl_loop(File1, DevName, false, - Result, PrevSyncResult, HandlerName); + State1 = write_to_dev(Bin,State), + file_ctrl_loop(State1); %% synchronous event {{log,Bin},{From,MRef}} -> - File1 = ensure(File, DevName), - Result = write_to_dev(File1, Bin, DevName, - PrevWriteResult, HandlerName), + State1 = ensure_file(State), + State2 = write_to_dev(Bin,State1), From ! {MRef,ok}, - file_ctrl_loop(File1, DevName, false, - Result, PrevSyncResult, HandlerName); + file_ctrl_loop(State2); filesync -> - File1 = ensure(File, DevName), - Result = sync_dev(File1, DevName, Synced, - PrevSyncResult, HandlerName), - file_ctrl_loop(File1, DevName, true, - PrevWriteResult, Result, HandlerName); + State1 = sync_dev(State), + file_ctrl_loop(State1); {filesync,{From,MRef}} -> - File1 = ensure(File, DevName), - Result = sync_dev(File1, DevName, Synced, - PrevSyncResult, HandlerName), + State1 = ensure_file(State), + State2 = sync_dev(State1), From ! {MRef,ok}, - file_ctrl_loop(File1, DevName, true, - PrevWriteResult, Result, HandlerName); + file_ctrl_loop(State2); + + {update_config,#{file_check:=FileCheck, + max_no_bytes:=Size, + max_no_files:=Count, + compress_on_rotate:=Compress}} -> + State1 = update_rotation({Size,Count,Compress},State), + file_ctrl_loop(State1#{file_check=>FileCheck}); stop -> - _ = close_log_file(File), + close_log_file(State), stopped end. +maybe_ensure_file(#{file_check:=0}=State) -> + ensure_file(State); +maybe_ensure_file(#{last_check:=T0,file_check:=CheckInt}=State) + when is_integer(CheckInt) -> + T = timestamp(), + if T-T0 > CheckInt -> ensure_file(State); + true -> State + end; +maybe_ensure_file(State) -> + State. + %% In order to play well with tools like logrotate, we need to be able %% to re-create the file if it has disappeared (e.g. if rotated by %% logrotate) -ensure(Fd,DevName) when is_atom(DevName) -> - Fd; -ensure({Fd,INode},FileName) -> - case file:read_file_info(FileName) of - {ok,#file_info{inode=INode}} -> - {Fd,INode}; +ensure_file(#{fd:=Fd0,inode:=INode0,file_name:=FileName,modes:=Modes}=State) -> + case file:read_file_info(FileName,[raw]) of + {ok,#file_info{inode=INode0}} -> + State#{last_check=>timestamp()}; _ -> - _ = file:close(Fd), - _ = file:close(Fd), % delayed_write cause close not to close - case do_open_log_file({file,FileName}) of - {ok,File} -> - File; + close_log_file(Fd0), + case file:open(FileName,Modes) of + {ok,Fd} -> + {ok,#file_info{inode=INode}} = + file:read_file_info(FileName,[raw]), + State#{fd=>Fd,inode=>INode, + last_check=>timestamp(), + synced=>true,sync_res=>ok}; Error -> exit({could_not_reopen_file,Error}) end - end. + end; +ensure_file(State) -> + State. -write_to_dev(DevName, Bin, _DevName, _PrevWriteResult, _HandlerName) - when is_atom(DevName) -> - io:put_chars(DevName, Bin); -write_to_dev({Fd,_}, Bin, FileName, PrevWriteResult, HandlerName) -> +write_to_dev(Bin,#{dev:=DevName}=State) -> + io:put_chars(DevName, Bin), + State; +write_to_dev(Bin, State) -> + State1 = #{fd:=Fd} = maybe_ensure_file(State), Result = ?file_write(Fd, Bin), - maybe_notify_error(write,Result,PrevWriteResult,FileName,HandlerName). + State2 = maybe_rotate_file(Bin,State1), + maybe_notify_error(write,Result,State2), + State2#{synced=>false,write_res=>Result}. -sync_dev(_, _FileName, true, PrevSyncResult, _HandlerName) -> - PrevSyncResult; -sync_dev({Fd,_}, FileName, false, PrevSyncResult, HandlerName) -> +sync_dev(#{synced:=false}=State) -> + State1 = #{fd:=Fd} = maybe_ensure_file(State), Result = ?file_datasync(Fd), - maybe_notify_error(filesync,Result,PrevSyncResult,FileName,HandlerName). + maybe_notify_error(filesync,Result,State1), + State1#{synced=>true,sync_res=>Result}; +sync_dev(State) -> + State. -maybe_notify_error(_Op, ok, _PrevResult, _FileName, _HandlerName) -> +update_rotation({infinity,_,_},State) -> + maybe_remove_archives(0,State), + maps:remove(rotation,State); +update_rotation({Size,Count,Compress},#{file_name:=FileName} = State) -> + maybe_remove_archives(Count,State), + {ok,#file_info{size=CurrSize}} = file:read_file_info(FileName,[raw]), + State1 = State#{rotation=>#{size=>Size, + count=>Count, + compress=>Compress, + curr_size=>CurrSize}}, + maybe_update_compress(0,State1), + maybe_rotate_file(0,State1). + +maybe_remove_archives(Count,#{file_name:=FileName}=State) -> + Archive = rot_file_name(FileName,Count,false), + CompressedArchive = rot_file_name(FileName,Count,true), + case {file:read_file_info(Archive,[raw]), + file:read_file_info(CompressedArchive,[raw])} of + {{error,enoent},{error,enoent}} -> + ok; + _ -> + _ = file:delete(Archive), + _ = file:delete(CompressedArchive), + maybe_remove_archives(Count+1,State) + end. + +maybe_update_compress(Count,#{rotation:=#{count:=Count}}) -> + ok; +maybe_update_compress(N,#{file_name:=FileName, + rotation:=#{compress:=Compress}}=State) -> + Archive = rot_file_name(FileName,N,not Compress), + case file:read_file_info(Archive,[raw]) of + {ok,_} when Compress -> + compress_file(Archive); + {ok,_} -> + decompress_file(Archive); + _ -> + ok + end, + maybe_update_compress(N+1,State). + +maybe_rotate_file(Bin,#{rotation:=_}=State) when is_binary(Bin) -> + maybe_rotate_file(byte_size(Bin),State); +maybe_rotate_file(AddSize,#{rotation:=#{size:=RotSize, + curr_size:=CurrSize}=Rotation}=State) -> + NewSize = CurrSize + AddSize, + if NewSize>RotSize -> + rotate_file(State#{rotation=>Rotation#{curr_size=>NewSize}}); + true -> + State#{rotation=>Rotation#{curr_size=>NewSize}} + end; +maybe_rotate_file(_Bin,State) -> + State. + +rotate_file(#{fd:=Fd0,file_name:=FileName,modes:=Modes,rotation:=Rotation}=State) -> + State1 = sync_dev(State), + _ = file:close(Fd0), + _ = file:close(Fd0), + rotate_files(FileName,maps:get(count,Rotation),maps:get(compress,Rotation)), + case file:open(FileName,Modes) of + {ok,Fd} -> + {ok,#file_info{inode=INode}} = file:read_file_info(FileName,[raw]), + State1#{fd=>Fd,inode=>INode,rotation=>Rotation#{curr_size=>0}}; + Error -> + exit({could_not_reopen_file,Error}) + end. + +rotate_files(FileName,0,_Compress) -> + _ = file:delete(FileName), + ok; +rotate_files(FileName,1,Compress) -> + FileName0 = FileName++".0", + _ = file:rename(FileName,FileName0), + if Compress -> compress_file(FileName0); + true -> ok + end, ok; -maybe_notify_error(_Op, PrevResult, PrevResult, _FileName, _HandlerName) -> +rotate_files(FileName,Count,Compress) -> + _ = file:rename(rot_file_name(FileName,Count-2,Compress), + rot_file_name(FileName,Count-1,Compress)), + rotate_files(FileName,Count-1,Compress). + +rot_file_name(FileName,Count,false) -> + FileName ++ "." ++ integer_to_list(Count); +rot_file_name(FileName,Count,true) -> + rot_file_name(FileName,Count,false) ++ ".gz". + +compress_file(FileName) -> + {ok,In} = file:open(FileName,[read,binary]), + {ok,Out} = file:open(FileName++".gz",[write]), + Z = zlib:open(), + zlib:deflateInit(Z, default, deflated, 31, 8, default), + compress_data(Z,In,Out), + zlib:deflateEnd(Z), + zlib:close(Z), + _ = file:close(In), + _ = file:close(Out), + _ = file:delete(FileName), + ok. + +compress_data(Z,In,Out) -> + case file:read(In,100000) of + {ok,Data} -> + Compressed = zlib:deflate(Z, Data), + _ = file:write(Out,Compressed), + compress_data(Z,In,Out); + eof -> + Compressed = zlib:deflate(Z, <<>>, finish), + _ = file:write(Out,Compressed), + ok + end. + +decompress_file(FileName) -> + {ok,In} = file:open(FileName,[read,binary]), + {ok,Out} = file:open(filename:rootname(FileName,".gz"),[write]), + Z = zlib:open(), + zlib:inflateInit(Z, 31), + decompress_data(Z,In,Out), + zlib:inflateEnd(Z), + zlib:close(Z), + _ = file:close(In), + _ = file:close(Out), + _ = file:delete(FileName), + ok. + +decompress_data(Z,In,Out) -> + case file:read(In,1000) of + {ok,Data} -> + Decompressed = zlib:inflate(Z, Data), + _ = file:write(Out,Decompressed), + decompress_data(Z,In,Out); + eof -> + ok + end. + +maybe_notify_error(_Op, ok, _State) -> + ok; +maybe_notify_error(Op, Result, #{write_res:=WR,sync_res:=SR}) + when (Op==write andalso Result==WR) orelse + (Op==filesync andalso Result==SR) -> %% don't report same error twice - PrevResult; -maybe_notify_error(Op, Error, _PrevResult, FileName, HandlerName) -> + ok; +maybe_notify_error(Op, Error, #{handler_name:=HandlerName,file_name:=FileName}) -> logger_h_common:error_notify({HandlerName,Op,FileName,Error}), - Error. + ok. + +timestamp() -> + erlang:monotonic_time(millisecond). diff --git a/lib/kernel/test/kernel.spec b/lib/kernel/test/kernel.spec index 62afc9f97b..eaa17f3a59 100644 --- a/lib/kernel/test/kernel.spec +++ b/lib/kernel/test/kernel.spec @@ -2,3 +2,4 @@ {config, "../test_server/ts.unix.config"}. {suites,"../kernel_test", all}. +{skip_suites,"../kernel_test",[logger_stress_SUITE],"Benchmarks only"}. diff --git a/lib/kernel/test/kernel_config_SUITE.erl b/lib/kernel/test/kernel_config_SUITE.erl index 9207025a2c..57c44c2498 100644 --- a/lib/kernel/test/kernel_config_SUITE.erl +++ b/lib/kernel/test/kernel_config_SUITE.erl @@ -21,7 +21,8 @@ -include_lib("common_test/include/ct.hrl"). --export([all/0, suite/0,groups/0,init_per_group/2,end_per_group/2, sync/1]). +-export([all/0, suite/0,groups/0,init_per_group/2,end_per_group/2, + start_distribution_false/1, sync/1]). -export([init_per_suite/1, end_per_suite/1]). @@ -30,7 +31,7 @@ suite() -> {timetrap,{minutes,2}}]. all() -> - [sync]. + [sync, start_distribution_false]. groups() -> []. @@ -59,12 +60,9 @@ from(H, [H | T]) -> T; from(H, [_ | T]) -> from(H, T); from(_, []) -> []. -%%----------------------------------------------------------------- -%% Test suite for sync_nodes. This is quite tricky. -%% +%% Test sync_nodes. This is quite tricky. %% Should be started in a CC view with: %% erl -sname XXX where XX not in [cp1, cp2] -%%----------------------------------------------------------------- sync(Conf) when is_list(Conf) -> %% Write a config file Dir = proplists:get_value(priv_dir,Conf), @@ -106,9 +104,18 @@ wait_for_node(Node) -> _Other -> wait_for_node(Node) end. - stop_node(Node) -> M = list_to_atom(lists:concat([Node, [$@], from($@,atom_to_list(node()))])), rpc:cast(M, erlang, halt, []). + +start_distribution_false(Config) when is_list(Config) -> + %% When distribution is disabled, -sname/-name has no effect + Str = os:cmd(ct:get_progname() + ++ " -kernel start_distribution false" + ++ " -sname no_distribution" + ++ " -eval \"erlang:display(node())\"" + ++ " -noshell -s erlang halt"), + "'nonode@nohost'" ++ _ = Str, + ok. diff --git a/lib/kernel/test/logger_SUITE.erl b/lib/kernel/test/logger_SUITE.erl index 70bb775db8..035e5d8974 100644 --- a/lib/kernel/test/logger_SUITE.erl +++ b/lib/kernel/test/logger_SUITE.erl @@ -1048,8 +1048,11 @@ kernel_config(Config) -> ok = rpc:call(Node,logger,internal_init_logger,[]), ok = rpc:call(Node,logger,add_handlers,[kernel]), #{primary:=#{filter_default:=log,filters:=[]}, - handlers:=[#{id:=default,filters:=DF,config:=#{type:={file,F}}}], + handlers:=[#{id:=default,filters:=DF, + config:=#{type:=file,file:=F,modes:=Modes}}], module_levels:=[]} = rpc:call(Node,logger,get_config,[]), + [append,delayed_write,raw] = lists:sort(Modes), + %% Same, but using 'logger' parameter instead of 'error_logger' ok = rpc:call(Node,logger,remove_handler,[default]),% so it can be added again @@ -1060,26 +1063,27 @@ kernel_config(Config) -> ok = rpc:call(Node,logger,internal_init_logger,[]), ok = rpc:call(Node,logger,add_handlers,[kernel]), #{primary:=#{filter_default:=log,filters:=[]}, - handlers:=[#{id:=default,filters:=DF,config:=#{type:={file,F}}}], + handlers:=[#{id:=default,filters:=DF, + config:=#{type:=file,file:=F,modes:=Modes}}], module_levels:=[]} = rpc:call(Node,logger,get_config,[]), %% Same, but with type={file,File,Modes} ok = rpc:call(Node,logger,remove_handler,[default]),% so it can be added again ok = rpc:call(Node,application,unset_env,[kernel,error_logger]), - M = [raw,write,delayed_write], + M = [raw,write], ok = rpc:call(Node,application,set_env,[kernel,logger, [{handler,default,logger_std_h, #{config=>#{type=>{file,F,M}}}}]]), ok = rpc:call(Node,logger,internal_init_logger,[]), ok = rpc:call(Node,logger,add_handlers,[kernel]), #{primary:=#{filter_default:=log,filters:=[]}, - handlers:=[#{id:=default,filters:=DF,config:=#{type:={file,F,M}}}], + handlers:=[#{id:=default,filters:=DF, + config:=#{type:=file,file:=F,modes:=[delayed_write|M]}}], module_levels:=[]} = rpc:call(Node,logger,get_config,[]), %% Same, but with disk_log handler ok = rpc:call(Node,logger,remove_handler,[default]),% so it can be added again ok = rpc:call(Node,application,unset_env,[kernel,error_logger]), - M = [raw,write,delayed_write], ok = rpc:call(Node,application,set_env,[kernel,logger, [{handler,default,logger_disk_log_h, #{config=>#{file=>F}}}]]), diff --git a/lib/kernel/test/logger_std_h_SUITE.erl b/lib/kernel/test/logger_std_h_SUITE.erl index b2c2c8ba67..0c5516f82b 100644 --- a/lib/kernel/test/logger_std_h_SUITE.erl +++ b/lib/kernel/test/logger_std_h_SUITE.erl @@ -112,6 +112,8 @@ all() -> add_remove_instance_standard_error, add_remove_instance_file1, add_remove_instance_file2, + add_remove_instance_file3, + add_remove_instance_file4, default_formatter, filter_config, errors, @@ -142,7 +144,12 @@ all() -> restart_after, handler_requests_under_load, recreate_deleted_log, - reopen_changed_log + reopen_changed_log, + rotate_size, + rotate_size_compressed, + rotate_size_reopen, + rotation_opts, + rotation_opts_restart_handler ]. add_remove_instance_tty(_Config) -> @@ -179,10 +186,27 @@ add_remove_instance_file2(Config) -> add_remove_instance_file2(cleanup,_Config) -> logger_std_h_remove(). -add_remove_instance_file(Log, Type) -> +add_remove_instance_file3(_Config) -> + Log = atom_to_list(?MODULE), + StdHConfig = #{type=>file}, + add_remove_instance_file(Log, StdHConfig). +add_remove_instance_file3(cleanup,_Config) -> + logger_std_h_remove(). + +add_remove_instance_file4(Config) -> + Dir = ?config(priv_dir,Config), + Log = filename:join(Dir,"stdlog4.txt"), + StdHConfig = #{file=>Log,modes=>[]}, + add_remove_instance_file(Log, StdHConfig). +add_remove_instance_file4(cleanup,_Config) -> + logger_std_h_remove(). + +add_remove_instance_file(Log, Type) when not is_map(Type) -> + add_remove_instance_file(Log,#{type=>Type}); +add_remove_instance_file(Log, StdHConfig) when is_map(StdHConfig) -> ok = logger:add_handler(?MODULE, logger_std_h, - #{config => #{type => Type}, + #{config => StdHConfig, filter_default=>stop, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}), @@ -257,9 +281,10 @@ errors(Config) -> end, {error, - {handler_not_added,{open_failed,Log,_}}} = + {handler_not_added, + {invalid_config,logger_std_h,#{modes:=bad_file_opt}}}} = logger:add_handler(myh3,logger_std_h, - #{config=>#{type=>{file,Log,[bad_file_opt]}}}), + #{config=>#{type=>{file,Log,bad_file_opt}}}), ok = logger:notice(?msg). @@ -607,24 +632,51 @@ reconfig(cleanup, _Config) -> file_opts(Config) -> Dir = ?config(priv_dir,Config), Log = filename:join(Dir, lists:concat([?FUNCTION_NAME,".log"])), - BadFileOpts = [raw], - BadType = {file,Log,BadFileOpts}, - {error,{handler_not_added,{open_failed,Log,enoent}}} = - logger:add_handler(?MODULE, logger_std_h, - #{config => #{type => BadType}}), + MissingOpts = [raw], + Type1 = {file,Log,MissingOpts}, + ok = logger:add_handler(?MODULE, logger_std_h, + #{config => #{type => Type1}}), + {ok,#{config:=#{type:=file,file:=Log,modes:=Modes1}}} = + logger:get_handler_config(?MODULE), + [append,delayed_write,raw] = lists:sort(Modes1), + ok = logger:remove_handler(?MODULE), OkFileOpts = [raw,append], OkType = {file,Log,OkFileOpts}, ok = logger:add_handler(?MODULE, logger_std_h, - #{config => #{type => OkType}, + #{config => #{type => OkType}, % old format filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}), - #{cb_state := #{handler_state := #{type := OkType}}} = + ModOpts = [delayed_write|OkFileOpts], + #{cb_state := #{handler_state := #{type:=file, + file:=Log, + modes:=ModOpts}}} = logger_olp:info(h_proc_name()), - {ok,#{config := #{type := OkType}}} = logger:get_handler_config(?MODULE), + {ok,#{config := #{type:=file, + file:=Log, + modes:=ModOpts}}} = logger:get_handler_config(?MODULE), + ok = logger:remove_handler(?MODULE), + + ok = logger:add_handler(?MODULE, + logger_std_h, + #{config => #{type => file, + file => Log, + modes => OkFileOpts}, % new format + filter_default=>log, + filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), + formatter=>{?MODULE,self()}}), + + #{cb_state := #{handler_state := #{type:=file, + file:=Log, + modes:=ModOpts}}} = + logger_olp:info(h_proc_name()), + {ok,#{config := #{type:=file, + file:=Log, + modes:=ModOpts}}} = + logger:get_handler_config(?MODULE), logger:notice(M1=?msg,?domain), ?check(M1), B1 = ?bin(M1), @@ -640,13 +692,14 @@ sync(Config) -> Type = {file,Log}, ok = logger:add_handler(?MODULE, logger_std_h, - #{config => #{type => Type}, + #{config => #{type => Type, + file_check => 10000}, filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,nl}}), %% check repeated filesync happens - start_tracer([{logger_std_h, write_to_dev, 5}, + start_tracer([{logger_std_h, write_to_dev, 2}, {file, datasync, 1}], [{logger_std_h, write_to_dev, <<"first\n">>}, {file,datasync}]), @@ -656,7 +709,7 @@ sync(Config) -> check_tracer(filesync_rep_int()*2), %% check that explicit filesync is only done once - start_tracer([{logger_std_h, write_to_dev, 5}, + start_tracer([{logger_std_h, write_to_dev, 2}, {file, datasync, 1}], [{logger_std_h, write_to_dev, <<"second\n">>}, {file,datasync}, @@ -675,7 +728,7 @@ sync(Config) -> #{filesync_repeat_interval => no_repeat}), no_repeat = maps:get(filesync_repeat_interval, maps:get(cb_state, logger_olp:info(h_proc_name()))), - start_tracer([{logger_std_h, write_to_dev, 5}, + start_tracer([{logger_std_h, write_to_dev, 2}, {file, datasync, 1}], [{logger_std_h, write_to_dev, <<"third\n">>}, {file,datasync}, @@ -1285,6 +1338,331 @@ reopen_changed_log(Config) -> reopen_changed_log(cleanup, _Config) -> ok = stop_handler(?MODULE). +rotate_size(Config) -> + {Log,_HConfig,_StdHConfig} = + start_handler(?MODULE, ?FUNCTION_NAME, Config), + ok = logger:update_handler_config(?MODULE,#{config=>#{max_no_bytes=>1000, + max_no_files=>2}}), + + Str = lists:duplicate(19,$a), + [logger:notice(Str,?domain) || _ <- lists:seq(1,50)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=1000}} = file:read_file_info(Log), + {error,enoent} = file:read_file_info(Log++".0"), + + logger:notice(Str,?domain), + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=0}} = file:read_file_info(Log), + {ok,#file_info{size=1020}} = file:read_file_info(Log++".0"), + {error,enoent} = file:read_file_info(Log++".1"), + + [logger:notice(Str,?domain) || _ <- lists:seq(1,51)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=0}} = file:read_file_info(Log), + {ok,#file_info{size=1020}} = file:read_file_info(Log++".0"), + {ok,#file_info{size=1020}} = file:read_file_info(Log++".1"), + {error,enoent} = file:read_file_info(Log++".2"), + + [logger:notice(Str,?domain) || _ <- lists:seq(1,50)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=1000}} = file:read_file_info(Log), + {ok,#file_info{size=1020}} = file:read_file_info(Log++".0"), + {ok,#file_info{size=1020}} = file:read_file_info(Log++".1"), + {error,enoent} = file:read_file_info(Log++".2"), + + logger:notice("bbbb",?domain), + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=0}} = file:read_file_info(Log), + {ok,#file_info{size=1005}} = file:read_file_info(Log++".0"), + {ok,#file_info{size=1020}} = file:read_file_info(Log++".1"), + {error,enoent} = file:read_file_info(Log++".2"), + + ok. +rotate_size(cleanup,_Config) -> + ok = stop_handler(?MODULE). + +rotate_size_compressed(Config) -> + {Log,_HConfig,_StdHConfig} = + start_handler(?MODULE, ?FUNCTION_NAME, Config), + ok = logger:update_handler_config(?MODULE, + #{config=>#{max_no_bytes=>1000, + max_no_files=>2, + compress_on_rotate=>true}}), + Str = lists:duplicate(19,$a), + [logger:notice(Str,?domain) || _ <- lists:seq(1,50)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=1000}} = file:read_file_info(Log), + {error,enoent} = file:read_file_info(Log++".0"), + {error,enoent} = file:read_file_info(Log++".0.gz"), + + logger:notice(Str,?domain), + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=0}} = file:read_file_info(Log), + {error,enoent} = file:read_file_info(Log++".0"), + {ok,#file_info{size=35}} = file:read_file_info(Log++".0.gz"), + {error,enoent} = file:read_file_info(Log++".1"), + {error,enoent} = file:read_file_info(Log++".1.gz"), + + [logger:notice(Str,?domain) || _ <- lists:seq(1,51)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=0}} = file:read_file_info(Log), + {error,enoent} = file:read_file_info(Log++".0"), + {ok,#file_info{size=35}} = file:read_file_info(Log++".0.gz"), + {error,enoent} = file:read_file_info(Log++".1"), + {ok,#file_info{size=35}} = file:read_file_info(Log++".1.gz"), + {error,enoent} = file:read_file_info(Log++".2"), + {error,enoent} = file:read_file_info(Log++".2.gz"), + + [logger:notice(Str,?domain) || _ <- lists:seq(1,50)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=1000}} = file:read_file_info(Log), + {error,enoent} = file:read_file_info(Log++".0"), + {ok,#file_info{size=35}} = file:read_file_info(Log++".0.gz"), + {error,enoent} = file:read_file_info(Log++".1"), + {ok,#file_info{size=35}} = file:read_file_info(Log++".1.gz"), + {error,enoent} = file:read_file_info(Log++".2"), + {error,enoent} = file:read_file_info(Log++".2.gz"), + + logger:notice("bbbb",?domain), + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=0}} = file:read_file_info(Log), + {error,enoent} = file:read_file_info(Log++".0"), + {ok,#file_info{size=38}} = file:read_file_info(Log++".0.gz"), + {error,enoent} = file:read_file_info(Log++".1"), + {ok,#file_info{size=35}} = file:read_file_info(Log++".1.gz"), + {error,enoent} = file:read_file_info(Log++".2"), + {error,enoent} = file:read_file_info(Log++".2.gz"), + + ok. +rotate_size_compressed(cleanup,_Config) -> + ok = stop_handler(?MODULE). + +rotate_size_reopen(Config) -> + {Log,_HConfig,_StdHConfig} = + start_handler(?MODULE, ?FUNCTION_NAME, Config), + ok = logger:update_handler_config(?MODULE,#{config=>#{max_no_bytes=>1000, + max_no_files=>2}}), + + Str = lists:duplicate(19,$a), + [logger:notice(Str,?domain) || _ <- lists:seq(1,40)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=800}} = file:read_file_info(Log), + + {ok,HConfig} = logger:get_handler_config(?MODULE), + ok = logger:remove_handler(?MODULE), + ok = logger:add_handler(?MODULE,maps:get(module,HConfig),HConfig), + {ok,#file_info{size=800}} = file:read_file_info(Log), + + [logger:notice(Str,?domain) || _ <- lists:seq(1,40)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=580}} = file:read_file_info(Log), + {ok,#file_info{size=1020}} = file:read_file_info(Log++".0"), + ok. +rotate_size_reopen(cleanup,_Config) -> + ok = stop_handler(?MODULE). + +rotation_opts(Config) -> + {Log,_HConfig,StdHConfig} = + start_handler(?MODULE, ?FUNCTION_NAME, Config), + #{max_no_bytes:=infinity, + max_no_files:=0, + compress_on_rotate:=false} = StdHConfig, + + %% Test bad rotation config + {error,{invalid_config,_,_}} = + logger:update_handler_config(?MODULE,config,#{max_no_bytes=>0}), + {error,{invalid_config,_,_}} = + logger:update_handler_config(?MODULE,config,#{max_no_files=>infinity}), + {error,{invalid_config,_,_}} = + logger:update_handler_config(?MODULE,config, + #{compress_on_rotate=>undefined}), + + + %% Test good rotation config - start with no rotation + Str = lists:duplicate(19,$a), + [logger:notice(Str,?domain) || _ <- lists:seq(1,10)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=200}} = file:read_file_info(Log), + [] = filelib:wildcard(Log++".*"), + + %% Turn on rotation, check that existing file is rotated since its + %% size exceeds max_no_bytes + ok = logger:update_handler_config(?MODULE, + config, + #{max_no_bytes=>100, + max_no_files=>2}), + timer:sleep(100), % give some time to execute config_changed + {ok,#file_info{size=0}} = file:read_file_info(Log), + Log0 = Log++".0", + {ok,#file_info{size=200}} = file:read_file_info(Log0), + [Log0] = filelib:wildcard(Log++".*"), + + %% Fill all logs + [logger:notice(Str,?domain) || _ <- lists:seq(1,13)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=20}} = file:read_file_info(Log), + {ok,#file_info{size=120}} = file:read_file_info(Log0), + {ok,#file_info{size=120}} = file:read_file_info(Log++".1"), + [_,_] = filelib:wildcard(Log++".*"), + + %% Extend size and count and check that nothing changes with existing files + ok = logger:update_handler_config(?MODULE, + config, + #{max_no_bytes=>200, + max_no_files=>3}), + timer:sleep(100), % give some time to execute config_changed + {ok,#file_info{size=20}} = file:read_file_info(Log), + {ok,#file_info{size=120}} = file:read_file_info(Log0), + {ok,#file_info{size=120}} = file:read_file_info(Log++".1"), + [_,_] = filelib:wildcard(Log++".*"), + + %% Add more log events and see that extended size and count works + [logger:notice(Str,?domain) || _ <- lists:seq(1,10)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=0}} = file:read_file_info(Log), + {ok,#file_info{size=220}} = file:read_file_info(Log0), + {ok,#file_info{size=120}} = file:read_file_info(Log++".1"), + {ok,#file_info{size=120}} = file:read_file_info(Log++".2"), + [_,_,_] = filelib:wildcard(Log++".*"), + + %% Reduce count and check that archive files that exceed the new + %% count are moved + ok = logger:update_handler_config(?MODULE, + config, + #{max_no_files=>1}), + timer:sleep(100), % give some time to execute config_changed + {ok,#file_info{size=0}} = file:read_file_info(Log), + {ok,#file_info{size=220}} = file:read_file_info(Log0), + [Log0] = filelib:wildcard(Log++".*"), + + %% Extend size and count again, and turn on compression. Check + %% that archives are compressed + ok = logger:update_handler_config(?MODULE, + config, + #{max_no_bytes=>100, + max_no_files=>2, + compress_on_rotate=>true}), + timer:sleep(100), % give some time to execute config_changed + {ok,#file_info{size=0}} = file:read_file_info(Log), + Log0gz = Log0++".gz", + {ok,#file_info{size=29}} = file:read_file_info(Log0gz), + [Log0gz] = filelib:wildcard(Log++".*"), + + %% Fill all logs + [logger:notice(Str,?domain) || _ <- lists:seq(1,13)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=20}} = file:read_file_info(Log), + {ok,#file_info{size=29}} = file:read_file_info(Log0gz), + {ok,#file_info{size=29}} = file:read_file_info(Log++".1.gz"), + [_,_] = filelib:wildcard(Log++".*"), + + %% Reduce count and turn off compression. Check that archives that + %% exceeds the new count are removed, and the rest are + %% uncompressed. + ok = logger:update_handler_config(?MODULE, + config, + #{max_no_files=>1, + compress_on_rotate=>false}), + timer:sleep(100), % give some time to execute config_changed + {ok,#file_info{size=20}} = file:read_file_info(Log), + {ok,#file_info{size=120}} = file:read_file_info(Log0), + [Log0] = filelib:wildcard(Log++".*"), + + %% Check that config and handler state agree on the current rotation settings + {ok,#{config:=#{max_no_bytes:=100, + max_no_files:=1, + compress_on_rotate:=false}}} = + logger:get_handler_config(?MODULE), + #{cb_state:=#{handler_state:=#{max_no_bytes:=100, + max_no_files:=1, + compress_on_rotate:=false}}} = + logger_olp:info(h_proc_name()), + ok. +rotation_opts(cleanup,_Config) -> + ok = stop_handler(?MODULE). + +rotation_opts_restart_handler(Config) -> + {Log,_HConfig,_StdHConfig} = + start_handler(?MODULE, ?FUNCTION_NAME, Config), + ok = logger:update_handler_config(?MODULE, + config, + #{max_no_bytes=>100, + max_no_files=>2}), + + %% Fill all logs + Str = lists:duplicate(19,$a), + [logger:notice(Str,?domain) || _ <- lists:seq(1,15)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=60}} = file:read_file_info(Log), + {ok,#file_info{size=120}} = file:read_file_info(Log++".0"), + {ok,#file_info{size=120}} = file:read_file_info(Log++".1"), + [_,_] = filelib:wildcard(Log++".*"), + + %% Stop/start handler and turn off rotation. Check that archives are removed. + {ok,#{config:=StdHConfig1}=HConfig1} = logger:get_handler_config(?MODULE), + ok = logger:remove_handler(?MODULE), + ok = logger:add_handler( + ?MODULE,logger_std_h, + HConfig1#{config=>StdHConfig1#{max_no_bytes=>infinity}}), + timer:sleep(100), + {ok,#file_info{size=60}} = file:read_file_info(Log), + [] = filelib:wildcard(Log++".*"), + + %% Add some log events and check that file is no longer rotated. + [logger:notice(Str,?domain) || _ <- lists:seq(1,10)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=260}} = file:read_file_info(Log), + [] = filelib:wildcard(Log++".*"), + + %% Stop/start handler and trun on rotation. Check that file is rotated. + {ok,#{config:=StdHConfig2}=HConfig2} = logger:get_handler_config(?MODULE), + ok = logger:remove_handler(?MODULE), + ok = logger:add_handler( + ?MODULE,logger_std_h, + HConfig2#{config=>StdHConfig2#{max_no_bytes=>100, + max_no_files=>2}}), + timer:sleep(100), + {ok,#file_info{size=0}} = file:read_file_info(Log), + {ok,#file_info{size=260}} = file:read_file_info(Log++".0"), + [_] = filelib:wildcard(Log++".*"), + + %% Fill all logs + [logger:notice(Str,?domain) || _ <- lists:seq(1,10)], + logger_std_h:filesync(?MODULE), + {ok,#file_info{size=80}} = file:read_file_info(Log), + {ok,#file_info{size=120}} = file:read_file_info(Log++".0"), + {ok,#file_info{size=260}} = file:read_file_info(Log++".1"), + + %% Stop/start handler, reduce count and turn on compression. Check + %% that excess archives are removed, and the rest compressed. + {ok,#{config:=StdHConfig3}=HConfig3} = logger:get_handler_config(?MODULE), + ok = logger:remove_handler(?MODULE), + ok = logger:add_handler( + ?MODULE,logger_std_h, + HConfig3#{config=>StdHConfig3#{max_no_bytes=>75, + max_no_files=>1, + compress_on_rotate=>true}}), + timer:sleep(100), + {ok,#file_info{size=0}} = file:read_file_info(Log), + {ok,#file_info{size=29}} = file:read_file_info(Log++".0.gz"), + [_] = filelib:wildcard(Log++".*"), + + %% Stop/start handler and turn off compression. Check that achives + %% are decompressed. + {ok,#{config:=StdHConfig4}=HConfig4} = logger:get_handler_config(?MODULE), + ok = logger:remove_handler(?MODULE), + ok = logger:add_handler( + ?MODULE,logger_std_h, + HConfig4#{config=>StdHConfig4#{compress_on_rotate=>false}}), + timer:sleep(100), + {ok,#file_info{size=0}} = file:read_file_info(Log), + {ok,#file_info{size=80}} = file:read_file_info(Log++".0"), + [_] = filelib:wildcard(Log++".*"), + + ok. +rotation_opts_restart_handler(cleanup,_Config) -> + ok = stop_handler(?MODULE). + %%%----------------------------------------------------------------- %%% send_requests(TO, Reqs = [{Mod,Func,Args,Res}|Rs]) -> @@ -1305,8 +1683,8 @@ start_handler(Name, TTY, _Config) when TTY == standard_io; ok = logger:add_handler(Name, logger_std_h, #{config => #{type => TTY}, - filter_default=>log, - filters=>?DEFAULT_HANDLER_FILTERS([Name]), + filter_default=>stop, + filters=>filter_only_this_domain(Name), formatter=>{?MODULE,op}}), {ok,HConfig = #{config := StdHConfig}} = logger:get_handler_config(Name), {HConfig,StdHConfig}; @@ -1320,12 +1698,17 @@ start_handler(Name, FuncName, Config) -> ok = logger:add_handler(Name, logger_std_h, #{config => #{type => Type}, - filter_default=>log, - filters=>?DEFAULT_HANDLER_FILTERS([Name]), + filter_default=>stop, + filters=>filter_only_this_domain(Name), formatter=>{?MODULE,op}}), {ok,HConfig = #{config := StdHConfig}} = logger:get_handler_config(Name), {Log,HConfig,StdHConfig}. + +filter_only_this_domain(Name) -> + [{remote_gl,{fun logger_filters:remote_gl/2,stop}}, + {domain,{fun logger_filters:domain/2,{log,super,[Name]}}}]. + stop_handler(Name) -> R = logger:remove_handler(Name), ct:pal("Handler ~p stopped! Result: ~p", [Name,R]), @@ -1658,7 +2041,7 @@ tpl([]) -> tracer({trace,_,call,{logger_h_common,handle_cast,[Op|_]}}, {Pid,[{Mod,Func,Op}|Expected]}) -> maybe_tracer_done(Pid,Expected,{Mod,Func,Op}); -tracer({trace,_,call,{Mod=logger_std_h,Func=write_to_dev,[_,Data,_,_,_]}}, +tracer({trace,_,call,{Mod=logger_std_h,Func=write_to_dev,[Data,_]}}, {Pid,[{Mod,Func,Data}|Expected]}) -> maybe_tracer_done(Pid,Expected,{Mod,Func,Data}); tracer({trace,_,call,{Mod,Func,_}}, {Pid,[{Mod,Func}|Expected]}) -> @@ -1742,4 +2125,3 @@ filesync_rep_int() -> file_delete(Log) -> file:delete(Log). - diff --git a/lib/kernel/test/logger_stress_SUITE.erl b/lib/kernel/test/logger_stress_SUITE.erl index 4072e8c86a..1a278fb1b2 100644 --- a/lib/kernel/test/logger_stress_SUITE.erl +++ b/lib/kernel/test/logger_stress_SUITE.erl @@ -67,6 +67,10 @@ all() -> reject_events, std_handler, disk_log_handler, + std_handler_time, + std_handler_time_big, + disk_log_handler_time, + disk_log_handler_time_big, emulator_events, remote_events, remote_to_disk_log, @@ -123,6 +127,20 @@ std_handler(cleanup,_Config) -> _ = file:delete("default.log"), ok. +%% Disable overload protection and just print a lot - measure time. +%% The IOPS reported is the number of log events written per millisecond. +std_handler_time(Config) -> + measure_handler_time(logger_std_h,#{type=>{file,"default.log"}},Config). +std_handler_time(cleanup,_Config) -> + _ = file:delete("default.log"), + ok. + +std_handler_time_big(Config) -> + measure_handler_time_big(logger_std_h,#{type=>{file,"default.log"}},Config). +std_handler_time_big(cleanup,_Config) -> + _ = file:delete("default.log"), + ok. + %% Cascading failure that produce gen_server and proc_lib reports - %% how many of the produced log events are actually written to a log %% with logger_disk_log_h wrap file handler. @@ -140,6 +158,20 @@ disk_log_handler(cleanup,_Config) -> [_ = file:delete(F) || F <- Files], ok. +%% Disable overload protection and just print a lot - measure time. +%% The IOPS reported is the number of log events written per millisecond. +disk_log_handler_time(Config) -> + measure_handler_time(logger_disk_log_h,#{type=>halt},Config). +disk_log_handler_time(cleanup,_Config) -> + _ = file:delete("default"), + ok. + +disk_log_handler_time_big(Config) -> + measure_handler_time_big(logger_disk_log_h,#{type=>halt},Config). +disk_log_handler_time_big(cleanup,_Config) -> + _ = file:delete("default"), + ok. + %% Cascading failure that produce log events from the emulator - how %% many of the produced log events pass through the proxy. emulator_events(Config) -> @@ -221,6 +253,68 @@ remote_emulator_to_disk_log(cleanup,_Config) -> %%%----------------------------------------------------------------- %%% Internal functions +measure_handler_time(Module,HCfg,Config) -> + measure_handler_time(Module,100000,small_fa(),millisecond,HCfg,#{},Config). + +measure_handler_time_big(Module,HCfg,Config) -> + FCfg = #{chars_limit=>4096, max_size=>1024}, + measure_handler_time(Module,100,big_fa(),second,HCfg,FCfg,Config). + +measure_handler_time(Module,N,FA,Unit,HCfg,FCfg,Config) -> + {ok,_,Node} = + logger_test_lib:setup( + Config, + [{logger, + [{handler,default,Module, + #{formatter => {logger_formatter, + maps:merge(#{legacy_header=>false, + single_line=>true},FCfg)}, + config=>maps:merge(#{sync_mode_qlen => N+1, + drop_mode_qlen => N+1, + flush_qlen => N+1, + burst_limit_enable => false, + filesync_repeat_interval => no_repeat}, + HCfg)}}]}]), + %% HPid = rpc:call(Node,erlang,whereis,[?name_to_reg_name(Module,default)]), + %% {links,[_,FCPid]} = rpc:call(Node,erlang,process_info,[HPid,links]), + T0 = erlang:monotonic_time(millisecond), + ok = rpc:call(Node,?MODULE,nlogs_wait,[N,FA,Module]), + %% ok = rpc:call(Node,fprof,apply,[fun ?MODULE:nlogs_wait/2,[N div 10,FA,Module],[{procs,[HPid,FCPid]}]]), + T1 = erlang:monotonic_time(millisecond), + T = T1-T0, + M = case Unit of + millisecond -> 1; + second -> 1000 + end, + IOPS = M*N/T, + ct:pal("N: ~p~nT: ~p~nIOPS: ~.2f events pr ~w",[N,T,IOPS,Unit]), + %% Stats = rpc:call(Node,logger_olp,info,[?name_to_reg_name(Module,default)]), + %% ct:pal("Stats: ~p",[Stats]), + ct_event:notify(#event{name = benchmark_data, + data = [{value,IOPS}]}), + {comment,io_lib:format("~.2f events written pr ~w",[IOPS,Unit])}. + +nlogs_wait(N,{F,A},Module) -> + group_leader(whereis(user),self()), + [?LOG_NOTICE(F,A) || _ <- lists:seq(1,N)], + wait(Module). + +wait(Module) -> + case Module:filesync(default) of + {error,handler_busy} -> + wait(Module); + ok -> + ok + end. + +small_fa() -> + Str = "\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "[\\]^_`abcdefghijklmnopqr", + {"~ts",[Str]}. + +big_fa() -> + {"~p",[lists:duplicate(1048576,"a")]}. + nlogs(N) -> group_leader(whereis(user),self()), Str = "\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ" diff --git a/lib/kernel/vsn.mk b/lib/kernel/vsn.mk index 4b43c6ae9d..7bebe1ba70 100644 --- a/lib/kernel/vsn.mk +++ b/lib/kernel/vsn.mk @@ -1 +1 @@ -KERNEL_VSN = 6.2 +KERNEL_VSN = 6.3 diff --git a/lib/mnesia/doc/src/notes.xml b/lib/mnesia/doc/src/notes.xml index 8fc3610bb6..01d1666b8d 100644 --- a/lib/mnesia/doc/src/notes.xml +++ b/lib/mnesia/doc/src/notes.xml @@ -39,7 +39,22 @@ thus constitutes one section in this document. The title of each section is the version number of Mnesia.</p> - <section><title>Mnesia 4.15.5</title> + <section><title>Mnesia 4.15.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Avoid overload warnings caused by a race condition.</p> + <p> + Own Id: OTP-15619 Aux Id: ERIERL-310 </p> + </item> + </list> + </section> + +</section> + +<section><title>Mnesia 4.15.5</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/mnesia/vsn.mk b/lib/mnesia/vsn.mk index 1cfb35c774..781a4830a0 100644 --- a/lib/mnesia/vsn.mk +++ b/lib/mnesia/vsn.mk @@ -1 +1 @@ -MNESIA_VSN = 4.15.5 +MNESIA_VSN = 4.15.6 diff --git a/lib/observer/doc/src/notes.xml b/lib/observer/doc/src/notes.xml index 22035af982..2d914f8c61 100644 --- a/lib/observer/doc/src/notes.xml +++ b/lib/observer/doc/src/notes.xml @@ -32,6 +32,45 @@ <p>This document describes the changes made to the Observer application.</p> +<section><title>Observer 2.9</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Since Logger was introduced in OTP-21.0, menu choice + <em>Log > Toggle Log View</em> in observer would cause + a crash unless an <c>error_logger</c> event handler was + explicitly installed. This is now corrected.</p> + <p> + Own Id: OTP-15553 Aux Id: ERL-848 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Since persistent_term was introduced, observer would + sometimes crash when expanding a term from a process + state. This is now corrected.</p> + <p> + Own Id: OTP-15493 Aux Id: ERL-810 </p> + </item> + <item> + <p> + Add <c>OBSERVER_SCALE</c> environment variable for HiDPI + support.</p> + <p> + Own Id: OTP-15586 Aux Id: PR-2105 </p> + </item> + </list> + </section> + +</section> + <section><title>Observer 2.8.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/observer/vsn.mk b/lib/observer/vsn.mk index 5ce0aca589..0e9c8b302c 100644 --- a/lib/observer/vsn.mk +++ b/lib/observer/vsn.mk @@ -1 +1 @@ -OBSERVER_VSN = 2.8.2 +OBSERVER_VSN = 2.9 diff --git a/lib/odbc/doc/src/notes.xml b/lib/odbc/doc/src/notes.xml index dba7663bb9..696fcaa479 100644 --- a/lib/odbc/doc/src/notes.xml +++ b/lib/odbc/doc/src/notes.xml @@ -32,7 +32,22 @@ <p>This document describes the changes made to the odbc application. </p> - <section><title>ODBC 2.12.2</title> + <section><title>ODBC 2.12.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Enhance error handling to avoid stack corruption</p> + <p> + Own Id: OTP-15667 Aux Id: ERL-808, PR-2065 </p> + </item> + </list> + </section> + +</section> + +<section><title>ODBC 2.12.2</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/odbc/vsn.mk b/lib/odbc/vsn.mk index bb21016fad..ff023e666b 100644 --- a/lib/odbc/vsn.mk +++ b/lib/odbc/vsn.mk @@ -1 +1 @@ -ODBC_VSN = 2.12.2 +ODBC_VSN = 2.12.3 diff --git a/lib/public_key/doc/src/notes.xml b/lib/public_key/doc/src/notes.xml index b3e6023c41..f6bc0dc797 100644 --- a/lib/public_key/doc/src/notes.xml +++ b/lib/public_key/doc/src/notes.xml @@ -35,6 +35,21 @@ <file>notes.xml</file> </header> +<section><title>Public_Key 1.6.5</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add export of dialyzer type</p> + <p> + Own Id: OTP-15624</p> + </item> + </list> + </section> + +</section> + <section><title>Public_Key 1.6.4</title> <section><title>Improvements and New Features</title> diff --git a/lib/public_key/vsn.mk b/lib/public_key/vsn.mk index 5e2643f0ea..11c06fb158 100644 --- a/lib/public_key/vsn.mk +++ b/lib/public_key/vsn.mk @@ -1 +1 @@ -PUBLIC_KEY_VSN = 1.6.4 +PUBLIC_KEY_VSN = 1.6.5 diff --git a/lib/runtime_tools/doc/src/notes.xml b/lib/runtime_tools/doc/src/notes.xml index 810bb8207c..58a2a66c4b 100644 --- a/lib/runtime_tools/doc/src/notes.xml +++ b/lib/runtime_tools/doc/src/notes.xml @@ -32,6 +32,21 @@ <p>This document describes the changes made to the Runtime_Tools application.</p> +<section><title>Runtime_Tools 1.13.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Update of systemtap trace example scripts.</p> + <p> + Own Id: OTP-15670</p> + </item> + </list> + </section> + +</section> + <section><title>Runtime_Tools 1.13.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/runtime_tools/vsn.mk b/lib/runtime_tools/vsn.mk index aa3d702997..fa2f338ec2 100644 --- a/lib/runtime_tools/vsn.mk +++ b/lib/runtime_tools/vsn.mk @@ -1 +1 @@ -RUNTIME_TOOLS_VSN = 1.13.1 +RUNTIME_TOOLS_VSN = 1.13.2 diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml index 0bc4baf5eb..17f14bdea2 100644 --- a/lib/ssh/doc/src/notes.xml +++ b/lib/ssh/doc/src/notes.xml @@ -30,6 +30,23 @@ <file>notes.xml</file> </header> +<section><title>Ssh 4.7.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + SSH sftp daemon now accepts an SSH_FXP_STAT message + encoded according to the wrong sftp version. Some clients + sends such messages.</p> + <p> + Own Id: OTP-15498 Aux Id: ERL-822, PR-2077 </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 4.7.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssh/test/ssh_protocol_SUITE.erl b/lib/ssh/test/ssh_protocol_SUITE.erl index 3e3e151781..b12ddfeef6 100644 --- a/lib/ssh/test/ssh_protocol_SUITE.erl +++ b/lib/ssh/test/ssh_protocol_SUITE.erl @@ -38,7 +38,11 @@ -define(EXTRA_KEX, 'diffie-hellman-group1-sha1'). -define(CIPHERS, ['aes256-ctr','aes192-ctr','aes128-ctr','aes128-cbc','3des-cbc']). --define(DEFAULT_CIPHERS, [{client2server,?CIPHERS}, {server2client,?CIPHERS}]). +-define(DEFAULT_CIPHERS, (fun() -> Ciphs = filter_supported(cipher, ?CIPHERS), + [{client2server,Ciphs}, {server2client,Ciphs}] + end)() + ). + -define(v(Key, Config), proplists:get_value(Key, Config)). -define(v(Key, Config, Default), proplists:get_value(Key, Config, Default)). diff --git a/lib/ssh/vsn.mk b/lib/ssh/vsn.mk index 2890d7fe5b..0f9eee887c 100644 --- a/lib/ssh/vsn.mk +++ b/lib/ssh/vsn.mk @@ -1,4 +1,4 @@ #-*-makefile-*- ; force emacs to enter makefile-mode -SSH_VSN = 4.7.3 +SSH_VSN = 4.7.4 APP_VSN = "ssh-$(SSH_VSN)" diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 82eb8ff700..732fdc71e7 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -27,6 +27,93 @@ </header> <p>This document describes the changes made to the SSL application.</p> +<section><title>SSL 9.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix bug that an incorrect return value for gen_statem + could be created when alert was a result of handling + renegotiation info extension</p> + <p> + Own Id: OTP-15502</p> + </item> + <item> + <p> + Correct check for 3des_ede_cbc, could cause ssl to claim + to support 3des_ede_cbc when cryptolib does not.</p> + <p> + Own Id: OTP-15539</p> + </item> + <item> + <p> + Improved DTLS error handling, avoids unexpected + connection failure in rare cases.</p> + <p> + Own Id: OTP-15561</p> + </item> + <item> + <p> + Corrected active once emulation bug that could cause the + ssl_closed meassage to not be sent. Bug introduced by + OTP-15449</p> + <p> + Own Id: OTP-15666 Aux Id: ERIERL-316, </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add client option {reuse_session, SessionID::binary()} + that can be used together with new option value + {reuse_sessions, save}. This makes it possible to reuse a + session from a specific connection establishment.</p> + <p> + Own Id: OTP-15369</p> + </item> + <item> + <p> + The Reason part of of the error return from the functions + connect and handshake has a better and documented format. + This will sometimes differ from previous returned + reasons, however those where only documented as term() + and should for that reason not be relied on.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15423</p> + </item> + <item> + <p> + Refactor of state handling to improve TLS application + data throughput and reduce CPU overhead</p> + <p> + Own Id: OTP-15445</p> + </item> + <item> + <p> + The SSL code has been optimized in many small ways to + reduce CPU load for encryption/decryption, especially for + Erlang's distribution protocol over TLS.</p> + <p> + Own Id: OTP-15529</p> + </item> + <item> + <p> + Add support for active N</p> + <p> + Own Id: OTP-15665 Aux Id: ERL-811, PR-2072 </p> + </item> + </list> + </section> + +</section> + <section><title>SSL 9.1.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index 254b7f3f4d..1e97fe046b 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -2737,6 +2737,11 @@ ssl_options_list([Key | Keys], [Value | Values], Acc) -> handle_active_option(false, connection = StateName, To, Reply, State) -> hibernate_after(StateName, State, [{reply, To, Reply}]); +handle_active_option(_, connection = StateName, To, _Reply, #state{connection_env = #connection_env{terminated = true}, + user_data_buffer = {_,0,_}} = State) -> + handle_normal_shutdown(?ALERT_REC(?FATAL, ?CLOSE_NOTIFY, all_data_deliverd), StateName, + State#state{start_or_recv_from = To}), + {stop,{shutdown, peer_close}, State}; handle_active_option(_, connection = StateName0, To, Reply, #state{static_env = #static_env{protocol_cb = Connection}, user_data_buffer = {_,0,_}} = State0) -> case Connection:next_event(StateName0, no_record, State0) of diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index 3527062a8a..0d9f907d5c 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1 +1 @@ -SSL_VSN = 9.1.2 +SSL_VSN = 9.2 diff --git a/lib/stdlib/doc/src/ets.xml b/lib/stdlib/doc/src/ets.xml index d2ac6a75b1..2cb677785d 100644 --- a/lib/stdlib/doc/src/ets.xml +++ b/lib/stdlib/doc/src/ets.xml @@ -642,12 +642,11 @@ Error: fun containing local Erlang function calls <p><marker id="info_2_safe_fixed_monotonic_time"/></p> <p><c>Item=safe_fixed|safe_fixed_monotonic_time, Value={FixationTime,Info}|false</c></p> - <p>If the table has been fixed using + <p>If the table is fixed using <seealso marker="#safe_fixtable/2"> <c>safe_fixtable/2</c></seealso>, the call returns a tuple where <c>FixationTime</c> is the - time when the table was first fixed by a process, which either - is or is not one of the processes it is fixed by now.</p> + last time when the table changed from unfixed to fixed.</p> <p>The format and value of <c>FixationTime</c> depends on <c>Item</c>:</p> <taglist> @@ -679,8 +678,15 @@ Error: fun containing local Erlang function calls table is fixed by now. <c>RefCount</c> is the value of the reference counter and it keeps track of how many times the table has been fixed by the process.</p> - <p>If the table never has been fixed, the call returns - <c>false</c>.</p> + <p>Table fixations are not limited to <seealso marker="#safe_fixtable/2"> + <c>safe_fixtable/2</c></seealso>. Temporary fixations may also + be done by for example <seealso marker="#traversal">traversing + functions</seealso> like <c>select</c> and <c>match</c>. Such + table fixations are automatically released before the + corresponding functions returns, but they may be seen by a + concurrent call to + <c>ets:info(T,safe_fixed|safe_fixed_monotonic_time)</c>.</p> + <p>If the table is not fixed at all, the call returns <c>false</c>.</p> </item> <item> <p><c>Item=stats, Value=tuple()</c></p> diff --git a/lib/stdlib/doc/src/gen_statem.xml b/lib/stdlib/doc/src/gen_statem.xml index 0cad5929ca..d4a4ba268b 100644 --- a/lib/stdlib/doc/src/gen_statem.xml +++ b/lib/stdlib/doc/src/gen_statem.xml @@ -320,12 +320,13 @@ erlang:'!' -----> Module:StateName/3 </p> <p> There is also a server start option - <seealso marker="#type-hibernate_after_opt"> + <seealso marker="#type-enter_loop_opt"> <c>{hibernate_after, Timeout}</c> </seealso> for - <seealso marker="#start/3"><c>start/3,4</c></seealso> or - <seealso marker="#start_link/3"><c>start_link/3,4</c></seealso> + <seealso marker="#start/3"><c>start/3,4</c></seealso>, + <seealso marker="#start_link/3"><c>start_link/3,4</c></seealso> or + <seealso marker="#enter_loop/4"><c>enter_loop/4,5,6</c></seealso>, that may be used to automatically hibernate the server. </p> </description> diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index dee7136eb1..23c3f6e981 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -31,6 +31,57 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 3.8</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug in the Erlang Pretty Printer: long atom + names in combination with <c><<>></c> could + cause a crash. </p> + <p> + Own Id: OTP-15592 Aux Id: ERL-818 </p> + </item> + <item> + <p> Fix bugs that could cause wrong results or bad + performance when formatting lists of characters using the + control sequences <c>p</c> or <c>P</c> and limiting the + output with the option <c>chars_limit</c>. </p> + <p> + Own Id: OTP-15639</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Improved ETS documentation about safe table traversal and + the partially bound key optimization for + <c>ordered_set</c>.</p> + <p> + Own Id: OTP-15545 Aux Id: PR-2103, PR-2139 </p> + </item> + <item> + <p> Optimize <c>calendar:gregorian_days_to_date/1</c>. + </p> + <p> + Own Id: OTP-15572 Aux Id: PR-2121 </p> + </item> + <item> + <p> Optimize functions + <c>calendar:rfc3339_to_system_time()</c> and + <c>calendar:system_time_to_rfc3339()</c>. </p> + <p> + Own Id: OTP-15630</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 3.7.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/src/io_lib_pretty.erl b/lib/stdlib/src/io_lib_pretty.erl index 5483ea87b5..8f2fd7ea8f 100644 --- a/lib/stdlib/src/io_lib_pretty.erl +++ b/lib/stdlib/src/io_lib_pretty.erl @@ -721,7 +721,7 @@ printable_list(_L, 1, _T, _Enc) -> printable_list(L, _D, T, latin1) when T < 0 -> io_lib:printable_latin1_list(L); printable_list(L, _D, T, Enc) when T >= 0 -> - case slice(L, tsub(T, 2)) of + case slice(L, tsub(T, 2), Enc) of false -> false; {prefix, Prefix} when Enc =:= latin1 -> @@ -737,20 +737,46 @@ printable_list(L, _D, T, Enc) when T >= 0 -> printable_list(L, _D, T, _Uni) when T < 0-> io_lib:printable_list(L). -slice(L, N) -> - try io_lib:chars_length(L) =< N of - true -> +slice(L, N, latin1) -> + try lists:split(N, L) of + {_, []} -> all; - false -> - case string:slice(L, 0, N) of - "" -> - false; - Prefix -> - {prefix, Prefix} + {[], _} -> + false; + {L1, _} -> + {prefix, L1} + catch + _:_ -> + all + end; +slice(L, N, _Uni) -> + %% Be careful not to traverse more of L than necessary. + try string:slice(L, 0, N) of + "" -> + false; + Prefix -> + %% Assume no binaries are introduced by string:slice(). + case is_flat(L, lists:flatlength(Prefix)) of + true -> + case string:equal(Prefix, L) of + true -> + all; + false -> + {prefix, Prefix} + end; + false -> + false end catch _:_ -> false end. +is_flat(_L, 0) -> + true; +is_flat([C|Cs], N) when is_integer(C) -> + is_flat(Cs, N - 1); +is_flat(_, _N) -> + false. + printable_bin0(Bin, D, T, Enc) -> Len = case D >= 0 of true -> diff --git a/lib/stdlib/src/stdlib.appup.src b/lib/stdlib/src/stdlib.appup.src index 9e5d6a3bd8..37ea97c353 100644 --- a/lib/stdlib/src/stdlib.appup.src +++ b/lib/stdlib/src/stdlib.appup.src @@ -40,7 +40,8 @@ {<<"^3\\.6$">>,[restart_new_emulator]}, {<<"^3\\.6\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.7$">>,[restart_new_emulator]}, - {<<"^3\\.7\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}], + {<<"^3\\.7\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^3\\.7\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], [{<<"^3\\.4$">>,[restart_new_emulator]}, {<<"^3\\.4\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.4\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, @@ -54,4 +55,5 @@ {<<"^3\\.6$">>,[restart_new_emulator]}, {<<"^3\\.6\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.7$">>,[restart_new_emulator]}, - {<<"^3\\.7\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}]}. + {<<"^3\\.7\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^3\\.7\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. diff --git a/lib/stdlib/test/ets_SUITE.erl b/lib/stdlib/test/ets_SUITE.erl index 22c77aa172..8561491d50 100644 --- a/lib/stdlib/test/ets_SUITE.erl +++ b/lib/stdlib/test/ets_SUITE.erl @@ -56,9 +56,11 @@ -export([t_match_spec_run/1]). -export([t_bucket_disappears/1]). -export([t_named_select/1]). +-export([select_fixtab_owner_change/1]). -export([otp_5340/1]). -export([otp_6338/1]). -export([otp_6842_select_1000/1]). +-export([select_mbuf_trapping/1]). -export([otp_7665/1]). -export([meta_wb/1]). -export([grow_shrink/1, grow_pseudo_deleted/1, shrink_pseudo_deleted/1]). @@ -129,9 +131,10 @@ all() -> t_insert_list, t_test_ms, t_select_delete, t_select_replace, t_select_replace_next_bug, t_ets_dets, memory, t_select_reverse, t_bucket_disappears, - t_named_select, + t_named_select, select_fixtab_owner_change, select_fail, t_insert_new, t_repair_continuation, otp_5340, otp_6338, otp_6842_select_1000, otp_7665, + select_mbuf_trapping, otp_8732, meta_wb, grow_shrink, grow_pseudo_deleted, shrink_pseudo_deleted, {group, meta_smp}, smp_insert, smp_fixed_delete, smp_unfix_fix, smp_select_replace, @@ -247,7 +250,64 @@ t_named_select_do(Opts) -> verify_etsmem(EtsMem). +%% Verify select and friends release fixtab as they should +%% even when owneship is changed between traps. +select_fixtab_owner_change(_Config) -> + T = ets:new(xxx, [protected]), + NKeys = 2000, + [ets:insert(T,{K,K band 7}) || K <- lists:seq(1,NKeys)], + %% Buddy and Papa will ping-pong table ownership between them + %% and the aim is to give Buddy the table when he is + %% in the middle of a yielding select* call. + {Buddy,_} = spawn_opt(fun() -> sfoc_buddy_loop(T, 1, undefined) end, + [link,monitor]), + + sfoc_papa_loop(T, Buddy), + + receive {'DOWN', _, process, Buddy, _} -> ok end, + ets:delete(T), + ok. + +sfoc_buddy_loop(T, I, State0) -> + receive + {'ETS-TRANSFER', T, Papa, _} -> + ets:give_away(T, Papa, State0), + case State0 of + done -> + ok; + _ -> + State1 = sfoc_traverse(T, I, State0), + %% Verify no fixation left + {I, false} = {I, ets:info(T, safe_fixed_monotonic_time)}, + sfoc_buddy_loop(T, I+1, State1) + end + end. + +sfoc_papa_loop(T, Buddy) -> + ets:give_away(T, Buddy, "Catch!"), + receive + {'ETS-TRANSFER', T, Buddy, State} -> + case State of + done -> + ok; + _ -> + sfoc_papa_loop(T, Buddy) + end + end. + +sfoc_traverse(T, 1, S) -> + ets:select(T, [{{'$1',7}, [], ['$1']}]), S; +sfoc_traverse(T, 2, S) -> + 0 = ets:select_count(T, [{{'$1',7}, [], [false]}]), S; +sfoc_traverse(T, 3, _) -> + Limit = ets:info(T, size) div 2, + {_, Continuation} = ets:select(T, [{{'$1',7}, [], ['$1']}], + Limit), + Continuation; +sfoc_traverse(_T, 4, Continuation) -> + _ = ets:select(Continuation), + done. %% Check ets:match_spec_run/2. t_match_spec_run(Config) when is_list(Config) -> @@ -5292,6 +5352,61 @@ otp_6338(Config) when is_list(Config) -> end), ok. +%% OTP-15660: Verify select not doing excessive trapping +%% when process have mbuf heap fragments. +select_mbuf_trapping(Config) when is_list(Config) -> + select_mbuf_trapping_do(set), + select_mbuf_trapping_do(ordered_set). + +select_mbuf_trapping_do(Type) -> + T = ets:new(xxx, [Type]), + NKeys = 50, + [ets:insert(T, {K, value}) || K <- lists:seq(1,NKeys)], + + {priority, Prio} = process_info(self(), priority), + Tracee = self(), + [SchedTracer] + = start_loopers(1, Prio, + fun (SC) -> + receive + {trace, Tracee, out, _} -> + SC+1; + done -> + Tracee ! {schedule_count, SC}, + exit(normal) + end + end, + 0), + + erlang:garbage_collect(), + 1 = erlang:trace(self(), true, [running,{tracer,SchedTracer}]), + + %% Artificially create an mbuf heap fragment + MbufTerm = "Frag me up", + MbufTerm = erts_debug:set_internal_state(mbuf, MbufTerm), + + Keys = ets:select(T, [{{'$1', value}, [], ['$1']}]), + NKeys = length(Keys), + + 1 = erlang:trace(self(), false, [running]), + Ref = erlang:trace_delivered(Tracee), + receive + {trace_delivered, Tracee, Ref} -> + SchedTracer ! done + end, + receive + {schedule_count, N} -> + io:format("~p context switches: ~p", [Type,N]), + if + N < 3 -> ok; + true -> ct:fail(failed) + end + end, + true = ets:delete(T), + ok. + + + %% Elements could come in the wrong order in a bag if a rehash occurred. otp_5340(Config) when is_list(Config) -> repeat_for_opts(fun otp_5340_do/1). diff --git a/lib/stdlib/test/io_SUITE.erl b/lib/stdlib/test/io_SUITE.erl index f097552e8c..824f5d19f2 100644 --- a/lib/stdlib/test/io_SUITE.erl +++ b/lib/stdlib/test/io_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2018. All Rights Reserved. +%% Copyright Ericsson AB 1999-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -32,7 +32,7 @@ io_with_huge_message_queue/1, format_string/1, maps/1, coverage/1, otp_14178_unicode_atoms/1, otp_14175/1, otp_14285/1, limit_term/1, otp_14983/1, otp_15103/1, otp_15076/1, - otp_15159/1]). + otp_15159/1, otp_15639/1]). -export([pretty/2, trf/3]). @@ -64,7 +64,8 @@ all() -> io_lib_print_binary_depth_one, otp_10302, otp_10755, otp_10836, io_lib_width_too_small, io_with_huge_message_queue, format_string, maps, coverage, otp_14178_unicode_atoms, otp_14175, - otp_14285, limit_term, otp_14983, otp_15103, otp_15076, otp_15159]. + otp_14285, limit_term, otp_14983, otp_15103, otp_15076, otp_15159, + otp_15639]. %% Error cases for output. error_1(Config) when is_list(Config) -> @@ -2647,3 +2648,35 @@ otp_15076(_Config) -> {'EXIT', {badarg, _}} = (catch io_lib:build_text(L)), {'EXIT', {badarg, _}} = (catch io_lib:build_text(L, [])), ok. + +otp_15639(_Config) -> + L = lists:duplicate(10, "a"), + LOpts = [{encoding, latin1}, {chars_limit, 10}], + UOpts = [{encoding, unicode}, {chars_limit, 10}], + "[[...]|...]" = pretty(L, LOpts), + "[[...]|...]" = pretty(L, UOpts), + "[\"a\",[...]|...]" = + pretty(L, [{chars_limit, 12}, {encoding, latin1}]), + "[\"a\",[...]|...]" = + pretty(L, [{chars_limit, 12}, {encoding, unicode}]), + + %% Latin-1 + "\"12345678\"" = pretty("12345678", LOpts), + "\"12345678\"..." = pretty("123456789", LOpts), + "\"\\r\\n123456\"..." = pretty("\r\n1234567", LOpts), + "\"\\r1234567\"..." = pretty("\r12345678", LOpts), + "\"\\r\\n123456\"..." = pretty("\r\n12345678", LOpts), + "\"12345678\"..." = pretty("12345678"++[x], LOpts), + "[49,50|...]" = pretty("1234567"++[x], LOpts), + "[49,x]" = pretty("1"++[x], LOpts), + "[[...]|...]" = pretty(["1","2","3","4","5","6","7","8"], LOpts), + %% Unicode + "\"12345678\"" = pretty("12345678", UOpts), + "\"12345678\"..." = pretty("123456789", UOpts), + "\"\\r\\n1234567\"" = pretty("\r\n1234567", UOpts), + "\"\\r1234567\"..." = pretty("\r12345678", UOpts), + "\"\\r\\n1234567\"..." = pretty("\r\n12345678", UOpts), + "[49,50|...]" = pretty("12345678"++[x], UOpts), + "\"12345678\"..." = pretty("123456789"++[x], UOpts), + "[[...]|...]" = pretty(["1","2","3","4","5","6","7","8"], UOpts), + ok. diff --git a/lib/stdlib/vsn.mk b/lib/stdlib/vsn.mk index d46173497b..cbefd6590a 100644 --- a/lib/stdlib/vsn.mk +++ b/lib/stdlib/vsn.mk @@ -1 +1 @@ -STDLIB_VSN = 3.7.1 +STDLIB_VSN = 3.8 diff --git a/lib/syntax_tools/doc/src/notes.xml b/lib/syntax_tools/doc/src/notes.xml index dc13fe474b..772f5e6e04 100644 --- a/lib/syntax_tools/doc/src/notes.xml +++ b/lib/syntax_tools/doc/src/notes.xml @@ -32,6 +32,20 @@ <p>This document describes the changes made to the Syntax_Tools application.</p> +<section><title>Syntax_Tools 2.1.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix pretty-printing of type funs. </p> + <p> + Own Id: OTP-15519 Aux Id: ERL-815 </p> + </item> + </list> + </section> + +</section> + <section><title>Syntax_Tools 2.1.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/syntax_tools/vsn.mk b/lib/syntax_tools/vsn.mk index 8959ebbd04..538c71dc24 100644 --- a/lib/syntax_tools/vsn.mk +++ b/lib/syntax_tools/vsn.mk @@ -1 +1 @@ -SYNTAX_TOOLS_VSN = 2.1.6 +SYNTAX_TOOLS_VSN = 2.1.7 diff --git a/lib/tools/c_src/Makefile.in b/lib/tools/c_src/Makefile.in index cfe91917f8..289322b6fa 100644 --- a/lib/tools/c_src/Makefile.in +++ b/lib/tools/c_src/Makefile.in @@ -29,7 +29,6 @@ CC=@CC@ LD=@LD@ AR=@AR@ RANLIB=@RANLIB@ -RM=@RM@ MKDIR=@MKDIR@ INSTALL=@INSTALL@ INSTALL_DIR=@INSTALL_DIR@ @@ -158,9 +157,9 @@ $(ERTS_LIB): docs: clean: - $(RM) -rf ../obj/* - $(RM) -rf ../bin/* - $(RM) -f ./*~ + $(RM) -r ../obj/* + $(RM) -r ../bin/* + $(RM) ./*~ .PHONY: all erts_lib docs clean diff --git a/lib/tools/doc/src/notes.xml b/lib/tools/doc/src/notes.xml index a6781dfdb3..28f8346a19 100644 --- a/lib/tools/doc/src/notes.xml +++ b/lib/tools/doc/src/notes.xml @@ -31,6 +31,45 @@ </header> <p>This document describes the changes made to the Tools application.</p> +<section><title>Tools 3.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Minor fixes for <c>make clean</c>.</p> + <p> + Own Id: OTP-15657</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + In the HTML file generated by + <c>cover:analyse_to_file/1,2</c>, a link is now added to + the line number. This makes it easier to share pointers + to specific lines.</p> + <p> + Own Id: OTP-15541</p> + </item> + <item> + <p> + Uncovered lines are now marked with a sad face, + <c>:-(</c>, in the HTML output from + <c>cover:analyse_to_file/1,2</c>. This is to make these + lines easier to find by search.</p> + <p> + Own Id: OTP-15542</p> + </item> + </list> + </section> + +</section> + <section><title>Tools 3.0.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/tools/vsn.mk b/lib/tools/vsn.mk index bb8305e9f1..5700885549 100644 --- a/lib/tools/vsn.mk +++ b/lib/tools/vsn.mk @@ -1 +1 @@ -TOOLS_VSN = 3.0.2 +TOOLS_VSN = 3.1 diff --git a/lib/wx/doc/src/notes.xml b/lib/wx/doc/src/notes.xml index 1061e73138..33d02e22ba 100644 --- a/lib/wx/doc/src/notes.xml +++ b/lib/wx/doc/src/notes.xml @@ -32,6 +32,25 @@ <p>This document describes the changes made to the wxErlang application.</p> +<section><title>Wx 1.8.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Improved support for wxWidgets 3.1.3 which have changed + <c>wxFONTWEIGTH</c>, also added <c>wxGCDC</c> and + <c>wxDisplay</c> modules.</p> + <p> + Fixed a crash on Mojave and check for events more often.</p> + <p> + Own Id: OTP-15587</p> + </item> + </list> + </section> + +</section> + <section><title>Wx 1.8.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/wx/vsn.mk b/lib/wx/vsn.mk index d241a7a1b4..dac219fa98 100644 --- a/lib/wx/vsn.mk +++ b/lib/wx/vsn.mk @@ -1 +1 @@ -WX_VSN = 1.8.6 +WX_VSN = 1.8.7 diff --git a/make/install_dir_data.sh.in b/make/install_dir_data.sh.in new file mode 100644 index 0000000000..8c1dc3d889 --- /dev/null +++ b/make/install_dir_data.sh.in @@ -0,0 +1,70 @@ +#!/bin/sh +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2019. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# + +# +# install_dir_data.sh <SourceDir> <DestDir> +# +# Install all content in <SourceDir> including subdirectories +# into <DestDir>. +# + +INSTALL="@INSTALL@" +INSTALL_DIR="@INSTALL_DIR@" +INSTALL_DATA="@INSTALL_DATA@" + +debug=yes + +error () { + echo "ERROR: $1" 1>&2 + exit 1 +} + +usage () { + error "$1\n Usage $progname <SourceDir> <DestDir>" +} + +cmd () { + [ $debug = no ] || echo "$@" + "$@" || exit 1 +} + +progname="$0" + +[ $# -eq 2 ] || usage "Invalid amount of arguments" + +src="$1" +dest="$2" + +cmd cd "$src" + +for dir in `find . -type d`; do + destdir="$dest" + [ "$dir" = "." ] || destdir="$dest/$dir" + cmd $INSTALL_DIR "$destdir" +done + +for file in `find . -type f`; do + subdir=`dirname "$file"` + destdir="$dest" + [ "$subdir" = "." ] || destdir="$dest/$subdir" + cmd $INSTALL_DATA "$file" "$destdir" +done + +exit 0 diff --git a/make/otp.mk.in b/make/otp.mk.in index df29d26833..ceff8f7c31 100644 --- a/make/otp.mk.in +++ b/make/otp.mk.in @@ -73,6 +73,7 @@ INSTALL_DIR = @INSTALL_DIR@ INSTALL_PROGRAM = @INSTALL_PROGRAM@ INSTALL_SCRIPT = @INSTALL_SCRIPT@ INSTALL_DATA = @INSTALL_DATA@ +INSTALL_DIR_DATA = $(ERL_TOP)/make/install_dir_data.sh CC = @CC@ GCC = @GCC@ @@ -274,6 +275,7 @@ XSLTPROC = @XSLTPROC@ FOP = @FOP@ XMLLINT = @XMLLINT@ CP = @CP@ +MKDIR = @MKDIR@ DOCGEN=$(ERL_TOP)/lib/erl_docgen FOP_CONFIG = $(DOCGEN)/priv/fop.xconf diff --git a/make/otp_version_tickets_in_merge b/make/otp_version_tickets_in_merge index e69de29bb2..77633c2ce2 100644 --- a/make/otp_version_tickets_in_merge +++ b/make/otp_version_tickets_in_merge @@ -0,0 +1,79 @@ +OTP-14702 +OTP-15229 +OTP-15298 +OTP-15369 +OTP-15375 +OTP-15398 +OTP-15423 +OTP-15442 +OTP-15445 +OTP-15447 +OTP-15460 +OTP-15479 +OTP-15484 +OTP-15490 +OTP-15493 +OTP-15494 +OTP-15498 +OTP-15502 +OTP-15503 +OTP-15508 +OTP-15514 +OTP-15518 +OTP-15519 +OTP-15527 +OTP-15529 +OTP-15539 +OTP-15540 +OTP-15541 +OTP-15542 +OTP-15545 +OTP-15552 +OTP-15553 +OTP-15555 +OTP-15556 +OTP-15557 +OTP-15558 +OTP-15561 +OTP-15562 +OTP-15567 +OTP-15569 +OTP-15570 +OTP-15572 +OTP-15576 +OTP-15577 +OTP-15578 +OTP-15580 +OTP-15583 +OTP-15584 +OTP-15586 +OTP-15587 +OTP-15592 +OTP-15599 +OTP-15600 +OTP-15601 +OTP-15602 +OTP-15604 +OTP-15605 +OTP-15619 +OTP-15624 +OTP-15625 +OTP-15629 +OTP-15630 +OTP-15634 +OTP-15637 +OTP-15639 +OTP-15642 +OTP-15647 +OTP-15650 +OTP-15654 +OTP-15657 +OTP-15659 +OTP-15660 +OTP-15662 +OTP-15663 +OTP-15665 +OTP-15666 +OTP-15667 +OTP-15669 +OTP-15670 @@ -30,7 +30,7 @@ AUTOCONF_SUBDIRS="$AUTOCONF_SUBDIRS make erts" # partly built in one of the bootstrap phases. Applications that # only get some static includes copied into the bootstrap directory # should not be included. -bootstrap_apps="erts lib/asn1 lib/compiler lib/hipe lib/ic lib/kernel lib/parsetools lib/sasl lib/snmp lib/stdlib lib/syntax_tools" +bootstrap_apps="erts lib/asn1 lib/compiler lib/erl_interface lib/hipe lib/kernel lib/jinterface lib/parsetools lib/sasl lib/snmp lib/stdlib lib/syntax_tools lib/wx" # We will quote a bit more than needed, but the important thing is that # all that needs quoting will be quoted... diff --git a/otp_patch_apply b/otp_patch_apply index 0127b96a73..3ff929ccbb 100755 --- a/otp_patch_apply +++ b/otp_patch_apply @@ -19,7 +19,7 @@ # %CopyrightEnd% # -version="1.0.1" +version="1.0.2" force= lib_path= @@ -388,9 +388,7 @@ if [ $install_docs = yes ]; then TESTROOT="$idir" release_docs) || exit 1 done - (cd "$sdir/system/doc/top" && $MAKE clean) - - (cd "$sdir/system/doc/top" && \ + (cd "$sdir/system/doc" && \ $MAKE MAKE="$MAKE" RELEASE_ROOT="$idir" RELEASE_PATH="$idir" \ TESTROOT="$idir" release_docs) || exit 1 diff --git a/otp_versions.table b/otp_versions.table index aea1e8d5cd..eef6a9c8fe 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -1,3 +1,5 @@ +OTP-21.3 : common_test-1.17 compiler-7.3.2 crypto-4.4.1 dialyzer-3.3.2 diameter-2.2 edoc-0.10 erl_docgen-0.9 erl_interface-3.11 erts-10.3 ftp-1.0.2 hipe-3.18.3 inets-7.0.6 kernel-6.3 mnesia-4.15.6 observer-2.9 odbc-2.12.3 public_key-1.6.5 runtime_tools-1.13.2 ssh-4.7.4 ssl-9.2 stdlib-3.8 syntax_tools-2.1.7 tools-3.1 wx-1.8.7 # asn1-5.0.8 debugger-4.2.6 eldap-1.2.6 et-1.6.4 eunit-2.3.7 jinterface-1.9.1 megaco-3.18.4 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 reltool-0.7.8 sasl-3.3 snmp-5.2.12 tftp-1.0.1 xmerl-1.3.19 : +OTP-21.2.7 : erts-10.2.5 kernel-6.2.1 # asn1-5.0.8 common_test-1.16.1 compiler-7.3.1 crypto-4.4 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 et-1.6.4 eunit-2.3.7 ftp-1.0.1 hipe-3.18.2 inets-7.0.5 jinterface-1.9.1 megaco-3.18.4 mnesia-4.15.5 observer-2.8.2 odbc-2.12.2 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.4 reltool-0.7.8 runtime_tools-1.13.1 sasl-3.3 snmp-5.2.12 ssh-4.7.3 ssl-9.1.2 stdlib-3.7.1 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.2 wx-1.8.6 xmerl-1.3.19 : OTP-21.2.6 : erts-10.2.4 stdlib-3.7.1 # asn1-5.0.8 common_test-1.16.1 compiler-7.3.1 crypto-4.4 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 et-1.6.4 eunit-2.3.7 ftp-1.0.1 hipe-3.18.2 inets-7.0.5 jinterface-1.9.1 kernel-6.2 megaco-3.18.4 mnesia-4.15.5 observer-2.8.2 odbc-2.12.2 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.4 reltool-0.7.8 runtime_tools-1.13.1 sasl-3.3 snmp-5.2.12 ssh-4.7.3 ssl-9.1.2 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.2 wx-1.8.6 xmerl-1.3.19 : OTP-21.2.5 : inets-7.0.5 # asn1-5.0.8 common_test-1.16.1 compiler-7.3.1 crypto-4.4 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 erts-10.2.3 et-1.6.4 eunit-2.3.7 ftp-1.0.1 hipe-3.18.2 jinterface-1.9.1 kernel-6.2 megaco-3.18.4 mnesia-4.15.5 observer-2.8.2 odbc-2.12.2 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.4 reltool-0.7.8 runtime_tools-1.13.1 sasl-3.3 snmp-5.2.12 ssh-4.7.3 ssl-9.1.2 stdlib-3.7 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.2 wx-1.8.6 xmerl-1.3.19 : OTP-21.2.4 : erts-10.2.3 inets-7.0.4 # asn1-5.0.8 common_test-1.16.1 compiler-7.3.1 crypto-4.4 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 et-1.6.4 eunit-2.3.7 ftp-1.0.1 hipe-3.18.2 jinterface-1.9.1 kernel-6.2 megaco-3.18.4 mnesia-4.15.5 observer-2.8.2 odbc-2.12.2 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.4 reltool-0.7.8 runtime_tools-1.13.1 sasl-3.3 snmp-5.2.12 ssh-4.7.3 ssl-9.1.2 stdlib-3.7 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.2 wx-1.8.6 xmerl-1.3.19 : diff --git a/system/doc/design_principles/Makefile b/system/doc/design_principles/Makefile index 242bf1c9a4..2fbd7d087f 100644 --- a/system/doc/design_principles/Makefile +++ b/system/doc/design_principles/Makefile @@ -88,7 +88,6 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) $(HTMLDIR)/%.gif: %.gif $(INSTALL_DATA) $< $@ @@ -107,8 +106,8 @@ images: $(IMAGE_FILES:%=$(HTMLDIR)/%) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/design_principles/statem.xml b/system/doc/design_principles/statem.xml index 07b4284cb8..45ea972ff2 100644 --- a/system/doc/design_principles/statem.xml +++ b/system/doc/design_principles/statem.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2016</year><year>2018</year> + <year>2016</year><year>2019</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -2415,17 +2415,18 @@ handle_event(enter, _OldState, {open,_}, _Data) -> <seealso marker="#Event Time-Outs">event time-out</seealso> to trigger hibernation after a certain time of inactivity. There is also a server start option - <seealso marker="stdlib:gen_statem#type-hibernate_after_opt"> + <seealso marker="stdlib:gen_statem#type-enter_loop_opt"> <c>{hibernate_after, Timeout}</c> </seealso> for - <seealso marker="stdlib:gen_statem#start/3"> - <c>start/3,4</c> - </seealso> - or + <seealso marker="stdlib:gen_statem#start/3"><c>start/3,4</c></seealso>, <seealso marker="stdlib:gen_statem#start_link/3"> <c>start_link/3,4</c> </seealso> + or + <seealso marker="stdlib:gen_statem#enter_loop/4"> + <c>enter_loop/4,5,6</c> + </seealso> that may be used to automatically hibernate the server. </p> <p> diff --git a/system/doc/efficiency_guide/Makefile b/system/doc/efficiency_guide/Makefile index 72bcd2ee73..a2742a1354 100644 --- a/system/doc/efficiency_guide/Makefile +++ b/system/doc/efficiency_guide/Makefile @@ -87,7 +87,6 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) docs: html @@ -98,8 +97,8 @@ html: $(GIF_FILES) $(HTML_UG_FILE) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/embedded/Makefile b/system/doc/embedded/Makefile index 396aef276b..1604075312 100644 --- a/system/doc/embedded/Makefile +++ b/system/doc/embedded/Makefile @@ -75,7 +75,6 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) docs: html @@ -86,8 +85,8 @@ html: $(GIF_FILES) $(HTML_UG_FILE) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/general_info/Makefile b/system/doc/general_info/Makefile index 70db7c525a..539075280e 100644 --- a/system/doc/general_info/Makefile +++ b/system/doc/general_info/Makefile @@ -68,7 +68,6 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) docs: html @@ -79,8 +78,8 @@ html: $(GIF_FILES) $(HTML_UG_FILE) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/general_info/deprecations.xml b/system/doc/general_info/deprecations.xml index ccfd553bad..c748e60059 100644 --- a/system/doc/general_info/deprecations.xml +++ b/system/doc/general_info/deprecations.xml @@ -61,6 +61,27 @@ library has also been <seealso marker="scheduled_for_removal#OTP-23">scheduled for removal</seealso>.</p> </section> + <section> + <title>System Events</title> + <p> + The format of "System Events" as defined in the man page for + <seealso marker="stdlib:sys">sys</seealso> + has been clarified and cleaned up. + Due to this, code that relied on the internal badly + documented previous (before this change) format + of OTP's "System Events", needs to be changed. + </p> + <p> + In the wake of this the function + <seealso marker="stdlib:sys#get_debug/3">sys:get_debug/3</seealso> + that returns data with undocumented and internal format + (and therefore is practically useless) has been deprecated, + and a new function + <seealso marker="stdlib:sys#get_log/1">sys:get_log/1</seealso> + has been added, + that hopefully does what the deprecated function was intended for. + </p> + </section> </section> <section> <marker id="OTP-18"/> diff --git a/system/doc/getting_started/Makefile b/system/doc/getting_started/Makefile index cdf1e121c2..1c917895d5 100644 --- a/system/doc/getting_started/Makefile +++ b/system/doc/getting_started/Makefile @@ -74,7 +74,6 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) docs: html @@ -85,8 +84,8 @@ html: $(GIF_FILES) $(HTML_UG_FILE) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/html/design_principles/.gitignore b/system/doc/html/design_principles/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/design_principles/.gitignore diff --git a/system/doc/html/efficiency_guide/.gitignore b/system/doc/html/efficiency_guide/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/efficiency_guide/.gitignore diff --git a/system/doc/html/embedded/.gitignore b/system/doc/html/embedded/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/embedded/.gitignore diff --git a/system/doc/html/general_info/.gitignore b/system/doc/html/general_info/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/general_info/.gitignore diff --git a/system/doc/html/getting_started/.gitignore b/system/doc/html/getting_started/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/getting_started/.gitignore diff --git a/system/doc/html/installation_guide/.gitignore b/system/doc/html/installation_guide/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/installation_guide/.gitignore diff --git a/system/doc/html/installation_guide/source/.gitignore b/system/doc/html/installation_guide/source/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/installation_guide/source/.gitignore diff --git a/system/doc/html/js/.gitignore b/system/doc/html/js/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/js/.gitignore diff --git a/system/doc/html/oam/.gitignore b/system/doc/html/oam/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/oam/.gitignore diff --git a/system/doc/html/programming_examples/.gitignore b/system/doc/html/programming_examples/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/programming_examples/.gitignore diff --git a/system/doc/html/reference_manual/.gitignore b/system/doc/html/reference_manual/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/reference_manual/.gitignore diff --git a/system/doc/html/system_architecture_intro/.gitignore b/system/doc/html/system_architecture_intro/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/system_architecture_intro/.gitignore diff --git a/system/doc/html/system_principles/.gitignore b/system/doc/html/system_principles/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/system_principles/.gitignore diff --git a/system/doc/html/tutorial/.gitignore b/system/doc/html/tutorial/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/html/tutorial/.gitignore diff --git a/system/doc/installation_guide/Makefile b/system/doc/installation_guide/Makefile index 4a1335cf31..38252757d6 100644 --- a/system/doc/installation_guide/Makefile +++ b/system/doc/installation_guide/Makefile @@ -91,7 +91,6 @@ $(XMLDIR)/%.xml: $(ERL_TOP)/HOWTO/%.md $(ERL_TOP)/make/emd2exml $(ERL_TOP)/make/emd2exml $< $@ $(REDIRECT_HTML_DIR)/%.html: Makefile - test -d $(REDIRECT_HTML_DIR) || $(INSTALL_DIR) $(REDIRECT_HTML_DIR) echo "<html><head><meta HTTP-EQUIV=\"REFRESH\"" > $@ echo " content=\"5; url=../"$(notdir $@)"\">" >> $@ echo "<title>This page has moved</title></head><body>" >> $@ @@ -112,8 +111,8 @@ debug opt: clean clean_docs: rm -f $(GENERATED_XML_FILES) - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/oam/Makefile b/system/doc/oam/Makefile index 147f56f885..2eb429e04d 100644 --- a/system/doc/oam/Makefile +++ b/system/doc/oam/Makefile @@ -71,10 +71,9 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) $(HTMLDIR)/%.gif: %.gif - $(INSTALL_DATA) $< $@ + $(CP) $< $@ docs: html @@ -87,8 +86,8 @@ gifs: $(GIF_FILES:%=$(HTMLDIR)/%) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/programming_examples/Makefile b/system/doc/programming_examples/Makefile index e4737ba069..9c67c24b64 100644 --- a/system/doc/programming_examples/Makefile +++ b/system/doc/programming_examples/Makefile @@ -74,7 +74,6 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) docs: html local_docs: PDFDIR=../../pdf @@ -84,8 +83,8 @@ html: $(GIF_FILES) $(HTML_UG_FILE) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/reference_manual/Makefile b/system/doc/reference_manual/Makefile index d034ad2ff8..809eb2c979 100644 --- a/system/doc/reference_manual/Makefile +++ b/system/doc/reference_manual/Makefile @@ -84,7 +84,6 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) docs: html @@ -95,8 +94,8 @@ html: $(GIF_FILES) $(HTML_UG_FILE) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/system_architecture_intro/Makefile b/system/doc/system_architecture_intro/Makefile index eb885a744d..ea9ee85105 100644 --- a/system/doc/system_architecture_intro/Makefile +++ b/system/doc/system_architecture_intro/Makefile @@ -69,7 +69,6 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) docs: html @@ -80,8 +79,8 @@ html: $(GIF_FILES) $(HTML_UG_FILE) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/system_principles/Makefile b/system/doc/system_principles/Makefile index 1979deda4c..5110b73373 100644 --- a/system/doc/system_principles/Makefile +++ b/system/doc/system_principles/Makefile @@ -70,7 +70,6 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) docs: html @@ -81,8 +80,8 @@ html: $(GIF_FILES) $(HTML_UG_FILE) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/top/Makefile b/system/doc/top/Makefile index 65d2fcfb6b..e3f9c4710a 100644 --- a/system/doc/top/Makefile +++ b/system/doc/top/Makefile @@ -101,6 +101,25 @@ PDFREFDIR= pdf TOP_PDF_FILE = $(PDFDIR)/$(APPLICATION)-$(VSN).pdf TOPDOC=true +ifdef RELEASE_PATH +INST_TYPE=rel +INST_TYPE_SRC_DIR=$(RELEASE_PATH) +# We build to the 'temporary' dir in order to be able to install +# results using INSTALL_DATA (in order to get correct access +# rights on installed files) +INST_TYPE_DEST_DIR=$(RELSYSDIR)/temporary +INST_TYPE_DEST_DIR_DEP=$(INST_TYPE_DEST_DIR) +INST_TYPE_JS_DEST_DIR=$(INST_TYPE_DEST_DIR) +INST_TYPE_VSN_FILE=$(INST_TYPE_DEST_DIR)/OTP_VERSION +else +INST_TYPE=src +INST_TYPE_SRC_DIR=$(ERL_TOP) +INST_TYPE_DEST_DIR=$(HTMLDIR) +INST_TYPE_DEST_DIR_DEP= +INST_TYPE_JS_DEST_DIR=$(INST_TYPE_DEST_DIR)/js +INST_TYPE_VSN_FILE=$(ERL_TOP)/OTP_VERSION +endif + #-------------------------------------------------------------------------- # We generate the index page from the installed system. This make # it important that this is done last. The file index.html.src @@ -110,17 +129,18 @@ EBIN = ebin INDEX_SCRIPT = $(EBIN)/erl_html_tools.$(EMULATOR) INDEX_SRC = src/erl_html_tools.erl -INDEX_FILES = \ - $(HTMLDIR)/index.html \ - $(HTMLDIR)/applications.html -JAVASCRIPT = $(HTMLDIR)/js/erlresolvelinks.js +INDEX_HTML=$(INST_TYPE_DEST_DIR)/index.html +APPLICATIONS_HTML=$(INST_TYPE_DEST_DIR)/applications.html +INDEX_FILES = $(INDEX_HTML) $(APPLICATIONS_HTML) + +JAVASCRIPT = $(INST_TYPE_JS_DEST_DIR)/erlresolvelinks.js JAVASCRIPT_BUILD_SCRIPT = $(EBIN)/erlresolvelinks.$(EMULATOR) JAVASCRIPT_BUILD_SCRIPT_SRC = src/erlresolvelinks.erl MAN_INDEX_SCRIPT = $(EBIN)/otp_man_index.$(EMULATOR) MAN_INDEX_SRC = src/otp_man_index.erl -MAN_INDEX = $(HTMLDIR)/man_index.html +MAN_INDEX = $(INST_TYPE_DEST_DIR)/man_index.html GLOSSARY = $(HTMLDIR)/glossary.html GLOSSARY_SRC = $(ERL_TOP)/system/internal_tools/doctools/src/glossary.erl @@ -135,45 +155,38 @@ TEMPLATES = \ $(INDEX_SCRIPT): $(INDEX_SRC) $(ERLC) -o$(EBIN) +warn_unused_vars $< -# We don't list toc_*.html as targets because we don't know -$(HTMLDIR)/index.html + $(HTMLDIR)/applications.html: $(INDEX_SCRIPT) $(TEMPLATES) - echo "Generating index $@" -# Check if we are building the index from source or an installed release - if test "$$RELEASE_ROOT" = "" ; then \ - $(ERL) -noshell -pa $(EBIN) -s erl_html_tools top_index src $(ERL_TOP) \ - $(HTMLDIR) `cat "$(ERL_TOP)/OTP_VERSION"` -s erlang halt ;\ +$(INST_TYPE_DEST_DIR)/OTP_VERSION: $(INST_TYPE_DEST_DIR_DEP) + if test -f "$(RELEASE_PATH)/releases/$(SYSTEM_VSN)/OTP_VERSION"; then \ + $(CP) "$(RELEASE_PATH)/releases/$(SYSTEM_VSN)/OTP_VERSION" $@; \ else \ - $(ERL) -noshell -pa $(EBIN) -s erl_html_tools top_index rel $(RELEASE_ROOT) \ - $(HTMLDIR) `cat "$(RELEASE_ROOT)/releases/$(SYSTEM_VSN)/OTP_VERSION"` \ - -s erlang halt ;\ + $(CP) $(ERL_TOP)/OTP_VERSION $@; \ fi +# We don't list toc_*.html as targets because we don't know +$(INDEX_HTML) + $(APPLICATIONS_HTML): $(INST_TYPE_DEST_DIR_DEP) $(INDEX_SCRIPT) $(TEMPLATES) $(INST_TYPE_VSN_FILE) + echo "Generating index $@" + $(ERL) -noshell -pa $(EBIN) -s erl_html_tools top_index $(INST_TYPE) \ + $(INST_TYPE_SRC_DIR) $(INST_TYPE_DEST_DIR) \ + `cat "$(INST_TYPE_VSN_FILE)"` -s erlang halt + #-------------------------------------------------------------------------- $(JAVASCRIPT_BUILD_SCRIPT): $(JAVASCRIPT_BUILD_SCRIPT_SRC) $(ERLC) -o$(EBIN) +warn_unused_vars $< -$(JAVASCRIPT): $(JAVASCRIPT_BUILD_SCRIPT) - erl -noshell -pa $(EBIN) -s erlresolvelinks make -s erlang halt - $(INSTALL_DIR) $(HTMLDIR)/js - $(INSTALL_DATA) erlresolvelinks.js $(JAVASCRIPT) +$(JAVASCRIPT): $(INST_TYPE_DEST_DIR_DEP) $(JAVASCRIPT_BUILD_SCRIPT) + erl -noshell -pa $(EBIN) -run erlresolvelinks make $(ERL_TOP) \ + $(INST_TYPE_SRC_DIR) $(INST_TYPE_JS_DEST_DIR) -s erlang halt #-------------------------------------------------------------------------- $(MAN_INDEX_SCRIPT): $(MAN_INDEX_SRC) $(ERLC) -o$(EBIN) +warn_unused_vars $< -$(MAN_INDEX): $(MAN_INDEX_SCRIPT) -# Check if we are building the index from source or an installed release - if test "$$RELEASE_ROOT" = "" ; then \ - $(ERL) -noshell -pa $(EBIN) -s otp_man_index gen src $(ERL_TOP) $@ \ - -s erlang halt ;\ - else \ - $(ERL) -noshell -pa $(EBIN) -s otp_man_index gen rel $(RELEASE_ROOT) $@ \ - -s erlang halt ;\ - fi - +$(MAN_INDEX): $(INST_TYPE_DEST_DIR_DEP) $(MAN_INDEX_SCRIPT) + $(ERL) -noshell -pa $(EBIN) -s otp_man_index gen $(INST_TYPE) \ + $(INST_TYPE_SRC_DIR) $@ -s erlang halt #-------------------------------------------------------------------------- @@ -251,18 +264,22 @@ html: $(INDEX_FILES) \ debug opt: clean: - rm -rf ../html/js - rm -f PR.template - rm -f $(INDEX_FILES) $(MAN_INDEX) - rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) - rm -f $(INDEX_SCRIPT) $(GLOSSARY_SCRIPT) \ - $(JAVASCRIPT_BUILD_SCRIPT) - rm -f erl_crash.dump errs core *~ + $(RM) ../html/js/*.js + $(RM) PR.template + $(RM) $(XMLDIR)/*.xml + $(RM) $(INDEX_FILES) $(MAN_INDEX) + $(RM) $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) + $(RM) $(INDEX_SCRIPT) $(GLOSSARY_SCRIPT) $(JAVASCRIPT_BUILD_SCRIPT) + $(RM) erl_crash.dump errs core *~ + # ---------------------------------------------------- # Release Target # ---------------------------------------------------- include $(ERL_TOP)/make/otp_release_targets.mk +$(RELSYSDIR)/temporary: + $(INSTALL_DIR) $(RELSYSDIR)/temporary + release_docs_spec: docs $(INSTALL_DIR) "$(RELEASE_PATH)" $(INSTALL_DATA) $(INFO_FILES) "$(RELEASE_PATH)" @@ -271,13 +288,13 @@ release_docs_spec: docs $(INSTALL_DATA) \ $(TOP_PDF_FILE) $(RELSYSDIR)/pdf $(INSTALL_DIR) $(RELSYSDIR)/js - $(INSTALL_DATA) \ - $(JAVASCRIPT) $(RELSYSDIR)/js + $(INSTALL_DATA) $(JAVASCRIPT) $(RELSYSDIR)/js $(INSTALL_DATA) $(INDEX_FILES) $(MAN_INDEX) $(RELSYSDIR) $(INSTALL_DIR) $(RELSYSDIR)/docbuild $(INSTALL_DATA) $(INDEX_SCRIPT) $(MAN_INDEX_SCRIPT) $(JAVASCRIPT_BUILD_SCRIPT) \ $(INDEX_SRC) $(MAN_INDEX_SRC) $(JAVASCRIPT_BUILD_SCRIPT_SRC) \ $(TEMPLATES) $(RELSYSDIR)/docbuild + $(RM) -r $(RELSYSDIR)/temporary release_spec: diff --git a/system/doc/top/src/erl_html_tools.erl b/system/doc/top/src/erl_html_tools.erl index 28a0649658..dee1871342 100644 --- a/system/doc/top/src/erl_html_tools.erl +++ b/system/doc/top/src/erl_html_tools.erl @@ -508,7 +508,7 @@ subst_app(App, [{VSN,_Path,Link,Text} | VerInfos]) -> " <a href=\"",Link,"\" target=\"_top\">",uc(App), "</a>\n", " <a href=\"",Link,"\" target=\"_top\">",VSN,"</a>\n", - " <td class=appnums>\n", + " <br/>\n", subst_vsn(VerInfos), " </td>\n", " <td>\n", @@ -522,7 +522,7 @@ subst_vsn([{VSN,_Path,Link,_Text} | VSNs]) -> [ " <font size=\"2\"><a class=anum href=\"",Link,"\" target=\"_top\">", VSN, - "</a></font><br>\n", + "</a></font><br/>\n", subst_vsn(VSNs) ]; subst_vsn([]) -> diff --git a/system/doc/top/src/erlresolvelinks.erl b/system/doc/top/src/erlresolvelinks.erl index cfe8d0fa0b..c1285fa1c3 100644 --- a/system/doc/top/src/erlresolvelinks.erl +++ b/system/doc/top/src/erlresolvelinks.erl @@ -27,40 +27,28 @@ %%----------------------------------------------------------------- -module(erlresolvelinks). --export([make/0, make/1]). +-export([make/1]). -include_lib("kernel/include/file.hrl"). -define(JAVASCRIPT_NAME, "erlresolvelinks.js"). -make() -> - case os:getenv("ERL_TOP") of - false -> - io:format("Variable ERL_TOP is required\n",[]); - Value -> - make_from_src(Value, ".") - end. - -make([RootDir, DestDir]) -> - do_make(RootDir, DestDir); -make(RootDir) when is_atom(RootDir) -> - DestDir = filename:join(RootDir, "doc"), - do_make(RootDir, DestDir). - -do_make(_RootDir, _DestDir) -> - ok. +make([ErlTop, RootDir, DestDir]) -> + make(ErlTop, RootDir, DestDir). -make_from_src(RootDir, DestDir) -> +make(ErlTop, RootDir, DestDir) -> %% doc/Dir %% erts-Vsn %% lib/App-Vsn Name = ?JAVASCRIPT_NAME, - DocDirs0 = get_dirs(filename:join([RootDir, "system/doc"])), + DocDirs0 = get_dirs(filename:join([ErlTop, "system/doc"])), DocDirs = lists:map(fun({Dir, _DirPath}) -> D = filename:join(["doc", Dir]), {D, D} end, DocDirs0), - ErtsDirs = latest_app_dirs(RootDir, ""), - AppDirs = latest_app_dirs(RootDir, "lib"), + Released = ErlTop /= RootDir, + + ErtsDirs = latest_app_dirs(Released, RootDir, ""), + AppDirs = latest_app_dirs(Released, RootDir, "lib"), AllAppDirs = lists:map( @@ -106,30 +94,46 @@ is_dir({File, AFile}) -> false end. -latest_app_dirs(RootDir, Dir) -> +released_app_vsns([]) -> + []; +released_app_vsns([{AppVsn, Dir} | AVDirs]) -> + try + {ok, _} = file:read_file_info(filename:join([Dir, "doc", "html"])), + [App, Vsn] = string:tokens(AppVsn, "-"), + VsnNumList = vsnstr_to_numlist(Vsn), + [_Maj, _Min | _] = VsnNumList, + [{{App, VsnNumList}, AppVsn} | released_app_vsns(AVDirs)] + catch + _:_ -> released_app_vsns(AVDirs) + end. + +latest_app_dirs(Release, RootDir, Dir) -> ADir = filename:join(RootDir, Dir), RDirs0 = get_dirs(ADir), - RDirs1 = lists:filter(fun is_app_dir/1, RDirs0), - - SDirs0 = - lists:map(fun({App, Dir1}) -> - File = filename:join(Dir1, "vsn.mk"), - case file:read_file(File) of - {ok, Bin} -> - case re:run(Bin, ".*VSN\s*=\s*([0-9\.]+).*",[{capture,[1],list}]) of - {match, [VsnStr]} -> - VsnNumList = vsnstr_to_numlist(VsnStr), - {{App, VsnNumList}, App++"-"++VsnStr}; - nomatch -> - io:format("No VSN variable found in ~s\n", [File]), - error - end; - {error, Reason} -> - io:format("~p : ~s\n", [Reason, File]), - error - end - end, - RDirs1), + SDirs0 = case Release of + true -> + released_app_vsns(RDirs0); + false -> + lists:map(fun({App, Dir1}) -> + File = filename:join(Dir1, "vsn.mk"), + case file:read_file(File) of + {ok, Bin} -> + case re:run(Bin, ".*VSN\s*=\s*([0-9\.]+).*",[{capture,[1],list}]) of + {match, [VsnStr]} -> + VsnNumList = vsnstr_to_numlist(VsnStr), + {{App, VsnNumList}, App++"-"++VsnStr}; + nomatch -> + io:format("No VSN variable found in ~s\n", [File]), + error + end; + {error, Reason} -> + io:format("~p : ~s\n", [Reason, File]), + error + end + end, + lists:filter(fun is_app_dir/1, RDirs0)) + end, + SDirs1 = lists:keysort(1, SDirs0), App2Dirs = lists:foldr(fun({{App, _VsnNumList}, AppVsn}, Acc) -> case lists:keymember(App, 1, Acc) of diff --git a/system/doc/tutorial/Makefile b/system/doc/tutorial/Makefile index 5867096fc8..4c62deeffd 100644 --- a/system/doc/tutorial/Makefile +++ b/system/doc/tutorial/Makefile @@ -93,10 +93,9 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -_create_dirs := $(shell mkdir -p $(HTMLDIR)) $(HTMLDIR)/%.gif: %.gif - $(INSTALL_DATA) $< $@ + $(CP) $< $@ docs: html @@ -109,8 +108,8 @@ gifs: $(GIF_FILES:%=$(HTMLDIR)/%) debug opt: clean clean_docs: - rm -rf $(HTMLDIR) - rm -rf $(XMLDIR) + rm -f $(XMLDIR)/*.xml + rm -f $(HTMLDIR)/*.gif $(HTMLDIR)/*.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/xml/design_principles/.gitignore b/system/doc/xml/design_principles/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/design_principles/.gitignore diff --git a/system/doc/xml/efficiency_guide/.gitignore b/system/doc/xml/efficiency_guide/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/efficiency_guide/.gitignore diff --git a/system/doc/xml/embedded/.gitignore b/system/doc/xml/embedded/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/embedded/.gitignore diff --git a/system/doc/xml/general_info/.gitignore b/system/doc/xml/general_info/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/general_info/.gitignore diff --git a/system/doc/xml/getting_started/.gitignore b/system/doc/xml/getting_started/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/getting_started/.gitignore diff --git a/system/doc/xml/installation_guide/.gitignore b/system/doc/xml/installation_guide/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/installation_guide/.gitignore diff --git a/system/doc/xml/oam/.gitignore b/system/doc/xml/oam/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/oam/.gitignore diff --git a/system/doc/xml/programming_examples/.gitignore b/system/doc/xml/programming_examples/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/programming_examples/.gitignore diff --git a/system/doc/xml/reference_manual/.gitignore b/system/doc/xml/reference_manual/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/reference_manual/.gitignore diff --git a/system/doc/xml/system_architecture_intro/.gitignore b/system/doc/xml/system_architecture_intro/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/system_architecture_intro/.gitignore diff --git a/system/doc/xml/system_principles/.gitignore b/system/doc/xml/system_principles/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/system_principles/.gitignore diff --git a/system/doc/xml/tutorial/.gitignore b/system/doc/xml/tutorial/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/system/doc/xml/tutorial/.gitignore |