diff options
138 files changed, 2670 insertions, 1195 deletions
diff --git a/.gitignore b/.gitignore index 9cd91245f5..3a9e53dc94 100644 --- a/.gitignore +++ b/.gitignore @@ -114,6 +114,8 @@ JAVADOC-GENERATED /bootstrap/lib/asn1 /bootstrap/lib/common_test +/bootstrap/lib/edoc +/bootstrap/lib/erl_docgen /bootstrap/lib/hipe /bootstrap/lib/ic /bootstrap/lib/orber diff --git a/HOWTO/DTRACE.md b/HOWTO/DTRACE.md index 8fa2fd9d50..90f4addefd 100644 --- a/HOWTO/DTRACE.md +++ b/HOWTO/DTRACE.md @@ -27,12 +27,10 @@ Goals Supported platforms ------------------- -* OS X 10.6.x / Snow Leopard. It should also work for 10.7 / Lion, - but I haven't personally tested it. +* OS X 10.6.x / Snow Leopard, OS X 10.7.x / Lion and probably newer versions. * Solaris 10. I have done limited testing on Solaris 11 and OpenIndiana release 151a, and both appear to work. -* FreeBSD 9.0, though please see the "FreeBSD 9.0 Release Notes" - section below! +* FreeBSD 9.0 and 10.0. * Linux via SystemTap compatibility. Please see [$ERL_TOP/HOWTO/SYSTEMTAP.md][] for more details. diff --git a/HOWTO/INSTALL-CROSS.md b/HOWTO/INSTALL-CROSS.md index 03ea4c6e76..b01de12e44 100644 --- a/HOWTO/INSTALL-CROSS.md +++ b/HOWTO/INSTALL-CROSS.md @@ -103,14 +103,6 @@ has to be provided in the `$PATH`. The Erlang/OTP for the target system will be built using this Erlang system, together with the cross compilation tools provided. -If you want to build the documentation out of the same source tree as you are -cross compiling in, you currently need a full Erlang/OTP system of the same -release as the one being built for the build machine. If this is the case, -build and install one for the build machine (or use one already built) and add -it to the `$PATH` before cross building, and building the documentation. See -the [How to Build the Documentation][] section in the [$ERL_TOP/HOWTO/INSTALL.md][] -document for information on how to build the documentation. - If you want to build using a compatible Erlang/OTP system in the `$PATH`, jump to (3). @@ -284,6 +276,14 @@ and then do the cross build of the system. `otp_build release -a` will do the same as (5), and you will after this have to do a manual install either by doing (6), or (7). +Building and Installing the Documentation +----------------------------------------- + +After the system has been cross built you can build and install the +documentation the same way as after a native build of the system. See the +[How to Build the Documentation][] section in the [$ERL_TOP/HOWTO/INSTALL.md][] +document for information on how to build the documentation. + Testing the cross compiled system --------------------------------- Some of the tests that come with erlang use native code to test. This means diff --git a/HOWTO/INSTALL.md b/HOWTO/INSTALL.md index 533960ef99..c9f4062b83 100644 --- a/HOWTO/INSTALL.md +++ b/HOWTO/INSTALL.md @@ -189,16 +189,16 @@ section below before proceeding. Step 1: Start by unpacking the Erlang/OTP distribution file with your GNU compatible TAR program. - $ gunzip -c otp_src_%OTP-REL%.tar.gz | tar xf - + $ gunzip -c otp_src_%OTP-VSN%.tar.gz | tar xf - alternatively: - $ zcat otp_src_%OTP-REL%.tar.gz | tar xf - + $ zcat otp_src_%OTP-VSN%.tar.gz | tar xf - Step 2: Now cd into the base directory (`$ERL_TOP`). - $ cd otp_src_%OTP-REL% + $ cd otp_src_%OTP-VSN% ### Configuring ### @@ -500,21 +500,11 @@ The Erlang/OTP Documentation ### How to Build the Documentation ### - $ cd $ERL_TOP - -If you have just built Erlang/OTP in the current source tree, you have -already ran `configure` and do not need to do this again; otherwise, run -`configure`. - - $ ./configure [Configure Args] - -When building the documentation you need a full Erlang/OTP-%OTP-REL% system in -the `$PATH`. - - $ export PATH=<Erlang/OTP-%OTP-REL% bin dir>:$PATH # Assuming bash/sh - -Build the documentation. +Before you can build the documentation you need to either [native build][] +or [cross build][] the Erlang/OTP system. After this you can build the +documentation as follows. + $ cd $ERL_TOP $ make docs The documentation can be installed either using the `install-docs` target, @@ -553,13 +543,13 @@ For some graphical tools to find the on-line help you have to install the HTML documentation on top of the installed OTP applications, i.e. $ cd <ReleaseDir> - $ gunzip -c otp_html_%OTP-REL%.tar.gz | tar xf - + $ gunzip -c otp_html_%OTP-VSN%.tar.gz | tar xf - For `erl -man <page>` to work the Unix manual pages have to be installed in the same way, i.e. $ cd <ReleaseDir> - $ gunzip -c otp_man_%OTP-REL%.tar.gz | tar xf - + $ gunzip -c otp_man_%OTP-VSN%.tar.gz | tar xf - Where `<ReleaseDir>` is @@ -803,9 +793,11 @@ Before modifying this document you need to have a look at the [Building in Git]: #How-to-Build-and-Install-ErlangOTP_Building-in-Git [Pre-built Source Release]: #How-to-Build-and-Install-ErlangOTP_Prebuilt-Source-Release [make and $ERL_TOP]: #How-to-Build-and-Install-ErlangOTP_make-and-ERLTOP - [html documentation]: http://www.erlang.org/download/otp_doc_html_%OTP-REL%.tar.gz - [man pages]: http://www.erlang.org/download/otp_doc_man_%OTP-REL%.tar.gz - [the released source tar ball]: http://www.erlang.org/download/otp_src_%OTP-REL%.tar.gz + [html documentation]: http://www.erlang.org/download/otp_doc_html_%OTP-VSN%.tar.gz + [man pages]: http://www.erlang.org/download/otp_doc_man_%OTP-VSN%.tar.gz + [the released source tar ball]: http://www.erlang.org/download/otp_src_%OTP-VSN%.tar.gz + [native build]: #How-to-Build-and-Install-ErlangOTP + [cross build]: INSTALL-CROSS.md [$ERL_TOP/HOWTO/MARKDOWN.md]: MARKDOWN.md [?TOC]: true diff --git a/HOWTO/MARKDOWN.md b/HOWTO/MARKDOWN.md index c9ad09a1be..8e8a5c02cf 100644 --- a/HOWTO/MARKDOWN.md +++ b/HOWTO/MARKDOWN.md @@ -223,8 +223,11 @@ places. Appropriate attributes to the `X` tag will also be generated. \%CopyrightBegin\% and \%CopyrightEnd\% "tags" will be removed from the output. -* All occurrences of \%OTP-REL% will be replaced by current release number - (e.g. R14A). +* All occurrences of \%OTP-REL% will be replaced by current OTP release number + (e.g. 17). + +* All occurrences of \%OTP-VSN% will be replaced by current OTP version + (e.g. 17.0). * All occurrences of \%ERTS-VSN% will be replaced by current ERTS version (e.g. 5.8). diff --git a/Makefile.in b/Makefile.in index d3d81c8817..a64c2872fd 100644 --- a/Makefile.in +++ b/Makefile.in @@ -172,6 +172,7 @@ endif TARGET := @TARGET@ include $(ERL_TOP)/make/target.mk export TARGET +include $(ERL_TOP)/make/otp_default_release_path.mk BOOTSTRAP_ONLY = @BOOTSTRAP_ONLY@ @@ -395,29 +396,35 @@ endif cd $(ERL_TOP)/erts && \ ERL_TOP=$(ERL_TOP) PATH=$(INST_PATH_PREFIX)"$${PATH}" \ $(MAKE) BUILD_ALL=1 TESTROOT="$(RELEASE_ROOT)" release +ifeq ($(RELEASE_ROOT),) + $(INSTALL_DATA) "$(ERL_TOP)/OTP_VERSION" "$(OTP_DEFAULT_RELEASE_PATH)" +else + $(INSTALL_DATA) "$(ERL_TOP)/OTP_VERSION" "$(RELEASE_ROOT)" +endif # --------------------------------------------------------------- # Target only used when building commercial ERTS patches # --------------------------------------------------------------- -release_docs docs: mod2app + +release_docs docs: doc_bootstrap_build doc_bootstrap_copy mod2app ifeq ($(OTP_SMALL_BUILD),true) cd $(ERL_TOP)/lib && \ - PATH=$(ERL_TOP)/bin:"$${PATH}" ERL_TOP=$(ERL_TOP) \ - $(MAKE) TESTROOT="$(RELEASE_ROOT)" $@ + PATH=$(BOOT_PREFIX)"$${PATH}" ERL_TOP=$(ERL_TOP) \ + $(MAKE) TESTROOT="$(RELEASE_ROOT)" DOCGEN=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen $@ else cd $(ERL_TOP)/lib && \ - PATH=$(ERL_TOP)/bin:"$${PATH}" ERL_TOP=$(ERL_TOP) \ - $(MAKE) BUILD_ALL=1 TESTROOT="$(RELEASE_ROOT)" $@ + PATH=$(BOOT_PREFIX)"$${PATH}" ERL_TOP=$(ERL_TOP) \ + $(MAKE) BUILD_ALL=1 TESTROOT="$(RELEASE_ROOT)" DOCGEN=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen $@ endif cd $(ERL_TOP)/erts && \ - PATH=$(ERL_TOP)/bin:"$${PATH}" ERL_TOP=$(ERL_TOP) \ - $(MAKE) BUILD_ALL=1 TESTROOT="$(RELEASE_ROOT)" $@ + PATH=$(BOOT_PREFIX)"$${PATH}" ERL_TOP=$(ERL_TOP) \ + $(MAKE) BUILD_ALL=1 TESTROOT="$(RELEASE_ROOT)" DOCGEN=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen $@ cd $(ERL_TOP)/system/doc && \ - PATH=$(ERL_TOP)/bin:"$${PATH}" \ - ERL_TOP=$(ERL_TOP) $(MAKE) TESTROOT="$(RELEASE_ROOT)" $@ + PATH=$(BOOT_PREFIX)"$${PATH}" \ + ERL_TOP=$(ERL_TOP) $(MAKE) TESTROOT="$(RELEASE_ROOT)" DOCGEN=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen $@ mod2app: - PATH=$(ERL_TOP)/bin:"$${PATH}" escript $(ERL_TOP)/lib/erl_docgen/priv/bin/xref_mod_app.escript -topdir $(ERL_TOP) -outfile $(ERL_TOP)/make/$(TARGET)/mod2app.xml + PATH=$(BOOT_PREFIX)"$${PATH}" escript $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen/priv/bin/xref_mod_app.escript -topdir $(ERL_TOP) -outfile $(ERL_TOP)/make/$(TARGET)/mod2app.xml # ---------------------------------------------------------------------- ERLANG_EARS=$(BOOTSTRAP_ROOT)/bootstrap/erts @@ -762,6 +769,80 @@ tertiary_bootstrap_copy: done # $(V_at)cp lib/syntax_tools/ebin/*.beam $(BOOTSTRAP_ROOT)/bootstrap/lib/syntax_tools/ebin +doc_bootstrap_build: + $(make_verbose)cd lib && \ + ERL_TOP=$(ERL_TOP) PATH=$(BOOT_PREFIX)"$${PATH}" \ + $(MAKE) opt DOC_BOOTSTRAP=true + +doc_bootstrap_copy: + $(make_verbose) +# XMERL + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/xmerl ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/xmerl ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/xmerl/ebin ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/xmerl/ebin ; fi + $(V_at)for x in lib/xmerl/ebin/*.beam; do \ + BN=`basename $$x`; \ + TF=$(BOOTSTRAP_ROOT)/bootstrap/lib/xmerl/ebin/$$BN; \ + test -f $$TF && \ + test '!' -z "`find $$x -newer $$TF -print`" && \ + cp $$x $$TF; \ + test '!' -f $$TF && \ + cp $$x $$TF; \ + true; \ + done +# xmerl/include already copied in secondary bootstrap +# EDOC + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/edoc ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/edoc ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/edoc/ebin ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/edoc/ebin ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/edoc/include ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/edoc/include ; fi + $(V_at)for x in lib/edoc/ebin/*.beam; do \ + BN=`basename $$x`; \ + TF=$(BOOTSTRAP_ROOT)/bootstrap/lib/edoc/ebin/$$BN; \ + test -f $$TF && \ + test '!' -z "`find $$x -newer $$TF -print`" && \ + cp $$x $$TF; \ + test '!' -f $$TF && \ + cp $$x $$TF; \ + true; \ + done + $(V_at)for x in lib/edoc/include/*.hrl; do \ + BN=`basename $$x`; \ + TF=$(BOOTSTRAP_ROOT)/bootstrap/lib/edoc/include/$$BN; \ + test -f $$TF && \ + test '!' -z "`find $$x -newer $$TF -print`" && \ + cp $$x $$TF; \ + test '!' -f $$TF && \ + cp $$x $$TF; \ + true; \ + done +# ERL_DOCGEN + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen/ebin ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen/ebin ; fi + $(V_at)for x in lib/erl_docgen/ebin/*.beam; do \ + BN=`basename $$x`; \ + TF=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen/ebin/$$BN; \ + test -f $$TF && \ + test '!' -z "`find $$x -newer $$TF -print`" && \ + cp $$x $$TF; \ + test '!' -f $$TF && \ + cp $$x $$TF; \ + true; \ + done + $(V_at)for d in priv priv/bin priv/css priv/dtd priv/dtd_html_entities priv/dtd_man_entities priv/images priv/js priv/js/flipmenu priv/xsl; do \ + if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen/$$d ; then mkdir -p $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen/$$d ; fi; \ + for x in lib/erl_docgen/$$d/*; do \ + BN=`basename $$x`; \ + if test ! -d lib/erl_docgen/$$d/$$BN ; then \ + TF=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen/$$d/$$BN; \ + test -f $$TF && \ + test '!' -z "`find $$x -newer $$TF -print`" && \ + cp $$x $$TF; \ + test '!' -f $$TF && \ + cp $$x $$TF; \ + true; \ + fi; \ + done; \ + done + .PHONY: check_recreate_primary_bootstrap recreate_primary_bootstrap @@ -921,7 +1002,7 @@ $(TEST_DIRS): # # Order is important here, don't change it! # -INST_DEP += install.dirs install.emulator install.libs install.Install install.bin +INST_DEP += install.dirs install.emulator install.libs install.Install install.otp_version install.bin install: $(INST_DEP) @@ -950,6 +1031,13 @@ install.Install: (cd "$(ERLANG_LIBDIR)" \ && ./Install $(INSTALL_CROSS) -minimal "$(ERLANG_INST_LIBDIR)") +install.otp_version: +ifeq ($(ERLANG_LIBDIR),) + $(INSTALL_DATA) "$(ERL_TOP)/OTP_VERSION" "$(OTP_DEFAULT_RELEASE_PATH)" +else + $(INSTALL_DATA) "$(ERL_TOP)/OTP_VERSION" "$(ERLANG_LIBDIR)" +endif + # # Install erlang base public files # diff --git a/OTP_VERSION b/OTP_VERSION new file mode 100644 index 0000000000..c86282f25f --- /dev/null +++ b/OTP_VERSION @@ -0,0 +1 @@ +17.0-rc1 diff --git a/bootstrap/lib/compiler/ebin/beam_validator.beam b/bootstrap/lib/compiler/ebin/beam_validator.beam Binary files differindex c940cf7eed..8c685feb27 100644 --- a/bootstrap/lib/compiler/ebin/beam_validator.beam +++ b/bootstrap/lib/compiler/ebin/beam_validator.beam diff --git a/bootstrap/lib/compiler/ebin/compile.beam b/bootstrap/lib/compiler/ebin/compile.beam Binary files differindex 4e4f3cf4b7..1aa96eb292 100644 --- a/bootstrap/lib/compiler/ebin/compile.beam +++ b/bootstrap/lib/compiler/ebin/compile.beam diff --git a/bootstrap/lib/compiler/ebin/v3_core.beam b/bootstrap/lib/compiler/ebin/v3_core.beam Binary files differindex 89eb5bb272..52a1086004 100644 --- a/bootstrap/lib/compiler/ebin/v3_core.beam +++ b/bootstrap/lib/compiler/ebin/v3_core.beam diff --git a/bootstrap/lib/kernel/ebin/application.beam b/bootstrap/lib/kernel/ebin/application.beam Binary files differindex 7e2e98d5ab..2c8444fb95 100644 --- a/bootstrap/lib/kernel/ebin/application.beam +++ b/bootstrap/lib/kernel/ebin/application.beam diff --git a/bootstrap/lib/kernel/ebin/application_controller.beam b/bootstrap/lib/kernel/ebin/application_controller.beam Binary files differindex 590b9a54b0..659f29e35e 100644 --- a/bootstrap/lib/kernel/ebin/application_controller.beam +++ b/bootstrap/lib/kernel/ebin/application_controller.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam Binary files differindex c8f45fd838..0e72af0e35 100644 --- a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam +++ b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_lint.beam b/bootstrap/lib/stdlib/ebin/erl_lint.beam Binary files differindex 4a45d59de6..301785992c 100644 --- a/bootstrap/lib/stdlib/ebin/erl_lint.beam +++ b/bootstrap/lib/stdlib/ebin/erl_lint.beam diff --git a/bootstrap/lib/stdlib/ebin/qlc_pt.beam b/bootstrap/lib/stdlib/ebin/qlc_pt.beam Binary files differindex 5c89047e67..f53c840702 100644 --- a/bootstrap/lib/stdlib/ebin/qlc_pt.beam +++ b/bootstrap/lib/stdlib/ebin/qlc_pt.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor.beam b/bootstrap/lib/stdlib/ebin/supervisor.beam Binary files differindex d1d1131c06..5fe8747341 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam b/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam Binary files differindex 6480117b9e..54d7385738 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam diff --git a/configure.in b/configure.in index 88fd847030..081febd47b 100644 --- a/configure.in +++ b/configure.in @@ -180,18 +180,26 @@ AC_PROG_LN_S AC_PROG_RANLIB # -# Get erts version and otp release from erts/vsn.mk +# Get erts version from erts/vsn.mk # AC_MSG_CHECKING([ERTS version]) [ERTS_VSN=`sed -n "s/^VSN[ ]*=[ ]*\(.*\)/\1/p" < $ERL_TOP/erts/vsn.mk`] AC_MSG_RESULT([$ERTS_VSN]) AC_SUBST(ERTS_VSN) +# +# Get OTP release and OTP version from $ERL_TOP/OTP_VERSION +# AC_MSG_CHECKING([OTP release]) -[OTP_REL=`sed -n "s/^SYSTEM_VSN[ ]*=[ ]*\(.*\)/\1/p" < $ERL_TOP/erts/vsn.mk`] +[OTP_REL=`cat $ERL_TOP/OTP_VERSION | sed "s|\([0-9]*\).*|\1|"`] AC_MSG_RESULT([$OTP_REL]) AC_SUBST(OTP_REL) +AC_MSG_CHECKING([OTP version]) +[OTP_VSN=`cat $ERL_TOP/OTP_VERSION`] +AC_MSG_RESULT([$OTP_VSN]) +AC_SUBST(OTP_VSN) + AC_ARG_ENABLE(threads, AS_HELP_STRING([--enable-threads], [enable async thread support]) AS_HELP_STRING([--disable-threads], [disable async thread support])) diff --git a/erts/configure.in b/erts/configure.in index 8d245252b5..02e5d12918 100644 --- a/erts/configure.in +++ b/erts/configure.in @@ -357,6 +357,16 @@ else fi AC_SUBST(OTP_RELEASE) +AC_MSG_CHECKING([OTP release]) +[SYSTEM_VSN=`cat $ERL_TOP/OTP_VERSION | sed "s|\([0-9]*\).*|\1|"`] +AC_MSG_RESULT([$SYSTEM_VSN]) +AC_SUBST(SYSTEM_VSN) + +AC_MSG_CHECKING([OTP version]) +[OTP_VERSION=`cat $ERL_TOP/OTP_VERSION`] +AC_MSG_RESULT([$OTP_VERSION]) +AC_SUBST(OTP_VERSION) + dnl OK, we might have darwin switches off different kinds, lets dnl check it all before continuing. TMPSYS=`uname -s`-`uname -m` @@ -3848,28 +3858,26 @@ esac if test "$enable_dtrace_test" = "yes" ; then if test "$DTRACE" = "dtrace" ; then AC_CHECK_HEADERS(sys/sdt.h) + AC_MSG_CHECKING([for 1-stage DTrace precompilation]) # The OS X version of dtrace prints a spurious line here. if ! dtrace -h $DTRACE_CPP -Iemulator/beam -o ./foo-dtrace.h -s emulator/beam/erlang_dtrace.d; then AC_MSG_ERROR([Could not precompile erlang_dtrace.d: dtrace -h failed]) fi + AC_MSG_RESULT([yes]) - $RM -f dtest.{o,c} - cat > dtest.c <<_DTEST - #include "foo-dtrace.h" - int main(void) { ERLANG_DIST_PORT_BUSY_ENABLED(); return 0; } -_DTEST - $CC $CFLAGS -c -o dtest.o dtest.c - - $RM -f $DTRACE_2STEP_TEST - if dtrace -G $DTRACE_CPP $DTRACE_BITS_FLAG -Iemulator/beam -o $DTRACE_2STEP_TEST -s emulator/beam/erlang_dtrace.d dtest.o && \ - test -f $DTRACE_2STEP_TEST ; then - rm $DTRACE_2STEP_TEST + AC_MSG_CHECKING([for 2-stage DTrace precompilation]) + AC_TRY_COMPILE([ #include "foo-dtrace.h" ], + [ERLANG_DIST_PORT_BUSY_ENABLED();], + [$RM -f $DTRACE_2STEP_TEST + dtrace -G $DTRACE_CPP $DTRACE_BITS_FLAG -Iemulator/beam -o $DTRACE_2STEP_TEST -s emulator/beam/erlang_dtrace.d conftest.$OBJEXT 2>&AS_MESSAGE_LOG_FD + if test -f $DTRACE_2STEP_TEST; then + rm $DTRACE_2STEP_TEST DTRACE_ENABLED_2STEP=yes - AC_MSG_NOTICE([dtrace precompilation for 2-stage DTrace successful]) - else - AC_MSG_NOTICE([dtrace precompilation for 1-stage DTrace successful]) - fi - $RM -f dtest.{o,c} foo-dtrace.h + fi], + []) + AS_IF([test "x$DTRACE_ENABLED_2STEP" = "xyes"], + [AC_MSG_RESULT([yes])], + [AC_MSG_RESULT([no])]) DTRACE_ENABLED=yes case $OPSYS in diff --git a/erts/doc/src/Makefile b/erts/doc/src/Makefile index d4c6fe67d2..e8b856c3ff 100644 --- a/erts/doc/src/Makefile +++ b/erts/doc/src/Makefile @@ -139,7 +139,7 @@ man: $(MAN1_FILES) $(MAN3_FILES) gifs: $(GIF_FILES:%=$(HTMLDIR)/%) -$(INFO_FILE): $(INFO_FILE_SRC) ../../vsn.mk +$(INFO_FILE): $(INFO_FILE_SRC) $(ERL_TOP)/make/$(TARGET)/otp.mk sed -e 's;%RELEASE%;$(SYSTEM_VSN);' $(INFO_FILE_SRC) > $(INFO_FILE) diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 4cf5631727..0f312fb430 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -6094,16 +6094,19 @@ ok <seealso marker="#system_info_multi_scheduling">erlang:system_info(multi_scheduling)</seealso>, and <seealso marker="#system_info_schedulers">erlang:system_info(schedulers)</seealso>.</p> </item> - <tag><marker id="system_info_otp_correction_package"><c>otp_correction_package</c></marker></tag> - <item> - <p>Returns a string containing the OTP correction package version - number that currenly executing VM is part of. Note that other - OTP applications in the system may be part of other OTP correction - packages.</p> - </item> <tag><marker id="system_info_otp_release"><c>otp_release</c></marker></tag> <item> - <p>Returns a string containing the OTP release number.</p> + <p>Returns a string containing the OTP release number of the + OTP release that the currently executing ERTS application is + part of.</p> + <p>As of OTP release 17, the OTP release number corresponds to + the major OTP version number. There is no + <c>erlang:system_info()</c> argument giving the exact OTP + version. This since the exact OTP version in the general case + is hard to determine. For more information see + <seealso marker="doc/installation_guide:otp_version">the + documentation of the OTP version in the installation + guide</seealso>.</p> </item> <tag><marker id="system_info_port_parallelism"><c>port_parallelism</c></marker></tag> <item><p>Returns the default port parallelism scheduling hint used. diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in index f88a4ccc24..5e5b98bbd6 100644 --- a/erts/emulator/Makefile.in +++ b/erts/emulator/Makefile.in @@ -513,10 +513,9 @@ ifdef DTRACE_ENABLED # global.h causes problems by including dtrace-wrapper.h which includes # the autogenerated erlang_dtrace.h ... so make erlang_dtrace.h very early. DTRACE_HEADERS = $(TARGET)/erlang_dtrace.h -generate: $(DTRACE_HEADERS) $(GENERATE) +GENERATE+= $(DTRACE_HEADERS) else DTRACE_HEADERS = -generate: $(GENERATE) endif ifdef HIPE_ENABLED @@ -574,8 +573,8 @@ $(TTF_DIR)/erl_alloc_types.h: beam/erl_alloc.types utils/make_alloc_types GENERATE += $(TTF_DIR)/erl_alloc_types.h # version include file -$(TARGET)/erl_version.h: ../vsn.mk - $(gen_verbose)LANG=C $(PERL) utils/make_version -o $@ $(SYSTEM_VSN) $(SYSTEM_CP_VSN) $(VSN)$(SERIALNO) $(TARGET) +$(TARGET)/erl_version.h: ../vsn.mk $(ERL_TOP)/make/$(TARGET)/otp.mk + $(gen_verbose)LANG=C $(PERL) utils/make_version -o $@ $(SYSTEM_VSN) $(OTP_VERSION) $(VSN)$(SERIALNO) $(TARGET) GENERATE += $(TARGET)/erl_version.h # driver table diff --git a/erts/emulator/beam/beam_debug.c b/erts/emulator/beam/beam_debug.c index e36ec2a93e..a3cd08834f 100644 --- a/erts/emulator/beam/beam_debug.c +++ b/erts/emulator/beam/beam_debug.c @@ -635,6 +635,11 @@ print_op(int to, void *to_arg, int op, int size, BeamInstr* addr) case op_i_put_tuple_rI: case op_i_put_tuple_xI: case op_i_put_tuple_yI: + case op_new_map_jdII: + case op_update_map_assoc_jsdII: + case op_update_map_exact_jsdII: + case op_i_has_map_fields_fsI: + case op_i_get_map_elements_fsI: { int n = unpacked[-1]; diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index 1ae413d46e..6f295530b9 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -2355,6 +2355,126 @@ void process_main(void) Next(4+Arg(3)); } + OpCase(i_has_map_fields_fsI): { + map_t* mp; + Eterm map; + Eterm field; + Eterm *ks; + BeamInstr* fs; + Uint sz,n; + + GetArg1(1, map); + + /* this instruction assumes Arg1 is a map, + * i.e. that it follows a test is_map if needed. + */ + + mp = (map_t *)map_val(map); + sz = map_get_size(mp); + + if (sz == 0) { + SET_I((BeamInstr *) Arg(0)); + goto has_map_fields_fail; + } + + ks = map_get_keys(mp); + n = (Uint)Arg(2); + fs = &Arg(3); /* pattern fields */ + + ASSERT(n>0); + + while(sz) { + field = (Eterm)*fs; + if (EQ(field,*ks)) { + n--; + fs++; + if (n == 0) break; + } + ks++; sz--; + } + + if (n) { + SET_I((BeamInstr *) Arg(0)); + goto has_map_fields_fail; + } + + I += 4 + Arg(2); +has_map_fields_fail: + ASSERT(VALID_INSTR(*I)); + Goto(*I); + } + +#define PUT_TERM_REG(term, desc) \ +do { \ + switch ((desc) & _TAG_IMMED1_MASK) { \ + case (R_REG_DEF << _TAG_PRIMARY_SIZE) | TAG_PRIMARY_HEADER: \ + r(0) = (term); \ + break; \ + case (X_REG_DEF << _TAG_PRIMARY_SIZE) | TAG_PRIMARY_HEADER: \ + x((desc) >> _TAG_IMMED1_SIZE) = (term); \ + break; \ + case (Y_REG_DEF << _TAG_PRIMARY_SIZE) | TAG_PRIMARY_HEADER: \ + y((desc) >> _TAG_IMMED1_SIZE) = (term); \ + break; \ + default: \ + ASSERT(0); \ + break; \ + } \ +} while(0) + + OpCase(i_get_map_elements_fsI): { + Eterm map; + map_t *mp; + Eterm field; + Eterm *ks; + Eterm *vs; + BeamInstr *fs; + Uint sz,n; + + GetArg1(1, map); + + /* this instruction assumes Arg1 is a map, + * i.e. that it follows a test is_map if needed. + */ + + mp = (map_t *)map_val(map); + sz = map_get_size(mp); + + if (sz == 0) { + SET_I((BeamInstr *) Arg(0)); + goto get_map_elements_fail; + } + + n = (Uint)Arg(2) / 2; + fs = &Arg(3); /* pattern fields and target registers */ + ks = map_get_keys(mp); + vs = map_get_values(mp); + + while(sz) { + field = (Eterm)*fs; + if (EQ(field,*ks)) { + PUT_TERM_REG(*vs, fs[1]); + n--; + fs += 2; + /* no more values to fetch, we are done */ + if (n == 0) break; + } + ks++; sz--; + vs++; + } + + if (n) { + SET_I((BeamInstr *) Arg(0)); + goto get_map_elements_fail; + } + + I += 4 + Arg(2); +get_map_elements_fail: + ASSERT(VALID_INSTR(*I)); + Goto(*I); + } +#undef PUT_TERM_REG + OpCase(update_map_assoc_jsdII): { Eterm res; Eterm map; diff --git a/erts/emulator/beam/beam_load.c b/erts/emulator/beam/beam_load.c index b589d1c930..53df7876d8 100644 --- a/erts/emulator/beam/beam_load.c +++ b/erts/emulator/beam/beam_load.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2013. All Rights Reserved. + * Copyright Ericsson AB 1996-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -409,7 +409,7 @@ typedef struct LoaderState { __result = __result << 8 | *Stp->file_p++; \ } \ Dest = __result; \ - } while (0) + } #define GetByte(Stp, Dest) \ if ((Stp)->file_left < 1) { \ @@ -506,6 +506,9 @@ static GenOp* gen_select_literals(LoaderState* stp, GenOpArg S, static GenOp* const_select_val(LoaderState* stp, GenOpArg S, GenOpArg Fail, GenOpArg Size, GenOpArg* Rest); +static GenOp* gen_get_map_element(LoaderState* stp, GenOpArg Fail, GenOpArg Src, + GenOpArg Size, GenOpArg* Rest); + static int freeze_code(LoaderState* stp); static void final_touch(LoaderState* stp); @@ -3951,6 +3954,49 @@ tuple_append_put(LoaderState* stp, GenOpArg Arity, GenOpArg Dst, return op; } +/* + * Replace a get_map_elements with one key to an instruction with one + * element + */ + +static GenOp* +gen_get_map_element(LoaderState* stp, GenOpArg Fail, GenOpArg Src, + GenOpArg Size, GenOpArg* Rest) +{ + GenOp* op; + + ASSERT(Size.type == TAG_u); + + NEW_GENOP(stp, op); + op->next = NULL; + op->op = genop_get_map_element_4; + op->arity = 4; + + op->a[0] = Fail; + op->a[1] = Src; + op->a[2] = Rest[0]; + op->a[3] = Rest[1]; + return op; +} + +static GenOp* +gen_has_map_field(LoaderState* stp, GenOpArg Fail, GenOpArg Src, + GenOpArg Size, GenOpArg* Rest) +{ + GenOp* op; + + ASSERT(Size.type == TAG_u); + + NEW_GENOP(stp, op); + op->next = NULL; + op->op = genop_has_map_field_3; + op->arity = 4; + + op->a[0] = Fail; + op->a[1] = Src; + op->a[2] = Rest[0]; + return op; +} /* * Freeze the code in memory, move the string table into place, diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index f25b4dbae5..2adba9b240 100755 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -64,7 +64,7 @@ static Export *gather_gc_info_res_trap; #define DECL_AM(S) Eterm AM_ ## S = am_atom_put(#S, sizeof(#S) - 1) -static char otp_correction_package[] = ERLANG_OTP_CORRECTION_PACKAGE; +static char otp_version[] = ERLANG_OTP_VERSION; /* Keep erts_system_version as a global variable for easy access from a core */ static char erts_system_version[] = ("Erlang/OTP " ERLANG_OTP_RELEASE "%s" @@ -312,7 +312,7 @@ erts_print_system_version(int to, void *arg, Process *c_p) int i, rc = -1; char *rc_str = ""; char rc_buf[100]; - char *ocp = otp_correction_package; + char *ov = otp_version; #ifdef ERTS_SMP Uint total, online, active; #ifdef ERTS_DIRTY_SCHEDULERS @@ -323,9 +323,9 @@ erts_print_system_version(int to, void *arg, Process *c_p) (void) erts_schedulers_state(&total, &online, &active, NULL, NULL, NULL, 0); #endif #endif - for (i = 0; i < sizeof(otp_correction_package)-4; i++) { - if (ocp[i] == '-' && ocp[i+1] == 'r' && ocp[i+2] == 'c') - rc = atoi(&ocp[i+3]); + for (i = 0; i < sizeof(otp_version)-4; i++) { + if (ov[i] == '-' && ov[i+1] == 'r' && ov[i+2] == 'c') + rc = atoi(&ov[i+3]); } if (rc >= 0) { if (rc == 0) @@ -2448,10 +2448,6 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) DECL_AM(unknown); BIF_RET(AM_unknown); } - } else if (ERTS_IS_ATOM_STR("otp_correction_package", BIF_ARG_1)) { - int n = sizeof(ERLANG_OTP_CORRECTION_PACKAGE)-1; - hp = HAlloc(BIF_P, 2*n); - BIF_RET(buf_to_intlist(&hp, ERLANG_OTP_CORRECTION_PACKAGE, n, NIL)); } else if (ERTS_IS_ATOM_STR("otp_release", BIF_ARG_1)) { int n = sizeof(ERLANG_OTP_RELEASE)-1; hp = HAlloc(BIF_P, 2*n); diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c index 41e64fcd4f..a5d67571e2 100644 --- a/erts/emulator/beam/erl_db.c +++ b/erts/emulator/beam/erl_db.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2013. All Rights Reserved. + * Copyright Ericsson AB 1996-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -125,6 +125,7 @@ get_meta_main_tab_lock(unsigned slot) static erts_smp_spinlock_t meta_main_tab_main_lock; static Uint meta_main_tab_first_free; /* Index of first free slot */ static int meta_main_tab_cnt; /* Number of active tables */ +static int meta_main_tab_top; /* Highest ever used slot + 1 */ static Uint meta_main_tab_slot_mask; /* The slot index part of an unnamed table id */ static Uint meta_main_tab_seq_incr; static Uint meta_main_tab_seq_cnt = 0; /* To give unique(-ish) table identifiers */ @@ -1469,6 +1470,10 @@ BIF_RETTYPE ets_new_2(BIF_ALIST_2) ASSERT(slot>=0 && slot<db_max_tabs); meta_main_tab_first_free = GET_NEXT_FREE_SLOT(slot); meta_main_tab_cnt++; + if (slot >= meta_main_tab_top) { + ASSERT(slot == meta_main_tab_top); + meta_main_tab_top = slot + 1; + } if (is_named) { ret = BIF_ARG_1; @@ -2058,27 +2063,31 @@ BIF_RETTYPE ets_all_0(BIF_ALIST_0) { DbTable* tb; Eterm previous; - int i, j; + int i; Eterm* hp; Eterm* hendp; int t_tabs_cnt; - int t_max_tabs; + int t_top; erts_smp_spin_lock(&meta_main_tab_main_lock); t_tabs_cnt = meta_main_tab_cnt; - t_max_tabs = db_max_tabs; + t_top = meta_main_tab_top; erts_smp_spin_unlock(&meta_main_tab_main_lock); hp = HAlloc(BIF_P, 2*t_tabs_cnt); hendp = hp + 2*t_tabs_cnt; previous = NIL; - j = 0; - for(i = 0; (i < t_max_tabs && j < t_tabs_cnt); i++) { + for(i = 0; i < t_top; i++) { erts_smp_rwmtx_t *mmtl = get_meta_main_tab_lock(i); erts_smp_rwmtx_rlock(mmtl); if (IS_SLOT_ALIVE(i)) { - j++; + if (hp == hendp) { + /* Racing table creator, grab some more heap space */ + t_tabs_cnt = 10; + hp = HAlloc(BIF_P, 2*t_tabs_cnt); + hendp = hp + 2*t_tabs_cnt; + } tb = meta_main_tab[i].u.tb; previous = CONS(hp, tb->common.id, previous); hp += 2; @@ -2849,6 +2858,7 @@ void init_db(void) ERTS_ETS_MISC_MEM_ADD(size); meta_main_tab_cnt = 0; + meta_main_tab_top = 0; for (i=1; i<db_max_tabs; i++) { SET_NEXT_FREE_SLOT(i-1,i); } diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index c35f1fc2c6..4126df6f34 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -1237,6 +1237,19 @@ static void steal_resource_type(ErlNifResourceType* type) } } +/* The opened_rt_list is used by enif_open_resource_type() + * in order to rollback "creates" and "take-overs" in case the load fails. + */ +struct opened_resource_type +{ + struct opened_resource_type* next; + + ErlNifResourceFlags op; + ErlNifResourceType* type; + ErlNifResourceDtor* new_dtor; +}; +static struct opened_resource_type* opened_rt_list = NULL; + ErlNifResourceType* enif_open_resource_type(ErlNifEnv* env, const char* module_str, @@ -1258,22 +1271,21 @@ enif_open_resource_type(ErlNifEnv* env, if (type == NULL) { if (flags & ERL_NIF_RT_CREATE) { type = erts_alloc(ERTS_ALC_T_NIF, - sizeof(struct enif_resource_type_t)); - type->dtor = dtor; + sizeof(struct enif_resource_type_t)); type->module = module_am; type->name = name_am; erts_refc_init(&type->refc, 1); - type->owner = env->mod_nif; - type->prev = &resource_type_list; - type->next = resource_type_list.next; - type->next->prev = type; - type->prev->next = type; op = ERL_NIF_RT_CREATE; + #ifdef DEBUG + type->dtor = (void*)1; + type->owner = (void*)2; + type->prev = (void*)3; + type->next = (void*)4; + #endif } } else { - if (flags & ERL_NIF_RT_TAKEOVER) { - steal_resource_type(type); + if (flags & ERL_NIF_RT_TAKEOVER) { op = ERL_NIF_RT_TAKEOVER; } else { @@ -1281,12 +1293,13 @@ enif_open_resource_type(ErlNifEnv* env, } } if (type != NULL) { - type->owner = env->mod_nif; - type->dtor = dtor; - if (type->dtor != NULL) { - erts_refc_inc(&type->owner->rt_dtor_cnt, 1); - } - erts_refc_inc(&type->owner->rt_cnt, 1); + struct opened_resource_type* ort = erts_alloc(ERTS_ALC_T_TMP, + sizeof(struct opened_resource_type)); + ort->op = op; + ort->type = type; + ort->new_dtor = dtor; + ort->next = opened_rt_list; + opened_rt_list = ort; } if (tried != NULL) { *tried = op; @@ -1294,6 +1307,51 @@ enif_open_resource_type(ErlNifEnv* env, return type; } +static void commit_opened_resource_types(struct erl_module_nif* lib) +{ + while (opened_rt_list) { + struct opened_resource_type* ort = opened_rt_list; + + ErlNifResourceType* type = ort->type; + + if (ort->op == ERL_NIF_RT_CREATE) { + type->prev = &resource_type_list; + type->next = resource_type_list.next; + type->next->prev = type; + type->prev->next = type; + } + else { /* ERL_NIF_RT_TAKEOVER */ + steal_resource_type(type); + } + + type->owner = lib; + type->dtor = ort->new_dtor; + + if (type->dtor != NULL) { + erts_refc_inc(&lib->rt_dtor_cnt, 1); + } + erts_refc_inc(&lib->rt_cnt, 1); + + opened_rt_list = ort->next; + erts_free(ERTS_ALC_T_TMP, ort); + } +} + +static void rollback_opened_resource_types(void) +{ + while (opened_rt_list) { + struct opened_resource_type* ort = opened_rt_list; + + if (ort->op == ERL_NIF_RT_CREATE) { + erts_free(ERTS_ALC_T_NIF, ort->type); + } + + opened_rt_list = ort->next; + erts_free(ERTS_ALC_T_TMP, ort); + } +} + + static void nif_resource_dtor(Binary* bin) { ErlNifResource* resource = (ErlNifResource*) ERTS_MAGIC_BIN_DATA(bin); @@ -1319,6 +1377,8 @@ void* enif_alloc_resource(ErlNifResourceType* type, size_t size) { Binary* bin = erts_create_magic_binary(SIZEOF_ErlNifResource(size), &nif_resource_dtor); ErlNifResource* resource = ERTS_MAGIC_BIN_DATA(bin); + + ASSERT(type->owner && type->next && type->prev); /* not allowed in load/upgrade */ resource->type = type; erts_refc_inc(&bin->refc, 1); #ifdef DEBUG @@ -2040,9 +2100,15 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) lib->entry = entry; erts_refc_init(&lib->rt_cnt, 0); erts_refc_init(&lib->rt_dtor_cnt, 0); + ASSERT(opened_rt_list == NULL); lib->mod = mod; env.mod_nif = lib; - if (mod->curr.nif != NULL) { /* Reload */ + if (mod->curr.nif != NULL) { /*************** Reload ******************/ + /* + * Repeated load_nif calls from same Erlang module instance ("reload") + * is deprecated and was only ment as a development feature not to + * be used in production systems. (See warning below) + */ int k; lib->priv_data = mod->curr.nif->priv_data; @@ -2074,6 +2140,7 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) ret = load_nif_error(BIF_P, reload, "Library reload-call unsuccessful."); } else { + commit_opened_resource_types(lib); mod->curr.nif->entry = NULL; /* to prevent 'unload' callback */ erts_unload_nif(mod->curr.nif); reload_warning = 1; @@ -2081,7 +2148,7 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) } else { lib->priv_data = NULL; - if (mod->old.nif != NULL) { /* Upgrade */ + if (mod->old.nif != NULL) { /**************** Upgrade ***************/ void* prev_old_data = mod->old.nif->priv_data; if (entry->upgrade == NULL) { ret = load_nif_error(BIF_P, upgrade, "Upgrade not supported by this NIF library."); @@ -2094,17 +2161,18 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) mod->old.nif->priv_data = prev_old_data; ret = load_nif_error(BIF_P, upgrade, "Library upgrade-call unsuccessful."); } - /*else if (mod->old_nif->priv_data != prev_old_data) { - refresh_cached_nif_data(mod->old_code, mod->old_nif); - }*/ + else + commit_opened_resource_types(lib); } - else if (entry->load != NULL) { /* Initial load */ + else if (entry->load != NULL) { /********* Initial load ***********/ erts_pre_nif(&env, BIF_P, lib); veto = entry->load(&env, &lib->priv_data, BIF_ARG_2); erts_post_nif(&env); if (veto) { ret = load_nif_error(BIF_P, "load", "Library load-call unsuccessful."); } + else + commit_opened_resource_types(lib); } } if (ret == am_ok) { @@ -2133,6 +2201,7 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) } else { error: + rollback_opened_resource_types(); ASSERT(ret != am_ok); if (lib != NULL) { erts_free(ERTS_ALC_T_NIF, lib); diff --git a/erts/emulator/beam/erl_utils.h b/erts/emulator/beam/erl_utils.h index 5b81d814c6..43a19e97f1 100644 --- a/erts/emulator/beam/erl_utils.h +++ b/erts/emulator/beam/erl_utils.h @@ -202,17 +202,18 @@ int eq(Eterm, Eterm); #define EQ(x,y) (((x) == (y)) || (is_not_both_immed((x),(y)) && eq((x),(y)))) #if HALFWORD_HEAP -Sint cmp_rel_opt(Eterm, Eterm*, Eterm, Eterm*, int); -#define cmp_rel(A,A_BASE,B,B_BASE) cmp_rel_opt(A,A_BASE,B,B_BASE,0) -#define cmp_rel_term(A,A_BASE,B,B_BASE) cmp_rel_opt(A,A_BASE,B,B_BASE,1) -#define CMP(A,B) cmp_rel_opt(A,NULL,B,NULL,0) -#define CMP_TERM(A,B) cmp_rel_opt(A,NULL,B,NULL,1) +Sint erts_cmp_rel_opt(Eterm, Eterm*, Eterm, Eterm*, int); +#define cmp_rel(A,A_BASE,B,B_BASE) erts_cmp_rel_opt(A,A_BASE,B,B_BASE,0) +#define cmp_rel_term(A,A_BASE,B,B_BASE) erts_cmp_rel_opt(A,A_BASE,B,B_BASE,1) +#define CMP(A,B) erts_cmp_rel_opt(A,NULL,B,NULL,0) +#define CMP_TERM(A,B) erts_cmp_rel_opt(A,NULL,B,NULL,1) #else -Sint cmp(Eterm, Eterm, int); -#define cmp_rel(A,A_BASE,B,B_BASE) cmp(A,B,0) -#define cmp_rel_term(A,A_BASE,B,B_BASE) cmp(A,B,1) -#define CMP(A,B) cmp(A,B,0) -#define CMP_TERM(A,B) cmp(A,B,1) +Sint cmp(Eterm, Eterm); +Sint erts_cmp(Eterm, Eterm, int); +#define cmp_rel(A,A_BASE,B,B_BASE) erts_cmp(A,B,0) +#define cmp_rel_term(A,A_BASE,B,B_BASE) erts_cmp(A,B,1) +#define CMP(A,B) erts_cmp(A,B,0) +#define CMP_TERM(A,B) erts_cmp(A,B,1) #endif #define cmp_lt(a,b) (CMP((a),(b)) < 0) diff --git a/erts/emulator/beam/ops.tab b/erts/emulator/beam/ops.tab index 9ea3a3a8ea..8406f5344a 100644 --- a/erts/emulator/beam/ops.tab +++ b/erts/emulator/beam/ops.tab @@ -1469,11 +1469,6 @@ apply_last I P # Map instructions in R17. # -# put_map Fail Src Dst Live Size Rest=* => jump Fail -# is_map Fail Src => jump Fail -# has_map_field Fail Src Key => jump Fail -# get_map_element Fail Src Key Dst => jump Fail - put_map_assoc F n Dst Live Size Rest=* => new_map F Dst Live Size Rest put_map_exact F n Dst Live Size Rest=* => new_map F Dst Live Size Rest put_map_assoc F Src Dst Live Size Rest=* => \ @@ -1492,6 +1487,15 @@ is_map f r is_map f x is_map f y +## Transform has_map_field(s) #{ K1 := _, K2 := _ } + +has_map_field/3 + +has_map_fields Fail Src Size=u==1 Rest=* => gen_has_map_field(Fail,Src,Size,Rest) +has_map_fields Fail Src Size Rest=* => i_has_map_fields Fail Src Size Rest + +i_has_map_fields f s I + has_map_field Fail Src=rxy Key=arxy => i_has_map_field Fail Src Key has_map_field Fail Src Key => move Key x | i_has_map_field Fail Src x @@ -1509,6 +1513,15 @@ i_has_map_field f r y i_has_map_field f x y i_has_map_field f y y +## Transform get_map_elements(s) #{ K1 := V1, K2 := V2 } + +get_map_element/4 + +get_map_elements Fail Src=rxy Size=u==2 Rest=* => gen_get_map_element(Fail,Src,Size,Rest) +get_map_elements Fail Src Size Rest=* => i_get_map_elements Fail Src Size Rest + +i_get_map_elements f s I + get_map_element Fail Src=rxy Key=ax Dst => i_get_map_element Fail Src Key Dst get_map_element Fail Src=rxy Key=rycq Dst => \ move Key x | i_get_map_element Fail Src x Dst diff --git a/erts/emulator/beam/utils.c b/erts/emulator/beam/utils.c index bc4a05d385..7da555b18d 100644 --- a/erts/emulator/beam/utils.c +++ b/erts/emulator/beam/utils.c @@ -2425,14 +2425,24 @@ static int cmp_atoms(Eterm a, Eterm b) bb->name+3, bb->len-3); } -/* cmp(Eterm a, Eterm b, int exact) +#if !HALFWORD_HEAP +/* cmp(Eterm a, Eterm b) + * For compatibility with HiPE - arith-based compare. + */ +Sint cmp(Eterm a, Eterm b) +{ + return erts_cmp(a, b, 0); +} +#endif + +/* erts_cmp(Eterm a, Eterm b, int exact) * exact = 1 -> term-based compare * exact = 0 -> arith-based compare */ #if HALFWORD_HEAP -Sint cmp_rel_opt(Eterm a, Eterm* a_base, Eterm b, Eterm* b_base, int exact) +Sint erts_cmp_rel_opt(Eterm a, Eterm* a_base, Eterm b, Eterm* b_base, int exact) #else -Sint cmp(Eterm a, Eterm b, int exact) +Sint erts_cmp(Eterm a, Eterm b, int exact) #endif { DECLARE_WSTACK(stack); diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c index a5294ad84e..865cb50a56 100644 --- a/erts/emulator/sys/unix/sys.c +++ b/erts/emulator/sys/unix/sys.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2013. All Rights Reserved. + * Copyright Ericsson AB 1996-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -2561,7 +2561,7 @@ void *erts_sys_aligned_alloc(UWord alignment, UWord size) #ifdef HAVE_POSIX_MEMALIGN void *ptr = NULL; int error; - ASSERT(alignment && (alignment & ~alignment) == 0); /* power of 2 */ + ASSERT(alignment && (alignment & (alignment-1)) == 0); /* power of 2 */ error = posix_memalign(&ptr, (size_t) alignment, (size_t) size); #if HAVE_ERTS_MSEG if (error || !ptr) { @@ -2584,7 +2584,7 @@ void *erts_sys_aligned_alloc(UWord alignment, UWord size) void erts_sys_aligned_free(UWord alignment, void *ptr) { - ASSERT(alignment && (alignment & ~alignment) == 0); /* power of 2 */ + ASSERT(alignment && (alignment & (alignment-1)) == 0); /* power of 2 */ free(ptr); } diff --git a/erts/emulator/sys/win32/sys.c b/erts/emulator/sys/win32/sys.c index 5ea4703a7a..a11ac73ebc 100755 --- a/erts/emulator/sys/win32/sys.c +++ b/erts/emulator/sys/win32/sys.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2013. All Rights Reserved. + * Copyright Ericsson AB 1996-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -2757,7 +2757,7 @@ void erts_sys_free(ErtsAlcType_t t, void *x, void *p) void *erts_sys_aligned_alloc(UWord alignment, UWord size) { void *ptr; - ASSERT(alignment && (alignment & ~alignment) == 0); /* power of 2 */ + ASSERT(alignment && (alignment & (alignment-1)) == 0); /* power of 2 */ ptr = _aligned_malloc((size_t) size, (size_t) alignment); ASSERT(!ptr || (((UWord) ptr) & (alignment - 1)) == 0); return ptr; @@ -2765,14 +2765,14 @@ void *erts_sys_aligned_alloc(UWord alignment, UWord size) void erts_sys_aligned_free(UWord alignment, void *ptr) { - ASSERT(alignment && (alignment & ~alignment) == 0); /* power of 2 */ + ASSERT(alignment && (alignment & (alignment-1)) == 0); /* power of 2 */ _aligned_free(ptr); } void *erts_sys_aligned_realloc(UWord alignment, void *ptr, UWord size, UWord old_size) { void *new_ptr; - ASSERT(alignment && (alignment & ~alignment) == 0); /* power of 2 */ + ASSERT(alignment && (alignment & (alignment-1)) == 0); /* power of 2 */ new_ptr = _aligned_realloc(ptr, (size_t) size, (size_t) alignment); ASSERT(!new_ptr || (((UWord) new_ptr) & (alignment - 1)) == 0); return new_ptr; diff --git a/erts/emulator/test/driver_SUITE.erl b/erts/emulator/test/driver_SUITE.erl index 06211406b4..c62bc0c454 100644 --- a/erts/emulator/test/driver_SUITE.erl +++ b/erts/emulator/test/driver_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1945,6 +1945,14 @@ otp_9302(Config) when is_list(Config) -> end. thr_free_drv(Config) when is_list(Config) -> + case erlang:system_info(threads) of + false -> + {skipped, "No thread support"}; + true -> + thr_free_drv_do(Config) + end. + +thr_free_drv_do(Config) -> ?line Path = ?config(data_dir, Config), ?line erl_ddll:start(), ?line ok = load_driver(Path, thr_free_drv), diff --git a/erts/emulator/test/driver_SUITE_data/otp_9302_drv.c b/erts/emulator/test/driver_SUITE_data/otp_9302_drv.c index 7c144d20cf..ad29d17f06 100644 --- a/erts/emulator/test/driver_SUITE_data/otp_9302_drv.c +++ b/erts/emulator/test/driver_SUITE_data/otp_9302_drv.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2011-2013. All Rights Reserved. + * Copyright Ericsson AB 2011-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -94,7 +94,7 @@ DRIVER_INIT(otp_9302_drv) static void stop(ErlDrvData drv_data) { Otp9302Data *data = (Otp9302Data *) drv_data; - if (!data->smp) + if (data->msgq.mtx) erl_drv_mutex_destroy(data->msgq.mtx); driver_free(data); } @@ -114,13 +114,16 @@ static ErlDrvData start(ErlDrvPort port, driver_system_info(&sys_info, sizeof(ErlDrvSysInfo)); data->smp = sys_info.smp_support; + data->msgq.mtx = NULL; if (!data->smp) { data->msgq.start = NULL; data->msgq.end = NULL; - data->msgq.mtx = erl_drv_mutex_create(""); - if (!data->msgq.mtx) { - driver_free(data); - return ERL_DRV_ERROR_GENERAL; + if (sys_info.thread_support) { + data->msgq.mtx = erl_drv_mutex_create(""); + if (!data->msgq.mtx) { + driver_free(data); + return ERL_DRV_ERROR_GENERAL; + } } } @@ -143,19 +146,22 @@ static void enqueue_reply(Otp9302AsyncData *adata) Otp9302MsgQ *msgq = adata->msgq; adata->next = NULL; adata->refc++; - erl_drv_mutex_lock(msgq->mtx); + if (msgq->mtx) + erl_drv_mutex_lock(msgq->mtx); if (msgq->end) msgq->end->next = adata; else msgq->end = msgq->start = adata; msgq->end = adata; - erl_drv_mutex_unlock(msgq->mtx); + if (msgq->mtx) + erl_drv_mutex_unlock(msgq->mtx); } static void dequeue_replies(Otp9302AsyncData *adata) { Otp9302MsgQ *msgq = adata->msgq; - erl_drv_mutex_lock(msgq->mtx); + if (msgq->mtx) + erl_drv_mutex_lock(msgq->mtx); if (--adata->refc == 0) driver_free(adata); while (msgq->start) { @@ -166,7 +172,8 @@ static void dequeue_replies(Otp9302AsyncData *adata) driver_free(adata); } msgq->start = msgq->end = NULL; - erl_drv_mutex_unlock(msgq->mtx); + if (msgq->mtx) + erl_drv_mutex_unlock(msgq->mtx); } static void async_invoke(void *data) diff --git a/erts/emulator/test/nif_SUITE.erl b/erts/emulator/test/nif_SUITE.erl index bcc1f9e5af..a854d3f05b 100644 --- a/erts/emulator/test/nif_SUITE.erl +++ b/erts/emulator/test/nif_SUITE.erl @@ -850,6 +850,138 @@ resource_takeover(Config) when is_list(Config) -> ?line ok = forget_resource(AN4), ?line [] = nif_mod_call_history(), + + %% + %% Test rollback after failed upgrade of same lib-version + %% + + {A5,BinA5} = make_resource(2, Holder, "A5"), + {NA5,BinNA5} = make_resource(0, Holder, "NA5"), + {AN5,_BinAN5} = make_resource(1, Holder, "AN5"), + + {A6,BinA6} = make_resource(2, Holder, "A6"), + {NA6,BinNA6} = make_resource(0, Holder, "NA6"), + {AN6,_BinAN6} = make_resource(1, Holder, "AN6"), + + {module,nif_mod} = erlang:load_module(nif_mod,ModBin), + undefined = nif_mod:lib_version(), + {error,{upgrade,_}} = + nif_mod:load_nif_lib(Config, 1, + [{resource_type, 4, ?RT_TAKEOVER, "resource_type_A",resource_dtor_B, + ?RT_TAKEOVER}, + {resource_type, 4, ?RT_TAKEOVER bor ?RT_CREATE, "resource_type_null_A",null, + ?RT_TAKEOVER}, + {resource_type, 4, ?RT_TAKEOVER, "resource_type_A_null",resource_dtor_A, + ?RT_TAKEOVER}, + {resource_type, 4, ?RT_CREATE, "Mr Pink", resource_dtor_A, + ?RT_CREATE}, + + {return, 1} % FAIL + ]), + + undefined = nif_mod:lib_version(), + [{upgrade,1,5,105}] = nif_mod_call_history(), + + %% Make sure dtor was not changed (from A to B) + ok = forget_resource(A5), + [{{resource_dtor_A_v1,BinA5},1,6,106}] = nif_mod_call_history(), + + %% Make sure dtor was not nullified (from A to null) + ok = forget_resource(NA5), + [{{resource_dtor_A_v1,BinNA5},1,7,107}] = nif_mod_call_history(), + + %% Make sure dtor was not added (from null to A) + ok = forget_resource(AN5), + [] = nif_mod_call_history(), + + %% + %% Test rollback after failed upgrade of other lib-version + %% + + {error,{upgrade,_}} = + nif_mod:load_nif_lib(Config, 2, + [{resource_type, 4, ?RT_TAKEOVER, "resource_type_A",resource_dtor_B, + ?RT_TAKEOVER}, + {resource_type, 4, ?RT_TAKEOVER bor ?RT_CREATE, "resource_type_null_A",null, + ?RT_TAKEOVER}, + {resource_type, 4, ?RT_TAKEOVER, "resource_type_A_null",resource_dtor_A, + ?RT_TAKEOVER}, + {resource_type, null, ?RT_TAKEOVER, "Mr Pink", resource_dtor_A, + ?RT_TAKEOVER}, + {resource_type, 4, ?RT_CREATE, "Mr Pink", resource_dtor_A, + ?RT_CREATE}, + + {return, 1} % FAIL + ]), + + undefined = nif_mod:lib_version(), + [{upgrade,2,_,_}] = nif_mod_call_history(), + + %% Make sure dtor was not changed (from A to B) + ok = forget_resource(A6), + [{{resource_dtor_A_v1,BinA6},1,_,_}] = nif_mod_call_history(), + + %% Make sure dtor was not nullified (from A to null) + ok = forget_resource(NA6), + [{{resource_dtor_A_v1,BinNA6},1,_,_}] = nif_mod_call_history(), + + %% Make sure dtor was not added (from null to A) + ok = forget_resource(AN6), + [] = nif_mod_call_history(), + + %% + %% Test rolback after failed initial load + %% + false = code:purge(nif_mod), + [{unload,1,_,_}] = nif_mod_call_history(), + true = code:delete(nif_mod), + false = code:purge(nif_mod), + [] = nif_mod_call_history(), + + + {module,nif_mod} = erlang:load_module(nif_mod,ModBin), + undefined = nif_mod:lib_version(), + {error,{load,_}} = + nif_mod:load_nif_lib(Config, 1, + [{resource_type, null, ?RT_TAKEOVER, "resource_type_A",resource_dtor_A, + ?RT_TAKEOVER}, + {resource_type, 4, ?RT_TAKEOVER bor ?RT_CREATE, "resource_type_null_A",null, + ?RT_CREATE}, + {resource_type, 4, ?RT_CREATE, "resource_type_A_null",resource_dtor_A, + ?RT_CREATE}, + {resource_type, 4, ?RT_CREATE, "Mr Pink", resource_dtor_A, + ?RT_CREATE}, + + {return, 1} % FAIL + ]), + + undefined = nif_mod:lib_version(), + ok = nif_mod:load_nif_lib(Config, 1, + [{resource_type, null, ?RT_TAKEOVER, "resource_type_A",resource_dtor_A, + ?RT_TAKEOVER}, + {resource_type, 0, ?RT_TAKEOVER bor ?RT_CREATE, "resource_type_null_A", + resource_dtor_A, ?RT_CREATE}, + + {resource_type, 1, ?RT_CREATE, "resource_type_A_null", null, + ?RT_CREATE}, + {resource_type, null, ?RT_TAKEOVER, "Mr Pink", resource_dtor_A, + ?RT_TAKEOVER}, + + {return, 0} % SUCCESS + ]), + + ?line hold_nif_mod_priv_data(nif_mod:get_priv_data_ptr()), + ?line [{load,1,1,101},{get_priv_data_ptr,1,2,102}] = nif_mod_call_history(), + + {NA7,BinNA7} = make_resource(0, Holder, "NA7"), + {AN7,BinAN7} = make_resource(1, Holder, "AN7"), + + ok = forget_resource(NA7), + [{{resource_dtor_A_v1,BinNA7},1,_,_}] = nif_mod_call_history(), + + ok = forget_resource(AN7), + [] = nif_mod_call_history(), + ?line true = lists:member(?MODULE, erlang:system_info(taints)), ?line true = lists:member(nif_mod, erlang:system_info(taints)), ?line verify_tmpmem(TmpMem), @@ -1406,7 +1538,7 @@ dirty_nif(Config) when is_list(Config) -> {skipped,"No dirty scheduler support"} end. -next_msg(Pid) -> +next_msg(_Pid) -> receive M -> M after 100 -> diff --git a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c index b550d1f16d..160f4843ad 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c +++ b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c @@ -1477,7 +1477,6 @@ static ERL_NIF_TERM consume_timeslice_nif(ErlNifEnv* env, int argc, const ERL_NI { int percent; char atom[10]; - int do_repeat; if (!enif_get_int(env, argv[0], &percent) || !enif_get_atom(env, argv[1], atom, sizeof(atom), ERL_NIF_LATIN1)) { diff --git a/erts/emulator/test/nif_SUITE_data/nif_mod.c b/erts/emulator/test/nif_SUITE_data/nif_mod.c index e32d10057c..55a0d2ac4f 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_mod.c +++ b/erts/emulator/test/nif_SUITE_data/nif_mod.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2009-2010. All Rights Reserved. + * Copyright Ericsson AB 2009-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -41,6 +41,7 @@ static ERL_NIF_TERM am_null; static ERL_NIF_TERM am_resource_type; static ERL_NIF_TERM am_resource_dtor_A; static ERL_NIF_TERM am_resource_dtor_B; +static ERL_NIF_TERM am_return; static NifModPrivData* priv_data(ErlNifEnv* env) { @@ -54,6 +55,7 @@ static void init(ErlNifEnv* env) am_resource_type = enif_make_atom(env, "resource_type"); am_resource_dtor_A = enif_make_atom(env, "resource_dtor_A"); am_resource_dtor_B = enif_make_atom(env, "resource_dtor_B"); + am_return = enif_make_atom(env, "return"); } static void add_call_with_arg(ErlNifEnv* env, NifModPrivData* data, const char* func_name, @@ -105,19 +107,15 @@ static void resource_dtor_B(ErlNifEnv* env, void* a) } /* {resource_type, Ix|null, ErlNifResourceFlags in, "TypeName", dtor(A|B|null), ErlNifResourceFlags out}*/ -static void open_resource_type(ErlNifEnv* env, ERL_NIF_TERM op_tpl) +static void open_resource_type(ErlNifEnv* env, const ERL_NIF_TERM* arr) { NifModPrivData* data = priv_data(env); - const ERL_NIF_TERM* arr; - int arity; char rt_name[30]; union { ErlNifResourceFlags e; int i; } flags, exp_res, got_res; unsigned ix; ErlNifResourceDtor* dtor; ErlNifResourceType* got_ptr; - CHECK(enif_get_tuple(env, op_tpl, &arity, &arr)); - CHECK(arity == 6); CHECK(enif_is_identical(arr[0], am_resource_type)); CHECK(enif_get_int(env, arr[2], &flags.i)); CHECK(enif_get_string(env, arr[3], rt_name, sizeof(rt_name), ERL_NIF_LATIN1) > 0); @@ -147,18 +145,32 @@ static void open_resource_type(ErlNifEnv* env, ERL_NIF_TERM op_tpl) CHECK(got_res.e == exp_res.e); } -static void do_load_info(ErlNifEnv* env, ERL_NIF_TERM load_info) +static void do_load_info(ErlNifEnv* env, ERL_NIF_TERM load_info, int* retvalp) { NifModPrivData* data = priv_data(env); ERL_NIF_TERM head, tail; unsigned ix; + for (ix=0; ix<RT_MAX; ix++) { data->rt_arr[ix] = NULL; } for (head = load_info; enif_get_list_cell(env, head, &head, &tail); head = tail) { - - open_resource_type(env, head); + const ERL_NIF_TERM* arr; + int arity; + + CHECK(enif_get_tuple(env, head, &arity, &arr)); + switch (arity) { + case 6: + open_resource_type(env, arr); + break; + case 2: + CHECK(arr[0] == am_return); + CHECK(enif_get_int(env, arr[1], retvalp)); + break; + default: + CHECK(0); + } } CHECK(enif_is_empty_list(env, head)); } @@ -166,6 +178,7 @@ static void do_load_info(ErlNifEnv* env, ERL_NIF_TERM load_info) static int load(ErlNifEnv* env, void** priv, ERL_NIF_TERM load_info) { NifModPrivData* data; + int retval = 0; init(env); data = (NifModPrivData*) enif_alloc(sizeof(NifModPrivData)); @@ -177,38 +190,41 @@ static int load(ErlNifEnv* env, void** priv, ERL_NIF_TERM load_info) add_call(env, data, "load"); - do_load_info(env, load_info); - data->calls = 0; - return 0; + do_load_info(env, load_info, &retval); + if (retval) + NifModPrivData_release(data); + return retval; } static int reload(ErlNifEnv* env, void** priv, ERL_NIF_TERM load_info) { NifModPrivData* data = (NifModPrivData*) *priv; + int retval = 0; init(env); add_call(env, data, "reload"); - do_load_info(env, load_info); - return 0; + do_load_info(env, load_info, &retval); + return retval; } static int upgrade(ErlNifEnv* env, void** priv, void** old_priv_data, ERL_NIF_TERM load_info) { NifModPrivData* data = (NifModPrivData*) *old_priv_data; + int retval = 0; init(env); add_call(env, data, "upgrade"); data->ref_cnt++; *priv = *old_priv_data; - do_load_info(env, load_info); + do_load_info(env, load_info, &retval); - return 0; + return retval; } static void unload(ErlNifEnv* env, void* priv) { NifModPrivData* data = (NifModPrivData*) priv; - int is_last; + add_call(env, data, "unload"); NifModPrivData_release(data); } diff --git a/erts/emulator/test/nif_SUITE_data/nif_mod.h b/erts/emulator/test/nif_SUITE_data/nif_mod.h index cd0ecf4b54..fb14fee815 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_mod.h +++ b/erts/emulator/test/nif_SUITE_data/nif_mod.h @@ -14,7 +14,6 @@ typedef struct call_info_t typedef struct { ErlNifMutex* mtx; - int calls; int ref_cnt; CallInfo* call_history; ErlNifResourceType* rt_arr[RT_MAX]; @@ -28,6 +27,11 @@ typedef struct enif_mutex_unlock((NMPD)->mtx); \ if (is_last) { \ enif_mutex_destroy((NMPD)->mtx); \ + while ((NMPD)->call_history) { \ + CallInfo* next = (NMPD)->call_history->next; \ + enif_free((NMPD)->call_history); \ + (NMPD)->call_history = next; \ + } \ enif_free((NMPD)); \ } \ }while (0) diff --git a/erts/emulator/utils/make_version b/erts/emulator/utils/make_version index 02b68f2b39..0ba1c77930 100755 --- a/erts/emulator/utils/make_version +++ b/erts/emulator/utils/make_version @@ -39,10 +39,10 @@ if ($ARGV[0] eq '-o') { } my $release = shift; -defined $release or die "No release specified"; +defined $release or die "No otp release specified"; -my $correction_package = shift; -defined $correction_package or die "No correction package specified"; +my $otp_version = shift; +defined $otp_version or die "No otp version specified"; my $version = shift; defined $version or die "No version name specified"; @@ -56,7 +56,7 @@ open(FILE, ">$outputfile") or die "Can't create $outputfile: $!"; print FILE <<EOF; /* This file was created by 'make_version' -- don't modify. */ #define ERLANG_OTP_RELEASE "$release" -#define ERLANG_OTP_CORRECTION_PACKAGE "$correction_package" +#define ERLANG_OTP_VERSION "$otp_version" #define ERLANG_VERSION "$version" #define ERLANG_COMPILE_DATE "$time_str" #define ERLANG_ARCHITECTURE "$architecture" diff --git a/erts/etc/win32/nsis/erlang.nsi b/erts/etc/win32/nsis/erlang.nsi index f4fd2b4cdb..162e634148 100644 --- a/erts/etc/win32/nsis/erlang.nsi +++ b/erts/etc/win32/nsis/erlang.nsi @@ -93,6 +93,7 @@ SectionIn 1 RO skip_silent_mode: SetOutPath "$INSTDIR" + File "${TESTROOT}\OTP_VERSION" File "${TESTROOT}\Install.ini" File "${TESTROOT}\Install.exe" File /r "${TESTROOT}\releases" diff --git a/erts/etc/win32/nsis/erlang20.nsi b/erts/etc/win32/nsis/erlang20.nsi index 3333c4a9aa..3ee33e8121 100644 --- a/erts/etc/win32/nsis/erlang20.nsi +++ b/erts/etc/win32/nsis/erlang20.nsi @@ -144,6 +144,7 @@ Section "Development" SecErlangDev SectionIn 1 RO
SetOutPath "$INSTDIR"
+ File "${TESTROOT}\OTP_VERSION"
File "${TESTROOT}\Install.ini"
File "${TESTROOT}\Install.exe"
SetOutPath "$INSTDIR\releases"
diff --git a/erts/start_scripts/Makefile b/erts/start_scripts/Makefile index 32e65a227f..14d5d46195 100644 --- a/erts/start_scripts/Makefile +++ b/erts/start_scripts/Makefile @@ -52,7 +52,7 @@ endif ############################################################################## # Get version numbers from the VSN files -# VSN & SYSTEM_VSN +# VSN include ../vsn.mk include $(LIBPATH)/kernel/vsn.mk include $(LIBPATH)/stdlib/vsn.mk diff --git a/erts/vsn.mk b/erts/vsn.mk index 88a393f3d5..081fb66398 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -19,11 +19,6 @@ VSN = 6.0 -# OTP major version -SYSTEM_VSN = 17 -# OTP correction package version -SYSTEM_CP_VSN = 17.0-rc1 - # Port number 4365 in 4.2 # Port number 4366 in 4.3 # Port number 4368 in 4.4.0 - 4.6.2 diff --git a/lib/Makefile b/lib/Makefile index 6301c882b2..4b979fc28a 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -25,10 +25,11 @@ ERTS_APPLICATIONS = stdlib sasl kernel compiler # Then these have to be build ERLANG_APPLICATIONS = tools test_server common_test runtime_tools \ - inets xmerl edoc erl_docgen parsetools + inets parsetools # These are only build if -a is given to otp_build or make is used directly -ALL_ERLANG_APPLICATIONS = snmp otp_mibs erl_interface asn1 jinterface \ +ALL_ERLANG_APPLICATIONS = xmerl edoc erl_docgen snmp otp_mibs erl_interface \ + asn1 jinterface \ wx debugger reltool gs \ ic mnesia crypto orber os_mon syntax_tools \ public_key ssl observer odbc diameter \ @@ -59,10 +60,14 @@ else else ifdef TERTIARY_BOOTSTRAP SUB_DIRECTORIES = snmp sasl jinterface ic syntax_tools wx - else # Not bootstrap build - SUB_DIRECTORIES = $(ERTS_APPLICATIONS) \ - $(ERLANG_APPLICATIONS) \ - $(EXTRA_APPLICATIONS) + else + ifdef DOC_BOOTSTRAP + SUB_DIRECTORIES = xmerl edoc erl_docgen + else # Not bootstrap build + SUB_DIRECTORIES = $(ERTS_APPLICATIONS) \ + $(ERLANG_APPLICATIONS) \ + $(EXTRA_APPLICATIONS) + endif endif endif endif diff --git a/lib/asn1/src/asn1ct.erl b/lib/asn1/src/asn1ct.erl index 531a4935fe..30f493a2df 100644 --- a/lib/asn1/src/asn1ct.erl +++ b/lib/asn1/src/asn1ct.erl @@ -1040,7 +1040,7 @@ get_file_list1(Stream,Dir,Includes,Acc) -> Ret = io:get_line(Stream,''), case Ret of eof -> - file:close(Stream), + ok = file:close(Stream), lists:reverse(Acc); FileName -> SuffixedNameList = @@ -1926,8 +1926,9 @@ read_config_file(ModuleName) -> Includes = [I || {i,I} <- Options], read_config_file1(ModuleName,Includes); {error,Reason} -> - file:format_error(Reason), - throw({error,{"error reading asn1 config file",Reason}}) + Error = "error reading asn1 config file: " ++ + file:format_error(Reason), + throw({error,Error}) end. read_config_file1(ModuleName,[]) -> case filename:extension(ModuleName) of @@ -1945,8 +1946,9 @@ read_config_file1(ModuleName,[H|T]) -> {error,enoent} -> read_config_file1(ModuleName,T); {error,Reason} -> - file:format_error(Reason), - throw({error,{"error reading asn1 config file",Reason}}) + Error = "error reading asn1 config file: " ++ + file:format_error(Reason), + throw({error,Error}) end. get_config_info(CfgList,InfoType) -> diff --git a/lib/asn1/src/asn1ct_check.erl b/lib/asn1/src/asn1ct_check.erl index df034c96ad..b9f2cb876a 100644 --- a/lib/asn1/src/asn1ct_check.erl +++ b/lib/asn1/src/asn1ct_check.erl @@ -2464,7 +2464,7 @@ normalize_value(S0, Type, {'DEFAULT',Value}, NameList) -> {'BIT STRING',CType,_} -> normalize_bitstring(S,Value,CType); {'OCTET STRING',CType,_} -> - normalize_octetstring(S,Value,CType); + normalize_octetstring(S0, Value, CType); {'NULL',_CType,_} -> %%normalize_null(Value); 'NULL'; @@ -2612,20 +2612,9 @@ normalize_octetstring(S,Value,CType) -> fun normalize_octetstring/3,[]); {Name,String} when is_atom(Name) -> normalize_octetstring(S,String,CType); - List when is_list(List) -> - %% check if list elements are valid octet values - lists:map(fun([])-> ok; - (H)when H > 255-> - asn1ct:warning("not legal octet value ~p in OCTET STRING, ~p~n", - [H,List],S, - "not legal octet value ~p in OCTET STRING"); - (_)-> ok - end, List), - List; - Other -> - asn1ct:warning("unknown default value ~p~n",[Other],S, - "unknown default value"), - Value + _ -> + Item = S#state.value, + throw(asn1_error(S, Item, illegal_octet_string_value)) end. normalize_objectidentifier(S, Value) -> @@ -3091,7 +3080,6 @@ check_type(S=#state{recordtopname=TopName},Type,Ts) when is_record(Ts,type) -> Ct=maybe_illicit_implicit_tag(open_type,Tag), TempNewDef#newt{type='ASN1_OPEN_TYPE',tag=Ct}; 'INTEGER' -> - check_integer(S,[],Constr), TempNewDef#newt{tag= merge_tags(Tag,?TAG_PRIMITIVE(?N_INTEGER))}; @@ -3938,9 +3926,9 @@ check_constraint(S,{simpletable,Type}) -> #'Externaltypereference'{} -> ERef = check_externaltypereference(S,C), {simpletable,ERef#'Externaltypereference'.type}; - #type{def=#'Externaltypereference'{type=T}} -> - check_externaltypereference(S,C#type.def), - {simpletable,T}; + #type{def=#'Externaltypereference'{}=ExtTypeRef} -> + ERef = check_externaltypereference(S, ExtTypeRef), + {simpletable,ERef#'Externaltypereference'.type}; {valueset,#type{def=ERef=#'Externaltypereference'{}}} -> % this is an object set {_,TDef} = get_referenced_type(S,ERef), case TDef#typedef.typespec of @@ -6814,6 +6802,8 @@ asn1_error(#state{mname=Where}, Item, Error) -> format_error({already_defined,Name,PrevLine}) -> io_lib:format("the name ~p has already been defined at line ~p", [Name,PrevLine]); +format_error(illegal_octet_string_value) -> + "expecting a bstring or an hstring as value for an OCTET STRING"; format_error({invalid_fields,Fields,Obj}) -> io_lib:format("invalid ~s in ~p", [format_fields(Fields),Obj]); format_error({missing_mandatory_fields,Fields,Obj}) -> diff --git a/lib/asn1/src/asn1ct_gen.erl b/lib/asn1/src/asn1ct_gen.erl index 71d870b4ce..4707e517b4 100644 --- a/lib/asn1/src/asn1ct_gen.erl +++ b/lib/asn1/src/asn1ct_gen.erl @@ -71,7 +71,7 @@ pgen_module(OutFile,Erules,Module, HrlGenerated = pgen_hrl(Erules,Module,TypeOrVal,Options,Indent), asn1ct_name:start(), ErlFile = lists:concat([OutFile,".erl"]), - open_output_file(ErlFile), + _ = open_output_file(ErlFile), asn1ct_func:start_link(), gen_head(Erules,Module,HrlGenerated), pgen_exports(Erules,Module,TypeOrVal), @@ -190,7 +190,7 @@ pgen_check_defaultval(Erules,Module) -> "********~n~n",[X]) end, lists:foreach(Fun,CheckObjects), - file:close(IoDevice); + ok = file:close(IoDevice); _ -> ok end, gen_check_defaultval(Erules,Module,CheckObjects). @@ -1124,7 +1124,7 @@ pgen_info() -> open_hrl(OutFile,Module) -> File = lists:concat([OutFile,".hrl"]), - open_output_file(File), + _ = open_output_file(File), gen_hrlhead(Module). %% EMIT functions ************************ diff --git a/lib/asn1/src/asn1ct_table.erl b/lib/asn1/src/asn1ct_table.erl index a5eb6d0413..2eca80eda3 100644 --- a/lib/asn1/src/asn1ct_table.erl +++ b/lib/asn1/src/asn1ct_table.erl @@ -22,34 +22,25 @@ %% Table abstraction module for ASN.1 compiler -export([new/1]). --export([new/2]). -export([new_reuse/1]). --export([new_reuse/2]). -export([exists/1]). -export([size/1]). -export([insert/2]). -export([lookup/2]). -export([match/2]). -export([to_list/1]). --export([delete/1]). % TODO: Remove (since we run in a separate process) +-export([delete/1]). -%% Always creates a new table -new(Table) -> new(Table, []). -new(Table, Options) -> - TableId = case get(Table) of - undefined -> - ets:new(Table, Options); - _ -> - delete(Table), - ets:new(Table, Options) - end, +%% Always create a new table. +new(Table) -> + undefined = get(Table), %Assertion. + TableId = ets:new(Table, []), put(Table, TableId). -%% Only create it if it doesn't exist yet -new_reuse(Table) -> new_reuse(Table, []). -new_reuse(Table, Options) -> - not exists(Table) andalso new(Table, Options). +%% Only create it if it doesn't exist yet. +new_reuse(Table) -> + not exists(Table) andalso new(Table). exists(Table) -> get(Table) =/= undefined. @@ -63,14 +54,17 @@ match(Table, MatchSpec) -> ets:match(get(Table), MatchSpec). to_list(Table) -> ets:tab2list(get(Table)). +%% Deleting tables is no longer strictly necessary since each compilation +%% runs in separate process, but it will reduce memory consumption +%% especially when many compilations are run in parallel. + delete(Tables) when is_list(Tables) -> [delete(T) || T <- Tables], true; delete(Table) when is_atom(Table) -> - case get(Table) of + case erase(Table) of undefined -> true; TableId -> - ets:delete(TableId), - erase(Table) + ets:delete(TableId) end. diff --git a/lib/asn1/src/asn1ct_tok.erl b/lib/asn1/src/asn1ct_tok.erl index 85199c65ec..33f4379173 100644 --- a/lib/asn1/src/asn1ct_tok.erl +++ b/lib/asn1/src/asn1ct_tok.erl @@ -36,7 +36,7 @@ process(Stream,Lno,R) -> process(io:get_line(Stream, ''), Stream,Lno+1,R). process(eof, Stream,Lno,R) -> - file:close(Stream), + ok = file:close(Stream), lists:flatten(lists:reverse([{'$end',Lno}|R])); diff --git a/lib/asn1/src/asn1ct_value.erl b/lib/asn1/src/asn1ct_value.erl index a86c963b9d..be8b967cf0 100644 --- a/lib/asn1/src/asn1ct_value.erl +++ b/lib/asn1/src/asn1ct_value.erl @@ -352,7 +352,7 @@ random_unnamed_bit_string(M, C) -> random(Upper) -> {A1,A2,A3} = erlang:now(), - random:seed(A1,A2,A3), + _ = random:seed(A1, A2, A3), random:uniform(Upper). size_random(C) -> diff --git a/lib/asn1/test/error_SUITE.erl b/lib/asn1/test/error_SUITE.erl index 6451f81c01..930b44cea6 100644 --- a/lib/asn1/test/error_SUITE.erl +++ b/lib/asn1/test/error_SUITE.erl @@ -19,7 +19,7 @@ -module(error_SUITE). -export([suite/0,all/0,groups/0, - already_defined/1,enumerated/1,objects/1]). + already_defined/1,enumerated/1,objects/1,values/1]). -include_lib("test_server/include/test_server.hrl"). @@ -29,9 +29,11 @@ all() -> [{group,p}]. groups() -> - [{p,parallel(),[already_defined, - enumerated, - objects]}]. + [{p,parallel(), + [already_defined, + enumerated, + objects, + values]}]. parallel() -> case erlang:system_info(schedulers) > 1 of @@ -138,6 +140,25 @@ objects(Config) -> } = run(P, Config), ok. +values(Config) -> + M = 'Values', + P = {M, + <<"Values DEFINITIONS AUTOMATIC TAGS ::= BEGIN\n" + " os1 OCTET STRING ::= \"abc\"\n" + " os2 OCTET STRING ::= 42\n" + " os3 OCTET STRING ::= { 1, 3 }\n" + "END\n">>}, + {error, + [ + {structured_error,{M,2},asn1ct_check, + illegal_octet_string_value}, + {structured_error,{M,3},asn1ct_check, + illegal_octet_string_value}, + {structured_error,{M,4},asn1ct_check, + illegal_octet_string_value} + ] + } = run(P, Config), + ok. run({Mod,Spec}, Config) -> diff --git a/lib/common_test/test/ct_telnet_SUITE.erl b/lib/common_test/test/ct_telnet_SUITE.erl index 25debf09d4..acce4eca14 100644 --- a/lib/common_test/test/ct_telnet_SUITE.erl +++ b/lib/common_test/test/ct_telnet_SUITE.erl @@ -215,23 +215,18 @@ all_cases(Suite,Config) -> fun({group,G}) -> {value,{G,Props,GTCs}} = lists:keysearch(G,1,Suite:groups()), - GTCs1 = case lists:member(parallel,Props) of - true -> - %%! TEMPORARY WORKAROUND FOR PROBLEM - %%! WITH ct_test_support NOT HANDLING - %%! VERIFICATION OF PARALLEL GROUPS - %%! CORRECTLY! - []; - false -> - [[{?eh,tc_start,{Suite,GTC}}, - {?eh,tc_done,{Suite,GTC,ok}}] || - GTC <- GTCs] - end, - [{?eh,tc_start,{Suite,{init_per_group,G,Props}}}, - {?eh,tc_done,{Suite,{init_per_group,G,Props},ok}} | - GTCs1] ++ - [{?eh,tc_start,{Suite,{end_per_group,G,Props}}}, - {?eh,tc_done,{Suite,{end_per_group,G,Props},ok}}]; + GTCs1 = [[{?eh,tc_start,{Suite,GTC}}, + {?eh,tc_done,{Suite,GTC,ok}}] || + GTC <- GTCs], + GEvs = [{?eh,tc_start,{Suite,{init_per_group,G,Props}}}, + {?eh,tc_done,{Suite,{init_per_group,G,Props},ok}} | + GTCs1] ++ + [{?eh,tc_start,{Suite,{end_per_group,G,Props}}}, + {?eh,tc_done,{Suite,{end_per_group,G,Props},ok}}], + case lists:member(parallel, Props) of + true -> [{parallel,GEvs}]; + false -> GEvs + end; (TC) -> [{?eh,tc_done,{Suite,TC,ok}}] end, GroupsAndTCs), diff --git a/lib/compiler/src/beam_a.erl b/lib/compiler/src/beam_a.erl index 3dfa67a771..fe4f473846 100644 --- a/lib/compiler/src/beam_a.erl +++ b/lib/compiler/src/beam_a.erl @@ -92,6 +92,8 @@ rename_instr({put_map_assoc,Fail,S,D,R,L}) -> {put_map,Fail,assoc,S,D,R,L}; rename_instr({put_map_exact,Fail,S,D,R,L}) -> {put_map,Fail,exact,S,D,R,L}; +rename_instr({test,has_map_fields,Fail,Src,{list,List}}) -> + {test,has_map_fields,Fail,[Src|List]}; rename_instr({select_val=I,Reg,Fail,{list,List}}) -> {select,I,Reg,Fail,List}; rename_instr({select_tuple_arity=I,Reg,Fail,{list,List}}) -> diff --git a/lib/compiler/src/beam_asm.erl b/lib/compiler/src/beam_asm.erl index 112b087f3c..f8cf178d2e 100644 --- a/lib/compiler/src/beam_asm.erl +++ b/lib/compiler/src/beam_asm.erl @@ -324,6 +324,8 @@ make_op({gc_bif,Bif,Fail,Live,Args,Dest}, Dict) -> encode_op(BifOp, [Fail,Live,{extfunc,erlang,Bif,Arity}|Args++[Dest]],Dict); make_op({bs_add=Op,Fail,[Src1,Src2,Unit],Dest}, Dict) -> encode_op(Op, [Fail,Src1,Src2,Unit,Dest], Dict); +make_op({test,Cond,Fail,Src,{list,_}=Ops}, Dict) -> + encode_op(Cond, [Fail,Src,Ops], Dict); make_op({test,Cond,Fail,Ops}, Dict) when is_list(Ops) -> encode_op(Cond, [Fail|Ops], Dict); make_op({test,Cond,Fail,Live,[Op|Ops],Dst}, Dict) when is_list(Ops) -> diff --git a/lib/compiler/src/beam_block.erl b/lib/compiler/src/beam_block.erl index 3723cc19e1..7a30c68593 100644 --- a/lib/compiler/src/beam_block.erl +++ b/lib/compiler/src/beam_block.erl @@ -154,8 +154,8 @@ collect({get_list,S,D1,D2}) -> {set,[D1,D2],[S],get_list}; collect(remove_message) -> {set,[],[],remove_message}; collect({put_map,F,Op,S,D,R,{list,Puts}}) -> {set,[D],[S|Puts],{alloc,R,{put_map,Op,F}}}; -collect({get_map_element,F,S,K,D}) -> - {set,[D],[S],{get_map_element,K,F}}; +collect({get_map_elements,F,S,{list,Gets}}) -> + {set,Gets,[S],{get_map_elements,F}}; collect({'catch',R,L}) -> {set,[R],[],{'catch',L}}; collect(fclearerror) -> {set,[],[],fclearerror}; collect({fcheckerror,{f,0}}) -> {set,[],[],fcheckerror}; @@ -240,7 +240,7 @@ move_allocates_2(Alloc, [], Acc) -> alloc_may_pass({set,_,_,{alloc,_,_}}) -> false; alloc_may_pass({set,_,_,{set_tuple_element,_}}) -> false; -alloc_may_pass({set,_,_,{get_map_element,_,_}}) -> false; +alloc_may_pass({set,_,_,{get_map_elements,_}}) -> false; alloc_may_pass({set,_,_,put_list}) -> false; alloc_may_pass({set,_,_,put}) -> false; alloc_may_pass({set,_,_,_}) -> true. @@ -291,7 +291,11 @@ opt_moves([X0,Y0], Is0) -> not_possible -> {[X,Y0],Is2}; {X,_} -> {[X,Y0],Is2}; {Y,Is} -> {[X,Y],Is} - end. + end; +opt_moves(Ds, Is) -> + %% multiple destinations -> pass through + {Ds,Is}. + %% opt_move(Dest, [Instruction]) -> {UpdatedDest,[Instruction]} | not_possible %% If there is a {move,Dest,FinalDest} instruction diff --git a/lib/compiler/src/beam_clean.erl b/lib/compiler/src/beam_clean.erl index 55f985ad0e..b653998252 100644 --- a/lib/compiler/src/beam_clean.erl +++ b/lib/compiler/src/beam_clean.erl @@ -262,8 +262,8 @@ replace([{bs_utf16_size=I,{f,Lbl},Src,Dst}|Is], Acc, D) when Lbl =/= 0 -> replace([{put_map=I,{f,Lbl},Op,Src,Dst,Live,List}|Is], Acc, D) when Lbl =/= 0 -> replace(Is, [{I,{f,label(Lbl, D)},Op,Src,Dst,Live,List}|Acc], D); -replace([{get_map_element=I,{f,Lbl},Src,Key,Dst}|Is], Acc, D) when Lbl =/= 0 -> - replace(Is, [{I,{f,label(Lbl, D)},Src,Key,Dst}|Acc], D); +replace([{get_map_elements=I,{f,Lbl},Src,List}|Is], Acc, D) when Lbl =/= 0 -> + replace(Is, [{I,{f,label(Lbl, D)},Src,List}|Acc], D); replace([I|Is], Acc, D) -> replace(Is, [I|Acc], D); replace([], Acc, _) -> Acc. diff --git a/lib/compiler/src/beam_disasm.erl b/lib/compiler/src/beam_disasm.erl index 57fdf95677..363989f4c9 100644 --- a/lib/compiler/src/beam_disasm.erl +++ b/lib/compiler/src/beam_disasm.erl @@ -366,9 +366,13 @@ disasm_instr(B, Bs, Atoms, Literals) -> select_tuple_arity -> disasm_select_inst(select_tuple_arity, Bs, Atoms, Literals); put_map_assoc -> - disasm_map_inst(put_map_assoc, Bs, Atoms, Literals); + disasm_map_inst(put_map_assoc, Arity, Bs, Atoms, Literals); put_map_exact -> - disasm_map_inst(put_map_exact, Bs, Atoms, Literals); + disasm_map_inst(put_map_exact, Arity, Bs, Atoms, Literals); + get_map_elements -> + disasm_map_inst(get_map_elements, Arity, Bs, Atoms, Literals); + has_map_fields -> + disasm_map_inst(has_map_fields, Arity, Bs, Atoms, Literals); _ -> try decode_n_args(Arity, Bs, Atoms, Literals) of {Args, RestBs} -> @@ -399,16 +403,15 @@ disasm_select_inst(Inst, Bs, Atoms, Literals) -> {List, RestBs} = decode_n_args(Len, Bs4, Atoms, Literals), {{Inst, [X,F,{Z,U,List}]}, RestBs}. -disasm_map_inst(Inst, Bs0, Atoms, Literals) -> - {F, Bs1} = decode_arg(Bs0, Atoms, Literals), - {S, Bs2} = decode_arg(Bs1, Atoms, Literals), - {X, Bs3} = decode_arg(Bs2, Atoms, Literals), - {N, Bs4} = decode_arg(Bs3, Atoms, Literals), - {Z, Bs5} = decode_arg(Bs4, Atoms, Literals), - {U, Bs6} = decode_arg(Bs5, Atoms, Literals), - {u, Len} = U, - {List, RestBs} = decode_n_args(Len, Bs6, Atoms, Literals), - {{Inst, [F,S,X,N,{Z,U,List}]}, RestBs}. +disasm_map_inst(Inst, Arity, Bs0, Atoms, Literals) -> + {Args0,Bs1} = decode_n_args(Arity, Bs0, Atoms, Literals), + %% no droplast .. + [Z|Args1] = lists:reverse(Args0), + Args = lists:reverse(Args1), + {U, Bs2} = decode_arg(Bs1, Atoms, Literals), + {u, Len} = U, + {List, RestBs} = decode_n_args(Len, Bs2, Atoms, Literals), + {{Inst, Args ++ [{Z,U,List}]}, RestBs}. %%----------------------------------------------------------------------- %% decode_arg([Byte]) -> {Arg, [Byte]} @@ -1150,13 +1153,15 @@ resolve_inst({is_map,Args0},_,_,_) -> [FLbl|Args] = resolve_args(Args0), {test, is_map, FLbl, Args}; -resolve_inst({has_map_field,Args0},_,_,_) -> - [FLbl|Args] = resolve_args(Args0), - {test,has_map_field,FLbl,Args}; +resolve_inst({has_map_fields,Args0},_,_,_) -> + [FLbl,Src,{{z,1},{u,_Len},List0}] = Args0, + List = resolve_args(List0), + {test,has_map_fields,FLbl,Src,{list,List}}; -resolve_inst({get_map_element,Args},_,_,_) -> - [FLbl,Src,Key,Dst] = resolve_args(Args), - {get_map_element,FLbl,Src,Key,Dst}; +resolve_inst({get_map_elements,Args0},_,_,_) -> + [FLbl,Src,{{z,1},{u,_Len},List0}] = Args0, + List = resolve_args(List0), + {get_map_elements,FLbl,Src,{list,List}}; %% %% Catches instructions that are not yet handled. diff --git a/lib/compiler/src/beam_flatten.erl b/lib/compiler/src/beam_flatten.erl index 534bc6d954..46835bece1 100644 --- a/lib/compiler/src/beam_flatten.erl +++ b/lib/compiler/src/beam_flatten.erl @@ -63,8 +63,8 @@ norm({set,[],[S,D],{set_tuple_element,I}}) -> {set_tuple_element,S,D,I}; norm({set,[D1,D2],[S],get_list}) -> {get_list,S,D1,D2}; norm({set,[D],[S|Puts],{alloc,R,{put_map,Op,F}}}) -> {put_map,F,Op,S,D,R,{list,Puts}}; -norm({set,[D],[S],{get_map_element,K,F}}) -> - {get_map_element,F,S,K,D}; +norm({set,Gets,[S],{get_map_elements,F}}) -> + {get_map_elements,F,S,{list,Gets}}; norm({set,[],[],remove_message}) -> remove_message; norm({set,[],[],fclearerror}) -> fclearerror; norm({set,[],[],fcheckerror}) -> {fcheckerror,{f,0}}. diff --git a/lib/compiler/src/beam_jump.erl b/lib/compiler/src/beam_jump.erl index 1f720b94c3..0fc8d45c80 100644 --- a/lib/compiler/src/beam_jump.erl +++ b/lib/compiler/src/beam_jump.erl @@ -529,7 +529,7 @@ ulbl({bs_put,Lbl,_,_}, Used) -> mark_used(Lbl, Used); ulbl({put_map,Lbl,_Op,_Src,_Dst,_Live,_List}, Used) -> mark_used(Lbl, Used); -ulbl({get_map_element,Lbl,_Src,_Key,_Dst}, Used) -> +ulbl({get_map_elements,Lbl,_Src,_List}, Used) -> mark_used(Lbl, Used); ulbl(_, Used) -> Used. diff --git a/lib/compiler/src/beam_split.erl b/lib/compiler/src/beam_split.erl index 638a4826ea..688bba9a94 100644 --- a/lib/compiler/src/beam_split.erl +++ b/lib/compiler/src/beam_split.erl @@ -53,9 +53,9 @@ split_block([{set,[D],[S|Puts],{alloc,R,{put_map,Op,{f,Lbl}=Fail}}}|Is], Bl, Acc) when Lbl =/= 0 -> split_block(Is, [], [{put_map,Fail,Op,S,D,R,{list,Puts}}| make_block(Bl, Acc)]); -split_block([{set,[D],[S],{get_map_element,K,{f,Lbl}=Fail}}|Is], Bl, Acc) +split_block([{set,Gets,[S],{get_map_elements,{f,Lbl}=Fail}}|Is], Bl, Acc) when Lbl =/= 0 -> - split_block(Is, [], [{get_map_element,Fail,S,K,D}|make_block(Bl, Acc)]); + split_block(Is, [], [{get_map_elements,Fail,S,{list,Gets}}|make_block(Bl, Acc)]); split_block([{set,[R],[],{'catch',L}}|Is], Bl, Acc) -> split_block(Is, [], [{'catch',R,L}|make_block(Bl, Acc)]); split_block([{set,[],[],{line,_}=Line}|Is], Bl, Acc) -> diff --git a/lib/compiler/src/beam_utils.erl b/lib/compiler/src/beam_utils.erl index a3f16cfa8f..27034aecce 100644 --- a/lib/compiler/src/beam_utils.erl +++ b/lib/compiler/src/beam_utils.erl @@ -185,6 +185,7 @@ is_pure_test({test,is_lt,_,[_,_]}) -> true; is_pure_test({test,is_nil,_,[_]}) -> true; is_pure_test({test,is_nonempty_list,_,[_]}) -> true; is_pure_test({test,test_arity,_,[_,_]}) -> true; +is_pure_test({test,has_map_fields,_,[_,{list,_}]}) -> true; is_pure_test({test,Op,_,Ops}) -> erl_internal:new_type_test(Op, length(Ops)). diff --git a/lib/compiler/src/beam_validator.erl b/lib/compiler/src/beam_validator.erl index 2486d486ed..7e1324cf61 100644 --- a/lib/compiler/src/beam_validator.erl +++ b/lib/compiler/src/beam_validator.erl @@ -770,6 +770,10 @@ valfun_4({test,is_nonempty_list,{f,Lbl},[Cons]}, Vst) -> valfun_4({test,test_arity,{f,Lbl},[Tuple,Sz]}, Vst) when is_integer(Sz) -> assert_type(tuple, Tuple, Vst), set_type_reg({tuple,Sz}, Tuple, branch_state(Lbl, Vst)); +valfun_4({test,has_map_fields,{f,Lbl},Src,{list,List}}, Vst) -> + validate_src([Src], Vst), + assert_strict_literal_termorder(List), + branch_state(Lbl, Vst); valfun_4({test,_Op,{f,Lbl},Src}, Vst) -> validate_src(Src, Vst), branch_state(Lbl, Vst); @@ -871,14 +875,23 @@ valfun_4({put_map_assoc,{f,Fail},Src,Dst,Live,{list,List}}, Vst) -> verify_put_map(Fail, Src, Dst, Live, List, Vst); valfun_4({put_map_exact,{f,Fail},Src,Dst,Live,{list,List}}, Vst) -> verify_put_map(Fail, Src, Dst, Live, List, Vst); -valfun_4({get_map_element,{f,Fail},Src,Key,Dst}, Vst0) -> - assert_term(Src, Vst0), - assert_term(Key, Vst0), - Vst = branch_state(Fail, Vst0), - set_type_reg(term, Dst, Vst); +valfun_4({get_map_elements,{f,Fail},Src,{list,List}}, Vst) -> + verify_get_map(Fail, Src, List, Vst); valfun_4(_, _) -> error(unknown_instruction). +verify_get_map(Fail, Src, List, Vst0) -> + assert_term(Src, Vst0), + Vst1 = branch_state(Fail, Vst0), + Lits = mmap(fun(L,_R) -> [L] end, List), + assert_strict_literal_termorder(Lits), + verify_get_map_pair(List,Vst0,Vst1). + +verify_get_map_pair([],_,Vst) -> Vst; +verify_get_map_pair([Src,Dst|Vs],Vst0,Vsti) -> + assert_term(Src, Vst0), + verify_get_map_pair(Vs,Vst0,set_type_reg(term,Dst,Vsti)). + verify_put_map(Fail, Src, Dst, Live, List, Vst0) -> verify_live(Live, Vst0), verify_y_init(Vst0), @@ -1099,6 +1112,39 @@ assert_freg_set({fr,Fr}=Freg, #vst{current=#st{f=Fregs}}) end; assert_freg_set(Fr, _) -> error({bad_source,Fr}). +%%% Maps + +%% ensure that a list of literals has a strict +%% ascending term order (also meaning unique literals) +assert_strict_literal_termorder(Ls) -> + Vs = lists:map(fun (L) -> get_literal(L) end, Ls), + case check_strict_value_termorder(Vs) of + true -> ok; + false -> error({not_strict_order, Ls}) + end. + +%% usage: +%% mmap(fun(A,B) -> [{A,B}] end, [1,2,3,4]), +%% [{1,2},{3,4}] + +mmap(F,List) -> + {arity,Ar} = erlang:fun_info(F,arity), + mmap(F,Ar,List). +mmap(_F,_,[]) -> []; +mmap(F,Ar,List) -> + {Hd,Tl} = lists:split(Ar,List), + apply(F,Hd) ++ mmap(F,Ar,Tl). + +check_strict_value_termorder([]) -> true; +check_strict_value_termorder([_]) -> true; +check_strict_value_termorder([V1,V2]) -> + erts_internal:cmp_term(V1,V2) < 0; +check_strict_value_termorder([V1,V2|Vs]) -> + case erts_internal:cmp_term(V1,V2) < 0 of + true -> check_strict_value_termorder([V2|Vs]); + false -> false + end. + %%% %%% Binary matching. %%% @@ -1334,6 +1380,7 @@ assert_term(Src, Vst) -> %% number Integer or Float of unknown value %% + assert_type(WantedType, Term, Vst) -> assert_type(WantedType, get_term_type(Term, Vst)). @@ -1349,7 +1396,6 @@ assert_type({tuple_element,I}, {tuple,Sz}) assert_type(Needed, Actual) -> error({bad_type,{needed,Needed},{actual,Actual}}). - %% upgrade_tuple_type(NewTupleType, OldType) -> TupleType. %% upgrade_tuple_type/2 is used when linear code finds out more and %% more information about a tuple type, so that the type gets more @@ -1430,6 +1476,15 @@ get_term_type_1({y,Y}=Reg, #vst{current=#st{y=Ys}}) when is_integer(Y) -> get_term_type_1(Src, _) -> error({bad_source,Src}). +%% get_literal(Src) -> literal_value(). +get_literal(nil) -> []; +get_literal({atom,A}) when is_atom(A) -> A; +get_literal({float,F}) when is_float(F) -> F; +get_literal({integer,I}) when is_integer(I) -> I; +get_literal({literal,L}) -> L; +get_literal(T) -> error({not_literal,T}). + + branch_arities([], _, #vst{}=Vst) -> Vst; branch_arities([Sz,{f,L}|T], Tuple, #vst{current=St}=Vst0) when is_integer(Sz) -> diff --git a/lib/compiler/src/beam_z.erl b/lib/compiler/src/beam_z.erl index 9953a48710..c2a6ef604e 100644 --- a/lib/compiler/src/beam_z.erl +++ b/lib/compiler/src/beam_z.erl @@ -78,6 +78,18 @@ undo_rename({put_map,Fail,assoc,S,D,R,L}) -> {put_map_assoc,Fail,S,D,R,L}; undo_rename({put_map,Fail,exact,S,D,R,L}) -> {put_map_exact,Fail,S,D,R,L}; +undo_rename({test,has_map_fields,Fail,[Src|List]}) -> + {test,has_map_fields,Fail,Src,{list,[to_typed_literal(V)||V<-List]}}; +undo_rename({get_map_elements,Fail,Src,{list, List}}) -> + {get_map_elements,Fail,Src,{list,[to_typed_literal(V)||V<-List]}}; undo_rename({select,I,Reg,Fail,List}) -> {I,Reg,Fail,{list,List}}; undo_rename(I) -> I. + +%% to_typed_literal(Arg) +%% transform Arg to specific literal i.e. float | integer | atom if applicable +to_typed_literal({literal, V}) when is_float(V) -> {float, V}; +to_typed_literal({literal, V}) when is_atom(V) -> {atom, V}; +to_typed_literal({literal, V}) when is_integer(V) -> {integer, V}; +to_typed_literal({literal, []}) -> nil; +to_typed_literal(V) -> V. diff --git a/lib/compiler/src/genop.tab b/lib/compiler/src/genop.tab index 79b467f949..7d6bf56ccb 100755 --- a/lib/compiler/src/genop.tab +++ b/lib/compiler/src/genop.tab @@ -529,10 +529,10 @@ BEAM_FORMAT_NUMBER=0 153: line/1 -# R16 +# R17 154: put_map_assoc/5 155: put_map_exact/5 156: is_map/2 -157: has_map_field/3 -158: get_map_element/4 +157: has_map_fields/3 +158: get_map_elements/3 diff --git a/lib/compiler/src/sys_core_fold.erl b/lib/compiler/src/sys_core_fold.erl index e2b9213891..eb9c302334 100644 --- a/lib/compiler/src/sys_core_fold.erl +++ b/lib/compiler/src/sys_core_fold.erl @@ -3146,9 +3146,6 @@ format_error(result_ignored) -> "(suppress the warning by assigning the expression to the _ variable)"; format_error(useless_building) -> "a term is constructed, but never used"; -format_error({map_pair_key_overloaded,K}) -> - M = io_lib:format("the key ~p is used multiple times in map value association",[K]), - flatten(M); format_error(bin_opt_alias) -> "INFO: the '=' operator will prevent delayed sub binary optimization"; format_error(bin_partition) -> diff --git a/lib/compiler/src/v3_codegen.erl b/lib/compiler/src/v3_codegen.erl index 4d155c0fd0..e00ee1f3ad 100644 --- a/lib/compiler/src/v3_codegen.erl +++ b/lib/compiler/src/v3_codegen.erl @@ -938,22 +938,39 @@ select_map_val(V, Es, B, Fail, I, Vdb, Bef, St0) -> {Bis,Aft,St2} = match_cg(B, Fail, Int, St1), {Eis++Bis,Aft,St2}. +select_extract_map(_, [], _, _, _, Bef, St) -> {[],Bef,St}; select_extract_map(Src, Vs, Fail, I, Vdb, Bef, St) -> - F = fun ({map_pair,Key,{var,V}}, Int0) -> - Rsrc = fetch_var(Src, Int0), + %% First split the instruction flow + %% We want one set of each + %% 1) has_map_fields (no target registers) + %% 2) get_map_elements (with target registers) + %% Assume keys are term-sorted + Rsrc = fetch_var(Src, Bef), + + {{HasKs,GetVs},Aft} = lists:foldr(fun + ({map_pair,Key,{var,V}},{{HasKsi,GetVsi},Int0}) -> case vdb_find(V, Vdb) of {V,_,L} when L =< I -> - {[{test,has_map_field,{f,Fail},[Rsrc,Key]}],Int0}; + {{[Key|HasKsi],GetVsi},Int0}; _Other -> Reg1 = put_reg(V, Int0#sr.reg), Int1 = Int0#sr{reg=Reg1}, - {[{get_map_element,{f,Fail}, - Rsrc,Key,fetch_reg(V, Reg1)}], - Int1} + {{HasKsi,[Key,fetch_reg(V, Reg1)|GetVsi]},Int1} end - end, - {Es,Aft} = flatmapfoldl(F, Bef, Vs), - {Es,Aft,St}. + end, {{[],[]},Bef}, Vs), + + Code = case {HasKs,GetVs} of + {[],[]} -> {[],Aft,St}; + {HasKs,[]} -> + [{test,has_map_fields,{f,Fail},Rsrc,{list,HasKs}}]; + {[],GetVs} -> + [{get_map_elements, {f,Fail},Rsrc,{list,GetVs}}]; + {HasKs,GetVs} -> + [{test,has_map_fields,{f,Fail},Rsrc,{list,HasKs}}, + {get_map_elements, {f,Fail},Rsrc,{list,GetVs}}] + end, + {Code, Aft, St}. + select_extract_cons(Src, [{var,Hd}, {var,Tl}], I, Vdb, Bef, St) -> {Es,Aft} = case {vdb_find(Hd, Vdb), vdb_find(Tl, Vdb)} of diff --git a/lib/compiler/src/v3_core.erl b/lib/compiler/src/v3_core.erl index ec5deb6905..a50b46bd7b 100644 --- a/lib/compiler/src/v3_core.erl +++ b/lib/compiler/src/v3_core.erl @@ -101,6 +101,8 @@ -record(ireceive2, {anno=#a{},clauses,timeout,action}). -record(iset, {anno=#a{},var,arg}). -record(itry, {anno=#a{},args,vars,body,evars,handler}). +-record(ifilter, {anno=#a{},arg}). +-record(igen, {anno=#a{},acc_pat,acc_guard,skip_pat,tail,tail_pat,arg}). -type iapply() :: #iapply{}. -type ibinary() :: #ibinary{}. @@ -117,10 +119,13 @@ -type ireceive2() :: #ireceive2{}. -type iset() :: #iset{}. -type itry() :: #itry{}. +-type ifilter() :: #ifilter{}. +-type igen() :: #igen{}. -type i() :: iapply() | ibinary() | icall() | icase() | icatch() | iclause() | ifun() | iletrec() | imatch() | iprimop() - | iprotect() | ireceive1() | ireceive2() | iset() | itry(). + | iprotect() | ireceive1() | ireceive2() | iset() | itry() + | ifilter() | igen(). -type warning() :: {file:filename(), [{integer(), module(), term()}]}. @@ -479,8 +484,9 @@ expr({cons,L,H0,T0}, St0) -> {T1,Tps,St2} = safe(T0, St1), A = lineno_anno(L, St2), {ann_c_cons(A, H1, T1),Hps ++ Tps,St2}; -expr({lc,L,E,Qs}, St) -> - lc_tq(L, E, Qs, #c_literal{anno=lineno_anno(L, St),val=[]}, St); +expr({lc,L,E,Qs0}, St0) -> + {Qs1,St1} = preprocess_quals(L, Qs0, St0), + lc_tq(L, E, Qs1, #c_literal{anno=lineno_anno(L, St1),val=[]}, St1); expr({bc,L,E,Qs}, St) -> bc_tq(L, E, Qs, {nil,L}, St); expr({tuple,L,Es0}, St0) -> @@ -647,7 +653,7 @@ expr({match,L,P0,E0}, St0) -> Other when not is_atom(Other) -> {#imatch{anno=#a{anno=Lanno},pat=P2,arg=E2,fc=Fc},Eps,St4} end; -expr({op,_,'++',{lc,Llc,E,Qs},More}, St0) -> +expr({op,_,'++',{lc,Llc,E,Qs0},More}, St0) -> %% Optimise '++' here because of the list comprehension algorithm. %% %% To avoid achieving quadratic complexity if there is a chain of @@ -655,7 +661,8 @@ expr({op,_,'++',{lc,Llc,E,Qs},More}, St0) -> %% evaluation of More now. Evaluating More here could also reduce the %% number variables in the environment for letrec. {Mc,Mps,St1} = safe(More, St0), - {Y,Yps,St} = lc_tq(Llc, E, Qs, Mc, St1), + {Qs,St2} = preprocess_quals(Llc, Qs0, St1), + {Y,Yps,St} = lc_tq(Llc, E, Qs, Mc, St2), {Y,Mps++Yps,St}; expr({op,L,'andalso',E1,E2}, St0) -> {#c_var{name=V0},St} = new_var(L, St0), @@ -889,136 +896,45 @@ fun_tq({_,_,Name}=Id, Cs0, L, St0, NameInfo) -> %% lc_tq(Line, Exp, [Qualifier], Mc, State) -> {LetRec,[PreExp],State}. %% This TQ from Simon PJ pp 127-138. -%% This gets a bit messy as we must transform all directly here. We -%% recognise guard tests and try to fold them together and join to a -%% preceding generators, this should give us better and more compact -%% code. -lc_tq(Line, E, [{generate,Lg,P,G}|Qs0], Mc, St0) -> - {Gs,Qs1} = splitwith(fun is_guard_test/1, Qs0), +lc_tq(Line, E, [#igen{anno=GAnno,acc_pat=AccPat,acc_guard=AccGuard, + skip_pat=SkipPat,tail=Tail,tail_pat=TailPat, + arg={Pre,Arg}}|Qs], Mc, St0) -> {Name,St1} = new_fun_name("lc", St0), - {Head,St2} = new_var(St1), - {Tname,St3} = new_var_name(St2), - LA = lineno_anno(Line, St3), - CGAnno = #a{anno=[list_comprehension|LA]}, - LAnno = #a{anno=LA}, - Tail = #c_var{anno=LA,name=Tname}, - {Arg,St4} = new_var(St3), - {Nc,[],St5} = expr({call,Lg,{atom,Lg,Name},[{var,Lg,Tname}]}, St4), - {Guardc,St6} = lc_guard_tests(Gs, St5), %These are always flat! - {Lc,Lps,St7} = lc_tq(Line, E, Qs1, Nc, St6), - {Pc,St8} = list_gen_pattern(P, Line, St7), - {Gc,Gps,St9} = safe(G, St8), %Will be a function argument! - Fc = function_clause([Arg], LA, {Name,1}), - - %% Avoid constructing a default clause if the list comprehension - %% only has a variable as generator and there are no guard - %% tests. In other words, if the comprehension is equivalent to - %% lists:map/2. - Cs0 = case {Guardc, Pc} of - {[], #c_var{}} -> - [#iclause{anno=LAnno, - pats=[#c_literal{anno=LA,val=[]}],guard=[], - body=[Mc]}]; - _ -> - [#iclause{anno=#a{anno=[compiler_generated|LA]}, - pats=[ann_c_cons(LA, Head, Tail)], - guard=[], - body=[Nc]}, - #iclause{anno=LAnno, - pats=[#c_literal{anno=LA,val=[]}],guard=[], - body=[Mc]}] - end, - Cs = case Pc of - nomatch -> Cs0; - _ -> - [#iclause{anno=LAnno, - pats=[ann_c_cons(LA, Pc, Tail)], - guard=Guardc, - body=Lps ++ [Lc]}|Cs0] - end, - Fun = #ifun{anno=LAnno,id=[],vars=[Arg],clauses=Cs,fc=Fc}, - {#iletrec{anno=CGAnno,defs=[{{Name,1},Fun}], - body=Gps ++ [#iapply{anno=LAnno, - op=#c_var{anno=LA,name={Name,1}}, - args=[Gc]}]}, - [],St9}; -lc_tq(Line, E, [{b_generate,Lg,P,G}|Qs0], Mc, St0) -> - {Gs,Qs1} = splitwith(fun is_guard_test/1, Qs0), - {Name,St1} = new_fun_name("blc", St0), LA = lineno_anno(Line, St1), LAnno = #a{anno=LA}, - CGAnno = #a{anno=[list_comprehension|LA]}, - HeadBinPattern = pattern(P, St1), - #c_binary{segments=Ps0} = HeadBinPattern, - {Ps,Tail,St2} = append_tail_segment(Ps0, St1), - {EPs,St3} = emasculate_segments(Ps, St2), - Pattern = HeadBinPattern#c_binary{segments=Ps}, - EPattern = HeadBinPattern#c_binary{segments=EPs}, - {Arg,St4} = new_var(St3), - {Guardc,St5} = lc_guard_tests(Gs, St4), %These are always flat! - Tname = Tail#c_var.name, - {Nc,[],St6} = expr({call,Lg,{atom,Lg,Name},[{var,Lg,Tname}]}, St5), - {Bc,Bps,St7} = lc_tq(Line, E, Qs1, Nc, St6), - {Gc,Gps,St10} = safe(G, St7), %Will be a function argument! - Fc = function_clause([Arg], LA, {Name,1}), - {TailSegList,_,St} = append_tail_segment([], St10), - Cs = [#iclause{anno=#a{anno=[compiler_generated|LA]}, - pats=[Pattern], - guard=Guardc, - body=Bps ++ [Bc]}, - #iclause{anno=#a{anno=[compiler_generated|LA]}, - pats=[EPattern], - guard=[], - body=[#iapply{anno=LAnno, - op=#c_var{anno=LA,name={Name,1}}, - args=[Tail]}]}, - #iclause{anno=LAnno, - pats=[#c_binary{anno=LA,segments=TailSegList}],guard=[], - body=[Mc]}], - Fun = #ifun{anno=LAnno,id=[],vars=[Arg],clauses=Cs,fc=Fc}, - {#iletrec{anno=CGAnno,defs=[{{Name,1},Fun}], - body=Gps ++ [#iapply{anno=LAnno, - op=#c_var{anno=LA,name={Name,1}}, - args=[Gc]}]}, - [],St}; -lc_tq(Line, E, [Fil0|Qs0], Mc, St0) -> - %% Special case sequences guard tests. - LA = lineno_anno(element(2, Fil0), St0), - LAnno = #a{anno=LA}, - CGAnno = #a{anno=[list_comprehension|LA]}, - case is_guard_test(Fil0) of - true -> - {Gs0,Qs1} = splitwith(fun is_guard_test/1, Qs0), - {Lc,Lps,St1} = lc_tq(Line, E, Qs1, Mc, St0), - {Gs,St2} = lc_guard_tests([Fil0|Gs0], St1), %These are always flat! - {#icase{anno=CGAnno, - args=[], - clauses=[#iclause{anno=LAnno,pats=[], - guard=Gs,body=Lps ++ [Lc]}], - fc=#iclause{anno=LAnno#a{anno=[compiler_generated|LA]}, - pats=[],guard=[],body=[Mc]}}, - [],St2}; - false -> - {Lc,Lps,St1} = lc_tq(Line, E, Qs0, Mc, St0), - {Fpat,St2} = new_var(St1), - Fc = fail_clause([Fpat], LA, - c_tuple([#c_literal{val=case_clause},Fpat])), - %% Do a novars little optimisation here. - {Filc,Fps,St3} = novars(Fil0, St2), - {#icase{anno=CGAnno, - args=[Filc], - clauses=[#iclause{anno=LAnno, - pats=[#c_literal{anno=LA,val=true}], - guard=[], - body=Lps ++ [Lc]}, - #iclause{anno=LAnno#a{anno=[compiler_generated|LA]}, - pats=[#c_literal{anno=LA,val=false}], - guard=[], - body=[Mc]}], - fc=Fc}, - Fps,St3} - end; + F = #c_var{anno=LA,name={Name,1}}, + Nc = #iapply{anno=GAnno,op=F,args=[Tail]}, + {Var,St2} = new_var(St1), + Fc = function_clause([Var], LA, {Name,1}), + TailClause = #iclause{anno=LAnno,pats=[TailPat],guard=[],body=[Mc]}, + Cs0 = case {AccPat,AccGuard} of + {SkipPat,[]} -> + %% Skip and accumulator patterns are the same and there is + %% no guard, no need to generate a skip clause. + [TailClause]; + _ -> + [#iclause{anno=#a{anno=[compiler_generated|LA]}, + pats=[SkipPat],guard=[],body=[Nc]}, + TailClause] + end, + {Cs,St4} = case AccPat of + nomatch -> + %% The accumulator pattern never matches, no need + %% for an accumulator clause. + {Cs0,St2}; + _ -> + {Lc,Lps,St3} = lc_tq(Line, E, Qs, Nc, St2), + {[#iclause{anno=LAnno,pats=[AccPat],guard=AccGuard, + body=Lps ++ [Lc]}|Cs0], + St3} + end, + Fun = #ifun{anno=LAnno,id=[],vars=[Var],clauses=Cs,fc=Fc}, + {#iletrec{anno=LAnno#a{anno=[list_comprehension|LA]},defs=[{{Name,1},Fun}], + body=Pre ++ [#iapply{anno=LAnno,op=F,args=[Arg]}]}, + [],St4}; +lc_tq(Line, E, [#ifilter{}=Filter|Qs], Mc, St) -> + filter_tq(Line, E, Filter, Mc, St, Qs, fun lc_tq/5); lc_tq(Line, E0, [], Mc0, St0) -> {H1,Hps,St1} = safe(E0, St0), {T1,Tps,St} = force_safe(Mc0, St1), @@ -1028,146 +944,60 @@ lc_tq(Line, E0, [], Mc0, St0) -> %% bc_tq(Line, Exp, [Qualifier], More, State) -> {LetRec,[PreExp],State}. %% This TQ from Gustafsson ERLANG'05. -%% This gets a bit messy as we must transform all directly here. We -%% recognise guard tests and try to fold them together and join to a -%% preceding generators, this should give us better and more compact -%% code. %% More could be transformed before calling bc_tq. -bc_tq(Line, Exp, Qualifiers, _, St0) -> +bc_tq(Line, Exp, Qs0, _, St0) -> {BinVar,St1} = new_var(St0), - {Sz,SzPre,St2} = bc_initial_size(Exp, Qualifiers, St1), - {E,BcPre,St} = bc_tq1(Line, Exp, Qualifiers, BinVar, St2), + {Sz,SzPre,St2} = bc_initial_size(Exp, Qs0, St1), + {Qs,St3} = preprocess_quals(Line, Qs0, St2), + {E,BcPre,St} = bc_tq1(Line, Exp, Qs, BinVar, St3), Pre = SzPre ++ [#iset{var=BinVar, arg=#iprimop{name=#c_literal{val=bs_init_writable}, args=[Sz]}}] ++ BcPre, {E,Pre,St}. -bc_tq1(Line, E, [{generate,Lg,P,G}|Qs0], AccExpr, St0) -> - {Gs,Qs1} = splitwith(fun is_guard_test/1, Qs0), - {Name,St1} = new_fun_name("lbc", St0), - LA = lineno_anno(Line, St1), - {[Head,Tail,AccVar],St2} = new_vars(LA, 3, St1), - LAnno = #a{anno=LA}, - CGAnno = #a{anno=[list_comprehension|LA]}, - {Arg,St3} = new_var(St2), - NewMore = {call,Lg,{atom,Lg,Name},[{var,Lg,Tail#c_var.name}, - {var,Lg,AccVar#c_var.name}]}, - {Guardc,St4} = lc_guard_tests(Gs, St3), %These are always flat! - {Lc,Lps,St5} = bc_tq1(Line, E, Qs1, AccVar, St4), - {Nc,Nps,St6} = expr(NewMore, St5), - {Pc,St7} = list_gen_pattern(P, Line, St6), - {Gc,Gps,St8} = safe(G, St7), %Will be a function argument! - Fc = function_clause([Arg,AccVar], LA, {Name,2}), - Cs0 = case {Guardc, Pc} of - {[], #c_var{}} -> - [#iclause{anno=LAnno, - pats=[#c_literal{anno=LA,val=[]},AccVar],guard=[], - body=[AccVar]}]; - _ -> - [#iclause{anno=#a{anno=[compiler_generated|LA]}, - pats=[ann_c_cons(LA, Head, Tail),AccVar], - guard=[], - body=Nps ++ [Nc]}, - #iclause{anno=LAnno, - pats=[#c_literal{anno=LA,val=[]},AccVar],guard=[], - body=[AccVar]}] - end, - Cs = case Pc of - nomatch -> Cs0; - _ -> - Body = Lps ++ Nps ++ [#iset{var=AccVar,arg=Lc},Nc], - [#iclause{anno=LAnno, - pats=[ann_c_cons(LA,Pc,Tail),AccVar], - guard=Guardc, - body=Body}|Cs0] - end, - Fun = #ifun{anno=LAnno,id=[],vars=[Arg,AccVar],clauses=Cs,fc=Fc}, - {#iletrec{anno=CGAnno,defs=[{{Name,2},Fun}], - body=Gps ++ [#iapply{anno=LAnno, - op=#c_var{anno=LA,name={Name,2}}, - args=[Gc,AccExpr]}]}, - [],St8}; -bc_tq1(Line, E, [{b_generate,Lg,P,G}|Qs0], AccExpr, St0) -> - {Gs,Qs1} = splitwith(fun is_guard_test/1, Qs0), +bc_tq1(Line, E, [#igen{anno=GAnno,acc_pat=AccPat,acc_guard=AccGuard, + skip_pat=SkipPat,tail=Tail,tail_pat=TailPat, + arg={Pre,Arg}}|Qs], Mc, St0) -> {Name,St1} = new_fun_name("lbc", St0), LA = lineno_anno(Line, St1), - {AccVar,St2} = new_var(LA, St1), LAnno = #a{anno=LA}, - CGAnno = #a{anno=[list_comprehension|LA]}, - HeadBinPattern = pattern(P, St2), - #c_binary{segments=Ps0} = HeadBinPattern, - {Ps,Tail,St3} = append_tail_segment(Ps0, St2), - {EPs,St4} = emasculate_segments(Ps, St3), - Pattern = HeadBinPattern#c_binary{segments=Ps}, - EPattern = HeadBinPattern#c_binary{segments=EPs}, - {Arg,St5} = new_var(St4), - NewMore = {call,Lg,{atom,Lg,Name},[{var,Lg,Tail#c_var.name}, - {var,Lg,AccVar#c_var.name}]}, - {Guardc,St6} = lc_guard_tests(Gs, St5), %These are always flat! - {Bc,Bps,St7} = bc_tq1(Line, E, Qs1, AccVar, St6), - {Nc,Nps,St8} = expr(NewMore, St7), - {Gc,Gps,St9} = safe(G, St8), %Will be a function argument! - Fc = function_clause([Arg,AccVar], LA, {Name,2}), - Body = Bps ++ Nps ++ [#iset{var=AccVar,arg=Bc},Nc], - {TailSegList,_,St} = append_tail_segment([], St9), - Cs = [#iclause{anno=LAnno, - pats=[Pattern,AccVar], - guard=Guardc, - body=Body}, - #iclause{anno=#a{anno=[compiler_generated|LA]}, - pats=[EPattern,AccVar], - guard=[], - body=Nps ++ [Nc]}, - #iclause{anno=LAnno, - pats=[#c_binary{anno=LA,segments=TailSegList},AccVar], - guard=[], - body=[AccVar]}], - Fun = #ifun{anno=CGAnno,id=[],vars=[Arg,AccVar],clauses=Cs,fc=Fc}, - {#iletrec{anno=LAnno,defs=[{{Name,2},Fun}], - body=Gps ++ [#iapply{anno=LAnno, - op=#c_var{anno=LA,name={Name,2}}, - args=[Gc,AccExpr]}]}, - [],St}; -bc_tq1(Line, E, [Fil0|Qs0], AccVar, St0) -> - %% Special case sequences guard tests. - LA = lineno_anno(element(2, Fil0), St0), - LAnno = #a{anno=LA}, - CGAnno = #a{anno=[list_comprehension|LA]}, - case is_guard_test(Fil0) of - true -> - {Gs0,Qs1} = splitwith(fun is_guard_test/1, Qs0), - {Bc,Bps,St1} = bc_tq1(Line, E, Qs1, AccVar, St0), - {Gs,St} = lc_guard_tests([Fil0|Gs0], St1), %These are always flat! - {#icase{anno=CGAnno, - args=[], - clauses=[#iclause{anno=LAnno, - pats=[], - guard=Gs,body=Bps ++ [Bc]}], - fc=#iclause{anno=LAnno#a{anno=[compiler_generated|LA]}, - pats=[],guard=[],body=[AccVar]}}, - [],St}; - false -> - {Bc,Bps,St1} = bc_tq1(Line, E, Qs0, AccVar, St0), - {Fpat,St2} = new_var(St1), - Fc = fail_clause([Fpat], LA, - c_tuple([#c_literal{val=case_clause},Fpat])), - %% Do a novars little optimisation here. - {Filc,Fps,St} = novars(Fil0, St2), - {#icase{anno=CGAnno, - args=[Filc], - clauses=[#iclause{anno=LAnno, - pats=[#c_literal{anno=LA,val=true}], - guard=[], - body=Bps ++ [Bc]}, - #iclause{anno=LAnno#a{anno=[compiler_generated|LA]}, - pats=[#c_literal{anno=LA,val=false}], - guard=[], - body=[AccVar]}], - fc=Fc}, - Fps,St} - end; + {Vars=[_,AccVar],St2} = new_vars(LA, 2, St1), + F = #c_var{anno=LA,name={Name,2}}, + Nc = #iapply{anno=GAnno,op=F,args=[Tail,AccVar]}, + Fc = function_clause(Vars, LA, {Name,2}), + TailClause = #iclause{anno=LAnno,pats=[TailPat,AccVar],guard=[], + body=[AccVar]}, + Cs0 = case {AccPat,AccGuard} of + {SkipPat,[]} -> + %% Skip and accumulator patterns are the same and there is + %% no guard, no need to generate a skip clause. + [TailClause]; + _ -> + [#iclause{anno=#a{anno=[compiler_generated|LA]}, + pats=[SkipPat,AccVar],guard=[],body=[Nc]}, + TailClause] + end, + {Cs,St4} = case AccPat of + nomatch -> + %% The accumulator pattern never matches, no need + %% for an accumulator clause. + {Cs0,St2}; + _ -> + {Bc,Bps,St3} = bc_tq1(Line, E, Qs, AccVar, St2), + Body = Bps ++ [#iset{var=AccVar,arg=Bc},Nc], + {[#iclause{anno=LAnno, + pats=[AccPat,AccVar],guard=AccGuard, + body=Body}|Cs0], + St3} + end, + Fun = #ifun{anno=LAnno,id=[],vars=Vars,clauses=Cs,fc=Fc}, + {#iletrec{anno=LAnno#a{anno=[list_comprehension|LA]},defs=[{{Name,2},Fun}], + body=Pre ++ [#iapply{anno=LAnno,op=F,args=[Arg,Mc]}]}, + [],St4}; +bc_tq1(Line, E, [#ifilter{}=Filter|Qs], Mc, St) -> + filter_tq(Line, E, Filter, Mc, St, Qs, fun bc_tq1/5); bc_tq1(_, {bin,Bl,Elements}, [], AccVar, St0) -> {E,Pre,St} = expr({bin,Bl,[{bin_element,Bl, {var,Bl,AccVar#c_var.name}, @@ -1175,16 +1005,154 @@ bc_tq1(_, {bin,Bl,Elements}, [], AccVar, St0) -> [binary,{unit,1}]}|Elements]}, St0), #a{anno=A} = Anno0 = get_anno(E), Anno = Anno0#a{anno=[compiler_generated,single_use|A]}, - %%Anno = Anno0#a{anno=[compiler_generated|A]}, {set_anno(E, Anno),Pre,St}. +%% filter_tq(Line, Expr, Filter, Mc, State, [Qualifier], TqFun) -> +%% {Case,[PreExpr],State}. +%% Transform an intermediate comprehension filter to its intermediate case +%% representation. + +filter_tq(Line, E, #ifilter{anno=#a{anno=LA}=LAnno,arg={Pre,Arg}}, + Mc, St0, Qs, TqFun) -> + %% The filter is an expression, it is compiled to a case of degree 1 with + %% 3 clauses, one accumulating, one skipping and the final one throwing + %% {case_clause,Value} where Value is the result of the filter and is not a + %% boolean. + {Lc,Lps,St1} = TqFun(Line, E, Qs, Mc, St0), + {FailPat,St2} = new_var(St1), + Fc = fail_clause([FailPat], LA, + c_tuple([#c_literal{val=case_clause},FailPat])), + {#icase{anno=LAnno#a{anno=[list_comprehension|LA]},args=[Arg], + clauses=[#iclause{anno=LAnno, + pats=[#c_literal{val=true}],guard=[], + body=Lps ++ [Lc]}, + #iclause{anno=LAnno#a{anno=[compiler_generated|LA]}, + pats=[#c_literal{val=false}],guard=[], + body=[Mc]}], + fc=Fc}, + Pre,St2}; +filter_tq(Line, E, #ifilter{anno=#a{anno=LA}=LAnno,arg=Guard}, + Mc, St0, Qs, TqFun) when is_list(Guard) -> + %% Otherwise it is a guard, compiled to a case of degree 0 with 2 clauses, + %% the first matches if the guard succeeds and the comprehension continues + %% or the second one is selected and the current element is skipped. + {Lc,Lps,St1} = TqFun(Line, E, Qs, Mc, St0), + {#icase{anno=LAnno#a{anno=[list_comprehension|LA]},args=[], + clauses=[#iclause{anno=LAnno,pats=[],guard=Guard,body=Lps ++ [Lc]}], + fc=#iclause{anno=LAnno#a{anno=[compiler_generated|LA]}, + pats=[],guard=[],body=[Mc]}}, + [],St1}. + +%% preprocess_quals(Line, [Qualifier], State) -> {[Qualifier'],State}. +%% Preprocess a list of Erlang qualifiers into its intermediate representation, +%% represented as a list of #igen{} and #ifilter{} records. We recognise guard +%% tests and try to fold them together and join to a preceding generators, this +%% should give us better and more compact code. + +preprocess_quals(Line, Qs, St) -> + preprocess_quals(Line, Qs, St, []). + +preprocess_quals(Line, [Q|Qs0], St0, Acc) -> + case is_generator(Q) of + true -> + {Gs,Qs} = splitwith(fun is_guard_test/1, Qs0), + {Gen,St} = generator(Line, Q, Gs, St0), + preprocess_quals(Line, Qs, St, [Gen|Acc]); + false -> + LAnno = #a{anno=lineno_anno(get_anno(Q), St0)}, + case is_guard_test(Q) of + true -> + %% When a filter is a guard test, its argument in the + %% #ifilter{} record is a list as returned by + %% lc_guard_tests/2. + {Gs,Qs} = splitwith(fun is_guard_test/1, Qs0), + {Cg,St} = lc_guard_tests([Q|Gs], St0), + Filter = #ifilter{anno=LAnno,arg=Cg}, + preprocess_quals(Line, Qs, St, [Filter|Acc]); + false -> + %% Otherwise, it is a pair {Pre,Arg} as in a generator + %% input. + {Ce,Pre,St} = novars(Q, St0), + Filter = #ifilter{anno=LAnno,arg={Pre,Ce}}, + preprocess_quals(Line, Qs0, St, [Filter|Acc]) + end + end; +preprocess_quals(_, [], St, Acc) -> + {reverse(Acc),St}. + +is_generator({generate,_,_,_}) -> true; +is_generator({b_generate,_,_,_}) -> true; +is_generator(_) -> false. + +%% +%% Generators are abstracted as sextuplets: +%% - acc_pat is the accumulator pattern, e.g. [Pat|Tail] for Pat <- Expr. +%% - acc_guard is the list of guards immediately following the current +%% generator in the qualifier list input. +%% - skip_pat is the skip pattern, e.g. <<X,_:X,Tail/bitstring>> for +%% <<X,1:X>> <= Expr. +%% - tail is the variable used in AccPat and SkipPat bound to the rest of the +%% generator input. +%% - tail_pat is the tail pattern, respectively [] and <<_/bitstring>> for list +%% and bit string generators. +%% - arg is a pair {Pre,Arg} where Pre is the list of expressions to be +%% inserted before the comprehension function and Arg is the expression +%% that it should be passed. +%% + +%% generator(Line, Generator, Guard, State) -> {Generator',State}. +%% Transform a given generator into its #igen{} representation. + +generator(Line, {generate,Lg,P0,E}, Gs, St0) -> + LA = lineno_anno(Line, St0), + GA = lineno_anno(Lg, St0), + {Head,St1} = list_gen_pattern(P0, Line, St0), + {[Tail,Skip],St2} = new_vars(2, St1), + {Cg,St3} = lc_guard_tests(Gs, St2), + {AccPat,SkipPat} = case Head of + #c_var{} -> + %% If the generator pattern is a variable, the + %% pattern from the accumulator clause can be + %% reused in the skip one. lc_tq and bc_tq1 takes + %% care of dismissing the latter in that case. + Cons = ann_c_cons(LA, Head, Tail), + {Cons,Cons}; + nomatch -> + %% If it never matches, there is no need for + %% an accumulator clause. + {nomatch,ann_c_cons(LA, Skip, Tail)}; + _ -> + {ann_c_cons(LA, Head, Tail), + ann_c_cons(LA, Skip, Tail)} + end, + {Ce,Pre,St4} = safe(E, St3), + Gen = #igen{anno=#a{anno=GA},acc_pat=AccPat,acc_guard=Cg,skip_pat=SkipPat, + tail=Tail,tail_pat=#c_literal{anno=LA,val=[]},arg={Pre,Ce}}, + {Gen,St4}; +generator(Line, {b_generate,Lg,P,E}, Gs, St0) -> + LA = lineno_anno(Line, St0), + GA = lineno_anno(Lg, St0), + Cp = #c_binary{segments=Segs} = pattern(P, St0), + %% The function append_tail_segment/2 keeps variable patterns as-is, making + %% it possible to have the same skip clause removal as with list generators. + {AccSegs,Tail,TailSeg,St1} = append_tail_segment(Segs, St0), + AccPat = Cp#c_binary{segments=AccSegs}, + {Cg,St2} = lc_guard_tests(Gs, St1), + {SkipSegs,St3} = emasculate_segments(AccSegs, St2), + SkipPat = Cp#c_binary{segments=SkipSegs}, + {Ce,Pre,St4} = safe(E, St3), + Gen = #igen{anno=#a{anno=GA},acc_pat=AccPat,acc_guard=Cg,skip_pat=SkipPat, + tail=Tail,tail_pat=#c_binary{anno=LA,segments=[TailSeg]}, + arg={Pre,Ce}}, + {Gen,St4}. + append_tail_segment(Segs, St0) -> {Var,St} = new_var(St0), Tail = #c_bitstr{val=Var,size=#c_literal{val=all}, unit=#c_literal{val=1}, type=#c_literal{val=binary}, flags=#c_literal{val=[unsigned,big]}}, - {Segs++[Tail],Var,St}. + {Segs++[Tail],Var,Tail,St}. emasculate_segments(Segs, St) -> emasculate_segments(Segs, St, []). @@ -1195,7 +1163,7 @@ emasculate_segments([B|Rest], St0, Acc) -> {Var,St1} = new_var(St0), emasculate_segments(Rest, St1, [B#c_bitstr{val=Var}|Acc]); emasculate_segments([], St, Acc) -> - {lists:reverse(Acc),St}. + {reverse(Acc),St}. lc_guard_tests([], St) -> {[],St}; lc_guard_tests(Gs0, St0) -> @@ -1511,8 +1479,50 @@ pattern({cons,L,H,T}, St) -> pattern({tuple,L,Ps}, St) -> ann_c_tuple(lineno_anno(L, St), pattern_list(Ps, St)); pattern({map,L,Ps}, St) -> - #c_map{anno=lineno_anno(L, St), es=sort(pattern_list(Ps, St))}; -pattern({map_field_exact,L,K,V}, St) -> + #c_map{anno=lineno_anno(L, St), es=pattern_map_pairs(Ps, St)}; +pattern({bin,L,Ps}, St) -> + %% We don't create a #ibinary record here, since there is + %% no need to hold any used/new annotations in a pattern. + #c_binary{anno=lineno_anno(L, St),segments=pat_bin(Ps, St)}; +pattern({match,_,P1,P2}, St) -> + pat_alias(pattern(P1, St), pattern(P2, St)). + +%% pattern_map_pairs([MapFieldExact],State) -> [#c_map_pairs{}] +pattern_map_pairs(Ps, St) -> + %% check literal key uniqueness (dict is needed) + %% pattern all pairs + {CMapPairs, Kdb} = lists:mapfoldl(fun + (P,Kdbi) -> + #c_map_pair{key=Ck,val=Cv} = CMapPair = pattern_map_pair(P,St), + K = core_lib:literal_value(Ck), + case dict:find(K,Kdbi) of + {ok, Vs} -> + {CMapPair, dict:store(K,[Cv|Vs],Kdbi)}; + _ -> + {CMapPair, dict:store(K,[Cv],Kdbi)} + end + end, dict:new(), Ps), + pattern_alias_map_pairs(CMapPairs,Kdb,dict:new(),St). + +pattern_alias_map_pairs([],_,_,_) -> []; +pattern_alias_map_pairs([#c_map_pair{key=Ck}=Pair|Pairs],Kdb,Kset,St) -> + %% alias same keys if needed + K = core_lib:literal_value(Ck), + case dict:find(K,Kset) of + {ok,processed} -> + pattern_alias_map_pairs(Pairs,Kdb,Kset,St); + _ -> + Cvs = dict:fetch(K,Kdb), + Cv = pattern_alias_map_pair_patterns(Cvs), + Kset1 = dict:store(K, processed, Kset), + [Pair#c_map_pair{val=Cv}|pattern_alias_map_pairs(Pairs,Kdb,Kset1,St)] + end. + +pattern_alias_map_pair_patterns([Cv]) -> Cv; +pattern_alias_map_pair_patterns([Cv1,Cv2|Cvs]) -> + pattern_alias_map_pair_patterns([pat_alias(Cv1,Cv2)|Cvs]). + +pattern_map_pair({map_field_exact,L,K,V}, St) -> %% FIXME: Better way to construct literals? or missing case %% {Key,_,_} = expr(K, St), Key = case K of @@ -1529,13 +1539,8 @@ pattern({map_field_exact,L,K,V}, St) -> #c_map_pair{anno=lineno_anno(L, St), op=#c_literal{val=exact}, key=Key, - val=pattern(V, St)}; -pattern({bin,L,Ps}, St) -> - %% We don't create a #ibinary record here, since there is - %% no need to hold any used/new annotations in a pattern. - #c_binary{anno=lineno_anno(L, St),segments=pat_bin(Ps, St)}; -pattern({match,_,P1,P2}, St) -> - pat_alias(pattern(P1, St), pattern(P2, St)). + val=pattern(V, St)}. + %% pat_bin([BinElement], State) -> [BinSeg]. diff --git a/lib/compiler/src/v3_kernel.erl b/lib/compiler/src/v3_kernel.erl index bc5ca0314a..5675572092 100644 --- a/lib/compiler/src/v3_kernel.erl +++ b/lib/compiler/src/v3_kernel.erl @@ -496,7 +496,6 @@ translate_match_fail_1(Anno, As, Sub, #kern{ff=FF}) -> translate_fc(Args) -> [#c_literal{val=function_clause},make_list(Args)]. - %{Kes,Ep,St2} = map_pairs(Ces, Sub, St1), map_split_pairs(A, Var, Ces, Sub, St0) -> %% two steps %% 1. force variables @@ -718,12 +717,8 @@ pattern(#c_tuple{anno=A,es=Ces}, Isub, Osub0, St0) -> {Kes,Osub1,St1} = pattern_list(Ces, Isub, Osub0, St0), {#k_tuple{anno=A,es=Kes},Osub1,St1}; pattern(#c_map{anno=A,es=Ces}, Isub, Osub0, St0) -> - {Kes,Osub1,St1} = pattern_list(Ces, Isub, Osub0, St0), + {Kes,Osub1,St1} = pattern_map_pairs(Ces, Isub, Osub0, St0), {#k_map{anno=A,op=exact,es=Kes},Osub1,St1}; -pattern(#c_map_pair{op=#c_literal{val=exact},anno=A,key=Ck,val=Cv},Isub, Osub0, St0) -> - {Kk,Osub1,St1} = pattern(Ck, Isub, Osub0, St0), - {Kv,Osub2,St2} = pattern(Cv, Isub, Osub1, St1), - {#k_map_pair{anno=A,key=Kk,val=Kv},Osub2,St2}; pattern(#c_binary{anno=A,segments=Cv}, Isub, Osub0, St0) -> {Kv,Osub1,St1} = pattern_bin(Cv, Isub, Osub0, St0), {#k_binary{anno=A,segs=Kv},Osub1,St1}; @@ -738,6 +733,25 @@ flatten_alias(#c_alias{var=V,pat=P}) -> {[V|Vs],Pat}; flatten_alias(Pat) -> {[],Pat}. +pattern_map_pairs(Ces0, Isub, Osub0, St0) -> + %% It is assumed that all core keys are literals + %% It is later assumed that these keys are term sorted + %% so we need to sort them here + Ces1 = lists:sort(fun + (#c_map_pair{key=CkA},#c_map_pair{key=CkB}) -> + A = core_lib:literal_value(CkA), + B = core_lib:literal_value(CkB), + erts_internal:cmp_term(A,B) < 0 + end, Ces0), + %% pattern the pair keys and values as normal + {Kes,{Osub1,St1}} = lists:mapfoldl(fun + (#c_map_pair{anno=A,key=Ck,val=Cv},{Osubi0,Sti0}) -> + {Kk,Osubi1,Sti1} = pattern(Ck, Isub, Osubi0, Sti0), + {Kv,Osubi2,Sti2} = pattern(Cv, Isub, Osubi1, Sti1), + {#k_map_pair{anno=A,key=Kk,val=Kv},{Osubi2,Sti2}} + end, {Osub0, St0}, Ces1), + {Kes,Osub1,St1}. + pattern_bin(Es, Isub, Osub0, St0) -> {Kbin,{_,Osub},St} = pattern_bin_1(Es, Isub, Osub0, St0), {Kbin,Osub,St}. @@ -1388,13 +1402,13 @@ get_match(#k_bin_int{}=BinInt, St0) -> get_match(#k_tuple{es=Es}, St0) -> {Mes,St1} = new_vars(length(Es), St0), {#k_tuple{es=Mes},Mes,St1}; -get_match(#k_map{es=Es0}, St0) -> +get_match(#k_map{op=exact,es=Es0}, St0) -> {Mes,St1} = new_vars(length(Es0), St0), {Es,_} = mapfoldl(fun (#k_map_pair{}=Pair, [V|Vs]) -> {Pair#k_map_pair{val=V},Vs} end, Mes, Es0), - {#k_map{es=Es},Mes,St1}; + {#k_map{op=exact,es=Es},Mes,St1}; get_match(M, St) -> {M,[],St}. @@ -1519,7 +1533,7 @@ arg_val(Arg, C) -> end || Pair <- Es], %% multiple keys may have the same name %% do not use ordsets - lists:sort(Keys) + lists:sort(fun(A,B) -> erts_internal:cmp_term(A,B) < 0 end, Keys) end. %% ubody_used_vars(Expr, State) -> [UsedVar] diff --git a/lib/compiler/src/v3_kernel_pp.erl b/lib/compiler/src/v3_kernel_pp.erl index 639c6737e2..b4e486f97c 100644 --- a/lib/compiler/src/v3_kernel_pp.erl +++ b/lib/compiler/src/v3_kernel_pp.erl @@ -115,7 +115,7 @@ format_1(#k_map{op=assoc,es=Es}, Ctxt) -> format_hseq(Es, ",", ctxt_bump_indent(Ctxt, 1), fun format/2), "}~" ]; -format_1(#k_map{op=exact,es=Es}, Ctxt) -> +format_1(#k_map{es=Es}, Ctxt) -> ["::{", format_hseq(Es, ",", ctxt_bump_indent(Ctxt, 1), fun format/2), "}::" diff --git a/lib/compiler/test/error_SUITE.erl b/lib/compiler/test/error_SUITE.erl index 859c4571ea..5cdf429a5f 100644 --- a/lib/compiler/test/error_SUITE.erl +++ b/lib/compiler/test/error_SUITE.erl @@ -23,7 +23,7 @@ -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, init_per_group/2,end_per_group/2, head_mismatch_line/1,warnings_as_errors/1, bif_clashes/1, - transforms/1]). + transforms/1,forbidden_maps/1]). %% Used by transforms/1 test case. -export([parse_transform/2]). @@ -36,7 +36,7 @@ all() -> groups() -> [{p,test_lib:parallel(), - [head_mismatch_line,warnings_as_errors,bif_clashes,transforms]}]. + [head_mismatch_line,warnings_as_errors,bif_clashes,transforms,forbidden_maps]}]. init_per_suite(Config) -> Config. @@ -240,6 +240,21 @@ parse_transform(_, _) -> error(too_bad). +forbidden_maps(Config) when is_list(Config) -> + Ts1 = [{map_illegal_use_of_pattern, + <<" + -export([t/0]). + t() -> + V = 32, + #{<<\"hi\",V,\"all\">> := 1} = id(#{<<\"hi all\">> => 1}). + id(I) -> I. + ">>, + [return], + {error,[{5,erl_lint,{illegal_map_key_variable,'V'}}], []}}], + [] = run2(Config, Ts1), + ok. + + run(Config, Tests) -> ?line File = test_filename(Config), run(Tests, File, dont_write_beam). diff --git a/lib/compiler/test/map_SUITE.erl b/lib/compiler/test/map_SUITE.erl index 86f65c3ed6..7d35ffc8d5 100644 --- a/lib/compiler/test/map_SUITE.erl +++ b/lib/compiler/test/map_SUITE.erl @@ -30,6 +30,7 @@ t_list_comprehension/1, t_map_sort_literals/1, t_map_size/1, + t_build_and_match_aliasing/1, %% warnings t_warn_useless_build/1, @@ -54,6 +55,8 @@ all() -> [ t_guard_bifs, t_guard_sequence, t_guard_update, t_guard_receive,t_guard_fun, t_list_comprehension, t_map_sort_literals, + t_map_size, + t_build_and_match_aliasing, %% warnings t_warn_useless_build, @@ -103,8 +106,6 @@ t_build_and_match_literals(Config) when is_list(Config) -> id(#{ map_1=>#{ map_2=>#{value_3 => third}, value_2=> second}, value_1=>first}), %% error case - %V = 32, - %{'EXIT',{{badmatch,_},_}} = (catch (#{<<"hi all">> => 1} = id(#{<<"hi",V,"all">> => 1}))), {'EXIT',{{badmatch,_},_}} = (catch (#{x:=3,x:=2} = id(#{x=>3}))), {'EXIT',{{badmatch,_},_}} = (catch (#{x:=2} = id(#{x=>3}))), {'EXIT',{{badmatch,_},_}} = (catch (#{x:=3} = id({a,b,c}))), @@ -112,6 +113,20 @@ t_build_and_match_literals(Config) when is_list(Config) -> {'EXIT',{{badmatch,_},_}} = (catch (#{x:=3} = id(#{x=>"three"}))), ok. +t_build_and_match_aliasing(Config) when is_list(Config) -> + M1 = id(#{a=>1,b=>2,c=>3,d=>4}), + #{c:=C1=_=_=C2} = M1, + true = C1 =:= C2, + #{a:=A,a:=A,a:=A,b:=B,b:=B} = M1, + #{a:=A,a:=A,a:=A,b:=B,b:=B,b:=2} = M1, + #{a:=A=1,a:=A,a:=A,b:=B=2,b:=B,b:=2} = M1, + #{c:=C1, c:=_, c:=3, c:=_, c:=C2} = M1, + #{c:=C=_=3=_=C} = M1, + + M2 = id(#{"a"=>1,"b"=>2,"c"=>3,"d"=>4}), + #{"a":=A2,"a":=A2,"a":=A2,"b":=B2,"b":=B2,"b":=2} = M2, + #{"a":=_,"a":=_,"a":=_,"b":=_,"b":=_,"b":=2} = M2, + ok. t_map_size(Config) when is_list(Config) -> 0 = map_size(id(#{})), diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c index 925ad0c091..2e1a21178f 100644 --- a/lib/crypto/c_src/crypto.c +++ b/lib/crypto/c_src/crypto.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2010-2013. All Rights Reserved. + * Copyright Ericsson AB 2010-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -444,6 +444,15 @@ static ERL_NIF_TERM atom_ppbasis; static ERL_NIF_TERM atom_onbasis; #endif +static ErlNifResourceType* hmac_context_rtype; +struct hmac_context +{ + ErlNifMutex* mtx; + int alive; + HMAC_CTX ctx; +}; +static void hmac_context_dtor(ErlNifEnv* env, struct hmac_context*); + /* #define PRINTF_ERR0(FMT) enif_fprintf(stderr, FMT "\n") #define PRINTF_ERR1(FMT, A1) enif_fprintf(stderr, FMT "\n", A1) @@ -498,6 +507,15 @@ static int init(ErlNifEnv* env, ERL_NIF_TERM load_info) return 0; } + hmac_context_rtype = enif_open_resource_type(env, NULL, "hmac_context", + (ErlNifResourceDtor*) hmac_context_dtor, + ERL_NIF_RT_CREATE|ERL_NIF_RT_TAKEOVER, + NULL); + if (!hmac_context_rtype) { + PRINTF_ERR0("CRYPTO: Could not open resource type 'hmac_context'"); + return 0; + } + if (library_refc > 0) { /* Repeated loading of this library (module upgrade). * Atoms and callbacks are already set, we are done. @@ -1280,11 +1298,19 @@ static ERL_NIF_TERM sha512_mac_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM #endif } +static void hmac_context_dtor(ErlNifEnv* env, struct hmac_context *obj) +{ + if (obj->alive) { + HMAC_CTX_cleanup(&obj->ctx); + obj->alive = 0; + } + enif_mutex_destroy(obj->mtx); +} + static ERL_NIF_TERM hmac_init(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (Type, Key) */ ErlNifBinary key; - ERL_NIF_TERM ret; - unsigned char * ctx_buf; + struct hmac_context* obj; const EVP_MD *md; if (argv[0] == atom_sha) md = EVP_sha1(); @@ -1309,57 +1335,60 @@ static ERL_NIF_TERM hmac_init(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[ return enif_make_badarg(env); } - ctx_buf = enif_make_new_binary(env, sizeof(HMAC_CTX), &ret); - HMAC_CTX_init((HMAC_CTX *) ctx_buf); - HMAC_Init((HMAC_CTX *) ctx_buf, key.data, key.size, md); + obj = enif_alloc_resource(hmac_context_rtype, sizeof(struct hmac_context)); + obj->mtx = enif_mutex_create("crypto.hmac"); + obj->alive = 1; + HMAC_CTX_init(&obj->ctx); + HMAC_Init(&obj->ctx, key.data, key.size, md); - return ret; + return enif_make_resource(env, obj); } static ERL_NIF_TERM hmac_update(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (Context, Data) */ - ErlNifBinary context, data; - ERL_NIF_TERM ret; - unsigned char * ctx_buf; + ErlNifBinary data; + struct hmac_context* obj; - if (!enif_inspect_binary(env, argv[0], &context) - || !enif_inspect_iolist_as_binary(env, argv[1], &data) - || context.size != sizeof(HMAC_CTX)) { + if (!enif_get_resource(env, argv[0], hmac_context_rtype, (void**)&obj) + || !enif_inspect_iolist_as_binary(env, argv[1], &data)) { + return enif_make_badarg(env); + } + enif_mutex_lock(obj->mtx); + if (!obj->alive) { + enif_mutex_unlock(obj->mtx); return enif_make_badarg(env); } + HMAC_Update(&obj->ctx, data.data, data.size); + enif_mutex_unlock(obj->mtx); - ctx_buf = enif_make_new_binary(env, sizeof(HMAC_CTX), &ret); - memcpy(ctx_buf, context.data, context.size); - HMAC_Update((HMAC_CTX *)ctx_buf, data.data, data.size); CONSUME_REDS(env,data); - - return ret; + return argv[0]; } static ERL_NIF_TERM hmac_final(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (Context) or (Context, HashLen) */ - ErlNifBinary context; ERL_NIF_TERM ret; - HMAC_CTX ctx; + struct hmac_context* obj; unsigned char mac_buf[EVP_MAX_MD_SIZE]; unsigned char * mac_bin; unsigned int req_len = 0; unsigned int mac_len; - if (!enif_inspect_binary(env, argv[0], &context)) { - return enif_make_badarg(env); - } - if (argc == 2 && !enif_get_uint(env, argv[1], &req_len)) { + if (!enif_get_resource(env,argv[0],hmac_context_rtype, (void**)&obj) + || (argc == 2 && !enif_get_uint(env, argv[1], &req_len))) { return enif_make_badarg(env); } - if (context.size != sizeof(ctx)) { - return enif_make_badarg(env); + enif_mutex_lock(obj->mtx); + if (!obj->alive) { + enif_mutex_unlock(obj->mtx); + return enif_make_badarg(env); } - memcpy(&ctx, context.data, context.size); - HMAC_Final(&ctx, mac_buf, &mac_len); - HMAC_CTX_cleanup(&ctx); + HMAC_Final(&obj->ctx, mac_buf, &mac_len); + HMAC_CTX_cleanup(&obj->ctx); + obj->alive = 0; + enif_mutex_unlock(obj->mtx); if (argc == 2 && req_len < mac_len) { /* Only truncate to req_len bytes if asked. */ diff --git a/lib/crypto/c_src/crypto_callback.c b/lib/crypto/c_src/crypto_callback.c index 81106b4cc2..a08dcec463 100644 --- a/lib/crypto/c_src/crypto_callback.c +++ b/lib/crypto/c_src/crypto_callback.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2012. All Rights Reserved. + * Copyright Ericsson AB 2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -17,6 +17,7 @@ * %CopyrightEnd% */ +#include <stdio.h> #include <string.h> #include <openssl/opensslconf.h> @@ -51,13 +52,28 @@ DLLEXPORT struct crypto_callbacks* get_crypto_callbacks(int nlocks); static ErlNifRWLock** lock_vec = NULL; /* Static locks used by openssl */ +static void nomem(size_t size, const char* op) +{ + fprintf(stderr, "Out of memory abort. Crypto failed to %s %zu bytes.\r\n", + op, size); + abort(); +} + static void* crypto_alloc(size_t size) { - return enif_alloc(size); + void *ret = enif_alloc(size); + + if (!ret && size) + nomem(size, "allocate"); + return ret; } static void* crypto_realloc(void* ptr, size_t size) { - return enif_realloc(ptr, size); + void* ret = enif_realloc(ptr, size); + + if (!ret && size) + nomem(size, "reallocate"); + return ret; } static void crypto_free(void* ptr) { diff --git a/lib/crypto/doc/src/crypto.xml b/lib/crypto/doc/src/crypto.xml index 40f829e704..c95827c371 100644 --- a/lib/crypto/doc/src/crypto.xml +++ b/lib/crypto/doc/src/crypto.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1999</year><year>2013</year> + <year>1999</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -366,7 +366,11 @@ or to one of the functions <seealso marker="#hmac_final-1">hmac_final</seealso> and <seealso marker="#hmac_final_n-2">hmac_final_n</seealso> </p> - + <warning><p>Do not use a <c>Context</c> as argument in more than one + call to hmac_update or hmac_final. The semantics of reusing old contexts + in any way is undefined and could even crash the VM in earlier releases. + The reason for this limitation is a lack of support in the underlying + OpenSSL API.</p></warning> </desc> </func> diff --git a/lib/debugger/test/bs_construct_SUITE.erl b/lib/debugger/test/bs_construct_SUITE.erl index 4870c87d74..8a6798c6ad 100644 --- a/lib/debugger/test/bs_construct_SUITE.erl +++ b/lib/debugger/test/bs_construct_SUITE.erl @@ -647,19 +647,27 @@ make_sub_bin(Bin0) -> %% give the same result. dynamic(Config) when is_list(Config) -> - ?line dynamic_1(fun dynamic_big/5), - ?line dynamic_1(fun dynamic_little/5), + Ps = [spawn_monitor(fun() -> + dynamic_1(Fun) + end) || Fun <- [fun dynamic_big/5, + fun dynamic_little/5]], + [receive + {'DOWN',Ref,process,Pid,normal} -> + ok; + {'DOWN',Ref,process,Pid,Exit} -> + ?t:fail({Pid,Exit}) + end || {Pid,Ref} <- Ps], ok. dynamic_1(Dynamic) -> - <<Lpad:128>> = erlang:md5([0]), - <<Rpad:128>> = erlang:md5([1]), - <<Int:128>> = erlang:md5([2]), - 8385 = dynamic_2(0, {Int,Lpad,Rpad,Dynamic}, 0). + <<Lpad:64,_/binary>> = erlang:md5([0]), + <<Rpad:64,_/binary>> = erlang:md5([1]), + <<Int:64,_/binary>> = erlang:md5([2]), + 2145 = dynamic_2(0, {Int,Lpad,Rpad,Dynamic}, 0). -dynamic_2(129, _, Count) -> Count; +dynamic_2(64+1, _, Count) -> Count; dynamic_2(Bef, Data, Count0) -> - Count = dynamic_3(Bef, 128-Bef, Data, Count0), + Count = dynamic_3(Bef, 64-Bef, Data, Count0), dynamic_2(Bef+1, Data, Count). dynamic_3(_, -1, _, Count) -> Count; @@ -680,13 +688,13 @@ dynamic_big(Bef, N, Int, Lpad, Rpad) -> <<MaskedInt:N>> = NumBin, %% Construct the binary in two different ways. - Bin = id(<<Lpad:Bef,NumBin/bitstring,Rpad:(128-Bef-N)>>), - Bin = <<Lpad:Bef,Int:N,Rpad:(128-Bef-N)>>, + Bin = id(<<Lpad:Bef,NumBin/bitstring,Rpad:(64-Bef-N)>>), + Bin = <<Lpad:Bef,Int:N,Rpad:(64-Bef-N)>>, %% Further verify the result by matching. LpadMasked = Lpad band ((1 bsl Bef) - 1), - RpadMasked = Rpad band ((1 bsl (128-Bef-N)) - 1), - Rbits = (128-Bef-N), + RpadMasked = Rpad band ((1 bsl (64-Bef-N)) - 1), + Rbits = (64-Bef-N), <<LpadMasked:Bef,MaskedInt:N,RpadMasked:Rbits>> = id(Bin), ok. @@ -696,13 +704,13 @@ dynamic_little(Bef, N, Int, Lpad, Rpad) -> <<MaskedInt:N/little>> = NumBin, %% Construct the binary in two different ways. - Bin = id(<<Lpad:Bef/little,NumBin/bitstring,Rpad:(128-Bef-N)/little>>), - Bin = <<Lpad:Bef/little,Int:N/little,Rpad:(128-Bef-N)/little>>, + Bin = id(<<Lpad:Bef/little,NumBin/bitstring,Rpad:(64-Bef-N)/little>>), + Bin = <<Lpad:Bef/little,Int:N/little,Rpad:(64-Bef-N)/little>>, %% Further verify the result by matching. LpadMasked = Lpad band ((1 bsl Bef) - 1), - RpadMasked = Rpad band ((1 bsl (128-Bef-N)) - 1), - Rbits = (128-Bef-N), + RpadMasked = Rpad band ((1 bsl (64-Bef-N)) - 1), + Rbits = (64-Bef-N), <<LpadMasked:Bef/little,MaskedInt:N/little,RpadMasked:Rbits/little>> = id(Bin), ok. diff --git a/lib/dialyzer/src/dialyzer_contracts.erl b/lib/dialyzer/src/dialyzer_contracts.erl index 3467ab4e65..5b2c52b2d0 100644 --- a/lib/dialyzer/src/dialyzer_contracts.erl +++ b/lib/dialyzer/src/dialyzer_contracts.erl @@ -404,7 +404,8 @@ contract_from_form([{type, _, 'fun', [_, _]} = Form | Left], RecDict, throw({error, NewMsg}) end, NewType = erl_types:t_solve_remote(Type, ExpTypes, AllRecords), - {NewType, []} + NewTypeNoVars = erl_types:subst_all_vars_to_any(NewType), + {NewTypeNoVars, []} end, NewTypeAcc = [TypeFun | TypeAcc], NewFormAcc = [{Form, []} | FormAcc], @@ -418,7 +419,8 @@ contract_from_form([{type, _L1, bounded_fun, process_constraints(Constr, RecDict, ExpTypes, AllRecords), Type = erl_types:t_from_form(Form, RecDict, VarDict), NewType = erl_types:t_solve_remote(Type, ExpTypes, AllRecords), - {NewType, Constr1} + NewTypeNoVars = erl_types:subst_all_vars_to_any(NewType), + {NewTypeNoVars, Constr1} end, NewTypeAcc = [TypeFun | TypeAcc], NewFormAcc = [{Form, Constr} | FormAcc], diff --git a/lib/dialyzer/src/dialyzer_dataflow.erl b/lib/dialyzer/src/dialyzer_dataflow.erl index 03f9684b02..b00e0465e0 100644 --- a/lib/dialyzer/src/dialyzer_dataflow.erl +++ b/lib/dialyzer/src/dialyzer_dataflow.erl @@ -58,10 +58,12 @@ t_fun_range/2, t_integer/0, t_integers/1, t_is_any/1, t_is_atom/1, t_is_atom/2, t_is_any_atom/3, t_is_boolean/2, - t_is_integer/2, t_is_nil/2, t_is_none/1, t_is_none_or_unit/1, + t_is_integer/2, t_is_list/1, + t_is_nil/2, t_is_none/1, t_is_none_or_unit/1, t_is_number/2, t_is_reference/2, t_is_pid/2, t_is_port/2, t_is_unit/1, - t_limit/2, t_list/0, t_maybe_improper_list/0, t_module/0, + t_limit/2, t_list/0, t_list_elements/2, + t_maybe_improper_list/0, t_module/0, t_none/0, t_non_neg_integer/0, t_number/0, t_number_vals/2, t_pid/0, t_port/0, t_product/1, t_reference/0, t_to_string/2, t_to_tlist/1, @@ -293,6 +295,7 @@ traverse(Tree, Map, State) -> t_is_any(ArgType) orelse t_is_simple(ArgType, State) orelse is_call_to_send(Arg) + orelse is_lc_simple_list(Arg, ArgType, State) of true -> % do not warn in these cases State1; @@ -2713,6 +2716,13 @@ is_call_to_send(Tree) -> andalso (Arity =:= 2) end. +is_lc_simple_list(Tree, TreeType, State) -> + Opaques = State#state.opaques, + Ann = cerl:get_ann(Tree), + lists:member(list_comprehension, Ann) + andalso t_is_list(TreeType) + andalso t_is_simple(t_list_elements(TreeType, Opaques), State). + filter_match_fail([Clause] = Cls) -> Body = cerl:clause_body(Clause), case cerl:type(Body) of diff --git a/lib/dialyzer/test/small_SUITE_data/results/contracts_with_subtypes b/lib/dialyzer/test/small_SUITE_data/results/contracts_with_subtypes index 173ff3a9f1..bfa33cd296 100644 --- a/lib/dialyzer/test/small_SUITE_data/results/contracts_with_subtypes +++ b/lib/dialyzer/test/small_SUITE_data/results/contracts_with_subtypes @@ -8,15 +8,15 @@ contracts_with_subtypes.erl:111: The call contracts_with_subtypes:rec_arg({'b',{ contracts_with_subtypes.erl:142: The pattern 1 can never match the type string() contracts_with_subtypes.erl:145: The pattern 'alpha' can never match the type {'ok',_} | {'ok',_,string()} contracts_with_subtypes.erl:147: The pattern 42 can never match the type {'ok',_} | {'ok',_,string()} -contracts_with_subtypes.erl:163: The pattern 'alpha' can never match the type {'ok',X} -contracts_with_subtypes.erl:165: The pattern 42 can never match the type {'ok',X} -contracts_with_subtypes.erl:183: The pattern 'alpha' can never match the type {'ok',X} -contracts_with_subtypes.erl:185: The pattern 42 can never match the type {'ok',X} +contracts_with_subtypes.erl:163: The pattern 'alpha' can never match the type {'ok',_} +contracts_with_subtypes.erl:165: The pattern 42 can never match the type {'ok',_} +contracts_with_subtypes.erl:183: The pattern 'alpha' can never match the type {'ok',_} +contracts_with_subtypes.erl:185: The pattern 42 can never match the type {'ok',_} contracts_with_subtypes.erl:202: The pattern 1 can never match the type string() -contracts_with_subtypes.erl:205: The pattern {'ok', _} can never match the type {'ok',X,string()} -contracts_with_subtypes.erl:206: The pattern 'alpha' can never match the type {'ok',X,string()} -contracts_with_subtypes.erl:207: The pattern {'ok', 42} can never match the type {'ok',X,string()} -contracts_with_subtypes.erl:208: The pattern 42 can never match the type {'ok',X,string()} +contracts_with_subtypes.erl:205: The pattern {'ok', _} can never match the type {'ok',_,string()} +contracts_with_subtypes.erl:206: The pattern 'alpha' can never match the type {'ok',_,string()} +contracts_with_subtypes.erl:207: The pattern {'ok', 42} can never match the type {'ok',_,string()} +contracts_with_subtypes.erl:208: The pattern 42 can never match the type {'ok',_,string()} contracts_with_subtypes.erl:234: Function flat_ets_new_t/0 has no local return contracts_with_subtypes.erl:235: The call contracts_with_subtypes:flat_ets_new(12,[]) breaks the contract (Name,Options) -> atom() when is_subtype(Name,atom()), is_subtype(Options,[Option]), is_subtype(Option,'set' | 'ordered_set' | 'bag' | 'duplicate_bag' | 'public' | 'protected' | 'private' | 'named_table' | {'keypos',integer()} | {'heir',pid(),term()} | {'heir','none'} | {'write_concurrency',boolean()} | {'read_concurrency',boolean()} | 'compressed') contracts_with_subtypes.erl:23: Invalid type specification for function contracts_with_subtypes:extract2/0. The success typing is () -> 'something' diff --git a/lib/dialyzer/test/unmatched_returns_SUITE_data/dialyzer_options b/lib/dialyzer/test/unmatched_returns_SUITE_data/dialyzer_options new file mode 100644 index 0000000000..49ac917f61 --- /dev/null +++ b/lib/dialyzer/test/unmatched_returns_SUITE_data/dialyzer_options @@ -0,0 +1 @@ +{dialyzer_options, [{warnings, [unmatched_returns]}]}. diff --git a/lib/dialyzer/test/unmatched_returns_SUITE_data/results/lc_warnings b/lib/dialyzer/test/unmatched_returns_SUITE_data/results/lc_warnings new file mode 100644 index 0000000000..2784f2119e --- /dev/null +++ b/lib/dialyzer/test/unmatched_returns_SUITE_data/results/lc_warnings @@ -0,0 +1,5 @@ + +lc_warnings.erl:32: Expression produces a value of type [opaque_atom_adt:opaque_atom()], but this value is unmatched +lc_warnings.erl:43: Expression produces a value of type [array()], but this value is unmatched +lc_warnings.erl:65: Expression produces a value of type [lc_warnings:opaque_tuple()], but this value is unmatched +lc_warnings.erl:7: Expression produces a value of type ['ok' | {'error',atom()}], but this value is unmatched diff --git a/lib/dialyzer/test/unmatched_returns_SUITE_data/src/lc_warnings/lc_warnings.erl b/lib/dialyzer/test/unmatched_returns_SUITE_data/src/lc_warnings/lc_warnings.erl new file mode 100644 index 0000000000..cb01a8fde3 --- /dev/null +++ b/lib/dialyzer/test/unmatched_returns_SUITE_data/src/lc_warnings/lc_warnings.erl @@ -0,0 +1,95 @@ +-module(lc_warnings). +-compile([export_all]). + +close(Fs) -> + %% There should be a warning since we ignore a potential + %% {error,Error} return from file:close/1. + [file:close(F) || F <- Fs], + + %% No warning because the type of unmatched return will be ['ok'] + %% (which is a list of a simple type). + [ok = file:close(F) || F <- Fs], + + %% Suppressed. + _ = [file:close(F) || F <- Fs], + ok. + +format(X) -> + %% No warning since the result of the list comprehension is + %% a list of simple. + [io:format("~p\n", [E]) || E <- X], + + %% Warning explicitly suppressed. + _ = [io:format("~p\n", [E]) || E <- X], + ok. + +opaque1() -> + List = gen_atom(), + %% This is a list of an externally defined opaque type. Since + %% we are not allowed to peek inside opaque types, there should + %% be a warning (even though the type in this case happens to be + %% an atom). + [E || E <- List], + + %% Suppressed. + _ = [E || E <- List], + ok. + +opaque2() -> + List = gen_array(), + %% This is an list of an externally defined opaque type. Since + %% we are not allowed to peek inside opaque types, there should + %% be a warning. + [E || E <- List], + + %% Suppressed. + _ = [E || E <- List], + ok. + +opaque3() -> + List = gen_int(), + + %% No warning, since we are allowed to look into the type and can + %% see that it is a simple type. + [E || E <- List], + + %% Suppressed. + _ = [E || E <- List], + ok. + +opaque4() -> + List = gen_tuple(), + + %% There should be a warning, since we are allowed to look inside + %% the opaque type and see that it is a tuple (non-simple). + [E || E <- List], + + %% Suppressed. + _ = [E || E <- List], + ok. + +gen_atom() -> + [opaque_atom_adt:atom(ok)]. + +gen_array() -> + [array:new()]. + + +gen_int() -> + [opaque_int(42)]. + +gen_tuple() -> + [opaque_tuple(x, 25)]. + +-opaque opaque_int() :: integer(). + +-spec opaque_int(integer()) -> opaque_int(). + +opaque_int(Int) -> Int. + +-opaque opaque_tuple() :: {any(),any()}. + +-spec opaque_tuple(any(), any()) -> opaque_tuple(). + +opaque_tuple(X, Y) -> + {X,Y}. diff --git a/lib/dialyzer/test/unmatched_returns_SUITE_data/src/lc_warnings/opaque_atom_adt.erl b/lib/dialyzer/test/unmatched_returns_SUITE_data/src/lc_warnings/opaque_atom_adt.erl new file mode 100644 index 0000000000..b5b51fe75b --- /dev/null +++ b/lib/dialyzer/test/unmatched_returns_SUITE_data/src/lc_warnings/opaque_atom_adt.erl @@ -0,0 +1,9 @@ +-module(opaque_atom_adt). +-export([atom/1]). + +-opaque opaque_atom() :: atom(). + +-spec atom(atom()) -> opaque_atom(). + +atom(Atom) -> + Atom. diff --git a/lib/diameter/src/transport/diameter_sctp.erl b/lib/diameter/src/transport/diameter_sctp.erl index f5275e66b5..d0a01351f3 100644 --- a/lib/diameter/src/transport/diameter_sctp.erl +++ b/lib/diameter/src/transport/diameter_sctp.erl @@ -171,18 +171,33 @@ start_link(T) -> info({gen_sctp, Sock}) -> lists:flatmap(fun(K) -> info(K, Sock) end, - [{socket, sockname}, - {peer, peername}, + [{socket, socknames}, + {peer, peernames}, {statistics, getstat}]). info({K,F}, Sock) -> case inet:F(Sock) of {ok, V} -> - [{K,V}]; + [{K, map(F,V)}]; _ -> [] end. +%% inet:{sock,peer}names/1 returns [{Addr, Port}] but the port number +%% should be the same in each tuple. Map to a {[Addr], Port} tuple if +%% so. +map(K, [{_, Port} | _] = APs) + when K == socknames; + K == peernames -> + try [A || {A,P} <- APs, P == Port orelse throw(?MODULE)] of + As -> {As, Port} + catch + ?MODULE -> APs + end; + +map(_, V) -> + V. + %% --------------------------------------------------------------------------- %% # init/1 %% --------------------------------------------------------------------------- @@ -549,7 +564,7 @@ accept_peer(_, []) -> ok; accept_peer(Sock, Matches) -> - {RAddrs, _} = ok(inet:peername(Sock)), + RAddrs = [A || {A,_} <- ok(inet:peernames(Sock))], diameter_peer:match(RAddrs, Matches) orelse x({accept, RAddrs, Matches}), ok. diff --git a/lib/hipe/cerl/erl_types.erl b/lib/hipe/cerl/erl_types.erl index af34e355bb..329a71b099 100644 --- a/lib/hipe/cerl/erl_types.erl +++ b/lib/hipe/cerl/erl_types.erl @@ -645,15 +645,16 @@ decorate_with_opaque(Type, ?opaque(Set2), Opaques) -> end. decoration([#opaque{struct = S} = Opaque|OpaqueTypes], Type, Opaques, - NewOpaqueTypes, All) -> + NewOpaqueTypes0, All) -> IsOpaque = is_opaque_type2(Opaque, Opaques), I = t_inf(Type, S), - case not IsOpaque orelse t_is_none(I = t_inf(Type, S)) of - true -> decoration(OpaqueTypes, Type, Opaques, NewOpaqueTypes, All); + case not IsOpaque orelse t_is_none(I) of + true -> decoration(OpaqueTypes, Type, Opaques, NewOpaqueTypes0, All); false -> NewOpaque = Opaque#opaque{struct = decorate(I, S, Opaques)}, NewAll = All orelse t_is_equal(I, Type), - decoration(OpaqueTypes, Type, Opaques, [NewOpaque|NewOpaqueTypes], NewAll) + NewOpaqueTypes = [NewOpaque|NewOpaqueTypes0], + decoration(OpaqueTypes, Type, Opaques, NewOpaqueTypes, NewAll) end; decoration([], _Type, _Opaques, NewOpaqueTypes, All) -> {NewOpaqueTypes, All}. @@ -733,13 +734,14 @@ t_opaque_from_records(RecDict) -> end end, RecDict), OpaqueTypeDict = - dict:map(fun({opaque, Name, _Arity}, {Module, Type, ArgNames}) -> - case ArgNames of - [] -> - t_opaque(Module, Name, [], t_from_form(Type, RecDict)); - _ -> - throw({error,"Polymorphic opaque types not supported yet"}) - end + dict:map(fun({opaque, Name, _Arity}, {Module, _Type, ArgNames}) -> + %% Args = args_to_types(ArgNames), + %% List = lists:zip(ArgNames, Args), + %% TmpVarDict = dict:from_list(List), + %% Rep = t_from_form(Type, RecDict, TmpVarDict), + Rep = t_none(), % not used for anything right now + Args = [t_any() || _ <- ArgNames], + skip_opaque_alias(Rep, Module, Name, Args) end, OpaqueRecDict), [OpaqueType || {_Key, OpaqueType} <- dict:to_list(OpaqueTypeDict)]. @@ -833,8 +835,12 @@ t_solve_remote(?union(List), ET, R, C) -> {t_sup(RL), RR}; t_solve_remote(T, _ET, _R, _C) -> {T, []}. -t_solve_remote_type(#remote{mod = RemMod, name = Name, args = Args} = RemType, +t_solve_remote_type(#remote{mod = RemMod, name = Name, args = Args0} = RemType, ET, R, C) -> + Args = lists:map(fun(A) -> + {Arg, _} = t_solve_remote(A, ET, R, C), + Arg + end, Args0), ArgsLen = length(Args), case dict:find(RemMod, R) of error -> @@ -880,9 +886,7 @@ t_solve_remote_type(#remote{mod = RemMod, name = Name, args = Args} = RemType, true -> t_limit(NewRep, ?REC_TYPE_LIMIT); false -> NewRep end, - {t_from_form({opaque, -1, Name, {Mod, Args, RT1}}, - RemDict, TmpVarDict), - RetRR}; + {skip_opaque_alias(RT1, Mod, Name, Args), RetRR}; error -> Msg = io_lib:format("Unable to find remote type ~w:~w()\n", [RemMod, Name]), @@ -1962,16 +1966,20 @@ t_array() -> t_tuple([t_atom('array'), t_sup([t_atom('undefined'), t_non_neg_integer()]), t_sup([t_atom('undefined'), t_non_neg_integer()]), - t_any(), t_any()])). + t_any(), + t_any()])). -spec t_dict() -> erl_type(). t_dict() -> t_opaque(dict, dict, [], t_tuple([t_atom('dict'), - t_non_neg_integer(), t_non_neg_integer(), - t_non_neg_integer(), t_non_neg_integer(), - t_non_neg_integer(), t_non_neg_integer(), + t_sup([t_atom('undefined'), t_non_neg_integer()]), + t_sup([t_atom('undefined'), t_non_neg_integer()]), + t_sup([t_atom('undefined'), t_non_neg_integer()]), + t_sup([t_atom('undefined'), t_non_neg_integer()]), + t_sup([t_atom('undefined'), t_non_neg_integer()]), + t_sup([t_atom('undefined'), t_non_neg_integer()]), t_sup([t_atom('undefined'), t_tuple()]), t_sup([t_atom('undefined'), t_tuple()])])). @@ -2082,11 +2090,11 @@ t_has_var(?tuple(Elements, _, _)) -> t_has_var_list(Elements); t_has_var(?tuple_set(_) = T) -> t_has_var_list(t_tuple_subtypes(T)); -%% t_has_var(?opaque(_)=T) -> -%% %% "Polymorphic opaque types not supported yet" -%% t_has_var(t_opaque_structure(T)); -%% t_has_var(?union(_) = U) -> -%% exit(flat_format("Union happens in t_has_var/1 ~p\n",[U])); +t_has_var(?opaque(Set)) -> + %% Assume variables in 'args' are also present i 'struct' + t_has_var_list([O#opaque.struct || O <- set_to_list(Set)]); +t_has_var(?union(List)) -> + t_has_var_list(List); t_has_var(_) -> false. -spec t_has_var_list([erl_type()]) -> boolean(). @@ -2117,9 +2125,10 @@ t_collect_vars(?tuple(Types, _, _), Acc) -> t_collect_vars(?tuple_set(_) = TS, Acc) -> lists:foldl(fun(T, TmpAcc) -> t_collect_vars(T, TmpAcc) end, Acc, t_tuple_subtypes(TS)); -%% t_collect_vars(?opaque(_)=T, Acc) -> -%% %% "Polymorphic opaque types not supported yet" -%% t_collect_vars(t_opaque_structure(T), Acc); +t_collect_vars(?opaque(Set), Acc) -> + %% Assume variables in 'args' are also present i 'struct' + lists:foldl(fun(T, TmpAcc) -> t_collect_vars(T, TmpAcc) end, Acc, + [O#opaque.struct || O <- set_to_list(Set)]); t_collect_vars(_, Acc) -> Acc. @@ -2442,8 +2451,8 @@ sup_opaque(List) -> ?opaque(ordsets:from_list(L)). sup_opaq(L0) -> - L1 = [{{Mod,Name}, T} || - #opaque{mod = Mod, name = Name}=T <- L0], + L1 = [{{Mod,Name,Args}, T} || + #opaque{mod = Mod, name = Name, args = Args}=T <- L0], F = family(L1), [supl(Ts) || {_, Ts} <- F]. @@ -2809,31 +2818,35 @@ inf_collect(_T1, [], _Opaques, OpL) -> OpL. combine(S, T1, T2) -> - #opaque{mod = Mod1, name = Name1} = T1, - #opaque{mod = Mod2, name = Name2} = T2, - case {Mod1, Name1} =:= {Mod2, Name2} of - true -> [comb(Mod1, Name1, S, T1)]; - false -> [comb(Mod1, Name1, S, T1), comb(Mod2, Name2, S, T2)] + #opaque{mod = Mod1, name = Name1, args = Args1} = T1, + #opaque{mod = Mod2, name = Name2, args = Args2} = T2, + case is_same_type_name({Mod1, Name1, Args1}, {Mod2, Name2, Args2}) of + true -> [comb(Mod1, Name1, Args1, S, T1)]; + false -> [comb(Mod1, Name1, Args1, S, T1), comb(Mod2, Name2, Args2, S, T2)] end. -comb(Mod, Name, S, T) -> - case is_same_name(Mod, Name, S) of +comb(Mod, Name, Args, S, T) -> + case is_same_name(Mod, Name, Args, S) of true -> S; false -> T#opaque{struct = S} end. -is_same_name(Mod, Name, ?opaque([#opaque{mod = Mod, name = Name}])) -> true; -is_same_name(_Mod, _Name, _Opaque) -> false. +is_same_name(Mod, Name, Args, + ?opaque([#opaque{mod = Mod, name = Name, args = Args}])) -> + true; +is_same_name(_, _, _, _) -> false. %% Combining two lists this way can be very time consuming... +%% Note: two parameterized opaque types are not the same if their +%% actual parameters differ inf_opaque(Set1, Set2, Opaques) -> List1 = inf_look_up(Set1, 1, Opaques), List2 = inf_look_up(Set2, 2, Opaques), List0 = [combine(Inf, T1, T2) || - {Is1, ModName1, T1} <- List1, - {Is2, ModName2, T2} <- List2, - not t_is_none(Inf = inf_opaque_types(Is1, ModName1, T1, - Is2, ModName2, T2, + {Is1, ModNameArgs1, T1} <- List1, + {Is2, ModNameArgs2, T2} <- List2, + not t_is_none(Inf = inf_opaque_types(Is1, ModNameArgs1, T1, + Is2, ModNameArgs2, T2, Opaques))], List = lists:sort(lists:append(List0)), sup_opaque(List). @@ -2841,18 +2854,22 @@ inf_opaque(Set1, Set2, Opaques) -> %% Optimization: do just one lookup. inf_look_up(Set, Pos, Opaques) -> [{Opaques =:= 'universe' orelse inf_is_opaque_type2(T, Pos, Opaques), - {M, N}, T} || - #opaque{mod = M, name = N} = T <- set_to_list(Set)]. + {M, N, Args}, T} || + #opaque{mod = M, name = N, args = Args} = T <- set_to_list(Set)]. inf_is_opaque_type2(T, Pos, {match, Opaques}) -> is_opaque_type2(T, Opaques) orelse throw(Pos); inf_is_opaque_type2(T, _Pos, Opaques) -> is_opaque_type2(T, Opaques). -inf_opaque_types(IsOpaque1, ModName1, T1, IsOpaque2, ModName2, T2, Opaques) -> +inf_opaque_types(IsOpaque1, ModNameArgs1, T1, + IsOpaque2, ModNameArgs2, T2, Opaques) -> #opaque{struct = S1}=T1, #opaque{struct = S2}=T2, - case Opaques =:= 'universe' orelse ModName1 =:= ModName2 of + case + Opaques =:= 'universe' orelse + is_same_type_name(ModNameArgs1, ModNameArgs2) + of true -> t_inf(S1, S2, Opaques); false -> case {IsOpaque1, IsOpaque2} of @@ -3060,11 +3077,13 @@ t_subst_dict(?tuple(Elements, _Arity, _Tag), Dict) -> t_tuple([t_subst_dict(E, Dict) || E <- Elements]); t_subst_dict(?tuple_set(_) = TS, Dict) -> t_sup([t_subst_dict(T, Dict) || T <- t_tuple_subtypes(TS)]); -%% t_subst_dict(?opaque(Es), Dict) -> -%% %% "Polymorphic opaque types not supported yet" -%% List = [Opaque#opaque{struct = t_subst_dict(S, Dict)} || -%% Opaque = #opaque{struct = S} <- set_to_list(Es)], -%% ?opaque(ordsets:from_list(List)); +t_subst_dict(?opaque(Es), Dict) -> + List = [Opaque#opaque{args = [t_subst_dict(Arg, Dict) || Arg <- Args], + struct = t_subst_dict(S, Dict)} || + Opaque = #opaque{args = Args, struct = S} <- set_to_list(Es)], + ?opaque(ordsets:from_list(List)); +t_subst_dict(?union(List), Dict) -> + ?union([t_subst_dict(E, Dict) || E <- List]); t_subst_dict(T, _Dict) -> T. @@ -3107,11 +3126,13 @@ t_subst_aux(?tuple(Elements, _Arity, _Tag), VarMap) -> t_tuple([t_subst_aux(E, VarMap) || E <- Elements]); t_subst_aux(?tuple_set(_) = TS, VarMap) -> t_sup([t_subst_aux(T, VarMap) || T <- t_tuple_subtypes(TS)]); -%% t_subst_aux(?opaque(Es), VarMap) -> -%% %% "Polymorphic opaque types not supported yet" -%% List = [Opaque#opaque{struct = t_subst_aux(S, VarMap)} || -%% Opaque = #opaque{struct = S} <- set_to_list(Es)], -%% ?opaque(ordsets:from_list(List)); +t_subst_aux(?opaque(Es), VarMap) -> + List = [Opaque#opaque{args = [t_subst_aux(Arg, VarMap) || Arg <- Args], + struct = t_subst_aux(S, VarMap)} || + Opaque = #opaque{args = Args, struct = S} <- set_to_list(Es)], + ?opaque(ordsets:from_list(List)); +t_subst_aux(?union(List), VarMap) -> + ?union([t_subst_aux(E, VarMap) || E <- List]); t_subst_aux(T, _VarMap) -> T. @@ -3232,15 +3253,20 @@ unify_union(List) -> is_opaque_type(?opaque(Elements), Opaques) -> lists:any(fun(Opaque) -> is_opaque_type2(Opaque, Opaques) end, Elements). -is_opaque_type2(#opaque{mod = Mod1, name = Name1}, Opaques) -> +is_opaque_type2(#opaque{mod = Mod1, name = Name1, args = Args1}, Opaques) -> F1 = fun(?opaque(Es)) -> - F2 = fun(#opaque{mod = Mod, name = Name}) -> - Mod1 =:= Mod andalso Name1 =:= Name + F2 = fun(#opaque{mod = Mod, name = Name, args = Args}) -> + is_type_name(Mod1, Name1, Args1, Mod, Name, Args) end, lists:any(F2, Es) end, lists:any(F1, Opaques). +is_type_name(Mod, Name, Args1, Mod, Name, Args2) -> + length(Args1) =:= length(Args2); +is_type_name(Mod1, Name1, Args1, Mod2, Name2, Args2) -> + is_same_type_name2(Mod1, Name1, Args1, Mod2, Name2, Args2). + %% Two functions since t_unify is not symmetric. unify_tuple_set_and_tuple1(?tuple_set([{Arity, List}]), ?tuple(Elements2, Arity, _), VarMap) -> @@ -3885,8 +3911,8 @@ t_to_string(?identifier(Set), _RecDict) -> string:join([flat_format("~w()", [T]) || T <- set_to_list(Set)], " | ") end; t_to_string(?opaque(Set), RecDict) -> - string:join([opaque_type(Mod, Name, S, RecDict) || - #opaque{mod = Mod, name = Name, struct = S} + string:join([opaque_type(Mod, Name, Args, S, RecDict) || + #opaque{mod = Mod, name = Name, struct = S, args = Args} <- set_to_list(Set)], " | "); t_to_string(?matchstate(Pres, Slots), RecDict) -> @@ -4059,11 +4085,14 @@ union_sequence(Types, RecDict) -> string:join(List, " | "). -ifdef(DEBUG). -opaque_type(Mod, Name, S, RecDict) -> - opaque_name(Mod, Name, t_to_string(S, RecDict)). +opaque_type(Mod, Name, _Args, S, RecDict) -> + ArgsString = "[ARGS:" ++ comma_sequence(_Args, RecDict) ++ "]", + String = t_to_string(S, RecDict), + opaque_name(Mod, Name, ArgsString ++ String). -else. -opaque_type(Mod, Name, _S, _RecDict) -> - opaque_name(Mod, Name, ""). +opaque_type(Mod, Name, Args, _S, RecDict) -> + ArgsString = comma_sequence(Args, RecDict), + opaque_name(Mod, Name, ArgsString). -endif. opaque_name(Mod, Name, Extra) -> @@ -4288,18 +4317,19 @@ t_from_form({type, _L, union, Args}, TypeNames, RecDict, VarDict) -> {L, R} = list_from_form(Args, TypeNames, RecDict, VarDict), {t_sup(L), R}; t_from_form({type, _L, Name, Args}, TypeNames, RecDict, VarDict) -> + type_from_form(Name, Args, TypeNames, RecDict, VarDict); +t_from_form({opaque, _L, Name, {Mod, Args, Rep}}, _TypeNames, + _RecDict, _VarDict) -> + {t_opaque(Mod, Name, Args, Rep), []}. + +type_from_form(Name, Args, TypeNames, RecDict, VarDict) -> ArgsLen = length(Args), + ArgTypes = forms_to_types(Args, TypeNames, RecDict, VarDict), case lookup_type(Name, ArgsLen, RecDict) of {type, {_Module, Type, ArgNames}} -> case can_unfold_more({type, Name}, TypeNames) of true -> - List = lists:zipwith( - fun(ArgName, ArgType) -> - {Ttemp, _R} = t_from_form(ArgType, TypeNames, - RecDict, VarDict), - {ArgName, Ttemp} - end, - ArgNames, Args), + List = lists:zip(ArgNames, ArgTypes), TmpVarDict = dict:from_list(List), {T, R} = t_from_form(Type, [{type, Name}|TypeNames], RecDict, TmpVarDict), @@ -4313,13 +4343,7 @@ t_from_form({type, _L, Name, Args}, TypeNames, RecDict, VarDict) -> {Rep, Rret} = case can_unfold_more({opaque, Name}, TypeNames) of true -> - List = lists:zipwith( - fun(ArgName, ArgType) -> - {Ttemp, _R} = t_from_form(ArgType, TypeNames, - RecDict, VarDict), - {ArgName, Ttemp} - end, - ArgNames, Args), + List = lists:zip(ArgNames, ArgTypes), TmpVarDict = dict:from_list(List), {T, R} = t_from_form(Type, [{opaque, Name}|TypeNames], RecDict, TmpVarDict), @@ -4329,20 +4353,21 @@ t_from_form({type, _L, Name, Args}, TypeNames, RecDict, VarDict) -> end; false -> {t_any(), [{opaque, Name}]} end, - Tret = t_from_form({opaque, -1, Name, {Module, Args, Rep}}, - RecDict, VarDict), - {Tret, Rret}; + Args2 = [subst_all_vars_to_any(ArgType) || ArgType <- ArgTypes], + {skip_opaque_alias(Rep, Module, Name, Args2), Rret}; error -> Msg = io_lib:format("Unable to find type ~w/~w\n", [Name, ArgsLen]), throw({error, Msg}) - end; -t_from_form({opaque, _L, Name, {Mod, Args, Rep}}, _TypeNames, - _RecDict, _VarDict) -> - case Args of - [] -> {t_opaque(Mod, Name, Args, Rep), []}; - _ -> throw({error, "Polymorphic opaque types not supported yet"}) end. +forms_to_types(Forms, TypeNames, RecDict, VarDict) -> + {Types, _} = list_from_form(Forms, TypeNames, RecDict, VarDict), + Types. + +skip_opaque_alias(?opaque(_) = T, _Mod, _Name, _Args) -> T; +skip_opaque_alias(T, Module, Name, Args) -> + t_opaque(Module, Name, Args, T). + record_from_form({atom, _, Name}, ModFields, TypeNames, RecDict, VarDict) -> case can_unfold_more({record, Name}, TypeNames) of true -> @@ -4403,8 +4428,10 @@ build_field_dict([], _TypeNames, _RecDict, _VarDict, Acc) -> get_mod_record([{FieldName, DeclType}|Left1], [{FieldName, ModType}|Left2], Acc) -> - case t_is_var(ModType) orelse t_is_remote(ModType) orelse - t_is_subtype(ModType, DeclType) of + ModTypeNoVars = subst_all_vars_to_any(ModType), + case + t_is_remote(ModTypeNoVars) orelse t_is_subtype(ModTypeNoVars, DeclType) + of false -> {error, FieldName}; true -> get_mod_record(Left1, Left2, [{FieldName, ModType}|Acc]) end; @@ -4627,6 +4654,23 @@ do_opaque(?union(List) = Type, Opaques, Pred) -> do_opaque(Type, _Opaques, Pred) -> Pred(Type). +is_same_type_name({Mod, Name, Args}, {Mod, Name, Args}) -> true; +is_same_type_name({Mod1, Name1, Args1}, {Mod2, Name2, Args2}) -> + is_same_type_name2(Mod1, Name1, Args1, Mod2, Name2, Args2). + +%% Compatibility. In Erlang/OTP 17 the pre-defined opaque types +%% digraph() and so on can be used, but there are also new types such +%% as digraph:graph() with the exact same meaning. In Erlang/OTP R18.0 +%% all but the last clause can be removed. + +is_same_type_name2(digraph, digraph, [], digraph, graph, []) -> true; +is_same_type_name2(digraph, graph, [], digraph, digraph, []) -> true; +is_same_type_name2(gb_sets, gb_set, [], gb_sets, set, [?any]) -> true; +is_same_type_name2(gb_sets, set, [?any], gb_sets, gb_set, []) -> true; +is_same_type_name2(gb_trees, gb_tree, [], gb_trees, tree, [?any, ?any]) -> true; +is_same_type_name2(gb_trees, tree, [?any, ?any], gb_trees, gb_tree, []) -> true; +is_same_type_name2(_, _, _, _, _, _) -> false. + %% ----------------------------------- %% Set %% diff --git a/lib/kernel/src/application_master.erl b/lib/kernel/src/application_master.erl index 68f78c6eb8..bc15b5a7de 100644 --- a/lib/kernel/src/application_master.erl +++ b/lib/kernel/src/application_master.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -27,7 +27,7 @@ -include("application_master.hrl"). --record(state, {child, appl_data, children = [], procs = 0, gleader}). +-record(state, {child, appl_data, children = [], procs = 0, gleader, req=[]}). %%----------------------------------------------------------------- %% Func: start_link/1 @@ -205,22 +205,25 @@ terminate_loop(Child, State) -> %%----------------------------------------------------------------- %% The Application Master is linked to *all* processes in the group -%% (application). +%% (application). %%----------------------------------------------------------------- handle_msg({get_child, Tag, From}, State) -> - From ! {Tag, get_child_i(State#state.child)}, - State; + get_child_i(State, Tag, From); handle_msg({stop, Tag, From}, State) -> catch terminate(normal, State), From ! {Tag, ok}, exit(normal); +handle_msg({child, Ref, GrandChild, Mod}, #state{req=Reqs0}=State) -> + {value, {_, Tag, From}, Reqs} = lists:keytake(Ref, 1, Reqs0), + From ! {Tag, {GrandChild, Mod}}, + State#state{req=Reqs}; handle_msg(_, State) -> State. - -terminate(Reason, State) -> - terminate_child(State#state.child, State), - kill_children(State#state.children), +terminate(Reason, State = #state{child=Child, children=Children, req=Reqs}) -> + _ = [From ! {Tag, error} || {_, Tag, From} <- Reqs], + terminate_child(Child, State), + kill_children(Children), exit(Reason). @@ -342,8 +345,8 @@ start_supervisor(Type, M, A) -> loop_it(Parent, Child, Mod, AppState) -> receive - {Parent, get_child} -> - Parent ! {self(), Child, Mod}, + {Parent, get_child, Ref} -> + Parent ! {child, Ref, Child, Mod}, loop_it(Parent, Child, Mod, AppState); {Parent, terminate} -> NewAppState = prep_stop(Mod, AppState), @@ -382,10 +385,15 @@ prep_stop(Mod, AppState) -> NewAppState end. -get_child_i(Child) -> - Child ! {self(), get_child}, - receive - {Child, GrandChild, Mod} -> {GrandChild, Mod} +get_child_i(#state{child=Child, req=Reqs}=State, Tag, From) -> + Ref = erlang:make_ref(), + case erlang:is_process_alive(Child) of + true -> + Child ! {self(), get_child, Ref}, + State#state{req=[{Ref, Tag, From}|Reqs]}; + false -> + From ! {Tag, error}, + State end. terminate_child_i(Child, State) -> diff --git a/lib/kernel/src/inet.erl b/lib/kernel/src/inet.erl index 792593246a..41d422d7d4 100644 --- a/lib/kernel/src/inet.erl +++ b/lib/kernel/src/inet.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -597,8 +597,7 @@ getservbyname(Name, Protocol) when is_atom(Name) -> Error -> Error end. --spec ntoa(IpAddress) -> - {ok, Address} | {error, einval} when +-spec ntoa(IpAddress) -> Address | {error, einval} when Address :: string(), IpAddress :: ip_address(). ntoa(Addr) -> diff --git a/lib/kernel/src/kernel.appup.src b/lib/kernel/src/kernel.appup.src index b946c2d1af..f8f4cc1ec2 100644 --- a/lib/kernel/src/kernel.appup.src +++ b/lib/kernel/src/kernel.appup.src @@ -1,7 +1,7 @@ %% -*- erlang -*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2013. All Rights Reserved. +%% Copyright Ericsson AB 1999-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -16,12 +16,10 @@ %% %% %CopyrightEnd% {"%VSN%", - %% Up from - max two major revisions back + %% Up from - max one major revision back [{<<"3\\.0(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R17 - {<<"2\\.16(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R16 - {<<"2\\.15(\\.[0-9]+)*">>,[restart_new_emulator]}],%% R15 - %% Down to - max two major revisions back + {<<"2\\.16(\\.[0-9]+)*">>,[restart_new_emulator]}],%% R16 + %% Down to - max one major revision back [{<<"3\\.0(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R17 - {<<"2\\.16(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R16 - {<<"2\\.15(\\.[0-9]+)*">>,[restart_new_emulator]}] %% R15 + {<<"2\\.16(\\.[0-9]+)*">>,[restart_new_emulator]}] %% R16 }. diff --git a/lib/kernel/test/application_SUITE.erl b/lib/kernel/test/application_SUITE.erl index ff62297f2d..ccb3760309 100644 --- a/lib/kernel/test/application_SUITE.erl +++ b/lib/kernel/test/application_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -36,7 +36,7 @@ -export([config_change/1, persistent_env/1, distr_changed_tc1/1, distr_changed_tc2/1, ensure_started/1, ensure_all_started/1, - shutdown_func/1, do_shutdown/1, shutdown_timeout/1]). + shutdown_func/1, do_shutdown/1, shutdown_timeout/1, shutdown_deadlock/1]). -define(TESTCASE, testcase_name). -define(testcase, ?config(?TESTCASE, Config)). @@ -54,6 +54,7 @@ all() -> script_start, nodedown_start, permit_false_start_local, permit_false_start_dist, get_key, get_env, ensure_all_started, {group, distr_changed}, config_change, shutdown_func, shutdown_timeout, + shutdown_deadlock, persistent_env]. groups() -> @@ -961,7 +962,7 @@ nodedown_start(Conf) when is_list(Conf) -> ensure_started(suite) -> []; ensure_started(doc) -> ["Test application:ensure_started/1."]; -ensure_started(Conf) -> +ensure_started(_Conf) -> {ok, Fd} = file:open("app1.app", [write]), w_app1(Fd), @@ -981,7 +982,7 @@ ensure_started(Conf) -> ensure_all_started(suite) -> []; ensure_all_started(doc) -> ["Test application:ensure_all_started/1-2."]; -ensure_all_started(Conf) -> +ensure_all_started(_Conf) -> {ok, Fd1} = file:open("app1.app", [write]), w_app1(Fd1), @@ -2096,7 +2097,31 @@ shutdown_timeout(Config) when is_list(Config) -> end, ok. +%%%----------------------------------------------------------------- +%%% Provokes a (previous) application shutdown deadlock +%%%----------------------------------------------------------------- +shutdown_deadlock(Config) when is_list(Config) -> + DataDir = ?config(data_dir,Config), + code:add_path(filename:join([DataDir,deadlock])), + %% ok = rpc:call(Cp1, application, start, [sasl]), + ok = application:start(deadlock), + Tester = self(), + application:set_env(deadlock, fail_stop, Tester), + spawn(fun() -> Tester ! {stop, application:stop(deadlock)} end), + receive + {deadlock, Server} -> + spawn(fun() -> + Master = application_controller:get_master(deadlock), + Child = application_master:get_child(Master), + Tester ! {child, Child} + end), + timer:sleep(100), + erlang:display({self(), "Sending Continue", Server}), + Server ! continue + end, + [_|_] = application:which_applications(), + ok. %%----------------------------------------------------------------- diff --git a/lib/kernel/test/application_SUITE_data/deadlock/deadlock.app b/lib/kernel/test/application_SUITE_data/deadlock/deadlock.app index 0c1001bed6..233c7a3f76 100644 --- a/lib/kernel/test/application_SUITE_data/deadlock/deadlock.app +++ b/lib/kernel/test/application_SUITE_data/deadlock/deadlock.app @@ -4,5 +4,5 @@ {applications, [kernel, stdlib, sasl]}, {modules, [deadlock]}, {mod, {deadlock, []}}, - {env, [{fail_start, false}]} + {env, [{fail_start, false}, {fail_stop, false}]} ]}. diff --git a/lib/kernel/test/application_SUITE_data/deadlock/deadlock.erl b/lib/kernel/test/application_SUITE_data/deadlock/deadlock.erl index 5f68bf9078..3ef6105371 100644 --- a/lib/kernel/test/application_SUITE_data/deadlock/deadlock.erl +++ b/lib/kernel/test/application_SUITE_data/deadlock/deadlock.erl @@ -21,7 +21,7 @@ init([sup]) -> {ok, {{one_for_one, 5, 10}, [ { sasl_syslog_dm, {?MODULE, start_link, []}, - permanent, brutal_kill, worker, + permanent, 25000, worker, [deadlock] } ]}}; @@ -32,6 +32,8 @@ init([sup]) -> init([child]) -> case application:get_env(deadlock, fail_start) of {ok, false} -> + process_flag(trap_exit, true), + io:format("~p: Traps exit~n",[?MODULE]), %% we must not fail on the first init, otherwise supervisor %% terminates immediately {ok, []}; @@ -50,6 +52,14 @@ handle_info(_Msg, State) -> {noreply, State}. terminate(_Reason, _State) -> + case application:get_env(deadlock, fail_stop) of + {ok, false} -> ok; + {ok, Tester} -> + Tester ! {deadlock, self()}, + io:format("~p: Waiting in terminate (~p)~n",[?MODULE,Tester]), + receive continue -> ok end + end, + io:format("~p: terminates~n", [?MODULE]), ok. code_change(_OldVsn, State, _Extra) -> diff --git a/lib/kernel/test/kernel_SUITE.erl b/lib/kernel/test/kernel_SUITE.erl index 75f51eb076..2d8123c9c4 100644 --- a/lib/kernel/test/kernel_SUITE.erl +++ b/lib/kernel/test/kernel_SUITE.erl @@ -78,17 +78,18 @@ app_test(Config) when is_list(Config) -> ok. -%% Test that appup allows upgrade from/downgrade to a maximum of two -%% major releases back. +%% Test that appup allows upgrade from/downgrade to a maximum of one +%% major release back. appup_test(_Config) -> - do_appup_tests(create_test_vsns()). + appup_tests(kernel,create_test_vsns(kernel)). -do_appup_tests({[],[]}) -> +appup_tests(_App,{[],[]}) -> {skip,"no previous releases available"}; -do_appup_tests({OkVsns,NokVsns}) -> - application:load(kernel), - {_,_,Vsn} = lists:keyfind(kernel,1,application:loaded_applications()), - AppupFile = filename:join([code:lib_dir(kernel),ebin,"kernel.appup"]), +appup_tests(App,{OkVsns,NokVsns}) -> + application:load(App), + {_,_,Vsn} = lists:keyfind(App,1,application:loaded_applications()), + AppupFileName = atom_to_list(App) ++ ".appup", + AppupFile = filename:join([code:lib_dir(App),ebin,AppupFileName]), {ok,[{Vsn,UpFrom,DownTo}=AppupScript]} = file:consult(AppupFile), ct:log("~p~n",[AppupScript]), ct:log("Testing ok versions: ~p~n",[OkVsns]), @@ -99,13 +100,12 @@ do_appup_tests({OkVsns,NokVsns}) -> check_appup(NokVsns,DownTo,error), ok. -create_test_vsns() -> +create_test_vsns(App) -> This = erlang:system_info(otp_release), FirstMajor = previous_major(This), SecondMajor = previous_major(FirstMajor), - ThirdMajor = previous_major(SecondMajor), - Ok = kernel_vsn([FirstMajor,SecondMajor]), - Nok0 = kernel_vsn([ThirdMajor]), + Ok = app_vsn(App,[FirstMajor]), + Nok0 = app_vsn(App,[SecondMajor]), Nok = case Ok of [Ok1|_] -> [Ok1 ++ ",1" | Nok0]; % illegal @@ -121,18 +121,36 @@ previous_major("r"++Rel) -> previous_major(Rel) -> integer_to_list(list_to_integer(Rel)-1). -kernel_vsn([R|Rs]) -> - case test_server:is_release_available(R) of - true -> - {ok,N} = test_server:start_node(prevrel,peer,[{erl,[{release,R}]}]), - As = rpc:call(N,application,which_applications,[]), - {_,_,KV} = lists:keyfind(kernel,1,As), - test_server:stop_node(N), - [KV|kernel_vsn(Rs)]; +app_vsn(App,[R|Rs]) -> + OldRel = + case test_server:is_release_available(R) of + true -> + {release,R}; + false -> + case ct:get_config({otp_releases,list_to_atom(R)}) of + undefined -> + false; + Prog0 -> + case os:find_executable(Prog0) of + false -> + false; + Prog -> + {prog,Prog} + end + end + end, + case OldRel of false -> - kernel_vsn(Rs) + app_vsn(App,Rs); + _ -> + {ok,N} = test_server:start_node(prevrel,peer,[{erl,[OldRel]}]), + _ = rpc:call(N,application,load,[App]), + As = rpc:call(N,application,loaded_applications,[]), + {_,_,V} = lists:keyfind(App,1,As), + test_server:stop_node(N), + [V|app_vsn(App,Rs)] end; -kernel_vsn([]) -> +app_vsn(_App,[]) -> []. check_appup([Vsn|Vsns],Instrs,Expected) -> diff --git a/lib/mnesia/doc/src/mnesia.xml b/lib/mnesia/doc/src/mnesia.xml index 914ec77721..72e9bd7e8f 100644 --- a/lib/mnesia/doc/src/mnesia.xml +++ b/lib/mnesia/doc/src/mnesia.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2013</year> + <year>1996</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -1204,7 +1204,11 @@ mnesia:create_table(person, <desc> <p>Performs a user initiated dump of the local log file. This is usually not necessary since Mnesia, by default, - manages this automatically.</p> + manages this automatically. + See configuration parameters + <seealso marker="#dump_log_time_threshold">dump_log_time_threshold</seealso> and + <seealso marker="#dump_log_write_threshold">dump_log_write_threshold</seealso>. + </p> </desc> </func> <func> @@ -2208,6 +2212,18 @@ mnesia:create_table(employee, </desc> </func> <func> + <name>sync_log() -> ok | {error, Reason} </name> + <fsummary>Perform a file sync of the local log file.</fsummary> + <desc> + <p>Ensures that the local transaction log file is synced to disk. + On a single node system data written to disk tables, since the last dump, + can be lost in case of a power outage. + See <seealso marker="#dump_log/0">dump_log/0</seealso>. + </p> + </desc> + </func> + + <func> <name>sync_transaction(Fun, [[, Args], Retries]) -> {aborted, Reason} | {atomic, ResultOfFun} </name> <fsummary>Synchronously execute a transaction.</fsummary> <desc> @@ -2445,7 +2461,7 @@ mnesia:create_table(employee, <name>table(Tab [,[Option]]) -> QueryHandle </name> <fsummary>Return a QLC query handle.</fsummary> <desc> - <p> <marker id="qlc_table"></marker> + <p><marker id="qlc_table"></marker> Returns a QLC (Query List Comprehension) query handle, see <seealso marker="stdlib:qlc">qlc(3)</seealso>.The module <c>qlc</c> implements a query language, it can use mnesia tables as sources of data. Calling @@ -3015,6 +3031,7 @@ raise(Name, Amount) -> performed on the original data file. The default is <c>true</c></p> </item> <item> + <marker id=" dump_log_write_threshold"></marker> <p><c>-mnesia dump_log_write_threshold Max</c>, where <c>Max</c> is an integer which specifies the maximum number of writes allowed to the transaction log before a new dump of the log @@ -3022,13 +3039,14 @@ raise(Name, Amount) -> </p> </item> <item> + <marker id=" dump_log_time_threshold"></marker> <p><c>-mnesia dump_log_time_threshold Max</c>, where <c>Max</c> is an integer which specifies the dump log interval in milliseconds. It defaults to 3 minutes. If a dump has not been performed within <c>dump_log_time_threshold</c> milliseconds, then a new dump is performed regardless of how many writes have been - performed. + performed. </p> </item> <item> diff --git a/lib/mnesia/src/mnesia.erl b/lib/mnesia/src/mnesia.erl index 70466d10d7..b7d80c1370 100644 --- a/lib/mnesia/src/mnesia.erl +++ b/lib/mnesia/src/mnesia.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -104,7 +104,8 @@ set_master_nodes/1, set_master_nodes/2, %% Misc admin - dump_log/0, subscribe/1, unsubscribe/1, report_event/1, + dump_log/0, sync_log/0, + subscribe/1, unsubscribe/1, report_event/1, %% Snmp snmp_open_table/2, snmp_close_table/1, @@ -1808,7 +1809,7 @@ do_dirty_rpc(Tab, Node, M, F, Args) -> {badrpc, Reason} -> timer:sleep(20), %% Do not be too eager, and can't use yield on SMP %% Sync with mnesia_monitor - try sys:get_status(mnesia_monitor) catch _:_ -> ok end, + _ = try sys:get_status(mnesia_monitor) catch _:_ -> ok end, case mnesia_controller:call({check_w2r, Node, Tab}) of % Sync NewNode when NewNode =:= Node -> ErrorTag = mnesia_lib:dirty_rpc_error_tag(Reason), @@ -2554,6 +2555,9 @@ set_master_nodes(Tab, Nodes) -> dump_log() -> mnesia_controller:sync_dump_log(user). +sync_log() -> + mnesia_monitor:sync_log(latest_log). + subscribe(What) -> mnesia_subscr:subscribe(self(), What). diff --git a/lib/mnesia/src/mnesia_controller.erl b/lib/mnesia/src/mnesia_controller.erl index 78f7bfa325..a83e55ac62 100644 --- a/lib/mnesia/src/mnesia_controller.erl +++ b/lib/mnesia/src/mnesia_controller.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -198,7 +198,8 @@ sync_dump_log(InitBy) -> call({sync_dump_log, InitBy}). async_dump_log(InitBy) -> - ?SERVER_NAME ! {async_dump_log, InitBy}. + ?SERVER_NAME ! {async_dump_log, InitBy}, + ok. %% Wait for tables to be active %% If needed, we will wait for Mnesia to start @@ -293,10 +294,11 @@ update(Fun) -> mnesia_down(Node) -> - case cast({mnesia_down, Node}) of - {error, _} -> mnesia_monitor:mnesia_down(?SERVER_NAME, Node); - _Pid -> ok + case whereis(?SERVER_NAME) of + undefined -> mnesia_monitor:mnesia_down(?SERVER_NAME, Node); + Pid -> gen_server:cast(Pid, {mnesia_down, Node}) end. + wait_for_schema_commit_lock() -> link(whereis(?SERVER_NAME)), unsafe_call(wait_for_schema_commit_lock). @@ -467,7 +469,7 @@ connect_nodes2(Father, Ns, UserFun) -> process_flag(trap_exit, true), Res = try_merge_schema(New, [], UserFun), Msg = {schema_is_merged, [], late_merge, []}, - multicall([node()|Ns], Msg), + _ = multicall([node()|Ns], Msg), After = val({current, db_nodes}), Father ! {?MODULE, self(), Res, mnesia_lib:intersect(Ns,After)}, unlink(Father), @@ -548,7 +550,7 @@ schema_is_merged() -> cast(Msg) -> case whereis(?SERVER_NAME) of - undefined ->{error, {node_not_running, node()}}; + undefined -> ok; Pid -> gen_server:cast(Pid, Msg) end. @@ -1789,7 +1791,7 @@ sync_and_block_table_whereabouts(Tab, ToNode, RemoteS, AccessMode) when Tab /= s true -> Current -- [ToNode]; false -> Current end, - remote_call(ToNode, block_table, [Tab]), + _ = remote_call(ToNode, block_table, [Tab]), [remote_call(Node, add_active_replica, [Tab, ToNode, RemoteS, AccessMode]) || Node <- [ToNode | Ns]], ok. diff --git a/lib/mnesia/src/mnesia_dumper.erl b/lib/mnesia/src/mnesia_dumper.erl index e2a0aa3bda..14665797a0 100644 --- a/lib/mnesia/src/mnesia_dumper.erl +++ b/lib/mnesia/src/mnesia_dumper.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -85,7 +85,7 @@ adjust_log_writes(DoCast) -> %% Don't care if we lost a few writes mnesia_lib:set_counter(trans_log_writes_left, Max), Diff = Max - Left, - mnesia_lib:incr_counter(trans_log_writes, Diff), + _ = mnesia_lib:incr_counter(trans_log_writes, Diff), global:del_lock(Token, [node()]) end. @@ -451,7 +451,8 @@ disc_delete_table(Tab, Storage) -> Storage == disc_only_copies; Tab == schema -> mnesia_monitor:unsafe_close_dets(Tab), Dat = mnesia_lib:tab2dat(Tab), - file:delete(Dat); + file:delete(Dat), + ok; true -> DclFile = mnesia_lib:tab2dcl(Tab), case get({?MODULE,Tab}) of @@ -466,13 +467,14 @@ disc_delete_table(Tab, Storage) -> file:delete(DcdFile), ok end, - erase({?MODULE, Tab}); + erase({?MODULE, Tab}), + ok; false -> - ignore + ok end. disc_delete_indecies(_Tab, _Cs, Storage) when Storage /= disc_only_copies -> - ignore; + ok; disc_delete_indecies(Tab, Cs, disc_only_copies) -> Indecies = Cs#cstruct.index, mnesia_index:del_transient(Tab, Indecies, disc_only_copies). @@ -522,10 +524,11 @@ insert_op(Tid, _, {op, change_table_copy_type, N, FromS, ToS, TabDef}, InPlace, {disc_copies, ram_copies} when Tab == schema -> mnesia_lib:set(use_dir, false), mnesia_monitor:unsafe_close_dets(Tab), - file:delete(Dat); + ok = file:delete(Dat); {disc_copies, ram_copies} -> - file:delete(Dcl), - file:delete(Dcd); + _ = file:delete(Dcl), + _ = file:delete(Dcd), + ok; {ram_copies, disc_only_copies} -> ok = ensure_rename(Dmp, Dat), true = open_files(Tab, disc_only_copies, InPlace, InitBy), @@ -544,7 +547,8 @@ insert_op(Tid, _, {op, change_table_copy_type, N, FromS, ToS, TabDef}, InPlace, startup -> ignore; _ -> - mnesia_controller:get_disc_copy(Tab) + mnesia_controller:get_disc_copy(Tab), + ok end, disc_delete_table(Tab, disc_only_copies); {disc_copies, disc_only_copies} -> @@ -553,8 +557,9 @@ insert_op(Tid, _, {op, change_table_copy_type, N, FromS, ToS, TabDef}, InPlace, mnesia_schema:ram_delete_table(Tab, FromS), PosList = Cs#cstruct.index, mnesia_index:init_indecies(Tab, disc_only_copies, PosList), - file:delete(Dcl), - file:delete(Dcd); + _ = file:delete(Dcl), + _ = file:delete(Dcd), + ok; {disc_only_copies, disc_copies} -> mnesia_monitor:unsafe_close_dets(Tab), disc_delete_indecies(Tab, Cs, disc_only_copies), diff --git a/lib/mnesia/src/mnesia_event.erl b/lib/mnesia/src/mnesia_event.erl index 35fe2d4035..67ec9d7399 100644 --- a/lib/mnesia/src/mnesia_event.erl +++ b/lib/mnesia/src/mnesia_event.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -63,7 +63,7 @@ handle_event(Event, State) -> %%----------------------------------------------------------------- handle_info(Msg, State) -> - handle_any_event(Msg, State), + {ok, _} = handle_any_event(Msg, State), {ok, State}. %%----------------------------------------------------------------- diff --git a/lib/mnesia/src/mnesia_index.erl b/lib/mnesia/src/mnesia_index.erl index 54db45e3ba..8fef611a48 100644 --- a/lib/mnesia/src/mnesia_index.erl +++ b/lib/mnesia/src/mnesia_index.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -229,7 +229,7 @@ del_transient(Tab, Storage) -> PosList = val({Tab, index}), del_transient(Tab, PosList, Storage). -del_transient(_, [], _) -> done; +del_transient(_, [], _) -> ok; del_transient(Tab, [Pos | Tail], Storage) -> delete_transient_index(Tab, Pos, Storage), del_transient(Tab, Tail, Storage). @@ -237,7 +237,7 @@ del_transient(Tab, [Pos | Tail], Storage) -> delete_transient_index(Tab, Pos, disc_only_copies) -> Tag = {Tab, index, Pos}, mnesia_monitor:unsafe_close_dets(Tag), - file:delete(tab2filename(Tab, Pos)), + _ = file:delete(tab2filename(Tab, Pos)), del_index_info(Tab, Pos), %% Uses val(..) mnesia_lib:unset({Tab, {index, Pos}}); @@ -255,7 +255,7 @@ init_disc_index(_Tab, []) -> init_disc_index(Tab, [Pos | Tail]) when is_integer(Pos) -> Fn = tab2filename(Tab, Pos), IxTag = {Tab, index, Pos}, - file:delete(Fn), + _ = file:delete(Fn), Args = [{file, Fn}, {keypos, 1}, {type, bag}], mnesia_monitor:open_dets(IxTag, Args), Storage = disc_only_copies, diff --git a/lib/mnesia/src/mnesia_lib.erl b/lib/mnesia/src/mnesia_lib.erl index ae6631646c..109e924971 100644 --- a/lib/mnesia/src/mnesia_lib.erl +++ b/lib/mnesia/src/mnesia_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -296,11 +296,7 @@ active_here(Tab) -> not_active_here(Tab) -> not active_here(Tab). -exists(Fname) -> - case file:open(Fname, [raw,read]) of - {ok, F} ->file:close(F), true; - _ -> false - end. +exists(Fname) -> filelib:is_regular(Fname). dir() -> mnesia_monitor:get_env(dir). @@ -596,7 +592,7 @@ coredump(CrashInfo) -> Core = mkcore(CrashInfo), Out = core_file(), important("Writing Mnesia core to file: ~p...~p~n", [Out, CrashInfo]), - file:write_file(Out, Core), + _ = file:write_file(Out, Core), Out. core_file() -> @@ -620,7 +616,7 @@ mkcore(CrashInfo) -> Core = [ CrashInfo, {time, {date(), time()}}, - {self, catch process_info(self())}, + {self, proc_dbg_info(self())}, {nodes, catch rpc:multicall(Nodes, ?MODULE, get_node_number, [])}, {applications, catch lists:sort(application:loaded_applications())}, {flags, catch init:get_arguments()}, @@ -697,7 +693,7 @@ relatives() -> Info = fun(Name) -> case whereis(Name) of undefined -> false; - Pid -> {true, {Name, Pid, catch process_info(Pid)}} + Pid -> {true, {Name, Pid, proc_dbg_info(Pid)}} end end, lists:zf(Info, mnesia:ms()). @@ -706,14 +702,14 @@ workers({workers, Loaders, Senders, Dumper}) -> Info = fun({Pid, {send_table, Tab, _Receiver, _St}}) -> case Pid of undefined -> false; - Pid -> {true, {Pid, Tab, catch process_info(Pid)}} + Pid -> {true, {Pid, Tab, proc_dbg_info(Pid)}} end; ({Pid, What}) when is_pid(Pid) -> - {true, {Pid, What, catch process_info(Pid)}}; + {true, {Pid, What, proc_dbg_info(Pid)}}; ({Name, Pid}) -> case Pid of undefined -> false; - Pid -> {true, {Name, Pid, catch process_info(Pid)}} + Pid -> {true, {Name, Pid, proc_dbg_info(Pid)}} end end, SInfo = lists:zf(Info, Senders), @@ -727,13 +723,21 @@ locking_procs(LockList) when is_list(LockList) -> Pid = Tid#tid.pid, case node(Pid) == node() of true -> - {true, {Pid, catch process_info(Pid)}}; + {true, {Pid, proc_dbg_info(Pid)}}; _ -> false end end, lists:zf(Info, UT). +proc_dbg_info(Pid) -> + try + [process_info(Pid, current_stacktrace)| + process_info(Pid)] + catch _:R -> + [{process_info,crashed,R}] + end. + view() -> Bin = mkcore({crashinfo, {"view only~n", []}}), vcore(Bin). @@ -806,9 +810,9 @@ vcore(File) -> vcore_elem({schema_file, {ok, B}}) -> Fname = "/tmp/schema.DAT", - file:write_file(Fname, B), - dets:view(Fname), - file:delete(Fname); + _ = file:write_file(Fname, B), + _ = dets:view(Fname), + _ = file:delete(Fname); vcore_elem({logfile, {ok, BinList}}) -> Fun = fun({F, Info}) -> @@ -922,7 +926,7 @@ random_time(Retries, _Counter0) -> case get(random_seed) of undefined -> {X, Y, Z} = erlang:now(), %% time() - random:seed(X, Y, Z), + _ = random:seed(X, Y, Z), Time = Dup + random:uniform(MaxIntv), %% dbg_out("---random_test rs ~w max ~w val ~w---~n", [Retries, MaxIntv, Time]), Time; @@ -958,20 +962,17 @@ report_system_event({'EXIT', Reason}, Event) -> unlink(Pid), %% We get an exit signal if server dies - receive - {'EXIT', Pid, _Reason} -> - {error, {node_not_running, node()}} - after 0 -> - gen_event:stop(mnesia_event), - ok + receive {'EXIT', Pid, _Reason} -> ok + after 0 -> gen_event:stop(mnesia_event) end; Error -> Msg = "Mnesia(~p): Cannot report event ~p: ~p (~p)~n", error_logger:format(Msg, [node(), Event, Reason, Error]) - end; + end, + ok; report_system_event(_Res, _Event) -> - ignore. + ok. %% important messages are reported regardless of debug level important(Format, Args) -> @@ -1025,8 +1026,8 @@ copy_file(From, To) -> case file:open(To, [raw, binary, write]) of {ok, T} -> Res = copy_file_loop(F, T, 8000), - file:close(F), - file:close(T), + ok = file:close(F), + ok = file:close(T), Res; {error, Reason} -> {error, Reason} @@ -1038,7 +1039,7 @@ copy_file(From, To) -> copy_file_loop(F, T, ChunkSize) -> case file:read(F, ChunkSize) of {ok, Bin} -> - file:write(T, Bin), + ok = file:write(T, Bin), copy_file_loop(F, T, ChunkSize); eof -> ok; @@ -1205,7 +1206,7 @@ dets_to_ets(Tabname, Tab, File, Type, Rep, Lock) -> {keypos, 2}, {repair, Rep}]) of {ok, Tabname} -> Res = dets:to_ets(Tabname, Tab), - Close(Tabname), + ok = Close(Tabname), trav_ret(Res, Tab); Other -> Other diff --git a/lib/mnesia/src/mnesia_locker.erl b/lib/mnesia/src/mnesia_locker.erl index c4fe370ec1..32cea903c9 100644 --- a/lib/mnesia/src/mnesia_locker.erl +++ b/lib/mnesia/src/mnesia_locker.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -103,7 +103,8 @@ val(Var) -> end. reply(From, R) -> - From ! {?MODULE, node(), R}. + From ! {?MODULE, node(), R}, + true. %% Quiets dialyzer l_request(Node, X, Store) -> {?MODULE, Node} ! {self(), X}, diff --git a/lib/mnesia/src/mnesia_log.erl b/lib/mnesia/src/mnesia_log.erl index 18303869ed..d2fd04a60b 100644 --- a/lib/mnesia/src/mnesia_log.erl +++ b/lib/mnesia/src/mnesia_log.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -393,13 +393,15 @@ unsafe_close_log(Log) -> purge_some_logs() -> mnesia_monitor:unsafe_close_log(latest_log), - file:delete(latest_log_file()), - file:delete(decision_tab_file()). + _ = file:delete(latest_log_file()), + _ = file:delete(decision_tab_file()), + ok. purge_all_logs() -> - file:delete(previous_log_file()), - file:delete(latest_log_file()), - file:delete(decision_tab_file()). + _ = file:delete(previous_log_file()), + _ = file:delete(latest_log_file()), + _ = file:delete(decision_tab_file()), + ok. %% Prepare dump by renaming the open logfile if possible %% Returns a tuple on the following format: {Res, OpenLog} diff --git a/lib/mnesia/src/mnesia_monitor.erl b/lib/mnesia/src/mnesia_monitor.erl index c7b905a1bf..6fc1a394a6 100644 --- a/lib/mnesia/src/mnesia_monitor.erl +++ b/lib/mnesia/src/mnesia_monitor.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -44,6 +44,7 @@ set_env/2, start/0, start_proc/4, + sync_log/1, terminate_proc/3, unsafe_close_dets/1, unsafe_close_log/1, @@ -118,6 +119,9 @@ open_log(Args) -> reopen_log(Name, Fname, Head) -> unsafe_call({reopen_log, Name, Fname, Head}). +sync_log(Name) -> + unsafe_call({sync_log, Name}). + close_log(Name) -> unsafe_call({close_log, Name}). @@ -202,7 +206,7 @@ needs_protocol_conversion(Node) -> cast(Msg) -> case whereis(?MODULE) of - undefined -> ignore; + undefined -> ok; Pid -> gen_server:cast(Pid, Msg) end. @@ -382,6 +386,9 @@ handle_call({reopen_log, Name, Fname, Head}, _From, State) -> {noreply, State} end; +handle_call({sync_log, Name}, _From, State) -> + {reply, disk_log:sync(Name), State}; + handle_call({close_log, Name}, _From, State) -> case disk_log:close(Name) of ok -> @@ -395,7 +402,7 @@ handle_call({close_log, Name}, _From, State) -> end; handle_call({unsafe_close_log, Name}, _From, State) -> - disk_log:close(Name), + _ = disk_log:close(Name), {reply, ok, State}; handle_call({negotiate_protocol, Mon, _Version, _Protocols}, _From, State) @@ -439,7 +446,7 @@ handle_call({negotiate_protocol, Nodes}, From, State) -> end; handle_call(init, _From, State) -> - net_kernel:monitor_nodes(true), + _ = net_kernel:monitor_nodes(true), EarlyNodes = State#state.early_connects, State2 = State#state{tm_started = true}, {reply, EarlyNodes, State2}; diff --git a/lib/mnesia/src/mnesia_recover.erl b/lib/mnesia/src/mnesia_recover.erl index 7aa03bda37..0548a25ebf 100644 --- a/lib/mnesia/src/mnesia_recover.erl +++ b/lib/mnesia/src/mnesia_recover.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/mnesia/src/mnesia_tm.erl b/lib/mnesia/src/mnesia_tm.erl index 17af0cad44..af658150da 100644 --- a/lib/mnesia/src/mnesia_tm.erl +++ b/lib/mnesia/src/mnesia_tm.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -183,7 +183,8 @@ mnesia_down(Node) -> undefined -> mnesia_monitor:mnesia_down(?MODULE, Node); Pid -> - Pid ! {mnesia_down, Node} + Pid ! {mnesia_down, Node}, + ok end. prepare_checkpoint(Nodes, Cp) -> diff --git a/lib/mnesia/test/mnesia_durability_test.erl b/lib/mnesia/test/mnesia_durability_test.erl index 4434abaa1e..366fda7044 100644 --- a/lib/mnesia/test/mnesia_durability_test.erl +++ b/lib/mnesia/test/mnesia_durability_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -48,19 +48,19 @@ groups() -> load_directly_when_all_are_ram_copiesB, {group, late_load_when_all_are_ram_copies_on_ram_nodes}, load_when_last_replica_becomes_available, - load_when_we_have_down_from_all_other_replica_nodes, + load_when_down_from_all_other_replica_nodes, late_load_transforms_into_disc_load, late_load_leads_to_hanging, force_load_when_nobody_intents_to_load, force_load_when_someone_has_decided_to_load, - force_load_when_someone_else_already_has_loaded, + force_load_when_someone_else_has_loaded, force_load_when_we_has_loaded, force_load_on_a_non_local_table, force_load_when_the_table_does_not_exist, {group, load_tables_with_master_tables}]}, {late_load_when_all_are_ram_copies_on_ram_nodes, [], - [late_load_when_all_are_ram_copies_on_ram_nodes1, - late_load_when_all_are_ram_copies_on_ram_nodes2]}, + [late_load_all_ram_cs_ram_nodes1, + late_load_all_ram_cs_ram_nodes2]}, {load_tables_with_master_tables, [], [master_nodes, starting_master_nodes, master_on_non_local_tables, @@ -292,8 +292,8 @@ load_directly_when_all_are_ram_copiesB(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -late_load_when_all_are_ram_copies_on_ram_nodes1(suite) -> []; -late_load_when_all_are_ram_copies_on_ram_nodes1(Config) when is_list(Config) -> +late_load_all_ram_cs_ram_nodes1(suite) -> []; +late_load_all_ram_cs_ram_nodes1(Config) when is_list(Config) -> [N1, N2] = mnesia_test_lib:prepare_test_case([{init_test_case, [mnesia]}, delete_schema, {reload_appls, [mnesia]}], @@ -303,8 +303,8 @@ late_load_when_all_are_ram_copies_on_ram_nodes1(Config) when is_list(Config) -> 2, Config, ?FILE, ?LINE), Res. -late_load_when_all_are_ram_copies_on_ram_nodes2(suite) -> []; -late_load_when_all_are_ram_copies_on_ram_nodes2(Config) when is_list(Config) -> +late_load_all_ram_cs_ram_nodes2(suite) -> []; +late_load_all_ram_cs_ram_nodes2(Config) when is_list(Config) -> [N1, N2, N3] = mnesia_test_lib:prepare_test_case([{init_test_case, [mnesia]}, delete_schema, {reload_appls, [mnesia]}], @@ -439,13 +439,13 @@ load_when_last_replica_becomes_available(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -load_when_we_have_down_from_all_other_replica_nodes(doc) -> +load_when_down_from_all_other_replica_nodes(doc) -> ["The table can be loaded if this node was the last one surviving. ", "Check this by having N1, N2, N3 and a table replicated on all those ", "nodes. Then kill them in the N1, N2, N3 order. Then start N3 and ", "verify that the table is available with correct contents."]; -load_when_we_have_down_from_all_other_replica_nodes(suite) -> []; -load_when_we_have_down_from_all_other_replica_nodes(Config) when is_list(Config) -> +load_when_down_from_all_other_replica_nodes(suite) -> []; +load_when_down_from_all_other_replica_nodes(Config) when is_list(Config) -> [N1, N2, N3] = Nodes = ?acquire_nodes(3, Config), ?match({atomic,ok}, mnesia:create_table(test_rec, @@ -773,14 +773,14 @@ wait_for_signal() -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -force_load_when_someone_else_already_has_loaded(doc) -> +force_load_when_someone_else_has_loaded(doc) -> ["Normal case. Do a force load when somebody else has loaded the table. ", "Start N1, N2, kill in N1, N2 order. Start N2 load the table, start N1 ", "force load. Did it work? (i.e: did N1 load the table from N2 as that", "one is the latest version and it is available on N2)"]; -force_load_when_someone_else_already_has_loaded(suite) -> []; -force_load_when_someone_else_already_has_loaded(Config) when is_list(Config) -> +force_load_when_someone_else_has_loaded(suite) -> []; +force_load_when_someone_else_has_loaded(Config) when is_list(Config) -> [N1, N2] = Nodes = ?acquire_nodes(2, Config), Table = test_rec, Trec1 = #test_rec{key=1,val=111}, diff --git a/lib/mnesia/test/mnesia_nice_coverage_test.erl b/lib/mnesia/test/mnesia_nice_coverage_test.erl index 78eab67b11..4b28ac634f 100644 --- a/lib/mnesia/test/mnesia_nice_coverage_test.erl +++ b/lib/mnesia/test/mnesia_nice_coverage_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2010. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -189,6 +189,7 @@ adm(Attrs, Node1, Node2) -> ?match({atomic, ok}, mnesia:move_table_copy(nice_tab, Node2, Node1)), ?match(yes, mnesia:force_load_table(nice_counter_tab)), + ?match(ok, mnesia:sync_log()), ?match(dumped, mnesia:dump_log()), ok. diff --git a/lib/mnesia/test/mnesia_schema_recovery_test.erl b/lib/mnesia/test/mnesia_schema_recovery_test.erl index 0fe26efd0b..2301b291c2 100644 --- a/lib/mnesia/test/mnesia_schema_recovery_test.erl +++ b/lib/mnesia/test/mnesia_schema_recovery_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2010. All Rights Reserved. +%% Copyright Ericsson AB 1998-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -49,66 +49,69 @@ groups() -> [{interrupted_before_log_dump, [], [interrupted_before_create_ram, interrupted_before_create_disc, - interrupted_before_create_disc_only, + interrupted_before_create_do, interrupted_before_create_nostore, interrupted_before_delete_ram, interrupted_before_delete_disc, - interrupted_before_delete_disc_only, - interrupted_before_add_ram, interrupted_before_add_disc, - interrupted_before_add_disc_only, + interrupted_before_delete_do, + interrupted_before_add_ram, + interrupted_before_add_disc, + interrupted_before_add_do, interrupted_before_add_kill_copier, interrupted_before_move_ram, interrupted_before_move_disc, - interrupted_before_move_disc_only, + interrupted_before_move_do, interrupted_before_move_kill_copier, interrupted_before_delcopy_ram, interrupted_before_delcopy_disc, - interrupted_before_delcopy_disc_only, + interrupted_before_delcopy_do, interrupted_before_delcopy_kill_copier, interrupted_before_addindex_ram, interrupted_before_addindex_disc, - interrupted_before_addindex_disc_only, + interrupted_before_addindex_do, interrupted_before_delindex_ram, interrupted_before_delindex_disc, - interrupted_before_delindex_disc_only, + interrupted_before_delindex_do, interrupted_before_change_type_ram2disc, - interrupted_before_change_type_ram2disc_only, + interrupted_before_change_type_ram2do, interrupted_before_change_type_disc2ram, - interrupted_before_change_type_disc2disc_only, - interrupted_before_change_type_disc_only2ram, - interrupted_before_change_type_disc_only2disc, + interrupted_before_change_type_disc2do, + interrupted_before_change_type_do2ram, + interrupted_before_change_type_do2disc, interrupted_before_change_type_other_node, interrupted_before_change_schema_type]}, {interrupted_after_log_dump, [], [interrupted_after_create_ram, interrupted_after_create_disc, - interrupted_after_create_disc_only, + interrupted_after_create_do, interrupted_after_create_nostore, interrupted_after_delete_ram, interrupted_after_delete_disc, - interrupted_after_delete_disc_only, - interrupted_after_add_ram, interrupted_after_add_disc, - interrupted_after_add_disc_only, + interrupted_after_delete_do, + interrupted_after_add_ram, + interrupted_after_add_disc, + interrupted_after_add_do, interrupted_after_add_kill_copier, - interrupted_after_move_ram, interrupted_after_move_disc, - interrupted_after_move_disc_only, + interrupted_after_move_ram, + interrupted_after_move_disc, + interrupted_after_move_do, interrupted_after_move_kill_copier, interrupted_after_delcopy_ram, interrupted_after_delcopy_disc, - interrupted_after_delcopy_disc_only, + interrupted_after_delcopy_do, interrupted_after_delcopy_kill_copier, interrupted_after_addindex_ram, interrupted_after_addindex_disc, - interrupted_after_addindex_disc_only, + interrupted_after_addindex_do, interrupted_after_delindex_ram, interrupted_after_delindex_disc, - interrupted_after_delindex_disc_only, + interrupted_after_delindex_do, interrupted_after_change_type_ram2disc, - interrupted_after_change_type_ram2disc_only, + interrupted_after_change_type_ram2do, interrupted_after_change_type_disc2ram, - interrupted_after_change_type_disc2disc_only, - interrupted_after_change_type_disc_only2ram, - interrupted_after_change_type_disc_only2disc, + interrupted_after_change_type_disc2do, + interrupted_after_change_type_do2ram, + interrupted_after_change_type_do2disc, interrupted_after_change_type_other_node, interrupted_after_change_schema_type]}]. @@ -128,8 +131,8 @@ interrupted_before_create_disc(Config) when is_list(Config) -> KillAt = {mnesia_dumper, dump_schema_op}, interrupted_create(Config, disc_copies, all, KillAt). -interrupted_before_create_disc_only(suite) -> []; -interrupted_before_create_disc_only(Config) when is_list(Config) -> +interrupted_before_create_do(suite) -> []; +interrupted_before_create_do(Config) when is_list(Config) -> KillAt = {mnesia_dumper, dump_schema_op}, interrupted_create(Config, disc_only_copies, all, KillAt). @@ -148,8 +151,8 @@ interrupted_after_create_disc(Config) when is_list(Config) -> KillAt = {mnesia_dumper, post_dump}, interrupted_create(Config, disc_copies, all, KillAt). -interrupted_after_create_disc_only(suite) -> []; -interrupted_after_create_disc_only(Config) when is_list(Config) -> +interrupted_after_create_do(suite) -> []; +interrupted_after_create_do(Config) when is_list(Config) -> KillAt = {mnesia_dumper, post_dump}, interrupted_create(Config, disc_only_copies, all, KillAt). @@ -204,8 +207,8 @@ interrupted_before_delete_disc(suite) -> []; interrupted_before_delete_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_delete(Config, disc_copies, Debug_Point). -interrupted_before_delete_disc_only(suite) -> []; -interrupted_before_delete_disc_only(Config) when is_list(Config) -> +interrupted_before_delete_do(suite) -> []; +interrupted_before_delete_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_delete(Config, disc_only_copies, Debug_Point). @@ -217,8 +220,8 @@ interrupted_after_delete_disc(suite) -> []; interrupted_after_delete_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_delete(Config, disc_copies, Debug_Point). -interrupted_after_delete_disc_only(suite) -> []; -interrupted_after_delete_disc_only(Config) when is_list(Config) -> +interrupted_after_delete_do(suite) -> []; +interrupted_after_delete_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_delete(Config, disc_only_copies, Debug_Point). @@ -249,8 +252,8 @@ interrupted_before_add_disc(suite) -> []; interrupted_before_add_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_add(Config, disc_copies, kill_reciever, Debug_Point). -interrupted_before_add_disc_only(suite) -> []; -interrupted_before_add_disc_only(Config) when is_list(Config) -> +interrupted_before_add_do(suite) -> []; +interrupted_before_add_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_add(Config, disc_only_copies, kill_reciever, Debug_Point). interrupted_before_add_kill_copier(suite) -> []; @@ -266,8 +269,8 @@ interrupted_after_add_disc(suite) -> []; interrupted_after_add_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_add(Config, disc_copies, kill_reciever, Debug_Point). -interrupted_after_add_disc_only(suite) -> []; -interrupted_after_add_disc_only(Config) when is_list(Config) -> +interrupted_after_add_do(suite) -> []; +interrupted_after_add_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_add(Config, disc_only_copies, kill_reciever, Debug_Point). interrupted_after_add_kill_copier(suite) -> []; @@ -327,8 +330,8 @@ interrupted_before_move_disc(suite) -> []; interrupted_before_move_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_move(Config, disc_copies, kill_reciever, Debug_Point). -interrupted_before_move_disc_only(suite) -> []; -interrupted_before_move_disc_only(Config) when is_list(Config) -> +interrupted_before_move_do(suite) -> []; +interrupted_before_move_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_move(Config, disc_only_copies, kill_reciever, Debug_Point). interrupted_before_move_kill_copier(suite) -> []; @@ -344,8 +347,8 @@ interrupted_after_move_disc(suite) -> []; interrupted_after_move_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_move(Config, disc_copies, kill_reciever, Debug_Point). -interrupted_after_move_disc_only(suite) -> []; -interrupted_after_move_disc_only(Config) when is_list(Config) -> +interrupted_after_move_do(suite) -> []; +interrupted_after_move_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_move(Config, disc_only_copies, kill_reciever, Debug_Point). interrupted_after_move_kill_copier(suite) -> []; @@ -408,8 +411,8 @@ interrupted_before_delcopy_disc(suite) -> []; interrupted_before_delcopy_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_delcopy(Config, disc_copies, kill_reciever, Debug_Point). -interrupted_before_delcopy_disc_only(suite) -> []; -interrupted_before_delcopy_disc_only(Config) when is_list(Config) -> +interrupted_before_delcopy_do(suite) -> []; +interrupted_before_delcopy_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_delcopy(Config, disc_only_copies, kill_reciever, Debug_Point). interrupted_before_delcopy_kill_copier(suite) -> []; @@ -425,8 +428,8 @@ interrupted_after_delcopy_disc(suite) -> []; interrupted_after_delcopy_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_delcopy(Config, disc_copies, kill_reciever, Debug_Point). -interrupted_after_delcopy_disc_only(suite) -> []; -interrupted_after_delcopy_disc_only(Config) when is_list(Config) -> +interrupted_after_delcopy_do(suite) -> []; +interrupted_after_delcopy_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_delcopy(Config, disc_only_copies, kill_reciever, Debug_Point). interrupted_after_delcopy_kill_copier(suite) -> []; @@ -487,8 +490,8 @@ interrupted_before_addindex_disc(suite) -> []; interrupted_before_addindex_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_addindex(Config, disc_copies, Debug_Point). -interrupted_before_addindex_disc_only(suite) -> []; -interrupted_before_addindex_disc_only(Config) when is_list(Config) -> +interrupted_before_addindex_do(suite) -> []; +interrupted_before_addindex_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_addindex(Config, disc_only_copies, Debug_Point). @@ -500,8 +503,8 @@ interrupted_after_addindex_disc(suite) -> []; interrupted_after_addindex_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_addindex(Config, disc_copies, Debug_Point). -interrupted_after_addindex_disc_only(suite) -> []; -interrupted_after_addindex_disc_only(Config) when is_list(Config) -> +interrupted_after_addindex_do(suite) -> []; +interrupted_after_addindex_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_addindex(Config, disc_only_copies, Debug_Point). @@ -555,8 +558,8 @@ interrupted_before_delindex_disc(suite) -> []; interrupted_before_delindex_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_delindex(Config, disc_copies, Debug_Point). -interrupted_before_delindex_disc_only(suite) -> []; -interrupted_before_delindex_disc_only(Config) when is_list(Config) -> +interrupted_before_delindex_do(suite) -> []; +interrupted_before_delindex_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_delindex(Config, disc_only_copies, Debug_Point). @@ -568,8 +571,8 @@ interrupted_after_delindex_disc(suite) -> []; interrupted_after_delindex_disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_delindex(Config, disc_copies, Debug_Point). -interrupted_after_delindex_disc_only(suite) -> []; -interrupted_after_delindex_disc_only(Config) when is_list(Config) -> +interrupted_after_delindex_do(suite) -> []; +interrupted_after_delindex_do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_delindex(Config, disc_only_copies, Debug_Point). @@ -613,24 +616,24 @@ interrupted_before_change_type_ram2disc(suite) -> []; interrupted_before_change_type_ram2disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_change_type(Config, ram_copies, disc_copies, changer, Debug_Point). -interrupted_before_change_type_ram2disc_only(suite) -> []; -interrupted_before_change_type_ram2disc_only(Config) when is_list(Config) -> +interrupted_before_change_type_ram2do(suite) -> []; +interrupted_before_change_type_ram2do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_change_type(Config, ram_copies, disc_only_copies, changer, Debug_Point). interrupted_before_change_type_disc2ram(suite) -> []; interrupted_before_change_type_disc2ram(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_change_type(Config, disc_copies, ram_copies, changer, Debug_Point). -interrupted_before_change_type_disc2disc_only(suite) -> []; -interrupted_before_change_type_disc2disc_only(Config) when is_list(Config) -> +interrupted_before_change_type_disc2do(suite) -> []; +interrupted_before_change_type_disc2do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_change_type(Config, disc_copies, disc_only_copies, changer, Debug_Point). -interrupted_before_change_type_disc_only2ram(suite) -> []; -interrupted_before_change_type_disc_only2ram(Config) when is_list(Config) -> +interrupted_before_change_type_do2ram(suite) -> []; +interrupted_before_change_type_do2ram(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_change_type(Config, disc_only_copies, ram_copies, changer, Debug_Point). -interrupted_before_change_type_disc_only2disc(suite) -> []; -interrupted_before_change_type_disc_only2disc(Config) when is_list(Config) -> +interrupted_before_change_type_do2disc(suite) -> []; +interrupted_before_change_type_do2disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, dump_schema_op}, interrupted_change_type(Config, disc_only_copies, disc_copies, changer, Debug_Point). interrupted_before_change_type_other_node(suite) -> []; @@ -642,24 +645,24 @@ interrupted_after_change_type_ram2disc(suite) -> []; interrupted_after_change_type_ram2disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_change_type(Config, ram_copies, disc_copies, changer, Debug_Point). -interrupted_after_change_type_ram2disc_only(suite) -> []; -interrupted_after_change_type_ram2disc_only(Config) when is_list(Config) -> +interrupted_after_change_type_ram2do(suite) -> []; +interrupted_after_change_type_ram2do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_change_type(Config, ram_copies, disc_only_copies, changer, Debug_Point). interrupted_after_change_type_disc2ram(suite) -> []; interrupted_after_change_type_disc2ram(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_change_type(Config, disc_copies, ram_copies, changer, Debug_Point). -interrupted_after_change_type_disc2disc_only(suite) -> []; -interrupted_after_change_type_disc2disc_only(Config) when is_list(Config) -> +interrupted_after_change_type_disc2do(suite) -> []; +interrupted_after_change_type_disc2do(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_change_type(Config, disc_copies, disc_only_copies, changer, Debug_Point). -interrupted_after_change_type_disc_only2ram(suite) -> []; -interrupted_after_change_type_disc_only2ram(Config) when is_list(Config) -> +interrupted_after_change_type_do2ram(suite) -> []; +interrupted_after_change_type_do2ram(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_change_type(Config, disc_only_copies, ram_copies, changer, Debug_Point). -interrupted_after_change_type_disc_only2disc(suite) -> []; -interrupted_after_change_type_disc_only2disc(Config) when is_list(Config) -> +interrupted_after_change_type_do2disc(suite) -> []; +interrupted_after_change_type_do2disc(Config) when is_list(Config) -> Debug_Point = {mnesia_dumper, post_dump}, interrupted_change_type(Config, disc_only_copies, disc_copies, changer, Debug_Point). interrupted_after_change_type_other_node(suite) -> []; diff --git a/lib/orber/src/Makefile b/lib/orber/src/Makefile index 1c6781e5fd..d96350f4d5 100644 --- a/lib/orber/src/Makefile +++ b/lib/orber/src/Makefile @@ -21,9 +21,6 @@ include $(ERL_TOP)/make/target.mk include $(ERL_TOP)/make/$(TARGET)/otp.mk -# To get hold of SYSTEM_VSN (e.g. R9C). -#include $(ERL_TOP)/erts/vsn.mk - # ---------------------------------------------------- # Application version # ---------------------------------------------------- diff --git a/lib/runtime_tools/src/observer_backend.erl b/lib/runtime_tools/src/observer_backend.erl index 68ef04f20c..fea0854042 100644 --- a/lib/runtime_tools/src/observer_backend.erl +++ b/lib/runtime_tools/src/observer_backend.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2013. All Rights Reserved. +%% Copyright Ericsson AB 2002-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -227,7 +227,9 @@ fetch_stats(Parent, Time) -> fetch_stats_loop(Parent, Time) -> erlang:system_flag(scheduler_wall_time, true), receive - _Msg -> erlang:system_flag(scheduler_wall_time, false) + _Msg -> + %% erlang:system_flag(scheduler_wall_time, false) + ok after Time -> _M = Parent ! {stats, 1, erlang:statistics(scheduler_wall_time), @@ -244,17 +246,6 @@ etop_collect(Collector) -> %% utilization in etop). Next time the flag will be true and then %% there will be a measurement. SchedulerWallTime = erlang:statistics(scheduler_wall_time), - - %% Turn off the flag while collecting data per process etc. - case erlang:system_flag(scheduler_wall_time,false) of - false -> - %% First time and the flag was false - start a monitoring - %% process to set the flag back to false when etop is stopped. - spawn(fun() -> flag_holder_proc(Collector) end); - _ -> - ok - end, - ProcInfo = etop_collect(processes(), []), Collector ! {self(),#etop_info{now = now(), @@ -264,13 +255,22 @@ etop_collect(Collector) -> memi = etop_memi(), procinfo = ProcInfo }}, + + case SchedulerWallTime of + undefined -> + spawn(fun() -> flag_holder_proc(Collector) end); + _ -> + ok + end, + erlang:system_flag(scheduler_wall_time,true). flag_holder_proc(Collector) -> Ref = erlang:monitor(process,Collector), receive {'DOWN',Ref,_,_,_} -> - erlang:system_flag(scheduler_wall_time,false) + %% erlang:system_flag(scheduler_wall_time,false) + ok end. etop_memi() -> diff --git a/lib/sasl/src/sasl.appup.src b/lib/sasl/src/sasl.appup.src index 0c4d80a74f..e789853eea 100644 --- a/lib/sasl/src/sasl.appup.src +++ b/lib/sasl/src/sasl.appup.src @@ -1,7 +1,7 @@ %% -*- erlang -*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2013. All Rights Reserved. +%% Copyright Ericsson AB 1999-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -16,12 +16,10 @@ %% %% %CopyrightEnd% {"%VSN%", - %% Up from - max two major revisions back + %% Up from - max one major revision back [{<<"2\\.4(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R17 - {<<"2\\.3(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R16 - {<<"2\\.2(\\.[0-9]+)*">>,[restart_new_emulator]}], %% R15 - %% Down to - max two major revisions back + {<<"2\\.3(\\.[0-9]+)*">>,[restart_new_emulator]}], %% R16 + %% Down to - max one major revision back [{<<"2\\.4(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R17 - {<<"2\\.3(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R16 - {<<"2\\.2(\\.[0-9]+)*">>,[restart_new_emulator]}] %% R15 + {<<"2\\.3(\\.[0-9]+)*">>,[restart_new_emulator]}] %% R16 }. diff --git a/lib/sasl/src/systools_rc.erl b/lib/sasl/src/systools_rc.erl index 54c327410d..76f753c3d0 100644 --- a/lib/sasl/src/systools_rc.erl +++ b/lib/sasl/src/systools_rc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -365,14 +365,22 @@ translate_application_instrs(Script, Appls, PreAppls) -> case lists:keysearch(Appl, #application.name, Appls) of {value, PostApplication} -> PostMods = PostApplication#application.modules, + Type = PostApplication#application.type, + Apply = + case Type of + none -> []; + load -> [{apply, {application, load, + [Appl]}}]; + _ -> [{apply, {application, start, + [Appl, Type]}}] + end, [{apply, {application, stop, [Appl]}}] ++ [{remove, {M, brutal_purge, brutal_purge}} || M <- PreMods] ++ [{purge, PreMods}] ++ [{add_module, M, []} || M <- PostMods] ++ - [{apply, {application, start, - [Appl, permanent]}}]; + Apply; false -> throw({error, {no_such_application, Appl}}) end; diff --git a/lib/sasl/test/sasl_SUITE.erl b/lib/sasl/test/sasl_SUITE.erl index e799230338..ad63a31cbd 100644 --- a/lib/sasl/test/sasl_SUITE.erl +++ b/lib/sasl/test/sasl_SUITE.erl @@ -22,7 +22,6 @@ %% Default timetrap timeout (set in init_per_testcase). -define(default_timeout, ?t:minutes(1)). --define(application, sasl). %% Test server specific exports -export([all/0,groups/0,init_per_group/2,end_per_group/2]). @@ -58,17 +57,18 @@ app_test(Config) when is_list(Config) -> ?t:app_test(sasl, allow), ok. -%% Test that appup allows upgrade from/downgrade to a maximum of two -%% major releases back. +%% Test that appup allows upgrade from/downgrade to a maximum of one +%% major release back. appup_test(_Config) -> - do_appup_tests(create_test_vsns()). + appup_tests(sasl,create_test_vsns(sasl)). -do_appup_tests({[],[]}) -> +appup_tests(_App,{[],[]}) -> {skip,"no previous releases available"}; -do_appup_tests({OkVsns,NokVsns}) -> - application:load(sasl), - {_,_,Vsn} = lists:keyfind(sasl,1,application:loaded_applications()), - AppupFile = filename:join([code:lib_dir(sasl),ebin,"sasl.appup"]), +appup_tests(App,{OkVsns,NokVsns}) -> + application:load(App), + {_,_,Vsn} = lists:keyfind(App,1,application:loaded_applications()), + AppupFileName = atom_to_list(App) ++ ".appup", + AppupFile = filename:join([code:lib_dir(App),ebin,AppupFileName]), {ok,[{Vsn,UpFrom,DownTo}=AppupScript]} = file:consult(AppupFile), ct:log("~p~n",[AppupScript]), ct:log("Testing ok versions: ~p~n",[OkVsns]), @@ -79,13 +79,12 @@ do_appup_tests({OkVsns,NokVsns}) -> check_appup(NokVsns,DownTo,error), ok. -create_test_vsns() -> +create_test_vsns(App) -> This = erlang:system_info(otp_release), FirstMajor = previous_major(This), SecondMajor = previous_major(FirstMajor), - ThirdMajor = previous_major(SecondMajor), - Ok = sasl_vsn([FirstMajor,SecondMajor]), - Nok0 = sasl_vsn([ThirdMajor]), + Ok = app_vsn(App,[FirstMajor]), + Nok0 = app_vsn(App,[SecondMajor]), Nok = case Ok of [Ok1|_] -> [Ok1 ++ ",1" | Nok0]; % illegal @@ -101,19 +100,36 @@ previous_major("r"++Rel) -> previous_major(Rel) -> integer_to_list(list_to_integer(Rel)-1). -sasl_vsn([R|Rs]) -> - case test_server:is_release_available(R) of - true -> - {ok,N} = test_server:start_node(prevrel,peer,[{erl,[{release,R}]}]), - _ = rpc:call(N,application,load,[sasl]), +app_vsn(App,[R|Rs]) -> + OldRel = + case test_server:is_release_available(R) of + true -> + {release,R}; + false -> + case ct:get_config({otp_releases,list_to_atom(R)}) of + undefined -> + false; + Prog0 -> + case os:find_executable(Prog0) of + false -> + false; + Prog -> + {prog,Prog} + end + end + end, + case OldRel of + false -> + app_vsn(App,Rs); + _ -> + {ok,N} = test_server:start_node(prevrel,peer,[{erl,[OldRel]}]), + _ = rpc:call(N,application,load,[App]), As = rpc:call(N,application,loaded_applications,[]), - {_,_,V} = lists:keyfind(sasl,1,As), + {_,_,V} = lists:keyfind(App,1,As), test_server:stop_node(N), - [V|sasl_vsn(Rs)]; - false -> - sasl_vsn(Rs) + [V|app_vsn(App,Rs)] end; -sasl_vsn([]) -> +app_vsn(_App,[]) -> []. check_appup([Vsn|Vsns],Instrs,Expected) -> diff --git a/lib/sasl/test/systools_rc_SUITE.erl b/lib/sasl/test/systools_rc_SUITE.erl index 0cb6e63cf3..5efab7c028 100644 --- a/lib/sasl/test/systools_rc_SUITE.erl +++ b/lib/sasl/test/systools_rc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2012. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -436,25 +436,19 @@ translate(Config) when is_list(Config) -> translate_app(Config) when is_list(Config) -> PreApps = - [#application{name = test, - description = "TEST", - vsn = "1.0", - modules = [foo,bar,baz], - regs = [], - mod = {sasl, []}}, + [Test = #application{name = test, + description = "TEST", + vsn = "1.0", + modules = [foo,bar,baz], + regs = [], + mod = {sasl, []}}, #application{name = pelle, description = "PELLE", vsn = "1.0", modules = [pelle, kalle], regs = [], mod = {pelle, []}}], - Apps = - [#application{name = test, - description = "TEST", - vsn = "1.0", - modules = [foo,bar,baz], - regs = [], - mod = {sasl, []}}], + Apps = [Test], %% Simple translation (1) Up1 = [{add_module, foo}, {add_module, bar}], @@ -475,6 +469,56 @@ translate_app(Config) when is_list(Config) -> {load,{baz,brutal_purge,brutal_purge}}, {apply,{application,start,[test,permanent]}}] = X2, + %% Translate add_application with different restart types + %% permanent + Up2_1 = [{add_application, test, permanent}], + {ok, X2_1} = systools_rc:translate_scripts([Up2_1], Apps, []), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}, + {apply,{application,start,[test,permanent]}}] = X2_1, + + %% transient + Up2_2 = [{add_application, test, transient}], + {ok, X2_2} = systools_rc:translate_scripts([Up2_2], Apps, []), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}, + {apply,{application,start,[test,transient]}}] = X2_2, + + %% temporary + Up2_3 = [{add_application, test, temporary}], + {ok, X2_3} = systools_rc:translate_scripts([Up2_3], Apps, []), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}, + {apply,{application,start,[test,temporary]}}] = X2_3, + + %% load + Up2_4 = [{add_application, test, load}], + {ok, X2_4} = systools_rc:translate_scripts([Up2_4], Apps, []), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}, + {apply,{application,load,[test]}}] = X2_4, + + %% none + Up2_5 = [{add_application, test, none}], + {ok, X2_5} = systools_rc:translate_scripts([Up2_5], Apps, []), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}] = X2_5, + %% Simple translation (3) Up3 = [{remove_application, pelle}], {ok, X3} = systools_rc:translate_scripts([Up3], Apps, PreApps), @@ -484,6 +528,102 @@ translate_app(Config) when is_list(Config) -> {remove,{kalle,brutal_purge,brutal_purge}}, {purge,[pelle,kalle]}, {apply,{application,unload,[pelle]}}] = X3, + + %% Simple translation (4) + Up4 = [{restart_application, test}], + {ok, X4} = systools_rc:translate_scripts([Up4], Apps, PreApps), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {apply,{application,stop,[test]}}, + {remove,{foo,brutal_purge,brutal_purge}}, + {remove,{bar,brutal_purge,brutal_purge}}, + {remove,{baz,brutal_purge,brutal_purge}}, + {purge,[foo,bar,baz]}, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}, + {apply,{application,start,[test,permanent]}}] = X4, + + %% Translate restart_application with different restart types + %% permanent + {ok, X4_1} = systools_rc:translate_scripts([Up4], + [Test#application{type=permanent}], + [Test]), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {apply,{application,stop,[test]}}, + {remove,{foo,brutal_purge,brutal_purge}}, + {remove,{bar,brutal_purge,brutal_purge}}, + {remove,{baz,brutal_purge,brutal_purge}}, + {purge,[foo,bar,baz]}, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}, + {apply,{application,start,[test,permanent]}}] = X4_1, + + %% transient + {ok, X4_2} = systools_rc:translate_scripts([Up4], + [Test#application{type=transient}], + [Test]), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {apply,{application,stop,[test]}}, + {remove,{foo,brutal_purge,brutal_purge}}, + {remove,{bar,brutal_purge,brutal_purge}}, + {remove,{baz,brutal_purge,brutal_purge}}, + {purge,[foo,bar,baz]}, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}, + {apply,{application,start,[test,transient]}}] = X4_2, + + %% temporary + {ok, X4_3} = systools_rc:translate_scripts([Up4], + [Test#application{type=temporary}], + [Test]), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {apply,{application,stop,[test]}}, + {remove,{foo,brutal_purge,brutal_purge}}, + {remove,{bar,brutal_purge,brutal_purge}}, + {remove,{baz,brutal_purge,brutal_purge}}, + {purge,[foo,bar,baz]}, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}, + {apply,{application,start,[test,temporary]}}] = X4_3, + + %% load + {ok, X4_4} = systools_rc:translate_scripts([Up4], + [Test#application{type=load}], + [Test]), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {apply,{application,stop,[test]}}, + {remove,{foo,brutal_purge,brutal_purge}}, + {remove,{bar,brutal_purge,brutal_purge}}, + {remove,{baz,brutal_purge,brutal_purge}}, + {purge,[foo,bar,baz]}, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}, + {apply,{application,load,[test]}}] = X4_4, + + %% none + {ok, X4_5} = systools_rc:translate_scripts([Up4], + [Test#application{type=none}], + [Test]), + [{load_object_code,{test,"1.0",[foo,bar,baz]}}, + point_of_no_return, + {apply,{application,stop,[test]}}, + {remove,{foo,brutal_purge,brutal_purge}}, + {remove,{bar,brutal_purge,brutal_purge}}, + {remove,{baz,brutal_purge,brutal_purge}}, + {purge,[foo,bar,baz]}, + {load,{foo,brutal_purge,brutal_purge}}, + {load,{bar,brutal_purge,brutal_purge}}, + {load,{baz,brutal_purge,brutal_purge}}] = X4_5, + ok. diff --git a/lib/sasl/test/test_lib.hrl b/lib/sasl/test/test_lib.hrl index 37aa44c198..c8a4e92f24 100644 --- a/lib/sasl/test/test_lib.hrl +++ b/lib/sasl/test/test_lib.hrl @@ -1,3 +1,3 @@ -define(ertsvsn,"4.4"). --define(kernelvsn,"2.15.3"). --define(stdlibvsn,"1.18.3"). +-define(kernelvsn,"2.16.4"). +-define(stdlibvsn,"1.19.4"). diff --git a/lib/stdlib/doc/src/ets.xml b/lib/stdlib/doc/src/ets.xml index 21cf8e4149..3df24bf688 100644 --- a/lib/stdlib/doc/src/ets.xml +++ b/lib/stdlib/doc/src/ets.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2013</year> + <year>1996</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -171,6 +171,10 @@ <p>Returns a list of all tables at the node. Named tables are given by their names, unnamed tables are given by their table identifiers.</p> + <p>There is no guarantee of consistency in the returned list. Tables created + or deleted by other processes "during" the ets:all() call may or may + not be included in the list. Only tables created/deleted <em>before</em> + ets:all() is called are guaranteed to be included/excluded.</p> </desc> </func> <func> diff --git a/lib/stdlib/src/edlin_expand.erl b/lib/stdlib/src/edlin_expand.erl index 0033010dce..a2b4663219 100644 --- a/lib/stdlib/src/edlin_expand.erl +++ b/lib/stdlib/src/edlin_expand.erl @@ -73,7 +73,7 @@ to_atom(Str) -> error end. -match(Prefix, Alts, Extra) -> +match(Prefix, Alts, Extra0) -> Len = length(Prefix), Matches = lists:sort( [{S, A} || {H, A} <- Alts, @@ -89,13 +89,11 @@ match(Prefix, Alts, Extra) -> {yes, Remain, []} end; {complete, Str} -> - {_, Arity} = lists:keyfind(Str, 1, Matches), - case {Arity, Extra} of - {0, "("} -> - {yes, nthtail(Len, Str) ++ "()", []}; - _ -> - {yes, nthtail(Len, Str) ++ Extra, []} - end; + Extra = case {Extra0,Matches} of + {"(",[{Str,0}]} -> "()"; + {_,_} -> Extra0 + end, + {yes, nthtail(Len, Str) ++ Extra, []}; no -> {no, [], []} end. diff --git a/lib/stdlib/src/epp.erl b/lib/stdlib/src/epp.erl index 4fd302e612..dc25b6b9f4 100644 --- a/lib/stdlib/src/epp.erl +++ b/lib/stdlib/src/epp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -640,7 +640,7 @@ leave_file(From, St) -> Ms = dict:store({atom,'FILE'}, {none,[{string,CurrLoc,OldName2}]}, St#epp.macs), - NextSt = OldSt#epp{sstk=Sts,macs=Ms}, + NextSt = OldSt#epp{sstk=Sts,macs=Ms,uses=St#epp.uses}, enter_file_reply(From, OldName, CurrLoc, CurrLoc), case OldName2 =:= OldName of true -> diff --git a/lib/stdlib/src/stdlib.appup.src b/lib/stdlib/src/stdlib.appup.src index 749a9a4201..22eefb2514 100644 --- a/lib/stdlib/src/stdlib.appup.src +++ b/lib/stdlib/src/stdlib.appup.src @@ -1,7 +1,7 @@ %% -*- erlang -*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2013. All Rights Reserved. +%% Copyright Ericsson AB 1999-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -16,12 +16,10 @@ %% %% %CopyrightEnd% {"%VSN%", - %% Up from - max two major revisions back + %% Up from - max one major revision back [{<<"2\\.0(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R17 - {<<"1\\.19(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R16 - {<<"1\\.18(\\.[0-9]+)*">>,[restart_new_emulator]}],%% R15 - %% Down to - max two major revisions back + {<<"1\\.19(\\.[0-9]+)*">>,[restart_new_emulator]}],%% R16 + %% Down to - max one major revision back [{<<"2\\.0(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R17 - {<<"1\\.19(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R16 - {<<"1\\.18(\\.[0-9]+)*">>,[restart_new_emulator]}] %% R15 + {<<"1\\.19(\\.[0-9]+)*">>,[restart_new_emulator]}] %% R16 }. diff --git a/lib/stdlib/test/edlin_expand_SUITE.erl b/lib/stdlib/test/edlin_expand_SUITE.erl index 2c1c5acf5a..43c980e994 100644 --- a/lib/stdlib/test/edlin_expand_SUITE.erl +++ b/lib/stdlib/test/edlin_expand_SUITE.erl @@ -26,11 +26,11 @@ -include_lib("test_server/include/test_server.hrl"). -% Default timetrap timeout (set in init_per_testcase). +%% Default timetrap timeout (set in init_per_testcase). -define(default_timeout, ?t:minutes(1)). init_per_testcase(_Case, Config) -> - ?line Dog = ?t:timetrap(?default_timeout), + Dog = ?t:timetrap(?default_timeout), [{watchdog, Dog} | Config]. end_per_testcase(_Case, Config) -> Dog = ?config(watchdog, Config), @@ -67,23 +67,21 @@ normal(doc) -> normal(suite) -> []; normal(Config) when is_list(Config) -> - ?line {module,expand_test} = c:l(expand_test), - % These tests might fail if another module with the prefix "expand_" happens - % to also be loaded. - ?line {yes, "test:", []} = edlin_expand:expand(lists:reverse("expand_")), - ?line {no, [], []} = edlin_expand:expand(lists:reverse("expandXX_")), - ?line {no,[], - [{"a_fun_name",1}, - {"a_less_fun_name",1}, - {"b_comes_after_a",1}, - {"expand0arity_entirely",0}, - {"module_info",0}, - {"module_info",1}]} = edlin_expand:expand(lists:reverse("expand_test:")), - ?line {yes,[],[{"a_fun_name",1}, - {"a_less_fun_name",1}]} = edlin_expand:expand( - lists:reverse("expand_test:a_")), - ?line {yes,"arity_entirely()",[]} = edlin_expand:expand( - lists:reverse("expand_test:expand0")), + {module,expand_test} = c:l(expand_test), + %% These tests might fail if another module with the prefix + %% "expand_" happens to also be loaded. + {yes, "test:", []} = do_expand("expand_"), + {no, [], []} = do_expand("expandXX_"), + {no,[], + [{"a_fun_name",1}, + {"a_less_fun_name",1}, + {"b_comes_after_a",1}, + {"expand0arity_entirely",0}, + {"module_info",0}, + {"module_info",1}]} = do_expand("expand_test:"), + {yes,[],[{"a_fun_name",1}, + {"a_less_fun_name",1}]} = do_expand("expand_test:a_"), + {yes,"arity_entirely()",[]} = do_expand("expand_test:expand0"), ok. quoted_fun(doc) -> @@ -91,38 +89,35 @@ quoted_fun(doc) -> quoted_fun(suite) -> []; quoted_fun(Config) when is_list(Config) -> - ?line {module,expand_test} = c:l(expand_test), - ?line {module,expand_test1} = c:l(expand_test1), + {module,expand_test} = c:l(expand_test), + {module,expand_test1} = c:l(expand_test1), %% should be no colon after test this time - ?line {yes, "test", []} = edlin_expand:expand(lists:reverse("expand_")), - ?line {no, [], []} = edlin_expand:expand(lists:reverse("expandXX_")), - ?line {no,[],[{"'#weird-fun-name'",1}, - {"'Quoted_fun_name'",0}, - {"'Quoted_fun_too'",0}, - {"a_fun_name",1}, - {"a_less_fun_name",1}, - {"b_comes_after_a",1}, - {"module_info",0}, - {"module_info",1}]} = edlin_expand:expand( - lists:reverse("expand_test1:")), - ?line {yes,"_",[]} = edlin_expand:expand( - lists:reverse("expand_test1:a")), - ?line {yes,[],[{"a_fun_name",1}, - {"a_less_fun_name",1}]} = edlin_expand:expand( - lists:reverse("expand_test1:a_")), - ?line {yes,[], - [{"'#weird-fun-name'",1}, + {yes, "test", []} = do_expand("expand_"), + {no, [], []} = do_expand("expandXX_"), + {no,[],[{"'#weird-fun-name'",1}, {"'Quoted_fun_name'",0}, - {"'Quoted_fun_too'",0}]} = edlin_expand:expand( - lists:reverse("expand_test1:'")), - ?line {yes,"uoted_fun_",[]} = edlin_expand:expand( - lists:reverse("expand_test1:'Q")), - ?line {yes,[], - [{"'Quoted_fun_name'",0}, - {"'Quoted_fun_too'",0}]} = edlin_expand:expand( - lists:reverse("expand_test1:'Quoted_fun_")), - ?line {yes,"weird-fun-name'(",[]} = edlin_expand:expand( - lists:reverse("expand_test1:'#")), + {"'Quoted_fun_too'",0}, + {"a_fun_name",1}, + {"a_less_fun_name",1}, + {"b_comes_after_a",1}, + {"module_info",0}, + {"module_info",1}]} = do_expand("expand_test1:"), + {yes,"_",[]} = do_expand("expand_test1:a"), + {yes,[],[{"a_fun_name",1}, + {"a_less_fun_name",1}]} = do_expand("expand_test1:a_"), + {yes,[], + [{"'#weird-fun-name'",1}, + {"'Quoted_fun_name'",0}, + {"'Quoted_fun_too'",0}]} = do_expand("expand_test1:'"), + {yes,"uoted_fun_",[]} = do_expand("expand_test1:'Q"), + {yes,[], + [{"'Quoted_fun_name'",0}, + {"'Quoted_fun_too'",0}]} = do_expand("expand_test1:'Quoted_fun_"), + {yes,"weird-fun-name'(",[]} = do_expand("expand_test1:'#"), + + %% Since there is a module_info/1 as well as a module_info/0 + %% there should not be a closing parenthesis added. + {yes,"(",[]} = do_expand("expand_test:module_info"), ok. quoted_module(doc) -> @@ -130,51 +125,46 @@ quoted_module(doc) -> quoted_module(suite) -> []; quoted_module(Config) when is_list(Config) -> - ?line {module,'ExpandTestCaps'} = c:l('ExpandTestCaps'), - ?line {yes, "Caps':", []} = edlin_expand:expand(lists:reverse("'ExpandTest")), - ?line {no,[], - [{"a_fun_name",1}, - {"a_less_fun_name",1}, - {"b_comes_after_a",1}, - {"module_info",0}, - {"module_info",1}]} = edlin_expand:expand(lists:reverse("'ExpandTestCaps':")), - ?line {yes,[],[{"a_fun_name",1}, - {"a_less_fun_name",1}]} = edlin_expand:expand( - lists:reverse("'ExpandTestCaps':a_")), + {module,'ExpandTestCaps'} = c:l('ExpandTestCaps'), + {yes, "Caps':", []} = do_expand("'ExpandTest"), + {no,[], + [{"a_fun_name",1}, + {"a_less_fun_name",1}, + {"b_comes_after_a",1}, + {"module_info",0}, + {"module_info",1}]} = do_expand("'ExpandTestCaps':"), + {yes,[],[{"a_fun_name",1}, + {"a_less_fun_name",1}]} = do_expand("'ExpandTestCaps':a_"), ok. quoted_both(suite) -> []; quoted_both(Config) when is_list(Config) -> - ?line {module,'ExpandTestCaps'} = c:l('ExpandTestCaps'), - ?line {module,'ExpandTestCaps1'} = c:l('ExpandTestCaps1'), + {module,'ExpandTestCaps'} = c:l('ExpandTestCaps'), + {module,'ExpandTestCaps1'} = c:l('ExpandTestCaps1'), %% should be no colon (or quote) after test this time - ?line {yes, "Caps", []} = edlin_expand:expand(lists:reverse("'ExpandTest")), - ?line {no,[],[{"'#weird-fun-name'",0}, - {"'Quoted_fun_name'",0}, - {"'Quoted_fun_too'",0}, - {"a_fun_name",1}, - {"a_less_fun_name",1}, - {"b_comes_after_a",1}, - {"module_info",0}, - {"module_info",1}]} = edlin_expand:expand( - lists:reverse("'ExpandTestCaps1':")), - ?line {yes,"_",[]} = edlin_expand:expand( - lists:reverse("'ExpandTestCaps1':a")), - ?line {yes,[],[{"a_fun_name",1}, - {"a_less_fun_name",1}]} = edlin_expand:expand( - lists:reverse("'ExpandTestCaps1':a_")), - ?line {yes,[], - [{"'#weird-fun-name'",0}, + {yes, "Caps", []} = do_expand("'ExpandTest"), + {no,[],[{"'#weird-fun-name'",0}, {"'Quoted_fun_name'",0}, - {"'Quoted_fun_too'",0}]} = edlin_expand:expand( - lists:reverse("'ExpandTestCaps1':'")), - ?line {yes,"uoted_fun_",[]} = edlin_expand:expand( - lists:reverse("'ExpandTestCaps1':'Q")), - ?line {yes,[], - [{"'Quoted_fun_name'",0}, - {"'Quoted_fun_too'",0}]} = edlin_expand:expand( - lists:reverse("'ExpandTestCaps1':'Quoted_fun_")), - ?line {yes,"weird-fun-name'()",[]} = edlin_expand:expand( - lists:reverse("'ExpandTestCaps1':'#")), + {"'Quoted_fun_too'",0}, + {"a_fun_name",1}, + {"a_less_fun_name",1}, + {"b_comes_after_a",1}, + {"module_info",0}, + {"module_info",1}]} = do_expand("'ExpandTestCaps1':"), + {yes,"_",[]} = do_expand("'ExpandTestCaps1':a"), + {yes,[],[{"a_fun_name",1}, + {"a_less_fun_name",1}]} = do_expand("'ExpandTestCaps1':a_"), + {yes,[], + [{"'#weird-fun-name'",0}, + {"'Quoted_fun_name'",0}, + {"'Quoted_fun_too'",0}]} = do_expand("'ExpandTestCaps1':'"), + {yes,"uoted_fun_",[]} = do_expand("'ExpandTestCaps1':'Q"), + {yes,[], + [{"'Quoted_fun_name'",0}, + {"'Quoted_fun_too'",0}]} = do_expand("'ExpandTestCaps1':'Quoted_fun_"), + {yes,"weird-fun-name'()",[]} = do_expand("'ExpandTestCaps1':'#"), ok. + +do_expand(String) -> + edlin_expand:expand(lists:reverse(String)). diff --git a/lib/stdlib/test/epp_SUITE.erl b/lib/stdlib/test/epp_SUITE.erl index 0cbdf76270..0b4726c07a 100644 --- a/lib/stdlib/test/epp_SUITE.erl +++ b/lib/stdlib/test/epp_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2013. All Rights Reserved. +%% Copyright Ericsson AB 1998-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -25,7 +25,8 @@ variable_1/1, otp_4870/1, otp_4871/1, otp_5362/1, pmod/1, not_circular/1, skip_header/1, otp_6277/1, otp_7702/1, otp_8130/1, overload_mac/1, otp_8388/1, otp_8470/1, otp_8503/1, - otp_8562/1, otp_8665/1, otp_8911/1, otp_10302/1, otp_10820/1]). + otp_8562/1, otp_8665/1, otp_8911/1, otp_10302/1, otp_10820/1, + otp_11728/1]). -export([epp_parse_erl_form/2]). @@ -67,7 +68,7 @@ all() -> {group, variable}, otp_4870, otp_4871, otp_5362, pmod, not_circular, skip_header, otp_6277, otp_7702, otp_8130, overload_mac, otp_8388, otp_8470, otp_8503, otp_8562, - otp_8665, otp_8911, otp_10302, otp_10820]. + otp_8665, otp_8911, otp_10302, otp_10820, otp_11728]. groups() -> [{upcase_mac, [], [upcase_mac_1, upcase_mac_2]}, @@ -1387,6 +1388,31 @@ do_otp_10820(File, C, PC) -> true = test_server:stop_node(Node), ok. +otp_11728(doc) -> + ["OTP-11728. Bugfix circular macro."]; +otp_11728(suite) -> + []; +otp_11728(Config) when is_list(Config) -> + Dir = ?config(priv_dir, Config), + H = <<"-define(MACRO,[[]++?MACRO]).">>, + HrlFile = filename:join(Dir, "otp_11728.hrl"), + ok = file:write_file(HrlFile, H), + C = <<"-module(otp_11728). + -compile(export_all). + + -include(\"otp_11728.hrl\"). + + function_name()-> + A=?MACRO, % line 7 + ok">>, + ErlFile = filename:join(Dir, "otp_11728.erl"), + ok = file:write_file(ErlFile, C), + {ok, L} = epp:parse_file(ErlFile, [Dir], []), + true = lists:member({error,{7,epp,{circular,'MACRO',none}}}, L), + _ = file:delete(HrlFile), + _ = file:delete(ErlFile), + ok. + check(Config, Tests) -> eval_tests(Config, fun check_test/2, Tests). diff --git a/lib/stdlib/test/ets_SUITE.erl b/lib/stdlib/test/ets_SUITE.erl index 82c3e7ecaf..8dc8b2c291 100644 --- a/lib/stdlib/test/ets_SUITE.erl +++ b/lib/stdlib/test/ets_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -75,6 +75,7 @@ -export([otp_9932/1]). -export([otp_9423/1]). -export([otp_10182/1]). +-export([ets_all/1]). -export([memory_check_summary/1]). -export([init_per_testcase/2, end_per_testcase/2]). @@ -151,6 +152,7 @@ all() -> otp_10182, otp_9932, otp_9423, + ets_all, memory_check_summary]. % MUST BE LAST @@ -5565,7 +5567,19 @@ otp_10182(Config) when is_list(Config) -> ets:delete(Db), In = Out. - +%% Test that ets:all include/exclude tables that we know are created/deleted +ets_all(Config) when is_list(Config) -> + Pids = [spawn_link(fun() -> ets_all_run() end) || _ <- [1,2]], + receive after 3*1000 -> ok end, + [begin unlink(P), exit(P,kill) end || P <- Pids], + ok. + +ets_all_run() -> + Table = ets:new(undefined, []), + true = lists:member(Table, ets:all()), + ets:delete(Table), + false = lists:member(Table, ets:all()), + ets_all_run(). % diff --git a/lib/stdlib/test/stdlib_SUITE.erl b/lib/stdlib/test/stdlib_SUITE.erl index 8fff5e2e05..b0d6913636 100644 --- a/lib/stdlib/test/stdlib_SUITE.erl +++ b/lib/stdlib/test/stdlib_SUITE.erl @@ -25,7 +25,6 @@ % Default timetrap timeout (set in init_per_testcase). -define(default_timeout, ?t:minutes(1)). --define(application, stdlib). % Test server specific exports -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, @@ -78,17 +77,18 @@ app_test(Config) when is_list(Config) -> ?t:app_test(stdlib), ok. -%% Test that appup allows upgrade from/downgrade to a maximum of two -%% major releases back. +%% Test that appup allows upgrade from/downgrade to a maximum of one +%% major release back. appup_test(_Config) -> - do_appup_tests(create_test_vsns()). + appup_tests(stdlib,create_test_vsns(stdlib)). -do_appup_tests({[],[]}) -> +appup_tests(_App,{[],[]}) -> {skip,"no previous releases available"}; -do_appup_tests({OkVsns,NokVsns}) -> - application:load(stdlib), - {_,_,Vsn} = lists:keyfind(stdlib,1,application:loaded_applications()), - AppupFile = filename:join([code:lib_dir(stdlib),ebin,"stdlib.appup"]), +appup_tests(App,{OkVsns,NokVsns}) -> + application:load(App), + {_,_,Vsn} = lists:keyfind(App,1,application:loaded_applications()), + AppupFileName = atom_to_list(App) ++ ".appup", + AppupFile = filename:join([code:lib_dir(App),ebin,AppupFileName]), {ok,[{Vsn,UpFrom,DownTo}=AppupScript]} = file:consult(AppupFile), ct:log("~p~n",[AppupScript]), ct:log("Testing ok versions: ~p~n",[OkVsns]), @@ -99,13 +99,12 @@ do_appup_tests({OkVsns,NokVsns}) -> check_appup(NokVsns,DownTo,error), ok. -create_test_vsns() -> +create_test_vsns(App) -> This = erlang:system_info(otp_release), FirstMajor = previous_major(This), SecondMajor = previous_major(FirstMajor), - ThirdMajor = previous_major(SecondMajor), - Ok = stdlib_vsn([FirstMajor,SecondMajor]), - Nok0 = stdlib_vsn([ThirdMajor]), + Ok = app_vsn(App,[FirstMajor]), + Nok0 = app_vsn(App,[SecondMajor]), Nok = case Ok of [Ok1|_] -> [Ok1 ++ ",1" | Nok0]; % illegal @@ -121,18 +120,36 @@ previous_major("r"++Rel) -> previous_major(Rel) -> integer_to_list(list_to_integer(Rel)-1). -stdlib_vsn([R|Rs]) -> - case test_server:is_release_available(R) of - true -> - {ok,N} = test_server:start_node(prevrel,peer,[{erl,[{release,R}]}]), - As = rpc:call(N,application,which_applications,[]), - {_,_,KV} = lists:keyfind(stdlib,1,As), - test_server:stop_node(N), - [KV|stdlib_vsn(Rs)]; +app_vsn(App,[R|Rs]) -> + OldRel = + case test_server:is_release_available(R) of + true -> + {release,R}; + false -> + case ct:get_config({otp_releases,list_to_atom(R)}) of + undefined -> + false; + Prog0 -> + case os:find_executable(Prog0) of + false -> + false; + Prog -> + {prog,Prog} + end + end + end, + case OldRel of false -> - stdlib_vsn(Rs) + app_vsn(App,Rs); + _ -> + {ok,N} = test_server:start_node(prevrel,peer,[{erl,[OldRel]}]), + _ = rpc:call(N,application,load,[App]), + As = rpc:call(N,application,loaded_applications,[]), + {_,_,V} = lists:keyfind(App,1,As), + test_server:stop_node(N), + [V|app_vsn(App,Rs)] end; -stdlib_vsn([]) -> +app_vsn(_App,[]) -> []. check_appup([Vsn|Vsns],Instrs,Expected) -> diff --git a/lib/syntax_tools/src/erl_syntax.erl b/lib/syntax_tools/src/erl_syntax.erl index 2f4c9ac309..ed9decd4b5 100644 --- a/lib/syntax_tools/src/erl_syntax.erl +++ b/lib/syntax_tools/src/erl_syntax.erl @@ -453,33 +453,38 @@ %% </tr><tr> %% <td>list_comp</td> %% <td>macro</td> +%% <td>map_expr</td> +%% <td>map_field_assoc</td> +%% </tr><tr> +%% <td>map_field_exact</td> %% <td>match_expr</td> %% <td>module_qualifier</td> -%% </tr><tr> %% <td>named_fun_expr</td> +%% </tr><tr> %% <td>nil</td> %% <td>operator</td> %% <td>parentheses</td> -%% </tr><tr> %% <td>prefix_expr</td> +%% </tr><tr> %% <td>receive_expr</td> %% <td>record_access</td> %% <td>record_expr</td> -%% </tr><tr> %% <td>record_field</td> +%% </tr><tr> %% <td>record_index_expr</td> %% <td>rule</td> %% <td>size_qualifier</td> -%% </tr><tr> %% <td>string</td> +%% </tr><tr> %% <td>text</td> %% <td>try_expr</td> %% <td>tuple</td> -%% </tr><tr> %% <td>underscore</td> +%% </tr><tr> %% <td>variable</td> %% <td>warning_marker</td> %% <td></td> +%% <td></td> %% </tr> %% </table></center> %% @@ -520,6 +525,9 @@ %% @see list/2 %% @see list_comp/2 %% @see macro/2 +%% @see map_expr/2 +%% @see map_field_assoc/2 +%% @see map_field_exact/2 %% @see match_expr/2 %% @see module_qualifier/2 %% @see named_fun_expr/2 diff --git a/lib/test_server/src/configure.in b/lib/test_server/src/configure.in index 067663feb4..cd723bcd4d 100644 --- a/lib/test_server/src/configure.in +++ b/lib/test_server/src/configure.in @@ -2,7 +2,7 @@ dnl Process this file with autoconf to produce a configure script for Erlang. dnl dnl %CopyrightBegin% dnl -dnl Copyright Ericsson AB 1997-2013. All Rights Reserved. +dnl Copyright Ericsson AB 1997-2014. All Rights Reserved. dnl dnl The contents of this file are subject to the Erlang Public License, dnl Version 1.1, (the "License"); you may not use this file except in @@ -170,7 +170,30 @@ case $system in fi SHLIB_EXTRACT_ALL="" ;; - *-netbsd*|*-freebsd*|*-openbsd*|*-dragonfly*) + *-openbsd*) + # Not available on all versions: check for include file. + AC_CHECK_HEADER(dlfcn.h, [ + SHLIB_CFLAGS="-fpic" + SHLIB_LD="${CC}" + SHLIB_LDFLAGS="$LDFLAGS -shared" + SHLIB_SUFFIX=".so" + if test X${enable_m64_build} = Xyes; then + AC_MSG_ERROR(don't know how to link 64-bit dynamic drivers) + fi + if test X${enable_m32_build} = Xyes; then + AC_MSG_ERROR(don't know how to link 32-bit dynamic drivers) + fi + ], [ + # No dynamic loading. + SHLIB_CFLAGS="" + SHLIB_LD="ld" + SHLIB_LDFLAGS="" + SHLIB_SUFFIX="" + AC_MSG_ERROR(don't know how to compile and link dynamic drivers) + ]) + SHLIB_EXTRACT_ALL="" + ;; + *-netbsd*|*-freebsd*|*-dragonfly*) # Not available on all versions: check for include file. AC_CHECK_HEADER(dlfcn.h, [ SHLIB_CFLAGS="-fpic" diff --git a/lib/test_server/src/test_server_ctrl.erl b/lib/test_server/src/test_server_ctrl.erl index e24d6ceacb..dcf905db24 100644 --- a/lib/test_server/src/test_server_ctrl.erl +++ b/lib/test_server/src/test_server_ctrl.erl @@ -487,6 +487,7 @@ init([]) -> ok end, test_server_sup:cleanup_crash_dumps(), + test_server_sup:util_start(), State = #state{jobs=[],finish=false}, TI0 = test_server:init_target_info(), TargetHost = test_server_sup:hoststr(), @@ -1055,6 +1056,7 @@ handle_info(_, State) -> %% test suites (if any) and any possible remainting slave node terminate(_Reason, State) -> + test_server_sup:util_stop(), case State#state.trc of false -> ok; Sock -> test_server_node:stop_tracer_node(Sock) @@ -1725,30 +1727,33 @@ make_html_link(LinkName, Target, Explanation) -> ok = write_html_file(LinkName, H). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% start_minor_log_file(Mod, Func) -> AbsName +%% start_minor_log_file(Mod, Func, ParallelTC) -> AbsName %% Mod = atom() %% Func = atom() +%% ParallelTC = bool() %% AbsName = string() %% %% Create a minor log file for the test case Mod,Func,Args. The log file -%% will be stored in the log directory under the name <Mod>.<Func>.log. -%% Some header info will also be inserted into the log file. +%% will be stored in the log directory under the name <Mod>.<Func>.html. +%% Some header info will also be inserted into the log file. If the test +%% case runs in a parallel group, then to avoid clashing file names if the +%% case is executed more than once, the name <Mod>.<Func>.<Timestamp>.html +%% is used. -start_minor_log_file(Mod, Func) -> +start_minor_log_file(Mod, Func, ParallelTC) -> MFA = {Mod,Func,1}, LogDir = get(test_server_log_dir_base), Name0 = lists:flatten(io_lib:format("~w.~w~ts", [Mod,Func,?html_ext])), Name = downcase(Name0), AbsName = filename:join(LogDir, Name), - case file:read_file_info(AbsName) of - {error,_} -> %% normal case, unique name + case (ParallelTC orelse (element(1,file:read_file_info(AbsName))==ok)) of + false -> %% normal case, unique name start_minor_log_file1(Mod, Func, LogDir, AbsName, MFA); - {ok,_} -> %% special case, duplicate names - {_,S,Us} = now(), + true -> %% special case, duplicate names + Tag = test_server_sup:unique_name(), Name1_0 = - lists:flatten(io_lib:format("~w.~w.~w.~w~ts", [Mod,Func,S, - trunc(Us/1000), - ?html_ext])), + lists:flatten(io_lib:format("~w.~w.~ts~ts", [Mod,Func,Tag, + ?html_ext])), Name1 = downcase(Name1_0), AbsName1 = filename:join(LogDir, Name1), start_minor_log_file1(Mod, Func, LogDir, AbsName1, MFA) @@ -3631,7 +3636,7 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, TSDir = get(test_server_dir), print(major, "=case ~w:~w", [Mod, Func]), - MinorName = start_minor_log_file(Mod, Func), + MinorName = start_minor_log_file(Mod, Func, self() /= Main), print(minor, "<a name=\"top\"></a>", [], internal_raw), MinorBase = filename:basename(MinorName), print(major, "=logfile ~ts", [filename:basename(MinorName)]), diff --git a/lib/test_server/src/test_server_sup.erl b/lib/test_server/src/test_server_sup.erl index 377aa21018..70ead42b1b 100644 --- a/lib/test_server/src/test_server_sup.erl +++ b/lib/test_server/src/test_server_sup.erl @@ -29,10 +29,12 @@ hostatom/0, hostatom/1, hoststr/0, hoststr/1, framework_call/2,framework_call/3,framework_call/4, format_loc/1, + util_start/0, util_stop/0, unique_name/0, call_trace/1]). -include("test_server_internal.hrl"). -define(crash_dump_tar,"crash_dumps.tar.gz"). -define(src_listing_ext, ".src.html"). +-record(util_state, {starter, latest_name}). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% timetrap(Timeout,Scale,Pid) -> Handle @@ -583,6 +585,69 @@ downcase([], Result) -> lists:reverse(Result). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% util_start() -> ok +%% +%% Start local utility process +util_start() -> + Starter = self(), + case whereis(?MODULE) of + undefined -> + spawn_link(fun() -> + register(?MODULE, self()), + util_loop(#util_state{starter=Starter}) + end); + _Pid -> + ok + end. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% util_stop() -> ok +%% +%% Stop local utility process +util_stop() -> + try (?MODULE ! {self(),stop}) of + _ -> + receive {?MODULE,stopped} -> ok + after 5000 -> exit(whereis(?MODULE), kill) + end + catch + _:_ -> + ok + end. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% unique_name() -> string() +%% +unique_name() -> + ?MODULE ! {self(),unique_name}, + receive {?MODULE,Name} -> Name + after 5000 -> exit({?MODULE,no_util_process}) + end. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% util_loop(State) -> ok +%% +util_loop(State) -> + receive + {From,unique_name} -> + {_,S,Us} = now(), + Ms = trunc(Us/1000), + Name = lists:flatten(io_lib:format("~w.~w", [S,Ms])), + if Name == State#util_state.latest_name -> + timer:sleep(1), + self() ! {From,unique_name}, + util_loop(State); + true -> + From ! {?MODULE,Name}, + util_loop(State#util_state{latest_name = Name}) + end; + {From,stop} -> + catch unlink(State#util_state.starter), + From ! {?MODULE,stopped}, + ok + end. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% call_trace(TraceSpecFile) -> ok %% %% Read terms on format {m,Mod} | {f,Mod,Func} diff --git a/lib/tools/emacs/erlang.el b/lib/tools/emacs/erlang.el index 3a868f1300..f007f780eb 100644 --- a/lib/tools/emacs/erlang.el +++ b/lib/tools/emacs/erlang.el @@ -664,6 +664,7 @@ resulting regexp is surrounded by \\_< and \\_>." "is_function" "is_integer" "is_list" + "is_map" "is_number" "is_pid" "is_port" @@ -715,7 +716,8 @@ resulting regexp is surrounded by \\_< and \\_>." "pos_integer" "string" "term" - "timeout") + "timeout" + "map") "Erlang type specs types")) (eval-and-compile @@ -772,6 +774,7 @@ resulting regexp is surrounded by \\_< and \\_>." "is_function" "is_integer" "is_list" + "is_map" "is_number" "is_pid" "is_port" @@ -791,6 +794,7 @@ resulting regexp is surrounded by \\_< and \\_>." "list_to_tuple" "load_module" "make_ref" + "map_size" "max" "min" "module_loaded" diff --git a/lib/tools/test/eprof_SUITE.erl b/lib/tools/test/eprof_SUITE.erl index 26685a6a84..1227d5b841 100644 --- a/lib/tools/test/eprof_SUITE.erl +++ b/lib/tools/test/eprof_SUITE.erl @@ -127,6 +127,14 @@ basic(Config) when is_list(Config) -> ok. basic_option(Config) when is_list(Config) -> + %% Eprof is not supported on native-compile code. + case lists:module_info(native_addresses) of + [] -> basic_option_1(Config); + [_|_] -> {skip,"lists is native-compiled"} + end. + +basic_option_1(Config) -> + %% load eprof_test and change directory {ok, OldCurDir} = file:get_cwd(), diff --git a/make/emd2exml.in b/make/emd2exml.in index 5bfe89894e..48473349d0 100644 --- a/make/emd2exml.in +++ b/make/emd2exml.in @@ -334,6 +334,8 @@ text(Line) -> text("%ERTS-VSN%" ++ Cs, Acc) -> text(Cs, ["@ERTS_VSN@"|Acc]); +text("%OTP-VSN%" ++ Cs, Acc) -> + text(Cs, ["@OTP_VSN@"|Acc]); text("%OTP-REL%" ++ Cs, Acc) -> text(Cs, ["@OTP_REL@"|Acc]); @@ -357,6 +359,8 @@ put_text(#state{c = CTag, emphasis = EmTag} = S, Line) -> put_text(S, "%ERTS-VSN%"++Cs, CTag, EmTag, Acc) -> put_text(S, Cs, CTag, EmTag, ["@ERTS_VSN@"|Acc]); +put_text(S, "%OTP-VSN%"++Cs, CTag, EmTag, Acc) -> + put_text(S, Cs, CTag, EmTag, ["@OTP_VSN@"|Acc]); put_text(S, "%OTP-REL%"++Cs, CTag, EmTag, Acc) -> put_text(S, Cs, CTag, EmTag, ["@OTP_REL@"|Acc]); @@ -560,6 +564,8 @@ code(Line) -> code("%ERTS-VSN%" ++ Cs, Acc) -> code(Cs, ["@ERTS_VSN@"|Acc]); +code("%OTP-VSN%" ++ Cs, Acc) -> + code(Cs, ["@OTP_VSN@"|Acc]); code("%OTP-REL%" ++ Cs, Acc) -> code(Cs, ["@OTP_REL@"|Acc]); diff --git a/make/otp.mk.in b/make/otp.mk.in index 785926b997..a89bc37820 100644 --- a/make/otp.mk.in +++ b/make/otp.mk.in @@ -30,6 +30,13 @@ include $(ERL_TOP)/make/output.mk # ---------------------------------------------------- +# Version +# ---------------------------------------------------- + +OTP_VERSION = @OTP_VERSION@ +SYSTEM_VSN = @SYSTEM_VSN@ + +# ---------------------------------------------------- # Cross Compiling # ---------------------------------------------------- CROSS_COMPILING = @CROSS_COMPILING@ diff --git a/make/otp_default_release_path.mk b/make/otp_default_release_path.mk new file mode 100644 index 0000000000..932bbb2f6d --- /dev/null +++ b/make/otp_default_release_path.mk @@ -0,0 +1,24 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2014. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# + +# +# Where to release to by default +# + +OTP_DEFAULT_RELEASE_PATH="$(ERL_TOP)/release/$(TARGET)" diff --git a/make/otp_release_targets.mk b/make/otp_release_targets.mk index b6afcd1c8b..fcac2ff695 100644 --- a/make/otp_release_targets.mk +++ b/make/otp_release_targets.mk @@ -17,6 +17,8 @@ # %CopyrightEnd% # +include $(ERL_TOP)/make/otp_default_release_path.mk + # ---------------------------------------------------- # Targets for the new documentation support # ---------------------------------------------------- @@ -137,7 +139,7 @@ endif ifeq ($(TESTROOT),) release release_docs release_tests release_html: - $(MAKE) $(MFLAGS) RELEASE_PATH="$(ERL_TOP)/release/$(TARGET)" \ + $(MAKE) $(MFLAGS) RELEASE_PATH=$(OTP_DEFAULT_RELEASE_PATH) \ $(TARGET_MAKEFILE) $@_spec else @@ -1216,8 +1216,12 @@ do_patch_app () echo "No OTP $otp_major_vsn installation in $target_dir" 1>&2 exit 1 fi + shift + otp_version=`cat "$target_dir/OTP_VERSION"` || { echo "Not able to read $target_dir/OTP_VERSION" 1>&2; exit 1; } + { echo "$otp_version" | sed "s|^\([^\*]*\)\**|\1\*\*|g" > $target_dir/OTP_VERSION; } 2>/dev/null || { echo "Not able to update $target_dir/OTP_VERSION" 1>&2; exit 1; } + # Build all applications to target for app in "$@"; do if [ "$app" = "erts" ] && [ -d $ERL_TOP/$app ]; then @@ -1430,9 +1434,9 @@ cd $ERL_TOP determine_version_controller -# Unset ERL_FLAGS and ERL_OTP<Major-VSN>_FLAGS during bootstrap to +# Unset ERL_FLAGS and ERL_OTP<OTP Release>_FLAGS during bootstrap to # prevent potential problems -otp_major_vsn=`cat erts/vsn.mk | grep SYSTEM_VSN | sed "s|SYSTEM_VSN[^=]*=[^0-9]*\([0-9]*\).*|\1|g"` +otp_major_vsn=`cat OTP_VERSION | sed "s|\([0-9]*\).*|\1|"` erl_otp_flags="ERL_OTP${otp_major_vsn}_FLAGS" unset ERL_FLAGS unset ${erl_otp_flags} diff --git a/system/doc/installation_guide/otp_version.xml b/system/doc/installation_guide/otp_version.xml new file mode 100644 index 0000000000..dcba2d1702 --- /dev/null +++ b/system/doc/installation_guide/otp_version.xml @@ -0,0 +1,71 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2014</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>OTP version</title> + <prepared></prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date>2014-02-19</date> + <rev></rev> + <file>otp_version.xml</file> + </header> + <p>As of OTP release 17, the OTP release number corresponds to + the major OTP version number. In the normal case, the OTP version + number will be constructed as + <c><Major>.<Minor>.<Patch></c>. However more + dot separated parts may exist. If all parts less significant than + <c>Minor</c> equals <c>0</c>, they are omitted when printing the + version. Release candidates have an <c>-rc<N></c> suffix. The + OTP version as a concept was introduced in OTP 17.</p> + <p>OTP of a specific version is a set of applications of + specific versions. The application versions identified by + an OTP version corresponds to application versions that + have been tested together by the Erlang/OTP team at Ericsson AB. + An OTP system can however be put together with applications from + different OTP versions. Such a combination of application versions + has not been tested by the Erlang/OTP team. It is therefore + <em>always preferred to use OTP applications from one single OTP + version</em>.</p> + <p>In an OTP source code tree as well as in an installed OTP + development system, the OTP version can be read from the text + file <c>OTP_VERSION</c> in the OTP installation root directory + (<seealso marker="kernel:code#root_dir/0"><c>code:root_dir()</c></seealso>).</p> + <p>If the version read from the <c>OTP_VERSION</c> file in a + development system has a <c>**</c> suffix, the system has been + patched using the <c>$ERL_TOP/otp_build patch_app</c> tool. In + this case, the system consists of application versions from + multiple OTP versions. The version preceding the <c>**</c> + suffix corresponds to the OTP version of the base system that + has been patched.</p> + <p>On a target system (see the + <seealso marker="doc/system_principles:create_target">system principles</seealso> + documentation) no <c>OTP_VERSION</c> file will exist. This since + one easily can create a target system where it is hard to even + determine the base OTP version.</p> + <p>Note that if a development system is updated by other means + than <c>$ERL_TOP/otp_build patch_app</c>, the <c>OTP_VERSION</c> + file may identify wrong OTP version.</p> +</chapter> + diff --git a/system/doc/installation_guide/part.xml b/system/doc/installation_guide/part.xml index 19808fd165..150df39512 100644 --- a/system/doc/installation_guide/part.xml +++ b/system/doc/installation_guide/part.xml @@ -31,6 +31,7 @@ <description> <p>How to install Erlang/OTP on UNIX or Windows.</p> </description> + <xi:include href="otp_version.xml"/> <xi:include href="install-binary.xml"/> <xi:include href="verification.xml"/> <xi:include href="INSTALL.xml"/> diff --git a/system/doc/installation_guide/xmlfiles.mk b/system/doc/installation_guide/xmlfiles.mk index 3995c607af..245491ab94 100644 --- a/system/doc/installation_guide/xmlfiles.mk +++ b/system/doc/installation_guide/xmlfiles.mk @@ -17,6 +17,7 @@ # %CopyrightEnd% # INST_GUIDE_CHAPTER_FILES = \ + otp_version.xml \ install-binary.xml \ verification.xml \ INSTALL.xml \ diff --git a/system/doc/reference_manual/code_loading.xml b/system/doc/reference_manual/code_loading.xml index 23871dfa83..b5b5704df5 100644 --- a/system/doc/reference_manual/code_loading.xml +++ b/system/doc/reference_manual/code_loading.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2013</year> + <year>2003</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -121,9 +121,22 @@ loop() -> <title>Running a function when a module is loaded</title> <warning> - <p>We recommend that the feature described in this section is to - be used only for loading NIF libraries. For other usages this - feature should be considered experimental.</p> + <p>The <c>on_load</c> feature should be considered experimental + as there are a number of known weak points in current semantics + which therefore might also change in future releases:</p> + <list> + <item><p>Doing external call in on_load to the module itself + leads to deadlock.</p></item> + <item><p>At module upgrade, other processes calling the module + get suspended waiting for on_load to finish. This can be very bad + for applications with demands on realtime characteristics.</p></item> + <item><p>At module upgrade, no rollback is done if the on_load function fails. + The system will be left in a bad limbo state without any working + and reachable instance of the module.</p></item> + </list> + <p>The problems with module upgrade described above could be fixed in future + releases by changing the behaviour to not make the module reachable until + after the on_load function has successfully returned.</p> </warning> <p>The <c>-on_load()</c> directive names a function that should diff --git a/system/doc/top/Makefile b/system/doc/top/Makefile index 20ef9fe781..79fd875d68 100644 --- a/system/doc/top/Makefile +++ b/system/doc/top/Makefile @@ -198,7 +198,7 @@ $(GLOSSARY): $(GLOSSARY_SCRIPT) #-------------------------------------------------------------------------- -PR.template: PR.template.src $(ERL_TOP)/erts/vsn.mk +PR.template: PR.template.src $(ERL_TOP)/make/$(TARGET)/otp.mk sed -e 's;%VSN%;$(VSN);' \ -e 's;%SYSTEM_VSN%;$(SYSTEM_VSN);' \ $< > $@ |