diff options
63 files changed, 1924 insertions, 713 deletions
diff --git a/OTP_VERSION b/OTP_VERSION index d79aa6586c..7f4e82686d 100644 --- a/OTP_VERSION +++ b/OTP_VERSION @@ -1 +1 @@ -21.0.6 +21.0.8 diff --git a/erts/configure.in b/erts/configure.in index 9e8ffd5ec5..2563a29d08 100644 --- a/erts/configure.in +++ b/erts/configure.in @@ -581,7 +581,8 @@ if test "X$PROFILE_INSTR_GENERATE" = "Xtrue"; then PROFILE_INSTR_USE=false]) rm -f default.profdata fi], - []) + [], + [AC_MSG_NOTICE([Disabling PGO when cross-compiling])]) rm -f *.profraw CFLAGS=$saved_CFLAGS; fi diff --git a/erts/doc/src/escript.xml b/erts/doc/src/escript.xml index 9b0d42185e..be1664b39f 100644 --- a/erts/doc/src/escript.xml +++ b/erts/doc/src/escript.xml @@ -4,7 +4,7 @@ <comref> <header> <copyright> - <year>2007</year><year>2017</year> + <year>2007</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -155,9 +155,12 @@ io:setopts([{encoding, unicode}])</code> for example:</p> <pre> halt(1).</pre> - <p>To retrieve the pathname of the script, call - <seealso marker="#script_name_0"> - <c>escript:script_name()</c></seealso> from your script + <p> + To retrieve the pathname of the script, call + <seealso marker="#script_name-0"> + <c>escript:script_name()</c> + </seealso> + from your script (the pathname is usually, but not always, absolute).</p> <p>If the file contains source code (as in the example above), it is processed by the @@ -229,6 +232,7 @@ $ <input>escript factorial.beam 5</input> factorial 5 = 120 $ <input>escript factorial.zip 5</input> factorial 5 = 120</pre> + <marker id="create-2"/> </desc> </func> @@ -259,7 +263,7 @@ factorial 5 = 120</pre> zip:create_option()</seealso>]</v> </type> <desc> - <p><marker id="create_2"></marker> + <p> Creates an escript from a list of sections. The sections can be specified in any order. An escript begins with an optional <c>Header</c> followed by a mandatory <c>Body</c>. If @@ -344,6 +348,7 @@ ok {{2010,3,2},{0,59,22}}, 54,1,0,0,0,0,0}, <<"%% demo.erl\n-module(demo).\n-export([main/1]).\n\n%% Demo\nmain(_Arg"...>>}]}</pre> + <marker id="extract-2"/> </desc> </func> @@ -368,9 +373,11 @@ ok <v>SourceCode = BeamCode = ZipArchive = binary()</v> </type> <desc> - <p><marker id="extract_2"></marker> - Parses an escript and extracts its sections. This is the reverse - of <seealso marker="#create_2"><c>create/2</c></seealso>.</p> + <p> + Parses an escript and extracts its sections. + This is the reverse of + <seealso marker="#create-2"><c>create/2</c></seealso>. + </p> <p>All sections are returned even if they do not exist in the escript. If a particular section happens to have the same value as the default value, the extracted value is set to the @@ -393,6 +400,7 @@ ok {ok,[{{archive,<<80,75,3,4,20,0,0,0,8,0,118,7,98,60,105, 152,61,93,107,0,0,0,118,0,...>>} {emu_args,undefined}]}</pre> + <marker id="script_name-0"/> </desc> </func> @@ -403,7 +411,7 @@ ok <v>File = filename()</v> </type> <desc> - <p><marker id="script_name_0"></marker> + <p> Returns the name of the escript that is executed. If the function is invoked outside the context of an escript, the behavior is undefined.</p> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index 6b4db48f61..653a917823 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -31,6 +31,60 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 10.0.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A process could get stuck in an infinite rescheduling + loop between normal and dirty schedulers. This bug was + introduced in ERTS version 10.0.</p> + <p> + Thanks to Maxim Fedorov for finding and fixing this + issue.</p> + <p> + Own Id: OTP-15275 Aux Id: PR-1943 </p> + </item> + <item> + <p> + Garbage collection of a distribution entry could cause an + emulator crash if <c>net_kernel</c> had not brought + previous connection attempts on it down properly.</p> + <p> + Own Id: OTP-15279 Aux Id: ERIERL-226 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.0.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A race between termination of a process and resume of the + same process via <c>erlang:resume_process/1</c> could + cause the VM to crash. This bug was introduced in erts + version 10.0 (OTP 21.0).</p> + <p> + Own Id: OTP-15237</p> + </item> + <item> + <p> + When tracing on <c>running</c>, <c>in</c> trace events + could be lost when a process was rescheduled between a + dirty and a normal scheduler.</p> + <p> + Own Id: OTP-15269 Aux Id: ERL-713 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 10.0.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/emulator/beam/dist.c b/erts/emulator/beam/dist.c index 146c00b07d..ceb89a6910 100644 --- a/erts/emulator/beam/dist.c +++ b/erts/emulator/beam/dist.c @@ -3615,6 +3615,12 @@ static Sint abort_connection(DistEntry* dep, Uint32 conn_id) return 0; } +Sint +erts_abort_connection(DistEntry *dep, Uint32 conn_id) +{ + return abort_connection(dep, conn_id); +} + BIF_RETTYPE erts_internal_abort_connection_2(BIF_ALIST_2) { DistEntry* dep; diff --git a/erts/emulator/beam/dist.h b/erts/emulator/beam/dist.h index dda2029a4c..30b4b35c20 100644 --- a/erts/emulator/beam/dist.h +++ b/erts/emulator/beam/dist.h @@ -399,5 +399,7 @@ extern void erts_kill_dist_connection(DistEntry *dep, Uint32); extern Uint erts_dist_cache_size(void); +extern Sint erts_abort_connection(DistEntry *dep, Uint32 conn_id); + #endif diff --git a/erts/emulator/beam/erl_db_hash.c b/erts/emulator/beam/erl_db_hash.c index b988a19cf4..752d3ae3a8 100644 --- a/erts/emulator/beam/erl_db_hash.c +++ b/erts/emulator/beam/erl_db_hash.c @@ -150,6 +150,22 @@ static ERTS_INLINE Uint hash_to_ix(DbTableHash* tb, HashValue hval) } +static ERTS_INLINE FixedDeletion* alloc_fixdel(DbTableHash* tb) +{ + FixedDeletion* fixd = (FixedDeletion*) erts_db_alloc(ERTS_ALC_T_DB_FIX_DEL, + (DbTable *) tb, + sizeof(FixedDeletion)); + ERTS_ETS_MISC_MEM_ADD(sizeof(FixedDeletion)); + return fixd; +} + +static ERTS_INLINE void free_fixdel(DbTableHash* tb, FixedDeletion* fixd) +{ + erts_db_free(ERTS_ALC_T_DB_FIX_DEL, (DbTable*)tb, + fixd, sizeof(FixedDeletion)); + ERTS_ETS_MISC_MEM_ADD(-sizeof(FixedDeletion)); +} + static ERTS_INLINE int link_fixdel(DbTableHash* tb, FixedDeletion* fixd, erts_aint_t fixated_by_me) @@ -160,8 +176,7 @@ static ERTS_INLINE int link_fixdel(DbTableHash* tb, was_next = erts_atomic_read_acqb(&tb->fixdel); do { /* Lockless atomic insertion in linked list: */ if (NFIXED(tb) <= fixated_by_me) { - erts_db_free(ERTS_ALC_T_DB_FIX_DEL, (DbTable*)tb, - fixd, sizeof(FixedDeletion)); + free_fixdel(tb, fixd); return 0; /* raced by unfixer */ } exp_next = was_next; @@ -180,10 +195,7 @@ static ERTS_INLINE int link_fixdel(DbTableHash* tb, static int add_fixed_deletion(DbTableHash* tb, int ix, erts_aint_t fixated_by_me) { - FixedDeletion* fixd = (FixedDeletion*) erts_db_alloc(ERTS_ALC_T_DB_FIX_DEL, - (DbTable *) tb, - sizeof(FixedDeletion)); - ERTS_ETS_MISC_MEM_ADD(sizeof(FixedDeletion)); + FixedDeletion* fixd = alloc_fixdel(tb); fixd->slot = ix; fixd->all = 0; return link_fixdel(tb, fixd, fixated_by_me); @@ -637,11 +649,7 @@ restart: free_me = fixdel; fixdel = fixdel->next; - erts_db_free(ERTS_ALC_T_DB_FIX_DEL, - (DbTable *) tb, - (void *) free_me, - sizeof(FixedDeletion)); - ERTS_ETS_MISC_MEM_ADD(-sizeof(FixedDeletion)); + free_fixdel(tb, free_me); work++; } @@ -2338,11 +2346,10 @@ static SWord db_mark_all_deleted_hash(DbTable *tbl, SWord reds) } else { /* First call */ - fixdel = erts_db_alloc(ERTS_ALC_T_DB_FIX_DEL, - (DbTable *) tb, - sizeof(FixedDeletion)); - ERTS_ETS_MISC_MEM_ADD(sizeof(FixedDeletion)); - link_fixdel(tb, fixdel, 0); + int ok; + fixdel = alloc_fixdel(tb); + ok = link_fixdel(tb, fixdel, 0); + ASSERT(ok); (void)ok; i = 0; } @@ -2444,11 +2451,7 @@ static SWord db_free_table_continue_hash(DbTable *tbl, SWord reds) FixedDeletion *fx = fixdel; fixdel = fx->next; - erts_db_free(ERTS_ALC_T_DB_FIX_DEL, - (DbTable *) tb, - (void *) fx, - sizeof(FixedDeletion)); - ERTS_ETS_MISC_MEM_ADD(-sizeof(FixedDeletion)); + free_fixdel(tb, fx); if (--reds < 0) { erts_atomic_set_relb(&tb->fixdel, (erts_aint_t)fixdel); return reds; /* Not done */ diff --git a/erts/emulator/beam/erl_node_tables.c b/erts/emulator/beam/erl_node_tables.c index 1f147011a8..9b34af1480 100644 --- a/erts/emulator/beam/erl_node_tables.c +++ b/erts/emulator/beam/erl_node_tables.c @@ -412,6 +412,44 @@ static void schedule_delete_dist_entry(DistEntry* dep) static void start_timer_delete_dist_entry(void *vdep) { + DistEntry *dep = vdep; + Eterm sysname; + enum dist_entry_state state; + Uint32 connection_id; + + erts_de_rlock(dep); + state = dep->state; + connection_id = dep->connection_id; + sysname = dep->sysname; + erts_de_runlock(dep); + + if (state != ERTS_DE_STATE_IDLE) { + char *state_str; + erts_dsprintf_buf_t *dsbuf = erts_create_logger_dsbuf(); + switch (state) { + case ERTS_DE_STATE_CONNECTED: + state_str = "connected"; + break; + case ERTS_DE_STATE_PENDING: + state_str = "pending connect"; + break; + case ERTS_DE_STATE_EXITING: + state_str = "exiting"; + break; + case ERTS_DE_STATE_IDLE: + state_str = "idle"; + break; + default: + state_str = "unknown"; + break; + } + erts_dsprintf(dsbuf, "Garbage collecting distribution " + "entry for node %T in state: %s", + sysname, state_str); + erts_send_error_to_logger_nogl(dsbuf); + erts_abort_connection(dep, connection_id); + } + if (node_tab_delete_delay == 0) { prepare_try_delete_dist_entry(vdep); } diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index 9386f79b56..0f7f1598fd 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -9079,6 +9079,9 @@ unlock_lock_rq(int pre_free, void *vrq) } +static void trace_schedule_in(Process *p, erts_aint32_t state); +static void trace_schedule_out(Process *p, erts_aint32_t state); + /* * schedule() is called from BEAM (process_main()) or HiPE * (hipe_mode_switch()) when the current process is to be @@ -9184,22 +9187,8 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) state = erts_atomic32_read_nob(&p->state); - if (IS_TRACED(p)) { - if (IS_TRACED_FL(p, F_TRACE_CALLS) && !(state & ERTS_PSFLG_FREE)) - erts_schedule_time_break(p, ERTS_BP_CALL_TIME_SCHEDULE_OUT); - if ((state & (ERTS_PSFLG_FREE|ERTS_PSFLG_EXITING)) == ERTS_PSFLG_EXITING) { - if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_EXIT)) - trace_sched(p, ERTS_PROC_LOCK_MAIN, - ((state & ERTS_PSFLG_FREE) - ? am_out_exited - : am_out_exiting)); - } - else { - if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED) || - ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_PROCS)) - trace_sched(p, ERTS_PROC_LOCK_MAIN, am_out); - } - } + if (IS_TRACED(p)) + trace_schedule_out(p, state); erts_proc_lock(p, ERTS_PROC_LOCK_STATUS|ERTS_PROC_LOCK_TRACE); @@ -9610,6 +9599,8 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) /* Migrate to dirty scheduler... */ sunlock_sched_out_proc: erts_proc_unlock(p, ERTS_PROC_LOCK_STATUS); + if (IS_TRACED(p)) + trace_schedule_in(p, state); goto sched_out_proc; } } @@ -9643,29 +9634,14 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) erts_proc_unlock(p, ERTS_PROC_LOCK_STATUS); - /* Clear tracer if it has been removed */ - if (IS_TRACED(p) && erts_is_tracer_proc_enabled( - p, ERTS_PROC_LOCK_MAIN, &p->common)) { - - if (state & ERTS_PSFLG_EXITING) { - if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_EXIT)) - trace_sched(p, ERTS_PROC_LOCK_MAIN, am_in_exiting); - } - else { - if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED) || - ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_PROCS)) - trace_sched(p, ERTS_PROC_LOCK_MAIN, am_in); - } - if (IS_TRACED_FL(p, F_TRACE_CALLS)) { - erts_schedule_time_break(p, ERTS_BP_CALL_TIME_SCHEDULE_IN); - } - } + if (IS_TRACED(p)) + trace_schedule_in(p, state); if (is_normal_sched) { if (state & ERTS_PSFLG_RUNNING_SYS) { if (state & (ERTS_PSFLG_SIG_Q|ERTS_PSFLG_SIG_IN_Q)) { int local_only = (!!(p->flags & F_LOCAL_SIGS_ONLY) - & !(state & ERTS_PSFLG_SUSPENDED)); + & !(state & (ERTS_PSFLG_SUSPENDED|ERTS_PSFLGS_DIRTY_WORK))); if (!local_only | !!(state & ERTS_PSFLG_SIG_Q)) { int sig_reds; /* @@ -9823,6 +9799,53 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) } } +static void +trace_schedule_in(Process *p, erts_aint32_t state) +{ + ASSERT(IS_TRACED(p)); + ERTS_LC_ASSERT(erts_proc_lc_my_proc_locks(p) == ERTS_PROC_LOCK_MAIN); + + /* Clear tracer if it has been removed */ + if (erts_is_tracer_proc_enabled(p, ERTS_PROC_LOCK_MAIN, &p->common)) { + + if (state & ERTS_PSFLG_EXITING) { + if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_EXIT)) + trace_sched(p, ERTS_PROC_LOCK_MAIN, am_in_exiting); + } + else { + if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED) || + ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_PROCS)) + trace_sched(p, ERTS_PROC_LOCK_MAIN, am_in); + } + if (IS_TRACED_FL(p, F_TRACE_CALLS)) + erts_schedule_time_break(p, ERTS_BP_CALL_TIME_SCHEDULE_IN); + } + +} + +static void +trace_schedule_out(Process *p, erts_aint32_t state) +{ + ASSERT(IS_TRACED(p)); + ERTS_LC_ASSERT(erts_proc_lc_my_proc_locks(p) == ERTS_PROC_LOCK_MAIN); + + if (IS_TRACED_FL(p, F_TRACE_CALLS) && !(state & ERTS_PSFLG_FREE)) + erts_schedule_time_break(p, ERTS_BP_CALL_TIME_SCHEDULE_OUT); + + if ((state & (ERTS_PSFLG_FREE|ERTS_PSFLG_EXITING)) == ERTS_PSFLG_EXITING) { + if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_EXIT)) + trace_sched(p, ERTS_PROC_LOCK_MAIN, + ((state & ERTS_PSFLG_FREE) + ? am_out_exited + : am_out_exiting)); + } + else { + if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED) || + ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_PROCS)) + trace_sched(p, ERTS_PROC_LOCK_MAIN, am_out); + } +} + static int notify_sys_task_executed(Process *c_p, ErtsProcSysTask *st, Eterm st_result, int normal_sched) diff --git a/erts/emulator/beam/ops.tab b/erts/emulator/beam/ops.tab index c51e4ef784..e76d896ffc 100644 --- a/erts/emulator/beam/ops.tab +++ b/erts/emulator/beam/ops.tab @@ -244,7 +244,7 @@ if_end # Optimize for that case. raise x==2 x==1 => i_raise raise Trace=y Value=y => move Trace x=2 | move Value x=1 | i_raise -raise Trace Value => move Trace x=3 | move Value x=1 | move x=3 x=2 | i_raise +raise Trace Value => move Trace x | move Value x=1 | move x x=2 | i_raise i_raise diff --git a/erts/emulator/pcre/LICENCE b/erts/emulator/pcre/LICENCE new file mode 100644 index 0000000000..f6ef7fd766 --- /dev/null +++ b/erts/emulator/pcre/LICENCE @@ -0,0 +1,93 @@ +PCRE LICENCE +------------ + +PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + +Release 8 of PCRE is distributed under the terms of the "BSD" licence, as +specified below. The documentation for PCRE, supplied in the "doc" +directory, is distributed under the same terms as the software itself. The data +in the testdata directory is not copyrighted and is in the public domain. + +The basic library functions are written in C and are freestanding. Also +included in the distribution is a set of C++ wrapper functions, and a +just-in-time compiler that can be used to optimize pattern matching. These +are both optional features that can be omitted when the library is built. + + +THE BASIC LIBRARY FUNCTIONS +--------------------------- + +Written by: Philip Hazel +Email local part: ph10 +Email domain: cam.ac.uk + +University of Cambridge Computing Service, +Cambridge, England. + +Copyright (c) 1997-2018 University of Cambridge +All rights reserved. + + +PCRE JUST-IN-TIME COMPILATION SUPPORT +------------------------------------- + +Written by: Zoltan Herczeg +Email local part: hzmester +Emain domain: freemail.hu + +Copyright(c) 2010-2018 Zoltan Herczeg +All rights reserved. + + +STACK-LESS JUST-IN-TIME COMPILER +-------------------------------- + +Written by: Zoltan Herczeg +Email local part: hzmester +Emain domain: freemail.hu + +Copyright(c) 2009-2018 Zoltan Herczeg +All rights reserved. + + +THE C++ WRAPPER FUNCTIONS +------------------------- + +Contributed by: Google Inc. + +Copyright (c) 2007-2012, Google Inc. +All rights reserved. + + +THE "BSD" LICENCE +----------------- + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the name of Google + Inc. nor the names of their contributors may be used to endorse or + promote products derived from this software without specific prior + written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. + +End diff --git a/erts/emulator/pcre/README.pcre_update.md b/erts/emulator/pcre/README.pcre_update.md index 599e3d0d12..5df1e15bde 100644 --- a/erts/emulator/pcre/README.pcre_update.md +++ b/erts/emulator/pcre/README.pcre_update.md @@ -723,6 +723,12 @@ requires thorough reading of all new text. For the upgrade from 7.6 to 8.33, the update of the pcrepattern part of our manual page took about eight hours. +## Update Licence + +Copy the LICENCE file to `erts/emulator/pcre/LICENCE` and update +the `[PCRE]` section in `system/COPYRIGHT` with the content of +the `LICENCE` file. + ## Add new relevant options to re Then, when all this is done, you should add any new relevant options diff --git a/erts/emulator/pcre/local_config.h b/erts/emulator/pcre/local_config.h index c6af423d72..c3b4dab586 100644 --- a/erts/emulator/pcre/local_config.h +++ b/erts/emulator/pcre/local_config.h @@ -86,4 +86,4 @@ #define SUPPORT_UTF /* Version number of package */ -#define VERSION "8.41" +#define VERSION "8.42" diff --git a/erts/emulator/pcre/pcre-8.41.tar.bz2 b/erts/emulator/pcre/pcre-8.41.tar.bz2 Binary files differdeleted file mode 100644 index 1798432dc9..0000000000 --- a/erts/emulator/pcre/pcre-8.41.tar.bz2 +++ /dev/null diff --git a/erts/emulator/pcre/pcre-8.42.tar.bz2 b/erts/emulator/pcre/pcre-8.42.tar.bz2 Binary files differnew file mode 100644 index 0000000000..61bfa38970 --- /dev/null +++ b/erts/emulator/pcre/pcre-8.42.tar.bz2 diff --git a/erts/emulator/pcre/pcre.h b/erts/emulator/pcre/pcre.h index ab8f40cfc1..3563791223 100644 --- a/erts/emulator/pcre/pcre.h +++ b/erts/emulator/pcre/pcre.h @@ -43,9 +43,9 @@ POSSIBILITY OF SUCH DAMAGE. /* The current PCRE version information. */ #define PCRE_MAJOR 8 -#define PCRE_MINOR 41 +#define PCRE_MINOR 42 #define PCRE_PRERELEASE -#define PCRE_DATE 2017-07-05 +#define PCRE_DATE 2018-03-20 /* When an application links to a PCRE DLL in Windows, the symbols that are imported have to be identified as such. When building PCRE, the appropriate @@ -328,11 +328,11 @@ these bits, just add new ones on the end, in order to remain compatible. */ /* Types */ -struct real_pcre; /* declaration; the definition is private */ -typedef struct real_pcre pcre; +struct real_pcre8_or_16; /* declaration; the definition is private */ +typedef struct real_pcre8_or_16 pcre; -struct real_pcre16; /* declaration; the definition is private */ -typedef struct real_pcre16 pcre16; +struct real_pcre8_or_16; /* declaration; the definition is private */ +typedef struct real_pcre8_or_16 pcre16; struct real_pcre32; /* declaration; the definition is private */ typedef struct real_pcre32 pcre32; diff --git a/erts/emulator/pcre/pcre_chartables.c b/erts/emulator/pcre/pcre_chartables.c index b3d9020f25..06482c08d2 100644 --- a/erts/emulator/pcre/pcre_chartables.c +++ b/erts/emulator/pcre/pcre_chartables.c @@ -19,7 +19,9 @@ array definition from the final binary if PCRE is built into a static library and dead code stripping is activated. This leads to link errors. Pulling in the header ensures that the array gets flagged as "someone outside this compilation unit might reference this" and so it will always be supplied to the linker. */ + /* %ExternalCopyright% */ + #ifdef HAVE_CONFIG_H #include "config.h" #endif diff --git a/erts/emulator/pcre/pcre_compile.c b/erts/emulator/pcre/pcre_compile.c index e79284ab79..ae7f6e2a2a 100644 --- a/erts/emulator/pcre/pcre_compile.c +++ b/erts/emulator/pcre/pcre_compile.c @@ -8061,7 +8061,7 @@ for (;; ptr++) single group (i.e. not to a duplicated name. */ HANDLE_REFERENCE: - if (firstcharflags == REQ_UNSET) firstcharflags = REQ_NONE; + if (firstcharflags == REQ_UNSET) zerofirstcharflags = firstcharflags = REQ_NONE; previous = code; item_hwm_offset = cd->hwm - cd->start_workspace; *code++ = ((options & PCRE_CASELESS) != 0)? OP_REFI : OP_REF; diff --git a/erts/emulator/pcre/pcre_dfa_exec.c b/erts/emulator/pcre/pcre_dfa_exec.c index c859d67fc7..c101656fd7 100644 --- a/erts/emulator/pcre/pcre_dfa_exec.c +++ b/erts/emulator/pcre/pcre_dfa_exec.c @@ -2288,12 +2288,14 @@ for (;;) case OP_NOTI: if (clen > 0) { - unsigned int otherd; + pcre_uint32 otherd; #ifdef SUPPORT_UTF if (utf && d >= 128) { #ifdef SUPPORT_UCP otherd = UCD_OTHERCASE(d); +#else + otherd = d; #endif /* SUPPORT_UCP */ } else diff --git a/erts/emulator/pcre/pcre_exec.c b/erts/emulator/pcre/pcre_exec.c index 6708ba92a6..1946e97a72 100644 --- a/erts/emulator/pcre/pcre_exec.c +++ b/erts/emulator/pcre/pcre_exec.c @@ -6,7 +6,7 @@ and semantics are as close as possible to those of the Perl 5 language. Written by Philip Hazel - Copyright (c) 1997-2014 University of Cambridge + Copyright (c) 1997-2018 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -2407,7 +2407,7 @@ for (;;) case OP_ANY: if (IS_NEWLINE(eptr)) RRETURN(MATCH_NOMATCH); if (md->partial != 0 && - eptr + 1 >= md->end_subject && + eptr == md->end_subject - 1 && NLBLOCK->nltype == NLTYPE_FIXED && NLBLOCK->nllen == 2 && UCHAR21TEST(eptr) == NLBLOCK->nl[0]) @@ -3167,7 +3167,7 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, eptrb, RM18); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (eptr-- == pp) break; /* Stop if tried at original pos */ + if (eptr-- <= pp) break; /* Stop if tried at original pos */ BACKCHAR(eptr); } } @@ -3326,7 +3326,7 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, eptrb, RM21); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (eptr-- == pp) break; /* Stop if tried at original pos */ + if (eptr-- <= pp) break; /* Stop if tried at original pos */ #ifdef SUPPORT_UTF if (utf) BACKCHAR(eptr); #endif diff --git a/erts/emulator/pcre/pcre_jit_compile.c b/erts/emulator/pcre/pcre_jit_compile.c index 932ca2c389..926e40f6d3 100644 --- a/erts/emulator/pcre/pcre_jit_compile.c +++ b/erts/emulator/pcre/pcre_jit_compile.c @@ -164,7 +164,6 @@ typedef struct jit_arguments { const pcre_uchar *begin; const pcre_uchar *end; int *offsets; - pcre_uchar *uchar_ptr; pcre_uchar *mark_ptr; void *callout_data; /* Everything else after. */ @@ -214,7 +213,7 @@ enum control_types { type_then_trap = 1 }; -typedef int (SLJIT_CALL *jit_function)(jit_arguments *args); +typedef int (SLJIT_FUNC *jit_function)(jit_arguments *args); /* The following structure is the key data type for the recursive code generator. It is allocated by compile_matchingpath, and contains @@ -489,9 +488,24 @@ typedef struct compare_context { /* Used for accessing the elements of the stack. */ #define STACK(i) ((i) * (int)sizeof(sljit_sw)) +#ifdef SLJIT_PREF_SHIFT_REG +#if SLJIT_PREF_SHIFT_REG == SLJIT_R2 +/* Nothing. */ +#elif SLJIT_PREF_SHIFT_REG == SLJIT_R3 +#define SHIFT_REG_IS_R3 +#else +#error "Unsupported shift register" +#endif +#endif + #define TMP1 SLJIT_R0 +#ifdef SHIFT_REG_IS_R3 +#define TMP2 SLJIT_R3 +#define TMP3 SLJIT_R2 +#else #define TMP2 SLJIT_R2 #define TMP3 SLJIT_R3 +#endif #define STR_PTR SLJIT_S0 #define STR_END SLJIT_S1 #define STACK_TOP SLJIT_R1 @@ -520,13 +534,10 @@ the start pointers when the end of the capturing group has not yet reached. */ #if defined COMPILE_PCRE8 #define MOV_UCHAR SLJIT_MOV_U8 -#define MOVU_UCHAR SLJIT_MOVU_U8 #elif defined COMPILE_PCRE16 #define MOV_UCHAR SLJIT_MOV_U16 -#define MOVU_UCHAR SLJIT_MOVU_U16 #elif defined COMPILE_PCRE32 #define MOV_UCHAR SLJIT_MOV_U32 -#define MOVU_UCHAR SLJIT_MOVU_U32 #else #error Unsupported compiling mode #endif @@ -2383,12 +2394,25 @@ if (length < 8) } else { - GET_LOCAL_BASE(SLJIT_R1, 0, OVECTOR_START); - OP1(SLJIT_MOV, SLJIT_R2, 0, SLJIT_IMM, length - 1); - loop = LABEL(); - OP1(SLJIT_MOVU, SLJIT_MEM1(SLJIT_R1), sizeof(sljit_sw), SLJIT_R0, 0); - OP2(SLJIT_SUB | SLJIT_SET_Z, SLJIT_R2, 0, SLJIT_R2, 0, SLJIT_IMM, 1); - JUMPTO(SLJIT_NOT_ZERO, loop); + if (sljit_emit_mem(compiler, SLJIT_MOV | SLJIT_MEM_SUPP | SLJIT_MEM_STORE | SLJIT_MEM_PRE, SLJIT_R0, SLJIT_MEM1(SLJIT_R1), sizeof(sljit_sw)) == SLJIT_SUCCESS) + { + GET_LOCAL_BASE(SLJIT_R1, 0, OVECTOR_START); + OP1(SLJIT_MOV, SLJIT_R2, 0, SLJIT_IMM, length - 1); + loop = LABEL(); + sljit_emit_mem(compiler, SLJIT_MOV | SLJIT_MEM_STORE | SLJIT_MEM_PRE, SLJIT_R0, SLJIT_MEM1(SLJIT_R1), sizeof(sljit_sw)); + OP2(SLJIT_SUB | SLJIT_SET_Z, SLJIT_R2, 0, SLJIT_R2, 0, SLJIT_IMM, 1); + JUMPTO(SLJIT_NOT_ZERO, loop); + } + else + { + GET_LOCAL_BASE(SLJIT_R1, 0, OVECTOR_START + sizeof(sljit_sw)); + OP1(SLJIT_MOV, SLJIT_R2, 0, SLJIT_IMM, length - 1); + loop = LABEL(); + OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_R1), 0, SLJIT_R0, 0); + OP2(SLJIT_ADD, SLJIT_R1, 0, SLJIT_R1, 0, SLJIT_IMM, sizeof(sljit_sw)); + OP2(SLJIT_SUB | SLJIT_SET_Z, SLJIT_R2, 0, SLJIT_R2, 0, SLJIT_IMM, 1); + JUMPTO(SLJIT_NOT_ZERO, loop); + } } } @@ -2421,12 +2445,25 @@ if (length < 8) } else { - GET_LOCAL_BASE(TMP2, 0, OVECTOR_START + sizeof(sljit_sw)); - OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_IMM, length - 2); - loop = LABEL(); - OP1(SLJIT_MOVU, SLJIT_MEM1(TMP2), sizeof(sljit_sw), TMP1, 0); - OP2(SLJIT_SUB | SLJIT_SET_Z, STACK_TOP, 0, STACK_TOP, 0, SLJIT_IMM, 1); - JUMPTO(SLJIT_NOT_ZERO, loop); + if (sljit_emit_mem(compiler, SLJIT_MOV | SLJIT_MEM_SUPP | SLJIT_MEM_STORE | SLJIT_MEM_PRE, TMP1, SLJIT_MEM1(TMP2), sizeof(sljit_sw)) == SLJIT_SUCCESS) + { + GET_LOCAL_BASE(TMP2, 0, OVECTOR_START + sizeof(sljit_sw)); + OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_IMM, length - 2); + loop = LABEL(); + sljit_emit_mem(compiler, SLJIT_MOV | SLJIT_MEM_STORE | SLJIT_MEM_PRE, TMP1, SLJIT_MEM1(TMP2), sizeof(sljit_sw)); + OP2(SLJIT_SUB | SLJIT_SET_Z, STACK_TOP, 0, STACK_TOP, 0, SLJIT_IMM, 1); + JUMPTO(SLJIT_NOT_ZERO, loop); + } + else + { + GET_LOCAL_BASE(TMP2, 0, OVECTOR_START + 2 * sizeof(sljit_sw)); + OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_IMM, length - 2); + loop = LABEL(); + OP1(SLJIT_MOV, SLJIT_MEM1(TMP2), 0, TMP1, 0); + OP2(SLJIT_ADD, TMP2, 0, TMP2, 0, SLJIT_IMM, sizeof(sljit_sw)); + OP2(SLJIT_SUB | SLJIT_SET_Z, STACK_TOP, 0, STACK_TOP, 0, SLJIT_IMM, 1); + JUMPTO(SLJIT_NOT_ZERO, loop); + } } OP1(SLJIT_MOV, STACK_TOP, 0, ARGUMENTS, 0); @@ -2436,10 +2473,10 @@ if (common->control_head_ptr != 0) OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), common->control_head_ptr, SLJIT_IMM, 0); OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_MEM1(STACK_TOP), SLJIT_OFFSETOF(jit_arguments, stack)); OP1(SLJIT_MOV, TMP1, 0, SLJIT_MEM1(SLJIT_SP), common->start_ptr); -OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_MEM1(STACK_TOP), SLJIT_OFFSETOF(struct sljit_stack, base)); +OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_MEM1(STACK_TOP), SLJIT_OFFSETOF(struct sljit_stack, end)); } -static sljit_sw SLJIT_CALL do_search_mark(sljit_sw *current, const pcre_uchar *skip_arg) +static sljit_sw SLJIT_FUNC do_search_mark(sljit_sw *current, const pcre_uchar *skip_arg) { while (current != NULL) { @@ -2460,7 +2497,7 @@ while (current != NULL) SLJIT_ASSERT(current[0] == 0 || current < (sljit_sw*)current[0]); current = (sljit_sw*)current[0]; } -return -1; +return 0; } static SLJIT_INLINE void copy_ovector(compiler_common *common, int topbracket) @@ -2468,6 +2505,7 @@ static SLJIT_INLINE void copy_ovector(compiler_common *common, int topbracket) DEFINE_COMPILER; struct sljit_label *loop; struct sljit_jump *early_quit; +BOOL has_pre; /* At this point we can freely use all registers. */ OP1(SLJIT_MOV, SLJIT_S2, 0, SLJIT_MEM1(SLJIT_SP), OVECTOR(1)); @@ -2481,17 +2519,30 @@ if (common->mark_ptr != 0) OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_R0), SLJIT_OFFSETOF(jit_arguments, mark_ptr), SLJIT_R2, 0); OP2(SLJIT_SUB, SLJIT_R2, 0, SLJIT_MEM1(SLJIT_R0), SLJIT_OFFSETOF(jit_arguments, offsets), SLJIT_IMM, sizeof(int)); OP1(SLJIT_MOV, SLJIT_R0, 0, SLJIT_MEM1(SLJIT_R0), SLJIT_OFFSETOF(jit_arguments, begin)); -GET_LOCAL_BASE(SLJIT_S0, 0, OVECTOR_START); + +has_pre = sljit_emit_mem(compiler, SLJIT_MOV | SLJIT_MEM_SUPP | SLJIT_MEM_PRE, SLJIT_S1, SLJIT_MEM1(SLJIT_S0), sizeof(sljit_sw)) == SLJIT_SUCCESS; +GET_LOCAL_BASE(SLJIT_S0, 0, OVECTOR_START - (has_pre ? sizeof(sljit_sw) : 0)); + /* Unlikely, but possible */ early_quit = CMP(SLJIT_EQUAL, SLJIT_R1, 0, SLJIT_IMM, 0); loop = LABEL(); -OP2(SLJIT_SUB, SLJIT_S1, 0, SLJIT_MEM1(SLJIT_S0), 0, SLJIT_R0, 0); -OP2(SLJIT_ADD, SLJIT_S0, 0, SLJIT_S0, 0, SLJIT_IMM, sizeof(sljit_sw)); + +if (has_pre) + sljit_emit_mem(compiler, SLJIT_MOV | SLJIT_MEM_PRE, SLJIT_S1, SLJIT_MEM1(SLJIT_S0), sizeof(sljit_sw)); +else + { + OP1(SLJIT_MOV, SLJIT_S1, 0, SLJIT_MEM1(SLJIT_S0), 0); + OP2(SLJIT_ADD, SLJIT_S0, 0, SLJIT_S0, 0, SLJIT_IMM, sizeof(sljit_sw)); + } + +OP2(SLJIT_ADD, SLJIT_R2, 0, SLJIT_R2, 0, SLJIT_IMM, sizeof(int)); +OP2(SLJIT_SUB, SLJIT_S1, 0, SLJIT_S1, 0, SLJIT_R0, 0); /* Copy the integer value to the output buffer */ #if defined COMPILE_PCRE16 || defined COMPILE_PCRE32 OP2(SLJIT_ASHR, SLJIT_S1, 0, SLJIT_S1, 0, SLJIT_IMM, UCHAR_SHIFT); #endif -OP1(SLJIT_MOVU_S32, SLJIT_MEM1(SLJIT_R2), sizeof(int), SLJIT_S1, 0); + +OP1(SLJIT_MOV_S32, SLJIT_MEM1(SLJIT_R2), 0, SLJIT_S1, 0); OP2(SLJIT_SUB | SLJIT_SET_Z, SLJIT_R1, 0, SLJIT_R1, 0, SLJIT_IMM, 1); JUMPTO(SLJIT_NOT_ZERO, loop); JUMPHERE(early_quit); @@ -2499,14 +2550,29 @@ JUMPHERE(early_quit); /* Calculate the return value, which is the maximum ovector value. */ if (topbracket > 1) { - GET_LOCAL_BASE(SLJIT_R0, 0, OVECTOR_START + topbracket * 2 * sizeof(sljit_sw)); - OP1(SLJIT_MOV, SLJIT_R1, 0, SLJIT_IMM, topbracket + 1); + if (sljit_emit_mem(compiler, SLJIT_MOV | SLJIT_MEM_SUPP | SLJIT_MEM_PRE, SLJIT_R2, SLJIT_MEM1(SLJIT_R0), -(2 * (sljit_sw)sizeof(sljit_sw))) == SLJIT_SUCCESS) + { + GET_LOCAL_BASE(SLJIT_R0, 0, OVECTOR_START + topbracket * 2 * sizeof(sljit_sw)); + OP1(SLJIT_MOV, SLJIT_R1, 0, SLJIT_IMM, topbracket + 1); - /* OVECTOR(0) is never equal to SLJIT_S2. */ - loop = LABEL(); - OP1(SLJIT_MOVU, SLJIT_R2, 0, SLJIT_MEM1(SLJIT_R0), -(2 * (sljit_sw)sizeof(sljit_sw))); - OP2(SLJIT_SUB, SLJIT_R1, 0, SLJIT_R1, 0, SLJIT_IMM, 1); - CMPTO(SLJIT_EQUAL, SLJIT_R2, 0, SLJIT_S2, 0, loop); + /* OVECTOR(0) is never equal to SLJIT_S2. */ + loop = LABEL(); + sljit_emit_mem(compiler, SLJIT_MOV | SLJIT_MEM_PRE, SLJIT_R2, SLJIT_MEM1(SLJIT_R0), -(2 * (sljit_sw)sizeof(sljit_sw))); + OP2(SLJIT_SUB, SLJIT_R1, 0, SLJIT_R1, 0, SLJIT_IMM, 1); + CMPTO(SLJIT_EQUAL, SLJIT_R2, 0, SLJIT_S2, 0, loop); + } + else + { + GET_LOCAL_BASE(SLJIT_R0, 0, OVECTOR_START + (topbracket - 1) * 2 * sizeof(sljit_sw)); + OP1(SLJIT_MOV, SLJIT_R1, 0, SLJIT_IMM, topbracket + 1); + + /* OVECTOR(0) is never equal to SLJIT_S2. */ + loop = LABEL(); + OP1(SLJIT_MOV, SLJIT_R2, 0, SLJIT_MEM1(SLJIT_R0), 0); + OP2(SLJIT_SUB, SLJIT_R0, 0, SLJIT_R0, 0, SLJIT_IMM, 2 * (sljit_sw)sizeof(sljit_sw)); + OP2(SLJIT_SUB, SLJIT_R1, 0, SLJIT_R1, 0, SLJIT_IMM, 1); + CMPTO(SLJIT_EQUAL, SLJIT_R2, 0, SLJIT_S2, 0, loop); + } OP1(SLJIT_MOV, SLJIT_RETURN_REG, 0, SLJIT_R1, 0); } else @@ -5167,93 +5233,190 @@ OP_FLAGS(SLJIT_OR | SLJIT_SET_Z, TMP2, 0, SLJIT_EQUAL); sljit_emit_fast_return(compiler, RETURN_ADDR, 0); } -#define CHAR1 STR_END -#define CHAR2 STACK_TOP - static void do_casefulcmp(compiler_common *common) { DEFINE_COMPILER; struct sljit_jump *jump; struct sljit_label *label; +int char1_reg; +int char2_reg; -sljit_emit_fast_enter(compiler, RETURN_ADDR, 0); +if (sljit_get_register_index(TMP3) < 0) + { + char1_reg = STR_END; + char2_reg = STACK_TOP; + } +else + { + char1_reg = TMP3; + char2_reg = RETURN_ADDR; + } + +sljit_emit_fast_enter(compiler, SLJIT_MEM1(SLJIT_SP), LOCALS0); OP2(SLJIT_SUB, STR_PTR, 0, STR_PTR, 0, TMP2, 0); -OP1(SLJIT_MOV, TMP3, 0, CHAR1, 0); -OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), LOCALS0, CHAR2, 0); -OP2(SLJIT_SUB, TMP1, 0, TMP1, 0, SLJIT_IMM, IN_UCHARS(1)); -OP2(SLJIT_SUB, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1)); -label = LABEL(); -OP1(MOVU_UCHAR, CHAR1, 0, SLJIT_MEM1(TMP1), IN_UCHARS(1)); -OP1(MOVU_UCHAR, CHAR2, 0, SLJIT_MEM1(STR_PTR), IN_UCHARS(1)); -jump = CMP(SLJIT_NOT_EQUAL, CHAR1, 0, CHAR2, 0); -OP2(SLJIT_SUB | SLJIT_SET_Z, TMP2, 0, TMP2, 0, SLJIT_IMM, IN_UCHARS(1)); -JUMPTO(SLJIT_NOT_ZERO, label); +if (char1_reg == STR_END) + { + OP1(SLJIT_MOV, TMP3, 0, char1_reg, 0); + OP1(SLJIT_MOV, RETURN_ADDR, 0, char2_reg, 0); + } -JUMPHERE(jump); -OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1)); -OP1(SLJIT_MOV, CHAR1, 0, TMP3, 0); -OP1(SLJIT_MOV, CHAR2, 0, SLJIT_MEM1(SLJIT_SP), LOCALS0); -sljit_emit_fast_return(compiler, RETURN_ADDR, 0); -} +if (sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_SUPP | SLJIT_MEM_POST, char1_reg, SLJIT_MEM1(TMP1), IN_UCHARS(1)) == SLJIT_SUCCESS) + { + label = LABEL(); + sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_POST, char1_reg, SLJIT_MEM1(TMP1), IN_UCHARS(1)); + sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_POST, char2_reg, SLJIT_MEM1(STR_PTR), IN_UCHARS(1)); + jump = CMP(SLJIT_NOT_EQUAL, char1_reg, 0, char2_reg, 0); + OP2(SLJIT_SUB | SLJIT_SET_Z, TMP2, 0, TMP2, 0, SLJIT_IMM, IN_UCHARS(1)); + JUMPTO(SLJIT_NOT_ZERO, label); + + JUMPHERE(jump); + OP1(SLJIT_MOV, TMP1, 0, SLJIT_MEM1(SLJIT_SP), LOCALS0); + } +else if (sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_SUPP | SLJIT_MEM_PRE, char1_reg, SLJIT_MEM1(TMP1), IN_UCHARS(1)) == SLJIT_SUCCESS) + { + OP2(SLJIT_SUB, TMP1, 0, TMP1, 0, SLJIT_IMM, IN_UCHARS(1)); + OP2(SLJIT_SUB, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1)); + + label = LABEL(); + sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_PRE, char1_reg, SLJIT_MEM1(TMP1), IN_UCHARS(1)); + sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_PRE, char2_reg, SLJIT_MEM1(STR_PTR), IN_UCHARS(1)); + jump = CMP(SLJIT_NOT_EQUAL, char1_reg, 0, char2_reg, 0); + OP2(SLJIT_SUB | SLJIT_SET_Z, TMP2, 0, TMP2, 0, SLJIT_IMM, IN_UCHARS(1)); + JUMPTO(SLJIT_NOT_ZERO, label); -#define LCC_TABLE STACK_LIMIT + JUMPHERE(jump); + OP1(SLJIT_MOV, TMP1, 0, SLJIT_MEM1(SLJIT_SP), LOCALS0); + OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1)); + } +else + { + label = LABEL(); + OP1(MOV_UCHAR, char1_reg, 0, SLJIT_MEM1(TMP1), 0); + OP1(MOV_UCHAR, char2_reg, 0, SLJIT_MEM1(STR_PTR), 0); + OP2(SLJIT_ADD, TMP1, 0, TMP1, 0, SLJIT_IMM, IN_UCHARS(1)); + OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1)); + jump = CMP(SLJIT_NOT_EQUAL, char1_reg, 0, char2_reg, 0); + OP2(SLJIT_SUB | SLJIT_SET_Z, TMP2, 0, TMP2, 0, SLJIT_IMM, IN_UCHARS(1)); + JUMPTO(SLJIT_NOT_ZERO, label); + + JUMPHERE(jump); + OP1(SLJIT_MOV, TMP1, 0, SLJIT_MEM1(SLJIT_SP), LOCALS0); + } + +if (char1_reg == STR_END) + { + OP1(SLJIT_MOV, char1_reg, 0, TMP3, 0); + OP1(SLJIT_MOV, char2_reg, 0, RETURN_ADDR, 0); + } + +sljit_emit_fast_return(compiler, TMP1, 0); +} static void do_caselesscmp(compiler_common *common) { DEFINE_COMPILER; struct sljit_jump *jump; struct sljit_label *label; +int char1_reg = STR_END; +int char2_reg; +int lcc_table; +int opt_type = 0; -sljit_emit_fast_enter(compiler, RETURN_ADDR, 0); +if (sljit_get_register_index(TMP3) < 0) + { + char2_reg = STACK_TOP; + lcc_table = STACK_LIMIT; + } +else + { + char2_reg = RETURN_ADDR; + lcc_table = TMP3; + } + +if (sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_SUPP | SLJIT_MEM_POST, char1_reg, SLJIT_MEM1(TMP1), IN_UCHARS(1)) == SLJIT_SUCCESS) + opt_type = 1; +else if (sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_SUPP | SLJIT_MEM_PRE, char1_reg, SLJIT_MEM1(TMP1), IN_UCHARS(1)) == SLJIT_SUCCESS) + opt_type = 2; + +sljit_emit_fast_enter(compiler, SLJIT_MEM1(SLJIT_SP), LOCALS0); OP2(SLJIT_SUB, STR_PTR, 0, STR_PTR, 0, TMP2, 0); -OP1(SLJIT_MOV, TMP3, 0, LCC_TABLE, 0); -OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), LOCALS0, CHAR1, 0); -OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), LOCALS1, CHAR2, 0); -OP1(SLJIT_MOV, LCC_TABLE, 0, SLJIT_IMM, common->lcc); -OP2(SLJIT_SUB, TMP1, 0, TMP1, 0, SLJIT_IMM, IN_UCHARS(1)); -OP2(SLJIT_SUB, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1)); +OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), LOCALS1, char1_reg, 0); + +if (char2_reg == STACK_TOP) + { + OP1(SLJIT_MOV, TMP3, 0, char2_reg, 0); + OP1(SLJIT_MOV, RETURN_ADDR, 0, lcc_table, 0); + } + +OP1(SLJIT_MOV, lcc_table, 0, SLJIT_IMM, common->lcc); + +if (opt_type == 1) + { + label = LABEL(); + sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_POST, char1_reg, SLJIT_MEM1(TMP1), IN_UCHARS(1)); + sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_POST, char2_reg, SLJIT_MEM1(STR_PTR), IN_UCHARS(1)); + } +else if (opt_type == 2) + { + OP2(SLJIT_SUB, TMP1, 0, TMP1, 0, SLJIT_IMM, IN_UCHARS(1)); + OP2(SLJIT_SUB, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1)); + + label = LABEL(); + sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_PRE, char1_reg, SLJIT_MEM1(TMP1), IN_UCHARS(1)); + sljit_emit_mem(compiler, MOV_UCHAR | SLJIT_MEM_PRE, char2_reg, SLJIT_MEM1(STR_PTR), IN_UCHARS(1)); + } +else + { + label = LABEL(); + OP1(MOV_UCHAR, char1_reg, 0, SLJIT_MEM1(TMP1), 0); + OP1(MOV_UCHAR, char2_reg, 0, SLJIT_MEM1(STR_PTR), 0); + OP2(SLJIT_ADD, TMP1, 0, TMP1, 0, SLJIT_IMM, IN_UCHARS(1)); + } -label = LABEL(); -OP1(MOVU_UCHAR, CHAR1, 0, SLJIT_MEM1(TMP1), IN_UCHARS(1)); -OP1(MOVU_UCHAR, CHAR2, 0, SLJIT_MEM1(STR_PTR), IN_UCHARS(1)); #ifndef COMPILE_PCRE8 -jump = CMP(SLJIT_GREATER, CHAR1, 0, SLJIT_IMM, 255); +jump = CMP(SLJIT_GREATER, char1_reg, 0, SLJIT_IMM, 255); #endif -OP1(SLJIT_MOV_U8, CHAR1, 0, SLJIT_MEM2(LCC_TABLE, CHAR1), 0); +OP1(SLJIT_MOV_U8, char1_reg, 0, SLJIT_MEM2(lcc_table, char1_reg), 0); #ifndef COMPILE_PCRE8 JUMPHERE(jump); -jump = CMP(SLJIT_GREATER, CHAR2, 0, SLJIT_IMM, 255); +jump = CMP(SLJIT_GREATER, char2_reg, 0, SLJIT_IMM, 255); #endif -OP1(SLJIT_MOV_U8, CHAR2, 0, SLJIT_MEM2(LCC_TABLE, CHAR2), 0); +OP1(SLJIT_MOV_U8, char2_reg, 0, SLJIT_MEM2(lcc_table, char2_reg), 0); #ifndef COMPILE_PCRE8 JUMPHERE(jump); #endif -jump = CMP(SLJIT_NOT_EQUAL, CHAR1, 0, CHAR2, 0); + +if (opt_type == 0) + OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1)); + +jump = CMP(SLJIT_NOT_EQUAL, char1_reg, 0, char2_reg, 0); OP2(SLJIT_SUB | SLJIT_SET_Z, TMP2, 0, TMP2, 0, SLJIT_IMM, IN_UCHARS(1)); JUMPTO(SLJIT_NOT_ZERO, label); JUMPHERE(jump); -OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1)); -OP1(SLJIT_MOV, LCC_TABLE, 0, TMP3, 0); -OP1(SLJIT_MOV, CHAR1, 0, SLJIT_MEM1(SLJIT_SP), LOCALS0); -OP1(SLJIT_MOV, CHAR2, 0, SLJIT_MEM1(SLJIT_SP), LOCALS1); -sljit_emit_fast_return(compiler, RETURN_ADDR, 0); -} +OP1(SLJIT_MOV, TMP1, 0, SLJIT_MEM1(SLJIT_SP), LOCALS0); + +if (opt_type == 2) + OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1)); -#undef LCC_TABLE -#undef CHAR1 -#undef CHAR2 +if (char2_reg == STACK_TOP) + { + OP1(SLJIT_MOV, char2_reg, 0, TMP3, 0); + OP1(SLJIT_MOV, lcc_table, 0, RETURN_ADDR, 0); + } + +OP1(SLJIT_MOV, char1_reg, 0, SLJIT_MEM1(SLJIT_SP), LOCALS1); +sljit_emit_fast_return(compiler, TMP1, 0); +} #if defined SUPPORT_UTF && defined SUPPORT_UCP -static const pcre_uchar * SLJIT_CALL do_utf_caselesscmp(pcre_uchar *src1, jit_arguments *args, pcre_uchar *end1) +static const pcre_uchar * SLJIT_FUNC do_utf_caselesscmp(pcre_uchar *src1, pcre_uchar *src2, pcre_uchar *end1, pcre_uchar *end2) { /* This function would be ineffective to do in JIT level. */ sljit_u32 c1, c2; -const pcre_uchar *src2 = args->uchar_ptr; -const pcre_uchar *end2 = args->end; const ucd_record *ur; const sljit_u32 *pp; @@ -6776,32 +6939,37 @@ else #if defined SUPPORT_UTF && defined SUPPORT_UCP if (common->utf && *cc == OP_REFI) { - SLJIT_ASSERT(TMP1 == SLJIT_R0 && STACK_TOP == SLJIT_R1 && TMP2 == SLJIT_R2); + SLJIT_ASSERT(TMP1 == SLJIT_R0 && STACK_TOP == SLJIT_R1); if (ref) - OP1(SLJIT_MOV, TMP2, 0, SLJIT_MEM1(SLJIT_SP), OVECTOR(offset + 1)); + OP1(SLJIT_MOV, SLJIT_R2, 0, SLJIT_MEM1(SLJIT_SP), OVECTOR(offset + 1)); else - OP1(SLJIT_MOV, TMP2, 0, SLJIT_MEM1(TMP2), sizeof(sljit_sw)); + OP1(SLJIT_MOV, SLJIT_R2, 0, SLJIT_MEM1(TMP2), sizeof(sljit_sw)); if (withchecks) - jump = CMP(SLJIT_EQUAL, TMP1, 0, TMP2, 0); + jump = CMP(SLJIT_EQUAL, TMP1, 0, SLJIT_R2, 0); - /* Needed to save important temporary registers. */ + /* No free saved registers so save data on stack. */ OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), LOCALS0, STACK_TOP, 0); - OP1(SLJIT_MOV, SLJIT_R1, 0, ARGUMENTS, 0); - OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_R1), SLJIT_OFFSETOF(jit_arguments, uchar_ptr), STR_PTR, 0); - sljit_emit_ijump(compiler, SLJIT_CALL3, SLJIT_IMM, SLJIT_FUNC_OFFSET(do_utf_caselesscmp)); + OP1(SLJIT_MOV, SLJIT_R1, 0, STR_PTR, 0); + OP1(SLJIT_MOV, SLJIT_R3, 0, STR_END, 0); + sljit_emit_icall(compiler, SLJIT_CALL, SLJIT_RET(SW) | SLJIT_ARG1(SW) | SLJIT_ARG2(SW) | SLJIT_ARG3(SW) | SLJIT_ARG4(SW), SLJIT_IMM, SLJIT_FUNC_OFFSET(do_utf_caselesscmp)); OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_MEM1(SLJIT_SP), LOCALS0); + OP1(SLJIT_MOV, STR_PTR, 0, SLJIT_RETURN_REG, 0); + if (common->mode == JIT_COMPILE) add_jump(compiler, backtracks, CMP(SLJIT_LESS_EQUAL, SLJIT_RETURN_REG, 0, SLJIT_IMM, 1)); else { - add_jump(compiler, backtracks, CMP(SLJIT_EQUAL, SLJIT_RETURN_REG, 0, SLJIT_IMM, 0)); - nopartial = CMP(SLJIT_NOT_EQUAL, SLJIT_RETURN_REG, 0, SLJIT_IMM, 1); + OP2(SLJIT_SUB | SLJIT_SET_Z | SLJIT_SET_LESS, SLJIT_UNUSED, 0, SLJIT_RETURN_REG, 0, SLJIT_IMM, 1); + + add_jump(compiler, backtracks, JUMP(SLJIT_LESS)); + + nopartial = JUMP(SLJIT_NOT_EQUAL); + OP1(SLJIT_MOV, STR_PTR, 0, STR_END, 0); check_partial(common, FALSE); add_jump(compiler, backtracks, JUMP(SLJIT_JUMP)); JUMPHERE(nopartial); } - OP1(SLJIT_MOV, STR_PTR, 0, SLJIT_RETURN_REG, 0); } else #endif /* SUPPORT_UTF && SUPPORT_UCP */ @@ -7125,7 +7293,7 @@ add_jump(compiler, &backtrack->topbacktracks, CMP(SLJIT_EQUAL, TMP1, 0, SLJIT_IM return cc + 1 + LINK_SIZE; } -static int SLJIT_CALL do_callout(struct jit_arguments *arguments, PUBL(callout_block) *callout_block, pcre_uchar **jit_ovector) +static sljit_s32 SLJIT_FUNC do_callout(struct jit_arguments *arguments, PUBL(callout_block) *callout_block, pcre_uchar **jit_ovector) { const pcre_uchar *begin = arguments->begin; int *offset_vector = arguments->offsets; @@ -7207,18 +7375,17 @@ OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), LOCALS0, STACK_TOP, 0); /* SLJIT_R0 = arguments */ OP1(SLJIT_MOV, SLJIT_R1, 0, STACK_TOP, 0); GET_LOCAL_BASE(SLJIT_R2, 0, OVECTOR_START); -sljit_emit_ijump(compiler, SLJIT_CALL3, SLJIT_IMM, SLJIT_FUNC_OFFSET(do_callout)); -OP1(SLJIT_MOV_S32, SLJIT_RETURN_REG, 0, SLJIT_RETURN_REG, 0); +sljit_emit_icall(compiler, SLJIT_CALL, SLJIT_RET(S32) | SLJIT_ARG1(SW) | SLJIT_ARG2(SW) | SLJIT_ARG3(SW), SLJIT_IMM, SLJIT_FUNC_OFFSET(do_callout)); OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_MEM1(SLJIT_SP), LOCALS0); free_stack(common, CALLOUT_ARG_SIZE / sizeof(sljit_sw)); /* Check return value. */ -OP2(SLJIT_SUB | SLJIT_SET_Z | SLJIT_SET_SIG_GREATER, SLJIT_UNUSED, 0, SLJIT_RETURN_REG, 0, SLJIT_IMM, 0); -add_jump(compiler, &backtrack->topbacktracks, JUMP(SLJIT_SIG_GREATER)); +OP2(SLJIT_SUB32 | SLJIT_SET_Z | SLJIT_SET_SIG_GREATER, SLJIT_UNUSED, 0, SLJIT_RETURN_REG, 0, SLJIT_IMM, 0); +add_jump(compiler, &backtrack->topbacktracks, JUMP(SLJIT_SIG_GREATER32)); if (common->forced_quit_label == NULL) - add_jump(compiler, &common->forced_quit, JUMP(SLJIT_NOT_EQUAL) /* SIG_LESS */); + add_jump(compiler, &common->forced_quit, JUMP(SLJIT_NOT_EQUAL32) /* SIG_LESS */); else - JUMPTO(SLJIT_NOT_EQUAL /* SIG_LESS */, common->forced_quit_label); + JUMPTO(SLJIT_NOT_EQUAL32 /* SIG_LESS */, common->forced_quit_label); return cc + 2 + 2 * LINK_SIZE; } @@ -10439,11 +10606,11 @@ if (opcode == OP_SKIP_ARG) OP1(SLJIT_MOV, TMP1, 0, SLJIT_MEM1(SLJIT_SP), common->control_head_ptr); OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), LOCALS0, STACK_TOP, 0); OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_IMM, (sljit_sw)(current->cc + 2)); - sljit_emit_ijump(compiler, SLJIT_CALL2, SLJIT_IMM, SLJIT_FUNC_OFFSET(do_search_mark)); + sljit_emit_icall(compiler, SLJIT_CALL, SLJIT_RET(SW) | SLJIT_ARG1(SW) | SLJIT_ARG2(SW), SLJIT_IMM, SLJIT_FUNC_OFFSET(do_search_mark)); OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_MEM1(SLJIT_SP), LOCALS0); OP1(SLJIT_MOV, STR_PTR, 0, TMP1, 0); - add_jump(compiler, &common->reset_match, CMP(SLJIT_NOT_EQUAL, STR_PTR, 0, SLJIT_IMM, -1)); + add_jump(compiler, &common->reset_match, CMP(SLJIT_NOT_EQUAL, STR_PTR, 0, SLJIT_IMM, 0)); return; } @@ -11031,7 +11198,7 @@ if (!compiler) common->compiler = compiler; /* Main pcre_jit_exec entry. */ -sljit_emit_enter(compiler, 0, 1, 5, 5, 0, 0, private_data_size); +sljit_emit_enter(compiler, 0, SLJIT_ARG1(SW), 5, 5, 0, 0, private_data_size); /* Register init. */ reset_ovector(common, (re->top_bracket + 1) * 2); @@ -11044,8 +11211,8 @@ OP1(SLJIT_MOV, STR_PTR, 0, SLJIT_MEM1(TMP1), SLJIT_OFFSETOF(jit_arguments, str)) OP1(SLJIT_MOV, STR_END, 0, SLJIT_MEM1(TMP1), SLJIT_OFFSETOF(jit_arguments, end)); OP1(SLJIT_MOV, TMP2, 0, SLJIT_MEM1(TMP1), SLJIT_OFFSETOF(jit_arguments, stack)); OP1(SLJIT_MOV_U32, TMP1, 0, SLJIT_MEM1(TMP1), SLJIT_OFFSETOF(jit_arguments, limit_match)); -OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_MEM1(TMP2), SLJIT_OFFSETOF(struct sljit_stack, base)); -OP1(SLJIT_MOV, STACK_LIMIT, 0, SLJIT_MEM1(TMP2), SLJIT_OFFSETOF(struct sljit_stack, limit)); +OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_MEM1(TMP2), SLJIT_OFFSETOF(struct sljit_stack, end)); +OP1(SLJIT_MOV, STACK_LIMIT, 0, SLJIT_MEM1(TMP2), SLJIT_OFFSETOF(struct sljit_stack, start)); OP2(SLJIT_ADD, TMP1, 0, TMP1, 0, SLJIT_IMM, 1); OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), LIMIT_MATCH, TMP1, 0); @@ -11251,20 +11418,22 @@ common->quit_label = quit_label; set_jumps(common->stackalloc, LABEL()); /* RETURN_ADDR is not a saved register. */ sljit_emit_fast_enter(compiler, SLJIT_MEM1(SLJIT_SP), LOCALS0); -OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), LOCALS1, TMP2, 0); -OP1(SLJIT_MOV, TMP1, 0, ARGUMENTS, 0); -OP1(SLJIT_MOV, TMP1, 0, SLJIT_MEM1(TMP1), SLJIT_OFFSETOF(jit_arguments, stack)); -OP1(SLJIT_MOV, SLJIT_MEM1(TMP1), SLJIT_OFFSETOF(struct sljit_stack, top), STACK_TOP, 0); -OP2(SLJIT_SUB, TMP2, 0, SLJIT_MEM1(TMP1), SLJIT_OFFSETOF(struct sljit_stack, limit), SLJIT_IMM, STACK_GROWTH_RATE); -sljit_emit_ijump(compiler, SLJIT_CALL2, SLJIT_IMM, SLJIT_FUNC_OFFSET(sljit_stack_resize)); -jump = CMP(SLJIT_NOT_EQUAL, SLJIT_RETURN_REG, 0, SLJIT_IMM, 0); -OP1(SLJIT_MOV, TMP1, 0, ARGUMENTS, 0); -OP1(SLJIT_MOV, TMP1, 0, SLJIT_MEM1(TMP1), SLJIT_OFFSETOF(jit_arguments, stack)); -OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_MEM1(TMP1), SLJIT_OFFSETOF(struct sljit_stack, top)); -OP1(SLJIT_MOV, STACK_LIMIT, 0, SLJIT_MEM1(TMP1), SLJIT_OFFSETOF(struct sljit_stack, limit)); -OP1(SLJIT_MOV, TMP2, 0, SLJIT_MEM1(SLJIT_SP), LOCALS1); -sljit_emit_fast_return(compiler, SLJIT_MEM1(SLJIT_SP), LOCALS0); +SLJIT_ASSERT(TMP1 == SLJIT_R0 && STACK_TOP == SLJIT_R1); + +OP1(SLJIT_MOV, SLJIT_MEM1(SLJIT_SP), LOCALS1, STACK_TOP, 0); +OP1(SLJIT_MOV, SLJIT_R0, 0, ARGUMENTS, 0); +OP2(SLJIT_SUB, SLJIT_R1, 0, STACK_LIMIT, 0, SLJIT_IMM, STACK_GROWTH_RATE); +OP1(SLJIT_MOV, SLJIT_R0, 0, SLJIT_MEM1(SLJIT_R0), SLJIT_OFFSETOF(jit_arguments, stack)); +OP1(SLJIT_MOV, STACK_LIMIT, 0, TMP2, 0); + +sljit_emit_icall(compiler, SLJIT_CALL, SLJIT_RET(SW) | SLJIT_ARG1(SW) | SLJIT_ARG2(SW), SLJIT_IMM, SLJIT_FUNC_OFFSET(sljit_stack_resize)); +jump = CMP(SLJIT_EQUAL, SLJIT_RETURN_REG, 0, SLJIT_IMM, 0); +OP1(SLJIT_MOV, TMP2, 0, STACK_LIMIT, 0); +OP1(SLJIT_MOV, STACK_LIMIT, 0, SLJIT_RETURN_REG, 0); +OP1(SLJIT_MOV, TMP1, 0, SLJIT_MEM1(SLJIT_SP), LOCALS0); +OP1(SLJIT_MOV, STACK_TOP, 0, SLJIT_MEM1(SLJIT_SP), LOCALS1); +sljit_emit_fast_return(compiler, TMP1, 0); /* Allocation failed. */ JUMPHERE(jump); @@ -11409,9 +11578,9 @@ union { sljit_u8 local_space[MACHINE_STACK_SIZE]; struct sljit_stack local_stack; -local_stack.max_limit = local_space; -local_stack.limit = local_space; -local_stack.base = local_space + MACHINE_STACK_SIZE; +local_stack.min_start = local_space; +local_stack.start = local_space; +local_stack.end = local_space + MACHINE_STACK_SIZE; local_stack.top = local_space + MACHINE_STACK_SIZE; arguments->stack = &local_stack; convert_executable_func.executable_func = executable_func; @@ -11536,7 +11705,7 @@ if ((options & PCRE_PARTIAL_HARD) != 0) else if ((options & PCRE_PARTIAL_SOFT) != 0) mode = JIT_PARTIAL_SOFT_COMPILE; -if (functions->executable_funcs[mode] == NULL) +if (functions == NULL || functions->executable_funcs[mode] == NULL) return PCRE_ERROR_JIT_BADOPTION; /* Sanity checks should be handled by pcre_exec. */ diff --git a/erts/emulator/pcre/pcre_latin_1_table.c b/erts/emulator/pcre/pcre_latin_1_table.c index aa29275a94..d6cf38fa3b 100644 --- a/erts/emulator/pcre/pcre_latin_1_table.c +++ b/erts/emulator/pcre/pcre_latin_1_table.c @@ -14,6 +14,7 @@ Pulling in the header ensures that the array gets flagged as "someone outside this compilation unit might reference this" and so it will always be supplied to the linker. */ /* %ExternalCopyright% */ + #ifdef HAVE_CONFIG_H #include "config.h" #endif @@ -120,7 +121,7 @@ print, punct, and cntrl. Other classes are built from combinations. */ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, 0x00,0x00,0x00,0x00,0xfe,0xff,0xff,0x07, - 0x00,0x00,0x00,0x00,0x00,0x00,0x20,0x00, + 0x00,0x00,0x00,0x00,0x00,0x04,0x20,0x04, 0x00,0x00,0x00,0x80,0xff,0xff,0x7f,0xff, 0x00,0x00,0x00,0x00,0x00,0x00,0xff,0x03, diff --git a/erts/emulator/sys/common/erl_sys_common_misc.c b/erts/emulator/sys/common/erl_sys_common_misc.c index 2541ab5d31..d34e1a9ec0 100644 --- a/erts/emulator/sys/common/erl_sys_common_misc.c +++ b/erts/emulator/sys/common/erl_sys_common_misc.c @@ -176,6 +176,7 @@ sys_double_to_chars_fast(double f, char *buffer, int buffer_size, int decimals, double af; Uint64 int_part, frac_part; int neg; + int has_decimals = decimals != 0; char *p = buffer; if (decimals < 0) @@ -257,7 +258,7 @@ sys_double_to_chars_fast(double f, char *buffer, int buffer_size, int decimals, } /* Delete trailing zeroes */ - if (compact) + if (compact && has_decimals) p = find_first_trailing_zero(p); *p = '\0'; return p - buffer; diff --git a/erts/emulator/test/node_container_SUITE.erl b/erts/emulator/test/node_container_SUITE.erl index 7df001fec5..55135fbcbc 100644 --- a/erts/emulator/test/node_container_SUITE.erl +++ b/erts/emulator/test/node_container_SUITE.erl @@ -50,7 +50,8 @@ bad_nc/1, unique_pid/1, iter_max_procs/1, - magic_ref/1]). + magic_ref/1, + dist_entry_gc/1]). suite() -> [{ct_hooks,[ts_install_cth]}, @@ -58,7 +59,7 @@ suite() -> all() -> - [term_to_binary_to_term_eq, round_trip_eq, cmp, ref_eq, + [dist_entry_gc, term_to_binary_to_term_eq, round_trip_eq, cmp, ref_eq, node_table_gc, dist_link_refc, dist_monitor_refc, node_controller_refc, ets_refc, match_spec_refc, timer_refc, pid_wrap, port_wrap, bad_nc, @@ -894,6 +895,29 @@ magic_ref(Config) when is_list(Config) -> true = is_reference(MRef2), true = erts_debug:get_internal_state({magic_ref,MRef2}), ok. + + +lost_pending_connection(Node) -> + _ = (catch erts_internal:new_connection(Node)), + ok. + +dist_entry_gc(Config) when is_list(Config) -> + Me = self(), + {ok, Node} = start_node(get_nodefirstname(), "+zdntgc 0"), + P = spawn_link(Node, + fun () -> + LostNode = list_to_atom("lost_pending_connection@" ++ hostname()), + lost_pending_connection(LostNode), + garbage_collect(), %% Could crash... + Me ! {self(), ok} + end), + receive + {P, ok} -> ok + end, + unlink(P), + stop_node(Node), + ok. + %% %% -- Internal utils --------------------------------------------------------- %% diff --git a/erts/emulator/test/num_bif_SUITE.erl b/erts/emulator/test/num_bif_SUITE.erl index 700734cd0b..f15217814a 100644 --- a/erts/emulator/test/num_bif_SUITE.erl +++ b/erts/emulator/test/num_bif_SUITE.erl @@ -161,6 +161,7 @@ t_float_to_string(Config) when is_list(Config) -> test_fts("1.000",1.0, [{decimals, 3}]), test_fts("1.0",1.0, [{decimals, 1}]), test_fts("1.0",1.0, [{decimals, 3}, compact]), + test_fts("10",10.0, [{decimals, 0}, compact]), test_fts("1.12",1.123, [{decimals, 2}]), test_fts("1.123",1.123, [{decimals, 3}]), test_fts("1.123",1.123, [{decimals, 3}, compact]), diff --git a/erts/vsn.mk b/erts/vsn.mk index c0444fa483..293b555b18 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -18,7 +18,7 @@ # %CopyrightEnd% # -VSN = 10.0.5 +VSN = 10.0.7 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/compiler/src/v3_core.erl b/lib/compiler/src/v3_core.erl index 3b746ab5bf..c9517c3e51 100644 --- a/lib/compiler/src/v3_core.erl +++ b/lib/compiler/src/v3_core.erl @@ -328,14 +328,16 @@ gexpr({protect,Line,Arg}, Bools0, St0) -> Anno = lineno_anno(Line, St), {#iprotect{anno=#a{anno=Anno},body=Eps++[E]},[],Bools0,St} end; -gexpr({op,L,'andalso',E1,E2}, Bools, St0) -> +gexpr({op,_,'andalso',_,_}=E0, Bools, St0) -> + {op,L,'andalso',E1,E2} = right_assoc(E0, 'andalso', St0), Anno = lineno_anno(L, St0), {#c_var{name=V0},St} = new_var(Anno, St0), V = {var,L,V0}, False = {atom,L,false}, E = make_bool_switch_guard(L, E1, V, E2, False), gexpr(E, Bools, St); -gexpr({op,L,'orelse',E1,E2}, Bools, St0) -> +gexpr({op,_,'orelse',_,_}=E0, Bools, St0) -> + {op,L,'orelse',E1,E2} = right_assoc(E0, 'orelse', St0), Anno = lineno_anno(L, St0), {#c_var{name=V0},St} = new_var(Anno, St0), V = {var,L,V0}, @@ -2054,6 +2056,19 @@ fail_clause(Pats, Anno, Arg) -> body=[#iprimop{anno=#a{anno=Anno},name=#c_literal{val=match_fail}, args=[Arg]}]}. +%% Optimization for Dialyzer. +right_assoc(E, Op, St) -> + case member(dialyzer, St#core.opts) of + true -> + right_assoc2(E, Op); + false -> + E + end. + +right_assoc2({op,L1,Op,{op,L2,Op,E1,E2},E3}, Op) -> + right_assoc2({op,L2,Op,E1,{op,L1,Op,E2,E3}}, Op); +right_assoc2(E, _Op) -> E. + annotate_tuple(A, Es, St) -> case member(dialyzer, St#core.opts) of true -> diff --git a/lib/crypto/c_src/otp_test_engine.c b/lib/crypto/c_src/otp_test_engine.c index b6c9067964..34c825059f 100644 --- a/lib/crypto/c_src/otp_test_engine.c +++ b/lib/crypto/c_src/otp_test_engine.c @@ -64,7 +64,8 @@ static int test_init(ENGINE *e) { printf("OTP Test Engine Initializatzion!\r\n"); /* Load all digest and cipher algorithms. Needed for password protected private keys */ - OpenSSL_add_all_algorithms(); + OpenSSL_add_all_ciphers(); + OpenSSL_add_all_digests(); return 111; } diff --git a/lib/crypto/src/crypto.erl b/lib/crypto/src/crypto.erl index a9d933f5d7..17351d10ea 100644 --- a/lib/crypto/src/crypto.erl +++ b/lib/crypto/src/crypto.erl @@ -298,8 +298,10 @@ next_iv(Type, Data) when is_binary(Data) -> IVecSize = case Type of des_cbc -> 8; des3_cbc -> 8; + blowfish_cbc -> 8; aes_cbc -> 16; - aes_ige -> 32 + aes_ige -> 32; % For compatibility if someone has bug-adapted code + aes_ige256 -> 32 % The name used in block_encrypt et al end, {_, IVec} = split_binary(Data, size(Data) - IVecSize), IVec; diff --git a/lib/dialyzer/test/small_SUITE_data/results/left_assoc b/lib/dialyzer/test/small_SUITE_data/results/left_assoc new file mode 100644 index 0000000000..58cdad29de --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/results/left_assoc @@ -0,0 +1,2 @@ + +left_assoc.erl:93: The variable __@2 can never match since previous clauses completely covered the type binary() diff --git a/lib/dialyzer/test/small_SUITE_data/src/left_assoc.erl b/lib/dialyzer/test/small_SUITE_data/src/left_assoc.erl new file mode 100644 index 0000000000..0250e4ab49 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/left_assoc.erl @@ -0,0 +1,96 @@ +-module(left_assoc). + +%% As pointed out in ERL-680, analyzing guards with short circuit +%% operators becomes very slow as the number of left associations +%% grows. + +-spec from_iso8601('Elixir.String':t(), 'Elixir.Calendar':calendar()) -> + {ok, t()} | {error, atom()}. + +-export_type([t/0]). + +-type t() :: + #{'__struct__' := 'Elixir.Date', + calendar := 'Elixir.Calendar':calendar(), + day := 'Elixir.Calendar':day(), + month := 'Elixir.Calendar':month(), + year := 'Elixir.Calendar':year()}. + +-export([from_iso8601/1, + from_iso8601/2]). + +from_iso8601(__@1) -> + from_iso8601(__@1, 'Elixir.Calendar.ISO'). + +from_iso8601(<<45/integer,_rest@1/binary>>, _calendar@1) -> + case raw_from_iso8601(_rest@1, _calendar@1) of + {ok,#{year := _year@1} = _date@1} -> + {ok,_date@1#{year := - _year@1}}; + __@1 -> + __@1 + end; +from_iso8601(<<_rest@1/binary>>, _calendar@1) -> + raw_from_iso8601(_rest@1, _calendar@1). + +raw_from_iso8601(_string@1, _calendar@1) -> + case _string@1 of + <<_y1@1/integer, + _y2@1/integer, + _y3@1/integer, + _y4@1/integer, + 45/integer, + _m1@1/integer, + _m2@1/integer, + 45/integer, + _d1@1/integer, + _d2@1/integer>> + when + ((((((((((((((_y1@1 >= 48 + andalso + _y1@1 =< 57) + andalso + _y2@1 >= 48) + andalso + _y2@1 =< 57) + andalso + _y3@1 >= 48) + andalso + _y3@1 =< 57) + andalso + _y4@1 >= 48) + andalso + _y4@1 =< 57) + andalso + _m1@1 >= 48) + andalso + _m1@1 =< 57) + andalso + _m2@1 >= 48) + andalso + _m2@1 =< 57) + andalso + _d1@1 >= 48) + andalso + _d1@1 =< 57) + andalso + _d2@1 >= 48) + andalso + _d2@1 =< 57 -> + {ok, + #{year => (_y1@1 - 48) * 1000 + (_y2@1 - 48) * 100 + + + (_y3@1 - 48) * 10 + + + (_y4@1 - 48), + month => (_m1@1 - 48) * 10 + (_m2@1 - 48), + day => (_d1@1 - 48) * 10 + (_d2@1 - 48), + calendar => _calendar@1, + '__struct__' => 'Elixir.Date'}}; + __@1 -> + case __@1 of + _ -> + {error,invalid_format}; + __@2 -> + error({with_clause,__@2}) + end + end. diff --git a/lib/erl_docgen/priv/xsl/db_eix.xsl b/lib/erl_docgen/priv/xsl/db_eix.xsl index b496614854..6bce577f08 100644 --- a/lib/erl_docgen/priv/xsl/db_eix.xsl +++ b/lib/erl_docgen/priv/xsl/db_eix.xsl @@ -3,7 +3,7 @@ # # %CopyrightBegin% # - # Copyright Ericsson AB 2009-2016. All Rights Reserved. + # Copyright Ericsson AB 2009-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -199,11 +199,34 @@ <xsl:template name="name"> <xsl:param name="lastfuncsblock"/> + <xsl:variable name="signature"> + <xsl:variable name="signature1"> + <xsl:choose> + <xsl:when test="ancestor::cref"> + <xsl:value-of + select="normalize-space(nametext)"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of + select="normalize-space(substring-before(., '->'))"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <xsl:choose> + <xsl:when test="string-length($signature1) > 0"> + <xsl:value-of select="$signature1"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="normalize-space(.)"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <xsl:variable name="tmpstring"> - <xsl:value-of select="substring-before(substring-after(., '('), '->')"/> + <xsl:value-of select="substring-after($signature, '(')"/> </xsl:variable> - <xsl:variable name="ustring"> + <xsl:variable name="argstring"> <xsl:choose> <xsl:when test="string-length($tmpstring) > 0"> <xsl:call-template name="remove-paren"> @@ -219,10 +242,19 @@ </xsl:variable> <xsl:variable name="arity"> - <xsl:call-template name="calc-arity"> - <xsl:with-param name="string" select="substring-before($ustring, ')')"/> - <xsl:with-param name="no-of-pars" select="0"/> - </xsl:call-template> + <xsl:choose> + <xsl:when + test="string-length(substring-before(., '->')) > 0"> + <xsl:call-template name="calc-arity"> + <xsl:with-param + name="string" + select="substring-before($argstring, ')')"/> + <xsl:with-param name="no-of-pars" select="0"/> + </xsl:call-template> + </xsl:when> + <xsl:otherwise/> + </xsl:choose> + </xsl:variable> <xsl:variable name="fname"> @@ -250,10 +282,18 @@ </xsl:variable> <xsl:text> {"</xsl:text><xsl:value-of select="$fname"/> + <xsl:text>", "</xsl:text> + <xsl:call-template name="escape-doublequotes"> + <xsl:with-param name="string" select="$signature"/> + </xsl:call-template> <xsl:text>", "</xsl:text><xsl:value-of select="$fname"/> - <xsl:text>(</xsl:text><xsl:value-of select="normalize-space($tmpstring)"/> - <xsl:text>", "</xsl:text><xsl:value-of select="$fname"/> - <xsl:text>-</xsl:text><xsl:value-of select="$arity"/><xsl:text>"}</xsl:text> + <xsl:choose> + <xsl:when test="string-length($arity) > 0"> + <xsl:text>-</xsl:text><xsl:value-of select="$arity"/> + </xsl:when> + <xsl:otherwise/> + </xsl:choose> + <xsl:text>"}</xsl:text> <xsl:choose> <xsl:when test="($lastfuncsblock = 'true') and (position() = last())"> @@ -345,6 +385,27 @@ </xsl:template> + <xsl:template name="escape-doublequotes"> + <xsl:param name="string"/> + <xsl:param name="pPat">"</xsl:param> + <xsl:param name="pRep">\"</xsl:param> + + <xsl:choose> + <xsl:when test="not(contains($string, $pPat))"> + <xsl:copy-of select="$string"/> + </xsl:when> + <xsl:otherwise> + <xsl:copy-of select="substring-before($string, $pPat)"/> + <xsl:copy-of select="$pRep"/> + <xsl:call-template name="escape-doublequotes"> + <xsl:with-param + name="string" + select="substring-after($string, $pPat)"/> + </xsl:call-template> + </xsl:otherwise> + </xsl:choose> + </xsl:template> + <!-- default content handling --> <xsl:template match="text()"/> diff --git a/lib/erl_interface/src/legacy/erl_marshal.c b/lib/erl_interface/src/legacy/erl_marshal.c index c18067b9bc..932bba43bf 100644 --- a/lib/erl_interface/src/legacy/erl_marshal.c +++ b/lib/erl_interface/src/legacy/erl_marshal.c @@ -1803,7 +1803,7 @@ static int cmp_exe2(unsigned char **e1, unsigned char **e2) k = 0; while (1) { if (k++ == min){ - if (i == j) return 0; + if (i == j) return compare_top_ext(e1 , e2); if (i < j) return -1; return 1; } diff --git a/lib/erl_interface/test/erl_ext_SUITE_data/ext_test.c b/lib/erl_interface/test/erl_ext_SUITE_data/ext_test.c index 1e986feacf..04e8ca322e 100644 --- a/lib/erl_interface/test/erl_ext_SUITE_data/ext_test.c +++ b/lib/erl_interface/test/erl_ext_SUITE_data/ext_test.c @@ -88,6 +88,11 @@ TESTCASE(compare_list) { // erlang:term_to_binary([0, 1000]) unsigned char term4[] = {131,108,0,0,0,2,97,0,98,0,0,3,232,106}; + // erlang:term_to_binary([a|b]) + unsigned char term5a[] = {131,108,0,0,0,1,100,0,1,97,100,0,1,98}; + // erlang:term_to_binary([a|c]) + unsigned char term5b[] = {131,108,0,0,0,1,100,0,1,97,100,0,1,99}; + erl_init(NULL, 0); start_a = term1; start_b = term2; @@ -103,6 +108,13 @@ TESTCASE(compare_list) { test_compare_ext("lists1", start_a, end_a, start_b, end_b, -1); + start_a = term5a; + start_b = term5b; + end_a = term5a + sizeof(term5a); + end_b = term5b + sizeof(term5b); + + test_compare_ext("lists5", start_a, end_a, start_b, end_b, -1); + report(1); } diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml index 6c4d4cb107..c5105dcba2 100644 --- a/lib/inets/doc/src/notes.xml +++ b/lib/inets/doc/src/notes.xml @@ -108,6 +108,34 @@ </section> + <section><title>Inets 6.5.2.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Do not use chunked-encoding with 1xx, 204 and 304 + responses when using mod_esi. Old behavior was not + compliant with HTTP/1.1 RFC and could cause clients to + hang when they received 1xx, 204 or 304 responses that + included an empty chunked-encoded body.</p> + <p> + Own Id: OTP-15241</p> + </item> + <item> + <p> + Add robust handling of chunked-encoded HTTP responses + with an empty body (1xx, 204, 304). Old behavior could + cause the client to hang when connecting to a faulty + server implementation.</p> + <p> + Own Id: OTP-15242</p> + </item> + </list> + </section> + + </section> + <section><title>Inets 6.5.2.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/doc/src/logger_chapter.xml b/lib/kernel/doc/src/logger_chapter.xml index 30172f6ca6..d58c4a4d42 100644 --- a/lib/kernel/doc/src/logger_chapter.xml +++ b/lib/kernel/doc/src/logger_chapter.xml @@ -507,7 +507,7 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <c>logger_level</c></seealso>. It is changed during runtime with <seealso marker="logger#set_primary_config-2"> <c>logger:set_primary_config(level,Level)</c></seealso>.</p> - <p>Defaults to <c>info</c>.</p> + <p>Defaults to <c>notice</c>.</p> </item> <tag><c>filters = [{FilterId,Filter}]</c></tag> <item> diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index 5884f93878..c766c18233 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -31,6 +31,23 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 6.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed bug in <c>net_kernel</c> that could cause an + emulator crash if certain connection attempts failed. Bug + exists since kernel-6.0 (OTP-21.0).</p> + <p> + Own Id: OTP-15280 Aux Id: ERIERL-226, OTP-15279 </p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 6.0</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/src/net_kernel.erl b/lib/kernel/src/net_kernel.erl index bea08242d8..ef92f9f4d1 100644 --- a/lib/kernel/src/net_kernel.erl +++ b/lib/kernel/src/net_kernel.erl @@ -279,24 +279,18 @@ passive_connect_monitor(From, Node) -> ok = monitor_nodes(true,[{node_type,all}]), Reply = case lists:member(Node,nodes([connected])) of true -> - io:format("~p: passive_connect_monitor ~p\n", [self(), ?LINE]), true; _ -> receive {nodeup,Node,_} -> - io:format("~p: passive_connect_monitor ~p\n", [self(), ?LINE]), true after connecttime() -> - io:format("~p: passive_connect_monitor ~p\n", [self(), ?LINE]), false end end, ok = monitor_nodes(false,[{node_type,all}]), - io:format("~p: passive_connect_monitor ~p\n", [self(), ?LINE]), {Pid, Tag} = From, - io:format("~p: passive_connect_monitor ~p\n", [self(), ?LINE]), - erlang:send(Pid, {Tag, Reply}), - io:format("~p: passive_connect_monitor ~p\n", [self(), ?LINE]). + erlang:send(Pid, {Tag, Reply}). %% If the net_kernel isn't running we ignore all requests to the @@ -358,20 +352,34 @@ init({Name, LongOrShortNames, TickT, CleanHalt}) -> {stop, Error} end. - -do_auto_connect(Type, Node, ConnId, WaitForBarred, From, State) -> - ConnLookup = ets:lookup(sys_dist, Node), - - case ConnLookup of +do_auto_connect_1(Node, ConnId, From, State) -> + case ets:lookup(sys_dist, Node) of [#barred_connection{}] -> - case WaitForBarred of - false -> - {reply, false, State}; - true -> + case ConnId of + passive_cnct -> spawn(?MODULE,passive_connect_monitor,[From,Node]), - {noreply, State} + {noreply, State}; + _ -> + erts_internal:abort_connection(Node, ConnId), + {reply, false, State} end; + ConnLookup -> + do_auto_connect_2(Node, ConnId, From, State, ConnLookup) + end. + +do_auto_connect_2(Node, passive_cnct, From, State, ConnLookup) -> + case (catch erts_internal:new_connection(Node)) of + {Nr,_DHandle}=ConnId when is_integer(Nr) -> + do_auto_connect_2(Node, ConnId, From, State, ConnLookup); + + _Error -> + error_logger:error_msg("~n** Cannot get connection id for node ~w~n", + [Node]), + {reply, false, State} + end; +do_auto_connect_2(Node, ConnId, From, State, ConnLookup) -> + case ConnLookup of [#connection{conn_id=ConnId, state = up}] -> {reply, true, State}; [#connection{conn_id=ConnId, waiting=Waiting}=Conn] -> @@ -385,6 +393,7 @@ do_auto_connect(Type, Node, ConnId, WaitForBarred, From, State) -> case application:get_env(kernel, dist_auto_connect) of {ok, never} -> ?connect_failure(Node,{dist_auto_connect,never}), + erts_internal:abort_connection(Node, ConnId), {reply, false, State}; %% This might happen due to connection close @@ -394,14 +403,16 @@ do_auto_connect(Type, Node, ConnId, WaitForBarred, From, State) -> (hd(ConnLookup))#connection.state =:= up -> ?connect_failure(Node,{barred_connection, ets:lookup(sys_dist, Node)}), + erts_internal:abort_connection(Node, ConnId), {reply, false, State}; _ -> - case setup(ConnLookup, Node,ConnId,Type,From,State) of + case setup(ConnLookup, Node,ConnId,normal,From,State) of {ok, SetupPid} -> Owners = [{SetupPid, Node} | State#state.conn_owners], {noreply,State#state{conn_owners=Owners}}; _Error -> ?connect_failure(Node, {setup_call, failed, _Error}), + erts_internal:abort_connection(Node, ConnId), {reply, false, State} end end @@ -454,18 +465,7 @@ handle_call({passive_cnct, Node}, From, State) when Node =:= node() -> async_reply({reply, true, State}, From); handle_call({passive_cnct, Node}, From, State) -> verbose({passive_cnct, Node}, 1, State), - Type = normal, - WaitForBarred = true, - R = case (catch erts_internal:new_connection(Node)) of - {Nr,_DHandle}=ConnId when is_integer(Nr) -> - do_auto_connect(Type, Node, ConnId, WaitForBarred, From, State); - - _Error -> - error_logger:error_msg("~n** Cannot get connection id for node ~w~n", - [Node]), - {reply, false, State} - end, - + R = do_auto_connect_1(Node, passive_cnct, From, State), return_call(R, From); %% @@ -479,7 +479,16 @@ handle_call({connect, Type, Node}, From, State) -> ConnLookup = ets:lookup(sys_dist, Node), R = case (catch erts_internal:new_connection(Node)) of {Nr,_DHandle}=ConnId when is_integer(Nr) -> - do_explicit_connect(ConnLookup, Type, Node, ConnId, From, State); + R1 = do_explicit_connect(ConnLookup, Type, Node, ConnId, From, State), + case R1 of + {reply, true, _S} -> %% already connected + ok; + {noreply, _S} -> %% connection pending + ok; + {reply, false, _S} -> %% connection refused + erts_internal:abort_connection(Node, ConnId) + end, + R1; _Error -> error_logger:error_msg("~n** Cannot get connection id for node ~w~n", @@ -703,7 +712,7 @@ handle_info({auto_connect,Node, Nr, DHandle}, State) -> verbose({auto_connect, Node, Nr, DHandle}, 1, State), ConnId = {Nr, DHandle}, NewState = - case do_auto_connect(normal, Node, ConnId, false, noreply, State) of + case do_auto_connect_1(Node, ConnId, noreply, State) of {noreply, S} -> %% Pending connection S; @@ -711,7 +720,6 @@ handle_info({auto_connect,Node, Nr, DHandle}, State) -> S; {reply, false, S} -> %% Connection refused - erts_internal:abort_connection(Node, ConnId), S end, {noreply, NewState}; diff --git a/lib/kernel/vsn.mk b/lib/kernel/vsn.mk index aa8e4dc119..fe22e2af98 100644 --- a/lib/kernel/vsn.mk +++ b/lib/kernel/vsn.mk @@ -1 +1 @@ -KERNEL_VSN = 6.0 +KERNEL_VSN = 6.0.1 diff --git a/lib/public_key/src/public_key.erl b/lib/public_key/src/public_key.erl index b34f905fc3..3704503f1e 100644 --- a/lib/public_key/src/public_key.erl +++ b/lib/public_key/src/public_key.erl @@ -593,8 +593,6 @@ verify(DigestOrPlainText, DigestType, Signature, Key) -> %% Backwards compatible verify(Digest, none, Signature, Key = {_, #'Dss-Parms'{}}, Options) when is_binary(Digest) -> verify({digest, Digest}, sha, Signature, Key, Options); -verify(Digest, none, Signature, Key = #'DSAPrivateKey'{}, Options) when is_binary(Digest) -> - verify({digest, Digest}, sha, Signature, Key, Options); verify(DigestOrPlainText, DigestType, Signature, Key, Options) when is_binary(Signature) -> case format_verify_key(Key) of badarg -> diff --git a/lib/sasl/test/sasl_report_SUITE.erl b/lib/sasl/test/sasl_report_SUITE.erl index a03932133e..e639b55cee 100644 --- a/lib/sasl/test/sasl_report_SUITE.erl +++ b/lib/sasl/test/sasl_report_SUITE.erl @@ -106,6 +106,9 @@ gen_server_crash(Config, Encoding) -> ok = rpc:call(Node,?MODULE,crash_me,[]), + ok = rpc:call(Node,logger_std_h,filesync,[default]), + ok = rpc:call(Node,logger_std_h,filesync,[sasl]), + test_server:stop_node(Node), ok = logger:remove_primary_filter(no_remote), diff --git a/lib/ssl/src/Makefile b/lib/ssl/src/Makefile index c0c55c6eb7..8d1341f594 100644 --- a/lib/ssl/src/Makefile +++ b/lib/ssl/src/Makefile @@ -66,6 +66,7 @@ MODULES= \ ssl_srp_primes \ tls_connection \ dtls_connection \ + tls_sender\ ssl_config \ ssl_connection \ tls_handshake \ diff --git a/lib/ssl/src/dtls_connection.erl b/lib/ssl/src/dtls_connection.erl index bf3ff3a9a7..2a0b2b317d 100644 --- a/lib/ssl/src/dtls_connection.erl +++ b/lib/ssl/src/dtls_connection.erl @@ -36,7 +36,7 @@ %% Internal application API %% Setup --export([start_fsm/8, start_link/7, init/1]). +-export([start_fsm/8, start_link/7, init/1, pids/1]). %% State transition handling -export([next_record/1, next_event/3, next_event/4, handle_common_event/4]). @@ -44,10 +44,10 @@ %% Handshake handling -export([renegotiate/2, send_handshake/2, queue_handshake/2, queue_change_cipher/2, - reinit_handshake_data/1, select_sni_extension/1, empty_connection_state/2]). + reinit/1, reinit_handshake_data/1, select_sni_extension/1, empty_connection_state/2]). %% Alert and close handling --export([encode_alert/3,send_alert/2, close/5, protocol_name/0]). +-export([encode_alert/3, send_alert/2, send_alert_in_connection/2, close/5, protocol_name/0]). %% Data handling -export([encode_data/3, passive_receive/2, next_record_if_active/1, @@ -72,7 +72,7 @@ start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = false},_, Tracker} try {ok, Pid} = dtls_connection_sup:start_child([Role, Host, Port, Socket, Opts, User, CbInfo]), - {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, Pid, CbModule, Tracker), + {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, [Pid], CbModule, Tracker), ssl_connection:handshake(SslSocket, Timeout) catch error:{badmatch, {error, _} = Error} -> @@ -100,6 +100,10 @@ init([Role, Host, Port, Socket, Options, User, CbInfo]) -> EState = State0#state{protocol_specific = Map#{error => Error}}, gen_statem:enter_loop(?MODULE, [], error, EState) end. + +pids(_) -> + [self()]. + %%==================================================================== %% State transition handling %%==================================================================== @@ -328,10 +332,14 @@ queue_change_cipher(ChangeCipher, #state{flight_buffer = Flight, dtls_record:next_epoch(ConnectionStates0, write), State#state{flight_buffer = Flight#{change_cipher_spec => ChangeCipher}, connection_states = ConnectionStates}. + +reinit(State) -> + %% To be API compatible with TLS NOOP here + reinit_handshake_data(State). reinit_handshake_data(#state{protocol_buffers = Buffers} = State) -> State#state{premaster_secret = undefined, public_key_info = undefined, - tls_handshake_history = ssl_handshake:init_handshake_history(), + tls_handshake_history = ssl_handshake:init_handshake_history(), flight_state = {retransmit, ?INITIAL_RETRANSMIT_TIMEOUT}, flight_buffer = new_flight(), protocol_buffers = @@ -365,6 +373,10 @@ send_alert(Alert, #state{negotiated_version = Version, send(Transport, Socket, BinMsg), State0#state{connection_states = ConnectionStates}. +send_alert_in_connection(Alert, State) -> + _ = send_alert(Alert, State), + ok. + close(downgrade, _,_,_,_) -> ok; %% Other @@ -710,6 +722,12 @@ connection(internal, #client_hello{}, #state{role = server, allow_renegotiate = State1 = send_alert(Alert, State0), {Record, State} = ssl_connection:prepare_connection(State1, ?MODULE), next_event(?FUNCTION_NAME, Record, State); +connection({call, From}, {application_data, Data}, State) -> + try + send_application_data(Data, From, ?FUNCTION_NAME, State) + catch throw:Error -> + ssl_connection:hibernate_after(?FUNCTION_NAME, State, [{reply, From, Error}]) + end; connection(Type, Event, State) -> ssl_connection:?FUNCTION_NAME(Type, Event, State, ?MODULE). @@ -1131,3 +1149,42 @@ log_ignore_alert(true, StateName, Alert, Role) -> [Role, StateName, Txt]); log_ignore_alert(false, _, _,_) -> ok. + +send_application_data(Data, From, _StateName, + #state{socket = Socket, + negotiated_version = Version, + protocol_cb = Connection, + transport_cb = Transport, + connection_states = ConnectionStates0, + ssl_options = #ssl_options{renegotiate_at = RenegotiateAt}} = State0) -> + + case time_to_renegotiate(Data, ConnectionStates0, RenegotiateAt) of + true -> + renegotiate(State0#state{renegotiation = {true, internal}}, + [{next_event, {call, From}, {application_data, Data}}]); + false -> + {Msgs, ConnectionStates} = + Connection:encode_data(Data, Version, ConnectionStates0), + State = State0#state{connection_states = ConnectionStates}, + case Connection:send(Transport, Socket, Msgs) of + ok -> + ssl_connection:hibernate_after(connection, State, [{reply, From, ok}]); + Result -> + ssl_connection:hibernate_after(connection, State, [{reply, From, Result}]) + end + end. + +time_to_renegotiate(_Data, + #{current_write := #{sequence_number := Num}}, + RenegotiateAt) -> + + %% We could do test: + %% is_time_to_renegotiate((erlang:byte_size(_Data) div + %% ?MAX_PLAIN_TEXT_LENGTH) + 1, RenegotiateAt), but we chose to + %% have a some what lower renegotiateAt and a much cheaper test + is_time_to_renegotiate(Num, RenegotiateAt). + +is_time_to_renegotiate(N, M) when N < M-> + false; +is_time_to_renegotiate(_,_) -> + true. diff --git a/lib/ssl/src/dtls_socket.erl b/lib/ssl/src/dtls_socket.erl index b26d3ae41a..2001afd02f 100644 --- a/lib/ssl/src/dtls_socket.erl +++ b/lib/ssl/src/dtls_socket.erl @@ -48,7 +48,7 @@ accept(dtls, #config{transport_info = {Transport,_,_,_}, dtls_handler = {Listner, _}}, _Timeout) -> case dtls_packet_demux:accept(Listner, self()) of {ok, Pid, Socket} -> - {ok, socket(Pid, Transport, {Listner, Socket}, ConnectionCb)}; + {ok, socket([Pid], Transport, {Listner, Socket}, ConnectionCb)}; {error, Reason} -> {error, Reason} end. @@ -73,12 +73,12 @@ close(gen_udp, {_Client, _Socket}) -> close(Transport, {_Client, Socket}) -> Transport:close(Socket). -socket(Pid, gen_udp = Transport, {{_, _}, Socket}, ConnectionCb) -> - #sslsocket{pid = Pid, +socket(Pids, gen_udp = Transport, {{_, _}, Socket}, ConnectionCb) -> + #sslsocket{pid = Pids, %% "The name "fd" is keept for backwards compatibility fd = {Transport, Socket, ConnectionCb}}; -socket(Pid, Transport, Socket, ConnectionCb) -> - #sslsocket{pid = Pid, +socket(Pids, Transport, Socket, ConnectionCb) -> + #sslsocket{pid = Pids, %% "The name "fd" is keept for backwards compatibility fd = {Transport, Socket, ConnectionCb}}. setopts(_, #sslsocket{pid = {dtls, #config{dtls_handler = {ListenPid, _}}}}, Options) -> diff --git a/lib/ssl/src/inet_tls_dist.erl b/lib/ssl/src/inet_tls_dist.erl index aa3d7e3f72..ca059603ae 100644 --- a/lib/ssl/src/inet_tls_dist.erl +++ b/lib/ssl/src/inet_tls_dist.erl @@ -69,14 +69,14 @@ is_node_name(Node) -> %% ------------------------------------------------------------------------- -hs_data_common(#sslsocket{pid = DistCtrl} = SslSocket) -> +hs_data_common(#sslsocket{pid = [_, DistCtrl|_]} = SslSocket) -> #hs_data{ f_send = - fun (Ctrl, Packet) when Ctrl == DistCtrl -> + fun (_Ctrl, Packet) -> f_send(SslSocket, Packet) end, f_recv = - fun (Ctrl, Length, Timeout) when Ctrl == DistCtrl -> + fun (_, Length, Timeout) -> f_recv(SslSocket, Length, Timeout) end, f_setopts_pre_nodeup = @@ -175,8 +175,7 @@ mf_getopts(SslSocket, Opts) -> ssl:getopts(SslSocket, Opts). f_handshake_complete(DistCtrl, Node, DHandle) -> - ssl_connection:handshake_complete(DistCtrl, Node, DHandle). - + tls_sender:dist_handshake_complete(DistCtrl, Node, DHandle). setopts_filter(Opts) -> [Opt || {K,_} = Opt <- Opts, @@ -244,7 +243,7 @@ accept_loop(Driver, Listen, Kernel, Socket) -> trace([{active, false},{packet, 4}|Opts]), net_kernel:connecttime()) of - {ok, #sslsocket{pid = DistCtrl} = SslSocket} -> + {ok, #sslsocket{pid = [_, DistCtrl| _]} = SslSocket} -> trace( Kernel ! {accept, self(), DistCtrl, @@ -404,7 +403,7 @@ gen_accept_connection( do_accept( _Driver, AcceptPid, DistCtrl, MyNode, Allowed, SetupTime, Kernel) -> - SslSocket = ssl_connection:get_sslsocket(DistCtrl), + {ok, SslSocket} = tls_sender:dist_tls_socket(DistCtrl), receive {AcceptPid, controller} -> Timer = dist_util:start_timer(SetupTime), @@ -529,7 +528,7 @@ do_setup_connect(Driver, Kernel, Node, Address, Ip, TcpPort, Version, Type, MyNo [binary, {active, false}, {packet, 4}, Driver:family(), nodelay()] ++ Opts, net_kernel:connecttime()) of - {ok, #sslsocket{pid = DistCtrl} = SslSocket} -> + {ok, #sslsocket{pid = [_, DistCtrl| _]} = SslSocket} -> _ = monitor_pid(DistCtrl), ok = ssl:controlling_process(SslSocket, self()), HSData0 = hs_data_common(SslSocket), diff --git a/lib/ssl/src/ssl.app.src b/lib/ssl/src/ssl.app.src index 41871260fa..936df12e70 100644 --- a/lib/ssl/src/ssl.app.src +++ b/lib/ssl/src/ssl.app.src @@ -10,6 +10,7 @@ tls_v1, ssl_v3, tls_connection_sup, + tls_sender, %% DTLS dtls_connection, dtls_handshake, diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index 71d1a28f98..4cf56035ba 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -231,7 +231,7 @@ handshake(#sslsocket{fd = {_, _, _, Tracker}} = Socket, SslOpts, Timeout) when catch Error = {error, _Reason} -> Error end; -handshake(#sslsocket{pid = Pid, fd = {_, _, _}} = Socket, SslOpts, Timeout) when +handshake(#sslsocket{pid = [Pid|_], fd = {_, _, _}} = Socket, SslOpts, Timeout) when (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity)-> try {ok, EmOpts, _} = dtls_packet_demux:get_all_opts(Pid), @@ -291,7 +291,7 @@ handshake_cancel(Socket) -> %% %% Description: Close an ssl connection %%-------------------------------------------------------------------- -close(#sslsocket{pid = Pid}) when is_pid(Pid) -> +close(#sslsocket{pid = [Pid|_]}) when is_pid(Pid) -> ssl_connection:close(Pid, {close, ?DEFAULT_TIMEOUT}); close(#sslsocket{pid = {dtls, #config{dtls_handler = {Pid, _}}}}) -> dtls_packet_demux:close(Pid); @@ -303,12 +303,12 @@ close(#sslsocket{pid = {ListenSocket, #config{transport_info={Transport,_, _, _} %% %% Description: Close an ssl connection %%-------------------------------------------------------------------- -close(#sslsocket{pid = TLSPid}, +close(#sslsocket{pid = [TLSPid|_]}, {Pid, Timeout} = DownGrade) when is_pid(TLSPid), is_pid(Pid), (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity) -> ssl_connection:close(TLSPid, {close, DownGrade}); -close(#sslsocket{pid = TLSPid}, Timeout) when is_pid(TLSPid), +close(#sslsocket{pid = [TLSPid|_]}, Timeout) when is_pid(TLSPid), (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity) -> ssl_connection:close(TLSPid, {close, Timeout}); close(#sslsocket{pid = {ListenSocket, #config{transport_info={Transport,_, _, _}}}}, _) -> @@ -319,8 +319,10 @@ close(#sslsocket{pid = {ListenSocket, #config{transport_info={Transport,_, _, _} %% %% Description: Sends data over the ssl connection %%-------------------------------------------------------------------- -send(#sslsocket{pid = Pid}, Data) when is_pid(Pid) -> +send(#sslsocket{pid = [Pid]}, Data) when is_pid(Pid) -> ssl_connection:send(Pid, Data); +send(#sslsocket{pid = [_, Pid]}, Data) when is_pid(Pid) -> + tls_sender:send_data(Pid, erlang:iolist_to_binary(Data)); send(#sslsocket{pid = {_, #config{transport_info={_, udp, _, _}}}}, _) -> {error,enotconn}; %% Emulate connection behaviour send(#sslsocket{pid = {dtls,_}}, _) -> @@ -336,7 +338,7 @@ send(#sslsocket{pid = {ListenSocket, #config{transport_info={Transport, _, _, _} %%-------------------------------------------------------------------- recv(Socket, Length) -> recv(Socket, Length, infinity). -recv(#sslsocket{pid = Pid}, Length, Timeout) when is_pid(Pid), +recv(#sslsocket{pid = [Pid|_]}, Length, Timeout) when is_pid(Pid), (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity)-> ssl_connection:recv(Pid, Length, Timeout); recv(#sslsocket{pid = {dtls,_}}, _, _) -> @@ -351,7 +353,7 @@ recv(#sslsocket{pid = {Listen, %% Description: Changes process that receives the messages when active = true %% or once. %%-------------------------------------------------------------------- -controlling_process(#sslsocket{pid = Pid}, NewOwner) when is_pid(Pid), is_pid(NewOwner) -> +controlling_process(#sslsocket{pid = [Pid|_]}, NewOwner) when is_pid(Pid), is_pid(NewOwner) -> ssl_connection:new_user(Pid, NewOwner); controlling_process(#sslsocket{pid = {dtls, _}}, NewOwner) when is_pid(NewOwner) -> @@ -369,7 +371,7 @@ controlling_process(#sslsocket{pid = {Listen, %% %% Description: Return SSL information for the connection %%-------------------------------------------------------------------- -connection_information(#sslsocket{pid = Pid}) when is_pid(Pid) -> +connection_information(#sslsocket{pid = [Pid|_]}) when is_pid(Pid) -> case ssl_connection:connection_information(Pid, false) of {ok, Info} -> {ok, [Item || Item = {_Key, Value} <- Info, Value =/= undefined]}; @@ -386,7 +388,7 @@ connection_information(#sslsocket{pid = {dtls,_}}) -> %% %% Description: Return SSL information for the connection %%-------------------------------------------------------------------- -connection_information(#sslsocket{pid = Pid}, Items) when is_pid(Pid) -> +connection_information(#sslsocket{pid = [Pid|_]}, Items) when is_pid(Pid) -> case ssl_connection:connection_information(Pid, include_security_info(Items)) of {ok, Info} -> {ok, [Item || Item = {Key, Value} <- Info, lists:member(Key, Items), @@ -400,9 +402,9 @@ connection_information(#sslsocket{pid = Pid}, Items) when is_pid(Pid) -> %% %% Description: same as inet:peername/1. %%-------------------------------------------------------------------- -peername(#sslsocket{pid = Pid, fd = {Transport, Socket, _}}) when is_pid(Pid)-> +peername(#sslsocket{pid = [Pid|_], fd = {Transport, Socket, _}}) when is_pid(Pid)-> dtls_socket:peername(Transport, Socket); -peername(#sslsocket{pid = Pid, fd = {Transport, Socket, _, _}}) when is_pid(Pid)-> +peername(#sslsocket{pid = [Pid|_], fd = {Transport, Socket, _, _}}) when is_pid(Pid)-> tls_socket:peername(Transport, Socket); peername(#sslsocket{pid = {dtls, #config{dtls_handler = {_Pid, _}}}}) -> dtls_socket:peername(dtls, undefined); @@ -416,7 +418,7 @@ peername(#sslsocket{pid = {dtls,_}}) -> %% %% Description: Returns the peercert. %%-------------------------------------------------------------------- -peercert(#sslsocket{pid = Pid}) when is_pid(Pid) -> +peercert(#sslsocket{pid = [Pid|_]}) when is_pid(Pid) -> case ssl_connection:peer_certificate(Pid) of {ok, undefined} -> {error, no_peercert}; @@ -434,7 +436,7 @@ peercert(#sslsocket{pid = {Listen, _}}) when is_port(Listen) -> %% Description: Returns the protocol that has been negotiated. If no %% protocol has been negotiated will return {error, protocol_not_negotiated} %%-------------------------------------------------------------------- -negotiated_protocol(#sslsocket{pid = Pid}) -> +negotiated_protocol(#sslsocket{pid = [Pid|_]}) when is_pid(Pid) -> ssl_connection:negotiated_protocol(Pid). %%-------------------------------------------------------------------- @@ -571,7 +573,7 @@ eccs_filter_supported(Curves) -> %% %% Description: Gets options %%-------------------------------------------------------------------- -getopts(#sslsocket{pid = Pid}, OptionTags) when is_pid(Pid), is_list(OptionTags) -> +getopts(#sslsocket{pid = [Pid|_]}, OptionTags) when is_pid(Pid), is_list(OptionTags) -> ssl_connection:get_opts(Pid, OptionTags); getopts(#sslsocket{pid = {dtls, #config{transport_info = {Transport,_,_,_}}}} = ListenSocket, OptionTags) when is_list(OptionTags) -> try dtls_socket:getopts(Transport, ListenSocket, OptionTags) of @@ -602,7 +604,7 @@ getopts(#sslsocket{}, OptionTags) -> %% %% Description: Sets options %%-------------------------------------------------------------------- -setopts(#sslsocket{pid = Pid}, Options0) when is_pid(Pid), is_list(Options0) -> +setopts(#sslsocket{pid = [Pid|_]}, Options0) when is_pid(Pid), is_list(Options0) -> try proplists:expand([{binary, [{mode, binary}]}, {list, [{mode, list}]}], Options0) of Options -> @@ -657,7 +659,7 @@ getstat(Socket) -> getstat(#sslsocket{pid = {Listen, #config{transport_info = {Transport, _, _, _}}}}, Options) when is_port(Listen), is_list(Options) -> tls_socket:getstat(Transport, Listen, Options); -getstat(#sslsocket{pid = Pid, fd = {Transport, Socket, _, _}}, Options) when is_pid(Pid), is_list(Options) -> +getstat(#sslsocket{pid = [Pid|_], fd = {Transport, Socket, _, _}}, Options) when is_pid(Pid), is_list(Options) -> tls_socket:getstat(Transport, Socket, Options). %%--------------------------------------------------------------- @@ -670,7 +672,7 @@ shutdown(#sslsocket{pid = {Listen, #config{transport_info = {Transport,_, _, _}} Transport:shutdown(Listen, How); shutdown(#sslsocket{pid = {dtls,_}},_) -> {error, enotconn}; -shutdown(#sslsocket{pid = Pid}, How) -> +shutdown(#sslsocket{pid = [Pid|_]}, How) when is_pid(Pid) -> ssl_connection:shutdown(Pid, How). %%-------------------------------------------------------------------- @@ -682,9 +684,9 @@ sockname(#sslsocket{pid = {Listen, #config{transport_info = {Transport, _, _, _ tls_socket:sockname(Transport, Listen); sockname(#sslsocket{pid = {dtls, #config{dtls_handler = {Pid, _}}}}) -> dtls_packet_demux:sockname(Pid); -sockname(#sslsocket{pid = Pid, fd = {Transport, Socket, _}}) when is_pid(Pid) -> +sockname(#sslsocket{pid = [Pid|_], fd = {Transport, Socket, _}}) when is_pid(Pid) -> dtls_socket:sockname(Transport, Socket); -sockname(#sslsocket{pid = Pid, fd = {Transport, Socket, _, _}}) when is_pid(Pid) -> +sockname(#sslsocket{pid = [Pid| _], fd = {Transport, Socket, _, _}}) when is_pid(Pid) -> tls_socket:sockname(Transport, Socket). %%--------------------------------------------------------------- @@ -713,7 +715,15 @@ versions() -> %% %% Description: Initiates a renegotiation. %%-------------------------------------------------------------------- -renegotiate(#sslsocket{pid = Pid}) when is_pid(Pid) -> +renegotiate(#sslsocket{pid = [Pid, Sender |_]}) when is_pid(Pid), + is_pid(Sender) -> + case tls_sender:renegotiate(Sender) of + {ok, Write} -> + tls_connection:renegotiation(Pid, Write); + Error -> + Error + end; +renegotiate(#sslsocket{pid = [Pid |_]}) when is_pid(Pid) -> ssl_connection:renegotiation(Pid); renegotiate(#sslsocket{pid = {dtls,_}}) -> {error, enotconn}; @@ -727,7 +737,7 @@ renegotiate(#sslsocket{pid = {Listen,_}}) when is_port(Listen) -> %% %% Description: use a ssl sessions TLS PRF to generate key material %%-------------------------------------------------------------------- -prf(#sslsocket{pid = Pid}, +prf(#sslsocket{pid = [Pid|_]}, Secret, Label, Seed, WantedLength) when is_pid(Pid) -> ssl_connection:prf(Pid, Secret, Label, Seed, WantedLength); prf(#sslsocket{pid = {dtls,_}}, _,_,_,_) -> diff --git a/lib/ssl/src/ssl_api.hrl b/lib/ssl/src/ssl_api.hrl index 144323c572..7579b56ab0 100644 --- a/lib/ssl/src/ssl_api.hrl +++ b/lib/ssl/src/ssl_api.hrl @@ -42,7 +42,8 @@ {verify, verify_type()} | {verify_fun, {fun(), InitialUserState::term()}} | {fail_if_no_peer_cert, boolean()} | {depth, integer()} | - {cert, Der::binary()} | {certfile, path()} | {key, Der::binary()} | + {cert, Der::binary()} | {certfile, path()} | + {key, {private_key_type(), Der::binary()}} | {keyfile, path()} | {password, string()} | {cacerts, [Der::binary()]} | {cacertfile, path()} | {dh, Der::binary()} | {dhfile, path()} | {user_lookup_fun, {fun(), InitialUserState::term()}} | @@ -65,4 +66,11 @@ ClosedTag::atom(), ErrTag::atom()}}. -type prf_random() :: client_random | server_random. +-type private_key_type() :: rsa | %% Backwards compatibility + dsa | %% Backwards compatibility + 'RSAPrivateKey' | + 'DSAPrivateKey' | + 'ECPrivateKey' | + 'PrivateKeyInfo'. + -endif. % -ifdef(ssl_api). diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index 05433b15ba..5ea1924d40 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -55,7 +55,7 @@ ]). %% Data handling --export([write_application_data/3, read_application_data/2]). +-export([read_application_data/2, internal_renegotiation/2]). %% Help functions for tls|dtls_connection.erl -export([handle_session/7, ssl_config/3, @@ -64,13 +64,13 @@ %% General gen_statem state functions with extra callback argument %% to determine if it is an SSL/TLS or DTLS gen_statem machine -export([init/4, error/4, hello/4, user_hello/4, abbreviated/4, certify/4, cipher/4, - connection/4, death_row/4, downgrade/4]). + connection/4, downgrade/4]). %% gen_statem callbacks -export([terminate/3, format_status/2]). %% Erlang Distribution export --export([get_sslsocket/1, handshake_complete/3]). +-export([get_sslsocket/1, dist_handshake_complete/2]). %%==================================================================== %% Setup @@ -118,7 +118,7 @@ handshake(Connection, Port, Socket, Opts, User, CbInfo, Timeout) -> %% %% Description: Starts ssl handshake. %%-------------------------------------------------------------------- -handshake(#sslsocket{pid = Pid} = Socket, Timeout) -> +handshake(#sslsocket{pid = [Pid|_]} = Socket, Timeout) -> case call(Pid, {start, Timeout}) of connected -> {ok, Socket}; @@ -134,7 +134,7 @@ handshake(#sslsocket{pid = Pid} = Socket, Timeout) -> %% %% Description: Starts ssl handshake with some new options %%-------------------------------------------------------------------- -handshake(#sslsocket{pid = Pid} = Socket, SslOptions, Timeout) -> +handshake(#sslsocket{pid = [Pid|_]} = Socket, SslOptions, Timeout) -> case call(Pid, {start, SslOptions, Timeout}) of connected -> {ok, Socket}; @@ -148,7 +148,7 @@ handshake(#sslsocket{pid = Pid} = Socket, SslOptions, Timeout) -> %% %% Description: Continues handshake with new options %%-------------------------------------------------------------------- -handshake_continue(#sslsocket{pid = Pid} = Socket, SslOptions, Timeout) -> +handshake_continue(#sslsocket{pid = [Pid|_]} = Socket, SslOptions, Timeout) -> case call(Pid, {handshake_continue, SslOptions, Timeout}) of connected -> {ok, Socket}; @@ -160,7 +160,7 @@ handshake_continue(#sslsocket{pid = Pid} = Socket, SslOptions, Timeout) -> %% %% Description: Cancels connection %%-------------------------------------------------------------------- -handshake_cancel(#sslsocket{pid = Pid}) -> +handshake_cancel(#sslsocket{pid = [Pid|_]}) -> case call(Pid, cancel) of closed -> ok; @@ -168,7 +168,7 @@ handshake_cancel(#sslsocket{pid = Pid}) -> Error end. %-------------------------------------------------------------------- --spec socket_control(tls_connection | dtls_connection, port(), pid(), atom()) -> +-spec socket_control(tls_connection | dtls_connection, port(), [pid()], atom()) -> {ok, #sslsocket{}} | {error, reason()}. %% %% Description: Set the ssl process to own the accept socket @@ -177,24 +177,24 @@ socket_control(Connection, Socket, Pid, Transport) -> socket_control(Connection, Socket, Pid, Transport, undefined). %-------------------------------------------------------------------- --spec socket_control(tls_connection | dtls_connection, port(), pid(), atom(), pid()| undefined) -> +-spec socket_control(tls_connection | dtls_connection, port(), [pid()], atom(), pid()| atom()) -> {ok, #sslsocket{}} | {error, reason()}. %%-------------------------------------------------------------------- -socket_control(Connection, Socket, Pid, Transport, udp_listener) -> +socket_control(Connection, Socket, Pids, Transport, udp_listener) -> %% dtls listener process must have the socket control - {ok, Connection:socket(Pid, Transport, Socket, Connection, undefined)}; + {ok, Connection:socket(Pids, Transport, Socket, Connection, undefined)}; -socket_control(tls_connection = Connection, Socket, Pid, Transport, ListenTracker) -> +socket_control(tls_connection = Connection, Socket, [Pid|_] = Pids, Transport, ListenTracker) -> case Transport:controlling_process(Socket, Pid) of ok -> - {ok, Connection:socket(Pid, Transport, Socket, Connection, ListenTracker)}; + {ok, Connection:socket(Pids, Transport, Socket, Connection, ListenTracker)}; {error, Reason} -> {error, Reason} end; -socket_control(dtls_connection = Connection, {_, Socket}, Pid, Transport, ListenTracker) -> +socket_control(dtls_connection = Connection, {_, Socket}, [Pid|_] = Pids, Transport, ListenTracker) -> case Transport:controlling_process(Socket, Pid) of ok -> - {ok, Connection:socket(Pid, Transport, Socket, Connection, ListenTracker)}; + {ok, Connection:socket(Pids, Transport, Socket, Connection, ListenTracker)}; {error, Reason} -> {error, Reason} end. @@ -306,12 +306,20 @@ peer_certificate(ConnectionPid) -> renegotiation(ConnectionPid) -> call(ConnectionPid, renegotiate). +%%-------------------------------------------------------------------- +-spec internal_renegotiation(pid(), ssl_record:connection_states()) -> + ok. +%% +%% Description: Starts a renegotiation of the ssl session. +%%-------------------------------------------------------------------- +internal_renegotiation(ConnectionPid, #{current_write := WriteState}) -> + gen_statem:cast(ConnectionPid, {internal_renegotiate, WriteState}). get_sslsocket(ConnectionPid) -> call(ConnectionPid, get_sslsocket). -handshake_complete(ConnectionPid, Node, DHandle) -> - call(ConnectionPid, {handshake_complete, Node, DHandle}). +dist_handshake_complete(ConnectionPid, DHandle) -> + gen_statem:cast(ConnectionPid, {dist_handshake_complete, DHandle}). %%-------------------------------------------------------------------- -spec prf(pid(), binary() | 'master_secret', binary(), @@ -335,7 +343,7 @@ handle_own_alert(Alert, Version, StateName, ssl_options = SslOpts} = State) -> try %% Try to tell the other side {BinMsg, _} = - Connection:encode_alert(Alert, Version, ConnectionStates), + Connection:encode_alert(Alert, Version, ConnectionStates), Connection:send(Transport, Socket, BinMsg) catch _:_ -> %% Can crash if we are in a uninitialized state ignore @@ -353,8 +361,9 @@ handle_normal_shutdown(Alert, _, #state{socket = Socket, protocol_cb = Connection, start_or_recv_from = StartFrom, tracker = Tracker, - role = Role, renegotiation = {false, first}}) -> - alert_user(Transport, Tracker,Socket, StartFrom, Alert, Role, Connection); + role = Role, renegotiation = {false, first}} = State) -> + Pids = Connection:pids(State), + alert_user(Pids, Transport, Tracker,Socket, StartFrom, Alert, Role, Connection); handle_normal_shutdown(Alert, StateName, #state{socket = Socket, socket_options = Opts, @@ -362,8 +371,9 @@ handle_normal_shutdown(Alert, StateName, #state{socket = Socket, protocol_cb = Connection, user_application = {_Mon, Pid}, tracker = Tracker, - start_or_recv_from = RecvFrom, role = Role}) -> - alert_user(Transport, Tracker, Socket, StateName, Opts, Pid, RecvFrom, Alert, Role, Connection). + start_or_recv_from = RecvFrom, role = Role} = State) -> + Pids = Connection:pids(State), + alert_user(Pids, Transport, Tracker, Socket, StateName, Opts, Pid, RecvFrom, Alert, Role, Connection). handle_alert(#alert{level = ?FATAL} = Alert, StateName, #state{socket = Socket, transport_cb = Transport, @@ -374,7 +384,8 @@ handle_alert(#alert{level = ?FATAL} = Alert, StateName, invalidate_session(Role, Host, Port, Session), log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), - alert_user(Transport, Tracker, Socket, StateName, Opts, Pid, From, Alert, Role, Connection), + Pids = Connection:pids(State), + alert_user(Pids, Transport, Tracker, Socket, StateName, Opts, Pid, From, Alert, Role, Connection), stop(normal, State); handle_alert(#alert{level = ?WARNING, description = ?CLOSE_NOTIFY} = Alert, @@ -383,12 +394,24 @@ handle_alert(#alert{level = ?WARNING, description = ?CLOSE_NOTIFY} = Alert, stop({shutdown, peer_close}, State); handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, StateName, - #state{role = Role, ssl_options = SslOpts, protocol_cb = Connection, renegotiation = {true, internal}} = State) -> + #state{role = Role, ssl_options = SslOpts, protocol_cb = Connection, + renegotiation = {true, internal}} = State) -> log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), handle_normal_shutdown(Alert, StateName, State), stop({shutdown, peer_close}, State); +handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, connection = StateName, + #state{role = Role, + ssl_options = SslOpts, renegotiation = {true, From}, + protocol_cb = Connection} = State0) -> + log_alert(SslOpts#ssl_options.log_alert, Role, + Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), + gen_statem:reply(From, {error, renegotiation_rejected}), + State1 = Connection:reinit_handshake_data(State0), + {Record, State} = Connection:next_record(State1#state{renegotiation = undefined}), + Connection:next_event(connection, Record, State); + handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, StateName, #state{role = Role, ssl_options = SslOpts, renegotiation = {true, From}, @@ -398,7 +421,7 @@ handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, gen_statem:reply(From, {error, renegotiation_rejected}), {Record, State1} = Connection:next_record(State0), %% Go back to connection! - State = Connection:reinit_handshake_data(State1#state{renegotiation = undefined}), + State = Connection:reinit(State1#state{renegotiation = undefined}), Connection:next_event(connection, Record, State); %% Gracefully log and ignore all other warning alerts @@ -412,36 +435,6 @@ handle_alert(#alert{level = ?WARNING} = Alert, StateName, %%==================================================================== %% Data handling %%==================================================================== -write_application_data(Data0, {FromPid, _} = From, - #state{socket = Socket, - negotiated_version = Version, - protocol_cb = Connection, - transport_cb = Transport, - connection_states = ConnectionStates0, - socket_options = SockOpts, - ssl_options = #ssl_options{renegotiate_at = RenegotiateAt}} = State) -> - Data = encode_packet(Data0, SockOpts), - - case time_to_renegotiate(Data, ConnectionStates0, RenegotiateAt) of - true -> - Connection:renegotiate(State#state{renegotiation = {true, internal}}, - [{next_event, {call, From}, {application_data, Data0}}]); - false -> - {Msgs, ConnectionStates} = - Connection:encode_data(Data, Version, ConnectionStates0), - NewState = State#state{connection_states = ConnectionStates}, - case Connection:send(Transport, Socket, Msgs) of - ok when FromPid =:= self() -> - hibernate_after(connection, NewState, []); - Error when FromPid =:= self() -> - stop({shutdown, Error}, NewState); - ok -> - hibernate_after(connection, NewState, [{reply, From, ok}]); - Result -> - hibernate_after(connection, NewState, [{reply, From, Result}]) - end - end. - read_application_data(Data, #state{user_application = {_Mon, Pid}, socket = Socket, protocol_cb = Connection, @@ -459,58 +452,74 @@ read_application_data(Data, #state{user_application = {_Mon, Pid}, end, case get_data(SOpts, BytesToRead, Buffer1) of {ok, ClientData, Buffer} -> % Send data - case State0 of - #state{ - ssl_options = #ssl_options{erl_dist = true}, - protocol_specific = #{d_handle := DHandle}} -> - State = - State0#state{ - user_data_buffer = Buffer, - bytes_to_read = undefined}, - try erlang:dist_ctrl_put_data(DHandle, ClientData) of - _ - when SOpts#socket_options.active =:= false; - Buffer =:= <<>> -> - %% Passive mode, wait for active once or recv - %% Active and empty, get more data - Connection:next_record_if_active(State); - _ -> %% We have more data - read_application_data(<<>>, State) - catch error:_ -> - death_row(State, disconnect) - end; - _ -> - SocketOpt = - deliver_app_data( - Transport, Socket, SOpts, - ClientData, Pid, RecvFrom, Tracker, Connection), - cancel_timer(Timer), - State = - State0#state{ - user_data_buffer = Buffer, + #state{ssl_options = #ssl_options{erl_dist = Dist}, + erl_dist_data = DistData} = State0, + case Dist andalso is_dist_up(DistData) of + true -> + dist_app_data(ClientData, State0#state{user_data_buffer = Buffer, + bytes_to_read = undefined}); + _ -> + SocketOpt = + deliver_app_data(Connection:pids(State0), + Transport, Socket, SOpts, + ClientData, Pid, RecvFrom, Tracker, Connection), + cancel_timer(Timer), + State = + State0#state{ + user_data_buffer = Buffer, start_or_recv_from = undefined, timer = undefined, bytes_to_read = undefined, socket_options = SocketOpt - }, - if - SocketOpt#socket_options.active =:= false; - Buffer =:= <<>> -> - %% Passive mode, wait for active once or recv + }, + if + SocketOpt#socket_options.active =:= false; + Buffer =:= <<>> -> + %% Passive mode, wait for active once or recv %% Active and empty, get more data - Connection:next_record_if_active(State); - true -> %% We have more data - read_application_data(<<>>, State) - end - end; + Connection:next_record_if_active(State); + true -> %% We have more data + read_application_data(<<>>, State) + end + end; {more, Buffer} -> % no reply, we need more data Connection:next_record(State0#state{user_data_buffer = Buffer}); {passive, Buffer} -> Connection:next_record_if_active(State0#state{user_data_buffer = Buffer}); {error,_Reason} -> %% Invalid packet in packet mode - deliver_packet_error(Transport, Socket, SOpts, Buffer1, Pid, RecvFrom, Tracker, Connection), + deliver_packet_error(Connection:pids(State0), + Transport, Socket, SOpts, Buffer1, Pid, RecvFrom, Tracker, Connection), stop(normal, State0) end. + +dist_app_data(ClientData, #state{protocol_cb = Connection, + erl_dist_data = #{dist_handle := undefined, + dist_buffer := DistBuff} = DistData} = State) -> + Connection:next_record_if_active(State#state{erl_dist_data = DistData#{dist_buffer => [ClientData, DistBuff]}}); +dist_app_data(ClientData, #state{erl_dist_data = #{dist_handle := DHandle, + dist_buffer := DistBuff} = ErlDistData, + protocol_cb = Connection, + user_data_buffer = Buffer, + socket_options = SOpts} = State) -> + Data = merge_dist_data(DistBuff, ClientData), + try erlang:dist_ctrl_put_data(DHandle, Data) of + _ when SOpts#socket_options.active =:= false; + Buffer =:= <<>> -> + %% Passive mode, wait for active once or recv + %% Active and empty, get more data + Connection:next_record_if_active(State#state{erl_dist_data = ErlDistData#{dist_buffer => <<>>}}); + _ -> %% We have more data + read_application_data(<<>>, State) + catch error:_ -> + stop(State, disconnect) + end. + +merge_dist_data(<<>>, ClientData) -> + ClientData; +merge_dist_data(DistBuff, <<>>) -> + DistBuff; +merge_dist_data(DistBuff, ClientData) -> + [DistBuff, ClientData]. %%==================================================================== %% Help functions for tls|dtls_connection.erl %%==================================================================== @@ -610,12 +619,6 @@ init({call, From}, {start, {Opts, EmOpts}, Timeout}, socket_options = SockOpts} = State0, Connection) -> try SslOpts = ssl:handle_options(Opts, OrigSSLOptions), - case SslOpts of - #ssl_options{erl_dist = true} -> - process_flag(priority, max); - _ -> - ok - end, State = ssl_config(SslOpts, Role, State0), init({call, From}, {start, Timeout}, State#state{ssl_options = SslOpts, @@ -728,8 +731,8 @@ abbreviated(internal, #next_protocol{selected_protocol = SelectedProtocol}, Connection:next_event(?FUNCTION_NAME, Record, State#state{expecting_next_protocol_negotiation = false}); abbreviated(internal, - #change_cipher_spec{type = <<1>>}, #state{connection_states = ConnectionStates0} = - State0, Connection) -> + #change_cipher_spec{type = <<1>>}, + #state{connection_states = ConnectionStates0} = State0, Connection) -> ConnectionStates1 = ssl_record:activate_pending_connection_state(ConnectionStates0, read, Connection), {Record, State} = Connection:next_record(State0#state{connection_states = @@ -1025,22 +1028,6 @@ cipher(Type, Msg, State, Connection) -> #state{}, tls_connection | dtls_connection) -> gen_statem:state_function_result(). %%-------------------------------------------------------------------- -connection({call, {FromPid, _} = From}, {application_data, Data}, - #state{protocol_cb = Connection} = State, Connection) -> - %% We should look into having a worker process to do this to - %% parallize send and receive decoding and not block the receiver - %% if sending is overloading the socket. - try - write_application_data(Data, From, State) - catch throw:Error -> - case self() of - FromPid -> - stop({shutdown, Error}, State); - _ -> - hibernate_after( - ?FUNCTION_NAME, State, [{reply, From, Error}]) - end - end; connection({call, RecvFrom}, {recv, N, Timeout}, #state{protocol_cb = Connection, socket_options = #socket_options{active = false}} = State0, Connection) -> @@ -1067,64 +1054,25 @@ connection({call, From}, negotiated_protocol, #state{negotiated_protocol = SelectedProtocol} = State, _) -> hibernate_after(?FUNCTION_NAME, State, [{reply, From, {ok, SelectedProtocol}}]); -connection( - {call, From}, {handshake_complete, _Node, DHandle}, - #state{ - ssl_options = #ssl_options{erl_dist = true}, - socket_options = SockOpts, - protocol_specific = ProtocolSpecific} = State, - Connection) -> - %% From now on we execute on normal priority - process_flag(priority, normal), - try erlang:dist_ctrl_get_data_notification(DHandle) of - _ -> - NewState = - State#state{ - socket_options = - SockOpts#socket_options{active = true}, - protocol_specific = - ProtocolSpecific#{d_handle => DHandle}}, - {Record, NewerState} = Connection:next_record_if_active(NewState), - Connection:next_event(connection, Record, NewerState, [{reply, From, ok}]) - catch error:_ -> - death_row(State, disconnect) - end; connection({call, From}, Msg, State, Connection) -> handle_call(Msg, From, ?FUNCTION_NAME, State, Connection); -connection( - info, dist_data = Msg, - #state{ - ssl_options = #ssl_options{erl_dist = true}, - protocol_specific = #{d_handle := DHandle}} = State, - _) -> - eat_msgs(Msg), - try send_dist_data(?FUNCTION_NAME, State, DHandle, []) - catch error:_ -> - death_row(State, disconnect) - end; -connection( - info, {send, From, Ref, Data}, - #state{ - ssl_options = #ssl_options{erl_dist = true}, - protocol_specific = #{d_handle := _}}, - _) -> - %% This is for testing only! - %% - %% Needed by some OTP distribution - %% test suites... - From ! {Ref, ok}, - {keep_state_and_data, - [{next_event, {call, {self(), undefined}}, - {application_data, iolist_to_binary(Data)}}]}; -connection( - info, tick = Msg, - #state{ - ssl_options = #ssl_options{erl_dist = true}, - protocol_specific = #{d_handle := _}}, - _) -> - eat_msgs(Msg), - {keep_state_and_data, - [{next_event, {call, {self(), undefined}}, {application_data, <<>>}}]}; +connection(cast, {internal_renegotiate, WriteState}, #state{protocol_cb = Connection, + connection_states = ConnectionStates} + = State, Connection) -> + Connection:renegotiate(State#state{renegotiation = {true, internal}, + connection_states = ConnectionStates#{current_write => WriteState}}, []); +connection(cast, {dist_handshake_complete, DHandle}, + #state{ssl_options = #ssl_options{erl_dist = true}, + erl_dist_data = ErlDistData, + socket_options = SockOpts} = State0, Connection) -> + process_flag(priority, normal), + State1 = + State0#state{ + socket_options = + SockOpts#socket_options{active = true}, + erl_dist_data = ErlDistData#{dist_handle => DHandle}}, + {Record, State} = dist_app_data(<<>>, State1), + Connection:next_event(connection, Record, State); connection(info, Msg, State, _) -> handle_info(Msg, ?FUNCTION_NAME, State); connection(internal, {recv, _}, State, Connection) -> @@ -1133,32 +1081,6 @@ connection(Type, Msg, State, Connection) -> handle_common_event(Type, Msg, ?FUNCTION_NAME, State, Connection). %%-------------------------------------------------------------------- --spec death_row(gen_statem:event_type(), term(), - #state{}, tls_connection | dtls_connection) -> - gen_statem:state_function_result(). -%%-------------------------------------------------------------------- -%% We just wait for the owner to die which triggers the monitor, -%% or the socket may die too -death_row( - info, {'DOWN', MonitorRef, _, _, Reason}, - #state{user_application={MonitorRef,_Pid}}, - _) -> - {stop, {shutdown, Reason}}; -death_row( - info, {'EXIT', Socket, Reason}, #state{socket = Socket}, _) -> - {stop, {shutdown, Reason}}; -death_row(state_timeout, Reason, _State, _Connection) -> - {stop, {shutdown,Reason}}; -death_row(_Type, _Msg, _State, _Connection) -> - %% Waste all other events - keep_state_and_data. - -%% State entry function -death_row(State, Reason) -> - {next_state, death_row, State, - [{state_timeout, 5000, Reason}]}. - -%%-------------------------------------------------------------------- -spec downgrade(gen_statem:event_type(), term(), #state{}, tls_connection | dtls_connection) -> gen_statem:state_function_result(). @@ -1208,7 +1130,14 @@ handle_common_event(internal, {application_data, Data}, StateName, State0, Conne {stop, _, _} = Stop-> Stop; {Record, State} -> - Connection:next_event(StateName, Record, State) + case Connection:next_event(StateName, Record, State) of + {next_state, StateName, State} -> + hibernate_after(StateName, State, []); + {next_state, StateName, State, Actions} -> + hibernate_after(StateName, State, Actions); + {stop, _, _} = Stop -> + Stop + end end; handle_common_event(internal, #change_cipher_spec{type = <<1>>}, StateName, #state{negotiated_version = Version} = State, _) -> @@ -1294,12 +1223,8 @@ handle_call({set_opts, Opts0}, From, StateName, handle_call(renegotiate, From, StateName, _, _) when StateName =/= connection -> {keep_state_and_data, [{reply, From, {error, already_renegotiating}}]}; -handle_call( - get_sslsocket, From, _StateName, - #state{transport_cb = Transport, socket = Socket, tracker = Tracker}, - Connection) -> - SslSocket = - Connection:socket(self(), Transport, Socket, Connection, Tracker), +handle_call(get_sslsocket, From, _StateName, State, Connection) -> + SslSocket = Connection:socket(State), {keep_state_and_data, [{reply, From, SslSocket}]}; handle_call({prf, Secret, Label, Seed, WantedLength}, From, _, @@ -1336,7 +1261,8 @@ handle_info({ErrorTag, Socket, econnaborted}, StateName, start_or_recv_from = StartFrom, role = Role, error_tag = ErrorTag, tracker = Tracker} = State) when StateName =/= connection -> - alert_user(Transport, Tracker,Socket, + Pids = Connection:pids(State), + alert_user(Pids, Transport, Tracker,Socket, StartFrom, ?ALERT_REC(?FATAL, ?CLOSE_NOTIFY), Role, Connection), stop(normal, State); @@ -1347,23 +1273,18 @@ handle_info({ErrorTag, Socket, Reason}, StateName, #state{socket = Socket, handle_normal_shutdown(?ALERT_REC(?FATAL, ?CLOSE_NOTIFY), StateName, State), stop(normal, State); -handle_info( - {'DOWN', MonitorRef, _, _, Reason}, _, - #state{ - user_application = {MonitorRef, _Pid}, - ssl_options = #ssl_options{erl_dist = true}}) -> +handle_info({'DOWN', MonitorRef, _, _, Reason}, _, + #state{user_application = {MonitorRef, _Pid}, + ssl_options = #ssl_options{erl_dist = true}}) -> {stop, {shutdown, Reason}}; -handle_info( - {'DOWN', MonitorRef, _, _, _}, _, - #state{user_application = {MonitorRef, _Pid}}) -> +handle_info({'DOWN', MonitorRef, _, _, _}, _, + #state{user_application = {MonitorRef, _Pid}}) -> {stop, normal}; -handle_info( - {'EXIT', Pid, _Reason}, StateName, - #state{user_application = {_MonitorRef, Pid}} = State) -> +handle_info({'EXIT', Pid, _Reason}, StateName, + #state{user_application = {_MonitorRef, Pid}} = State) -> %% It seems the user application has linked to us %% - ignore that and let the monitor handle this {next_state, StateName, State}; - %%% So that terminate will be run when supervisor issues shutdown handle_info({'EXIT', _Sup, shutdown}, _StateName, State) -> stop(shutdown, State); @@ -1411,7 +1332,7 @@ terminate({shutdown, transport_closed} = Reason, socket = Socket, transport_cb = Transport} = State) -> handle_trusted_certs_db(State), Connection:close(Reason, Socket, Transport, undefined, undefined); -terminate({shutdown, own_alert}, _StateName, #state{%%send_queue = SendQueue, +terminate({shutdown, own_alert}, _StateName, #state{ protocol_cb = Connection, socket = Socket, transport_cb = Transport} = State) -> @@ -1422,15 +1343,14 @@ terminate({shutdown, own_alert}, _StateName, #state{%%send_queue = SendQueue, _ -> Connection:close({timeout, ?DEFAULT_TIMEOUT}, Socket, Transport, undefined, undefined) end; -terminate(Reason, connection, #state{negotiated_version = Version, - protocol_cb = Connection, - connection_states = ConnectionStates0, +terminate(Reason, connection, #state{protocol_cb = Connection, + connection_states = ConnectionStates, ssl_options = #ssl_options{padding_check = Check}, transport_cb = Transport, socket = Socket } = State) -> handle_trusted_certs_db(State), - {BinAlert, ConnectionStates} = terminate_alert(Reason, Version, ConnectionStates0, Connection), - Connection:send(Transport, Socket, BinAlert), + Alert = terminate_alert(Reason), + ok = Connection:send_alert_in_connection(Alert, State), Connection:close(Reason, Socket, Transport, ConnectionStates, Check); terminate(Reason, _StateName, #state{transport_cb = Transport, protocol_cb = Connection, socket = Socket @@ -2379,18 +2299,13 @@ map_extensions(#hello_extensions{renegotiation_info = RenegotiationInfo, elliptic_curves => ssl_handshake:extension_value(ECCCurves), sni => ssl_handshake:extension_value(SNI)}. -terminate_alert(normal, Version, ConnectionStates, Connection) -> - Connection:encode_alert(?ALERT_REC(?WARNING, ?CLOSE_NOTIFY), - Version, ConnectionStates); -terminate_alert({Reason, _}, Version, ConnectionStates, Connection) when Reason == close; - Reason == shutdown -> - Connection:encode_alert(?ALERT_REC(?WARNING, ?CLOSE_NOTIFY), - Version, ConnectionStates); - -terminate_alert(_, Version, ConnectionStates, Connection) -> - {BinAlert, _} = Connection:encode_alert(?ALERT_REC(?FATAL, ?INTERNAL_ERROR), - Version, ConnectionStates), - BinAlert. +terminate_alert(normal) -> + ?ALERT_REC(?WARNING, ?CLOSE_NOTIFY); +terminate_alert({Reason, _}) when Reason == close; + Reason == shutdown -> + ?ALERT_REC(?WARNING, ?CLOSE_NOTIFY); +terminate_alert(_) -> + ?ALERT_REC(?FATAL, ?INTERNAL_ERROR). handle_trusted_certs_db(#state{ssl_options = #ssl_options{cacertfile = <<>>, cacerts = []}}) -> @@ -2420,16 +2335,15 @@ prepare_connection(#state{renegotiation = Renegotiate, start_or_recv_from = RecvFrom} = State0, Connection) when Renegotiate =/= {false, first}, RecvFrom =/= undefined -> - State1 = Connection:reinit_handshake_data(State0), + State1 = Connection:reinit(State0), {Record, State} = Connection:next_record(State1), {Record, ack_connection(State)}; prepare_connection(State0, Connection) -> - State = Connection:reinit_handshake_data(State0), + State = Connection:reinit(State0), {no_record, ack_connection(State)}. -ack_connection(#state{renegotiation = {true, Initiater}} = State) - when Initiater == internal; - Initiater == peer -> +ack_connection(#state{renegotiation = {true, Initiater}} = State) when Initiater == peer; + Initiater == internal -> State#state{renegotiation = undefined}; ack_connection(#state{renegotiation = {true, From}} = State) -> gen_statem:reply(From, ok), @@ -2578,35 +2492,6 @@ handle_active_option(_, StateName0, To, Reply, #state{protocol_cb = Connection} end end. -encode_packet(Data, #socket_options{packet=Packet}) -> - case Packet of - 1 -> encode_size_packet(Data, 8, (1 bsl 8) - 1); - 2 -> encode_size_packet(Data, 16, (1 bsl 16) - 1); - 4 -> encode_size_packet(Data, 32, (1 bsl 32) - 1); - _ -> Data - end. - -encode_size_packet(Bin, Size, Max) -> - Len = erlang:byte_size(Bin), - case Len > Max of - true -> throw({error, {badarg, {packet_to_large, Len, Max}}}); - false -> <<Len:Size, Bin/binary>> - end. - -time_to_renegotiate(_Data, - #{current_write := #{sequence_number := Num}}, - RenegotiateAt) -> - - %% We could do test: - %% is_time_to_renegotiate((erlang:byte_size(_Data) div ?MAX_PLAIN_TEXT_LENGTH) + 1, RenegotiateAt), - %% but we chose to have a some what lower renegotiateAt and a much cheaper test - is_time_to_renegotiate(Num, RenegotiateAt). - -is_time_to_renegotiate(N, M) when N < M-> - false; -is_time_to_renegotiate(_,_) -> - true. - %% Picks ClientData get_data(_, _, <<>>) -> @@ -2653,9 +2538,10 @@ decode_packet(Type, Buffer, PacketOpts) -> %% Note that if the user has explicitly configured the socket to expect %% HTTP headers using the {packet, httph} option, we don't do any automatic %% switching of states. -deliver_app_data(Transport, Socket, SOpts = #socket_options{active=Active, packet=Type}, +deliver_app_data(CPids, Transport, Socket, SOpts = #socket_options{active=Active, packet=Type}, Data, Pid, From, Tracker, Connection) -> - send_or_reply(Active, Pid, From, format_reply(Transport, Socket, SOpts, Data, Tracker, Connection)), + send_or_reply(Active, Pid, From, + format_reply(CPids, Transport, Socket, SOpts, Data, Tracker, Connection)), SO = case Data of {P, _, _, _} when ((P =:= http_request) or (P =:= http_response)), ((Type =:= http) or (Type =:= http_bin)) -> @@ -2674,21 +2560,24 @@ deliver_app_data(Transport, Socket, SOpts = #socket_options{active=Active, packe SO end. -format_reply(_, _,#socket_options{active = false, mode = Mode, packet = Packet, +format_reply(_, _, _,#socket_options{active = false, mode = Mode, packet = Packet, header = Header}, Data, _, _) -> {ok, do_format_reply(Mode, Packet, Header, Data)}; -format_reply(Transport, Socket, #socket_options{active = _, mode = Mode, packet = Packet, +format_reply(CPids, Transport, Socket, #socket_options{active = _, mode = Mode, packet = Packet, header = Header}, Data, Tracker, Connection) -> - {ssl, Connection:socket(self(), Transport, Socket, Connection, Tracker), + {ssl, Connection:socket(CPids, Transport, Socket, Connection, Tracker), do_format_reply(Mode, Packet, Header, Data)}. -deliver_packet_error(Transport, Socket, SO= #socket_options{active = Active}, Data, Pid, From, Tracker, Connection) -> - send_or_reply(Active, Pid, From, format_packet_error(Transport, Socket, SO, Data, Tracker, Connection)). +deliver_packet_error(CPids, Transport, Socket, + SO= #socket_options{active = Active}, Data, Pid, From, Tracker, Connection) -> + send_or_reply(Active, Pid, From, format_packet_error(CPids, + Transport, Socket, SO, Data, Tracker, Connection)). -format_packet_error(_, _,#socket_options{active = false, mode = Mode}, Data, _, _) -> +format_packet_error(_, _, _,#socket_options{active = false, mode = Mode}, Data, _, _) -> {error, {invalid_packet, do_format_reply(Mode, raw, 0, Data)}}; -format_packet_error(Transport, Socket, #socket_options{active = _, mode = Mode}, Data, Tracker, Connection) -> - {ssl_error, Connection:socket(self(), Transport, Socket, Connection, Tracker), +format_packet_error(CPids, Transport, Socket, #socket_options{active = _, mode = Mode}, + Data, Tracker, Connection) -> + {ssl_error, Connection:socket(CPids, Transport, Socket, Connection, Tracker), {invalid_packet, do_format_reply(Mode, raw, 0, Data)}}. do_format_reply(binary, _, N, Data) when N > 0 -> % Header mode @@ -2726,29 +2615,29 @@ send_user(Pid, Msg) -> Pid ! Msg, ok. -alert_user(Transport, Tracker, Socket, connection, Opts, Pid, From, Alert, Role, Connection) -> - alert_user(Transport, Tracker, Socket, Opts#socket_options.active, Pid, From, Alert, Role, Connection); -alert_user(Transport, Tracker, Socket,_, _, _, From, Alert, Role, Connection) -> - alert_user(Transport, Tracker, Socket, From, Alert, Role, Connection). +alert_user(Pids, Transport, Tracker, Socket, connection, Opts, Pid, From, Alert, Role, Connection) -> + alert_user(Pids, Transport, Tracker, Socket, Opts#socket_options.active, Pid, From, Alert, Role, Connection); +alert_user(Pids, Transport, Tracker, Socket,_, _, _, From, Alert, Role, Connection) -> + alert_user(Pids, Transport, Tracker, Socket, From, Alert, Role, Connection). -alert_user(Transport, Tracker, Socket, From, Alert, Role, Connection) -> - alert_user(Transport, Tracker, Socket, false, no_pid, From, Alert, Role, Connection). +alert_user(Pids, Transport, Tracker, Socket, From, Alert, Role, Connection) -> + alert_user(Pids, Transport, Tracker, Socket, false, no_pid, From, Alert, Role, Connection). -alert_user(_, _, _, false = Active, Pid, From, Alert, Role, _) when From =/= undefined -> +alert_user(_, _, _, _, false = Active, Pid, From, Alert, Role, _) when From =/= undefined -> %% If there is an outstanding ssl_accept | recv %% From will be defined and send_or_reply will %% send the appropriate error message. ReasonCode = ssl_alert:reason_code(Alert, Role), send_or_reply(Active, Pid, From, {error, ReasonCode}); -alert_user(Transport, Tracker, Socket, Active, Pid, From, Alert, Role, Connection) -> +alert_user(Pids, Transport, Tracker, Socket, Active, Pid, From, Alert, Role, Connection) -> case ssl_alert:reason_code(Alert, Role) of closed -> send_or_reply(Active, Pid, From, - {ssl_closed, Connection:socket(self(), + {ssl_closed, Connection:socket(Pids, Transport, Socket, Connection, Tracker)}); ReasonCode -> send_or_reply(Active, Pid, From, - {ssl_error, Connection:socket(self(), + {ssl_error, Connection:socket(Pids, Transport, Socket, Connection, Tracker), ReasonCode}) end. @@ -2817,42 +2706,14 @@ new_emulated([], EmOpts) -> EmOpts; new_emulated(NewEmOpts, _) -> NewEmOpts. -%%---------------Erlang distribution -------------------------------------- - -send_dist_data(StateName, State, DHandle, Acc) -> - case erlang:dist_ctrl_get_data(DHandle) of - none -> - erlang:dist_ctrl_get_data_notification(DHandle), - hibernate_after(StateName, State, lists:reverse(Acc)); - Data -> - send_dist_data( - StateName, State, DHandle, - [{next_event, {call, {self(), undefined}}, {application_data, Data}} - |Acc]) - end. - -%% Overload mitigation -eat_msgs(Msg) -> - receive Msg -> eat_msgs(Msg) - after 0 -> ok - end. -%% When acting as distribution controller map the exit reason -%% to follow the documented nodedown_reason for net_kernel stop(Reason, State) -> - {stop, erl_dist_stop_reason(Reason, State), State}. + {stop, Reason, State}. stop_and_reply(Reason, Replies, State) -> - {stop_and_reply, erl_dist_stop_reason(Reason, State), Replies, State}. - -erl_dist_stop_reason( - Reason, #state{ssl_options = #ssl_options{erl_dist = true}}) -> - case Reason of - normal -> - %% We can not exit with normal since that will not bring - %% down the rest of the distribution processes - {shutdown, normal}; - _ -> Reason - end; -erl_dist_stop_reason(Reason, _State) -> - Reason. + {stop_and_reply, Reason, Replies, State}. + +is_dist_up(#{dist_handle := Handle}) when Handle =/= undefined -> + true; +is_dist_up(_) -> + false. diff --git a/lib/ssl/src/ssl_connection.hrl b/lib/ssl/src/ssl_connection.hrl index 9cef0c9605..66e3182313 100644 --- a/lib/ssl/src/ssl_connection.hrl +++ b/lib/ssl/src/ssl_connection.hrl @@ -44,6 +44,7 @@ host :: string() | inet:ip_address(), port :: integer(), socket :: port() | tuple(), %% TODO: dtls socket + sender :: pid() | undefined, ssl_options :: #ssl_options{}, socket_options :: #socket_options{}, connection_states :: ssl_record:connection_states() | secret_printout(), @@ -74,6 +75,7 @@ cert_db_ref :: certdb_ref() | 'undefined', bytes_to_read :: undefined | integer(), %% bytes to read in passive mode user_data_buffer :: undefined | binary() | secret_printout(), + erl_dist_data = #{} :: map(), renegotiation :: undefined | {boolean(), From::term() | internal | peer}, start_or_recv_from :: term(), timer :: undefined | reference(), % start_or_recive_timer diff --git a/lib/ssl/src/ssl_internal.hrl b/lib/ssl/src/ssl_internal.hrl index ae1c3ea47c..fd246e2550 100644 --- a/lib/ssl/src/ssl_internal.hrl +++ b/lib/ssl/src/ssl_internal.hrl @@ -120,7 +120,7 @@ %% undefined if not hibernating, or number of ms of %% inactivity after which ssl_connection will go into %% hibernation - hibernate_after :: timeout(), + hibernate_after :: timeout(), %% This option should only be set to true by inet_tls_dist erl_dist = false :: boolean(), alpn_advertised_protocols = undefined :: [binary()] | undefined , diff --git a/lib/ssl/src/tls_connection.erl b/lib/ssl/src/tls_connection.erl index 4d1122f804..6c7511d2b3 100644 --- a/lib/ssl/src/tls_connection.erl +++ b/lib/ssl/src/tls_connection.erl @@ -43,30 +43,35 @@ %% Internal application API %% Setup --export([start_fsm/8, start_link/7, init/1]). +-export([start_fsm/8, start_link/8, init/1, pids/1]). %% State transition handling --export([next_record/1, next_event/3, next_event/4, handle_common_event/4]). +-export([next_record/1, next_event/3, next_event/4, + handle_common_event/4]). %% Handshake handling --export([renegotiate/2, send_handshake/2, +-export([renegotiation/2, renegotiate/2, send_handshake/2, queue_handshake/2, queue_change_cipher/2, - reinit_handshake_data/1, select_sni_extension/1, empty_connection_state/2]). + reinit/1, reinit_handshake_data/1, select_sni_extension/1, + empty_connection_state/2]). %% Alert and close handling --export([encode_alert/3, send_alert/2, close/5, protocol_name/0]). +-export([send_alert/2, send_alert_in_connection/2, encode_alert/3, close/5, protocol_name/0]). %% Data handling --export([encode_data/3, passive_receive/2, next_record_if_active/1, send/3, - socket/5, setopts/3, getopts/3]). +-export([encode_data/3, passive_receive/2, next_record_if_active/1, + send/3, socket/5, setopts/3, getopts/3]). %% gen_statem state functions -export([init/3, error/3, downgrade/3, %% Initiation and take down states hello/3, user_hello/3, certify/3, cipher/3, abbreviated/3, %% Handshake states - connection/3, death_row/3]). + connection/3]). %% gen_statem callbacks -export([callback_mode/0, terminate/3, code_change/4, format_status/2]). + +-define(DIST_CNTRL_SPAWN_OPTS, [{priority, max}]). + %%==================================================================== %% Internal application API %%==================================================================== @@ -77,9 +82,10 @@ start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = false},_, Tracker} User, {CbModule, _,_, _} = CbInfo, Timeout) -> try - {ok, Pid} = tls_connection_sup:start_child([Role, Host, Port, Socket, + {ok, Sender} = tls_sender:start(), + {ok, Pid} = tls_connection_sup:start_child([Role, Sender, Host, Port, Socket, Opts, User, CbInfo]), - {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, Pid, CbModule, Tracker), + {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, [Pid, Sender], CbModule, Tracker), ssl_connection:handshake(SslSocket, Timeout) catch error:{badmatch, {error, _} = Error} -> @@ -90,9 +96,10 @@ start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = true},_, Tracker} = User, {CbModule, _,_, _} = CbInfo, Timeout) -> try - {ok, Pid} = tls_connection_sup:start_child_dist([Role, Host, Port, Socket, + {ok, Sender} = tls_sender:start([{spawn_opt, ?DIST_CNTRL_SPAWN_OPTS}]), + {ok, Pid} = tls_connection_sup:start_child_dist([Role, Sender, Host, Port, Socket, Opts, User, CbInfo]), - {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, Pid, CbModule, Tracker), + {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, [Pid, Sender], CbModule, Tracker), ssl_connection:handshake(SslSocket, Timeout) catch error:{badmatch, {error, _} = Error} -> @@ -100,25 +107,37 @@ start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = true},_, Tracker} = end. %%-------------------------------------------------------------------- --spec start_link(atom(), host(), inet:port_number(), port(), list(), pid(), tuple()) -> +-spec start_link(atom(), pid(), host(), inet:port_number(), port(), list(), pid(), tuple()) -> {ok, pid()} | ignore | {error, reason()}. %% %% Description: Creates a gen_statem process which calls Module:init/1 to %% initialize. %%-------------------------------------------------------------------- -start_link(Role, Host, Port, Socket, Options, User, CbInfo) -> - {ok, proc_lib:spawn_link(?MODULE, init, [[Role, Host, Port, Socket, Options, User, CbInfo]])}. +start_link(Role, Sender, Host, Port, Socket, Options, User, CbInfo) -> + {ok, proc_lib:spawn_link(?MODULE, init, [[Role, Sender, Host, Port, Socket, Options, User, CbInfo]])}. -init([Role, Host, Port, Socket, Options, User, CbInfo]) -> +init([Role, Sender, Host, Port, Socket, {SslOpts, _, _} = Options, User, CbInfo]) -> process_flag(trap_exit, true), - State0 = #state{protocol_specific = Map} = initial_state(Role, Host, Port, Socket, Options, User, CbInfo), + case SslOpts#ssl_options.erl_dist of + true -> + process_flag(priority, max); + _ -> + ok + end, + State0 = #state{protocol_specific = Map} = initial_state(Role, Sender, + Host, Port, Socket, Options, User, CbInfo), try State = ssl_connection:ssl_config(State0#state.ssl_options, Role, State0), - gen_statem:enter_loop(?MODULE, [], init, State) + initialize_tls_sender(State), + gen_statem:enter_loop(?MODULE, [], init, State) catch throw:Error -> EState = State0#state{protocol_specific = Map#{error => Error}}, gen_statem:enter_loop(?MODULE, [], error, EState) end. + +pids(#state{protocol_specific = #{sender := Sender}}) -> + [self(), Sender]. + %%==================================================================== %% State transition handling %%==================================================================== @@ -235,13 +254,15 @@ handle_common_event(internal, #ssl_tls{type = _Unknown}, StateName, State) -> %%==================================================================== %% Handshake handling %%==================================================================== +renegotiation(Pid, WriteState) -> + gen_statem:call(Pid, {user_renegotiate, WriteState}). + renegotiate(#state{role = client} = State, Actions) -> %% Handle same way as if server requested %% the renegotiation Hs0 = ssl_handshake:init_handshake_history(), {next_state, connection, State#state{tls_handshake_history = Hs0}, [{next_event, internal, #hello_request{}} | Actions]}; - renegotiate(#state{role = server, socket = Socket, transport_cb = Transport, @@ -286,6 +307,12 @@ queue_change_cipher(Msg, #state{negotiated_version = Version, State0#state{connection_states = ConnectionStates, flight_buffer = Flight0 ++ [BinChangeCipher]}. +reinit(#state{protocol_specific = #{sender := Sender}, + negotiated_version = Version, + connection_states = #{current_write := Write}} = State) -> + tls_sender:update_connection_state(Sender, Write, Version), + reinit_handshake_data(State). + reinit_handshake_data(State) -> %% premaster_secret, public_key_info and tls_handshake_info %% are only needed during the handshake phase. @@ -307,15 +334,6 @@ empty_connection_state(ConnectionEnd, BeastMitigation) -> %%==================================================================== %% Alert and close handling %%==================================================================== -send_alert(Alert, #state{negotiated_version = Version, - socket = Socket, - transport_cb = Transport, - connection_states = ConnectionStates0} = State0) -> - {BinMsg, ConnectionStates} = - encode_alert(Alert, Version, ConnectionStates0), - send(Transport, Socket, BinMsg), - State0#state{connection_states = ConnectionStates}. - %%-------------------------------------------------------------------- -spec encode_alert(#alert{}, ssl_record:ssl_version(), ssl_record:connection_states()) -> {iolist(), ssl_record:connection_states()}. @@ -324,6 +342,20 @@ send_alert(Alert, #state{negotiated_version = Version, %%-------------------------------------------------------------------- encode_alert(#alert{} = Alert, Version, ConnectionStates) -> tls_record:encode_alert_record(Alert, Version, ConnectionStates). + +send_alert(Alert, #state{negotiated_version = Version, + socket = Socket, + protocol_cb = Connection, + transport_cb = Transport, + connection_states = ConnectionStates0} = StateData0) -> + {BinMsg, ConnectionStates} = + Connection:encode_alert(Alert, Version, ConnectionStates0), + Connection:send(Transport, Socket, BinMsg), + StateData0#state{connection_states = ConnectionStates}. + +send_alert_in_connection(Alert, #state{protocol_specific = #{sender := Sender}}) -> + tls_sender:send_alert(Sender, Alert). + %% User closes or recursive call! close({close, Timeout}, Socket, Transport = gen_tcp, _,_) -> tls_socket:setopts(Transport, Socket, [{active, false}]), @@ -378,8 +410,8 @@ next_record_if_active(State) -> send(Transport, Socket, Data) -> tls_socket:send(Transport, Socket, Data). -socket(Pid, Transport, Socket, Connection, Tracker) -> - tls_socket:socket(Pid, Transport, Socket, Connection, Tracker). +socket(Pids, Transport, Socket, Connection, Tracker) -> + tls_socket:socket(Pids, Transport, Socket, Connection, Tracker). setopts(Transport, Socket, Other) -> tls_socket:setopts(Transport, Socket, Other). @@ -448,15 +480,17 @@ error(_, _, _) -> #state{}) -> gen_statem:state_function_result(). %%-------------------------------------------------------------------- -hello(internal, #client_hello{extensions = Extensions} = Hello, #state{ssl_options = #ssl_options{handshake = hello}, - start_or_recv_from = From} = State) -> - {next_state, user_hello, State#state{start_or_recv_from = undefined, +hello(internal, #client_hello{extensions = Extensions} = Hello, + #state{ssl_options = #ssl_options{handshake = hello}, + start_or_recv_from = From} = State) -> + {next_state, user_hello, State#state{start_or_recv_from = undefined, hello = Hello}, [{reply, From, {ok, ssl_connection:map_extensions(Extensions)}}]}; -hello(internal, #server_hello{extensions = Extensions} = Hello, #state{ssl_options = #ssl_options{handshake = hello}, - start_or_recv_from = From} = State) -> +hello(internal, #server_hello{extensions = Extensions} = Hello, + #state{ssl_options = #ssl_options{handshake = hello}, + start_or_recv_from = From} = State) -> {next_state, user_hello, State#state{start_or_recv_from = undefined, - hello = Hello}, + hello = Hello}, [{reply, From, {ok, ssl_connection:map_extensions(Extensions)}}]}; hello(internal, #client_hello{client_version = ClientVersion} = Hello, #state{connection_states = ConnectionStates0, @@ -540,14 +574,19 @@ cipher(Type, Event, State) -> %%-------------------------------------------------------------------- connection(info, Event, State) -> gen_info(Event, ?FUNCTION_NAME, State); +connection({call, From}, {user_renegotiate, WriteState}, + #state{connection_states = ConnectionStates} = State) -> + {next_state, ?FUNCTION_NAME, State#state{connection_states = ConnectionStates#{current_write => WriteState}}, + [{next_event,{call, From}, renegotiate}]}; connection(internal, #hello_request{}, - #state{role = client, host = Host, port = Port, + #state{role = client, + renegotiation = {Renegotiation, _}, + host = Host, port = Port, session = #session{own_certificate = Cert} = Session0, session_cache = Cache, session_cache_cb = CacheCb, - ssl_options = SslOpts, - connection_states = ConnectionStates0, - renegotiation = {Renegotiation, _}} = State0) -> - Hello = tls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, + ssl_options = SslOpts, + connection_states = ConnectionStates} = State0) -> + Hello = tls_handshake:client_hello(Host, Port, ConnectionStates, SslOpts, Cache, CacheCb, Renegotiation, Cert), {State1, Actions} = send_handshake(Hello, State0), {Record, State} = @@ -556,7 +595,10 @@ connection(internal, #hello_request{}, = Hello#client_hello.session_id}}), next_event(hello, Record, State, Actions); connection(internal, #client_hello{} = Hello, - #state{role = server, allow_renegotiate = true} = State0) -> + #state{role = server, allow_renegotiate = true, connection_states = CS, + %%protocol_cb = Connection, + protocol_specific = #{sender := Sender} + } = State0) -> %% Mitigate Computational DoS attack %% http://www.educatedguesswork.org/2011/10/ssltls_and_computational_dos.html %% http://www.thc.org/thc-ssl-dos/ Rather than disabling client @@ -565,24 +607,21 @@ connection(internal, #client_hello{} = Hello, erlang:send_after(?WAIT_TO_ALLOW_RENEGOTIATION, self(), allow_renegotiate), {Record, State} = next_record(State0#state{allow_renegotiate = false, renegotiation = {true, peer}}), - next_event(hello, Record, State, [{next_event, internal, Hello}]); + {ok, Write} = tls_sender:renegotiate(Sender), + next_event(hello, Record, State#state{connection_states = CS#{current_write => Write}}, + [{next_event, internal, Hello}]); connection(internal, #client_hello{}, - #state{role = server, allow_renegotiate = false} = State0) -> + #state{role = server, allow_renegotiate = false, + protocol_cb = Connection} = State0) -> Alert = ?ALERT_REC(?WARNING, ?NO_RENEGOTIATION), - State1 = send_alert(Alert, State0), - {Record, State} = ssl_connection:prepare_connection(State1, ?MODULE), + send_alert_in_connection(Alert, State0), + State1 = Connection:reinit_handshake_data(State0), + {Record, State} = next_record(State1), next_event(?FUNCTION_NAME, Record, State); connection(Type, Event, State) -> ssl_connection:?FUNCTION_NAME(Type, Event, State, ?MODULE). %%-------------------------------------------------------------------- --spec death_row(gen_statem:event_type(), term(), #state{}) -> - gen_statem:state_function_result(). -%%-------------------------------------------------------------------- -death_row(Type, Event, State) -> - ssl_connection:death_row(Type, Event, State, ?MODULE). - -%%-------------------------------------------------------------------- -spec downgrade(gen_statem:event_type(), term(), #state{}) -> gen_statem:state_function_result(). %%-------------------------------------------------------------------- @@ -596,6 +635,7 @@ callback_mode() -> state_functions. terminate(Reason, StateName, State) -> + ensure_sender_terminate(Reason, State), catch ssl_connection:terminate(Reason, StateName, State). format_status(Type, Data) -> @@ -607,11 +647,13 @@ code_change(_OldVsn, StateName, State, _) -> %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- -initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions, Tracker}, User, +initial_state(Role, Sender, Host, Port, Socket, {SSLOptions, SocketOptions, Tracker}, User, {CbModule, DataTag, CloseTag, ErrorTag}) -> - #ssl_options{beast_mitigation = BeastMitigation} = SSLOptions, + #ssl_options{beast_mitigation = BeastMitigation, + erl_dist = IsErlDist} = SSLOptions, ConnectionStates = tls_record:init_connection_states(Role, BeastMitigation), + ErlDistData = erl_dist_data(IsErlDist), SessionCacheCb = case application:get_env(ssl, session_cb) of {ok, Cb} when is_atom(Cb) -> Cb; @@ -619,7 +661,7 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions, Tracker}, Us ssl_session_cache end, - Monitor = erlang:monitor(process, User), + UserMonitor = erlang:monitor(process, User), #state{socket_options = SocketOptions, ssl_options = SSLOptions, @@ -632,9 +674,10 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions, Tracker}, Us host = Host, port = Port, socket = Socket, + erl_dist_data = ErlDistData, connection_states = ConnectionStates, protocol_buffers = #protocol_buffers{}, - user_application = {Monitor, User}, + user_application = {UserMonitor, User}, user_data_buffer = <<>>, session_cache_cb = SessionCacheCb, renegotiation = {false, first}, @@ -642,9 +685,37 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions, Tracker}, Us start_or_recv_from = undefined, protocol_cb = ?MODULE, tracker = Tracker, - flight_buffer = [] + flight_buffer = [], + protocol_specific = #{sender => Sender} }. +erl_dist_data(true) -> + #{dist_handle => undefined, + dist_buffer => <<>>}; +erl_dist_data(false) -> + #{}. + +initialize_tls_sender(#state{role = Role, + socket = Socket, + socket_options = SockOpts, + tracker = Tracker, + protocol_cb = Connection, + transport_cb = Transport, + negotiated_version = Version, + ssl_options = #ssl_options{renegotiate_at = RenegotiateAt}, + connection_states = #{current_write := ConnectionWriteState}, + protocol_specific = #{sender := Sender}}) -> + Init = #{current_write => ConnectionWriteState, + role => Role, + socket => Socket, + socket_options => SockOpts, + tracker => Tracker, + protocol_cb => Connection, + transport_cb => Transport, + negotiated_version => Version, + renegotiate_at => RenegotiateAt}, + tls_sender:initialize(Sender, Init). + next_tls_record(Data, StateName, #state{protocol_buffers = #protocol_buffers{tls_record_buffer = Buf0, tls_cipher_texts = CT0} = Buffers} @@ -716,6 +787,9 @@ handle_info({CloseTag, Socket}, StateName, %% and then receive the final message. next_event(StateName, no_record, State) end; +handle_info({'EXIT', Pid, Reason}, _, + #state{protocol_specific = Pid} = State) -> + {stop, {shutdown, sender_died, Reason}, State}; handle_info(Msg, StateName, State) -> ssl_connection:StateName(info, Msg, State, ?MODULE). @@ -784,7 +858,8 @@ unprocessed_events(Events) -> erlang:length(Events)-1. -assert_buffer_sanity(<<?BYTE(_Type), ?UINT24(Length), Rest/binary>>, #ssl_options{max_handshake_size = Max}) when +assert_buffer_sanity(<<?BYTE(_Type), ?UINT24(Length), Rest/binary>>, + #ssl_options{max_handshake_size = Max}) when Length =< Max -> case size(Rest) of N when N < Length -> @@ -804,3 +879,16 @@ assert_buffer_sanity(Bin, _) -> throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, malformed_handshake_data)) end. + +ensure_sender_terminate(downgrade, _) -> + ok; %% Do not terminate sender during downgrade phase +ensure_sender_terminate(_, #state{protocol_specific = #{sender := Sender}}) -> + %% Make sure TLS sender dies when connection process is terminated normally + %% This is needed if the tls_sender is blocked in prim_inet:send + Kill = fun() -> + receive + after 5000 -> + catch (exit(Sender, kill)) + end + end, + spawn(Kill). diff --git a/lib/ssl/src/tls_sender.erl b/lib/ssl/src/tls_sender.erl new file mode 100644 index 0000000000..007fd345dd --- /dev/null +++ b/lib/ssl/src/tls_sender.erl @@ -0,0 +1,397 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018-2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +-module(tls_sender). + +-behaviour(gen_statem). + +-include("ssl_internal.hrl"). +-include("ssl_alert.hrl"). +-include("ssl_handshake.hrl"). +-include("ssl_api.hrl"). + +%% API +-export([start/0, start/1, initialize/2, send_data/2, send_alert/2, renegotiate/1, + update_connection_state/3, dist_tls_socket/1, dist_handshake_complete/3]). + +%% gen_statem callbacks +-export([callback_mode/0, init/1, terminate/3, code_change/4]). +-export([init/3, connection/3, handshake/3, death_row/3]). + +-define(SERVER, ?MODULE). + +-record(data, {connection_pid, + connection_states = #{}, + role, + socket, + socket_options, + tracker, + protocol_cb, + transport_cb, + negotiated_version, + renegotiate_at, + connection_monitor, + dist_handle + }). + +%%%=================================================================== +%%% API +%%%=================================================================== +%%-------------------------------------------------------------------- +-spec start() -> {ok, Pid :: pid()} | + ignore | + {error, Error :: term()}. +-spec start(list()) -> {ok, Pid :: pid()} | + ignore | + {error, Error :: term()}. + +%% Description: Start sender process to avoid dead lock that +%% may happen when a socket is busy (busy port) and the +%% same process is sending and receiving +%%-------------------------------------------------------------------- +start() -> + gen_statem:start_link(?MODULE, [], []). +start(SpawnOpts) -> + gen_statem:start_link(?MODULE, [], SpawnOpts). + +%%-------------------------------------------------------------------- +-spec initialize(pid(), map()) -> ok. +%% Description: So TLS connection process can initialize it sender +%% process. +%%-------------------------------------------------------------------- +initialize(Pid, InitMsg) -> + gen_statem:call(Pid, {self(), InitMsg}). + +%%-------------------------------------------------------------------- +-spec send_data(pid(), iodata()) -> ok. +%% Description: Send application data +%%-------------------------------------------------------------------- +send_data(Pid, AppData) -> + %% Needs error handling for external API + call(Pid, {application_data, AppData}). + +%%-------------------------------------------------------------------- +-spec send_alert(pid(), #alert{}) -> _. +%% Description: TLS connection process wants to end an Alert +%% in the connection state. +%%-------------------------------------------------------------------- +send_alert(Pid, Alert) -> + gen_statem:cast(Pid, Alert). + +%%-------------------------------------------------------------------- +-spec renegotiate(pid()) -> {ok, WriteState::map()} | {error, closed}. +%% Description: So TLS connection process can synchronize the +%% encryption state to be used when handshaking. +%%-------------------------------------------------------------------- +renegotiate(Pid) -> + %% Needs error handling for external API + call(Pid, renegotiate). +%%-------------------------------------------------------------------- +-spec update_connection_state(pid(), WriteState::map(), tls_record:tls_version()) -> ok. +%% Description: So TLS connection process can synchronize the +%% encryption state to be used when sending application data. +%%-------------------------------------------------------------------- +update_connection_state(Pid, NewState, Version) -> + gen_statem:cast(Pid, {new_write, NewState, Version}). +%%-------------------------------------------------------------------- +-spec dist_handshake_complete(pid(), node(), term()) -> ok. +%% Description: Erlang distribution callback +%%-------------------------------------------------------------------- +dist_handshake_complete(ConnectionPid, Node, DHandle) -> + gen_statem:call(ConnectionPid, {dist_handshake_complete, Node, DHandle}). +%%-------------------------------------------------------------------- +-spec dist_tls_socket(pid()) -> {ok, #sslsocket{}}. +%% Description: To enable distribution startup to get a proper "#sslsocket{}" +%%-------------------------------------------------------------------- +dist_tls_socket(Pid) -> + gen_statem:call(Pid, dist_get_tls_socket). + +%%%=================================================================== +%%% gen_statem callbacks +%%%=================================================================== +%%-------------------------------------------------------------------- +-spec callback_mode() -> gen_statem:callback_mode_result(). +%%-------------------------------------------------------------------- +callback_mode() -> + state_functions. + +%%-------------------------------------------------------------------- +-spec init(Args :: term()) -> + gen_statem:init_result(atom()). +%%-------------------------------------------------------------------- +init(_) -> + %% Note: Should not trap exits so that this process + %% will be terminated if tls_connection process is + %% killed brutally + {ok, init, #data{}}. + +%%-------------------------------------------------------------------- +-spec init(gen_statem:event_type(), + Msg :: term(), + StateData :: term()) -> + gen_statem:event_handler_result(atom()). +%%-------------------------------------------------------------------- +init({call, From}, {Pid, #{current_write := WriteState, + role := Role, + socket := Socket, + socket_options := SockOpts, + tracker := Tracker, + protocol_cb := Connection, + transport_cb := Transport, + negotiated_version := Version, + renegotiate_at := RenegotiateAt}}, + #data{connection_states = ConnectionStates} = StateData0) -> + Monitor = erlang:monitor(process, Pid), + StateData = + StateData0#data{connection_pid = Pid, + connection_monitor = Monitor, + connection_states = + ConnectionStates#{current_write => WriteState}, + role = Role, + socket = Socket, + socket_options = SockOpts, + tracker = Tracker, + protocol_cb = Connection, + transport_cb = Transport, + negotiated_version = Version, + renegotiate_at = RenegotiateAt}, + {next_state, handshake, StateData, [{reply, From, ok}]}; +init(info, Msg, StateData) -> + handle_info(Msg, ?FUNCTION_NAME, StateData). +%%-------------------------------------------------------------------- +-spec connection(gen_statem:event_type(), + Msg :: term(), + StateData :: term()) -> + gen_statem:event_handler_result(atom()). +%%-------------------------------------------------------------------- +connection({call, From}, renegotiate, + #data{connection_states = #{current_write := Write}} = StateData) -> + {next_state, handshake, StateData, [{reply, From, {ok, Write}}]}; +connection({call, From}, {application_data, AppData}, + #data{socket_options = SockOpts} = StateData) -> + case encode_packet(AppData, SockOpts) of + {error, _} = Error -> + {next_state, ?FUNCTION_NAME, StateData, [{reply, From, Error}]}; + Data -> + send_application_data(Data, From, ?FUNCTION_NAME, StateData) + end; +connection({call, From}, dist_get_tls_socket, + #data{protocol_cb = Connection, + transport_cb = Transport, + socket = Socket, + connection_pid = Pid, + tracker = Tracker} = StateData) -> + TLSSocket = Connection:socket([Pid, self()], Transport, Socket, Connection, Tracker), + {next_state, ?FUNCTION_NAME, StateData, [{reply, From, {ok, TLSSocket}}]}; +connection({call, From}, {dist_handshake_complete, _Node, DHandle}, #data{connection_pid = Pid} = StateData) -> + ok = erlang:dist_ctrl_input_handler(DHandle, Pid), + ok = ssl_connection:dist_handshake_complete(Pid, DHandle), + %% From now on we execute on normal priority + process_flag(priority, normal), + Events = dist_data_events(DHandle, []), + {next_state, ?FUNCTION_NAME, StateData#data{dist_handle = DHandle}, [{reply, From, ok} | Events]}; +connection(cast, #alert{} = Alert, StateData0) -> + StateData = send_tls_alert(Alert, StateData0), + {next_state, ?FUNCTION_NAME, StateData}; +connection(cast, {new_write, WritesState, Version}, + #data{connection_states = ConnectionStates0} = StateData) -> + {next_state, connection, + StateData#data{connection_states = + ConnectionStates0#{current_write => WritesState}, + negotiated_version = Version}}; +connection(info, dist_data, #data{dist_handle = DHandle} = StateData) -> + Events = dist_data_events(DHandle, []), + {next_state, ?FUNCTION_NAME, StateData, Events}; +connection(info, tick, StateData) -> + consume_ticks(), + {next_state, ?FUNCTION_NAME, StateData, + [{next_event, {call, {self(), undefined}}, + {application_data, <<>>}}]}; +connection(info, {send, From, Ref, Data}, _StateData) -> + %% This is for testing only! + %% + %% Needed by some OTP distribution + %% test suites... + From ! {Ref, ok}, + {keep_state_and_data, + [{next_event, {call, {self(), undefined}}, + {application_data, iolist_to_binary(Data)}}]}; +connection(info, Msg, StateData) -> + handle_info(Msg, ?FUNCTION_NAME, StateData). +%%-------------------------------------------------------------------- +-spec handshake(gen_statem:event_type(), + Msg :: term(), + StateData :: term()) -> + gen_statem:event_handler_result(atom()). +%%-------------------------------------------------------------------- +handshake({call, _}, _, _) -> + {keep_state_and_data, [postpone]}; +handshake(cast, {new_write, WritesState, Version}, + #data{connection_states = ConnectionStates0} = StateData) -> + {next_state, connection, + StateData#data{connection_states = + ConnectionStates0#{current_write => WritesState}, + negotiated_version = Version}}; +handshake(info, Msg, StateData) -> + handle_info(Msg, ?FUNCTION_NAME, StateData). + +%%-------------------------------------------------------------------- +-spec death_row(gen_statem:event_type(), + Msg :: term(), + StateData :: term()) -> + gen_statem:event_handler_result(atom()). +%%-------------------------------------------------------------------- +death_row(state_timeout, Reason, _State) -> + {stop, {shutdown, Reason}}; +death_row(_Type, _Msg, _State) -> + %% Waste all other events + keep_state_and_data. + +%%-------------------------------------------------------------------- +-spec terminate(Reason :: term(), State :: term(), Data :: term()) -> + any(). +%%-------------------------------------------------------------------- +terminate(_Reason, _State, _Data) -> + void. + +%%-------------------------------------------------------------------- +-spec code_change( + OldVsn :: term() | {down,term()}, + State :: term(), Data :: term(), Extra :: term()) -> + {ok, NewState :: term(), NewData :: term()} | + (Reason :: term()). +%% Convert process state when code is changed +%%-------------------------------------------------------------------- +code_change(_OldVsn, State, Data, _Extra) -> + {ok, State, Data}. + +%%%=================================================================== +%%% Internal functions +%%%=================================================================== +handle_info({'DOWN', Monitor, _, _, Reason}, _, + #data{connection_monitor = Monitor, + dist_handle = Handle} = StateData) when Handle =/= undefined-> + {next_state, death_row, StateData, [{state_timeout, 5000, Reason}]}; +handle_info({'DOWN', Monitor, _, _, _}, _, + #data{connection_monitor = Monitor} = StateData) -> + {stop, normal, StateData}; +handle_info(_,_,_) -> + {keep_state_and_data}. + +send_tls_alert(Alert, #data{negotiated_version = Version, + socket = Socket, + protocol_cb = Connection, + transport_cb = Transport, + connection_states = ConnectionStates0} = StateData0) -> + {BinMsg, ConnectionStates} = + Connection:encode_alert(Alert, Version, ConnectionStates0), + Connection:send(Transport, Socket, BinMsg), + StateData0#data{connection_states = ConnectionStates}. + +send_application_data(Data, From, StateName, + #data{connection_pid = Pid, + socket = Socket, + dist_handle = DistHandle, + negotiated_version = Version, + protocol_cb = Connection, + transport_cb = Transport, + connection_states = ConnectionStates0, + renegotiate_at = RenegotiateAt} = StateData0) -> + case time_to_renegotiate(Data, ConnectionStates0, RenegotiateAt) of + true -> + ssl_connection:internal_renegotiation(Pid, ConnectionStates0), + {next_state, handshake, StateData0, + [{next_event, {call, From}, {application_data, Data}}]}; + false -> + {Msgs, ConnectionStates} = + Connection:encode_data(Data, Version, ConnectionStates0), + StateData = StateData0#data{connection_states = ConnectionStates}, + case Connection:send(Transport, Socket, Msgs) of + ok when DistHandle =/= undefined -> + {next_state, StateName, StateData, []}; + Reason when DistHandle =/= undefined -> + {next_state, death_row, StateData, [{state_timeout, 5000, Reason}]}; + ok -> + {next_state, StateName, StateData, [{reply, From, ok}]}; + Result -> + {next_state, StateName, StateData, [{reply, From, Result}]} + end + end. + +encode_packet(Data, #socket_options{packet=Packet}) -> + case Packet of + 1 -> encode_size_packet(Data, 8, (1 bsl 8) - 1); + 2 -> encode_size_packet(Data, 16, (1 bsl 16) - 1); + 4 -> encode_size_packet(Data, 32, (1 bsl 32) - 1); + _ -> Data + end. + +encode_size_packet(Bin, Size, Max) -> + Len = erlang:byte_size(Bin), + case Len > Max of + true -> + {error, {badarg, {packet_to_large, Len, Max}}}; + false -> + <<Len:Size, Bin/binary>> + end. +time_to_renegotiate(_Data, + #{current_write := #{sequence_number := Num}}, + RenegotiateAt) -> + + %% We could do test: + %% is_time_to_renegotiate((erlang:byte_size(_Data) div + %% ?MAX_PLAIN_TEXT_LENGTH) + 1, RenegotiateAt), but we chose to + %% have a some what lower renegotiateAt and a much cheaper test + is_time_to_renegotiate(Num, RenegotiateAt). + +is_time_to_renegotiate(N, M) when N < M-> + false; +is_time_to_renegotiate(_,_) -> + true. + +call(FsmPid, Event) -> + try gen_statem:call(FsmPid, Event) + catch + exit:{noproc, _} -> + {error, closed}; + exit:{normal, _} -> + {error, closed}; + exit:{{shutdown, _},_} -> + {error, closed} + end. + +%%---------------Erlang distribution -------------------------------------- + +dist_data_events(DHandle, Events) -> + case erlang:dist_ctrl_get_data(DHandle) of + none -> + erlang:dist_ctrl_get_data_notification(DHandle), + lists:reverse(Events); + Data -> + Event = {next_event, {call, {self(), undefined}}, {application_data, Data}}, + dist_data_events(DHandle, [Event | Events]) + end. + +consume_ticks() -> + receive tick -> + consume_ticks() + after 0 -> + ok + end. diff --git a/lib/ssl/src/tls_socket.erl b/lib/ssl/src/tls_socket.erl index 154281f1c2..a391bc53de 100644 --- a/lib/ssl/src/tls_socket.erl +++ b/lib/ssl/src/tls_socket.erl @@ -64,11 +64,12 @@ accept(ListenSocket, #config{transport_info = {Transport,_,_,_} = CbInfo, {ok, Socket} -> {ok, EmOpts} = get_emulated_opts(Tracker), {ok, Port} = tls_socket:port(Transport, Socket), - ConnArgs = [server, "localhost", Port, Socket, + {ok, Sender} = tls_sender:start(), + ConnArgs = [server, Sender, "localhost", Port, Socket, {SslOpts, emulated_socket_options(EmOpts, #socket_options{}), Tracker}, self(), CbInfo], case tls_connection_sup:start_child(ConnArgs) of {ok, Pid} -> - ssl_connection:socket_control(ConnectionCb, Socket, Pid, Transport, Tracker); + ssl_connection:socket_control(ConnectionCb, Socket, [Pid, Sender], Transport, Tracker); {error, Reason} -> {error, Reason} end; @@ -112,8 +113,8 @@ connect(Address, Port, {error, {options, {socket_options, UserOpts}}} end. -socket(Pid, Transport, Socket, ConnectionCb, Tracker) -> - #sslsocket{pid = Pid, +socket(Pids, Transport, Socket, ConnectionCb, Tracker) -> + #sslsocket{pid = Pids, %% "The name "fd" is keept for backwards compatibility fd = {Transport, Socket, ConnectionCb, Tracker}}. setopts(gen_tcp, #sslsocket{pid = {ListenSocket, #config{emulated = Tracker}}}, Options) -> diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index 930ca60c5e..b6d38ee9db 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -3270,7 +3270,7 @@ no_reuses_session_server_restart_new_cert(Config) when is_list(Config) -> ssl_test_lib:start_server([{node, ServerNode}, {port, Port}, {from, self()}, {mfa, {ssl_test_lib, no_result, []}}, - {options, DsaServerOpts}]), + {options, [{reuseaddr, true} | DsaServerOpts]}]), Client1 = ssl_test_lib:start_client([{node, ClientNode}, @@ -3331,7 +3331,7 @@ no_reuses_session_server_restart_new_cert_file(Config) when is_list(Config) -> ssl_test_lib:start_server([{node, ServerNode}, {port, Port}, {from, self()}, {mfa, {ssl_test_lib, no_result, []}}, - {options, NewServerOpts1}]), + {options, [{reuseaddr, true} | NewServerOpts1]}]), Client1 = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, {host, Hostname}, @@ -3674,7 +3674,7 @@ hibernate(Config) -> {mfa, {ssl_test_lib, send_recv_result_active, []}}, {options, ServerOpts}]), Port = ssl_test_lib:inet_port(Server), - {Client, #sslsocket{pid=Pid}} = ssl_test_lib:start_client([return_socket, + {Client, #sslsocket{pid=[Pid|_]}} = ssl_test_lib:start_client([return_socket, {node, ClientNode}, {port, Port}, {host, Hostname}, {from, self()}, @@ -3717,7 +3717,7 @@ hibernate_right_away(Config) -> Server1 = ssl_test_lib:start_server(StartServerOpts), Port1 = ssl_test_lib:inet_port(Server1), - {Client1, #sslsocket{pid = Pid1}} = ssl_test_lib:start_client(StartClientOpts ++ + {Client1, #sslsocket{pid = [Pid1|_]}} = ssl_test_lib:start_client(StartClientOpts ++ [{port, Port1}, {options, [{hibernate_after, 0}|ClientOpts]}]), ssl_test_lib:check_result(Server1, ok, Client1, ok), @@ -3729,7 +3729,7 @@ hibernate_right_away(Config) -> Server2 = ssl_test_lib:start_server(StartServerOpts), Port2 = ssl_test_lib:inet_port(Server2), - {Client2, #sslsocket{pid = Pid2}} = ssl_test_lib:start_client(StartClientOpts ++ + {Client2, #sslsocket{pid = [Pid2|_]}} = ssl_test_lib:start_client(StartClientOpts ++ [{port, Port2}, {options, [{hibernate_after, 1}|ClientOpts]}]), ssl_test_lib:check_result(Server2, ok, Client2, ok), @@ -3965,13 +3965,13 @@ tls_tcp_error_propagation_in_active_mode(Config) when is_list(Config) -> {mfa, {ssl_test_lib, no_result, []}}, {options, ServerOpts}]), Port = ssl_test_lib:inet_port(Server), - {Client, #sslsocket{pid=Pid} = SslSocket} = ssl_test_lib:start_client([return_socket, - {node, ClientNode}, {port, Port}, - {host, Hostname}, - {from, self()}, - {mfa, {?MODULE, receive_msg, []}}, - {options, ClientOpts}]), - + {Client, #sslsocket{pid=[Pid|_]} = SslSocket} = ssl_test_lib:start_client([return_socket, + {node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, receive_msg, []}}, + {options, ClientOpts}]), + {status, _, _, StatusInfo} = sys:get_status(Pid), [_, _,_, _, Prop] = StatusInfo, State = ssl_test_lib:state(Prop), @@ -4645,6 +4645,7 @@ renegotiate_rejected(Socket) -> ok; %% Handle 1/n-1 splitting countermeasure Rizzo/Duong-Beast {ssl, Socket, "H"} -> + receive {ssl, Socket, "ello world"} -> ok diff --git a/lib/stdlib/doc/src/supervisor.xml b/lib/stdlib/doc/src/supervisor.xml index 6d5065ca02..0e8075a578 100644 --- a/lib/stdlib/doc/src/supervisor.xml +++ b/lib/stdlib/doc/src/supervisor.xml @@ -208,8 +208,16 @@ child_spec() = #{id => child_id(), % mandatory the child process is unconditionally terminated using <c>exit(Child,kill)</c>.</p> <p>If the child process is another supervisor, the shutdown time - is to be set to <c>infinity</c> to give the subtree ample - time to shut down. It is also allowed to set it to <c>infinity</c>, + must be set to <c>infinity</c> to give the subtree ample + time to shut down.</p> + <warning> + <p>Setting the shutdown time to anything other + than <c>infinity</c> for a child of type <c>supervisor</c> + can cause a race condition where the child in question + unlinks its own children, but fails to terminate them + before it is killed.</p> + </warning> + <p>It is also allowed to set it to <c>infinity</c>, if the child process is a worker.</p> <warning> <p>Be careful when setting the shutdown time to diff --git a/lib/stdlib/src/ms_transform.erl b/lib/stdlib/src/ms_transform.erl index d117481d2e..3845e35e9b 100644 --- a/lib/stdlib/src/ms_transform.erl +++ b/lib/stdlib/src/ms_transform.erl @@ -224,10 +224,12 @@ transform_from_shell(Dialect, Clauses, BoundEnvironment) -> %% Called when translating during compiling %% --spec parse_transform(Forms, Options) -> Forms2 when +-spec parse_transform(Forms, Options) -> Forms2 | Errors | Warnings when Forms :: [erl_parse:abstract_form() | erl_parse:form_info()], Forms2 :: [erl_parse:abstract_form() | erl_parse:form_info()], - Options :: term(). + Options :: term(), + Errors :: {error, ErrInfo :: [tuple()], WarnInfo :: []}, + Warnings :: {warning, Forms2, WarnInfo :: [tuple()]}. parse_transform(Forms, _Options) -> SaveFilename = setup_filename(), diff --git a/lib/stdlib/test/ets_SUITE.erl b/lib/stdlib/test/ets_SUITE.erl index 7a48d1d55e..d8912e548c 100644 --- a/lib/stdlib/test/ets_SUITE.erl +++ b/lib/stdlib/test/ets_SUITE.erl @@ -66,7 +66,7 @@ meta_lookup_named_read/1, meta_lookup_named_write/1, meta_newdel_unnamed/1, meta_newdel_named/1]). -export([smp_insert/1, smp_fixed_delete/1, smp_unfix_fix/1, smp_select_delete/1, - smp_select_replace/1, otp_8166/1, otp_8732/1]). + smp_select_replace/1, otp_8166/1, otp_8732/1, delete_unfix_race/1]). -export([exit_large_table_owner/1, exit_many_large_table_owner/1, exit_many_tables_owner/1, @@ -142,7 +142,8 @@ all() -> ets_all, massive_ets_all, take, - whereis_table]. + whereis_table, + delete_unfix_race]. groups() -> [{new, [], @@ -5489,6 +5490,46 @@ smp_fixed_delete_do() -> %%verify_table_load(T), ets:delete(T). +%% ERL-720 +%% Provoke race between ets:delete and table unfix (by select_count) +%% that caused ets_misc memory counter to indicate false leak. +delete_unfix_race(Config) when is_list(Config) -> + EtsMem = etsmem(), + Table = ets:new(t,[set,public,{write_concurrency,true}]), + InsertOp = + fun() -> + receive stop -> + false + after 0 -> + ets:insert(Table, {rand:uniform(10)}), + true + end + end, + DeleteOp = + fun() -> + receive stop -> + false + after 0 -> + ets:delete(Table, rand:uniform(10)), + true + end + end, + SelectOp = + fun() -> + ets:select_count(Table, ets:fun2ms(fun(X) -> true end)) + end, + Main = self(), + Ins = spawn(fun()-> repeat_while(InsertOp), Main ! self() end), + Del = spawn(fun()-> repeat_while(DeleteOp), Main ! self() end), + spawn(fun()-> + repeat(SelectOp, 10000), + Del ! stop, + Ins ! stop + end), + [receive Pid -> ok end || Pid <- [Ins,Del]], + ets:delete(Table), + verify_etsmem(EtsMem). + num_of_buckets(T) -> element(1,ets:info(T,stats)). diff --git a/lib/stdlib/test/re_SUITE_data/testoutput2 b/lib/stdlib/test/re_SUITE_data/testoutput2 index 811bbefc84..61ed8d9d4e 100644 --- a/lib/stdlib/test/re_SUITE_data/testoutput2 +++ b/lib/stdlib/test/re_SUITE_data/testoutput2 @@ -14705,4 +14705,20 @@ No options No first char No need char +"(?<=(a))\1?b" + ab + 0: b + 1: a + aaab + 0: ab + 1: a + +"(?=(a))\1?b" + ab + 0: ab + 1: a + aaab + 0: ab + 1: a + /-- End of testinput2 --/ diff --git a/lib/stdlib/test/re_SUITE_data/testoutput5 b/lib/stdlib/test/re_SUITE_data/testoutput5 index bab989ca7e..090e1e1c85 100644 --- a/lib/stdlib/test/re_SUITE_data/testoutput5 +++ b/lib/stdlib/test/re_SUITE_data/testoutput5 @@ -1942,4 +1942,12 @@ Need char = 'z' 0: \x{17f} 0+ +/\C[^\v]+\x80/8 + [AΏBŀC] +No match + +/\C[^\d]+\x80/8 + [AΏBŀC] +No match + /-- End of testinput5 --/ diff --git a/otp_versions.table b/otp_versions.table index cffea1d2ec..c33bb63002 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -1,3 +1,5 @@ +OTP-21.0.8 : erts-10.0.7 kernel-6.0.1 # asn1-5.0.6 common_test-1.16 compiler-7.2.3 crypto-4.3.2 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0.1 jinterface-1.9 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6.1 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0.1 stdlib-3.5.1 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : +OTP-21.0.7 : erts-10.0.6 # asn1-5.0.6 common_test-1.16 compiler-7.2.3 crypto-4.3.2 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0.1 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6.1 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0.1 stdlib-3.5.1 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0.6 : crypto-4.3.2 inets-7.0.1 ssl-9.0.1 # asn1-5.0.6 common_test-1.16 compiler-7.2.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 erts-10.0.5 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6.1 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 stdlib-3.5.1 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0.5 : compiler-7.2.3 crypto-4.3.1 erts-10.0.5 # asn1-5.0.6 common_test-1.16 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6.1 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 stdlib-3.5.1 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0.4 : erts-10.0.4 # asn1-5.0.6 common_test-1.16 compiler-7.2.2 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6.1 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 stdlib-3.5.1 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : @@ -5,6 +7,7 @@ OTP-21.0.3 : erts-10.0.3 # asn1-5.0.6 common_test-1.16 compiler-7.2.2 crypto-4.3 OTP-21.0.2 : compiler-7.2.2 erts-10.0.2 public_key-1.6.1 stdlib-3.5.1 # asn1-5.0.6 common_test-1.16 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0.1 : compiler-7.2.1 erts-10.0.1 # asn1-5.0.6 common_test-1.16 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 stdlib-3.5 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0 : asn1-5.0.6 common_test-1.16 compiler-7.2 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 erts-10.0 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 mnesia-4.15.4 observer-2.8 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 ssh-4.7 ssl-9.0 stdlib-3.5 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 # megaco-3.18.3 odbc-2.12.1 snmp-5.2.11 : +OTP-20.3.8.8 : inets-6.5.2.4 # asn1-5.0.5.1 common_test-1.15.4 compiler-7.1.5.1 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 ssl-8.2.6.2 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.8.7 : crypto-4.2.2.2 mnesia-4.15.3.2 # asn1-5.0.5.1 common_test-1.15.4 compiler-7.1.5.1 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.3 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 ssl-8.2.6.2 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.8.6 : inets-6.5.2.3 # asn1-5.0.5.1 common_test-1.15.4 compiler-7.1.5.1 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.1 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.1 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 ssl-8.2.6.2 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.8.5 : compiler-7.1.5.1 crypto-4.2.2.1 erts-9.3.3.3 mnesia-4.15.3.1 ssl-8.2.6.2 # asn1-5.0.5.1 common_test-1.15.4 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 erl_interface-3.10.2.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.2 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : diff --git a/system/COPYRIGHT b/system/COPYRIGHT index 5d47e0ca38..ff5cc30793 100644 --- a/system/COPYRIGHT +++ b/system/COPYRIGHT @@ -41,12 +41,15 @@ PCRE LICENCE PCRE is a library of functions to support regular expressions whose syntax and semantics are as close as possible to those of the Perl 5 language. -Release 7 of PCRE is distributed under the terms of the "BSD" licence, as +Release 8 of PCRE is distributed under the terms of the "BSD" licence, as specified below. The documentation for PCRE, supplied in the "doc" -directory, is distributed under the same terms as the software itself. +directory, is distributed under the same terms as the software itself. The data +in the testdata directory is not copyrighted and is in the public domain. The basic library functions are written in C and are freestanding. Also -included in the distribution is a set of C++ wrapper functions. +included in the distribution is a set of C++ wrapper functions, and a +just-in-time compiler that can be used to optimize pattern matching. These +are both optional features that can be omitted when the library is built. THE BASIC LIBRARY FUNCTIONS @@ -59,7 +62,29 @@ Email domain: cam.ac.uk University of Cambridge Computing Service, Cambridge, England. -Copyright (c) 1997-2008 University of Cambridge +Copyright (c) 1997-2018 University of Cambridge +All rights reserved. + + +PCRE JUST-IN-TIME COMPILATION SUPPORT +------------------------------------- + +Written by: Zoltan Herczeg +Email local part: hzmester +Emain domain: freemail.hu + +Copyright(c) 2010-2018 Zoltan Herczeg +All rights reserved. + + +STACK-LESS JUST-IN-TIME COMPILER +-------------------------------- + +Written by: Zoltan Herczeg +Email local part: hzmester +Emain domain: freemail.hu + +Copyright(c) 2009-2018 Zoltan Herczeg All rights reserved. @@ -68,7 +93,7 @@ THE C++ WRAPPER FUNCTIONS Contributed by: Google Inc. -Copyright (c) 2007-2008, Google Inc. +Copyright (c) 2007-2012, Google Inc. All rights reserved. diff --git a/system/doc/design_principles/sup_princ.xml b/system/doc/design_principles/sup_princ.xml index 06ca44a9f6..a2e0d95408 100644 --- a/system/doc/design_principles/sup_princ.xml +++ b/system/doc/design_principles/sup_princ.xml @@ -312,12 +312,17 @@ child_spec() = #{id => child_id(), % mandatory signal back. If no exit signal is received within the specified time, the child process is unconditionally terminated using <c>exit(Child, kill)</c>.</item> - <item>If the child process is another supervisor, it is to be + <item>If the child process is another supervisor, it must be set to <c>infinity</c> to give the subtree enough time to shut down. It is also allowed to set it to <c>infinity</c>, - if the child process is a worker. See the warning below:</item> + if the child process is a worker. See the warning below:</item> </list> <warning> + <p>Setting the shutdown time to anything other + than <c>infinity</c> for a child of type <c>supervisor</c> + can cause a race condition where the child in question + unlinks its own children, but fails to terminate them + before it is killed.</p> <p>Be careful when setting the shutdown time to <c>infinity</c> when the child process is a worker. Because, in this situation, the termination of the supervision tree depends on the |