diff options
66 files changed, 2283 insertions, 187 deletions
diff --git a/erts/doc/src/Makefile b/erts/doc/src/Makefile index fccdd744f3..40f74b78ff 100644 --- a/erts/doc/src/Makefile +++ b/erts/doc/src/Makefile @@ -53,19 +53,16 @@ XML_REF3_EFILES = \ erl_tracer.xml \ init.xml \ persistent_term.xml \ + atomics.xml \ + counters.xml \ zlib.xml XML_REF3_FILES = \ + $(XML_REF3_EFILES) \ driver_entry.xml \ erl_nif.xml \ - erl_tracer.xml \ erl_driver.xml \ - erl_prim_loader.xml \ - erlang.xml \ - erts_alloc.xml \ - init.xml \ - persistent_term.xml \ - zlib.xml + erts_alloc.xml XML_PART_FILES = \ part.xml diff --git a/erts/doc/src/atomics.xml b/erts/doc/src/atomics.xml new file mode 100644 index 0000000000..3fca92fb97 --- /dev/null +++ b/erts/doc/src/atomics.xml @@ -0,0 +1,183 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>2018</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + </legalnotice> + + <title>atomics</title> + </header> + <module>atomics</module> + <modulesummary>Atomic Functions</modulesummary> + <description> + <p>This module provides a set of functions to do atomic operations towards + mutable atomic variables. The implementation utilizes only + atomic hardware instructions without any software level locking, which makes + it very efficient for concurrent access. The atomics are organized into + arrays with the follwing semantics:</p> + <list type="bulleted"> + <item> + <p>Atomics are 64 bit integers.</p> + </item> + <item> + <p>Atomics can be represented as either signed or unsigned.</p> + </item> + <item> + <p>Atomics wrap around at overflow and underflow operations.</p> + </item> + <item> + <p>All operations guarantee atomicity. No intermediate results can be + seen. The result of one mutation can only be the input to one + following mutation.</p> + </item> + <item> + <p>All atomic operations are mutually ordered. If atomic B is updated + <em>after</em> atomic A, then that is how it will appear to any + concurrent readers. No one can read the new value of B and then read the + old value of A.</p> + </item> + <item> + <p>Indexes into atomic arrays are one-based. An atomic array of + arity N contains N atomics with index from 1 to N.</p> + </item> + </list> + </description> + + <datatypes> + <datatype> + <name name="atomics_ref"/> + <desc><p>Identifies an atomic array returned from + <seealso marker="#new/2"><c>new/2</c></seealso>.</p> + </desc> + </datatype> + </datatypes> + + <funcs> + <func> + <name name="new" arity="2"/> + <fsummary>Create atomic array</fsummary> + <desc> + <p>Create a new atomic array of <c><anno>Arity</anno></c> atomics.</p> + <p>Argument <c><anno>Opts</anno></c> is a list of the following possible + options:</p> + <taglist> + <tag><c>{signed, boolean()}</c></tag> + <item><p>Indicate if the elements of the array will be treated + as signed or unsigned integers. Default is <c>true</c> (signed).</p> + <p>The integer interval for signed atomics are from <c>-(1 bsl 63)</c> + to <c>(1 bsl 63)-1</c> and for unsigned atomics from <c>0</c> to <c>(1 + bsl 64)-1</c>.</p> + </item> + </taglist> + </desc> + </func> + + <func> + <name name="put" arity="3"/> + <fsummary>Set atomic value</fsummary> + <desc> + <p>Set atomic to <c><anno>Value</anno></c>.</p> + </desc> + </func> + + <func> + <name name="get" arity="2"/> + <fsummary>Read atomic value</fsummary> + <desc> + <p>Read atomic value.</p> + </desc> + </func> + + <func> + <name name="add" arity="3"/> + <fsummary>Add to atomic</fsummary> + <desc> + <p>Add <c><anno>Incr</anno></c> to atomic.</p> + </desc> + </func> + + <func> + <name name="add_get" arity="3"/> + <fsummary>Atomic add and get</fsummary> + <desc> + <p>Atomic addition and return of the result.</p> + </desc> + </func> + + <func> + <name name="sub" arity="3"/> + <fsummary>Subtract from atomic</fsummary> + <desc> + <p>Subtract <c><anno>Decr</anno></c> from atomic.</p> + </desc> + </func> + + <func> + <name name="sub_get" arity="3"/> + <fsummary>Atomic sub and get</fsummary> + <desc> + <p>Atomic subtraction and return of the result.</p> + </desc> + </func> + + <func> + <name name="exchange" arity="3"/> + <fsummary>Atomic exchange.</fsummary> + <desc> + <p>Atomically replaces the value of the atomic with + <c><anno>Desired</anno></c> and returns the value it held + previously.</p> + </desc> + </func> + + <func> + <name name="compare_exchange" arity="4"/> + <fsummary>Atomic compare and exchange.</fsummary> + <desc> + <p>Atomically compares the atomic with <c><anno>Expected</anno></c>, + and if those are equal, set atomic to <c><anno>Desired</anno></c>. + Returns <c>ok</c> if <c><anno>Desired</anno></c> was written. Returns + the actual atomic value if not equal to <c><anno>Expected</anno></c>.</p> + </desc> + </func> + + <func> + <name name="info" arity="1"/> + <fsummary>Get information about atomic array.</fsummary> + <desc> + <p>Return information about an atomic array in a map. The map + has the following keys:</p> + <taglist> + <tag><c>size</c></tag> + <item><p>The number of atomics in the array.</p></item> + <tag><c>max</c></tag> + <item><p>The highest possible value an atomic in this array can + hold.</p></item> + <tag><c>min</c></tag> + <item><p>The lowest possible value an atomic in this array can + hold.</p></item> + <tag><c>memory</c></tag> + <item><p>Approximate memory consumption for the array in + bytes.</p></item> + </taglist> + </desc> + </func> + + </funcs> +</erlref> diff --git a/erts/doc/src/counters.xml b/erts/doc/src/counters.xml new file mode 100644 index 0000000000..85eedfdadc --- /dev/null +++ b/erts/doc/src/counters.xml @@ -0,0 +1,142 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>2018</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + </legalnotice> + + <title>counters</title> + </header> + <module>counters</module> + <modulesummary>Counter Functions</modulesummary> + <description> + <p>This module provides a set of functions to do operations towards + shared mutable counter variables. The implementation does not utilize any + software level locking, which makes it very efficient for concurrent + access. The counters are organized into arrays with the follwing + semantics:</p> + <list type="bulleted"> + <item> + <p>Counters are 64 bit signed integers.</p> + </item> + <item> + <p>Counters wrap around at overflow and underflow operations.</p> + </item> + <item><p>Counters are initialized to zero and can then only be written to + by adding or subtracting.</p> + </item> + <item> + <p>Write operations guarantee atomicity. No intermediate results can be + seen from a single write operation.</p> + </item> + <item> + <p>Two types of counter arrays can be created with options <c>atomics</c> or + <c>write_concurrency</c>. The <c>atomics</c> counters have good allround + performance with nice consistent semantics while + <c>write_concurrency</c> counters offers even better concurrent + write performance at the expense of some potential read + inconsistencies. See <seealso marker="#new/2"><c>new/2</c></seealso>.</p> + </item> + <item> + <p>Indexes into counter arrays are one-based. A counter array of + size N contains N counters with index from 1 to N.</p> + </item> + </list> + </description> + + <datatypes> + <datatype> + <name name="counters_ref"/> + <desc><p>Identifies a counter array returned from + <seealso marker="#new/2"><c>new/2</c></seealso>.</p> + </desc> + </datatype> + </datatypes> + + <funcs> + <func> + <name name="new" arity="2"/> + <fsummary>Create counter array</fsummary> + <desc> + <p>Create a new counter array of <c><anno>Size</anno></c> counters.</p> + <p>Argument <c><anno>Opts</anno></c> is a list of the following possible + options:</p> + <taglist> + <tag><c>atomics</c> (Default)</tag> + <item><p>Counters will be sequentially consistent. If write + operation A is done sequencially before write operation B, then a concurrent reader + may see none of them, only A, or both A and B. It cannot see only B.</p> + </item> + <tag><c>write_concurrency</c></tag> + <item><p>This is an optimization to achieve very efficient concurrent + write operations at the expense of potential read inconsistency and memory + consumption per counter.</p> + <p>Read operations may see sequentially inconsistent results with + regard to concurrent write operations. Even if write operation A is done + sequencially before write operation B, a concurrent reader may see any + combination of A and B, including only B. A read operation is only + guaranteed to see all writes done sequentially before the read. No writes + are ever lost, but will eventually all be seen.</p> + </item> + </taglist> + </desc> + </func> + + <func> + <name name="get" arity="2"/> + <fsummary>Read counter value</fsummary> + <desc> + <p>Read counter value.</p> + </desc> + </func> + + <func> + <name name="add" arity="3"/> + <fsummary>Add to counter</fsummary> + <desc> + <p>Add <c><anno>Incr</anno></c> to counter.</p> + </desc> + </func> + + <func> + <name name="sub" arity="3"/> + <fsummary>Subtract from counter</fsummary> + <desc> + <p>Subtract <c><anno>Decr</anno></c> from counter.</p> + </desc> + </func> + + <func> + <name name="info" arity="1"/> + <fsummary>Get information about counter array.</fsummary> + <desc> + <p>Return information about a counter array in a map. The map + has the following keys (at least):</p> + <taglist> + <tag><c>size</c></tag> + <item><p>The number of counters in the array.</p></item> + <tag><c>memory</c></tag> + <item><p>Approximate memory consumption for the array in + bytes.</p></item> + </taglist> + </desc> + </func> + + </funcs> +</erlref> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index a71d1eddf5..a6c4f6e87b 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -31,6 +31,22 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 10.1.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a rare bug where files could be closed on a + normal instead of an IO scheduler, resulting in system + instability if the operation blocked.</p> + <p> + Own Id: OTP-15421</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 10.1.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -1355,6 +1371,21 @@ </section> +<section><title>Erts 9.3.3.6</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>List subtraction (The <c>--</c> operator) will now + yield properly on large inputs.</p> + <p> + Own Id: OTP-15371</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 9.3.3.5</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -4799,6 +4830,21 @@ </section> +<section><title>Erts 7.3.1.6</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>List subtraction (The <c>--</c> operator) will now + yield properly on large inputs.</p> + <p> + Own Id: OTP-15371</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 7.3.1.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/doc/src/ref_man.xml b/erts/doc/src/ref_man.xml index ff64aa86b8..a78aaa449e 100644 --- a/erts/doc/src/ref_man.xml +++ b/erts/doc/src/ref_man.xml @@ -50,5 +50,7 @@ <xi:include href="erts_alloc.xml"/> <xi:include href="erl_nif.xml"/> <xi:include href="erl_tracer.xml"/> + <xi:include href="atomics.xml"/> + <xi:include href="counters.xml"/> </application> diff --git a/erts/doc/src/specs.xml b/erts/doc/src/specs.xml index 1ab6cdc19e..0b943e6295 100644 --- a/erts/doc/src/specs.xml +++ b/erts/doc/src/specs.xml @@ -6,4 +6,6 @@ <xi:include href="../specs/specs_init.xml"/> <xi:include href="../specs/specs_persistent_term.xml"/> <xi:include href="../specs/specs_zlib.xml"/> + <xi:include href="../specs/specs_atomics.xml"/> + <xi:include href="../specs/specs_counters.xml"/> </specs> diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in index 57c562ec8c..b74f8371f5 100644 --- a/erts/emulator/Makefile.in +++ b/erts/emulator/Makefile.in @@ -648,6 +648,8 @@ PRELOAD_BEAM = $(ERL_TOP)/erts/preloaded/ebin/erts_code_purger.beam \ $(ERL_TOP)/erts/preloaded/ebin/erl_tracer.beam \ $(ERL_TOP)/erts/preloaded/ebin/erts_literal_area_collector.beam \ $(ERL_TOP)/erts/preloaded/ebin/erts_dirty_process_signal_handler.beam \ + $(ERL_TOP)/erts/preloaded/ebin/atomics.beam \ + $(ERL_TOP)/erts/preloaded/ebin/counters.beam \ $(ERL_TOP)/erts/preloaded/ebin/persistent_term.beam ifeq ($(TARGET),win32) @@ -841,6 +843,7 @@ RUN_OBJS += \ $(OBJDIR)/erl_bif_info.o $(OBJDIR)/erl_bif_op.o \ $(OBJDIR)/erl_bif_os.o $(OBJDIR)/erl_bif_lists.o \ $(OBJDIR)/erl_bif_persistent.o \ + $(OBJDIR)/erl_bif_atomics.o $(OBJDIR)/erl_bif_counters.o \ $(OBJDIR)/erl_bif_trace.o $(OBJDIR)/erl_bif_unique.o \ $(OBJDIR)/erl_bif_wrap.o $(OBJDIR)/erl_nfunc_sched.o \ $(OBJDIR)/erl_guard_bifs.o $(OBJDIR)/erl_dirty_bif_wrap.o \ diff --git a/erts/emulator/beam/atom.names b/erts/emulator/beam/atom.names index 0167286bbf..3f7aa0983c 100644 --- a/erts/emulator/beam/atom.names +++ b/erts/emulator/beam/atom.names @@ -396,6 +396,7 @@ atom microsecond atom microstate_accounting atom milli_seconds atom millisecond +atom min atom min_heap_size atom min_bin_vheap_size atom minor diff --git a/erts/emulator/beam/bif.tab b/erts/emulator/beam/bif.tab index 9a11e6a455..6015fc0691 100644 --- a/erts/emulator/beam/bif.tab +++ b/erts/emulator/beam/bif.tab @@ -709,3 +709,17 @@ bif persistent_term:get/0 bif persistent_term:erase/1 bif persistent_term:info/0 bif erts_internal:erase_persistent_terms/0 + +bif erts_internal:atomics_new/2 +bif atomics:get/2 +bif atomics:put/3 +bif atomics:add/3 +bif atomics:add_get/3 +bif atomics:exchange/3 +bif atomics:compare_exchange/4 +bif atomics:info/1 + +bif erts_internal:counters_new/1 +bif erts_internal:counters_get/2 +bif erts_internal:counters_add/3 +bif erts_internal:counters_info/1 diff --git a/erts/emulator/beam/erl_alloc.types b/erts/emulator/beam/erl_alloc.types index 02a9581c28..08dcc5ea9a 100644 --- a/erts/emulator/beam/erl_alloc.types +++ b/erts/emulator/beam/erl_alloc.types @@ -338,6 +338,8 @@ type GC_INFO_REQ SHORT_LIVED SYSTEM gc_info_request type PORT_DATA_HEAP STANDARD SYSTEM port_data_heap type MSACC DRIVER SYSTEM microstate_accounting type SYS_CHECK_REQ SHORT_LIVED SYSTEM system_check_request +type ATOMICS STANDARD SYSTEM erl_bif_atomics +type COUNTERS STANDARD SYSTEM erl_bif_counters # # Types used by system specific code diff --git a/erts/emulator/beam/erl_bif_atomics.c b/erts/emulator/beam/erl_bif_atomics.c new file mode 100644 index 0000000000..092dbb3bd3 --- /dev/null +++ b/erts/emulator/beam/erl_bif_atomics.c @@ -0,0 +1,256 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2018. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * %CopyrightEnd% + */ + +/* + * Purpose: High performance atomics. + */ + +#ifdef HAVE_CONFIG_H +# include "config.h" +#endif + +#include <stddef.h> /* offsetof */ + +#include "sys.h" +#include "export.h" +#include "bif.h" +#include "erl_threads.h" +#include "big.h" +#include "erl_binary.h" +#include "erl_bif_unique.h" +#include "erl_map.h" + +typedef struct +{ + int is_signed; + UWord vlen; + erts_atomic64_t v[1]; +}AtomicsRef; + +static int atomics_destructor(Binary *unused) +{ + return 1; +} + +#define OPT_SIGNED (1 << 0) + +BIF_RETTYPE erts_internal_atomics_new_2(BIF_ALIST_2) +{ + AtomicsRef* p; + Binary* mbin; + UWord i, cnt, opts; + Uint bytes; + Eterm* hp; + + if (!term_to_UWord(BIF_ARG_1, &cnt) + || cnt == 0 + || !term_to_UWord(BIF_ARG_2, &opts)) { + + BIF_ERROR(BIF_P, BADARG); + } + + if (cnt > (ERTS_UWORD_MAX / sizeof(p->v[0]))) + BIF_ERROR(BIF_P, SYSTEM_LIMIT); + + bytes = offsetof(AtomicsRef, v) + cnt*sizeof(p->v[0]); + mbin = erts_create_magic_binary_x(bytes, + atomics_destructor, + ERTS_ALC_T_ATOMICS, + 0); + p = ERTS_MAGIC_BIN_DATA(mbin); + p->is_signed = opts & OPT_SIGNED; + p->vlen = cnt; + for (i=0; i < cnt; i++) + erts_atomic64_init_nob(&p->v[i], 0); + hp = HAlloc(BIF_P, ERTS_MAGIC_REF_THING_SIZE); + return erts_mk_magic_ref(&hp, &MSO(BIF_P), mbin); +} + +static ERTS_INLINE int get_ref(Eterm ref, AtomicsRef** pp) +{ + Binary* mbin; + if (!is_internal_magic_ref(ref)) + return 0; + + mbin = erts_magic_ref2bin(ref); + if (ERTS_MAGIC_BIN_DESTRUCTOR(mbin) != atomics_destructor) + return 0; + *pp = ERTS_MAGIC_BIN_DATA(mbin); + return 1; +} + +static ERTS_INLINE int get_ref_ix(Eterm ref, Eterm ix, + AtomicsRef** pp, UWord* ixp) +{ + return (get_ref(ref, pp) + && term_to_UWord(ix, ixp) + && --(*ixp) < (*pp)->vlen); +} + +static ERTS_INLINE int get_value(AtomicsRef* p, Eterm term, erts_aint64_t *valp) +{ + return (p->is_signed ? + term_to_Sint64(term, (Sint64*)valp) : + term_to_Uint64(term, (Uint64*)valp)); +} + +static ERTS_INLINE int get_incr(AtomicsRef* p, Eterm term, erts_aint64_t *valp) +{ + return (term_to_Sint64(term, (Sint64*)valp) + || term_to_Uint64(term, (Uint64*)valp)); +} + +static ERTS_INLINE Eterm bld_atomic(Process* proc, AtomicsRef* p, + erts_aint64_t val) +{ + if (p->is_signed) { + if (IS_SSMALL(val)) + return make_small((Sint) val); + else { + Uint hsz = ERTS_SINT64_HEAP_SIZE(val); + Eterm* hp = HAlloc(proc, hsz); + return erts_sint64_to_big(val, &hp); + } + } + else { + if ((Uint64)val <= MAX_SMALL) + return make_small((Sint) val); + else { + Uint hsz = ERTS_UINT64_HEAP_SIZE(val); + Eterm* hp = HAlloc(proc, hsz); + return erts_uint64_to_big(val, &hp); + } + } +} + +BIF_RETTYPE atomics_put_3(BIF_ALIST_3) +{ + AtomicsRef* p; + UWord ix; + erts_aint64_t val; + + if (!get_ref_ix(BIF_ARG_1, BIF_ARG_2, &p, &ix) + || !get_value(p, BIF_ARG_3, &val)) { + BIF_ERROR(BIF_P, BADARG); + } + erts_atomic64_set_mb(&p->v[ix], val); + return am_ok; +} + +BIF_RETTYPE atomics_get_2(BIF_ALIST_2) +{ + AtomicsRef* p; + UWord ix; + + if (!get_ref_ix(BIF_ARG_1, BIF_ARG_2, &p, &ix)) { + BIF_ERROR(BIF_P, BADARG); + } + return bld_atomic(BIF_P, p, erts_atomic64_read_mb(&p->v[ix])); +} + +BIF_RETTYPE atomics_add_3(BIF_ALIST_3) +{ + AtomicsRef* p; + UWord ix; + erts_aint64_t incr; + + if (!get_ref_ix(BIF_ARG_1, BIF_ARG_2, &p, &ix) + || !get_incr(p, BIF_ARG_3, &incr)) { + BIF_ERROR(BIF_P, BADARG); + } + erts_atomic64_add_mb(&p->v[ix], incr); + return am_ok; +} + +BIF_RETTYPE atomics_add_get_3(BIF_ALIST_3) +{ + AtomicsRef* p; + UWord ix; + erts_aint64_t incr; + + if (!get_ref_ix(BIF_ARG_1, BIF_ARG_2, &p, &ix) + || !get_incr(p, BIF_ARG_3, &incr)) { + BIF_ERROR(BIF_P, BADARG); + } + return bld_atomic(BIF_P, p, erts_atomic64_add_read_mb(&p->v[ix], incr)); +} + +BIF_RETTYPE atomics_exchange_3(BIF_ALIST_3) +{ + AtomicsRef* p; + UWord ix; + erts_aint64_t desired, was; + + if (!get_ref_ix(BIF_ARG_1, BIF_ARG_2, &p, &ix) + || !get_value(p, BIF_ARG_3, &desired)) { + BIF_ERROR(BIF_P, BADARG); + } + was = erts_atomic64_xchg_mb(&p->v[ix], desired); + return bld_atomic(BIF_P, p, was); +} + +BIF_RETTYPE atomics_compare_exchange_4(BIF_ALIST_4) +{ + AtomicsRef* p; + UWord ix; + erts_aint64_t expected, desired, was; + + if (!get_ref_ix(BIF_ARG_1, BIF_ARG_2, &p, &ix) + || !get_value(p, BIF_ARG_3, &expected) + || !get_value(p, BIF_ARG_4, &desired)) { + BIF_ERROR(BIF_P, BADARG); + } + was = erts_atomic64_cmpxchg_mb(&p->v[ix], desired, expected); + return was == expected ? am_ok : bld_atomic(BIF_P, p, was); +} + +BIF_RETTYPE atomics_info_1(BIF_ALIST_1) +{ + AtomicsRef* p; + Uint hsz = MAP4_SZ; + Eterm *hp; + Uint64 max; + Sint64 min; + UWord memory; + Eterm max_val, min_val, sz_val, mem_val; + + if (!get_ref(BIF_ARG_1, &p)) + BIF_ERROR(BIF_P, BADARG); + + max = p->is_signed ? ERTS_SINT64_MAX : ERTS_UINT64_MAX; + min = p->is_signed ? ERTS_SINT64_MIN : 0; + memory = erts_magic_ref2bin(BIF_ARG_1)->orig_size; + + erts_bld_uint64(NULL, &hsz, max); + erts_bld_sint64(NULL, &hsz, min); + erts_bld_uword(NULL, &hsz, p->vlen); + erts_bld_uword(NULL, &hsz, memory); + + hp = HAlloc(BIF_P, hsz); + max_val = erts_bld_uint64(&hp, NULL, max); + min_val = erts_bld_sint64(&hp, NULL, min); + sz_val = erts_bld_uword(&hp, NULL, p->vlen); + mem_val = erts_bld_uword(&hp, NULL, memory); + + return MAP4(hp, am_max, max_val, + am_memory, mem_val, + am_min, min_val, + am_size, sz_val); +} diff --git a/erts/emulator/beam/erl_bif_counters.c b/erts/emulator/beam/erl_bif_counters.c new file mode 100644 index 0000000000..a46b462225 --- /dev/null +++ b/erts/emulator/beam/erl_bif_counters.c @@ -0,0 +1,219 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2018. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * %CopyrightEnd% + */ + +/* + * Purpose: High performance atomics. + */ + +#ifdef HAVE_CONFIG_H +# include "config.h" +#endif + +#include <stddef.h> /* offsetof */ + +#include "sys.h" +#include "export.h" +#include "bif.h" +#include "erl_threads.h" +#include "big.h" +#include "erl_binary.h" +#include "erl_bif_unique.h" +#include "erl_map.h" + + +#define COUNTERS_PER_CACHE_LINE (ERTS_CACHE_LINE_SIZE / sizeof(erts_atomic64_t)) + +typedef struct +{ + UWord arity; +#ifdef DEBUG + UWord ulen; +#endif + union { + erts_atomic64_t v[COUNTERS_PER_CACHE_LINE]; + byte cache_line__[ERTS_CACHE_LINE_SIZE]; + } u[1]; +}CountersRef; + +static int counters_destructor(Binary *unused) +{ + return 1; +} + + +static UWord ERTS_INLINE div_ceil(UWord dividend, UWord divisor) +{ + return (dividend + divisor - 1) / divisor; +} + +BIF_RETTYPE erts_internal_counters_new_1(BIF_ALIST_1) +{ + CountersRef* p; + Binary* mbin; + UWord ui, vi, cnt; + Uint bytes, cache_lines; + Eterm* hp; + + if (!term_to_UWord(BIF_ARG_1, &cnt) + || cnt == 0) { + BIF_ERROR(BIF_P, BADARG); + } + + if (cnt > (ERTS_UWORD_MAX / (sizeof(erts_atomic64_t)*2*erts_no_schedulers))) + BIF_ERROR(BIF_P, SYSTEM_LIMIT); + + cache_lines = erts_no_schedulers * div_ceil(cnt, COUNTERS_PER_CACHE_LINE); + bytes = offsetof(CountersRef, u) + cache_lines * ERTS_CACHE_LINE_SIZE; + mbin = erts_create_magic_binary_x(bytes, + counters_destructor, + ERTS_ALC_T_ATOMICS, + 0); + p = ERTS_MAGIC_BIN_DATA(mbin); + p->arity = cnt; +#ifdef DEBUG + p->ulen = cache_lines; +#endif + ASSERT((byte*)&p->u[cache_lines] <= ((byte*)p + bytes)); + for (ui=0; ui < cache_lines; ui++) + for (vi=0; vi < COUNTERS_PER_CACHE_LINE; vi++) + erts_atomic64_init_nob(&p->u[ui].v[vi], 0); + hp = HAlloc(BIF_P, ERTS_MAGIC_REF_THING_SIZE); + return erts_mk_magic_ref(&hp, &MSO(BIF_P), mbin); +} + +static ERTS_INLINE int get_ref(Eterm ref, CountersRef** pp) +{ + Binary* mbin; + if (!is_internal_magic_ref(ref)) + return 0; + + mbin = erts_magic_ref2bin(ref); + if (ERTS_MAGIC_BIN_DESTRUCTOR(mbin) != counters_destructor) + return 0; + *pp = ERTS_MAGIC_BIN_DATA(mbin); + return 1; +} + +static ERTS_INLINE int get_ref_cnt(Eterm ref, Eterm index, + CountersRef** pp, + erts_atomic64_t** app, + UWord sched_ix) +{ + CountersRef* p; + UWord ix, ui, vi; + if (!get_ref(ref, &p) || !term_to_UWord(index, &ix) || --ix >= p->arity) + return 0; + ui = (ix / COUNTERS_PER_CACHE_LINE) * erts_no_schedulers + sched_ix; + vi = ix % COUNTERS_PER_CACHE_LINE; + ASSERT(ui < p->ulen); + *pp = p; + *app = &p->u[ui].v[vi]; + return 1; +} + +static ERTS_INLINE int get_ref_my_cnt(Eterm ref, Eterm index, + CountersRef** pp, + erts_atomic64_t** app) +{ + ErtsSchedulerData *esdp = erts_get_scheduler_data(); + ASSERT(esdp && !ERTS_SCHEDULER_IS_DIRTY(esdp)); + return get_ref_cnt(ref, index, pp, app, esdp->no - 1); +} + +static ERTS_INLINE int get_ref_first_cnt(Eterm ref, Eterm index, + CountersRef** pp, + erts_atomic64_t** app) +{ + return get_ref_cnt(ref, index, pp, app, 0); +} + +static ERTS_INLINE int get_incr(CountersRef* p, Eterm term, erts_aint64_t *valp) +{ + return (term_to_Sint64(term, (Sint64*)valp) + || term_to_Uint64(term, (Uint64*)valp)); +} + +static ERTS_INLINE Eterm bld_counter(Process* proc, CountersRef* p, + erts_aint64_t val) +{ + if (IS_SSMALL(val)) + return make_small((Sint) val); + else { + Uint hsz = ERTS_SINT64_HEAP_SIZE(val); + Eterm* hp = HAlloc(proc, hsz); + return erts_sint64_to_big(val, &hp); + } +} + +BIF_RETTYPE erts_internal_counters_get_2(BIF_ALIST_2) +{ + CountersRef* p; + erts_atomic64_t* ap; + erts_aint64_t acc = 0; + int j; + + if (!get_ref_first_cnt(BIF_ARG_1, BIF_ARG_2, &p, &ap)) { + BIF_ERROR(BIF_P, BADARG); + } + for (j = erts_no_schedulers; j ; --j) { + acc += erts_atomic64_read_nob(ap); + ap = (erts_atomic64_t*) ((byte*)ap + ERTS_CACHE_LINE_SIZE); + } + return bld_counter(BIF_P, p, acc); +} + +BIF_RETTYPE erts_internal_counters_add_3(BIF_ALIST_3) +{ + CountersRef* p; + erts_atomic64_t* ap; + erts_aint64_t incr, sum; + + if (!get_ref_my_cnt(BIF_ARG_1, BIF_ARG_2, &p, &ap) + || !get_incr(p, BIF_ARG_3, &incr)) { + BIF_ERROR(BIF_P, BADARG); + } + sum = incr + erts_atomic64_read_nob(ap); + erts_atomic64_set_nob(ap, sum); + return am_ok; +} + + +BIF_RETTYPE erts_internal_counters_info_1(BIF_ALIST_1) +{ + CountersRef* p; + Uint hsz = MAP2_SZ; + Eterm *hp; + UWord memory; + Eterm sz_val, mem_val; + + if (!get_ref(BIF_ARG_1, &p)) + BIF_ERROR(BIF_P, BADARG); + + memory = erts_magic_ref2bin(BIF_ARG_1)->orig_size; + erts_bld_uword(NULL, &hsz, p->arity); + erts_bld_uword(NULL, &hsz, memory); + + hp = HAlloc(BIF_P, hsz); + sz_val = erts_bld_uword(&hp, NULL, p->arity); + mem_val = erts_bld_uword(&hp, NULL, memory); + + return MAP2(hp, am_memory, mem_val, + am_size, sz_val); +} diff --git a/erts/emulator/beam/erl_monitor_link.h b/erts/emulator/beam/erl_monitor_link.h index 9ff8aa509a..ed7bf7d54a 100644 --- a/erts/emulator/beam/erl_monitor_link.h +++ b/erts/emulator/beam/erl_monitor_link.h @@ -1387,11 +1387,14 @@ ERTS_GLB_INLINE void erts_monitor_release(ErtsMonitor *mon) { ErtsMonitorData *mdp = erts_monitor_to_data(mon); - ERTS_ML_ASSERT(!(mon->flags & ERTS_ML_FLG_IN_TABLE)); ERTS_ML_ASSERT(erts_atomic32_read_nob(&mdp->refc) > 0); - if (erts_atomic32_dec_read_nob(&mdp->refc) == 0) + if (erts_atomic32_dec_read_nob(&mdp->refc) == 0) { + ERTS_ML_ASSERT(!(mdp->origin.flags & ERTS_ML_FLG_IN_TABLE)); + ERTS_ML_ASSERT(!(mdp->target.flags & ERTS_ML_FLG_IN_TABLE)); + erts_monitor_destroy__(mdp); + } } ERTS_GLB_INLINE void @@ -1399,12 +1402,14 @@ erts_monitor_release_both(ErtsMonitorData *mdp) { ERTS_ML_ASSERT((mdp->origin.flags & ERTS_ML_FLGS_SAME) == (mdp->target.flags & ERTS_ML_FLGS_SAME)); - ERTS_ML_ASSERT(!(mdp->origin.flags & ERTS_ML_FLG_IN_TABLE)); - ERTS_ML_ASSERT(!(mdp->target.flags & ERTS_ML_FLG_IN_TABLE)); ERTS_ML_ASSERT(erts_atomic32_read_nob(&mdp->refc) >= 2); - if (erts_atomic32_add_read_nob(&mdp->refc, (erts_aint32_t) -2) == 0) + if (erts_atomic32_add_read_nob(&mdp->refc, (erts_aint32_t) -2) == 0) { + ERTS_ML_ASSERT(!(mdp->origin.flags & ERTS_ML_FLG_IN_TABLE)); + ERTS_ML_ASSERT(!(mdp->target.flags & ERTS_ML_FLG_IN_TABLE)); + erts_monitor_destroy__(mdp); + } } ERTS_GLB_INLINE int diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h index bb22548587..869a575cb4 100644 --- a/erts/emulator/beam/sys.h +++ b/erts/emulator/beam/sys.h @@ -325,6 +325,7 @@ typedef long Sint erts_align_attribute(sizeof(long)); #define UWORD_CONSTANT(Const) Const##UL #define ERTS_UWORD_MAX ULONG_MAX #define ERTS_SWORD_MAX LONG_MAX +#define ERTS_SWORD_MIN LONG_MIN #define ERTS_SIZEOF_ETERM SIZEOF_LONG #define ErtsStrToSint strtol #elif SIZEOF_VOID_P == SIZEOF_INT @@ -335,6 +336,7 @@ typedef int Sint erts_align_attribute(sizeof(int)); #define UWORD_CONSTANT(Const) Const##U #define ERTS_UWORD_MAX UINT_MAX #define ERTS_SWORD_MAX INT_MAX +#define ERTS_SWORD_MIN INT_MIN #define ERTS_SIZEOF_ETERM SIZEOF_INT #define ErtsStrToSint strtol #elif SIZEOF_VOID_P == SIZEOF_LONG_LONG @@ -345,6 +347,7 @@ typedef long long Sint erts_align_attribute(sizeof(long long)); #define UWORD_CONSTANT(Const) Const##ULL #define ERTS_UWORD_MAX ULLONG_MAX #define ERTS_SWORD_MAX LLONG_MAX +#define ERTS_SWORD_MIN LLONG_MIN #define ERTS_SIZEOF_ETERM SIZEOF_LONG_LONG #if defined(__WIN32__) #define ErtsStrToSint _strtoi64 diff --git a/erts/emulator/nifs/common/prim_file_nif.c b/erts/emulator/nifs/common/prim_file_nif.c index 009761a7f8..0ada345442 100644 --- a/erts/emulator/nifs/common/prim_file_nif.c +++ b/erts/emulator/nifs/common/prim_file_nif.c @@ -38,6 +38,9 @@ static void unload(ErlNifEnv *env, void* priv_data); static ErlNifResourceType *efile_resource_type; +static ERL_NIF_TERM am_erts_prim_file; +static ERL_NIF_TERM am_close; + static ERL_NIF_TERM am_ok; static ERL_NIF_TERM am_error; static ERL_NIF_TERM am_continue; @@ -96,11 +99,14 @@ static ERL_NIF_TERM set_cwd_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM arg static ERL_NIF_TERM read_file_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); +static ERL_NIF_TERM open_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); +static ERL_NIF_TERM close_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); + +/* Internal ops */ +static ERL_NIF_TERM delayed_close_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM get_handle_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM altname_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM open_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); - /* All file handle operations are passed through a wrapper that handles state * transitions, marking it as busy during the course of the operation, and * closing on completion if the owner died in the middle of an operation. @@ -128,7 +134,11 @@ static ERL_NIF_TERM open_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[] * * CLOSE_PENDING -> * CLOSED (file_handle_wrapper) - */ + * + * Should the owner of a file die, we can't close it immediately as that could + * potentially block a normal scheduler. When entering the CLOSED state from + * owner_death_callback, we will instead send a message to the erts_prim_file + * process that will then close the file through delayed_close_nif. */ typedef ERL_NIF_TERM (*file_op_impl_t)(efile_data_t *d, ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]); @@ -142,7 +152,6 @@ static ERL_NIF_TERM file_handle_wrapper(file_op_impl_t operation, ErlNifEnv *env return file_handle_wrapper( name ## _impl , env, argc, argv); \ } -WRAP_FILE_HANDLE_EXPORT(close_nif) WRAP_FILE_HANDLE_EXPORT(read_nif) WRAP_FILE_HANDLE_EXPORT(write_nif) WRAP_FILE_HANDLE_EXPORT(pread_nif) @@ -193,18 +202,27 @@ static ErlNifFunc nif_funcs[] = { /* Internal ops. */ {"get_handle_nif", 1, get_handle_nif}, + {"delayed_close_nif", 1, delayed_close_nif, ERL_NIF_DIRTY_JOB_IO_BOUND}, {"altname_nif", 1, altname_nif, ERL_NIF_DIRTY_JOB_IO_BOUND}, }; ERL_NIF_INIT(prim_file, nif_funcs, load, NULL, upgrade, unload) +static ErlNifPid erts_prim_file_pid; + static void owner_death_callback(ErlNifEnv* env, void* obj, ErlNifPid* pid, ErlNifMonitor* mon); -static void gc_callback(ErlNifEnv *env, void* data); -static int load(ErlNifEnv *env, void** priv_data, ERL_NIF_TERM load_info) +static int load(ErlNifEnv *env, void** priv_data, ERL_NIF_TERM prim_file_pid) { ErlNifResourceTypeInit callbacks; + if(!enif_get_local_pid(env, prim_file_pid, &erts_prim_file_pid)) { + ASSERT(!"bad pid passed to prim_file_nif"); + } + + am_erts_prim_file = enif_make_atom(env, "erts_prim_file"); + am_close = enif_make_atom(env, "close"); + am_ok = enif_make_atom(env, "ok"); am_error = enif_make_atom(env, "error"); am_continue = enif_make_atom(env, "continue"); @@ -239,7 +257,7 @@ static int load(ErlNifEnv *env, void** priv_data, ERL_NIF_TERM load_info) am_eof = enif_make_atom(env, "eof"); callbacks.down = owner_death_callback; - callbacks.dtor = gc_callback; + callbacks.dtor = NULL; callbacks.stop = NULL; efile_resource_type = enif_open_resource_type_x(env, "efile", &callbacks, @@ -305,8 +323,10 @@ static ERL_NIF_TERM file_handle_wrapper(file_op_impl_t operation, ErlNifEnv *env /* This is the only point where a change from CLOSE_PENDING is * possible, and we're running synchronously, so we can't race with * anything else here. */ + posix_errno_t ignored; + erts_atomic32_set_acqb(&d->state, EFILE_STATE_CLOSED); - efile_close(d); + efile_close(d, &ignored); } } else { /* CLOSE_PENDING should be impossible at this point since it requires @@ -319,6 +339,24 @@ static ERL_NIF_TERM file_handle_wrapper(file_op_impl_t operation, ErlNifEnv *env return result; } +/* This is a special close operation used by the erts_prim_file process for + * cleaning up orphaned files. It differs from the ordinary close_nif in that + * it only works for files that have already entered the CLOSED state. */ +static ERL_NIF_TERM delayed_close_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { + posix_errno_t ignored; + efile_data_t *d; + + ASSERT(argc == 1); + if(!get_file_data(env, argv[0], &d)) { + return enif_make_badarg(env); + } + + ASSERT(erts_atomic32_read_acqb(&d->state) == EFILE_STATE_CLOSED); + efile_close(d, &ignored); + + return am_ok; +} + static void owner_death_callback(ErlNifEnv* env, void* obj, ErlNifPid* pid, ErlNifMonitor* mon) { efile_data_t *d = (efile_data_t*)obj; @@ -334,8 +372,24 @@ static void owner_death_callback(ErlNifEnv* env, void* obj, ErlNifPid* pid, ErlN switch(previous_state) { case EFILE_STATE_IDLE: - efile_close(d); - return; + { + /* We cannot close the file here as that could block a normal + * scheduler, so we tell erts_prim_file to do it for us. + * + * This can in turn become a bottleneck (especially in cases + * like NFS failure), but it's less problematic than blocking + * thread progress. */ + ERL_NIF_TERM message, file_ref; + + file_ref = enif_make_resource(env, d); + message = enif_make_tuple2(env, am_close, file_ref); + + if(!enif_send(env, &erts_prim_file_pid, NULL, message)) { + ERTS_INTERNAL_ERROR("Failed to defer prim_file close."); + } + + return; + } case EFILE_STATE_CLOSE_PENDING: case EFILE_STATE_CLOSED: /* We're either already closed or managed to mark ourselves for @@ -352,24 +406,6 @@ static void owner_death_callback(ErlNifEnv* env, void* obj, ErlNifPid* pid, ErlN } } -static void gc_callback(ErlNifEnv *env, void* data) { - efile_data_t *d = (efile_data_t*)data; - - enum efile_state_t previous_state; - - (void)env; - - previous_state = erts_atomic32_cmpxchg_acqb(&d->state, - EFILE_STATE_CLOSED, EFILE_STATE_IDLE); - - ASSERT(previous_state != EFILE_STATE_CLOSE_PENDING && - previous_state != EFILE_STATE_BUSY); - - if(previous_state == EFILE_STATE_IDLE) { - efile_close(d); - } -} - static ERL_NIF_TERM efile_filetype_to_atom(enum efile_filetype_t type) { switch(type) { case EFILE_FILETYPE_DEVICE: return am_device; @@ -455,38 +491,62 @@ static ERL_NIF_TERM open_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[] return posix_error_to_tuple(env, posix_errno); } - result = enif_make_resource(env, d); - enif_release_resource(d); - enif_self(env, &controlling_process); if(enif_monitor_process(env, d, &controlling_process, &d->monitor)) { + /* We need to close the file manually as we haven't registered a + * destructor. */ + posix_errno_t ignored; + + erts_atomic32_set_acqb(&d->state, EFILE_STATE_CLOSED); + efile_close(d, &ignored); + return posix_error_to_tuple(env, EINVAL); } + /* Note that we do not call enif_release_resource at this point. While it's + * normally safe to leave resource management to the GC, efile_close is a + * blocking operation which must not be done in the GC callback, and we + * can't defer it as the resource is gone as soon as it returns. + * + * We instead keep the resource alive until efile_close is called, after + * which it's safe to leave things to the GC. If the controlling process + * were to die before the user had a chance to close their file, the above + * monitor will tell the erts_prim_file process to close it for them. */ + result = enif_make_resource(env, d); + return enif_make_tuple2(env, am_ok, result); } -static ERL_NIF_TERM close_nif_impl(efile_data_t *d, ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { +static ERL_NIF_TERM close_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { enum efile_state_t previous_state; + efile_data_t *d; - ASSERT(argc == 0); + ASSERT(argc == 1); + if(!get_file_data(env, argv[0], &d)) { + return enif_make_badarg(env); + } previous_state = erts_atomic32_cmpxchg_acqb(&d->state, - EFILE_STATE_CLOSED, EFILE_STATE_BUSY); + EFILE_STATE_CLOSED, EFILE_STATE_IDLE); - ASSERT(previous_state == EFILE_STATE_CLOSE_PENDING || - previous_state == EFILE_STATE_BUSY); + if(previous_state == EFILE_STATE_IDLE) { + posix_errno_t error; - if(previous_state == EFILE_STATE_BUSY) { enif_demonitor_process(env, d, &d->monitor); - if(!efile_close(d)) { - return posix_error_to_tuple(env, d->posix_errno); + if(!efile_close(d, &error)) { + return posix_error_to_tuple(env, error); } - } - return am_ok; + return am_ok; + } else { + /* CLOSE_PENDING should be impossible at this point since it requires + * a transition from BUSY; the only valid state here is CLOSED. */ + ASSERT(previous_state == EFILE_STATE_CLOSED); + + return posix_error_to_tuple(env, EINVAL); + } } static ERL_NIF_TERM read_nif_impl(efile_data_t *d, ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { @@ -1174,7 +1234,7 @@ static posix_errno_t read_file(efile_data_t *d, size_t size, ErlNifBinary *resul } static ERL_NIF_TERM read_file_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { - posix_errno_t posix_errno; + posix_errno_t posix_errno, ignored; efile_fileinfo_t info = {0}; efile_path_t path; @@ -1193,7 +1253,9 @@ static ERL_NIF_TERM read_file_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM a } posix_errno = read_file(d, info.size, &result); - enif_release_resource(d); + + erts_atomic32_set_acqb(&d->state, EFILE_STATE_CLOSED); + efile_close(d, &ignored); if(posix_errno) { return posix_error_to_tuple(env, posix_errno); diff --git a/erts/emulator/nifs/common/prim_file_nif.h b/erts/emulator/nifs/common/prim_file_nif.h index 099c06c48c..b2e30c59dd 100644 --- a/erts/emulator/nifs/common/prim_file_nif.h +++ b/erts/emulator/nifs/common/prim_file_nif.h @@ -159,8 +159,11 @@ posix_errno_t efile_open(const efile_path_t *path, enum efile_modes_t modes, ErlNifResourceType *nif_type, efile_data_t **d); /** @brief Closes a file. The file must have entered the CLOSED state prior to - * calling this to prevent double close. */ -int efile_close(efile_data_t *d); + * calling this to prevent double close. + * + * Note that the file is completely invalid after this point, so the error code + * is provided in \c error rather than d->posix_errno. */ +int efile_close(efile_data_t *d, posix_errno_t *error); /* **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** */ diff --git a/erts/emulator/nifs/unix/unix_prim_file.c b/erts/emulator/nifs/unix/unix_prim_file.c index dea73db18a..169b193993 100644 --- a/erts/emulator/nifs/unix/unix_prim_file.c +++ b/erts/emulator/nifs/unix/unix_prim_file.c @@ -202,21 +202,24 @@ posix_errno_t efile_open(const efile_path_t *path, enum efile_modes_t modes, return errno; } -int efile_close(efile_data_t *d) { +int efile_close(efile_data_t *d, posix_errno_t *error) { efile_unix_t *u = (efile_unix_t*)d; int fd; + ASSERT(enif_thread_type() == ERL_NIF_THR_DIRTY_IO_SCHEDULER); ASSERT(erts_atomic32_read_nob(&d->state) == EFILE_STATE_CLOSED); ASSERT(u->fd != -1); fd = u->fd; u->fd = -1; + enif_release_resource(d); + /* close(2) either always closes (*BSD, Linux) or leaves the fd in an * undefined state (POSIX 2008, Solaris), so we must not retry on EINTR. */ if(close(fd) < 0) { - u->common.posix_errno = errno; + *error = errno; return 0; } diff --git a/erts/emulator/nifs/win32/win_prim_file.c b/erts/emulator/nifs/win32/win_prim_file.c index 602a282dd1..d0aa70542f 100644 --- a/erts/emulator/nifs/win32/win_prim_file.c +++ b/erts/emulator/nifs/win32/win_prim_file.c @@ -466,18 +466,21 @@ posix_errno_t efile_open(const efile_path_t *path, enum efile_modes_t modes, } } -int efile_close(efile_data_t *d) { +int efile_close(efile_data_t *d, posix_errno_t *error) { efile_win_t *w = (efile_win_t*)d; HANDLE handle; + ASSERT(enif_thread_type() == ERL_NIF_THR_DIRTY_IO_SCHEDULER); ASSERT(erts_atomic32_read_nob(&d->state) == EFILE_STATE_CLOSED); ASSERT(w->handle != INVALID_HANDLE_VALUE); handle = w->handle; w->handle = INVALID_HANDLE_VALUE; + enif_release_resource(d); + if(!CloseHandle(handle)) { - w->common.posix_errno = windows_to_posix_errno(GetLastError()); + *error = windows_to_posix_errno(GetLastError()); return 0; } diff --git a/erts/emulator/test/Makefile b/erts/emulator/test/Makefile index d201ea6ee1..6a064ec8d4 100644 --- a/erts/emulator/test/Makefile +++ b/erts/emulator/test/Makefile @@ -33,6 +33,7 @@ MODULES= \ after_SUITE \ alloc_SUITE \ async_ports_SUITE \ + atomics_SUITE \ beam_SUITE \ beam_literals_SUITE \ bif_SUITE \ @@ -50,6 +51,7 @@ MODULES= \ call_trace_SUITE \ code_SUITE \ code_parallel_load_SUITE \ + counters_SUITE \ crypto_SUITE \ ddll_SUITE \ decode_packet_SUITE \ diff --git a/erts/emulator/test/atomics_SUITE.erl b/erts/emulator/test/atomics_SUITE.erl new file mode 100644 index 0000000000..8c42354770 --- /dev/null +++ b/erts/emulator/test/atomics_SUITE.erl @@ -0,0 +1,147 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% +-module(atomics_SUITE). + +-include_lib("common_test/include/ct.hrl"). + +-compile(export_all). + +suite() -> [{ct_hooks,[ts_install_cth]}]. + +all() -> + [signed, unsigned, bad, signed_limits, unsigned_limits]. + +signed(Config) when is_list(Config) -> + Size = 10, + Ref = atomics:new(Size,[]), + #{size:=Size, memory:=Memory} = atomics:info(Ref), + {_,true} = {Memory, Memory > Size*8}, + {_,true} = {Memory, Memory < Size*max_atomic_sz() + 100}, + [signed_do(Ref, Ix) || Ix <- lists:seq(1, Size)], + ok. + +signed_do(Ref, Ix) -> + 0 = atomics:get(Ref, Ix), + ok = atomics:put(Ref, Ix, 3), + ok = atomics:add(Ref, Ix, 14), + 17 = atomics:get(Ref, Ix), + 20 = atomics:add_get(Ref, Ix, 3), + -3 = atomics:add_get(Ref, Ix, -23), + 17 = atomics:add_get(Ref, Ix, 20), + ok = atomics:sub(Ref, Ix, 4), + 13 = atomics:get(Ref, Ix), + -7 = atomics:sub_get(Ref, Ix, 20), + 3 = atomics:sub_get(Ref, Ix, -10), + 3 = atomics:exchange(Ref, Ix, 666), + ok = atomics:compare_exchange(Ref, Ix, 666, 777), + 777 = atomics:compare_exchange(Ref, Ix, 666, -666), + ok. + +unsigned(Config) when is_list(Config) -> + Size = 10, + Ref = atomics:new(Size,[{signed, false}]), + #{size:=Size, memory:=Memory} = atomics:info(Ref), + true = Memory > Size*8, + true = Memory < Size*max_atomic_sz() + 100, + [unsigned_do(Ref, Ix) || Ix <- lists:seq(1, Size)], + ok. + +unsigned_do(Ref, Ix) -> + 0 = atomics:get(Ref, Ix), + ok = atomics:put(Ref, Ix, 3), + ok = atomics:add(Ref, Ix, 14), + 17 = atomics:get(Ref, Ix), + 20 = atomics:add_get(Ref, Ix, 3), + ok = atomics:sub(Ref, Ix, 7), + 13 = atomics:get(Ref, Ix), + 3 = atomics:sub_get(Ref, Ix, 10), + 3 = atomics:exchange(Ref, Ix, 666), + ok = atomics:compare_exchange(Ref, Ix, 666, 777), + 777 = atomics:compare_exchange(Ref, Ix, 666, 888), + ok. + +bad(Config) when is_list(Config) -> + {'EXIT',{badarg,_}} = (catch atomics:new(0,[])), + {'EXIT',{badarg,_}} = (catch atomics:new(10,[bad])), + {'EXIT',{badarg,_}} = (catch atomics:new(10,[{signed,bad}])), + {'EXIT',{badarg,_}} = (catch atomics:new(10,[{signed,true}, bad])), + {'EXIT',{badarg,_}} = (catch atomics:new(10,[{signed,false} | bad])), + Ref = atomics:new(10,[]), + {'EXIT',{badarg,_}} = (catch atomics:get(1742, 7)), + {'EXIT',{badarg,_}} = (catch atomics:get(make_ref(), 7)), + {'EXIT',{badarg,_}} = (catch atomics:get(Ref, -1)), + {'EXIT',{badarg,_}} = (catch atomics:get(Ref, 0)), + {'EXIT',{badarg,_}} = (catch atomics:get(Ref, 11)), + {'EXIT',{badarg,_}} = (catch atomics:get(Ref, 7.0)), + ok. + + +signed_limits(Config) when is_list(Config) -> + Bits = 64, + Max = (1 bsl (Bits-1)) - 1, + Min = -(1 bsl (Bits-1)), + + Ref = atomics:new(1,[{signed, true}]), + #{max:=Max, min:=Min} = atomics:info(Ref), + 0 = atomics:get(Ref, 1), + ok = atomics:add(Ref, 1, Max), + Min = atomics:add_get(Ref, 1, 1), + Max = atomics:sub_get(Ref, 1, 1), + + IncrMax = (Max bsl 1) bor 1, + ok = atomics:put(Ref, 1, 0), + ok = atomics:add(Ref, 1, IncrMax), + -1 = atomics:get(Ref, 1), + {'EXIT',{badarg,_}} = (catch atomics:add(Ref, 1, IncrMax+1)), + {'EXIT',{badarg,_}} = (catch atomics:add(Ref, 1, Min-1)), + + ok. + +unsigned_limits(Config) when is_list(Config) -> + Bits = 64, + Max = (1 bsl Bits) - 1, + Min = 0, + + Ref = atomics:new(1,[{signed,false}]), + #{max:=Max, min:=Min} = atomics:info(Ref), + 0 = atomics:get(Ref, 1), + ok = atomics:add(Ref, 1, Max), + Min = atomics:add_get(Ref, 1, 1), + Max = atomics:sub_get(Ref, 1, 1), + + {'EXIT',{badarg,_}} = (catch atomics:add(Ref, 1, Max+1)), + IncrMin = -(1 bsl (Bits-1)), + ok = atomics:put(Ref, 1, -IncrMin), + ok = atomics:add(Ref, 1, IncrMin), + 0 = atomics:get(Ref, 1), + {'EXIT',{badarg,_}} = (catch atomics:add(Ref, 1, IncrMin-1)), + + ok. + +max_atomic_sz() -> + case erlang:system_info({wordsize, external}) of + 4 -> 16; + 8 -> + EI = erlang:system_info(ethread_info), + case lists:keyfind("64-bit native atomics", 1, EI) of + {_, "no", _} -> 16; + _ -> 8 + end + end. diff --git a/erts/emulator/test/counters_SUITE.erl b/erts/emulator/test/counters_SUITE.erl new file mode 100644 index 0000000000..7de164096b --- /dev/null +++ b/erts/emulator/test/counters_SUITE.erl @@ -0,0 +1,112 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% +-module(counters_SUITE). + +-include_lib("common_test/include/ct.hrl"). + +-compile(export_all). + +suite() -> [{ct_hooks,[ts_install_cth]}]. + +all() -> + [basic, bad, limits]. + +basic(Config) when is_list(Config) -> + Size = 10, + [begin + Ref = counters:new(Size,[Type]), + #{size:=Size, memory:=Memory} = counters:info(Ref), + check_memory(Type, Memory, Size), + [basic_do(Ref, Ix) || Ix <- lists:seq(1, Size)] + end + || Type <- [atomics, write_concurrency]], + ok. + +basic_do(Ref, Ix) -> + 0 = counters:get(Ref, Ix), + ok = counters:add(Ref, Ix, 3), + 3 = counters:get(Ref, Ix), + ok = counters:add(Ref, Ix, 14), + 17 = counters:get(Ref, Ix), + ok = counters:add(Ref, Ix, -20), + -3 = counters:get(Ref, Ix), + ok = counters:add(Ref, Ix, 100), + 97 = counters:get(Ref, Ix), + ok = counters:sub(Ref, Ix, 20), + 77 = counters:get(Ref, Ix), + ok = counters:sub(Ref, Ix, -10), + 87 = counters:get(Ref, Ix), + ok. + +check_memory(atomics, Memory, Size) -> + {_,true} = {Memory, Memory > Size*8}, + {_,true} = {Memory, Memory < Size*max_atomic_sz() + 100}; +check_memory(write_concurrency, Memory, Size) -> + NScheds = erlang:system_info(schedulers), + {_,true} = {Memory, Memory > NScheds*Size*8}, + {_,true} = {Memory, Memory < NScheds*(Size+7)*max_atomic_sz() + 100}. + +max_atomic_sz() -> + case erlang:system_info({wordsize, external}) of + 4 -> 16; + 8 -> + EI = erlang:system_info(ethread_info), + case lists:keyfind("64-bit native atomics", 1, EI) of + {_, "no", _} -> 16; + _ -> 8 + end + end. + +bad(Config) when is_list(Config) -> + {'EXIT',{badarg,_}} = (catch counters:new(0,[])), + {'EXIT',{badarg,_}} = (catch counters:new(10,[bad])), + {'EXIT',{badarg,_}} = (catch counters:new(10,[atomic, bad])), + {'EXIT',{badarg,_}} = (catch counters:new(10,[write_concurrency | bad])), + Ref = counters:new(10,[]), + {'EXIT',{badarg,_}} = (catch counters:get(1742, 7)), + {'EXIT',{badarg,_}} = (catch counters:get(make_ref(), 7)), + {'EXIT',{badarg,_}} = (catch counters:get(Ref, -1)), + {'EXIT',{badarg,_}} = (catch counters:get(Ref, 0)), + {'EXIT',{badarg,_}} = (catch counters:get(Ref, 11)), + {'EXIT',{badarg,_}} = (catch counters:get(Ref, 7.0)), + ok. + + +limits(Config) when is_list(Config) -> + Bits = 64, + Max = (1 bsl (Bits-1)) - 1, + Min = -(1 bsl (Bits-1)), + + Ref = counters:new(1,[]), + 0 = counters:get(Ref, 1), + ok = counters:add(Ref, 1, Max), + ok = counters:add(Ref, 1, 1), + Min = counters:get(Ref, 1), + ok = counters:sub(Ref, 1, 1), + Max = counters:get(Ref, 1), + + IncrMax = (Max bsl 1) bor 1, + ok = counters:sub(Ref, 1, counters:get(Ref, 1)), + ok = counters:add(Ref, 1, IncrMax), + -1 = counters:get(Ref, 1), + {'EXIT',{badarg,_}} = (catch counters:add(Ref, 1, IncrMax+1)), + {'EXIT',{badarg,_}} = (catch counters:add(Ref, 1, Min-1)), + + ok. diff --git a/erts/epmd/src/epmd.c b/erts/epmd/src/epmd.c index 44e997e609..8313678b5d 100644 --- a/erts/epmd/src/epmd.c +++ b/erts/epmd/src/epmd.c @@ -437,6 +437,11 @@ static void usage(EpmdVars *g) fprintf(stderr, " epmd -kill even if there " "are registered nodes.\n"); fprintf(stderr, " Also allows forced unregister (epmd -stop).\n"); +#ifdef HAVE_SYSTEMD_DAEMON + fprintf(stderr, " -systemd\n"); + fprintf(stderr, " Wait for socket from systemd. The option makes sense\n"); + fprintf(stderr, " when started from .socket unit.\n"); +#endif /* HAVE_SYSTEMD_DAEMON */ fprintf(stderr, "\nDbgExtra options\n"); fprintf(stderr, " -packet_timeout Seconds\n"); fprintf(stderr, " Set the number of seconds a connection can be\n"); @@ -462,11 +467,6 @@ static void usage(EpmdVars *g) fprintf(stderr, " Forcibly unregisters a name with epmd\n"); fprintf(stderr, " (only allowed if -relaxed_command_check was given when \n"); fprintf(stderr, " epmd was started).\n"); -#ifdef HAVE_SYSTEMD_DAEMON - fprintf(stderr, " -systemd\n"); - fprintf(stderr, " Wait for socket from systemd. The option makes sense\n"); - fprintf(stderr, " when started from .socket unit.\n"); -#endif /* HAVE_SYSTEMD_DAEMON */ epmd_cleanup_exit(g,1); } diff --git a/erts/include/internal/gcc/ethr_dw_atomic.h b/erts/include/internal/gcc/ethr_dw_atomic.h index 47158b7295..dd116c81ce 100644 --- a/erts/include/internal/gcc/ethr_dw_atomic.h +++ b/erts/include/internal/gcc/ethr_dw_atomic.h @@ -78,7 +78,7 @@ typedef volatile ETHR_NATIVE_SU_DW_SINT_T * ethr_native_dw_ptr_t; * runtime. We, therefore, need an extra word allocated. */ #define ETHR_DW_NATMC_MEM__(VAR) \ - (&var->c[(int) ((ethr_uint_t) &(VAR)->c[0]) & ETHR_DW_NATMC_ALIGN_MASK__]) + (&(VAR)->c[(int) ((ethr_uint_t) &(VAR)->c[0]) & ETHR_DW_NATMC_ALIGN_MASK__]) typedef union { volatile ETHR_NATIVE_SU_DW_SINT_T dw_sint; volatile ethr_sint_t sint[3]; diff --git a/erts/include/internal/i386/ethr_dw_atomic.h b/erts/include/internal/i386/ethr_dw_atomic.h index 91acdb0483..3c47da9758 100644 --- a/erts/include/internal/i386/ethr_dw_atomic.h +++ b/erts/include/internal/i386/ethr_dw_atomic.h @@ -73,7 +73,7 @@ typedef volatile ethr_native_sint128_t__ * ethr_native_dw_ptr_t; * runtime. We, therefore, need an extra word allocated. */ #define ETHR_DW_NATMC_MEM__(VAR) \ - (&var->c[(int) ((ethr_uint_t) &(VAR)->c[0]) & ETHR_DW_NATMC_ALIGN_MASK__]) + (&(VAR)->c[(int) ((ethr_uint_t) &(VAR)->c[0]) & ETHR_DW_NATMC_ALIGN_MASK__]) typedef union { #ifdef ETHR_NATIVE_SU_DW_SINT_T volatile ETHR_NATIVE_SU_DW_SINT_T dw_sint; diff --git a/erts/include/internal/win/ethr_dw_atomic.h b/erts/include/internal/win/ethr_dw_atomic.h index a6b26ab7bb..f7a1900a82 100644 --- a/erts/include/internal/win/ethr_dw_atomic.h +++ b/erts/include/internal/win/ethr_dw_atomic.h @@ -80,7 +80,7 @@ typedef volatile __int64 * ethr_native_dw_ptr_t; * runtime. We, therefore, need an extra word allocated. */ #define ETHR_DW_NATMC_MEM__(VAR) \ - (&var->c[(int) ((ethr_uint_t) &(VAR)->c[0]) & ETHR_DW_NATMC_ALIGN_MASK__]) + (&(VAR)->c[(int) ((ethr_uint_t) &(VAR)->c[0]) & ETHR_DW_NATMC_ALIGN_MASK__]) typedef union { #ifdef ETHR_NATIVE_SU_DW_SINT_T volatile ETHR_NATIVE_SU_DW_SINT_T dw_sint; diff --git a/erts/preloaded/ebin/atomics.beam b/erts/preloaded/ebin/atomics.beam Binary files differnew file mode 100644 index 0000000000..1de97fa668 --- /dev/null +++ b/erts/preloaded/ebin/atomics.beam diff --git a/erts/preloaded/ebin/counters.beam b/erts/preloaded/ebin/counters.beam Binary files differnew file mode 100644 index 0000000000..caaa6167e1 --- /dev/null +++ b/erts/preloaded/ebin/counters.beam diff --git a/erts/preloaded/ebin/erts_internal.beam b/erts/preloaded/ebin/erts_internal.beam Binary files differindex a8a0dfcd62..2b98e87843 100644 --- a/erts/preloaded/ebin/erts_internal.beam +++ b/erts/preloaded/ebin/erts_internal.beam diff --git a/erts/preloaded/ebin/prim_file.beam b/erts/preloaded/ebin/prim_file.beam Binary files differindex 6a9528d1e2..37a1a563a9 100644 --- a/erts/preloaded/ebin/prim_file.beam +++ b/erts/preloaded/ebin/prim_file.beam diff --git a/erts/preloaded/src/Makefile b/erts/preloaded/src/Makefile index 7564086b1b..c655bff72e 100644 --- a/erts/preloaded/src/Makefile +++ b/erts/preloaded/src/Makefile @@ -48,6 +48,8 @@ PRE_LOADED_ERL_MODULES = \ erl_tracer \ erts_literal_area_collector \ erts_dirty_process_signal_handler \ + atomics \ + counters \ persistent_term PRE_LOADED_BEAM_MODULES = \ diff --git a/erts/preloaded/src/atomics.erl b/erts/preloaded/src/atomics.erl new file mode 100644 index 0000000000..d1fe5e65cf --- /dev/null +++ b/erts/preloaded/src/atomics.erl @@ -0,0 +1,119 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +%% Purpose : Main atomics API module. + +-module(atomics). + +-export([new/2, + put/3, get/2, + add/3, add_get/3, + sub/3, sub_get/3, + exchange/3, compare_exchange/4, + info/1]). + +-export_type([atomics_ref/0]). + +-opaque atomics_ref() :: reference(). + +-define(OPT_SIGNED, (1 bsl 0)). +-define(OPT_DEFAULT, ?OPT_SIGNED). + +-spec new(Arity, Opts) -> atomics_ref() when + Arity :: pos_integer(), + Opts :: [Opt], + Opt :: {signed, boolean()}. +new(Arity, Opts) -> + erts_internal:atomics_new(Arity, encode_opts(Opts, ?OPT_DEFAULT)). + +encode_opts([{signed, true}|T], Acc) -> + encode_opts(T, Acc bor ?OPT_SIGNED); +encode_opts([{signed, false}|T], Acc) -> + encode_opts(T, Acc band (bnot ?OPT_SIGNED)); +encode_opts([], Acc) -> + Acc; +encode_opts(_, _) -> + erlang:error(badarg). + +-spec put(Ref, Ix, Value) -> ok when + Ref :: atomics_ref(), + Ix :: integer(), + Value :: integer(). +put(_Ref, _Ix, _Value) -> + erlang:nif_error(undef). + +-spec get(Ref, Ix) -> integer() when + Ref :: atomics_ref(), + Ix :: integer(). +get(_Ref, _Ix) -> + erlang:nif_error(undef). + +-spec add(Ref, Ix, Incr) -> ok when + Ref :: atomics_ref(), + Ix :: integer(), + Incr :: integer(). +add(_Ref, _Ix, _Incr) -> + erlang:nif_error(undef). + +-spec add_get(Ref, Ix, Incr) -> integer() when + Ref :: atomics_ref(), + Ix :: integer(), + Incr :: integer(). +add_get(_Ref, _Ix, _Incr) -> + erlang:nif_error(undef). + +-spec sub(Ref, Ix, Decr) -> ok when + Ref :: atomics_ref(), + Ix :: integer(), + Decr :: integer(). +sub(Ref, Ix, Decr) -> + ?MODULE:add(Ref, Ix, -Decr). + +-spec sub_get(Ref, Ix, Decr) -> integer() when + Ref :: atomics_ref(), + Ix :: integer(), + Decr :: integer(). +sub_get(Ref, Ix, Decr) -> + ?MODULE:add_get(Ref, Ix, -Decr). + +-spec exchange(Ref, Ix, Desired) -> integer() when + Ref :: atomics_ref(), + Ix :: integer(), + Desired :: integer(). +exchange(_Ref, _Ix, _Desired) -> + erlang:nif_error(undef). + +-spec compare_exchange(Ref, Ix, Expected, Desired) -> ok | integer() when + Ref :: atomics_ref(), + Ix :: integer(), + Expected :: integer(), + Desired :: integer(). +compare_exchange(_Ref, _Ix, _Expected, _Desired) -> + erlang:nif_error(undef). + +-spec info(Ref) -> Info when + Ref :: atomics_ref(), + Info :: #{'size':=Size,'max':=Max,'min':=Min,'memory':=Memory}, + Size :: non_neg_integer(), + Max :: integer(), + Min :: integer(), + Memory :: non_neg_integer(). +info(_Ref) -> + erlang:nif_error(undef). diff --git a/erts/preloaded/src/counters.erl b/erts/preloaded/src/counters.erl new file mode 100644 index 0000000000..67354f648d --- /dev/null +++ b/erts/preloaded/src/counters.erl @@ -0,0 +1,90 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +%% Purpose : Main atomics API module. + +-module(counters). + +-export([new/2, + get/2, + add/3, + sub/3, + info/1]). + +-export_type([counters_ref/0]). + +-opaque counters_ref() :: {atomics, reference()} | {write_concurrency, reference()}. + +-spec new(Size, Opts) -> counters_ref() when + Size :: pos_integer(), + Opts :: [Opt], + Opt :: atomics | write_concurrency. +new(Size, [atomics]) -> + {atomics, atomics:new(Size, [{signed, true}])}; +new(Size, [write_concurrency]) -> + {write_concurrency, erts_internal:counters_new(Size)}; +new(Size, []) -> + new(Size, [atomics]); +new(_, _) -> + erlang:error(badarg). + +-spec get(Ref, Ix) -> integer() when + Ref :: counters_ref(), + Ix :: integer(). +get({atomics,Ref}, Ix) -> + atomics:get(Ref, Ix); +get({write_concurrency, Ref}, Ix) -> + erts_internal:counters_get(Ref, Ix); +get(_, _) -> + erlang:error(badarg). + + + +-spec add(Ref, Ix, Incr) -> ok when + Ref :: counters_ref(), + Ix :: integer(), + Incr :: integer(). +add({atomics, Ref}, Ix, Incr) -> + atomics:add(Ref, Ix, Incr); +add({write_concurrency, Ref}, Ix, Incr) -> + erts_internal:counters_add(Ref, Ix, Incr); +add(_, _, _) -> + erlang:error(badarg). + + +-spec sub(Ref, Ix, Decr) -> ok when + Ref :: counters_ref(), + Ix :: integer(), + Decr :: integer(). +sub(Ref, Ix, Decr) -> + add(Ref, Ix, -Decr). + +-spec info(Ref) -> Info when + Ref :: counters_ref(), + Info :: #{'size':=Size, 'memory':=Memory}, + Size :: non_neg_integer(), + Memory :: non_neg_integer(). +info({atomics, Ref}) -> + atomics:info(Ref); +info({write_concurrency, Ref}) -> + erts_internal:counters_info(Ref); +info(_) -> + erlang:error(badarg). + diff --git a/erts/preloaded/src/erts.app.src b/erts/preloaded/src/erts.app.src index dc97b86d8c..13fbbc7ff7 100644 --- a/erts/preloaded/src/erts.app.src +++ b/erts/preloaded/src/erts.app.src @@ -33,12 +33,14 @@ prim_file, prim_inet, prim_zip, + atomics, + counters, zlib ]}, {registered, []}, {applications, []}, {env, []}, - {runtime_dependencies, ["stdlib-3.5", "kernel-6.1", "sasl-3.0.1"]} + {runtime_dependencies, ["stdlib-3.5", "kernel-6.1", "sasl-@OTP-13468@"]} ]}. %% vim: ft=erlang diff --git a/erts/preloaded/src/erts_internal.erl b/erts/preloaded/src/erts_internal.erl index 63b786a473..d491a505c6 100644 --- a/erts/preloaded/src/erts_internal.erl +++ b/erts/preloaded/src/erts_internal.erl @@ -92,6 +92,11 @@ -export([erase_persistent_terms/0]). +-export([atomics_new/2]). + +-export([counters_new/1, counters_get/2, counters_add/3, + counters_info/1]). + %% %% Await result of send to port %% @@ -697,3 +702,23 @@ create_dist_channel(_Node, _DistCtrlr, _Flags, _Ver) -> -spec erase_persistent_terms() -> 'ok'. erase_persistent_terms() -> erlang:nif_error(undefined). + +-spec atomics_new(pos_integer(), pos_integer()) -> reference(). +atomics_new(_Arity, _EncOpts) -> + erlang:nif_error(undef). + +-spec counters_new(pos_integer()) -> reference(). +counters_new(_Size) -> + erlang:nif_error(undef). + +-spec counters_get(reference(), pos_integer()) -> integer(). +counters_get(_Ref, _Ix) -> + erlang:nif_error(undef). + +-spec counters_add(reference(), pos_integer(), integer()) -> ok. +counters_add(_Ref, _Ix, _Incr) -> + erlang:nif_error(undef). + +-spec counters_info(reference()) -> #{}. +counters_info(_Ref) -> + erlang:nif_error(undef). diff --git a/erts/preloaded/src/prim_file.erl b/erts/preloaded/src/prim_file.erl index 6d85868183..5fc22bc582 100644 --- a/erts/preloaded/src/prim_file.erl +++ b/erts/preloaded/src/prim_file.erl @@ -83,6 +83,15 @@ internal_normalize_utf8(_) -> is_translatable(_) -> erlang:nif_error(undefined). +%% This is a janitor process used to close files whose controlling process has +%% died. The emulator will be torn down if this is killed. +delayed_close_loop() -> + receive + {close, FRef} when is_reference(FRef) -> delayed_close_nif(FRef); + _ -> ok + end, + delayed_close_loop(). + %% %% Returns {error, Reason} | {ok, BytesCopied} @@ -95,7 +104,12 @@ copy(#file_descriptor{module = ?MODULE} = Source, file:copy_opened(Source, Dest, Length). on_load() -> - ok = erlang:load_nif(atom_to_list(?MODULE), 0). + Pid = spawn(fun() -> + process_flag(trap_exit, true), + delayed_close_loop() + end), + true = register(erts_prim_file, Pid), + ok = erlang:load_nif(atom_to_list(?MODULE), Pid). open(Name, Modes) -> %% The try/catch pattern seen here is used throughout the file to adhere to @@ -482,6 +496,8 @@ truncate_nif(_FileRef) -> erlang:nif_error(undef). get_handle_nif(_FileRef) -> erlang:nif_error(undef). +delayed_close_nif(_FileRef) -> + erlang:nif_error(undef). %% %% Quality-of-life helpers diff --git a/erts/vsn.mk b/erts/vsn.mk index 01c19aff2f..241d5a3b85 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -18,7 +18,7 @@ # %CopyrightEnd% # -VSN = 10.1.1 +VSN = 10.1.2 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/compiler/doc/src/notes.xml b/lib/compiler/doc/src/notes.xml index 5024310788..e0e5bc832b 100644 --- a/lib/compiler/doc/src/notes.xml +++ b/lib/compiler/doc/src/notes.xml @@ -32,6 +32,21 @@ <p>This document describes the changes made to the Compiler application.</p> +<section><title>Compiler 7.2.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a bug where incorrect code was generated + following a binary match guard.</p> + <p> + Own Id: OTP-15353 Aux Id: ERL-753 </p> + </item> + </list> + </section> + +</section> + <section><title>Compiler 7.2.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/compiler/src/beam_ssa_dead.erl b/lib/compiler/src/beam_ssa_dead.erl index c20652580d..067d9a6741 100644 --- a/lib/compiler/src/beam_ssa_dead.erl +++ b/lib/compiler/src/beam_ssa_dead.erl @@ -135,7 +135,8 @@ shortcut_terminator(Last, _Is, _Bs, _St) -> Last. shortcut_switch([{Lit,L0}|T], Bool, Bs, St0) -> - St = St0#st{rel_op=normalize_op({bif,'=:='}, [Bool,Lit])}, + RelOp = {'=:=',Bool,Lit}, + St = St0#st{rel_op=RelOp}, #b_br{bool=#b_literal{val=true},succ=L} = shortcut(L0, bind_var(Bool, Lit, Bs), St#st{target=one_way}), [{Lit,L}|shortcut_switch(T, Bool, Bs, St0)]; @@ -388,41 +389,43 @@ eval_terminator(#b_switch{arg=Arg,fail=Fail,list=List}=Sw, Bs, St) -> %% Literal argument. Simplify to a `br`. beam_ssa:normalize(Sw#b_switch{arg=Val}); #b_var{} -> - case St of - #st{rel_op=none} -> - %% No previous relational operator is stored. - %% Give up. + %% Try optimizing the switch. + case eval_switch(List, Arg, St, Fail) of + none -> none; - #st{} -> - %% There is a previous relational operator stored. - %% Try optimizing the switch. - case eval_switch(List, Arg, St, Fail) of - none -> - none; - To when is_integer(To) -> - %% Either one of the values in the switch - %% matched a previous value in a '=:=' test, or - %% none of the values matched a previous test. - #b_br{bool=#b_literal{val=true},succ=To,fail=To} - end + To when is_integer(To) -> + %% Either one of the values in the switch + %% matched a previous value in a '=:=' test, or + %% none of the values matched a previous test. + #b_br{bool=#b_literal{val=true},succ=To,fail=To} end end; eval_terminator(#b_ret{}, _Bs, _St) -> none. -eval_switch([{Lit,Lbl}|T], Arg, St, Fail) -> - case eval_rel_op({bif,'=:='}, [Arg,Lit], St) of - none -> - %% This label could be reached. - eval_switch(T, Arg, St, none); - #b_literal{val=false} -> - %% This branch will never be taken. - eval_switch(T, Arg, St, Fail); - #b_literal{val=true} -> +eval_switch(List, Arg, #st{rel_op={_,Arg,_}=PrevOp}, Fail) -> + %% There is a previous relational operator testing the same variable. + %% Optimization may be possible. + eval_switch_1(List, Arg, PrevOp, Fail); +eval_switch(_, _, _, _) -> + %% There is either no previous relational operator, or it tests + %% a different variable. Nothing to optimize. + none. + +eval_switch_1([{Lit,Lbl}|T], Arg, PrevOp, Fail) -> + RelOp = {'=:=',Arg,Lit}, + case will_succeed(PrevOp, RelOp) of + yes -> %% Success. This branch will always be taken. - Lbl + Lbl; + no -> + %% This branch will never be taken. + eval_switch_1(T, Arg, PrevOp, Fail); + maybe -> + %% This label could be reached. + eval_switch_1(T, Arg, PrevOp, none) end; -eval_switch([], _Arg, _St, Fail) -> +eval_switch_1([], _Arg, _PrevOp, Fail) -> %% Fail is now either the failure label or 'none'. Fail. diff --git a/lib/compiler/src/beam_ssa_pre_codegen.erl b/lib/compiler/src/beam_ssa_pre_codegen.erl index 9175931375..c60c6da9ea 100644 --- a/lib/compiler/src/beam_ssa_pre_codegen.erl +++ b/lib/compiler/src/beam_ssa_pre_codegen.erl @@ -2211,7 +2211,13 @@ linear_scan(#st{intervals=Intervals0,res=Res}=St0) -> Free = init_free(maps:to_list(Res)), Intervals1 = [init_interval(Int, Res) || Int <- Intervals0], Intervals = sort(Intervals1), - IsReserved = fun (#i{reg=Reg}) -> Reg =/= none end, + IsReserved = fun(#i{reg=Reg}) -> + case Reg of + none -> false; + {prefer,{_,_}} -> false; + {_,_} -> true + end + end, {UnhandledRes,Unhandled} = partition(IsReserved, Intervals), L = #l{unhandled_res=UnhandledRes, unhandled_any=Unhandled,free=Free}, diff --git a/lib/compiler/src/beam_ssa_type.erl b/lib/compiler/src/beam_ssa_type.erl index 18e6e73a46..95fc3bb0e9 100644 --- a/lib/compiler/src/beam_ssa_type.erl +++ b/lib/compiler/src/beam_ssa_type.erl @@ -27,9 +27,10 @@ -define(UNICODE_INT, #t_integer{elements={0,16#10FFFF}}). --record(d, {ds :: #{beam_ssa:var_name():=beam_ssa:b_set()}, +-record(d, {ds :: #{beam_ssa:b_var():=beam_ssa:b_set()}, ls :: #{beam_ssa:label():=type_db()}, - sub :: #{beam_ssa:var_name():=beam_ssa:value()} + once :: cerl_sets:set(beam_ssa:b_var()), + sub :: #{beam_ssa:b_var():=beam_ssa:value()} }). -define(ATOM_SET_SIZE, 5). @@ -56,13 +57,15 @@ Block :: beam_ssa:b_blk(). opt(Linear, Args) -> + UsedOnce = used_once(Linear, Args), Ts = maps:from_list([{V,any} || #b_var{}=V <- Args]), FakeCall = #b_set{op=call,args=[#b_remote{mod=#b_literal{val=unknown}, name=#b_literal{val=unknown}, arity=0}]}, Defs = maps:from_list([{Var,FakeCall#b_set{dst=Var}} || #b_var{}=Var <- Args]), - D = #d{ds=Defs,ls=#{0=>Ts},sub=#{}}, + D = #d{ds=Defs,ls=#{0=>Ts,?BADARG_BLOCK=>#{}}, + once=UsedOnce,sub=#{}}, opt_1(Linear, D). opt_1([{L,Blk}|Bs], #d{ls=Ls}=D) -> @@ -425,16 +428,43 @@ opt_terminator(#b_ret{}=Ret, _Ts, _Ds) -> Ret. update_successors(#b_br{bool=#b_literal{val=true},succ=S}, Ts, D) -> update_successor(S, Ts, D); -update_successors(#b_br{bool=#b_var{}=Bool,succ=Succ,fail=Fail}, Ts, D0) -> - D = update_successor_bool(Bool, false, Fail, Ts, D0), - SuccTs = infer_types(Bool, Ts, D0), - update_successor_bool(Bool, true, Succ, SuccTs, D); -update_successors(#b_switch{arg=#b_var{}=V,fail=Fail,list=List}, Ts, D0) -> - D = update_successor(Fail, Ts, D0), - foldl(fun({Val,S}, A) -> - T = get_type(Val, Ts), - update_successor(S, Ts#{V=>T}, A) - end, D, List); +update_successors(#b_br{bool=#b_var{}=Bool,succ=Succ,fail=Fail}, Ts0, D0) -> + case cerl_sets:is_element(Bool, D0#d.once) of + true -> + %% This variable is defined in this block and is only + %% referenced by this br terminator. Therefore, there is + %% no need to include the type database passed on to the + %% successors of this block. + Ts = maps:remove(Bool, Ts0), + D = update_successor(Fail, Ts, D0), + SuccTs = infer_types(Bool, Ts, D0), + update_successor(Succ, SuccTs, D); + false -> + D = update_successor_bool(Bool, false, Fail, Ts0, D0), + SuccTs = infer_types(Bool, Ts0, D0), + update_successor_bool(Bool, true, Succ, SuccTs, D) + end; +update_successors(#b_switch{arg=#b_var{}=V,fail=Fail,list=List}, Ts0, D0) -> + case cerl_sets:is_element(V, D0#d.once) of + true -> + %% This variable is defined in this block and is only + %% referenced by this switch terminator. Therefore, there is + %% no need to include the type database passed on to the + %% successors of this block. + Ts = maps:remove(V, Ts0), + D = update_successor(Fail, Ts, D0), + F = fun({_Val,S}, A) -> + update_successor(S, Ts, A) + end, + foldl(F, D, List); + false -> + D = update_successor(Fail, Ts0, D0), + F = fun({Val,S}, A) -> + T = get_type(Val, Ts0), + update_successor(S, Ts0#{V=>T}, A) + end, + foldl(F, D, List) + end; update_successors(#b_ret{}, _Ts, D) -> D. update_successor_bool(#b_var{}=Var, BoolValue, S, Ts, D) -> @@ -447,6 +477,11 @@ update_successor_bool(#b_var{}=Var, BoolValue, S, Ts, D) -> update_successor(S, Ts, D) end. +update_successor(?BADARG_BLOCK, _Ts, #d{}=D) -> + %% We KNOW that no variables are used in the ?BADARG_BLOCK, + %% so there is no need to update the type information. That + %% can be a huge timesaver for huge functions. + D; update_successor(S, Ts0, #d{ls=Ls}=D) -> case Ls of #{S:=Ts1} -> @@ -766,6 +801,48 @@ simplify_not(#b_br{bool=#b_var{}=V,succ=Succ,fail=Fail}=Br0, Ts, Ds) -> Br0 end. +%%% +%%% Calculate the set of variables that are only used once in the +%%% block that they are defined in. That will allow us to discard type +%%% information for variables that will never be referenced by the +%%% successor blocks, potentially improving compilation times. +%%% + +used_once(Linear, Args) -> + Map0 = used_once_1(reverse(Linear), #{}), + Map = maps:without(Args, Map0), + cerl_sets:from_list(maps:keys(Map)). + +used_once_1([{L,#b_blk{is=Is,last=Last}}|Bs], Uses0) -> + Uses = used_once_2([Last|reverse(Is)], L, Uses0), + used_once_1(Bs, Uses); +used_once_1([], Uses) -> Uses. + +used_once_2([I|Is], L, Uses0) -> + Uses = used_once_uses(beam_ssa:used(I), L, Uses0), + case I of + #b_set{dst=Dst} -> + case Uses of + #{Dst:=[L]} -> + used_once_2(Is, L, Uses); + #{} -> + used_once_2(Is, L, maps:remove(Dst, Uses)) + end; + _ -> + used_once_2(Is, L, Uses) + end; +used_once_2([], _, Uses) -> Uses. + +used_once_uses([V|Vs], L, Uses) -> + case Uses of + #{V:=Us} -> + used_once_uses(Vs, L, Uses#{V:=[L|Us]}); + #{} -> + used_once_uses(Vs, L, Uses#{V=>[L]}) + end; +used_once_uses([], _, Uses) -> Uses. + + get_types(Values, Ts) -> [get_type(Val, Ts) || Val <- Values]. -spec get_type(beam_ssa:value(), type_db()) -> type(). diff --git a/lib/compiler/vsn.mk b/lib/compiler/vsn.mk index ab707885f4..92f8aec424 100644 --- a/lib/compiler/vsn.mk +++ b/lib/compiler/vsn.mk @@ -1 +1 @@ -COMPILER_VSN = 7.2.6 +COMPILER_VSN = 7.2.7 diff --git a/lib/crypto/c_src/crypto_callback.c b/lib/crypto/c_src/crypto_callback.c index 23d2bed057..0cc7dd609d 100644 --- a/lib/crypto/c_src/crypto_callback.c +++ b/lib/crypto/c_src/crypto_callback.c @@ -179,6 +179,10 @@ DLLEXPORT struct crypto_callbacks* get_crypto_callbacks(int nlocks) /* This is not really a NIF library, but we use ERL_NIF_INIT in order to * get access to the erl_nif API (on Windows). */ -ERL_NIF_INIT(dummy, (ErlNifFunc*)NULL , NULL, NULL, NULL, NULL) +static struct { + int dummy__; + ErlNifFunc funcv[0]; +} empty; +ERL_NIF_INIT(dummy, empty.funcv, NULL, NULL, NULL, NULL) #endif diff --git a/lib/kernel/test/code_SUITE.erl b/lib/kernel/test/code_SUITE.erl index abb22b5035..64e0b9d8dd 100644 --- a/lib/kernel/test/code_SUITE.erl +++ b/lib/kernel/test/code_SUITE.erl @@ -1021,6 +1021,13 @@ mult_lib_remove_prefix([H|T1], [H|T2]) -> mult_lib_remove_prefix([$/|T], []) -> T. bad_erl_libs(Config) when is_list(Config) -> + %% Preserve ERL_LIBS if set. + BadLibs0 = "/no/such/dir", + BadLibs = + case os:getenv("ERL_LIBS") of + false -> BadLibs0; + Libs -> BadLibs0 ++ ":" ++ Libs + end, {ok,Node} = test_server:start_node(bad_erl_libs, slave, []), Code = rpc:call(Node,code,get_path,[]), @@ -1028,10 +1035,9 @@ bad_erl_libs(Config) when is_list(Config) -> {ok,Node2} = test_server:start_node(bad_erl_libs, slave, - [{args,"-env ERL_LIBS /no/such/dir"}]), + [{args,"-env ERL_LIBS " ++ BadLibs}]), Code2 = rpc:call(Node,code,get_path,[]), test_server:stop_node(Node2), - %% Test that code path is not affected by the faulty ERL_LIBS Code = Code2, diff --git a/lib/public_key/doc/src/notes.xml b/lib/public_key/doc/src/notes.xml index 62b4b4ca1b..7ed60ed3ca 100644 --- a/lib/public_key/doc/src/notes.xml +++ b/lib/public_key/doc/src/notes.xml @@ -35,6 +35,22 @@ <file>notes.xml</file> </header> +<section><title>Public_Key 1.6.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Add DSA SHA2 oids in public_keys ASN1-spec and + public_key:pkix_sign_types/1</p> + <p> + Own Id: OTP-15367</p> + </item> + </list> + </section> + +</section> + <section><title>Public_Key 1.6.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/public_key/vsn.mk b/lib/public_key/vsn.mk index 4e52028c36..96eaf4f962 100644 --- a/lib/public_key/vsn.mk +++ b/lib/public_key/vsn.mk @@ -1 +1 @@ -PUBLIC_KEY_VSN = 1.6.2 +PUBLIC_KEY_VSN = 1.6.3 diff --git a/lib/sasl/src/sasl.app.src b/lib/sasl/src/sasl.app.src index 688aff16f1..5d45af0b50 100644 --- a/lib/sasl/src/sasl.app.src +++ b/lib/sasl/src/sasl.app.src @@ -43,5 +43,5 @@ {env, []}, {mod, {sasl, []}}, {runtime_dependencies, ["tools-2.6.14","stdlib-3.4","kernel-5.3", - "erts-9.0"]}]}. + "erts-@OTP-13468@"]}]}. diff --git a/lib/sasl/src/systools_make.erl b/lib/sasl/src/systools_make.erl index 072d4ed555..101701cec6 100644 --- a/lib/sasl/src/systools_make.erl +++ b/lib/sasl/src/systools_make.erl @@ -1562,7 +1562,7 @@ mandatory_modules() -> preloaded() -> %% Sorted - [erl_init,erl_prim_loader,erl_tracer,erlang, + [atomics,counters,erl_init,erl_prim_loader,erl_tracer,erlang, erts_code_purger,erts_dirty_process_signal_handler, erts_internal,erts_literal_area_collector, init,persistent_term,prim_buffer,prim_eval,prim_file, diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml index c9809ce1e3..1012c80fb1 100644 --- a/lib/ssh/doc/src/notes.xml +++ b/lib/ssh/doc/src/notes.xml @@ -230,6 +230,22 @@ </section> </section> +<section><title>Ssh 4.6.9.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Incompatibility with newer OpenSSH fixed. Previously + versions 7.8 and later could cause Erlang SSH to exit.</p> + <p> + Own Id: OTP-15413</p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 4.6.9.1</title> <section><title>Fixed Bugs and Malfunctions</title> <list> @@ -3869,4 +3885,3 @@ </section> </chapter> - diff --git a/lib/ssh/doc/src/ssh.xml b/lib/ssh/doc/src/ssh.xml index 9bfe7f5e9b..fcf79969d3 100644 --- a/lib/ssh/doc/src/ssh.xml +++ b/lib/ssh/doc/src/ssh.xml @@ -183,22 +183,6 @@ </datatype> <datatype> - <name name="pref_public_key_algs_client_option"/> - <desc> - <p>List of user (client) public key algorithms to try to use.</p> - <p>The default value is the <c>public_key</c> entry in the list returned by - <seealso marker="#default_algorithms/0">ssh:default_algorithms/0</seealso>. - </p> - <p>If there is no public key of a specified type available, the corresponding entry is ignored. - Note that the available set is dependent on the underlying cryptolib and current user's public keys. - </p> - <p>See also the option <seealso marker="ssh_file#type-user_dir_common_option"><c>user_dir</c></seealso> - for specifying the path to the user's keys. - </p> - </desc> - </datatype> - - <datatype> <name name="host_accepting_client_options"/> <name name="accept_hosts"/> <name name="fp_digest_alg"/> @@ -794,6 +778,22 @@ </datatype> <datatype> + <name name="pref_public_key_algs_common_option"/> + <desc> + <p>List of user (client) public key algorithms to try to use.</p> + <p>The default value is the <c>public_key</c> entry in the list returned by + <seealso marker="#default_algorithms/0">ssh:default_algorithms/0</seealso>. + </p> + <p>If there is no public key of a specified type available, the corresponding entry is ignored. + Note that the available set is dependent on the underlying cryptolib and current user's public keys. + </p> + <p>See also the option <seealso marker="ssh_file#type-user_dir_common_option"><c>user_dir</c></seealso> + for specifying the path to the user's keys. + </p> + </desc> + </datatype> + + <datatype> <name name="disconnectfun_common_option"/> <desc> <p>Provides a fun to implement your own logging when the peer disconnects.</p> diff --git a/lib/ssh/src/ssh.hrl b/lib/ssh/src/ssh.hrl index 233f3caa5d..f645201c4f 100644 --- a/lib/ssh/src/ssh.hrl +++ b/lib/ssh/src/ssh.hrl @@ -182,6 +182,7 @@ | ssh_msg_debug_fun_common_option() | rekey_limit_common_option() | id_string_common_option() + | pref_public_key_algs_common_option() | preferred_algorithms_common_option() | modify_algorithms_common_option() | auth_methods_common_option() @@ -209,6 +210,7 @@ {ssh_msg_debug_fun, fun((ssh:connection_ref(),AlwaysDisplay::boolean(),Msg::binary(),LanguageTag::binary()) -> any()) } . -type id_string_common_option() :: {id_string, string() | random | {random,Nmin::pos_integer(),Nmax::pos_integer()} }. +-type pref_public_key_algs_common_option() :: {pref_public_key_algs, [pubkey_alg()] } . -type preferred_algorithms_common_option():: {preferred_algorithms, algs_list()}. -type modify_algorithms_common_option() :: {modify_algorithms, modify_algs_list()}. -type auth_methods_common_option() :: {auth_methods, string() }. @@ -227,8 +229,7 @@ -type client_option() :: - pref_public_key_algs_client_option() - | ssh_file:pubkey_passphrase_client_options() + ssh_file:pubkey_passphrase_client_options() | host_accepting_client_options() | authentication_client_options() | diffie_hellman_group_exchange_client_option() @@ -242,8 +243,6 @@ {keyboard_interact_fun, fun((term(),term(),term()) -> term())} | opaque_common_options(). --type pref_public_key_algs_client_option() :: {pref_public_key_algs, [pubkey_alg()] } . - -type host_accepting_client_options() :: {silently_accept_hosts, accept_hosts()} | {user_interaction, boolean()} diff --git a/lib/ssh/src/ssh_connection_handler.erl b/lib/ssh/src/ssh_connection_handler.erl index 4b41c10cbb..30eafc2f2a 100644 --- a/lib/ssh/src/ssh_connection_handler.erl +++ b/lib/ssh/src/ssh_connection_handler.erl @@ -983,6 +983,10 @@ handle_event(_, #ssh_msg_userauth_info_request{}, {userauth_keyboard_interactive %%% ######## {connected, client|server} #### +%% Skip ext_info messages in connected state (for example from OpenSSH >= 7.7) +handle_event(_, #ssh_msg_ext_info{}, {connected,_Role}, D) -> + {keep_state, D}; + handle_event(_, {#ssh_msg_kexinit{},_}, {connected,Role}, D0) -> {KeyInitMsg, SshPacket, Ssh} = ssh_transport:key_exchange_init_msg(D0#data.ssh_params), D = D0#data{ssh_params = Ssh, diff --git a/lib/ssh/src/ssh_options.erl b/lib/ssh/src/ssh_options.erl index bc9f2156bc..bc9b0b6eda 100644 --- a/lib/ssh/src/ssh_options.erl +++ b/lib/ssh/src/ssh_options.erl @@ -452,12 +452,6 @@ default(client) -> class => user_options }, - {pref_public_key_algs, def} => - #{default => ssh_transport:default_algorithms(public_key), - chk => fun check_pref_public_key_algs/1, - class => user_options - }, - {dh_gex_limits, def} => #{default => {1024, 6144, 8192}, % FIXME: Is this true nowadays? chk => fun({Min,I,Max}) -> @@ -523,6 +517,12 @@ default(common) -> class => user_options }, + {pref_public_key_algs, def} => + #{default => ssh_transport:default_algorithms(public_key), + chk => fun check_pref_public_key_algs/1, + class => user_options + }, + {preferred_algorithms, def} => #{default => ssh:default_algorithms(), chk => fun check_preferred_algorithms/1, diff --git a/lib/ssh/src/ssh_transport.erl b/lib/ssh/src/ssh_transport.erl index c5b0704925..7424c9bcaf 100644 --- a/lib/ssh/src/ssh_transport.erl +++ b/lib/ssh/src/ssh_transport.erl @@ -759,8 +759,7 @@ ext_info_message(#ssh{role=server, send_ext_info=true, opts = Opts} = Ssh0) -> AlgsList = lists:map(fun erlang:atom_to_list/1, - proplists:get_value(public_key, - ?GET_OPT(preferred_algorithms, Opts))), + ?GET_OPT(pref_public_key_algs, Opts)), Msg = #ssh_msg_ext_info{nr_extensions = 1, data = [{"server-sig-algs", string:join(AlgsList,",")}] }, diff --git a/lib/ssh/test/ssh_compat_SUITE.erl b/lib/ssh/test/ssh_compat_SUITE.erl index 1c607bebe8..af85ef7aee 100644 --- a/lib/ssh/test/ssh_compat_SUITE.erl +++ b/lib/ssh/test/ssh_compat_SUITE.erl @@ -648,6 +648,7 @@ setup_remote_priv_and_local_auth_keys(KeyAlg, IP, Port, UserDir, Config) -> {silently_accept_hosts,true}, {user_interaction,false} ]), + rm_id_in_remote_dir(Ch, ".ssh"), _ = ssh_sftp:make_dir(Ch, ".ssh"), DstFile = filename:join(".ssh", dst_filename(user,KeyAlg)), ok = ssh_sftp:write_file(Ch, DstFile, Priv), @@ -658,6 +659,18 @@ setup_remote_priv_and_local_auth_keys(KeyAlg, IP, Port, UserDir, Config) -> ok = ssh:close(Cc), UserDir. +rm_id_in_remote_dir(Ch, Dir) -> + case ssh_sftp:list_dir(Ch, Dir) of + {error,_Error} -> + ok; + {ok,FileNames} -> + lists:foreach(fun("id_"++_ = F) -> + ok = ssh_sftp:delete(Ch, filename:join(Dir,F)); + (_) -> + leave + end, FileNames) + end. + user_priv_pub_keys(Config, KeyAlg) -> priv_pub_keys("users_keys", user, Config, KeyAlg). host_priv_pub_keys(Config, KeyAlg) -> priv_pub_keys("host_keys", host, Config, KeyAlg). diff --git a/lib/ssh/test/ssh_compat_SUITE_data/build_scripts/create_all b/lib/ssh/test/ssh_compat_SUITE_data/build_scripts/create_all index 0dcf8cb570..c2e77fcc79 100755 --- a/lib/ssh/test/ssh_compat_SUITE_data/build_scripts/create_all +++ b/lib/ssh/test/ssh_compat_SUITE_data/build_scripts/create_all @@ -18,6 +18,12 @@ SSH_SSL_VERSIONS=(\ openssh 7.6p1 openssl 1.0.2n \ \ openssh 7.6p1 libressl 2.6.4 \ + \ + openssh 7.7p1 openssl 1.0.2p \ + openssh 7.8p1 openssl 1.0.2p \ + openssh 7.9p1 openssl 1.0.2p \ + \ + openssh 7.9p1 libressl 2.6.4 \ ) if [ "x$1" == "x-b" ] diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index d117641496..4baedf7431 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -333,6 +333,38 @@ </section> +<section><title>SSL 8.2.6.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Add engine support for RSA key exchange</p> + <p> + Own Id: OTP-15420</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 8.2.6.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Extend check for undelivered data at closing, could under + some circumstances fail to deliverd all data that was + acctualy recivied.</p> + <p> + Own Id: OTP-15412</p> + </item> + </list> + </section> + +</section> + <section><title>SSL 8.2.6.2</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -3163,5 +3195,3 @@ </section> </section> </chapter> - - diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_cipher.erl index e12faba824..ff3e0d9c90 100644 --- a/lib/ssl/src/ssl_cipher.erl +++ b/lib/ssl/src/ssl_cipher.erl @@ -44,7 +44,7 @@ hash_algorithm/1, sign_algorithm/1, is_acceptable_hash/2, is_fallback/1, random_bytes/1, calc_mac_hash/4, is_stream_ciphersuite/1, signature_scheme/1, - scheme_to_components/1]). + scheme_to_components/1, hash_size/1]). -compile(inline). @@ -651,6 +651,29 @@ is_stream_ciphersuite(#{cipher := rc4_128}) -> true; is_stream_ciphersuite(_) -> false. + +-spec hash_size(atom()) -> integer(). +hash_size(null) -> + 0; +%% The AEAD MAC hash size is not used in the context +%% of calculating the master secret. See RFC 5246 Section 6.2.3.3. +hash_size(aead) -> + 0; +hash_size(md5) -> + 16; +hash_size(sha) -> + 20; +%% Uncomment when adding cipher suite that needs it +%hash_size(sha224) -> +% 28; +hash_size(sha256) -> + 32; +hash_size(sha384) -> + 48. +%% Uncomment when adding cipher suite that needs it +%hash_size(sha512) -> +% 64. + %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- @@ -861,29 +884,6 @@ scheme_to_components(rsa_pss_pss_sha512) -> {sha512, rsa_pss_pss, undefined}; scheme_to_components(rsa_pkcs1_sha1) -> {sha1, rsa_pkcs1, undefined}; scheme_to_components(ecdsa_sha1) -> {sha1, ecdsa, undefined}. - - -hash_size(null) -> - 0; -%% The AEAD MAC hash size is not used in the context -%% of calculating the master secret. See RFC 5246 Section 6.2.3.3. -hash_size(aead) -> - 0; -hash_size(md5) -> - 16; -hash_size(sha) -> - 20; -%% Uncomment when adding cipher suite that needs it -%hash_size(sha224) -> -% 28; -hash_size(sha256) -> - 32; -hash_size(sha384) -> - 48. -%% Uncomment when adding cipher suite that needs it -%hash_size(sha512) -> -% 64. - %% RFC 5246: 6.2.3.2. CBC Block Cipher %% %% Implementation note: Canvel et al. [CBCTIME] have demonstrated a diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index da2e92a76b..39a627c0a2 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -1000,6 +1000,13 @@ premaster_secret(EncSecret, #'RSAPrivateKey'{} = RSAPrivateKey) -> catch _:_ -> throw(?ALERT_REC(?FATAL, ?DECRYPT_ERROR)) + end; +premaster_secret(EncSecret, #{algorithm := rsa} = Engine) -> + try crypto:private_decrypt(rsa, EncSecret, maps:remove(algorithm, Engine), + [{rsa_pad, rsa_pkcs1_padding}]) + catch + _:_ -> + throw(?ALERT_REC(?FATAL, ?DECRYPT_ERROR)) end. %%==================================================================== %% Extensions handling diff --git a/lib/ssl/src/tls_v1.erl b/lib/ssl/src/tls_v1.erl index e7218c8c8a..68ba598612 100644 --- a/lib/ssl/src/tls_v1.erl +++ b/lib/ssl/src/tls_v1.erl @@ -36,6 +36,8 @@ default_signature_schemes/1, signature_schemes/2, groups/1, groups/2, group_to_enum/1, enum_to_group/1]). +-export([derive_secret/4, hkdf_expand_label/5, hkdf_extract/3, hkdf_expand/4]). + -type named_curve() :: sect571r1 | sect571k1 | secp521r1 | brainpoolP512r1 | sect409k1 | sect409r1 | brainpoolP384r1 | secp384r1 | sect283k1 | sect283r1 | brainpoolP256r1 | secp256k1 | secp256r1 | @@ -52,6 +54,44 @@ %% Internal application API %%==================================================================== +%% TLS 1.3 --------------------------------------------------- +-spec derive_secret(Secret::binary(), Label::binary(), + Messages::binary(), Algo::ssl_cipher_format:hash()) -> Key::binary(). +derive_secret(Secret, Label, Messages, Algo) -> + Hash = crypto:hash(mac_algo(Algo), Messages), + hkdf_expand_label(Secret, Label, + Hash, ssl_cipher:hash_size(Algo), Algo). + +-spec hkdf_expand_label(Secret::binary(), Label0::binary(), + Context::binary(), Length::integer(), + Algo::ssl_cipher_format:hash()) -> KeyingMaterial::binary(). +hkdf_expand_label(Secret, Label0, Context, Length, Algo) -> + %% struct { + %% uint16 length = Length; + %% opaque label<7..255> = "tls13 " + Label; + %% opaque context<0..255> = Context; + %% } HkdfLabel; + Content = << <<"tls13">>/binary, Label0/binary, Context/binary>>, + Len = size(Content), + HkdfLabel = <<?UINT16(Len), Content/binary>>, + hkdf_expand(Secret, HkdfLabel, Length, Algo). + +-spec hkdf_extract(MacAlg::ssl_cipher_format:hash(), Salt::binary(), + KeyingMaterial::binary()) -> PseudoRandKey::binary(). + +hkdf_extract(MacAlg, Salt, KeyingMaterial) -> + hmac_hash(MacAlg, Salt, KeyingMaterial). + + +-spec hkdf_expand(PseudoRandKey::binary(), ContextInfo::binary(), + Length::integer(), Algo::ssl_cipher_format:hash()) -> KeyingMaterial::binary(). + +hkdf_expand(PseudoRandKey, ContextInfo, Length, Algo) -> + Iterations = erlang:ceil(Length / ssl_cipher:hash_size(Algo)), + hkdf_expand(Algo, PseudoRandKey, ContextInfo, Length, 1, Iterations, <<>>, <<>>). +%% TLS 1.3 --------------------------------------------------- + +%% TLS 1.0 -1.2 --------------------------------------------------- -spec master_secret(integer(), binary(), binary(), binary()) -> binary(). master_secret(PrfAlgo, PreMasterSecret, ClientRandom, ServerRandom) -> @@ -61,9 +101,10 @@ master_secret(PrfAlgo, PreMasterSecret, ClientRandom, ServerRandom) -> prf(PrfAlgo, PreMasterSecret, <<"master secret">>, [ClientRandom, ServerRandom], 48). +%% TLS 1.0 -1.2 --------------------------------------------------- -spec finished(client | server, integer(), integer(), binary(), [binary()]) -> binary(). - +%% TLS 1.0 -1.1 --------------------------------------------------- finished(Role, Version, PrfAlgo, MasterSecret, Handshake) when Version == 1; Version == 2; PrfAlgo == ?MD5SHA -> %% RFC 2246 & 4346 - 7.4.9. Finished @@ -77,9 +118,11 @@ finished(Role, Version, PrfAlgo, MasterSecret, Handshake) MD5 = crypto:hash(md5, Handshake), SHA = crypto:hash(sha, Handshake), prf(?MD5SHA, MasterSecret, finished_label(Role), [MD5, SHA], 12); +%% TLS 1.0 -1.1 --------------------------------------------------- +%% TLS 1.2 --------------------------------------------------- finished(Role, Version, PrfAlgo, MasterSecret, Handshake) - when Version == 3; Version == 4 -> + when Version == 3 -> %% RFC 5246 - 7.4.9. Finished %% struct { %% opaque verify_data[12]; @@ -89,22 +132,28 @@ finished(Role, Version, PrfAlgo, MasterSecret, Handshake) %% PRF(master_secret, finished_label, Hash(handshake_messages)) [0..11]; Hash = crypto:hash(mac_algo(PrfAlgo), Handshake), prf(PrfAlgo, MasterSecret, finished_label(Role), Hash, 12). +%% TLS 1.2 --------------------------------------------------- +%% TODO 1.3 finished -spec certificate_verify(md5sha | sha, integer(), [binary()]) -> binary(). +%% TLS 1.0 -1.1 --------------------------------------------------- certificate_verify(md5sha, _Version, Handshake) -> MD5 = crypto:hash(md5, Handshake), SHA = crypto:hash(sha, Handshake), <<MD5/binary, SHA/binary>>; +%% TLS 1.0 -1.1 --------------------------------------------------- +%% TLS 1.2 --------------------------------------------------- certificate_verify(HashAlgo, _Version, Handshake) -> crypto:hash(HashAlgo, Handshake). +%% TLS 1.2 --------------------------------------------------- -spec setup_keys(integer(), integer(), binary(), binary(), binary(), integer(), integer(), integer()) -> {binary(), binary(), binary(), binary(), binary(), binary()}. - +%% TLS v1.0 --------------------------------------------------- setup_keys(Version, _PrfAlgo, MasterSecret, ServerRandom, ClientRandom, HashSize, KeyMatLen, IVSize) when Version == 1 -> @@ -129,8 +178,9 @@ setup_keys(Version, _PrfAlgo, MasterSecret, ServerRandom, ClientRandom, HashSize ClientIV:IVSize/binary, ServerIV:IVSize/binary>> = KeyBlock, {ClientWriteMacSecret, ServerWriteMacSecret, ClientWriteKey, ServerWriteKey, ClientIV, ServerIV}; +%% TLS v1.0 --------------------------------------------------- -%% TLS v1.1 +%% TLS v1.1 --------------------------------------------------- setup_keys(Version, _PrfAlgo, MasterSecret, ServerRandom, ClientRandom, HashSize, KeyMatLen, IVSize) when Version == 2 -> @@ -156,8 +206,9 @@ setup_keys(Version, _PrfAlgo, MasterSecret, ServerRandom, ClientRandom, HashSize ClientIV:IVSize/binary, ServerIV:IVSize/binary>> = KeyBlock, {ClientWriteMacSecret, ServerWriteMacSecret, ClientWriteKey, ServerWriteKey, ClientIV, ServerIV}; +%% TLS v1.1 --------------------------------------------------- -%% TLS v1.2 +%% TLS v1.2 --------------------------------------------------- setup_keys(Version, PrfAlgo, MasterSecret, ServerRandom, ClientRandom, HashSize, KeyMatLen, IVSize) when Version == 3; Version == 4 -> @@ -182,8 +233,10 @@ setup_keys(Version, PrfAlgo, MasterSecret, ServerRandom, ClientRandom, HashSize, ClientIV:IVSize/binary, ServerIV:IVSize/binary>> = KeyBlock, {ClientWriteMacSecret, ServerWriteMacSecret, ClientWriteKey, ServerWriteKey, ClientIV, ServerIV}. +%% TLS v1.2 --------------------------------------------------- --spec mac_hash(integer(), binary(), integer(), integer(), tls_record:tls_version(), +%% TLS 1.0 -1.2 --------------------------------------------------- +-spec mac_hash(integer() | atom(), binary(), integer(), integer(), tls_record:tls_version(), integer(), binary()) -> binary(). mac_hash(Method, Mac_write_secret, Seq_num, Type, {Major, Minor}, @@ -197,6 +250,9 @@ mac_hash(Method, Mac_write_secret, Seq_num, Type, {Major, Minor}, ?BYTE(Major), ?BYTE(Minor), ?UINT16(Length)>>, Fragment]), Mac. +%% TLS 1.0 -1.2 --------------------------------------------------- + +%% TODO 1.3 same as above? -spec suites(1|2|3|4) -> [ssl_cipher_format:cipher_suite()]. @@ -345,7 +401,6 @@ signature_schemes(Version, SignatureSchemes) when is_tuple(Version) signature_schemes(_, _) -> []. - default_signature_schemes(Version) -> Default = [ rsa_pkcs1_sha256, @@ -371,12 +426,21 @@ default_signature_schemes(Version) -> %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- +hkdf_expand(Algo, PseudoRandKey, ContextInfo, Length, N, N, Prev, Acc) -> + Keyingmaterial = hmac_hash(Algo, PseudoRandKey, <<Prev/binary, ContextInfo/binary, ?BYTE(N)>>), + binary:part(<<Acc/binary, Keyingmaterial/binary>>, {0, Length}); +hkdf_expand(Algo, PseudoRandKey, ContextInfo, Length, M, N, Prev, Acc) -> + Keyingmaterial = hmac_hash(Algo, PseudoRandKey, <<Prev/binary, ContextInfo/binary, ?BYTE(M)>>), + hkdf_expand(Algo, PseudoRandKey, ContextInfo, Length, M + 1, N, Keyingmaterial, <<Acc/binary, Keyingmaterial/binary>>). + %%%% HMAC and the Pseudorandom Functions RFC 2246 & 4346 - 5.%%%% hmac_hash(?NULL, _, _) -> <<>>; hmac_hash(Alg, Key, Value) -> crypto:hmac(mac_algo(Alg), Key, Value). +mac_algo(Alg) when is_atom(Alg) -> + Alg; mac_algo(?MD5) -> md5; mac_algo(?SHA) -> sha; mac_algo(?SHA256) -> sha256; diff --git a/lib/ssl/test/Makefile b/lib/ssl/test/Makefile index d5ba105478..a4adc7561b 100644 --- a/lib/ssl/test/Makefile +++ b/lib/ssl/test/Makefile @@ -62,6 +62,7 @@ MODULES = \ ssl_upgrade_SUITE\ ssl_sni_SUITE \ ssl_eqc_SUITE \ + ssl_rfc_5869_SUITE \ make_certs\ x509_test diff --git a/lib/ssl/test/ssl_bench_test_lib.erl b/lib/ssl/test/ssl_bench_test_lib.erl index e5cbb911bd..47bcd41608 100644 --- a/lib/ssl/test/ssl_bench_test_lib.erl +++ b/lib/ssl/test/ssl_bench_test_lib.erl @@ -58,13 +58,13 @@ setup(Name) -> Path = code:get_path(), true = rpc:call(Node, code, set_path, [Path]), ok = rpc:call(Node, ?MODULE, setup_server, [node()]), - io:format("Client (~p) using ~s~n",[node(), code:which(ssl)]), + io:format("Client (~p) using ~ts~n",[node(), code:which(ssl)]), (Node =:= node()) andalso restrict_schedulers(client), Node. setup_server(ClientNode) -> (ClientNode =:= node()) andalso restrict_schedulers(server), - io:format("Server (~p) using ~s~n",[node(), code:which(ssl)]), + io:format("Server (~p) using ~ts~n",[node(), code:which(ssl)]), ok. restrict_schedulers(Type) -> diff --git a/lib/ssl/test/ssl_engine_SUITE.erl b/lib/ssl/test/ssl_engine_SUITE.erl index 1423c99dc2..e6c82d3eb5 100644 --- a/lib/ssl/test/ssl_engine_SUITE.erl +++ b/lib/ssl/test/ssl_engine_SUITE.erl @@ -90,12 +90,14 @@ end_per_testcase(_TestCase, Config) -> private_key(Config) when is_list(Config) -> ClientFileBase = filename:join([proplists:get_value(priv_dir, Config), "client_engine"]), ServerFileBase = filename:join([proplists:get_value(priv_dir, Config), "server_engine"]), + Ext = x509_test:extensions([{key_usage, [digitalSignature, keyEncipherment]}]), #{server_config := ServerConf, client_config := ClientConf} = GenCertData = public_key:pkix_test_data(#{server_chain => #{root => [{key, ssl_test_lib:hardcode_rsa_key(1)}], intermediates => [[{key, ssl_test_lib:hardcode_rsa_key(2)}]], - peer => [{key, ssl_test_lib:hardcode_rsa_key(3)} + peer => [{extensions, Ext}, + {key, ssl_test_lib:hardcode_rsa_key(3)} ]}, client_chain => #{root => [{key, ssl_test_lib:hardcode_rsa_key(4)}], @@ -131,6 +133,12 @@ private_key(Config) when is_list(Config) -> %% Test with engine test_tls_connection(EngineServerConf, EngineClientConf, Config), + %% Test with engine and rsa keyexchange + RSASuites = all_kex_rsa_suites([{tls_version, 'tlsv1.2'} | Config]), + + test_tls_connection([{ciphers, RSASuites}, {versions, ['tlsv1.2']} | EngineServerConf], + [{ciphers, RSASuites}, {versions, ['tlsv1.2']} | EngineClientConf], Config), + %% Test with engine and present file arugments test_tls_connection(EngineFileServerConf, EngineFileClientConf, Config), @@ -160,3 +168,8 @@ test_tls_connection(ServerConf, ClientConf, Config) -> ssl_test_lib:check_result(Server, ok, Client, ok), ssl_test_lib:close(Server), ssl_test_lib:close(Client). + +all_kex_rsa_suites(Config) -> + Version = proplists:get_value(tls_version, Config), + All = ssl:cipher_suites(all, Version), + ssl:filter_cipher_suites(All,[{key_exchange, fun(rsa) -> true;(_) -> false end}]). diff --git a/lib/ssl/test/ssl_rfc_5869_SUITE.erl b/lib/ssl/test/ssl_rfc_5869_SUITE.erl new file mode 100644 index 0000000000..8b2d1c2082 --- /dev/null +++ b/lib/ssl/test/ssl_rfc_5869_SUITE.erl @@ -0,0 +1,316 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018-2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +%% +-module(ssl_rfc_5869_SUITE). + +%% Note: This directive should only be used in test suites. +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% Common Test interface functions ----------------------------------- +%%-------------------------------------------------------------------- +all() -> + [sha_256_basic, + sha_256_long, + sha_256_no_salt, + sha_basic, + sha_long, + sha_no_salt, + sha_default_salt + ]. + +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + catch crypto:stop(), + try crypto:start() of + ok -> + Config + catch _:_ -> + {skip, "Crypto did not start"} + end. + +end_per_suite(_Config) -> + application:stop(crypto). + +%%-------------------------------------------------------------------- +init_per_testcase(_TestCase, Config) -> + ct:timetrap({seconds, 5}), + Config. + +end_per_testcase(_TestCase, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Test Cases -------------------------------------------------------- +%%-------------------------------------------------------------------- + +sha_256_basic() -> + [{doc, "Basic test case with SHA-256"}]. +sha_256_basic(Config) when is_list(Config) -> + %% Hash = SHA-256 + %% IKM = 0x0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b (22 octets) + %% salt = 0x000102030405060708090a0b0c (13 octets) + %% info = 0xf0f1f2f3f4f5f6f7f8f9 (10 octets) + %% L = 42 + %% PRK = 0x077709362c2e32df0ddc3f0dc47bba63 + %% 90b6c73bb50f9c3122ec844ad7c2b3e5 (32 octets) + %% OKM = 0x3cb25f25faacd57a90434f64d0362f2a + %% 2d2d0a90cf1a5a4c5db02d56ecc4c5bf + %% 34007208d5b887185865 (42 octets) + IKM = hexstr2bin("0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b"), + Salt = hexstr2bin("000102030405060708090a0b0c"), + Info = hexstr2bin("f0f1f2f3f4f5f6f7f8f9"), + PRK = hexstr2bin("077709362c2e32df0ddc3f0dc47bba63" + "90b6c73bb50f9c3122ec844ad7c2b3e5"), + OKM = hexstr2bin("3cb25f25faacd57a90434f64d0362f2a" + "2d2d0a90cf1a5a4c5db02d56ecc4c5bf" + "34007208d5b887185865"), + hkdf_test(sha256, Salt, IKM, PRK, Info, 42, OKM). + +sha_256_long() -> + [{doc, "Test with SHA-256 and longer inputs/outputs"}]. +sha_256_long(Config) when is_list(Config) -> + %% Hash = SHA-256 + %% IKM = 0x000102030405060708090a0b0c0d0e0f + %% 101112131415161718191a1b1c1d1e1f + %% 202122232425262728292a2b2c2d2e2f + %% 303132333435363738393a3b3c3d3e3f + %% 404142434445464748494a4b4c4d4e4f (80 octets) + %% salt = 0x606162636465666768696a6b6c6d6e6f + %% 707172737475767778797a7b7c7d7e7f + %% 808182838485868788898a8b8c8d8e8f + %% 909192939495969798999a9b9c9d9e9f + %% a0a1a2a3a4a5a6a7a8a9aaabacadaeaf (80 octets) + %% info = 0xb0b1b2b3b4b5b6b7b8b9babbbcbdbebf + %% c0c1c2c3c4c5c6c7c8c9cacbcccdcecf + %% d0d1d2d3d4d5d6d7d8d9dadbdcdddedf + %% e0e1e2e3e4e5e6e7e8e9eaebecedeeef + %% f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff (80 octets) + %% L = 82 + + %% PRK = 0x06a6b88c5853361a06104c9ceb35b45c + %% ef760014904671014a193f40c15fc244 (32 octets) + %% OKM = 0xb11e398dc80327a1c8e7f78c596a4934 + %% 4f012eda2d4efad8a050cc4c19afa97c + %% 59045a99cac7827271cb41c65e590e09 + %% da3275600c2f09b8367793a9aca3db71 + %% cc30c58179ec3e87c14c01d5c1f3434f + %% 1d87 (82 octets) + IKM = hexstr2bin("000102030405060708090a0b0c0d0e0f" + "101112131415161718191a1b1c1d1e1f" + "202122232425262728292a2b2c2d2e2f" + "303132333435363738393a3b3c3d3e3f" + "404142434445464748494a4b4c4d4e4f" + ), + Salt = hexstr2bin("606162636465666768696a6b6c6d6e6f" + "707172737475767778797a7b7c7d7e7f" + "808182838485868788898a8b8c8d8e8f" + "909192939495969798999a9b9c9d9e9f" + "a0a1a2a3a4a5a6a7a8a9aaabacadaeaf" + ), + Info = hexstr2bin("b0b1b2b3b4b5b6b7b8b9babbbcbdbebf" + "c0c1c2c3c4c5c6c7c8c9cacbcccdcecf" + "d0d1d2d3d4d5d6d7d8d9dadbdcdddedf" + "e0e1e2e3e4e5e6e7e8e9eaebecedeeef" + "f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff" + ), + PRK = hexstr2bin("06a6b88c5853361a06104c9ceb35b45c" + "ef760014904671014a193f40c15fc244"), + OKM = hexstr2bin("b11e398dc80327a1c8e7f78c596a4934" + "4f012eda2d4efad8a050cc4c19afa97c" + "59045a99cac7827271cb41c65e590e09" + "da3275600c2f09b8367793a9aca3db71" + "cc30c58179ec3e87c14c01d5c1f3434f" + "1d87" + ), + hkdf_test(sha256, Salt, IKM, PRK, Info, 82, OKM). +sha_256_no_salt() -> + [{doc, "Test with SHA-256 and zero-length salt/info"}]. +sha_256_no_salt(Config) when is_list(Config) -> + %% Hash = SHA-256 + %% IKM = 0x0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b (22 octets) + %% salt = (0 octets) + %% info = (0 octets) + %% L = 42 + + %% PRK = 0x19ef24a32c717b167f33a91d6f648bdf + %% 96596776afdb6377ac434c1c293ccb04 (32 octets) + %% OKM = 0x8da4e775a563c18f715f802a063c5a31 + %% b8a11f5c5ee1879ec3454e5f3c738d2d + %% 9d201395faa4b61a96c8 (42 octets) + IKM = hexstr2bin("0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b"), + Salt = <<>>, + Info = <<>>, + PRK = hexstr2bin("19ef24a32c717b167f33a91d6f648bdf" + "96596776afdb6377ac434c1c293ccb04"), + OKM = hexstr2bin("8da4e775a563c18f715f802a063c5a31" + "b8a11f5c5ee1879ec3454e5f3c738d2d" + "9d201395faa4b61a96c8"), + hkdf_test(sha256, Salt, IKM, PRK, Info, 42, OKM). + +sha_basic() -> + [{doc, " Basic test case with SHA-1"}]. +sha_basic(Config) when is_list(Config) -> + %% Hash = SHA-1 + %% IKM = 0x0b0b0b0b0b0b0b0b0b0b0b (11 octets) + %% salt = 0x000102030405060708090a0b0c (13 octets) + %% info = 0xf0f1f2f3f4f5f6f7f8f9 (10 octets) + %% L = 42 + + %% PRK = 0x9b6c18c432a7bf8f0e71c8eb88f4b30baa2ba243 (20 octets) + %% OKM = 0x085a01ea1b10f36933068b56efa5ad81 + %% a4f14b822f5b091568a9cdd4f155fda2 + %% c22e422478d305f3f896 (42 octets) + IKM = hexstr2bin("0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b"), + Salt = hexstr2bin("000102030405060708090a0b0c"), + Info = hexstr2bin("f0f1f2f3f4f5f6f7f8f9"), + PRK = hexstr2bin("077709362c2e32df0ddc3f0dc47bba63" + "90b6c73bb50f9c3122ec844ad7c2b3e5"), + OKM = hexstr2bin("3cb25f25faacd57a90434f64d0362f2a" + "2d2d0a90cf1a5a4c5db02d56ecc4c5bf" + "34007208d5b887185865"), + hkdf_test(sha256, Salt, IKM, PRK, Info, 42, OKM). + +sha_long() -> + [{doc, "Test with SHA-1 and longer inputs/outputs"}]. +sha_long(Config) when is_list(Config) -> + %% Hash = SHA-1 + %% IKM = 0x000102030405060708090a0b0c0d0e0f + %% 101112131415161718191a1b1c1d1e1f + %% 202122232425262728292a2b2c2d2e2f + %% 303132333435363738393a3b3c3d3e3f + %% 404142434445464748494a4b4c4d4e4f (80 octets) + %% salt = 0x606162636465666768696a6b6c6d6e6f + %% 707172737475767778797a7b7c7d7e7f + %% 808182838485868788898a8b8c8d8e8f + %% 909192939495969798999a9b9c9d9e9f + %% a0a1a2a3a4a5a6a7a8a9aaabacadaeaf (80 octets) + %% info = 0xb0b1b2b3b4b5b6b7b8b9babbbcbdbebf + %% c0c1c2c3c4c5c6c7c8c9cacbcccdcecf + %% d0d1d2d3d4d5d6d7d8d9dadbdcdddedf + %% e0e1e2e3e4e5e6e7e8e9eaebecedeeef + %% f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff (80 octets) + %% L = 82 + + %% PRK = 0x8adae09a2a307059478d309b26c4115a224cfaf6 (20 octets) + %% OKM = 0x0bd770a74d1160f7c9f12cd5912a06eb + %% ff6adcae899d92191fe4305673ba2ffe + %% 8fa3f1a4e5ad79f3f334b3b202b2173c + %% 486ea37ce3d397ed034c7f9dfeb15c5e + %% 927336d0441f4c4300e2cff0d0900b52 + %% d3b4 (82 octets) + IKM = hexstr2bin("000102030405060708090a0b0c0d0e0f" + "101112131415161718191a1b1c1d1e1f" + "202122232425262728292a2b2c2d2e2f" + "303132333435363738393a3b3c3d3e3f" + "404142434445464748494a4b4c4d4e4f" + ), + Salt = hexstr2bin("606162636465666768696a6b6c6d6e6f" + "707172737475767778797a7b7c7d7e7f" + "808182838485868788898a8b8c8d8e8f" + "909192939495969798999a9b9c9d9e9f" + "a0a1a2a3a4a5a6a7a8a9aaabacadaeaf" + ), + Info = hexstr2bin("b0b1b2b3b4b5b6b7b8b9babbbcbdbebf" + "c0c1c2c3c4c5c6c7c8c9cacbcccdcecf" + "d0d1d2d3d4d5d6d7d8d9dadbdcdddedf" + "e0e1e2e3e4e5e6e7e8e9eaebecedeeef" + "f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff" + ), + PRK = hexstr2bin("8adae09a2a307059478d309b26c4115a224cfaf6"), + OKM = hexstr2bin("0bd770a74d1160f7c9f12cd5912a06eb" + "ff6adcae899d92191fe4305673ba2ffe" + "8fa3f1a4e5ad79f3f334b3b202b2173c" + "486ea37ce3d397ed034c7f9dfeb15c5e" + "927336d0441f4c4300e2cff0d0900b52" + "d3b4" + ), + hkdf_test(sha, Salt, IKM, PRK, Info, 82, OKM). + +sha_no_salt() -> + [{doc, "Test with SHA-1 and zero-length salt/info"}]. +sha_no_salt(Config) when is_list(Config) -> + %% Hash = SHA-1 + %% IKM = 0x0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b (22 octets) + %% salt = (0 octets) + %% info = (0 octets) + %% L = 42 + + %% PRK = 0xda8c8a73c7fa77288ec6f5e7c297786aa0d32d01 (20 octets) + %% OKM = 0x0ac1af7002b3d761d1e55298da9d0506 + %% b9ae52057220a306e07b6b87e8df21d0 + %% ea00033de03984d34918 (42 octets) + IKM = hexstr2bin("0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b"), + Salt = <<>>, + Info = <<>>, + PRK = hexstr2bin("da8c8a73c7fa77288ec6f5e7c297786aa0d32d01"), + OKM = hexstr2bin("0ac1af7002b3d761d1e55298da9d0506" + "b9ae52057220a306e07b6b87e8df21d0" + "ea00033de03984d34918"), + hkdf_test(sha, Salt, IKM, PRK, Info, 42, OKM). + + +sha_default_salt() -> + [{doc, "Test with SHA-1, salt not provided (defaults to HashLen zero octets), + zero-length info"}]. +sha_default_salt(Config) when is_list(Config) -> + %% Hash = SHA-1 + %% IKM = 0x0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c (22 octets) + %% salt = not provided (defaults to HashLen zero octets) + %% info = (0 octets) + %% L = 42 + + %% PRK = 0x2adccada18779e7c2077ad2eb19d3f3e731385dd (20 octets) + %% OKM = 0x2c91117204d745f3500d636a62f64f0a + %% b3bae548aa53d423b0d1f27ebba6f5e5 + %% 673a081d70cce7acfc48 (42 octets) + IKM = hexstr2bin("0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c"), + Salt = binary:copy(<<0>>, 20), + Info = <<>>, + PRK = hexstr2bin("2adccada18779e7c2077ad2eb19d3f3e731385dd"), + OKM = hexstr2bin("2c91117204d745f3500d636a62f64f0a" + "b3bae548aa53d423b0d1f27ebba6f5e5" + "673a081d70cce7acfc48"), + hkdf_test(sha, Salt, IKM, PRK, Info, 42, OKM). + +hkdf_test(HashAlg, Salt, KeyingMaterial, PsedoRandKey, ContextInfo, Length, Key) -> + PsedoRandKey = tls_v1:hkdf_extract(HashAlg, Salt, KeyingMaterial), + Key = tls_v1:hkdf_expand(PsedoRandKey, ContextInfo, Length, HashAlg). + +hexstr2bin(S) when is_binary(S) -> + list_to_binary(hexstr2list(binary_to_list(S))); +hexstr2bin(S) -> + list_to_binary(hexstr2list(S)). + +hexstr2list([$ |T]) -> + hexstr2list(T); +hexstr2list([X,Y|T]) -> + [mkint(X)*16 + mkint(Y) | hexstr2list(T)]; +hexstr2list([]) -> + []. +mkint(C) when $0 =< C, C =< $9 -> + C - $0; +mkint(C) when $A =< C, C =< $F -> + C - $A + 10; +mkint(C) when $a =< C, C =< $f -> + C - $a + 10. diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index 318643aef8..64b97fad7f 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -504,6 +504,21 @@ </section> +<section><title>STDLIB 3.4.5.1</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>List subtraction (The <c>--</c> operator) will now + yield properly on large inputs.</p> + <p> + Own Id: OTP-15371</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 3.4.5</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -1658,6 +1673,21 @@ </section> +<section><title>STDLIB 2.8.0.1</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>List subtraction (The <c>--</c> operator) will now + yield properly on large inputs.</p> + <p> + Own Id: OTP-15371</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 2.8</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -7827,4 +7857,3 @@ </section> </section> </chapter> - diff --git a/lib/stdlib/test/lists_SUITE.erl b/lib/stdlib/test/lists_SUITE.erl index 316b7a1ffa..5dab6f6697 100644 --- a/lib/stdlib/test/lists_SUITE.erl +++ b/lib/stdlib/test/lists_SUITE.erl @@ -2618,6 +2618,13 @@ subtract(Config) when is_list(Config) -> %% certain thresholds, and we need proper coverage for all corner cases. [sub_thresholds(N) || N <- lists:seq(0, 32)], + %% Trapping, both crashing and otherwise. + [sub_trapping(N) || N <- lists:seq(0, 18)], + + %% The current implementation chooses which algorithm to use based on + %% certain thresholds, and we need proper coverage for all corner cases. + [sub_thresholds(N) || N <- lists:seq(0, 32)], + ok. sub_non_matching(A, B) -> diff --git a/otp_versions.table b/otp_versions.table index 523b3d94ab..8ab4b72804 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -1,3 +1,4 @@ +OTP-21.1.2 : compiler-7.2.7 erts-10.1.2 public_key-1.6.3 # asn1-5.0.7 common_test-1.16.1 crypto-4.3.3 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 et-1.6.3 eunit-2.3.7 ftp-1.0.1 hipe-3.18.1 inets-7.0.2 jinterface-1.9.1 kernel-6.1 megaco-3.18.4 mnesia-4.15.5 observer-2.8.1 odbc-2.12.2 os_mon-2.4.6 otp_mibs-1.2.1 parsetools-2.1.8 reltool-0.7.7 runtime_tools-1.13.1 sasl-3.2.1 snmp-5.2.12 ssh-4.7.1 ssl-9.0.3 stdlib-3.6 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.1 wx-1.8.5 xmerl-1.3.18 : OTP-21.1.1 : compiler-7.2.6 eldap-1.2.6 erts-10.1.1 ssl-9.0.3 # asn1-5.0.7 common_test-1.16.1 crypto-4.3.3 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 erl_docgen-0.8.1 erl_interface-3.10.4 et-1.6.3 eunit-2.3.7 ftp-1.0.1 hipe-3.18.1 inets-7.0.2 jinterface-1.9.1 kernel-6.1 megaco-3.18.4 mnesia-4.15.5 observer-2.8.1 odbc-2.12.2 os_mon-2.4.6 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.2 reltool-0.7.7 runtime_tools-1.13.1 sasl-3.2.1 snmp-5.2.12 ssh-4.7.1 stdlib-3.6 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.1 wx-1.8.5 xmerl-1.3.18 : OTP-21.1 : asn1-5.0.7 common_test-1.16.1 compiler-7.2.5 crypto-4.3.3 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.5 erl_docgen-0.8.1 erl_interface-3.10.4 erts-10.1 et-1.6.3 eunit-2.3.7 ftp-1.0.1 hipe-3.18.1 inets-7.0.2 jinterface-1.9.1 kernel-6.1 megaco-3.18.4 mnesia-4.15.5 observer-2.8.1 odbc-2.12.2 os_mon-2.4.6 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.2 reltool-0.7.7 runtime_tools-1.13.1 sasl-3.2.1 snmp-5.2.12 ssh-4.7.1 ssl-9.0.2 stdlib-3.6 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.1 wx-1.8.5 xmerl-1.3.18 # : OTP-21.0.9 : compiler-7.2.4 erts-10.0.8 # asn1-5.0.6 common_test-1.16 crypto-4.3.2 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0.1 jinterface-1.9 kernel-6.0.1 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6.1 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0.1 stdlib-3.5.1 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : @@ -10,6 +11,9 @@ OTP-21.0.3 : erts-10.0.3 # asn1-5.0.6 common_test-1.16 compiler-7.2.2 crypto-4.3 OTP-21.0.2 : compiler-7.2.2 erts-10.0.2 public_key-1.6.1 stdlib-3.5.1 # asn1-5.0.6 common_test-1.16 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0.1 : compiler-7.2.1 erts-10.0.1 # asn1-5.0.6 common_test-1.16 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 stdlib-3.5 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0 : asn1-5.0.6 common_test-1.16 compiler-7.2 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 erts-10.0 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 mnesia-4.15.4 observer-2.8 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 ssh-4.7 ssl-9.0 stdlib-3.5 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 # megaco-3.18.3 odbc-2.12.1 snmp-5.2.11 : +OTP-20.3.8.14 : ssh-4.6.9.2 # asn1-5.0.5.1 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.6 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : +OTP-20.3.8.13 : ssl-8.2.6.4 # asn1-5.0.5.1 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.6 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : +OTP-20.3.8.12 : erts-9.3.3.6 ssl-8.2.6.3 stdlib-3.4.5.1 # asn1-5.0.5.1 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.8.11 : erts-9.3.3.5 # asn1-5.0.5.1 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 ssl-8.2.6.2 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.8.10 : eldap-1.2.3.1 erts-9.3.3.4 # asn1-5.0.5.1 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 erl_docgen-0.7.3 erl_interface-3.10.2.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 ssl-8.2.6.2 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.8.9 : compiler-7.1.5.2 # asn1-5.0.5.1 common_test-1.15.4 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 ssl-8.2.6.2 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : @@ -90,6 +94,7 @@ OTP-19.0.3 : inets-6.3.2 kernel-5.0.1 ssl-8.0.1 # asn1-4.0.3 common_test-1.12.2 OTP-19.0.2 : compiler-7.0.1 erts-8.0.2 stdlib-3.0.1 # asn1-4.0.3 common_test-1.12.2 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7 debugger-4.2 dialyzer-3.0.1 diameter-1.12 edoc-0.7.19 eldap-1.2.2 erl_docgen-0.5 erl_interface-3.9 et-1.6 eunit-2.3 gs-1.6.1 hipe-3.15.1 ic-4.4.1 inets-6.3.1 jinterface-1.7 kernel-5.0 megaco-3.18.1 mnesia-4.14 observer-2.2.1 odbc-2.11.2 orber-3.8.2 os_mon-2.4.1 otp_mibs-1.1.1 parsetools-2.1.2 percept-0.9 public_key-1.2 reltool-0.7.1 runtime_tools-1.10 sasl-3.0 snmp-5.2.3 ssh-4.3.1 ssl-8.0 syntax_tools-2.0 tools-2.8.5 typer-0.9.11 wx-1.7 xmerl-1.3.11 : OTP-19.0.1 : dialyzer-3.0.1 erts-8.0.1 inets-6.3.1 observer-2.2.1 ssh-4.3.1 tools-2.8.5 # asn1-4.0.3 common_test-1.12.2 compiler-7.0 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7 debugger-4.2 diameter-1.12 edoc-0.7.19 eldap-1.2.2 erl_docgen-0.5 erl_interface-3.9 et-1.6 eunit-2.3 gs-1.6.1 hipe-3.15.1 ic-4.4.1 jinterface-1.7 kernel-5.0 megaco-3.18.1 mnesia-4.14 odbc-2.11.2 orber-3.8.2 os_mon-2.4.1 otp_mibs-1.1.1 parsetools-2.1.2 percept-0.9 public_key-1.2 reltool-0.7.1 runtime_tools-1.10 sasl-3.0 snmp-5.2.3 ssl-8.0 stdlib-3.0 syntax_tools-2.0 typer-0.9.11 wx-1.7 xmerl-1.3.11 : OTP-19.0 : asn1-4.0.3 common_test-1.12.2 compiler-7.0 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7 debugger-4.2 dialyzer-3.0 diameter-1.12 edoc-0.7.19 eldap-1.2.2 erl_docgen-0.5 erl_interface-3.9 erts-8.0 et-1.6 eunit-2.3 gs-1.6.1 hipe-3.15.1 ic-4.4.1 inets-6.3 jinterface-1.7 kernel-5.0 megaco-3.18.1 mnesia-4.14 observer-2.2 odbc-2.11.2 orber-3.8.2 os_mon-2.4.1 otp_mibs-1.1.1 parsetools-2.1.2 percept-0.9 public_key-1.2 reltool-0.7.1 runtime_tools-1.10 sasl-3.0 snmp-5.2.3 ssh-4.3 ssl-8.0 stdlib-3.0 syntax_tools-2.0 tools-2.8.4 typer-0.9.11 wx-1.7 xmerl-1.3.11 # : +OTP-18.3.4.11 : erts-7.3.1.6 stdlib-2.8.0.1 # asn1-4.0.2 common_test-1.12.1.1 compiler-6.0.3.1 cosEvent-2.2 cosEventDomain-1.2 cosFileTransfer-1.2 cosNotification-1.2.1 cosProperty-1.2 cosTime-1.2.1 cosTransactions-1.3.1 crypto-3.6.3.1 debugger-4.1.2 dialyzer-2.9 diameter-1.11.2 edoc-0.7.18 eldap-1.2.1.1 erl_docgen-0.4.2 erl_interface-3.8.2 et-1.5.1 eunit-2.2.13 gs-1.6 hipe-3.15 ic-4.4 inets-6.2.4.1 jinterface-1.6.1 kernel-4.2 megaco-3.18 mnesia-4.13.4 observer-2.1.2 odbc-2.11.1 orber-3.8.1 os_mon-2.4 ose-1.1 otp_mibs-1.1 parsetools-2.1.1 percept-0.8.11 public_key-1.1.1 reltool-0.7 runtime_tools-1.9.3 sasl-2.7 snmp-5.2.2 ssh-4.2.2.6 ssl-7.3.3.2 syntax_tools-1.7 test_server-3.10 tools-2.8.3 typer-0.9.10 webtool-0.9.1 wx-1.6.1 xmerl-1.3.10 : OTP-18.3.4.10 : erts-7.3.1.5 # asn1-4.0.2 common_test-1.12.1.1 compiler-6.0.3.1 cosEvent-2.2 cosEventDomain-1.2 cosFileTransfer-1.2 cosNotification-1.2.1 cosProperty-1.2 cosTime-1.2.1 cosTransactions-1.3.1 crypto-3.6.3.1 debugger-4.1.2 dialyzer-2.9 diameter-1.11.2 edoc-0.7.18 eldap-1.2.1.1 erl_docgen-0.4.2 erl_interface-3.8.2 et-1.5.1 eunit-2.2.13 gs-1.6 hipe-3.15 ic-4.4 inets-6.2.4.1 jinterface-1.6.1 kernel-4.2 megaco-3.18 mnesia-4.13.4 observer-2.1.2 odbc-2.11.1 orber-3.8.1 os_mon-2.4 ose-1.1 otp_mibs-1.1 parsetools-2.1.1 percept-0.8.11 public_key-1.1.1 reltool-0.7 runtime_tools-1.9.3 sasl-2.7 snmp-5.2.2 ssh-4.2.2.6 ssl-7.3.3.2 stdlib-2.8 syntax_tools-1.7 test_server-3.10 tools-2.8.3 typer-0.9.10 webtool-0.9.1 wx-1.6.1 xmerl-1.3.10 : OTP-18.3.4.9 : ssh-4.2.2.6 # asn1-4.0.2 common_test-1.12.1.1 compiler-6.0.3.1 cosEvent-2.2 cosEventDomain-1.2 cosFileTransfer-1.2 cosNotification-1.2.1 cosProperty-1.2 cosTime-1.2.1 cosTransactions-1.3.1 crypto-3.6.3.1 debugger-4.1.2 dialyzer-2.9 diameter-1.11.2 edoc-0.7.18 eldap-1.2.1.1 erl_docgen-0.4.2 erl_interface-3.8.2 erts-7.3.1.4 et-1.5.1 eunit-2.2.13 gs-1.6 hipe-3.15 ic-4.4 inets-6.2.4.1 jinterface-1.6.1 kernel-4.2 megaco-3.18 mnesia-4.13.4 observer-2.1.2 odbc-2.11.1 orber-3.8.1 os_mon-2.4 ose-1.1 otp_mibs-1.1 parsetools-2.1.1 percept-0.8.11 public_key-1.1.1 reltool-0.7 runtime_tools-1.9.3 sasl-2.7 snmp-5.2.2 ssl-7.3.3.2 stdlib-2.8 syntax_tools-1.7 test_server-3.10 tools-2.8.3 typer-0.9.10 webtool-0.9.1 wx-1.6.1 xmerl-1.3.10 : OTP-18.3.4.8 : ssh-4.2.2.5 # asn1-4.0.2 common_test-1.12.1.1 compiler-6.0.3.1 cosEvent-2.2 cosEventDomain-1.2 cosFileTransfer-1.2 cosNotification-1.2.1 cosProperty-1.2 cosTime-1.2.1 cosTransactions-1.3.1 crypto-3.6.3.1 debugger-4.1.2 dialyzer-2.9 diameter-1.11.2 edoc-0.7.18 eldap-1.2.1.1 erl_docgen-0.4.2 erl_interface-3.8.2 erts-7.3.1.4 et-1.5.1 eunit-2.2.13 gs-1.6 hipe-3.15 ic-4.4 inets-6.2.4.1 jinterface-1.6.1 kernel-4.2 megaco-3.18 mnesia-4.13.4 observer-2.1.2 odbc-2.11.1 orber-3.8.1 os_mon-2.4 ose-1.1 otp_mibs-1.1 parsetools-2.1.1 percept-0.8.11 public_key-1.1.1 reltool-0.7 runtime_tools-1.9.3 sasl-2.7 snmp-5.2.2 ssl-7.3.3.2 stdlib-2.8 syntax_tools-1.7 test_server-3.10 tools-2.8.3 typer-0.9.10 webtool-0.9.1 wx-1.6.1 xmerl-1.3.10 : |