diff options
73 files changed, 1732 insertions, 403 deletions
diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index d38f29b8d1..412675fd2b 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -32,6 +32,288 @@ <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 8.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix bug for calls from hipe code to BIFs that disable GC + while yielding. Has been causing Dialyzer crashes on ARM + (and presumably all other non-intel platforms).</p> + <p> + Own Id: OTP-13724 Aux Id: PR-1116 </p> + </item> + <item> + <p> + Fix a bug where changing the current working directory of + the VM would not change the current working directory of + programs spawned using <c>erlang:open_port({spawn,""}, + ...)</c>.</p> + <p> + Own Id: OTP-13733 Aux Id: ERL-175 </p> + </item> + <item> + <p> + Fix a bug where disabling tracing from a process that had + return_to tracing enabled and was tracing on + <c>erlang:trace/3</c> would cause a segmentation fault.</p> + <p> + Own Id: OTP-13734</p> + </item> + <item> + <p> + Update all erts documentation to use simpler English, use + consistent terminology and be easier to navigate.</p> + <p> + Own Id: OTP-13740</p> + </item> + <item> + <p> + Add dirty schedulers to the microstate accounting + statistics.</p> + <p> + Own Id: OTP-13744</p> + </item> + <item> + <p> + Fixed dirty scheduler build support on 32-bit windows.</p> + <p> + Own Id: OTP-13759</p> + </item> + <item> + <p> + inet:getstat(Socket) on an SCTP socket returned 0 for + send stats. This bug has now been corrected. Reported by + systra as issue ERL-102 on bugs.erlang.org.</p> + <p> + Own Id: OTP-13773 Aux Id: ERL-102 </p> + </item> + <item> + <p> + AF_UNSPEC and unknown address families were misread by + UDP receive in prim_inet resulting in an exception. This + bug has now been corrected.</p> + <p> + Own Id: OTP-13775</p> + </item> + <item> + <p> + Sweep HiPE stack for literals during code purge.</p> + <p> + Own Id: OTP-13777 Aux Id: PR-1122 </p> + </item> + <item> + <p> + Fix bug in run_erl for OpenBSD that could cause it on + rare occations to exit without starting the program (erl) + at all.</p> + <p> + Own Id: OTP-13795</p> + </item> + <item> + <p> + Update build scripts to not make assumtions about where + env, cp and perl are located.</p> + <p> + Own Id: OTP-13800</p> + </item> + <item> + <p> + Fixed a bug where init:stop could deadlock if a process + with infinite shutdown timeout (e.g. a supervisor) + attempted to load code while terminating.</p> + <p> + Own Id: OTP-13802</p> + </item> + <item> + <p> + Fixed a segmentation fault on sparc CPUs when free'ing a + tracer module's state.</p> + <p> + Own Id: OTP-13803</p> + </item> + <item> + <p> + <c>fun</c>s was not properly handled during purge of a + module. This could cause a crash of the VM after a purge + of a module.</p> + <p> + Own Id: OTP-13809</p> + </item> + <item> + <p> + Fixed a memory leak when the process monitoring a port + crashed.</p> + <p> + Own Id: OTP-13818</p> + </item> + <item> + <p> + Fixed multiple dirty scheduler related tracing bugs.</p> + <p> + Own Id: OTP-13822</p> + </item> + <item> + <p> + Fix error handling in beam code runtime loader for a + number of cases when index and size fields got corrupted + (negative) values.</p> + <p> + Own Id: OTP-13848 Aux Id: ERL-216 </p> + </item> + <item> + <p> + Minor fix of dirty scheduler implementation.</p> + <p> + Own Id: OTP-13852</p> + </item> + <item> + <p> + Calls to <c>erl_drv_send_term()</c> or + <c>erl_drv_output_term()</c> from a non-scheduler thread + while the corresponding port was invalid caused the + emulator to enter an inconsistent state which eventually + caused an emulator crash.</p> + <p> + Own Id: OTP-13866</p> + </item> + <item> + <p> + Fix a rare race condition in <c>erlang:open_port({spawn, + ""}, ...)</c> that would result in the erl_child_setup + program aborting and cause the emulator to exit.</p> + <p> + Own Id: OTP-13868</p> + </item> + <item> + <p>Driver and NIF operations accessing processes or ports + could cause an emulator crash when used from + non-scheduler threads. Those operations are:</p> <list> + <item><c>erl_drv_send_term()</c></item> + <item><c>driver_send_term()</c></item> + <item><c>erl_drv_output_term()</c></item> + <item><c>driver_output_term()</c></item> + <item><c>enif_send()</c></item> + <item><c>enif_port_command()</c></item> </list> + <p> + Own Id: OTP-13869</p> + </item> + <item> + <p> + Fix start scripts generation dependency in Makefile</p> + <p> + Own Id: OTP-13871 Aux Id: ERL-241 </p> + </item> + <item> + <p> + The VM could crash if <c>erlang:get_stacktrace()</c> was + called after a rescheduled NIF had thrown an exception.</p> + <p> + Own Id: OTP-13877</p> + </item> + <item> + <p>Calling <c>code:delete/1</c> before a loading a module + with an on_load function, the old code would be + overwritten, causing a memory or a crash if NIFs were + involved. (Thanks to vans163 for reporting this bug.)</p> + <p> + Own Id: OTP-13893 Aux Id: ERL-240 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Improved accuracy of timeouts on MacOS X. This by setting + premature timeouts followed by a short actual timeout + during scheduler wait.</p> + <p> + Own Id: OTP-13698</p> + </item> + <item> + <p>Added the following symbolic time unit representations + to the <seealso + marker="erlang#type-time_unit"><c>erlang:time_unit()</c></seealso> + type:</p> <list> <item><c>second</c></item> + <item><c>millisecond</c></item> + <item><c>microsecond</c></item> + <item><c>nanosecond</c></item> </list> <p>The following + symbolic time unit representations are now + <em>deprecated</em>, but still part of the + <c>erlang:time_unit()</c> type:</p> <list> + <item><c>seconds</c></item> + <item><c>milli_seconds</c></item> + <item><c>micro_seconds</c></item> + <item><c>nano_seconds</c></item> </list> + <p> + Own Id: OTP-13735</p> + </item> + <item> + <p> + Fix maps hashing entropy of maps with maps keys.</p> + <p> + Own Id: OTP-13763 Aux Id: ERL-199 </p> + </item> + <item> + <p> + Improved dirty scheduler support. A purge of a module can + now be performed without having to wait for completion of + all ongoing dirty NIF calls.</p> + <p> + Note that when enabling support for dirty schedulers, a + new purge strategy will as of ERTS version 8.1 be + enabled. This new strategy is not fully backwards + compatible with the strategy used by default. For more + information see the documentation of <seealso + marker="erts:erlang#check_process_code/3"><c>erlang:check_process_code/3</c></seealso>.</p> + <p> + Own Id: OTP-13808 Aux Id: OTP-13833 </p> + </item> + <item> + <p> + A new purge strategy has been introduced. The new + strategy will by default be disabled during the OTP 19 + release, but will be the only strategy available as of + the OTP 20 release.</p> + <p> + The new strategy is slightly incompatible with the + strategy being used by default in OTP 19. Using the + default strategy, processes holding <c>fun</c>s that + refer to the module being purged either fail a soft + purge, or will be killed during a hard purge. The new + strategy completely ignores <c>fun</c>s. If <c>fun</c>s + referring to the code being purged exist, and are used + after a purge, an exception will be raised upon usage. + That is, the behavior will be exactly the same as the + case when a <c>fun</c> is received by a process after the + purge.</p> + <p> + The new strategy can optionally be enabled when building + OTP during OTP 19, and will automatically be enabled if + the runtime system is built with support for dirty + schedulers.</p> + <p> + For more information see the documentation of <seealso + marker="erts:erlang#check_process_code/3"><c>erlang:check_process_code/3</c></seealso>.</p> + <p> + Own Id: OTP-13833</p> + </item> + <item> + <p> + Fixed unnecessary overestimation of heap size need during + garbage collection.</p> + <p> + Own Id: OTP-13851</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 8.0.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/doc/src/run_erl.xml b/erts/doc/src/run_erl.xml index ad7b2c5b85..a9b6a7e2c6 100644 --- a/erts/doc/src/run_erl.xml +++ b/erts/doc/src/run_erl.xml @@ -49,7 +49,7 @@ <funcs> <func> <name>run_erl [-daemon] pipe_dir/ log_dir "exec command - [command_arguments]"</name> + arg1 arg2 ..."</name> <fsummary>Start the Erlang emulator without attached terminal.</fsummary> <desc> <p>Arguments:</p> @@ -92,11 +92,10 @@ </item> </list> </item> - <tag><c>"exec command [command_arguments]"</c></tag> + <tag><c>"exec command arg1 arg2 ..."</c></tag> <item> - <p>In the third argument, <c><![CDATA[command]]></c> is the - executable to execute where everything written to <c>stdin</c> - and <c>stdout</c> is logged to <c><![CDATA[log_dir]]></c>.</p> + <p>A space-separated string specifying the program to be executed. + The second field is typically a command name such as <c>erl</c>.</p> </item> </taglist> </desc> @@ -105,7 +104,7 @@ <section> <title>Notes concerning the Log Files</title> - <p>While running, <c>run_erl</c> (as stated earlier) sends all output, + <p>While running, <c>run_erl</c> sends all output, uninterpreted, to a log file. The file is named <c><![CDATA[erlang.log.N]]></c>, where <c>N</c> is an integer. When the log is "full" (default log size is 100 KB), <c>run_erl</c> starts to log diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index cb7278696f..75389107bb 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -3542,8 +3542,14 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1) else if (ERTS_IS_ATOM_STR("DbTable_words", BIF_ARG_1)) { /* Used by ets_SUITE (stdlib) */ size_t words = (sizeof(DbTable) + sizeof(Uint) - 1)/sizeof(Uint); - BIF_RET(make_small((Uint) words)); + Eterm* hp = HAlloc(BIF_P ,3); + BIF_RET(TUPLE2(hp, make_small((Uint) words), + erts_ets_hash_sizeof_ext_segtab())); } + else if (ERTS_IS_ATOM_STR("DbTable_meta", BIF_ARG_1)) { + /* Used by ets_SUITE (stdlib) */ + BIF_RET(erts_ets_get_meta_state(BIF_P)); + } else if (ERTS_IS_ATOM_STR("check_io_debug", BIF_ARG_1)) { /* Used by driver_SUITE (emulator) */ Uint sz, *szp; @@ -4277,6 +4283,10 @@ BIF_RETTYPE erts_debug_set_internal_state_2(BIF_ALIST_2) } BIF_RET(am_ok); } + else if (ERTS_IS_ATOM_STR("DbTable_meta", BIF_ARG_1)) { + /* Used by ets_SUITE (stdlib) */ + BIF_RET(erts_ets_restore_meta_state(BIF_P, BIF_ARG_2)); + } } BIF_ERROR(BIF_P, BADARG); diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c index 128a7b3865..485f68ca98 100644 --- a/erts/emulator/beam/erl_db.c +++ b/erts/emulator/beam/erl_db.c @@ -3978,6 +3978,36 @@ erts_ets_colliding_names(Process* p, Eterm name, Uint cnt) return list; } +/* + * For testing only + * Retreive meta table size state + */ +Eterm erts_ets_get_meta_state(Process* p) +{ + Eterm* hp = HAlloc(p, 3); + return TUPLE2(hp, + erts_ets_hash_get_memstate(p, &meta_pid_to_tab->hash), + erts_ets_hash_get_memstate(p, &meta_pid_to_fixed_tab->hash)); +} +/* + * For testing only + * Restore a previously retrieved meta table size state. + * We do this to suppress failed memory checks + * caused by the hysteresis of meta tables grow/shrink limits. + */ +Eterm erts_ets_restore_meta_state(Process* p, Eterm meta_state) +{ + Eterm* tv; + Eterm* hp; + if (!is_tuple_arity(meta_state, 2)) + return am_badarg; + + tv = tuple_val(meta_state); + hp = HAlloc(p, 3); + return TUPLE2(hp, + erts_ets_hash_restore_memstate(&meta_pid_to_tab->hash, tv[1]), + erts_ets_hash_restore_memstate(&meta_pid_to_fixed_tab->hash, tv[2])); +} #ifdef HARDDEBUG /* Here comes some debug functions */ diff --git a/erts/emulator/beam/erl_db.h b/erts/emulator/beam/erl_db.h index 1d26c49652..f7eb3dc45c 100644 --- a/erts/emulator/beam/erl_db.h +++ b/erts/emulator/beam/erl_db.h @@ -90,6 +90,8 @@ extern Export ets_select_continue_exp; extern erts_smp_atomic_t erts_ets_misc_mem_size; Eterm erts_ets_colliding_names(Process*, Eterm name, Uint cnt); +Eterm erts_ets_get_meta_state(Process* p); +Eterm erts_ets_restore_meta_state(Process* p, Eterm target_state); Uint erts_db_get_max_tabs(void); @@ -267,7 +269,5 @@ erts_db_free_nt(ErtsAlcType_t type, void *ptr, Uint size) #endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ -#undef ERTS_DB_ALC_MEM_UPDATE_ - #endif /* #if defined(ERTS_WANT_DB_INTERNAL__) && !defined(ERTS_HAVE_DB_INTERNAL__) */ diff --git a/erts/emulator/beam/erl_db_hash.c b/erts/emulator/beam/erl_db_hash.c index 5e6fe4f460..939ae57192 100644 --- a/erts/emulator/beam/erl_db_hash.c +++ b/erts/emulator/beam/erl_db_hash.c @@ -84,25 +84,28 @@ #include "erl_db_hash.h" -#ifdef MYDEBUG /* Will fail test case ets_SUITE:memory */ -# define IF_DEBUG(x) x -# define MY_ASSERT(x) ASSERT(x) -#else -# define IF_DEBUG(x) -# define MY_ASSERT(x) -#endif - /* * The following symbols can be manipulated to "tune" the linear hash array */ -#define CHAIN_LEN 6 /* Medium bucket chain len */ +#define GROW_LIMIT(NACTIVE) ((NACTIVE)*1) +#define SHRINK_LIMIT(NACTIVE) ((NACTIVE) / 2) -/* Number of slots per segment */ -#define SEGSZ_EXP 8 -#define SEGSZ (1 << SEGSZ_EXP) -#define SEGSZ_MASK (SEGSZ-1) +/* +** We want the first mandatory segment to be small (to reduce minimal footprint) +** and larger extra segments (to reduce number of alloc/free calls). +*/ -#define NSEG_1 2 /* Size of first segment table (must be at least 2) */ +/* Number of slots in first segment */ +#define FIRST_SEGSZ_EXP 8 +#define FIRST_SEGSZ (1 << FIRST_SEGSZ_EXP) +#define FIRST_SEGSZ_MASK (FIRST_SEGSZ - 1) + +/* Number of slots per extra segment */ +#define EXT_SEGSZ_EXP 11 +#define EXT_SEGSZ (1 << EXT_SEGSZ_EXP) +#define EXT_SEGSZ_MASK (EXT_SEGSZ-1) + +#define NSEG_1 (ErtsSizeofMember(DbTableHash,first_segtab) / sizeof(struct segment*)) #define NSEG_2 256 /* Size of second segment table */ #define NSEG_INC 128 /* Number of segments to grow after that */ @@ -123,7 +126,9 @@ #define NACTIVE(tb) ((int)erts_smp_atomic_read_nob(&(tb)->nactive)) #define NITEMS(tb) ((int)erts_smp_atomic_read_nob(&(tb)->common.nitems)) -#define BUCKET(tb, i) SEGTAB(tb)[(i) >> SEGSZ_EXP]->buckets[(i) & SEGSZ_MASK] +#define SLOT_IX_TO_SEG_IX(i) (((i)+(EXT_SEGSZ-FIRST_SEGSZ)) >> EXT_SEGSZ_EXP) + +#define BUCKET(tb, i) SEGTAB(tb)[SLOT_IX_TO_SEG_IX(i)]->buckets[(i) & EXT_SEGSZ_MASK] /* * When deleting a table, the number of records to delete. @@ -189,6 +194,7 @@ static ERTS_INLINE int add_fixed_deletion(DbTableHash* tb, int ix, #ifdef ERTS_SMP # define DB_HASH_LOCK_MASK (DB_HASH_LOCK_CNT-1) # define GET_LOCK(tb,hval) (&(tb)->locks->lck_vec[(hval) & DB_HASH_LOCK_MASK].lck) +# define GET_LOCK_MAYBE(tb,hval) ((tb)->common.is_thread_safe ? NULL : GET_LOCK(tb,hval)) /* Fine grained read lock */ static ERTS_INLINE erts_smp_rwmtx_t* RLOCK_HASH(DbTableHash* tb, HashValue hval) @@ -318,27 +324,23 @@ struct mp_info { /* A table segment */ struct segment { - HashDbTerm* buckets[SEGSZ]; -#ifdef MYDEBUG - int is_ext_segment; -#endif + HashDbTerm* buckets[1]; }; +#define SIZEOF_SEGMENT(N) \ + (offsetof(struct segment,buckets) + sizeof(HashDbTerm*)*(N)) -/* A segment that also contains a segment table */ -struct ext_segment { - struct segment s; /* The segment itself. Must be first */ - +/* An extended segment table */ +struct ext_segtab { +#ifdef ERTS_SMP + ErtsThrPrgrLaterOp lop; +#endif struct segment** prev_segtab; /* Used when table is shrinking */ - int nsegs; /* Size of segtab */ + int prev_nsegs; /* Size of prev_segtab */ + int nsegs; /* Size of this segtab */ struct segment* segtab[1]; /* The segment table */ }; -#define SIZEOF_EXTSEG(NSEGS) \ - (offsetof(struct ext_segment,segtab) + sizeof(struct segment*)*(NSEGS)) - -#if defined(DEBUG) || defined(VALGRIND) -# define EXTSEG(SEGTAB_PTR) \ - ((struct ext_segment*) (((char*)(SEGTAB_PTR)) - offsetof(struct ext_segment,segtab))) -#endif +#define SIZEOF_EXT_SEGTAB(NSEGS) \ + (offsetof(struct ext_segtab,segtab) + sizeof(struct segment*)*(NSEGS)) static ERTS_INLINE void SET_SEGTAB(DbTableHash* tb, @@ -348,49 +350,22 @@ static ERTS_INLINE void SET_SEGTAB(DbTableHash* tb, erts_smp_atomic_set_wb(&tb->segtab, (erts_aint_t) segtab); else erts_smp_atomic_set_nob(&tb->segtab, (erts_aint_t) segtab); -#ifdef VALGRIND - tb->top_ptr_to_segment_with_active_segtab = EXTSEG(segtab); -#endif } -/* How the table segments relate to each other: - - ext_segment: ext_segment: "plain" segment - #=================# #================# #=============# - | bucket[0] |<--+ +------->| bucket[256] | +->| bucket[512] | - | bucket[1] | | | | [257] | | | [513] | - : : | | : : | : : - | bucket[255] | | | | [511] | | | [767] | - |-----------------| | | |----------------| | #=============# - | prev_segtab=NULL| | | +--<---prev_segtab | | - | nsegs = 2 | | | | | nsegs = 256 | | -+->| segtab[0] -->-------+---|---|--<---segtab[0] |<-+ | -| | segtab[1] -->-----------+---|--<---segtab[1] | | | -| #=================# | | segtab[2] -->-----|--+ ext_segment: -| | : : | #================# -+----------------<---------------+ | segtab[255] ->----|----->| bucket[255*256]| - #================# | | | - | : : - | |----------------| - +----<---prev_segtab | - : : -*/ - /* ** Forward decl's (static functions) */ -static struct ext_segment* alloc_ext_seg(DbTableHash* tb, unsigned seg_ix, - struct segment** old_segtab); -static int alloc_seg(DbTableHash *tb); +static struct ext_segtab* alloc_ext_segtab(DbTableHash* tb, unsigned seg_ix); +static void alloc_seg(DbTableHash *tb); static int free_seg(DbTableHash *tb, int free_records); static HashDbTerm* next(DbTableHash *tb, Uint *iptr, erts_smp_rwmtx_t** lck_ptr, HashDbTerm *list); static HashDbTerm* search_list(DbTableHash* tb, Eterm key, HashValue hval, HashDbTerm *list); -static void shrink(DbTableHash* tb, int nactive); -static void grow(DbTableHash* tb, int nactive); +static void shrink(DbTableHash* tb, int nitems); +static void grow(DbTableHash* tb, int nitems); static Eterm build_term_list(Process* p, HashDbTerm* ptr1, HashDbTerm* ptr2, Uint sz, DbTableHash*); static int analyze_pattern(DbTableHash *tb, Eterm pattern, @@ -463,9 +438,10 @@ db_finalize_dbterm_hash(int cret, DbUpdateHandle* handle); static ERTS_INLINE void try_shrink(DbTableHash* tb) { int nactive = NACTIVE(tb); - if (nactive > SEGSZ && NITEMS(tb) < (nactive * CHAIN_LEN) + int nitems = NITEMS(tb); + if (nactive > FIRST_SEGSZ && nitems < SHRINK_LIMIT(nactive) && !IS_FIXED(tb)) { - shrink(tb, nactive); + shrink(tb, nitems); } } @@ -662,16 +638,20 @@ int db_create_hash(Process *p, DbTable *tbl) { DbTableHash *tb = &tbl->hash; - erts_smp_atomic_init_nob(&tb->szm, SEGSZ_MASK); - erts_smp_atomic_init_nob(&tb->nactive, SEGSZ); + erts_smp_atomic_init_nob(&tb->szm, FIRST_SEGSZ_MASK); + erts_smp_atomic_init_nob(&tb->nactive, FIRST_SEGSZ); erts_smp_atomic_init_nob(&tb->fixdel, (erts_aint_t)NULL); erts_smp_atomic_init_nob(&tb->segtab, (erts_aint_t)NULL); - SET_SEGTAB(tb, alloc_ext_seg(tb,0,NULL)->segtab); + SET_SEGTAB(tb, tb->first_segtab); tb->nsegs = NSEG_1; - tb->nslots = SEGSZ; + tb->nslots = FIRST_SEGSZ; + tb->first_segtab[0] = (struct segment*) erts_db_alloc(ERTS_ALC_T_DB_SEG, + (DbTable *) tb, + SIZEOF_SEGMENT(FIRST_SEGSZ)); + sys_memset(tb->first_segtab[0], 0, SIZEOF_SEGMENT(FIRST_SEGSZ)); - erts_smp_atomic_init_nob(&tb->is_resizing, 0); #ifdef ERTS_SMP + erts_smp_atomic_init_nob(&tb->is_resizing, 0); if (tb->common.type & DB_FINE_LOCKED) { erts_smp_rwmtx_opt_t rwmtx_opt = ERTS_SMP_RWMTX_OPT_DEFAULT_INITER; int i; @@ -686,7 +666,7 @@ int db_create_hash(Process *p, DbTable *tbl) erts_smp_rwmtx_init_opt_x(&tb->locks->lck_vec[i].lck, &rwmtx_opt, "db_hash_slot", make_small(i)); } - /* This important property is needed to guarantee that the buckets + /* This important property is needed to guarantee the two buckets * involved in a grow/shrink operation it protected by the same lock: */ ASSERT(erts_smp_atomic_read_nob(&tb->nactive) % DB_HASH_LOCK_CNT == 0); @@ -862,8 +842,8 @@ Lnew: WUNLOCK_HASH(lck); { int nactive = NACTIVE(tb); - if (nitems > nactive * (CHAIN_LEN+1) && !IS_FIXED(tb)) { - grow(tb, nactive); + if (nitems > GROW_LIMIT(nactive) && !IS_FIXED(tb)) { + grow(tb, nitems); } } CHECK_TABLES(); @@ -1514,6 +1494,7 @@ static int db_select_chunk_hash(Process *p, DbTable *tbl, match_list = CONS(hp, match_res, match_list); ++got; } + --num_left; } current = current->next; } @@ -1531,7 +1512,6 @@ static int db_select_chunk_hash(Process *p, DbTable *tbl, } } else { /* Key is variable */ - --num_left; if ((slot_ix=next_slot(tb,slot_ix,&lck)) == 0) { slot_ix = -1; @@ -2250,12 +2230,12 @@ static int db_free_table_continue_hash(DbTable *tbl) done /= 2; while(tb->nslots != 0) { - free_seg(tb, 1); + done += 1 + EXT_SEGSZ/64 + free_seg(tb, 1); /* * If we have done enough work, get out here. */ - if (++done >= (DELETE_RECORD_LIMIT / CHAIN_LEN / SEGSZ)) { + if (done >= DELETE_RECORD_LIMIT) { return 0; /* Not done */ } } @@ -2414,90 +2394,81 @@ static int analyze_pattern(DbTableHash *tb, Eterm pattern, return DB_ERROR_NONE; } -static struct ext_segment* alloc_ext_seg(DbTableHash* tb, unsigned seg_ix, - struct segment** old_segtab) +static struct ext_segtab* alloc_ext_segtab(DbTableHash* tb, unsigned seg_ix) { - int nsegs; - struct ext_segment* eseg; + struct segment** old_segtab = SEGTAB(tb); + int nsegs = 0; + struct ext_segtab* est; + ASSERT(seg_ix >= NSEG_1); switch (seg_ix) { - case 0: nsegs = NSEG_1; break; - case 1: nsegs = NSEG_2; break; - default: nsegs = seg_ix + NSEG_INC; break; - } - eseg = (struct ext_segment*) erts_db_alloc_fnf(ERTS_ALC_T_DB_SEG, - (DbTable *) tb, - SIZEOF_EXTSEG(nsegs)); - ASSERT(eseg != NULL); - sys_memset(&eseg->s, 0, sizeof(struct segment)); - IF_DEBUG(eseg->s.is_ext_segment = 1); - eseg->prev_segtab = old_segtab; - eseg->nsegs = nsegs; - if (old_segtab) { - ASSERT(nsegs > tb->nsegs); - sys_memcpy(eseg->segtab, old_segtab, tb->nsegs*sizeof(struct segment*)); - } + case NSEG_1: nsegs = NSEG_2; break; + default: nsegs = seg_ix + NSEG_INC; break; + } + ASSERT(nsegs > tb->nsegs); + est = (struct ext_segtab*) erts_db_alloc(ERTS_ALC_T_DB_SEG, + (DbTable *) tb, + SIZEOF_EXT_SEGTAB(nsegs)); + est->nsegs = nsegs; + est->prev_segtab = old_segtab; + est->prev_nsegs = tb->nsegs; + sys_memcpy(est->segtab, old_segtab, tb->nsegs*sizeof(struct segment*)); #ifdef DEBUG - sys_memset(&eseg->segtab[seg_ix], 0, (nsegs-seg_ix)*sizeof(struct segment*)); + sys_memset(&est->segtab[seg_ix], 0, (nsegs-seg_ix)*sizeof(struct segment*)); #endif - eseg->segtab[seg_ix] = &eseg->s; - return eseg; + return est; } /* Extend table with one new segment */ -static int alloc_seg(DbTableHash *tb) +static void alloc_seg(DbTableHash *tb) { - int seg_ix = tb->nslots >> SEGSZ_EXP; - - if (seg_ix+1 == tb->nsegs) { /* New segtab needed (extended segment) */ - struct segment** segtab = SEGTAB(tb); - struct ext_segment* seg = alloc_ext_seg(tb, seg_ix, segtab); - if (seg == NULL) return 0; - segtab[seg_ix] = &seg->s; - /* We don't use the new segtab until next call (see "shrink race") */ - } - else { /* Just a new plain segment */ - struct segment** segtab; - if (seg_ix == tb->nsegs) { /* Time to start use segtab from last call */ - struct ext_segment* eseg; - eseg = (struct ext_segment*) SEGTAB(tb)[seg_ix-1]; - MY_ASSERT(eseg!=NULL && eseg->s.is_ext_segment); - SET_SEGTAB(tb, eseg->segtab); - tb->nsegs = eseg->nsegs; - } - ASSERT(seg_ix < tb->nsegs); - segtab = SEGTAB(tb); - ASSERT(segtab[seg_ix] == NULL); - segtab[seg_ix] = (struct segment*) erts_db_alloc_fnf(ERTS_ALC_T_DB_SEG, - (DbTable *) tb, - sizeof(struct segment)); - if (segtab[seg_ix] == NULL) return 0; - sys_memset(segtab[seg_ix], 0, sizeof(struct segment)); - } - tb->nslots += SEGSZ; - return 1; + int seg_ix = SLOT_IX_TO_SEG_IX(tb->nslots); + struct segment** segtab; + + ASSERT(seg_ix > 0); + if (seg_ix == tb->nsegs) { /* New segtab needed */ + struct ext_segtab* est = alloc_ext_segtab(tb, seg_ix); + SET_SEGTAB(tb, est->segtab); + tb->nsegs = est->nsegs; + } + ASSERT(seg_ix < tb->nsegs); + segtab = SEGTAB(tb); + segtab[seg_ix] = (struct segment*) erts_db_alloc(ERTS_ALC_T_DB_SEG, + (DbTable *) tb, + SIZEOF_SEGMENT(EXT_SEGSZ)); + sys_memset(segtab[seg_ix], 0, SIZEOF_SEGMENT(EXT_SEGSZ)); + tb->nslots += EXT_SEGSZ; +} + +#ifdef ERTS_SMP +static void dealloc_ext_segtab(void* lop_data) +{ + struct ext_segtab* est = (struct ext_segtab*) lop_data; + + erts_free(ERTS_ALC_T_DB_SEG, est); } +#endif /* Shrink table by freeing the top segment ** free_records: 1=free any records in segment, 0=assume segment is empty */ static int free_seg(DbTableHash *tb, int free_records) { - const int seg_ix = (tb->nslots >> SEGSZ_EXP) - 1; + const int seg_ix = SLOT_IX_TO_SEG_IX(tb->nslots) - 1; struct segment** const segtab = SEGTAB(tb); - struct ext_segment* const top = (struct ext_segment*) segtab[seg_ix]; - int bytes; + struct segment* const segp = segtab[seg_ix]; + Uint seg_sz; int nrecords = 0; - ASSERT(top != NULL); + ASSERT(segp != NULL); #ifndef DEBUG if (free_records) #endif { - int i; - for (i=0; i<SEGSZ; ++i) { - HashDbTerm* p = top->s.buckets[i]; + int i = (seg_ix == 0) ? FIRST_SEGSZ : EXT_SEGSZ; + while (i--) { + HashDbTerm* p = segp->buckets[i]; while(p != 0) { HashDbTerm* nxt = p->next; ASSERT(free_records); /* segment not empty as assumed? */ @@ -2507,55 +2478,46 @@ static int free_seg(DbTableHash *tb, int free_records) } } } - - /* The "shrink race": - * We must avoid deallocating an extended segment while its segtab may - * still be used by other threads. - * The trick is to stop use a segtab one call earlier. That is, stop use - * a segtab when the segment above it is deallocated. When the segtab is - * later deallocated, it has not been used for a very long time. - * It is even theoretically safe as we have by then rehashed the entire - * segment, seizing *all* locks, so there cannot exist any retarded threads - * still hanging in BUCKET macro with an old segtab pointer. - * For this to work, we must of course allocate a new segtab one call - * earlier in alloc_seg() as well. And this is also the reason why - * the minimum size of the first segtab is 2 and not 1 (NSEG_1). - */ - if (seg_ix == tb->nsegs-1 || seg_ix==0) { /* Dealloc extended segment */ - MY_ASSERT(top->s.is_ext_segment); - ASSERT(segtab != top->segtab || seg_ix==0); - bytes = SIZEOF_EXTSEG(top->nsegs); - } - else { /* Dealloc plain segment */ - struct ext_segment* newtop = (struct ext_segment*) segtab[seg_ix-1]; - MY_ASSERT(!top->s.is_ext_segment); - - if (segtab == newtop->segtab) { /* New top segment is extended */ - MY_ASSERT(newtop->s.is_ext_segment); - if (newtop->prev_segtab != NULL) { - /* Time to use a smaller segtab */ - SET_SEGTAB(tb, newtop->prev_segtab); - tb->nsegs = seg_ix; - ASSERT(tb->nsegs == EXTSEG(SEGTAB(tb))->nsegs); - } - else { - ASSERT(NSEG_1 > 2 && seg_ix==1); - } - } - bytes = sizeof(struct segment); + if (seg_ix >= NSEG_1) { + struct ext_segtab* est = ErtsContainerStruct_(segtab,struct ext_segtab,segtab); + + if (seg_ix == est->prev_nsegs) { /* Dealloc extended segtab */ + ASSERT(est->prev_segtab != NULL); + SET_SEGTAB(tb, est->prev_segtab); + tb->nsegs = est->prev_nsegs; + +#ifdef ERTS_SMP + if (!tb->common.is_thread_safe) { + /* + * Table is doing a graceful shrink operation and we must avoid + * deallocating this segtab while it may still be read by other + * threads. Schedule deallocation with thread progress to make + * sure no lingering threads are still hanging in BUCKET macro + * with an old segtab pointer. + */ + Uint sz = SIZEOF_EXT_SEGTAB(est->nsegs); + ASSERT(sz == ERTS_ALC_DBG_BLK_SZ(est)); + ERTS_DB_ALC_MEM_UPDATE_(tb, sz, 0); + erts_schedule_thr_prgr_later_cleanup_op(dealloc_ext_segtab, + est, + &est->lop, + sz); + } + else +#endif + erts_db_free(ERTS_ALC_T_DB_SEG, (DbTable*)tb, est, + SIZEOF_EXT_SEGTAB(est->nsegs)); + } } + seg_sz = (seg_ix == 0) ? FIRST_SEGSZ : EXT_SEGSZ; + erts_db_free(ERTS_ALC_T_DB_SEG, (DbTable *)tb, segp, SIZEOF_SEGMENT(seg_sz)); - erts_db_free(ERTS_ALC_T_DB_SEG, (DbTable *)tb, - (void*)top, bytes); #ifdef DEBUG - if (seg_ix > 0) { - segtab[seg_ix] = NULL; - } else { - SET_SEGTAB(tb, NULL); - } + if (seg_ix < tb->nsegs) + SEGTAB(tb)[seg_ix] = NULL; #endif - tb->nslots -= SEGSZ; + tb->nslots -= seg_sz; ASSERT(tb->nslots >= 0); return nrecords; } @@ -2604,104 +2566,111 @@ static Eterm build_term_list(Process* p, HashDbTerm* ptr1, HashDbTerm* ptr2, static ERTS_INLINE int begin_resizing(DbTableHash* tb) { +#ifdef ERTS_SMP if (DB_USING_FINE_LOCKING(tb)) - return !erts_smp_atomic_xchg_acqb(&tb->is_resizing, 1); - else { - if (erts_smp_atomic_read_nob(&tb->is_resizing)) - return 0; - erts_smp_atomic_set_nob(&tb->is_resizing, 1); - return 1; - } + return !erts_atomic_xchg_acqb(&tb->is_resizing, 1); + else + ERTS_LC_ASSERT(erts_lc_rwmtx_is_rwlocked(&tb->common.rwlock)); +#endif + return 1; } static ERTS_INLINE void done_resizing(DbTableHash* tb) { +#ifdef ERTS_SMP if (DB_USING_FINE_LOCKING(tb)) - erts_smp_atomic_set_relb(&tb->is_resizing, 0); - else - erts_smp_atomic_set_nob(&tb->is_resizing, 0); + erts_atomic_set_relb(&tb->is_resizing, 0); +#endif } -/* Grow table with one new bucket. +/* Grow table with one or more new buckets. ** Allocate new segment if needed. */ -static void grow(DbTableHash* tb, int nactive) +static void grow(DbTableHash* tb, int nitems) { HashDbTerm** pnext; HashDbTerm** to_pnext; HashDbTerm* p; erts_smp_rwmtx_t* lck; - int from_ix; + int nactive; + int from_ix, to_ix; int szm; + int loop_limit = 5; - if (!begin_resizing(tb)) - return; /* already in progress */ - if (NACTIVE(tb) != nactive) { - goto abort; /* already done (race) */ - } - - /* Ensure that the slot nactive exists */ - if (nactive == tb->nslots) { - /* Time to get a new segment */ - ASSERT((nactive & SEGSZ_MASK) == 0); - if (!alloc_seg(tb)) goto abort; - } - ASSERT(nactive < tb->nslots); + do { + if (!begin_resizing(tb)) + return; /* already in progress */ + nactive = NACTIVE(tb); + if (nitems <= GROW_LIMIT(nactive)) { + goto abort; /* already done (race) */ + } - szm = erts_smp_atomic_read_nob(&tb->szm); - if (nactive <= szm) { - from_ix = nactive & (szm >> 1); - } else { - ASSERT(nactive == szm+1); - from_ix = 0; - szm = (szm<<1) | 1; - } + /* Ensure that the slot nactive exists */ + if (nactive == tb->nslots) { + /* Time to get a new segment */ + ASSERT(((nactive-FIRST_SEGSZ) & EXT_SEGSZ_MASK) == 0); + alloc_seg(tb); + } + ASSERT(nactive < tb->nslots); - lck = WLOCK_HASH(tb, from_ix); - /* Now a final double check (with the from_ix lock held) - * that we did not get raced by a table fixer. - */ - if (IS_FIXED(tb)) { - WUNLOCK_HASH(lck); - goto abort; - } - erts_smp_atomic_inc_nob(&tb->nactive); - if (from_ix == 0) { - if (DB_USING_FINE_LOCKING(tb)) - erts_smp_atomic_set_relb(&tb->szm, szm); - else - erts_smp_atomic_set_nob(&tb->szm, szm); - } - done_resizing(tb); + szm = erts_smp_atomic_read_nob(&tb->szm); + if (nactive <= szm) { + from_ix = nactive & (szm >> 1); + } else { + ASSERT(nactive == szm+1); + from_ix = 0; + szm = (szm<<1) | 1; + } + to_ix = nactive; + + lck = WLOCK_HASH(tb, from_ix); + ERTS_SMP_ASSERT(lck == GET_LOCK_MAYBE(tb,to_ix)); + /* Now a final double check (with the from_ix lock held) + * that we did not get raced by a table fixer. + */ + if (IS_FIXED(tb)) { + WUNLOCK_HASH(lck); + goto abort; + } + erts_smp_atomic_set_nob(&tb->nactive, ++nactive); + if (from_ix == 0) { + if (DB_USING_FINE_LOCKING(tb)) + erts_smp_atomic_set_relb(&tb->szm, szm); + else + erts_smp_atomic_set_nob(&tb->szm, szm); + } + done_resizing(tb); + + /* Finally, let's split the bucket. We try to do it in a smart way + to keep link order and avoid unnecessary updates of next-pointers */ + pnext = &BUCKET(tb, from_ix); + p = *pnext; + to_pnext = &BUCKET(tb, to_ix); + while (p != NULL) { + if (p->hvalue == INVALID_HASH) { /* rare but possible with fine locking */ + *pnext = p->next; + free_term(tb, p); + p = *pnext; + } + else { + int ix = p->hvalue & szm; + if (ix != from_ix) { + ASSERT(ix == (from_ix ^ ((szm+1)>>1))); + *to_pnext = p; + /* Swap "from" and "to": */ + from_ix = ix; + to_pnext = pnext; + } + pnext = &p->next; + p = *pnext; + } + } + *to_pnext = NULL; + WUNLOCK_HASH(lck); - /* Finally, let's split the bucket. We try to do it in a smart way - to keep link order and avoid unnecessary updates of next-pointers */ - pnext = &BUCKET(tb, from_ix); - p = *pnext; - to_pnext = &BUCKET(tb, nactive); - while (p != NULL) { - if (p->hvalue == INVALID_HASH) { /* rare but possible with fine locking */ - *pnext = p->next; - free_term(tb, p); - p = *pnext; - } - else { - int ix = p->hvalue & szm; - if (ix != from_ix) { - ASSERT(ix == (from_ix ^ ((szm+1)>>1))); - *to_pnext = p; - /* Swap "from" and "to": */ - from_ix = ix; - to_pnext = pnext; - } - pnext = &p->next; - p = *pnext; - } - } - *to_pnext = NULL; + }while (--loop_limit && nitems > GROW_LIMIT(nactive)); - WUNLOCK_HASH(lck); return; abort: @@ -2712,60 +2681,78 @@ abort: /* Shrink table by joining top bucket. ** Remove top segment if it gets empty. */ -static void shrink(DbTableHash* tb, int nactive) -{ - if (!begin_resizing(tb)) - return; /* already in progress */ - if (NACTIVE(tb) == nactive) { - erts_smp_rwmtx_t* lck; - int src_ix = nactive - 1; - int low_szm = erts_smp_atomic_read_nob(&tb->szm) >> 1; - int dst_ix = src_ix & low_szm; - - ASSERT(dst_ix < src_ix); - ASSERT(nactive > SEGSZ); - lck = WLOCK_HASH(tb, dst_ix); - /* Double check for racing table fixers */ - if (!IS_FIXED(tb)) { - HashDbTerm** src_bp = &BUCKET(tb, src_ix); - HashDbTerm** dst_bp = &BUCKET(tb, dst_ix); - HashDbTerm** bp = src_bp; - - /* Q: Why join lists by appending "dst" at the end of "src"? - A: Must step through "src" anyway to purge pseudo deleted. */ - while(*bp != NULL) { - if ((*bp)->hvalue == INVALID_HASH) { - HashDbTerm* deleted = *bp; - *bp = deleted->next; - free_term(tb, deleted); - } else { - bp = &(*bp)->next; - } - } - *bp = *dst_bp; - *dst_bp = *src_bp; - *src_bp = NULL; - - erts_smp_atomic_set_nob(&tb->nactive, src_ix); - if (dst_ix == 0) { - erts_smp_atomic_set_relb(&tb->szm, low_szm); - } - WUNLOCK_HASH(lck); - - if (tb->nslots - src_ix >= SEGSZ) { - free_seg(tb, 0); - } - } - else { - WUNLOCK_HASH(lck); - } +static void shrink(DbTableHash* tb, int nitems) +{ + HashDbTerm** src_bp; + HashDbTerm** dst_bp; + HashDbTerm** bp; + erts_smp_rwmtx_t* lck; + int src_ix, dst_ix, low_szm; + int nactive; + int loop_limit = 5; - } - /*else already done */ + do { + if (!begin_resizing(tb)) + return; /* already in progress */ + nactive = NACTIVE(tb); + if (!(nactive > FIRST_SEGSZ && nitems < SHRINK_LIMIT(nactive))) { + goto abort; /* already done (race) */ + } + src_ix = nactive - 1; + low_szm = erts_smp_atomic_read_nob(&tb->szm) >> 1; + dst_ix = src_ix & low_szm; + + ASSERT(dst_ix < src_ix); + ASSERT(nactive > FIRST_SEGSZ); + lck = WLOCK_HASH(tb, dst_ix); + ERTS_SMP_ASSERT(lck == GET_LOCK_MAYBE(tb,src_ix)); + /* Double check for racing table fixers */ + if (IS_FIXED(tb)) { + WUNLOCK_HASH(lck); + goto abort; + } + + src_bp = &BUCKET(tb, src_ix); + dst_bp = &BUCKET(tb, dst_ix); + bp = src_bp; + + /* + * We join lists by appending "dst" at the end of "src" + * as we must step through "src" anyway to purge pseudo deleted. + */ + while(*bp != NULL) { + if ((*bp)->hvalue == INVALID_HASH) { + HashDbTerm* deleted = *bp; + *bp = deleted->next; + free_term(tb, deleted); + } else { + bp = &(*bp)->next; + } + } + *bp = *dst_bp; + *dst_bp = *src_bp; + *src_bp = NULL; + + nactive = src_ix; + erts_smp_atomic_set_nob(&tb->nactive, nactive); + if (dst_ix == 0) { + erts_smp_atomic_set_relb(&tb->szm, low_szm); + } + WUNLOCK_HASH(lck); + + if (tb->nslots - src_ix >= EXT_SEGSZ) { + free_seg(tb, 0); + } + done_resizing(tb); + + } while (--loop_limit + && nactive > FIRST_SEGSZ && nitems < SHRINK_LIMIT(nactive)); + return; + +abort: done_resizing(tb); } - /* Search a list of tuples for a matching key */ static HashDbTerm* search_list(DbTableHash* tb, Eterm key, @@ -2931,8 +2918,8 @@ db_finalize_dbterm_hash(int cret, DbUpdateHandle* handle) WUNLOCK_HASH(lck); nactive = NACTIVE(tb); - if (nitems > nactive * (CHAIN_LEN + 1) && !IS_FIXED(tb)) { - grow(tb, nactive); + if (nitems > GROW_LIMIT(nactive) && !IS_FIXED(tb)) { + grow(tb, nitems); } } else { WUNLOCK_HASH(lck); @@ -3016,6 +3003,51 @@ void db_calc_stats_hash(DbTableHash* tb, DbHashStats* stats) stats->std_dev_expected = sqrt(stats->avg_chain_len * (1 - 1.0/NACTIVE(tb))); stats->kept_items = kept_items; } + +/* For testing only */ +Eterm erts_ets_hash_sizeof_ext_segtab(void) +{ + return make_small(((SIZEOF_EXT_SEGTAB(0)-1) / sizeof(UWord)) + 1); +} +/* For testing only */ +Eterm erts_ets_hash_get_memstate(Process* p, DbTableHash* tb) +{ + Eterm seg_cnt; + while (!begin_resizing(tb)) + /*spinn*/; + + seg_cnt = make_small(SLOT_IX_TO_SEG_IX(tb->nslots)); + done_resizing(tb); + return seg_cnt; +} +/* For testing only */ +Eterm erts_ets_hash_restore_memstate(DbTableHash* tb, Eterm memstate) +{ + int seg_cnt, target; + + if (!is_small(memstate)) + return make_small(__LINE__); + + target = signed_val(memstate); + if (target < 1) + return make_small(__LINE__); + while (1) { + while (!begin_resizing(tb)) + /*spin*/; + seg_cnt = SLOT_IX_TO_SEG_IX(tb->nslots); + done_resizing(tb); + + if (target == seg_cnt) + return am_ok; + if (IS_FIXED(tb)) + return make_small(__LINE__); + if (target < seg_cnt) + shrink(tb, 0); + else + grow(tb, INT_MAX); + } +} + #ifdef HARDDEBUG void db_check_table_hash(DbTable *tbl) @@ -3024,7 +3056,7 @@ void db_check_table_hash(DbTable *tbl) HashDbTerm* list; int j; - for (j = 0; j < tb->nactive; j++) { + for (j = 0; j < NACTIVE(tb); j++) { if ((list = BUCKET(tb,j)) != 0) { while (list != 0) { if (!is_tuple(make_tuple(list->dbterm.tpl))) { diff --git a/erts/emulator/beam/erl_db_hash.h b/erts/emulator/beam/erl_db_hash.h index e654363cd5..6d25c73549 100644 --- a/erts/emulator/beam/erl_db_hash.h +++ b/erts/emulator/beam/erl_db_hash.h @@ -50,23 +50,23 @@ typedef struct db_table_hash_fine_locks { typedef struct db_table_hash { DbTableCommon common; - erts_smp_atomic_t segtab; /* The segment table (struct segment**) */ + /* SMP: szm and nactive are write-protected by is_resizing or table write lock */ erts_smp_atomic_t szm; /* current size mask. */ - + erts_smp_atomic_t nactive; /* Number of "active" slots */ + + erts_smp_atomic_t segtab; /* The segment table (struct segment**) */ + struct segment* first_segtab[1]; + /* SMP: nslots and nsegs are protected by is_resizing or table write lock */ int nslots; /* Total number of slots */ int nsegs; /* Size of segment table */ /* List of slots where elements have been deleted while table was fixed */ erts_smp_atomic_t fixdel; /* (FixedDeletion*) */ - erts_smp_atomic_t nactive; /* Number of "active" slots */ - erts_smp_atomic_t is_resizing; /* grow/shrink in progress */ #ifdef ERTS_SMP + erts_smp_atomic_t is_resizing; /* grow/shrink in progress */ DbTableHashFineLocks* locks; #endif -#ifdef VALGRIND - struct ext_segment* top_ptr_to_segment_with_active_segtab; -#endif } DbTableHash; @@ -109,5 +109,8 @@ typedef struct { }DbHashStats; void db_calc_stats_hash(DbTableHash* tb, DbHashStats*); +Eterm erts_ets_hash_sizeof_ext_segtab(void); +Eterm erts_ets_hash_get_memstate(Process*, DbTableHash* tb); +Eterm erts_ets_hash_restore_memstate(DbTableHash* tb, Eterm memstate); #endif /* _DB_HASH_H */ diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h index dfe82cab44..16e6c33367 100644 --- a/erts/emulator/beam/sys.h +++ b/erts/emulator/beam/sys.h @@ -99,6 +99,12 @@ #define ErtsContainerStruct(ptr, type, member) \ ((type *)((char *)(1 ? (ptr) : &((type *)0)->member) - offsetof(type, member))) +/* Use this variant when the member is an array */ +#define ErtsContainerStruct_(ptr, type, memberv) \ + ((type *)((char *)(1 ? (ptr) : ((type *)0)->memberv) - offsetof(type, memberv))) + +#define ErtsSizeofMember(type, member) sizeof(((type *)0)->member) + #if defined (__WIN32__) # include "erl_win_sys.h" #else diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 254d3baeb1..1885338ce5 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -777,6 +777,7 @@ static int is_nonzero(const char *s, size_t n) #define INET_LOPT_NETNS 38 /* Network namespace pathname */ #define INET_LOPT_TCP_SHOW_ECONNRESET 39 /* tell user about incoming RST */ #define INET_LOPT_LINE_DELIM 40 /* Line delimiting char */ +#define INET_OPT_TCLASS 41 /* IPv6 transport class */ /* SCTP options: a separate range, from 100: */ #define SCTP_OPT_RTOINFO 100 #define SCTP_OPT_ASSOCINFO 101 @@ -1330,6 +1331,7 @@ static ErlDrvTermData am_reuseaddr; static ErlDrvTermData am_dontroute; static ErlDrvTermData am_priority; static ErlDrvTermData am_tos; +static ErlDrvTermData am_tclass; static ErlDrvTermData am_ipv6_v6only; static ErlDrvTermData am_netns; #endif @@ -3720,6 +3722,7 @@ static void inet_init_sctp(void) { INIT_ATOM(dontroute); INIT_ATOM(priority); INIT_ATOM(tos); + INIT_ATOM(tclass); INIT_ATOM(ipv6_v6only); INIT_ATOM(netns); @@ -6228,6 +6231,15 @@ static int inet_set_opts(inet_descriptor* desc, char* ptr, int len) #else continue; #endif +#if defined(IPV6_TCLASS) && defined(SOL_IPV6) + case INET_OPT_TCLASS: + proto = SOL_IPV6; + type = IPV6_TCLASS; + propagate = 1; + DEBUGF(("inet_set_opts(%ld): s=%d, IPV6_TCLASS=%d\r\n", + (long)desc->port, desc->s, ival)); + break; +#endif case TCP_OPT_NODELAY: proto = IPPROTO_TCP; @@ -6661,6 +6673,19 @@ static int sctp_set_opts(inet_descriptor* desc, char* ptr, int len) continue; /* Option not supported -- ignore it */ # endif +# if defined(IPV6_TCLASS) && defined(SOL_IPV6) + case INET_OPT_TCLASS: + { + arg.ival= get_int32 (curr); curr += 4; + proto = SOL_IPV6; + type = IPV6_TCLASS; + arg_ptr = (char*) (&arg.ival); + arg_sz = sizeof ( arg.ival); + break; + } +# endif + + case INET_OPT_IPV6_V6ONLY: # if HAVE_DECL_IPV6_V6ONLY { @@ -7162,6 +7187,15 @@ static ErlDrvSSizeT inet_fill_opts(inet_descriptor* desc, put_int32(0, ptr); continue; #endif + case INET_OPT_TCLASS: +#if defined(IPV6_TCLASS) && defined(SOL_IPV6) + proto = SOL_IPV6; + type = IPV6_TCLASS; + break; +#else + TRUNCATE_TO(0,ptr); + continue; +#endif case INET_OPT_REUSEADDR: type = SO_REUSEADDR; break; @@ -7556,6 +7590,7 @@ static ErlDrvSSizeT sctp_fill_opts(inet_descriptor* desc, case INET_OPT_DONTROUTE: case INET_OPT_PRIORITY : case INET_OPT_TOS : + case INET_OPT_TCLASS : case INET_OPT_IPV6_V6ONLY: case SCTP_OPT_AUTOCLOSE: case SCTP_OPT_MAXSEG : @@ -7629,6 +7664,19 @@ static ErlDrvSSizeT sctp_fill_opts(inet_descriptor* desc, continue; # endif } + case INET_OPT_TCLASS: + { +# if defined(IPV6_TCLASS) && defined(SOL_IPV6) + proto = SOL_IPV6; + type = IPV6_TCLASS; + is_int = 1; + tag = am_tclass; + break; +# else + /* Not supported -- ignore */ + continue; +# endif + } case INET_OPT_IPV6_V6ONLY: # if HAVE_DECL_IPV6_V6ONLY { diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex ae4861c336..1a573ce297 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/src/prim_inet.erl b/erts/preloaded/src/prim_inet.erl index bcf16402b0..61f727e8a4 100644 --- a/erts/preloaded/src/prim_inet.erl +++ b/erts/preloaded/src/prim_inet.erl @@ -347,7 +347,17 @@ accept_opts(L, S) -> case getopts(L, [active, nodelay, keepalive, delay_send, priority, tos]) of {ok, Opts} -> case setopts(S, Opts) of - ok -> {ok, S}; + ok -> + case getopts(L, [tclass]) of + {ok, []} -> + {ok, S}; + {ok, TClassOpts} -> + case setopts(S, TClassOpts) of + ok -> + {ok, S}; + Error -> close(S), Error + end + end; Error -> close(S), Error end; Error -> @@ -1196,6 +1206,7 @@ enc_opt(sndbuf) -> ?INET_OPT_SNDBUF; enc_opt(recbuf) -> ?INET_OPT_RCVBUF; enc_opt(priority) -> ?INET_OPT_PRIORITY; enc_opt(tos) -> ?INET_OPT_TOS; +enc_opt(tclass) -> ?INET_OPT_TCLASS; enc_opt(nodelay) -> ?TCP_OPT_NODELAY; enc_opt(multicast_if) -> ?UDP_OPT_MULTICAST_IF; enc_opt(multicast_ttl) -> ?UDP_OPT_MULTICAST_TTL; @@ -1255,6 +1266,7 @@ dec_opt(?INET_OPT_SNDBUF) -> sndbuf; dec_opt(?INET_OPT_RCVBUF) -> recbuf; dec_opt(?INET_OPT_PRIORITY) -> priority; dec_opt(?INET_OPT_TOS) -> tos; +dec_opt(?INET_OPT_TCLASS) -> tclass; dec_opt(?TCP_OPT_NODELAY) -> nodelay; dec_opt(?UDP_OPT_MULTICAST_IF) -> multicast_if; dec_opt(?UDP_OPT_MULTICAST_TTL) -> multicast_ttl; @@ -1329,6 +1341,7 @@ type_opt_1(sndbuf) -> int; type_opt_1(recbuf) -> int; type_opt_1(priority) -> int; type_opt_1(tos) -> int; +type_opt_1(tclass) -> int; type_opt_1(nodelay) -> bool; type_opt_1(ipv6_v6only) -> bool; %% multicast diff --git a/erts/vsn.mk b/erts/vsn.mk index 95d0e7b08d..ad9148f61f 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -18,7 +18,7 @@ # %CopyrightEnd% # -VSN = 8.0.5 +VSN = 8.1 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/asn1/doc/src/notes.xml b/lib/asn1/doc/src/notes.xml index ac3d9c828e..499a7e40c3 100644 --- a/lib/asn1/doc/src/notes.xml +++ b/lib/asn1/doc/src/notes.xml @@ -32,6 +32,22 @@ <p>This document describes the changes made to the asn1 application.</p> +<section><title>Asn1 4.0.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Compiling multiple ASN.1 modules in the same directory + with parallel make (make -j) should now be safe.</p> + <p> + Own Id: OTP-13624</p> + </item> + </list> + </section> + +</section> + <section><title>Asn1 4.0.3</title> <section><title>Improvements and New Features</title> diff --git a/lib/asn1/vsn.mk b/lib/asn1/vsn.mk index 527af05da1..e4bf3e2236 100644 --- a/lib/asn1/vsn.mk +++ b/lib/asn1/vsn.mk @@ -1 +1 @@ -ASN1_VSN = 4.0.3 +ASN1_VSN = 4.0.4 diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index 32ae699c7a..7653670d30 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -33,6 +33,42 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.12.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + If the telnet server would pause during transmission of a + line of text before terminating the line, the + <c>ct_telnet:expect/3</c> function would print the line + twice in the test case HTML log. This problem has been + fixed.</p> + <p> + Own Id: OTP-13730 Aux Id: seq13135 </p> + </item> + <item> + <p> + The functions <c>ct:set_verbosity/2</c> and + <c>ct:get_verbosity/1</c> have been added in order to + make it possible for test cases, CT Hooks, or test + framework functions, to modify and read verbosity levels + for logging.</p> + <p> + Own Id: OTP-13841</p> + </item> + <item> + <p><c>make</c> (tools) and <c>ct_make</c> (common_test) + would crash if an Erlang source file contained a + <c>-warning()</c> directive.</p> + <p> + Own Id: OTP-13855</p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.12.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/common_test/vsn.mk b/lib/common_test/vsn.mk index c68750886a..ab5cfd7a80 100644 --- a/lib/common_test/vsn.mk +++ b/lib/common_test/vsn.mk @@ -1 +1 @@ -COMMON_TEST_VSN = 1.12.2 +COMMON_TEST_VSN = 1.12.3 diff --git a/lib/compiler/doc/src/notes.xml b/lib/compiler/doc/src/notes.xml index dd6b132a92..6aaf16e9a5 100644 --- a/lib/compiler/doc/src/notes.xml +++ b/lib/compiler/doc/src/notes.xml @@ -32,6 +32,43 @@ <p>This document describes the changes made to the Compiler application.</p> +<section><title>Compiler 7.0.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + If the compiler fails to write the BEAM file, it will now + report the reason of the error for the write operation.</p> + <p> + Own Id: OTP-13701</p> + </item> + <item> + <p> + Fixed an internal compiler error. (Thanks to Svilen + Ivanov for reporting this bug.)</p> + <p> + Own Id: OTP-13780 Aux Id: ERL-202 </p> + </item> + <item> + <p> + The compiler could crash when trying to compile a + complicated expression with multiple catches all on one + line . (Thanks to Thomas Arts for reporting this bug.)</p> + <p> + Own Id: OTP-13804 Aux Id: ERL-209 </p> + </item> + <item> + <p> + Eliminated a few internal compiler failures.</p> + <p> + Own Id: OTP-13863</p> + </item> + </list> + </section> + +</section> + <section><title>Compiler 7.0.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/compiler/vsn.mk b/lib/compiler/vsn.mk index 334784657e..87fde38f2b 100644 --- a/lib/compiler/vsn.mk +++ b/lib/compiler/vsn.mk @@ -1 +1 @@ -COMPILER_VSN = 7.0.1 +COMPILER_VSN = 7.0.2 diff --git a/lib/crypto/doc/src/notes.xml b/lib/crypto/doc/src/notes.xml index 56e165a1d1..4ae64e059e 100644 --- a/lib/crypto/doc/src/notes.xml +++ b/lib/crypto/doc/src/notes.xml @@ -31,6 +31,32 @@ </header> <p>This document describes the changes made to the Crypto application.</p> +<section><title>Crypto 3.7.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Crypto has been fixed to work against OpenSSL versions + with disabled DES ciphers. Correct spelling of cipher + algorithm 'des3_cfb' has been introduced; the previous + misspeling still works.</p> + <p> + Own Id: OTP-13783 Aux Id: ERL-203 </p> + </item> + <item> + <p> + The size of an internal array in crypto has been fixed to + not segfault when having all possible ciphers. Bug fix by + Duncan Overbruck.</p> + <p> + Own Id: OTP-13789 Aux Id: PR-1140 </p> + </item> + </list> + </section> + +</section> + <section><title>Crypto 3.7</title> <section><title>Improvements and New Features</title> diff --git a/lib/crypto/vsn.mk b/lib/crypto/vsn.mk index 96466869d1..bbee24554a 100644 --- a/lib/crypto/vsn.mk +++ b/lib/crypto/vsn.mk @@ -1 +1 @@ -CRYPTO_VSN = 3.7 +CRYPTO_VSN = 3.7.1 diff --git a/lib/debugger/doc/src/notes.xml b/lib/debugger/doc/src/notes.xml index 2e0d834269..93bc46ddbe 100644 --- a/lib/debugger/doc/src/notes.xml +++ b/lib/debugger/doc/src/notes.xml @@ -33,6 +33,22 @@ <p>This document describes the changes made to the Debugger application.</p> +<section><title>Debugger 4.2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Update build scripts to not make assumtions about where + env, cp and perl are located.</p> + <p> + Own Id: OTP-13800</p> + </item> + </list> + </section> + +</section> + <section><title>Debugger 4.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/debugger/vsn.mk b/lib/debugger/vsn.mk index dd496013cd..f5440865ef 100644 --- a/lib/debugger/vsn.mk +++ b/lib/debugger/vsn.mk @@ -1 +1 @@ -DEBUGGER_VSN = 4.2 +DEBUGGER_VSN = 4.2.1 diff --git a/lib/dialyzer/doc/src/notes.xml b/lib/dialyzer/doc/src/notes.xml index 6400072b1f..b0f0a9aef0 100644 --- a/lib/dialyzer/doc/src/notes.xml +++ b/lib/dialyzer/doc/src/notes.xml @@ -32,6 +32,29 @@ <p>This document describes the changes made to the Dialyzer application.</p> +<section><title>Dialyzer 3.0.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The translation of forms to types is improved for + opaque types in a few cases. </p> + <p> + Own Id: OTP-13682</p> + </item> + <item> + <p> Add warning suppression to compiler-generated case + statements. Warnings about clauses that cannot match and + are also compiler generated are suppressed unless none of + the clauses return. </p> + <p> + Own Id: OTP-13723 Aux Id: ERL-159, PR-1121 </p> + </item> + </list> + </section> + +</section> + <section><title>Dialyzer 3.0.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/dialyzer/vsn.mk b/lib/dialyzer/vsn.mk index b9a28afdd9..6723876208 100644 --- a/lib/dialyzer/vsn.mk +++ b/lib/dialyzer/vsn.mk @@ -1 +1 @@ -DIALYZER_VSN = 3.0.1 +DIALYZER_VSN = 3.0.2 diff --git a/lib/diameter/doc/src/notes.xml b/lib/diameter/doc/src/notes.xml index b1be7bdcf7..c2bbed2e5a 100644 --- a/lib/diameter/doc/src/notes.xml +++ b/lib/diameter/doc/src/notes.xml @@ -43,6 +43,31 @@ first.</p> <!-- ===================================================================== --> +<section><title>diameter 1.12.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Close diameter_tcp/sctp listening sockets at + diameter:stop_service/1.</p> + <p> + Broken by OTP-13611.</p> + <p> + Own Id: OTP-13787 Aux Id: OTP-13611 </p> + </item> + <item> + <p> + Update build scripts to not make assumtions about where + env, cp and perl are located.</p> + <p> + Own Id: OTP-13800</p> + </item> + </list> + </section> + +</section> + <section><title>diameter 1.12</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/edoc/doc/src/notes.xml b/lib/edoc/doc/src/notes.xml index ae8147c564..e6ad2c683f 100644 --- a/lib/edoc/doc/src/notes.xml +++ b/lib/edoc/doc/src/notes.xml @@ -32,6 +32,21 @@ <p>This document describes the changes made to the EDoc application.</p> +<section><title>Edoc 0.8</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Improve types and specs in OTP documentation generated + from Erlang source files. </p> + <p> + Own Id: OTP-13720 Aux Id: ERL-120 </p> + </item> + </list> + </section> + +</section> + <section><title>Edoc 0.7.19</title> <section><title>Improvements and New Features</title> diff --git a/lib/edoc/vsn.mk b/lib/edoc/vsn.mk index f38800b3e0..d3cc732e9c 100644 --- a/lib/edoc/vsn.mk +++ b/lib/edoc/vsn.mk @@ -1 +1 @@ -EDOC_VSN = 0.7.19 +EDOC_VSN = 0.8 diff --git a/lib/erl_docgen/doc/src/notes.xml b/lib/erl_docgen/doc/src/notes.xml index 21dc617d55..cf24161d43 100644 --- a/lib/erl_docgen/doc/src/notes.xml +++ b/lib/erl_docgen/doc/src/notes.xml @@ -31,7 +31,22 @@ </header> <p>This document describes the changes made to the <em>erl_docgen</em> application.</p> - <section><title>Erl_Docgen 0.5</title> + <section><title>Erl_Docgen 0.6</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Improve types and specs in OTP documentation generated + from Erlang source files. </p> + <p> + Own Id: OTP-13720 Aux Id: ERL-120 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Docgen 0.5</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/erl_docgen/vsn.mk b/lib/erl_docgen/vsn.mk index 98d1c3f7be..6489d26327 100644 --- a/lib/erl_docgen/vsn.mk +++ b/lib/erl_docgen/vsn.mk @@ -1 +1 @@ -ERL_DOCGEN_VSN = 0.5 +ERL_DOCGEN_VSN = 0.6 diff --git a/lib/erl_interface/doc/src/notes.xml b/lib/erl_interface/doc/src/notes.xml index a69c5aac11..4ef5454f44 100644 --- a/lib/erl_interface/doc/src/notes.xml +++ b/lib/erl_interface/doc/src/notes.xml @@ -31,6 +31,23 @@ </header> <p>This document describes the changes made to the Erl_interface application.</p> +<section><title>Erl_Interface 3.9.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Look for .erlang.cookie in windows system directory if + HOMEDRIVE and HOMEPATH is not set. The same behaviour as + the VM.</p> + <p> + Own Id: OTP-13849</p> + </item> + </list> + </section> + +</section> + <section><title>Erl_Interface 3.9</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/erl_interface/vsn.mk b/lib/erl_interface/vsn.mk index 33705d1e8b..82be43b7df 100644 --- a/lib/erl_interface/vsn.mk +++ b/lib/erl_interface/vsn.mk @@ -1,2 +1,2 @@ -EI_VSN = 3.9 +EI_VSN = 3.9.1 ERL_INTERFACE_VSN = $(EI_VSN) diff --git a/lib/eunit/doc/src/notes.xml b/lib/eunit/doc/src/notes.xml index 88602e8222..6ae3c04bc8 100644 --- a/lib/eunit/doc/src/notes.xml +++ b/lib/eunit/doc/src/notes.xml @@ -33,6 +33,23 @@ </header> <p>This document describes the changes made to the EUnit application.</p> +<section><title>Eunit 2.3.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + When asserts were moved out to a separate header file, + the automatic enabling of asserts when testing is enabled + stopped working.</p> + <p> + Own Id: OTP-13892</p> + </item> + </list> + </section> + +</section> + <section><title>Eunit 2.3</title> <section><title>Improvements and New Features</title> diff --git a/lib/eunit/vsn.mk b/lib/eunit/vsn.mk index b551ee6eb6..83d826f8b6 100644 --- a/lib/eunit/vsn.mk +++ b/lib/eunit/vsn.mk @@ -1 +1 @@ -EUNIT_VSN = 2.3 +EUNIT_VSN = 2.3.1 diff --git a/lib/hipe/doc/src/notes.xml b/lib/hipe/doc/src/notes.xml index e2a1524be6..fc529fba61 100644 --- a/lib/hipe/doc/src/notes.xml +++ b/lib/hipe/doc/src/notes.xml @@ -31,6 +31,46 @@ </header> <p>This document describes the changes made to HiPE.</p> +<section><title>Hipe 3.15.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed various hipe compiler backend bugs affecting mostly + ARM and SPARC.</p> + <p> + Own Id: OTP-13846 Aux Id: PR-1146 </p> + </item> + <item> + <p> + Fixed some Dialyzer warnings and code cleanup for the + Sparc compiler backend.</p> + <p> + Own Id: OTP-13861 Aux Id: PR-1148 </p> + </item> + <item> + <p> Fix erl_bif_types opaque bug. </p> + <p> + Own Id: OTP-13878 Aux Id: PR-1161, ERL-249 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Fix erl_types opaque match order</p> + <p> + Own Id: OTP-13876</p> + </item> + </list> + </section> + +</section> + <section><title>Hipe 3.15.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/hipe/vsn.mk b/lib/hipe/vsn.mk index e61c1a042c..f00ff0cf2e 100644 --- a/lib/hipe/vsn.mk +++ b/lib/hipe/vsn.mk @@ -1 +1 @@ -HIPE_VSN = 3.15.1 +HIPE_VSN = 3.15.2 diff --git a/lib/ic/doc/src/notes.xml b/lib/ic/doc/src/notes.xml index 08b02bc4a4..ea8bf758cf 100644 --- a/lib/ic/doc/src/notes.xml +++ b/lib/ic/doc/src/notes.xml @@ -31,7 +31,29 @@ <file>notes.xml</file> </header> - <section><title>IC 4.4.1</title> + <section><title>IC 4.4.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Correct bugs when path to mib or idl spec files + contains UTF-8 characters. </p> + <p> + Own Id: OTP-13718 Aux Id: ERL-179 </p> + </item> + <item> + <p> + Update build scripts to not make assumtions about where + env, cp and perl are located.</p> + <p> + Own Id: OTP-13800</p> + </item> + </list> + </section> + +</section> + +<section><title>IC 4.4.1</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/ic/vsn.mk b/lib/ic/vsn.mk index 7d00ae0170..f0e5e7c266 100644 --- a/lib/ic/vsn.mk +++ b/lib/ic/vsn.mk @@ -1 +1 @@ -IC_VSN = 4.4.1 +IC_VSN = 4.4.2 diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml index caa5a083a3..0c7604ef65 100644 --- a/lib/inets/doc/src/notes.xml +++ b/lib/inets/doc/src/notes.xml @@ -33,7 +33,24 @@ <file>notes.xml</file> </header> - <section><title>Inets 6.3.2</title> + <section><title>Inets 6.3.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The legacy option 'inet6fb4' for inets had stopped + working. This bug has now been corrected. Fix by Edwin + Fine in bugs.erlang.org ERL-200 and Github PR#1132.</p> + <p> + Own Id: OTP-13776 Aux Id: ERL-200 PR-1132 </p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 6.3.2</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/jinterface/doc/src/notes.xml b/lib/jinterface/doc/src/notes.xml index 3151fc4b5e..30f607c357 100644 --- a/lib/jinterface/doc/src/notes.xml +++ b/lib/jinterface/doc/src/notes.xml @@ -31,6 +31,22 @@ </header> <p>This document describes the changes made to the Jinterface application.</p> +<section><title>Jinterface 1.7.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Update build scripts to not make assumtions about where + env, cp and perl are located.</p> + <p> + Own Id: OTP-13800</p> + </item> + </list> + </section> + +</section> + <section><title>Jinterface 1.7</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/jinterface/vsn.mk b/lib/jinterface/vsn.mk index 752b34e78c..c29d9df3d7 100644 --- a/lib/jinterface/vsn.mk +++ b/lib/jinterface/vsn.mk @@ -1 +1 @@ -JINTERFACE_VSN = 1.7 +JINTERFACE_VSN = 1.7.1 diff --git a/lib/kernel/doc/src/inet.xml b/lib/kernel/doc/src/inet.xml index 3f4e3684f4..d907cef7d3 100644 --- a/lib/kernel/doc/src/inet.xml +++ b/lib/kernel/doc/src/inet.xml @@ -1092,6 +1092,15 @@ setcap cap_sys_admin,cap_sys_ptrace,cap_dac_read_search+epi beam.smp</code> The option is ignored on platforms where it is not implemented. Use with caution.</p> </item> + <tag><c>{tclass, Integer}</c></tag> + <item> + <p> + Sets <c>IPV6_TCLASS IP</c> level options on platforms + where this is implemented. The behavior and allowed range + varies between different systems. + The option is ignored on platforms where it is not + implemented. Use with caution.</p> + </item> </taglist> <p>In addition to these options, <em>raw</em> option specifications can be used. The raw options are diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index 9e9be3f661..5bcc0b7c09 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -31,6 +31,74 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 5.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix a memory leak when calling + seq_trace:get_system_tracer().</p> + <p> + Own Id: OTP-13742</p> + </item> + <item> + <p> + Fix for the problem that when adding the ebin directory + of an application to the code path, the + <c>code:priv_dir/1</c> function returns an incorrect path + to the priv directory of the same application.</p> + <p> + Own Id: OTP-13758 Aux Id: ERL-195 </p> + </item> + <item> + <p> + Fix code_server crash when adding code paths of two + levels.</p> + <p> + Own Id: OTP-13765 Aux Id: ERL-194 </p> + </item> + <item> + <p> + Respect -proto_dist switch while connection to EPMD</p> + <p> + Own Id: OTP-13770 Aux Id: PR-1129 </p> + </item> + <item> + <p> + Fixed a bug where init:stop could deadlock if a process + with infinite shutdown timeout (e.g. a supervisor) + attempted to load code while terminating.</p> + <p> + Own Id: OTP-13802</p> + </item> + <item> + <p> + Close stdin of commands run in os:cmd. This is a + backwards compatiblity fix that restores the behaviour of + pre 19.0 os:cmd.</p> + <p> + Own Id: OTP-13867 Aux Id: seq13178 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add <c>net_kernel:setopts/2</c> and + <c>net_kernel:getopts/2</c> to control options for + distribution sockets in runtime.</p> + <p> + Own Id: OTP-13564</p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 5.0.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/src/inet.erl b/lib/kernel/src/inet.erl index 75dd800c6b..f5c13ecdd7 100644 --- a/lib/kernel/src/inet.erl +++ b/lib/kernel/src/inet.erl @@ -677,7 +677,7 @@ parse_strict_address(Addr) -> %% Return a list of available options options() -> [ - tos, priority, reuseaddr, keepalive, dontroute, linger, + tos, tclass, priority, reuseaddr, keepalive, dontroute, linger, broadcast, sndbuf, recbuf, nodelay, ipv6_v6only, buffer, header, active, packet, deliver, mode, multicast_if, multicast_ttl, multicast_loop, @@ -698,7 +698,7 @@ stats() -> %% Available options for tcp:connect %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% connect_options() -> - [tos, priority, reuseaddr, keepalive, linger, sndbuf, recbuf, nodelay, + [tos, tclass, priority, reuseaddr, keepalive, linger, sndbuf, recbuf, nodelay, header, active, packet, packet_size, buffer, mode, deliver, line_delimiter, exit_on_close, high_watermark, low_watermark, high_msgq_watermark, low_msgq_watermark, send_timeout, send_timeout_close, delay_send, raw, @@ -766,7 +766,7 @@ con_add(Name, Val, #connect_opts{} = R, Opts, AllOpts) -> %% Available options for tcp:listen %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% listen_options() -> - [tos, priority, reuseaddr, keepalive, linger, sndbuf, recbuf, nodelay, + [tos, tclass, priority, reuseaddr, keepalive, linger, sndbuf, recbuf, nodelay, header, active, packet, buffer, mode, deliver, backlog, ipv6_v6only, exit_on_close, high_watermark, low_watermark, high_msgq_watermark, low_msgq_watermark, send_timeout, send_timeout_close, delay_send, @@ -846,7 +846,7 @@ tcp_module_1(Opts, Address) -> %% Available options for udp:open %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% udp_options() -> - [tos, priority, reuseaddr, sndbuf, recbuf, header, active, buffer, mode, + [tos, tclass, priority, reuseaddr, sndbuf, recbuf, header, active, buffer, mode, deliver, ipv6_v6only, broadcast, dontroute, multicast_if, multicast_ttl, multicast_loop, add_membership, drop_membership, read_packets,raw, @@ -917,7 +917,7 @@ udp_module(Opts) -> % (*) passing of open FDs ("fdopen") is not supported. sctp_options() -> [ % The following are generic inet options supported for SCTP sockets: - mode, active, buffer, tos, priority, dontroute, reuseaddr, linger, sndbuf, + mode, active, buffer, tos, tclass, priority, dontroute, reuseaddr, linger, sndbuf, recbuf, ipv6_v6only, high_msgq_watermark, low_msgq_watermark, % Other options are SCTP-specific (though they may be similar to their diff --git a/lib/kernel/src/inet_int.hrl b/lib/kernel/src/inet_int.hrl index c8a8962e78..4e8f59a3b9 100644 --- a/lib/kernel/src/inet_int.hrl +++ b/lib/kernel/src/inet_int.hrl @@ -153,6 +153,7 @@ -define(INET_LOPT_NETNS, 38). -define(INET_LOPT_TCP_SHOW_ECONNRESET, 39). -define(INET_LOPT_LINE_DELIM, 40). +-define(INET_OPT_TCLASS, 41). % Specific SCTP options: separate range: -define(SCTP_OPT_RTOINFO, 100). -define(SCTP_OPT_ASSOCINFO, 101). diff --git a/lib/kernel/test/gen_tcp_api_SUITE.erl b/lib/kernel/test/gen_tcp_api_SUITE.erl index 77ec89b4f4..92a74465b7 100644 --- a/lib/kernel/test/gen_tcp_api_SUITE.erl +++ b/lib/kernel/test/gen_tcp_api_SUITE.erl @@ -38,7 +38,7 @@ t_local_basic/1, t_local_unbound/1, t_local_fdopen/1, t_local_fdopen_listen/1, t_local_fdopen_listen_unbound/1, t_local_fdopen_connect/1, t_local_fdopen_connect_unbound/1, - t_local_abstract/1]). + t_local_abstract/1, t_accept_inet6_tclass/1]). -export([getsockfd/0,closesockfd/1]). @@ -50,6 +50,7 @@ all() -> [{group, t_accept}, {group, t_connect}, {group, t_recv}, t_shutdown_write, t_shutdown_both, t_shutdown_error, t_shutdown_async, t_fdopen, t_fdconnect, t_implicit_inet6, + t_accept_inet6_tclass, {group, t_local}]. groups() -> @@ -521,6 +522,24 @@ local_handshake(S, SAddr, C, CAddr) -> SData = ok(gen_tcp:recv(C, length(SData))), ok. +t_accept_inet6_tclass(Config) when is_list(Config) -> + TClassOpt = {tclass,8#56 bsl 2}, % Expedited forwarding + case gen_tcp:listen(0, [inet6,TClassOpt]) of + {ok,L} -> + LPort = ok(inet:port(L)), + Loopback = {0,0,0,0,0,0,0,1}, + Sa = ok(gen_tcp:connect(Loopback, LPort, [])), + Sb = ok(gen_tcp:accept(L)), + [TClassOpt] = ok(inet:getopts(Sb, [tclass])), + ok = gen_tcp:close(Sb), + ok = gen_tcp:close(Sa), + ok = gen_tcp:close(L), + ok; + {error,_} -> + {skip,"IPv6 TCLASS not supported"} + end. + + %%% Utilities %% Calls M:F/length(A), which should return a timeout error, and complete diff --git a/lib/kernel/vsn.mk b/lib/kernel/vsn.mk index cdd200a234..d3b2d18ae5 100644 --- a/lib/kernel/vsn.mk +++ b/lib/kernel/vsn.mk @@ -1 +1 @@ -KERNEL_VSN = 5.0.2 +KERNEL_VSN = 5.1 diff --git a/lib/mnesia/doc/src/notes.xml b/lib/mnesia/doc/src/notes.xml index 7d8e8d0c44..e621bd593c 100644 --- a/lib/mnesia/doc/src/notes.xml +++ b/lib/mnesia/doc/src/notes.xml @@ -39,7 +39,21 @@ thus constitutes one section in this document. The title of each section is the version number of Mnesia.</p> - <section><title>Mnesia 4.14</title> + <section><title>Mnesia 4.14.1</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Correct some minor documentation issues. </p> + <p> + Own Id: OTP-13891</p> + </item> + </list> + </section> + +</section> + +<section><title>Mnesia 4.14</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/mnesia/vsn.mk b/lib/mnesia/vsn.mk index fb4200f62d..f08e364276 100644 --- a/lib/mnesia/vsn.mk +++ b/lib/mnesia/vsn.mk @@ -1 +1 @@ -MNESIA_VSN = 4.14 +MNESIA_VSN = 4.14.1 diff --git a/lib/observer/doc/src/notes.xml b/lib/observer/doc/src/notes.xml index f79f75fead..659eb28292 100644 --- a/lib/observer/doc/src/notes.xml +++ b/lib/observer/doc/src/notes.xml @@ -32,6 +32,22 @@ <p>This document describes the changes made to the Observer application.</p> +<section><title>Observer 2.2.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed error handling in observer when mnesia tables was + requested and not available.</p> + <p> + Own Id: OTP-13845 Aux Id: ERL-237 </p> + </item> + </list> + </section> + +</section> + <section><title>Observer 2.2.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/observer/vsn.mk b/lib/observer/vsn.mk index 9a7c6a546f..444089151e 100644 --- a/lib/observer/vsn.mk +++ b/lib/observer/vsn.mk @@ -1 +1 @@ -OBSERVER_VSN = 2.2.1 +OBSERVER_VSN = 2.2.2 diff --git a/lib/odbc/doc/src/notes.xml b/lib/odbc/doc/src/notes.xml index 40f9339028..7fb19a072e 100644 --- a/lib/odbc/doc/src/notes.xml +++ b/lib/odbc/doc/src/notes.xml @@ -32,7 +32,23 @@ <p>This document describes the changes made to the odbc application. </p> - <section><title>ODBC 2.11.2</title> + <section><title>ODBC 2.11.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + ODBC build configure has been updated to accept Mac OS X + El Capitan. Fixed by Lee Bannard.</p> + <p> + Own Id: OTP-13781</p> + </item> + </list> + </section> + +</section> + +<section><title>ODBC 2.11.2</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/odbc/vsn.mk b/lib/odbc/vsn.mk index 957c6b42eb..a7c8f8079b 100644 --- a/lib/odbc/vsn.mk +++ b/lib/odbc/vsn.mk @@ -1 +1 @@ -ODBC_VSN = 2.11.2 +ODBC_VSN = 2.11.3 diff --git a/lib/parsetools/doc/src/notes.xml b/lib/parsetools/doc/src/notes.xml index b826b4d03a..30a9374e81 100644 --- a/lib/parsetools/doc/src/notes.xml +++ b/lib/parsetools/doc/src/notes.xml @@ -31,6 +31,22 @@ </header> <p>This document describes the changes made to the Parsetools application.</p> +<section><title>Parsetools 2.1.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Yecc generates Dialyzer suppressions to avoid + warnings when operator precedence declarations are used. + </p> + <p> + Own Id: OTP-13681</p> + </item> + </list> + </section> + +</section> + <section><title>Parsetools 2.1.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/parsetools/vsn.mk b/lib/parsetools/vsn.mk index befdd82d6e..07cd959c98 100644 --- a/lib/parsetools/vsn.mk +++ b/lib/parsetools/vsn.mk @@ -1 +1 @@ -PARSETOOLS_VSN = 2.1.2 +PARSETOOLS_VSN = 2.1.3 diff --git a/lib/reltool/doc/src/notes.xml b/lib/reltool/doc/src/notes.xml index 25d983afd3..2365a68feb 100644 --- a/lib/reltool/doc/src/notes.xml +++ b/lib/reltool/doc/src/notes.xml @@ -38,7 +38,38 @@ thus constitutes one section in this document. The title of each section is the version number of Reltool.</p> - <section><title>Reltool 0.7.1</title> + <section><title>Reltool 0.7.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Dependencies specified in .app files would earlier only + be followed for applications that are included in a 'rel' + spec in the reltool config. For other applications, only + xref would decide the dependencies.</p> + <p> + Some dependency chains would even be missed for + applications that are included in a 'rel' spec in the + reltool config. E.g.</p> + <p> + <list> <item>Application x has y as included application, + and y in turn has z as included application. Then z is + not included. </item> <item>Application x has y in its + 'applications' tag in the .app file, and y in turn has z + as included application. Then z is not included.</item> + </list></p> + <p> + These bugs are now corrected.</p> + <p> + Own Id: OTP-11993</p> + </item> + </list> + </section> + +</section> + +<section><title>Reltool 0.7.1</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/reltool/vsn.mk b/lib/reltool/vsn.mk index 76f69fd294..2b23ff6f20 100644 --- a/lib/reltool/vsn.mk +++ b/lib/reltool/vsn.mk @@ -1 +1 @@ -RELTOOL_VSN = 0.7.1 +RELTOOL_VSN = 0.7.2 diff --git a/lib/runtime_tools/doc/src/notes.xml b/lib/runtime_tools/doc/src/notes.xml index b6dfb2dd28..0b830593b4 100644 --- a/lib/runtime_tools/doc/src/notes.xml +++ b/lib/runtime_tools/doc/src/notes.xml @@ -32,6 +32,20 @@ <p>This document describes the changes made to the Runtime_Tools application.</p> +<section><title>Runtime_Tools 1.10.1</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Correct some minor documentation issues. </p> + <p> + Own Id: OTP-13891</p> + </item> + </list> + </section> + +</section> + <section><title>Runtime_Tools 1.10</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/runtime_tools/vsn.mk b/lib/runtime_tools/vsn.mk index b33f6f4721..0fc86e42f7 100644 --- a/lib/runtime_tools/vsn.mk +++ b/lib/runtime_tools/vsn.mk @@ -1 +1 @@ -RUNTIME_TOOLS_VSN = 1.10 +RUNTIME_TOOLS_VSN = 1.10.1 diff --git a/lib/sasl/doc/src/notes.xml b/lib/sasl/doc/src/notes.xml index bd713af932..055d433524 100644 --- a/lib/sasl/doc/src/notes.xml +++ b/lib/sasl/doc/src/notes.xml @@ -31,6 +31,59 @@ </header> <p>This document describes the changes made to the SASL application.</p> +<section><title>SASL 3.0.1</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Improved dirty scheduler support. A purge of a module can + now be performed without having to wait for completion of + all ongoing dirty NIF calls.</p> + <p> + Note that when enabling support for dirty schedulers, a + new purge strategy will as of ERTS version 8.1 be + enabled. This new strategy is not fully backwards + compatible with the strategy used by default. For more + information see the documentation of <seealso + marker="erts:erlang#check_process_code/3"><c>erlang:check_process_code/3</c></seealso>.</p> + <p> + Own Id: OTP-13808 Aux Id: OTP-13833 </p> + </item> + <item> + <p> + A new purge strategy has been introduced. The new + strategy will by default be disabled during the OTP 19 + release, but will be the only strategy available as of + the OTP 20 release.</p> + <p> + The new strategy is slightly incompatible with the + strategy being used by default in OTP 19. Using the + default strategy, processes holding <c>fun</c>s that + refer to the module being purged either fail a soft + purge, or will be killed during a hard purge. The new + strategy completely ignores <c>fun</c>s. If <c>fun</c>s + referring to the code being purged exist, and are used + after a purge, an exception will be raised upon usage. + That is, the behavior will be exactly the same as the + case when a <c>fun</c> is received by a process after the + purge.</p> + <p> + The new strategy can optionally be enabled when building + OTP during OTP 19, and will automatically be enabled if + the runtime system is built with support for dirty + schedulers.</p> + <p> + For more information see the documentation of <seealso + marker="erts:erlang#check_process_code/3"><c>erlang:check_process_code/3</c></seealso>.</p> + <p> + Own Id: OTP-13833</p> + </item> + </list> + </section> + +</section> + <section><title>SASL 3.0</title> <section><title>Improvements and New Features</title> diff --git a/lib/sasl/vsn.mk b/lib/sasl/vsn.mk index fd0fc9b8b5..a7d7c09cde 100644 --- a/lib/sasl/vsn.mk +++ b/lib/sasl/vsn.mk @@ -1 +1 @@ -SASL_VSN = 3.0 +SASL_VSN = 3.0.1 diff --git a/lib/snmp/doc/src/notes.xml b/lib/snmp/doc/src/notes.xml index 0f5c35b300..3323d32878 100644 --- a/lib/snmp/doc/src/notes.xml +++ b/lib/snmp/doc/src/notes.xml @@ -34,7 +34,35 @@ </header> - <section><title>SNMP 5.2.3</title> + <section><title>SNMP 5.2.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Correct bugs when path to mib or idl spec files + contains UTF-8 characters. </p> + <p> + Own Id: OTP-13718 Aux Id: ERL-179 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Solves snmp config string handling as reported by ERL-164 + and solved by PR-1100</p> + <p> + Own Id: OTP-13706</p> + </item> + </list> + </section> + +</section> + +<section><title>SNMP 5.2.3</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/snmp/vsn.mk b/lib/snmp/vsn.mk index f95b428290..28eba0d0d6 100644 --- a/lib/snmp/vsn.mk +++ b/lib/snmp/vsn.mk @@ -19,6 +19,6 @@ # %CopyrightEnd% APPLICATION = snmp -SNMP_VSN = 5.2.3 +SNMP_VSN = 5.2.4 PRE_VSN = APP_VSN = "$(APPLICATION)-$(SNMP_VSN)$(PRE_VSN)" diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml index f9d11b2a60..b990c18e9a 100644 --- a/lib/ssh/doc/src/notes.xml +++ b/lib/ssh/doc/src/notes.xml @@ -30,6 +30,68 @@ <file>notes.xml</file> </header> +<section><title>Ssh 4.3.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Upgrade of an established client connection could crash + because the ssh client supervisors children had wrong + type. This is fixed now.</p> + <p> + Own Id: OTP-13782 Aux Id: seq13158 </p> + </item> + <item> + <p> + Partly checks the public key early in public key + authorization</p> + <p> + Own Id: OTP-13847 Aux Id: + defensics-ssh3.1.0-190243,205277,219318 </p> + </item> + <item> + <p> + Corrected handling of SHA for ECDSA (Elliptic curve + public keys)</p> + <p> + Own Id: OTP-13850 Aux Id: defensics-ssh3.1.0-214168 </p> + </item> + <item> + <p> + Problems found by test suites as well as by + Codenomicon/Defensics fixed: - reduce max random padding + to 15 bytes (Codenomicon/Defensics) - inclomplete pdu + handling (Codenomicon/Defensics) - badmatch in test suite + - non-blocking send fixes deadlock in + ssh_connection_SUITE:interrupted_send</p> + <p> + Own Id: OTP-13854</p> + </item> + <item> + <p> + Caller is now notified when a tcp close is received.</p> + <p> + Own Id: OTP-13859 Aux Id: seq13177 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Use application:ensure_all_started/2 instead of + hard-coding deps</p> + <p> + Own Id: OTP-13843 Aux Id: PR-1147 </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 4.3.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 9d68ee0eee..d02e21a013 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -28,6 +28,45 @@ <p>This document describes the changes made to the SSL application.</p> +<section><title>SSL 8.0.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Correctly formed handshake messages received out of order + will now correctly fail the connection with unexpected + message.</p> + <p> + Own Id: OTP-13853</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + ssl application now behaves gracefully also on partially + incorrect input from peer.</p> + <p> + Own Id: OTP-13834</p> + </item> + <item> + <p> + Add application environment configuration + bypass_pem_cache. This can be used as a workaround for + the current implementation of the PEM-cache that has + proven to be a bottleneck.</p> + <p> + Own Id: OTP-13883</p> + </item> + </list> + </section> + +</section> + <section><title>SSL 8.0.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index f0347703e7..554150380f 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -31,6 +31,66 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 3.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The <c>zip:unzip/1,2</c> and <c>zip:extract/1,2</c> + functions have been updated to handle directory traversal + exploits. Any element in the zip file that contains a + path that points to a directory above the top level + working directory, <c>cwd</c>, will instead be extracted + in <c>cwd</c>. An error message is printed for any such + element in the zip file during the unzip operation. The + <c>keep_old_files</c> option determines if a file will + overwrite a previous file with the same name within the + zip file.</p> + <p> + Own Id: OTP-13633</p> + </item> + <item> + <p> Correct the contracts for + <c>ets:match_object/1,3</c>. </p> + <p> + Own Id: OTP-13721 Aux Id: PR-1113 </p> + </item> + <item> + <p> + Errors in type specification and Emacs template + generation for <c>gen_statem:code_change/4</c> has been + fixed from bugs.erlang.org's Jira cases ERL-172 and + ERL-187.</p> + <p> + Own Id: OTP-13746 Aux Id: ERL-172, ERL-187 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + gen_statem has been changed to set the callback mode for + a server to what Module:callback_mode/0 returns. This + facilitates e.g code downgrade since the callback mode + now becomes a property of the currently active code, not + of the server process.</p> + <p> + Exception handling from Module:init/1 has also been + improved.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-13752</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 3.0.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/test/ets_SUITE.erl b/lib/stdlib/test/ets_SUITE.erl index b02d17bdb6..e726a29c9e 100644 --- a/lib/stdlib/test/ets_SUITE.erl +++ b/lib/stdlib/test/ets_SUITE.erl @@ -218,7 +218,7 @@ memory_check_summary(_Config) -> receive {get_failed_memchecks, FailedMemchecks} -> ok end, io:format("Failed memchecks: ~p\n",[FailedMemchecks]), NoFailedMemchecks = length(FailedMemchecks), - if NoFailedMemchecks > 3 -> + if NoFailedMemchecks > 1 -> ct:fail("Too many failed (~p) memchecks", [NoFailedMemchecks]); true -> ok @@ -591,12 +591,6 @@ select_fail_do(Opts) -> -define(S(T),ets:info(T,memory)). --define(TAB_STRUCT_SZ, erts_debug:get_internal_state('DbTable_words')). -%%-define(NORMAL_TAB_STRUCT_SZ, 26). %% SunOS5.8, 32-bit, non smp, private heap -%% -%% The hardcoded expected memory sizes (in words) are the ones we expect on: -%% SunOS5.8, 32-bit, non smp, private heap -%% %% Whitebox test of ets:info(X, memory). memory(Config) when is_list(Config) -> @@ -607,11 +601,11 @@ memory(Config) when is_list(Config) -> memory_do(Opts) -> L = [T1,T2,T3,T4] = fill_sets_int(1000,Opts), XR1 = case mem_mode(T1) of - {normal,_} -> {13836,13046,13046,13052}; %{13862,13072,13072,13078}; - {compressed,4} -> {11041,10251,10251,10252}; %{11067,10277,10277,10278}; - {compressed,8} -> {10050,9260,9260,9260} %{10076,9286,9286,9286} + {normal,_} -> {13836, 15346, 15346, 15346+6}; + {compressed,4} -> {11041, 12551, 12551, 12551+1}; + {compressed,8} -> {10050, 11560, 11560, 11560} end, - XRes1 = adjust_xmem(L, XR1), + XRes1 = adjust_xmem(L, XR1, 1), Res1 = {?S(T1),?S(T2),?S(T3),?S(T4)}, lists:foreach(fun(T) -> Before = ets:info(T,size), @@ -623,11 +617,11 @@ memory_do(Opts) -> end, L), XR2 = case mem_mode(T1) of - {normal,_} -> {13826,13037,13028,13034}; %{13852,13063,13054,13060}; - {compressed,4} -> {11031,10242,10233,10234}; %{11057,10268,10259,10260}; - {compressed,8} -> {10040,9251,9242,9242} %10066,9277,9268,9268} + {normal,_} -> {13826, 15337, 15337-9, 15337-3}; + {compressed,4} -> {11031, 12542, 12542-9, 12542-8}; + {compressed,8} -> {10040, 11551, 11551-9, 11551-9} end, - XRes2 = adjust_xmem(L, XR2), + XRes2 = adjust_xmem(L, XR2, 1), Res2 = {?S(T1),?S(T2),?S(T3),?S(T4)}, lists:foreach(fun(T) -> Before = ets:info(T,size), @@ -639,17 +633,17 @@ memory_do(Opts) -> end, L), XR3 = case mem_mode(T1) of - {normal,_} -> {13816,13028,13010,13016}; %{13842,13054,13036,13042}; - {compressed,4} -> {11021,10233,10215,10216}; %{11047,10259,10241,10242}; - {compressed,8} -> {10030,9242,9224,9224} %{10056,9268,9250,9250} + {normal,_} -> {13816, 15328, 15328-18, 15328-12}; + {compressed,4} -> {11021, 12533, 12533-18, 12533-17}; + {compressed,8} -> {10030, 11542, 11542-18, 11542-18} end, - XRes3 = adjust_xmem(L, XR3), + XRes3 = adjust_xmem(L, XR3, 1), Res3 = {?S(T1),?S(T2),?S(T3),?S(T4)}, lists:foreach(fun(T) -> ets:delete_all_objects(T) end, L), - XRes4 = adjust_xmem(L, {50,260,260,260}), %{76,286,286,286}), + XRes4 = adjust_xmem(L, {50, 256, 256, 256}, 0), Res4 = {?S(T1),?S(T2),?S(T3),?S(T4)}, lists:foreach(fun(T) -> ets:delete(T) @@ -660,7 +654,7 @@ memory_do(Opts) -> ets:select_delete(T,[{'_',[],[true]}]) end, L2), - XRes5 = adjust_xmem(L2, {50,260,260,260}), %{76,286,286,286}), + XRes5 = adjust_xmem(L2, {50, 256, 256, 256}, 0), Res5 = {?S(T11),?S(T12),?S(T13),?S(T14)}, io:format("XRes1 = ~p~n" " Res1 = ~p~n~n" @@ -698,15 +692,15 @@ chk_normal_tab_struct_size() -> erlang:system_info(smp_support), erlang:system_info(heap_type)}, io:format("System = ~p~n", [System]), - io:format("?TAB_STRUCT_SZ=~p~n", [?TAB_STRUCT_SZ]), ok. -adjust_xmem([_T1,_T2,_T3,_T4], {A0,B0,C0,D0} = _Mem0) -> +adjust_xmem([_T1,_T2,_T3,_T4], {A0,B0,C0,D0} = _Mem0, EstCnt) -> %% Adjust for 64-bit, smp, and os: %% Table struct size may differ. - TabDiff = ?TAB_STRUCT_SZ, - {A0+TabDiff, B0+TabDiff, C0+TabDiff, D0+TabDiff}. + {TabSz, EstSz} = erts_debug:get_internal_state('DbTable_words'), + HTabSz = TabSz + EstCnt*EstSz, + {A0+TabSz, B0+HTabSz, C0+HTabSz, D0+HTabSz}. %% Misc. whitebox tests t_whitebox(Config) when is_list(Config) -> @@ -1908,7 +1902,7 @@ evil_counter(I,Opts) -> end, Start = Start0 + rand:uniform(100000), ets:insert(T, {dracula,Start}), - Iter = 40000, + Iter = 40000 div syrup_factor(), End = Start + Iter, End = evil_counter_1(Iter, T), ets:delete(T). @@ -3302,7 +3296,8 @@ evil_delete_owner(Name, Flags, Data, Fix) -> exit_large_table_owner(Config) when is_list(Config) -> %%Data = [{erlang:phash2(I, 16#ffffff),I} || I <- lists:seq(1, 500000)], - FEData = fun(Do) -> repeat_while(fun(500000) -> {false,ok}; + Laps = 500000 div syrup_factor(), + FEData = fun(Do) -> repeat_while(fun(I) when I =:= Laps -> {false,ok}; (I) -> Do({erlang:phash2(I, 16#ffffff),I}), {true, I+1} end, 1) @@ -3318,7 +3313,8 @@ exit_large_table_owner_do(Opts,{FEData,Config}) -> exit_many_large_table_owner(Config) when is_list(Config) -> ct:timetrap({minutes,30}), %% valgrind needs a lot %%Data = [{erlang:phash2(I, 16#ffffff),I} || I <- lists:seq(1, 500000)], - FEData = fun(Do) -> repeat_while(fun(500000) -> {false,ok}; + Laps = 500000 div syrup_factor(), + FEData = fun(Do) -> repeat_while(fun(I) when I =:= Laps -> {false,ok}; (I) -> Do({erlang:phash2(I, 16#ffffff),I}), {true, I+1} end, 1) @@ -4271,7 +4267,8 @@ heavy_lookup_element_do(Opts) -> Tab = ets_new(foobar_table, [set, protected, {keypos, 2} | Opts]), ok = fill_tab2(Tab, 0, 7000), %% lookup ALL elements 50 times - _ = [do_lookup_element(Tab, 6999, 1) || _ <- lists:seq(1, 50)], + Laps = 50 div syrup_factor(), + _ = [do_lookup_element(Tab, 6999, 1) || _ <- lists:seq(1, Laps)], true = ets:delete(Tab), verify_etsmem(EtsMem). @@ -4295,6 +4292,7 @@ heavy_concurrent(Config) when is_list(Config) -> do_heavy_concurrent(Opts) -> Size = 10000, + Laps = 10000 div syrup_factor(), EtsMem = etsmem(), Tab = ets_new(blupp, [set, public, {keypos, 2} | Opts]), ok = fill_tab2(Tab, 0, Size), @@ -4302,7 +4300,7 @@ do_heavy_concurrent(Opts) -> fun (N) -> my_spawn_link( fun () -> - do_heavy_concurrent_proc(Tab, Size, N) + do_heavy_concurrent_proc(Tab, Laps, N) end) end, lists:seq(1, 500)), @@ -5363,12 +5361,12 @@ verify_table_load(T) -> Stats = ets:info(T,stats), {Buckets,AvgLen,StdDev,ExpSD,_MinLen,_MaxLen,_} = Stats, ok = if - AvgLen > 7 -> + AvgLen > 1.2 -> io:format("Table overloaded: Stats=~p\n~p\n", [Stats, ets:info(T)]), false; - Buckets>256, AvgLen < 6 -> + Buckets>256, AvgLen < 0.47 -> io:format("Table underloaded: Stats=~p\n~p\n", [Stats, ets:info(T)]), false; @@ -5442,7 +5440,8 @@ smp_select_delete(Config) when is_list(Config) -> Eq+1 end, 0, TotCnts), - verify_table_load(T), + %% May fail as select_delete does not shrink table (enough) + %%verify_table_load(T), LeftInTab = ets:select_delete(T, [{{'$1','$1'}, [], [true]}]), 0 = ets:info(T,size), false = ets:info(T,fixed), @@ -5718,27 +5717,45 @@ etsmem() -> {Bl0+Bl,BlSz0+BlSz} end, {0,0}, CS) end}, - {Mem,AllTabs}. + {Mem,AllTabs, erts_debug:get_internal_state('DbTable_meta')}. -verify_etsmem({MemInfo,AllTabs}) -> +verify_etsmem(EtsMem) -> wait_for_test_procs(), + verify_etsmem(EtsMem, false). + +verify_etsmem({MemInfo,AllTabs,MetaState}=EtsMem, Adjusted) -> case etsmem() of - {MemInfo,_} -> + {MemInfo,_,_} -> io:format("Ets mem info: ~p", [MemInfo]), case MemInfo of {ErlMem,EtsAlloc} when ErlMem == notsup; EtsAlloc == undefined -> %% Use 'erl +Mea max' to do more complete memory leak testing. {comment,"Incomplete or no mem leak testing"}; _ -> - ok + case Adjusted of + true -> + {comment, "Meta state adjusted"}; + false -> + ok + end end; - {MemInfo2, AllTabs2} -> + + {MemInfo2, AllTabs2, MetaState2} -> io:format("Expected: ~p", [MemInfo]), io:format("Actual: ~p", [MemInfo2]), io:format("Changed tables before: ~p\n",[AllTabs -- AllTabs2]), io:format("Changed tables after: ~p\n", [AllTabs2 -- AllTabs]), - ets_test_spawn_logger ! {failed_memcheck, get('__ETS_TEST_CASE__')}, - {comment, "Failed memory check"} + io:format("Meta state before: ~p\n", [MetaState]), + io:format("Meta state after: ~p\n", [MetaState2]), + case {MetaState =:= MetaState2, Adjusted} of + {false, false} -> + io:format("Adjust meta state and retry...\n\n",[]), + {ok,ok} = erts_debug:set_internal_state('DbTable_meta', MetaState), + verify_etsmem(EtsMem, true); + _ -> + ets_test_spawn_logger ! {failed_memcheck, get('__ETS_TEST_CASE__')}, + {comment, "Failed memory check"} + end end. @@ -6261,3 +6278,9 @@ do_tc(Do, Report) -> T2 = erlang:monotonic_time(), Elapsed = erlang:convert_time_unit(T2 - T1, native, milli_seconds), Report(Elapsed). + +syrup_factor() -> + case erlang:system_info(build_type) of + valgrind -> 20; + _ -> 1 + end. diff --git a/lib/syntax_tools/doc/src/notes.xml b/lib/syntax_tools/doc/src/notes.xml index ef207f7c3d..82c4484d96 100644 --- a/lib/syntax_tools/doc/src/notes.xml +++ b/lib/syntax_tools/doc/src/notes.xml @@ -32,6 +32,40 @@ <p>This document describes the changes made to the Syntax_Tools application.</p> +<section><title>Syntax_Tools 2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug where <c>erl_tidy</c> crashed on the tilde + character when printing to standard output. </p> + <p> + Own Id: OTP-13725 Aux Id: ERL-151, PR-1071 </p> + </item> + <item> + <p><c>merl_transform</c> could get into an infinite loop + when syntactically incorrect text was passed to a + <c>merl:qquote/2,3</c> call.</p> + <p> + Own Id: OTP-13755</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Improve types and specs in OTP documentation generated + from Erlang source files. </p> + <p> + Own Id: OTP-13720 Aux Id: ERL-120 </p> + </item> + </list> + </section> + +</section> + <section><title>Syntax_Tools 2.0</title> <section><title>Improvements and New Features</title> diff --git a/lib/syntax_tools/vsn.mk b/lib/syntax_tools/vsn.mk index f09c2a01d0..c0ca083c38 100644 --- a/lib/syntax_tools/vsn.mk +++ b/lib/syntax_tools/vsn.mk @@ -1 +1 @@ -SYNTAX_TOOLS_VSN = 2.0 +SYNTAX_TOOLS_VSN = 2.1 diff --git a/lib/tools/doc/src/notes.xml b/lib/tools/doc/src/notes.xml index a0a817c0f2..2d9bee0dd1 100644 --- a/lib/tools/doc/src/notes.xml +++ b/lib/tools/doc/src/notes.xml @@ -31,6 +31,38 @@ </header> <p>This document describes the changes made to the Tools application.</p> +<section><title>Tools 2.8.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Errors in type specification and Emacs template + generation for <c>gen_statem:code_change/4</c> has been + fixed from bugs.erlang.org's Jira cases ERL-172 and + ERL-187.</p> + <p> + Own Id: OTP-13746 Aux Id: ERL-172, ERL-187 </p> + </item> + <item> + <p> + Fix gc_start/gc_end in fprof tags when parsing old trace + logs.</p> + <p> + Own Id: OTP-13778 Aux Id: PR-1136 </p> + </item> + <item> + <p><c>make</c> (tools) and <c>ct_make</c> (common_test) + would crash if an Erlang source file contained a + <c>-warning()</c> directive.</p> + <p> + Own Id: OTP-13855</p> + </item> + </list> + </section> + +</section> + <section><title>Tools 2.8.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/wx/doc/src/notes.xml b/lib/wx/doc/src/notes.xml index 4f0e166924..70ff0a92b7 100644 --- a/lib/wx/doc/src/notes.xml +++ b/lib/wx/doc/src/notes.xml @@ -32,6 +32,23 @@ <p>This document describes the changes made to the wxErlang application.</p> +<section><title>Wx 1.7.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Increased the stacksize for the wx thread. The default + stacksize on Windows is 1MB which is not enough if the + user created many nested dialogs.</p> + <p> + Own Id: OTP-13816</p> + </item> + </list> + </section> + +</section> + <section><title>Wx 1.7</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/wx/vsn.mk b/lib/wx/vsn.mk index de4e5e1935..0ce63d9f71 100644 --- a/lib/wx/vsn.mk +++ b/lib/wx/vsn.mk @@ -1 +1 @@ -WX_VSN = 1.7 +WX_VSN = 1.7.1 diff --git a/lib/xmerl/doc/src/notes.xml b/lib/xmerl/doc/src/notes.xml index 0abcb87998..12e64537ed 100644 --- a/lib/xmerl/doc/src/notes.xml +++ b/lib/xmerl/doc/src/notes.xml @@ -32,6 +32,21 @@ <p>This document describes the changes made to the Xmerl application.</p> +<section><title>Xmerl 1.3.12</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a number of broken links in the xmerl + documentation. </p> + <p> + Own Id: OTP-13880</p> + </item> + </list> + </section> + +</section> + <section><title>Xmerl 1.3.11</title> <section><title>Improvements and New Features</title> diff --git a/otp_versions.table b/otp_versions.table index 225bbfec71..999c2c6d46 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -1,3 +1,4 @@ +OTP-19.1 : asn1-4.0.4 common_test-1.12.3 compiler-7.0.2 crypto-3.7.1 debugger-4.2.1 dialyzer-3.0.2 diameter-1.12.1 edoc-0.8 erl_docgen-0.6 erl_interface-3.9.1 erts-8.1 eunit-2.3.1 gs-1.6.2 hipe-3.15.2 ic-4.4.2 inets-6.3.3 jinterface-1.7.1 kernel-5.1 mnesia-4.14.1 observer-2.2.2 odbc-2.11.3 parsetools-2.1.3 reltool-0.7.2 runtime_tools-1.10.1 sasl-3.0.1 snmp-5.2.4 ssh-4.3.2 ssl-8.0.2 stdlib-3.1 syntax_tools-2.1 tools-2.8.6 wx-1.7.1 xmerl-1.3.12 # cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 eldap-1.2.2 et-1.6 megaco-3.18.1 orber-3.8.2 os_mon-2.4.1 otp_mibs-1.1.1 percept-0.9 public_key-1.2 typer-0.9.11 : OTP-19.0.7 : erts-8.0.5 # asn1-4.0.3 common_test-1.12.2 compiler-7.0.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7 debugger-4.2 dialyzer-3.0.1 diameter-1.12 edoc-0.7.19 eldap-1.2.2 erl_docgen-0.5 erl_interface-3.9 et-1.6 eunit-2.3 gs-1.6.1 hipe-3.15.1 ic-4.4.1 inets-6.3.2 jinterface-1.7 kernel-5.0.2 megaco-3.18.1 mnesia-4.14 observer-2.2.1 odbc-2.11.2 orber-3.8.2 os_mon-2.4.1 otp_mibs-1.1.1 parsetools-2.1.2 percept-0.9 public_key-1.2 reltool-0.7.1 runtime_tools-1.10 sasl-3.0 snmp-5.2.3 ssh-4.3.1 ssl-8.0.1 stdlib-3.0.1 syntax_tools-2.0 tools-2.8.5 typer-0.9.11 wx-1.7 xmerl-1.3.11 : OTP-19.0.6 : erts-8.0.4 # asn1-4.0.3 common_test-1.12.2 compiler-7.0.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7 debugger-4.2 dialyzer-3.0.1 diameter-1.12 edoc-0.7.19 eldap-1.2.2 erl_docgen-0.5 erl_interface-3.9 et-1.6 eunit-2.3 gs-1.6.1 hipe-3.15.1 ic-4.4.1 inets-6.3.2 jinterface-1.7 kernel-5.0.2 megaco-3.18.1 mnesia-4.14 observer-2.2.1 odbc-2.11.2 orber-3.8.2 os_mon-2.4.1 otp_mibs-1.1.1 parsetools-2.1.2 percept-0.9 public_key-1.2 reltool-0.7.1 runtime_tools-1.10 sasl-3.0 snmp-5.2.3 ssh-4.3.1 ssl-8.0.1 stdlib-3.0.1 syntax_tools-2.0 tools-2.8.5 typer-0.9.11 wx-1.7 xmerl-1.3.11 : OTP-19.0.5 : kernel-5.0.2 # asn1-4.0.3 common_test-1.12.2 compiler-7.0.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7 debugger-4.2 dialyzer-3.0.1 diameter-1.12 edoc-0.7.19 eldap-1.2.2 erl_docgen-0.5 erl_interface-3.9 erts-8.0.3 et-1.6 eunit-2.3 gs-1.6.1 hipe-3.15.1 ic-4.4.1 inets-6.3.2 jinterface-1.7 megaco-3.18.1 mnesia-4.14 observer-2.2.1 odbc-2.11.2 orber-3.8.2 os_mon-2.4.1 otp_mibs-1.1.1 parsetools-2.1.2 percept-0.9 public_key-1.2 reltool-0.7.1 runtime_tools-1.10 sasl-3.0 snmp-5.2.3 ssh-4.3.1 ssl-8.0.1 stdlib-3.0.1 syntax_tools-2.0 tools-2.8.5 typer-0.9.11 wx-1.7 xmerl-1.3.11 : |