diff options
23 files changed, 481 insertions, 327 deletions
diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 7699f64c25..fabca87e9f 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -3339,7 +3339,7 @@ RealSystem = system + MissedSystem</code> <func> <name name="monitor" arity="2" clause_i="1" since=""/> - <name name="monitor" arity="2" clause_i="2" since="?"/> + <name name="monitor" arity="2" clause_i="2" since="OTP 19.0"/> <name name="monitor" arity="2" clause_i="3" since="OTP 18.0"/> <fsummary>Start monitoring.</fsummary> <type name="registered_name"/> @@ -4526,7 +4526,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="5" since="?"/> + <name name="port_info" arity="2" clause_i="5" since="OTP R16B"/> <fsummary>Information about the locking of a port.</fsummary> <desc> <p><c><anno>Locking</anno></c> is one of the following:</p> @@ -4547,7 +4547,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="6" since="?"/> + <name name="port_info" arity="2" clause_i="6" since="OTP R16B"/> <fsummary>Information about the memory size of a port.</fsummary> <desc> <p><c><anno>Bytes</anno></c> is the total number of @@ -4565,7 +4565,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="7" since="?"/> + <name name="port_info" arity="2" clause_i="7" since="OTP R16B"/> <fsummary>Information about the monitors of a port.</fsummary> <desc> <p><c><anno>Monitors</anno></c> represent processes monitored by @@ -4581,7 +4581,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="8" since="?"/> + <name name="port_info" arity="2" clause_i="8" since="OTP 19.0"/> <fsummary>Which processes are monitoring this port.</fsummary> <desc> <p>Returns list of pids that are monitoring given port at the @@ -4613,7 +4613,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="10" since="?"/> + <name name="port_info" arity="2" clause_i="10" since="OTP R16B"/> <fsummary>Information about the OS pid of a port.</fsummary> <desc> <p><c><anno>OsPid</anno></c> is the process identifier (or equivalent) @@ -4651,7 +4651,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="12" since="?"/> + <name name="port_info" arity="2" clause_i="12" since="OTP R16B"/> <fsummary>Information about the parallelism hint of a port.</fsummary> <desc> <p><c><anno>Boolean</anno></c> corresponds to the port parallelism @@ -4662,7 +4662,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="13" since="?"/> + <name name="port_info" arity="2" clause_i="13" since="OTP R16B"/> <fsummary>Information about the queue size of a port.</fsummary> <desc> <p><c><anno>Bytes</anno></c> is the total number @@ -4782,7 +4782,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="process_flag" arity="2" clause_i="4" since="?"/> + <name name="process_flag" arity="2" clause_i="4" since="OTP R13B04"/> <fsummary>Set process flag min_bin_vheap_size for the calling process. </fsummary> <desc> @@ -4794,7 +4794,7 @@ RealSystem = system + MissedSystem</code> <func> <name name="process_flag" arity="2" clause_i="5" - anchor="process_flag_max_heap_size" since="?"/> + anchor="process_flag_max_heap_size" since="OTP 19.0"/> <fsummary>Set process flag max_heap_size for the calling process. </fsummary> <type name="max_heap_size"/> @@ -4868,7 +4868,7 @@ RealSystem = system + MissedSystem</code> <func> <name name="process_flag" arity="2" clause_i="6" - anchor="process_flag_message_queue_data" since="?"/> + anchor="process_flag_message_queue_data" since="OTP 19.0"/> <fsummary>Set process flag message_queue_data for the calling process. </fsummary> <type name="message_queue_data"/> @@ -5047,7 +5047,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="process_flag" arity="3" since="?"/> + <name name="process_flag" arity="3" since=""/> <fsummary>Set process flags for a process.</fsummary> <desc> <p>Sets certain flags for the process <c><anno>Pid</anno></c>, @@ -6399,7 +6399,7 @@ true</pre> <func> <name name="statistics" arity="1" clause_i="1" - anchor="statistics_active_tasks" since="?"/> + anchor="statistics_active_tasks" since="OTP 18.3"/> <fsummary>Information about active processes and ports.</fsummary> <desc> <p>Returns the same as @@ -6414,7 +6414,7 @@ true</pre> <func> <name name="statistics" arity="1" clause_i="2" - anchor="statistics_active_tasks_all" since="?"/> + anchor="statistics_active_tasks_all" since="OTP 20.0"/> <fsummary>Information about active processes and ports.</fsummary> <desc> <p>Returns a list where each element represents the amount @@ -6503,7 +6503,7 @@ true</pre> <func> <name name="statistics" arity="1" clause_i="7" - anchor="statistics_microstate_accounting" since="?"/> + anchor="statistics_microstate_accounting" since="OTP 19.0"/> <fsummary>Information about microstate accounting.</fsummary> <desc> <p>Microstate accounting can be used to measure how much time the Erlang @@ -6682,7 +6682,7 @@ lists:map( <func> <name name="statistics" arity="1" clause_i="10" - anchor="statistics_run_queue_lengths" since="?"/> + anchor="statistics_run_queue_lengths" since="OTP 18.3"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc> <p>Returns the same as @@ -6697,7 +6697,7 @@ lists:map( <func> <name name="statistics" arity="1" clause_i="11" - anchor="statistics_run_queue_lengths_all" since="?"/> + anchor="statistics_run_queue_lengths_all" since="OTP 20.0"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc> <p>Returns a list where each element represents the amount @@ -6758,7 +6758,7 @@ lists:map( <func> <name name="statistics" arity="1" clause_i="13" - anchor="statistics_scheduler_wall_time" since="?"/> + anchor="statistics_scheduler_wall_time" since="OTP R15B01"/> <fsummary>Information about each schedulers work time.</fsummary> <desc> <p>Returns a list of tuples with @@ -6882,7 +6882,7 @@ ok <func> <name name="statistics" arity="1" clause_i="14" - anchor="statistics_scheduler_wall_time_all" since="?"/> + anchor="statistics_scheduler_wall_time_all" since="OTP 20.0"/> <fsummary>Information about each schedulers work time.</fsummary> <desc> <p>The same as @@ -6910,7 +6910,7 @@ ok </func> <func> <name name="statistics" arity="1" clause_i="15" - anchor="statistics_total_active_tasks" since="?"/> + anchor="statistics_total_active_tasks" since="OTP 18.3"/> <fsummary>Information about active processes and ports.</fsummary> <desc> <p>The same as calling @@ -6921,7 +6921,7 @@ ok <func> <name name="statistics" arity="1" clause_i="16" - anchor="statistics_total_active_tasks_all" since="?"/> + anchor="statistics_total_active_tasks_all" since="OTP 20.0"/> <fsummary>Information about active processes and ports.</fsummary> <desc> <p>The same as calling @@ -6932,7 +6932,7 @@ ok <func> <name name="statistics" arity="1" clause_i="17" - anchor="statistics_total_run_queue_lengths" since="?"/> + anchor="statistics_total_run_queue_lengths" since="OTP 18.3"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc> <p>The same as calling @@ -6943,7 +6943,7 @@ ok <func> <name name="statistics" arity="1" clause_i="18" - anchor="statistics_total_run_queue_lengths_all" since="?"/> + anchor="statistics_total_run_queue_lengths_all" since="OTP 20.0"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc> <p>The same as calling @@ -7186,7 +7186,7 @@ ok <func> <name name="system_flag" arity="2" clause_i="3" - anchor="system_flag_dirty_cpu_schedulers_online" since="?"/> + anchor="system_flag_dirty_cpu_schedulers_online" since="OTP 17.0"/> <fsummary>Set system_flag_dirty_cpu_schedulers_online.</fsummary> <desc> <p> @@ -7214,7 +7214,7 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="4" since="?"/> + <name name="system_flag" arity="2" clause_i="4" since="OTP 20.2.3"/> <fsummary>Set system flag for erts_alloc.</fsummary> <desc> <p>Sets system flags for @@ -7251,7 +7251,7 @@ ok <func> <name name="system_flag" arity="2" clause_i="6" - anchor="system_flag_microstate_accounting" since="?"/> + anchor="system_flag_microstate_accounting" since="OTP 19.0"/> <fsummary>Set system flag microstate_accounting.</fsummary> <desc> <p> @@ -7279,7 +7279,7 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="8" since="?"/> + <name name="system_flag" arity="2" clause_i="8" since="OTP R13B04"/> <fsummary>Set system flag min_bin_vheap_size.</fsummary> <desc> <p>Sets the default minimum binary virtual heap size for @@ -7297,7 +7297,7 @@ ok <func> <name name="system_flag" arity="2" clause_i="9" - anchor="system_flag_max_heap_size" since="?"/> + anchor="system_flag_max_heap_size" since="OTP 19.0"/> <fsummary>Set system flag max_heap_size.</fsummary> <type name="max_heap_size"/> <desc> @@ -7498,7 +7498,7 @@ ok <func> <name name="system_flag" arity="2" clause_i="12" - anchor="system_flag_scheduler_wall_time" since="?"/> + anchor="system_flag_scheduler_wall_time" since="OTP R15B01"/> <fsummary>Set system flag scheduler_wall_time.</fsummary> <desc> <p> @@ -7586,7 +7586,7 @@ Metadata = #{ pid => pid(), <func> <name name="system_flag" arity="2" clause_i="16" - anchor="system_flag_time_offset" since="?"/> + anchor="system_flag_time_offset" since="OTP 18.0"/> <fsummary>Finalize the time offset.</fsummary> <desc> <p> @@ -7909,7 +7909,7 @@ Metadata = #{ pid => pid(), anchor="system_info_cpu_topology" since=""/> <!-- cpu_topology --> <name name="system_info" arity="1" clause_i="13" since=""/> <!-- {cpu_topology, _} --> <name name="system_info" arity="1" clause_i="38" since=""/> <!-- logical_processors --> - <name name="system_info" arity="1" clause_i="74" since="?"/> <!-- update_cpu_info --> + <name name="system_info" arity="1" clause_i="74" since="OTP R14B"/> <!-- update_cpu_info --> <fsummary>Information about the CPU topology of the system.</fsummary> <type name="cpu_topology"/> <type name="level_entry"/> @@ -8061,14 +8061,14 @@ Metadata = #{ pid => pid(), <func> <name name="system_info" arity="1" clause_i="31" - anchor="system_info_process" since="?"/> <!-- fullsweep_after --> + anchor="system_info_process" since=""/> <!-- fullsweep_after --> <name name="system_info" arity="1" clause_i="32" since=""/> <!-- garbage_collection --> <name name="system_info" arity="1" clause_i="33" since=""/> <!-- heap_sizes --> <name name="system_info" arity="1" clause_i="34" since=""/> <!-- heap_type --> - <name name="system_info" arity="1" clause_i="40" since="?"/> <!-- max_heap_size --> - <name name="system_info" arity="1" clause_i="41" since="?"/> <!-- message_queue_data --> - <name name="system_info" arity="1" clause_i="42" since="?"/> <!-- min_heap_size --> - <name name="system_info" arity="1" clause_i="43" since="?"/> <!-- min_bin_vheap_size --> + <name name="system_info" arity="1" clause_i="40" since="OTP 19.0"/> <!-- max_heap_size --> + <name name="system_info" arity="1" clause_i="41" since="OTP 19.0"/> <!-- message_queue_data --> + <name name="system_info" arity="1" clause_i="42" since="OTP R13B04"/> <!-- min_heap_size --> + <name name="system_info" arity="1" clause_i="43" since="OTP R13B04"/> <!-- min_bin_vheap_size --> <name name="system_info" arity="1" clause_i="57" since=""/> <!-- procs --> <fsummary>Information about the default process heap settings.</fsummary> <type name="message_queue_data"/> @@ -8179,12 +8179,12 @@ Metadata = #{ pid => pid(), </func> <func> - <name name="system_info" arity="1" clause_i="6" anchor="system_info_limits" since="?"/> <!-- atom_count --> - <name name="system_info" arity="1" clause_i="7" since="?"/> <!-- atom_limit --> - <name name="system_info" arity="1" clause_i="29" since="?"/> <!-- ets_count --> - <name name="system_info" arity="1" clause_i="30" since="?"/> <!-- ets_limit --> - <name name="system_info" arity="1" clause_i="53" since="?"/> <!-- port_count --> - <name name="system_info" arity="1" clause_i="54" since="?"/> <!-- port_limit --> + <name name="system_info" arity="1" clause_i="6" anchor="system_info_limits" since="OTP 20.0"/> <!-- atom_count --> + <name name="system_info" arity="1" clause_i="7" since="OTP 20.0"/> <!-- atom_limit --> + <name name="system_info" arity="1" clause_i="29" since="OTP 21.1"/> <!-- ets_count --> + <name name="system_info" arity="1" clause_i="30" since="OTP R16B03"/> <!-- ets_limit --> + <name name="system_info" arity="1" clause_i="53" since="OTP R16B"/> <!-- port_count --> + <name name="system_info" arity="1" clause_i="54" since="OTP R16B"/> <!-- port_limit --> <name name="system_info" arity="1" clause_i="55" since=""/> <!-- process_count --> <name name="system_info" arity="1" clause_i="56" since=""/> <!-- process_limit --> <fsummary>Information about various system limits.</fsummary> @@ -8267,7 +8267,7 @@ Metadata = #{ pid => pid(), <name name="system_info" arity="1" clause_i="69" since="OTP 18.0"/> <!-- time_correction --> <name name="system_info" arity="1" clause_i="70" since="OTP 18.0"/> <!-- time_offset --> <name name="system_info" arity="1" clause_i="71" since="OTP 18.0"/> <!-- time_warp_mode --> - <name name="system_info" arity="1" clause_i="72" since="?"/> <!-- tolerant_timeofday --> + <name name="system_info" arity="1" clause_i="72" since="OTP 17.1"/> <!-- tolerant_timeofday --> <fsummary>Information about system time.</fsummary> <desc> <marker id="system_info_time_tags"/> @@ -8488,12 +8488,12 @@ Metadata = #{ pid => pid(), <func> <name name="system_info" arity="1" clause_i="17" - anchor="system_info_scheduler" since="?"/> <!-- dirty_cpu_schedulers --> - <name name="system_info" arity="1" clause_i="18" since="?"/> <!-- dirty_cpu_schedulers_online --> - <name name="system_info" arity="1" clause_i="19" since="?"/> <!-- dirty_io_schedulers --> + anchor="system_info_scheduler" since="OTP 17.0"/> <!-- dirty_cpu_schedulers --> + <name name="system_info" arity="1" clause_i="18" since="OTP 17.0"/> <!-- dirty_cpu_schedulers_online --> + <name name="system_info" arity="1" clause_i="19" since="OTP 17.0"/> <!-- dirty_io_schedulers --> <name name="system_info" arity="1" clause_i="45" since=""/> <!-- multi_scheduling --> <name name="system_info" arity="1" clause_i="46" since=""/> <!-- multi_scheduling_blockers --> - <name name="system_info" arity="1" clause_i="49" since="?"/> <!-- normal_multi_scheduling_blockers --> + <name name="system_info" arity="1" clause_i="49" since="OTP 19.0"/> <!-- normal_multi_scheduling_blockers --> <name name="system_info" arity="1" clause_i="58" since=""/> <!-- scheduler_bind_type --> <name name="system_info" arity="1" clause_i="59" since=""/> <!-- scheduler_bindings --> <name name="system_info" arity="1" clause_i="60" since=""/> <!-- scheduler_id --> @@ -8789,9 +8789,9 @@ Metadata = #{ pid => pid(), <func> <name name="system_info" arity="1" clause_i="14" anchor="system_info_dist" since=""/> <!-- creation --> - <name name="system_info" arity="1" clause_i="16" since="?"/> <!-- delayed_node_table_gc --> + <name name="system_info" arity="1" clause_i="16" since="OTP 18.0"/> <!-- delayed_node_table_gc --> <name name="system_info" arity="1" clause_i="20" since=""/> <!-- dist --> - <name name="system_info" arity="1" clause_i="21" since="?"/> <!-- dist_buf_busy_limit --> + <name name="system_info" arity="1" clause_i="21" since="OTP R14B01"/> <!-- dist_buf_busy_limit --> <name name="system_info" arity="1" clause_i="22" since=""/> <!-- dist_ctrl --> <fsummary>Information about erlang distribution.</fsummary> <desc> @@ -8866,7 +8866,7 @@ Metadata = #{ pid => pid(), <!-- <name name="system_info" arity="1" clause_i="6"/> atom_count --> <!-- <name name="system_info" arity="1" clause_i="7"/> atom_limit --> <name name="system_info" arity="1" clause_i="8" - anchor="system_info_misc" since="?"/> <!-- build_type --> + anchor="system_info_misc" since="OTP R14B"/> <!-- build_type --> <name name="system_info" arity="1" clause_i="9" since=""/> <!-- c_compiler_used --> <name name="system_info" arity="1" clause_i="10" since=""/> <!-- check_io --> <name name="system_info" arity="1" clause_i="11" since=""/> <!-- compat_rel --> @@ -8882,8 +8882,8 @@ Metadata = #{ pid => pid(), <!-- <name name="system_info" arity="1" clause_i="21"/> dist_buf_busy_limit --> <!-- <name name="system_info" arity="1" clause_i="22"/> dist_ctrl --> <name name="system_info" arity="1" clause_i="23" since=""/> <!-- driver_version --> - <name name="system_info" arity="1" clause_i="24" since="?"/> <!-- dynamic_trace --> - <name name="system_info" arity="1" clause_i="25" since="?"/> <!-- dynamic_trace_probes --> + <name name="system_info" arity="1" clause_i="24" since="OTP R15B01"/> <!-- dynamic_trace --> + <name name="system_info" arity="1" clause_i="25" since="OTP R15B01"/> <!-- dynamic_trace_probes --> <!-- <name name="system_info" arity="1" clause_i="26"/> end_time --> <!-- <name name="system_info" arity="1" clause_i="27"/> elib_malloc --> <!-- <name name="system_info" arity="1" clause_i="28"/> eager_check_io, removed --> @@ -8905,12 +8905,12 @@ Metadata = #{ pid => pid(), <name name="system_info" arity="1" clause_i="44" since=""/> <!-- modified_timing_level --> <!-- <name name="system_info" arity="1" clause_i="45"/> multi_scheduling --> <!-- <name name="system_info" arity="1" clause_i="46"/> multi_scheduling_blockers --> - <name name="system_info" arity="1" clause_i="47" since="?"/> <!-- nif_version --> + <name name="system_info" arity="1" clause_i="47" since="OTP 17.4"/> <!-- nif_version --> <!-- n<name name="system_info" arity="1" clause_i="48"/> ormal_multi_scheduling_blockers --> <name name="system_info" arity="1" clause_i="49" since=""/> <!-- otp_release --> <!-- <name name="system_info" arity="1" clause_i="50"/> os_monotonic_time_source --> <!-- <name name="system_info" arity="1" clause_i="51"/> os_system_time_source --> - <name name="system_info" arity="1" clause_i="52" since="?"/> <!-- port_parallelism --> + <name name="system_info" arity="1" clause_i="52" since="OTP R16B"/> <!-- port_parallelism --> <!-- <name name="system_info" arity="1" clause_i="53"/> port_count --> <!-- <name name="system_info" arity="1" clause_i="54"/> port_limit --> <!-- <name name="system_info" arity="1" clause_i="55"/> process_count --> @@ -10537,7 +10537,7 @@ timestamp() -> </func> <func> - <name name="trace_pattern" arity="3" clause_i="1" since="?"/> + <name name="trace_pattern" arity="3" clause_i="1" since="OTP 19.0"/> <fsummary>Set trace pattern for message sending.</fsummary> <type name="trace_match_spec"/> <type name="match_variable"/> @@ -10608,7 +10608,7 @@ timestamp() -> </func> <func> - <name name="trace_pattern" arity="3" clause_i="2" since="?"/> + <name name="trace_pattern" arity="3" clause_i="2" since="OTP 19.0"/> <fsummary>Set trace pattern for tracing of message receiving.</fsummary> <type name="trace_match_spec"/> <type name="match_variable"/> diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index 19093ebfdd..dca502c939 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -340,6 +340,7 @@ erts_sched_stat_t erts_sched_stat; static erts_tsd_key_t ERTS_WRITE_UNLIKELY(sched_data_key); #if ERTS_POLL_USE_SCHEDULER_POLLING +static erts_atomic32_t function_calls; static erts_atomic32_t doing_sys_schedule; #endif static erts_atomic32_t no_empty_run_queues; @@ -3247,6 +3248,7 @@ poll_thread(void *arg) static ERTS_INLINE void clear_sys_scheduling(void) { + erts_atomic32_set_relb(&function_calls, 0); erts_atomic32_set_mb(&doing_sys_schedule, 0); } @@ -3269,28 +3271,6 @@ prepare_for_sys_schedule(void) return 0; } -static void -check_io_timer(void *null) -{ - ErtsSchedulerData *esdp = erts_get_scheduler_data(); - if (prepare_for_sys_schedule()) { - erts_check_io(esdp->ssi->psi, ERTS_POLL_NO_TIMEOUT); - clear_sys_scheduling(); - } - - /* The timer is cleared if this schedulers run-queue became empty - or if the CHECKIO flag was cleared. The CHECKIO flags is cleared - when a check_balance assigns another scheduler to be the poller in - the overload scenario. */ - if ((ERTS_RUNQ_FLGS_GET_NOB(esdp->run_queue) & (ERTS_RUNQ_FLG_OUT_OF_WORK|ERTS_RUNQ_FLG_CHECKIO)) - == ERTS_RUNQ_FLG_CHECKIO) { - erts_start_timer_callback(ERTS_POLL_SCHEDULER_POLLING_TIMEOUT, - check_io_timer, NULL); - } else { - ERTS_RUNQ_FLGS_UNSET(esdp->run_queue, ERTS_RUNQ_FLG_CHECKIO); - } -} - #else #define clear_sys_scheduling() #define prepare_for_sys_schedule() 0 @@ -3448,6 +3428,7 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) current_time = erts_get_monotonic_time(esdp); } } + *fcalls = 0; clear_sys_scheduling(); } else { if (!ERTS_SCHEDULER_IS_DIRTY(esdp)) { @@ -4711,15 +4692,6 @@ check_balance(ErtsRunQueue *c_rq) if (blnc_no_rqs == 1) { c_rq->check_balance_reds = INT_MAX; erts_atomic32_set_nob(&balance_info.checking_balance, 0); -#if ERTS_POLL_USE_SCHEDULER_POLLING - c_rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; - if ((ERTS_RUNQ_FLGS_GET_NOB(c_rq) & (ERTS_RUNQ_FLG_OUT_OF_WORK|ERTS_RUNQ_FLG_CHECKIO)) - == 0) { - ERTS_RUNQ_FLGS_SET(c_rq, ERTS_RUNQ_FLG_CHECKIO); - erts_start_timer_callback(ERTS_POLL_SCHEDULER_POLLING_TIMEOUT, check_io_timer, NULL); - } - ERTS_RUNQ_FLGS_UNSET(c_rq, ERTS_RUNQ_FLGS_MIGRATION_INFO); -#endif return; } @@ -5239,19 +5211,6 @@ erts_fprintf(stderr, "--------------------------------\n"); /* Publish new migration paths... */ erts_atomic_set_wb(&erts_migration_paths, (erts_aint_t) new_mpaths); -#if ERTS_POLL_USE_SCHEDULER_POLLING - if (full_scheds == current_active) { - ERTS_ASSERT(full_scheds <= current_active); - /* All active schedulers ran for full, we need to do active polling, - so we setup a timer that does active polling */ - if (!(ERTS_RUNQ_FLGS_GET_NOB(c_rq) & ERTS_RUNQ_FLG_CHECKIO)) { - /* Active polling is not running, start it */ - erts_start_timer_callback(ERTS_POLL_SCHEDULER_POLLING_TIMEOUT, check_io_timer, NULL); - } - run_queue_info[c_rq->ix].flags |= ERTS_RUNQ_FLG_CHECKIO; - } -#endif - /* Reset balance statistics in all online queues */ for (qix = 0; qix < blnc_no_rqs; qix++) { Uint32 flags = run_queue_info[qix].flags; @@ -5261,8 +5220,6 @@ erts_fprintf(stderr, "--------------------------------\n"); ASSERT(!(flags & ERTS_RUNQ_FLG_OUT_OF_WORK)); if (rq->waiting) flags |= ERTS_RUNQ_FLG_OUT_OF_WORK; - if (rq != c_rq) - flags &= ~ERTS_RUNQ_FLG_CHECKIO; rq->full_reds_history_sum = run_queue_info[qix].full_reds_history_sum; @@ -5272,7 +5229,7 @@ erts_fprintf(stderr, "--------------------------------\n"); ERTS_DBG_CHK_FULL_REDS_HISTORY(rq); rq->out_of_work_count = 0; - (void) ERTS_RUNQ_FLGS_READ_BSET(rq, ERTS_RUNQ_FLGS_MIGRATION_INFO|ERTS_RUNQ_FLG_CHECKIO, flags); + (void) ERTS_RUNQ_FLGS_READ_BSET(rq, ERTS_RUNQ_FLGS_MIGRATION_INFO, flags); rq->max_len = erts_atomic32_read_dirty(&rq->len); for (pix = 0; pix < ERTS_NO_PRIO_LEVELS; pix++) { ErtsRunQueueInfo *rqi; @@ -5923,6 +5880,7 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online, int no_poll_th erts_alloc_permanent_cache_aligned(ERTS_ALC_T_RUNQS, size_runqs); #if ERTS_POLL_USE_SCHEDULER_POLLING erts_atomic32_init_nob(&doing_sys_schedule, 0); + erts_atomic32_init_nob(&function_calls, 0); #endif erts_atomic32_init_nob(&no_empty_run_queues, 0); @@ -9277,7 +9235,7 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) Process *proxy_p = NULL; ErtsRunQueue *rq; int context_reds; - int fcalls; + int fcalls = 0; int actual_reds; int reds; Uint32 flags; @@ -9351,6 +9309,10 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) reds = ERTS_PROC_MIN_CONTEXT_SWITCH_REDS_COST; esdp->virtual_reds = 0; +#if ERTS_POLL_USE_SCHEDULER_POLLING + fcalls = (int) erts_atomic32_add_read_acqb(&function_calls, reds); +#endif + ASSERT(esdp && esdp == erts_get_scheduler_data()); rq = erts_get_runq_current(esdp); @@ -9567,7 +9529,33 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) non_empty_runq(rq); goto check_activities_to_run; - } + } else if (is_normal_sched && + fcalls > (2 * context_reds) && + prepare_for_sys_schedule()) { + ErtsMonotonicTime current_time; + /* + * Schedule system-level activities. + */ + + ERTS_MSACC_PUSH_STATE_CACHED_M(); + + erts_runq_unlock(rq); + + ERTS_MSACC_SET_STATE_CACHED_M(ERTS_MSACC_STATE_CHECK_IO); + LTTNG2(scheduler_poll, esdp->no, 1); + + erts_check_io(esdp->ssi->psi, ERTS_POLL_NO_TIMEOUT); + ERTS_MSACC_POP_STATE_M(); + + current_time = erts_get_monotonic_time(esdp); + if (current_time >= erts_next_timeout_time(esdp->next_tmo_ref)) + erts_bump_timers(esdp->timer_wheel, current_time); + + erts_runq_lock(rq); + fcalls = 0; + clear_sys_scheduling(); + goto continue_check_activities_to_run; + } if (flags & ERTS_RUNQ_FLG_MISC_OP) exec_misc_ops(rq); diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index 0aa19e7bde..43937f216c 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -173,8 +173,6 @@ extern int erts_dio_sched_thread_suggested_stack_size; (((Uint32) 1) << (ERTS_RUNQ_FLG_BASE2 + 9)) #define ERTS_RUNQ_FLG_HALTING \ (((Uint32) 1) << (ERTS_RUNQ_FLG_BASE2 + 10)) -#define ERTS_RUNQ_FLG_CHECKIO \ - (((Uint32) 1) << (ERTS_RUNQ_FLG_BASE2 + 11)) #define ERTS_RUNQ_FLG_MAX (ERTS_RUNQ_FLG_BASE2 + 12) diff --git a/erts/emulator/beam/erl_trace.c b/erts/emulator/beam/erl_trace.c index 701fb38147..ae7084b7f4 100644 --- a/erts/emulator/beam/erl_trace.c +++ b/erts/emulator/beam/erl_trace.c @@ -72,7 +72,7 @@ static ErtsTracer default_port_tracer; static Eterm system_monitor; static Eterm system_profile; -static erts_aint_t system_logger; +static erts_atomic_t system_logger; #ifdef HAVE_ERTS_NOW_CPU int erts_cpu_timestamp; diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index b44464d6da..ed687b8d70 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -12960,38 +12960,40 @@ make_noninheritable_handle(SOCKET s) static void fire_multi_timers(tcp_descriptor *desc, ErlDrvPort port, ErlDrvData data) { - ErlDrvTime next_timeout; - MultiTimerData *curr = desc->mtd; - if (!curr) { - ASSERT(0); - return; + ErlDrvTime next_timeout = 0; + if (!desc->mtd) { + ASSERT(0); + return; } #ifdef DEBUG { ErlDrvTime chk = erl_drv_monotonic_time(ERL_DRV_MSEC); - ASSERT(chk >= curr->when); + ASSERT(chk >= desc->mtd->when); } #endif do { - MultiTimerData *save = curr; + MultiTimerData save = *desc->mtd; - (*(save->timeout_function))(data,save->caller); + /* We first remove the timer so that the timeout_functions has + can call clean_multi_timers without breaking anything */ + if (desc->mtd_cache == NULL) { + desc->mtd_cache = desc->mtd; + } else { + FREE(desc->mtd); + } - curr = curr->next; + desc->mtd = save.next; + if (desc->mtd != NULL) + desc->mtd->prev = NULL; - if (desc->mtd_cache == NULL) - desc->mtd_cache = save; - else - FREE(save); + (*(save.timeout_function))(data,save.caller); - if (curr == NULL) { - desc->mtd = NULL; + if (desc->mtd == NULL) return; - } - curr->prev = NULL; - next_timeout = curr->when - erl_drv_monotonic_time(ERL_DRV_MSEC); + + next_timeout = desc->mtd->when - erl_drv_monotonic_time(ERL_DRV_MSEC); } while (next_timeout <= 0); - desc->mtd = curr; + driver_set_timer(port, (unsigned long) next_timeout); } diff --git a/erts/emulator/sys/common/erl_check_io.c b/erts/emulator/sys/common/erl_check_io.c index c681fa481f..ac9a070bce 100644 --- a/erts/emulator/sys/common/erl_check_io.c +++ b/erts/emulator/sys/common/erl_check_io.c @@ -2262,14 +2262,14 @@ erts_check_io_info(void *proc) #if ERTS_POLL_USE_FALLBACK erts_poll_info_flbk(get_fallback_pollset(), &piv[0]); - piv[0].poll_threads = 1; + piv[0].poll_threads = 0; piv[0].active_fds = 0; piv++; #endif #if ERTS_POLL_USE_SCHEDULER_POLLING erts_poll_info(get_scheduler_pollset(0), &piv[0]); - piv[0].poll_threads = 1; + piv[0].poll_threads = 0; piv[0].active_fds = 0; piv++; #endif diff --git a/erts/emulator/sys/common/erl_poll.c b/erts/emulator/sys/common/erl_poll.c index 51d50933ff..27ffba58bd 100644 --- a/erts/emulator/sys/common/erl_poll.c +++ b/erts/emulator/sys/common/erl_poll.c @@ -2326,6 +2326,7 @@ uint32_t epoll_events(int kp_fd, int fd) { /* For epoll we read the information about what is selected upon from the proc fs.*/ char fname[30]; + char s[256]; FILE *f; unsigned int pos, flags, mnt_id; int line = 0; @@ -2343,12 +2344,12 @@ uint32_t epoll_events(int kp_fd, int fd) } if (fscanf(f,"\nmnt_id:\t%x\n", &mnt_id)); line += 3; - while (!feof(f)) { + while (fgets(s, sizeof(s) / sizeof(*s), f)) { /* tfd: 10 events: 40000019 data: 180000000a */ int ev_fd; uint32_t events; uint64_t data; - if (fscanf(f,"tfd:%d events:%x data:%llx\n", &ev_fd, &events, + if (sscanf(s,"tfd:%d events:%x data:%llx", &ev_fd, &events, (unsigned long long*)&data) != 3) { fprintf(stderr,"failed to parse file %s on line %d, errno = %d\n", fname, line, @@ -2392,6 +2393,7 @@ ERTS_POLL_EXPORT(erts_poll_get_selected_events)(ErtsPollSet *ps, /* For epoll we read the information about what is selected upon from the proc fs.*/ char fname[30]; + char s[256]; FILE *f; unsigned int pos, flags, mnt_id; int line = 0; @@ -2410,12 +2412,12 @@ ERTS_POLL_EXPORT(erts_poll_get_selected_events)(ErtsPollSet *ps, } if (fscanf(f,"\nmnt_id:\t%x\n", &mnt_id)); line += 3; - while (!feof(f)) { + while (fgets(s, sizeof(s) / sizeof(*s), f)) { /* tfd: 10 events: 40000019 data: 180000000a */ int fd; uint32_t events; uint64_t data; - if (fscanf(f,"tfd:%d events:%x data:%llx\n", &fd, &events, + if (sscanf(s,"tfd:%d events:%x data:%llx", &fd, &events, (unsigned long long*)&data) != 3) { fprintf(stderr,"failed to parse file %s on line %d, errno = %d\n", fname, line, errno); diff --git a/erts/emulator/sys/unix/sys_drivers.c b/erts/emulator/sys/unix/sys_drivers.c index 816bdea9c5..2f5459bee5 100644 --- a/erts/emulator/sys/unix/sys_drivers.c +++ b/erts/emulator/sys/unix/sys_drivers.c @@ -998,9 +998,9 @@ static void clear_fd_data(ErtsSysFdData *fdd) fdd->psz = 0; } -static void nbio_stop_fd(ErlDrvPort prt, ErtsSysFdData *fdd) +static void nbio_stop_fd(ErlDrvPort prt, ErtsSysFdData *fdd, int use) { - driver_select(prt, abs(fdd->fd), ERL_DRV_USE_NO_CALLBACK|DO_READ|DO_WRITE, 0); + driver_select(prt, abs(fdd->fd), use ? ERL_DRV_USE_NO_CALLBACK : 0|DO_READ|DO_WRITE, 0); clear_fd_data(fdd); SET_BLOCKING(abs(fdd->fd)); @@ -1020,11 +1020,11 @@ static void fd_stop(ErlDrvData ev) /* Does not close the fds */ if (dd->ifd) { sz += sizeof(ErtsSysFdData); - nbio_stop_fd(prt, dd->ifd); + nbio_stop_fd(prt, dd->ifd, 1); } if (dd->ofd && dd->ofd != dd->ifd) { sz += sizeof(ErtsSysFdData); - nbio_stop_fd(prt, dd->ofd); + nbio_stop_fd(prt, dd->ofd, 1); } erts_free(ERTS_ALC_T_DRV_TAB, dd); @@ -1070,12 +1070,12 @@ static void stop(ErlDrvData ev) ErlDrvPort prt = dd->port_num; if (dd->ifd) { - nbio_stop_fd(prt, dd->ifd); + nbio_stop_fd(prt, dd->ifd, 0); driver_select(prt, abs(dd->ifd->fd), ERL_DRV_USE, 0); /* close(ifd); */ } if (dd->ofd && dd->ofd != dd->ifd) { - nbio_stop_fd(prt, dd->ofd); + nbio_stop_fd(prt, dd->ofd, 0); driver_select(prt, abs(dd->ofd->fd), ERL_DRV_USE, 0); /* close(ofd); */ } diff --git a/erts/emulator/test/driver_SUITE.erl b/erts/emulator/test/driver_SUITE.erl index bd62708aa7..1d2ae4fb51 100644 --- a/erts/emulator/test/driver_SUITE.erl +++ b/erts/emulator/test/driver_SUITE.erl @@ -1069,14 +1069,19 @@ get_stable_check_io_info(N) -> get_check_io_total(ChkIo) -> ct:log("ChkIo = ~p~n",[ChkIo]), {Fallback, Rest} = get_fallback(ChkIo), + OnlyPollThreads = [PS || PS <- Rest, not is_scheduler_pollset(PS)], add_fallback_infos(Fallback, - lists:foldl(fun(Pollset, Acc) -> - lists:zipwith(fun(A, B) -> - add_pollset_infos(A,B) - end, - Pollset, Acc) - end, - hd(Rest), tl(Rest))). + lists:foldl( + fun(Pollset, Acc) -> + lists:zipwith(fun(A, B) -> + add_pollset_infos(A,B) + end, + Pollset, Acc) + end, + hd(OnlyPollThreads), tl(OnlyPollThreads))). + +is_scheduler_pollset(Pollset) -> + proplists:get_value(poll_threads, Pollset) == 0. add_pollset_infos({Tag, A}=TA , {Tag, B}=TB) -> case tag_type(Tag) of diff --git a/erts/emulator/test/scheduler_SUITE.erl b/erts/emulator/test/scheduler_SUITE.erl index 2e0dfa42f3..f61949c75b 100644 --- a/erts/emulator/test/scheduler_SUITE.erl +++ b/erts/emulator/test/scheduler_SUITE.erl @@ -1450,28 +1450,26 @@ poll_threads(Config) when is_list(Config) -> {Conc, PollType, KP} = get_ioconfig(Config), {Sched, SchedOnln, _} = get_sstate(Config, ""), + [1, 1] = get_ionum(Config,"+IOt 2 +IOp 2"), + [1, 1, 1, 1, 1] = get_ionum(Config,"+IOt 5 +IOp 5"), + [1, 1] = get_ionum(Config, "+S 2 +IOPt 100 +IOPp 100"), + if Conc -> - [1, 1, 1] = get_ionum(Config,"+IOt 2 +IOp 2"), - [1, 1, 1, 1, 1, 1] = get_ionum(Config,"+IOt 5 +IOp 5"), - [1, 1, 1] = get_ionum(Config, "+S 2 +IOPt 100 +IOPp 100"), - [5, 1] = get_ionum(Config,"+IOt 5 +IOp 1"), - [3, 2, 1] = get_ionum(Config,"+IOt 5 +IOp 2"), - [2, 2, 2, 2, 2, 1] = get_ionum(Config,"+IOt 10 +IOPp 50"), + [5] = get_ionum(Config,"+IOt 5 +IOp 1"), + [3, 2] = get_ionum(Config,"+IOt 5 +IOp 2"), + [2, 2, 2, 2, 2] = get_ionum(Config,"+IOt 10 +IOPp 50"), - [2, 1] = get_ionum(Config, "+S 2 +IOPt 100"), - [4, 1] = get_ionum(Config, "+S 4 +IOPt 100"), - [4, 1] = get_ionum(Config, "+S 4:2 +IOPt 100"), - [4, 4, 1] = get_ionum(Config, "+S 8 +IOPt 100 +IOPp 25"), + [2] = get_ionum(Config, "+S 2 +IOPt 100"), + [4] = get_ionum(Config, "+S 4 +IOPt 100"), + [4] = get_ionum(Config, "+S 4:2 +IOPt 100"), + [4, 4] = get_ionum(Config, "+S 8 +IOPt 100 +IOPp 25"), fail = get_ionum(Config, "+IOt 1 +IOp 2"), ok; not Conc -> - [1, 1] = get_ionum(Config,"+IOt 2 +IOp 2"), - [1, 1, 1, 1, 1] = get_ionum(Config,"+IOt 5 +IOp 5"), - [1, 1] = get_ionum(Config, "+S 2 +IOPt 100 +IOPp 100"), [1, 1, 1, 1, 1] = get_ionum(Config,"+IOt 5 +IOp 1"), [1, 1, 1, 1, 1] = get_ionum(Config,"+IOt 5 +IOp 2"), @@ -1515,7 +1513,8 @@ get_iostate(Config, Cmd)-> erlang:system_info(check_io) end]), IO = [IOState || IOState <- IOStates, - proplists:get_value(fallback, IOState) == false], + proplists:get_value(fallback, IOState) == false, + proplists:get_value(poll_threads, IOState) /= 0], stop_node(Node), IO; {error,timeout} -> diff --git a/erts/emulator/test/z_SUITE.erl b/erts/emulator/test/z_SUITE.erl index 1c52e1a934..6549108126 100644 --- a/erts/emulator/test/z_SUITE.erl +++ b/erts/emulator/test/z_SUITE.erl @@ -251,7 +251,7 @@ pollset_size(Config) when is_list(Config) -> end. check_io_debug(Config) when is_list(Config) -> - case lists:keysearch(name, 1, erlang:system_info(check_io)) of + case lists:keysearch(name, 1, hd(erlang:system_info(check_io))) of {value, {name, erts_poll}} -> check_io_debug_test(); _ -> {skipped, "Not implemented in this emulator"} end. diff --git a/lib/common_test/doc/src/ct_netconfc.xml b/lib/common_test/doc/src/ct_netconfc.xml index 32a1175d81..8fbe5f3df6 100644 --- a/lib/common_test/doc/src/ct_netconfc.xml +++ b/lib/common_test/doc/src/ct_netconfc.xml @@ -412,11 +412,11 @@ </func> <func> - <name since="OTP 18.3">create_subscription(Client) -> Result</name> - <name since="OTP 18.3">create_subscription(Client, Stream) -> Result</name> - <name since="OTP 18.3">create_subscription(Client, Stream, Filter) -> Result</name> - <name since="OTP 18.3">create_subscription(Client, Stream, Filter, Timeout) -> Result</name> - <name name="create_subscription" arity="5" clause_i="2" since="OTP 18.3"/> + <name since="OTP R15B02">create_subscription(Client) -> Result</name> + <name since="OTP R15B02">create_subscription(Client, Stream) -> Result</name> + <name since="OTP R15B02">create_subscription(Client, Stream, Filter) -> Result</name> + <name since="OTP R15B02">create_subscription(Client, Stream, Filter, Timeout) -> Result</name> + <name name="create_subscription" arity="5" clause_i="2" since="OTP R15B02"/> <name name="create_subscription" arity="6" since="OTP R15B02"/> <fsummary>Creates a subscription for event notifications.</fsummary> <desc> @@ -515,7 +515,7 @@ create_subscription(Client, Stream, Filter, StartTime, StopTime, Timeout)</pre> <func> <name name="edit_config" arity="3" since="OTP R15B02"/> - <name name="edit_config" arity="4" clause_i="1" since="OTP R15B02"/> + <name name="edit_config" arity="4" clause_i="1" since="OTP 18.0"/> <name name="edit_config" arity="4" clause_i="2" since="OTP R15B02"/> <name name="edit_config" arity="5" since="OTP 18.0"/> <fsummary>Edits configuration data.</fsummary> @@ -599,7 +599,7 @@ create_subscription(Client, Stream, Filter, StartTime, StopTime, Timeout)</pre> <func> <name name="get_event_streams" arity="1" since="OTP 20.0"/> <name name="get_event_streams" arity="2" clause_i="1" since="OTP R15B02"/> - <name name="get_event_streams" arity="2" clause_i="2" since="OTP R15B02"/> + <name name="get_event_streams" arity="2" clause_i="2" since="OTP 20.0"/> <name name="get_event_streams" arity="3" since="OTP R15B02"/> <fsummary>Sends a request to get the specified event streams.</fsummary> <desc> diff --git a/lib/inets/src/http_client/httpc_handler.erl b/lib/inets/src/http_client/httpc_handler.erl index 1bf5d25c98..8b356d8026 100644 --- a/lib/inets/src/http_client/httpc_handler.erl +++ b/lib/inets/src/http_client/httpc_handler.erl @@ -805,11 +805,12 @@ handle_unix_socket_options(#request{unix_socket = UnixSocket}, error({badarg, [{ipfamily, Else}, {unix_socket, UnixSocket}]}) end. -connect_and_send_first_request(Address, Request, #state{options = Options0} = State) -> +connect_and_send_first_request(Address, #request{ipv6_host_with_brackets = HasBrackets} = Request, + #state{options = Options0} = State) -> SocketType = socket_type(Request), ConnTimeout = (Request#request.settings)#http_options.connect_timeout, Options = handle_unix_socket_options(Request, Options0), - case connect(SocketType, Address, Options, ConnTimeout) of + case connect(SocketType, format_address(Address, HasBrackets), Options, ConnTimeout) of {ok, Socket} -> ClientClose = httpc_request:is_client_closing( @@ -1739,3 +1740,8 @@ update_session(ProfileName, #session{id = SessionId} = Session, Pos, Value) -> end. +format_address({Host, Port}, true) when is_list(Host)-> + {ok, Address} = inet:parse_address(string:strip(string:strip(Host, right, $]), left, $[)), + {Address, Port}; +format_address(HostPort, _) -> + HostPort. diff --git a/lib/kernel/doc/src/gen_sctp.xml b/lib/kernel/doc/src/gen_sctp.xml index 1e7009b3a8..f70d6c24db 100644 --- a/lib/kernel/doc/src/gen_sctp.xml +++ b/lib/kernel/doc/src/gen_sctp.xml @@ -284,7 +284,7 @@ connect(Socket, Ip, Port>, <func> <name name="listen" arity="2" clause_i="1" since=""/> - <name name="listen" arity="2" clause_i="2" since=""/> + <name name="listen" arity="2" clause_i="2" since="OTP R15B"/> <fsummary>Set up a socket to listen.</fsummary> <desc> <p>Sets up a socket to listen on the IP address and port number diff --git a/lib/kernel/test/gen_tcp_misc_SUITE.erl b/lib/kernel/test/gen_tcp_misc_SUITE.erl index 244bd7e2a0..52edfaee29 100644 --- a/lib/kernel/test/gen_tcp_misc_SUITE.erl +++ b/lib/kernel/test/gen_tcp_misc_SUITE.erl @@ -53,7 +53,7 @@ active_once_closed/1, send_timeout/1, send_timeout_active/1, otp_7731/1, zombie_sockets/1, otp_7816/1, otp_8102/1, wrapping_oct/0, wrapping_oct/1, otp_9389/1, otp_13939/1, - otp_12242/1]). + otp_12242/1, delay_send_error/1]). %% Internal exports. -export([sender/3, not_owner/1, passive_sockets_server/2, priority_server/1, @@ -97,7 +97,7 @@ all() -> active_once_closed, send_timeout, send_timeout_active, otp_7731, wrapping_oct, zombie_sockets, otp_7816, otp_8102, otp_9389, - otp_12242]. + otp_12242, delay_send_error]. groups() -> []. @@ -3427,3 +3427,32 @@ otp_12242(Addr) when tuple_size(Addr) =:= 4 -> wait(Mref) -> receive {'DOWN',Mref,_,_,Reason} -> Reason end. + +%% OTP-15536 +%% Test that send error works correctly for delay_send +delay_send_error(Config) -> + {ok, LS} = gen_tcp:listen(0, [{reuseaddr, true}, {packet, 1}, {active, false}]), + {ok,{{0,0,0,0},PortNum}}=inet:sockname(LS), + P = spawn_link( + fun() -> + {ok, S} = gen_tcp:accept(LS), + receive die -> gen_tcp:close(S) end + end), + erlang:monitor(process, P), + {ok, S} = gen_tcp:connect("localhost", PortNum, + [{packet, 1}, {active, false}, {delay_send, true}]), + + %% Do a couple of sends first to see that it works + ok = gen_tcp:send(S, "hello"), + ok = gen_tcp:send(S, "hello"), + ok = gen_tcp:send(S, "hello"), + + %% Make the receiver close + P ! die, + receive _Down -> ok end, + + ok = gen_tcp:send(S, "hello"), + timer:sleep(500), %% Sleep in order for delay_send to have time to trigger + + %% This used to result in a double free + {error, closed} = gen_tcp:send(S, "hello"). diff --git a/lib/mnesia/doc/src/mnesia.xml b/lib/mnesia/doc/src/mnesia.xml index 94f1af34bf..11b0b8e987 100644 --- a/lib/mnesia/doc/src/mnesia.xml +++ b/lib/mnesia/doc/src/mnesia.xml @@ -2077,6 +2077,13 @@ mnesia:create_table(employee, <fsummary>Starts a local Mnesia system.</fsummary> <desc> <marker id="start"></marker> + <p>Mnesia startup is asynchronous. The function call + <c>mnesia:start()</c> returns the atom <c>ok</c> and then + starts to initialize the different tables. Depending on the + size of the database, this can take some time, and the + application programmer must wait for the tables that the + application needs before they can be used. This is achieved + by using the function <c>mnesia:wait_for_tables/2</c>.</p> <p>The startup procedure for a set of Mnesia nodes is a fairly complicated operation. A Mnesia system consists of a set of nodes, with Mnesia started locally on all diff --git a/lib/ssl/src/dtls_connection.erl b/lib/ssl/src/dtls_connection.erl index 2583667fa2..855cd0d123 100644 --- a/lib/ssl/src/dtls_connection.erl +++ b/lib/ssl/src/dtls_connection.erl @@ -107,9 +107,11 @@ pids(_) -> %%==================================================================== %% State transition handling %%==================================================================== -next_record(#state{unprocessed_handshake_events = N} = State) when N > 0 -> - {no_record, State#state{unprocessed_handshake_events = N-1}}; - +next_record(#state{handshake_env = + #handshake_env{unprocessed_handshake_events = N} = HsEnv} + = State) when N > 0 -> + {no_record, State#state{handshake_env = + HsEnv#handshake_env{unprocessed_handshake_events = N-1}}}; next_record(#state{protocol_buffers = #protocol_buffers{dtls_cipher_texts = [#ssl_tls{epoch = Epoch} = CT | Rest]} = Buffers, @@ -249,19 +251,22 @@ handle_protocol_record(#ssl_tls{type = ?HANDSHAKE, fragment = Data}, StateName, #state{protocol_buffers = Buffers0, - negotiated_version = Version} = State0) -> + negotiated_version = Version} = State) -> try case dtls_handshake:get_dtls_handshake(Version, Data, Buffers0) of {[], Buffers} -> - next_event(StateName, no_record, State0#state{protocol_buffers = Buffers}); + next_event(StateName, no_record, State#state{protocol_buffers = Buffers}); {Packets, Buffers} -> - State = State0#state{protocol_buffers = Buffers}, + HsEnv = State#state.handshake_env, Events = dtls_handshake_events(Packets), {next_state, StateName, - State#state{unprocessed_handshake_events = unprocessed_events(Events)}, Events} + State#state{protocol_buffers = Buffers, + handshake_env = + HsEnv#handshake_env{unprocessed_handshake_events + = unprocessed_events(Events)}}, Events} end catch throw:#alert{} = Alert -> - handle_own_alert(Alert, Version, StateName, State0) + handle_own_alert(Alert, Version, StateName, State) end; %%% DTLS record protocol level change cipher messages handle_protocol_record(#ssl_tls{type = ?CHANGE_CIPHER_SPEC, fragment = Data}, StateName, State) -> @@ -299,7 +304,7 @@ send_handshake(Handshake, #state{connection_states = ConnectionStates} = State) #{epoch := Epoch} = ssl_record:current_connection_state(ConnectionStates, write), send_handshake_flight(queue_handshake(Handshake, State), Epoch). -queue_handshake(Handshake0, #state{tls_handshake_history = Hist0, +queue_handshake(Handshake0, #state{handshake_env = #handshake_env{tls_handshake_history = Hist0} = HsEnv, negotiated_version = Version, flight_buffer = #{handshakes := HsBuffer0, change_cipher_spec := undefined, @@ -308,9 +313,9 @@ queue_handshake(Handshake0, #state{tls_handshake_history = Hist0, Hist = update_handshake_history(Handshake0, Handshake, Hist0), State#state{flight_buffer = Flight0#{handshakes => [Handshake | HsBuffer0], next_sequence => Seq +1}, - tls_handshake_history = Hist}; + handshake_env = HsEnv#handshake_env{tls_handshake_history = Hist}}; -queue_handshake(Handshake0, #state{tls_handshake_history = Hist0, +queue_handshake(Handshake0, #state{handshake_env = #handshake_env{tls_handshake_history = Hist0} = HsEnv, negotiated_version = Version, flight_buffer = #{handshakes_after_change_cipher_spec := Buffer0, next_sequence := Seq} = Flight0} = State) -> @@ -318,7 +323,7 @@ queue_handshake(Handshake0, #state{tls_handshake_history = Hist0, Hist = update_handshake_history(Handshake0, Handshake, Hist0), State#state{flight_buffer = Flight0#{handshakes_after_change_cipher_spec => [Handshake | Buffer0], next_sequence => Seq +1}, - tls_handshake_history = Hist}. + handshake_env = HsEnv#handshake_env{tls_handshake_history = Hist}}. queue_change_cipher(ChangeCipher, #state{flight_buffer = Flight, connection_states = ConnectionStates0} = State) -> @@ -330,10 +335,11 @@ queue_change_cipher(ChangeCipher, #state{flight_buffer = Flight, reinit(State) -> %% To be API compatible with TLS NOOP here reinit_handshake_data(State). -reinit_handshake_data(#state{protocol_buffers = Buffers} = State) -> +reinit_handshake_data(#state{protocol_buffers = Buffers, + handshake_env = HsEnv} = State) -> State#state{premaster_secret = undefined, public_key_info = undefined, - tls_handshake_history = ssl_handshake:init_handshake_history(), + handshake_env = HsEnv#handshake_env{tls_handshake_history = ssl_handshake:init_handshake_history()}, flight_state = {retransmit, ?INITIAL_RETRANSMIT_TIMEOUT}, flight_buffer = new_flight(), protocol_buffers = @@ -417,10 +423,10 @@ init({call, From}, {start, Timeout}, role = client, session_cache = Cache, session_cache_cb = CacheCb}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, ssl_options = SslOpts, session = #session{own_certificate = Cert} = Session0, - connection_states = ConnectionStates0, - renegotiation = {Renegotiation, _} + connection_states = ConnectionStates0 } = State0) -> Timer = ssl_connection:start_or_recv_cancel_timer(Timeout, From), Hello = dtls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, @@ -487,6 +493,7 @@ hello(internal, #client_hello{cookie = <<>>, #state{static_env = #static_env{role = server, transport_cb = Transport, socket = Socket}, + handshake_env = HsEnv, protocol_specific = #{current_cookie_secret := Secret}} = State0) -> {ok, {IP, Port}} = dtls_socket:peername(Transport, Socket), Cookie = dtls_handshake:cookie(Secret, IP, Port, Hello), @@ -500,24 +507,30 @@ hello(internal, #client_hello{cookie = <<>>, State1 = prepare_flight(State0#state{negotiated_version = Version}), {State2, Actions} = send_handshake(VerifyRequest, State1), {Record, State} = next_record(State2), - next_event(?FUNCTION_NAME, Record, State#state{tls_handshake_history = ssl_handshake:init_handshake_history()}, Actions); + next_event(?FUNCTION_NAME, Record, + State#state{handshake_env = HsEnv#handshake_env{ + tls_handshake_history = + ssl_handshake:init_handshake_history()}}, + Actions); hello(internal, #hello_verify_request{cookie = Cookie}, #state{static_env = #static_env{role = client, host = Host, port = Port, session_cache = Cache, session_cache_cb = CacheCb}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}} = HsEnv, ssl_options = SslOpts, session = #session{own_certificate = OwnCert} = Session0, - connection_states = ConnectionStates0, - renegotiation = {Renegotiation, _} + connection_states = ConnectionStates0 } = State0) -> Hello = dtls_handshake:client_hello(Host, Port, Cookie, ConnectionStates0, SslOpts, Cache, CacheCb, Renegotiation, OwnCert), Version = Hello#client_hello.client_version, - State1 = prepare_flight(State0#state{tls_handshake_history = ssl_handshake:init_handshake_history()}), + State1 = prepare_flight(State0#state{handshake_env = + HsEnv#handshake_env{tls_handshake_history + = ssl_handshake:init_handshake_history()}}), {State2, Actions} = send_handshake(Hello, State1), State = State2#state{negotiated_version = Version, %% Requested version @@ -560,9 +573,9 @@ hello(internal, #client_hello{cookie = Cookie} = Hello, #state{static_env = #sta hello(internal, #server_hello{} = Hello, #state{ static_env = #static_env{role = client}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, connection_states = ConnectionStates0, negotiated_version = ReqVersion, - renegotiation = {Renegotiation, _}, ssl_options = SslOptions} = State) -> case dtls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation) of #alert{} = Alert -> @@ -676,11 +689,12 @@ connection(internal, #hello_request{}, #state{static_env = #static_env{host = Ho session_cache = Cache, session_cache_cb = CacheCb }, + handshake_env = #handshake_env{ renegotiation = {Renegotiation, _}}, session = #session{own_certificate = Cert} = Session0, ssl_options = SslOpts, - connection_states = ConnectionStates0, - renegotiation = {Renegotiation, _}} = State0) -> + connection_states = ConnectionStates0 + } = State0) -> Hello = dtls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, Cache, CacheCb, Renegotiation, Cert), @@ -702,7 +716,8 @@ connection(internal, #client_hello{} = Hello, #state{static_env = #static_env{ro %% initiated renegotiation we will disallow many client initiated %% renegotiations immediately after each other. erlang:send_after(?WAIT_TO_ALLOW_RENEGOTIATION, self(), allow_renegotiate), - {next_state, hello, State#state{allow_renegotiate = false, renegotiation = {true, peer}}, + {next_state, hello, State#state{allow_renegotiate = false, + handshake_env = #handshake_env{renegotiation = {true, peer}}}, [{next_event, internal, Hello}]}; connection(internal, #client_hello{}, #state{static_env = #static_env{role = server}, allow_renegotiate = false} = State0) -> @@ -774,6 +789,10 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions, _}, User, }, #state{static_env = InitStatEnv, + handshake_env = #handshake_env{ + tls_handshake_history = ssl_handshake:init_handshake_history(), + renegotiation = {false, first} + }, socket_options = SocketOptions, %% We do not want to save the password in the state so that %% could be written in the clear into error logs. @@ -783,7 +802,6 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions, _}, User, protocol_buffers = #protocol_buffers{}, user_application = {Monitor, User}, user_data_buffer = <<>>, - renegotiation = {false, first}, allow_renegotiate = SSLOptions#ssl_options.client_renegotiation, start_or_recv_from = undefined, flight_buffer = new_flight(), @@ -836,9 +854,8 @@ handle_client_hello(#client_hello{client_version = ClientVersion} = Hello, static_env = #static_env{port = Port, session_cache = Cache, session_cache_cb = CacheCb}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}} = HsEnv, session = #session{own_certificate = Cert} = Session0, - renegotiation = {Renegotiation, _}, - negotiated_protocol = CurrentProtocol, key_algorithm = KeyExAlg, ssl_options = SslOpts} = State0) -> @@ -857,7 +874,7 @@ handle_client_hello(#client_hello{client_version = ClientVersion} = Hello, State = prepare_flight(State0#state{connection_states = ConnectionStates, negotiated_version = Version, hashsign_algorithm = HashSign, - client_hello_version = ClientVersion, + handshake_env = HsEnv#handshake_env{client_hello_version = ClientVersion}, session = Session, negotiated_protocol = Protocol}), @@ -1146,13 +1163,14 @@ send_application_data(Data, From, _StateName, #state{static_env = #static_env{socket = Socket, protocol_cb = Connection, transport_cb = Transport}, + handshake_env = HsEnv, negotiated_version = Version, connection_states = ConnectionStates0, ssl_options = #ssl_options{renegotiate_at = RenegotiateAt}} = State0) -> case time_to_renegotiate(Data, ConnectionStates0, RenegotiateAt) of true -> - renegotiate(State0#state{renegotiation = {true, internal}}, + renegotiate(State0#state{handshake_env = HsEnv#handshake_env{renegotiation = {true, internal}}}, [{next_event, {call, From}, {application_data, Data}}]); false -> {Msgs, ConnectionStates} = diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index 7d7da2dcec..2bfa9a52cd 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -356,8 +356,8 @@ handle_normal_shutdown(Alert, _, #state{static_env = #static_env{role = Role, transport_cb = Transport, protocol_cb = Connection, tracker = Tracker}, - start_or_recv_from = StartFrom, - renegotiation = {false, first}} = State) -> + handshake_env = #handshake_env{renegotiation = {false, first}}, + start_or_recv_from = StartFrom} = State) -> Pids = Connection:pids(State), alert_user(Pids, Transport, Tracker,Socket, StartFrom, Alert, Role, Connection); @@ -401,8 +401,8 @@ handle_alert(#alert{level = ?WARNING, description = ?CLOSE_NOTIFY} = Alert, handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, StateName, #state{static_env = #static_env{role = Role, protocol_cb = Connection}, - ssl_options = SslOpts, - renegotiation = {true, internal}} = State) -> + handshake_env = #handshake_env{renegotiation = {true, internal}}, + ssl_options = SslOpts} = State) -> log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), handle_normal_shutdown(Alert, StateName, State), @@ -411,26 +411,26 @@ handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, connection = StateName, #state{static_env = #static_env{role = Role, protocol_cb = Connection}, - ssl_options = SslOpts, - renegotiation = {true, From} + handshake_env = #handshake_env{renegotiation = {true, From}} = HsEnv, + ssl_options = SslOpts } = State0) -> log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), gen_statem:reply(From, {error, renegotiation_rejected}), State = Connection:reinit_handshake_data(State0), - Connection:next_event(connection, no_record, State#state{renegotiation = undefined}); + Connection:next_event(connection, no_record, State#state{handshake_env = HsEnv#handshake_env{renegotiation = undefined}}); handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, StateName, #state{static_env = #static_env{role = Role, protocol_cb = Connection}, - ssl_options = SslOpts, - renegotiation = {true, From} + handshake_env = #handshake_env{renegotiation = {true, From}} = HsEnv, + ssl_options = SslOpts } = State0) -> log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), gen_statem:reply(From, {error, renegotiation_rejected}), %% Go back to connection! - State = Connection:reinit(State0#state{renegotiation = undefined}), + State = Connection:reinit(State0#state{handshake_env = HsEnv#handshake_env{renegotiation = undefined}}), Connection:next_event(connection, no_record, State); %% Gracefully log and ignore all other warning alerts @@ -607,7 +607,8 @@ handle_session(#server_hello{cipher_suite = CipherSuite, ssl_config(Opts, Role, State) -> ssl_config(Opts, Role, State, new). -ssl_config(Opts, Role, #state{static_env = InitStatEnv0} =State0, Type) -> +ssl_config(Opts, Role, #state{static_env = InitStatEnv0, + handshake_env = HsEnv} = State0, Type) -> {ok, #{cert_db_ref := Ref, cert_db_handle := CertDbHandle, fileref_db_handle := FileRefHandle, @@ -634,8 +635,8 @@ ssl_config(Opts, Role, #state{static_env = InitStatEnv0} =State0, Type) -> ssl_options = Opts}, case Type of new -> - Handshake = ssl_handshake:init_handshake_history(), - State#state{tls_handshake_history = Handshake}; + Hist = ssl_handshake:init_handshake_history(), + State#state{handshake_env = HsEnv#handshake_env{tls_handshake_history = Hist}}; continue -> State end. @@ -728,15 +729,15 @@ abbreviated({call, From}, Msg, State, Connection) -> handle_call(Msg, From, ?FUNCTION_NAME, State, Connection); abbreviated(internal, #finished{verify_data = Data} = Finished, #state{static_env = #static_env{role = server}, + handshake_env = #handshake_env{tls_handshake_history = Hist}, negotiated_version = Version, expecting_finished = true, - tls_handshake_history = Handshake, session = #session{master_secret = MasterSecret}, connection_states = ConnectionStates0} = State0, Connection) -> case ssl_handshake:verify_connection(ssl:tls_version(Version), Finished, client, get_current_prf(ConnectionStates0, write), - MasterSecret, Handshake) of + MasterSecret, Hist) of verified -> ConnectionStates = ssl_record:set_client_verify_data(current_both, Data, ConnectionStates0), @@ -748,13 +749,13 @@ abbreviated(internal, #finished{verify_data = Data} = Finished, end; abbreviated(internal, #finished{verify_data = Data} = Finished, #state{static_env = #static_env{role = client}, - tls_handshake_history = Handshake0, + handshake_env = #handshake_env{tls_handshake_history = Hist0}, session = #session{master_secret = MasterSecret}, negotiated_version = Version, connection_states = ConnectionStates0} = State0, Connection) -> case ssl_handshake:verify_connection(ssl:tls_version(Version), Finished, server, get_pending_prf(ConnectionStates0, write), - MasterSecret, Handshake0) of + MasterSecret, Hist0) of verified -> ConnectionStates1 = ssl_record:set_server_verify_data(current_read, Data, ConnectionStates0), @@ -1003,18 +1004,18 @@ cipher(info, Msg, State, _) -> cipher(internal, #certificate_verify{signature = Signature, hashsign_algorithm = CertHashSign}, #state{static_env = #static_env{role = server}, + handshake_env = #handshake_env{tls_handshake_history = Hist}, key_algorithm = KexAlg, public_key_info = PublicKeyInfo, negotiated_version = Version, - session = #session{master_secret = MasterSecret}, - tls_handshake_history = Handshake + session = #session{master_secret = MasterSecret} } = State, Connection) -> TLSVersion = ssl:tls_version(Version), %% Use negotiated value if TLS-1.2 otherwhise return default HashSign = negotiated_hashsign(CertHashSign, KexAlg, PublicKeyInfo, TLSVersion), case ssl_handshake:certificate_verify(Signature, PublicKeyInfo, - TLSVersion, HashSign, MasterSecret, Handshake) of + TLSVersion, HashSign, MasterSecret, Hist) of valid -> Connection:next_event(?FUNCTION_NAME, no_record, State#state{cert_hashsign_algorithm = HashSign}); @@ -1038,11 +1039,11 @@ cipher(internal, #finished{verify_data = Data} = Finished, = Session0, ssl_options = SslOpts, connection_states = ConnectionStates0, - tls_handshake_history = Handshake0} = State, Connection) -> + handshake_env = #handshake_env{tls_handshake_history = Hist}} = State, Connection) -> case ssl_handshake:verify_connection(ssl:tls_version(Version), Finished, opposite_role(Role), get_current_prf(ConnectionStates0, read), - MasterSecret, Handshake0) of + MasterSecret, Hist) of verified -> Session = handle_session(Role, SslOpts, Host, Port, Session0), cipher_role(Role, Data, Session, @@ -1084,9 +1085,10 @@ connection({call, RecvFrom}, {recv, N, Timeout}, start_or_recv_from = RecvFrom, timer = Timer}, ?FUNCTION_NAME, Connection); -connection({call, From}, renegotiate, #state{static_env = #static_env{protocol_cb = Connection}} = State, +connection({call, From}, renegotiate, #state{static_env = #static_env{protocol_cb = Connection}, + handshake_env = HsEnv} = State, Connection) -> - Connection:renegotiate(State#state{renegotiation = {true, From}}, []); + Connection:renegotiate(State#state{handshake_env = HsEnv#handshake_env{renegotiation = {true, From}}}, []); connection({call, From}, peer_certificate, #state{session = #session{peer_certificate = Cert}} = State, _) -> hibernate_after(?FUNCTION_NAME, State, [{reply, From, {ok, Cert}}]); @@ -1106,9 +1108,10 @@ connection({call, From}, negotiated_protocol, connection({call, From}, Msg, State, Connection) -> handle_call(Msg, From, ?FUNCTION_NAME, State, Connection); connection(cast, {internal_renegotiate, WriteState}, #state{static_env = #static_env{protocol_cb = Connection}, + handshake_env = HsEnv, connection_states = ConnectionStates} = State, Connection) -> - Connection:renegotiate(State#state{renegotiation = {true, internal}, + Connection:renegotiate(State#state{handshake_env = HsEnv#handshake_env{renegotiation = {true, internal}}, connection_states = ConnectionStates#{current_write => WriteState}}, []); connection(cast, {dist_handshake_complete, DHandle}, #state{ssl_options = #ssl_options{erl_dist = true}, @@ -1141,15 +1144,17 @@ downgrade(Type, Event, State, Connection) -> %% common or unexpected events for the state. %%-------------------------------------------------------------------- handle_common_event(internal, {handshake, {#hello_request{} = Handshake, _}}, connection = StateName, - #state{static_env = #static_env{role = client}} = State, _) -> + #state{static_env = #static_env{role = client}, + handshake_env = HsEnv} = State, _) -> %% Should not be included in handshake history - {next_state, StateName, State#state{renegotiation = {true, peer}}, [{next_event, internal, Handshake}]}; + {next_state, StateName, State#state{handshake_env = HsEnv#handshake_env{renegotiation = {true, peer}}}, + [{next_event, internal, Handshake}]}; handle_common_event(internal, {handshake, {#hello_request{}, _}}, StateName, #state{static_env = #static_env{role = client}}, _) when StateName =/= connection -> keep_state_and_data; handle_common_event(internal, {handshake, {Handshake, Raw}}, StateName, - #state{tls_handshake_history = Hs0} = State0, + #state{handshake_env = #handshake_env{tls_handshake_history = Hist0} = HsEnv} = State0, Connection) -> PossibleSNI = Connection:select_sni_extension(Handshake), @@ -1157,8 +1162,9 @@ handle_common_event(internal, {handshake, {Handshake, Raw}}, StateName, %% a client_hello, which needs to be determined by the connection callback. %% In other cases this is a noop State = handle_sni_extension(PossibleSNI, State0), - HsHist = ssl_handshake:update_handshake_history(Hs0, iolist_to_binary(Raw)), - {next_state, StateName, State#state{tls_handshake_history = HsHist}, + + Hist = ssl_handshake:update_handshake_history(Hist0, Raw), + {next_state, StateName, State#state{handshake_env = HsEnv#handshake_env{tls_handshake_history = Hist}}, [{next_event, internal, Handshake}]}; handle_common_event(internal, {protocol_record, TLSorDTLSRecord}, StateName, State, Connection) -> Connection:handle_protocol_record(TLSorDTLSRecord, StateName, State); @@ -1321,7 +1327,7 @@ handle_info(allow_renegotiate, StateName, State) -> {next_state, StateName, State#state{allow_renegotiate = true}}; handle_info({cancel_start_or_recv, StartFrom}, StateName, - #state{renegotiation = {false, first}} = State) when StateName =/= connection -> + #state{handshake_env = #handshake_env{renegotiation = {false, first}}} = State) when StateName =/= connection -> {stop_and_reply, {shutdown, user_timeout}, {reply, StartFrom, {error, timeout}}, @@ -1406,7 +1412,7 @@ format_status(terminate, [_, StateName, State]) -> [{data, [{"State", {StateName, State#state{connection_states = ?SECRET_PRINTOUT, protocol_buffers = ?SECRET_PRINTOUT, user_data_buffer = ?SECRET_PRINTOUT, - tls_handshake_history = ?SECRET_PRINTOUT, + handshake_env = ?SECRET_PRINTOUT, session = ?SECRET_PRINTOUT, private_key = ?SECRET_PRINTOUT, diffie_hellman_params = ?SECRET_PRINTOUT, @@ -1562,16 +1568,16 @@ certify_client(#state{client_certificate_requested = false} = State, _) -> State. verify_client_cert(#state{static_env = #static_env{role = client}, + handshake_env = #handshake_env{tls_handshake_history = Hist}, client_certificate_requested = true, negotiated_version = Version, private_key = PrivateKey, session = #session{master_secret = MasterSecret, own_certificate = OwnCert}, - cert_hashsign_algorithm = HashSign, - tls_handshake_history = Handshake0} = State, Connection) -> + cert_hashsign_algorithm = HashSign} = State, Connection) -> case ssl_handshake:client_certificate_verify(OwnCert, MasterSecret, - ssl:tls_version(Version), HashSign, PrivateKey, Handshake0) of + ssl:tls_version(Version), HashSign, PrivateKey, Hist) of #certificate_verify{} = Verified -> Connection:queue_handshake(Verified, State); ignore -> @@ -1607,7 +1613,9 @@ server_certify_and_key_exchange(State0, Connection) -> request_client_cert(State2, Connection). certify_client_key_exchange(#encrypted_premaster_secret{premaster_secret= EncPMS}, - #state{private_key = Key, client_hello_version = {Major, Minor} = Version} = State, Connection) -> + #state{private_key = Key, + handshake_env = #handshake_env{client_hello_version = {Major, Minor} = Version}} + = State, Connection) -> FakeSecret = make_premaster_secret(Version, rsa), %% Countermeasure for Bleichenbacher attack always provide some kind of premaster secret %% and fail handshake later.RFC 5246 section 7.4.7.1. @@ -2034,14 +2042,15 @@ cipher_protocol(State, Connection) -> Connection:queue_change_cipher(#change_cipher_spec{}, State). finished(#state{static_env = #static_env{role = Role}, + handshake_env = #handshake_env{tls_handshake_history = Hist}, negotiated_version = Version, session = Session, - connection_states = ConnectionStates0, - tls_handshake_history = Handshake0} = State0, StateName, Connection) -> + connection_states = ConnectionStates0} = State0, + StateName, Connection) -> MasterSecret = Session#session.master_secret, Finished = ssl_handshake:finished(ssl:tls_version(Version), Role, get_current_prf(ConnectionStates0, write), - MasterSecret, Handshake0), + MasterSecret, Hist), ConnectionStates = save_verify_data(Role, Finished, ConnectionStates0, StateName), Connection:send_handshake(Finished, State0#state{connection_states = ConnectionStates}). @@ -2369,7 +2378,7 @@ handle_trusted_certs_db(#state{static_env = #static_env{cert_db_ref = Ref, ok end. -prepare_connection(#state{renegotiation = Renegotiate, +prepare_connection(#state{handshake_env = #handshake_env{renegotiation = Renegotiate}, start_or_recv_from = RecvFrom} = State0, Connection) when Renegotiate =/= {false, first}, RecvFrom =/= undefined -> @@ -2379,18 +2388,18 @@ prepare_connection(State0, Connection) -> State = Connection:reinit(State0), {no_record, ack_connection(State)}. -ack_connection(#state{renegotiation = {true, Initiater}} = State) when Initiater == peer; - Initiater == internal -> - State#state{renegotiation = undefined}; -ack_connection(#state{renegotiation = {true, From}} = State) -> +ack_connection(#state{handshake_env = #handshake_env{renegotiation = {true, Initiater}} = HsEnv} = State) when Initiater == peer; + Initiater == internal -> + State#state{handshake_env = HsEnv#handshake_env{renegotiation = undefined}}; +ack_connection(#state{handshake_env = #handshake_env{renegotiation = {true, From}} = HsEnv} = State) -> gen_statem:reply(From, ok), - State#state{renegotiation = undefined}; -ack_connection(#state{renegotiation = {false, first}, + State#state{handshake_env = HsEnv#handshake_env{renegotiation = undefined}}; +ack_connection(#state{handshake_env = #handshake_env{renegotiation = {false, first}} = HsEnv, start_or_recv_from = StartFrom, timer = Timer} = State) when StartFrom =/= undefined -> gen_statem:reply(StartFrom, connected), cancel_timer(Timer), - State#state{renegotiation = undefined, + State#state{handshake_env = HsEnv#handshake_env{renegotiation = undefined}, start_or_recv_from = undefined, timer = undefined}; ack_connection(State) -> State. diff --git a/lib/ssl/src/ssl_connection.hrl b/lib/ssl/src/ssl_connection.hrl index dc8aa7619b..177fa37b83 100644 --- a/lib/ssl/src/ssl_connection.hrl +++ b/lib/ssl/src/ssl_connection.hrl @@ -51,8 +51,18 @@ cert_db_ref :: certdb_ref() | 'undefined', tracker :: pid() | 'undefined' %% Tracker process for listen socket }). + +-record(handshake_env, { + client_hello_version :: ssl_record:ssl_version() | 'undefined', + unprocessed_handshake_events = 0 :: integer(), + tls_handshake_history :: ssl_handshake:ssl_handshake_history() | secret_printout() + | 'undefined', + renegotiation :: undefined | {boolean(), From::term() | internal | peer} + }). + -record(state, { static_env :: #static_env{}, + handshake_env :: #handshake_env{} | secret_printout(), %% Change seldome user_application :: {Monitor::reference(), User::pid()}, ssl_options :: #ssl_options{}, @@ -68,12 +78,9 @@ connection_states :: ssl_record:connection_states() | secret_printout(), protocol_buffers :: term() | secret_printout() , %% #protocol_buffers{} from tls_record.hrl or dtls_recor.hr user_data_buffer :: undefined | binary() | secret_printout(), - + %% Used only in HS - unprocessed_handshake_events = 0 :: integer(), - tls_handshake_history :: ssl_handshake:ssl_handshake_history() | secret_printout() - | 'undefined', - client_hello_version :: ssl_record:ssl_version() | 'undefined', + client_certificate_requested = false :: boolean(), key_algorithm :: ssl_cipher_format:key_algo(), hashsign_algorithm = {undefined, undefined}, @@ -86,7 +93,6 @@ srp_params :: #srp_user{} | secret_printout() | 'undefined', srp_keys ::{PublicKey :: binary(), PrivateKey :: binary()} | secret_printout() | 'undefined', premaster_secret :: binary() | secret_printout() | 'undefined', - renegotiation :: undefined | {boolean(), From::term() | internal | peer}, start_or_recv_from :: term(), timer :: undefined | reference(), % start_or_recive_timer hello, %%:: #client_hello{} | #server_hello{}, diff --git a/lib/ssl/src/tls_connection.erl b/lib/ssl/src/tls_connection.erl index 8b24151d9f..e034cb20e9 100644 --- a/lib/ssl/src/tls_connection.erl +++ b/lib/ssl/src/tls_connection.erl @@ -143,22 +143,24 @@ pids(#state{protocol_specific = #{sender := Sender}}) -> %%==================================================================== %% State transition handling %%==================================================================== -next_record(#state{unprocessed_handshake_events = N} = State) when N > 0 -> - {no_record, State#state{unprocessed_handshake_events = N-1}}; - +next_record(#state{handshake_env = + #handshake_env{unprocessed_handshake_events = N} = HsEnv} + = State) when N > 0 -> + {no_record, State#state{handshake_env = + HsEnv#handshake_env{unprocessed_handshake_events = N-1}}}; next_record(#state{protocol_buffers = - #protocol_buffers{tls_packets = [], tls_cipher_texts = [CT | Rest]} - = Buffers, - connection_states = ConnStates0, - ssl_options = #ssl_options{padding_check = Check}} = State) -> - case tls_record:decode_cipher_text(CT, ConnStates0, Check) of - {Plain, ConnStates} -> - {Plain, State#state{protocol_buffers = - Buffers#protocol_buffers{tls_cipher_texts = Rest}, - connection_states = ConnStates}}; - #alert{} = Alert -> - {Alert, State} - end; + #protocol_buffers{tls_packets = [], tls_cipher_texts = [#ssl_tls{type = Type}| _] = CipherTexts0} + = Buffers, + connection_states = ConnectionStates0, + ssl_options = #ssl_options{padding_check = Check}} = State) -> + case decode_cipher_texts(Type, CipherTexts0, ConnectionStates0, Check, <<>>) of + {#ssl_tls{} = Record, ConnectionStates, CipherTexts} -> + {Record, State#state{protocol_buffers = Buffers#protocol_buffers{tls_cipher_texts = CipherTexts}, + connection_states = ConnectionStates}}; + {#alert{} = Alert, ConnectionStates, CipherTexts} -> + {Alert, State#state{protocol_buffers = Buffers#protocol_buffers{tls_cipher_texts = CipherTexts}, + connection_states = ConnectionStates}} + end; next_record(#state{protocol_buffers = #protocol_buffers{tls_packets = [], tls_cipher_texts = []}, protocol_specific = #{active_n_toggle := true, active_n := N} = ProtocolSpec, static_env = #static_env{socket = Socket, @@ -196,6 +198,22 @@ next_event(StateName, Record, State, Actions) -> {next_state, StateName, State, [{next_event, internal, Alert} | Actions]} end. +decode_cipher_texts(Type, [] = CipherTexts, ConnectionStates, _, Acc) -> + {#ssl_tls{type = Type, fragment = Acc}, ConnectionStates, CipherTexts}; +decode_cipher_texts(Type, + [#ssl_tls{type = Type} = CT | CipherTexts], ConnectionStates0, Check, Acc) -> + case tls_record:decode_cipher_text(CT, ConnectionStates0, Check) of + {#ssl_tls{type = ?APPLICATION_DATA, fragment = Plain}, ConnectionStates} -> + decode_cipher_texts(Type, CipherTexts, + ConnectionStates, Check, <<Acc/binary, Plain/binary>>); + {#ssl_tls{type = Type, fragment = Plain}, ConnectionStates} -> + {#ssl_tls{type = Type, fragment = Plain}, ConnectionStates, CipherTexts}; + #alert{} = Alert -> + {Alert, ConnectionStates0, CipherTexts} + end; +decode_cipher_texts(Type, CipherTexts, ConnectionStates, _, Acc) -> + {#ssl_tls{type = Type, fragment = Acc}, ConnectionStates, CipherTexts}. + %%% TLS record protocol level application data messages handle_protocol_record(#ssl_tls{type = ?APPLICATION_DATA, fragment = Data}, StateName, State0) -> @@ -227,8 +245,12 @@ handle_protocol_record(#ssl_tls{type = ?HANDSHAKE, fragment = Data}, connection -> ssl_connection:hibernate_after(StateName, State, Events); _ -> + HsEnv = State#state.handshake_env, {next_state, StateName, - State#state{unprocessed_handshake_events = unprocessed_events(Events)}, Events} + State#state{protocol_buffers = Buffers, + handshake_env = + HsEnv#handshake_env{unprocessed_handshake_events + = unprocessed_events(Events)}}, Events} end end catch throw:#alert{} = Alert -> @@ -263,15 +285,17 @@ handle_protocol_record(#ssl_tls{type = _Unknown}, StateName, State) -> renegotiation(Pid, WriteState) -> gen_statem:call(Pid, {user_renegotiate, WriteState}). -renegotiate(#state{static_env = #static_env{role = client}} = State, Actions) -> +renegotiate(#state{static_env = #static_env{role = client}, + handshake_env = HsEnv} = State, Actions) -> %% Handle same way as if server requested %% the renegotiation Hs0 = ssl_handshake:init_handshake_history(), - {next_state, connection, State#state{tls_handshake_history = Hs0}, + {next_state, connection, State#state{handshake_env = HsEnv#handshake_env{tls_handshake_history = Hs0}}, [{next_event, internal, #hello_request{}} | Actions]}; renegotiate(#state{static_env = #static_env{role = server, socket = Socket, transport_cb = Transport}, + handshake_env = HsEnv, negotiated_version = Version, connection_states = ConnectionStates0} = State0, Actions) -> HelloRequest = ssl_handshake:hello_request(), @@ -282,20 +306,20 @@ renegotiate(#state{static_env = #static_env{role = server, send(Transport, Socket, BinMsg), State = State0#state{connection_states = ConnectionStates, - tls_handshake_history = Hs0}, + handshake_env = HsEnv#handshake_env{tls_handshake_history = Hs0}}, next_event(hello, no_record, State, Actions). send_handshake(Handshake, State) -> send_handshake_flight(queue_handshake(Handshake, State)). queue_handshake(Handshake, #state{negotiated_version = Version, - tls_handshake_history = Hist0, + handshake_env = #handshake_env{tls_handshake_history = Hist0} = HsEnv, flight_buffer = Flight0, connection_states = ConnectionStates0} = State0) -> {BinHandshake, ConnectionStates, Hist} = encode_handshake(Handshake, Version, ConnectionStates0, Hist0), State0#state{connection_states = ConnectionStates, - tls_handshake_history = Hist, + handshake_env = HsEnv#handshake_env{tls_handshake_history = Hist}, flight_buffer = Flight0 ++ [BinHandshake]}. send_handshake_flight(#state{static_env = #static_env{socket = Socket, @@ -318,14 +342,14 @@ reinit(#state{protocol_specific = #{sender := Sender}, tls_sender:update_connection_state(Sender, Write, Version), reinit_handshake_data(State). -reinit_handshake_data(State) -> +reinit_handshake_data(#state{handshake_env = HsEnv} =State) -> %% premaster_secret, public_key_info and tls_handshake_info %% are only needed during the handshake phase. %% To reduce memory foot print of a connection reinitialize them. State#state{ premaster_secret = undefined, public_key_info = undefined, - tls_handshake_history = ssl_handshake:init_handshake_history() + handshake_env = HsEnv#handshake_env{tls_handshake_history = ssl_handshake:init_handshake_history()} }. select_sni_extension(#client_hello{extensions = HelloExtensions}) -> @@ -440,10 +464,10 @@ init({call, From}, {start, Timeout}, socket = Socket, session_cache = Cache, session_cache_cb = CacheCb}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}} = HsEnv, ssl_options = SslOpts, session = #session{own_certificate = Cert} = Session0, - connection_states = ConnectionStates0, - renegotiation = {Renegotiation, _} + connection_states = ConnectionStates0 } = State0) -> Timer = ssl_connection:start_or_recv_cancel_timer(Timeout, From), Hello = tls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, @@ -459,7 +483,7 @@ init({call, From}, {start, Timeout}, negotiated_version = Version, %% Requested version session = Session0#session{session_id = Hello#client_hello.session_id}, - tls_handshake_history = Handshake, + handshake_env = HsEnv#handshake_env{tls_handshake_history = Handshake}, start_or_recv_from = From, timer = Timer}, next_event(hello, no_record, State); @@ -505,8 +529,8 @@ hello(internal, #client_hello{client_version = ClientVersion} = Hello, port = Port, session_cache = Cache, session_cache_cb = CacheCb}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}} = HsEnv, session = #session{own_certificate = Cert} = Session0, - renegotiation = {Renegotiation, _}, negotiated_protocol = CurrentProtocol, key_algorithm = KeyExAlg, ssl_options = SslOpts} = State) -> @@ -526,7 +550,7 @@ hello(internal, #client_hello{client_version = ClientVersion} = Hello, State#state{connection_states = ConnectionStates, negotiated_version = Version, hashsign_algorithm = HashSign, - client_hello_version = ClientVersion, + handshake_env = HsEnv#handshake_env{client_hello_version = ClientVersion}, session = Session, negotiated_protocol = Protocol}) end; @@ -534,7 +558,7 @@ hello(internal, #server_hello{} = Hello, #state{connection_states = ConnectionStates0, negotiated_version = ReqVersion, static_env = #static_env{role = client}, - renegotiation = {Renegotiation, _}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, ssl_options = SslOptions} = State) -> case tls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation) of #alert{} = Alert -> @@ -620,7 +644,7 @@ connection(internal, #hello_request{}, port = Port, session_cache = Cache, session_cache_cb = CacheCb}, - renegotiation = {Renegotiation, peer}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, peer}}, session = #session{own_certificate = Cert} = Session0, ssl_options = SslOpts, protocol_specific = #{sender := Pid}, @@ -642,7 +666,7 @@ connection(internal, #hello_request{}, port = Port, session_cache = Cache, session_cache_cb = CacheCb}, - renegotiation = {Renegotiation, _}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, session = #session{own_certificate = Cert} = Session0, ssl_options = SslOpts, connection_states = ConnectionStates} = State0) -> @@ -653,6 +677,7 @@ connection(internal, #hello_request{}, = Hello#client_hello.session_id}}, Actions); connection(internal, #client_hello{} = Hello, #state{static_env = #static_env{role = server}, + handshake_env = HsEnv, allow_renegotiate = true, connection_states = CS, protocol_specific = #{sender := Sender} @@ -666,7 +691,7 @@ connection(internal, #client_hello{} = Hello, {ok, Write} = tls_sender:renegotiate(Sender), next_event(hello, no_record, State#state{connection_states = CS#{current_write => Write}, allow_renegotiate = false, - renegotiation = {true, peer} + handshake_env = HsEnv#handshake_env{renegotiation = {true, peer}} }, [{next_event, internal, Hello}]); connection(internal, #client_hello{}, @@ -762,6 +787,10 @@ initial_state(Role, Sender, Host, Port, Socket, {SSLOptions, SocketOptions, Trac }, #state{ static_env = InitStatEnv, + handshake_env = #handshake_env{ + tls_handshake_history = ssl_handshake:init_handshake_history(), + renegotiation = {false, first} + }, socket_options = SocketOptions, ssl_options = SSLOptions, session = #session{is_resumable = new}, @@ -769,7 +798,6 @@ initial_state(Role, Sender, Host, Port, Socket, {SSLOptions, SocketOptions, Trac protocol_buffers = #protocol_buffers{}, user_application = {UserMonitor, User}, user_data_buffer = <<>>, - renegotiation = {false, first}, allow_renegotiate = SSLOptions#ssl_options.client_renegotiation, start_or_recv_from = undefined, flight_buffer = [], diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index 3778530a47..cfc4ec5770 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -4888,20 +4888,24 @@ run_suites(Ciphers, Config, Type) -> ssl_test_lib:ssl_options(server_psk_anon_hint, Config)]}; srp -> {ssl_test_lib:ssl_options(client_srp, Config), - ssl_test_lib:ssl_options(server_srp, Config)}; + [{ciphers, Ciphers} | + ssl_test_lib:ssl_options(server_srp, Config)]}; srp_anon -> {ssl_test_lib:ssl_options(client_srp, Config), - ssl_test_lib:ssl_options(server_srp_anon, Config)}; + [{ciphers, Ciphers} | + ssl_test_lib:ssl_options(server_srp_anon, Config)]}; srp_dsa -> {ssl_test_lib:ssl_options(client_srp_dsa, Config), - ssl_test_lib:ssl_options(server_srp_dsa, Config)}; + [{ciphers, Ciphers} | + ssl_test_lib:ssl_options(server_srp_dsa, Config)]}; ecdsa -> {ssl_test_lib:ssl_options(client_ecdsa_opts, Config), [{ciphers, Ciphers} | ssl_test_lib:ssl_options(server_ecdsa_opts, Config)]}; ecdh_rsa -> {ssl_test_lib:ssl_options(client_ecdh_rsa_opts, Config), - ssl_test_lib:ssl_options(server_ecdh_rsa_opts, Config)}; + [{ciphers, Ciphers} | + ssl_test_lib:ssl_options(server_ecdh_rsa_opts, Config)]}; rc4_rsa -> {ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), [{ciphers, Ciphers} | diff --git a/lib/stdlib/doc/src/ets.xml b/lib/stdlib/doc/src/ets.xml index 70d1aaa74d..ccccf7de88 100644 --- a/lib/stdlib/doc/src/ets.xml +++ b/lib/stdlib/doc/src/ets.xml @@ -138,23 +138,56 @@ operation. In database terms the isolation level can be seen as "serializable", as if all isolated operations are carried out serially, one after the other in a strict order.</p> + </section> - <p>No other support is available within this module that would guarantee - consistency between objects. However, function - <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso> - can be used to guarantee that a sequence of - <seealso marker="#first/1"><c>first/1</c></seealso> and - <seealso marker="#next/2"><c>next/2</c></seealso> calls traverse the - table without errors and that each existing object in the table is - visited exactly once, even if another (or the same) process - simultaneously deletes or inserts objects into the table. - Nothing else is guaranteed; in particular objects that are inserted - or deleted during such a traversal can be visited once or not at all. - Functions that internally traverse over a table, like - <seealso marker="#select/1"><c>select</c></seealso> and - <seealso marker="#match/1"><c>match</c></seealso>, - give the same guarantee as - <seealso marker="#safe_fixtable/2"><c>safe_fixtable</c></seealso>.</p> + <section><marker id="traversal"></marker> + <title>Table traversal</title> + <p>There are different ways to traverse through the objects of a table.</p> + <list type="bulleted"> + <item><p><em>Single-step</em> traversal one key at at time, using + <seealso marker="#first/1"><c>first/1</c></seealso>, + <seealso marker="#next/2"><c>next/2</c></seealso>, + <seealso marker="#last/1"><c>last/1</c></seealso> and + <seealso marker="#prev/2"><c>prev/2</c></seealso>.</p> + </item> + <item><p>Search with simple <em>match patterns</em>, using + <seealso marker="#match/1"><c>match/1/2/3</c></seealso>, + <seealso marker="#match_delete/2"><c>match_delete/2</c></seealso> and + <seealso marker="#match_object/1"><c>match_object/1/2/3</c></seealso>.</p> + </item> + <item><p>Search with more powerful <em>match specifications</em>, using + <seealso marker="#select/1"><c>select/1/2/3</c></seealso>, + <seealso marker="#select_count/2"><c>select_count/2</c></seealso>, + <seealso marker="#select_delete/2"><c>select_delete/2</c></seealso>, + <seealso marker="#select_replace/2"><c>select_replace/2</c></seealso> and + <seealso marker="#select_reverse/1"><c>select_reverse/1/2/3</c></seealso>.</p> + </item> + <item><p><em>Table conversions</em>, using + <seealso marker="#tab2file/2"><c>tab2file/2/3</c></seealso> and + <seealso marker="#tab2list/1"><c>tab2list/1</c></seealso>.</p> + </item> + </list> + <p>None of these ways of table traversal will guarantee a consistent table snapshot + if the table is also updated during the traversal. Moreover, traversals not + done in a <em>safe</em> way, on tables where keys are inserted or deleted + during the traversal, may yield the following undesired effects:</p> + <list type="bulleted"> + <item><p>Any key may be missed.</p></item> + <item><p>Any key may be found more than once.</p></item> + <item><p>The traversal may fail with <c>badarg</c> exception if keys are deleted.</p> + </item> + </list> + <p>A table traversal is <em>safe</em> if either</p> + <list type="bulleted"> + <item><p>the table is of type <c>ordered_set</c>.</p> + </item> + <item><p>the entire table traversal is done within one ETS function + call.</p> + </item> + <item><p>function <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso> + is used to keep the table fixated during the entire traversal.</p> + </item> + </list> </section> <section> @@ -871,6 +904,9 @@ ets:is_compiled_ms(Broken).</code> <seealso marker="#first/1"><c>first/1</c></seealso> and <seealso marker="#next/2"><c>next/2</c></seealso>.</p> <p>If the table is empty, <c>'$end_of_table'</c> is returned.</p> + <p>Use <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso> + to guarantee <seealso marker="#traversal">safe traversal</seealso> + for subsequent calls to <seealso marker="#match/1"><c>match/1</c></seealso>.</p> </desc> </func> @@ -936,6 +972,10 @@ ets:is_compiled_ms(Broken).</code> <seealso marker="#first/1"><c>first/1</c></seealso> and <seealso marker="#next/2"><c>next/2</c></seealso>.</p> <p>If the table is empty, <c>'$end_of_table'</c> is returned.</p> + <p>Use <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso> + to guarantee <seealso marker="#traversal">safe traversal</seealso> + for subsequent calls to <seealso marker="#match_object/1"> + <c>match_object/1</c></seealso>.</p> </desc> </func> @@ -1192,12 +1232,13 @@ ets:select(Table, MatchSpec),</code> <p>To find the first key in the table, use <seealso marker="#first/1"><c>first/1</c></seealso>.</p> <p>Unless a table of type <c>set</c>, <c>bag</c>, or - <c>duplicate_bag</c> is protected using + <c>duplicate_bag</c> is fixated using <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso>, - a traversal can fail if - concurrent updates are made to the table. For table - type <c>ordered_set</c>, the function returns the next key in - order, even if the object does no longer exist.</p> + a call to <c>next/2</c> will fail if <c><anno>Key1</anno></c> no longer + exists in the table. For table type <c>ordered_set</c>, the function + always returns the next key after <c><anno>Key1</anno></c> in term + order, regardless whether <c><anno>Key1</anno></c> ever existed in the + table.</p> </desc> </func> @@ -1212,7 +1253,7 @@ ets:select(Table, MatchSpec),</code> table types, the function is synonymous to <seealso marker="#next/2"><c>next/2</c></seealso>. If no previous key exists, <c>'$end_of_table'</c> is returned.</p> - <p>To find the last key in the table, use + <p>To find the last key in an <c>ordered_set</c> table, use <seealso marker="#last/1"><c>last/1</c></seealso>.</p> </desc> </func> @@ -1287,7 +1328,16 @@ ets:select(ets:repair_continuation(Broken,MS)).</code> <fsummary>Fix an ETS table for safe traversal.</fsummary> <desc> <p>Fixes a table of type <c>set</c>, <c>bag</c>, or - <c>duplicate_bag</c> for safe traversal.</p> + <c>duplicate_bag</c> for <seealso marker="#traversal"> + safe traversal</seealso> using + <seealso marker="#first/1"><c>first/1</c></seealso> & + <seealso marker="#next/2"><c>next/2</c></seealso>, + <seealso marker="#match/3"><c>match/3</c></seealso> & + <seealso marker="#match/1"><c>match/1</c></seealso>, + <seealso marker="#match_object/3"><c>match_object/3</c></seealso> & + <seealso marker="#match_object/1"><c>match_object/1</c></seealso>, or + <seealso marker="#select/3"><c>select/3</c></seealso> & + <seealso marker="#select/1"><c>select/1</c></seealso>.</p> <p>A process fixes a table by calling <c>safe_fixtable(<anno>Tab</anno>, true)</c>. The table remains fixed until the process releases it by calling @@ -1300,11 +1350,11 @@ ets:select(ets:repair_continuation(Broken,MS)).</code> <p>When a table is fixed, a sequence of <seealso marker="#first/1"><c>first/1</c></seealso> and <seealso marker="#next/2"><c>next/2</c></seealso> calls are - guaranteed to succeed, and each object in - the table is returned only once, even if objects - are removed or inserted during the traversal. The keys for new - objects inserted during the traversal <em>can</em> be returned by - <c>next/2</c> (it depends on the internal ordering of the keys).</p> + guaranteed to succeed even if keys are removed during the + traversal. The keys for objects inserted or deleted during a + traversal may or may not be returned by <c>next/2</c> depending on + the ordering of keys within the table and if the key exists at the time + <c>next/2</c> is called.</p> <p><em>Example:</em></p> <code type="none"> clean_all_with_value(Tab,X) -> @@ -1322,7 +1372,7 @@ clean_all_with_value(Tab,X,Key) -> true end, clean_all_with_value(Tab,X,ets:next(Tab,Key)).</code> - <p>Notice that no deleted objects are removed from a + <p>Notice that deleted objects are not freed from a fixed table until it has been released. If a process fixes a table but never releases it, the memory used by the deleted objects is never freed. The performance of operations on @@ -1332,9 +1382,9 @@ clean_all_with_value(Tab,X,Key) -> <c>info(Tab, safe_fixed_monotonic_time)</c></seealso>. A system with many processes fixing tables can need a monitor that sends alarms when tables have been fixed for too long.</p> - <p>Notice that for table type <c>ordered_set</c>, - <c>safe_fixtable/2</c> is not necessary, as calls to - <c>first/1</c> and <c>next/2</c> always succeed.</p> + <p>Notice that <c>safe_fixtable/2</c> is not necessary for table type + <c>ordered_set</c> and for traversals done by a single ETS function call, + like <seealso marker="#select/2"><c>select/2</c></seealso>.</p> </desc> </func> @@ -1462,7 +1512,10 @@ is_integer(X), is_integer(Y), X + Y < 4711]]></code> table, which is still faster than traversing the table object by object using <seealso marker="#first/1"><c>first/1</c></seealso> and <seealso marker="#next/2"><c>next/2</c></seealso>.</p> - <p>If the table is empty, <c>'$end_of_table'</c> is returned.</p> + <p>If the table is empty, <c>'$end_of_table'</c> is returned.</p> + <p>Use <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso> + to guarantee <seealso marker="#traversal">safe traversal</seealso> + for subsequent calls to <seealso marker="#select/1"><c>select/1</c></seealso>.</p> </desc> </func> @@ -1519,7 +1572,7 @@ is_integer(X), is_integer(Y), X + Y < 4711]]></code> the match specification result.</p> <p>The match-and-replace operation for each individual object is guaranteed to be <seealso marker="#concurrency">atomic and isolated</seealso>. The - <c>select_replace</c> table iteration as a whole, like all other select functions, + <c>select_replace</c> table traversal as a whole, like all other select functions, does not give such guarantees.</p> <p>The match specifiction must be guaranteed to <em>retain the key</em> of any matched object. If not, <c>select_replace</c> will fail with <c>badarg</c> diff --git a/lib/stdlib/doc/src/io_lib.xml b/lib/stdlib/doc/src/io_lib.xml index cd4ca0a3a7..4d527f8ed3 100644 --- a/lib/stdlib/doc/src/io_lib.xml +++ b/lib/stdlib/doc/src/io_lib.xml @@ -385,7 +385,7 @@ <func> <name name="write" arity="1" since=""/> <name name="write" arity="2" clause_i="1" since=""/> - <name name="write" arity="2" clause_i="2" since=""/> + <name name="write" arity="2" clause_i="2" since="OTP 20.0"/> <fsummary>Write a term.</fsummary> <desc> <p>Returns a character list that represents <c><anno>Term</anno></c>. |