diff options
337 files changed, 14050 insertions, 7157 deletions
diff --git a/OTP_VERSION b/OTP_VERSION index e9099b773b..cd23cb3bd7 100644 --- a/OTP_VERSION +++ b/OTP_VERSION @@ -1 +1 @@ -21.2.2 +21.2.6 diff --git a/bootstrap/lib/stdlib/ebin/io_lib.beam b/bootstrap/lib/stdlib/ebin/io_lib.beam Binary files differindex 05894640cb..1e29538db9 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_format.beam b/bootstrap/lib/stdlib/ebin/io_lib_format.beam Binary files differindex 9b8d7d8a9e..5fa6974ac7 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_format.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_format.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam b/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam Binary files differindex bd65ecfc30..17ff848921 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam diff --git a/erts/doc/src/atomics.xml b/erts/doc/src/atomics.xml index f552c11e18..455973f011 100644 --- a/erts/doc/src/atomics.xml +++ b/erts/doc/src/atomics.xml @@ -85,6 +85,8 @@ bsl 64)-1</c>.</p> </item> </taglist> + <p>Atomics are not tied to the current process and are automatically + garbage collected when they are no longer referenced.</p> </desc> </func> diff --git a/erts/doc/src/counters.xml b/erts/doc/src/counters.xml index 3d26093a59..36816bd68d 100644 --- a/erts/doc/src/counters.xml +++ b/erts/doc/src/counters.xml @@ -103,6 +103,8 @@ acceptable.</p> </item> </taglist> + <p>Counters are not tied to the current process and are automatically + garbage collected when they are no longer referenced.</p> </desc> </func> diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index 05a9895687..133f160dc9 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -1552,6 +1552,15 @@ parameter determines. The lingering prevents repeated deletions and insertions in the tables from occurring.</p> </item> + <tag><marker id="+ztma"/><c>+ztma true | false</c></tag> + <item> + <p>Enables or disables support for tuple module apply in + the emulator. This is a transitional flag for running code + that uses parameterized modules and was compiled under OTP 20 + or earlier. For future compatibility, the modules will need + to be recompiled with the +tuple_calls compiler option. + Defaults to false.</p> + </item> </taglist> </item> </taglist> diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index a42323b13d..6932b18571 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -3339,7 +3339,7 @@ RealSystem = system + MissedSystem</code> <func> <name name="monitor" arity="2" clause_i="1" since=""/> - <name name="monitor" arity="2" clause_i="2" since="?"/> + <name name="monitor" arity="2" clause_i="2" since="OTP 19.0"/> <name name="monitor" arity="2" clause_i="3" since="OTP 18.0"/> <fsummary>Start monitoring.</fsummary> <type name="registered_name"/> @@ -4170,9 +4170,16 @@ RealSystem = system + MissedSystem</code> </item> <tag><c>badarg</c></tag> <item> - If the port driver so decides for any reason (probably + <p>If the port driver so decides for any reason (probably something wrong with <c><anno>Operation</anno></c> - or <c><anno>Data</anno></c>). + or <c><anno>Data</anno></c>).</p> + <warning> + <p>Do not call <c>port_call</c> with an unknown + <c><anno>Port</anno></c> identifier and expect <c>badarg</c> + exception. Any undefined behavior is possible (including node + crash) depending on how the port driver interprets the supplied + arguments.</p> + </warning> </item> </taglist> </desc> @@ -4262,6 +4269,11 @@ RealSystem = system + MissedSystem</code> <p>If <c><anno>Data</anno></c> is an invalid I/O list.</p> </item> </taglist> + <warning> + <p>Do not send data to an unknown port. Any undefined behavior is + possible (including node crash) depending on how the port driver + interprets the data.</p> + </warning> </desc> </func> @@ -4321,6 +4333,11 @@ RealSystem = system + MissedSystem</code> a busy port. </item> </taglist> + <warning> + <p>Do not send data to an unknown port. Any undefined behavior is + possible (including node crash) depending on how the port driver + interprets the data.</p> + </warning> </desc> </func> @@ -4425,6 +4442,13 @@ RealSystem = system + MissedSystem</code> If the port driver so decides for any reason (probably something wrong with <c><anno>Operation</anno></c> or <c><anno>Data</anno></c>). + <warning> + <p>Do not call <c>port_control/3</c> with an unknown + <c><anno>Port</anno></c> identifier and expect <c>badarg</c> + exception. Any undefined behavior is possible (including node + crash) depending on how the port driver interprets the supplied + arguments.</p> + </warning> </item> </taglist> </desc> @@ -4526,7 +4550,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="5" since="?"/> + <name name="port_info" arity="2" clause_i="5" since="OTP R16B"/> <fsummary>Information about the locking of a port.</fsummary> <desc> <p><c><anno>Locking</anno></c> is one of the following:</p> @@ -4547,7 +4571,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="6" since="?"/> + <name name="port_info" arity="2" clause_i="6" since="OTP R16B"/> <fsummary>Information about the memory size of a port.</fsummary> <desc> <p><c><anno>Bytes</anno></c> is the total number of @@ -4565,7 +4589,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="7" since="?"/> + <name name="port_info" arity="2" clause_i="7" since="OTP R16B"/> <fsummary>Information about the monitors of a port.</fsummary> <desc> <p><c><anno>Monitors</anno></c> represent processes monitored by @@ -4581,7 +4605,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="8" since="?"/> + <name name="port_info" arity="2" clause_i="8" since="OTP 19.0"/> <fsummary>Which processes are monitoring this port.</fsummary> <desc> <p>Returns list of pids that are monitoring given port at the @@ -4613,7 +4637,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="10" since="?"/> + <name name="port_info" arity="2" clause_i="10" since="OTP R16B"/> <fsummary>Information about the OS pid of a port.</fsummary> <desc> <p><c><anno>OsPid</anno></c> is the process identifier (or equivalent) @@ -4651,7 +4675,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="12" since="?"/> + <name name="port_info" arity="2" clause_i="12" since="OTP R16B"/> <fsummary>Information about the parallelism hint of a port.</fsummary> <desc> <p><c><anno>Boolean</anno></c> corresponds to the port parallelism @@ -4662,7 +4686,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="port_info" arity="2" clause_i="13" since="?"/> + <name name="port_info" arity="2" clause_i="13" since="OTP R16B"/> <fsummary>Information about the queue size of a port.</fsummary> <desc> <p><c><anno>Bytes</anno></c> is the total number @@ -4782,7 +4806,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="process_flag" arity="2" clause_i="4" since="?"/> + <name name="process_flag" arity="2" clause_i="4" since="OTP R13B04"/> <fsummary>Set process flag min_bin_vheap_size for the calling process. </fsummary> <desc> @@ -4794,7 +4818,7 @@ RealSystem = system + MissedSystem</code> <func> <name name="process_flag" arity="2" clause_i="5" - anchor="process_flag_max_heap_size" since="?"/> + anchor="process_flag_max_heap_size" since="OTP 19.0"/> <fsummary>Set process flag max_heap_size for the calling process. </fsummary> <type name="max_heap_size"/> @@ -4868,7 +4892,7 @@ RealSystem = system + MissedSystem</code> <func> <name name="process_flag" arity="2" clause_i="6" - anchor="process_flag_message_queue_data" since="?"/> + anchor="process_flag_message_queue_data" since="OTP 19.0"/> <fsummary>Set process flag message_queue_data for the calling process. </fsummary> <type name="message_queue_data"/> @@ -5047,7 +5071,7 @@ RealSystem = system + MissedSystem</code> </func> <func> - <name name="process_flag" arity="3" since="?"/> + <name name="process_flag" arity="3" since=""/> <fsummary>Set process flags for a process.</fsummary> <desc> <p>Sets certain flags for the process <c><anno>Pid</anno></c>, @@ -6399,7 +6423,7 @@ true</pre> <func> <name name="statistics" arity="1" clause_i="1" - anchor="statistics_active_tasks" since="?"/> + anchor="statistics_active_tasks" since="OTP 18.3"/> <fsummary>Information about active processes and ports.</fsummary> <desc> <p>Returns the same as @@ -6414,7 +6438,7 @@ true</pre> <func> <name name="statistics" arity="1" clause_i="2" - anchor="statistics_active_tasks_all" since="?"/> + anchor="statistics_active_tasks_all" since="OTP 20.0"/> <fsummary>Information about active processes and ports.</fsummary> <desc> <p>Returns a list where each element represents the amount @@ -6503,7 +6527,7 @@ true</pre> <func> <name name="statistics" arity="1" clause_i="7" - anchor="statistics_microstate_accounting" since="?"/> + anchor="statistics_microstate_accounting" since="OTP 19.0"/> <fsummary>Information about microstate accounting.</fsummary> <desc> <p>Microstate accounting can be used to measure how much time the Erlang @@ -6682,7 +6706,7 @@ lists:map( <func> <name name="statistics" arity="1" clause_i="10" - anchor="statistics_run_queue_lengths" since="?"/> + anchor="statistics_run_queue_lengths" since="OTP 18.3"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc> <p>Returns the same as @@ -6697,7 +6721,7 @@ lists:map( <func> <name name="statistics" arity="1" clause_i="11" - anchor="statistics_run_queue_lengths_all" since="?"/> + anchor="statistics_run_queue_lengths_all" since="OTP 20.0"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc> <p>Returns a list where each element represents the amount @@ -6758,7 +6782,7 @@ lists:map( <func> <name name="statistics" arity="1" clause_i="13" - anchor="statistics_scheduler_wall_time" since="?"/> + anchor="statistics_scheduler_wall_time" since="OTP R15B01"/> <fsummary>Information about each schedulers work time.</fsummary> <desc> <p>Returns a list of tuples with @@ -6882,7 +6906,7 @@ ok <func> <name name="statistics" arity="1" clause_i="14" - anchor="statistics_scheduler_wall_time_all" since="?"/> + anchor="statistics_scheduler_wall_time_all" since="OTP 20.0"/> <fsummary>Information about each schedulers work time.</fsummary> <desc> <p>The same as @@ -6910,7 +6934,7 @@ ok </func> <func> <name name="statistics" arity="1" clause_i="15" - anchor="statistics_total_active_tasks" since="?"/> + anchor="statistics_total_active_tasks" since="OTP 18.3"/> <fsummary>Information about active processes and ports.</fsummary> <desc> <p>The same as calling @@ -6921,7 +6945,7 @@ ok <func> <name name="statistics" arity="1" clause_i="16" - anchor="statistics_total_active_tasks_all" since="?"/> + anchor="statistics_total_active_tasks_all" since="OTP 20.0"/> <fsummary>Information about active processes and ports.</fsummary> <desc> <p>The same as calling @@ -6932,7 +6956,7 @@ ok <func> <name name="statistics" arity="1" clause_i="17" - anchor="statistics_total_run_queue_lengths" since="?"/> + anchor="statistics_total_run_queue_lengths" since="OTP 18.3"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc> <p>The same as calling @@ -6943,7 +6967,7 @@ ok <func> <name name="statistics" arity="1" clause_i="18" - anchor="statistics_total_run_queue_lengths_all" since="?"/> + anchor="statistics_total_run_queue_lengths_all" since="OTP 20.0"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc> <p>The same as calling @@ -7186,7 +7210,7 @@ ok <func> <name name="system_flag" arity="2" clause_i="3" - anchor="system_flag_dirty_cpu_schedulers_online" since="?"/> + anchor="system_flag_dirty_cpu_schedulers_online" since="OTP 17.0"/> <fsummary>Set system_flag_dirty_cpu_schedulers_online.</fsummary> <desc> <p> @@ -7214,7 +7238,7 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="4" since="?"/> + <name name="system_flag" arity="2" clause_i="4" since="OTP 20.2.3"/> <fsummary>Set system flag for erts_alloc.</fsummary> <desc> <p>Sets system flags for @@ -7251,7 +7275,7 @@ ok <func> <name name="system_flag" arity="2" clause_i="6" - anchor="system_flag_microstate_accounting" since="?"/> + anchor="system_flag_microstate_accounting" since="OTP 19.0"/> <fsummary>Set system flag microstate_accounting.</fsummary> <desc> <p> @@ -7279,7 +7303,7 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="8" since="?"/> + <name name="system_flag" arity="2" clause_i="8" since="OTP R13B04"/> <fsummary>Set system flag min_bin_vheap_size.</fsummary> <desc> <p>Sets the default minimum binary virtual heap size for @@ -7297,7 +7321,7 @@ ok <func> <name name="system_flag" arity="2" clause_i="9" - anchor="system_flag_max_heap_size" since="?"/> + anchor="system_flag_max_heap_size" since="OTP 19.0"/> <fsummary>Set system flag max_heap_size.</fsummary> <type name="max_heap_size"/> <desc> @@ -7498,7 +7522,7 @@ ok <func> <name name="system_flag" arity="2" clause_i="12" - anchor="system_flag_scheduler_wall_time" since="?"/> + anchor="system_flag_scheduler_wall_time" since="OTP R15B01"/> <fsummary>Set system flag scheduler_wall_time.</fsummary> <desc> <p> @@ -7539,7 +7563,39 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="14" since=""/> + <name name="system_flag" arity="2" clause_i="14" since="OTP 21.3"/> + <fsummary>Set system logger process.</fsummary> + <desc> + <p>Sets the process that will receive the logging + messages generated by ERTS. If set to <c>undefined</c>, + all logging messages generated by ERTS will be dropped. + The messages will be in the format:</p> + <code> +{log,Level,Format,ArgList,Metadata} where + +Level = atom(), +Format = string(), +ArgList = list(term()), +Metadata = #{ pid => pid(), + group_leader => pid(), + time := logger:timestamp(), + error_logger := #{ emulator := true, tag := atom() } + </code> + <p>If the <c>system_logger</c> process dies, + this flag will be reset to <c>logger</c>.</p> + <p>The default is the process named <c>logger</c>.</p> + <p>Returns the old value of the flag.</p> + <note><p>This function is designed to be used by the + KERNEL <seealso marker="kernel:logger"><c>logger</c></seealso>. + Be careful if you change it to something else as + log messages may be lost. If you want to intercept + emulator log messages, do it by adding a specialized handler + to the KERNEL logger.</p></note> + </desc> + </func> + + <func> + <name name="system_flag" arity="2" clause_i="15" since=""/> <fsummary>Set system flag trace_control_word.</fsummary> <desc> <p>Sets the value of the node trace control word to @@ -7553,8 +7609,8 @@ ok </func> <func> - <name name="system_flag" arity="2" clause_i="15" - anchor="system_flag_time_offset" since="?"/> + <name name="system_flag" arity="2" clause_i="16" + anchor="system_flag_time_offset" since="OTP 18.0"/> <fsummary>Finalize the time offset.</fsummary> <desc> <p> @@ -7703,8 +7759,9 @@ ok <seealso marker="#system_info_nif_version"><c>nif_version</c></seealso>, <seealso marker="#system_info_otp_release"><c>otp_release</c></seealso>, <seealso marker="#system_info_port_parallelism"><c>port_parallelism</c></seealso>, - <seealso marker="#system_info_system_version"><c>system_version</c></seealso>, <seealso marker="#system_info_system_architecture"><c>system_architecture</c></seealso>, + <seealso marker="#system_info_system_logger"><c>system_logger</c></seealso>, + <seealso marker="#system_info_system_version"><c>system_version</c></seealso>, <seealso marker="#system_info_trace_control_word"><c>trace_control_word</c></seealso>, <seealso marker="#system_info_version"><c>version</c></seealso>, <seealso marker="#system_info_wordsize"><c>wordsize</c></seealso> @@ -7876,7 +7933,7 @@ ok anchor="system_info_cpu_topology" since=""/> <!-- cpu_topology --> <name name="system_info" arity="1" clause_i="13" since=""/> <!-- {cpu_topology, _} --> <name name="system_info" arity="1" clause_i="38" since=""/> <!-- logical_processors --> - <name name="system_info" arity="1" clause_i="73" since="?"/> <!-- update_cpu_info --> + <name name="system_info" arity="1" clause_i="74" since="OTP R14B"/> <!-- update_cpu_info --> <fsummary>Information about the CPU topology of the system.</fsummary> <type name="cpu_topology"/> <type name="level_entry"/> @@ -8028,14 +8085,14 @@ ok <func> <name name="system_info" arity="1" clause_i="31" - anchor="system_info_process" since="?"/> <!-- fullsweep_after --> + anchor="system_info_process" since=""/> <!-- fullsweep_after --> <name name="system_info" arity="1" clause_i="32" since=""/> <!-- garbage_collection --> <name name="system_info" arity="1" clause_i="33" since=""/> <!-- heap_sizes --> <name name="system_info" arity="1" clause_i="34" since=""/> <!-- heap_type --> - <name name="system_info" arity="1" clause_i="40" since="?"/> <!-- max_heap_size --> - <name name="system_info" arity="1" clause_i="41" since="?"/> <!-- message_queue_data --> - <name name="system_info" arity="1" clause_i="42" since="?"/> <!-- min_heap_size --> - <name name="system_info" arity="1" clause_i="43" since="?"/> <!-- min_bin_vheap_size --> + <name name="system_info" arity="1" clause_i="40" since="OTP 19.0"/> <!-- max_heap_size --> + <name name="system_info" arity="1" clause_i="41" since="OTP 19.0"/> <!-- message_queue_data --> + <name name="system_info" arity="1" clause_i="42" since="OTP R13B04"/> <!-- min_heap_size --> + <name name="system_info" arity="1" clause_i="43" since="OTP R13B04"/> <!-- min_bin_vheap_size --> <name name="system_info" arity="1" clause_i="57" since=""/> <!-- procs --> <fsummary>Information about the default process heap settings.</fsummary> <type name="message_queue_data"/> @@ -8146,12 +8203,12 @@ ok </func> <func> - <name name="system_info" arity="1" clause_i="6" anchor="system_info_limits" since="?"/> <!-- atom_count --> - <name name="system_info" arity="1" clause_i="7" since="?"/> <!-- atom_limit --> - <name name="system_info" arity="1" clause_i="29" since="?"/> <!-- ets_count --> - <name name="system_info" arity="1" clause_i="30" since="?"/> <!-- ets_limit --> - <name name="system_info" arity="1" clause_i="53" since="?"/> <!-- port_count --> - <name name="system_info" arity="1" clause_i="54" since="?"/> <!-- port_limit --> + <name name="system_info" arity="1" clause_i="6" anchor="system_info_limits" since="OTP 20.0"/> <!-- atom_count --> + <name name="system_info" arity="1" clause_i="7" since="OTP 20.0"/> <!-- atom_limit --> + <name name="system_info" arity="1" clause_i="29" since="OTP 21.1"/> <!-- ets_count --> + <name name="system_info" arity="1" clause_i="30" since="OTP R16B03"/> <!-- ets_limit --> + <name name="system_info" arity="1" clause_i="53" since="OTP R16B"/> <!-- port_count --> + <name name="system_info" arity="1" clause_i="54" since="OTP R16B"/> <!-- port_limit --> <name name="system_info" arity="1" clause_i="55" since=""/> <!-- process_count --> <name name="system_info" arity="1" clause_i="56" since=""/> <!-- process_limit --> <fsummary>Information about various system limits.</fsummary> @@ -8231,10 +8288,10 @@ ok <name name="system_info" arity="1" clause_i="50" since="OTP 18.0"/> <!-- os_monotonic_time_source --> <name name="system_info" arity="1" clause_i="51" since="OTP 18.0"/> <!-- os_system_time_source --> <name name="system_info" arity="1" clause_i="63" since="OTP 18.0"/> <!-- start_time --> - <name name="system_info" arity="1" clause_i="68" since="OTP 18.0"/> <!-- time_correction --> - <name name="system_info" arity="1" clause_i="69" since="OTP 18.0"/> <!-- time_offset --> - <name name="system_info" arity="1" clause_i="70" since="OTP 18.0"/> <!-- time_warp_mode --> - <name name="system_info" arity="1" clause_i="71" since="?"/> <!-- tolerant_timeofday --> + <name name="system_info" arity="1" clause_i="69" since="OTP 18.0"/> <!-- time_correction --> + <name name="system_info" arity="1" clause_i="70" since="OTP 18.0"/> <!-- time_offset --> + <name name="system_info" arity="1" clause_i="71" since="OTP 18.0"/> <!-- time_warp_mode --> + <name name="system_info" arity="1" clause_i="72" since="OTP 17.1"/> <!-- tolerant_timeofday --> <fsummary>Information about system time.</fsummary> <desc> <marker id="system_info_time_tags"/> @@ -8324,7 +8381,7 @@ ok system time</seealso> that is used by the runtime system.</p> <p>The list contains two-tuples with <c>Key</c>s as first element, and <c>Value</c>s as second element. The - order if these tuples is undefined. The following + order of these tuples is undefined. The following tuples can be part of the list, but more tuples can be introduced in the future:</p> <taglist> @@ -8455,19 +8512,19 @@ ok <func> <name name="system_info" arity="1" clause_i="17" - anchor="system_info_scheduler" since="?"/> <!-- dirty_cpu_schedulers --> - <name name="system_info" arity="1" clause_i="18" since="?"/> <!-- dirty_cpu_schedulers_online --> - <name name="system_info" arity="1" clause_i="19" since="?"/> <!-- dirty_io_schedulers --> + anchor="system_info_scheduler" since="OTP 17.0"/> <!-- dirty_cpu_schedulers --> + <name name="system_info" arity="1" clause_i="18" since="OTP 17.0"/> <!-- dirty_cpu_schedulers_online --> + <name name="system_info" arity="1" clause_i="19" since="OTP 17.0"/> <!-- dirty_io_schedulers --> <name name="system_info" arity="1" clause_i="45" since=""/> <!-- multi_scheduling --> <name name="system_info" arity="1" clause_i="46" since=""/> <!-- multi_scheduling_blockers --> - <name name="system_info" arity="1" clause_i="49" since="?"/> <!-- normal_multi_scheduling_blockers --> + <name name="system_info" arity="1" clause_i="49" since="OTP 19.0"/> <!-- normal_multi_scheduling_blockers --> <name name="system_info" arity="1" clause_i="58" since=""/> <!-- scheduler_bind_type --> <name name="system_info" arity="1" clause_i="59" since=""/> <!-- scheduler_bindings --> <name name="system_info" arity="1" clause_i="60" since=""/> <!-- scheduler_id --> <name name="system_info" arity="1" clause_i="61" since=""/> <!-- schedulers --> <name name="system_info" arity="1" clause_i="62" since=""/> <!-- smp_support --> - <name name="system_info" arity="1" clause_i="66" since=""/> <!-- threads --> - <name name="system_info" arity="1" clause_i="67" since=""/> <!-- thread_pool_size --> + <name name="system_info" arity="1" clause_i="67" since=""/> <!-- threads --> + <name name="system_info" arity="1" clause_i="68" since=""/> <!-- thread_pool_size --> <fsummary>Information about system schedulers.</fsummary> <desc> <marker id="system_info_scheduler_tags"/> @@ -8756,9 +8813,9 @@ ok <func> <name name="system_info" arity="1" clause_i="14" anchor="system_info_dist" since=""/> <!-- creation --> - <name name="system_info" arity="1" clause_i="16" since="?"/> <!-- delayed_node_table_gc --> + <name name="system_info" arity="1" clause_i="16" since="OTP 18.0"/> <!-- delayed_node_table_gc --> <name name="system_info" arity="1" clause_i="20" since=""/> <!-- dist --> - <name name="system_info" arity="1" clause_i="21" since="?"/> <!-- dist_buf_busy_limit --> + <name name="system_info" arity="1" clause_i="21" since="OTP R14B01"/> <!-- dist_buf_busy_limit --> <name name="system_info" arity="1" clause_i="22" since=""/> <!-- dist_ctrl --> <fsummary>Information about erlang distribution.</fsummary> <desc> @@ -8833,7 +8890,7 @@ ok <!-- <name name="system_info" arity="1" clause_i="6"/> atom_count --> <!-- <name name="system_info" arity="1" clause_i="7"/> atom_limit --> <name name="system_info" arity="1" clause_i="8" - anchor="system_info_misc" since="?"/> <!-- build_type --> + anchor="system_info_misc" since="OTP R14B"/> <!-- build_type --> <name name="system_info" arity="1" clause_i="9" since=""/> <!-- c_compiler_used --> <name name="system_info" arity="1" clause_i="10" since=""/> <!-- check_io --> <name name="system_info" arity="1" clause_i="11" since=""/> <!-- compat_rel --> @@ -8849,8 +8906,8 @@ ok <!-- <name name="system_info" arity="1" clause_i="21"/> dist_buf_busy_limit --> <!-- <name name="system_info" arity="1" clause_i="22"/> dist_ctrl --> <name name="system_info" arity="1" clause_i="23" since=""/> <!-- driver_version --> - <name name="system_info" arity="1" clause_i="24" since="?"/> <!-- dynamic_trace --> - <name name="system_info" arity="1" clause_i="25" since="?"/> <!-- dynamic_trace_probes --> + <name name="system_info" arity="1" clause_i="24" since="OTP R15B01"/> <!-- dynamic_trace --> + <name name="system_info" arity="1" clause_i="25" since="OTP R15B01"/> <!-- dynamic_trace_probes --> <!-- <name name="system_info" arity="1" clause_i="26"/> end_time --> <!-- <name name="system_info" arity="1" clause_i="27"/> elib_malloc --> <!-- <name name="system_info" arity="1" clause_i="28"/> eager_check_io, removed --> @@ -8872,12 +8929,12 @@ ok <name name="system_info" arity="1" clause_i="44" since=""/> <!-- modified_timing_level --> <!-- <name name="system_info" arity="1" clause_i="45"/> multi_scheduling --> <!-- <name name="system_info" arity="1" clause_i="46"/> multi_scheduling_blockers --> - <name name="system_info" arity="1" clause_i="47" since="?"/> <!-- nif_version --> + <name name="system_info" arity="1" clause_i="47" since="OTP 17.4"/> <!-- nif_version --> <!-- n<name name="system_info" arity="1" clause_i="48"/> ormal_multi_scheduling_blockers --> <name name="system_info" arity="1" clause_i="49" since=""/> <!-- otp_release --> <!-- <name name="system_info" arity="1" clause_i="50"/> os_monotonic_time_source --> <!-- <name name="system_info" arity="1" clause_i="51"/> os_system_time_source --> - <name name="system_info" arity="1" clause_i="52" since="?"/> <!-- port_parallelism --> + <name name="system_info" arity="1" clause_i="52" since="OTP R16B"/> <!-- port_parallelism --> <!-- <name name="system_info" arity="1" clause_i="53"/> port_count --> <!-- <name name="system_info" arity="1" clause_i="54"/> port_limit --> <!-- <name name="system_info" arity="1" clause_i="55"/> process_count --> @@ -8889,19 +8946,20 @@ ok <!-- <name name="system_info" arity="1" clause_i="61"/> schedulers --> <!-- <name name="system_info" arity="1" clause_i="62"/> smp_support --> <!-- <name name="system_info" arity="1" clause_i="63"/> start_time --> - <name name="system_info" arity="1" clause_i="64" since=""/> <!-- system_version --> - <name name="system_info" arity="1" clause_i="65" since=""/> <!-- system_architecture --> - <!-- <name name="system_info" arity="1" clause_i="66"/> threads --> - <!-- <name name="system_info" arity="1" clause_i="67"/> thread_pool_size --> - <!-- <name name="system_info" arity="1" clause_i="68"/> time_correction --> - <!-- <name name="system_info" arity="1" clause_i="69"/> time_offset --> - <!-- <name name="system_info" arity="1" clause_i="70"/> time_warp_mode --> - <!-- <name name="system_info" arity="1" clause_i="71"/> tolerant_timeofday --> - <name name="system_info" arity="1" clause_i="72" since=""/> <!-- trace_control_word --> - <!-- <name name="system_info" arity="1" clause_i="73"/> update_cpu_info --> - <name name="system_info" arity="1" clause_i="74" since=""/> <!-- version --> - <name name="system_info" arity="1" clause_i="75" since=""/> <!-- wordsize --> - <!-- <name name="system_info" arity="1" clause_i="76"/> overview --> + <name name="system_info" arity="1" clause_i="64" since=""/> <!-- system_architecture --> + <name name="system_info" arity="1" clause_i="65" since="OTP 21.3"/> <!-- system_logger --> + <name name="system_info" arity="1" clause_i="66" since=""/> <!-- system_version --> + <!-- <name name="system_info" arity="1" clause_i="67"/> threads --> + <!-- <name name="system_info" arity="1" clause_i="68"/> thread_pool_size --> + <!-- <name name="system_info" arity="1" clause_i="69"/> time_correction --> + <!-- <name name="system_info" arity="1" clause_i="70"/> time_offset --> + <!-- <name name="system_info" arity="1" clause_i="71"/> time_warp_mode --> + <!-- <name name="system_info" arity="1" clause_i="72"/> tolerant_timeofday --> + <name name="system_info" arity="1" clause_i="73" since=""/> <!-- trace_control_word --> + <!-- <name name="system_info" arity="1" clause_i="74"/> update_cpu_info --> + <name name="system_info" arity="1" clause_i="75" since=""/> <!-- version --> + <name name="system_info" arity="1" clause_i="76" since=""/> <!-- wordsize --> + <!-- <name name="system_info" arity="1" clause_i="77"/> overview --> <fsummary>Information about the system.</fsummary> <desc> <marker id="system_info_misc_tags"/> @@ -9057,18 +9115,24 @@ ok <seealso marker="erl#+spp"><c>+spp</c></seealso> in <c>erl(1)</c>.</p> </item> - <tag><marker id="system_info_system_version"/> - <c>system_version</c></tag> - <item> - <p>Returns a string containing version number and - some important properties, such as the number of schedulers.</p> - </item> <tag><marker id="system_info_system_architecture"/> <c>system_architecture</c></tag> <item> <p>Returns a string containing the processor and OS architecture the emulator is built for.</p> </item> + <tag><marker id="system_info_system_logger"/> + <c>system_logger</c></tag> + <item> + <p>Returns the current <c>system_logger</c> as set by + <seealso marker="#system_flag/2"><c>erlang:system_flag(system_logger, _)</c></seealso>.</p> + </item> + <tag><marker id="system_info_system_version"/> + <c>system_version</c></tag> + <item> + <p>Returns a string containing version number and + some important properties, such as the number of schedulers.</p> + </item> <tag><marker id="system_info_trace_control_word"/> <c>trace_control_word</c></tag> <item> @@ -10497,7 +10561,7 @@ timestamp() -> </func> <func> - <name name="trace_pattern" arity="3" clause_i="1" since="?"/> + <name name="trace_pattern" arity="3" clause_i="1" since="OTP 19.0"/> <fsummary>Set trace pattern for message sending.</fsummary> <type name="trace_match_spec"/> <type name="match_variable"/> @@ -10568,7 +10632,7 @@ timestamp() -> </func> <func> - <name name="trace_pattern" arity="3" clause_i="2" since="?"/> + <name name="trace_pattern" arity="3" clause_i="2" since="OTP 19.0"/> <fsummary>Set trace pattern for tracing of message receiving.</fsummary> <type name="trace_match_spec"/> <type name="match_variable"/> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index f5fe9721c0..9ea1beb887 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -31,6 +31,93 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 10.2.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + When using the <c>{linger,{true,T}}</c> option; + <c>gen_tcp:listen/2</c> used the full linger time before + returning for example <c>eaddrinuse</c>. This bug has now + been corrected.</p> + <p> + Own Id: OTP-14728 Aux Id: ERIERL-303 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.2.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix bug where doing a <c>gen_tcp:send</c> on a socket + with <c>delay_send</c> set to true could cause a segfault + if the other side closes the connection.</p> + <p> + Bug was introduced in erts-10.2 (OTP-21.2).</p> + <p> + Own Id: OTP-15536 Aux Id: ERL-827 </p> + </item> + <item> + <p> + Fix a race condition when a port program closes that + could result in the next started port to hang during + startup.</p> + <p> + When this fault happens the following error is normally + (but not always) logged:</p> + <p> + <c> =ERROR REPORT==== 14-Jan-2019::10:45:52.868246 + ===</c><br/><c> Bad input fd in erts_poll()! fd=11, + port=#Port<0.505>, driver=spawn, name=/bin/sh -s + unix:cmd </c></p> + <p> + Bug was introduced in erts-10.0 (OTP-21.0).</p> + <p> + Own Id: OTP-15537</p> + </item> + <item> + <p> + Fix a bug where polling for external events could be + delayed for a very long time if all active schedulers + were 100% loaded.</p> + <p> + Bug was introduced in erts-10.2 (OTP-21.2).</p> + <p> + Own Id: OTP-15538 Aux Id: ERIERL-229 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.2.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a crash when dangling files were closed after + <c>init:restart/0</c>.</p> + <p> + Own Id: OTP-15495 Aux Id: ERL-821 </p> + </item> + <item> + <p> + A bug that could cause dirty schedulers to become + unresponsive has been fixed.</p> + <p> + Own Id: OTP-15509 Aux Id: PR-2027, PR-2093 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 10.2.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -1625,6 +1712,62 @@ </section> +<section><title>Erts 9.3.3.9</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Added an optional <c>./configure</c> flag to compile + the emulator with spectre mitigation: + <c>--with-spectre-mitigation</c></p> + <p>Note that this requires a recent version of GCC with + support for spectre mitigation and the + <c>--mindirect-branch=thunk</c> flag, such as + <c>8.1</c>.</p> + <p> + Own Id: OTP-15430 Aux Id: ERIERL-237 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 9.3.3.8</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A bug that could cause dirty schedulers to become + unresponsive has been fixed.</p> + <p> + Own Id: OTP-15509 Aux Id: PR-2027, PR-2093 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 9.3.3.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed bug in operator <c>band</c> of two negative + operands causing erroneous result if the absolute value + of one of the operands have the lowest <c>N*W</c> bits as + zero and the other absolute value is not larger than + <c>N*W</c> bits. <c>N</c> is an integer of 1 or larger + and <c>W</c> is 32 or 64 depending on word size.</p> + <p> + Own Id: OTP-15487 Aux Id: ERL-804 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 9.3.3.6</title> <section><title>Improvements and New Features</title> @@ -3200,6 +3343,43 @@ </section> +<section><title>Erts 8.3.5.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed bug in operator <c>band</c> of two negative + operands causing erroneous result if the absolute value + of one of the operands have the lowest <c>N*W</c> bits as + zero and the other absolute value is not larger than + <c>N*W</c> bits. <c>N</c> is an integer of 1 or larger + and <c>W</c> is 32 or 64 depending on word size.</p> + <p> + Own Id: OTP-15487 Aux Id: ERL-804 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Added an optional <c>./configure</c> flag to compile + the emulator with spectre mitigation: + <c>--with-spectre-mitigation</c></p> + <p>Note that this requires a recent version of GCC with + support for spectre mitigation and the + <c>--mindirect-branch=thunk</c> flag, such as + <c>8.1</c>.</p> + <p> + Own Id: OTP-15430 Aux Id: ERIERL-237 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 8.3.5.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/doc/src/persistent_term.xml b/erts/doc/src/persistent_term.xml index 1eda7f8d76..9d3c9afd80 100644 --- a/erts/doc/src/persistent_term.xml +++ b/erts/doc/src/persistent_term.xml @@ -256,6 +256,22 @@ will be slower as the number of persistent terms increases.</pre> </func> <func> + <name name="get" arity="2" since="OTP 21.3"/> + <fsummary>Get the value for a persistent term.</fsummary> + <desc> + <p>Retrieve the value for the persistent term associated with + the key <c><anno>Key</anno></c>. The lookup will be made in + constant time and the value will not be copied to the heap + of the calling process.</p> + <p>This function returns <c><anno>Default</anno></c> if no + term has been stored with the key <c><anno>Key</anno></c>.</p> + <p>If the calling process holds on to the value of the + persistent term and the persistent term is deleted in the future, + the term will be copied to the process.</p> + </desc> + </func> + + <func> <name name="info" arity="0" since="OTP 21.2"/> <fsummary>Get information about persistent terms.</fsummary> <desc> diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index e909a0b4da..4351dda5a7 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -379,6 +379,7 @@ do { \ # define NOINLINE #endif +int tuple_module_apply; /* * The following functions are called directly by process_main(). @@ -2210,6 +2211,7 @@ apply(Process* p, Eterm* reg, BeamInstr *I, Uint stack_offset) Eterm module = reg[0]; Eterm function = reg[1]; Eterm args = reg[2]; + Eterm this; /* * Check the arguments which should be of the form apply(Module, @@ -2232,8 +2234,20 @@ apply(Process* p, Eterm* reg, BeamInstr *I, Uint stack_offset) while (1) { Eterm m, f, a; - - if (is_not_atom(module)) goto error; + /* The module argument may be either an atom or an abstract module + * (currently implemented using tuples, but this might change). + */ + this = THE_NON_VALUE; + if (is_not_atom(module)) { + Eterm* tp; + + if (!tuple_module_apply || is_not_tuple(module)) goto error; + tp = tuple_val(module); + if (arityval(tp[0]) < 1) goto error; + this = module; + module = tp[1]; + if (is_not_atom(module)) goto error; + } if (module != am_erlang || function != am_apply) break; @@ -2268,7 +2282,9 @@ apply(Process* p, Eterm* reg, BeamInstr *I, Uint stack_offset) } /* * Walk down the 3rd parameter of apply (the argument list) and copy - * the parameters to the x registers (reg[]). + * the parameters to the x registers (reg[]). If the module argument + * was an abstract module, add 1 to the function arity and put the + * module argument in the n+1st x register as a THIS reference. */ tmp = args; @@ -2285,6 +2301,9 @@ apply(Process* p, Eterm* reg, BeamInstr *I, Uint stack_offset) if (is_not_nil(tmp)) { /* Must be well-formed list */ goto error; } + if (this != THE_NON_VALUE) { + reg[arity++] = this; + } /* * Get the index into the export table, or failing that the export @@ -2323,7 +2342,18 @@ fixed_apply(Process* p, Eterm* reg, Uint arity, return 0; } - if (is_not_atom(module)) goto error; + /* The module argument may be either an atom or an abstract module + * (currently implemented using tuples, but this might change). + */ + if (is_not_atom(module)) { + Eterm* tp; + if (!tuple_module_apply || is_not_tuple(module)) goto error; + tp = tuple_val(module); + if (arityval(tp[0]) < 1) goto error; + module = tp[1]; + if (is_not_atom(module)) goto error; + ++arity; + } /* Handle apply of apply/3... */ if (module == am_erlang && function == am_apply && arity == 3) { diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index 04498332b0..000397e790 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -2810,38 +2810,110 @@ BIF_RETTYPE list_to_existing_atom_1(BIF_ALIST_1) /* convert an integer to a list of ascii integers */ -BIF_RETTYPE integer_to_list_1(BIF_ALIST_1) +static Eterm integer_to_list(Process *c_p, Eterm num, int base) { - Eterm* hp; + Eterm *hp; + Eterm res; Uint need; + if (is_small(num)) { + char s[128]; + char *c = s; + Uint digits; + + digits = Sint_to_buf(signed_val(num), base, &c, sizeof(s)); + need = 2 * digits; + + hp = HAlloc(c_p, need); + res = buf_to_intlist(&hp, c, digits, NIL); + } else { + const int DIGITS_PER_RED = 16; + Eterm *hp_end; + Uint digits; + + digits = big_integer_estimate(num, base); + + if ((digits / DIGITS_PER_RED) > ERTS_BIF_REDS_LEFT(c_p)) { + ErtsSchedulerData *esdp = erts_get_scheduler_data(); + + /* This could take a very long time, tell the caller to reschedule + * us to a dirty CPU scheduler if we aren't already on one. */ + if (esdp->type == ERTS_SCHED_NORMAL) { + return THE_NON_VALUE; + } + } else { + BUMP_REDS(c_p, digits / DIGITS_PER_RED); + } + + need = 2 * digits; + + hp = HAlloc(c_p, need); + hp_end = hp + need; + + res = erts_big_to_list(num, base, &hp); + HRelease(c_p, hp_end, hp); + } + + return res; +} + +BIF_RETTYPE integer_to_list_1(BIF_ALIST_1) +{ + Eterm res; + if (is_not_integer(BIF_ARG_1)) { - BIF_ERROR(BIF_P, BADARG); + BIF_ERROR(BIF_P, BADARG); } - if (is_small(BIF_ARG_1)) { - char *c; - int n; - struct Sint_buf ibuf; + res = integer_to_list(BIF_P, BIF_ARG_1, 10); - c = Sint_to_buf(signed_val(BIF_ARG_1), &ibuf); - n = sys_strlen(c); - need = 2*n; - hp = HAlloc(BIF_P, need); - BIF_RET(buf_to_intlist(&hp, c, n, NIL)); + if (is_non_value(res)) { + Eterm args[1]; + args[0] = BIF_ARG_1; + return erts_schedule_bif(BIF_P, + args, + BIF_I, + integer_to_list_1, + ERTS_SCHED_DIRTY_CPU, + am_erlang, + am_integer_to_list, + 1); } - else { - int n = big_decimal_estimate(BIF_ARG_1); - Eterm res; - Eterm* hp_end; - need = 2*n; - hp = HAlloc(BIF_P, need); - hp_end = hp + need; - res = erts_big_to_list(BIF_ARG_1, &hp); - HRelease(BIF_P,hp_end,hp); - BIF_RET(res); + return res; +} + +BIF_RETTYPE integer_to_list_2(BIF_ALIST_2) +{ + Eterm res; + SWord base; + + if (is_not_integer(BIF_ARG_1) || is_not_small(BIF_ARG_2)) { + BIF_ERROR(BIF_P, BADARG); } + + base = signed_val(BIF_ARG_2); + if (base < 2 || base > 36) { + BIF_ERROR(BIF_P, BADARG); + } + + res = integer_to_list(BIF_P, BIF_ARG_1, base); + + if (is_non_value(res)) { + Eterm args[2]; + args[0] = BIF_ARG_1; + args[1] = BIF_ARG_2; + return erts_schedule_bif(BIF_P, + args, + BIF_I, + integer_to_list_2, + ERTS_SCHED_DIRTY_CPU, + am_erlang, + am_integer_to_list, + 2); + } + + return res; } /**********************************************************************/ @@ -4629,6 +4701,9 @@ BIF_RETTYPE system_flag_2(BIF_ALIST_2) return erts_bind_schedulers(BIF_P, BIF_ARG_2); } else if (ERTS_IS_ATOM_STR("erts_alloc", BIF_ARG_1)) { return erts_alloc_set_dyn_param(BIF_P, BIF_ARG_2); + } else if (ERTS_IS_ATOM_STR("system_logger", BIF_ARG_1)) { + Eterm res = erts_set_system_logger(BIF_ARG_2); + if (is_value(res)) BIF_RET(res); } error: BIF_ERROR(BIF_P, BADARG); @@ -5131,61 +5206,6 @@ erts_call_dirty_bif(ErtsSchedulerData *esdp, Process *c_p, BeamInstr *I, Eterm * return exiting; } - - -#ifdef HARDDEBUG -/* -You'll need this line in bif.tab to be able to use this debug bif - -bif erlang:send_to_logger/2 - -*/ -BIF_RETTYPE send_to_logger_2(BIF_ALIST_2) -{ - byte *buf; - ErlDrvSizeT len; - if (!is_atom(BIF_ARG_1) || !(is_list(BIF_ARG_2) || - is_nil(BIF_ARG_1))) { - BIF_ERROR(BIF_P,BADARG); - } - if (erts_iolist_size(BIF_ARG_2, &len) != 0) - BIF_ERROR(BIF_P,BADARG); - else if (len == 0) - buf = ""; - else { -#ifdef DEBUG - ErlDrvSizeT len2; -#endif - buf = (byte *) erts_alloc(ERTS_ALC_T_TMP, len+1); -#ifdef DEBUG - len2 = -#else - (void) -#endif - erts_iolist_to_buf(BIF_ARG_2, buf, len); - ASSERT(len2 == len); - buf[len] = '\0'; - switch (BIF_ARG_1) { - case am_info: - erts_send_info_to_logger(BIF_P->group_leader, buf, len); - break; - case am_warning: - erts_send_warning_to_logger(BIF_P->group_leader, buf, len); - break; - case am_error: - erts_send_error_to_logger(BIF_P->group_leader, buf, len); - break; - default: - { - BIF_ERROR(BIF_P,BADARG); - } - } - erts_free(ERTS_ALC_T_TMP, (void *) buf); - } - BIF_RET(am_true); -} -#endif /* HARDDEBUG */ - BIF_RETTYPE get_module_info_1(BIF_ALIST_1) { Eterm ret = erts_module_info_0(BIF_P, BIF_ARG_1); diff --git a/erts/emulator/beam/bif.tab b/erts/emulator/beam/bif.tab index d4ba90a61a..8419244832 100644 --- a/erts/emulator/beam/bif.tab +++ b/erts/emulator/beam/bif.tab @@ -725,3 +725,17 @@ bif erts_internal:counters_get/2 bif erts_internal:counters_add/3 bif erts_internal:counters_put/3 bif erts_internal:counters_info/1 + +# +# New in 21.2.3 +# + +bif erts_internal:spawn_system_process/3 + +# +# New in 21.3 +# + +bif erlang:integer_to_list/2 +bif erlang:integer_to_binary/2 +bif persistent_term:get/2 diff --git a/erts/emulator/beam/big.c b/erts/emulator/beam/big.c index 7e0be2a2a7..522f50287a 100644 --- a/erts/emulator/beam/big.c +++ b/erts/emulator/beam/big.c @@ -429,6 +429,7 @@ static const byte digits_per_sint_lookup[36-1]; static const byte digits_per_small_lookup[36-1]; static const Sint largest_power_of_base_lookup[36-1]; +static const double lg2_lookup[36-1]; static ERTS_INLINE byte get_digits_per_signed_int(Uint base) { return digits_per_sint_lookup[base-2]; @@ -442,6 +443,10 @@ static ERTS_INLINE Sint get_largest_power_of_base(Uint base) { return largest_power_of_base_lookup[base-2]; } +static ERTS_INLINE double lookup_log2(Uint base) { + return lg2_lookup[base - 2]; +} + /* ** compare two number vectors */ @@ -1720,23 +1725,23 @@ double_to_big(double x, Eterm *heap, Uint hsz) /* - ** Estimate the number of decimal digits (include sign) + ** Estimate the number of digits in given base (include sign) */ -int big_decimal_estimate(Wterm x) +int big_integer_estimate(Wterm x, Uint base) { Eterm* xp = big_val(x); int lg = I_lg(BIG_V(xp), BIG_SIZE(xp)); - int lg10 = ((lg+1)*28/93)+1; + int lgBase = ((lg + 1) / lookup_log2(base)) + 1; - if (BIG_SIGN(xp)) lg10++; /* add sign */ - return lg10+1; /* add null */ + if (BIG_SIGN(xp)) lgBase++; /* add sign */ + return lgBase + 1; /* add null */ } /* -** Convert a bignum into a string of decimal numbers +** Convert a bignum into a string of numbers in given base */ - -static Uint write_big(Wterm x, void (*write_func)(void *, char), void *arg) +static Uint write_big(Wterm x, int base, void (*write_func)(void *, char), + void *arg) { Eterm* xp = big_val(x); ErtsDigit* dx = BIG_V(xp); @@ -1744,48 +1749,72 @@ static Uint write_big(Wterm x, void (*write_func)(void *, char), void *arg) short sign = BIG_SIGN(xp); ErtsDigit rem; Uint n = 0; - const Uint digits_per_Sint = get_digits_per_signed_int(10); - const Sint largest_pow_of_base = get_largest_power_of_base(10); + const Uint digits_per_Sint = get_digits_per_signed_int(base); + const Sint largest_pow_of_base = get_largest_power_of_base(base); if (xl == 1 && *dx < largest_pow_of_base) { - rem = *dx; - if (rem == 0) { - (*write_func)(arg, '0'); n++; - } else { - while(rem) { - (*write_func)(arg, (rem % 10) + '0'); n++; - rem /= 10; - } - } + rem = *dx; + if (rem == 0) { + (*write_func)(arg, '0'); n++; + } else { + while(rem) { + int digit = rem % base; + + if (digit < 10) { + (*write_func)(arg, digit + '0'); n++; + } else { + (*write_func)(arg, 'A' + (digit - 10)); n++; + } + + rem /= base; + } + } } else { - ErtsDigit* tmp = (ErtsDigit*) erts_alloc(ERTS_ALC_T_TMP, - sizeof(ErtsDigit)*xl); - dsize_t tmpl = xl; + ErtsDigit* tmp = (ErtsDigit*) erts_alloc(ERTS_ALC_T_TMP, + sizeof(ErtsDigit) * xl); + dsize_t tmpl = xl; - MOVE_DIGITS(tmp, dx, xl); + MOVE_DIGITS(tmp, dx, xl); - while(1) { + while(1) { tmpl = D_div(tmp, tmpl, largest_pow_of_base, tmp, &rem); - if (tmpl == 1 && *tmp == 0) { - while(rem) { - (*write_func)(arg, (rem % 10)+'0'); n++; - rem /= 10; - } - break; - } else { + + if (tmpl == 1 && *tmp == 0) { + while(rem) { + int digit = rem % base; + + if (digit < 10) { + (*write_func)(arg, digit + '0'); n++; + } else { + (*write_func)(arg, 'A' + (digit - 10)); n++; + } + + rem /= base; + } + break; + } else { Uint i = digits_per_Sint; - while(i--) { - (*write_func)(arg, (rem % 10)+'0'); n++; - rem /= 10; - } - } - } - erts_free(ERTS_ALC_T_TMP, (void *) tmp); + + while(i--) { + int digit = rem % base; + + if (digit < 10) { + (*write_func)(arg, digit + '0'); n++; + } else { + (*write_func)(arg, 'A' + (digit - 10)); n++; + } + + rem /= base; + } + } + } + erts_free(ERTS_ALC_T_TMP, (void *) tmp); } if (sign) { - (*write_func)(arg, '-'); n++; + (*write_func)(arg, '-'); n++; } + return n; } @@ -1802,12 +1831,12 @@ write_list(void *arg, char c) blp->hp += 2; } -Eterm erts_big_to_list(Eterm x, Eterm **hpp) +Eterm erts_big_to_list(Eterm x, int base, Eterm **hpp) { struct big_list__ bl; bl.hp = *hpp; bl.res = NIL; - write_big(x, write_list, (void *) &bl); + write_big(x, base, write_list, (void *) &bl); *hpp = bl.hp; return bl.res; } @@ -1818,11 +1847,11 @@ write_string(void *arg, char c) *(--(*((char **) arg))) = c; } -char *erts_big_to_string(Wterm x, char *buf, Uint buf_sz) +char *erts_big_to_string(Wterm x, int base, char *buf, Uint buf_sz) { char *big_str = buf + buf_sz - 1; *big_str = '\0'; - write_big(x, write_string, (void *) &big_str); + write_big(x, base, write_string, (void*)&big_str); ASSERT(buf <= big_str && big_str <= buf + buf_sz - 1); return big_str; } @@ -1831,11 +1860,11 @@ char *erts_big_to_string(Wterm x, char *buf, Uint buf_sz) * e.g. 1 bsl 64 -> "18446744073709551616" */ -Uint erts_big_to_binary_bytes(Eterm x, char *buf, Uint buf_sz) +Uint erts_big_to_binary_bytes(Eterm x, int base, char *buf, Uint buf_sz) { char *big_str = buf + buf_sz; Uint n; - n = write_big(x, write_string, (void *) &big_str); + n = write_big(x, base, write_string, (void *) &big_str); ASSERT(buf <= big_str && big_str <= buf + buf_sz); return n; } @@ -2578,9 +2607,6 @@ static const double lg2_lookup[36-1] = { 4.32193, 4.39232, 4.45943, 4.52356, 4.58496, 4.64386, 4.70044, 4.75489, 4.80735, 4.85798, 4.90689, 4.9542, 5.0, 5.04439, 5.08746, 5.12928, 5.16993 }; -static ERTS_INLINE double lookup_log2(Uint base) { - return lg2_lookup[base - 2]; -} /* * How many digits can fit into a signed int (Sint) for given base, we take diff --git a/erts/emulator/beam/big.h b/erts/emulator/beam/big.h index 7556205063..274482a0d2 100644 --- a/erts/emulator/beam/big.h +++ b/erts/emulator/beam/big.h @@ -81,7 +81,11 @@ typedef Uint dsize_t; /* Vector size type */ * a Uint64 argument. Therefore, we must test the size of the argument * to ensure that the cast does not discard the high-order 32 bits. */ -#define _IS_SSMALL32(x) (((Uint32) ((((x)) >> (SMALL_BITS-1)) + 1)) < 2) +#if defined(ARCH_32) +# define _IS_SSMALL32(x) (((Uint32) ((((x)) >> (SMALL_BITS-1)) + 1)) < 2) +#else +# define _IS_SSMALL32(x) (1) +#endif #define _IS_SSMALL64(x) (((Uint64) ((((x)) >> (SMALL_BITS-1)) + 1)) < 2) #define IS_SSMALL(x) (sizeof(x) == sizeof(Uint32) ? _IS_SSMALL32(x) : _IS_SSMALL64(x)) @@ -119,10 +123,10 @@ typedef Uint dsize_t; /* Vector size type */ #endif -int big_decimal_estimate(Wterm); -Eterm erts_big_to_list(Eterm, Eterm**); -char *erts_big_to_string(Wterm x, char *buf, Uint buf_sz); -Uint erts_big_to_binary_bytes(Eterm x, char *buf, Uint buf_sz); +int big_integer_estimate(Wterm, Uint base); +Eterm erts_big_to_list(Eterm, int base, Eterm**); +char *erts_big_to_string(Wterm x, int base, char *buf, Uint buf_sz); +Uint erts_big_to_binary_bytes(Eterm x, int base, char *buf, Uint buf_sz); Eterm small_times(Sint, Sint, Eterm*); diff --git a/erts/emulator/beam/binary.c b/erts/emulator/beam/binary.c index 6a349764b2..a18228b84a 100644 --- a/erts/emulator/beam/binary.c +++ b/erts/emulator/beam/binary.c @@ -322,40 +322,102 @@ BIF_RETTYPE binary_to_integer_2(BIF_ALIST_2) } +static Eterm integer_to_binary(Process *c_p, Eterm num, int base) +{ + Eterm res; + + if (is_small(num)) { + char s[128]; + char *c = s; + Uint digits; + + digits = Sint_to_buf(signed_val(num), base, &c, sizeof(s)); + res = new_binary(c_p, (byte*)c, digits); + } else { + const int DIGITS_PER_RED = 16; + Uint digits, n; + byte *bytes; + + digits = big_integer_estimate(num, base); + + if ((digits / DIGITS_PER_RED) > ERTS_BIF_REDS_LEFT(c_p)) { + ErtsSchedulerData *esdp = erts_get_scheduler_data(); + + /* This could take a very long time, tell the caller to reschedule + * us to a dirty CPU scheduler if we aren't already on one. */ + if (esdp->type == ERTS_SCHED_NORMAL) { + return THE_NON_VALUE; + } + } else { + BUMP_REDS(c_p, digits / DIGITS_PER_RED); + } + + bytes = (byte*)erts_alloc(ERTS_ALC_T_TMP, sizeof(byte) * digits); + n = erts_big_to_binary_bytes(num, base, (char*)bytes, digits); + res = new_binary(c_p, bytes + digits - n, n); + erts_free(ERTS_ALC_T_TMP, (void*)bytes); + } + + return res; +} + BIF_RETTYPE integer_to_binary_1(BIF_ALIST_1) -{ - Uint size; +{ Eterm res; if (is_not_integer(BIF_ARG_1)) { - BIF_ERROR(BIF_P, BADARG); + BIF_ERROR(BIF_P, BADARG); } - if (is_small(BIF_ARG_1)) { - char *c; - struct Sint_buf ibuf; + res = integer_to_binary(BIF_P, BIF_ARG_1, 10); + + if (is_non_value(res)) { + Eterm args[1]; + args[0] = BIF_ARG_1; + return erts_schedule_bif(BIF_P, + args, + BIF_I, + integer_to_binary_1, + ERTS_SCHED_DIRTY_CPU, + am_erlang, + am_integer_to_binary, + 1); + } - /* Enhancement: If we can calculate the buffer size exactly - * we could avoid an unnecessary copy of buffers. - * Useful if size determination is faster than a copy. - */ - c = Sint_to_buf(signed_val(BIF_ARG_1), &ibuf); - size = sys_strlen(c); - res = new_binary(BIF_P, (byte *)c, size); - } else { - byte* bytes; - Uint n = 0; + return res; +} - /* Here we also have multiple copies of buffers - * due to new_binary interface - */ - size = big_decimal_estimate(BIF_ARG_1) - 1; /* remove null */ - bytes = (byte*) erts_alloc(ERTS_ALC_T_TMP, sizeof(byte)*size); - n = erts_big_to_binary_bytes(BIF_ARG_1, (char *)bytes, size); - res = new_binary(BIF_P, bytes + size - n, n); - erts_free(ERTS_ALC_T_TMP, (void *) bytes); +BIF_RETTYPE integer_to_binary_2(BIF_ALIST_2) +{ + Eterm res; + SWord base; + + if (is_not_integer(BIF_ARG_1) || is_not_small(BIF_ARG_2)) { + BIF_ERROR(BIF_P, BADARG); + } + + base = signed_val(BIF_ARG_2); + if (base < 2 || base > 36) { + BIF_ERROR(BIF_P, BADARG); } - BIF_RET(res); + + res = integer_to_binary(BIF_P, BIF_ARG_1, base); + + if (is_non_value(res)) { + Eterm args[2]; + args[0] = BIF_ARG_1; + args[1] = BIF_ARG_2; + return erts_schedule_bif(BIF_P, + args, + BIF_I, + integer_to_binary_2, + ERTS_SCHED_DIRTY_CPU, + am_erlang, + am_integer_to_binary, + 2); + } + + return res; } #define ERTS_B2L_BYTES_PER_REDUCTION 256 diff --git a/erts/emulator/beam/break.c b/erts/emulator/beam/break.c index 9ff52c92b8..92009c2345 100644 --- a/erts/emulator/beam/break.c +++ b/erts/emulator/beam/break.c @@ -82,7 +82,7 @@ process_info(fmtfn_t to, void *to_arg) * they are most likely just created and has invalid data */ if (!ERTS_PROC_IS_EXITING(p) && p->heap != NULL) - print_process_info(to, to_arg, p); + print_process_info(to, to_arg, p, 0); } } @@ -101,7 +101,7 @@ process_killer(void) rp = erts_pix2proc(i); if (rp && rp->i != ENULL) { int br; - print_process_info(ERTS_PRINT_STDOUT, NULL, rp); + print_process_info(ERTS_PRINT_STDOUT, NULL, rp, 0); erts_printf("(k)ill (n)ext (r)eturn:\n"); while(1) { if ((j = sys_get_key(0)) <= 0) @@ -199,13 +199,14 @@ static void doit_print_monitor(ErtsMonitor *mon, void *vpcontext) /* Display info about an individual Erlang process */ void -print_process_info(fmtfn_t to, void *to_arg, Process *p) +print_process_info(fmtfn_t to, void *to_arg, Process *p, ErtsProcLocks orig_locks) { int garbing = 0; int running = 0; Sint len; struct saved_calls *scb; erts_aint32_t state; + ErtsProcLocks locks = orig_locks; /* display the PID */ erts_print(to, to_arg, "=proc:%T\n", p->common.id); @@ -222,6 +223,22 @@ print_process_info(fmtfn_t to, void *to_arg, Process *p) | ERTS_PSFLG_DIRTY_RUNNING)) running = 1; + if (!(locks & ERTS_PROC_LOCK_MAIN)) { + locks |= ERTS_PROC_LOCK_MAIN; + if (ERTS_IS_CRASH_DUMPING && running) { + if (erts_proc_trylock(p, locks)) { + /* crash dumping and main lock taken, this probably means that + the process is doing a GC on a dirty-scheduler... so we cannot + do erts_proc_sig_fetch as that would potentially cause a segfault */ + locks = 0; + } + } else { + erts_proc_lock(p, locks); + } + } else { + ERTS_ASSERT(locks == ERTS_PROC_LOCK_MAIN && "Only main lock should be held"); + } + /* * If the process is registered as a global process, display the * registered name @@ -251,13 +268,19 @@ print_process_info(fmtfn_t to, void *to_arg, Process *p) erts_print(to, to_arg, "Spawned by: %T\n", p->parent); - erts_proc_lock(p, ERTS_PROC_LOCK_MSGQ); - len = erts_proc_sig_fetch(p); - erts_proc_unlock(p, ERTS_PROC_LOCK_MSGQ); + if (locks & ERTS_PROC_LOCK_MAIN) { + erts_proc_lock(p, ERTS_PROC_LOCK_MSGQ); + len = erts_proc_sig_fetch(p); + erts_proc_unlock(p, ERTS_PROC_LOCK_MSGQ); + } else { + len = p->sig_qs.len; + } erts_print(to, to_arg, "Message queue length: %d\n", len); - /* display the message queue only if there is anything in it */ - if (!ERTS_IS_CRASH_DUMPING && p->sig_qs.first != NULL && !garbing) { + /* display the message queue only if there is anything in it + and we can do it safely */ + if (!ERTS_IS_CRASH_DUMPING && p->sig_qs.first != NULL && !garbing + && (locks & ERTS_PROC_LOCK_MAIN)) { erts_print(to, to_arg, "Message queue: ["); ERTS_FOREACH_SIG_PRIVQS( p, mp, @@ -357,6 +380,8 @@ print_process_info(fmtfn_t to, void *to_arg, Process *p) /* Display all states */ erts_print(to, to_arg, "Internal State: "); erts_dump_extended_process_state(to, to_arg, state); + + erts_proc_unlock(p, locks & ~orig_locks); } static void diff --git a/erts/emulator/beam/erl_ao_firstfit_alloc.c b/erts/emulator/beam/erl_ao_firstfit_alloc.c index 3f0ab33597..917cb1cf10 100644 --- a/erts/emulator/beam/erl_ao_firstfit_alloc.c +++ b/erts/emulator/beam/erl_ao_firstfit_alloc.c @@ -100,8 +100,8 @@ #define AOFF_BLK_SZ(B) MBC_FBLK_SZ(&(B)->hdr) -#define LIST_NEXT(N) (((AOFF_RBTree_t*)(N))->u.next) -#define LIST_PREV(N) (((AOFF_RBTree_t*)(N))->parent) +#define AOFF_LIST_NEXT(N) (((AOFF_RBTree_t*)(N))->u.next) +#define AOFF_LIST_PREV(N) (((AOFF_RBTree_t*)(N))->parent) typedef struct AOFF_Carrier_t_ AOFF_Carrier_t; @@ -152,13 +152,13 @@ static ERTS_INLINE Uint node_max_size(AOFF_RBTree_t *x) static ERTS_INLINE void lower_max_size(AOFF_RBTree_t *node, AOFF_RBTree_t* stop_at) { - AOFF_RBTree_t* x = node; + AOFF_RBTree_t* x = node; Uint old_max = x->max_sz; Uint new_max = node_max_size(x); if (new_max < old_max) { x->max_sz = new_max; - while ((x=x->parent) != stop_at && x->max_sz == old_max) { + while ((x=x->parent) != stop_at && x->max_sz == old_max) { x->max_sz = node_max_size(x); } ASSERT(x == stop_at || x->max_sz > old_max); @@ -352,7 +352,7 @@ left_rotate(AOFF_RBTree_t **root, AOFF_RBTree_t *x) x->parent = y; y->max_sz = x->max_sz; - x->max_sz = node_max_size(x); + x->max_sz = node_max_size(x); ASSERT(y->max_sz >= x->max_sz); } @@ -377,7 +377,7 @@ right_rotate(AOFF_RBTree_t **root, AOFF_RBTree_t *x) y->right = x; x->parent = y; y->max_sz = x->max_sz; - x->max_sz = node_max_size(x); + x->max_sz = node_max_size(x); ASSERT(y->max_sz >= x->max_sz); } @@ -523,23 +523,23 @@ aoff_unlink_free_block(Allctr_t *allctr, Block_t *blk) ASSERT(del->flags & IS_BF_FLG); if (IS_LIST_ELEM(del)) { /* Remove from list */ - ASSERT(LIST_PREV(del)); - ASSERT(LIST_PREV(del)->flags & IS_BF_FLG); - LIST_NEXT(LIST_PREV(del)) = LIST_NEXT(del); - if (LIST_NEXT(del)) { - ASSERT(LIST_NEXT(del)->flags & IS_BF_FLG); - LIST_PREV(LIST_NEXT(del)) = LIST_PREV(del); + ASSERT(AOFF_LIST_PREV(del)); + ASSERT(AOFF_LIST_PREV(del)->flags & IS_BF_FLG); + AOFF_LIST_NEXT(AOFF_LIST_PREV(del)) = AOFF_LIST_NEXT(del); + if (AOFF_LIST_NEXT(del)) { + ASSERT(AOFF_LIST_NEXT(del)->flags & IS_BF_FLG); + AOFF_LIST_PREV(AOFF_LIST_NEXT(del)) = AOFF_LIST_PREV(del); } return; } - else if (LIST_NEXT(del)) { + else if (AOFF_LIST_NEXT(del)) { /* Replace tree node by next element in list... */ - - ASSERT(AOFF_BLK_SZ(LIST_NEXT(del)) == AOFF_BLK_SZ(del)); - ASSERT(IS_LIST_ELEM(LIST_NEXT(del))); - - replace(&crr->root, (AOFF_RBTree_t*)del, LIST_NEXT(del)); - + + ASSERT(AOFF_BLK_SZ(AOFF_LIST_NEXT(del)) == AOFF_BLK_SZ(del)); + ASSERT(IS_LIST_ELEM(AOFF_LIST_NEXT(del))); + + replace(&crr->root, (AOFF_RBTree_t*)del, AOFF_LIST_NEXT(del)); + HARD_CHECK_TREE(&crr->crr, alc->blk_order, crr->root, 0); return; } @@ -550,13 +550,13 @@ aoff_unlink_free_block(Allctr_t *allctr, Block_t *blk) HARD_CHECK_TREE(&crr->crr, alc->blk_order, crr->root, 0); /* Update the carrier tree with a potentially new (lower) max_sz - */ + */ if (crr->root) { if (crr->rbt_node.hdr.bhdr == crr->root->max_sz) { return; } ASSERT(crr->rbt_node.hdr.bhdr > crr->root->max_sz); - crr->rbt_node.hdr.bhdr = crr->root->max_sz; + crr->rbt_node.hdr.bhdr = crr->root->max_sz; } else { crr->rbt_node.hdr.bhdr = 0; @@ -773,7 +773,7 @@ rbt_insert(enum AOFFSortOrder order, AOFF_RBTree_t** root, AOFF_RBTree_t* blk) #ifdef DEBUG blk->flags = (order == FF_BF) ? IS_BF_FLG : 0; #else - blk->flags = 0; + blk->flags = 0; #endif blk->left = NULL; blk->right = NULL; @@ -787,7 +787,7 @@ rbt_insert(enum AOFFSortOrder order, AOFF_RBTree_t** root, AOFF_RBTree_t* blk) else { AOFF_RBTree_t *x = *root; while (1) { - SWord diff; + SWord diff; if (x->max_sz < blk_sz) { x->max_sz = blk_sz; } @@ -810,14 +810,14 @@ rbt_insert(enum AOFFSortOrder order, AOFF_RBTree_t** root, AOFF_RBTree_t* blk) } else { ASSERT(order == FF_BF); - ASSERT(blk->flags & IS_BF_FLG); - ASSERT(x->flags & IS_BF_FLG); + ASSERT(blk->flags & IS_BF_FLG); + ASSERT(x->flags & IS_BF_FLG); SET_LIST_ELEM(blk); - LIST_NEXT(blk) = LIST_NEXT(x); - LIST_PREV(blk) = x; - if (LIST_NEXT(x)) - LIST_PREV(LIST_NEXT(x)) = blk; - LIST_NEXT(x) = blk; + AOFF_LIST_NEXT(blk) = AOFF_LIST_NEXT(x); + AOFF_LIST_PREV(blk) = x; + if (AOFF_LIST_NEXT(x)) + AOFF_LIST_PREV(AOFF_LIST_NEXT(x)) = blk; + AOFF_LIST_NEXT(x) = blk; return; } } @@ -831,7 +831,7 @@ rbt_insert(enum AOFFSortOrder order, AOFF_RBTree_t** root, AOFF_RBTree_t* blk) } if (order == FF_BF) { SET_TREE_NODE(blk); - LIST_NEXT(blk) = NULL; + AOFF_LIST_NEXT(blk) = NULL; } } @@ -878,7 +878,7 @@ aoff_get_free_block(Allctr_t *allctr, Uint size, #ifdef HARD_DEBUG AOFF_RBTree_t* dbg_blk; #endif - + ASSERT(!cand_blk || cand_size >= size); /* Get first-fit carrier @@ -964,7 +964,7 @@ static void aoff_add_mbc(Allctr_t *allctr, Carrier_t *carrier) AOFF_RBTree_t **root = &alc->mbc_root; ASSERT(!IS_CRR_IN_TREE(crr, *root)); - HARD_CHECK_TREE(NULL, alc->crr_order, *root, 0); + HARD_CHECK_TREE(NULL, alc->crr_order, *root, 0); rbt_insert(alc->crr_order, root, &crr->rbt_node); @@ -1105,7 +1105,7 @@ info_options(Allctr_t *allctr, } if (hpp || szp) { - + if (!atoms_initialized) erts_exit(ERTS_ERROR_EXIT, "%s:%d: Internal error: Atoms not initialized", __FILE__, __LINE__);; @@ -1132,7 +1132,7 @@ erts_aoffalc_test(UWord op, UWord a1, UWord a2) switch (op) { case 0x500: return (UWord) ((AOFFAllctr_t *) a1)->blk_order == FF_AOBF; case 0x501: { - AOFF_RBTree_t *node = ((AOFFAllctr_t *) a1)->mbc_root; + AOFF_RBTree_t *node = ((AOFFAllctr_t *) a1)->mbc_root; Uint size = (Uint) a2; node = node ? rbt_search(node, size) : NULL; return (UWord) (node ? RBT_NODE_TO_MBC(node)->root : NULL); @@ -1140,13 +1140,13 @@ erts_aoffalc_test(UWord op, UWord a1, UWord a2) case 0x502: return (UWord) ((AOFF_RBTree_t *) a1)->parent; case 0x503: return (UWord) ((AOFF_RBTree_t *) a1)->left; case 0x504: return (UWord) ((AOFF_RBTree_t *) a1)->right; - case 0x505: return (UWord) LIST_NEXT(a1); + case 0x505: return (UWord) AOFF_LIST_NEXT(a1); case 0x506: return (UWord) IS_BLACK((AOFF_RBTree_t *) a1); case 0x507: return (UWord) IS_TREE_NODE((AOFF_RBTree_t *) a1); case 0x508: return (UWord) 0; /* IS_BF_ALGO */ case 0x509: return (UWord) ((AOFF_RBTree_t *) a1)->max_sz; case 0x50a: return (UWord) ((AOFFAllctr_t *) a1)->blk_order == FF_BF; - case 0x50b: return (UWord) LIST_PREV(a1); + case 0x50b: return (UWord) AOFF_LIST_PREV(a1); default: ASSERT(0); return ~((UWord) 0); } } @@ -1166,7 +1166,7 @@ static int rbt_is_member(AOFF_RBTree_t* root, AOFF_RBTree_t* node) return 0; } node = node->parent; - } + } return 1; } @@ -1279,15 +1279,15 @@ check_tree(Carrier_t* within_crr, enum AOFFSortOrder order, AOFF_RBTree_t* root, } if (order == FF_BF) { AOFF_RBTree_t* y = x; - AOFF_RBTree_t* nxt = LIST_NEXT(y); + AOFF_RBTree_t* nxt = AOFF_LIST_NEXT(y); ASSERT(IS_TREE_NODE(x)); while (nxt) { ASSERT(IS_LIST_ELEM(nxt)); ASSERT(AOFF_BLK_SZ(nxt) == AOFF_BLK_SZ(x)); ASSERT(FBLK_TO_MBC(&nxt->hdr) == within_crr); - ASSERT(LIST_PREV(nxt) == y); + ASSERT(AOFF_LIST_PREV(nxt) == y); y = nxt; - nxt = LIST_NEXT(nxt); + nxt = AOFF_LIST_NEXT(nxt); } } @@ -1301,13 +1301,13 @@ check_tree(Carrier_t* within_crr, enum AOFFSortOrder order, AOFF_RBTree_t* root, if (x->left) { ASSERT(x->left->parent == x); ASSERT(cmp_blocks(order, x->left, x) < 0); - ASSERT(x->left->max_sz <= x->max_sz); + ASSERT(x->left->max_sz <= x->max_sz); } if (x->right) { ASSERT(x->right->parent == x); ASSERT(cmp_blocks(order, x->right, x) > 0); - ASSERT(x->right->max_sz <= x->max_sz); + ASSERT(x->right->max_sz <= x->max_sz); } ASSERT(x->max_sz >= AOFF_BLK_SZ(x)); ASSERT(x->max_sz == AOFF_BLK_SZ(x) @@ -1327,7 +1327,7 @@ check_tree(Carrier_t* within_crr, enum AOFFSortOrder order, AOFF_RBTree_t* root, x = x->parent; --depth; } - ASSERT(depth == 0 || (!root && depth==1)); + ASSERT(depth == 0 || (!root && depth==1)); ASSERT(curr_blacks == 0); ASSERT((1 << (max_depth/2)) <= node_cnt); @@ -1373,4 +1373,3 @@ print_tree(AOFF_RBTree_t* root) #endif /* PRINT_TREE */ #endif /* HARD_DEBUG */ - diff --git a/erts/emulator/beam/erl_bestfit_alloc.c b/erts/emulator/beam/erl_bestfit_alloc.c index 9cb1199c2a..3f981ca2bc 100644 --- a/erts/emulator/beam/erl_bestfit_alloc.c +++ b/erts/emulator/beam/erl_bestfit_alloc.c @@ -122,8 +122,8 @@ typedef struct { RBTree_t *next; } RBTreeList_t; -#define LIST_NEXT(N) (((RBTreeList_t *) (N))->next) -#define LIST_PREV(N) (((RBTreeList_t *) (N))->t.parent) +#define BF_LIST_NEXT(N) (((RBTreeList_t *) (N))->next) +#define BF_LIST_PREV(N) (((RBTreeList_t *) (N))->t.parent) #ifdef DEBUG @@ -593,7 +593,6 @@ aobf_link_free_block(Allctr_t *allctr, Block_t *block) RBTree_t *blk = (RBTree_t *) block; Uint blk_sz = BF_BLK_SZ(blk); - blk->flags = 0; blk->left = NULL; @@ -673,7 +672,7 @@ aobf_get_free_block(Allctr_t *allctr, Uint size, x = x->left; } } - + if (!blk) return NULL; @@ -729,11 +728,11 @@ bf_link_free_block(Allctr_t *allctr, Block_t *block) if (blk_sz == size) { SET_LIST_ELEM(blk); - LIST_NEXT(blk) = LIST_NEXT(x); - LIST_PREV(blk) = x; - if (LIST_NEXT(x)) - LIST_PREV(LIST_NEXT(x)) = blk; - LIST_NEXT(x) = blk; + BF_LIST_NEXT(blk) = BF_LIST_NEXT(x); + BF_LIST_PREV(blk) = x; + if (BF_LIST_NEXT(x)) + BF_LIST_PREV(BF_LIST_NEXT(x)) = blk; + BF_LIST_NEXT(x) = blk; return; /* Finnished */ } @@ -764,7 +763,7 @@ bf_link_free_block(Allctr_t *allctr, Block_t *block) } SET_TREE_NODE(blk); - LIST_NEXT(blk) = NULL; + BF_LIST_NEXT(blk) = NULL; #ifdef HARD_DEBUG check_tree(root, 0, 0); @@ -780,22 +779,22 @@ bf_unlink_free_block(Allctr_t *allctr, Block_t *block) if (IS_LIST_ELEM(x)) { /* Remove from list */ - ASSERT(LIST_PREV(x)); - LIST_NEXT(LIST_PREV(x)) = LIST_NEXT(x); - if (LIST_NEXT(x)) - LIST_PREV(LIST_NEXT(x)) = LIST_PREV(x); + ASSERT(BF_LIST_PREV(x)); + BF_LIST_NEXT(BF_LIST_PREV(x)) = BF_LIST_NEXT(x); + if (BF_LIST_NEXT(x)) + BF_LIST_PREV(BF_LIST_NEXT(x)) = BF_LIST_PREV(x); } - else if (LIST_NEXT(x)) { + else if (BF_LIST_NEXT(x)) { /* Replace tree node by next element in list... */ - ASSERT(BF_BLK_SZ(LIST_NEXT(x)) == BF_BLK_SZ(x)); + ASSERT(BF_BLK_SZ(BF_LIST_NEXT(x)) == BF_BLK_SZ(x)); ASSERT(IS_TREE_NODE(x)); - ASSERT(IS_LIST_ELEM(LIST_NEXT(x))); + ASSERT(IS_LIST_ELEM(BF_LIST_NEXT(x))); #ifdef HARD_DEBUG check_tree(root, 0, 0); #endif - replace(root, x, LIST_NEXT(x)); + replace(root, x, BF_LIST_NEXT(x)); #ifdef HARD_DEBUG check_tree(bfallctr, 0); @@ -834,7 +833,7 @@ bf_get_free_block(Allctr_t *allctr, Uint size, x = x->left; } } - + if (!blk) return NULL; @@ -853,7 +852,7 @@ bf_get_free_block(Allctr_t *allctr, Uint size, /* Use next block if it exist in order to avoid replacing the tree node */ - blk = LIST_NEXT(blk) ? LIST_NEXT(blk) : blk; + blk = BF_LIST_NEXT(blk) ? BF_LIST_NEXT(blk) : blk; bf_unlink_free_block(allctr, (Block_t *) blk); return (Block_t *) blk; @@ -938,7 +937,7 @@ info_options(Allctr_t *allctr, } if (hpp || szp) { - + if (!atoms_initialized) erts_exit(ERTS_ERROR_EXIT, "%s:%d: Internal error: Atoms not initialized", __FILE__, __LINE__);; @@ -969,12 +968,12 @@ erts_bfalc_test(UWord op, UWord a1, UWord a2) case 0x202: return (UWord) ((RBTree_t *) a1)->parent; case 0x203: return (UWord) ((RBTree_t *) a1)->left; case 0x204: return (UWord) ((RBTree_t *) a1)->right; - case 0x205: return (UWord) LIST_NEXT(a1); + case 0x205: return (UWord) BF_LIST_NEXT(a1); case 0x206: return (UWord) IS_BLACK((RBTree_t *) a1); case 0x207: return (UWord) IS_TREE_NODE((RBTree_t *) a1); case 0x208: return (UWord) 1; /* IS_BF_ALGO */ case 0x20a: return (UWord) !((BFAllctr_t *) a1)->address_order; /* IS_BF */ - case 0x20b: return (UWord) LIST_PREV(a1); + case 0x20b: return (UWord) BF_LIST_PREV(a1); default: ASSERT(0); return ~((UWord) 0); } } diff --git a/erts/emulator/beam/erl_bif_binary.c b/erts/emulator/beam/erl_bif_binary.c index a2610bf2e1..ae1bf6e652 100644 --- a/erts/emulator/beam/erl_bif_binary.c +++ b/erts/emulator/beam/erl_bif_binary.c @@ -2762,7 +2762,7 @@ static BIF_RETTYPE do_encode_unsigned(Process *p, Eterm uns, Eterm endianess) dsize_t num_parts = BIG_SIZE(bigp); Eterm res; byte *b; - ErtsDigit d; + ErtsDigit d = 0; if(BIG_SIGN(bigp)) { goto badarg; @@ -2778,26 +2778,22 @@ static BIF_RETTYPE do_encode_unsigned(Process *p, Eterm uns, Eterm endianess) if (endianess == am_big) { Sint i,j; j = 0; - d = BIG_DIGIT(bigp,0); for (i=n-1;i>=0;--i) { - b[i] = d & 0xFF; - if (!((++j) % sizeof(ErtsDigit))) { + if (!((j++) % sizeof(ErtsDigit))) { d = BIG_DIGIT(bigp,j / sizeof(ErtsDigit)); - } else { - d >>= 8; } + b[i] = d & 0xFF; + d >>= 8; } } else { Sint i,j; j = 0; - d = BIG_DIGIT(bigp,0); for (i=0;i<n;++i) { - b[i] = d & 0xFF; - if (!((++j) % sizeof(ErtsDigit))) { + if (!((j++) % sizeof(ErtsDigit))) { d = BIG_DIGIT(bigp,j / sizeof(ErtsDigit)); - } else { - d >>= 8; } + b[i] = d & 0xFF; + d >>= 8; } } diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 7fada0d548..f6a98c85fd 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -2967,7 +2967,7 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) } else if (ERTS_IS_ATOM_STR("context_reductions", BIF_ARG_1)) { BIF_RET(make_small(CONTEXT_REDS)); } else if (ERTS_IS_ATOM_STR("kernel_poll", BIF_ARG_1)) { -#ifdef ERTS_ENABLE_KERNEL_POLL +#if ERTS_ENABLE_KERNEL_POLL BIF_RET(am_true); #else BIF_RET(am_false); @@ -3139,6 +3139,8 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) DECL_AM(tag); BIF_RET(AM_tag); #endif + } else if (ERTS_IS_ATOM_STR("system_logger", BIF_ARG_1)) { + BIF_RET(erts_get_system_logger()); } BIF_ERROR(BIF_P, BADARG); @@ -4608,6 +4610,7 @@ BIF_RETTYPE erts_debug_set_internal_state_2(BIF_ALIST_2) } } else if (ERTS_IS_ATOM_STR("broken_halt", BIF_ARG_1)) { + erts_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); broken_halt_test(BIF_ARG_2); } else if (ERTS_IS_ATOM_STR("unique_monotonic_integer_state", BIF_ARG_1)) { diff --git a/erts/emulator/beam/erl_bif_persistent.c b/erts/emulator/beam/erl_bif_persistent.c index 9dca768a18..5a78a043ce 100644 --- a/erts/emulator/beam/erl_bif_persistent.c +++ b/erts/emulator/beam/erl_bif_persistent.c @@ -332,6 +332,23 @@ BIF_RETTYPE persistent_term_get_1(BIF_ALIST_1) BIF_ERROR(BIF_P, BADARG); } +BIF_RETTYPE persistent_term_get_2(BIF_ALIST_2) +{ + Eterm key = BIF_ARG_1; + Eterm result = BIF_ARG_2; + HashTable* hash_table = (HashTable *) erts_atomic_read_nob(&the_hash_table); + Uint entry_index; + Eterm term; + + entry_index = lookup(hash_table, key); + term = hash_table->term[entry_index]; + if (is_boxed(term)) { + ASSERT(is_tuple_arity(term, 2)); + result = tuple_val(term)[2]; + } + BIF_RET(result); +} + BIF_RETTYPE persistent_term_erase_1(BIF_ALIST_1) { Eterm key = BIF_ARG_1; diff --git a/erts/emulator/beam/erl_bif_port.c b/erts/emulator/beam/erl_bif_port.c index 7fe4e02782..ed825d3dda 100644 --- a/erts/emulator/beam/erl_bif_port.c +++ b/erts/emulator/beam/erl_bif_port.c @@ -891,10 +891,6 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_typep, int *err_nump) driver = &spawn_driver; } else if (*tp == am_fd) { /* An fd port */ - int n; - struct Sint_buf sbuf; - char* p; - if (arity != make_arityval(3)) { goto badarg; } @@ -904,15 +900,9 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_typep, int *err_nump) opts.ifd = unsigned_val(tp[1]); opts.ofd = unsigned_val(tp[2]); - /* Syntesize name from input and output descriptor. */ - name_buf = erts_alloc(ERTS_ALC_T_TMP, - 2*sizeof(struct Sint_buf) + 2); - p = Sint_to_buf(opts.ifd, &sbuf); - n = sys_strlen(p); - sys_strncpy(name_buf, p, n); - name_buf[n] = '/'; - p = Sint_to_buf(opts.ofd, &sbuf); - sys_strcpy(name_buf+n+1, p); + /* Syntesize name from input and output descriptor. */ + name_buf = erts_alloc(ERTS_ALC_T_TMP, 256); + erts_snprintf(name_buf, 256, "%i/%i", opts.ifd, opts.ofd); driver = &fd_driver; } else { diff --git a/erts/emulator/beam/erl_db_util.c b/erts/emulator/beam/erl_db_util.c index f1d47326b4..e2c029c244 100644 --- a/erts/emulator/beam/erl_db_util.c +++ b/erts/emulator/beam/erl_db_util.c @@ -2470,7 +2470,7 @@ restart: case matchProcessDump: { erts_dsprintf_buf_t *dsbufp = erts_create_tmp_dsbuf(0); ASSERT(c_p == self); - print_process_info(ERTS_PRINT_DSBUF, (void *) dsbufp, c_p); + print_process_info(ERTS_PRINT_DSBUF, (void *) dsbufp, c_p, ERTS_PROC_LOCK_MAIN); *esp++ = new_binary(build_proc, (byte *)dsbufp->str, dsbufp->str_len); erts_destroy_tmp_dsbuf(dsbufp); diff --git a/erts/emulator/beam/erl_hl_timer.c b/erts/emulator/beam/erl_hl_timer.c index ef7a55fa38..75ad6de2c9 100644 --- a/erts/emulator/beam/erl_hl_timer.c +++ b/erts/emulator/beam/erl_hl_timer.c @@ -29,8 +29,6 @@ # include "config.h" #endif -/* #define ERTS_MAGIC_REF_BIF_TIMERS */ - #include "sys.h" #include "global.h" #include "bif.h" @@ -39,9 +37,6 @@ #include "erl_time.h" #include "erl_hl_timer.h" #include "erl_proc_sig_queue.h" -#ifdef ERTS_MAGIC_REF_BIF_TIMERS -#include "erl_binary.h" -#endif #define ERTS_TMR_CHECK_CANCEL_ON_CREATE 0 @@ -195,14 +190,9 @@ struct ErtsBifTimer_ { } type; struct { erts_atomic32_t state; -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - ErtsMagicBinary *mbin; - ErtsHLTimerList proc_list; -#else Uint32 refn[ERTS_REF_NUMBERS]; ErtsBifTimerTree proc_tree; ErtsBifTimerTree tree; -#endif Eterm message; ErlHeapFragment *bp; } btm; @@ -220,11 +210,7 @@ typedef ErtsTimer *(*ErtsCreateTimerFunc)(ErtsSchedulerData *esdp, int short_time, ErtsTmrType type, void *rcvrp, Eterm rcvr, Eterm msg, -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - ErtsMagicBinary *mbin, -#else Uint32 *refn, -#endif void (*callback)(void *), void *arg); #ifdef SMALL_MEMORY @@ -303,16 +289,12 @@ typedef struct { struct ErtsHLTimerService_ { ErtsHLTCncldTmrQ canceled_queue; ErtsHLTimer *time_tree; -#ifndef ERTS_MAGIC_REF_BIF_TIMERS ErtsBifTimer *btm_tree; -#endif ErtsHLTimer *next_timeout; ErtsYieldingTimeoutState yield; ErtsTWheelTimer service_timer; }; -#ifndef ERTS_MAGIC_REF_BIF_TIMERS - static ERTS_INLINE int refn_is_lt(Uint32 *x, Uint32 *y) { @@ -334,8 +316,6 @@ refn_is_eq(Uint32 *x, Uint32 *y) return (x[0] == y[0]) & (x[1] == y[1]) & (x[2] == y[2]); } -#endif - #define ERTS_RBT_PREFIX time #define ERTS_RBT_T ErtsHLTimer #define ERTS_RBT_KEY_T ErtsMonotonicTime @@ -525,13 +505,7 @@ same_time_list_lookup(ErtsHLTimer *root, ErtsHLTimer *x) #endif /* ERTS_HLT_HARD_DEBUG */ -#ifdef ERTS_MAGIC_REF_BIF_TIMERS -#define ERTS_BTM_HLT2REFN(T) ((T)->btm.mbin->refn) -#else #define ERTS_BTM_HLT2REFN(T) ((T)->btm.refn) -#endif - -#ifndef ERTS_MAGIC_REF_BIF_TIMERS #define ERTS_RBT_PREFIX btm #define ERTS_RBT_T ErtsBifTimer @@ -576,87 +550,12 @@ same_time_list_lookup(ErtsHLTimer *root, ErtsHLTimer *x) #define ERTS_RBT_IS_EQ(KX, KY) refn_is_eq((KX), (KY)) #define ERTS_RBT_WANT_DELETE #define ERTS_RBT_WANT_INSERT -#ifndef ERTS_MAGIC_REF_BIF_TIMERS #define ERTS_RBT_WANT_LOOKUP -#endif #define ERTS_RBT_WANT_FOREACH #define ERTS_RBT_UNDEF #include "erl_rbtree.h" -#endif /* !ERTS_MAGIC_REF_BIF_TIMERS */ - -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - -static ERTS_INLINE void -proc_btm_list_insert(ErtsBifTimer **list, ErtsBifTimer *x) -{ - ErtsBifTimer *y = *list; - if (!y) { - x->btm.proc_list.next = x; - x->btm.proc_list.prev = x; - *list = x; - } - else { - ERTS_HLT_ASSERT(y->btm.proc_list.prev->btm.proc_list.next == y); - x->btm.proc_list.next = y; - x->btm.proc_list.prev = y->btm.proc_list.prev; - y->btm.proc_list.prev->btm.proc_list.next = x; - y->btm.proc_list.prev = x; - } -} - -static ERTS_INLINE void -proc_btm_list_delete(ErtsBifTimer **list, ErtsBifTimer *x) -{ - ErtsBifTimer *y = *list; - if (y == x && x->btm.proc_list.next == x) { - ERTS_HLT_ASSERT(x->btm.proc_list.prev == x); - *list = NULL; - } - else { - if (y == x) - *list = x->btm.proc_list.next; - ERTS_HLT_ASSERT(x->btm.proc_list.prev->btm.proc_list.next == x); - ERTS_HLT_ASSERT(x->btm.proc_list.next->btm.proc_list.prev == x); - x->btm.proc_list.prev->btm.proc_list.next = x->btm.proc_list.next; - x->btm.proc_list.next->btm.proc_list.prev = x->btm.proc_list.prev; - } - x->btm.proc_list.next = NULL; -} - -static ERTS_INLINE int -proc_btm_list_foreach_destroy_yielding(ErtsBifTimer **list, - void (*destroy)(ErtsBifTimer *, void *), - void *arg, - int limit) -{ - int i; - ErtsBifTimer *first, *last; - - first = *list; - if (!first) - return 0; - - last = first->btm.proc_list.prev; - for (i = 0; i < limit; i++) { - ErtsBifTimer *x = last; - last = last->btm.proc_list.prev; - (*destroy)(x, arg); - x->btm.proc_list.next = NULL; - if (x == first) { - *list = NULL; - return 0; - } - } - - last->btm.proc_list.next = first; - first->btm.proc_list.prev = last; - return 1; -} - -#else /* !ERTS_MAGIC_REF_BIF_TIMERS */ - #define ERTS_RBT_PREFIX proc_btm #define ERTS_RBT_T ErtsBifTimer #define ERTS_RBT_KEY_T Uint32 * @@ -700,16 +599,12 @@ proc_btm_list_foreach_destroy_yielding(ErtsBifTimer **list, #define ERTS_RBT_IS_EQ(KX, KY) refn_is_eq((KX), (KY)) #define ERTS_RBT_WANT_DELETE #define ERTS_RBT_WANT_INSERT -#ifndef ERTS_MAGIC_REF_BIF_TIMERS #define ERTS_RBT_WANT_LOOKUP -#endif #define ERTS_RBT_WANT_FOREACH_DESTROY_YIELDING #define ERTS_RBT_UNDEF #include "erl_rbtree.h" -#endif /* !ERTS_MAGIC_REF_BIF_TIMERS */ - static void init_canceled_queue(ErtsHLTCncldTmrQ *cq); void @@ -728,9 +623,7 @@ erts_create_timer_service(void) srv = erts_alloc_permanent_cache_aligned(ERTS_ALC_T_TIMER_SERVICE, sizeof(ErtsHLTimerService)); srv->time_tree = NULL; -#ifndef ERTS_MAGIC_REF_BIF_TIMERS srv->btm_tree = NULL; -#endif srv->next_timeout = NULL; srv->yield = init_yield; erts_twheel_init_timer(&srv->service_timer); @@ -805,40 +698,10 @@ port_timeout_common(Port *port, void *tmr) return 0; } -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - -static erts_atomic_t * -mbin_to_btmref__(ErtsMagicBinary *mbin) -{ - return erts_binary_to_magic_indirection((Binary *) mbin); -} - -static ERTS_INLINE void -magic_binary_init(ErtsMagicBinary *mbin, ErtsBifTimer *tmr) -{ - erts_atomic_t *aptr = mbin_to_btmref__(mbin); - erts_atomic_init_nob(aptr, (erts_aint_t) tmr); -} - -static ERTS_INLINE ErtsBifTimer * -magic_binary_to_btm(ErtsMagicBinary *mbin) -{ - erts_atomic_t *aptr = mbin_to_btmref__(mbin); - ErtsBifTimer *tmr = (ErtsBifTimer *) erts_atomic_read_nob(aptr); - ERTS_HLT_ASSERT(!tmr || tmr->btm.mbin == mbin); - return tmr; -} - -#endif /* ERTS_MAGIC_REF_BIF_TIMERS */ - static ERTS_INLINE erts_aint_t init_btm_specifics(ErtsSchedulerData *esdp, ErtsBifTimer *tmr, Eterm msg, -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - ErtsMagicBinary *mbin -#else Uint32 *refn -#endif ) { Uint hsz = is_immed(msg) ? ((Uint) 0) : size_object(msg); @@ -853,13 +716,6 @@ init_btm_specifics(ErtsSchedulerData *esdp, tmr->btm.message = copy_struct(msg, hsz, &hp, &bp->off_heap); tmr->btm.bp = bp; } -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - refc = 1; - tmr->btm.mbin = mbin; - erts_refc_inc(&mbin->refc, 1); - magic_binary_init(mbin, tmr); - tmr->btm.proc_list.next = NULL; -#else refc = 0; tmr->btm.refn[0] = refn[0]; tmr->btm.refn[1] = refn[1]; @@ -868,7 +724,6 @@ init_btm_specifics(ErtsSchedulerData *esdp, tmr->btm.proc_tree.parent = ERTS_HLT_PFIELD_NOT_IN_TABLE; btm_rbt_insert(&esdp->timer_service->btm_tree, tmr); -#endif erts_atomic32_init_nob(&tmr->btm.state, ERTS_TMR_STATE_ACTIVE); return refc; /* refc from magic binary... */ @@ -886,11 +741,6 @@ timer_destroy(ErtsTimer *tmr, int twt, int btm) erts_free(ERTS_ALC_T_HL_PTIMER, tmr); } else { -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - Binary *bp = (Binary *) tmr->btm.btm.mbin; - if (erts_refc_dectest(&bp->refc, 0) == 0) - erts_bin_free(bp); -#endif if (tmr->head.roflgs & ERTS_TMR_ROFLG_PRE_ALC) bif_timer_pre_free(&tmr->btm); else @@ -940,9 +790,6 @@ schedule_tw_timer_destroy(ErtsTWTimer *tmr) else { /* Message buffer already dropped... */ size = sizeof(ErtsBifTimer); -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - size += sizeof(ErtsMagicIndirectionWord); -#endif } erts_schedule_thr_prgr_later_cleanup_op( @@ -1006,11 +853,7 @@ create_tw_timer(ErtsSchedulerData *esdp, int short_time, ErtsTmrType type, void *rcvrp, Eterm rcvr, Eterm msg, -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - ErtsMagicBinary *mbin, -#else Uint32 *refn, -#endif void (*callback)(void *), void *arg) { ErtsTWTimer *tmr; @@ -1087,11 +930,7 @@ create_tw_timer(ErtsSchedulerData *esdp, refc += init_btm_specifics(esdp, (ErtsBifTimer *) tmr, msg, -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - mbin -#else refn -#endif ); break; @@ -1152,9 +991,6 @@ schedule_hl_timer_destroy(ErtsHLTimer *tmr, Uint32 roflgs) else { /* Message buffer already dropped... */ size = sizeof(ErtsBifTimer); -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - size += sizeof(ErtsMagicIndirectionWord); -#endif } erts_schedule_thr_prgr_later_cleanup_op( @@ -1192,34 +1028,6 @@ check_canceled_queue(ErtsSchedulerData *esdp, ErtsHLTimerService *srv) #endif } -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - -static int -bif_timer_ref_destructor(Binary *unused) -{ - return 1; -} - -static ERTS_INLINE void -btm_clear_magic_binary(ErtsBifTimer *tmr) -{ - erts_atomic_t *aptr = mbin_to_btmref__(tmr->btm.mbin); - Uint32 roflgs = tmr->type.head.roflgs; -#ifdef ERTS_HLT_DEBUG - erts_aint_t tval = erts_atomic_xchg_nob(aptr, - (erts_aint_t) NULL); - ERTS_HLT_ASSERT(tval == (erts_aint_t) tmr); -#else - erts_atomic_set_nob(aptr, (erts_aint_t) NULL); -#endif - if (roflgs & ERTS_TMR_ROFLG_HLT) - hl_timer_dec_refc(&tmr->type.hlt, roflgs); - else - tw_timer_dec_refc(&tmr->type.twt); -} - -#endif /* ERTS_MAGIC_REF_BIF_TIMERS */ - static ERTS_INLINE void bif_timer_timeout(ErtsHLTimerService *srv, ErtsBifTimer *tmr, @@ -1240,10 +1048,6 @@ bif_timer_timeout(ErtsHLTimerService *srv, if (state == ERTS_TMR_STATE_ACTIVE) { Process *proc; -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - btm_clear_magic_binary(tmr); -#endif - if (roflgs & ERTS_TMR_ROFLG_REG_NAME) { Eterm term; term = tmr->type.head.receiver.name; @@ -1266,18 +1070,11 @@ bif_timer_timeout(ErtsHLTimerService *srv, erts_proc_lock(proc, ERTS_PROC_LOCK_BTM); /* If the process is exiting do not disturb the cleanup... */ if (!ERTS_PROC_IS_EXITING(proc)) { -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - if (tmr->btm.proc_list.next) { - proc_btm_list_delete(&proc->bif_timers, tmr); - dec_refc = 1; - } -#else if (tmr->btm.proc_tree.parent != ERTS_HLT_PFIELD_NOT_IN_TABLE) { proc_btm_rbt_delete(&proc->bif_timers, tmr); tmr->btm.proc_tree.parent = ERTS_HLT_PFIELD_NOT_IN_TABLE; dec_refc = 1; } -#endif } erts_proc_unlock(proc, ERTS_PROC_LOCK_BTM); if (dec_refc) @@ -1287,25 +1084,18 @@ bif_timer_timeout(ErtsHLTimerService *srv, free_message_buffer(tmr->btm.bp); } -#ifndef ERTS_MAGIC_REF_BIF_TIMERS if (tmr->btm.tree.parent != ERTS_HLT_PFIELD_NOT_IN_TABLE) { btm_rbt_delete(&srv->btm_tree, tmr); tmr->btm.tree.parent = ERTS_HLT_PFIELD_NOT_IN_TABLE; } -#endif - } static void tw_bif_timer_timeout(void *vbtmp) { -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - ErtsHLTimerService *srv = NULL; -#else ErtsSchedulerData *esdp = erts_get_scheduler_data(); ErtsHLTimerService *srv = esdp->timer_service; -#endif ErtsBifTimer *btmp = (ErtsBifTimer *) vbtmp; bif_timer_timeout(srv, btmp, btmp->type.head.roflgs); tw_timer_dec_refc(&btmp->type.twt); @@ -1317,11 +1107,7 @@ create_hl_timer(ErtsSchedulerData *esdp, int short_time, ErtsTmrType type, void *rcvrp, Eterm rcvr, Eterm msg, -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - ErtsMagicBinary *mbin, -#else Uint32 *refn, -#endif void (*callback)(void *), void *arg) { ErtsHLTimerService *srv = esdp->timer_service; @@ -1407,11 +1193,7 @@ create_hl_timer(ErtsSchedulerData *esdp, refc += init_btm_specifics(esdp, (ErtsBifTimer *) tmr, msg, -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - mbin -#else refn -#endif ); } @@ -1628,7 +1410,6 @@ cleanup_sched_local_canceled_timer(ErtsSchedulerData *esdp, ERTS_HLT_ASSERT((tmr->head.roflgs & ERTS_TMR_ROFLG_SID_MASK) == (Uint32) esdp->no); -#ifndef ERTS_MAGIC_REF_BIF_TIMERS if (roflgs & ERTS_TMR_ROFLG_BIF_TMR) { ErtsBifTimer *btm = (ErtsBifTimer *) tmr; if (btm->btm.tree.parent != ERTS_HLT_PFIELD_NOT_IN_TABLE) { @@ -1636,7 +1417,6 @@ cleanup_sched_local_canceled_timer(ErtsSchedulerData *esdp, btm->btm.tree.parent = ERTS_HLT_PFIELD_NOT_IN_TABLE; } } -#endif if (roflgs & ERTS_TMR_ROFLG_HLT) { hlt_delete_timer(esdp, &tmr->hlt); @@ -1909,9 +1689,6 @@ setup_bif_timer(Process *c_p, int twheel, ErtsMonotonicTime timeout_pos, Eterm ref, tmo_msg, *hp; ErtsBifTimer *tmr; ErtsSchedulerData *esdp; -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - Binary *mbin; -#endif Eterm tmp_hp[4]; ErtsCreateTimerFunc create_timer; @@ -1920,18 +1697,10 @@ setup_bif_timer(Process *c_p, int twheel, ErtsMonotonicTime timeout_pos, esdp = erts_proc_sched_data(c_p); -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - mbin = erts_create_magic_indirection(bif_timer_ref_destructor); - hp = HAlloc(c_p, ERTS_MAGIC_REF_THING_SIZE); - ref = erts_mk_magic_ref(&hp, &c_p->off_heap, mbin); - ASSERT(erts_get_ref_numbers_thr_id(((ErtsMagicBinary *)mbin)->refn) - == (Uint32) esdp->no); -#else hp = HAlloc(c_p, ERTS_REF_THING_SIZE); ref = erts_sched_make_ref_in_buffer(esdp, hp); ASSERT(erts_get_ref_numbers_thr_id(internal_ordinary_ref_numbers(ref)) == (Uint32) esdp->no); -#endif tmo_msg = wrap ? TUPLE3(tmp_hp, am_timeout, ref, msg) : msg; @@ -1939,11 +1708,7 @@ setup_bif_timer(Process *c_p, int twheel, ErtsMonotonicTime timeout_pos, tmr = (ErtsBifTimer *) create_timer(esdp, timeout_pos, short_time, ERTS_TMR_BIF, NULL, rcvr, tmo_msg, -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - (ErtsMagicBinary *) mbin, -#else internal_ordinary_ref_numbers(ref), -#endif NULL, NULL); if (is_internal_pid(rcvr)) { @@ -1951,14 +1716,10 @@ setup_bif_timer(Process *c_p, int twheel, ErtsMonotonicTime timeout_pos, rcvr, ERTS_PROC_LOCK_BTM, ERTS_P2P_FLG_INC_REFC); if (!proc) { -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - btm_clear_magic_binary(tmr); -#else if (tmr->btm.tree.parent != ERTS_HLT_PFIELD_NOT_IN_TABLE) { btm_rbt_delete(&esdp->timer_service->btm_tree, tmr); tmr->btm.tree.parent = ERTS_HLT_PFIELD_NOT_IN_TABLE; } -#endif if (tmr->btm.bp) free_message_buffer(tmr->btm.bp); if (twheel) @@ -1968,11 +1729,7 @@ setup_bif_timer(Process *c_p, int twheel, ErtsMonotonicTime timeout_pos, timer_destroy((ErtsTimer *) tmr, twheel, 1); } else { -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - proc_btm_list_insert(&proc->bif_timers, tmr); -#else proc_btm_rbt_insert(&proc->bif_timers, tmr); -#endif erts_proc_unlock(proc, ERTS_PROC_LOCK_BTM); tmr->type.head.receiver.proc = proc; } @@ -2000,10 +1757,6 @@ cancel_bif_timer(ErtsBifTimer *tmr) if (state != ERTS_TMR_STATE_ACTIVE) return 0; -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - btm_clear_magic_binary(tmr); -#endif - if (tmr->btm.bp) free_message_buffer(tmr->btm.bp); @@ -2022,19 +1775,12 @@ cancel_bif_timer(ErtsBifTimer *tmr) * the btm tree by itself (it may be in * the middle of tree destruction). */ -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - if (!ERTS_PROC_IS_EXITING(proc) && tmr->btm.proc_list.next) { - proc_btm_list_delete(&proc->bif_timers, tmr); - res = 1; - } -#else if (!ERTS_PROC_IS_EXITING(proc) && tmr->btm.proc_tree.parent != ERTS_HLT_PFIELD_NOT_IN_TABLE) { proc_btm_rbt_delete(&proc->bif_timers, tmr); tmr->btm.proc_tree.parent = ERTS_HLT_PFIELD_NOT_IN_TABLE; res = 1; } -#endif erts_proc_unlock(proc, ERTS_PROC_LOCK_BTM); } @@ -2075,12 +1821,10 @@ access_btm(ErtsBifTimer *tmr, Uint32 sid, ErtsSchedulerData *esdp, int cancel) queue_canceled_timer(esdp, sid, (ErtsTimer *) tmr); } else { -#ifndef ERTS_MAGIC_REF_BIF_TIMERS if (tmr->btm.tree.parent != ERTS_HLT_PFIELD_NOT_IN_TABLE) { btm_rbt_delete(&esdp->timer_service->btm_tree, tmr); tmr->btm.tree.parent = ERTS_HLT_PFIELD_NOT_IN_TABLE; } -#endif if (is_hlt) { if (cncl_res > 0) hl_timer_dec_refc(&tmr->type.hlt, tmr->type.hlt.head.roflgs); @@ -2157,52 +1901,6 @@ send_async_info(Process *proc, ErtsProcLocks initial_locks, return am_ok; } -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - -static BIF_RETTYPE -access_bif_timer(Process *c_p, Eterm tref, int cancel, int async, int info) -{ - BIF_RETTYPE ret; - Eterm res; - Sint64 time_left; - - if (!is_internal_magic_ref(tref)) { - if (is_not_ref(tref)) { - ERTS_BIF_PREP_ERROR(ret, c_p, BADARG); - return ret; - } - time_left = -1; - } - else { - ErtsMagicBinary *mbin; - mbin = (ErtsMagicBinary *) erts_magic_ref2bin(tref); - if (mbin->destructor != bif_timer_ref_destructor) - time_left = -1; - else { - ErtsBifTimer *tmr; - Uint32 sid; - tmr = magic_binary_to_btm(mbin); - sid = erts_get_ref_numbers_thr_id(internal_magic_ref_numbers(tref)); - ASSERT(1 <= sid && sid <= erts_no_schedulers); - time_left = access_btm(tmr, sid, erts_proc_sched_data(c_p), cancel); - } - } - - if (!info) - res = am_ok; - else if (!async) - res = return_info(c_p, time_left); - else - res = send_async_info(c_p, ERTS_PROC_LOCK_MAIN, - tref, cancel, time_left); - - ERTS_BIF_PREP_RET(ret, res); - - return ret; -} - -#else /* !ERTS_MAGIC_REF_BIF_TIMERS */ - static ERTS_INLINE Eterm send_sync_info(Process *proc, ErtsProcLocks initial_locks, Uint32 *refn, int cancel, Sint64 time_left) @@ -2505,8 +2203,6 @@ no_timer: return no_timer_result(c_p, tref, cancel, async, info); } -#endif /* !ERTS_MAGIC_REF_BIF_TIMERS */ - static ERTS_INLINE int bool_arg(Eterm val, int *argp) { @@ -2584,18 +2280,11 @@ exit_cancel_bif_timer(ErtsBifTimer *tmr, void *vesdp) is_hlt = !!(roflgs & ERTS_TMR_ROFLG_HLT); ERTS_HLT_ASSERT(sid == erts_get_ref_numbers_thr_id(ERTS_BTM_HLT2REFN(tmr))); -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - ERTS_HLT_ASSERT(tmr->btm.proc_list.next); -#else ERTS_HLT_ASSERT(tmr->btm.proc_tree.parent != ERTS_HLT_PFIELD_NOT_IN_TABLE); tmr->btm.proc_tree.parent = ERTS_HLT_PFIELD_NOT_IN_TABLE; -#endif if (state == ERTS_TMR_STATE_ACTIVE) { -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - btm_clear_magic_binary(tmr); -#endif if (tmr->btm.bp) free_message_buffer(tmr->btm.bp); @@ -2604,12 +2293,10 @@ exit_cancel_bif_timer(ErtsBifTimer *tmr, void *vesdp) return; } -#ifndef ERTS_MAGIC_REF_BIF_TIMERS if (tmr->btm.tree.parent != ERTS_HLT_PFIELD_NOT_IN_TABLE) { btm_rbt_delete(&esdp->timer_service->btm_tree, tmr); tmr->btm.tree.parent = ERTS_HLT_PFIELD_NOT_IN_TABLE; } -#endif if (is_hlt) hlt_delete_timer(esdp, &tmr->type.hlt); else @@ -2627,28 +2314,17 @@ exit_cancel_bif_timer(ErtsBifTimer *tmr, void *vesdp) # define ERTS_BTM_MAX_DESTROY_LIMIT 50 #endif -#ifndef ERTS_MAGIC_REF_BIF_TIMERS typedef struct { ErtsBifTimers *bif_timers; union { proc_btm_rbt_yield_state_t proc_btm_yield_state; } u; } ErtsBifTimerYieldState; -#endif int erts_cancel_bif_timers(Process *p, ErtsBifTimers **btm, void **vyspp) { ErtsSchedulerData *esdp = erts_proc_sched_data(p); -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - - return proc_btm_list_foreach_destroy_yielding(btm, - exit_cancel_bif_timer, - (void *) esdp, - ERTS_BTM_MAX_DESTROY_LIMIT); - -#else /* !ERTS_MAGIC_REF_BIF_TIMERS */ - ErtsBifTimerYieldState ys = {*btm, {ERTS_RBT_YIELD_STAT_INITER}}; ErtsBifTimerYieldState *ysp; int res; @@ -2682,7 +2358,6 @@ int erts_cancel_bif_timers(Process *p, ErtsBifTimers **btm, void **vyspp) return res; -#endif /* !ERTS_MAGIC_REF_BIF_TIMERS */ } static ERTS_INLINE int @@ -3116,11 +2791,6 @@ btm_print(ErtsBifTimer *tmr, void *vbtmp, ErtsMonotonicTime tpos, int is_hlt) ErtsMonotonicTime left; Eterm receiver; -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - if (!(tmr->type.head.roflgs & ERTS_TMR_ROFLG_BIF_TMR)) - return; -#endif - if (is_hlt) { ERTS_HLT_ASSERT(tmr->type.head.roflgs & ERTS_TMR_ROFLG_HLT); if (tmr->type.hlt.timeout <= btmp->now) @@ -3149,22 +2819,6 @@ btm_print(ErtsBifTimer *tmr, void *vbtmp, ErtsMonotonicTime tpos, int is_hlt) (Sint64) left); } -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - -static void -hlt_btm_print(ErtsHLTimer *tmr, void *vbtmp) -{ - btm_print((ErtsBifTimer *) tmr, vbtmp, 0, 1); -} - -static void -twt_btm_print(void *vbtmp, ErtsMonotonicTime tpos, void *vtwtp) -{ - btm_print((ErtsBifTimer *) vtwtp, vbtmp, tpos, 0); -} - -#else - static void btm_tree_print(ErtsBifTimer *tmr, void *vbtmp) { @@ -3177,8 +2831,6 @@ btm_tree_print(ErtsBifTimer *tmr, void *vbtmp) btm_print(tmr, vbtmp, tpos, is_hlt); } -#endif - void erts_print_bif_timer_info(fmtfn_t to, void *to_arg) { @@ -3196,15 +2848,7 @@ erts_print_bif_timer_info(fmtfn_t to, void *to_arg) for (six = 0; six < erts_no_schedulers; six++) { ErtsHLTimerService *srv = erts_aligned_scheduler_data[six].esd.timer_service; -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - ErtsTimerWheel *twheel = - erts_aligned_scheduler_data[six].esd.timer_wheel; - erts_twheel_debug_foreach(twheel, tw_bif_timer_timeout, - twt_btm_print, (void *) &btmp); - time_rbt_foreach(srv->time_tree, hlt_btm_print, (void *) &btmp); -#else btm_rbt_foreach(srv->btm_tree, btm_tree_print, (void *) &btmp); -#endif } } @@ -3219,10 +2863,6 @@ typedef struct { static void debug_btm_foreach(ErtsBifTimer *tmr, void *vbtmfd) { -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - if (!(tmr->type.head.roflgs & ERTS_TMR_ROFLG_BIF_TMR)) - return; -#endif if (erts_atomic32_read_nob(&tmr->btm.state) == ERTS_TMR_STATE_ACTIVE) { ErtsBTMForeachDebug *btmfd = (ErtsBTMForeachDebug *) vbtmfd; Eterm id = ((tmr->type.head.roflgs & ERTS_TMR_ROFLG_REG_NAME) @@ -3232,22 +2872,6 @@ debug_btm_foreach(ErtsBifTimer *tmr, void *vbtmfd) } } -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - -static void -hlt_debug_btm_foreach(ErtsHLTimer *tmr, void *vbtmfd) -{ - debug_btm_foreach((ErtsBifTimer *) tmr, vbtmfd); -} - -static void -twt_debug_btm_foreach(void *vbtmfd, ErtsMonotonicTime tpos, void *vtwtp) -{ - debug_btm_foreach((ErtsBifTimer *) vtwtp, vbtmfd); -} - -#endif - void erts_debug_bif_timer_foreach(void (*func)(Eterm, Eterm, @@ -3267,20 +2891,9 @@ erts_debug_bif_timer_foreach(void (*func)(Eterm, for (six = 0; six < erts_no_schedulers; six++) { ErtsHLTimerService *srv = erts_aligned_scheduler_data[six].esd.timer_service; -#ifdef ERTS_MAGIC_REF_BIF_TIMERS - ErtsTimerWheel *twheel = - erts_aligned_scheduler_data[six].esd.timer_wheel; - erts_twheel_debug_foreach(twheel, tw_bif_timer_timeout, - twt_debug_btm_foreach, - (void *) &btmfd); - time_rbt_foreach(srv->time_tree, - hlt_debug_btm_foreach, - (void *) &btmfd); -#else btm_rbt_foreach(srv->btm_tree, debug_btm_foreach, (void *) &btmfd); -#endif } } @@ -3403,9 +3016,7 @@ st_hdbg_func(ErtsHLTimer *tmr, void *vhdbg) } ERTS_HLT_ASSERT(tmr->time.tree.u.l.next->time.tree.u.l.prev == tmr); ERTS_HLT_ASSERT(tmr->time.tree.u.l.prev->time.tree.u.l.next == tmr); -#ifndef ERTS_MAGIC_REF_BIF_TIMERS ERTS_HLT_ASSERT(btm_rbt_lookup(hdbg->srv->btm_tree, ERTS_BTM_HLT2REFN(tmr)) == tmr); -#endif } static void @@ -3434,10 +3045,8 @@ tt_hdbg_func(ErtsHLTimer *tmr, void *vhdbg) & ~ERTS_HLT_PFLGS_MASK); ERTS_HLT_ASSERT(tmr == prnt); } -#ifndef ERTS_MAGIC_REF_BIF_TIMERS if (tmr->head.roflgs & ERTS_TMR_ROFLG_BIF_TMR) ERTS_HLT_ASSERT(btm_rbt_lookup(hdbg->srv->btm_tree, ERTS_BTM_HLT2REFN(tmr)) == tmr); -#endif if (tmr->time.tree.same_time) { ErtsHdbgHLT st_hdbg; st_hdbg.srv = hdbg->srv; @@ -3503,7 +3112,6 @@ hdbg_chk_srv(ErtsHLTimerService *srv) time_rbt_foreach(srv->time_tree, tt_hdbg_func, (void *) &hdbg); ERTS_HLT_ASSERT(hdbg.found_root); } -#ifndef ERTS_MAGIC_REF_BIF_TIMERS if (srv->btm_tree) { ErtsHdbgHLT hdbg; hdbg.srv = srv; @@ -3512,7 +3120,6 @@ hdbg_chk_srv(ErtsHLTimerService *srv) btm_rbt_foreach(srv->btm_tree, bt_hdbg_func, (void *) &hdbg); ERTS_HLT_ASSERT(hdbg.found_root); } -#endif } #endif /* ERTS_HLT_HARD_DEBUG */ diff --git a/erts/emulator/beam/erl_init.c b/erts/emulator/beam/erl_init.c index 99e788c718..163724ed3c 100644 --- a/erts/emulator/beam/erl_init.c +++ b/erts/emulator/beam/erl_init.c @@ -78,7 +78,7 @@ const char etp_erts_version[] = ERLANG_VERSION; const char etp_otp_release[] = ERLANG_OTP_RELEASE; const char etp_compile_date[] = ERLANG_COMPILE_DATE; const char etp_arch[] = ERLANG_ARCHITECTURE; -#ifdef ERTS_ENABLE_KERNEL_POLL +#if ERTS_ENABLE_KERNEL_POLL const int erts_use_kernel_poll = 1; const int etp_kernel_poll_support = 1; #else @@ -376,6 +376,27 @@ erl_init(int ncpu, } static Eterm +erl_spawn_system_process(Process* parent, Eterm mod, Eterm func, Eterm args, + ErlSpawnOpts *so) +{ + Eterm res; + int arity; + + ERTS_LC_ASSERT(ERTS_PROC_LOCK_MAIN & erts_proc_lc_my_proc_locks(parent)); + arity = erts_list_length(args); + + if (erts_find_function(mod, func, arity, erts_active_code_ix()) == NULL) { + erts_exit(ERTS_ERROR_EXIT, "No function %T:%T/%i\n", mod, func, arity); + } + + so->flags |= SPO_SYSTEM_PROC; + + res = erl_create_process(parent, mod, func, args, so); + + return res; +} + +static Eterm erl_first_process_otp(char* modname, void* code, unsigned size, int argc, char** argv) { int i; @@ -386,12 +407,6 @@ erl_first_process_otp(char* modname, void* code, unsigned size, int argc, char** Process parent; ErlSpawnOpts so; Eterm env; - - start_mod = erts_atom_put((byte *) modname, sys_strlen(modname), ERTS_ATOM_ENC_LATIN1, 1); - if (erts_find_function(start_mod, am_start, 2, - erts_active_code_ix()) == NULL) { - erts_exit(ERTS_ERROR_EXIT, "No function %s:start/2\n", modname); - } /* * We need a dummy parent process to be able to call erl_create_process(). @@ -399,6 +414,7 @@ erl_first_process_otp(char* modname, void* code, unsigned size, int argc, char** erts_init_empty_process(&parent); erts_proc_lock(&parent, ERTS_PROC_LOCK_MAIN); + hp = HAlloc(&parent, argc*2 + 4); args = NIL; for (i = argc-1; i >= 0; i--) { @@ -411,44 +427,73 @@ erl_first_process_otp(char* modname, void* code, unsigned size, int argc, char** hp += 2; args = CONS(hp, env, args); - so.flags = erts_default_spo_flags|SPO_SYSTEM_PROC; - res = erl_create_process(&parent, start_mod, am_start, args, &so); + start_mod = erts_atom_put((byte *) modname, sys_strlen(modname), + ERTS_ATOM_ENC_LATIN1, 1); + + so.flags = erts_default_spo_flags; + res = erl_spawn_system_process(&parent, start_mod, am_start, args, &so); + ASSERT(is_internal_pid(res)); + erts_proc_unlock(&parent, ERTS_PROC_LOCK_MAIN); erts_cleanup_empty_process(&parent); + return res; } static Eterm erl_system_process_otp(Eterm parent_pid, char* modname, int off_heap_msgq, int prio) -{ - Eterm start_mod; - Process* parent; +{ + Process *parent; ErlSpawnOpts so; - Eterm res; - - start_mod = erts_atom_put((byte *) modname, sys_strlen(modname), ERTS_ATOM_ENC_LATIN1, 1); - if (erts_find_function(start_mod, am_start, 0, - erts_active_code_ix()) == NULL) { - erts_exit(ERTS_ERROR_EXIT, "No function %s:start/0\n", modname); - } + Eterm mod, res; parent = erts_pid2proc(NULL, 0, parent_pid, ERTS_PROC_LOCK_MAIN); + mod = erts_atom_put((byte *) modname, sys_strlen(modname), + ERTS_ATOM_ENC_LATIN1, 1); + + so.flags = erts_default_spo_flags|SPO_USE_ARGS; - so.flags = erts_default_spo_flags|SPO_SYSTEM_PROC|SPO_USE_ARGS; - if (off_heap_msgq) + if (off_heap_msgq) { so.flags |= SPO_OFF_HEAP_MSGQ; - so.min_heap_size = H_MIN_SIZE; - so.min_vheap_size = BIN_VH_MIN_SIZE; - so.max_heap_size = H_MAX_SIZE; - so.max_heap_flags = H_MAX_FLAGS; - so.priority = prio; - so.max_gen_gcs = (Uint16) erts_atomic32_read_nob(&erts_max_gen_gcs); - so.scheduler = 0; - res = erl_create_process(parent, start_mod, am_start, NIL, &so); + } + + so.min_heap_size = H_MIN_SIZE; + so.min_vheap_size = BIN_VH_MIN_SIZE; + so.max_heap_size = H_MAX_SIZE; + so.max_heap_flags = H_MAX_FLAGS; + so.priority = prio; + so.max_gen_gcs = (Uint16) erts_atomic32_read_nob(&erts_max_gen_gcs); + so.scheduler = 0; + + res = erl_spawn_system_process(parent, mod, am_start, NIL, &so); + ASSERT(is_internal_pid(res)); + erts_proc_unlock(parent, ERTS_PROC_LOCK_MAIN); + return res; } +Eterm erts_internal_spawn_system_process_3(BIF_ALIST_3) { + Eterm mod, func, args, res; + ErlSpawnOpts so; + + mod = BIF_ARG_1; + func = BIF_ARG_2; + args = BIF_ARG_3; + + ASSERT(is_atom(mod)); + ASSERT(is_atom(func)); + ASSERT(erts_list_length(args) >= 0); + + so.flags = erts_default_spo_flags; + res = erl_spawn_system_process(BIF_P, mod, func, args, &so); + + if (is_non_value(res)) { + BIF_ERROR(BIF_P, so.error_code); + } + + BIF_RET(res); +} Eterm erts_preloaded(Process* p) @@ -684,6 +729,9 @@ void erts_usage(void) erts_fprintf(stderr, "-zebwt val set ets busy wait threshold, valid values are:\n"); erts_fprintf(stderr, " none|very_short|short|medium|long|very_long|extremely_long\n"); #endif + erts_fprintf(stderr, "-ztma bool enable/disable tuple module apply support in emulator\n"); + erts_fprintf(stderr, " (transitional flag for parameterized modules; recompile\n"); + erts_fprintf(stderr, " with +tuple_calls for compatibility with future versions)\n"); erts_fprintf(stderr, "\n"); erts_fprintf(stderr, "Note that if the emulator is started with erlexec (typically\n"); erts_fprintf(stderr, "from the erl script), these flags should be specified with +.\n"); @@ -2167,6 +2215,17 @@ erl_start(int argc, char **argv) erts_usage(); } } + else if (has_prefix("tma", sub_param)) { + arg = get_arg(sub_param+3, argv[i+1], &i); + if (sys_strcmp(arg,"true") == 0) { + tuple_module_apply = 1; + } else if (sys_strcmp(arg,"false") == 0) { + tuple_module_apply = 0; + } else { + erts_fprintf(stderr, "bad tuple module apply %s\n", arg); + erts_usage(); + } + } else { erts_fprintf(stderr, "bad -z option %s\n", argv[i]); erts_usage(); diff --git a/erts/emulator/beam/erl_map.c b/erts/emulator/beam/erl_map.c index cba17d3e6a..8f96dc3d23 100644 --- a/erts/emulator/beam/erl_map.c +++ b/erts/emulator/beam/erl_map.c @@ -475,7 +475,7 @@ Eterm erts_hashmap_from_array(ErtsHeapFactory* factory, Eterm *leafs, Uint n, Eterm erts_map_from_ks_and_vs(ErtsHeapFactory *factory, Eterm *ks0, Eterm *vs0, Uint n) { - if (n < MAP_SMALL_MAP_LIMIT) { + if (n <= MAP_SMALL_MAP_LIMIT) { Eterm *ks, *vs, *hp; flatmap_t *mp; Eterm keys; diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index ee6e6085b6..ebef485b04 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -2354,6 +2354,13 @@ static void dtor_demonitor(ErtsMonitor* mon, void* context) erts_proc_sig_send_demonitor(mon); } +#ifdef DEBUG +int erts_dbg_is_resource_dying(ErtsResource* resource) +{ + return resource->monitors && rmon_is_dying(resource->monitors); +} +#endif + # define NIF_RESOURCE_DTOR &nif_resource_dtor static int nif_resource_dtor(Binary* bin) @@ -2694,8 +2701,12 @@ int enif_consume_timeslice(ErlNifEnv* env, int percent) { Process *proc; Sint reds; + int sched; - execution_state(env, &proc, NULL); + execution_state(env, &proc, &sched); + + if (sched < 0) + return 0; /* no-op on dirty scheduler */ ASSERT(is_proc_bound(env) && percent >= 1 && percent <= 100); if (percent < 1) percent = 1; diff --git a/erts/emulator/beam/erl_port.h b/erts/emulator/beam/erl_port.h index 25976d38cc..039d8cf67a 100644 --- a/erts/emulator/beam/erl_port.h +++ b/erts/emulator/beam/erl_port.h @@ -1018,6 +1018,6 @@ int erts_port_output_async(Port *, Eterm, Eterm); /* * Signals from ports to ports. Used by sys drivers. */ -int erl_drv_port_control(Eterm, char, char*, ErlDrvSizeT); +int erl_drv_port_control(Eterm, unsigned int, char*, ErlDrvSizeT); #endif diff --git a/erts/emulator/beam/erl_printf_term.c b/erts/emulator/beam/erl_printf_term.c index 990a01b96f..9cfb7fc681 100644 --- a/erts/emulator/beam/erl_printf_term.c +++ b/erts/emulator/beam/erl_printf_term.c @@ -364,13 +364,13 @@ print_term(fmtfn_t fn, void* arg, Eterm obj, long *dcount) { int print_res; char def_buf[64]; char *buf, *big_str; - Uint sz = (Uint) big_decimal_estimate(wobj); + Uint sz = (Uint) big_integer_estimate(wobj, 10); sz++; if (sz <= 64) buf = &def_buf[0]; else buf = erts_alloc(ERTS_ALC_T_TMP, sz); - big_str = erts_big_to_string(wobj, buf, sz); + big_str = erts_big_to_string(wobj, 10, buf, sz); print_res = erts_printf_string(fn, arg, big_str); if (buf != &def_buf[0]) erts_free(ERTS_ALC_T_TMP, (void *) buf); diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index 2427d87f66..cc02fbad1e 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -92,10 +92,6 @@ #undef HARDDEBUG #endif -#ifdef HARDDEBUG -#define HARDDEBUG_RUNQS -#endif - #ifdef HIPE #include "hipe_mode_switch.h" /* for hipe_init_process() */ #include "hipe_signal.h" /* for hipe_thread_signal_init() */ @@ -344,6 +340,7 @@ erts_sched_stat_t erts_sched_stat; static erts_tsd_key_t ERTS_WRITE_UNLIKELY(sched_data_key); #if ERTS_POLL_USE_SCHEDULER_POLLING +static erts_atomic32_t function_calls; static erts_atomic32_t doing_sys_schedule; #endif static erts_atomic32_t no_empty_run_queues; @@ -2465,6 +2462,13 @@ handle_reap_ports(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting) erts_port_lock(prt); + if (prt->common.u.alive.reg && + prt->common.u.alive.reg->name == am_heart_port) { + /* Leave heart port to not get killed before flushing is done*/ + erts_port_release(prt); + continue; + } + state = erts_atomic32_read_nob(&prt->state); if (!(state & (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP | ERTS_PORT_SFLG_HALT))) { @@ -3251,6 +3255,7 @@ poll_thread(void *arg) static ERTS_INLINE void clear_sys_scheduling(void) { + erts_atomic32_set_relb(&function_calls, 0); erts_atomic32_set_mb(&doing_sys_schedule, 0); } @@ -3273,31 +3278,49 @@ prepare_for_sys_schedule(void) return 0; } -static void -check_io_timer(void *null) +#else +#define clear_sys_scheduling() +#define prepare_for_sys_schedule() 0 +#endif + +#ifdef HARDDEBUG +#define ERTS_HDBG_CHK_SLEEP_LIST(SL, L, F, FN) \ + check_sleepers_list((SL), (L), (F), (FN)) +static void check_sleepers_list(ErtsSchedulerSleepList *sl, + int lock, + ErtsSchedulerSleepInfo *find, + ErtsSchedulerSleepInfo *find_not) { - ErtsSchedulerData *esdp = erts_get_scheduler_data(); - if (prepare_for_sys_schedule()) { - erts_check_io(esdp->ssi->psi, ERTS_POLL_NO_TIMEOUT); - clear_sys_scheduling(); - } + ErtsSchedulerSleepInfo *last_out; + int found = 0; - /* The timer is cleared if this schedulers run-queue became empty - or if the CHECKIO flag was cleared. The CHECKIO flags is cleared - when a check_balance assigns another scheduler to be the poller in - the overload scenario. */ - if ((ERTS_RUNQ_FLGS_GET_NOB(esdp->run_queue) & (ERTS_RUNQ_FLG_OUT_OF_WORK|ERTS_RUNQ_FLG_CHECKIO)) - == ERTS_RUNQ_FLG_CHECKIO) { - erts_start_timer_callback(ERTS_POLL_SCHEDULER_POLLING_TIMEOUT, - check_io_timer, NULL); - } else { - ERTS_RUNQ_FLGS_UNSET(esdp->run_queue, ERTS_RUNQ_FLG_CHECKIO); - } -} + if (lock) + erts_spin_lock(&sl->lock); + ERTS_ASSERT(!find_not || (!find_not->next && !find_not->prev)); + + last_out = sl->list; + if (last_out) { + ErtsSchedulerSleepInfo *tmp = last_out; + do { + ERTS_ASSERT(tmp->next); + ERTS_ASSERT(tmp->prev); + ERTS_ASSERT(tmp->next->prev == tmp); + ERTS_ASSERT(tmp->prev->next == tmp); + ERTS_ASSERT(tmp != find_not); + if (tmp == find) + found = !0; + tmp = tmp->next; + + } while (tmp != last_out); + } + ERTS_ASSERT(!find || found); + + if (lock) + erts_spin_unlock(&sl->lock); +} #else -#define clear_sys_scheduling() -#define prepare_for_sys_schedule() 0 +#define ERTS_HDBG_CHK_SLEEP_LIST(SL, L, F, FN) ((void) 0) #endif static void @@ -3313,23 +3336,29 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) ERTS_LC_ASSERT(erts_lc_runq_is_locked(rq)); - if (ERTS_RUNQ_IX_IS_DIRTY(rq->ix)) - erts_spin_lock(&rq->sleepers.lock); flgs = sched_prep_spin_wait(ssi); if (flgs & ERTS_SSI_FLG_SUSPENDED) { /* Go suspend instead... */ - if (ERTS_RUNQ_IX_IS_DIRTY(rq->ix)) - erts_spin_unlock(&rq->sleepers.lock); return; } if (ERTS_RUNQ_IX_IS_DIRTY(rq->ix)) { - ssi->prev = NULL; - ssi->next = rq->sleepers.list; - if (rq->sleepers.list) - rq->sleepers.list->prev = ssi; - rq->sleepers.list = ssi; - erts_spin_unlock(&rq->sleepers.lock); + erts_spin_lock(&rq->sleepers.lock); + ERTS_HDBG_CHK_SLEEP_LIST(&rq->sleepers, 0, NULL, ssi); + ASSERT(!ssi->next); /* Not in sleepers list */ + ASSERT(!ssi->prev); + if (!rq->sleepers.list) { + ssi->next = ssi->prev = ssi; + rq->sleepers.list = ssi; + } + else { + ssi->prev = rq->sleepers.list; + ssi->next = rq->sleepers.list->next; + ssi->prev->next = ssi; + ssi->next->prev = ssi; + } + ERTS_HDBG_CHK_SLEEP_LIST(&rq->sleepers, 0, ssi, NULL); + erts_spin_unlock(&rq->sleepers.lock); dirty_active(esdp, -1); } @@ -3406,6 +3435,7 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) current_time = erts_get_monotonic_time(esdp); } } + *fcalls = 0; clear_sys_scheduling(); } else { if (!ERTS_SCHEDULER_IS_DIRTY(esdp)) { @@ -3470,8 +3500,28 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) (ERTS_SSI_FLG_SUSPENDED | ERTS_SSI_FLG_MSB_EXEC)); - if (ERTS_SCHEDULER_IS_DIRTY(esdp)) + if (ERTS_SCHEDULER_IS_DIRTY(esdp)) { dirty_sched_wall_time_change(esdp, working = 1); + erts_spin_lock(&rq->sleepers.lock); + ERTS_HDBG_CHK_SLEEP_LIST(&rq->sleepers, 0, ssi->next ? ssi : NULL, NULL); + if (ssi->next) { /* Still in list... */ + if (ssi->next == ssi) { + ASSERT(rq->sleepers.list == ssi); + ASSERT(ssi->prev == ssi); + rq->sleepers.list = NULL; + } + else { + ASSERT(ssi->prev != ssi); + if (rq->sleepers.list == ssi) + rq->sleepers.list = ssi->next; + ssi->prev->next = ssi->next; + ssi->next->prev = ssi->prev; + } + ssi->next = ssi->prev = NULL; + } + ERTS_HDBG_CHK_SLEEP_LIST(&rq->sleepers, 0, NULL, ssi); + erts_spin_unlock(&rq->sleepers.lock); + } else if (!thr_prgr_active) { erts_thr_progress_active(erts_thr_prgr_data(esdp), thr_prgr_active = 1); sched_wall_time_change(esdp, 1); @@ -3559,56 +3609,44 @@ wake_scheduler(ErtsRunQueue *rq) } static void -wake_dirty_schedulers(ErtsRunQueue *rq, int one) +wake_dirty_scheduler(ErtsRunQueue *rq) { - ErtsSchedulerSleepInfo *ssi; + ErtsSchedulerSleepInfo *lo_ssi, *fo_ssi; ErtsSchedulerSleepList *sl; ASSERT(ERTS_RUNQ_IX_IS_DIRTY(rq->ix)); sl = &rq->sleepers; erts_spin_lock(&sl->lock); - ssi = sl->list; - if (!ssi) { + ERTS_HDBG_CHK_SLEEP_LIST(&rq->sleepers, 0, NULL, NULL); + lo_ssi = sl->list; + if (!lo_ssi) { erts_spin_unlock(&sl->lock); - if (one) - wake_scheduler(rq); - } else if (one) { + wake_scheduler(rq); + } + else { erts_aint32_t flgs; - if (ssi->prev) - ssi->prev->next = ssi->next; - else { - ASSERT(sl->list == ssi); - sl->list = ssi->next; + fo_ssi = lo_ssi->next; + ASSERT(fo_ssi->prev == lo_ssi); + if (fo_ssi == lo_ssi) { + ASSERT(lo_ssi->prev == lo_ssi); + sl->list = NULL; + } + else { + ASSERT(lo_ssi->prev != lo_ssi); + lo_ssi->next = fo_ssi->next; + fo_ssi->next->prev = fo_ssi->prev; } - if (ssi->next) - ssi->next->prev = ssi->prev; - - erts_spin_unlock(&sl->lock); - - ERTS_THR_MEMORY_BARRIER; - flgs = ssi_flags_set_wake(ssi); - erts_sched_finish_poke(ssi, flgs); - } else { - sl->list = NULL; + fo_ssi->next = fo_ssi->prev = NULL; + ERTS_HDBG_CHK_SLEEP_LIST(&rq->sleepers, 0, NULL, fo_ssi); erts_spin_unlock(&sl->lock); ERTS_THR_MEMORY_BARRIER; - do { - ErtsSchedulerSleepInfo *wake_ssi = ssi; - ssi = ssi->next; - erts_sched_finish_poke(wake_ssi, ssi_flags_set_wake(wake_ssi)); - } while (ssi); + flgs = ssi_flags_set_wake(fo_ssi); + erts_sched_finish_poke(fo_ssi, flgs); } } -static void -wake_dirty_scheduler(ErtsRunQueue *rq) -{ - wake_dirty_schedulers(rq, 1); -} - - #define ERTS_NO_USED_RUNQS_SHIFT 16 #define ERTS_NO_RUNQS_MASK 0xffffU @@ -4661,15 +4699,6 @@ check_balance(ErtsRunQueue *c_rq) if (blnc_no_rqs == 1) { c_rq->check_balance_reds = INT_MAX; erts_atomic32_set_nob(&balance_info.checking_balance, 0); -#if ERTS_POLL_USE_SCHEDULER_POLLING - c_rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; - if ((ERTS_RUNQ_FLGS_GET_NOB(c_rq) & (ERTS_RUNQ_FLG_OUT_OF_WORK|ERTS_RUNQ_FLG_CHECKIO)) - == 0) { - ERTS_RUNQ_FLGS_SET(c_rq, ERTS_RUNQ_FLG_CHECKIO); - erts_start_timer_callback(ERTS_POLL_SCHEDULER_POLLING_TIMEOUT, check_io_timer, NULL); - } - ERTS_RUNQ_FLGS_UNSET(c_rq, ERTS_RUNQ_FLGS_MIGRATION_INFO); -#endif return; } @@ -5189,19 +5218,6 @@ erts_fprintf(stderr, "--------------------------------\n"); /* Publish new migration paths... */ erts_atomic_set_wb(&erts_migration_paths, (erts_aint_t) new_mpaths); -#if ERTS_POLL_USE_SCHEDULER_POLLING - if (full_scheds == current_active) { - ERTS_ASSERT(full_scheds <= current_active); - /* All active schedulers ran for full, we need to do active polling, - so we setup a timer that does active polling */ - if (!(ERTS_RUNQ_FLGS_GET_NOB(c_rq) & ERTS_RUNQ_FLG_CHECKIO)) { - /* Active polling is not running, start it */ - erts_start_timer_callback(ERTS_POLL_SCHEDULER_POLLING_TIMEOUT, check_io_timer, NULL); - } - run_queue_info[c_rq->ix].flags |= ERTS_RUNQ_FLG_CHECKIO; - } -#endif - /* Reset balance statistics in all online queues */ for (qix = 0; qix < blnc_no_rqs; qix++) { Uint32 flags = run_queue_info[qix].flags; @@ -5211,8 +5227,6 @@ erts_fprintf(stderr, "--------------------------------\n"); ASSERT(!(flags & ERTS_RUNQ_FLG_OUT_OF_WORK)); if (rq->waiting) flags |= ERTS_RUNQ_FLG_OUT_OF_WORK; - if (rq != c_rq) - flags &= ~ERTS_RUNQ_FLG_CHECKIO; rq->full_reds_history_sum = run_queue_info[qix].full_reds_history_sum; @@ -5222,7 +5236,7 @@ erts_fprintf(stderr, "--------------------------------\n"); ERTS_DBG_CHK_FULL_REDS_HISTORY(rq); rq->out_of_work_count = 0; - (void) ERTS_RUNQ_FLGS_READ_BSET(rq, ERTS_RUNQ_FLGS_MIGRATION_INFO|ERTS_RUNQ_FLG_CHECKIO, flags); + (void) ERTS_RUNQ_FLGS_READ_BSET(rq, ERTS_RUNQ_FLGS_MIGRATION_INFO, flags); rq->max_len = erts_atomic32_read_dirty(&rq->len); for (pix = 0; pix < ERTS_NO_PRIO_LEVELS; pix++) { ErtsRunQueueInfo *rqi; @@ -5873,6 +5887,7 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online, int no_poll_th erts_alloc_permanent_cache_aligned(ERTS_ALC_T_RUNQS, size_runqs); #if ERTS_POLL_USE_SCHEDULER_POLLING erts_atomic32_init_nob(&doing_sys_schedule, 0); + erts_atomic32_init_nob(&function_calls, 0); #endif erts_atomic32_init_nob(&no_empty_run_queues, 0); @@ -5988,6 +6003,8 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online, int no_poll_th for (ix = 0; ix < no_dirty_cpu_schedulers; ix++) { ErtsSchedulerSleepInfo *ssi = &aligned_dirty_cpu_sched_sleep_info[ix].ssi; erts_atomic32_init_nob(&ssi->flags, 0); + ssi->next = NULL; + ssi->prev = NULL; ssi->event = NULL; /* initialized in sched_dirty_cpu_thread_func */ erts_atomic32_init_nob(&ssi->aux_work, 0); } @@ -5998,6 +6015,8 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online, int no_poll_th for (ix = 0; ix < no_dirty_io_schedulers; ix++) { ErtsSchedulerSleepInfo *ssi = &aligned_dirty_io_sched_sleep_info[ix].ssi; erts_atomic32_init_nob(&ssi->flags, 0); + ssi->next = NULL; + ssi->prev = NULL; ssi->event = NULL; /* initialized in sched_dirty_io_thread_func */ erts_atomic32_init_nob(&ssi->aux_work, 0); } @@ -7517,7 +7536,13 @@ suspend_scheduler(ErtsSchedulerData *esdp) return; } +#ifdef HARDDEBUG + if (sched_type != ERTS_SCHED_NORMAL) + ERTS_HDBG_CHK_SLEEP_LIST(&esdp->run_queue->sleepers, !0, NULL, ssi); +#endif + if (erts_atomic32_read_nob(&ssi->flags) & ERTS_SSI_FLG_MSB_EXEC) { + ASSERT(no == 1); if (!msb_scheduler_type_switch(sched_type, esdp, no)) return; @@ -9217,7 +9242,7 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) Process *proxy_p = NULL; ErtsRunQueue *rq; int context_reds; - int fcalls; + int fcalls = 0; int actual_reds; int reds; Uint32 flags; @@ -9291,6 +9316,10 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) reds = ERTS_PROC_MIN_CONTEXT_SWITCH_REDS_COST; esdp->virtual_reds = 0; +#if ERTS_POLL_USE_SCHEDULER_POLLING + fcalls = (int) erts_atomic32_add_read_acqb(&function_calls, reds); +#endif + ASSERT(esdp && esdp == erts_get_scheduler_data()); rq = erts_get_runq_current(esdp); @@ -9507,7 +9536,33 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) non_empty_runq(rq); goto check_activities_to_run; - } + } else if (is_normal_sched && + fcalls > (2 * context_reds) && + prepare_for_sys_schedule()) { + ErtsMonotonicTime current_time; + /* + * Schedule system-level activities. + */ + + ERTS_MSACC_PUSH_STATE_CACHED_M(); + + erts_runq_unlock(rq); + + ERTS_MSACC_SET_STATE_CACHED_M(ERTS_MSACC_STATE_CHECK_IO); + LTTNG2(scheduler_poll, esdp->no, 1); + + erts_check_io(esdp->ssi->psi, ERTS_POLL_NO_TIMEOUT); + ERTS_MSACC_POP_STATE_M(); + + current_time = erts_get_monotonic_time(esdp); + if (current_time >= erts_next_timeout_time(esdp->next_tmo_ref)) + erts_bump_timers(esdp->timer_wheel, current_time); + + erts_runq_lock(rq); + fcalls = 0; + clear_sys_scheduling(); + goto continue_check_activities_to_run; + } if (flags & ERTS_RUNQ_FLG_MISC_OP) exec_misc_ops(rq); diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index a1b029adbe..43937f216c 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -173,8 +173,6 @@ extern int erts_dio_sched_thread_suggested_stack_size; (((Uint32) 1) << (ERTS_RUNQ_FLG_BASE2 + 9)) #define ERTS_RUNQ_FLG_HALTING \ (((Uint32) 1) << (ERTS_RUNQ_FLG_BASE2 + 10)) -#define ERTS_RUNQ_FLG_CHECKIO \ - (((Uint32) 1) << (ERTS_RUNQ_FLG_BASE2 + 11)) #define ERTS_RUNQ_FLG_MAX (ERTS_RUNQ_FLG_BASE2 + 12) @@ -359,7 +357,7 @@ typedef struct ErtsSchedulerSleepInfo_ ErtsSchedulerSleepInfo; typedef struct { erts_spinlock_t lock; - ErtsSchedulerSleepInfo *list; + ErtsSchedulerSleepInfo *list; /* circular lifo list; points to last out */ } ErtsSchedulerSleepList; struct ErtsSchedulerSleepInfo_ { diff --git a/erts/emulator/beam/erl_trace.c b/erts/emulator/beam/erl_trace.c index 2350d4c02f..ae7084b7f4 100644 --- a/erts/emulator/beam/erl_trace.c +++ b/erts/emulator/beam/erl_trace.c @@ -72,6 +72,7 @@ static ErtsTracer default_port_tracer; static Eterm system_monitor; static Eterm system_profile; +static erts_atomic_t system_logger; #ifdef HAVE_ERTS_NOW_CPU int erts_cpu_timestamp; @@ -340,6 +341,7 @@ void erts_init_trace(void) { default_port_trace_flags = F_INITIAL_TRACE_FLAGS; default_port_tracer = erts_tracer_nil; system_seq_tracer = erts_tracer_nil; + erts_atomic_init_nob(&system_logger, am_logger); init_sys_msg_dispatcher(); init_tracer_nif(); } @@ -2027,10 +2029,24 @@ enqueue_sys_msg(enum ErtsSysMsgType type, erts_mtx_unlock(&smq_mtx); } +Eterm +erts_get_system_logger(void) +{ + return (Eterm)erts_atomic_read_nob(&system_logger); +} + +Eterm +erts_set_system_logger(Eterm logger) +{ + if (logger != am_logger && logger != am_undefined && !is_internal_pid(logger)) + return THE_NON_VALUE; + return (Eterm)erts_atomic_xchg_nob(&system_logger, logger); +} + void erts_queue_error_logger_message(Eterm from, Eterm msg, ErlHeapFragment *bp) { - enqueue_sys_msg(SYS_MSG_TYPE_ERRLGR, from, am_logger, msg, bp); + enqueue_sys_msg(SYS_MSG_TYPE_ERRLGR, from, erts_get_system_logger(), msg, bp); } void @@ -2271,7 +2287,7 @@ sys_msg_dispatcher_func(void *unused) } break; case SYS_MSG_TYPE_ERRLGR: - receiver = am_logger; + receiver = smqp->to; break; default: receiver = NIL; @@ -2285,8 +2301,15 @@ sys_msg_dispatcher_func(void *unused) if (is_internal_pid(receiver)) { proc = erts_pid2proc(NULL, 0, receiver, proc_locks); if (!proc) { - /* Bad tracer */ - goto failure; + if (smqp->type == SYS_MSG_TYPE_ERRLGR) { + /* Bad logger process, send to kernel 'logger' process */ + erts_set_system_logger(am_logger); + receiver = erts_get_system_logger(); + goto logger; + } else { + /* Bad tracer */ + goto failure; + } } else { ErtsMessage *mp; @@ -2299,9 +2322,9 @@ sys_msg_dispatcher_func(void *unused) #endif erts_proc_unlock(proc, proc_locks); } - } - else if (receiver == am_logger) { - proc = erts_whereis_process(NULL,0,receiver,proc_locks,0); + } else if (receiver == am_logger) { + logger: + proc = erts_whereis_process(NULL,0,am_logger,proc_locks,0); if (!proc) goto failure; else if (smqp->from == proc->common.id) @@ -2309,7 +2332,10 @@ sys_msg_dispatcher_func(void *unused) else goto queue_proc_msg; } - else if (is_internal_port(receiver)) { + else if (receiver == am_undefined) { + goto drop_sys_msg; + } + else if (is_internal_port(receiver)) { port = erts_thr_id2port_sflgs(receiver, ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); if (!port) @@ -2366,7 +2392,7 @@ erts_foreach_sys_msg_in_q(void (*func)(Eterm, to = erts_get_system_profile(); break; case SYS_MSG_TYPE_ERRLGR: - to = am_logger; + to = erts_get_system_logger(); break; default: to = NIL; diff --git a/erts/emulator/beam/erl_trace.h b/erts/emulator/beam/erl_trace.h index bccf31606e..b7844d1cb0 100644 --- a/erts/emulator/beam/erl_trace.h +++ b/erts/emulator/beam/erl_trace.h @@ -94,6 +94,8 @@ void erts_foreach_sys_msg_in_q(void (*func)(Eterm, Eterm, Eterm, ErlHeapFragment *)); +Eterm erts_set_system_logger(Eterm); +Eterm erts_get_system_logger(void); void erts_queue_error_logger_message(Eterm, Eterm, ErlHeapFragment *); void erts_send_sys_msg_proc(Eterm, Eterm, Eterm, ErlHeapFragment *); diff --git a/erts/emulator/beam/erl_vm.h b/erts/emulator/beam/erl_vm.h index 4089fac48e..d37c2940c4 100644 --- a/erts/emulator/beam/erl_vm.h +++ b/erts/emulator/beam/erl_vm.h @@ -167,6 +167,8 @@ extern const int num_instructions; /* Number of instruction in opc[]. */ extern Uint erts_instr_count[]; +extern int tuple_module_apply; + /* some constants for various table sizes etc */ #define ATOM_TEXT_SIZE 32768 /* Increment for allocating atom text space */ diff --git a/erts/emulator/beam/external.c b/erts/emulator/beam/external.c index 9a66e491f3..1ded5f031c 100644 --- a/erts/emulator/beam/external.c +++ b/erts/emulator/beam/external.c @@ -102,7 +102,7 @@ static byte* enc_term(ErtsAtomCacheMap *, Eterm, byte*, Uint32, struct erl_off_h struct TTBEncodeContext_; static int enc_term_int(struct TTBEncodeContext_*,ErtsAtomCacheMap *acmp, Eterm obj, byte* ep, Uint32 dflags, struct erl_off_heap_header** off_heap, Sint *reds, byte **res); -static Uint is_external_string(Eterm obj, int* p_is_string); +static int is_external_string(Eterm obj, Uint* lenp); static byte* enc_atom(ErtsAtomCacheMap *, Eterm, byte*, Uint32); static byte* enc_pid(ErtsAtomCacheMap *, Eterm, byte*, Uint32); struct B2TContext_t; @@ -2481,11 +2481,21 @@ enc_term_int(TTBEncodeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, byte* ep, { Eterm* cons = list_val(obj); Eterm tl; + Uint len_cnt = WSTACK_POP(s); obj = CAR(cons); tl = CDR(cons); - WSTACK_PUSH2(s, (is_list(tl) ? ENC_ONE_CONS : ENC_TERM), - tl); + if (is_list(tl)) { + len_cnt++; + WSTACK_PUSH3(s, len_cnt, ENC_ONE_CONS, tl); + } + else { + byte* list_lenp = (byte*) WSTACK_POP(s); + ASSERT(list_lenp[-1] == LIST_EXT); + put_int32(len_cnt, list_lenp); + + WSTACK_PUSH2(s, ENC_TERM, tl); + } } break; case ENC_PATCH_FUN_SIZE: @@ -2689,10 +2699,7 @@ enc_term_int(TTBEncodeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, byte* ep, } case LIST_DEF: { - int is_str; - - i = is_external_string(obj, &is_str); - if (is_str) { + if (is_external_string(obj, &i)) { *ep++ = STRING_EXT; put_int16(i, ep); ep += 2; @@ -2701,9 +2708,12 @@ enc_term_int(TTBEncodeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, byte* ep, *ep++ = unsigned_val(CAR(cons)); obj = CDR(cons); } + r -= i; } else { + r -= i/2; *ep++ = LIST_EXT; - put_int32(i, ep); + /* Patch list length when we find end of list */ + WSTACK_PUSH2(s, (UWord)ep, 1); ep += 4; goto encode_one_cons; } @@ -2961,9 +2971,13 @@ enc_term_int(TTBEncodeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, byte* ep, return 0; } +/** @brief Is it a list of bytes not longer than MAX_STRING_LEN? + * @param lenp out: string length or number of list cells traversed + * @return true/false + */ static -Uint -is_external_string(Eterm list, int* p_is_string) +int +is_external_string(Eterm list, Uint* lenp) { Uint len = 0; @@ -2975,29 +2989,15 @@ is_external_string(Eterm list, int* p_is_string) Eterm* consp = list_val(list); Eterm hd = CAR(consp); - if (!is_byte(hd)) { - break; + if (!is_byte(hd) || ++len > MAX_STRING_LEN) { + *lenp = len; + return 0; } - len++; list = CDR(consp); } - /* - * If we have reached the end of the list, and we have - * not exceeded the maximum length of a string, this - * is a string. - */ - *p_is_string = is_nil(list) && len < MAX_STRING_LEN; - - /* - * Continue to calculate the length. - */ - while (is_list(list)) { - Eterm* consp = list_val(list); - len++; - list = CDR(consp); - } - return len; + *lenp = len; + return is_nil(list); } @@ -4075,8 +4075,8 @@ encode_size_struct_int(TTBSizeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, for (;;) { ASSERT(!is_header(obj)); - if (ctx && --r == 0) { - *reds = r; + if (ctx && --r <= 0) { + *reds = 0; ctx->obj = obj; ctx->result = result; WSTACK_SAVE(s, &ctx->wstack); @@ -4166,8 +4166,10 @@ encode_size_struct_int(TTBSizeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, result += (1 + encode_size_struct2(acmp, port_node_name(obj), dflags) + 4 + 1); break; - case LIST_DEF: - if ((m = is_string(obj)) && (m < MAX_STRING_LEN)) { + case LIST_DEF: { + int is_str = is_external_string(obj, &m); + r -= m/2; + if (is_str) { result += m + 2 + 1; } else { result += 5; @@ -4176,6 +4178,7 @@ encode_size_struct_int(TTBSizeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, continue; /* big loop */ } break; + } case TUPLE_DEF: { Eterm* ptr = tuple_val(obj); @@ -4317,7 +4320,7 @@ encode_size_struct_int(TTBSizeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, if (is_header(obj)) { switch (obj) { - case LIST_TAIL_OP: + case LIST_TAIL_OP: obj = (Eterm) WSTACK_POP(s); if (is_list(obj)) { Eterm* cons = list_val(obj); @@ -4343,7 +4346,7 @@ encode_size_struct_int(TTBSizeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, WSTACK_DESTROY(s); if (ctx) { ASSERT(ctx->wstack.wstart == NULL); - *reds = r; + *reds = r < 0 ? 0 : r; } *res = result; return 0; diff --git a/erts/emulator/beam/global.h b/erts/emulator/beam/global.h index 0631404599..f564472081 100644 --- a/erts/emulator/beam/global.h +++ b/erts/emulator/beam/global.h @@ -113,6 +113,9 @@ extern Eterm erts_bld_resource_ref(Eterm** hp, ErlOffHeap*, ErtsResource*); extern void erts_pre_nif(struct enif_environment_t*, Process*, struct erl_module_nif*, Process* tracee); extern void erts_post_nif(struct enif_environment_t* env); +#ifdef DEBUG +int erts_dbg_is_resource_dying(ErtsResource*); +#endif extern void erts_resource_stop(ErtsResource*, ErlNifEvent, int is_direct_call); void erts_fire_nif_monitor(ErtsMonitor *tmon); void erts_nif_demonitored(ErtsResource* resource); @@ -962,7 +965,7 @@ void init_break_handler(void); void erts_set_ignore_break(void); void erts_replace_intr(void); void process_info(fmtfn_t, void *); -void print_process_info(fmtfn_t, void *, Process*); +void print_process_info(fmtfn_t, void *, Process*, ErtsProcLocks); void info(fmtfn_t, void *); void loaded(fmtfn_t, void *); void erts_print_base64(fmtfn_t to, void *to_arg, byte* src, Uint size); @@ -1303,14 +1306,7 @@ Sint intlist_to_buf(Eterm, char*, Sint); /* most callers pass plain char*'s */ int erts_unicode_list_to_buf(Eterm list, byte *buf, Sint len, Sint* written); Sint erts_unicode_list_to_buf_len(Eterm list); -struct Sint_buf { -#if defined(ARCH_64) - char s[22]; -#else - char s[12]; -#endif -}; -char* Sint_to_buf(Sint, struct Sint_buf*); +int Sint_to_buf(Sint num, int base, char **buf_p, size_t buf_size); #define ERTS_IOLIST_STATE_INITER(C_P, OBJ) \ {(C_P), 0, 0, (OBJ), {NULL, NULL, NULL, ERTS_ALC_T_INVALID}, 0, 0} diff --git a/erts/emulator/beam/io.c b/erts/emulator/beam/io.c index 5325480901..7322239a73 100644 --- a/erts/emulator/beam/io.c +++ b/erts/emulator/beam/io.c @@ -4073,7 +4073,7 @@ done: * to the caller. */ int -erl_drv_port_control(Eterm port_num, char cmd, char* buff, ErlDrvSizeT size) +erl_drv_port_control(Eterm port_num, unsigned int cmd, char* buff, ErlDrvSizeT size) { ErtsProc2PortSigData *sigdp = erts_port_task_alloc_p2p_sig_data(); diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h index 869a575cb4..a69da4d762 100644 --- a/erts/emulator/beam/sys.h +++ b/erts/emulator/beam/sys.h @@ -1291,4 +1291,13 @@ erts_raw_env_next_char(byte *p, int encoding) #endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ +/* + * Magic numbers for our driver port_control callbacks. + * Kept them below 1<<27 to not inflict extra bignum garbage on 32-bit. + */ +#define ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER 0x018b0900U +#define ERTS_INET_DRV_CONTROL_MAGIC_NUMBER 0x03f1a300U +#define ERTS_SPAWN_DRV_CONTROL_MAGIC_NUMBER 0x04c76a00U +#define ERTS_FORKER_DRV_CONTROL_MAGIC_NUMBER 0x050a7800U + #endif diff --git a/erts/emulator/beam/utils.c b/erts/emulator/beam/utils.c index d81bd89a48..a231638b50 100644 --- a/erts/emulator/beam/utils.c +++ b/erts/emulator/beam/utils.c @@ -3681,30 +3681,47 @@ erts_unicode_list_to_buf_len(Eterm list) } } -/* -** Convert an integer to a byte list -** return pointer to converted stuff (need not to be at start of buf!) -*/ -char* Sint_to_buf(Sint n, struct Sint_buf *buf) +/* Prints an integer in the given base, returning the number of digits printed. + * + * (*buf) is a pointer to the buffer, and is set to the start of the string + * when returning. */ +int Sint_to_buf(Sint n, int base, char **buf, size_t buf_size) { - char* p = &buf->s[sizeof(buf->s)-1]; - int sign = 0; - - *p-- = '\0'; /* null terminate */ - if (n == 0) - *p-- = '0'; - else if (n < 0) { - sign = 1; - n = -n; + char *p = &(*buf)[buf_size - 1]; + int sign = 0, size = 0; + + ASSERT(base >= 2 && base <= 36); + + if (n == 0) { + *p-- = '0'; + size++; + } else if (n < 0) { + sign = 1; + n = -n; } while (n != 0) { - *p-- = (n % 10) + '0'; - n /= 10; + int digit = n % base; + + if (digit < 10) { + *p-- = '0' + digit; + } else { + *p-- = 'A' + (digit - 10); + } + + size++; + + n /= base; } - if (sign) - *p-- = '-'; - return p+1; + + if (sign) { + *p-- = '-'; + size++; + } + + *buf = p + 1; + + return size; } /* Build a list of integers in some safe memory area @@ -4771,58 +4788,3 @@ erts_ptr_id(void *ptr) return ptr; } -#ifdef DEBUG -/* - * Handy functions when using a debugger - don't use in the code! - */ - -void upp(byte *buf, size_t sz) -{ - bin_write(ERTS_PRINT_STDERR, NULL, buf, sz); -} - -void pat(Eterm atom) -{ - upp(atom_tab(atom_val(atom))->name, - atom_tab(atom_val(atom))->len); -} - - -void pinfo() -{ - process_info(ERTS_PRINT_STDOUT, NULL); -} - - -void pp(p) -Process *p; -{ - if(p) - print_process_info(ERTS_PRINT_STDERR, NULL, p); -} - -void ppi(Eterm pid) -{ - pp(erts_proc_lookup(pid)); -} - -void td(Eterm x) -{ - erts_fprintf(stderr, "%T\n", x); -} - -void -ps(Process* p, Eterm* stop) -{ - Eterm* sp = STACK_START(p) - 1; - - if (stop <= STACK_END(p)) { - stop = STACK_END(p) + 1; - } - - while(sp >= stop) { - erts_printf("%p: %.75T\n", sp, *sp); - sp--; - } -} -#endif diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index b44464d6da..a6a5c20266 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -9955,6 +9955,7 @@ static ErlDrvSSizeT tcp_inet_ctl(ErlDrvData e, unsigned int cmd, { tcp_descriptor* desc = (tcp_descriptor*)e; + cmd -= ERTS_INET_DRV_CONTROL_MAGIC_NUMBER; switch(cmd) { case INET_REQ_OPEN: { /* open socket and return internal index */ int domain; @@ -12184,6 +12185,7 @@ static ErlDrvSSizeT packet_inet_ctl(ErlDrvData e, unsigned int cmd, char* buf, int type = SOCK_DGRAM; int af = AF_INET; + cmd -= ERTS_INET_DRV_CONTROL_MAGIC_NUMBER; switch(cmd) { case INET_REQ_OPEN: /* open socket and return internal index */ DEBUGF(("packet_inet_ctl(%ld): OPEN\r\n", (long)desc->port)); @@ -12960,38 +12962,40 @@ make_noninheritable_handle(SOCKET s) static void fire_multi_timers(tcp_descriptor *desc, ErlDrvPort port, ErlDrvData data) { - ErlDrvTime next_timeout; - MultiTimerData *curr = desc->mtd; - if (!curr) { - ASSERT(0); - return; + ErlDrvTime next_timeout = 0; + if (!desc->mtd) { + ASSERT(0); + return; } #ifdef DEBUG { ErlDrvTime chk = erl_drv_monotonic_time(ERL_DRV_MSEC); - ASSERT(chk >= curr->when); + ASSERT(chk >= desc->mtd->when); } #endif do { - MultiTimerData *save = curr; + MultiTimerData save = *desc->mtd; - (*(save->timeout_function))(data,save->caller); + /* We first remove the timer so that the timeout_functions has + can call clean_multi_timers without breaking anything */ + if (desc->mtd_cache == NULL) { + desc->mtd_cache = desc->mtd; + } else { + FREE(desc->mtd); + } - curr = curr->next; + desc->mtd = save.next; + if (desc->mtd != NULL) + desc->mtd->prev = NULL; - if (desc->mtd_cache == NULL) - desc->mtd_cache = save; - else - FREE(save); + (*(save.timeout_function))(data,save.caller); - if (curr == NULL) { - desc->mtd = NULL; + if (desc->mtd == NULL) return; - } - curr->prev = NULL; - next_timeout = curr->when - erl_drv_monotonic_time(ERL_DRV_MSEC); + + next_timeout = desc->mtd->when - erl_drv_monotonic_time(ERL_DRV_MSEC); } while (next_timeout <= 0); - desc->mtd = curr; + driver_set_timer(port, (unsigned long) next_timeout); } diff --git a/erts/emulator/drivers/unix/ttsl_drv.c b/erts/emulator/drivers/unix/ttsl_drv.c index 28c6cc0f94..d2a524cb6c 100644 --- a/erts/emulator/drivers/unix/ttsl_drv.c +++ b/erts/emulator/drivers/unix/ttsl_drv.c @@ -394,6 +394,8 @@ static ErlDrvSSizeT ttysl_control(ErlDrvData drv_data, { char resbuff[2*sizeof(Uint32)]; ErlDrvSizeT res_size; + + command -= ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER; switch (command) { case CTRL_OP_GET_WINSIZE: { @@ -419,7 +421,7 @@ static ErlDrvSSizeT ttysl_control(ErlDrvData drv_data, } break; default: - return 0; + return -1; } if (rlen < res_size) { *rbuf = driver_alloc(res_size); diff --git a/erts/emulator/drivers/win32/ttsl_drv.c b/erts/emulator/drivers/win32/ttsl_drv.c index 99e7fb25a4..d19bfa3079 100644 --- a/erts/emulator/drivers/win32/ttsl_drv.c +++ b/erts/emulator/drivers/win32/ttsl_drv.c @@ -176,6 +176,8 @@ static ErlDrvSSizeT ttysl_control(ErlDrvData drv_data, { char resbuff[2*sizeof(Uint32)]; ErlDrvSizeT res_size; + + command -= ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER; switch (command) { case CTRL_OP_GET_WINSIZE: { @@ -201,7 +203,7 @@ static ErlDrvSSizeT ttysl_control(ErlDrvData drv_data, } break; default: - return 0; + return -1; } if (rlen < res_size) { *rbuf = driver_alloc(res_size); diff --git a/erts/emulator/nifs/common/prim_file_nif.c b/erts/emulator/nifs/common/prim_file_nif.c index 5fff55b467..a78ee62677 100644 --- a/erts/emulator/nifs/common/prim_file_nif.c +++ b/erts/emulator/nifs/common/prim_file_nif.c @@ -38,7 +38,6 @@ static void unload(ErlNifEnv *env, void* priv_data); static ErlNifResourceType *efile_resource_type; -static ERL_NIF_TERM am_erts_prim_file; static ERL_NIF_TERM am_close; static ERL_NIF_TERM am_ok; @@ -220,7 +219,6 @@ static int load(ErlNifEnv *env, void** priv_data, ERL_NIF_TERM prim_file_pid) ASSERT(!"bad pid passed to prim_file_nif"); } - am_erts_prim_file = enif_make_atom(env, "erts_prim_file"); am_close = enif_make_atom(env, "close"); am_ok = enif_make_atom(env, "ok"); diff --git a/erts/emulator/sys/common/erl_check_io.c b/erts/emulator/sys/common/erl_check_io.c index c681fa481f..c39cd01e1c 100644 --- a/erts/emulator/sys/common/erl_check_io.c +++ b/erts/emulator/sys/common/erl_check_io.c @@ -962,7 +962,7 @@ enif_select(ErlNifEnv* env, ErtsDrvSelectDataState *free_select = NULL; ErtsNifSelectDataState *free_nif = NULL; - ASSERT(!resource->monitors); + ASSERT(!erts_dbg_is_resource_dying(resource)); #ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS if (!grow_drv_ev_state(fd)) { @@ -2262,14 +2262,14 @@ erts_check_io_info(void *proc) #if ERTS_POLL_USE_FALLBACK erts_poll_info_flbk(get_fallback_pollset(), &piv[0]); - piv[0].poll_threads = 1; + piv[0].poll_threads = 0; piv[0].active_fds = 0; piv++; #endif #if ERTS_POLL_USE_SCHEDULER_POLLING erts_poll_info(get_scheduler_pollset(0), &piv[0]); - piv[0].poll_threads = 1; + piv[0].poll_threads = 0; piv[0].active_fds = 0; piv++; #endif diff --git a/erts/emulator/sys/common/erl_poll.c b/erts/emulator/sys/common/erl_poll.c index 51d50933ff..c71d23f58c 100644 --- a/erts/emulator/sys/common/erl_poll.c +++ b/erts/emulator/sys/common/erl_poll.c @@ -872,8 +872,8 @@ update_pollset(ErtsPollSet *ps, int fd, ErtsPollOp op, ErtsPollEvents events) } } -#if defined(EV_DISPATCH) && !defined(__OpenBSD__) - /* If we have EV_DISPATCH we use it, unless we are on OpenBSD as the +#if defined(EV_DISPATCH) && !(defined(__OpenBSD__) || defined(__NetBSD__)) + /* If we have EV_DISPATCH we use it, unless we are on OpenBSD/NetBSD as the behavior of EV_EOF seems to be edge triggered there and we need it to be level triggered. @@ -2326,6 +2326,7 @@ uint32_t epoll_events(int kp_fd, int fd) { /* For epoll we read the information about what is selected upon from the proc fs.*/ char fname[30]; + char s[256]; FILE *f; unsigned int pos, flags, mnt_id; int line = 0; @@ -2343,12 +2344,12 @@ uint32_t epoll_events(int kp_fd, int fd) } if (fscanf(f,"\nmnt_id:\t%x\n", &mnt_id)); line += 3; - while (!feof(f)) { + while (fgets(s, sizeof(s) / sizeof(*s), f)) { /* tfd: 10 events: 40000019 data: 180000000a */ int ev_fd; uint32_t events; uint64_t data; - if (fscanf(f,"tfd:%d events:%x data:%llx\n", &ev_fd, &events, + if (sscanf(s,"tfd:%d events:%x data:%llx", &ev_fd, &events, (unsigned long long*)&data) != 3) { fprintf(stderr,"failed to parse file %s on line %d, errno = %d\n", fname, line, @@ -2392,6 +2393,7 @@ ERTS_POLL_EXPORT(erts_poll_get_selected_events)(ErtsPollSet *ps, /* For epoll we read the information about what is selected upon from the proc fs.*/ char fname[30]; + char s[256]; FILE *f; unsigned int pos, flags, mnt_id; int line = 0; @@ -2410,12 +2412,12 @@ ERTS_POLL_EXPORT(erts_poll_get_selected_events)(ErtsPollSet *ps, } if (fscanf(f,"\nmnt_id:\t%x\n", &mnt_id)); line += 3; - while (!feof(f)) { + while (fgets(s, sizeof(s) / sizeof(*s), f)) { /* tfd: 10 events: 40000019 data: 180000000a */ int fd; uint32_t events; uint64_t data; - if (fscanf(f,"tfd:%d events:%x data:%llx\n", &fd, &events, + if (sscanf(s,"tfd:%d events:%x data:%llx", &fd, &events, (unsigned long long*)&data) != 3) { fprintf(stderr,"failed to parse file %s on line %d, errno = %d\n", fname, line, errno); diff --git a/erts/emulator/sys/unix/sys_drivers.c b/erts/emulator/sys/unix/sys_drivers.c index 816bdea9c5..042a091db1 100644 --- a/erts/emulator/sys/unix/sys_drivers.c +++ b/erts/emulator/sys/unix/sys_drivers.c @@ -732,7 +732,8 @@ static ErlDrvData spawn_start(ErlDrvPort port_num, char* name, proto->u.start.fds[1] = ifd[1]; proto->u.start.fds[2] = stderrfd; proto->u.start.port_id = opts->exit_status ? erts_drvport2id(port_num) : THE_NON_VALUE; - if (erl_drv_port_control(forker_port, 'S', (char*)proto, sizeof(*proto))) { + if (erl_drv_port_control(forker_port, ERTS_FORKER_DRV_CONTROL_MAGIC_NUMBER, + (char*)proto, sizeof(*proto))) { /* The forker port has been killed, we close both fd's which will make open_port throw an epipe error */ close(ofd[0]); @@ -759,6 +760,9 @@ static ErlDrvSSizeT spawn_control(ErlDrvData e, unsigned int cmd, char *buf, ErtsSysDriverData *dd = (ErtsSysDriverData*)e; ErtsSysForkerProto *proto = (ErtsSysForkerProto *)buf; + if (cmd != ERTS_SPAWN_DRV_CONTROL_MAGIC_NUMBER) + return -1; + ASSERT(len == sizeof(*proto)); ASSERT(proto->action == ErtsSysForkerProtoAction_SigChld); @@ -799,6 +803,8 @@ static ErlDrvSSizeT fd_control(ErlDrvData drv_data, { int fd = (int)(long)drv_data; char resbuff[2*sizeof(Uint32)]; + + command -= ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER; switch (command) { case FD_CTRL_OP_GET_WINSIZE: { @@ -810,7 +816,7 @@ static ErlDrvSSizeT fd_control(ErlDrvData drv_data, } break; default: - return 0; + return -1; } if (rlen < 2*sizeof(Uint32)) { *rbuf = driver_alloc(2*sizeof(Uint32)); @@ -998,9 +1004,9 @@ static void clear_fd_data(ErtsSysFdData *fdd) fdd->psz = 0; } -static void nbio_stop_fd(ErlDrvPort prt, ErtsSysFdData *fdd) +static void nbio_stop_fd(ErlDrvPort prt, ErtsSysFdData *fdd, int use) { - driver_select(prt, abs(fdd->fd), ERL_DRV_USE_NO_CALLBACK|DO_READ|DO_WRITE, 0); + driver_select(prt, abs(fdd->fd), use ? ERL_DRV_USE_NO_CALLBACK : 0|DO_READ|DO_WRITE, 0); clear_fd_data(fdd); SET_BLOCKING(abs(fdd->fd)); @@ -1020,11 +1026,11 @@ static void fd_stop(ErlDrvData ev) /* Does not close the fds */ if (dd->ifd) { sz += sizeof(ErtsSysFdData); - nbio_stop_fd(prt, dd->ifd); + nbio_stop_fd(prt, dd->ifd, 1); } if (dd->ofd && dd->ofd != dd->ifd) { sz += sizeof(ErtsSysFdData); - nbio_stop_fd(prt, dd->ofd); + nbio_stop_fd(prt, dd->ofd, 1); } erts_free(ERTS_ALC_T_DRV_TAB, dd); @@ -1070,12 +1076,12 @@ static void stop(ErlDrvData ev) ErlDrvPort prt = dd->port_num; if (dd->ifd) { - nbio_stop_fd(prt, dd->ifd); + nbio_stop_fd(prt, dd->ifd, 0); driver_select(prt, abs(dd->ifd->fd), ERL_DRV_USE, 0); /* close(ifd); */ } if (dd->ofd && dd->ofd != dd->ifd) { - nbio_stop_fd(prt, dd->ofd); + nbio_stop_fd(prt, dd->ofd, 0); driver_select(prt, abs(dd->ofd->fd), ERL_DRV_USE, 0); /* close(ofd); */ } @@ -1693,7 +1699,8 @@ static void forker_sigchld(Eterm port_id, int error) already used by the spawn_driver, we use control instead. Note that when using erl_drv_port_control it is an asynchronous control. */ - erl_drv_port_control(port_id, 'S', (char*)proto, sizeof(*proto)); + erl_drv_port_control(port_id, ERTS_SPAWN_DRV_CONTROL_MAGIC_NUMBER, + (char*)proto, sizeof(*proto)); } static void forker_ready_input(ErlDrvData e, ErlDrvEvent fd) @@ -1778,6 +1785,9 @@ static ErlDrvSSizeT forker_control(ErlDrvData e, unsigned int cmd, char *buf, ErlDrvPort port_num = (ErlDrvPort)e; int res; + if (cmd != ERTS_FORKER_DRV_CONTROL_MAGIC_NUMBER) + return -1; + if (first_call) { /* * Do driver_select here when schedulers and their pollsets have started. diff --git a/erts/emulator/test/driver_SUITE.erl b/erts/emulator/test/driver_SUITE.erl index bd62708aa7..1d2ae4fb51 100644 --- a/erts/emulator/test/driver_SUITE.erl +++ b/erts/emulator/test/driver_SUITE.erl @@ -1069,14 +1069,19 @@ get_stable_check_io_info(N) -> get_check_io_total(ChkIo) -> ct:log("ChkIo = ~p~n",[ChkIo]), {Fallback, Rest} = get_fallback(ChkIo), + OnlyPollThreads = [PS || PS <- Rest, not is_scheduler_pollset(PS)], add_fallback_infos(Fallback, - lists:foldl(fun(Pollset, Acc) -> - lists:zipwith(fun(A, B) -> - add_pollset_infos(A,B) - end, - Pollset, Acc) - end, - hd(Rest), tl(Rest))). + lists:foldl( + fun(Pollset, Acc) -> + lists:zipwith(fun(A, B) -> + add_pollset_infos(A,B) + end, + Pollset, Acc) + end, + hd(OnlyPollThreads), tl(OnlyPollThreads))). + +is_scheduler_pollset(Pollset) -> + proplists:get_value(poll_threads, Pollset) == 0. add_pollset_infos({Tag, A}=TA , {Tag, B}=TB) -> case tag_type(Tag) of diff --git a/erts/emulator/test/nif_SUITE.erl b/erts/emulator/test/nif_SUITE.erl index a2f3489943..ca5f90621f 100644 --- a/erts/emulator/test/nif_SUITE.erl +++ b/erts/emulator/test/nif_SUITE.erl @@ -1175,6 +1175,15 @@ maps(Config) when is_list(Config) -> M2 = maps_from_list_nif(maps:to_list(M2)), M3 = maps_from_list_nif(maps:to_list(M3)), + %% Test different map sizes (OTP-15567) + repeat_while(fun({35,_}) -> false; + ({K,Map}) -> + Map = maps_from_list_nif(maps:to_list(Map)), + Map = maps:filter(fun(K,V) -> V =:= K*100 end, Map), + {K+1, maps:put(K,K*100,Map)} + end, + {1,#{}}), + has_duplicate_keys = maps_from_list_nif([{1,1},{1,1}]), verify_tmpmem(TmpMem), @@ -2471,6 +2480,13 @@ repeat(0, _, Arg) -> repeat(N, Fun, Arg0) -> repeat(N-1, Fun, Fun(Arg0)). +repeat_while(Fun, Acc0) -> + case Fun(Acc0) of + false -> ok; + Acc1 -> + repeat_while(Fun, Acc1) + end. + check(Exp,Got,Line) -> case Got of Exp -> Exp; diff --git a/erts/emulator/test/num_bif_SUITE.erl b/erts/emulator/test/num_bif_SUITE.erl index f15217814a..6b834705cf 100644 --- a/erts/emulator/test/num_bif_SUITE.erl +++ b/erts/emulator/test/num_bif_SUITE.erl @@ -504,6 +504,10 @@ t_integer_to_string(Config) when is_list(Config) -> test_its("A", 10, 16), test_its("D4BE", 54462, 16), test_its("-D4BE", -54462, 16), + test_its("FFFFFFFFFF", 1099511627775, 16), + test_its("123456789ABCDEF123456789ABCDEF123456789ABCDEF", + 108977460683796539709587792812439445667270661579197935, + 16), lists:foreach(fun(Value) -> {'EXIT', {badarg, _}} = @@ -515,12 +519,14 @@ t_integer_to_string(Config) when is_list(Config) -> ok. test_its(List,Int) -> - Int = list_to_integer(List), - Int = binary_to_integer(list_to_binary(List)). + List = integer_to_list(Int), + Binary = list_to_binary(List), + Binary = integer_to_binary(Int). test_its(List,Int,Base) -> - Int = list_to_integer(List, Base), - Int = binary_to_integer(list_to_binary(List), Base). + List = integer_to_list(Int, Base), + Binary = list_to_binary(List), + Binary = integer_to_binary(Int, Base). %% Tests binary_to_integer/1. diff --git a/erts/emulator/test/persistent_term_SUITE.erl b/erts/emulator/test/persistent_term_SUITE.erl index 58cd3276b0..93eb026ced 100644 --- a/erts/emulator/test/persistent_term_SUITE.erl +++ b/erts/emulator/test/persistent_term_SUITE.erl @@ -6,7 +6,7 @@ %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. %% You may obtain a copy of the License at -%5 +%% %% http://www.apache.org/licenses/LICENSE-2.0 %% %% Unless required by applicable law or agreed to in writing, software @@ -21,7 +21,7 @@ -module(persistent_term_SUITE). -include_lib("common_test/include/ct.hrl"). --export([all/0,suite/0, +-export([all/0,suite/0,init_per_suite/1,end_per_suite/1, basic/1,purging/1,sharing/1,get_trapping/1, info/1,info_trapping/1,killed_while_trapping/1, off_heap_values/1,keys/1,collisions/1, @@ -39,39 +39,50 @@ all() -> killed_while_trapping,off_heap_values,keys,collisions, init_restart]. +init_per_suite(Config) -> + %% Put a term in the dict so that we know that the testcases handle + %% stray terms left by stdlib or other test suites. + persistent_term:put(init_per_suite, {?MODULE}), + Config. + +end_per_suite(Config) -> + persistent_term:erase(init_per_suite), + Config. + basic(_Config) -> Chk = chk(), N = 777, Seq = lists:seq(1, N), - par(2, N, Seq), - seq(3, Seq), - seq(3, Seq), %Same values. + par(2, N, Seq, Chk), + seq(3, Seq, Chk), + seq(3, Seq, Chk), %Same values. _ = [begin Key = {?MODULE,{key,I}}, true = persistent_term:erase(Key), false = persistent_term:erase(Key), - {'EXIT',{badarg,_}} = (catch persistent_term:get(Key)) + {'EXIT',{badarg,_}} = (catch persistent_term:get(Key)), + {not_present,Key} = persistent_term:get(Key, {not_present,Key}) end || I <- Seq], - [] = [P || {{?MODULE,_},_}=P <- persistent_term:get()], + [] = [P || {{?MODULE,_},_}=P <- pget(Chk)], chk(Chk). -par(C, N, Seq) -> +par(C, N, Seq, Chk) -> _ = [spawn_link(fun() -> ok = persistent_term:put({?MODULE,{key,I}}, {value,C*I}) end) || I <- Seq], - Result = wait(N), + Result = wait(N, Chk), _ = [begin Double = C*I, {{?MODULE,{key,I}},{value,Double}} = Res end || {I,Res} <- lists:zip(Seq, Result)], ok. -seq(C, Seq) -> +seq(C, Seq, Chk) -> _ = [ok = persistent_term:put({?MODULE,{key,I}}, {value,C*I}) || I <- Seq], - All = persistent_term:get(), - All = [P || {{?MODULE,_},_}=P <- persistent_term:get()], + All = pget(Chk), + All = [P || {{?MODULE,_},_}=P <- All], All = [{Key,persistent_term:get(Key)} || {Key,_} <- All], Result = lists:sort(All), _ = [begin @@ -80,15 +91,15 @@ seq(C, Seq) -> end || {I,Res} <- lists:zip(Seq, Result)], ok. -wait(N) -> - All = [P || {{?MODULE,_},_}=P <- persistent_term:get()], +wait(N, Chk) -> + All = [P || {{?MODULE,_},_}=P <- pget(Chk)], case length(All) of N -> All = [{Key,persistent_term:get(Key)} || {Key,_} <- All], lists:sort(All); _ -> receive after 10 -> ok end, - wait(N) + wait(N, Chk) end. %% Make sure that terms that have been erased are copied into all @@ -200,23 +211,23 @@ get_trapping(_Config) -> _ -> 1000 end, spawn_link(fun() -> get_trapping_create(N) end), - All = do_get_trapping(N, []), + All = do_get_trapping(N, [], Chk), N = get_trapping_check_result(lists:sort(All), 1), erlang:garbage_collect(), get_trapping_erase(N), chk(Chk). -do_get_trapping(N, Prev) -> - case persistent_term:get() of +do_get_trapping(N, Prev, Chk) -> + case pget(Chk) of Prev when length(Prev) >= N -> All = [P || {{?MODULE,{get_trapping,_}},_}=P <- Prev], case length(All) of N -> All; - _ -> do_get_trapping(N, Prev) + _ -> do_get_trapping(N, Prev, Chk) end; New -> receive after 1 -> ok end, - do_get_trapping(N, New) + do_get_trapping(N, New, Chk) end. get_trapping_create(0) -> @@ -331,25 +342,25 @@ info_trapping(_Config) -> _ -> 1000 end, spawn_link(fun() -> info_trapping_create(N) end), - All = do_info_trapping(N, 0), + All = do_info_trapping(N, 0, Chk), N = info_trapping_check_result(lists:sort(All), 1), erlang:garbage_collect(), info_trapping_erase(N), chk(Chk). -do_info_trapping(N, PrevMem) -> +do_info_trapping(N, PrevMem, Chk) -> case info_info() of - {N,Mem} -> + {M,Mem} when M >= N -> true = Mem >= PrevMem, - All = [P || {{?MODULE,{info_trapping,_}},_}=P <- persistent_term:get()], + All = [P || {{?MODULE,{info_trapping,_}},_}=P <- pget(Chk)], case length(All) of N -> All; - _ -> do_info_trapping(N, PrevMem) + _ -> do_info_trapping(N, PrevMem, Chk) end; {_,Mem} -> true = Mem >= PrevMem, receive after 1 -> ok end, - do_info_trapping(N, Mem) + do_info_trapping(N, Mem, Chk) end. info_trapping_create(0) -> @@ -462,17 +473,17 @@ collisions(_Config) -> _ = [V = persistent_term:get(K) || {K,V} <- Kvs], %% Now delete the persistent terms in random order. - collisions_delete(lists:keysort(2, Kvs)), + collisions_delete(lists:keysort(2, Kvs), Chk), chk(Chk). -collisions_delete([{Key,Val}|Kvs]) -> +collisions_delete([{Key,Val}|Kvs], Chk) -> Val = persistent_term:get(Key), true = persistent_term:erase(Key), - true = lists:sort(persistent_term:get()) =:= lists:sort(Kvs), + true = lists:sort(pget(Chk)) =:= lists:sort(Kvs), _ = [V = persistent_term:get(K) || {K,V} <- Kvs], - collisions_delete(Kvs); -collisions_delete([]) -> + collisions_delete(Kvs, Chk); +collisions_delete([], _) -> ok. colliding_keys() -> @@ -589,15 +600,16 @@ do_test_init_restart_cmd(File) -> %% and after each test case. chk() -> - persistent_term:info(). + {persistent_term:info(), persistent_term:get()}. -chk(Chk) -> - Chk = persistent_term:info(), +chk({Info, _Initial} = Chk) -> + Info = persistent_term:info(), Key = {?MODULE,?FUNCTION_NAME}, - ok = persistent_term:put(Key, {term,Chk}), + ok = persistent_term:put(Key, {term,Info}), Term = persistent_term:get(Key), true = persistent_term:erase(Key), chk_not_stuck(Term), + [persistent_term:erase(K) || {K, _} <- pget(Chk)], ok. chk_not_stuck(Term) -> @@ -612,3 +624,6 @@ chk_not_stuck(Term) -> _ -> ok end. + +pget({_, Initial}) -> + persistent_term:get() -- Initial. diff --git a/erts/emulator/test/process_SUITE.erl b/erts/emulator/test/process_SUITE.erl index 57eb082d64..b23f77a0b2 100644 --- a/erts/emulator/test/process_SUITE.erl +++ b/erts/emulator/test/process_SUITE.erl @@ -2104,6 +2104,13 @@ spawn_opt_max_heap_size(_Config) -> error_logger:add_report_handler(?MODULE, self()), + %% flush any prior messages in error_logger + Pid = spawn(fun() -> ok = nok end), + receive + {error, _, {emulator, _, [Pid|_]}} -> + flush() + end, + %% Test that numerical limit works max_heap_size_test(1024, 1024, true, true), @@ -2208,6 +2215,13 @@ receive_unexpected() -> ok end. +flush() -> + receive + _M -> flush() + after 0 -> + ok + end. + %% error_logger report handler proxy init(Pid) -> {ok, Pid}. diff --git a/erts/emulator/test/scheduler_SUITE.erl b/erts/emulator/test/scheduler_SUITE.erl index 2e0dfa42f3..f61949c75b 100644 --- a/erts/emulator/test/scheduler_SUITE.erl +++ b/erts/emulator/test/scheduler_SUITE.erl @@ -1450,28 +1450,26 @@ poll_threads(Config) when is_list(Config) -> {Conc, PollType, KP} = get_ioconfig(Config), {Sched, SchedOnln, _} = get_sstate(Config, ""), + [1, 1] = get_ionum(Config,"+IOt 2 +IOp 2"), + [1, 1, 1, 1, 1] = get_ionum(Config,"+IOt 5 +IOp 5"), + [1, 1] = get_ionum(Config, "+S 2 +IOPt 100 +IOPp 100"), + if Conc -> - [1, 1, 1] = get_ionum(Config,"+IOt 2 +IOp 2"), - [1, 1, 1, 1, 1, 1] = get_ionum(Config,"+IOt 5 +IOp 5"), - [1, 1, 1] = get_ionum(Config, "+S 2 +IOPt 100 +IOPp 100"), - [5, 1] = get_ionum(Config,"+IOt 5 +IOp 1"), - [3, 2, 1] = get_ionum(Config,"+IOt 5 +IOp 2"), - [2, 2, 2, 2, 2, 1] = get_ionum(Config,"+IOt 10 +IOPp 50"), + [5] = get_ionum(Config,"+IOt 5 +IOp 1"), + [3, 2] = get_ionum(Config,"+IOt 5 +IOp 2"), + [2, 2, 2, 2, 2] = get_ionum(Config,"+IOt 10 +IOPp 50"), - [2, 1] = get_ionum(Config, "+S 2 +IOPt 100"), - [4, 1] = get_ionum(Config, "+S 4 +IOPt 100"), - [4, 1] = get_ionum(Config, "+S 4:2 +IOPt 100"), - [4, 4, 1] = get_ionum(Config, "+S 8 +IOPt 100 +IOPp 25"), + [2] = get_ionum(Config, "+S 2 +IOPt 100"), + [4] = get_ionum(Config, "+S 4 +IOPt 100"), + [4] = get_ionum(Config, "+S 4:2 +IOPt 100"), + [4, 4] = get_ionum(Config, "+S 8 +IOPt 100 +IOPp 25"), fail = get_ionum(Config, "+IOt 1 +IOp 2"), ok; not Conc -> - [1, 1] = get_ionum(Config,"+IOt 2 +IOp 2"), - [1, 1, 1, 1, 1] = get_ionum(Config,"+IOt 5 +IOp 5"), - [1, 1] = get_ionum(Config, "+S 2 +IOPt 100 +IOPp 100"), [1, 1, 1, 1, 1] = get_ionum(Config,"+IOt 5 +IOp 1"), [1, 1, 1, 1, 1] = get_ionum(Config,"+IOt 5 +IOp 2"), @@ -1515,7 +1513,8 @@ get_iostate(Config, Cmd)-> erlang:system_info(check_io) end]), IO = [IOState || IOState <- IOStates, - proplists:get_value(fallback, IOState) == false], + proplists:get_value(fallback, IOState) == false, + proplists:get_value(poll_threads, IOState) /= 0], stop_node(Node), IO; {error,timeout} -> diff --git a/erts/emulator/test/system_info_SUITE.erl b/erts/emulator/test/system_info_SUITE.erl index 21ab6b378a..4e663fed7f 100644 --- a/erts/emulator/test/system_info_SUITE.erl +++ b/erts/emulator/test/system_info_SUITE.erl @@ -37,8 +37,9 @@ -export([process_count/1, system_version/1, misc_smoke_tests/1, heap_size/1, wordsize/1, memory/1, ets_limit/1, atom_limit/1, - ets_count/1, - atom_count/1]). + ets_count/1, atom_count/1, system_logger/1]). + +-export([init/1, handle_event/2, handle_call/2]). suite() -> [{ct_hooks,[ts_install_cth]}, @@ -46,8 +47,8 @@ suite() -> all() -> [process_count, system_version, misc_smoke_tests, - ets_count, - heap_size, wordsize, memory, ets_limit, atom_limit, atom_count]. + ets_count, heap_size, wordsize, memory, ets_limit, atom_limit, atom_count, + system_logger]. %%% %%% The test cases ------------------------------------------------------------- @@ -573,3 +574,78 @@ atom_count(Config) when is_list(Config) -> true = Limit >= Count2, true = Count2 > Count1, ok. + + +system_logger(Config) when is_list(Config) -> + + TC = self(), + + ok = error_logger:add_report_handler(?MODULE, [TC]), + + generate_log_event(), + + flush(1, report_handler), + + Initial = erlang:system_info(system_logger), + + {Logger,_} = spawn_monitor(fun F() -> receive M -> TC ! {system_logger,M}, F() end end), + + Initial = erlang:system_flag(system_logger, Logger), + Logger = erlang:system_info(system_logger), + + generate_log_event(), + flush(1, system_logger), + + Logger = erlang:system_flag(system_logger, Logger), + + generate_log_event(), + flush(1, system_logger), + + exit(Logger, die), + receive {'DOWN',_,_,_,_} -> ok end, + + generate_log_event(), + flush(1, report_handler), + + logger = erlang:system_info(system_logger), + + logger = erlang:system_flag(system_logger, undefined), + generate_log_event(), + flush(), + + undefined = erlang:system_flag(system_logger, Initial), + + ok. + +flush() -> + receive + M -> + ct:fail({unexpected_message, M}) + after 0 -> + ok + end. + +flush(0, _Pat) -> + flush(); +flush(Cnt, Pat) -> + receive + M when element(1,M) =:= Pat -> + ct:log("~p",[M]), + flush(Cnt-1, Pat) + after 500 -> + ct:fail({missing, Cnt, Pat}) + end. + +generate_log_event() -> + {_Pid, Ref} = spawn_monitor(fun() -> ok = nok end), + receive {'DOWN', Ref, _, _, _} -> ok end. + +init([To]) -> + {ok, To}. + +handle_call(Msg, State) -> + {ok, Msg, State}. + +handle_event(Event, State) -> + State ! {report_handler, Event}, + {ok, State}. diff --git a/erts/emulator/test/z_SUITE.erl b/erts/emulator/test/z_SUITE.erl index 1c52e1a934..6549108126 100644 --- a/erts/emulator/test/z_SUITE.erl +++ b/erts/emulator/test/z_SUITE.erl @@ -251,7 +251,7 @@ pollset_size(Config) when is_list(Config) -> end. check_io_debug(Config) when is_list(Config) -> - case lists:keysearch(name, 1, erlang:system_info(check_io)) of + case lists:keysearch(name, 1, hd(erlang:system_info(check_io))) of {value, {name, erts_poll}} -> check_io_debug_test(); _ -> {skipped, "Not implemented in this emulator"} end. diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c index 0cb01fd4ef..23bbb86333 100644 --- a/erts/etc/common/erlexec.c +++ b/erts/etc/common/erlexec.c @@ -174,6 +174,7 @@ static char *plusz_val_switches[] = { "dbbl", "dntgc", "ebwt", + "tma", NULL }; diff --git a/erts/etc/common/heart.c b/erts/etc/common/heart.c index bd218ff725..bb843a616b 100644 --- a/erts/etc/common/heart.c +++ b/erts/etc/common/heart.c @@ -500,7 +500,7 @@ message_loop(erlin_fd, erlout_fd) #if defined(__WIN32__) static void -kill_old_erlang(void){ +kill_old_erlang(int reason){ HANDLE erlh; DWORD exit_code; char* envvar = NULL; @@ -536,7 +536,8 @@ kill_old_erlang(void){ } #else static void -kill_old_erlang(void){ +kill_old_erlang(int reason) +{ pid_t pid; int i, res; int sig = SIGKILL; @@ -546,14 +547,25 @@ kill_old_erlang(void){ if (envvar && strcmp(envvar, "TRUE") == 0) return; - envvar = get_env(HEART_KILL_SIGNAL); - if (envvar && strcmp(envvar, "SIGABRT") == 0) { - print_error("kill signal SIGABRT requested"); - sig = SIGABRT; - } - if(heart_beat_kill_pid != 0){ - pid = (pid_t) heart_beat_kill_pid; + pid = (pid_t) heart_beat_kill_pid; + if (reason == R_CLOSED) { + print_error("Wait 5 seconds for Erlang to terminate nicely"); + for (i=0; i < 5; ++i) { + res = kill(pid, 0); /* check if alive */ + if (res < 0 && errno == ESRCH) + return; + sleep(1); + } + print_error("Erlang still alive, kill it"); + } + + envvar = get_env(HEART_KILL_SIGNAL); + if (envvar && strcmp(envvar, "SIGABRT") == 0) { + print_error("kill signal SIGABRT requested"); + sig = SIGABRT; + } + res = kill(pid,sig); for(i=0; i < 5 && res == 0; ++i){ sleep(1); @@ -677,7 +689,7 @@ do_terminate(int erlin_fd, int reason) { if(!command) print_error("Would reboot. Terminating."); else { - kill_old_erlang(); + kill_old_erlang(reason); /* High prio combined with system() works badly indeed... */ SetPriorityClass(GetCurrentProcess(), NORMAL_PRIORITY_CLASS); win_system(command); @@ -685,7 +697,7 @@ do_terminate(int erlin_fd, int reason) { } free_env_val(command); } else { - kill_old_erlang(); + kill_old_erlang(reason); /* High prio combined with system() works badly indeed... */ SetPriorityClass(GetCurrentProcess(), NORMAL_PRIORITY_CLASS); win_system(&cmd[0]); @@ -697,13 +709,13 @@ do_terminate(int erlin_fd, int reason) { if(!command) print_error("Would reboot. Terminating."); else { - kill_old_erlang(); + kill_old_erlang(reason); ret = system(command); print_error("Executed \"%s\" -> %d. Terminating.",command, ret); } free_env_val(command); } else { - kill_old_erlang(); + kill_old_erlang(reason); ret = system((char*)&cmd[0]); print_error("Executed \"%s\" -> %d. Terminating.",cmd, ret); } diff --git a/erts/etc/unix/Makefile b/erts/etc/unix/Makefile index 83c64d35fd..21a725cb88 100644 --- a/erts/etc/unix/Makefile +++ b/erts/etc/unix/Makefile @@ -30,7 +30,8 @@ opt debug lcnt: etc etc: etp-commands etp-commands: etp-commands.in - $(gen_verbose)sed 's:@ERL_TOP@:${ERL_TOP}:g' etp-commands.in > etp-commands + $(gen_verbose)sed -e 's:@ERL_TOP@:${ERL_TOP}:g' \ + etp-commands.in > etp-commands .PHONY: docs docs: diff --git a/erts/etc/unix/cerl.src b/erts/etc/unix/cerl.src index 2e034513b0..bcd64d242e 100644 --- a/erts/etc/unix/cerl.src +++ b/erts/etc/unix/cerl.src @@ -224,7 +224,13 @@ while [ $# -gt 0 ]; do shift cargs="$cargs -rr" run_rr=yes - skip_erlexec=yes + case "$1" in + "replay"|"ps") + ;; + *) + skip_erlexec=yes + ;; + esac ;; *) break @@ -307,7 +313,26 @@ if [ "x$GDB" = "x" ]; then exec $taskset1 valgrind $valgrind_xml $valgrind_log $valgrind_misc_flags $BINDIR/$EMU_NAME $sched_arg $emu_xargs "$@" elif [ $run_rr = yes ]; then - exec rr record --ignore-nested $BINDIR/$EMU_NAME $emu_xargs "$@" + if [ $1 = replay ]; then + shift + cmdfile="/tmp/.cerlgdb.$$" + echo "set \$etp_beam_executable = \"$BINDIR/$EMU_NAME\"" > $cmdfile + if [ "$1" = "-p" ]; then + echo 'set $etp_rr_run_until_beam = 1' >> $cmdfile + fi + cat $ROOTDIR/erts/etc/unix/etp-commands.in >> $cmdfile + exec rr replay -x $cmdfile $* + elif [ $1 = ps ]; then + shift + rr ps $* | head -1 + ChildSetup=`rr ps $* | grep 'erl_child_setup' | awk '{ print $2 }'` + for CS in $ChildSetup; do + rr ps $* | grep -E "^$CS" + done + exit 0 + else + exec rr record --ignore-nested $BINDIR/$EMU_NAME $emu_xargs "$@" + fi else exec $EXEC $xargs ${1+"$@"} fi diff --git a/erts/etc/unix/etp-commands.in b/erts/etc/unix/etp-commands.in index b12a205ba7..bb7b1a73f5 100644 --- a/erts/etc/unix/etp-commands.in +++ b/erts/etc/unix/etp-commands.in @@ -149,7 +149,7 @@ define etp-1 else # (($arg0) & 0x3) == 0 if (($arg0) == etp_the_non_value) - printf "<the non-value>" + printf "<the-non-value>" else etp-cp-1 ($arg0) end @@ -1241,7 +1241,7 @@ define etp-sig-int if $etp_sig_tag != etp_the_non_value etp-1 $etp_sig_tag 0 else - print "!ENCODED-DIST-MSG" + printf "!ENCODED-DIST-MSG" end if ($arg0)->m[1] != $etp_nil printf " @token= " @@ -1251,7 +1251,7 @@ define etp-sig-int etp-1 ($arg0)->m[2] 0 else if ($etp_sig_tag & 0x3f) != 0x30 - print "!INVALID-SIGNAL" + printf "!INVALID-SIGNAL" else set $etp_sig_op = (($etp_sig_tag >> 6) & 0xff) set $etp_sig_type = (($etp_sig_tag >> 14) & 0xff) @@ -4326,6 +4326,20 @@ document etp-show %--------------------------------------------------------------------------- end +define etp-rr-run-until-beam + source @ERL_TOP@/erts/etc/unix/etp-rr-run-until-beam.py +end + +document etp-rr-run-until-beam +%--------------------------------------------------------------------------- +% etp-rr-run-until-beam +% +% Use this gdb macro to make cerl -rr replay -p PID walk until +% the correct execute has been made. You may have to change the +% file that is used to debug with. +%--------------------------------------------------------------------------- +end + ############################################################################ # Init # @@ -4363,7 +4377,13 @@ define hook-run set $_exitsignal = -1 end +handle SIGPIPE nostop + etp-init help etp-init -etp-show -etp-system-info +if $etp_rr_run_until_beam + help etp-rr-run-until-beam +else + etp-show + etp-system-info +end diff --git a/erts/etc/unix/etp-rr-run-until-beam.py b/erts/etc/unix/etp-rr-run-until-beam.py new file mode 100644 index 0000000000..078998b910 --- /dev/null +++ b/erts/etc/unix/etp-rr-run-until-beam.py @@ -0,0 +1,45 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2013-2016. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# %CopyrightEnd% +# + +has_exited = False + +def stop_handler (event): + global has_exited + if isinstance(event, gdb.SignalEvent): + print("exit code: %s" % (event.stop_signal)) + has_exited = True + +gdb.events.stop.connect (stop_handler) + +gdb.execute('continue') + +while not has_exited: + r = gdb.execute('when', to_string=True) + m = re.match("[^0-9]*([0-9]+)", r) + if m: + event = int(m.group(1)); + gdb.execute('start ' + str(event + 1)); + gdb.execute('continue') + +gdb.events.stop.disconnect (stop_handler) + +gdb.execute('file ' + str(gdb.parse_and_eval("$etp_beam_executable"))) +gdb.execute('break main') +gdb.execute('reverse-continue') diff --git a/erts/preloaded/ebin/erl_prim_loader.beam b/erts/preloaded/ebin/erl_prim_loader.beam Binary files differindex eee9ad3ca8..7cb34c2d58 100644 --- a/erts/preloaded/ebin/erl_prim_loader.beam +++ b/erts/preloaded/ebin/erl_prim_loader.beam diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex 498d437616..ee96878ff7 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/ebin/erts_internal.beam b/erts/preloaded/ebin/erts_internal.beam Binary files differindex 0bd24c3466..bc53a2e431 100644 --- a/erts/preloaded/ebin/erts_internal.beam +++ b/erts/preloaded/ebin/erts_internal.beam diff --git a/erts/preloaded/ebin/init.beam b/erts/preloaded/ebin/init.beam Binary files differindex 3feed8a31a..7d848ba790 100644 --- a/erts/preloaded/ebin/init.beam +++ b/erts/preloaded/ebin/init.beam diff --git a/erts/preloaded/ebin/persistent_term.beam b/erts/preloaded/ebin/persistent_term.beam Binary files differindex e94ef983be..c882e4fad4 100644 --- a/erts/preloaded/ebin/persistent_term.beam +++ b/erts/preloaded/ebin/persistent_term.beam diff --git a/erts/preloaded/ebin/prim_file.beam b/erts/preloaded/ebin/prim_file.beam Binary files differindex 1e72f70c65..ffb349cdf3 100644 --- a/erts/preloaded/ebin/prim_file.beam +++ b/erts/preloaded/ebin/prim_file.beam diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex 990f57bf0a..d3614d5f16 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/src/erl_prim_loader.erl b/erts/preloaded/src/erl_prim_loader.erl index ae5f86e017..fefdd34292 100644 --- a/erts/preloaded/src/erl_prim_loader.erl +++ b/erts/preloaded/src/erl_prim_loader.erl @@ -302,7 +302,7 @@ check_file_result(Func, Target, {error,Reason}) -> logger ! {log,error,#{label=>{?MODULE,file_error},report=>Report}, #{pid=>self(), gl=>group_leader(), - time=>erlang:monotonic_time(microsecond), + time=>erlang:system_time(microsecond), error_logger=>#{tag=>error_report, type=>std_error}}}, error diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index f37a3fc5db..5730e999cb 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -2526,6 +2526,9 @@ subtract(_,_) -> OldSchedulersOnline when SchedulersOnline :: pos_integer(), OldSchedulersOnline :: pos_integer(); + (system_logger, Logger) -> PrevLogger when + Logger :: logger | undefined | pid(), + PrevLogger :: logger | undefined | pid(); (trace_control_word, TCW) -> OldTCW when TCW :: non_neg_integer(), OldTCW :: non_neg_integer(); @@ -2731,8 +2734,9 @@ tuple_to_list(_Tuple) -> (schedulers | schedulers_online) -> pos_integer(); (smp_support) -> boolean(); (start_time) -> integer(); - (system_version) -> string(); (system_architecture) -> string(); + (system_logger) -> logger | undefined | pid(); + (system_version) -> string(); (threads) -> boolean(); (thread_pool_size) -> non_neg_integer(); (time_correction) -> true | false; @@ -3395,60 +3399,15 @@ get_cookie() -> -spec integer_to_list(Integer, Base) -> string() when Integer :: integer(), Base :: 2..36. -integer_to_list(I, 10) -> - erlang:integer_to_list(I); -integer_to_list(I, Base) - when erlang:is_integer(I), erlang:is_integer(Base), - Base >= 2, Base =< 1+$Z-$A+10 -> - if I < 0 -> - [$-|integer_to_list(-I, Base, [])]; - true -> - integer_to_list(I, Base, []) - end; -integer_to_list(I, Base) -> - erlang:error(badarg, [I, Base]). - -integer_to_list(I0, Base, R0) -> - D = I0 rem Base, - I1 = I0 div Base, - R1 = if D >= 10 -> - [D-10+$A|R0]; - true -> - [D+$0|R0] - end, - if I1 =:= 0 -> - R1; - true -> - integer_to_list(I1, Base, R1) - end. +integer_to_list(_I, _Base) -> + erlang:nif_error(undefined). -spec integer_to_binary(Integer, Base) -> binary() when Integer :: integer(), Base :: 2..36. -integer_to_binary(I, 10) -> - erlang:integer_to_binary(I); -integer_to_binary(I, Base) - when erlang:is_integer(I), erlang:is_integer(Base), - Base >= 2, Base =< 1+$Z-$A+10 -> - if I < 0 -> - <<$-,(integer_to_binary(-I, Base, <<>>))/binary>>; - true -> - integer_to_binary(I, Base, <<>>) - end; -integer_to_binary(I, Base) -> - erlang:error(badarg, [I, Base]). - -integer_to_binary(I0, Base, R0) -> - D = I0 rem Base, - I1 = I0 div Base, - R1 = if - D >= 10 -> <<(D-10+$A),R0/binary>>; - true -> <<(D+$0),R0/binary>> - end, - if - I1 =:= 0 -> R1; - true -> integer_to_binary(I1, Base, R1) - end. +integer_to_binary(_I, _Base) -> + erlang:nif_error(undefined). + -record(cpu, {node = -1, processor = -1, diff --git a/erts/preloaded/src/erts_internal.erl b/erts/preloaded/src/erts_internal.erl index 8f29a569f2..305b524438 100644 --- a/erts/preloaded/src/erts_internal.erl +++ b/erts/preloaded/src/erts_internal.erl @@ -97,6 +97,8 @@ -export([counters_new/1, counters_get/2, counters_add/3, counters_put/3, counters_info/1]). +-export([spawn_system_process/3]). + %% %% Await result of send to port %% @@ -726,3 +728,10 @@ counters_put(_Ref, _Ix, _Value) -> -spec counters_info(reference()) -> #{}. counters_info(_Ref) -> erlang:nif_error(undef). + +-spec spawn_system_process(Mod, Func, Args) -> pid() when + Mod :: atom(), + Func :: atom(), + Args :: list(). +spawn_system_process(_Mod, _Func, _Args) -> + erlang:nif_error(undefined). diff --git a/erts/preloaded/src/init.erl b/erts/preloaded/src/init.erl index b4b8b3bf9b..bdcdf72c2f 100644 --- a/erts/preloaded/src/init.erl +++ b/erts/preloaded/src/init.erl @@ -485,7 +485,7 @@ do_handle_msg(Msg,State) -> X -> case whereis(user) of undefined -> - Time = erlang:monotonic_time(microsecond), + Time = erlang:system_time(microsecond), catch logger ! {log, info, "init got unexpected: ~p", [X], #{pid=>self(), gl=>self(), diff --git a/erts/preloaded/src/persistent_term.erl b/erts/preloaded/src/persistent_term.erl index 5d0c266127..ee7e49b6cb 100644 --- a/erts/preloaded/src/persistent_term.erl +++ b/erts/preloaded/src/persistent_term.erl @@ -19,7 +19,7 @@ %% -module(persistent_term). --export([erase/1,get/0,get/1,info/0,put/2]). +-export([erase/1,get/0,get/1,get/2,info/0,put/2]). -type key() :: term(). -type value() :: term(). @@ -41,6 +41,13 @@ get() -> get(_Key) -> erlang:nif_error(undef). +-spec get(Key, Default) -> Value when + Key :: key(), + Default :: value(), + Value :: value(). +get(_Key, _Default) -> + erlang:nif_error(undef). + -spec info() -> Info when Info :: #{'count':=Count,'memory':=Memory}, Count :: non_neg_integer(), diff --git a/erts/preloaded/src/prim_file.erl b/erts/preloaded/src/prim_file.erl index 5fc22bc582..0994e2a9f4 100644 --- a/erts/preloaded/src/prim_file.erl +++ b/erts/preloaded/src/prim_file.erl @@ -46,7 +46,7 @@ -define(MIN_READLINE_SIZE, 256). -define(LARGEFILESIZE, (1 bsl 63)). --export([copy/3]). +-export([copy/3, start/0]). -include("file_int.hrl"). @@ -85,14 +85,21 @@ is_translatable(_) -> %% This is a janitor process used to close files whose controlling process has %% died. The emulator will be torn down if this is killed. -delayed_close_loop() -> +start() -> + helper_loop(). + +helper_loop() -> receive {close, FRef} when is_reference(FRef) -> delayed_close_nif(FRef); _ -> ok end, - delayed_close_loop(). + helper_loop(). -%% +on_load() -> + %% This is spawned as a system process to prevent init:restart/0 from + %% killing it. + Pid = erts_internal:spawn_system_process(?MODULE, start, []), + ok = erlang:load_nif(atom_to_list(?MODULE), Pid). %% Returns {error, Reason} | {ok, BytesCopied} copy(#file_descriptor{module = ?MODULE} = Source, @@ -103,14 +110,6 @@ copy(#file_descriptor{module = ?MODULE} = Source, %% XXX Should be moved down to the driver for optimization. file:copy_opened(Source, Dest, Length). -on_load() -> - Pid = spawn(fun() -> - process_flag(trap_exit, true), - delayed_close_loop() - end), - true = register(erts_prim_file, Pid), - ok = erlang:load_nif(atom_to_list(?MODULE), Pid). - open(Name, Modes) -> %% The try/catch pattern seen here is used throughout the file to adhere to %% the public file interface, which has leaked through for ages because of diff --git a/erts/preloaded/src/prim_inet.erl b/erts/preloaded/src/prim_inet.erl index f1d938c9a4..2820a5bef4 100644 --- a/erts/preloaded/src/prim_inet.erl +++ b/erts/preloaded/src/prim_inet.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2018. All Rights Reserved. +%% Copyright Ericsson AB 2000-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -172,8 +172,18 @@ close(S) when is_port(S) -> %% and is a contradiction in itself. %% We have hereby done our best... %% - Tref = erlang:start_timer(T * 1000, self(), close_port), - close_pend_loop(S, Tref, undefined); + case subscribe(S, [subs_empty_out_q]) of + {ok, [{subs_empty_out_q,0}]} -> + close_port(S); + {ok, [{subs_empty_out_q,N}]} when N > 0 -> + %% Wait for pending output to be sent + Tref = erlang:start_timer(T * 1000, self(), close_port), + close_pend_loop(S, Tref, N); + _ -> + %% Subscribe failed - wait full time + Tref = erlang:start_timer(T * 1000, self(), close_port), + close_pend_loop(S, Tref, undefined) + end; _ -> % Regard this as {ok,{false,_}} case subscribe(S, [subs_empty_out_q]) of {ok, [{subs_empty_out_q,N}]} when N > 0 -> @@ -2679,12 +2689,13 @@ get_ip6([X1,X2,X3,X4,X5,X6,X7,X8,X9,X10,X11,X12,X13,X14,X15,X16 | T]) -> ?u16(X9,X10),?u16(X11,X12),?u16(X13,X14),?u16(X15,X16)}, T }. +-define(ERTS_INET_DRV_CONTROL_MAGIC_NUMBER, 16#03f1a300). %% Control command ctl_cmd(Port, Cmd, Args) -> ?DBG_FORMAT("prim_inet:ctl_cmd(~p, ~p, ~p)~n", [Port,Cmd,Args]), Result = - try erlang:port_control(Port, Cmd, Args) of + try erlang:port_control(Port, Cmd+?ERTS_INET_DRV_CONTROL_MAGIC_NUMBER, Args) of [?INET_REP_OK|Reply] -> {ok,Reply}; [?INET_REP] -> inet_reply; [?INET_REP_ERROR|Err] -> {error,list_to_atom(Err)} diff --git a/erts/vsn.mk b/erts/vsn.mk index 73199cc721..e4bdb1a8eb 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -18,7 +18,7 @@ # %CopyrightEnd% # -VSN = 10.2.1 +VSN = 10.2.4 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/common_test/doc/src/ct_netconfc.xml b/lib/common_test/doc/src/ct_netconfc.xml index 32a1175d81..8fbe5f3df6 100644 --- a/lib/common_test/doc/src/ct_netconfc.xml +++ b/lib/common_test/doc/src/ct_netconfc.xml @@ -412,11 +412,11 @@ </func> <func> - <name since="OTP 18.3">create_subscription(Client) -> Result</name> - <name since="OTP 18.3">create_subscription(Client, Stream) -> Result</name> - <name since="OTP 18.3">create_subscription(Client, Stream, Filter) -> Result</name> - <name since="OTP 18.3">create_subscription(Client, Stream, Filter, Timeout) -> Result</name> - <name name="create_subscription" arity="5" clause_i="2" since="OTP 18.3"/> + <name since="OTP R15B02">create_subscription(Client) -> Result</name> + <name since="OTP R15B02">create_subscription(Client, Stream) -> Result</name> + <name since="OTP R15B02">create_subscription(Client, Stream, Filter) -> Result</name> + <name since="OTP R15B02">create_subscription(Client, Stream, Filter, Timeout) -> Result</name> + <name name="create_subscription" arity="5" clause_i="2" since="OTP R15B02"/> <name name="create_subscription" arity="6" since="OTP R15B02"/> <fsummary>Creates a subscription for event notifications.</fsummary> <desc> @@ -515,7 +515,7 @@ create_subscription(Client, Stream, Filter, StartTime, StopTime, Timeout)</pre> <func> <name name="edit_config" arity="3" since="OTP R15B02"/> - <name name="edit_config" arity="4" clause_i="1" since="OTP R15B02"/> + <name name="edit_config" arity="4" clause_i="1" since="OTP 18.0"/> <name name="edit_config" arity="4" clause_i="2" since="OTP R15B02"/> <name name="edit_config" arity="5" since="OTP 18.0"/> <fsummary>Edits configuration data.</fsummary> @@ -599,7 +599,7 @@ create_subscription(Client, Stream, Filter, StartTime, StopTime, Timeout)</pre> <func> <name name="get_event_streams" arity="1" since="OTP 20.0"/> <name name="get_event_streams" arity="2" clause_i="1" since="OTP R15B02"/> - <name name="get_event_streams" arity="2" clause_i="2" since="OTP R15B02"/> + <name name="get_event_streams" arity="2" clause_i="2" since="OTP 20.0"/> <name name="get_event_streams" arity="3" since="OTP R15B02"/> <fsummary>Sends a request to get the specified event streams.</fsummary> <desc> diff --git a/lib/common_test/doc/src/ct_telnet.xml b/lib/common_test/doc/src/ct_telnet.xml index 9a12ce79ed..76f5305c46 100644 --- a/lib/common_test/doc/src/ct_telnet.xml +++ b/lib/common_test/doc/src/ct_telnet.xml @@ -239,18 +239,21 @@ <v>Connection = connection()</v> <v>Cmd = string()</v> <v>Opts = [Opt]</v> - <v>Opt = {timeout, timeout()} | {newline, boolean()}</v> + <v>Opt = {timeout, timeout()} | {newline, boolean() | string()}</v> <v>Data = [string()]</v> <v>Reason = term()</v> </type> <desc><marker id="cmd-3"/> <p>Sends a command through Telnet and waits for prompt.</p> - <p>By default, this function adds a new line to the end of the + <p>By default, this function adds "\n" to the end of the specified command. If this is not desired, use option <c>{newline,false}</c>. This is necessary, for example, when sending Telnet command sequences prefixed with character - Interprete As Command (IAC).</p> + Interpret As Command (IAC). Option <c>{newline,string()}</c> + can also be used if a different line end than "\n" is + required, for instance <c>{newline,"\r\n"}</c>, to add both + carriage return and newline characters.</p> <p>Option <c>timeout</c> specifies how long the client must wait for prompt. If the time expires, the function returns @@ -280,7 +283,7 @@ <v>CmdFormat = string()</v> <v>Args = list()</v> <v>Opts = [Opt]</v> - <v>Opt = {timeout, timeout()} | {newline, boolean()}</v> + <v>Opt = {timeout, timeout()} | {newline, boolean() | string()}</v> <v>Data = [string()]</v> <v>Reason = term()</v> </type> @@ -339,7 +342,7 @@ subexpression number <c>N</c>. Subexpressions are denoted with <c>'(' ')'</c> in the regular expression.</p> - <p>If a <c>Tag</c> is speciifed, the returned <c>Match</c> also + <p>If a <c>Tag</c> is specified, the returned <c>Match</c> also includes the matched <c>Tag</c>. Otherwise, only <c>RxMatch</c> is returned.</p> @@ -382,7 +385,7 @@ can abort the operation of waiting for prompt.</p></item> <tag><c>repeat | repeat, N</c></tag> <item><p>The pattern(s) must be matched multiple times. If <c>N</c> - is speciified, the pattern(s) are matched <c>N</c> times, and + is specified, the pattern(s) are matched <c>N</c> times, and the function returns <c>HaltReason = done</c>. This option can be interrupted by one or more <c>HaltPatterns</c>. <c>MatchList</c> is always returned, that is, a list of <c>Match</c> instead of @@ -547,17 +550,20 @@ <v>Connection = connection()</v> <v>Cmd = string()</v> <v>Opts = [Opt]</v> - <v>Opt = {newline, boolean()}</v> + <v>Opt = {newline, boolean() | string()}</v> <v>Reason = term()</v> </type> <desc><marker id="send-3"/> <p>Sends a Telnet command and returns immediately.</p> - <p>By default, this function adds a newline to the end of the + <p>By default, this function adds "\n" to the end of the specified command. If this is not desired, option <c>{newline,false}</c> can be used. This is necessary, for example, when sending Telnet command sequences prefixed with character - Interprete As Command (IAC).</p> + Interpret As Command (IAC). Option <c>{newline,string()}</c> + can also be used if a different line end than "\n" is + required, for instance <c>{newline,"\r\n"}</c>, to add both + carriage return and newline characters.</p> <p>The resulting output from the command can be read with <seealso marker="#get_data-1"><c>ct_telnet:get_data/2</c></seealso> or @@ -584,12 +590,15 @@ <v>CmdFormat = string()</v> <v>Args = list()</v> <v>Opts = [Opt]</v> - <v>Opt = {newline, boolean()}</v> + <v>Opt = {newline, boolean() | string()}</v> <v>Reason = term()</v> </type> <desc><marker id="sendf-4"/> <p>Sends a Telnet command and returns immediately (uses a format string and a list of arguments to build the command).</p> + + <p>For details, see + <seealso marker="#send-3"><c>ct_telnet:send/3</c></seealso>.</p> </desc> </func> </funcs> diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index 118dcd88bd..38fdc2442e 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -75,6 +75,44 @@ </section> +<section><title>Common_Test 1.15.4.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The status of a test case which failed with timetrap + timeout in <c>end_per_testcase</c> could not be modified + by returning <c>{fail,Reason}</c> from a + <c>post_end_per_testcase</c> hook function. This is now + corrected.</p> + <p> + Own Id: OTP-15584 Aux Id: ERIERL-282 </p> + </item> + </list> + </section> + +</section> + +<section><title>Common_Test 1.15.4.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The status of a test case which failed with timetrap + timeout in <c>end_per_testcase</c> could not be modified + by returning <c>{fail,Reason}</c> from a + <c>post_end_per_testcase</c> hook function. This is now + corrected.</p> + <p> + Own Id: OTP-15584 Aux Id: ERIERL-282 </p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.15.4</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -4026,8 +4064,3 @@ <section><title>common_test 1.3.0</title> </section> </chapter> - - - - - diff --git a/lib/common_test/src/ct_config.erl b/lib/common_test/src/ct_config.erl index a10d939919..a07e61199b 100644 --- a/lib/common_test/src/ct_config.erl +++ b/lib/common_test/src/ct_config.erl @@ -592,7 +592,7 @@ encrypt_config_file(SrcFileName, EncryptFileName, {file,KeyFile}) -> encrypt_config_file(SrcFileName, EncryptFileName, {key,Key}) -> _ = crypto:start(), - {Key,IVec} = make_crypto_key(Key), + {CryptoKey,IVec} = make_crypto_key(Key), case file:read_file(SrcFileName) of {ok,Bin0} -> Bin1 = term_to_binary({SrcFileName,Bin0}), @@ -600,7 +600,7 @@ encrypt_config_file(SrcFileName, EncryptFileName, {key,Key}) -> 0 -> Bin1; N -> list_to_binary([Bin1,random_bytes(8-N)]) end, - EncBin = crypto:block_encrypt(des3_cbc, Key, IVec, Bin2), + EncBin = crypto:block_encrypt(des3_cbc, CryptoKey, IVec, Bin2), case file:write_file(EncryptFileName, EncBin) of ok -> io:format("~ts --(encrypt)--> ~ts~n", @@ -631,10 +631,10 @@ decrypt_config_file(EncryptFileName, TargetFileName, {file,KeyFile}) -> decrypt_config_file(EncryptFileName, TargetFileName, {key,Key}) -> _ = crypto:start(), - {Key,IVec} = make_crypto_key(Key), + {CryptoKey,IVec} = make_crypto_key(Key), case file:read_file(EncryptFileName) of {ok,Bin} -> - DecBin = crypto:block_decrypt(des3_cbc, Key, IVec, Bin), + DecBin = crypto:block_decrypt(des3_cbc, CryptoKey, IVec, Bin), case catch binary_to_term(DecBin) of {'EXIT',_} -> {error,bad_file}; diff --git a/lib/common_test/src/ct_netconfc.erl b/lib/common_test/src/ct_netconfc.erl index 29188a648e..6a758c4ea3 100644 --- a/lib/common_test/src/ct_netconfc.erl +++ b/lib/common_test/src/ct_netconfc.erl @@ -583,7 +583,7 @@ get_config(Client, Source, Filter, Timeout) -> -spec edit_config(Client, Target, Config) -> Result when Client :: client(), Target :: netconf_db(), - Config :: simple_xml(), + Config :: simple_xml() | [simple_xml()], Result :: ok | {error,error_reason()}. edit_config(Client, Target, Config) -> edit_config(Client, Target, Config, ?DEFAULT_TIMEOUT). @@ -591,7 +591,7 @@ edit_config(Client, Target, Config) -> -spec edit_config(Client, Target, Config, OptParams) -> Result when Client :: client(), Target :: netconf_db(), - Config :: simple_xml(), + Config :: simple_xml() | [simple_xml()], OptParams :: [simple_xml()], Result :: ok | {error,error_reason()}; (Client, Target, Config, Timeout) -> Result when @@ -608,10 +608,12 @@ edit_config(Client, Target, Config, OptParams) when is_list(OptParams) -> -spec edit_config(Client, Target, Config, OptParams, Timeout) -> Result when Client :: client(), Target :: netconf_db(), - Config :: simple_xml(), + Config :: simple_xml() | [simple_xml()], OptParams :: [simple_xml()], Timeout :: timeout(), Result :: ok | {error,error_reason()}. +edit_config(Client, Target, Config, OptParams, Timeout) when not is_list(Config)-> + edit_config(Client, Target, [Config], OptParams, Timeout); edit_config(Client, Target, Config, OptParams, Timeout) -> call(Client, {send_rpc_op, edit_config, [Target,Config,OptParams], Timeout}). @@ -1113,7 +1115,7 @@ encode_rpc_operation(get,[Filter]) -> encode_rpc_operation(get_config,[Source,Filter]) -> {'get-config',[{source,[Source]}] ++ filter(Filter)}; encode_rpc_operation(edit_config,[Target,Config,OptParams]) -> - {'edit-config',[{target,[Target]}] ++ OptParams ++ [{config,[Config]}]}; + {'edit-config',[{target,[Target]}] ++ OptParams ++ [{config,Config}]}; encode_rpc_operation(delete_config,[Target]) -> {'delete-config',[{target,[Target]}]}; encode_rpc_operation(copy_config,[Target,Source]) -> diff --git a/lib/common_test/src/ct_telnet.erl b/lib/common_test/src/ct_telnet.erl index f9abecfd38..174008c790 100644 --- a/lib/common_test/src/ct_telnet.erl +++ b/lib/common_test/src/ct_telnet.erl @@ -194,6 +194,15 @@ send(Connection,Cmd,Opts) -> check_send_opts([{newline,Bool}|Opts]) when is_boolean(Bool) -> check_send_opts(Opts); +check_send_opts([{newline,String}|Opts]) when is_list(String) -> + case lists:all(fun(I) when is_integer(I), I>=0, I=<127 -> true; + (_) -> false + end, String) of + true -> + check_send_opts(Opts); + false -> + {error,{invalid_option,{newline,String}}} + end; check_send_opts([Invalid|_]) -> {error,{invalid_option,Invalid}}; check_send_opts([]) -> @@ -211,10 +220,16 @@ expect(Connection,Patterns) -> expect(Connection,Patterns,Opts) -> case get_handle(Connection) of - {ok,Pid} -> - call(Pid,{expect,Patterns,Opts}); - Error -> - Error + {ok,Pid} -> + case call(Pid,{expect,Patterns,Opts}) of + {error,Reason} when element(1,Reason)==bad_pattern -> + %% Faulty user input - should fail the test case + exit({Reason,{?MODULE,?FUNCTION_NAME,3}}); + Other -> + Other + end; + Error -> + Error end. %%%================================================================= @@ -674,60 +689,68 @@ silent_teln_expect(Name,Pid,Data,Pattern,Prx,Opts) -> %% 3b) Repeat (sequence): 2) is repeated either N times or until a %% halt condition is fulfilled. teln_expect(Name,Pid,Data,Pattern0,Prx,Opts) -> - HaltPatterns = + HaltPatterns0 = case get_ignore_prompt(Opts) of true -> get_haltpatterns(Opts); false -> [prompt | get_haltpatterns(Opts)] end, - - PromptCheck = get_prompt_check(Opts), - - {WaitForPrompt,Pattern1,Opts1} = wait_for_prompt(Pattern0,Opts), - - Seq = get_seq(Opts1), - Pattern2 = convert_pattern(Pattern1,Seq), - {IdleTimeout,TotalTimeout} = get_timeouts(Opts1), - - EO = #eo{teln_pid=Pid, - prx=Prx, - idle_timeout=IdleTimeout, - total_timeout=TotalTimeout, - seq=Seq, - haltpatterns=HaltPatterns, - prompt_check=PromptCheck}, + case convert_pattern(HaltPatterns0,false) of + {ok,HaltPatterns} -> + {WaitForPrompt,Pattern1,Opts1} = wait_for_prompt(Pattern0,Opts), + Seq = get_seq(Opts1), + case convert_pattern(Pattern1,Seq) of + {ok,Pattern2} -> + {IdleTimeout,TotalTimeout} = get_timeouts(Opts1), + PromptCheck = get_prompt_check(Opts1), + + EO = #eo{teln_pid=Pid, + prx=Prx, + idle_timeout=IdleTimeout, + total_timeout=TotalTimeout, + seq=Seq, + haltpatterns=HaltPatterns, + prompt_check=PromptCheck}, - case get_repeat(Opts1) of - false -> - case teln_expect1(Name,Pid,Data,Pattern2,[],EO) of - {ok,Matched,Rest} when WaitForPrompt -> - case lists:reverse(Matched) of - [{prompt,_},Matched1] -> - {ok,Matched1,Rest}; - [{prompt,_}|Matched1] -> - {ok,lists:reverse(Matched1),Rest} - end; - {ok,Matched,Rest} -> - {ok,Matched,Rest}; - {halt,Why,Rest} -> - {error,Why,Rest}; - {error,Reason} -> - {error,Reason} - end; - N -> - EO1 = EO#eo{repeat=N}, - repeat_expect(Name,Pid,Data,Pattern2,[],EO1) + case get_repeat(Opts1) of + false -> + case teln_expect1(Name,Pid,Data,Pattern2,[],EO) of + {ok,Matched,Rest} when WaitForPrompt -> + case lists:reverse(Matched) of + [{prompt,_},Matched1] -> + {ok,Matched1,Rest}; + [{prompt,_}|Matched1] -> + {ok,lists:reverse(Matched1),Rest} + end; + {ok,Matched,Rest} -> + {ok,Matched,Rest}; + {halt,Why,Rest} -> + {error,Why,Rest}; + {error,Reason} -> + {error,Reason} + end; + N -> + EO1 = EO#eo{repeat=N}, + repeat_expect(Name,Pid,Data,Pattern2,[],EO1) + end; + Error -> + Error + end; + Error -> + Error end. -convert_pattern(Pattern,Seq) - when is_list(Pattern) and not is_integer(hd(Pattern)) -> - case Seq of - true -> Pattern; - false -> rm_dupl(Pattern,[]) - end; +convert_pattern(Pattern0,Seq) + when Pattern0==[] orelse (is_list(Pattern0) and not is_integer(hd(Pattern0))) -> + Pattern = + case Seq of + true -> Pattern0; + false -> rm_dupl(Pattern0,[]) + end, + compile_pattern(Pattern,[]); convert_pattern(Pattern,_Seq) -> - [Pattern]. + compile_pattern([Pattern],[]). rm_dupl([P|Ps],Acc) -> case lists:member(P,Acc) of @@ -739,6 +762,25 @@ rm_dupl([P|Ps],Acc) -> rm_dupl([],Acc) -> lists:reverse(Acc). +compile_pattern([prompt|Patterns],Acc) -> + compile_pattern(Patterns,[prompt|Acc]); +compile_pattern([{prompt,_}=P|Patterns],Acc) -> + compile_pattern(Patterns,[P|Acc]); +compile_pattern([{Tag,Pattern}|Patterns],Acc) -> + try re:compile(Pattern,[unicode]) of + {ok,MP} -> compile_pattern(Patterns,[{Tag,MP}|Acc]); + {error,Error} -> {error,{bad_pattern,{Tag,Pattern},Error}} + catch error:badarg -> {error,{bad_pattern,{Tag,Pattern}}} + end; +compile_pattern([Pattern|Patterns],Acc) -> + try re:compile(Pattern,[unicode]) of + {ok,MP} -> compile_pattern(Patterns,[MP|Acc]); + {error,Error} -> {error,{bad_pattern,Pattern,Error}} + catch error:badarg -> {error,{bad_pattern,Pattern}} + end; +compile_pattern([],Acc) -> + {ok,lists:reverse(Acc)}. + get_timeouts(Opts) -> {case lists:keysearch(idle_timeout,1,Opts) of {value,{_,T}} -> @@ -772,7 +814,7 @@ get_seq(Opts) -> get_haltpatterns(Opts) -> case lists:keysearch(halt,1,Opts) of {value,{halt,HaltPatterns}} -> - convert_pattern(HaltPatterns,false); + HaltPatterns; false -> [] end. @@ -1068,7 +1110,7 @@ match_line(Name,Pid,Line,[{prompt,PromptType}|Patterns],FoundPrompt,Term, when PromptType=/=FoundPrompt -> match_line(Name,Pid,Line,Patterns,FoundPrompt,Term,EO,RetTag); match_line(Name,Pid,Line,[{Tag,Pattern}|Patterns],FoundPrompt,Term,EO,RetTag) -> - case re:run(Line,Pattern,[{capture,all,list},unicode]) of + case re:run(Line,Pattern,[{capture,all,list}]) of nomatch -> match_line(Name,Pid,Line,Patterns,FoundPrompt,Term,EO,RetTag); {match,Match} -> @@ -1076,7 +1118,7 @@ match_line(Name,Pid,Line,[{Tag,Pattern}|Patterns],FoundPrompt,Term,EO,RetTag) -> {RetTag,{Tag,Match}} end; match_line(Name,Pid,Line,[Pattern|Patterns],FoundPrompt,Term,EO,RetTag) -> - case re:run(Line,Pattern,[{capture,all,list},unicode]) of + case re:run(Line,Pattern,[{capture,all,list}]) of nomatch -> match_line(Name,Pid,Line,Patterns,FoundPrompt,Term,EO,RetTag); {match,Match} -> diff --git a/lib/common_test/src/ct_telnet_client.erl b/lib/common_test/src/ct_telnet_client.erl index 76e4b9ea70..007477c855 100644 --- a/lib/common_test/src/ct_telnet_client.erl +++ b/lib/common_test/src/ct_telnet_client.erl @@ -101,9 +101,11 @@ close(Pid) -> end. send_data(Pid, Data) -> - send_data(Pid, Data, true). + send_data(Pid, Data, "\n"). send_data(Pid, Data, true) -> - send_data(Pid, Data++"\n", false); + send_data(Pid, Data, "\n"); +send_data(Pid, Data, Newline) when is_list(Newline) -> + send_data(Pid, Data++Newline, false); send_data(Pid, Data, false) -> Pid ! {send_data, Data}, ok. diff --git a/lib/common_test/src/test_server.erl b/lib/common_test/src/test_server.erl index a896a0551b..9eda3f2152 100644 --- a/lib/common_test/src/test_server.erl +++ b/lib/common_test/src/test_server.erl @@ -850,17 +850,23 @@ spawn_fw_call(Mod,EPTC={end_per_testcase,Func},EndConf,Pid, "WARNING: end_per_testcase failed!</font>", {died,W} end, - try do_end_tc_call(Mod,EPTC,{Pid,Report,[EndConf]}, Why) of - _ -> ok - catch - _:FwEndTCErr -> - exit({fw_notify_done,end_tc,FwEndTCErr}) - end, - FailLoc = proplists:get_value(tc_fail_loc, EndConf), + FailLoc0 = proplists:get_value(tc_fail_loc, EndConf), + {RetVal1,FailLoc} = + try do_end_tc_call(Mod,EPTC,{Pid,Report,[EndConf]}, Why) of + Why -> + {RetVal,FailLoc0}; + {failed,_} = R -> + {R,[{Mod,Func}]}; + R -> + {R,FailLoc0} + catch + _:FwEndTCErr -> + exit({fw_notify_done,end_tc,FwEndTCErr}) + end, %% finished, report back (if end_per_testcase fails, a warning %% should be printed as part of the comment) SendTo ! {self(),fw_notify_done, - {Time,RetVal,FailLoc,[],Warn}} + {Time,RetVal1,FailLoc,[],Warn}} end, spawn_link(FwCall); @@ -902,14 +908,25 @@ spawn_fw_call(Mod,Func,CurrConf,Pid,Error,Loc,SendTo) -> FwErrorNotifyErr}) end, Conf = [{tc_status,{failed,Error}}|CurrConf], - try do_end_tc_call(Mod,EndTCFunc,{Pid,Error,[Conf]},Error) of - _ -> ok - catch - _:FwEndTCErr -> - exit({fw_notify_done,end_tc,FwEndTCErr}) - end, + {Time,RetVal,Loc1} = + try do_end_tc_call(Mod,EndTCFunc,{Pid,Error,[Conf]},Error) of + Error -> + {died, Error, Loc}; + {failed,Reason} = NewReturn -> + fw_error_notify(Mod,Func1,Conf,Reason), + {died, NewReturn, [{Mod,Func}]}; + NewReturn -> + T = case Error of + {timetrap_timeout,TT} -> TT; + _ -> 0 + end, + {T, NewReturn, Loc} + catch + _:FwEndTCErr -> + exit({fw_notify_done,end_tc,FwEndTCErr}) + end, %% finished, report back - SendTo ! {self(),fw_notify_done,{died,Error,Loc,[],undefined}} + SendTo ! {self(),fw_notify_done,{Time,RetVal,Loc1,[],undefined}} end, spawn_link(FwCall). diff --git a/lib/common_test/test/ct_hooks_SUITE.erl b/lib/common_test/test/ct_hooks_SUITE.erl index 0f5636a789..44b86b1dfe 100644 --- a/lib/common_test/test/ct_hooks_SUITE.erl +++ b/lib/common_test/test/ct_hooks_SUITE.erl @@ -84,7 +84,7 @@ all(suite) -> fail_post_suite_cth, skip_pre_suite_cth, skip_pre_end_cth, skip_pre_init_tc_cth, skip_post_suite_cth, recover_post_suite_cth, update_config_cth, - state_update_cth, options_cth, same_id_cth, + state_update_cth, update_result_cth, options_cth, same_id_cth, fail_n_skip_with_minimal_cth, prio_cth, no_config, no_init_suite_config, no_init_config, no_end_config, failed_sequence, repeat_force_stop, config_clash, @@ -209,6 +209,10 @@ state_update_cth(Config) when is_list(Config) -> do_test(state_update_cth, "ct_cth_fail_one_skip_one_SUITE.erl", [state_update_cth,state_update_cth],Config). +update_result_cth(Config) -> + do_test(update_result_cth, "ct_cth_update_result_post_end_tc_SUITE.erl", + [update_result_post_end_tc_cth],Config). + options_cth(Config) when is_list(Config) -> do_test(options_cth, "ct_cth_empty_SUITE.erl", [{empty_cth,[test]}],Config). @@ -1099,6 +1103,106 @@ test_events(state_update_cth) -> {?eh,stop_logging,[]} ]; +test_events(update_result_cth) -> + Suite = ct_cth_update_result_post_end_tc_SUITE, + [ + {?eh,start_logging,'_'}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,cth,{'_',init,['_',[]]}}, + {?eh,tc_start,{Suite,init_per_suite}}, + {?eh,tc_done,{Suite,init_per_suite,ok}}, + + {?eh,tc_start,{Suite,tc_ok_to_fail}}, + {?eh,cth,{'_',post_end_per_testcase,[Suite,tc_ok_to_fail,'_',ok,[]]}}, + {?eh,tc_done,{Suite,tc_ok_to_fail,{failed,{error,"Test failure"}}}}, + {?eh,cth,{'_',on_tc_fail,'_'}}, + {?eh,test_stats,{0,1,{0,0}}}, + + {?eh,tc_start,{Suite,tc_ok_to_skip}}, + {?eh,cth,{'_',post_end_per_testcase,[Suite,tc_ok_to_skip,'_',ok,[]]}}, + {?eh,tc_done,{Suite,tc_ok_to_skip,{skipped,"Test skipped"}}}, + {?eh,cth,{'_',on_tc_skip,'_'}}, + {?eh,test_stats,{0,1,{1,0}}}, + + {?eh,tc_start,{Suite,tc_fail_to_ok}}, + {?eh,cth,{'_',post_end_per_testcase, + [Suite,tc_fail_to_ok,'_', + {error,{test_case_failed,"should be changed to ok"}},[]]}}, + {?eh,tc_done,{Suite,tc_fail_to_ok,ok}}, + {?eh,test_stats,{1,1,{1,0}}}, + + {?eh,tc_start,{Suite,tc_fail_to_skip}}, + {?eh,cth,{'_',post_end_per_testcase, + [Suite,tc_fail_to_skip,'_', + {error,{test_case_failed,"should be changed to skip"}},[]]}}, + {?eh,tc_done,{Suite,tc_fail_to_skip,{skipped,"Test skipped"}}}, + {?eh,cth,{'_',on_tc_skip,'_'}}, + {?eh,test_stats,{1,1,{2,0}}}, + + {?eh,tc_start,{Suite,tc_timetrap_to_ok}}, + {?eh,cth,{'_',post_end_per_testcase, + [Suite,tc_timetrap_to_ok,'_',{timetrap_timeout,3000},[]]}}, + {?eh,tc_done,{Suite,tc_timetrap_to_ok,ok}}, + {?eh,test_stats,{2,1,{2,0}}}, + + {?eh,tc_start,{Suite,tc_timetrap_to_skip}}, + {?eh,cth,{'_',post_end_per_testcase, + [Suite,tc_timetrap_to_skip,'_',{timetrap_timeout,3000},[]]}}, + {?eh,tc_done,{Suite,tc_timetrap_to_skip,{skipped,"Test skipped"}}}, + {?eh,cth,{'_',on_tc_skip,'_'}}, + {?eh,test_stats,{2,1,{3,0}}}, + + {?eh,tc_start,{Suite,tc_skip_to_fail}}, + {?eh,cth,{'_',post_end_per_testcase, + [Suite,tc_skip_to_fail,'_', + {skip,"should be changed to fail"},[]]}}, + {?eh,tc_done,{Suite,tc_skip_to_fail,{failed,{error,"Test failure"}}}}, + {?eh,cth,{'_',on_tc_fail,'_'}}, + {?eh,test_stats,{2,2,{3,0}}}, + + {?eh,tc_start,{Suite,end_fail_to_fail}}, + {?eh,cth,{'_',post_end_per_testcase, + [Suite,end_fail_to_fail,'_', + {failed, + {Suite,end_per_testcase, + {'EXIT',{test_case_failed,"change result when end fails"}}}},[]]}}, + {?eh,tc_done,{Suite,end_fail_to_fail,{failed,{error,"Test failure"}}}}, + {?eh,cth,{'_',on_tc_fail,'_'}}, + {?eh,test_stats,{2,3,{3,0}}}, + + {?eh,tc_start,{Suite,end_fail_to_skip}}, + {?eh,cth,{'_',post_end_per_testcase, + [Suite,end_fail_to_skip,'_', + {failed, + {Suite,end_per_testcase, + {'EXIT',{test_case_failed,"change result when end fails"}}}},[]]}}, + {?eh,tc_done,{Suite,end_fail_to_skip,{skipped,"Test skipped"}}}, + {?eh,cth,{'_',on_tc_skip,'_'}}, + {?eh,test_stats,{2,3,{4,0}}}, + + {?eh,tc_start,{Suite,end_timetrap_to_fail}}, + {?eh,cth,{'_',post_end_per_testcase, + [Suite,end_timetrap_to_fail,'_', + {failed,{Suite,end_per_testcase,{timetrap_timeout,3000}}},[]]}}, + {?eh,tc_done,{Suite,end_timetrap_to_fail,{failed,{error,"Test failure"}}}}, + {?eh,cth,{'_',on_tc_fail,'_'}}, + {?eh,test_stats,{2,4,{4,0}}}, + + {?eh,tc_start,{Suite,end_timetrap_to_skip}}, + {?eh,cth,{'_',post_end_per_testcase, + [Suite,end_timetrap_to_skip,'_', + {failed,{Suite,end_per_testcase,{timetrap_timeout,3000}}},[]]}}, + {?eh,tc_done,{Suite,end_timetrap_to_skip,{skipped,"Test skipped"}}}, + {?eh,cth,{'_',on_tc_skip,'_'}}, + {?eh,test_stats,{2,4,{5,0}}}, + + {?eh,tc_start,{Suite,end_per_suite}}, + {?eh,tc_done,{Suite,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,cth,{'_',terminate,[[]]}}, + {?eh,stop_logging,[]} + ]; + test_events(options_cth) -> [ {?eh,start_logging,{'DEF','RUNDIR'}}, diff --git a/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/ct_cth_update_result_post_end_tc_SUITE.erl b/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/ct_cth_update_result_post_end_tc_SUITE.erl new file mode 100644 index 0000000000..a16138ce6f --- /dev/null +++ b/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/ct_cth_update_result_post_end_tc_SUITE.erl @@ -0,0 +1,101 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +-module(ct_cth_update_result_post_end_tc_SUITE). + +-compile(export_all). + +-include("ct.hrl"). + +suite() -> + [{timetrap,{seconds,3}}]. + +init_per_suite(Config) -> + Config. + +end_per_suite(Config) -> + ok. + +init_per_group(_,Config) -> + Config. + +end_per_group(_,_) -> + ok. + +init_per_testcase(_,Config) -> + Config. + +end_per_testcase(EndTimetrap,_) when EndTimetrap==end_timetrap_to_fail; + EndTimetrap==end_timetrap_to_skip-> + timer:sleep(10000); +end_per_testcase(EndFail,_) when EndFail==end_fail_to_fail; + EndFail==end_fail_to_skip-> + ct:fail("change result when end fails"); +end_per_testcase(_,_) -> + ok. + +all() -> + [tc_ok_to_fail, + tc_ok_to_skip, + tc_fail_to_ok, + tc_fail_to_skip, + tc_timetrap_to_ok, + tc_timetrap_to_skip, + tc_skip_to_fail, + end_fail_to_fail, + end_fail_to_skip, + end_timetrap_to_fail, + end_timetrap_to_skip]. + +%% Test cases starts here. +tc_ok_to_fail(_Config) -> + ok. + +tc_ok_to_skip(_Config) -> + ok. + +tc_fail_to_ok(_Config) -> + ct:fail("should be changed to ok"). + +tc_fail_to_skip(_Config) -> + ct:fail("should be changed to skip"). + +tc_timetrap_to_ok(_Config) -> + timer:sleep(10000), % will time out after 3 sek + ok. + +tc_timetrap_to_skip(_Config) -> + timer:sleep(10000), % will time out after 3 sek + ok. + +tc_skip_to_fail(_Config) -> + {skip,"should be changed to fail"}. + +end_fail_to_fail(_Config) -> + ok. + +end_fail_to_skip(_Config) -> + ok. + +end_timetrap_to_fail(_Config) -> + ok. + +end_timetrap_to_skip(_Config) -> + ok. diff --git a/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/update_result_post_end_tc_cth.erl b/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/update_result_post_end_tc_cth.erl new file mode 100644 index 0000000000..7afb3d8781 --- /dev/null +++ b/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/update_result_post_end_tc_cth.erl @@ -0,0 +1,98 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + + +-module(update_result_post_end_tc_cth). + + +-include_lib("common_test/src/ct_util.hrl"). +-include_lib("common_test/include/ct_event.hrl"). + + +%% CT Hooks +-compile(export_all). + +init(Id, Opts) -> + empty_cth:init(Id, Opts). + +pre_init_per_suite(Suite, Config, State) -> + empty_cth:pre_init_per_suite(Suite,Config,State). + +post_init_per_suite(Suite,Config,Return,State) -> + empty_cth:post_init_per_suite(Suite,Config,Return,State). + +pre_end_per_suite(Suite,Config,State) -> + empty_cth:pre_end_per_suite(Suite,Config,State). + +post_end_per_suite(Suite,Config,Return,State) -> + empty_cth:post_end_per_suite(Suite,Config,Return,State). + +pre_init_per_group(Suite,Group,Config,State) -> + empty_cth:pre_init_per_group(Suite,Group,Config,State). + +post_init_per_group(Suite,Group,Config,Return,State) -> + empty_cth:post_init_per_group(Suite,Group,Config,Return,State). + +pre_end_per_group(Suite,Group,Config,State) -> + empty_cth:pre_end_per_group(Suite,Group,Config,State). + +post_end_per_group(Suite,Group,Config,Return,State) -> + empty_cth:post_end_per_group(Suite,Group,Config,Return,State). + +pre_init_per_testcase(Suite,TC,Config,State) -> + empty_cth:pre_init_per_testcase(Suite,TC,Config,State). + +post_end_per_testcase(Suite,TC,Config,Return,State) -> + empty_cth:post_end_per_testcase(Suite,TC,Config,Return,State), + change_result(TC,Config,State). + +on_tc_fail(Suite,TC, Reason, State) -> + empty_cth:on_tc_fail(Suite,TC,Reason,State). + +on_tc_skip(Suite,TC, Reason, State) -> + empty_cth:on_tc_skip(Suite,TC,Reason,State). + +terminate(State) -> + empty_cth:terminate(State). + +%%%----------------------------------------------------------------- +%%% +change_result(tc_ok_to_fail,_Config,State) -> + {{fail, "Test failure"}, State}; +change_result(tc_ok_to_skip,_Config,State) -> + {{skip, "Test skipped"}, State}; +change_result(tc_fail_to_ok,Config,State) -> + {lists:keydelete(tc_status,1,Config),State}; +change_result(tc_fail_to_skip,Config,State) -> + {{skip,"Test skipped"},State}; +change_result(tc_timetrap_to_ok,Config,State) -> + {lists:keydelete(tc_status,1,Config),State}; +change_result(tc_timetrap_to_skip,Config,State) -> + {{skip,"Test skipped"},State}; +change_result(tc_skip_to_fail,_Config,State) -> + {{fail, "Test failure"}, State}; +change_result(end_fail_to_fail,_Config,State) -> + {{fail, "Test failure"}, State}; +change_result(end_fail_to_skip,_Config,State) -> + {{skip, "Test skipped"}, State}; +change_result(end_timetrap_to_fail,_Config,State) -> + {{fail, "Test failure"}, State}; +change_result(end_timetrap_to_skip,_Config,State) -> + {{skip, "Test skipped"}, State}. diff --git a/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl b/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl index a2fa099a8c..0d17481e95 100644 --- a/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl +++ b/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl @@ -440,6 +440,12 @@ edit_config(Config) -> ?ok = ct_netconfc:edit_config(Client,running, {server,[{xmlns,"myns"}], [{name,["myserver"]}]}), + ?NS:expect_reply('edit-config',ok), + ?ok = ct_netconfc:edit_config(Client,running, + [{server,[{xmlns,"myns"}], + [{name,["server1"]}]}, + {server,[{xmlns,"myns"}], + [{name,["server2"]}]}]), ?NS:expect_do_reply('close-session',close,ok), ?ok = ct_netconfc:close_session(Client), ok. diff --git a/lib/common_test/test/ct_telnet_SUITE.erl b/lib/common_test/test/ct_telnet_SUITE.erl index a0089c9bc9..f71b7c370f 100644 --- a/lib/common_test/test/ct_telnet_SUITE.erl +++ b/lib/common_test/test/ct_telnet_SUITE.erl @@ -50,10 +50,10 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. groups() -> - [{legacy, [], [unix_telnet,own_server,timetrap]}, - {raw, [], [unix_telnet,own_server,timetrap]}, - {html, [], [unix_telnet,own_server]}, - {silent, [], [unix_telnet,own_server]}]. + [{legacy, [], [unix_telnet,own_server,faulty_regexp,timetrap]}, + {raw, [], [unix_telnet,own_server,faulty_regexp,timetrap]}, + {html, [], [unix_telnet,own_server,faulty_regexp]}, + {silent, [], [unix_telnet,own_server,faulty_regexp]}]. all() -> [ @@ -119,6 +119,12 @@ own_server(Config) -> all_tests_in_suite(own_server,"ct_telnet_own_server_SUITE", CfgFile,Config). +faulty_regexp(Config) -> + CfgFile = "telnet.faulty_regexp." ++ + atom_to_list(groupname(Config)) ++ ".cfg", + all_tests_in_suite(faulty_regexp,"ct_telnet_faulty_regexp_SUITE", + CfgFile,Config). + timetrap(Config) -> CfgFile = "telnet.timetrap." ++ atom_to_list(groupname(Config)) ++ ".cfg", @@ -225,6 +231,31 @@ events_to_check(unix_telnet,Config) -> all_cases(ct_telnet_basic_SUITE,Config); events_to_check(own_server,Config) -> all_cases(ct_telnet_own_server_SUITE,Config); +events_to_check(faulty_regexp,_Config) -> + [{?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,tc_done, + {ct_telnet_faulty_regexp_SUITE,expect_pattern, + {failed, + {error,{{bad_pattern,"invalid(pattern",{"missing )",15}}, + {ct_telnet,expect,3}}}}}}, + {?eh,tc_done, + {ct_telnet_faulty_regexp_SUITE,expect_pattern_no_string, + {failed, + {error,{{bad_pattern,invalid_pattern}, + {ct_telnet,expect,3}}}}}}, + {?eh,tc_done, + {ct_telnet_faulty_regexp_SUITE,expect_tag_pattern, + {failed, + {error,{{bad_pattern,{tag,"invalid(pattern"},{"missing )",15}}, + {ct_telnet,expect,3}}}}}}, + {?eh,tc_done, + {ct_telnet_faulty_regexp_SUITE,expect_tag_pattern_no_string, + {failed, + {error,{{bad_pattern,{tag,invalid_pattern}}, + {ct_telnet,expect,3}}}}}}, + {?eh,tc_done,{ct_telnet_faulty_regexp_SUITE,expect_pattern_unicode,ok}}, + {?eh,tc_done,{ct_telnet_faulty_regexp_SUITE,expect_tag_pattern_unicode,ok}}, + {?eh,stop_logging,[]}]; events_to_check(timetrap,_Config) -> [{?eh,start_logging,{'DEF','RUNDIR'}}, {?eh,tc_done,{ct_telnet_timetrap_SUITE,expect_timetrap, diff --git a/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_faulty_regexp_SUITE.erl b/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_faulty_regexp_SUITE.erl new file mode 100644 index 0000000000..a5c9451a9c --- /dev/null +++ b/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_faulty_regexp_SUITE.erl @@ -0,0 +1,79 @@ +-module(ct_telnet_faulty_regexp_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +-define(name, telnet_server_conn1). + +%%-------------------------------------------------------------------- +%% TEST SERVER CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +init_per_suite(Config) -> + Config. + +end_per_suite(_Config) -> + ok. + +suite() -> [{require,?name,{unix,[telnet]}}, + {require,ct_conn_log}, + {ct_hooks, [{cth_conn_log,[]}]}]. + +all() -> + [expect_pattern, + expect_pattern_no_string, + expect_tag_pattern, + expect_tag_pattern_no_string, + expect_pattern_unicode, + expect_tag_pattern_unicode]. + +groups() -> + []. + +init_per_group(_GroupName, Config) -> + Config. + +end_per_group(_GroupName, Config) -> + Config. + +init_per_testcase(_,Config) -> + ct:log("init_per_testcase: opening telnet connection...",[]), + {ok,_} = ct_telnet:open(?name), + ct:log("...done",[]), + Config. + +end_per_testcase(_,_Config) -> + ct:log("end_per_testcase: closing telnet connection...",[]), + _ = ct_telnet:close(?name), + ct:log("...done",[]), + ok. + +expect_pattern(_) -> + ok = ct_telnet:send(?name, "echo ayt"), + ok = ct_telnet:expect(?name, "invalid(pattern"). + +expect_pattern_no_string(_) -> + ok = ct_telnet:send(?name, "echo ayt"), + ok = ct_telnet:expect(?name, invalid_pattern). + +expect_tag_pattern(_) -> + ok = ct_telnet:send(?name, "echo ayt"), + ok = ct_telnet:expect(?name, {tag,"invalid(pattern"}). + +expect_tag_pattern_no_string(_) -> + ok = ct_telnet:send(?name, "echo ayt"), + ok = ct_telnet:expect(?name, {tag,invalid_pattern}). + +%% Test that a unicode pattern can be given without the testcase +%% failing. Do however notice that there is no real unicode support +%% in ct_telnet yet, that is, the telnet binary mode is not supported. +expect_pattern_unicode(_) -> + ok = ct_telnet:send(?name, "echo ayt"), + {error,{prompt,_}} = ct_telnet:expect(?name, "pattern_with_unicode_αβ"), + ok. + +expect_tag_pattern_unicode(_) -> + ok = ct_telnet:send(?name, "echo ayt"), + {error,{prompt,_}} = ct_telnet:expect(?name, "pattern_with_unicode_αβ"), + ok. diff --git a/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_own_server_SUITE.erl b/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_own_server_SUITE.erl index 985fa40ad2..34df57027e 100644 --- a/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_own_server_SUITE.erl +++ b/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_own_server_SUITE.erl @@ -58,7 +58,8 @@ all() -> server_speaks, server_disconnects, newline_ayt, - newline_break + newline_break, + newline_string ]. groups() -> @@ -393,3 +394,11 @@ newline_break(_) -> "> " = lists:flatten(R), ok = ct_telnet:close(Handle), ok. + +%% Test option {newline,String} to specify an own newline, e.g. "\r\n" +newline_string(_) -> + {ok, Handle} = ct_telnet:open(telnet_server_conn1), + ok = ct_telnet:send(Handle, "echo hello-", [{newline,"own_nl\n"}]), + {ok,["hello-own_nl"]} = ct_telnet:expect(Handle, ["hello-own_nl"]), + ok = ct_telnet:close(Handle), + ok. diff --git a/lib/common_test/test_server/configure.in b/lib/common_test/test_server/configure.in index 0511d126b4..e07bd4c2aa 100644 --- a/lib/common_test/test_server/configure.in +++ b/lib/common_test/test_server/configure.in @@ -459,11 +459,11 @@ dnl Freely inspired by AC_TRY_LINK. (Maybe better to create a dnl AC_LANG_JAVA instead...) AC_DEFUN(ERL_TRY_LINK_JAVA, [java_link='$JAVAC conftest.java 1>&AC_FD_CC' -changequote(�, �)dnl +changequote(, )dnl cat > conftest.java <<EOF -�$1� +$1 class conftest { public static void main(String[] args) { - �$2� + $2 ; return; }} EOF changequote([, ])dnl diff --git a/lib/common_test/test_server/ts_erl_config.erl b/lib/common_test/test_server/ts_erl_config.erl index 537628e39a..f3972bea4e 100644 --- a/lib/common_test/test_server/ts_erl_config.erl +++ b/lib/common_test/test_server/ts_erl_config.erl @@ -208,7 +208,11 @@ erl_interface(Vars,OsType) -> {filename:join(Dir, "lib"), filename:join([Dir, "src", "eidefs.mk"])}; {srctree, _Root, Target} -> - {filename:join([Dir, "obj", Target]), + Obj = case is_debug_build() of + true -> "obj.debug"; + false -> "obj" + end, + {filename:join([Dir, Obj, Target]), filename:join([Dir, "src", Target, "eidefs.mk"])} end} end, diff --git a/lib/compiler/doc/src/notes.xml b/lib/compiler/doc/src/notes.xml index 7addadf82c..02e6203137 100644 --- a/lib/compiler/doc/src/notes.xml +++ b/lib/compiler/doc/src/notes.xml @@ -32,6 +32,26 @@ <p>This document describes the changes made to the Compiler application.</p> +<section><title>Compiler 7.3.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>An optimization that avoided allocation of a stack + frame for some <c>case</c> expressions was introduced in + OTP 21. (ERL-504/OTP-14808) It turns out that in rare + circumstances, this optimization is not safe. Therefore, + this optimization has been disabled.</p> + <p>A similar optimization will be included in OTP 22 in a + safe way.</p> + <p> + Own Id: OTP-15501 Aux Id: ERL-807, ERL-514, OTP-14808 </p> + </item> + </list> + </section> + +</section> + <section><title>Compiler 7.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/compiler/src/beam_type.erl b/lib/compiler/src/beam_type.erl index b5c979e529..a1e9eff8f3 100644 --- a/lib/compiler/src/beam_type.erl +++ b/lib/compiler/src/beam_type.erl @@ -1114,4 +1114,5 @@ verified_type(nonempty_list=T) -> T; verified_type({tuple,_,Sz,[]}=T) when is_integer(Sz) -> T; verified_type({tuple,_,Sz,[_]}=T) when is_integer(Sz) -> T; verified_type({tuple_element,_,_}=T) -> T; -verified_type(float=T) -> T. +verified_type(float=T) -> T; +verified_type(none=T) -> T. diff --git a/lib/compiler/src/v3_codegen.erl b/lib/compiler/src/v3_codegen.erl index e9152ba88f..d7a7778740 100644 --- a/lib/compiler/src/v3_codegen.erl +++ b/lib/compiler/src/v3_codegen.erl @@ -79,13 +79,9 @@ function(#k_fdef{anno=#k{a=Anno},func=Name,arity=Arity, try #k_match{} = Kb, %Assertion. - %% Try to suppress the stack frame unless it is - %% really needed. - Body0 = avoid_stack_frame(Kb), - %% Annotate kernel records with variable usage. Vdb0 = init_vars(As), - {Body,_,Vdb} = body(Body0, 1, Vdb0), + {Body,_,Vdb} = body(Kb, 1, Vdb0), %% Generate the BEAM assembly code. {Asm,EntryLabel,St} = cg_fun(Body, As, Vdb, AtomMod, @@ -98,136 +94,6 @@ function(#k_fdef{anno=#k{a=Anno},func=Name,arity=Arity, erlang:raise(Class, Error, Stack) end. - -%% avoid_stack_frame(Kernel) -> Kernel' -%% If possible, avoid setting up a stack frame. Functions -%% that only do matching, calls to guard BIFs, and tail-recursive -%% calls don't need a stack frame. - -avoid_stack_frame(#k_match{body=Body}=M) -> - try - M#k_match{body=avoid_stack_frame_1(Body)} - catch - impossible -> - M - end. - -avoid_stack_frame_1(#k_alt{first=First0,then=Then0}=Alt) -> - First = avoid_stack_frame_1(First0), - Then = avoid_stack_frame_1(Then0), - Alt#k_alt{first=First,then=Then}; -avoid_stack_frame_1(#k_bif{op=Op}=Bif) -> - case Op of - #k_internal{} -> - %% Most internal BIFs clobber the X registers. - throw(impossible); - _ -> - Bif - end; -avoid_stack_frame_1(#k_break{anno=Anno,args=Args}) -> - #k_guard_break{anno=Anno,args=Args}; -avoid_stack_frame_1(#k_guard_break{}=Break) -> - Break; -avoid_stack_frame_1(#k_enter{}=Enter) -> - %% Tail-recursive calls don't need a stack frame. - Enter; -avoid_stack_frame_1(#k_guard{clauses=Cs0}=Guard) -> - Cs = avoid_stack_frame_list(Cs0), - Guard#k_guard{clauses=Cs}; -avoid_stack_frame_1(#k_guard_clause{guard=G0,body=B0}=C) -> - G = avoid_stack_frame_1(G0), - B = avoid_stack_frame_1(B0), - C#k_guard_clause{guard=G,body=B}; -avoid_stack_frame_1(#k_match{anno=A,vars=Vs,body=B0,ret=Ret}) -> - %% Use #k_guard_match{} instead to avoid saving the X registers - %% to the stack before matching. - B = avoid_stack_frame_1(B0), - #k_guard_match{anno=A,vars=Vs,body=B,ret=Ret}; -avoid_stack_frame_1(#k_guard_match{body=B0}=M) -> - B = avoid_stack_frame_1(B0), - M#k_guard_match{body=B}; -avoid_stack_frame_1(#k_protected{arg=Arg0}=Prot) -> - Arg = avoid_stack_frame_1(Arg0), - Prot#k_protected{arg=Arg}; -avoid_stack_frame_1(#k_put{}=Put) -> - Put; -avoid_stack_frame_1(#k_return{}=Ret) -> - Ret; -avoid_stack_frame_1(#k_select{var=#k_var{anno=Vanno},types=Types0}=Select) -> - case member(reuse_for_context, Vanno) of - false -> - Types = avoid_stack_frame_list(Types0), - Select#k_select{types=Types}; - true -> - %% Including binary patterns that overwrite the register containing - %% the binary with the match context may not be safe. For example, - %% bs_match_SUITE:bin_tail_e/1 with inlining will be rejected by - %% beam_validator. - %% - %% Essentially the following code is produced: - %% - %% bs_match {x,0} => {x,0} - %% ... - %% bs_match {x,0} => {x,1} %% ILLEGAL - %% - %% A bs_match instruction will only accept a match context as the - %% source operand if the source and destination registers are the - %% the same (as in the first bs_match instruction above). - %% The second bs_match instruction is therefore illegal. - %% - %% This situation is avoided if there is a stack frame: - %% - %% move {x,0} => {y,0} - %% bs_match {x,0} => {x,0} - %% ... - %% bs_match {y,0} => {x,1} %% LEGAL - %% - throw(impossible) - end; -avoid_stack_frame_1(#k_seq{arg=#k_call{anno=Anno,op=Op}=Call, - body=#k_break{args=BrArgs0}}=Seq) -> - case Op of - #k_remote{mod=#k_atom{val=Mod}, - name=#k_atom{val=Name}, - arity=Arity} -> - case erl_bifs:is_exit_bif(Mod, Name, Arity) of - false -> - %% Will clobber X registers. Must have a stack frame. - throw(impossible); - true -> - %% The call to this BIF will never return. It is safe - %% to suppress the stack frame. - Bif = #k_bif{anno=Anno, - op=#k_internal{name=guard_error,arity=1}, - args=[Call],ret=[]}, - BrArgs = lists:duplicate(length(BrArgs0), #k_nil{}), - GB = #k_guard_break{anno=#k{us=[],ns=[],a=[]},args=BrArgs}, - Seq#k_seq{arg=Bif,body=GB} - end; - _ -> - %% Will clobber X registers. Must have a stack frame. - throw(impossible) - end; -avoid_stack_frame_1(#k_seq{arg=A0,body=B0}=Seq) -> - A = avoid_stack_frame_1(A0), - B = avoid_stack_frame_1(B0), - Seq#k_seq{arg=A,body=B}; -avoid_stack_frame_1(#k_test{}=Test) -> - Test; -avoid_stack_frame_1(#k_type_clause{values=Values0}=TC) -> - Values = avoid_stack_frame_list(Values0), - TC#k_type_clause{values=Values}; -avoid_stack_frame_1(#k_val_clause{body=B0}=VC) -> - B = avoid_stack_frame_1(B0), - VC#k_val_clause{body=B}; -avoid_stack_frame_1(_Body) -> - throw(impossible). - -avoid_stack_frame_list([H|T]) -> - [avoid_stack_frame_1(H)|avoid_stack_frame_list(T)]; -avoid_stack_frame_list([]) -> []. - - %% This pass creates beam format annotated with variable lifetime %% information. Each thing is given an index and for each variable we %% store the first and last index for its occurrence. The variable diff --git a/lib/compiler/src/v3_core.erl b/lib/compiler/src/v3_core.erl index c9517c3e51..66e578b776 100644 --- a/lib/compiler/src/v3_core.erl +++ b/lib/compiler/src/v3_core.erl @@ -766,14 +766,16 @@ expr({op,_,'++',{lc,Llc,E,Qs0},More}, St0) -> {Qs,St2} = preprocess_quals(Llc, Qs0, St1), {Y,Yps,St} = lc_tq(Llc, E, Qs, Mc, St2), {Y,Mps++Yps,St}; -expr({op,L,'andalso',E1,E2}, St0) -> +expr({op,_,'andalso',_,_}=E0, St0) -> + {op,L,'andalso',E1,E2} = right_assoc(E0, 'andalso', St0), Anno = lineno_anno(L, St0), {#c_var{name=V0},St} = new_var(Anno, St0), V = {var,L,V0}, False = {atom,L,false}, E = make_bool_switch(L, E1, V, E2, False, St0), expr(E, St); -expr({op,L,'orelse',E1,E2}, St0) -> +expr({op,_,'orelse',_,_}=E0, St0) -> + {op,L,'orelse',E1,E2} = right_assoc(E0, 'orelse', St0), Anno = lineno_anno(L, St0), {#c_var{name=V0},St} = new_var(Anno, St0), V = {var,L,V0}, @@ -2626,7 +2628,8 @@ cfun(#ifun{anno=A,id=Id,vars=Args,clauses=Lcs,fc=Lfc}, _As, St0) -> [],A#a.us,St2}. c_call_erl(Fun, Args) -> - cerl:c_call(cerl:c_atom(erlang), cerl:c_atom(Fun), Args). + As = [compiler_generated], + cerl:ann_c_call(As, cerl:c_atom(erlang), cerl:c_atom(Fun), Args). %% lit_vars(Literal) -> [Var]. diff --git a/lib/compiler/test/beam_type_SUITE.erl b/lib/compiler/test/beam_type_SUITE.erl index 061076b3ff..9f691716e3 100644 --- a/lib/compiler/test/beam_type_SUITE.erl +++ b/lib/compiler/test/beam_type_SUITE.erl @@ -157,6 +157,10 @@ coverage(Config) -> [_|_] -> ok end, + + %% Cover beam_type:verified_type(none). + {'EXIT',{badarith,_}} = (catch (id(2) / id(1)) band 16#ff), + ok. booleans(_Config) -> diff --git a/lib/compiler/test/match_SUITE.erl b/lib/compiler/test/match_SUITE.erl index e3f842b668..72e5356a8d 100644 --- a/lib/compiler/test/match_SUITE.erl +++ b/lib/compiler/test/match_SUITE.erl @@ -378,6 +378,13 @@ untuplify(Config) when is_list(Config) -> %% We do this to cover sys_core_fold:unalias_pat/1. {1,2,3,4,alias,{[1,2],{3,4},alias}} = untuplify_1([1,2], {3,4}, alias), error = untuplify_1([1,2], {3,4}, 42), + + %% Test that a previous bug in v3_codegen is gone. (The sinking of + %% stack frames into only the case arms that needed them was not always + %% safe.) + [33, -1, -33, 1] = untuplify_2(32, 65), + {33, 1, -33, -1} = untuplify_2(65, 32), + ok. untuplify_1(A, B, C) -> @@ -390,6 +397,21 @@ untuplify_1(A, B, C) -> error end. +untuplify_2(V1, V2) -> + {D1,D2,D3,D4} = + if V1 > V2 -> + %% The 1 value was overwritten by the value of V2-V1. + {V1-V2, 1, V2-V1, -1}; + true -> + {V2-V1, -1, V1-V2, 1} + end, + if + D2 > D4 -> + {D1, D2, D3, D4}; + true -> + [D1, D2, D3, D4] + end. + %% Coverage of beam_dead:shortcut_boolean_label/4. shortcut_boolean(Config) when is_list(Config) -> false = shortcut_boolean_1([0]), diff --git a/lib/compiler/test/warnings_SUITE.erl b/lib/compiler/test/warnings_SUITE.erl index 33d55996ad..1c23eba06d 100644 --- a/lib/compiler/test/warnings_SUITE.erl +++ b/lib/compiler/test/warnings_SUITE.erl @@ -42,7 +42,7 @@ comprehensions/1,maps/1,maps_bin_opt_info/1, redundant_boolean_clauses/1, latin1_fallback/1,underscore/1,no_warnings/1, - bit_syntax/1,inlining/1]). + bit_syntax/1,inlining/1,tuple_calls/1]). init_per_testcase(_Case, Config) -> Config. @@ -64,7 +64,8 @@ groups() -> bin_opt_info,bin_construction,comprehensions,maps, maps_bin_opt_info, redundant_boolean_clauses,latin1_fallback, - underscore,no_warnings,bit_syntax,inlining]}]. + underscore,no_warnings,bit_syntax,inlining, + tuple_calls]}]. init_per_suite(Config) -> test_lib:recompile(?MODULE), @@ -952,6 +953,20 @@ inlining(Config) -> run(Config, Ts), ok. +tuple_calls(Config) -> + %% Make sure that no spurious warnings are generated. + Ts = [{inlining_1, + <<"-compile(tuple_calls). + dispatch(X) -> + (list_to_atom(\"prefix_\" ++ + atom_to_list(suffix))):doit(X). + ">>, + [], + []} + ], + run(Config, Ts), + ok. + %%% %%% End of test cases. %%% diff --git a/lib/compiler/vsn.mk b/lib/compiler/vsn.mk index 97179b7fc4..efedb414ad 100644 --- a/lib/compiler/vsn.mk +++ b/lib/compiler/vsn.mk @@ -1 +1 @@ -COMPILER_VSN = 7.3 +COMPILER_VSN = 7.3.1 diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c index df607732bf..194a3d30e9 100644 --- a/lib/crypto/c_src/crypto.c +++ b/lib/crypto/c_src/crypto.c @@ -177,7 +177,8 @@ && !defined(HAS_LIBRESSL) \ && defined(HAVE_EC) # define HAVE_ED_CURVE_DH -# if OPENSSL_VERSION_NUMBER >= (PACKED_OPENSSL_VERSION_PLAIN(1,1,1)) +# if OPENSSL_VERSION_NUMBER >= (PACKED_OPENSSL_VERSION_PLAIN(1,1,1)) \ + && !defined(FIPS_SUPPORT) # define HAVE_EDDSA # endif #endif @@ -1425,8 +1426,6 @@ static void init_algorithms_types(ErlNifEnv* env) #endif algo_cipher[algo_cipher_cnt++] = enif_make_atom(env, "aes_cbc"); algo_cipher[algo_cipher_cnt++] = enif_make_atom(env, "aes_cbc128"); - algo_cipher[algo_cipher_cnt++] = enif_make_atom(env, "aes_cfb8"); - algo_cipher[algo_cipher_cnt++] = enif_make_atom(env, "aes_cfb128"); algo_cipher[algo_cipher_cnt++] = enif_make_atom(env, "aes_cbc256"); algo_cipher[algo_cipher_cnt++] = enif_make_atom(env, "aes_ctr"); algo_cipher[algo_cipher_cnt++] = enif_make_atom(env, "aes_ecb"); @@ -1441,6 +1440,8 @@ static void init_algorithms_types(ErlNifEnv* env) #ifdef HAVE_AES_IGE algo_cipher[algo_cipher_cnt++] = enif_make_atom(env,"aes_ige256"); #endif + algo_cipher[algo_cipher_cnt++] = enif_make_atom(env, "aes_cfb8"); + algo_cipher[algo_cipher_cnt++] = enif_make_atom(env, "aes_cfb128"); #ifndef OPENSSL_NO_DES algo_cipher[algo_cipher_cnt++] = enif_make_atom(env,"des_cbc"); algo_cipher[algo_cipher_cnt++] = enif_make_atom(env,"des_cfb"); @@ -2325,21 +2326,24 @@ static ERL_NIF_TERM block_crypt_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM return enif_raise_exception(env, atom_notsup); } - if (argv[0] == atom_aes_cfb8 - && (key.size == 24 || key.size == 32)) { - /* Why do EVP_CIPHER_CTX_set_key_length() fail on these key sizes? - * Fall back on low level API - */ - return aes_cfb_8_crypt(env, argc-1, argv+1); + if (argv[0] == atom_aes_cfb8) { + CHECK_NO_FIPS_MODE(); + if ((key.size == 24 || key.size == 32)) { + /* Why do EVP_CIPHER_CTX_set_key_length() fail on these key sizes? + * Fall back on low level API + */ + return aes_cfb_8_crypt(env, argc-1, argv+1); + } + } + else if (argv[0] == atom_aes_cfb128) { + CHECK_NO_FIPS_MODE(); + if ((key.size == 24 || key.size == 32)) { + /* Why do EVP_CIPHER_CTX_set_key_length() fail on these key sizes? + * Fall back on low level API + */ + return aes_cfb_128_crypt_nif(env, argc-1, argv+1); + } } - else if (argv[0] == atom_aes_cfb128 - && (key.size == 24 || key.size == 32)) { - /* Why do EVP_CIPHER_CTX_set_key_length() fail on these key sizes? - * Fall back on low level API - */ - return aes_cfb_128_crypt_nif(env, argc-1, argv+1); - } - ivec_size = EVP_CIPHER_iv_length(cipher); #ifdef HAVE_ECB_IVEC_BUG @@ -4357,8 +4361,11 @@ static int get_pkey_digest_type(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_ *md = NULL; if (type == atom_none && algorithm == atom_rsa) return PKEY_OK; + if (algorithm == atom_eddsa) #ifdef HAVE_EDDSA - if (algorithm == atom_eddsa) return PKEY_OK; + return PKEY_OK; +#else + return PKEY_NOTSUP; #endif digp = get_digest_type(type); if (!digp) return PKEY_BADARG; diff --git a/lib/crypto/doc/src/engine_keys.xml b/lib/crypto/doc/src/engine_keys.xml index feeb353d1e..5ac690eb90 100644 --- a/lib/crypto/doc/src/engine_keys.xml +++ b/lib/crypto/doc/src/engine_keys.xml @@ -51,7 +51,7 @@ <p> OTP/Crypto requires that the user provides two or three items of information about the key. The application used by the user is usually on a higher level, for example in - <seealso marker="ssl:ssl#key_option_def">SSL</seealso>. If using + <seealso marker="ssl:ssl#type-key">SSL</seealso>. If using the crypto application directly, it is required that: </p> <list> diff --git a/lib/crypto/src/crypto.erl b/lib/crypto/src/crypto.erl index 72cb9aabfd..bc8b124b10 100644 --- a/lib/crypto/src/crypto.erl +++ b/lib/crypto/src/crypto.erl @@ -512,17 +512,17 @@ block_encrypt(Type, Key, Ivec, PlainText) when Type =:= des_cbc; Type =:= aes_cbc256; Type =:= aes_cbc; Type =:= rc2_cbc -> - block_crypt_nif(Type, Key, Ivec, PlainText, true); + notsup_to_error(block_crypt_nif(Type, Key, Ivec, PlainText, true)); block_encrypt(Type, Key0, Ivec, PlainText) when Type =:= des3_cbc; Type =:= des_ede3 -> Key = check_des3_key(Key0), - block_crypt_nif(des_ede3_cbc, Key, Ivec, PlainText, true); + notsup_to_error(block_crypt_nif(des_ede3_cbc, Key, Ivec, PlainText, true)); block_encrypt(des3_cbf, Key0, Ivec, PlainText) -> % cfb misspelled Key = check_des3_key(Key0), - block_crypt_nif(des_ede3_cbf, Key, Ivec, PlainText, true); + notsup_to_error(block_crypt_nif(des_ede3_cbf, Key, Ivec, PlainText, true)); block_encrypt(des3_cfb, Key0, Ivec, PlainText) -> Key = check_des3_key(Key0), - block_crypt_nif(des_ede3_cfb, Key, Ivec, PlainText, true); + notsup_to_error(block_crypt_nif(des_ede3_cfb, Key, Ivec, PlainText, true)); block_encrypt(aes_ige256, Key, Ivec, PlainText) -> notsup_to_error(aes_ige_crypt_nif(Key, Ivec, PlainText, true)); block_encrypt(Type, Key, Ivec, {AAD, PlainText}) when Type =:= aes_gcm; @@ -549,17 +549,17 @@ block_decrypt(Type, Key, Ivec, Data) when Type =:= des_cbc; Type =:= aes_cfb128; Type =:= aes_cbc256; Type =:= rc2_cbc -> - block_crypt_nif(Type, Key, Ivec, Data, false); + notsup_to_error(block_crypt_nif(Type, Key, Ivec, Data, false)); block_decrypt(Type, Key0, Ivec, Data) when Type =:= des3_cbc; Type =:= des_ede3 -> Key = check_des3_key(Key0), - block_crypt_nif(des_ede3_cbc, Key, Ivec, Data, false); + notsup_to_error(block_crypt_nif(des_ede3_cbc, Key, Ivec, Data, false)); block_decrypt(des3_cbf, Key0, Ivec, Data) -> % cfb misspelled Key = check_des3_key(Key0), - block_crypt_nif(des_ede3_cbf, Key, Ivec, Data, false); + notsup_to_error(block_crypt_nif(des_ede3_cbf, Key, Ivec, Data, false)); block_decrypt(des3_cfb, Key0, Ivec, Data) -> Key = check_des3_key(Key0), - block_crypt_nif(des_ede3_cfb, Key, Ivec, Data, false); + notsup_to_error(block_crypt_nif(des_ede3_cfb, Key, Ivec, Data, false)); block_decrypt(aes_ige256, Key, Ivec, Data) -> notsup_to_error(aes_ige_crypt_nif(Key, Ivec, Data, false)); block_decrypt(Type, Key, Ivec, {AAD, Data, Tag}) when Type =:= aes_gcm; @@ -571,13 +571,13 @@ block_decrypt(Type, Key, Ivec, {AAD, Data, Tag}) when Type =:= aes_gcm; -spec block_encrypt(Type::block_cipher_without_iv(), Key::key(), PlainText::iodata()) -> binary(). block_encrypt(Type, Key, PlainText) -> - block_crypt_nif(Type, Key, PlainText, true). + notsup_to_error(block_crypt_nif(Type, Key, PlainText, true)). -spec block_decrypt(Type::block_cipher_without_iv(), Key::key(), Data::iodata()) -> binary(). block_decrypt(Type, Key, Data) -> - block_crypt_nif(Type, Key, Data, false). + notsup_to_error(block_crypt_nif(Type, Key, Data, false)). -spec next_iv(Type:: cbc_cipher(), Data) -> NextIVec when % Type :: cbc_cipher(), %des_cbc | des3_cbc | aes_cbc | aes_ige, diff --git a/lib/crypto/test/Makefile b/lib/crypto/test/Makefile index e046a25338..988d95a8bc 100644 --- a/lib/crypto/test/Makefile +++ b/lib/crypto/test/Makefile @@ -6,7 +6,7 @@ include $(ERL_TOP)/make/$(TARGET)/otp.mk # ---------------------------------------------------- MODULES = \ - blowfish_SUITE \ + crypto_bench_SUITE \ crypto_SUITE \ engine_SUITE @@ -77,7 +77,7 @@ release_spec: release_tests_spec: $(TEST_TARGET) $(INSTALL_DIR) "$(RELSYSDIR)" - $(INSTALL_DATA) crypto.spec crypto.cover $(RELTEST_FILES) "$(RELSYSDIR)" + $(INSTALL_DATA) crypto.spec crypto_bench.spec crypto.cover $(RELTEST_FILES) "$(RELSYSDIR)" @tar cfh - *_SUITE_data | (cd "$(RELSYSDIR)"; tar xf -) chmod -R u+w "$(RELSYSDIR)" diff --git a/lib/crypto/test/blowfish_SUITE.erl b/lib/crypto/test/blowfish_SUITE.erl deleted file mode 100644 index a931ebb47e..0000000000 --- a/lib/crypto/test/blowfish_SUITE.erl +++ /dev/null @@ -1,300 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2009-2018. All Rights Reserved. -%% -%% Licensed under the Apache License, Version 2.0 (the "License"); -%% you may not use this file except in compliance with the License. -%% You may obtain a copy of the License at -%% -%% http://www.apache.org/licenses/LICENSE-2.0 -%% -%% Unless required by applicable law or agreed to in writing, software -%% distributed under the License is distributed on an "AS IS" BASIS, -%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -%% See the License for the specific language governing permissions and -%% limitations under the License. -%% -%% %CopyrightEnd% -%% - -%% --module(blowfish_SUITE). - -%% Note: This directive should only be used in test suites. --compile(export_all). - --include_lib("common_test/include/ct.hrl"). - --define(TIMEOUT, 120000). % 2 min - --define(KEY, to_bin("0123456789ABCDEFF0E1D2C3B4A59687")). --define(IVEC, to_bin("FEDCBA9876543210")). -%% "7654321 Now is the time for " (includes trailing '\0') --define(DATA, to_bin("37363534333231204E6F77206973207468652074696D6520666F722000")). --define(DATA_PADDED, to_bin("37363534333231204E6F77206973207468652074696D6520666F722000000000")). - -%% Test server callback functions -%%-------------------------------------------------------------------- -%% Function: init_per_suite(Config) -> Config -%% Config - [tuple()] -%% A list of key/value pairs, holding the test case configuration. -%% Description: Initialization before the whole suite -%% -%% Note: This function is free to add any key/value pairs to the Config -%% variable, but should NOT alter/remove any existing entries. -%%-------------------------------------------------------------------- -init_per_suite(Config) -> - case catch crypto:start() of - ok -> - catch ct:comment("~s",[element(3,hd(crypto:info_lib()))]), - catch ct:log("crypto:info_lib() -> ~p~n" - "crypto:supports() -> ~p~n" - "crypto:version() -> ~p~n" - ,[crypto:info_lib(), crypto:supports(), crypto:version()]), - Config; - _Else -> - {skip,"Could not start crypto!"} - end. - -%%-------------------------------------------------------------------- -%% Function: end_per_suite(Config) -> _ -%% Config - [tuple()] -%% A list of key/value pairs, holding the test case configuration. -%% Description: Cleanup after the whole suite -%%-------------------------------------------------------------------- -end_per_suite(_Config) -> - crypto:stop(). - -%%-------------------------------------------------------------------- -%% Function: init_per_testcase(TestCase, Config) -> Config -%% Case - atom() -%% Name of the test case that is about to be run. -%% Config - [tuple()] -%% A list of key/value pairs, holding the test case configuration. -%% -%% Description: Initialization before each test case -%% -%% Note: This function is free to add any key/value pairs to the Config -%% variable, but should NOT alter/remove any existing entries. -%% Description: Initialization before each test case -%%-------------------------------------------------------------------- -init_per_testcase(_TestCase, Config0) -> - Config = lists:keydelete(watchdog, 1, Config0), - Dog = test_server:timetrap(?TIMEOUT), - [{watchdog, Dog} | Config]. - -%%-------------------------------------------------------------------- -%% Function: end_per_testcase(TestCase, Config) -> _ -%% Case - atom() -%% Name of the test case that is about to be run. -%% Config - [tuple()] -%% A list of key/value pairs, holding the test case configuration. -%% Description: Cleanup after each test case -%%-------------------------------------------------------------------- -end_per_testcase(_TestCase, Config) -> - Dog = ?config(watchdog, Config), - case Dog of - undefined -> - ok; - _ -> - test_server:timetrap_cancel(Dog) - end. - -%%-------------------------------------------------------------------- -%% Function: all(Clause) -> TestCases -%% Clause - atom() - suite | doc -%% TestCases - [Case] -%% Case - atom() -%% Name of a test case. -%% Description: Returns a list of all test cases in this test suite -%%-------------------------------------------------------------------- -suite() -> [{ct_hooks,[ts_install_cth]}]. - -all() -> -[{group, fips}, - {group, non_fips}]. - -groups() -> - [{fips, [], [no_ecb, no_cbc, no_cfb64, no_ofb64]}, - {non_fips, [], [ecb, cbc, cfb64, ofb64]}]. - -init_per_group(fips, Config) -> - case crypto:info_fips() of - enabled -> - Config; - not_enabled -> - case crypto:enable_fips_mode(true) of - true -> - enabled = crypto:info_fips(), - Config; - false -> - {skip, "Failed to enable FIPS mode"} - end; - not_supported -> - {skip, "FIPS mode not supported"} - end; -init_per_group(non_fips, Config) -> - case crypto:info_fips() of - enabled -> - true = crypto:enable_fips_mode(false), - not_enabled = crypto:info_fips(), - Config; - _NotEnabled -> - Config - end; -init_per_group(_GroupName, Config) -> - Config. - -end_per_group(_GroupName, Config) -> - Config. - - -%% Test cases start here. -%%-------------------------------------------------------------------- - -ecb_test(KeyBytes, ClearBytes, CipherBytes) -> - {Key, Clear, Cipher} = - {to_bin(KeyBytes), to_bin(ClearBytes), to_bin(CipherBytes)}, - ?line m(crypto:block_encrypt(blowfish_ecb, Key, Clear), Cipher), - true. - -ecb(doc) -> - "Test that ECB mode is OK"; -ecb(suite) -> - []; -ecb(Config) when is_list(Config) -> - true = ecb_test("0000000000000000", "0000000000000000", "4EF997456198DD78"), - true = ecb_test("FFFFFFFFFFFFFFFF", "FFFFFFFFFFFFFFFF", "51866FD5B85ECB8A"), - true = ecb_test("3000000000000000", "1000000000000001", "7D856F9A613063F2"), - true = ecb_test("1111111111111111", "1111111111111111", "2466DD878B963C9D"), - true = ecb_test("0123456789ABCDEF", "1111111111111111", "61F9C3802281B096"), - true = ecb_test("1111111111111111", "0123456789ABCDEF", "7D0CC630AFDA1EC7"), - true = ecb_test("0000000000000000", "0000000000000000", "4EF997456198DD78"), - true = ecb_test("FEDCBA9876543210", "0123456789ABCDEF", "0ACEAB0FC6A0A28D"), - true = ecb_test("7CA110454A1A6E57", "01A1D6D039776742", "59C68245EB05282B"), - true = ecb_test("0131D9619DC1376E", "5CD54CA83DEF57DA", "B1B8CC0B250F09A0"), - true = ecb_test("07A1133E4A0B2686", "0248D43806F67172", "1730E5778BEA1DA4"), - true = ecb_test("3849674C2602319E", "51454B582DDF440A", "A25E7856CF2651EB"), - true = ecb_test("04B915BA43FEB5B6", "42FD443059577FA2", "353882B109CE8F1A"), - true = ecb_test("0113B970FD34F2CE", "059B5E0851CF143A", "48F4D0884C379918"), - true = ecb_test("0170F175468FB5E6", "0756D8E0774761D2", "432193B78951FC98"), - true = ecb_test("43297FAD38E373FE", "762514B829BF486A", "13F04154D69D1AE5"), - true = ecb_test("07A7137045DA2A16", "3BDD119049372802", "2EEDDA93FFD39C79"), - true = ecb_test("04689104C2FD3B2F", "26955F6835AF609A", "D887E0393C2DA6E3"), - true = ecb_test("37D06BB516CB7546", "164D5E404F275232", "5F99D04F5B163969"), - true = ecb_test("1F08260D1AC2465E", "6B056E18759F5CCA", "4A057A3B24D3977B"), - true = ecb_test("584023641ABA6176", "004BD6EF09176062", "452031C1E4FADA8E"), - true = ecb_test("025816164629B007", "480D39006EE762F2", "7555AE39F59B87BD"), - true = ecb_test("49793EBC79B3258F", "437540C8698F3CFA", "53C55F9CB49FC019"), - true = ecb_test("4FB05E1515AB73A7", "072D43A077075292", "7A8E7BFA937E89A3"), - true = ecb_test("49E95D6D4CA229BF", "02FE55778117F12A", "CF9C5D7A4986ADB5"), - true = ecb_test("018310DC409B26D6", "1D9D5C5018F728C2", "D1ABB290658BC778"), - true = ecb_test("1C587F1C13924FEF", "305532286D6F295A", "55CB3774D13EF201"), - true = ecb_test("0101010101010101", "0123456789ABCDEF", "FA34EC4847B268B2"), - true = ecb_test("1F1F1F1F0E0E0E0E", "0123456789ABCDEF", "A790795108EA3CAE"), - true = ecb_test("E0FEE0FEF1FEF1FE", "0123456789ABCDEF", "C39E072D9FAC631D"), - true = ecb_test("0000000000000000", "FFFFFFFFFFFFFFFF", "014933E0CDAFF6E4"), - true = ecb_test("FFFFFFFFFFFFFFFF", "0000000000000000", "F21E9A77B71C49BC"), - true = ecb_test("0123456789ABCDEF", "0000000000000000", "245946885754369A"), - true = ecb_test("FEDCBA9876543210", "FFFFFFFFFFFFFFFF", "6B5C5A9C5D9E0A5A"), - ok. - -cbc(doc) -> - "Test that CBC mode is OK"; -cbc(suite) -> - []; -cbc(Config) when is_list(Config) -> - true = crypto:block_encrypt(blowfish_cbc, ?KEY, ?IVEC, ?DATA_PADDED) =:= - to_bin("6B77B4D63006DEE605B156E27403979358DEB9E7154616D959F1652BD5FF92CC"), - ok. - -cfb64(doc) -> - "Test that CFB64 mode is OK"; -cfb64(suite) -> - []; -cfb64(Config) when is_list(Config) -> - true = crypto:block_encrypt(blowfish_cfb64, ?KEY, ?IVEC, ?DATA) =:= - to_bin("E73214A2822139CAF26ECF6D2EB9E76E3DA3DE04D1517200519D57A6C3"), - ok. - -ofb64(doc) -> - "Test that OFB64 mode is OK"; -ofb64(suite) -> - []; -ofb64(Config) when is_list(Config) -> - true = crypto:block_encrypt(blowfish_ofb64, ?KEY, ?IVEC, ?DATA) =:= - to_bin("E73214A2822139CA62B343CC5B65587310DD908D0C241B2263C2CF80DA"), - ok. - -no_ecb(doc) -> - "Test that ECB mode is disabled"; -no_ecb(suite) -> - []; -no_ecb(Config) when is_list(Config) -> - notsup(fun crypto:block_encrypt/3, - [blowfish_ecb, - to_bin("0000000000000000"), - to_bin("FFFFFFFFFFFFFFFF")]). - -no_cbc(doc) -> - "Test that CBC mode is disabled"; -no_cbc(suite) -> - []; -no_cbc(Config) when is_list(Config) -> - notsup(fun crypto:block_encrypt/4, - [blowfish_cbc, ?KEY, ?IVEC, ?DATA_PADDED]). - -no_cfb64(doc) -> - "Test that CFB64 mode is disabled"; -no_cfb64(suite) -> - []; -no_cfb64(Config) when is_list(Config) -> - notsup(fun crypto:block_encrypt/4, - [blowfish_cfb64, ?KEY, ?IVEC, ?DATA]), - ok. - -no_ofb64(doc) -> - "Test that OFB64 mode is disabled"; -no_ofb64(suite) -> - []; -no_ofb64(Config) when is_list(Config) -> - notsup(fun crypto:block_encrypt/4, - [blowfish_ofb64, ?KEY, ?IVEC, ?DATA]). - -%% Helper functions - -%% Assert function fails with notsup error -notsup(Fun, Args) -> - ok = try - {error, {return, apply(Fun, Args)}} - catch - error:notsup -> - ok; - Class:Error -> - {error, {Class, Error}} - end. - - -%% Convert a hexadecimal string to a binary. --spec(to_bin(L::string()) -> binary()). -to_bin(L) -> - to_bin(L, []). - -%% @spec dehex(char()) -> integer() -%% @doc Convert a hex digit to its integer value. --spec(dehex(char()) -> integer()). -dehex(C) when C >= $0, C =< $9 -> - C - $0; -dehex(C) when C >= $a, C =< $f -> - C - $a + 10; -dehex(C) when C >= $A, C =< $F -> - C - $A + 10. - --spec(to_bin(L::string(), list()) -> binary()). -to_bin([], Acc) -> - iolist_to_binary(lists:reverse(Acc)); -to_bin([C1, C2 | Rest], Acc) -> - to_bin(Rest, [(dehex(C1) bsl 4) bor dehex(C2) | Acc]). - -m(X,X) -> ok. diff --git a/lib/crypto/test/crypto.spec b/lib/crypto/test/crypto.spec index cc09970cb3..4a95275687 100644 --- a/lib/crypto/test/crypto.spec +++ b/lib/crypto/test/crypto.spec @@ -1 +1,6 @@ {suites,"../crypto_test",all}. + +{skip_suites, "../crypto_test", [crypto_bench_SUITE + ], + "Benchmarks run separately"}. + diff --git a/lib/crypto/test/crypto_SUITE.erl b/lib/crypto/test/crypto_SUITE.erl index 6c6188f775..cbfa96cc16 100644 --- a/lib/crypto/test/crypto_SUITE.erl +++ b/lib/crypto/test/crypto_SUITE.erl @@ -99,6 +99,8 @@ groups() -> {group, rsa}, {group, dss}, {group, ecdsa}, + {group, no_ed25519}, + {group, no_ed448}, {group, dh}, {group, ecdh}, {group, no_srp}, @@ -113,8 +115,8 @@ groups() -> {group, no_blowfish_cfb64}, {group, no_blowfish_ofb64}, {group, aes_cbc128}, - {group, aes_cfb8}, - {group, aes_cfb128}, + {group, no_aes_cfb8}, + {group, no_aes_cfb128}, {group, aes_cbc256}, {group, no_aes_ige256}, {group, no_rc2_cbc}, @@ -156,7 +158,7 @@ groups() -> ]}, {dh, [], [generate_compute, compute_bug]}, - {ecdh, [], [generate_all_supported, compute, generate]}, + {ecdh, [], [use_all_elliptic_curves, compute, generate]}, {srp, [], [generate_compute]}, {des_cbc, [], [block]}, {des_cfb, [], [block]}, @@ -183,8 +185,16 @@ groups() -> {chacha20, [], [stream]}, {poly1305, [], [poly1305]}, {aes_cbc, [], [block]}, + {no_aes_cfb8,[], [no_support, no_block]}, + {no_aes_cfb128,[], [no_support, no_block]}, {no_md4, [], [no_support, no_hash]}, {no_md5, [], [no_support, no_hash, no_hmac]}, + {no_ed25519, [], [no_support, no_sign_verify + %% Does not work yet: ,public_encrypt, private_encrypt + ]}, + {no_ed448, [], [no_support, no_sign_verify + %% Does not work yet: ,public_encrypt, private_encrypt + ]}, {no_ripemd160, [], [no_support, no_hash]}, {no_srp, [], [no_support, no_generate_compute]}, {no_des_cbc, [], [no_support, no_block]}, @@ -251,7 +261,7 @@ init_per_group(fips, Config) -> enabled = crypto:info_fips(), FIPSConfig; false -> - {skip, "Failed to enable FIPS mode"} + {fail, "Failed to enable FIPS mode"} end; not_supported -> {skip, "FIPS mode not supported"} @@ -401,17 +411,6 @@ block() -> block(Config) when is_list(Config) -> Fips = proplists:get_bool(fips, Config), Type = ?config(type, Config), - %% See comment about EVP_CIPHER_CTX_set_key_length in - %% block_crypt_nif in crypto.c. - case {Fips, Type} of - {true, aes_cfb8} -> - throw({skip, "Cannot test aes_cfb8 in FIPS mode because of key length issue"}); - {true, aes_cfb128} -> - throw({skip, "Cannot test aes_cfb128 in FIPS mode because of key length issue"}); - _ -> - ok - end, - Blocks = lazy_eval(proplists:get_value(block, Config)), lists:foreach(fun block_cipher/1, Blocks), lists:foreach(fun block_cipher/1, block_iolistify(Blocks)), @@ -500,6 +499,13 @@ sign_verify(Config) when is_list(Config) -> SignVerify = proplists:get_value(sign_verify, Config), lists:foreach(fun do_sign_verify/1, SignVerify). +%%-------------------------------------------------------------------- +no_sign_verify() -> + [{doc, "Test disabled sign/verify digital signatures"}]. +no_sign_verify(Config) when is_list(Config) -> + [SignVerifyHd|_] = proplists:get_value(sign_verify, Config), + notsup(fun do_sign_verify/1, [SignVerifyHd]). + %%-------------------------------------------------------------------- public_encrypt() -> [{doc, "Test public_encrypt/decrypt "}]. @@ -563,32 +569,43 @@ compute(Config) when is_list(Config) -> Gen = proplists:get_value(compute, Config), lists:foreach(fun do_compute/1, Gen). %%-------------------------------------------------------------------- -generate_all_supported() -> - [{doc, " Test that all curves from crypto:ec_curves/0 returns two binaries"}]. -generate_all_supported(_Config) -> +use_all_elliptic_curves() -> + [{doc, " Test that all curves from crypto:ec_curves/0"}]. +use_all_elliptic_curves(_Config) -> + Msg = <<"hello world!">>, + Sups = crypto:supports(), + Curves = proplists:get_value(curves, Sups), + Hashs = proplists:get_value(hashs, Sups), + ct:log("Lib: ~p~nFIPS: ~p~nCurves:~n~p~nHashs: ~p", [crypto:info_lib(), + crypto:info_fips(), + Curves, + Hashs]), Results = - [try - crypto:generate_key(ecdh, C) - of - {B1,B2} when is_binary(B1) and is_binary(B2) -> - %% That is, seems like it works as expected. - {ok,C}; - Err -> - ct:log("ERROR: Curve ~p generated ~p", [C,Err]), - {error,{C,Err}} - catch - Cls:Err:Stack -> - ct:log("ERROR: Curve ~p exception ~p:~p~n~p", [C,Cls,Err,Stack]), - {error,{C,{Cls,Err}}} - end - || C <- crypto:ec_curves(), - not lists:member(C, [ed25519, ed448]) + [{{Curve,Hash}, + try + {Pub,Priv} = crypto:generate_key(ecdh, Curve), + true = is_binary(Pub), + true = is_binary(Priv), + Sig = crypto:sign(ecdsa, Hash, Msg, [Priv, Curve]), + crypto:verify(ecdsa, Hash, Msg, Sig, [Pub, Curve]) + catch + C:E -> + {C,E} + end} + || Curve <- Curves -- [ed25519, ed448, x25519, x448, ipsec3, ipsec4], + Hash <- Hashs -- [md4, md5, ripemd160, sha3_224, sha3_256, sha3_384, sha3_512] ], - OK = [C || {ok,C} <- Results], - ct:log("Ok (len=~p): ~p", [length(OK), OK]), - false = lists:any(fun({error,_}) -> true; - (_) -> false - end, Results). + Fails = + lists:filter(fun({_,true}) -> false; + (_) -> true + end, Results), + case Fails of + [] -> + ok; + _ -> + ct:log("Fails:~n~p",[Fails]), + ct:fail("Bad curve(s)",[]) + end. %%-------------------------------------------------------------------- generate() -> diff --git a/lib/crypto/test/crypto_bench.spec b/lib/crypto/test/crypto_bench.spec new file mode 100644 index 0000000000..b9a26d94db --- /dev/null +++ b/lib/crypto/test/crypto_bench.spec @@ -0,0 +1,3 @@ +{suites, "../crypto_test", [ + crypto_bench_SUITE + ]}. diff --git a/lib/crypto/test/crypto_bench_SUITE.erl b/lib/crypto/test/crypto_bench_SUITE.erl new file mode 100644 index 0000000000..c66a27f0c8 --- /dev/null +++ b/lib/crypto/test/crypto_bench_SUITE.erl @@ -0,0 +1,400 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +%% +-module(crypto_bench_SUITE). + +%% Note: This directive should only be used in test suites. +-compile(export_all). + +-include_lib("common_test/include/ct_event.hrl"). +-include_lib("common_test/include/ct.hrl"). + +suite() -> [%%{ct_hooks,[{ts_install_cth,[{nodenames,2}]}]}, + {timetrap,{minutes,2}} + ]. + +all() -> + [ + {group, textblock_256} + ]. + +groups() -> + [ + {textblock_256, [], [ + {group, ciphers_128}, + {group, ciphers_256} + ]}, + + {ciphers_128, [{repeat, 5}], [ + block, + stream + ]}, + + {ciphers_256, [{repeat, 5}], [ + block, + stream, + chacha + ]} + ]. + +%%%---------------------------------------------------------------- +%%% +init_per_suite(Config0) -> + try crypto:start() of + _ -> + [{_,_,Info}] = crypto:info_lib(), + ct:comment("~s",[Info]), + ct:pal("Crypto version: ~p~n~n~p",[Info,crypto:supports()]), + Config1 = measure_openssl_aes_cbc([128,256], Config0), + calibrate([{sec_goal,10} | Config1]) + + catch _:_ -> + {fail, "Crypto did not start"} + end. + +end_per_suite(_Config) -> + application:stop(crypto). + +%%%---------------------------------------------------------------- +%%% +init_per_group(Group, Config) -> + case atom_to_list(Group) of + "ciphers_"++KeySizeStr -> + KeySize = list_to_integer(KeySizeStr), + [{key_size,KeySize} | Config]; + + "textblock_"++BlockSizeStr -> + BlockSize = list_to_integer(BlockSizeStr), + [{block_size,BlockSize} | Config]; + + _ -> + Config + end. + +end_per_group(_Group, Config) -> + Config. + + +measure_openssl_aes_cbc(KeySizes, Config) -> + BLno_acc = [baseline(aes_cbc, KeySize, false) || KeySize <- KeySizes], + ct:pal("Non-accelerated baseline encryption time [µs/block]:~n~p", [BLno_acc]), + BLacc = [baseline(aes_cbc, KeySize, true) || KeySize <- KeySizes], + ct:pal("Possibly accelerated baseline encryption time [µs/block]:~n~p", [BLacc]), + [{acc,BLacc}, + {no_acc,BLno_acc} | Config]. + +calibrate(Config) -> + Secs = proplists:get_value(sec_goal, Config, 10), + {_,Empty} = data(empty, 0, 0), + {Ne,Te} = run1(Secs*3000, Empty), + report(["Overhead"], Te/Ne), + [{overhead,Te/Ne} | Config]. + +%%%================================================================ +%%% +%%% +block(Config) -> + run_cryptos([aes_cbc, aes_gcm, aes_ccm], + Config). + +stream(Config) -> + run_cryptos([aes_ctr], + Config). + +chacha(Config) -> + run_cryptos([chacha20, chacha20_poly1305], + Config). + + +%%%================================================================ +%%% +%%% + +run_cryptos(Cryptos, Config) -> + KeySize = proplists:get_value(key_size, Config), + BlockSize = proplists:get_value(block_size, Config), + MilliSecGoal = 1000*proplists:get_value(sec_goal,Config), + OverHead = proplists:get_value(overhead, Config, 0), + [try + TimePerOpBrutto = run(Crypto,KeySize,BlockSize,MilliSecGoal), + %% ct:pal("Brutto: ~p Overhead: ~p (~.2f %) Netto: ~p", + %% [TimePerOpBrutto, OverHead, 100*OverHead/TimePerOpBrutto,TimePerOpBrutto - OverHead]), + TimePerOpBrutto - OverHead + of + TimePerOp -> % µs + %% First, Report speed of encrypting blocks of 1000. [blocks/sec] + ReportUnit = 1000, + Label = [fmt(Crypto)," key:",KeySize," block:",BlockSize], + report(Label, + (BlockSize/ReportUnit)*1000000/TimePerOp + ), + + EffCrypto = case Crypto of + X -> X + end, + %% Percent of accelerated speed + case find_value([acc,{EffCrypto,KeySize},BlockSize], Config) of + undefined -> + ok; + TimePerOpBaseAcc -> + report(["Percent of acc OpenSSL "|Label], + 100*TimePerOpBaseAcc/TimePerOp % Percent of base *speed* + ) + end, + + %% Percent of non-accelerated speed + case find_value([no_acc,{EffCrypto,KeySize},BlockSize], Config) of + undefined -> + ok; + TimePerOpBaseNoAcc -> + report(["Percent of noacc OpenSSL "|Label], + 100*TimePerOpBaseNoAcc/TimePerOp % Percent of base *speed* + ) + end + catch + _:_ -> + ct:pal("~p unsupported",[{Crypto,KeySize,BlockSize}]) + end + || Crypto <- Cryptos, + supported(Crypto) + ]. + + +run(Crypto, KeySize, BlockSize, MilliSecGoal) -> + {_Type, Funs} = data(Crypto, KeySize, BlockSize), + {Nc,Tc} = run1(MilliSecGoal, Funs), + Tc/Nc. + +fmt(X) -> X. + + +find_value(KeyPath, PropList, Default) -> + try find_value(KeyPath, PropList) + of + undefined -> Default + catch + error:function_clause -> Default + end. + +find_value(KeyPath, PropList) -> + lists:foldl(fun(K, L) when is_list(L) -> proplists:get_value(K,L); + (_, _) -> undefined + end, PropList, KeyPath). + +%%%================================================================ +%%% +%%% +funs({block, {Type, Key, IV, Block}}) -> + {fun() -> ok end, + fun(_) -> crypto:block_encrypt(Type, Key, IV, Block) end, + fun(_) -> ok end}; + +funs({stream, {Type, Key, IV, Block}}) -> + {fun() -> {crypto:stream_init(Type, Key, IV),ok} end, + fun({Ctx,_}) -> crypto:stream_encrypt(Ctx, Block) end, + fun(_) -> ok end}. + + +data(aes_cbc, KeySize, BlockSize) -> + Type = case KeySize of + 128 -> aes_cbc128; + 256 -> aes_cbc256 + end, + Key = mk_bin(KeySize div 8), + IV = mk_bin(16), + Block = mk_bin(BlockSize), + {Type, funs({block, {Type, Key, IV, Block}})}; + +data(aes_gcm, KeySize, BlockSize) -> + Type = aes_gcm, + Key = mk_bin(KeySize div 8), + IV = mk_bin(12), + Block = mk_bin(BlockSize), + AAD = <<01,02,03,04>>, + {Type, funs({block, {Type, Key, IV, {AAD,Block,16}}})}; + +data(aes_ccm, KeySize, BlockSize) -> + Type = aes_ccm, + Key = mk_bin(KeySize div 8), + IV = mk_bin(12), + Block = mk_bin(BlockSize), + AAD = <<01,02,03,04>>, + {Type, funs({block, {Type, Key, IV, {AAD,Block,12}}})}; + +data(aes_ctr, KeySize, BlockSize) -> + Type = aes_ctr, + Key = mk_bin(KeySize div 8), + IV = mk_bin(16), + Block = mk_bin(BlockSize), + {Type, funs({stream, {Type, Key, IV, Block}})}; + +data(chacha20_poly1305, 256=KeySize, BlockSize) -> + Type = chacha20_poly1305, + Key = mk_bin(KeySize div 8), + IV = mk_bin(16), + AAD = <<01,02,03,04>>, + Block = mk_bin(BlockSize), + {Type, funs({block, {Type, Key, IV, {AAD,Block}}})}; + +data(chacha20, 256=KeySize, BlockSize) -> + Type = chacha20, + Key = mk_bin(KeySize div 8), + IV = mk_bin(16), + Block = mk_bin(BlockSize), + {Type, funs({stream, {Type, Key, IV, Block}})}; + +data(empty, 0, 0) -> + {undefined, + {fun() -> ok end, + fun(X) -> X end, + fun(_) -> ok end}}. + +%%%================================================================ +%%% +%%% +run1(MilliSecGoal, Funs) -> + Parent = self(), + Pid = spawn(fun() -> + {Fi,Fu,Ff} = Funs, + Ctx0 = Fi(), + erlang:garbage_collect(), + T0 = start_time(), + {N,Ctx} = loop(Fu, Ctx0, 0), + T = elapsed_time(T0), + Ff(Ctx), + Parent ! {result,N,microseconds(T)} + end), + Pid ! go, + receive + after MilliSecGoal -> + Pid ! stop + end, + receive + {result,N,MicroSecs} -> + {N,MicroSecs} + end. + + +loop(F, Ctx, N) -> + receive + stop -> + {N, Ctx} + after 0 -> + loop(F, F(Ctx), N+1) + end. + +%%%---------------------------------------------------------------- +report(LabelList, Value) -> + Label = report_chars(lists:concat(LabelList)), + ct:pal("ct_event:notify ~p: ~p", [Label, Value]), + ct_event:notify( + #event{name = benchmark_data, + data = [{name, Label}, + {value,Value}]}). + +report_chars(Cs) -> + [case C of + $- -> $_; + _ -> C + end || C <- Cs]. + +%%%---------------------------------------------------------------- +supported(Algorithm) -> + lists:member(Algorithm, + [A || {_,As} <- crypto:supports(), A <- As] + ). + +%%%---------------------------------------------------------------- +start_time() -> + erlang:system_time(). + +elapsed_time(StartTime) -> + erlang:system_time() - StartTime. + +microseconds(Time) -> + erlang:convert_time_unit(Time, native, microsecond). + +%%%---------------------------------------------------------------- + +%% Example output: +%% +DT:aes-128-cbc:3:16 +%% +R:135704772:aes-128-cbc:2.980000 +%% +DT:aes-128-cbc:3:64 +%% +R:36835089:aes-128-cbc:3.000000 +%% +DT:aes-128-cbc:3:256 +%% +R:9398616:aes-128-cbc:3.000000 +%% +DT:aes-128-cbc:3:1024 +%% +R:2355683:aes-128-cbc:2.990000 +%% +DT:aes-128-cbc:3:8192 +%% +R:294508:aes-128-cbc:2.990000 +%% +H:16:64:256:1024:8192 +%% +F:22:aes-128-cbc:728616225.50:785815232.00:802015232.00:806762338.46:806892821.40 + +baseline(Crypto, KeySize, EVP) -> + Spec= + case {Crypto,KeySize} of + {aes_cbc, 128} -> "aes-128-cbc"; + {aes_cbc, 256} -> "aes-256-cbc" + end, + {{Crypto,KeySize}, baseline(Spec, EVP)}. + +baseline(Spec, EVP) -> + Cmd = + case EVP of + true -> "openssl speed -mr -evp " ++ Spec; + false-> "openssl speed -mr " ++ Spec + end, + get_base_values(string:tokens(os:cmd(Cmd),"\n"), Spec, []). + + +get_base_values(["+DT:"++Sdt, + "+R:"++Sr + |T], Crypto, Acc) -> + [Crypto0,_GoalSecs0,BlockSize0] = string:tokens(Sdt, ":"), + [Nblocks0,Crypto0,RealSecs0] = string:tokens(Sr, ":"), + Crypto = fix_possible_space_bug(Crypto0), + RealSecs = list_to_float(RealSecs0), + BlockSize = list_to_integer(BlockSize0), + Nblocks = list_to_integer(Nblocks0), + get_base_values(T, Crypto, [{BlockSize, 1000000*RealSecs/Nblocks} | Acc]); + +get_base_values([_|T], Crypto, Acc) -> + get_base_values(T, Crypto, Acc); + +get_base_values([], _, Acc) -> + lists:sort(Acc). + +fix_possible_space_bug(S) -> lists:concat(lists:join("-",string:tokens(S,"- "))). + +%%%---------------------------------------------------------------- +mk_bin(Size) when Size =< 256 -> + list_to_binary(lists:seq(0,Size-1)); + +mk_bin(Size) when 1024 =< Size -> + B = mk_bin(Size div 4), + Brest = mk_bin(Size rem 4), + <<B/binary, B/binary, B/binary, B/binary, Brest/binary>>; + +mk_bin(Size) when 256 < Size -> + B = mk_bin(Size div 2), + Brest = mk_bin(Size rem 2), + <<B/binary, B/binary, Brest/binary>>. + diff --git a/lib/crypto/test/engine_SUITE.erl b/lib/crypto/test/engine_SUITE.erl index 8a45fc9076..869db516b4 100644 --- a/lib/crypto/test/engine_SUITE.erl +++ b/lib/crypto/test/engine_SUITE.erl @@ -345,13 +345,13 @@ engine_list(Config) when is_list(Config) -> {skip, "OTP Test engine not found"}; {ok, Engine} -> try - EngineList0 = crypto:engine_list(), case crypto:engine_load(<<"dynamic">>, [{<<"SO_PATH">>, Engine}, <<"LOAD">>], []) of {ok, E} -> EngineList0 = crypto:engine_list(), + false = lists:member(<<"MD5">>, EngineList0), ok = crypto:engine_add(E), [<<"MD5">>] = lists:subtract(crypto:engine_list(), EngineList0), ok = crypto:engine_remove(E), diff --git a/lib/dialyzer/src/dialyzer_codeserver.erl b/lib/dialyzer/src/dialyzer_codeserver.erl index 5587cf2bdf..c4e3c322e5 100644 --- a/lib/dialyzer/src/dialyzer_codeserver.erl +++ b/lib/dialyzer/src/dialyzer_codeserver.erl @@ -347,13 +347,11 @@ get_file_contract(Key, ContDict) -> lookup_mfa_contract(MFA, #codeserver{contracts = ContDict}) -> ets_dict_find(MFA, ContDict). --spec lookup_meta_info(module() | mfa(), codeserver()) -> meta_info(). +-spec lookup_meta_info(module() | mfa(), codeserver()) -> + {'ok', meta_info()} | 'error'. lookup_meta_info(MorMFA, #codeserver{fun_meta_info = FunMetaInfo}) -> - case ets_dict_find(MorMFA, FunMetaInfo) of - error -> []; - {ok, PropList} -> PropList - end. + ets_dict_find(MorMFA, FunMetaInfo). -spec get_contracts(codeserver()) -> dict:dict(mfa(), dialyzer_contracts:file_contract()). diff --git a/lib/dialyzer/src/dialyzer_contracts.erl b/lib/dialyzer/src/dialyzer_contracts.erl index af7f4385ad..9c36d745c3 100644 --- a/lib/dialyzer/src/dialyzer_contracts.erl +++ b/lib/dialyzer/src/dialyzer_contracts.erl @@ -25,7 +25,7 @@ %% get_contract_signature/1, is_overloaded/1, process_contract_remote_types/1, - store_tmp_contract/5]). + store_tmp_contract/6]). -export_type([file_contract/0, plt_contracts/0]). @@ -146,18 +146,18 @@ process_contract_remote_types(CodeServer) -> Mods = dialyzer_codeserver:all_temp_modules(CodeServer), RecordTable = dialyzer_codeserver:get_records_table(CodeServer), ExpTypes = dialyzer_codeserver:get_exported_types(CodeServer), - ContractFun = - fun({{_M, _F, _A}=MFA, {File, TmpContract, Xtra}}, C0) -> - #tmp_contract{contract_funs = CFuns, forms = Forms} = TmpContract, - {NewCs, C2} = lists:mapfoldl(fun(CFun, C1) -> - CFun(ExpTypes, RecordTable, C1) - end, C0, CFuns), - Args = general_domain(NewCs), - Contract = #contract{contracts = NewCs, args = Args, forms = Forms}, - {{MFA, {File, Contract, Xtra}}, C2} - end, ModuleFun = fun(ModuleName) -> + ContractFun = + fun({MFA, {File, TmpContract, Xtra}}, C0) -> + #tmp_contract{contract_funs = CFuns, forms = Forms} = TmpContract, + {NewCs, C2} = lists:mapfoldl(fun(CFun, C1) -> + CFun(ExpTypes, RecordTable, C1) + end, C0, CFuns), + Args = general_domain(NewCs), + Contract = #contract{contracts = NewCs, args = Args, forms = Forms}, + {{MFA, {File, Contract, Xtra}}, C2} + end, Cache = erl_types:cache__new(), {ContractMap, CallbackMap} = dialyzer_codeserver:get_temp_contracts(ModuleName, CodeServer), @@ -474,26 +474,29 @@ insert_constraints([], Map) -> Map. -type spec_data() :: {TypeSpec :: [_], Xtra:: [_]}. --spec store_tmp_contract(mfa(), file_line(), spec_data(), contracts(), types()) -> - contracts(). +-spec store_tmp_contract(module(), mfa(), file_line(), spec_data(), + contracts(), types()) -> contracts(). -store_tmp_contract(MFA, FileLine, {TypeSpec, Xtra}, SpecMap, RecordsDict) -> +store_tmp_contract(Module, MFA, FileLine, {TypeSpec, Xtra}, SpecMap, + RecordsDict) -> %% io:format("contract from form: ~tp\n", [TypeSpec]), - TmpContract = contract_from_form(TypeSpec, MFA, RecordsDict, FileLine), + TmpContract = contract_from_form(TypeSpec, Module, MFA, RecordsDict, FileLine), %% io:format("contract: ~tp\n", [TmpContract]), maps:put(MFA, {FileLine, TmpContract, Xtra}, SpecMap). -contract_from_form(Forms, MFA, RecDict, FileLine) -> - {CFuns, Forms1} = contract_from_form(Forms, MFA, RecDict, FileLine, [], []), +contract_from_form(Forms, Module, MFA, RecDict, FileLine) -> + {CFuns, Forms1} = + contract_from_form(Forms, Module, MFA, RecDict, FileLine, [], []), #tmp_contract{contract_funs = CFuns, forms = Forms1}. -contract_from_form([{type, _, 'fun', [_, _]} = Form | Left], MFA, RecDict, - FileLine, TypeAcc, FormAcc) -> +contract_from_form([{type, _, 'fun', [_, _]} = Form | Left], Module, MFA, + RecDict, FileLine, TypeAcc, FormAcc) -> TypeFun = fun(ExpTypes, RecordTable, Cache) -> {NewType, NewCache} = try - from_form_with_check(Form, ExpTypes, MFA, RecordTable, Cache) + from_form_with_check(Form, ExpTypes, Module, MFA, RecordTable, + Cache) catch throw:{error, Msg} -> {File, Line} = FileLine, @@ -506,68 +509,74 @@ contract_from_form([{type, _, 'fun', [_, _]} = Form | Left], MFA, RecDict, end, NewTypeAcc = [TypeFun | TypeAcc], NewFormAcc = [{Form, []} | FormAcc], - contract_from_form(Left, MFA, RecDict, FileLine, NewTypeAcc, NewFormAcc); + contract_from_form(Left, Module, MFA, RecDict, FileLine, NewTypeAcc, + NewFormAcc); contract_from_form([{type, _L1, bounded_fun, [{type, _L2, 'fun', [_, _]} = Form, Constr]}| Left], - MFA, RecDict, FileLine, TypeAcc, FormAcc) -> + Module, MFA, RecDict, FileLine, TypeAcc, FormAcc) -> TypeFun = fun(ExpTypes, RecordTable, Cache) -> {Constr1, VarTable, Cache1} = - process_constraints(Constr, MFA, RecDict, ExpTypes, RecordTable, - Cache), + process_constraints(Constr, Module, MFA, RecDict, ExpTypes, + RecordTable, Cache), {NewType, NewCache} = - from_form_with_check(Form, ExpTypes, MFA, RecordTable, + from_form_with_check(Form, ExpTypes, Module, MFA, RecordTable, VarTable, Cache1), NewTypeNoVars = erl_types:subst_all_vars_to_any(NewType), {{NewTypeNoVars, Constr1}, NewCache} end, NewTypeAcc = [TypeFun | TypeAcc], NewFormAcc = [{Form, Constr} | FormAcc], - contract_from_form(Left, MFA, RecDict, FileLine, NewTypeAcc, NewFormAcc); -contract_from_form([], _MFA, _RecDict, _FileLine, TypeAcc, FormAcc) -> + contract_from_form(Left, Module, MFA, RecDict, FileLine, NewTypeAcc, + NewFormAcc); +contract_from_form([], _Mod, _MFA, _RecDict, _FileLine, TypeAcc, FormAcc) -> {lists:reverse(TypeAcc), lists:reverse(FormAcc)}. -process_constraints(Constrs, MFA, RecDict, ExpTypes, RecordTable, Cache) -> - {Init0, NewCache} = initialize_constraints(Constrs, MFA, RecDict, ExpTypes, - RecordTable, Cache), +process_constraints(Constrs, Module, MFA, RecDict, ExpTypes, RecordTable, + Cache) -> + {Init0, NewCache} = initialize_constraints(Constrs, Module, MFA, RecDict, + ExpTypes, RecordTable, Cache), Init = remove_cycles(Init0), - constraints_fixpoint(Init, MFA, RecDict, ExpTypes, RecordTable, NewCache). + constraints_fixpoint(Init, Module, MFA, RecDict, ExpTypes, RecordTable, + NewCache). -initialize_constraints(Constrs, MFA, RecDict, ExpTypes, RecordTable, Cache) -> - initialize_constraints(Constrs, MFA, RecDict, ExpTypes, RecordTable, +initialize_constraints(Constrs, Module, MFA, RecDict, ExpTypes, RecordTable, + Cache) -> + initialize_constraints(Constrs, Module, MFA, RecDict, ExpTypes, RecordTable, Cache, []). -initialize_constraints([], _MFA, _RecDict, _ExpTypes, _RecordTable, +initialize_constraints([], _Module, _MFA, _RecDict, _ExpTypes, _RecordTable, Cache, Acc) -> {Acc, Cache}; -initialize_constraints([Constr|Rest], MFA, RecDict, ExpTypes, RecordTable, - Cache, Acc) -> +initialize_constraints([Constr|Rest], Module, MFA, RecDict, ExpTypes, + RecordTable, Cache, Acc) -> case Constr of {type, _, constraint, [{atom, _, is_subtype}, [Type1, Type2]]} -> VarTable = erl_types:var_table__new(), {T1, NewCache} = - final_form(Type1, ExpTypes, MFA, RecordTable, VarTable, Cache), + final_form(Type1, ExpTypes, Module, MFA, RecordTable, VarTable, Cache), Entry = {T1, Type2}, - initialize_constraints(Rest, MFA, RecDict, ExpTypes, RecordTable, - NewCache, [Entry|Acc]); + initialize_constraints(Rest, Module, MFA, RecDict, ExpTypes, + RecordTable, NewCache, [Entry|Acc]); {type, _, constraint, [{atom,_,Name}, List]} -> N = length(List), throw({error, io_lib:format("Unsupported type guard ~tw/~w\n", [Name, N])}) end. -constraints_fixpoint(Constrs, MFA, RecDict, ExpTypes, RecordTable, Cache) -> +constraints_fixpoint(Constrs, Module, MFA, RecDict, ExpTypes, RecordTable, + Cache) -> VarTable = erl_types:var_table__new(), {VarTab, NewCache} = - constraints_to_dict(Constrs, MFA, RecDict, ExpTypes, RecordTable, + constraints_to_dict(Constrs, Module, MFA, RecDict, ExpTypes, RecordTable, VarTable, Cache), - constraints_fixpoint(VarTab, MFA, Constrs, RecDict, ExpTypes, + constraints_fixpoint(VarTab, Module, MFA, Constrs, RecDict, ExpTypes, RecordTable, NewCache). -constraints_fixpoint(OldVarTab, MFA, Constrs, RecDict, ExpTypes, +constraints_fixpoint(OldVarTab, Module, MFA, Constrs, RecDict, ExpTypes, RecordTable, Cache) -> {NewVarTab, NewCache} = - constraints_to_dict(Constrs, MFA, RecDict, ExpTypes, RecordTable, + constraints_to_dict(Constrs, Module, MFA, RecDict, ExpTypes, RecordTable, OldVarTab, Cache), case NewVarTab of OldVarTab -> @@ -578,19 +587,23 @@ constraints_fixpoint(OldVarTab, MFA, Constrs, RecDict, ExpTypes, FinalConstrs = maps:fold(Fun, [], NewVarTab), {FinalConstrs, NewVarTab, NewCache}; _Other -> - constraints_fixpoint(NewVarTab, MFA, Constrs, RecDict, ExpTypes, + constraints_fixpoint(NewVarTab, Module, MFA, Constrs, RecDict, ExpTypes, RecordTable, NewCache) end. -final_form(Form, ExpTypes, MFA, RecordTable, VarTable, Cache) -> - from_form_with_check(Form, ExpTypes, MFA, RecordTable, VarTable, Cache). +final_form(Form, ExpTypes, Module, MFA, RecordTable, VarTable, Cache) -> + from_form_with_check(Form, ExpTypes, Module, MFA, RecordTable, VarTable, + Cache). -from_form_with_check(Form, ExpTypes, MFA, RecordTable, Cache) -> +from_form_with_check(Form, ExpTypes, Module, MFA, RecordTable, Cache) -> VarTable = erl_types:var_table__new(), - from_form_with_check(Form, ExpTypes, MFA, RecordTable, VarTable, Cache). + from_form_with_check(Form, ExpTypes, Module, MFA, RecordTable, VarTable, + Cache). -from_form_with_check(Form, ExpTypes, MFA, RecordTable, VarTable, Cache) -> - Site = {spec, MFA}, +from_form_with_check(Form, ExpTypes, Module, MFA, RecordTable, VarTable, + Cache) -> + {_, F, A} = MFA, + Site = {spec, {Module, F, A}}, C1 = erl_types:t_check_record_fields(Form, ExpTypes, Site, RecordTable, VarTable, Cache), %% The check costs some time, and with the assumption that contracts @@ -598,22 +611,22 @@ from_form_with_check(Form, ExpTypes, MFA, RecordTable, VarTable, Cache) -> %% erl_types:t_from_form_check_remote(Form, ExpTypes, MFA, RecordTable), erl_types:t_from_form(Form, ExpTypes, Site, RecordTable, VarTable, C1). -constraints_to_dict(Constrs, MFA, RecDict, ExpTypes, RecordTable, +constraints_to_dict(Constrs, Module, MFA, RecDict, ExpTypes, RecordTable, VarTab, Cache) -> {Subtypes, NewCache} = - constraints_to_subs(Constrs, MFA, RecDict, ExpTypes, RecordTable, + constraints_to_subs(Constrs, Module, MFA, RecDict, ExpTypes, RecordTable, VarTab, Cache, []), {insert_constraints(Subtypes), NewCache}. -constraints_to_subs([], _MFA, _RecDict, _ExpTypes, _RecordTable, +constraints_to_subs([], _Module, _MFA, _RecDict, _ExpTypes, _RecordTable, _VarTab, Cache, Acc) -> {Acc, Cache}; -constraints_to_subs([{T1, Form2}|Rest], MFA, RecDict, ExpTypes, RecordTable, - VarTab, Cache, Acc) -> +constraints_to_subs([{T1, Form2}|Rest], Module, MFA, RecDict, ExpTypes, + RecordTable, VarTab, Cache, Acc) -> {T2, NewCache} = - final_form(Form2, ExpTypes, MFA, RecordTable, VarTab, Cache), + final_form(Form2, ExpTypes, Module, MFA, RecordTable, VarTab, Cache), NewAcc = [{subtype, T1, T2}|Acc], - constraints_to_subs(Rest, MFA, RecDict, ExpTypes, RecordTable, + constraints_to_subs(Rest, Module, MFA, RecDict, ExpTypes, RecordTable, VarTab, NewCache, NewAcc). %% Replaces variables with '_' when necessary to break up cycles among @@ -898,6 +911,7 @@ is_remote_types_related(Contract, CSig, Sig, MFA, RecDict) -> t_from_forms_without_remote([{FType, []}], MFA, RecDict) -> Site = {spec, MFA}, + %% FIXME Type1 = erl_types:t_from_form_without_remote(FType, Site, RecDict), {ok, erl_types:subst_all_vars_to_any(Type1)}; t_from_forms_without_remote([{_FType, _Constrs}], _MFA, _RecDict) -> diff --git a/lib/dialyzer/src/dialyzer_utils.erl b/lib/dialyzer/src/dialyzer_utils.erl index abd89034f3..310301ee0b 100644 --- a/lib/dialyzer/src/dialyzer_utils.erl +++ b/lib/dialyzer/src/dialyzer_utils.erl @@ -450,8 +450,9 @@ get_spec_info([{Contract, Ln, [{Id, TypeSpec}]}|Left], error -> SpecData = {TypeSpec, Xtra}, NewActiveMap = - dialyzer_contracts:store_tmp_contract(MFA, {File, Ln}, SpecData, - ActiveMap, RecordsMap), + dialyzer_contracts:store_tmp_contract(ModName, MFA, {File, Ln}, + SpecData, ActiveMap, + RecordsMap), {NewSpecMap, NewCallbackMap} = case Contract of spec -> {NewActiveMap, CallbackMap}; @@ -599,24 +600,32 @@ collect_attribute([], _Tag, _File) -> -spec is_suppressed_fun(mfa(), codeserver()) -> boolean(). is_suppressed_fun(MFA, CodeServer) -> - lookup_fun_property(MFA, nowarn_function, CodeServer). + lookup_fun_property(MFA, nowarn_function, CodeServer, false). -spec is_suppressed_tag(mfa() | module(), dial_warn_tag(), codeserver()) -> boolean(). is_suppressed_tag(MorMFA, Tag, Codeserver) -> - not lookup_fun_property(MorMFA, Tag, Codeserver). - -lookup_fun_property({M, _F, _A}=MFA, Property, CodeServer) -> - MFAPropList = dialyzer_codeserver:lookup_meta_info(MFA, CodeServer), - case proplists:get_value(Property, MFAPropList, no) of - mod -> false; % suppressed in function - func -> true; % requested in function - no -> lookup_fun_property(M, Property, CodeServer) + not lookup_fun_property(MorMFA, Tag, Codeserver, true). + +lookup_fun_property({M, _F, _A}=MFA, Property, CodeServer, NoInfoReturn) -> + case dialyzer_codeserver:lookup_meta_info(MFA, CodeServer) of + error -> + lookup_fun_property(M, Property, CodeServer, NoInfoReturn); + {ok, MFAPropList} -> + case proplists:get_value(Property, MFAPropList, no) of + mod -> false; % suppressed in function + func -> true; % requested in function + no -> lookup_fun_property(M, Property, CodeServer, NoInfoReturn) + end end; -lookup_fun_property(M, Property, CodeServer) when is_atom(M) -> - MPropList = dialyzer_codeserver:lookup_meta_info(M, CodeServer), - proplists:is_defined(Property, MPropList). +lookup_fun_property(M, Property, CodeServer, NoInfoReturn) when is_atom(M) -> + case dialyzer_codeserver:lookup_meta_info(M, CodeServer) of + error -> + NoInfoReturn; + {ok, MPropList} -> + proplists:is_defined(Property, MPropList) + end. %% ============================================================================ %% diff --git a/lib/dialyzer/test/opaque_SUITE_data/results/para b/lib/dialyzer/test/opaque_SUITE_data/results/para index 37b5b7b44e..eca445315c 100644 --- a/lib/dialyzer/test/opaque_SUITE_data/results/para +++ b/lib/dialyzer/test/opaque_SUITE_data/results/para @@ -29,5 +29,6 @@ para4.erl:74: Attempt to test for equality between a term of type para4_adt:int( para4.erl:79: Attempt to test for equality between a term of type para4_adt:int(2 | 3 | 4) and a term of opaque type para4_adt:int(5 | 6 | 7) para4.erl:84: Attempt to test for equality between a term of type para4_adt:un(3 | 4) and a term of opaque type para4_adt:un(1 | 2) para4.erl:89: Attempt to test for equality between a term of type para4_adt:tup({_,_}) and a term of opaque type para4_adt:tup(tuple()) +para4.erl:94: Attempt to test for equality between a term of type para4_adt:t(#{1=>'a'}) and a term of opaque type para4_adt:t(#{2=>'b'}) para5.erl:13: Attempt to test for inequality between a term of type para5_adt:dd(atom()) and a term of opaque type para5_adt:d() para5.erl:8: The test para5_adt:d() =:= para5_adt:d() can never evaluate to 'true' diff --git a/lib/dialyzer/test/opaque_SUITE_data/src/para/para4.erl b/lib/dialyzer/test/opaque_SUITE_data/src/para/para4.erl index b9794672a9..8cd049169d 100644 --- a/lib/dialyzer/test/opaque_SUITE_data/src/para/para4.erl +++ b/lib/dialyzer/test/opaque_SUITE_data/src/para/para4.erl @@ -88,6 +88,11 @@ adt_tt13() -> I2 = adt_tup2(), I1 =:= I2. % opaque attempt +adt_tt14() -> + I1 = adt_map(), + I2 = adt_map2(), + I1 =:= I2. + y3() -> {a, 3}. @@ -132,3 +137,9 @@ adt_tup() -> adt_tup2() -> para4_adt:tup2(). + +adt_map() -> + para4_adt:map(). + +adt_map2() -> + para4_adt:map2(). diff --git a/lib/dialyzer/test/opaque_SUITE_data/src/para/para4_adt.erl b/lib/dialyzer/test/opaque_SUITE_data/src/para/para4_adt.erl index 407dd198a7..06a6c22677 100644 --- a/lib/dialyzer/test/opaque_SUITE_data/src/para/para4_adt.erl +++ b/lib/dialyzer/test/opaque_SUITE_data/src/para/para4_adt.erl @@ -8,6 +8,8 @@ -export([tup/0, tup2/0]). +-export([map/0, map2/0]). + -export_type([t/1, y/1, int/1, tup/1, un/1]). -type ai() :: atom() | integer(). @@ -106,3 +108,13 @@ tup() -> tup2() -> foo:tup2(). + +-spec map() -> t(#{2 => b}). + +map() -> + foo:map(). + +-spec map2() -> t(#{1 => a}). + +map2() -> + foo:map2(). diff --git a/lib/dialyzer/test/small_SUITE_data/results/spec_other_module b/lib/dialyzer/test/small_SUITE_data/results/spec_other_module new file mode 100644 index 0000000000..ab2e35cf55 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/results/spec_other_module @@ -0,0 +1,2 @@ + +spec_other_module.erl:7: Contract for function that does not exist: lists:flatten/1 diff --git a/lib/dialyzer/test/small_SUITE_data/src/lists_key_bug.erl b/lib/dialyzer/test/small_SUITE_data/src/lists_key_bug.erl new file mode 100644 index 0000000000..ad5cf3c503 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/lists_key_bug.erl @@ -0,0 +1,75 @@ +-module(lists_key_bug). + +%% OTP-15570 + +-export([is_1/1, is_2/1, i/1, t1/0, t2/0, im/0]). + +%% int_set([3]) +is_1(V) -> + K = ikey(V), + case lists:keyfind(K, 1, [{<<"foo">>, bar}]) of + false -> + a; + {_, _} -> + b + end. + +ikey(1) -> + 3; +ikey(2) -> + <<"foo">>. + +%% int_set([3, 5]) +is_2(V) -> + K = iskey(V), + case lists:keyfind(K, 1, [{<<"foo">>, bar}]) of + false -> + a; + {_, _} -> + b + end. + +iskey(1) -> + 12; +iskey(2) -> + 14; +iskey(3) -> + <<"foo">>. + +%% integer() +i(V) -> + K = intkey(V), + case lists:keyfind(K, 1, [{9.0, foo}]) of + false -> + a; + {_, _} -> + b + end. + +intkey(K) when is_integer(K) -> + K + 9999. + +t1() -> + case lists:keyfind({17}, 1, [{{17.0}, true}]) of + false -> + a; + {_, _} -> + b + end. + +t2() -> + case lists:keyfind({17.0}, 1, [{{17}, true}]) of + false -> + a; + {_, _} -> + b + end. + +%% Note: #{1.0 => a} =/= #{1 => a}. +im() -> + case lists:keyfind(#{1.0 => a}, 1, [{#{1 => a}, foo}]) of + false -> + a; + {_, _} -> + b + end. diff --git a/lib/dialyzer/test/small_SUITE_data/src/spec_other_module.erl b/lib/dialyzer/test/small_SUITE_data/src/spec_other_module.erl new file mode 100644 index 0000000000..b36742b1bd --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/spec_other_module.erl @@ -0,0 +1,7 @@ +-module(spec_other_module). + +%% OTP-15562 and ERL-845. Example provided by Kostis. + +-type deep_list(A) :: [A | deep_list(A)]. + +-spec lists:flatten(deep_list(A)) -> [A]. diff --git a/lib/diameter/doc/src/notes.xml b/lib/diameter/doc/src/notes.xml index 4bfc98de40..cc92bd99f0 100644 --- a/lib/diameter/doc/src/notes.xml +++ b/lib/diameter/doc/src/notes.xml @@ -78,6 +78,24 @@ first.</p> </section> +<section><title>diameter 2.1.4.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix failure of incoming answer message with faulty + Experimental-Result-Code. Failure to decode the AVP + resulted in an uncaught exception, with no no + handle_answer/error callback as a consequence.</p> + <p> + Own Id: OTP-15569 Aux Id: ERIERL-302 </p> + </item> + </list> + </section> + +</section> + <section><title>diameter 2.1.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/diameter/src/base/diameter_gen.erl b/lib/diameter/src/base/diameter_gen.erl index d110a3015e..564448de48 100644 --- a/lib/diameter/src/base/diameter_gen.erl +++ b/lib/diameter/src/base/diameter_gen.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2018. All Rights Reserved. +%% Copyright Ericsson AB 2010-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/diameter/src/base/diameter_traffic.erl b/lib/diameter/src/base/diameter_traffic.erl index d2856ae530..2d3e4a2ac9 100644 --- a/lib/diameter/src/base/diameter_traffic.erl +++ b/lib/diameter/src/base/diameter_traffic.erl @@ -1925,6 +1925,8 @@ get_avp(Dict, Name, [#diameter_header{} | Avps]) -> A = find_avp(Code, Vid, Avps), avp_decode(Dict, Name, ungroup(A)) catch + {diameter_gen, _} -> %% faulty Grouped AVP + undefined; error: _ -> undefined end; diff --git a/lib/diameter/src/diameter.appup.src b/lib/diameter/src/diameter.appup.src index 51830f5276..4e6b983bac 100644 --- a/lib/diameter/src/diameter.appup.src +++ b/lib/diameter/src/diameter.appup.src @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2018. All Rights Reserved. +%% Copyright Ericsson AB 2010-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -59,6 +59,7 @@ {"2.1.2", [{restart_application, diameter}]}, %% 20.1.3 {"2.1.3", [{restart_application, diameter}]}, %% 20.2 {"2.1.4", [{restart_application, diameter}]}, %% 20.3 + {"2.1.4.1", [{restart_application, diameter}]}, %% 20.3.8.19 {"2.1.5", [{update, diameter_peer_fsm}]} %% 21.0 ], [ @@ -100,6 +101,7 @@ {"2.1.2", [{restart_application, diameter}]}, {"2.1.3", [{restart_application, diameter}]}, {"2.1.4", [{restart_application, diameter}]}, + {"2.1.4.1", [{restart_application, diameter}]}, {"2.1.5", [{update, diameter_peer_fsm}]} ] }. diff --git a/lib/diameter/src/transport/diameter_tcp.erl b/lib/diameter/src/transport/diameter_tcp.erl index da059fa7d6..e5e766d2a0 100644 --- a/lib/diameter/src/transport/diameter_tcp.erl +++ b/lib/diameter/src/transport/diameter_tcp.erl @@ -92,9 +92,9 @@ -type connect_option() :: {raddr, inet:ip_address()} | {rport, pos_integer()} - | {ssl_options, true | [ssl:connect_option()]} + | {ssl_options, true | [ssl:tls_client_option()]} | option() - | ssl:connect_option() + | ssl:tls_client_option() | gen_tcp:connect_option(). -type match() :: inet:ip_address() @@ -102,9 +102,9 @@ | [match()]. -type listen_option() :: {accept, match()} - | {ssl_options, true | [ssl:listen_option()]} + | {ssl_options, true | [ssl:tls_server_option()]} | option() - | ssl:listen_option() + | ssl:tls_server_option() | gen_tcp:listen_option(). -type option() :: {port, non_neg_integer()} diff --git a/lib/edoc/src/edoc.erl b/lib/edoc/src/edoc.erl index b641118c5d..e9d62d3283 100644 --- a/lib/edoc/src/edoc.erl +++ b/lib/edoc/src/edoc.erl @@ -578,7 +578,7 @@ read_source(Name, Opts0) -> Opts = expand_opts(Opts0), case read_source_1(Name, Opts) of {ok, Forms} -> - check_forms(Forms, Name), + check_forms(Forms, Name, Opts), Forms; {error, R} -> edoc_report:error({"error reading file '~ts'.", @@ -692,13 +692,19 @@ fll([T | L], LastLine, Ts) -> fll(L, _LastLine, Ts) -> lists:reverse(L, Ts). -check_forms(Fs, Name) -> +check_forms(Fs, Name, Opts) -> Fun = fun (F) -> case erl_syntax:type(F) of error_marker -> case erl_syntax:error_marker_info(F) of {L, M, D} -> - edoc_report:error(L, Name, {format_error, M, D}); + edoc_report:error(L, Name, {format_error, M, D}), + case proplists:get_bool(preprocess, Opts) of + true -> + ok; + false -> + helpful_message(Name) + end; Other -> edoc_report:report(Name, "unknown error in " "source code: ~w.", [Other]) @@ -710,6 +716,11 @@ check_forms(Fs, Name) -> end, lists:foreach(Fun, Fs). +helpful_message(Name) -> + Ms = ["If the error is caused by too exotic macro", + "definitions or uses of macros, adding option", + "{preprocess, true} can help. See also edoc(3)."], + lists:foreach(fun(M) -> edoc_report:report(Name, M, []) end, Ms). %% @spec get_doc(File::filename()) -> {ModuleName, edoc_module()} %% @equiv get_doc(File, []) diff --git a/lib/erl_interface/configure.in b/lib/erl_interface/configure.in index a155ceef7e..747750c1fb 100644 --- a/lib/erl_interface/configure.in +++ b/lib/erl_interface/configure.in @@ -82,6 +82,15 @@ AC_ARG_ENABLE(threads, esac ], [ threads_disabled=maybe ]) +AC_ARG_ENABLE(mask-real-errno, +[ --disable-mask-real-errno do not mask real 'errno'], +[ case "$enableval" in + no) mask_real_errno=no ;; + *) mask_real_errno=yes ;; + esac ], +[ mask_real_errno=yes ]) + + dnl ---------------------------------------------------------------------- dnl Checks for programs dnl ---------------------------------------------------------------------- @@ -100,6 +109,10 @@ AC_CHECK_SIZEOF(long) AC_CHECK_SIZEOF(void *) AC_CHECK_SIZEOF(long long) +if test $mask_real_errno = yes; then + AC_DEFINE(EI_HIDE_REAL_ERRNO, 1, [Define if 'errno' should not be exposed as is in 'erl_errno']) +fi + dnl We set EI_64BIT mode when long is 8 bytes, this makes things dnl work on windows and unix correctly if test $ac_cv_sizeof_long = 8; then @@ -158,7 +171,7 @@ AC_CHECK_LIB([socket], [getpeername]) # Checks for header files. AC_HEADER_STDC AC_HEADER_SYS_WAIT -AC_CHECK_HEADERS([arpa/inet.h fcntl.h limits.h malloc.h netdb.h netinet/in.h stddef.h stdlib.h string.h sys/param.h sys/socket.h sys/select.h sys/time.h unistd.h sys/types.h]) +AC_CHECK_HEADERS([arpa/inet.h fcntl.h limits.h malloc.h netdb.h netinet/in.h stddef.h stdlib.h string.h sys/param.h sys/socket.h sys/select.h sys/time.h unistd.h sys/types.h sys/uio.h]) # Checks for typedefs, structures, and compiler characteristics. # fixme AC_C_CONST & AC_C_VOLATILE needed for Windows? @@ -193,7 +206,7 @@ AC_CHECK_FUNCS([dup2 gethostbyaddr gethostbyname \ gethostbyaddr_r \ gethostbyname_r gethostname writev \ gethrtime gettimeofday inet_ntoa memchr memmove memset select \ - socket strchr strerror strrchr strstr uname]) + socket strchr strerror strrchr strstr uname sysconf]) AC_CHECK_FUNC(res_gethostbyname, [], AC_CHECK_LIB(resolv, res_gethostbyname) ) @@ -255,6 +268,7 @@ AC_SUBST(EI_THREADS) case "$threads_disabled" in no|maybe) LM_CHECK_THR_LIB + ETHR_CHK_GCC_ATOMIC_OPS([]) case "$THR_LIB_NAME" in "") @@ -268,7 +282,7 @@ case "$threads_disabled" in EI_THREADS="true" THR_DEFS="$THR_DEFS -D_WIN32_WINNT=0x0600 -DWINVER=0x0600" ;; - pthread) + pthread) EI_THREADS="true" ;; *) diff --git a/lib/erl_interface/doc/src/ei.xml b/lib/erl_interface/doc/src/ei.xml index 16f4e18637..ae322255ad 100644 --- a/lib/erl_interface/doc/src/ei.xml +++ b/lib/erl_interface/doc/src/ei.xml @@ -733,6 +733,21 @@ ei_encode_tuple_header(buf, &i, 0);</pre> </func> <func> + <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_init(void)</nametext></name> + <fsummary>Initialize the ei library.</fsummary> + <desc> + <p>Initialize the <c>ei</c> library. This function should be called once + (and only once) before calling any other functionality in the <c>ei</c> + library. However, note the exception below.</p> + <p>If the <c>ei</c> library is used together with the <c>erl_interface</c> + library, this function should <em>not</em> be called directly. It will be + called by the <c>erl_init()</c> function which should be used to initialize + the combination of the two libraries instead.</p> + <p>On success zero is returned. On failure a posix error code is returned.</p> + </desc> + </func> + + <func> <name since=""><ret>int</ret><nametext>ei_print_term(FILE* fp, const char* buf, int* index)</nametext></name> <name since=""><ret>int</ret><nametext>ei_s_print_term(char** s, const char* buf, int* index)</nametext></name> <fsummary>Print a term in clear text.</fsummary> diff --git a/lib/erl_interface/doc/src/ei_connect.xml b/lib/erl_interface/doc/src/ei_connect.xml index 6f16c0652e..e318dd6664 100644 --- a/lib/erl_interface/doc/src/ei_connect.xml +++ b/lib/erl_interface/doc/src/ei_connect.xml @@ -85,6 +85,273 @@ the <c>_tmo</c> suffix.</p> </section> + <section> + <marker id="ussi"/> + <title>User Supplied Socket Implementation</title> + <p>By default <c>ei</c> supplies a TCP/IPv4 socket interface + that is used when communicating. The user can however plug in + his/her own IPv4 socket implementation. This, for example, in order + to communicate over TLS. A user supplied socket implementation + is plugged in by passing a + <seealso marker="#ei_socket_callbacks">callback structure</seealso> + to either + <seealso marker="#ei_connect_init"><c>ei_connect_init_ussi()</c></seealso> + or + <seealso marker="#ei_connect_init"><c>ei_connect_xinit_ussi()</c></seealso>.</p> + + <p>All callbacks in the <c>ei_socket_callbacks</c> structure + <em>should</em> return zero on success; and a posix error + code on failure.</p> + + <p>The <c>addr</c> argument of the <c>listen</c>, <c>accept</c>, + and <c>connect</c> callbacks refer to appropriate address + structure for currently used protocol. Currently <c>ei</c> + only supports IPv4. That is, at this time <c>addr</c> always + points to a <c>struct sockaddr_in</c> structure.</p> + + <p>The <c>ei_socket_callbacks</c> structure may be enlarged in + the future. All fields not set, <em>needs</em> to be zeroed out.</p> + + <marker id="ei_socket_callbacks"/> + <code type="none"><![CDATA[ +typedef struct { + int flags; + int (*socket)(void **ctx, void *setup_ctx); + int (*close)(void *ctx); + int (*listen)(void *ctx, void *addr, int *len, int backlog); + int (*accept)(void **ctx, void *addr, int *len, unsigned tmo); + int (*connect)(void *ctx, void *addr, int len, unsigned tmo); + int (*writev)(void *ctx, const void *iov, int iovcnt, ssize_t *len, unsigned tmo); + int (*write)(void *ctx, const char *buf, ssize_t *len, unsigned tmo); + int (*read)(void *ctx, char *buf, ssize_t *len, unsigned tmo); + int (*handshake_packet_header_size)(void *ctx, int *sz); + int (*connect_handshake_complete)(void *ctx); + int (*accept_handshake_complete)(void *ctx); + int (*get_fd)(void *ctx, int *fd); +} ei_socket_callbacks; + ]]></code> + + <taglist> + + <tag><c>flags</c></tag> + <item> + <p>Flags informing <c>ei</c> about the behaviour of the + callbacks. Flags should be bitwise or:ed together. If no flag, + is set, the <c>flags</c> field should contain <c>0</c>. Currently, + supported flags:</p> + <taglist> + <tag><c>EI_SCLBK_FLG_FULL_IMPL</c></tag> + <item> + <p> + If set, the <c>accept()</c>, <c>connect()</c>, + <c>writev()</c>, <c>write()</c>, and <c>read()</c> callbacks + implements timeouts. The timeout is passed in the <c>tmo</c> + argument and is given in milli seconds. Note that the + <c>tmo</c> argument to these callbacks differ from the + timeout arguments in the <c>ei</c> API. Zero means a zero + timeout. That is, poll and timeout immediately unless the + operation is successful. <c>EI_SCLBK_INF_TMO</c> + (max <c>unsigned</c>) means infinite timeout. The file + descriptor is in blocking mode when a callback is called, + and it must be in blocking mode when the callback returns. + </p> + <p> + If not set, <c>ei</c> will implement the timeout using + <c>select()</c> in order to determine when to call the + callbacks and when to time out. The <c>tmo</c> arguments + of the <c>accept()</c>, <c>connect()</c>, <c>writev()</c>, + <c>write()</c>, and <c>read()</c> callbacks should be + ignored. The callbacks may be called in non-blocking mode. + The callbacks are not allowed to change between blocking + and non-blocking mode. In order for this to work, + <c>select()</c> needs to interact with the socket primitives + used the same way as it interacts with the ordinary socket + primitives. If this is not the case, the callbacks + <em>need</em> to implement timeouts and this flag should + be set. + </p> + </item> + </taglist> + <p>More flags may be introduced in the future.</p> + </item> + + <tag><c>int (*socket)(void **ctx, void *setup_ctx)</c></tag> + <item> + <p>Create a socket and a context for the socket.</p> + + <p>On success it should set <c>*ctx</c> to point to a context for + the created socket. This context will be passed to all other + socket callbacks. This function will be passed the same + <c>setup_context</c> as passed to the preceeding + <seealso marker="#ei_connect_init"><c>ei_connect_init_ussi()</c></seealso> + or + <seealso marker="#ei_connect_init"><c>ei_connect_xinit_ussi()</c></seealso> + call.</p> + + <note><p>During the lifetime of a socket, the pointer <c>*ctx</c> + <em>has</em> to remain the same. That is, it cannot later be + relocated.</p></note> + + <p>This callback is mandatory.</p> + </item> + + <tag><c>int (*close)(void *ctx)</c></tag> + <item> + <p>Close the socket identified by <c>ctx</c> and destroy the context.</p> + + <p>This callback is mandatory.</p> + </item> + + <tag><c>int (*listen)(void *ctx, void *addr, int *len, int backlog)</c></tag> + <item> + <p>Bind the socket identified by <c>ctx</c> to a local interface + and then listen on it.</p> + + <p>The <c>addr</c> and <c>len</c> arguments are both input and output + arguments. When called <c>addr</c> points to an address structure of + lenght <c>*len</c> containing information on how to bind the socket. + Uppon return this callback should have updated the structure referred + by <c>addr</c> with information on how the socket actually was bound. + <c>*len</c> should be updated to reflect the size of <c>*addr</c> + updated. <c>backlog</c> identifies the size of the backlog for the + listen socket.</p> + + <p>This callback is mandatory.</p> + </item> + + <tag><c>int (*accept)(void **ctx, void *addr, int *len, unsigned tmo)</c></tag> + <item> + <p>Accept connections on the listen socket identified by + <c>*ctx</c>.</p> + + <p>When a connection is accepted, a new context for the accepted + connection should be created and <c>*ctx</c> should be updated + to point to the new context for the accepted connection. When + called <c>addr</c> points to an uninitialized address structure + of lenght <c>*len</c>. Uppon return this callback should have + updated this structure with information about the client address. + <c>*len</c> should be updated to reflect the size of <c>*addr</c> + updated. + </p> + + <p>If the <c>EI_SCLBK_FLG_FULL_IMPL</c> flag has been set, + <c>tmo</c> contains timeout time in milliseconds.</p> + + <note><p>During the lifetime of a socket, the pointer <c>*ctx</c> + <em>has</em> to remain the same. That is, it cannot later be + relocated.</p></note> + + <p>This callback is mandatory.</p> + </item> + + <tag><c>int (*connect)(void *ctx, void *addr, int len, unsigned tmo)</c></tag> + <item> + <p>Connect the socket identified by <c>ctx</c> to the address + identified by <c>addr</c>.</p> + + <p>When called <c>addr</c> points to an address structure of + lenght <c>len</c> containing information on where to connect.</p> + + <p>If the <c>EI_SCLBK_FLG_FULL_IMPL</c> flag has been set, + <c>tmo</c> contains timeout time in milliseconds.</p> + + <p>This callback is mandatory.</p> + </item> + + <tag><c>int (*writev)(void *ctx, const void *iov, long iovcnt, ssize_t *len, unsigned tmo)</c></tag> + <item> + <p>Write data on the connected socket identified by <c>ctx</c>.</p> + + <p><c>iov</c> points to an array of <c>struct iovec</c> structures of + length <c>iovcnt</c> containing data to write to the socket. On success, + this callback should set <c>*len</c> to the amount of bytes successfully + written on the socket.</p> + + <p>If the <c>EI_SCLBK_FLG_FULL_IMPL</c> flag has been set, + <c>tmo</c> contains timeout time in milliseconds.</p> + + <p>This callback is optional. Set the <c>writev</c> field + in the the <c>ei_socket_callbacks</c> structure to <c>NULL</c> if not + implemented.</p> + </item> + + <tag><c>int (*write)(void *ctx, const char *buf, ssize_t *len, unsigned tmo)</c></tag> + <item> + <p>Write data on the connected socket identified by <c>ctx</c>.</p> + + <p>When called <c>buf</c> points to a buffer of length <c>*len</c> + containing the data to write on the socket. On success, this callback + should set <c>*len</c> to the amount of bytes successfully written on + the socket.</p> + + <p>If the <c>EI_SCLBK_FLG_FULL_IMPL</c> flag has been set, + <c>tmo</c> contains timeout time in milliseconds.</p> + + <p>This callback is mandatory.</p> + </item> + + <tag><c>int (*read)(void *ctx, char *buf, ssize_t *len, unsigned tmo)</c></tag> + <item> + <p>Read data on the connected socket identified by <c>ctx</c>.</p> + + <p><c>buf</c> points to a buffer of length <c>*len</c> where the + read data should be placed. On success, this callback should update + <c>*len</c> to the amount of bytes successfully read on the socket.</p> + + <p>If the <c>EI_SCLBK_FLG_FULL_IMPL</c> flag has been set, + <c>tmo</c> contains timeout time in milliseconds.</p> + + <p>This callback is mandatory.</p> + </item> + + <tag><c>int (*handshake_packet_header_size)(void *ctx, int *sz)</c></tag> + <item> + <p>Inform about handshake packet header size to use during the Erlang + distribution handshake.</p> + + <p>On success, <c>*sz</c> should be set to the handshake packet header + size to use. Valid values are <c>2</c> and <c>4</c>. Erlang TCP + distribution use a handshake packet size of <c>2</c> and Erlang TLS + distribution use a handshake packet size of <c>4</c>.</p> + + <p>This callback is mandatory.</p> + </item> + + <tag><c>int (*connect_handshake_complete)(void *ctx)</c></tag> + <item> + <p>Called when a locally started handshake has completed successfully.</p> + + <p>This callback is optional. Set the <c>connect_handshake_complete</c> field + in the <c>ei_socket_callbacks</c> structure to <c>NULL</c> if not implemented.</p> + </item> + + <tag><c>int (*accept_handshake_complete)(void *ctx)</c></tag> + <item> + <p>Called when a remotely started handshake has completed successfully.</p> + + <p>This callback is optional. Set the <c>accept_handshake_complete</c> field in + the <c>ei_socket_callbacks</c> structure to <c>NULL</c> if not implemented.</p> + </item> + + <tag><c>int (*get_fd)(void *ctx, int *fd)</c></tag> + <item> + <p>Inform about file descriptor used by the socket which is identified + by <c>ctx</c>.</p> + + <note><p>During the lifetime of a socket, the file descriptor + <em>has</em> to remain the same. That is, repeated calls to this + callback with the same context <c>should</c> always report the same + file descriptor.</p> + <p>The file descriptor <em>has</em> to be a real file descriptor. + That is, no other operation should be able to get the same file + descriptor until it has been released by the <c>close()</c> + callback.</p> + </note> + + <p>This callback is mandatory.</p> + </item> + </taglist> + </section> <funcs> <func> <name since=""><ret>struct hostent *</ret><nametext>ei_gethostbyaddr(const char *addr, int len, int type)</nametext></name> @@ -96,6 +363,7 @@ <p>Convenience functions for some common name lookup functions.</p> </desc> </func> + <func> <name since=""><ret>int</ret><nametext>ei_accept(ei_cnode *ec, int listensock, ErlConnect *conp)</nametext></name> @@ -141,6 +409,14 @@ typedef struct { </func> <func> + <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_close_connection(int fd)</nametext></name> + <fsummary>Close a connection.</fsummary> + <desc> + <p>Closes a previously opened connection or listen socket.</p> + </desc> + </func> + + <func> <name since=""><ret>int</ret><nametext>ei_connect(ei_cnode* ec, char *nodename)</nametext></name> <name since=""><ret>int</ret><nametext>ei_xconnect(ei_cnode* ec, Erl_IpAddr adr, char *alivename)</nametext></name> <fsummary>Establish a connection to an Erlang node.</fsummary> @@ -193,7 +469,9 @@ fd = ei_xconnect(&ec, &addr, ALIVE); <func> <name since=""><ret>int</ret><nametext>ei_connect_init(ei_cnode* ec, const char* this_node_name, const char *cookie, short creation)</nametext></name> + <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_connect_init_ussi(ei_cnode* ec, const char* this_node_name, const char *cookie, short creation, ei_socket_callbacks *cbs, int cbs_sz, void *setup_context)</nametext></name> <name since=""><ret>int</ret><nametext>ei_connect_xinit(ei_cnode* ec, const char *thishostname, const char *thisalivename, const char *thisnodename, Erl_IpAddr thisipaddr, const char *cookie, short creation)</nametext></name> + <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_connect_xinit_ussi(ei_cnode* ec, const char *thishostname, const char *thisalivename, const char *thisnodename, Erl_IpAddr thisipaddr, const char *cookie, short creation, ei_socket_callbacks *cbs, int cbs_sz, void *setup_context)</nametext></name> <fsummary>Initialize for a connection.</fsummary> <desc> <p>Initializes the <c>ec</c> structure, to @@ -236,6 +514,21 @@ fd = ei_xconnect(&ec, &addr, ALIVE); <item> <p><c>thispaddr</c> if the IP address of the host.</p> </item> + <item> + <p><c>cbs</c> is a pointer to a + <seealso marker="#ei_socket_callbacks">callback structure</seealso> + implementing and alternative socket interface.</p> + </item> + <item> + <p><c>cbs_sz</c> is the size of the structure + pointed to by <c>cbs</c>.</p> + </item> + <item> + <p><c>setup_context</c> is a pointer to a structure that + will be passed as second argument to the <c>socket</c> callback + in the <c>cbs</c> structure.</p> + </item> + </list> <p>A C-node acting as a server is assigned a creation number when it calls <c>ei_publish()</c>.</p> @@ -299,6 +592,45 @@ if (ei_connect_init(&ec, "madonna", "cookie...", n++) < 0) { </func> <func> + <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_listen(ei_cnode *ec, int *port, int backlog)</nametext></name> + <name since="OTP @OTP-15442@"><ret>int</ret><nametext>ei_xlisten(ei_cnode *ec, Erl_IpAddr adr, int *port, int backlog)</nametext></name> + <fsummary>Create a listen socket.</fsummary> + <desc> + <p>Used by a server process to setup a listen socket which + later can be used for accepting connections from client processes. + </p> + <list type="bulleted"> + <item> + <p><c>ec</c> is the C-node structure.</p> + </item> + <item> + <p><c>adr</c> is local interface to bind to.</p> + </item> + <item> + <p><c>port</c> is a pointer to an integer containing the + port number to bind to. If <c>*port</c> equals <c>0</c> + when calling <c>ei_listen()</c>, the socket will be bound to + an ephemeral port. On success, <c>ei_listen()</c> will update + the value of <c>*port</c> to the port actually bound to. + </p> + </item> + <item> + <p><c>backlog</c> is maximum backlog of pending connections.</p> + </item> + </list> + <p><c>ei_listen</c> will create a socket, bind to a port on the + local interface identified by <c>adr</c> (or all local interfaces if + <c>ei_listen()</c> is called), and mark the socket as a passive socket + (that is, a socket that will be used for accepting incoming connections). + </p> + <p> + On success, a file descriptor is returned which can be used in a call to + <c>ei_accept()</c>. On failure, <c>ERL_ERROR</c> is returned and + <c>erl_errno</c> is set to <c>EIO</c>.</p> + </desc> + </func> + + <func> <name since=""><ret>int</ret><nametext>ei_publish(ei_cnode *ec, int port)</nametext></name> <fsummary>Publish a node name.</fsummary> <desc> diff --git a/lib/erl_interface/doc/src/ei_users_guide.xml b/lib/erl_interface/doc/src/ei_users_guide.xml index 0eed50b50b..2dfd99e35a 100644 --- a/lib/erl_interface/doc/src/ei_users_guide.xml +++ b/lib/erl_interface/doc/src/ei_users_guide.xml @@ -162,12 +162,20 @@ $ ld -L/usr/local/otp/lib/erl_interface-3.2.3/ </section> <section> - <title>Initializing the Erl_Interface Library</title> - <p>Before calling any of the other <c>Erl_Interface</c> functions, call - <c>erl_init()</c> exactly once to initialize the library. + <title>Initializing the Libraries</title> + <p> + Before calling any of the other functions in the <c>erl_interface</c> + and <c>ei</c> libraries, call <c>erl_init()</c> exactly once to initialize + both libraries. <c>erl_init()</c> takes two arguments. However, the arguments - are no longer used by <c>Erl_Interface</c> and are therefore to be - specified as <c>erl_init(NULL,0)</c>.</p> + are no longer used by <c>erl_interface</c> and are therefore to be + specified as <c>erl_init(NULL,0)</c>. + </p> + <p> + If you only use the <c>ei</c> library, instead initialize it by calling + <c>ei_init()</c> exactly once before calling any other functions in + the <c>ei</c> library. + </p> </section> <section> diff --git a/lib/erl_interface/include/ei.h b/lib/erl_interface/include/ei.h index 948f89be85..ca4960b252 100644 --- a/lib/erl_interface/include/ei.h +++ b/lib/erl_interface/include/ei.h @@ -35,6 +35,9 @@ #include <winsock2.h> #include <windows.h> #include <winbase.h> +typedef LONG_PTR ssize_t; /* Sigh... */ +#else +#include <sys/types.h> /* ssize_t */ #endif #include <stdio.h> /* Need type FILE */ @@ -286,6 +289,31 @@ typedef struct { char nodename[MAXNODELEN+1]; } ErlConnect; +#define EI_SCLBK_INF_TMO (~((unsigned) 0)) + +#define EI_SCLBK_FLG_FULL_IMPL (1 << 0) + +typedef struct { + int flags; + + int (*socket)(void **ctx, void *setup_ctx); + int (*close)(void *ctx); + int (*listen)(void *ctx, void *addr, int *len, int backlog); + int (*accept)(void **ctx, void *addr, int *len, unsigned tmo); + int (*connect)(void *ctx, void *addr, int len, unsigned tmo); + int (*writev)(void *ctx, const void *iov, int iovcnt, ssize_t *len, unsigned tmo); + int (*write)(void *ctx, const char *buf, ssize_t *len, unsigned tmo); + int (*read)(void *ctx, char *buf, ssize_t *len, unsigned tmo); + + int (*handshake_packet_header_size)(void *ctx, int *sz); + int (*connect_handshake_complete)(void *ctx); + int (*accept_handshake_complete)(void *ctx); + int (*get_fd)(void *ctx, int *fd); + + /* end of version 1 */ + +} ei_socket_callbacks; + typedef struct ei_cnode_s { char thishostname[EI_MAXHOSTNAMELEN+1]; char thisnodename[MAXNODELEN+1]; @@ -295,6 +323,8 @@ typedef struct ei_cnode_s { char ei_connect_cookie[EI_MAX_COOKIE_SIZE+1]; short creation; erlang_pid self; + ei_socket_callbacks *cbs; + void *setup_context; } ei_cnode; typedef struct in_addr *Erl_IpAddr; @@ -308,7 +338,6 @@ typedef struct ei_x_buff_TAG { int index; } ei_x_buff; - /* -------------------------------------------------------------------- */ /* Function definitions (listed in same order as documentation) */ /* -------------------------------------------------------------------- */ @@ -322,6 +351,16 @@ int ei_connect_xinit (ei_cnode* ec, const char *thishostname, Erl_IpAddr thisipaddr, const char *cookie, const short creation); +int ei_connect_init_ussi(ei_cnode* ec, const char* this_node_name, + const char *cookie, short creation, + ei_socket_callbacks *cbs, int cbs_sz, + void *setup_context); +int ei_connect_xinit_ussi(ei_cnode* ec, const char *thishostname, + const char *thisalivename, const char *thisnodename, + Erl_IpAddr thisipaddr, const char *cookie, + const short creation, ei_socket_callbacks *cbs, + int cbs_sz, void *setup_context); + int ei_connect(ei_cnode* ec, char *nodename); int ei_connect_tmo(ei_cnode* ec, char *nodename, unsigned ms); int ei_xconnect(ei_cnode* ec, Erl_IpAddr adr, char *alivename); @@ -348,11 +387,15 @@ int ei_rpc_from(ei_cnode* ec, int fd, int timeout, erlang_msg* msg, int ei_publish(ei_cnode* ec, int port); int ei_publish_tmo(ei_cnode* ec, int port, unsigned ms); +int ei_listen(ei_cnode *ec, int *port, int backlog); +int ei_xlisten(ei_cnode *ec, Erl_IpAddr adr, int *port, int backlog); int ei_accept(ei_cnode* ec, int lfd, ErlConnect *conp); int ei_accept_tmo(ei_cnode* ec, int lfd, ErlConnect *conp, unsigned ms); int ei_unpublish(ei_cnode* ec); int ei_unpublish_tmo(const char *alive, unsigned ms); +int ei_close_connection(int fd); + const char *ei_thisnodename(const ei_cnode* ec); const char *ei_thishostname(const ei_cnode* ec); const char *ei_thisalivename(const ei_cnode* ec); @@ -626,6 +669,8 @@ struct ei_reg_tabstat { }; +int ei_init(void); + /* -------------------------------------------------------------------- */ /* XXXXXXXXXXX */ /* -------------------------------------------------------------------- */ diff --git a/lib/erl_interface/src/Makefile.in b/lib/erl_interface/src/Makefile.in index 614e7325a9..b0bb9bfadf 100644 --- a/lib/erl_interface/src/Makefile.in +++ b/lib/erl_interface/src/Makefile.in @@ -31,12 +31,11 @@ .PHONY : debug opt release clean distclean depend -TARGET = @TARGET@ - # ---------------------------------------------------- # Application version and release dir specification # ---------------------------------------------------- include ../vsn.mk +include $(ERL_TOP)/make/target.mk include $(TARGET)/eidefs.mk include $(ERL_TOP)/make/output.mk @@ -417,7 +416,8 @@ MISCSRC = \ misc/eimd5.c \ misc/get_type.c \ misc/show_msg.c \ - misc/ei_compat.c + misc/ei_compat.c \ + misc/ei_init.c REGISTRYSRC = \ registry/hash_dohash.c \ diff --git a/lib/erl_interface/src/connect/ei_connect.c b/lib/erl_interface/src/connect/ei_connect.c index 9df4fa3b6c..7a304e6d4f 100644 --- a/lib/erl_interface/src/connect/ei_connect.c +++ b/lib/erl_interface/src/connect/ei_connect.c @@ -42,10 +42,8 @@ #include <inetLib.h> #include <unistd.h> -#include <sys/types.h> #include <sys/times.h> #include <unistd.h> -#include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <netinet/tcp.h> @@ -55,7 +53,6 @@ #else /* some other unix */ #include <unistd.h> -#include <sys/types.h> #include <sys/times.h> #if TIME_WITH_SYS_TIME @@ -84,6 +81,7 @@ #include <string.h> #include <errno.h> #include <ctype.h> +#include <stddef.h> #include "eiext.h" #include "ei_portio.h" @@ -98,11 +96,16 @@ #include "ei_epmd.h" #include "ei_internal.h" +static int ei_connect_initialized = 0; int ei_tracelevel = 0; #define COOKIE_FILE "/.erlang.cookie" #define EI_MAX_HOME_PATH 1024 +#define EI_SOCKET_CALLBACKS_SZ_V1 \ + (offsetof(ei_socket_callbacks, get_fd) \ + + sizeof(int (*)(void *))) + /* FIXME why not macro? */ static char *null_cookie = ""; @@ -113,35 +116,51 @@ static int get_home(char *buf, int size); static unsigned gen_challenge(void); static void gen_digest(unsigned challenge, char cookie[], unsigned char digest[16]); -static int send_status(int fd, char *status, unsigned ms); -static int recv_status(int fd, unsigned ms); -static int send_challenge(int fd, char *nodename, - unsigned challenge, unsigned version, unsigned ms); -static int recv_challenge(int fd, unsigned *challenge, - unsigned *version, - unsigned *flags, ErlConnect *namebuf, unsigned ms); -static int send_challenge_reply(int fd, unsigned char digest[16], +static int send_status(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, char *status, unsigned ms); +static int recv_status(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, unsigned ms); +static int send_challenge(ei_socket_callbacks *cbs, void *ctx, int pkt_sz, + char *nodename, unsigned challenge, + unsigned version, unsigned ms); +static int recv_challenge(ei_socket_callbacks *cbs, void *ctx, int pkt_sz, + unsigned *challenge, unsigned *version, + unsigned *flags, char *namebuf, unsigned ms); +static int send_challenge_reply(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, unsigned char digest[16], unsigned challenge, unsigned ms); -static int recv_challenge_reply(int fd, - unsigned our_challenge, +static int recv_challenge_reply(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, unsigned our_challenge, char cookie[], unsigned *her_challenge, unsigned ms); -static int send_challenge_ack(int fd, unsigned char digest[16], unsigned ms); -static int recv_challenge_ack(int fd, - unsigned our_challenge, +static int send_challenge_ack(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, unsigned char digest[16], + unsigned ms); +static int recv_challenge_ack(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, unsigned our_challenge, char cookie[], unsigned ms); -static int send_name(int fd, char *nodename, - unsigned version, unsigned ms); +static int send_name(ei_socket_callbacks *cbs, void *ctx, int pkt_sz, + char *nodename, unsigned version, unsigned ms); -/* Common for both handshake types */ -static int recv_name(int fd, - unsigned *version, - unsigned *flags, ErlConnect *namebuf, unsigned ms); +static int recv_name(ei_socket_callbacks *cbs, void *ctx, int pkt_sz, + unsigned *version, unsigned *flags, char *namebuf, + unsigned ms); static struct hostent* dyn_gethostbyname_r(const char *name, struct hostent *hostp, char **buffer_p, int buflen, int *h_errnop); +static void abort_connection(ei_socket_callbacks *cbs, void *ctx); +static int close_connection(ei_socket_callbacks *cbs, void *ctx, int fd); + +static char * +estr(int e) +{ + char *str = strerror(e); + if (!str) + return "unknown error"; + return str; +} /*************************************************************************** @@ -154,25 +173,208 @@ dyn_gethostbyname_r(const char *name, struct hostent *hostp, char **buffer_p, typedef struct ei_socket_info_s { int socket; + ei_socket_callbacks *cbs; + void *ctx; int dist_version; ei_cnode cnode; /* A copy, not a pointer. We don't know when freed */ char cookie[EI_MAX_COOKIE_SIZE+1]; } ei_socket_info; +/*************************************************************************** + * + * XXX + * + ***************************************************************************/ + +#ifndef ETHR_HAVE___atomic_compare_exchange_n +# define ETHR_HAVE___atomic_compare_exchange_n 0 +#endif +#ifndef ETHR_HAVE___atomic_load_n +# define ETHR_HAVE___atomic_load_n 0 +#endif +#ifndef ETHR_HAVE___atomic_store_n +# define ETHR_HAVE___atomic_store_n 0 +#endif + +#if defined(_REENTRANT) \ + && (!(ETHR_HAVE___atomic_compare_exchange_n & SIZEOF_VOID_P) \ + || !(ETHR_HAVE___atomic_load_n & SIZEOF_VOID_P) \ + || !(ETHR_HAVE___atomic_store_n & SIZEOF_VOID_P)) +# undef EI_DISABLE_SEQ_SOCKET_INFO +# define EI_DISABLE_SEQ_SOCKET_INFO +#endif + +#ifdef __WIN32__ +# undef EI_DISABLE_SEQ_SOCKET_INFO +# define EI_DISABLE_SEQ_SOCKET_INFO +#endif + +#ifndef EI_DISABLE_SEQ_SOCKET_INFO + +#ifdef _REENTRANT + +#define EI_ATOMIC_CMPXCHG_ACQ_REL(VARP, XCHGP, NEW) \ + __atomic_compare_exchange_n((VARP), (XCHGP), (NEW), 0, \ + __ATOMIC_ACQ_REL, __ATOMIC_ACQUIRE) +#define EI_ATOMIC_LOAD_ACQ(VARP) \ + __atomic_load_n((VARP), __ATOMIC_ACQUIRE) +#define EI_ATOMIC_STORE_REL(VARP, NEW) \ + __atomic_store_n((VARP), (NEW), __ATOMIC_RELEASE) + +#else /* ! _REENTRANT */ + +#define EI_ATOMIC_CMPXCHG_ACQ_REL(VARP, XCHGP, NEW) \ + (*(VARP) == *(XCHGP) \ + ? ((*(VARP) = (NEW)), !0) \ + : ((*(XCHGP) = *(VARP)), 0)) +#define EI_ATOMIC_LOAD_ACQ(VARP) (*(VARP)) +#define EI_ATOMIC_STORE_REL(VARP, NEW) (*(VARP) = (NEW)) + +#endif /* ! _REENTRANT */ + +#define EI_SOCKET_INFO_SEG_BITS 5 +#define EI_SOCKET_INFO_SEG_SIZE (1 << EI_SOCKET_INFO_SEG_BITS) +#define EI_SOCKET_INFO_SEG_MASK (EI_SOCKET_INFO_SEG_SIZE - 1) + +typedef struct { + int max_fds; + ei_socket_info *segments[1]; /* Larger in reality... */ +} ei_socket_info_data__; + +static ei_socket_info_data__ *socket_info_data = NULL; + +static int init_socket_info(int late) +{ + int max_fds; + int i; + size_t segments_len; + ei_socket_info_data__ *info_data, *xchg; + + if (EI_ATOMIC_LOAD_ACQ(&socket_info_data) != NULL) + return 0; /* Already initialized... */ + +#if defined(HAVE_SYSCONF) && defined(_SC_OPEN_MAX) + max_fds = sysconf(_SC_OPEN_MAX); +#else + max_fds = 1024; +#endif + + if (max_fds < 0) + return EIO; + + segments_len = ((max_fds-1)/EI_SOCKET_INFO_SEG_SIZE + 1); + + info_data = malloc(sizeof(ei_socket_info_data__) + + (sizeof(ei_socket_info *)*(segments_len-1))); + if (!info_data) + return ENOMEM; + + info_data->max_fds = max_fds; + for (i = 0; i < segments_len; i++) + info_data->segments[i] = NULL; + + xchg = NULL; + if (!EI_ATOMIC_CMPXCHG_ACQ_REL(&socket_info_data, &xchg, info_data)) + free(info_data); /* Already initialized... */ + + return 0; +} + +static int put_ei_socket_info(int fd, int dist_version, char* cookie, ei_cnode *ec, + ei_socket_callbacks *cbs, void *ctx) +{ + int six; + ei_socket_info *seg, *si; + int socket; + + if (fd < 0 || socket_info_data->max_fds <= fd) + return -1; + + socket = fd; + six = fd >> EI_SOCKET_INFO_SEG_BITS; + seg = EI_ATOMIC_LOAD_ACQ(&socket_info_data->segments[six]); + + if (!seg) { + ei_socket_info *xchg; + int i; + seg = malloc(sizeof(ei_socket_info)*EI_SOCKET_INFO_SEG_SIZE); + if (!seg) + return -1; + for (i = 0; i < EI_SOCKET_INFO_SEG_SIZE; i++) { + seg[i].socket = -1; + } + + xchg = NULL; + if (!EI_ATOMIC_CMPXCHG_ACQ_REL(&socket_info_data->segments[six], &xchg, seg)) { + free(seg); + seg = xchg; + } + } + + si = &seg[fd & EI_SOCKET_INFO_SEG_MASK]; + + if (dist_version < 0) { + socket = -1; + si->cbs = NULL; + si->ctx = NULL; + } + else { + si->dist_version = dist_version; + si->cnode = *ec; + si->cbs = cbs; + si->ctx = ctx; + strcpy(si->cookie, cookie); + } + + EI_ATOMIC_STORE_REL(&si->socket, socket); + + return 0; +} + +static ei_socket_info* get_ei_socket_info(int fd) +{ + int six, socket; + ei_socket_info *seg, *si; + + if (fd < 0 || socket_info_data->max_fds <= fd) + return NULL; + + six = fd >> EI_SOCKET_INFO_SEG_BITS; + seg = EI_ATOMIC_LOAD_ACQ(&socket_info_data->segments[six]); + + if (!seg) + return NULL; + + si = &seg[fd & EI_SOCKET_INFO_SEG_MASK]; + socket = EI_ATOMIC_LOAD_ACQ(&si->socket); + if (socket != fd) + return NULL; + return si; +} + +#else /* EI_DISABLE_SEQ_SOCKET_INFO */ + int ei_n_sockets = 0, ei_sz_sockets = 0; ei_socket_info *ei_sockets = NULL; + #ifdef _REENTRANT ei_mutex_t* ei_sockets_lock = NULL; #endif /* _REENTRANT */ +static int init_socket_info(int late) +{ +#ifdef _REENTRANT + if (late) + return ENOTSUP; /* Refuse doing unsafe initialization... */ + ei_sockets_lock = ei_mutex_create(); + if (!ei_sockets_lock) + return ENOMEM; +#endif /* _REENTRANT */ + return 0; +} -/*************************************************************************** - * - * XXX - * - ***************************************************************************/ - -static int put_ei_socket_info(int fd, int dist_version, char* cookie, ei_cnode *ec) +static int put_ei_socket_info(int fd, int dist_version, char* cookie, ei_cnode *ec, + ei_socket_callbacks *cbs, void *ctx) { int i; @@ -182,11 +384,13 @@ static int put_ei_socket_info(int fd, int dist_version, char* cookie, ei_cnode * for (i = 0; i < ei_n_sockets; ++i) { if (ei_sockets[i].socket == fd) { if (dist_version == -1) { - memmove(&ei_sockets[i], &ei_sockets[i+1], + memmove(&ei_sockets[i], &ei_sockets[i+1], sizeof(ei_sockets[0])*(ei_n_sockets-i-1)); } else { ei_sockets[i].dist_version = dist_version; /* Copy the content, see ei_socket_info */ + ei_sockets[i].cbs = cbs; + ei_sockets[i].ctx = ctx; ei_sockets[i].cnode = *ec; strcpy(ei_sockets[i].cookie, cookie); } @@ -209,7 +413,9 @@ static int put_ei_socket_info(int fd, int dist_version, char* cookie, ei_cnode * } ei_sockets[ei_n_sockets].socket = fd; ei_sockets[ei_n_sockets].dist_version = dist_version; - ei_sockets[i].cnode = *ec; + ei_sockets[ei_n_sockets].cnode = *ec; + ei_sockets[ei_n_sockets].cbs = cbs; + ei_sockets[ei_n_sockets].ctx = ctx; strcpy(ei_sockets[ei_n_sockets].cookie, cookie); ++ei_n_sockets; } @@ -219,14 +425,6 @@ static int put_ei_socket_info(int fd, int dist_version, char* cookie, ei_cnode * return 0; } -#if 0 -/* FIXME not used ?! */ -static int remove_ei_socket_info(int fd, int dist_version, char* cookie) -{ - return put_ei_socket_info(fd, -1, NULL); -} -#endif - static ei_socket_info* get_ei_socket_info(int fd) { int i; @@ -248,6 +446,13 @@ static ei_socket_info* get_ei_socket_info(int fd) return NULL; } +#endif /* EI_DISABLE_SEQ_SOCKET_INFO */ + +static int remove_ei_socket_info(int fd) +{ + return put_ei_socket_info(fd, -1, NULL, NULL, NULL, NULL); +} + ei_cnode *ei_fd_to_cnode(int fd) { ei_socket_info *sockinfo = get_ei_socket_info(fd); @@ -255,6 +460,19 @@ ei_cnode *ei_fd_to_cnode(int fd) return &sockinfo->cnode; } +int ei_get_cbs_ctx__(ei_socket_callbacks **cbs, void **ctx, int fd) +{ + ei_socket_info *sockinfo = get_ei_socket_info(fd); + if (sockinfo) { + *cbs = sockinfo->cbs; + *ctx = sockinfo->ctx; + return 0; + } + + *cbs = NULL; + *ctx = NULL; + return EBADF; +} /*************************************************************************** * Get/Set tracelevel @@ -333,21 +551,6 @@ const char *ei_getfdcookie(int fd) return r; } -/* call with cookie to set value to use on descriptor fd, -* or specify NULL to use default -*/ -/* FIXME why defined but not used? */ -#if 0 -static int ei_setfdcookie(ei_cnode* ec, int fd, char *cookie) -{ - int dist_version = ei_distversion(fd); - - if (cookie == NULL) - cookie = ec->ei_connect_cookie; - return put_ei_socket_info(fd, dist_version, cookie); -} -#endif - static int get_int32(unsigned char *s) { return ((s[0] << 24) | (s[1] << 16) | (s[2] << 8) | (s[3] )); @@ -400,34 +603,62 @@ static int initWinSock(void) } #endif +static int init_connect(int late) +{ + int error; + + /* + * 'late' is non-zero when not called via ei_init(). Such a + * call is not supported, but we for now save the day if + * it easy to do so; otherwise, return ENOTSUP. + */ + +#ifdef __WIN32__ + if (!initWinSock()) { + EI_TRACE_ERR0("ei_init_connect","can't initiate winsock"); + return EIO; + } +#endif /* win32 */ + + error = init_socket_info(late); + if (error) { + EI_TRACE_ERR0("ei_init_connect","can't initiate socket info"); + return error; + } + + ei_connect_initialized = !0; + return 0; +} + +int ei_init_connect(void) +{ + return init_connect(0); +} + /* * Perhaps run this routine instead of ei_connect_init/2 ? * Initailize by setting: * thishostname, thisalivename, thisnodename and thisipaddr */ -int ei_connect_xinit(ei_cnode* ec, const char *thishostname, - const char *thisalivename, const char *thisnodename, - Erl_IpAddr thisipaddr, const char *cookie, - const short creation) +int ei_connect_xinit_ussi(ei_cnode* ec, const char *thishostname, + const char *thisalivename, const char *thisnodename, + Erl_IpAddr thisipaddr, const char *cookie, + const short creation, ei_socket_callbacks *cbs, + int cbs_sz, void *setup_context) { char *dbglevel; - -/* FIXME this code was enabled for 'erl'_connect_xinit(), why not here? */ -#if 0 -#ifdef __WIN32__ - if (!initWinSock()) { - EI_TRACE_ERR0("ei_connect_xinit","can't initiate winsock"); - return ERL_ERROR; - } -#endif -#endif -#ifdef _REENTRANT - if (ei_sockets_lock == NULL) { - ei_sockets_lock = ei_mutex_create(); - } -#endif /* _REENTRANT */ + if (!ei_connect_initialized) + init_connect(!0); + if (cbs != &ei_default_socket_callbacks) + EI_SET_HAVE_PLUGIN_SOCKET_IMPL__; + + if (cbs_sz < EI_SOCKET_CALLBACKS_SZ_V1) { + EI_TRACE_ERR0("ei_connect_xinit","invalid size of ei_socket_callbacks struct"); + return ERL_ERROR; + } + ec->creation = creation & 0x3; /* 2 bits */ if (cookie) { @@ -469,6 +700,9 @@ int ei_connect_xinit(ei_cnode* ec, const char *thishostname, ec->self.serial = 0; ec->self.creation = creation & 0x3; /* 2 bits */ + ec->cbs = cbs; + ec->setup_context = setup_context; + if ((dbglevel = getenv("EI_TRACELEVEL")) != NULL || (dbglevel = getenv("ERL_DEBUG_DIST")) != NULL) ei_tracelevel = atoi(dbglevel); @@ -476,14 +710,27 @@ int ei_connect_xinit(ei_cnode* ec, const char *thishostname, return 0; } +int ei_connect_xinit(ei_cnode* ec, const char *thishostname, + const char *thisalivename, const char *thisnodename, + Erl_IpAddr thisipaddr, const char *cookie, + const short creation) +{ + return ei_connect_xinit_ussi(ec, thishostname, thisalivename, thisnodename, + thisipaddr, cookie, creation, + &ei_default_socket_callbacks, + sizeof(ei_default_socket_callbacks), + NULL); +} /* * Initialize by set: thishostname, thisalivename, * thisnodename and thisipaddr. At success return 0, * otherwise return -1. */ -int ei_connect_init(ei_cnode* ec, const char* this_node_name, - const char *cookie, short creation) +int ei_connect_init_ussi(ei_cnode* ec, const char* this_node_name, + const char *cookie, short creation, + ei_socket_callbacks *cbs, int cbs_sz, + void *setup_context) { char thishostname[EI_MAXHOSTNAMELEN+1]; char thisnodename[MAXNODELEN+1]; @@ -494,17 +741,8 @@ int ei_connect_init(ei_cnode* ec, const char* this_node_name, int ei_h_errno; int res; -#ifdef __WIN32__ - if (!initWinSock()) { - EI_TRACE_ERR0("ei_connect_xinit","can't initiate winsock"); - return ERL_ERROR; - } -#endif /* win32 */ -#ifdef _REENTRANT - if (ei_sockets_lock == NULL) { - ei_sockets_lock = ei_mutex_create(); - } -#endif /* _REENTRANT */ + if (!ei_connect_initialized) + init_connect(!0); /* gethostname requires len to be max(hostname) + 1 */ if (gethostname(thishostname, EI_MAXHOSTNAMELEN+1) == -1) { @@ -561,43 +799,22 @@ int ei_connect_init(ei_cnode* ec, const char* this_node_name, sprintf(thisnodename, "%s@%s", this_node_name, hp->h_name); } } - res = ei_connect_xinit(ec, thishostname, thisalivename, thisnodename, - (struct in_addr *)*hp->h_addr_list, cookie, creation); + res = ei_connect_xinit_ussi(ec, thishostname, thisalivename, thisnodename, + (struct in_addr *)*hp->h_addr_list, cookie, creation, + cbs, cbs_sz, setup_context); if (buf != buffer) free(buf); return res; } - -/* connects to port at ip-address ip_addr -* and returns fd to socket -* port has to be in host byte order -*/ -static int cnct(uint16 port, struct in_addr *ip_addr, int addr_len, unsigned ms) +int ei_connect_init(ei_cnode* ec, const char* this_node_name, + const char *cookie, short creation) { - int s, res; - struct sockaddr_in iserv_addr; - - if ((s = socket(AF_INET, SOCK_STREAM, 0)) < 0) { - erl_errno = errno; - return ERL_ERROR; - } - - memset((char*)&iserv_addr, 0, sizeof(struct sockaddr_in)); - memcpy((char*)&iserv_addr.sin_addr, (char*)ip_addr, addr_len); - iserv_addr.sin_family = AF_INET; - iserv_addr.sin_port = htons(port); - - if ((res = ei_connect_t(s, (struct sockaddr*)&iserv_addr, - sizeof(iserv_addr),ms)) < 0) { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - closesocket(s); - return ERL_ERROR; - } - - return s; -} /* cnct */ - + return ei_connect_init_ussi(ec, this_node_name, cookie, creation, + &ei_default_socket_callbacks, + sizeof(ei_default_socket_callbacks), + NULL); +} /* * Same as ei_gethostbyname_r, but also handles ERANGE error @@ -758,91 +975,218 @@ int ei_connect(ei_cnode* ec, char *nodename) * the node through epmd at that host * */ -int ei_xconnect_tmo(ei_cnode* ec, Erl_IpAddr adr, char *alivename, unsigned ms) +int ei_xconnect_tmo(ei_cnode* ec, Erl_IpAddr ip_addr, char *alivename, unsigned ms) { - struct in_addr *ip_addr=(struct in_addr *) adr; + ei_socket_callbacks *cbs = ec->cbs; + void *ctx; int rport = 0; /*uint16 rport = 0;*/ int sockd; - int one = 1; int dist = 0; - ErlConnect her_name; unsigned her_flags, her_version; - + unsigned our_challenge, her_challenge; + unsigned char our_digest[16]; + int err; + int pkt_sz; + struct sockaddr_in addr; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; + erl_errno = EIO; /* Default error code */ EI_TRACE_CONN1("ei_xconnect","-> CONNECT attempt to connect to %s", alivename); - if ((rport = ei_epmd_port_tmo(ip_addr,alivename,&dist, ms)) < 0) { + if ((rport = ei_epmd_port_tmo(ip_addr,alivename,&dist, tmo)) < 0) { EI_TRACE_ERR0("ei_xconnect","-> CONNECT can't get remote port"); /* ei_epmd_port_tmo() has set erl_errno */ return ERL_NO_PORT; } - - /* we now have port number to enode, try to connect */ - if((sockd = cnct((uint16)rport, ip_addr, sizeof(struct in_addr),ms)) < 0) { - EI_TRACE_ERR0("ei_xconnect","-> CONNECT socket connect failed"); - /* cnct() has set erl_errno */ - return ERL_CONNECT_FAIL; - } - - EI_TRACE_CONN0("ei_xconnect","-> CONNECT connected to remote"); - /* FIXME why connect before checking 'dist' output from ei_epmd_port() ?! */ if (dist <= 4) { EI_TRACE_ERR0("ei_xconnect","-> CONNECT remote version not compatible"); - goto error; + return ERL_ERROR; } - else { - unsigned our_challenge, her_challenge; - unsigned char our_digest[16]; - - if (send_name(sockd, ec->thisnodename, (unsigned) dist, ms)) - goto error; - if (recv_status(sockd, ms)) - goto error; - if (recv_challenge(sockd, &her_challenge, &her_version, - &her_flags, &her_name, ms)) - goto error; - our_challenge = gen_challenge(); - gen_digest(her_challenge, ec->ei_connect_cookie, our_digest); - if (send_challenge_reply(sockd, our_digest, our_challenge, ms)) - goto error; - if (recv_challenge_ack(sockd, our_challenge, - ec->ei_connect_cookie, ms)) - goto error; - put_ei_socket_info(sockd, dist, null_cookie, ec); /* FIXME check == 0 */ + + err = ei_socket_ctx__(cbs, &ctx, ec->setup_context); + if (err) { + EI_TRACE_ERR2("ei_xconnect","-> SOCKET failed: %s (%d)", + estr(err), err); + erl_errno = err; + return ERL_CONNECT_FAIL; + } + + memset((void *) &addr, 0, sizeof(struct sockaddr_in)); + memcpy((void *) &addr.sin_addr, (void *) ip_addr, sizeof(addr.sin_addr)); + addr.sin_family = AF_INET; + addr.sin_port = htons(rport); + + err = ei_connect_ctx_t__(cbs, ctx, (void *) &addr, sizeof(addr), tmo); + if (err) { + EI_TRACE_ERR2("ei_xconnect","-> CONNECT socket connect failed: %s (%d)", + estr(err), err); + abort_connection(cbs, ctx); + erl_errno = err; + return ERL_CONNECT_FAIL; } - setsockopt(sockd, IPPROTO_TCP, TCP_NODELAY, (char *)&one, sizeof(one)); - setsockopt(sockd, SOL_SOCKET, SO_KEEPALIVE, (char *)&one, sizeof(one)); + EI_TRACE_CONN0("ei_xconnect","-> CONNECT connected to remote"); - EI_TRACE_CONN1("ei_xconnect","-> CONNECT (ok) remote = %s",alivename); + err = EI_GET_FD__(cbs, ctx, &sockd); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + goto error; + } + + err = cbs->handshake_packet_header_size(ctx, &pkt_sz); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + goto error; + } + + if (send_name(cbs, ctx, pkt_sz, ec->thisnodename, (unsigned) dist, tmo)) + goto error; + if (recv_status(cbs, ctx, pkt_sz, tmo)) + goto error; + if (recv_challenge(cbs, ctx, pkt_sz, &her_challenge, + &her_version, &her_flags, NULL, tmo)) + goto error; + our_challenge = gen_challenge(); + gen_digest(her_challenge, ec->ei_connect_cookie, our_digest); + if (send_challenge_reply(cbs, ctx, pkt_sz, our_digest, our_challenge, tmo)) + goto error; + if (recv_challenge_ack(cbs, ctx, pkt_sz, our_challenge, + ec->ei_connect_cookie, tmo)) + goto error; + if (put_ei_socket_info(sockd, dist, null_cookie, ec, cbs, ctx) != 0) + goto error; + + if (cbs->connect_handshake_complete) { + err = cbs->connect_handshake_complete(ctx); + if (err) { + EI_TRACE_ERR2("ei_xconnect","-> CONNECT failed: %s (%d)", + estr(err), err); + close_connection(cbs, ctx, sockd); + EI_CONN_SAVE_ERRNO__(err); + return ERL_ERROR; + } + } + EI_TRACE_CONN1("ei_xconnect","-> CONNECT (ok) remote = %s",alivename); + erl_errno = 0; return sockd; error: EI_TRACE_ERR0("ei_xconnect","-> CONNECT failed"); - closesocket(sockd); + abort_connection(cbs, ctx); return ERL_ERROR; } /* ei_xconnect */ -int ei_xconnect(ei_cnode* ec, Erl_IpAddr adr, char *alivename) +int ei_xconnect(ei_cnode* ec, Erl_IpAddr ip_addr, char *alivename) { - return ei_xconnect_tmo(ec, adr, alivename, 0); + return ei_xconnect_tmo(ec, ip_addr, alivename, 0); } +int ei_listen(ei_cnode *ec, int *port, int backlog) +{ + struct in_addr ip_addr; + ip_addr.s_addr = htonl(INADDR_ANY); + return ei_xlisten(ec, &ip_addr, port, backlog); +} + +int ei_xlisten(ei_cnode *ec, struct in_addr *ip_addr, int *port, int backlog) +{ + ei_socket_callbacks *cbs = ec->cbs; + struct sockaddr_in sock_addr; + void *ctx; + int fd, err, len; + + err = ei_socket_ctx__(cbs, &ctx, ec->setup_context); + if (err) { + EI_TRACE_ERR2("ei_xlisten","-> SOCKET failed: %s (%d)", + estr(err), err); + erl_errno = err; + return ERL_ERROR; + } + + memset((void *) &sock_addr, 0, sizeof(struct sockaddr_in)); + memcpy((void *) &sock_addr.sin_addr, (void *) ip_addr, sizeof(*ip_addr)); + sock_addr.sin_family = AF_INET; + sock_addr.sin_port = htons((short) *port); + + len = sizeof(sock_addr); + err = ei_listen_ctx__(cbs, ctx, (void *) &sock_addr, &len, backlog); + if (err) { + EI_TRACE_ERR2("ei_xlisten","-> listen failed: %s (%d)", + estr(err), err); + erl_errno = err; + goto error; + } + + if (len != sizeof(sock_addr)) { + if (len < offsetof(struct sockaddr_in, sin_addr) + sizeof(sock_addr.sin_addr) + || len < offsetof(struct sockaddr_in, sin_port) + sizeof(sock_addr.sin_port)) { + erl_errno = EIO; + EI_TRACE_ERR0("ei_xlisten","-> get info failed"); + goto error; + } + } + + memcpy((void *) ip_addr, (void *) &sock_addr.sin_addr, sizeof(*ip_addr)); + *port = (int) ntohs(sock_addr.sin_port); + + err = EI_GET_FD__(cbs, ctx, &fd); + if (err) { + erl_errno = err; + goto error; + } + + if (put_ei_socket_info(fd, 0, null_cookie, ec, cbs, ctx) != 0) { + EI_TRACE_ERR0("ei_xlisten","-> save socket info failed"); + erl_errno = EIO; + goto error; + } + + erl_errno = 0; + + return fd; + +error: + abort_connection(cbs, ctx); + return ERL_ERROR; +} + +static int close_connection(ei_socket_callbacks *cbs, void *ctx, int fd) +{ + int err; + remove_ei_socket_info(fd); + err = ei_close_ctx__(cbs, ctx); + if (err) { + erl_errno = err; + return ERL_ERROR; + } + return 0; +} - /* - * For symmetry reasons -*/ -#if 0 int ei_close_connection(int fd) { - return closesocket(fd); + ei_socket_callbacks *cbs; + void *ctx; + int err = EI_GET_CBS_CTX__(&cbs, &ctx, fd); + if (err) + erl_errno = err; + else { + if (close_connection(cbs, ctx, fd) == 0) + return 0; + } + EI_TRACE_ERR2("ei_close_connection","<- CLOSE socket close failed: %s (%d)", + estr(erl_errno), erl_errno); + return ERL_ERROR; } /* ei_close_connection */ -#endif + +static void abort_connection(ei_socket_callbacks *cbs, void *ctx) +{ + (void) ei_close_ctx__(cbs, ctx); +} /* * Accept and initiate a connection from another @@ -857,25 +1201,71 @@ int ei_accept(ei_cnode* ec, int lfd, ErlConnect *conp) int ei_accept_tmo(ei_cnode* ec, int lfd, ErlConnect *conp, unsigned ms) { int fd; - struct sockaddr_in cli_addr; - int cli_addr_len=sizeof(struct sockaddr_in); unsigned her_version, her_flags; - ErlConnect her_name; + char tmp_nodename[MAXNODELEN+1]; + char *her_name; + int pkt_sz, err; + struct sockaddr_in addr; + int addr_len = sizeof(struct sockaddr_in); + ei_socket_callbacks *cbs; + void *ctx; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; erl_errno = EIO; /* Default error code */ + + err = EI_GET_CBS_CTX__(&cbs, &ctx, lfd); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return ERL_ERROR; + } + EI_TRACE_CONN0("ei_accept","<- ACCEPT waiting for connection"); + + if (conp) { + her_name = &conp->nodename[0]; + } + else { + her_name = &tmp_nodename[0]; + } - if ((fd = ei_accept_t(lfd, (struct sockaddr*) &cli_addr, - &cli_addr_len, ms )) < 0) { - EI_TRACE_ERR0("ei_accept","<- ACCEPT socket accept failed"); - erl_errno = (fd == -2) ? ETIMEDOUT : EIO; - goto error; + /* + * ei_accept_ctx_t__() replaces the pointer to the listen context + * with a pointer to the accepted connection context on success. + */ + err = ei_accept_ctx_t__(cbs, &ctx, (void *) &addr, &addr_len, tmo); + if (err) { + EI_TRACE_ERR2("ei_accept","<- ACCEPT socket accept failed: %s (%d)", + estr(err), err); + EI_CONN_SAVE_ERRNO__(err); + return ERL_ERROR; + } + + err = EI_GET_FD__(cbs, ctx, &fd); + if (err) { + EI_TRACE_ERR2("ei_accept","<- ACCEPT get fd failed: %s (%d)", + estr(err), err); + EI_CONN_SAVE_ERRNO__(err); + } + + if (addr_len != sizeof(struct sockaddr_in)) { + if (addr_len < (offsetof(struct sockaddr_in, sin_addr) + + sizeof(addr.sin_addr))) { + EI_TRACE_ERR0("ei_accept","<- ACCEPT get addr failed"); + goto error; + } + } + + err = cbs->handshake_packet_header_size(ctx, &pkt_sz); + if (err) { + EI_TRACE_ERR2("ei_accept","<- ACCEPT get packet size failed: %s (%d)", + estr(err), err); + EI_CONN_SAVE_ERRNO__(err); } EI_TRACE_CONN0("ei_accept","<- ACCEPT connected to remote"); - if (recv_name(fd, &her_version, &her_flags, &her_name, ms)) { + if (recv_name(cbs, ctx, pkt_sz, &her_version, &her_flags, her_name, tmo)) { EI_TRACE_ERR0("ei_accept","<- ACCEPT initial ident failed"); goto error; } @@ -888,34 +1278,45 @@ int ei_accept_tmo(ei_cnode* ec, int lfd, ErlConnect *conp, unsigned ms) unsigned our_challenge; unsigned her_challenge; unsigned char our_digest[16]; - - if (send_status(fd,"ok", ms)) + + if (send_status(cbs, ctx, pkt_sz, "ok", tmo)) goto error; our_challenge = gen_challenge(); - if (send_challenge(fd, ec->thisnodename, - our_challenge, her_version, ms)) + if (send_challenge(cbs, ctx, pkt_sz, ec->thisnodename, + our_challenge, her_version, tmo)) goto error; - if (recv_challenge_reply(fd, our_challenge, - ec->ei_connect_cookie, - &her_challenge, ms)) + if (recv_challenge_reply(cbs, ctx, pkt_sz, our_challenge, + ec->ei_connect_cookie, &her_challenge, tmo)) goto error; gen_digest(her_challenge, ec->ei_connect_cookie, our_digest); - if (send_challenge_ack(fd, our_digest, ms)) + if (send_challenge_ack(cbs, ctx, pkt_sz, our_digest, tmo)) goto error; - put_ei_socket_info(fd, her_version, null_cookie, ec); + if (put_ei_socket_info(fd, her_version, null_cookie, ec, cbs, ctx) != 0) + goto error; + } + if (conp) { + memcpy((void *) conp->ipadr, (void *) &addr.sin_addr, sizeof(conp->ipadr)); + } + + if (cbs->accept_handshake_complete) { + err = cbs->accept_handshake_complete(ctx); + if (err) { + EI_TRACE_ERR2("ei_xconnect","-> ACCEPT handshake failed: %s (%d)", + estr(err), err); + close_connection(cbs, ctx, fd); + EI_CONN_SAVE_ERRNO__(err); + return ERL_ERROR; + } } - if (conp) - *conp = her_name; - EI_TRACE_CONN1("ei_accept","<- ACCEPT (ok) remote = %s",her_name.nodename); + EI_TRACE_CONN1("ei_accept","<- ACCEPT (ok) remote = %s",her_name); erl_errno = 0; /* No error */ return fd; error: EI_TRACE_ERR0("ei_accept","<- ACCEPT failed"); - if (fd>=0) - closesocket(fd); + abort_connection(cbs, ctx); return ERL_ERROR; } /* ei_accept */ @@ -927,36 +1328,57 @@ error: */ int ei_receive_tmo(int fd, unsigned char *bufp, int bufsize, unsigned ms) { - int len; + ssize_t len; unsigned char fourbyte[4]={0,0,0,0}; - int res; - - if ((res = ei_read_fill_t(fd, (char *) bufp, 4, ms)) != 4) { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; + int err; + ei_socket_callbacks *cbs; + void *ctx; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; + + err = EI_GET_CBS_CTX__(&cbs, &ctx, fd); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return ERL_ERROR; + } + + len = (ssize_t) 4; + err = ei_read_fill_ctx_t__(cbs, ctx, (char *) bufp, &len, tmo); + if (!err && len != (ssize_t) 4) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); return ERL_ERROR; } /* Tick handling */ - if ((len = get_int32(bufp)) == ERL_TICK) - { - ei_write_fill_t(fd, (char *) fourbyte, 4, ms); + len = get_int32(bufp); + if (len == ERL_TICK) { + len = 4; + ei_write_fill_ctx_t__(cbs, ctx, (char *) fourbyte, &len, tmo); /* FIXME ok to ignore error or timeout? */ erl_errno = EAGAIN; return ERL_TICK; } - else if (len > bufsize) - { + + if (len > bufsize) { /* FIXME: We should drain the message. */ erl_errno = EMSGSIZE; return ERL_ERROR; } - else if ((res = ei_read_fill_t(fd, (char *) bufp, len, ms)) != len) - { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return ERL_ERROR; + else { + ssize_t need = len; + err = ei_read_fill_ctx_t__(cbs, ctx, (char *) bufp, &len, tmo); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return ERL_ERROR; + } + if (len != need) { + erl_errno = EIO; + return ERL_ERROR; + } } - return len; + return (int) len; } @@ -1112,36 +1534,11 @@ int ei_rpc_to(ei_cnode *ec, int fd, char *mod, char *fun, int ei_rpc_from(ei_cnode *ec, int fd, int timeout, erlang_msg *msg, ei_x_buff *x) { - fd_set readmask; - struct timeval tv; - struct timeval *t = NULL; - - if (timeout >= 0) { - tv.tv_sec = timeout / 1000; - tv.tv_usec = (timeout % 1000) * 1000; - t = &tv; - } - - FD_ZERO(&readmask); - FD_SET(fd,&readmask); - - switch (select(fd+1, &readmask, NULL, NULL, t)) { - case -1: - erl_errno = EIO; - return ERL_ERROR; - - case 0: - erl_errno = ETIMEDOUT; - return ERL_TIMEOUT; - - default: - if (FD_ISSET(fd, &readmask)) { - return ei_xreceive_msg(fd, msg, x); - } else { - erl_errno = EIO; - return ERL_ERROR; - } - } + unsigned tmo = timeout < 0 ? EI_SCLBK_INF_TMO : (unsigned) timeout; + int res = ei_xreceive_msg_tmo(fd, msg, x, tmo); + if (res < 0 && erl_errno == ETIMEDOUT) + return ERL_TIMEOUT; + return res; } /* rpc_from */ /* @@ -1295,19 +1692,34 @@ static char *hex(char digest[16], char buff[33]) return buff; } -static int read_2byte_package(int fd, char **buf, int *buflen, - int *is_static, unsigned ms) +static int read_hs_package(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, char **buf, int *buflen, + int *is_static, unsigned ms) { - unsigned char nbuf[2]; + unsigned char nbuf[4]; unsigned char *x = nbuf; - unsigned len; - int res; - - if((res = ei_read_fill_t(fd, (char *)nbuf, 2, ms)) != 2) { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; + ssize_t len, need; + int err; + + len = (ssize_t) pkt_sz; + err = ei_read_fill_ctx_t__(cbs, ctx, (char *)nbuf, &len, ms); + if (!err && len != (ssize_t) pkt_sz) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); return -1; } - len = get16be(x); + + switch (pkt_sz) { + case 2: + len = get16be(x); + break; + case 4: + len = get32be(x); + break; + default: + return -1; + } if (len > *buflen) { if (*is_static) { @@ -1329,20 +1741,26 @@ static int read_2byte_package(int fd, char **buf, int *buflen, *buflen = len; } } - if ((res = ei_read_fill_t(fd, *buf, len, ms)) != len) { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; + need = len; + err = ei_read_fill_ctx_t__(cbs, ctx, *buf, &len, ms); + if (!err && len != need) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); return -1; } return len; } -static int send_status(int fd, char *status, unsigned ms) +static int send_status(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, char *status, unsigned ms) { char *buf, *s; char dbuf[DEFBUF_SIZ]; - int siz = strlen(status) + 1 + 2; - int res; + int siz = strlen(status) + 1 + pkt_sz; + int err; + ssize_t len; buf = (siz > DEFBUF_SIZ) ? malloc(siz) : dbuf; if (!buf) { @@ -1350,14 +1768,28 @@ static int send_status(int fd, char *status, unsigned ms) return -1; } s = buf; - put16be(s,siz - 2); + switch (pkt_sz) { + case 2: + put16be(s,siz - 2); + break; + case 4: + put32be(s,siz - 4); + break; + default: + return -1; + } put8(s, 's'); memcpy(s, status, strlen(status)); - if ((res = ei_write_fill_t(fd, buf, siz, ms)) != siz) { - EI_TRACE_ERR0("send_status","-> SEND_STATUS socket write failed"); + len = (ssize_t) siz; + err = ei_write_fill_ctx_t__(cbs, ctx, buf, &len, ms); + if (!err && len != (ssize_t) siz) + err = EIO; + if (err) { + EI_TRACE_ERR2("send_status","-> SEND_STATUS socket write failed: %s (%d)", + estr(err), err); if (buf != dbuf) - free(buf); - erl_errno = (res == -2) ? ETIMEDOUT : EIO; + free(buf); + EI_CONN_SAVE_ERRNO__(err); return -1; } EI_TRACE_CONN1("send_status","-> SEND_STATUS (%s)",status); @@ -1367,7 +1799,8 @@ static int send_status(int fd, char *status, unsigned ms) return 0; } -static int recv_status(int fd, unsigned ms) +static int recv_status(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, unsigned ms) { char dbuf[DEFBUF_SIZ]; char *buf = dbuf; @@ -1375,7 +1808,8 @@ static int recv_status(int fd, unsigned ms) int buflen = DEFBUF_SIZ; int rlen; - if ((rlen = read_2byte_package(fd, &buf, &buflen, &is_static, ms)) <= 0) { + if ((rlen = read_hs_package(cbs, ctx, pkt_sz, + &buf, &buflen, &is_static, ms)) <= 0) { EI_TRACE_ERR1("recv_status", "<- RECV_STATUS socket read failed (%d)", rlen); goto error; @@ -1396,7 +1830,10 @@ error: return -1; } -static int send_name_or_challenge(int fd, char *nodename, +static int send_name_or_challenge(ei_socket_callbacks *cbs, + void *ctx, + int pkt_sz, + char *nodename, int f_chall, unsigned challenge, unsigned version, @@ -1405,9 +1842,10 @@ static int send_name_or_challenge(int fd, char *nodename, char *buf; unsigned char *s; char dbuf[DEFBUF_SIZ]; - int siz = 2 + 1 + 2 + 4 + strlen(nodename); + int siz = pkt_sz + 1 + 2 + 4 + strlen(nodename); const char* function[] = {"SEND_NAME", "SEND_CHALLENGE"}; - int res; + int err; + ssize_t len; if (f_chall) siz += 4; @@ -1417,7 +1855,16 @@ static int send_name_or_challenge(int fd, char *nodename, return -1; } s = (unsigned char *)buf; - put16be(s,siz - 2); + switch (pkt_sz) { + case 2: + put16be(s,siz - 2); + break; + case 4: + put32be(s,siz - 4); + break; + default: + return -1; + } put8(s, 'n'); put16be(s, version); put32be(s, (DFLAG_EXTENDED_REFERENCES @@ -1433,13 +1880,16 @@ static int send_name_or_challenge(int fd, char *nodename, if (f_chall) put32be(s, challenge); memcpy(s, nodename, strlen(nodename)); - - if ((res = ei_write_fill_t(fd, buf, siz, ms)) != siz) { + len = (ssize_t) siz; + err = ei_write_fill_ctx_t__(cbs, ctx, buf, &len, ms); + if (!err && len != (ssize_t) siz) + err = EIO; + if (err) { EI_TRACE_ERR1("send_name_or_challenge", "-> %s socket write failed", function[f_chall]); if (buf != dbuf) free(buf); - erl_errno = (res == -2) ? ETIMEDOUT : EIO; + EI_CONN_SAVE_ERRNO__(err); return -1; } @@ -1448,9 +1898,9 @@ static int send_name_or_challenge(int fd, char *nodename, return 0; } -static int recv_challenge(int fd, unsigned *challenge, - unsigned *version, - unsigned *flags, ErlConnect *namebuf, unsigned ms) +static int recv_challenge(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, unsigned *challenge, unsigned *version, + unsigned *flags, char *namebuf, unsigned ms) { char dbuf[DEFBUF_SIZ]; char *buf = dbuf; @@ -1458,13 +1908,13 @@ static int recv_challenge(int fd, unsigned *challenge, int buflen = DEFBUF_SIZ; int rlen; char *s; - struct sockaddr_in sin; - socklen_t sin_len = sizeof(sin); char tag; - + char tmp_nodename[MAXNODELEN+1]; + erl_errno = EIO; /* Default */ - if ((rlen = read_2byte_package(fd, &buf, &buflen, &is_static, ms)) <= 0) { + if ((rlen = read_hs_package(cbs, ctx, pkt_sz, &buf, &buflen, + &is_static, ms)) <= 0) { EI_TRACE_ERR1("recv_challenge", "<- RECV_CHALLENGE socket read failed (%d)",rlen); goto error; @@ -1505,22 +1955,19 @@ static int recv_challenge(int fd, unsigned *challenge, goto error; } - if (getpeername(fd, (struct sockaddr *) &sin, &sin_len) < 0) { - EI_TRACE_ERR0("recv_challenge","<- RECV_CHALLENGE can't get peername"); - erl_errno = errno; - goto error; - } - memcpy(namebuf->ipadr, &(sin.sin_addr.s_addr), - sizeof(sin.sin_addr.s_addr)); - memcpy(namebuf->nodename, s, rlen - 11); - namebuf->nodename[rlen - 11] = '\0'; + if (!namebuf) + namebuf = &tmp_nodename[0]; + + memcpy(namebuf, s, rlen - 11); + namebuf[rlen - 11] = '\0'; + if (!is_static) free(buf); EI_TRACE_CONN4("recv_challenge","<- RECV_CHALLENGE (ok) node = %s, " "version = %u, " "flags = %u, " "challenge = %d", - namebuf->nodename, + namebuf, *version, *flags, *challenge @@ -1533,24 +1980,40 @@ error: return -1; } -static int send_challenge_reply(int fd, unsigned char digest[16], +static int send_challenge_reply(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, unsigned char digest[16], unsigned challenge, unsigned ms) { char *s; char buf[DEFBUF_SIZ]; - int siz = 2 + 1 + 4 + 16; - int res; + int siz = pkt_sz + 1 + 4 + 16; + int err; + ssize_t len; s = buf; - put16be(s,siz - 2); + switch (pkt_sz) { + case 2: + put16be(s,siz - 2); + break; + case 4: + put32be(s,siz - 4); + break; + default: + return -1; + } put8(s, 'r'); put32be(s, challenge); memcpy(s, digest, 16); - - if ((res = ei_write_fill_t(fd, buf, siz, ms)) != siz) { - EI_TRACE_ERR0("send_challenge_reply", - "-> SEND_CHALLENGE_REPLY socket write failed"); - erl_errno = (res == -2) ? ETIMEDOUT : EIO; + + len = (ssize_t) siz; + err = ei_write_fill_ctx_t__(cbs, ctx, buf, &len, ms); + if (!err && len != (ssize_t) siz) + err = EIO; + if (err) { + EI_TRACE_ERR2("send_challenge_reply", + "-> SEND_CHALLENGE_REPLY socket write failed: %s (%d)", + estr(err), err); + EI_CONN_SAVE_ERRNO__(err); return -1; } @@ -1563,11 +2026,13 @@ static int send_challenge_reply(int fd, unsigned char digest[16], return 0; } -static int recv_challenge_reply (int fd, - unsigned our_challenge, - char cookie[], - unsigned *her_challenge, - unsigned ms) +static int recv_challenge_reply(ei_socket_callbacks *cbs, + void *ctx, + int pkt_sz, + unsigned our_challenge, + char cookie[], + unsigned *her_challenge, + unsigned ms) { char dbuf[DEFBUF_SIZ]; char *buf = dbuf; @@ -1580,7 +2045,7 @@ static int recv_challenge_reply (int fd, erl_errno = EIO; /* Default */ - if ((rlen = read_2byte_package(fd, &buf, &buflen, &is_static, ms)) != 21) { + if ((rlen = read_hs_package(cbs, ctx, pkt_sz, &buf, &buflen, &is_static, ms)) != 21) { EI_TRACE_ERR1("recv_challenge_reply", "<- RECV_CHALLENGE_REPLY socket read failed (%d)",rlen); goto error; @@ -1620,23 +2085,38 @@ error: return -1; } -static int send_challenge_ack(int fd, unsigned char digest[16], unsigned ms) +static int send_challenge_ack(ei_socket_callbacks *cbs, void *ctx, int pkt_sz, + unsigned char digest[16], unsigned ms) { char *s; char buf[DEFBUF_SIZ]; - int siz = 2 + 1 + 16; - int res; + int siz = pkt_sz + 1 + 16; + int err; + ssize_t len; s = buf; - - put16be(s,siz - 2); + switch (pkt_sz) { + case 2: + put16be(s,siz - 2); + break; + case 4: + put32be(s,siz - 4); + break; + default: + return -1; + } put8(s, 'a'); memcpy(s, digest, 16); - if ((res = ei_write_fill_t(fd, buf, siz, ms)) != siz) { - EI_TRACE_ERR0("recv_challenge_reply", - "-> SEND_CHALLENGE_ACK socket write failed"); - erl_errno = (res == -2) ? ETIMEDOUT : EIO; + len = (ssize_t) siz; + err = ei_write_fill_ctx_t__(cbs, ctx, buf, &len, ms); + if (!err && len != (ssize_t) siz) + err = EIO; + if (err) { + EI_TRACE_ERR2("recv_challenge_reply", + "-> SEND_CHALLENGE_ACK socket write failed: %s (%d)", + estr(err), err); + EI_CONN_SAVE_ERRNO__(err); return -1; } @@ -1649,8 +2129,8 @@ static int send_challenge_ack(int fd, unsigned char digest[16], unsigned ms) return 0; } -static int recv_challenge_ack(int fd, - unsigned our_challenge, +static int recv_challenge_ack(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, unsigned our_challenge, char cookie[], unsigned ms) { char dbuf[DEFBUF_SIZ]; @@ -1664,7 +2144,7 @@ static int recv_challenge_ack(int fd, erl_errno = EIO; /* Default */ - if ((rlen = read_2byte_package(fd, &buf, &buflen, &is_static, ms)) != 17) { + if ((rlen = read_hs_package(cbs, ctx, pkt_sz, &buf, &buflen, &is_static, ms)) != 17) { EI_TRACE_ERR1("recv_challenge_ack", "<- RECV_CHALLENGE_ACK socket read failed (%d)",rlen); goto error; @@ -1701,20 +2181,24 @@ error: return -1; } -static int send_name(int fd, char *nodename, unsigned version, unsigned ms) +static int send_name(ei_socket_callbacks *cbs, void *ctx, int pkt_sz, + char *nodename, unsigned version, unsigned ms) { - return send_name_or_challenge(fd, nodename, 0, 0, version, ms); + return send_name_or_challenge(cbs, ctx, pkt_sz, nodename, 0, + 0, version, ms); } -static int send_challenge(int fd, char *nodename, - unsigned challenge, unsigned version, unsigned ms) +static int send_challenge(ei_socket_callbacks *cbs, void *ctx, int pkt_sz, + char *nodename, unsigned challenge, unsigned version, + unsigned ms) { - return send_name_or_challenge(fd, nodename, 1, challenge, version, ms); + return send_name_or_challenge(cbs, ctx, pkt_sz, nodename, 1, + challenge, version, ms); } -static int recv_name(int fd, - unsigned *version, - unsigned *flags, ErlConnect *namebuf, unsigned ms) +static int recv_name(ei_socket_callbacks *cbs, void *ctx, + int pkt_sz, unsigned *version, + unsigned *flags, char *namebuf, unsigned ms) { char dbuf[DEFBUF_SIZ]; char *buf = dbuf; @@ -1722,13 +2206,13 @@ static int recv_name(int fd, int buflen = DEFBUF_SIZ; int rlen; char *s; - struct sockaddr_in sin; - socklen_t sin_len = sizeof(sin); + char tmp_nodename[MAXNODELEN+1]; char tag; erl_errno = EIO; /* Default */ - if ((rlen = read_2byte_package(fd, &buf, &buflen, &is_static, ms)) <= 0) { + if ((rlen = read_hs_package(cbs, ctx, pkt_sz, &buf, &buflen, + &is_static, ms)) <= 0) { EI_TRACE_ERR1("recv_name","<- RECV_NAME socket read failed (%d)",rlen); goto error; } @@ -1759,21 +2243,18 @@ static int recv_name(int fd, erl_errno = EIO; goto error; } - - if (getpeername(fd, (struct sockaddr *) &sin, &sin_len) < 0) { - EI_TRACE_ERR0("recv_name","<- RECV_NAME can't get peername"); - erl_errno = errno; - goto error; - } - memcpy(namebuf->ipadr, &(sin.sin_addr.s_addr), - sizeof(sin.sin_addr.s_addr)); - memcpy(namebuf->nodename, s, rlen - 7); - namebuf->nodename[rlen - 7] = '\0'; + + if (!namebuf) + namebuf = &tmp_nodename[0]; + + memcpy(namebuf, s, rlen - 7); + namebuf[rlen - 7] = '\0'; + if (!is_static) free(buf); EI_TRACE_CONN3("recv_name", "<- RECV_NAME (ok) node = %s, version = %u, flags = %u", - namebuf->nodename,*version,*flags); + namebuf,*version,*flags); erl_errno = 0; return 0; @@ -1867,3 +2348,4 @@ static int get_cookie(char *buf, int bufsize) return 1; /* Success! */ } + diff --git a/lib/erl_interface/src/connect/ei_resolve.c b/lib/erl_interface/src/connect/ei_resolve.c index 022a43d255..225fddc784 100644 --- a/lib/erl_interface/src/connect/ei_resolve.c +++ b/lib/erl_interface/src/connect/ei_resolve.c @@ -57,9 +57,9 @@ #ifdef HAVE_GETHOSTBYNAME_R -void ei_init_resolve(void) +int ei_init_resolve(void) { - return; /* Do nothing */ + return 0; /* Do nothing */ } #else /* !HAVE_GETHOSTBYNAME_R */ @@ -103,7 +103,7 @@ static int verify_dns_configuration(void); * our own, which are just wrappers around hostGetByName() and * hostGetByAddr(). Here we look up the functions. */ -void ei_init_resolve(void) +int ei_init_resolve(void) { #ifdef VXWORKS @@ -134,9 +134,12 @@ void ei_init_resolve(void) #ifdef _REENTRANT ei_gethost_sem = ei_mutex_create(); + if (!ei_gethost_sem) + return ENOMEM; #endif /* _REENTRANT */ ei_resolve_initialized = 1; + return 0; } #ifdef VXWORKS @@ -312,9 +315,11 @@ static struct hostent *my_gethostbyname_r(const char *name, struct hostent *src; struct hostent *rval = NULL; - /* FIXME this should have been done in 'erl'_init()? */ - if (!ei_resolve_initialized) ei_init_resolve(); - + if (!ei_resolve_initialized) { + *h_errnop = NO_RECOVERY; + return NULL; + } + #ifdef _REENTRANT /* === BEGIN critical section === */ if (ei_mutex_lock(ei_gethost_sem,0) != 0) { @@ -377,7 +382,10 @@ static struct hostent *my_gethostbyaddr_r(const char *addr, struct hostent *rval = NULL; /* FIXME this should have been done in 'erl'_init()? */ - if (!ei_resolve_initialized) ei_init_resolve(); + if (!ei_resolve_initialized) { + *h_errnop = NO_RECOVERY; + return NULL; + } #ifdef _REENTRANT /* === BEGIN critical section === */ diff --git a/lib/erl_interface/src/connect/ei_resolve.h b/lib/erl_interface/src/connect/ei_resolve.h index 10a49ffbc6..5711d7da76 100644 --- a/lib/erl_interface/src/connect/ei_resolve.h +++ b/lib/erl_interface/src/connect/ei_resolve.h @@ -20,6 +20,6 @@ #ifndef _EI_RESOLVE_H #define _EI_RESOLVE_H -void ei_init_resolve(void); +int ei_init_resolve(void); #endif /* _EI_RESOLVE_H */ diff --git a/lib/erl_interface/src/connect/eirecv.c b/lib/erl_interface/src/connect/eirecv.c index 7b9dbfc387..47eea06ced 100644 --- a/lib/erl_interface/src/connect/eirecv.c +++ b/lib/erl_interface/src/connect/eirecv.c @@ -60,22 +60,36 @@ ei_recv_internal (int fd, int arity; int version; int index = 0; - int i = 0; - int res; + int err; int show_this_msg = 0; + ei_socket_callbacks *cbs; + void *ctx; + ssize_t rlen; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; + + err = EI_GET_CBS_CTX__(&cbs, &ctx, fd); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return -1; + } /* get length field */ - if ((res = ei_read_fill_t(fd, header, 4, ms)) != 4) - { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; + rlen = 4; + err = ei_read_fill_ctx_t__(cbs, ctx, header, &rlen, tmo); + if (!err && rlen != 4) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); return -1; } + len = get32be(s); /* got tick - respond and return */ if (!len) { char tock[] = {0,0,0,0}; - ei_write_fill_t(fd, tock, sizeof(tock), ms); /* Failure no problem */ + ssize_t wlen = sizeof(tock); + ei_write_fill_ctx_t__(cbs, ctx, tock, &wlen, tmo); /* Failure no problem */ *msglenp = 0; return 0; /* maybe flag ERL_EAGAIN [sverkerw] */ } @@ -86,9 +100,12 @@ ei_recv_internal (int fd, ei_trace(-1,NULL); /* read enough to get at least entire header */ - bytesread = (len > EIRECVBUF ? EIRECVBUF : len); - if ((i = ei_read_fill_t(fd,header,bytesread,ms)) != bytesread) { - erl_errno = (i == -2) ? ETIMEDOUT : EIO; + rlen = bytesread = (len > EIRECVBUF ? EIRECVBUF : len); + err = ei_read_fill_ctx_t__(cbs, ctx, header, &rlen, tmo); + if (!err && rlen != bytesread) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); return -1; } @@ -212,12 +229,17 @@ ei_recv_internal (int fd, */ if (msglen > *bufsz) { if (staticbufp) { - int sz = EIRECVBUF; /* flush in rest of packet */ while (remain > 0) { - if (remain < sz) sz = remain; - if ((i=ei_read_fill_t(fd,header,sz,ms)) <= 0) break; - remain -= i; + rlen = remain > EIRECVBUF ? EIRECVBUF : remain; + err = ei_read_fill_ctx_t__(cbs, ctx, header, &rlen, tmo); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return -1; + } + if (rlen == 0) + break; + remain -= rlen; } erl_errno = EMSGSIZE; return -1; @@ -247,11 +269,15 @@ ei_recv_internal (int fd, /* read the rest of the message into callers buffer */ if (remain > 0) { - if ((i = ei_read_fill_t(fd,mbuf+bytesread-index,remain,ms)) != remain) { - *msglenp = bytesread-index+1; /* actual bytes in users buffer */ - erl_errno = (i == -2) ? ETIMEDOUT : EIO; - return -1; - } + rlen = remain; + err = ei_read_fill_ctx_t__(cbs, ctx, mbuf+bytesread-index, &rlen, tmo); + if (!err && rlen != remain) + err = EIO; + if (err) { + *msglenp = bytesread-index+1; /* actual bytes in users buffer */ + EI_CONN_SAVE_ERRNO__(err); + return -1; + } } if (show_this_msg) diff --git a/lib/erl_interface/src/connect/send.c b/lib/erl_interface/src/connect/send.c index 37d7db6d68..d97532d123 100644 --- a/lib/erl_interface/src/connect/send.c +++ b/lib/erl_interface/src/connect/send.c @@ -58,10 +58,17 @@ int ei_send_encoded_tmo(int fd, const erlang_pid *to, char *s, header[1200]; /* see size calculation below */ erlang_trace *token = NULL; int index = 5; /* reserve 5 bytes for control message */ - int res; -#ifdef HAVE_WRITEV - struct iovec v[2]; -#endif + int err; + ei_socket_callbacks *cbs; + void *ctx; + ssize_t len, tot_len; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; + + err = EI_GET_CBS_CTX__(&cbs, &ctx, fd); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return ERL_ERROR; + } /* are we tracing? */ /* check that he can receive trace tokens first */ @@ -91,30 +98,47 @@ int ei_send_encoded_tmo(int fd, const erlang_pid *to, if (ei_tracelevel >= 4) ei_show_sendmsg(stderr,header,msg); -#ifdef HAVE_WRITEV - - v[0].iov_base = (char *)header; - v[0].iov_len = index; - v[1].iov_base = (char *)msg; - v[1].iov_len = msglen; - - if ((res = ei_writev_fill_t(fd,v,2,ms)) != index+msglen) { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return -1; - } - -#else /* !HAVE_WRITEV */ - - if ((res = ei_write_fill_t(fd,header,index,ms)) != index) { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return -1; + +#ifdef EI_HAVE_STRUCT_IOVEC__ + if (ei_socket_callbacks_have_writev__(cbs)) { + struct iovec v[2]; + + v[0].iov_base = (char *)header; + v[0].iov_len = index; + v[1].iov_base = (char *)msg; + v[1].iov_len = msglen; + + len = tot_len = (ssize_t) index+msglen; + err = ei_writev_fill_ctx_t__(cbs, ctx, v, 2, &len, tmo); + if (!err && len != tot_len) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return -1; + } + + return 0; } - if ((res = ei_write_fill_t(fd,msg,msglen,ms)) != msglen) { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return -1; +#endif /* EI_HAVE_STRUCT_IOVEC__ */ + + /* no writev() */ + len = tot_len = (ssize_t) index; + err = ei_write_fill_ctx_t__(cbs, ctx, header, &len, tmo); + if (!err && len != tot_len) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return -1; } -#endif /* !HAVE_WRITEV */ + len = tot_len = (ssize_t) msglen; + err = ei_write_fill_ctx_t__(cbs, ctx, msg, &len, tmo); + if (!err && len != tot_len) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return -1; + } return 0; } diff --git a/lib/erl_interface/src/connect/send_exit.c b/lib/erl_interface/src/connect/send_exit.c index 2e298e3221..b4f7e14c7f 100644 --- a/lib/erl_interface/src/connect/send_exit.c +++ b/lib/erl_interface/src/connect/send_exit.c @@ -55,6 +55,17 @@ int ei_send_exit_tmo(int fd, const erlang_pid *from, const erlang_pid *to, char *s; int index = 0; int len = strlen(reason) + 1080; /* see below */ + ei_socket_callbacks *cbs; + void *ctx; + int err; + ssize_t wlen; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; + + err = EI_GET_CBS_CTX__(&cbs, &ctx, fd); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return ERL_ERROR; + } if (len > EISMALLBUF) if (!(dbuf = malloc(len))) @@ -92,10 +103,16 @@ int ei_send_exit_tmo(int fd, const erlang_pid *from, const erlang_pid *to, if (ei_tracelevel >= 4) ei_show_sendmsg(stderr,msgbuf,NULL); - ei_write_fill_t(fd,msgbuf,index,ms); - /* FIXME ignore timeout etc? erl_errno?! */ - - if (dbuf) free(dbuf); + wlen = (ssize_t) index; + err = ei_write_fill_ctx_t__(cbs, ctx, msgbuf, &wlen, tmo); + if (!err && wlen != (ssize_t) index) + err = EIO; + if (dbuf) + free(dbuf); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return ERL_ERROR; + } return 0; } diff --git a/lib/erl_interface/src/connect/send_reg.c b/lib/erl_interface/src/connect/send_reg.c index 62478f042d..80d61e57b5 100644 --- a/lib/erl_interface/src/connect/send_reg.c +++ b/lib/erl_interface/src/connect/send_reg.c @@ -51,11 +51,17 @@ int ei_send_reg_encoded_tmo(int fd, const erlang_pid *from, char *s, header[1400]; /* see size calculation below */ erlang_trace *token = NULL; int index = 5; /* reserve 5 bytes for control message */ - int res; + int err; + ei_socket_callbacks *cbs; + void *ctx; + ssize_t len, tot_len; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; -#ifdef HAVE_WRITEV - struct iovec v[2]; -#endif + err = EI_GET_CBS_CTX__(&cbs, &ctx, fd); + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return ERL_ERROR; + } /* are we tracing? */ /* check that he can receive trace tokens first */ @@ -86,29 +92,45 @@ int ei_send_reg_encoded_tmo(int fd, const erlang_pid *from, if (ei_tracelevel >= 4) ei_show_sendmsg(stderr,header,msg); -#ifdef HAVE_WRITEV +#ifdef EI_HAVE_STRUCT_IOVEC__ + if (ei_socket_callbacks_have_writev__(cbs)) { + struct iovec v[2]; - v[0].iov_base = (char *)header; - v[0].iov_len = index; - v[1].iov_base = (char *)msg; - v[1].iov_len = msglen; + v[0].iov_base = (char *)header; + v[0].iov_len = index; + v[1].iov_base = (char *)msg; + v[1].iov_len = msglen; - if ((res = ei_writev_fill_t(fd,v,2,ms)) != index+msglen) { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return -1; + len = tot_len = (ssize_t) index+msglen; + err = ei_writev_fill_ctx_t__(cbs, ctx, v, 2, &len, tmo); + if (!err && len != tot_len) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return -1; + } + return 0; } -#else - +#endif /* EI_HAVE_STRUCT_IOVEC__ */ + /* no writev() */ - if ((res = ei_write_fill_t(fd,header,index,ms)) != index) { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return -1; + len = tot_len = (ssize_t) index; + err = ei_write_fill_ctx_t__(cbs, ctx, header, &len, tmo); + if (!err && len != tot_len) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return -1; } - if ((res = ei_write_fill_t(fd,msg,msglen,ms)) != msglen) { - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return -1; + + len = tot_len = (ssize_t) msglen; + err = ei_write_fill_ctx_t__(cbs, ctx, msg, &len, tmo); + if (!err && len != tot_len) + err = EIO; + if (err) { + EI_CONN_SAVE_ERRNO__(err); + return -1; } -#endif return 0; } diff --git a/lib/erl_interface/src/epmd/epmd_port.c b/lib/erl_interface/src/epmd/epmd_port.c index 2ec418b24a..492c3fb3aa 100644 --- a/lib/erl_interface/src/epmd/epmd_port.c +++ b/lib/erl_interface/src/epmd/epmd_port.c @@ -62,31 +62,38 @@ int ei_epmd_connect_tmo(struct in_addr *inaddr, unsigned ms) { static unsigned int epmd_port = 0; - struct sockaddr_in saddr; - int sd; - int res; + int port, sd, err; + struct in_addr ip_addr; + struct sockaddr_in addr; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; + + err = ei_socket__(&sd); + if (err) { + erl_errno = err; + return -1; + } if (epmd_port == 0) { char* port_str = getenv("ERL_EPMD_PORT"); epmd_port = (port_str != NULL) ? atoi(port_str) : EPMD_PORT; } - memset(&saddr, 0, sizeof(saddr)); - saddr.sin_port = htons(epmd_port); - saddr.sin_family = AF_INET; - if (!inaddr) saddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); - else memmove(&saddr.sin_addr,inaddr,sizeof(saddr.sin_addr)); + port = (int) epmd_port; - if (((sd = socket(PF_INET, SOCK_STREAM, 0)) < 0)) - { - erl_errno = errno; - return -1; + if (!inaddr) { + ip_addr.s_addr = htonl(INADDR_LOOPBACK); + inaddr = &ip_addr; } + + memset((void *) &addr, 0, sizeof(struct sockaddr_in)); + memcpy((void *) &addr.sin_addr, (void *) inaddr, sizeof(addr.sin_addr)); + addr.sin_family = AF_INET; + addr.sin_port = htons(port); - if ((res = ei_connect_t(sd,(struct sockaddr *)&saddr,sizeof(saddr),ms)) < 0) - { - erl_errno = (res == -2) ? ETIMEDOUT : errno; - closesocket(sd); + err = ei_connect_t__(sd, (void *) &addr, sizeof(addr), tmo); + if (err) { + erl_errno = err; + ei_close__(sd); return -1; } @@ -104,6 +111,9 @@ static int ei_epmd_r4_port (struct in_addr *addr, const char *alive, int port; int dist_high, dist_low, proto; int res; + int err; + ssize_t dlen; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; #if defined(VXWORKS) char ntoabuf[32]; #endif @@ -124,10 +134,14 @@ static int ei_epmd_r4_port (struct in_addr *addr, const char *alive, return -1; } - if ((res = ei_write_fill_t(fd, buf, len+2, ms)) != len+2) { - closesocket(fd); - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return -1; + dlen = len + 2; + err = ei_write_fill_t__(fd, buf, &dlen, tmo); + if (!err && dlen != (ssize_t) len + 2) + erl_errno = EIO; + if (err) { + ei_close__(fd); + EI_CONN_SAVE_ERRNO__(err); + return -1; } #ifdef VXWORKS @@ -142,12 +156,15 @@ static int ei_epmd_r4_port (struct in_addr *addr, const char *alive, "-> PORT2_REQ alive=%s ip=%s",alive,inet_ntoa(*addr)); #endif - /* read first two bytes (response type, response) */ - if ((res = ei_read_fill_t(fd, buf, 2, ms)) != 2) { - EI_TRACE_ERR0("ei_epmd_r4_port","<- CLOSE"); - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - closesocket(fd); - return -2; /* version mismatch */ + dlen = (ssize_t) 2; + err = ei_read_fill_t__(fd, buf, &dlen, tmo); + if (!err && dlen != (ssize_t) 2) + erl_errno = EIO; + if (err) { + EI_TRACE_ERR0("ei_epmd_r4_port","<- CLOSE"); + ei_close__(fd); + EI_CONN_SAVE_ERRNO__(err); + return -2; } s = buf; @@ -156,7 +173,7 @@ static int ei_epmd_r4_port (struct in_addr *addr, const char *alive, if (res != EI_EPMD_PORT2_RESP) { /* response type */ EI_TRACE_ERR1("ei_epmd_r4_port","<- unknown (%d)",res); EI_TRACE_ERR0("ei_epmd_r4_port","-> CLOSE"); - closesocket(fd); + ei_close__(fd); erl_errno = EIO; return -1; } @@ -167,7 +184,7 @@ static int ei_epmd_r4_port (struct in_addr *addr, const char *alive, if ((res = get8(s))) { /* got negative response */ EI_TRACE_ERR1("ei_epmd_r4_port","<- PORT2_RESP result=%d (failure)",res); - closesocket(fd); + ei_close__(fd); erl_errno = EIO; return -1; } @@ -175,14 +192,18 @@ static int ei_epmd_r4_port (struct in_addr *addr, const char *alive, EI_TRACE_CONN1("ei_epmd_r4_port","<- PORT2_RESP result=%d (ok)",res); /* expecting remaining 8 bytes */ - if ((res = ei_read_fill_t(fd,buf,8,ms)) != 8) { + dlen = (ssize_t) 8; + err = ei_read_fill_t__(fd, buf, &dlen, tmo); + if (!err && dlen != (ssize_t) 8) + err = EIO; + if (err) { EI_TRACE_ERR0("ei_epmd_r4_port","<- CLOSE"); - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - closesocket(fd); + ei_close__(fd); + EI_CONN_SAVE_ERRNO__(err); return -1; } - closesocket(fd); + ei_close__(fd); s = buf; port = get16be(s); diff --git a/lib/erl_interface/src/epmd/epmd_publish.c b/lib/erl_interface/src/epmd/epmd_publish.c index 47d68a6db0..20b8e867e8 100644 --- a/lib/erl_interface/src/epmd/epmd_publish.c +++ b/lib/erl_interface/src/epmd/epmd_publish.c @@ -68,8 +68,10 @@ static int ei_epmd_r4_publish (int port, const char *alive, unsigned ms) int nlen = strlen(alive); int len = elen + nlen + 13; /* hard coded: be careful! */ int n; - int res, creation; - + int err, res, creation; + ssize_t dlen; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; + if (len > sizeof(buf)-2) { erl_errno = ERANGE; @@ -93,29 +95,39 @@ static int ei_epmd_r4_publish (int port, const char *alive, unsigned ms) if ((fd = ei_epmd_connect_tmo(NULL,ms)) < 0) return fd; - if ((res = ei_write_fill_t(fd, buf, len+2, ms)) != len+2) { - closesocket(fd); - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return -1; + dlen = (ssize_t) len+2; + err = ei_write_fill_t__(fd, buf, &dlen, tmo); + if (!err && dlen != (ssize_t) len + 2) + erl_errno = EIO; + if (err) { + ei_close__(fd); + EI_CONN_SAVE_ERRNO__(err); + return -1; } EI_TRACE_CONN6("ei_epmd_r4_publish", "-> ALIVE2_REQ alive=%s port=%d ntype=%d " "proto=%d dist-high=%d dist-low=%d", alive,port,'H',EI_MYPROTO,EI_DIST_HIGH,EI_DIST_LOW); - - if ((n = ei_read_fill_t(fd, buf, 4, ms)) != 4) { + + dlen = (ssize_t) 4; + err = ei_read_fill_t__(fd, buf, &dlen, tmo); + n = (int) dlen; + if (!err && n != 4) + err = EIO; + if (err) { EI_TRACE_ERR0("ei_epmd_r4_publish","<- CLOSE"); - closesocket(fd); - erl_errno = (n == -2) ? ETIMEDOUT : EIO; + ei_close__(fd); + EI_CONN_SAVE_ERRNO__(err); return -2; /* version mismatch */ } + /* Don't close fd here! It keeps us registered with epmd */ s = buf; if (((res=get8(s)) != EI_EPMD_ALIVE2_RESP)) { /* response */ EI_TRACE_ERR1("ei_epmd_r4_publish","<- unknown (%d)",res); EI_TRACE_ERR0("ei_epmd_r4_publish","-> CLOSE"); - closesocket(fd); + ei_close__(fd); erl_errno = EIO; return -1; } @@ -124,7 +136,7 @@ static int ei_epmd_r4_publish (int port, const char *alive, unsigned ms) if (((res=get8(s)) != 0)) { /* 0 == success */ EI_TRACE_ERR1("ei_epmd_r4_publish"," result=%d (fail)",res); - closesocket(fd); + ei_close__(fd); erl_errno = EIO; return -1; } diff --git a/lib/erl_interface/src/epmd/epmd_unpublish.c b/lib/erl_interface/src/epmd/epmd_unpublish.c index 255d0ffb59..c112f74147 100644 --- a/lib/erl_interface/src/epmd/epmd_unpublish.c +++ b/lib/erl_interface/src/epmd/epmd_unpublish.c @@ -58,7 +58,9 @@ int ei_unpublish_tmo(const char *alive, unsigned ms) char buf[EPMDBUF]; char *s = (char*)buf; int len = 1 + strlen(alive); - int fd, res; + int fd, err; + ssize_t dlen; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; if (len > sizeof(buf)-3) { erl_errno = ERANGE; @@ -72,20 +74,29 @@ int ei_unpublish_tmo(const char *alive, unsigned ms) /* FIXME can't connect, return success?! At least commen whats up */ if ((fd = ei_epmd_connect_tmo(NULL,ms)) < 0) return fd; - if ((res = ei_write_fill_t(fd, buf, len+2,ms)) != len+2) { - closesocket(fd); - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return -1; + dlen = (ssize_t) len+2; + err = ei_write_fill_t__(fd, buf, &dlen, tmo); + if (!err && dlen != (ssize_t) len + 2) + erl_errno = EIO; + if (err) { + ei_close__(fd); + EI_CONN_SAVE_ERRNO__(err); + return -1; } EI_TRACE_CONN1("ei_unpublish_tmo","-> STOP %s",alive); - - if ((res = ei_read_fill_t(fd, buf, 7, ms)) != 7) { - closesocket(fd); - erl_errno = (res == -2) ? ETIMEDOUT : EIO; - return -1; + + dlen = (ssize_t) 7; + err = ei_read_fill_t__(fd, buf, &dlen, tmo); + if (!err && dlen != (ssize_t) 7) + erl_errno = EIO; + if (err) { + ei_close__(fd); + EI_CONN_SAVE_ERRNO__(err); + return -1; } - closesocket(fd); + + ei_close__(fd); buf[7]=(char)0; /* terminate the string */ if (!strcmp("STOPPED",(char *)buf)) { diff --git a/lib/erl_interface/src/legacy/erl_connect.c b/lib/erl_interface/src/legacy/erl_connect.c index 7ffd545d3e..e2fd4611c0 100644 --- a/lib/erl_interface/src/legacy/erl_connect.c +++ b/lib/erl_interface/src/legacy/erl_connect.c @@ -179,15 +179,13 @@ int erl_xconnect(Erl_IpAddr addr, char *alivename) * * API: erl_close_connection() * - * Close a connection. FIXME call ei_close_connection() later. - * * Returns 0 on success and -1 on failure. * ***************************************************************************/ int erl_close_connection(int fd) { - return closesocket(fd); + return ei_close_connection(fd); } /* @@ -220,7 +218,10 @@ int erl_reg_send(int fd, char *server_name, ETERM *msg) ei_x_buff x; int r; - ei_x_new_with_version(&x); + if (ei_x_new_with_version(&x) < 0) { + erl_errno = ENOMEM; + return 0; + } if (ei_x_encode_term(&x, msg) < 0) { erl_errno = EINVAL; r = 0; diff --git a/lib/erl_interface/src/legacy/erl_eterm.c b/lib/erl_interface/src/legacy/erl_eterm.c index 9ad92121f4..7ed2bdbc93 100644 --- a/lib/erl_interface/src/legacy/erl_eterm.c +++ b/lib/erl_interface/src/legacy/erl_eterm.c @@ -65,7 +65,7 @@ void erl_init(void *hp,long heap_size) { erl_init_malloc(hp, heap_size); erl_init_marshal(); - ei_init_resolve(); + (void) ei_init(); } void erl_set_compat_rel(unsigned rel) diff --git a/lib/erl_interface/src/misc/ei_init.c b/lib/erl_interface/src/misc/ei_init.c new file mode 100644 index 0000000000..5357968657 --- /dev/null +++ b/lib/erl_interface/src/misc/ei_init.c @@ -0,0 +1,32 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2019. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * %CopyrightEnd% + */ + +#include "ei.h" +#include "ei_resolve.h" +#include "ei_internal.h" + +int +ei_init(void) +{ + int error = ei_init_connect(); + if (error) + return error; + return ei_init_resolve(); +} diff --git a/lib/erl_interface/src/misc/ei_internal.h b/lib/erl_interface/src/misc/ei_internal.h index aa6aacd703..f28dd6d668 100644 --- a/lib/erl_interface/src/misc/ei_internal.h +++ b/lib/erl_interface/src/misc/ei_internal.h @@ -22,19 +22,20 @@ #ifndef _EI_INTERNAL_H #define _EI_INTERNAL_H +#ifdef EI_HIDE_REAL_ERRNO +# define EI_CONN_SAVE_ERRNO__(E) \ + ((E) == ETIMEDOUT ? (erl_errno = ETIMEDOUT) : (erl_errno = EIO)) +#else +# define EI_CONN_SAVE_ERRNO__(E) \ + (erl_errno = (E)) +#endif + /* * Some useful stuff not to be exported to users. */ #ifdef __WIN32__ #define MAXPATHLEN 256 -#define writesocket(sock,buf,nbyte) send(sock,buf,nbyte,0) -#define readsocket(sock,buf,nbyte) recv(sock,buf,nbyte,0) -#else /* not __WIN32__ */ -#define writesocket write -#define readsocket read -#define closesocket close -#define ioctlsocket ioctl #endif /* @@ -152,7 +153,12 @@ extern int ei_tracelevel; +int ei_init_connect(void); + void ei_trace_printf(const char *name, int level, const char *format, ...); int ei_internal_use_r9_pids_ports(void); + +int ei_get_cbs_ctx__(ei_socket_callbacks **cbs, void **ctx, int fd); + #endif /* _EI_INTERNAL_H */ diff --git a/lib/erl_interface/src/misc/ei_portio.c b/lib/erl_interface/src/misc/ei_portio.c index 8cd35bf2e5..bccc86c1b1 100644 --- a/lib/erl_interface/src/misc/ei_portio.c +++ b/lib/erl_interface/src/misc/ei_portio.c @@ -19,9 +19,13 @@ * */ + +#include "eidef.h" + #ifdef __WIN32__ #include <winsock2.h> #include <windows.h> +#include <winbase.h> #include <process.h> #include <stdio.h> #include <stdlib.h> @@ -35,10 +39,6 @@ static unsigned long param_one = 1; #define SET_BLOCKING(Sock) ioctlsocket((Sock),FIONBIO,¶m_zero) #define SET_NONBLOCKING(Sock) ioctlsocket((Sock),FIONBIO,¶m_one) -#define ERROR_WOULDBLOCK WSAEWOULDBLOCK -#define ERROR_TIMEDOUT WSAETIMEDOUT -#define ERROR_INPROGRESS WSAEINPROGRESS -#define GET_SOCKET_ERROR() WSAGetLastError() #define MEANS_SOCKET_ERROR(Ret) ((Ret == SOCKET_ERROR)) #define IS_INVALID_SOCKET(Sock) ((Sock) == INVALID_SOCKET) @@ -50,125 +50,414 @@ static unsigned long param_one = 1; #include <taskLib.h> #include <inetLib.h> #include <selectLib.h> -#include <sys/types.h> #include <ioLib.h> #include <unistd.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <netinet/tcp.h> +#include <timers.h> static unsigned long param_zero = 0; static unsigned long param_one = 1; #define SET_BLOCKING(Sock) ioctl((Sock),FIONBIO,(int)¶m_zero) #define SET_NONBLOCKING(Sock) ioctl((Sock),FIONBIO,(int)¶m_one) -#define ERROR_WOULDBLOCK EWOULDBLOCK -#define ERROR_TIMEDOUT ETIMEDOUT -#define ERROR_INPROGRESS EINPROGRESS -#define GET_SOCKET_ERROR() (errno) #define MEANS_SOCKET_ERROR(Ret) ((Ret) == ERROR) #define IS_INVALID_SOCKET(Sock) ((Sock) < 0) #else /* other unix */ #include <stdlib.h> -#include <sys/types.h> #include <sys/socket.h> -#include <sys/uio.h> #include <unistd.h> #include <fcntl.h> #include <errno.h> +#include <netinet/in.h> +#include <netinet/tcp.h> +#include <arpa/inet.h> +#include <netdb.h> -#ifndef EWOULDBLOCK -#define ERROR_WOULDBLOCK EAGAIN -#else -#define ERROR_WOULDBLOCK EWOULDBLOCK -#endif #define SET_BLOCKING(fd) fcntl((fd), F_SETFL, \ fcntl((fd), F_GETFL, 0) & ~O_NONBLOCK) #define SET_NONBLOCKING(fd) fcntl((fd), F_SETFL, \ fcntl((fd), F_GETFL, 0) | O_NONBLOCK) -#define ERROR_TIMEDOUT ETIMEDOUT -#define ERROR_INPROGRESS EINPROGRESS -#define GET_SOCKET_ERROR() (errno) #define MEANS_SOCKET_ERROR(Ret) ((Ret) < 0) #define IS_INVALID_SOCKET(Sock) ((Sock) < 0) #endif /* common includes */ -#include "eidef.h" +#include <sys/types.h> #include <stdio.h> #include <stdlib.h> #include <string.h> -#include "ei_portio.h" -#include "ei_internal.h" - #ifdef HAVE_SYS_TIME_H #include <sys/time.h> #else #include <time.h> #endif +#ifdef HAVE_SYS_SELECT_H +#include <sys/select.h> +#endif +#include "ei_portio.h" +#include "ei_internal.h" + +#ifdef __WIN32__ -#ifdef HAVE_WRITEV -static int ei_writev_t(int fd, struct iovec *iov, int iovcnt, unsigned ms) +#define writesocket(sock,buf,nbyte) send(sock,buf,nbyte,0) +#define readsocket(sock,buf,nbyte) recv(sock,buf,nbyte,0) + +static int get_error(void) { - int res; - if (ms != 0) { - fd_set writemask; - struct timeval tv; - tv.tv_sec = (time_t) (ms / 1000U); - ms %= 1000U; - tv.tv_usec = (time_t) (ms * 1000U); - FD_ZERO(&writemask); - FD_SET(fd,&writemask); - switch (select(fd+1, NULL, &writemask, NULL, &tv)) { - case -1 : - return -1; /* i/o error */ - case 0: - return -2; /* timeout */ - default: - if (!FD_ISSET(fd, &writemask)) { - return -1; /* Other error */ - } - } + switch (WSAGetLastError()) { + case WSAEWOULDBLOCK: return EWOULDBLOCK; + case WSAETIMEDOUT: return ETIMEDOUT; + case WSAEINPROGRESS: return EINPROGRESS; + case WSA_NOT_ENOUGH_MEMORY: return ENOMEM; + case WSA_INVALID_PARAMETER: return EINVAL; + case WSAEBADF: return EBADF; + case WSAEINVAL: return EINVAL; + case WSAEADDRINUSE: return EADDRINUSE; + case WSAENETUNREACH: return ENETUNREACH; + case WSAECONNABORTED: return ECONNABORTED; + case WSAECONNRESET: return ECONNRESET; + case WSAECONNREFUSED: return ECONNREFUSED; + case WSAEHOSTUNREACH: return EHOSTUNREACH; + case WSAEMFILE: return EMFILE; + case WSAEALREADY: return EALREADY; + default: return EIO; } +} + +#else /* not __WIN32__ */ + +#define writesocket write +#define readsocket read +#define closesocket close +#define ioctlsocket ioctl + +static int get_error(void) +{ + int err = errno; + if (err == 0) + return EIO; /* Make sure never to return 0 as error code... */ + return err; +} + +#endif + +int ei_plugin_socket_impl__ = 0; + +/* + * Callbacks for communication over TCP/IPv4 + */ + +static int tcp_get_fd(void *ctx, int *fd) +{ + return EI_DFLT_CTX_TO_FD__(ctx, fd); +} + +static int tcp_hs_packet_header_size(void *ctx, int *sz) +{ + int fd; + *sz = 2; + return EI_DFLT_CTX_TO_FD__(ctx, &fd); +} + +static int tcp_handshake_complete(void *ctx) +{ + int res, fd, one = 1; + + res = EI_DFLT_CTX_TO_FD__(ctx, &fd); + if (res) + return res; + + res = setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, (char *)&one, sizeof(one)); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + + res = setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, (char *)&one, sizeof(one)); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + + return 0; +} + +static int tcp_socket(void **ctx, void *setup_ctx) +{ + int fd = socket(AF_INET, SOCK_STREAM, 0); + if (MEANS_SOCKET_ERROR(fd)) + return get_error(); + + *ctx = EI_FD_AS_CTX__(fd); + return 0; +} + +static int tcp_close(void *ctx) +{ + int fd, res; + + res = EI_DFLT_CTX_TO_FD__(ctx, &fd); + if (res) + return res; + + res = closesocket(fd); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + + return 0; +} + +static int tcp_listen(void *ctx, void *addr, int *len, int backlog) +{ + int res, fd; + socklen_t sz = (socklen_t) *len; + int on = 1; + + res = EI_DFLT_CTX_TO_FD__(ctx, &fd); + if (res) + return res; + + res = setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, (char *) &on, sizeof(on)); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + + res = bind(fd, (struct sockaddr *) addr, sz); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + + res = getsockname(fd, (struct sockaddr *) addr, (socklen_t *) &sz); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + *len = (int) sz; + + res = listen(fd, backlog); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + + return 0; +} + +static int tcp_accept(void **ctx, void *addr, int *len, unsigned unused) +{ + int fd, res; + socklen_t addr_len = (socklen_t) *len; + + if (!ctx) + return EINVAL; + + res = EI_DFLT_CTX_TO_FD__(*ctx, &fd); + if (res) + return res; + + res = accept(fd, (struct sockaddr*) addr, &addr_len); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + + *len = (int) addr_len; + + *ctx = EI_FD_AS_CTX__(res); + return 0; +} + +static int tcp_connect(void *ctx, void *addr, int len, unsigned unused) +{ + int res, fd; + + res = EI_DFLT_CTX_TO_FD__(ctx, &fd); + if (res) + return res; + + res = connect(fd, (struct sockaddr *) addr, len); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + + return 0; +} + +#if defined(EI_HAVE_STRUCT_IOVEC__) && defined(HAVE_WRITEV) + +static int tcp_writev(void *ctx, const void *viov, int iovcnt, ssize_t *len, unsigned unused) +{ + const struct iovec *iov = (const struct iovec *) viov; + int fd, error; + ssize_t res; + + error = EI_DFLT_CTX_TO_FD__(ctx, &fd); + if (error) + return error; + res = writev(fd, iov, iovcnt); - return (res < 0) ? -1 : res; + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + *len = res; + return 0; +} + +#endif + +static int tcp_write(void *ctx, const char* buf, ssize_t *len, unsigned unused) +{ + int error, fd; + ssize_t res; + + error = EI_DFLT_CTX_TO_FD__(ctx, &fd); + if (error) + return error; + + res = writesocket(fd, buf, *len); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + *len = res; + return 0; +} + +static int tcp_read(void *ctx, char* buf, ssize_t *len, unsigned unused) +{ + int error, fd; + ssize_t res; + + error = EI_DFLT_CTX_TO_FD__(ctx, &fd); + if (error) + return error; + + res = readsocket(fd, buf, *len); + if (MEANS_SOCKET_ERROR(res)) + return get_error(); + *len = res; + return 0; +} + +ei_socket_callbacks ei_default_socket_callbacks = { + 0, /* flags */ + tcp_socket, + tcp_close, + tcp_listen, + tcp_accept, + tcp_connect, +#if defined(EI_HAVE_STRUCT_IOVEC__) && defined(HAVE_WRITEV) + tcp_writev, +#else + NULL, +#endif + tcp_write, + tcp_read, + + tcp_hs_packet_header_size, + tcp_handshake_complete, + tcp_handshake_complete, + tcp_get_fd + +}; + + +/* + * + */ + +#if defined(EI_HAVE_STRUCT_IOVEC__) + +int ei_socket_callbacks_have_writev__(ei_socket_callbacks *cbs) +{ + return !!cbs->writev; } -int ei_writev_fill_t(int fd, const struct iovec *iov, int iovcnt, unsigned ms) +static int writev_ctx_t__(ei_socket_callbacks *cbs, void *ctx, + const struct iovec *iov, int iovcnt, + ssize_t *len, + unsigned ms) { - int i; - int done; + int error; + + if (!(cbs->flags & EI_SCLBK_FLG_FULL_IMPL) && ms != EI_SCLBK_INF_TMO) { + int fd; + + error = EI_GET_FD__(cbs, ctx, &fd); + if (error) + return error; + + do { + fd_set writemask; + struct timeval tv; + + tv.tv_sec = (time_t) (ms / 1000U); + ms %= 1000U; + tv.tv_usec = (time_t) (ms * 1000U); + FD_ZERO(&writemask); + FD_SET(fd,&writemask); + switch (select(fd+1, NULL, &writemask, NULL, &tv)) { + case -1 : + error = get_error(); + if (error != EINTR) + return error; + break; + case 0: + return ETIMEDOUT; /* timeout */ + default: + if (!FD_ISSET(fd, &writemask)) { + return EIO; /* Other error */ + } + error = 0; + break; + } + } while (error == EINTR); + } + do { + error = cbs->writev(ctx, (const void *) iov, iovcnt, len, ms); + } while (error == EINTR); + return error; +} + +int ei_writev_fill_ctx_t__(ei_socket_callbacks *cbs, void *ctx, + const struct iovec *iov, int iovcnt, + ssize_t *len, + unsigned ms) +{ + ssize_t i, done, sum; struct iovec *iov_base = NULL; struct iovec *current_iov; int current_iovcnt; - int sum; + int fd, error; + int basic; + + if (!cbs->writev) + return ENOTSUP; + + error = EI_GET_FD__(cbs, ctx, &fd); + if (error) + return error; + basic = !(cbs->flags & EI_SCLBK_FLG_FULL_IMPL); + for (sum = 0, i = 0; i < iovcnt; ++i) { sum += iov[i].iov_len; } - if (ms != 0U) { + if (basic && ms != 0U) { SET_NONBLOCKING(fd); } current_iovcnt = iovcnt; current_iov = (struct iovec *) iov; done = 0; for (;;) { - i = ei_writev_t(fd, current_iov, current_iovcnt, ms); - if (i <= 0) { /* ei_writev_t should always return at least 1 */ + + error = writev_ctx_t__(cbs, ctx, current_iov, current_iovcnt, &i, ms); + if (error) { + *len = done; if (ms != 0U) { SET_BLOCKING(fd); } if (iov_base != NULL) { free(iov_base); } - return (i); - } + return error; + } done += i; if (done < sum) { if (iov_base == NULL) { iov_base = malloc(sizeof(struct iovec) * iovcnt); if (iov_base == NULL) { - return -1; + *len = done; + return ENOMEM; } memcpy(iov_base, iov, sizeof(struct iovec) * iovcnt); current_iov = iov_base; @@ -189,195 +478,383 @@ int ei_writev_fill_t(int fd, const struct iovec *iov, int iovcnt, unsigned break; } } - if (ms != 0U) { + if (basic && ms != 0U) { SET_BLOCKING(fd); } if (iov_base != NULL) { free(iov_base); } - return (sum); + *len = done; + return 0; } +#endif /* defined(EI_HAVE_STRUCT_IOVEC__) */ -#endif - -int ei_connect_t(int fd, void *sinp, int sin_siz, unsigned ms) +int ei_socket_ctx__(ei_socket_callbacks *cbs, void **ctx, void *setup_ctx) { int res; - int error; - int s_res; - struct timeval tv; - fd_set writefds; - fd_set exceptfds; - - if (ms == 0) { - res = connect(fd, sinp, sin_siz); - return (res < 0) ? -1 : res; - } else { - SET_NONBLOCKING(fd); - res = connect(fd, sinp, sin_siz); - error = GET_SOCKET_ERROR(); - SET_BLOCKING(fd); - if (!MEANS_SOCKET_ERROR(res)) { - return (res < 0) ? -1 : res; - } else { - if (error != ERROR_WOULDBLOCK && - error != ERROR_INPROGRESS) { - return -1; - } else { - tv.tv_sec = (long) (ms/1000U); - ms %= 1000U; - tv.tv_usec = (long) (ms * 1000U); - FD_ZERO(&writefds); - FD_SET(fd,&writefds); - FD_ZERO(&exceptfds); - FD_SET(fd,&exceptfds); - s_res = select(fd + 1, NULL, &writefds, &exceptfds, &tv); - switch (s_res) { - case 0: - return -2; - case 1: - if (FD_ISSET(fd, &exceptfds)) { - return -1; - } else { - return 0; /* Connect completed */ - } - default: - return -1; - } - } - } - } + + do { + res = cbs->socket(ctx, setup_ctx); + } while (res == EINTR); + + return res; } -int ei_accept_t(int fd, void *addr, void *addrlen, unsigned ms) +int ei_close_ctx__(ei_socket_callbacks *cbs, void *ctx) { - int res; - if (ms != 0) { - fd_set readmask; - struct timeval tv; - tv.tv_sec = (time_t) (ms / 1000U); - ms %= 1000U; - tv.tv_usec = (time_t) (ms * 1000U); - FD_ZERO(&readmask); - FD_SET(fd,&readmask); - switch (select(fd+1, &readmask, NULL, NULL, &tv)) { - case -1 : - return -1; /* i/o error */ - case 0: - return -2; /* timeout */ - default: - if (!FD_ISSET(fd, &readmask)) { - return -1; /* Other error */ - } - } - } - res = (int) accept(fd,addr,addrlen); - return (res < 0) ? -1 : res; + return cbs->close(ctx); } + +int ei_connect_ctx_t__(ei_socket_callbacks *cbs, void *ctx, + void *addr, int len, unsigned ms) +{ + int res, fd; + + if ((cbs->flags & EI_SCLBK_FLG_FULL_IMPL) || ms == EI_SCLBK_INF_TMO) { + do { + res = cbs->connect(ctx, addr, len, ms); + } while (res == EINTR); + return res; + } + + res = EI_GET_FD__(cbs, ctx, &fd); + if (res) + return res; + SET_NONBLOCKING(fd); + do { + res = cbs->connect(ctx, addr, len, 0); + } while (res == EINTR); + SET_BLOCKING(fd); + switch (res) { + case EINPROGRESS: + case EAGAIN: +#ifdef EWOULDBLOCK +#if EWOULDBLOCK != EAGAIN + case EWOULDBLOCK: +#endif +#endif + break; + default: + return res; + } -static int ei_read_t(int fd, char* buf, int len, unsigned ms) + while (1) { + struct timeval tv; + fd_set writefds; + fd_set exceptfds; + + tv.tv_sec = (long) (ms/1000U); + ms %= 1000U; + tv.tv_usec = (long) (ms * 1000U); + FD_ZERO(&writefds); + FD_SET(fd,&writefds); + FD_ZERO(&exceptfds); + FD_SET(fd,&exceptfds); + res = select(fd + 1, NULL, &writefds, &exceptfds, &tv); + switch (res) { + case -1: + res = get_error(); + if (res != EINTR) + return res; + break; + case 0: + return ETIMEDOUT; + case 1: + if (!FD_ISSET(fd, &exceptfds)) + return 0; /* Connect completed */ + /* fall through... */ + default: + return EIO; + } + } +} + +int ei_listen_ctx__(ei_socket_callbacks *cbs, void *ctx, + void *adr, int *len, int backlog) { int res; - if (ms != 0) { - fd_set readmask; - struct timeval tv; - tv.tv_sec = (time_t) (ms / 1000U); - ms %= 1000U; - tv.tv_usec = (time_t) (ms * 1000U); - FD_ZERO(&readmask); - FD_SET(fd,&readmask); - switch (select(fd+1, &readmask, NULL, NULL, &tv)) { - case -1 : - return -1; /* i/o error */ - case 0: - return -2; /* timeout */ - default: - if (!FD_ISSET(fd, &readmask)) { - return -1; /* Other error */ - } - } + + do { + res = cbs->listen(ctx, adr, len, backlog); + } while (res == EINTR); + return res; +} + +int ei_accept_ctx_t__(ei_socket_callbacks *cbs, void **ctx, + void *addr, int *len, unsigned ms) +{ + int error; + + if (!(cbs->flags & EI_SCLBK_FLG_FULL_IMPL) && ms != EI_SCLBK_INF_TMO) { + int fd; + + error = EI_GET_FD__(cbs, *ctx, &fd); + if (error) + return error; + + do { + fd_set readmask; + struct timeval tv; + + tv.tv_sec = (time_t) (ms / 1000U); + ms %= 1000U; + tv.tv_usec = (time_t) (ms * 1000U); + FD_ZERO(&readmask); + FD_SET(fd,&readmask); + switch (select(fd+1, &readmask, NULL, NULL, &tv)) { + case -1 : + error = get_error(); + if (error != EINTR) + return error; + break; + case 0: + return ETIMEDOUT; /* timeout */ + default: + if (!FD_ISSET(fd, &readmask)) { + return EIO; /* Other error */ + } + error = 0; + break; + } + } while (error == EINTR); } - res = readsocket(fd, buf, len); - return (res < 0) ? -1 : res; + do { + error = cbs->accept(ctx, addr, len, ms); + } while (error == EINTR); + return error; } -static int ei_write_t(int fd, const char* buf, int len, unsigned ms) +static int read_ctx_t__(ei_socket_callbacks *cbs, void *ctx, + char* buf, ssize_t *len, unsigned ms) { - int res; - if (ms != 0) { - fd_set writemask; - struct timeval tv; - tv.tv_sec = (time_t) (ms / 1000U); - ms %= 1000U; - tv.tv_usec = (time_t) (ms * 1000U); - FD_ZERO(&writemask); - FD_SET(fd,&writemask); - switch (select(fd+1, NULL, &writemask, NULL, &tv)) { - case -1 : - return -1; /* i/o error */ - case 0: - return -2; /* timeout */ - default: - if (!FD_ISSET(fd, &writemask)) { - return -1; /* Other error */ - } - } + int error; + + if (!(cbs->flags & EI_SCLBK_FLG_FULL_IMPL) && ms != EI_SCLBK_INF_TMO) { + int fd; + + error = EI_GET_FD__(cbs, ctx, &fd); + if (error) + return error; + + do { + fd_set readmask; + struct timeval tv; + + tv.tv_sec = (time_t) (ms / 1000U); + ms %= 1000U; + tv.tv_usec = (time_t) (ms * 1000U); + FD_ZERO(&readmask); + FD_SET(fd,&readmask); + switch (select(fd+1, &readmask, NULL, NULL, &tv)) { + case -1 : + error = get_error(); + if (error != EINTR) + return error; + break; + case 0: + return ETIMEDOUT; /* timeout */ + default: + if (!FD_ISSET(fd, &readmask)) { + return EIO; /* Other error */ + } + error = 0; + break; + } + } while (error == EINTR); + } + do { + error = cbs->read(ctx, buf, len, ms); + } while (error == EINTR); + return error; +} + +static int write_ctx_t__(ei_socket_callbacks *cbs, void *ctx, const char* buf, ssize_t *len, unsigned ms) +{ + int error; + + if (!(cbs->flags & EI_SCLBK_FLG_FULL_IMPL) && ms != EI_SCLBK_INF_TMO) { + int fd; + + error = EI_GET_FD__(cbs, ctx, &fd); + if (error) + return error; + + do { + fd_set writemask; + struct timeval tv; + + tv.tv_sec = (time_t) (ms / 1000U); + ms %= 1000U; + tv.tv_usec = (time_t) (ms * 1000U); + FD_ZERO(&writemask); + FD_SET(fd,&writemask); + switch (select(fd+1, NULL, &writemask, NULL, &tv)) { + case -1 : + error = get_error(); + if (error != EINTR) + return error; + break; + case 0: + return ETIMEDOUT; /* timeout */ + default: + if (!FD_ISSET(fd, &writemask)) { + return EIO; /* Other error */ + } + error = 0; + break; + } + } while (error == EINTR); } - res = writesocket(fd, buf, len); - return (res < 0) ? -1 : res; + do { + error = cbs->write(ctx, buf, len, ms); + } while (error == EINTR); + return error; } /* * Fill buffer, return buffer length, 0 for EOF, < 0 (and sets errno) * for error. */ -int ei_read_fill_t(int fd, char* buf, int len, unsigned ms) +int ei_read_fill_ctx_t__(ei_socket_callbacks *cbs, void *ctx, char* buf, ssize_t *len, unsigned ms) { - int i,got=0; + ssize_t got = 0; + ssize_t want = *len; do { - i = ei_read_t(fd, buf+got, len-got, ms); - if (i <= 0) - return (i); - got += i; - } while (got < len); - return (len); - + ssize_t read_len = want-got; + int error; + + do { + error = read_ctx_t__(cbs, ctx, buf+got, &read_len, ms); + } while (error == EINTR); + if (error) + return error; + if (read_len == 0) { + *len = got; + return 0; + } + got += read_len; + } while (got < want); + + *len = got; + return 0; } /* read_fill */ -int ei_read_fill(int fd, char* buf, int len) +int ei_read_fill_ctx__(ei_socket_callbacks *cbs, void *ctx, char* buf, ssize_t *len) { - return ei_read_fill_t(fd, buf, len, 0); + return ei_read_fill_ctx_t__(cbs, ctx, buf, len, 0); } /* write entire buffer on fd or fail (setting errno) */ -int ei_write_fill_t(int fd, const char *buf, int len, unsigned ms) +int ei_write_fill_ctx_t__(ei_socket_callbacks *cbs, void *ctx, const char *buf, ssize_t *len, unsigned ms) { - int i,done=0; - if (ms != 0U) { + ssize_t tot = *len, done = 0; + int error, fd = -1, basic = !(cbs->flags & EI_SCLBK_FLG_FULL_IMPL); + + if (basic && ms != 0U) { + error = EI_GET_FD__(cbs, ctx, &fd); + if (error) + return error; SET_NONBLOCKING(fd); } do { - i = ei_write_t(fd, buf+done, len-done, ms); - if (i <= 0) { - if (ms != 0U) { + ssize_t write_len = tot-done; + error = write_ctx_t__(cbs, ctx, buf+done, &write_len, ms); + if (error) { + *len = done; + if (basic && ms != 0U) { SET_BLOCKING(fd); } - return (i); + return error; } - done += i; - } while (done < len); - if (ms != 0U) { + done += write_len; + } while (done < tot); + if (basic && ms != 0U) { SET_BLOCKING(fd); } - return (len); + *len = done; + return 0; +} + +int ei_write_fill_ctx__(ei_socket_callbacks *cbs, void *ctx, const char *buf, ssize_t *len) +{ + return ei_write_fill_ctx_t__(cbs, ctx, buf, len, 0); +} + +/* + * Internal API for TCP/IPv4 + */ + +int ei_connect_t__(int fd, void *addr, int len, unsigned ms) +{ + return ei_connect_ctx_t__(&ei_default_socket_callbacks, EI_FD_AS_CTX__(fd), + addr, len, ms); } -int ei_write_fill(int fd, const char *buf, int len) +int ei_socket__(int *fd) { - return ei_write_fill_t(fd, buf, len, 0); + void *ctx; + int error = ei_socket_ctx__(&ei_default_socket_callbacks, &ctx, NULL); + if (error) + return error; + return EI_GET_FD__(&ei_default_socket_callbacks, ctx, fd); } +int ei_close__(int fd) +{ + return ei_close_ctx__(&ei_default_socket_callbacks, EI_FD_AS_CTX__(fd)); +} + +int ei_listen__(int fd, void *adr, int *len, int backlog) +{ + return ei_listen_ctx__(&ei_default_socket_callbacks, EI_FD_AS_CTX__(fd), + adr, len, backlog); +} + +int ei_accept_t__(int *fd, void *addr, int *len, unsigned ms) +{ + void *ctx = EI_FD_AS_CTX__(*fd); + int error = ei_accept_ctx_t__(&ei_default_socket_callbacks, &ctx, + addr, len, ms); + if (error) + return error; + return EI_GET_FD__(&ei_default_socket_callbacks, ctx, fd); +} + +int ei_read_fill_t__(int fd, char* buf, ssize_t *len, unsigned ms) +{ + return ei_read_fill_ctx_t__(&ei_default_socket_callbacks, EI_FD_AS_CTX__(fd), + buf, len, ms); +} + +int ei_read_fill__(int fd, char* buf, ssize_t *len) +{ + return ei_read_fill_ctx_t__(&ei_default_socket_callbacks, EI_FD_AS_CTX__(fd), + buf, len, 0); +} + +int ei_write_fill_t__(int fd, const char *buf, ssize_t *len, unsigned ms) +{ + return ei_write_fill_ctx_t__(&ei_default_socket_callbacks, EI_FD_AS_CTX__(fd), + buf, len, ms); +} + +int ei_write_fill__(int fd, const char *buf, ssize_t *len) +{ + return ei_write_fill_ctx_t__(&ei_default_socket_callbacks, EI_FD_AS_CTX__(fd), + buf, len, 0); +} + +#if defined(EI_HAVE_STRUCT_IOVEC__) && defined(HAVE_WRITEV) + +int ei_writev_fill_t__(int fd, const struct iovec *iov, int iovcnt, ssize_t *len, unsigned ms) +{ + return ei_writev_fill_ctx_t__(&ei_default_socket_callbacks, EI_FD_AS_CTX__(fd), + iov, iovcnt, len, ms); +} + +#endif + diff --git a/lib/erl_interface/src/misc/ei_portio.h b/lib/erl_interface/src/misc/ei_portio.h index bded811a35..a84b5ca09c 100644 --- a/lib/erl_interface/src/misc/ei_portio.h +++ b/lib/erl_interface/src/misc/ei_portio.h @@ -21,21 +21,94 @@ */ #ifndef _EI_PORTIO_H #define _EI_PORTIO_H -#if !defined(__WIN32__) && !defined(VXWORKS) -#ifdef HAVE_WRITEV + +#undef EI_HAVE_STRUCT_IOVEC__ +#if !defined(__WIN32__) && !defined(VXWORKS) && defined(HAVE_SYS_UIO_H) /* Declaration of struct iovec *iov should be visible in this scope. */ -#include <sys/uio.h> +# include <sys/uio.h> +# define EI_HAVE_STRUCT_IOVEC__ #endif + +/* + * Internal API. Should not be used outside of the erl_interface application... + */ + +int ei_socket_ctx__(ei_socket_callbacks *cbs, void **ctx, void *setup); +int ei_close_ctx__(ei_socket_callbacks *cbs, void *ctx); +int ei_listen_ctx__(ei_socket_callbacks *cbs, void *ctx, void *adr, int *len, int backlog); +int ei_accept_ctx_t__(ei_socket_callbacks *cbs, void **ctx, void *addr, int *len, unsigned ms); +int ei_connect_ctx_t__(ei_socket_callbacks *cbs, void *ctx, void *addr, int len, unsigned ms); +int ei_read_fill_ctx__(ei_socket_callbacks *cbs, void *ctx, char* buf, ssize_t *len); +int ei_write_fill_ctx__(ei_socket_callbacks *cbs, void *ctx, const char *buf, ssize_t *len); +int ei_read_fill_ctx_t__(ei_socket_callbacks *cbs, void *ctx, char* buf, ssize_t *len, unsigned ms); +int ei_write_fill_ctx_t__(ei_socket_callbacks *cbs, void *ctx, const char *buf, ssize_t *len, unsigned ms); +#if defined(EI_HAVE_STRUCT_IOVEC__) +int ei_writev_fill_ctx_t__(ei_socket_callbacks *cbs, void *ctx, const struct iovec *iov, int iovcnt, ssize_t *len, unsigned ms); +int ei_socket_callbacks_have_writev__(ei_socket_callbacks *cbs); #endif -int ei_accept_t(int fd, void *addr, void *addrlen, unsigned ms); -int ei_connect_t(int fd, void *sinp, int sin_siz, unsigned ms); -int ei_read_fill(int fd, char* buf, int len); -int ei_write_fill(int fd, const char *buf, int len); -int ei_read_fill_t(int fd, char* buf, int len, unsigned ms); -int ei_write_fill_t(int fd, const char *buf, int len, unsigned ms); -#ifdef HAVE_WRITEV -int ei_writev_fill_t(int fd, const struct iovec *iov, int iovcnt, unsigned ms); +ei_socket_callbacks ei_default_socket_callbacks; + +#define EI_FD_AS_CTX__(FD) \ + ((void *) (long) (FD)) + +#define EI_DFLT_CTX_TO_FD__(CTX, FD) \ + ((int) (long) (CTX) < 0 \ + ? EBADF \ + : (*(FD) = (int) (long) (CTX), 0)) + +#define EI_GET_FD__(CBS, CTX, FD) \ + ((CBS) == &ei_default_socket_callbacks \ + ? EI_DFLT_CTX_TO_FD__((CTX), FD) \ + : (CBS)->get_fd((CTX), (FD))) + +extern int ei_plugin_socket_impl__; + +#if !defined(_REENTRANT) + +#define EI_HAVE_PLUGIN_SOCKET_IMPL__ \ + ei_plugin_socket_impl__ +#define EI_SET_HAVE_PLUGIN_SOCKET_IMPL__ \ + ei_plugin_socket_impl__ = 1 + +#elif ((ETHR_HAVE___atomic_load_n & SIZEOF_INT) \ + && (ETHR_HAVE___atomic_store_n & SIZEOF_INT)) + +#define EI_HAVE_PLUGIN_SOCKET_IMPL__ \ + __atomic_load_n(&ei_plugin_socket_impl__, __ATOMIC_ACQUIRE) +#define EI_SET_HAVE_PLUGIN_SOCKET_IMPL__ \ + __atomic_store_n(&ei_plugin_socket_impl__, 1, __ATOMIC_RELEASE) + +#else + +/* No gcc atomics; always lookup using ei_get_cbs_ctx()... */ +#define EI_HAVE_PLUGIN_SOCKET_IMPL__ 0 +#define EI_SET_HAVE_PLUGIN_SOCKET_IMPL__ (void) 0 + +#endif + +#define EI_GET_CBS_CTX__(CBS, CTX, FD) \ + (EI_HAVE_PLUGIN_SOCKET_IMPL__ \ + ? ei_get_cbs_ctx__((CBS), (CTX), (FD)) \ + : ((FD) < 0 \ + ? EBADF \ + : (*(CBS) = &ei_default_socket_callbacks, \ + *(CTX) = EI_FD_AS_CTX__((FD)), \ + 0))) +/* + * The following uses our own TCP/IPv4 socket implementation... + */ +int ei_socket__(int *fd); +int ei_close__(int fd); +int ei_listen__(int fd, void *adr, int *len, int backlog); +int ei_accept_t__(int *fd, void *addr, int *len, unsigned ms); +int ei_connect_t__(int fd, void *addr, int len, unsigned ms); +int ei_read_fill__(int fd, char* buf, ssize_t *len); +int ei_write_fill__(int fd, const char *buf, ssize_t *len); +int ei_read_fill_t__(int fd, char* buf, ssize_t *len, unsigned ms); +int ei_write_fill_t__(int fd, const char *buf, ssize_t *len, unsigned ms); +#if defined(EI_HAVE_STRUCT_IOVEC__) && defined(HAVE_WRITEV) +int ei_writev_fill_t__(int fd, const struct iovec *iov, int iovcnt, ssize_t *len, unsigned ms); #endif #endif /* _EI_PORTIO_H */ diff --git a/lib/erl_interface/src/not_used/send_link.c b/lib/erl_interface/src/not_used/send_link.c index 7be476fd93..38fae27df4 100644 --- a/lib/erl_interface/src/not_used/send_link.c +++ b/lib/erl_interface/src/not_used/send_link.c @@ -50,6 +50,7 @@ static int link_unlink(int fd, const erlang_pid *from, const erlang_pid *to, char *s; int index = 0; int n; + unsigned tmo = ms == 0 ? EI_SCLBK_INF_TMO : ms; index = 5; /* max sizes: */ ei_encode_version(msgbuf,&index); /* 1 */ @@ -69,7 +70,7 @@ static int link_unlink(int fd, const erlang_pid *from, const erlang_pid *to, if (ei_trace_distribution > 1) ei_show_sendmsg(stderr,msgbuf,NULL); #endif - n = ei_write_fill_t(fd,msgbuf,index,ms); + n = ei_write_fill_t__(fd,msgbuf,index,tmo); return (n==index ? 0 : -1); } diff --git a/lib/erl_interface/test/ei_accept_SUITE.erl b/lib/erl_interface/test/ei_accept_SUITE.erl index 78a433d21b..9c9c3f86b6 100644 --- a/lib/erl_interface/test/ei_accept_SUITE.erl +++ b/lib/erl_interface/test/ei_accept_SUITE.erl @@ -81,12 +81,10 @@ ei_accept(Config) when is_list(Config) -> ei_threaded_accept(Config) when is_list(Config) -> Einode = filename:join(proplists:get_value(data_dir, Config), "eiaccnode"), - N = 1, % 3, + N = 3, Host = atom_to_list(node()), - Port = 6767, - start_einode(Einode, N, Host, Port), + start_einode(Einode, N, Host), io:format("started eiaccnode"), - %%spawn_link(fun() -> start_einode(Einode, N, Host, Port) end), TestServerPid = self(), [spawn_link(fun() -> send_rec_einode(I, TestServerPid) end) || I <- lists:seq(0, N-1)], [receive I -> ok end || I <- lists:seq(0, N-1) ], @@ -159,10 +157,9 @@ send_rec_einode(N, TestServerPid) -> ct:fail(EINode) end. -start_einode(Einode, N, Host, Port) -> +start_einode(Einode, N, Host) -> Einodecmd = Einode ++ " " ++ atom_to_list(erlang:get_cookie()) - ++ " " ++ integer_to_list(N) ++ " " ++ Host ++ " " - ++ integer_to_list(Port) ++ " nothreads", + ++ " " ++ integer_to_list(N) ++ " " ++ Host, io:format("Einodecmd ~p ~n", [Einodecmd]), open_port({spawn, Einodecmd}, []), ok. diff --git a/lib/erl_interface/test/ei_accept_SUITE_data/ei_accept_test.c b/lib/erl_interface/test/ei_accept_SUITE_data/ei_accept_test.c index 50df848b69..c209f506b1 100644 --- a/lib/erl_interface/test/ei_accept_SUITE_data/ei_accept_test.c +++ b/lib/erl_interface/test/ei_accept_SUITE_data/ei_accept_test.c @@ -74,6 +74,8 @@ TESTCASE(interpret) int i; ei_term term; + ei_init(); + ei_x_new(&x); while (get_bin_term(&x, &term) == 0) { char* buf = x.buff, func[MAXATOMLEN]; @@ -125,45 +127,26 @@ static void cmd_ei_connect_init(char* buf, int len) ei_x_free(&res); } -static int my_listen(int port) -{ - int listen_fd; - struct sockaddr_in addr; - const char *on = "1"; - - if ((listen_fd = socket(AF_INET, SOCK_STREAM, 0)) < 0) - return -1; - - setsockopt(listen_fd, SOL_SOCKET, SO_REUSEADDR, on, sizeof(on)); - - memset((void*) &addr, 0, (size_t) sizeof(addr)); - addr.sin_family = AF_INET; - addr.sin_port = htons(port); - addr.sin_addr.s_addr = htonl(INADDR_ANY); - - if (bind(listen_fd, (struct sockaddr*) &addr, sizeof(addr)) < 0) - return -1; - - listen(listen_fd, 5); - return listen_fd; -} - static void cmd_ei_publish(char* buf, int len) { int index = 0; - int listen, r; - long port; + int iport, lfd, r; + long lport; ei_x_buff x; int i; /* get port */ - if (ei_decode_long(buf, &index, &port) < 0) + if (ei_decode_long(buf, &index, &lport) < 0) fail("expected int (port)"); /* Make a listen socket */ - if ((listen = my_listen(port)) <= 0) + + iport = (int) lport; + lfd = ei_listen(&ec, &iport, 5); + if (lfd < 0) fail("listen"); + lport = (long) iport; - if ((i = ei_publish(&ec, port)) == -1) + if ((i = ei_publish(&ec, lport)) == -1) fail("ei_publish"); #ifdef VXWORKS save_fd(i); @@ -171,7 +154,7 @@ static void cmd_ei_publish(char* buf, int len) /* send listen-fd, result and errno */ ei_x_new_with_version(&x); ei_x_encode_tuple_header(&x, 3); - ei_x_encode_long(&x, listen); + ei_x_encode_long(&x, (long) lfd); ei_x_encode_long(&x, i); ei_x_encode_long(&x, erl_errno); send_bin_term(&x); diff --git a/lib/erl_interface/test/ei_accept_SUITE_data/eiaccnode.c b/lib/erl_interface/test/ei_accept_SUITE_data/eiaccnode.c index 308f843530..90c7a2259f 100644 --- a/lib/erl_interface/test/ei_accept_SUITE_data/eiaccnode.c +++ b/lib/erl_interface/test/ei_accept_SUITE_data/eiaccnode.c @@ -47,8 +47,6 @@ #define MAIN main #endif -static int my_listen(int port); - /* A small einode. To be called from the test case ei_accept_SUITE:multi_thread @@ -64,7 +62,6 @@ static int my_listen(int port); */ static const char* cookie, * desthost; -static int port; /* actually base port */ #ifndef SD_SEND #ifdef SHUTWR @@ -74,10 +71,6 @@ static int port; /* actually base port */ #endif #endif -#ifndef __WIN32__ -#define closesocket(fd) close(fd) -#endif - #ifdef __WIN32__ static DWORD WINAPI #else @@ -86,26 +79,32 @@ static void* einode_thread(void* num) { int n = (int)num; + int port; ei_cnode ec; - char myname[100], destname[100]; + char myname[100], destname[100], filename[100]; int r, fd, listen; ErlConnect conn; erlang_msg msg; -/* FILE* f;*/ + FILE* file; - sprintf(myname, "eiacc%d", n); - printf("thread %d (%s) listening\n", n, myname, destname); + sprintf(filename, "eiacc%d_trace.txt", n); + file = fopen(filename, "w"); + + sprintf(myname, "eiacc%d", n); fflush(file); r = ei_connect_init(&ec, myname, cookie, 0); - if ((listen = my_listen(port+n)) <= 0) { - printf("listen err\n"); + port = 0; + listen = ei_listen(&ec, &port, 5); + if (listen <= 0) { + fprintf(file, "listen err\n"); fflush(file); exit(7); } - if (ei_publish(&ec, port + n) == -1) { - printf("ei_publish port %d\n", port+n); + fprintf(file, "thread %d (%s:%s) listening on port %d\n", n, myname, destname, port); + if (ei_publish(&ec, port) == -1) { + fprintf(file, "ei_publish port %d\n", port+n); fflush(file); exit(8); } fd = ei_accept(&ec, listen, &conn); - printf("ei_accept %d\n", fd); + fprintf(file, "ei_accept %d\n", fd); fflush(file); if (fd >= 0) { ei_x_buff x, xs; int index, version; @@ -117,37 +116,38 @@ static void* if (got == ERL_TICK) continue; if (got == ERL_ERROR) { - printf("receive error %d\n", n); + fprintf(file, "receive error %d\n", n); fflush(file); return 0; } - printf("received %d\n", got); + fprintf(file, "received %d\n", got); fflush(file); break; } index = 0; if (ei_decode_version(x.buff, &index, &version) != 0) { - printf("ei_decode_version %d\n", n); + fprintf(file, "ei_decode_version %d\n", n); fflush(file); return 0; } if (ei_decode_pid(x.buff, &index, &pid) != 0) { - printf("ei_decode_pid %d\n", n); + fprintf(file, "ei_decode_pid %d\n", n); fflush(file); return 0; } -/* fprintf(f, "got pid from %s \n", pid.node);*/ + fprintf(file, "got pid from %s \n", pid.node); fflush(file); ei_x_new_with_version(&xs); ei_x_encode_tuple_header(&xs, 2); ei_x_encode_long(&xs, n); ei_x_encode_pid(&xs, &pid); r = ei_send(fd, &pid, xs.buff, xs.index); -/* fprintf(f, "sent %d bytes %d\n", xs.index, r);*/ + fprintf(file, "sent %d bytes %d\n", xs.index, r); fflush(file); shutdown(fd, SD_SEND); - closesocket(fd); + ei_close_connection(fd); ei_x_free(&x); ei_x_free(&xs); } else { - printf("coudn't connect fd %d r %d\n", fd, r); + fprintf(file, "coudn't connect fd %d r %d\n", fd, r); fflush(file); } - printf("done thread %d\n", n); -/* fclose(f);*/ + ei_close_connection(listen); + fprintf(file, "done thread %d\n", n); + fclose(file); return 0; } @@ -170,12 +170,16 @@ MAIN(int argc, char *argv[]) if (n > 100) exit(2); desthost = argv[3]; - port = atoi(argv[4]); -#ifndef VXWORKS - no_threads = argv[5] != NULL && strcmp(argv[5], "nothreads") == 0; -#else + if (argc == 3) + no_threads = 0; + else + no_threads = argv[4] != NULL && strcmp(argv[4], "nothreads") == 0; +#ifdef VXWORKS no_threads = 1; #endif + + ei_init(); + for (i = 0; i < n; ++i) { if (!no_threads) { #ifndef VXWORKS @@ -209,27 +213,3 @@ MAIN(int argc, char *argv[]) printf("ok\n"); return 0; } - -static int my_listen(int port) -{ - int listen_fd; - struct sockaddr_in addr; - const char *on = "1"; - - if ((listen_fd = socket(AF_INET, SOCK_STREAM, 0)) < 0) - return -1; - - setsockopt(listen_fd, SOL_SOCKET, SO_REUSEADDR, on, sizeof(on)); - - memset((void*) &addr, 0, (size_t) sizeof(addr)); - addr.sin_family = AF_INET; - addr.sin_port = htons(port); - addr.sin_addr.s_addr = htonl(INADDR_ANY); - - if (bind(listen_fd, (struct sockaddr*) &addr, sizeof(addr)) < 0) - return -1; - - listen(listen_fd, 5); - return listen_fd; -} - diff --git a/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c b/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c index 29c03d7604..58c0c7f8d8 100644 --- a/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c +++ b/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c @@ -73,6 +73,8 @@ TESTCASE(interpret) int i; ei_term term; + ei_init(); + ei_x_new(&x); while (get_bin_term(&x, &term) == 0) { char* buf = x.buff, func[MAXATOMLEN]; diff --git a/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c b/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c index f945a7d378..e516f310b6 100644 --- a/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c +++ b/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c @@ -321,6 +321,8 @@ int ei_decode_my_string(const char *buf, int *index, char *to, TESTCASE(test_ei_decode_long) { + ei_init(); + EI_DECODE_2 (decode_long, 2, long, 0); EI_DECODE_2 (decode_long, 2, long, 255); EI_DECODE_2 (decode_long, 5, long, 256); @@ -363,6 +365,8 @@ TESTCASE(test_ei_decode_long) TESTCASE(test_ei_decode_ulong) { + ei_init(); + EI_DECODE_2 (decode_ulong, 2, unsigned long, 0); EI_DECODE_2 (decode_ulong, 2, unsigned long, 255); EI_DECODE_2 (decode_ulong, 5, unsigned long, 256); @@ -409,6 +413,8 @@ TESTCASE(test_ei_decode_ulong) TESTCASE(test_ei_decode_longlong) { + ei_init(); + #ifndef VXWORKS EI_DECODE_2 (decode_longlong, 2, EI_LONGLONG, 0); EI_DECODE_2 (decode_longlong, 2, EI_LONGLONG, 255); @@ -443,6 +449,8 @@ TESTCASE(test_ei_decode_longlong) TESTCASE(test_ei_decode_ulonglong) { + ei_init(); + #ifndef VXWORKS EI_DECODE_2 (decode_ulonglong, 2, EI_ULONGLONG, 0); EI_DECODE_2 (decode_ulonglong, 2, EI_ULONGLONG, 255); @@ -478,6 +486,8 @@ TESTCASE(test_ei_decode_ulonglong) TESTCASE(test_ei_decode_char) { + ei_init(); + EI_DECODE_2(decode_char, 2, char, 0); EI_DECODE_2(decode_char, 2, char, 0x7f); EI_DECODE_2(decode_char, 2, char, 0xff); @@ -491,6 +501,8 @@ TESTCASE(test_ei_decode_char) TESTCASE(test_ei_decode_nonoptimal) { + ei_init(); + EI_DECODE_2(decode_char, 2, char, 42); EI_DECODE_2(decode_char, 5, char, 42); EI_DECODE_2(decode_char, 4, char, 42); @@ -612,6 +624,8 @@ TESTCASE(test_ei_decode_nonoptimal) TESTCASE(test_ei_decode_misc) { + ei_init(); + /* EI_DECODE_0(decode_version); */ @@ -647,6 +661,7 @@ TESTCASE(test_ei_decode_misc) TESTCASE(test_ei_decode_utf8_atom) { + ei_init(); EI_DECODE_STRING_4(decode_my_atom_as, 4, P99({229,0}), /* LATIN1 "�" */ P99({ERLANG_ANY,ERLANG_LATIN1,ERLANG_LATIN1})); diff --git a/lib/erl_interface/test/ei_decode_encode_SUITE_data/ei_decode_encode_test.c b/lib/erl_interface/test/ei_decode_encode_SUITE_data/ei_decode_encode_test.c index 9977683d59..55d9ed1b1a 100644 --- a/lib/erl_interface/test/ei_decode_encode_SUITE_data/ei_decode_encode_test.c +++ b/lib/erl_interface/test/ei_decode_encode_SUITE_data/ei_decode_encode_test.c @@ -477,6 +477,8 @@ TESTCASE(test_ei_decode_encode) { int i; + ei_init(); + decode_encode_one(&fun_type); decode_encode_one(&pid_type); decode_encode_one(&port_type); diff --git a/lib/erl_interface/test/ei_encode_SUITE_data/ei_encode_test.c b/lib/erl_interface/test/ei_encode_SUITE_data/ei_encode_test.c index 32811fdf22..6f63cc5d7e 100644 --- a/lib/erl_interface/test/ei_encode_SUITE_data/ei_encode_test.c +++ b/lib/erl_interface/test/ei_encode_SUITE_data/ei_encode_test.c @@ -403,6 +403,8 @@ TESTCASE(test_ei_encode_long) { + ei_init(); + EI_ENCODE_1(encode_long, 0); EI_ENCODE_1(encode_long, 255); @@ -430,6 +432,8 @@ TESTCASE(test_ei_encode_long) TESTCASE(test_ei_encode_ulong) { + ei_init(); + EI_ENCODE_1(encode_ulong, 0); EI_ENCODE_1(encode_ulong, 255); @@ -454,6 +458,7 @@ TESTCASE(test_ei_encode_ulong) TESTCASE(test_ei_encode_longlong) { + ei_init(); #ifndef VXWORKS @@ -494,6 +499,7 @@ TESTCASE(test_ei_encode_longlong) TESTCASE(test_ei_encode_ulonglong) { + ei_init(); #ifndef VXWORKS @@ -527,6 +533,8 @@ TESTCASE(test_ei_encode_ulonglong) TESTCASE(test_ei_encode_char) { + ei_init(); + EI_ENCODE_1(encode_char, 0); EI_ENCODE_1(encode_char, 0x7f); @@ -540,6 +548,8 @@ TESTCASE(test_ei_encode_char) TESTCASE(test_ei_encode_misc) { + ei_init(); + EI_ENCODE_0(encode_version); EI_ENCODE_1(encode_double, 0.0); @@ -594,6 +604,8 @@ TESTCASE(test_ei_encode_fails) char buf[1024]; int index; + ei_init(); + /* FIXME the ei_x versions are not tested */ index = 0; @@ -660,6 +672,7 @@ TESTCASE(test_ei_encode_fails) TESTCASE(test_ei_encode_utf8_atom) { + ei_init(); EI_ENCODE_3(encode_atom_as, "�", ERLANG_LATIN1, ERLANG_UTF8); EI_ENCODE_3(encode_atom_as, "�", ERLANG_LATIN1, ERLANG_LATIN1); @@ -686,6 +699,7 @@ TESTCASE(test_ei_encode_utf8_atom) TESTCASE(test_ei_encode_utf8_atom_len) { + ei_init(); EI_ENCODE_4(encode_atom_len_as, "���", 1, ERLANG_LATIN1, ERLANG_UTF8); EI_ENCODE_4(encode_atom_len_as, "���", 2, ERLANG_LATIN1, ERLANG_LATIN1); diff --git a/lib/erl_interface/test/ei_format_SUITE_data/ei_format_test.c b/lib/erl_interface/test/ei_format_SUITE_data/ei_format_test.c index 8450332b28..1c0443c0f4 100644 --- a/lib/erl_interface/test/ei_format_SUITE_data/ei_format_test.c +++ b/lib/erl_interface/test/ei_format_SUITE_data/ei_format_test.c @@ -48,6 +48,8 @@ send_format(char* format) TESTCASE(atoms) { + ei_init(); + send_format("''"); send_format("'a'"); send_format("'A'"); @@ -82,6 +84,8 @@ TESTCASE(atoms) TESTCASE(tuples) { + ei_init(); + send_format("{}"); send_format("{a}"); send_format("{a, b}"); @@ -108,6 +112,8 @@ TESTCASE(lists) ei_x_buff x; static char str[65537]; + ei_init(); + send_format("[]"); send_format("[a]"); send_format("[a, b]"); @@ -177,6 +183,8 @@ TESTCASE(format_wo_ver) { */ ei_x_buff x; + ei_init(); + ei_x_new (&x); ei_x_format(&x, "[-1, +2, ~c, {~a,~s},{~a,~i}]", 'c', "a", "b", "c", 10); send_bin_term(&x); diff --git a/lib/erl_interface/test/ei_print_SUITE_data/ei_print_test.c b/lib/erl_interface/test/ei_print_SUITE_data/ei_print_test.c index 15cfbcae34..80be3016e6 100644 --- a/lib/erl_interface/test/ei_print_SUITE_data/ei_print_test.c +++ b/lib/erl_interface/test/ei_print_SUITE_data/ei_print_test.c @@ -84,6 +84,8 @@ static void send_printed3f(char* format, float f1, float f2) TESTCASE(atoms) { + ei_init(); + send_printed("''"); send_printed("'a'"); send_printed("'A'"); @@ -118,6 +120,8 @@ TESTCASE(atoms) TESTCASE(tuples) { + ei_init(); + send_printed("{}"); send_printed("{a}"); send_printed("{a, b}"); @@ -138,6 +142,8 @@ TESTCASE(lists) { ei_x_buff x; + ei_init(); + send_printed("[]"); send_printed("[a]"); send_printed("[a, b]"); @@ -164,6 +170,8 @@ TESTCASE(strings) { ei_x_buff x; + ei_init(); + send_printed("\"\n\""); send_printed("\"\r\n\""); send_printed("\"a\""); diff --git a/lib/erl_interface/test/ei_tmo_SUITE_data/ei_tmo_test.c b/lib/erl_interface/test/ei_tmo_SUITE_data/ei_tmo_test.c index 39846e4a58..693e405f75 100644 --- a/lib/erl_interface/test/ei_tmo_SUITE_data/ei_tmo_test.c +++ b/lib/erl_interface/test/ei_tmo_SUITE_data/ei_tmo_test.c @@ -96,6 +96,8 @@ TESTCASE(framework_check) int i; #endif + ei_init(); + OPEN_DEBUGFILE(1); DEBUGF(("B�rjar... \n")); @@ -340,6 +342,7 @@ TESTCASE(recv_tmo) int com_sock = -1; ei_cnode nodeinfo; + ei_init(); OPEN_DEBUGFILE(5); @@ -450,6 +453,7 @@ TESTCASE(send_tmo) int com_sock = -1; ei_cnode nodeinfo; + ei_init(); OPEN_DEBUGFILE(4); @@ -591,7 +595,7 @@ TESTCASE(connect_tmo) int com_sock = -1; ei_cnode nodeinfo; - + ei_init(); OPEN_DEBUGFILE(3); @@ -680,7 +684,7 @@ TESTCASE(accept_tmo) ErlConnect peer; ei_cnode nodeinfo; - + ei_init(); OPEN_DEBUGFILE(2); diff --git a/lib/erl_interface/test/erl_eterm_SUITE_data/cnode.c b/lib/erl_interface/test/erl_eterm_SUITE_data/cnode.c index bead0f8413..b87feb9dfc 100644 --- a/lib/erl_interface/test/erl_eterm_SUITE_data/cnode.c +++ b/lib/erl_interface/test/erl_eterm_SUITE_data/cnode.c @@ -20,7 +20,7 @@ #include <stdlib.h> #include <stdio.h> - +#include <string.h> #include "ei.h" #include "erl_interface.h" @@ -68,6 +68,7 @@ MAIN(int argc, char **argv) char host[80]; int number; ETERM *ref, *ref1, *ref2; + FILE *dfile = fopen("cnode_debug_printout", "w"); erl_init(NULL, 0); @@ -80,28 +81,30 @@ MAIN(int argc, char **argv) gethostname(host, sizeof(host)); sprintf(node, "c%d@%s", number, host); - printf("s = %d\n", s); + fprintf(dfile, "s = %d\n", s); fflush(dfile); sprintf(server, "test_server@%s", host); fd = erl_connect(server); - printf("fd = %d\n", fd); + fprintf(dfile, "fd = %d\n", fd); -/* printf("dist = %d\n", erl_distversion(fd)); */ +/* fprintf(dfile, "dist = %d\n", erl_distversion(fd)); */ #if 1 ref = erl_mk_long_ref(node, 4711, 113, 98, 0); #else ref = erl_mk_ref(node, 4711, 0); #endif - printf("ref = %d\n", ref); + fprintf(dfile, "ref = %p\n", ref); fflush(dfile); s = erl_reg_send(fd, "mip", ref); - printf("s = %d\n", s); + fprintf(dfile, "s = %d\n", s); fflush(dfile); { ETERM* emsg; emsg = SELF(fd); - erl_reg_send(fd,"mip",emsg); + fprintf(dfile, "pid = %p\n", emsg); fflush(dfile); + s = erl_reg_send(fd,"mip",emsg); + fprintf(dfile, "s2 = %d\n", s); fflush(dfile); erl_free_term(emsg); } @@ -116,28 +119,29 @@ MAIN(int argc, char **argv) #endif switch (s) { case ERL_TICK: - printf("tick\n"); + fprintf(dfile, "tick\n"); break; case ERL_ERROR: - printf("error\n"); + fprintf(dfile, "error: %s (%d)\n", strerror(erl_errno), erl_errno); break; case ERL_MSG: - printf("msg %d\n", msgsize); + fprintf(dfile, "msg %d\n", msgsize); break; default: - printf("unknown result %d\n", s); + fprintf(dfile, "unknown result %d\n", s); break; } + fflush(dfile); } while (s == ERL_TICK); s = erl_reg_send(fd, "mip", msg.msg); - printf("s = %d\n", s); + fprintf(dfile, "s = %d\n", s); fflush(dfile); s = erl_reg_send(fd, "mip", msg.to); - printf("s = %d\n", s); + fprintf(dfile, "s = %d\n", s); fflush(dfile); #if 0 /* from = NULL! */ s = erl_reg_send(fd, "mip", msg.from); - printf("s = %d\n", s); + fprintf(dfile, "s = %d\n", s); fflush(dfile); #endif #if 0 @@ -150,17 +154,19 @@ MAIN(int argc, char **argv) ref1 = erl_mk_long_ref(node, 4711, 113, 98, 0); ref2 = erl_mk_ref(node, 4711, 0); s = erl_encode(ref1, buf1); - printf("enc1 s = %d\n", s); + fprintf(dfile, "enc1 s = %d\n", s); fflush(dfile); s = erl_encode(ref2, buf2); - printf("enc2 s = %d\n", s); + fprintf(dfile, "enc2 s = %d\n", s); fflush(dfile); s = erl_compare_ext(buf1, buf2); - printf("comp s = %d\n", s); + fprintf(dfile, "comp s = %d\n", s); fflush(dfile); /* Compare, in another way */ s = erl_match(ref1, ref2); - printf("match s = %d\n", s); + fprintf(dfile, "match s = %d\n", s); fflush(dfile); #endif + fclose(dfile); + erl_close_connection(fd); return 0; diff --git a/lib/ftp/doc/src/ftp.xml b/lib/ftp/doc/src/ftp.xml index ead2367925..9645b03364 100644 --- a/lib/ftp/doc/src/ftp.xml +++ b/lib/ftp/doc/src/ftp.xml @@ -550,7 +550,7 @@ <v>ipfamily() = inet | inet6 | inet6fb4 (default is inet)</v> <v>port() = integer() > 0 (default is 21)</v> <v>mode() = active | passive (default is passive)</v> - <v>tls_options() = [<seealso marker="ssl:ssl#type-ssloption">ssl:ssloption()</seealso>]</v> + <v>tls_options() = [<seealso marker="ssl:ssl#type-tls_option">ssl:tls_option()</seealso>]</v> <v>sock_opts() = [<seealso marker="kernel:gen_tcp#type-option">gen_tcp:option()</seealso> except for ipv6_v6only, active, packet, mode, packet_size and header</v> <v>timeout() = integer() > 0 (default is 60000 milliseconds)</v> <v>dtimeout() = integer() > 0 | infinity (default is infinity)</v> diff --git a/lib/hipe/cerl/erl_bif_types.erl b/lib/hipe/cerl/erl_bif_types.erl index 48ce641ab9..8ae1cd4ab7 100644 --- a/lib/hipe/cerl/erl_bif_types.erl +++ b/lib/hipe/cerl/erl_bif_types.erl @@ -2225,10 +2225,7 @@ type_order() -> t_map(), t_list(), t_bitstr()]. key_comparisons_fail(X0, KeyPos, TupleList, Opaques) -> - X = case t_is_number(t_inf(X0, t_number(), Opaques), Opaques) of - false -> X0; - true -> t_number() - end, + X = erl_types:t_widen_to_number(X0), lists:all(fun(Tuple) -> Key = type(erlang, element, 2, [KeyPos, Tuple]), t_is_none(t_inf(Key, X, Opaques)) diff --git a/lib/hipe/cerl/erl_types.erl b/lib/hipe/cerl/erl_types.erl index 9abb4d31d9..d61cd8664c 100644 --- a/lib/hipe/cerl/erl_types.erl +++ b/lib/hipe/cerl/erl_types.erl @@ -66,7 +66,6 @@ t_find_opaque_mismatch/3, t_find_unknown_opaque/3, t_fixnum/0, - t_map/2, t_non_neg_fixnum/0, t_pos_fixnum/0, t_float/0, @@ -205,6 +204,7 @@ t_unopaque/1, t_unopaque/2, t_var/1, t_var_name/1, + t_widen_to_number/1, %% t_assign_variables_to_subtype/2, type_is_defined/4, record_field_diffs_to_string/2, @@ -1594,6 +1594,50 @@ lift_list_to_pos_empty(?nil) -> ?nil; lift_list_to_pos_empty(?list(Content, Termination, _)) -> ?list(Content, Termination, ?unknown_qual). +-spec t_widen_to_number(erl_type()) -> erl_type(). + +%% Widens integers and floats to t_number(). +%% Used by erl_bif_types:key_comparison_fail(). + +t_widen_to_number(?any) -> ?any; +t_widen_to_number(?none) -> ?none; +t_widen_to_number(?unit) -> ?unit; +t_widen_to_number(?atom(_Set) = T) -> T; +t_widen_to_number(?bitstr(_Unit, _Base) = T) -> T; +t_widen_to_number(?float) -> t_number(); +t_widen_to_number(?function(Domain, Range)) -> + ?function(t_widen_to_number(Domain), t_widen_to_number(Range)); +t_widen_to_number(?identifier(_Types) = T) -> T; +t_widen_to_number(?int_range(_From, _To)) -> t_number(); +t_widen_to_number(?int_set(_Set)) -> t_number(); +t_widen_to_number(?integer(_Types)) -> t_number(); +t_widen_to_number(?list(Type, Tail, Size)) -> + ?list(t_widen_to_number(Type), t_widen_to_number(Tail), Size); +t_widen_to_number(?map(Pairs, DefK, DefV)) -> + L = [{t_widen_to_number(K), MNess, t_widen_to_number(V)} || + {K, MNess, V} <- Pairs], + t_map(L, t_widen_to_number(DefK), t_widen_to_number(DefV)); +t_widen_to_number(?matchstate(_P, _Slots) = T) -> T; +t_widen_to_number(?nil) -> ?nil; +t_widen_to_number(?number(_Set, _Tag)) -> t_number(); +t_widen_to_number(?opaque(Set)) -> + L = [Opaque#opaque{struct = t_widen_to_number(S)} || + #opaque{struct = S} = Opaque <- set_to_list(Set)], + ?opaque(ordsets:from_list(L)); +t_widen_to_number(?product(Types)) -> + ?product(list_widen_to_number(Types)); +t_widen_to_number(?tuple(?any, _, _) = T) -> T; +t_widen_to_number(?tuple(Types, Arity, Tag)) -> + ?tuple(list_widen_to_number(Types), Arity, Tag); +t_widen_to_number(?tuple_set(_) = Tuples) -> + t_sup([t_widen_to_number(T) || T <- t_tuple_subtypes(Tuples)]); +t_widen_to_number(?union(List)) -> + ?union(list_widen_to_number(List)); +t_widen_to_number(?var(_Id)= T) -> T. + +list_widen_to_number(List) -> + [t_widen_to_number(E) || E <- List]. + %%----------------------------------------------------------------------------- %% Maps %% @@ -3104,9 +3148,18 @@ is_compat_arg(?list(Contents1, Termination1, Size1), is_compat_arg(?product(Types1), ?product(Types2)) -> is_compat_list(Types1, Types2); is_compat_arg(?map(Pairs1, DefK1, DefV1), ?map(Pairs2, DefK2, DefV2)) -> - (is_compat_list(Pairs1, Pairs2) andalso - is_compat_arg(DefK1, DefK2) andalso - is_compat_arg(DefV1, DefV2)); + {Ks1, _, Vs1} = lists:unzip3(Pairs1), + {Ks2, _, Vs2} = lists:unzip3(Pairs2), + Key1 = t_sup([DefK1 | Ks1]), + Key2 = t_sup([DefK2 | Ks2]), + case is_compat_arg(Key1, Key2) of + true -> + Value1 = t_sup([DefV1 | Vs1]), + Value2 = t_sup([DefV2 | Vs2]), + is_compat_arg(Value1, Value2); + false -> + false + end; is_compat_arg(?tuple(?any, ?any, ?any), ?tuple(_, _, _)) -> false; is_compat_arg(?tuple(_, _, _), ?tuple(?any, ?any, ?any)) -> false; is_compat_arg(?tuple(Elements1, Arity, _), @@ -4156,39 +4209,6 @@ t_abstract_records(?opaque(_)=Type, RecDict) -> t_abstract_records(T, _RecDict) -> T. -%% Map over types. Depth first. Used by the contract checker. ?list is -%% not fully implemented so take care when changing the type in Termination. - --spec t_map(fun((erl_type()) -> erl_type()), erl_type()) -> erl_type(). - -t_map(Fun, ?list(Contents, Termination, Size)) -> - Fun(?list(t_map(Fun, Contents), t_map(Fun, Termination), Size)); -t_map(Fun, ?function(Domain, Range)) -> - Fun(?function(t_map(Fun, Domain), t_map(Fun, Range))); -t_map(Fun, ?product(Types)) -> - Fun(?product([t_map(Fun, T) || T <- Types])); -t_map(Fun, ?union(Types)) -> - Fun(t_sup([t_map(Fun, T) || T <- Types])); -t_map(Fun, ?tuple(?any, ?any, ?any) = T) -> - Fun(T); -t_map(Fun, ?tuple(Elements, _Arity, _Tag)) -> - Fun(t_tuple([t_map(Fun, E) || E <- Elements])); -t_map(Fun, ?tuple_set(_) = Tuples) -> - Fun(t_sup([t_map(Fun, T) || T <- t_tuple_subtypes(Tuples)])); -t_map(Fun, ?opaque(Set)) -> - L = [Opaque#opaque{struct = NewS} || - #opaque{struct = S} = Opaque <- set_to_list(Set), - not t_is_none(NewS = t_map(Fun, S))], - Fun(case L of - [] -> ?none; - _ -> ?opaque(ordsets:from_list(L)) - end); -t_map(Fun, ?map(Pairs,DefK,DefV)) -> - %% TODO: - Fun(t_map(Pairs, Fun(DefK), Fun(DefV))); -t_map(Fun, T) -> - Fun(T). - %%============================================================================= %% %% Prettyprinter diff --git a/lib/inets/doc/src/httpd_util.xml b/lib/inets/doc/src/httpd_util.xml index 29971ba8ae..e0f947f860 100644 --- a/lib/inets/doc/src/httpd_util.xml +++ b/lib/inets/doc/src/httpd_util.xml @@ -45,8 +45,7 @@ <fsummary>Converts the date to the Erlang date format.</fsummary> <type> <v>DateString = string()</v> - <v>ErlDate = {{Year,Month,Date},{Hour,Min,Sec}}</v> - <v>Year = Month = Date = Hour = Min = Sec = integer()</v> + <v>ErlDate = calendar:datetime() </v> </type> <desc> <p><c>convert_request_date/1</c> converts <c>DateString</c> to @@ -281,10 +280,10 @@ <func> <name since="">rfc1123_date() -> RFC1123Date</name> - <name since="">rfc1123_date({{YYYY,MM,DD},{Hour,Min,Sec}}) -> RFC1123Date</name> + <name since="">rfc1123_date(Date) -> RFC1123Date</name> <fsummary>Returns the current date in RFC 1123 format.</fsummary> <type> - <v>YYYY = MM = DD = Hour = Min = Sec = integer()</v> + <v> Date = calendar:datetime()</v> <v>RFC1123Date = string()</v> </type> <desc> diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml index 12f5acb2e9..3624c6e3d7 100644 --- a/lib/inets/doc/src/notes.xml +++ b/lib/inets/doc/src/notes.xml @@ -33,7 +33,47 @@ <file>notes.xml</file> </header> - <section><title>Inets 7.0.3</title> + <section><title>Inets 7.0.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed bug that causes a crash in http client when using + hostnames (e.g. localhost) with the the option + ipv6_host_with_brackets set to true.</p> + <p> + This change also fixes a regression: httpc:request fails + with connection error (nxdomain) if option + ipv6_host_with_brackets set to true and host component of + the URI is an IPv6 address.</p> + <p> + Own Id: OTP-15554 Aux Id: ERIERL-289 </p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 7.0.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Make sure ipv6 addresses with brackets in URIs are + converted correctly before passing to lower level + functions like gen_tcp and ssl functions. Could cause + connection to fail.</p> + <p> + Own Id: OTP-15544 Aux Id: ERIERL-289 </p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 7.0.3</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/inets/src/http_client/httpc_handler.erl b/lib/inets/src/http_client/httpc_handler.erl index 1bf5d25c98..8d443a1477 100644 --- a/lib/inets/src/http_client/httpc_handler.erl +++ b/lib/inets/src/http_client/httpc_handler.erl @@ -809,7 +809,7 @@ connect_and_send_first_request(Address, Request, #state{options = Options0} = St SocketType = socket_type(Request), ConnTimeout = (Request#request.settings)#http_options.connect_timeout, Options = handle_unix_socket_options(Request, Options0), - case connect(SocketType, Address, Options, ConnTimeout) of + case connect(SocketType, format_address(Address), Options, ConnTimeout) of {ok, Socket} -> ClientClose = httpc_request:is_client_closing( @@ -1738,4 +1738,8 @@ update_session(ProfileName, #session{id = SessionId} = Session, Pos, Value) -> {stacktrace, Stacktrace}]}} end. - +format_address({[$[|T], Port}) -> + {ok, Address} = inet:parse_address(string:strip(T, right, $])), + {Address, Port}; +format_address(HostPort) -> + HostPort. diff --git a/lib/inets/vsn.mk b/lib/inets/vsn.mk index 52c05a7974..921161dce1 100644 --- a/lib/inets/vsn.mk +++ b/lib/inets/vsn.mk @@ -19,6 +19,6 @@ # %CopyrightEnd% APPLICATION = inets -INETS_VSN = 7.0.3 +INETS_VSN = 7.0.5 PRE_VSN = APP_VSN = "$(APPLICATION)-$(INETS_VSN)$(PRE_VSN)" diff --git a/lib/kernel/doc/src/gen_sctp.xml b/lib/kernel/doc/src/gen_sctp.xml index 1e7009b3a8..f70d6c24db 100644 --- a/lib/kernel/doc/src/gen_sctp.xml +++ b/lib/kernel/doc/src/gen_sctp.xml @@ -284,7 +284,7 @@ connect(Socket, Ip, Port>, <func> <name name="listen" arity="2" clause_i="1" since=""/> - <name name="listen" arity="2" clause_i="2" since=""/> + <name name="listen" arity="2" clause_i="2" since="OTP R15B"/> <fsummary>Set up a socket to listen.</fsummary> <desc> <p>Sets up a socket to listen on the IP address and port number diff --git a/lib/kernel/doc/src/logger.xml b/lib/kernel/doc/src/logger.xml index e09c5db5e3..df2d081d76 100644 --- a/lib/kernel/doc/src/logger.xml +++ b/lib/kernel/doc/src/logger.xml @@ -245,6 +245,12 @@ logger:error("error happened because: ~p", [Reason]). % Without macro </desc> </datatype> <datatype> + <name name="olp_config"/> + <desc> + <p></p> + </desc> + </datatype> + <datatype> <name name="primary_config"/> <desc> <p>Primary configuration data for Logger. The following @@ -597,8 +603,8 @@ start(_, []) -> <name name="get_config" arity="0" since="OTP 21.0"/> <fsummary>Look up the current Logger configuration</fsummary> <desc> - <p>Look up all current Logger configuration, including primary - and handler configuration, and module level settings.</p> + <p>Look up all current Logger configuration, including primary, + handler, and proxy configuration, and module level settings.</p> </desc> </func> @@ -636,6 +642,17 @@ start(_, []) -> </func> <func> + <name name="get_proxy_config" arity="0" since="OTP 21.3"/> + <fsummary>Look up the current configuration for the Logger proxy.</fsummary> + <desc> + <p>Look up the current configuration for the Logger proxy.</p> + <p>For more information about the proxy, see + section <seealso marker="logger_chapter#proxy">Logger + Proxy</seealso> in the Kernel User's Guide.</p> + </desc> + </func> + + <func> <name name="get_module_level" arity="0" since="OTP 21.0"/> <fsummary>Look up all current module levels.</fsummary> <desc> @@ -672,6 +689,15 @@ start(_, []) -> </func> <func> + <name name="i" arity="0" since="OTP 21.3"/> + <name name="i" arity="1" since="OTP 21.3"/> + <fsummary>Pretty print the Logger configuration.</fsummary> + <desc> + <p>Pretty print the Logger configuration.</p> + </desc> + </func> + + <func> <name name="remove_handler" arity="1" since="OTP 21.0"/> <fsummary>Remove the handler with the specified identity.</fsummary> <desc> @@ -801,6 +827,27 @@ start(_, []) -> </func> <func> + <name name="set_proxy_config" arity="1" since="OTP 21.3"/> + <fsummary>Set configuration data for the Logger proxy.</fsummary> + <desc> + <p>Set configuration data for the Logger proxy. This + overwrites the current proxy configuration. Keys that are not + specified in the <c><anno>Config</anno></c> map gets default + values.</p> + <p>To modify the existing configuration, + use <seealso marker="#update_proxy_config-1"> + <c>update_proxy_config/1</c></seealso>, or, if a more + complex merge is needed, read the current configuration + with <seealso marker="#get_proxy_config-0"><c>get_proxy_config/0</c> + </seealso>, then do the merge before writing the new + configuration back with this function.</p> + <p>For more information about the proxy, see + section <seealso marker="logger_chapter#proxy">Logger + Proxy</seealso> in the Kernel User's Guide.</p> + </desc> + </func> + + <func> <name name="set_module_level" arity="2" since="OTP 21.0"/> <fsummary>Set the log level for the specified modules.</fsummary> <desc> @@ -1013,6 +1060,25 @@ logger:set_process_metadata(maps:merge(logger:get_process_metadata(), Meta)). </seealso>.</p> </desc> </func> + + <func> + <name name="update_proxy_config" arity="1" since="OTP 21.3"/> + <fsummary>Update configuration data for the Logger proxy.</fsummary> + <desc> + <p>Update configuration data for the Logger proxy. This function + behaves as if it was implemented as follows:</p> + <code type="erl"> +Old = logger:get_proxy_config(), +logger:set_proxy_config(maps:merge(Old, Config)). + </code> + <p>To overwrite the existing configuration without any merge, + use <seealso marker="#set_proxy_config-1"><c>set_proxy_config/1</c> + </seealso>.</p> + <p>For more information about the proxy, see + section <seealso marker="logger_chapter#proxy">Logger + Proxy</seealso> in the Kernel User's Guide.</p> + </desc> + </func> </funcs> <section> diff --git a/lib/kernel/doc/src/logger_chapter.xml b/lib/kernel/doc/src/logger_chapter.xml index 458e61cef5..5ed7397135 100644 --- a/lib/kernel/doc/src/logger_chapter.xml +++ b/lib/kernel/doc/src/logger_chapter.xml @@ -693,8 +693,10 @@ logger:debug(#{got => connection_request, id => Id, state => State}, with <seealso marker="#logger_sasl_compatible"> <c>logger_sasl_compatible</c></seealso>.</p> <p>With this parameter, you can modify or disable the default - handler, add custom handlers and primary logger filters, and - set log levels per module.</p> + handler, add custom handlers and primary logger filters, set + log levels per module, and modify + the <seealso marker="#proxy">proxy</seealso> + configuration.</p> <p><c>Config</c> is any (zero or more) of the following:</p> <taglist> <tag><c>{handler, default, undefined}</c></tag> @@ -746,6 +748,14 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <p>for each <c>Module</c>.</p> <p>Multiple entries of this type are allowed.</p> </item> + <tag><c>{proxy, ProxyConfig}</c></tag> + <item> + <p>Sets the proxy configuration, equivalent to calling</p> + <pre><seealso marker="logger#set_proxy_config/1"> + logger:set_proxy_config(ProxyConfig) + </seealso></pre> + <p>Only one entry of this type is allowed.</p> + </item> </taglist> <p>See section <seealso marker="#config_examples">Configuration @@ -1334,9 +1344,50 @@ logger:add_handler(my_disk_log_h, logger_disk_log_h, </section> <section> + <marker id="proxy"/> + <title>Logger Proxy</title> + <p>The Logger proxy is an Erlang process which is part of the + Kernel application's supervision tree. During startup, the proxy + process registers itself as the <c>system_logger</c>, meaning + that log events produced by the emulator are sent to this + process.</p> + <p>When a log event is issued on a process which has its group + leader on a remote node, Logger automatically forwards the log + event to the group leader's node. To achieve this, it first + sends the log event as an Erlang message from the original + client process to the proxy on the local node, and the proxy in + turn forwards the event to the proxy on the remote node.</p> + <p>When receiving a log event, either from the emulator or from a + remote node, the proxy calls the Logger API to log the event.</p> + <p>The proxy process is overload protected in the same way as + described in + section <seealso marker="#overload_protection">Protecting the + Handler from Overload</seealso>, but with the following default + values:</p> + <code> + #{sync_mode_qlen => 500, + drop_mode_qlen => 1000, + flush_qlen => 5000, + burst_limit_enable => false, + overload_kill_enable => false}</code> + <p>For log events from the emulator, synchronous message passing + mode is not applicable, since all messages are passed + asynchronously by the emulator. Drop mode is achieved by setting + the <c>system_logger</c> to <c>undefined</c>, forcing the + emulator to drop events until it is set back to the proxy pid + again.</p> + <p>The proxy uses <seealso marker="erts:erlang#send_nosuspend/2"> + <c>erlang:send_nosuspend/2</c></seealso> when sending log + events to a remote node. If the message could not be sent + without suspending the sender, it is dropped. This is to avoid + blocking the proxy process.</p> + </section> + + <section> <title>See Also</title> <p> <seealso marker="disk_log"><c>disk_log(3)</c></seealso>, + <seealso marker="erts:erlang"><c>erlang(3)</c></seealso>, <seealso marker="error_logger"><c>error_logger(3)</c></seealso>, <seealso marker="logger"><c>logger(3)</c></seealso>, <seealso marker="logger_disk_log_h"><c>logger_disk_log_h(3)</c></seealso>, diff --git a/lib/kernel/src/Makefile b/lib/kernel/src/Makefile index 57f17defc8..3d1506ea08 100644 --- a/lib/kernel/src/Makefile +++ b/lib/kernel/src/Makefile @@ -118,6 +118,8 @@ MODULES = \ logger_h_common \ logger_filters \ logger_formatter \ + logger_olp \ + logger_proxy \ logger_server \ logger_simple_h \ logger_sup \ @@ -151,7 +153,7 @@ INTERNAL_HRL_FILES= application_master.hrl disk_log.hrl \ inet_dns.hrl inet_res.hrl \ inet_boot.hrl inet_config.hrl inet_int.hrl \ inet_dns_record_adts.hrl \ - logger_internal.hrl logger_h_common.hrl + logger_internal.hrl logger_olp.hrl logger_h_common.hrl ERL_FILES= $(MODULES:%=%.erl) @@ -279,6 +281,8 @@ $(EBIN)/logger_config.beam: logger_internal.hrl ../include/logger.hrl $(EBIN)/logger_disk_log_h.beam: logger_h_common.hrl logger_internal.hrl ../include/logger.hrl ../include/file.hrl $(EBIN)/logger_filters.beam: logger_internal.hrl ../include/logger.hrl $(EBIN)/logger_formatter.beam: logger_internal.hrl ../include/logger.hrl +$(EBIN)/logger_olp.beam: logger_olp.hrl logger_internal.hrl +$(EBIN)/logger_proxy.beam: logger_internal.hrl $(EBIN)/logger_server.beam: logger_internal.hrl ../include/logger.hrl $(EBIN)/logger_simple_h.beam: logger_internal.hrl ../include/logger.hrl $(EBIN)/logger_std_h.beam: logger_h_common.hrl logger_internal.hrl ../include/logger.hrl ../include/file.hrl diff --git a/lib/kernel/src/erl_epmd.erl b/lib/kernel/src/erl_epmd.erl index b7e8868911..7a14e2635c 100644 --- a/lib/kernel/src/erl_epmd.erl +++ b/lib/kernel/src/erl_epmd.erl @@ -77,8 +77,8 @@ stop() -> %% -spec port_please(Name, Host) -> {ok, Port, Version} | noport when - Name :: string(), - Host :: inet:ip_address(), + Name :: atom() | string(), + Host :: atom() | string() | inet:ip_address(), Port :: non_neg_integer(), Version :: non_neg_integer(). @@ -86,8 +86,8 @@ port_please(Node, Host) -> port_please(Node, Host, infinity). -spec port_please(Name, Host, Timeout) -> {ok, Port, Version} | noport when - Name :: string(), - Host :: inet:ip_address(), + Name :: atom() | string(), + Host :: atom() | string() | inet:ip_address(), Timeout :: non_neg_integer() | infinity, Port :: non_neg_integer(), Version :: non_neg_integer(). diff --git a/lib/kernel/src/kernel.app.src b/lib/kernel/src/kernel.app.src index fe073621c8..a1d9e8e215 100644 --- a/lib/kernel/src/kernel.app.src +++ b/lib/kernel/src/kernel.app.src @@ -68,6 +68,8 @@ logger_formatter, logger_h_common, logger_handler_watcher, + logger_olp, + logger_proxy, logger_server, logger_simple_h, logger_std_h, diff --git a/lib/kernel/src/logger.erl b/lib/kernel/src/logger.erl index 6762998d4f..7d36640f52 100644 --- a/lib/kernel/src/logger.erl +++ b/lib/kernel/src/logger.erl @@ -43,11 +43,14 @@ get_module_level/0, get_module_level/1, set_primary_config/1, set_primary_config/2, set_handler_config/2, set_handler_config/3, + set_proxy_config/1, update_primary_config/1, update_handler_config/2, update_handler_config/3, + update_proxy_config/1, update_formatter_config/2, update_formatter_config/3, get_primary_config/0, get_handler_config/1, get_handler_config/0, get_handler_ids/0, get_config/0, + get_proxy_config/0, add_handlers/1]). %% Private configuration @@ -57,6 +60,7 @@ -export([compare_levels/2]). -export([set_process_metadata/1, update_process_metadata/1, unset_process_metadata/0, get_process_metadata/0]). +-export([i/0, i/1]). %% Basic report formatting -export([format_report/1, format_otp_report/1]). @@ -122,6 +126,18 @@ {filters,log | stop,[{filter_id(),filter()}]} | {module_level,level(),[module()]}]. +-type olp_config() :: #{sync_mode_qlen => non_neg_integer(), + drop_mode_qlen => pos_integer(), + flush_qlen => pos_integer(), + burst_limit_enable => boolean(), + burst_limit_max_count => pos_integer(), + burst_limit_window_time => pos_integer(), + overload_kill_enable => boolean(), + overload_kill_qlen => pos_integer(), + overload_kill_mem_size => pos_integer(), + overload_kill_restart_after => + non_neg_integer() | infinity}. + -export_type([log_event/0, level/0, report/0, @@ -137,7 +153,8 @@ filter_arg/0, filter_return/0, config_handler/0, - formatter_config/0]). + formatter_config/0, + olp_config/0]). %%%----------------------------------------------------------------- %%% API @@ -390,6 +407,7 @@ set_primary_config(Key,Value) -> set_primary_config(Config) -> logger_server:set_config(primary,Config). + -spec set_handler_config(HandlerId,level,Level) -> Return when HandlerId :: handler_id(), Level :: level() | all | none, @@ -419,6 +437,11 @@ set_handler_config(HandlerId,Key,Value) -> set_handler_config(HandlerId,Config) -> logger_server:set_config(HandlerId,Config). +-spec set_proxy_config(Config) -> ok | {error,term()} when + Config :: olp_config(). +set_proxy_config(Config) -> + logger_server:set_config(proxy,Config). + -spec update_primary_config(Config) -> ok | {error,term()} when Config :: primary_config(). update_primary_config(Config) -> @@ -453,6 +476,11 @@ update_handler_config(HandlerId,Key,Value) -> update_handler_config(HandlerId,Config) -> logger_server:update_config(HandlerId,Config). +-spec update_proxy_config(Config) -> ok | {error,term()} when + Config :: olp_config(). +update_proxy_config(Config) -> + logger_server:update_config(proxy,Config). + -spec get_primary_config() -> Config when Config :: primary_config(). get_primary_config() -> @@ -486,6 +514,12 @@ get_handler_ids() -> {ok,#{handlers:=HandlerIds}} = logger_config:get(?LOGGER_TABLE,primary), HandlerIds. +-spec get_proxy_config() -> Config when + Config :: olp_config(). +get_proxy_config() -> + {ok,Config} = logger_config:get(?LOGGER_TABLE,proxy), + Config. + -spec update_formatter_config(HandlerId,FormatterConfig) -> ok | {error,term()} when HandlerId :: handler_id(), @@ -606,12 +640,150 @@ unset_process_metadata() -> -spec get_config() -> #{primary=>primary_config(), handlers=>[handler_config()], + proxy=>olp_config(), module_levels=>[{module(),level() | all | none}]}. get_config() -> #{primary=>get_primary_config(), handlers=>get_handler_config(), + proxy=>get_proxy_config(), module_levels=>lists:keysort(1,get_module_level())}. +-spec i() -> ok. +i() -> + #{primary := Primary, + handlers := HandlerConfigs, + proxy := Proxy, + module_levels := Modules} = get_config(), + M = modifier(), + i_primary(Primary,M), + i_handlers(HandlerConfigs,M), + i_proxy(Proxy,M), + i_modules(Modules,M). + +-spec i(What) -> ok when + What :: primary | handlers | proxy | modules | handler_id(). +i(primary) -> + i_primary(get_primary_config(),modifier()); +i(handlers) -> + i_handlers(get_handler_config(),modifier()); +i(proxy) -> + i_proxy(get_proxy_config(),modifier()); +i(modules) -> + i_modules(get_module_level(),modifier()); +i(HandlerId) when is_atom(HandlerId) -> + case get_handler_config(HandlerId) of + {ok,HandlerConfig} -> + i_handlers([HandlerConfig],modifier()); + Error -> + Error + end; +i(What) -> + erlang:error(badarg,[What]). + + +i_primary(#{level := Level, + filters := Filters, + filter_default := FilterDefault}, + M) -> + io:format("Primary configuration: ~n",[]), + io:format(" Level: ~p~n",[Level]), + io:format(" Filter Default: ~p~n", [FilterDefault]), + io:format(" Filters: ~n", []), + print_filters(" ",Filters,M). + +i_handlers(HandlerConfigs,M) -> + io:format("Handler configuration: ~n", []), + print_handlers(HandlerConfigs,M). + +i_proxy(Proxy,M) -> + io:format("Proxy configuration: ~n", []), + print_custom(" ",Proxy,M). + +i_modules(Modules,M) -> + io:format("Level set per module: ~n", []), + print_module_levels(Modules,M). + +encoding() -> + case lists:keyfind(encoding, 1, io:getopts()) of + false -> latin1; + {encoding, Enc} -> Enc + end. + +modifier() -> + modifier(encoding()). + +modifier(latin1) -> ""; +modifier(_) -> "t". + +print_filters(Indent, {Id, {Fun, Arg}}, M) -> + io:format("~sId: ~"++M++"p~n" + "~s Fun: ~"++M++"p~n" + "~s Arg: ~"++M++"p~n", + [Indent, Id, Indent, Fun, Indent, Arg]); +print_filters(Indent,[],_M) -> + io:format("~s(none)~n",[Indent]); +print_filters(Indent,Filters,M) -> + [print_filters(Indent,Filter,M) || Filter <- Filters], + ok. + +print_handlers(#{id := Id, + module := Module, + level := Level, + filters := Filters, filter_default := FilterDefault, + formatter := {FormatterModule,FormatterConfig}} = Config, M) -> + io:format(" Id: ~"++M++"p~n" + " Module: ~p~n" + " Level: ~p~n" + " Formatter:~n" + " Module: ~p~n" + " Config:~n", + [Id, Module, Level, FormatterModule]), + print_custom(" ",FormatterConfig,M), + io:format(" Filter Default: ~p~n" + " Filters:~n", + [FilterDefault]), + print_filters(" ",Filters,M), + case maps:find(config,Config) of + {ok,HandlerConfig} -> + io:format(" Handler Config:~n"), + print_custom(" ",HandlerConfig,M); + error -> + ok + end, + MyKeys = [filter_default, filters, formatter, level, module, id, config], + case maps:without(MyKeys,Config) of + Empty when Empty==#{} -> + ok; + Unhandled -> + io:format(" Custom Config:~n"), + print_custom(" ",Unhandled,M) + end; +print_handlers([], _M) -> + io:format(" (none)~n"); +print_handlers(HandlerConfigs, M) -> + [print_handlers(HandlerConfig, M) || HandlerConfig <- HandlerConfigs], + ok. + +print_custom(Indent, {Key, Value}, M) -> + io:format("~s~"++M++"p: ~"++M++"p~n",[Indent,Key,Value]); +print_custom(Indent, Map, M) when is_map(Map) -> + print_custom(Indent,lists:keysort(1,maps:to_list(Map)), M); +print_custom(Indent, List, M) when is_list(List), is_tuple(hd(List)) -> + [print_custom(Indent, X, M) || X <- List], + ok; +print_custom(Indent, Value, M) -> + io:format("~s~"++M++"p~n",[Indent,Value]). + +print_module_levels({Module,Level},M) -> + io:format(" Module: ~"++M++"p~n" + " Level: ~p~n", + [Module,Level]); +print_module_levels([],_M) -> + io:format(" (none)~n"); +print_module_levels(Modules,M) -> + [print_module_levels(Module,M) || Module <- Modules], + ok. + -spec internal_init_logger() -> ok | {error,term()}. %% This function is responsible for config of the logger %% This is done before add_handlers because we want the @@ -672,6 +844,17 @@ init_kernel_handlers(Env) -> %% This function is responsible for resolving the handler config %% and then starting the correct handlers. This is done after the %% kernel supervisor tree has been started as it needs the logger_sup. +add_handlers(kernel) -> + Env = get_logger_env(kernel), + case get_proxy_opts(Env) of + undefined -> + add_handlers(kernel,Env); + Opts -> + case set_proxy_config(Opts) of + ok -> add_handlers(kernel,Env); + {error, Reason} -> {error,{bad_proxy_config,Reason}} + end + end; add_handlers(App) when is_atom(App) -> add_handlers(App,get_logger_env(App)); add_handlers(HandlerConfig) -> @@ -729,6 +912,8 @@ check_logger_config(kernel,[{filters,_,_}|Env]) -> check_logger_config(kernel,Env); check_logger_config(kernel,[{module_level,_,_}|Env]) -> check_logger_config(kernel,Env); +check_logger_config(kernel,[{proxy,_}|Env]) -> + check_logger_config(kernel,Env); check_logger_config(_,Bad) -> throw(Bad). @@ -784,6 +969,13 @@ get_primary_filters(Env) -> _ -> throw({multiple_filters,Env}) end. +get_proxy_opts(Env) -> + case [P || P={proxy,_} <- Env] of + [{proxy,Opts}] -> Opts; + [] -> undefined; + _ -> throw({multiple_proxies,Env}) + end. + %% This function looks at the kernel logger environment %% and updates it so that the correct logger is configured init_default_config(Type,Env) when Type==standard_io; @@ -880,30 +1072,30 @@ log_allowed(Location,Level,Msg,Meta0) when is_map(Meta0) -> maps:merge(Location,maps:merge(proc_meta(),Meta0))), case node(maps:get(gl,Meta)) of Node when Node=/=node() -> - log_remote(Node,Level,Msg,Meta), - do_log_allowed(Level,Msg,Meta); + log_remote(Node,Level,Msg,Meta); _ -> - do_log_allowed(Level,Msg,Meta) - end. + ok + end, + do_log_allowed(Level,Msg,Meta,tid()). -do_log_allowed(Level,{Format,Args}=Msg,Meta) +do_log_allowed(Level,{Format,Args}=Msg,Meta,Tid) when ?IS_LEVEL(Level), is_list(Format), is_list(Args), is_map(Meta) -> - logger_backend:log_allowed(#{level=>Level,msg=>Msg,meta=>Meta},tid()); -do_log_allowed(Level,Report,Meta) + logger_backend:log_allowed(#{level=>Level,msg=>Msg,meta=>Meta},Tid); +do_log_allowed(Level,Report,Meta,Tid) when ?IS_LEVEL(Level), ?IS_REPORT(Report), is_map(Meta) -> logger_backend:log_allowed(#{level=>Level,msg=>{report,Report},meta=>Meta}, - tid()); -do_log_allowed(Level,String,Meta) + Tid); +do_log_allowed(Level,String,Meta,Tid) when ?IS_LEVEL(Level), ?IS_STRING(String), is_map(Meta) -> logger_backend:log_allowed(#{level=>Level,msg=>{string,String},meta=>Meta}, - tid()). + Tid). tid() -> ets:whereis(?LOGGER_TABLE). @@ -913,7 +1105,7 @@ log_remote(Node,Level,Msg,Meta) -> log_remote(Node,{log,Level,Msg,Meta}). log_remote(Node,Request) -> - {logger,Node} ! Request, + logger_proxy:log({remote,Node,Request}), ok. add_default_metadata(Meta) -> diff --git a/lib/kernel/src/logger_config.erl b/lib/kernel/src/logger_config.erl index 5e9faf332c..5024d20cfe 100644 --- a/lib/kernel/src/logger_config.erl +++ b/lib/kernel/src/logger_config.erl @@ -66,6 +66,8 @@ get(Tid,What) -> case ets:lookup(Tid,table_key(What)) of [{_,_,Config}] -> {ok,Config}; + [{_,Config}] when What=:=proxy -> + {ok,Config}; [] -> {error,{not_found,What}} end. @@ -79,10 +81,15 @@ get(Tid,What,Level) -> [Data] -> {ok,Data} end. +create(Tid,proxy,Config) -> + ets:insert(Tid,{table_key(proxy),Config}); create(Tid,What,Config) -> LevelInt = level_to_int(maps:get(level,Config)), ets:insert(Tid,{table_key(What),LevelInt,Config}). +set(Tid,proxy,Config) -> + ets:insert(Tid,{table_key(proxy),Config}), + ok; set(Tid,What,Config) -> LevelInt = level_to_int(maps:get(level,Config)), %% Should do this only if the level has actually changed. Possibly @@ -148,5 +155,6 @@ int_to_level(?LOG_ALL) -> all. %%%----------------------------------------------------------------- %%% Internal +table_key(proxy) -> ?PROXY_KEY; table_key(primary) -> ?PRIMARY_KEY; table_key(HandlerId) -> {?HANDLER_KEY,HandlerId}. diff --git a/lib/kernel/src/logger_disk_log_h.erl b/lib/kernel/src/logger_disk_log_h.erl index 41e0d51a9d..47b39da900 100644 --- a/lib/kernel/src/logger_disk_log_h.erl +++ b/lib/kernel/src/logger_disk_log_h.erl @@ -24,7 +24,7 @@ -include("logger_h_common.hrl"). %%% API --export([info/1, filesync/1, reset/1]). +-export([filesync/1]). %% logger_h_common callbacks -export([init/2, check_config/4, reset_state/2, @@ -47,25 +47,6 @@ filesync(Name) -> logger_h_common:filesync(?MODULE,Name). -%%%----------------------------------------------------------------- -%%% --spec info(Name) -> Info | {error,Reason} when - Name :: atom(), - Info :: term(), - Reason :: handler_busy | {badarg,term()}. - -info(Name) -> - logger_h_common:info(?MODULE,Name). - -%%%----------------------------------------------------------------- -%%% --spec reset(Name) -> ok | {error,Reason} when - Name :: atom(), - Reason :: handler_busy | {badarg,term()}. - -reset(Name) -> - logger_h_common:reset(?MODULE,Name). - %%%=================================================================== %%% logger callbacks %%%=================================================================== diff --git a/lib/kernel/src/logger_h_common.erl b/lib/kernel/src/logger_h_common.erl index 74a2d158fc..e69f6de38d 100644 --- a/lib/kernel/src/logger_h_common.erl +++ b/lib/kernel/src/logger_h_common.erl @@ -24,11 +24,11 @@ -include("logger_internal.hrl"). %% API --export([start_link/1, info/2, filesync/2, reset/2]). +-export([filesync/2]). -%% gen_server and proc_lib callbacks --export([init/1, handle_call/3, handle_cast/2, handle_info/2, - terminate/2, code_change/3]). +%% logger_olp callbacks +-export([init/1, handle_load/2, handle_call/3, handle_cast/2, handle_info/2, + terminate/2, code_change/3, notify/2, reset_state/1]). %% logger callbacks -export([log/2, adding_handler/1, removing_handler/1, changing_config/3, @@ -37,52 +37,45 @@ %% Library functions for handlers -export([error_notify/1]). -%%%----------------------------------------------------------------- --define(CONFIG_KEYS,[sync_mode_qlen, - drop_mode_qlen, - flush_qlen, - burst_limit_enable, - burst_limit_max_count, - burst_limit_window_time, - overload_kill_enable, - overload_kill_qlen, - overload_kill_mem_size, - overload_kill_restart_after, - filesync_repeat_interval]). --define(READ_ONLY_KEYS,[handler_pid,mode_tab]). +-define(OLP_KEYS,[sync_mode_qlen, + drop_mode_qlen, + flush_qlen, + burst_limit_enable, + burst_limit_max_count, + burst_limit_window_time, + overload_kill_enable, + overload_kill_qlen, + overload_kill_mem_size, + overload_kill_restart_after]). + +-define(COMMON_KEYS,[filesync_repeat_interval]). + +-define(READ_ONLY_KEYS,[olp]). %%%----------------------------------------------------------------- %%% API %% This function is called by the logger_sup supervisor -start_link(Args) -> - proc_lib:start_link(?MODULE,init,[Args]). - filesync(Module, Name) -> call(Module, Name, filesync). -info(Module, Name) -> - call(Module, Name, info). - -reset(Module, Name) -> - call(Module, Name, reset). - %%%----------------------------------------------------------------- %%% Handler being added adding_handler(#{id:=Name,module:=Module}=Config) -> HConfig0 = maps:get(config, Config, #{}), - HandlerConfig0 = maps:without(?CONFIG_KEYS,HConfig0), + HandlerConfig0 = maps:without(?OLP_KEYS++?COMMON_KEYS,HConfig0), case Module:check_config(Name,set,undefined,HandlerConfig0) of {ok,HandlerConfig} -> - ModifiedCommon = maps:with(?CONFIG_KEYS,HandlerConfig), - CommonConfig0 = maps:with(?CONFIG_KEYS,HConfig0), + ModifiedCommon = maps:with(?COMMON_KEYS,HandlerConfig), + CommonConfig0 = maps:with(?COMMON_KEYS,HConfig0), CommonConfig = maps:merge( maps:merge(get_default_config(), CommonConfig0), ModifiedCommon), case check_config(CommonConfig) of ok -> HConfig = maps:merge(CommonConfig,HandlerConfig), - start(Config#{config => HConfig}); + OlpOpts = maps:with(?OLP_KEYS,HConfig0), + start(OlpOpts, Config#{config => HConfig}); {error,Faulty} -> {error,{invalid_config,Module,Faulty}} end; @@ -92,11 +85,11 @@ adding_handler(#{id:=Name,module:=Module}=Config) -> %%%----------------------------------------------------------------- %%% Handler being removed -removing_handler(#{id:=Name, module:=Module}) -> +removing_handler(#{id:=Name, module:=Module, config:=#{olp:=Olp}}) -> case whereis(?name_to_reg_name(Module,Name)) of undefined -> ok; - Pid -> + _Pid -> %% We don't want to do supervisor:terminate_child here %% since we need to distinguish this explicit stop from a %% system termination in order to avoid circular attempts @@ -106,7 +99,7 @@ removing_handler(#{id:=Name, module:=Module}) -> %% the restart type is temporary, which means that the %% child specification is automatically removed from the %% supervisor when the process dies. - _ = gen_server:call(Pid, stop), + _ = logger_olp:stop(Olp), ok end. @@ -116,34 +109,52 @@ changing_config(SetOrUpdate, #{id:=Name,config:=OldHConfig,module:=Module}, NewConfig0) -> NewHConfig0 = maps:get(config, NewConfig0, #{}), - OldHandlerConfig = maps:without(?CONFIG_KEYS++?READ_ONLY_KEYS,OldHConfig), - NewHandlerConfig0 = maps:without(?CONFIG_KEYS++?READ_ONLY_KEYS,NewHConfig0), + NoHandlerKeys = ?OLP_KEYS++?COMMON_KEYS++?READ_ONLY_KEYS, + OldHandlerConfig = maps:without(NoHandlerKeys,OldHConfig), + NewHandlerConfig0 = maps:without(NoHandlerKeys,NewHConfig0), case Module:check_config(Name, SetOrUpdate, OldHandlerConfig,NewHandlerConfig0) of {ok, NewHandlerConfig} -> - ModifiedCommon = maps:with(?CONFIG_KEYS,NewHandlerConfig), - NewCommonConfig0 = maps:with(?CONFIG_KEYS,NewHConfig0), + ModifiedCommon = maps:with(?COMMON_KEYS,NewHandlerConfig), + NewCommonConfig0 = maps:with(?COMMON_KEYS,NewHConfig0), + OldCommonConfig = maps:with(?COMMON_KEYS,OldHConfig), CommonDefault = case SetOrUpdate of set -> get_default_config(); update -> - maps:with(?CONFIG_KEYS,OldHConfig) + OldCommonConfig end, NewCommonConfig = maps:merge( maps:merge(CommonDefault,NewCommonConfig0), ModifiedCommon), case check_config(NewCommonConfig) of ok -> - ReadOnly = maps:with(?READ_ONLY_KEYS,OldHConfig), - NewHConfig = maps:merge( - maps:merge(NewCommonConfig,NewHandlerConfig), - ReadOnly), - NewConfig = NewConfig0#{config=>NewHConfig}, - HPid = maps:get(handler_pid,OldHConfig), - case call(HPid, {change_config,NewConfig}) of - ok -> {ok,NewConfig}; - Error -> Error + OlpDefault = + case SetOrUpdate of + set -> + logger_olp:get_default_opts(); + update -> + maps:with(?OLP_KEYS,OldHConfig) + end, + Olp = maps:get(olp,OldHConfig), + NewOlpOpts = maps:merge(OlpDefault, + maps:with(?OLP_KEYS,NewHConfig0)), + case logger_olp:set_opts(Olp,NewOlpOpts) of + ok -> + maybe_set_repeated_filesync(Olp,OldCommonConfig, + NewCommonConfig), + ReadOnly = maps:with(?READ_ONLY_KEYS,OldHConfig), + NewHConfig = + maps:merge( + maps:merge( + maps:merge(NewCommonConfig,NewHandlerConfig), + ReadOnly), + NewOlpOpts), + NewConfig = NewConfig0#{config=>NewHConfig}, + {ok,NewConfig}; + Error -> + Error end; {error,Faulty} -> {error,{invalid_config,Module,Faulty}} @@ -158,14 +169,12 @@ changing_config(SetOrUpdate, LogEvent :: logger:log_event(), Config :: logger:handler_config(). -log(LogEvent, Config = #{id := Name, - config := #{handler_pid := HPid, - mode_tab := ModeTab}}) -> +log(LogEvent, Config = #{config := #{olp:=Olp}}) -> %% if the handler has crashed, we must drop this event %% and hope the handler restarts so we can try again - true = is_process_alive(HPid), + true = is_process_alive(logger_olp:get_pid(Olp)), Bin = log_to_binary(LogEvent, Config), - call_cast_or_drop(Name, HPid, ModeTab, Bin). + logger_olp:load(Olp,Bin). %%%----------------------------------------------------------------- %%% Remove internal fields from configuration @@ -180,18 +189,23 @@ filter_config(#{config:=HConfig}=Config) -> %%% %%% The handler process is linked to logger_sup, which is part of the %%% kernel application's supervision tree. -start(#{id := Name} = Config0) -> +start(OlpOpts0, #{id := Name, module:=Module, config:=HConfig} = Config0) -> + RegName = ?name_to_reg_name(Module,Name), ChildSpec = #{id => Name, - start => {?MODULE, start_link, [Config0]}, + start => {logger_olp, start_link, [RegName,?MODULE, + Config0, OlpOpts0]}, restart => temporary, shutdown => 2000, type => worker, modules => [?MODULE]}, case supervisor:start_child(logger_sup, ChildSpec) of - {ok,Pid,Config} -> + {ok,Pid,Olp} -> ok = logger_handler_watcher:register_handler(Name,Pid), - {ok,Config}; + OlpOpts = logger_olp:get_opts(Olp), + {ok,Config0#{config=>(maps:merge(HConfig,OlpOpts))#{olp=>Olp}}}; + {error,{Reason,Ch}} when is_tuple(Ch), element(1,Ch)==child -> + {error,Reason}; Error -> Error end. @@ -200,103 +214,50 @@ start(#{id := Name} = Config0) -> %%% gen_server callbacks %%%=================================================================== -init(#{id := Name, module := Module, - formatter := Formatter, config := HConfig0} = Config0) -> - RegName = ?name_to_reg_name(Module,Name), - register(RegName, self()), +init(#{id := Name, module := Module, config := HConfig}) -> process_flag(trap_exit, true), - process_flag(message_queue_data, off_heap), ?init_test_hooks(), - ?start_observation(Name), - case Module:init(Name, HConfig0) of + case Module:init(Name, HConfig) of {ok,HState} -> - try ets:new(Name, [public]) of - ModeTab -> - ?set_mode(ModeTab, async), - T0 = ?timestamp(), - HConfig = HConfig0#{handler_pid => self(), - mode_tab => ModeTab}, - Config = Config0#{config => HConfig}, - proc_lib:init_ack({ok,self(),Config}), - %% Storing common config in state to avoid copying - %% (sending) the config data for each log message - CommonConfig = maps:with(?CONFIG_KEYS,HConfig), - State = - ?merge_with_stats( - CommonConfig#{id => Name, - module => Module, - mode_tab => ModeTab, - mode => async, - ctrl_sync_count => - ?CONTROLLER_SYNC_INTERVAL, - last_qlen => 0, - last_log_ts => T0, - last_op => sync, - burst_win_ts => T0, - burst_msg_count => 0, - formatter => Formatter, - handler_state => HState}), - State1 = set_repeated_filesync(State), - unset_restart_flag(State1), - gen_server:enter_loop(?MODULE, [], State1) - catch - _:Error -> - unregister(RegName), - error_notify({init_handler,Name,Error}), - proc_lib:init_ack(Error) - end; + %% Storing common config in state to avoid copying + %% (sending) the config data for each log message + CommonConfig = maps:with(?COMMON_KEYS,HConfig), + State = CommonConfig#{id => Name, + module => Module, + ctrl_sync_count => + ?CONTROLLER_SYNC_INTERVAL, + last_op => sync, + handler_state => HState}, + State1 = set_repeated_filesync(State), + {ok,State1}; Error -> - unregister(RegName), - error_notify({init_handler,Name,Error}), - proc_lib:init_ack(Error) + Error end. -%% This is the synchronous log event. -handle_call({log, Bin}, _From, State) -> - {Result,State1} = do_log(Bin, call, State), - %% Result == ok | dropped - {reply,Result, State1}; +%% This is the log event. +handle_load(Bin, #{id:=Name, + module:=Module, + handler_state:=HandlerState, + ctrl_sync_count := CtrlSync}=State) -> + if CtrlSync==0 -> + {_,HS1} = Module:write(Name, sync, Bin, HandlerState), + State#{handler_state => HS1, + ctrl_sync_count => ?CONTROLLER_SYNC_INTERVAL, + last_op=>write}; + true -> + {_,HS1} = Module:write(Name, async, Bin, HandlerState), + State#{handler_state => HS1, + ctrl_sync_count => CtrlSync-1, + last_op=>write} + end. handle_call(filesync, _From, State = #{id := Name, module := Module, handler_state := HandlerState}) -> {Result,HandlerState1} = Module:filesync(Name,sync,HandlerState), - {reply, Result, State#{handler_state=>HandlerState1, last_op=>sync}}; - -handle_call({change_config, #{formatter:=Formatter, config:=NewHConfig}}, _From, - State = #{filesync_repeat_interval := FSyncInt0}) -> - %% In the future, if handler_state must be updated due to config - %% change, then we need to add a callback to Module here. - CommonConfig = maps:with(?CONFIG_KEYS,NewHConfig), - State1 = maps:merge(State, CommonConfig), - State2 = - case maps:get(filesync_repeat_interval, NewHConfig) of - FSyncInt0 -> - State1; - _FSyncInt1 -> - set_repeated_filesync(cancel_repeated_filesync(State1)) - end, - {reply, ok, State2#{formatter:=Formatter}}; - -handle_call(info, _From, State) -> - {reply, State, State}; - -handle_call(reset, _From, - #{id:=Name,module:=Module,handler_state:=HandlerState}=State) -> - State1 = ?merge_with_stats(State), - {reply, ok, State1#{last_qlen => 0, - last_log_ts => ?timestamp(), - handler_state => Module:reset_state(Name,HandlerState)}}; - -handle_call(stop, _From, State) -> - {stop, {shutdown,stopped}, ok, State}. - -%% This is the asynchronous log event. -handle_cast({log, Bin}, State) -> - {_,State1} = do_log(Bin, cast, State), - {noreply, State1}; + {reply, Result, State#{handler_state=>HandlerState1, last_op=>sync}}. %% If FILESYNC_REPEAT_INTERVAL is set to a millisec value, this %% clause gets called repeatedly by the handler. In order to @@ -319,168 +280,83 @@ handle_cast(repeated_filesync, {_,HS} = Module:filesync(Name, async, HandlerState), State#{handler_state => HS, last_op => sync} end, - {noreply,set_repeated_filesync(State1)}. + {noreply,set_repeated_filesync(State1)}; + +handle_cast({set_repeated_filesync,FSyncInt},State) -> + State1 = State#{filesync_repeat_interval=>FSyncInt}, + State2 = set_repeated_filesync(cancel_repeated_filesync(State1)), + {noreply, State2}. handle_info(Info, #{id := Name, module := Module, handler_state := HandlerState} = State) -> {noreply,State#{handler_state => Module:handle_info(Name,Info,HandlerState)}}. -terminate(Reason, State = #{id := Name, - module := Module, - handler_state := HandlerState}) -> +terminate(overloaded=Reason, #{id:=Name}=State) -> + _ = log_handler_info(Name,"Handler ~p overloaded and stopping",[Name],State), + do_terminate(Reason,State), + ConfigResult = logger:get_handler_config(Name), + case ConfigResult of + {ok,#{module:=Module}=HConfig0} -> + spawn(fun() -> logger:remove_handler(Name) end), + HConfig = try Module:filter_config(HConfig0) + catch _:_ -> HConfig0 + end, + {ok,fun() -> logger:add_handler(Name,Module,HConfig) end}; + Error -> + error_notify({Name,restart_impossible,Error}), + Error + end; +terminate(Reason, State) -> + do_terminate(Reason, State). + +do_terminate(Reason, State = #{id := Name, + module := Module, + handler_state := HandlerState}) -> _ = cancel_repeated_filesync(State), _ = Module:terminate(Name, Reason, HandlerState), - ok = stop_or_restart(Name, Reason, State), - unregister(?name_to_reg_name(Module, Name)), ok. code_change(_OldVsn, State, _Extra) -> {ok, State}. +reset_state(#{id:=Name, module:=Module, handler_state:=HandlerState} = State) -> + State#{handler_state=>Module:reset_state(Name, HandlerState)}. %%%----------------------------------------------------------------- %%% Internal functions call(Module, Name, Op) when is_atom(Name) -> - call(?name_to_reg_name(Module,Name), Op); + case logger_olp:call(?name_to_reg_name(Module,Name), Op) of + {error,busy} -> {error,handler_busy}; + Other -> Other + end; call(_, Name, Op) -> {error,{badarg,{Op,[Name]}}}. -call(Server, Msg) -> - try - gen_server:call(Server, Msg, ?DEFAULT_CALL_TIMEOUT) - catch - _:{timeout,_} -> {error,handler_busy} - end. - -%% check for overload between every event (and set Mode to async, -%% sync or drop accordingly), but never flush the whole mailbox -%% before LogWindowSize events have been handled -do_log(Bin, CallOrCast, State = #{id:=Name, mode:=Mode0}) -> - T1 = ?timestamp(), - - %% check if the handler is getting overloaded, or if it's - %% recovering from overload (the check must be done for each - %% event to react quickly to large bursts of events and - %% to ensure that the handler can never end up in drop mode - %% with an empty mailbox, which would stop operation) - {Mode1,QLen,Mem,State1} = check_load(State), - - if (Mode1 == drop) andalso (Mode0 =/= drop) -> - log_handler_info(Name, "Handler ~p switched to drop mode", - [Name], State); - (Mode0 == drop) andalso ((Mode1 == async) orelse (Mode1 == sync)) -> - log_handler_info(Name, "Handler ~p switched to ~w mode", - [Name,Mode1], State); - true -> - ok - end, - - %% kill the handler if it can't keep up with the load - kill_if_choked(Name, QLen, Mem, State), - - if Mode1 == flush -> - flush(Name, QLen, T1, State1); - true -> - write(Name, Mode1, T1, Bin, CallOrCast, State1) - end. - -%% this clause is called by do_log/3 after an overload check -%% has been performed, where QLen > FlushQLen -flush(Name, _QLen0, T1, State=#{last_log_ts := _T0, mode_tab := ModeTab}) -> - %% flush messages in the mailbox (a limited number in - %% order to not cause long delays) - NewFlushed = flush_log_events(?FLUSH_MAX_N), - - %% write info in log about flushed messages +notify({mode_change,Mode0,Mode1},#{id:=Name}=State) -> + log_handler_info(Name,"Handler ~p switched from ~p to ~p mode", + [Name,Mode0,Mode1], State); +notify({flushed,Flushed},#{id:=Name}=State) -> log_handler_info(Name, "Handler ~p flushed ~w log events", - [Name,NewFlushed], State), - - %% because of the receive loop when flushing messages, the - %% handler will be scheduled out often and the mailbox could - %% grow very large, so we'd better check the queue again here - {_,_QLen1} = process_info(self(), message_queue_len), - ?observe(Name,{max_qlen,_QLen1}), - - %% Add 1 for the current log event - ?observe(Name,{flushed,NewFlushed+1}), - - State1 = ?update_max_time(?diff_time(T1,_T0),State), - State2 = ?update_max_qlen(_QLen1,State1), - {dropped,?update_other(flushed,FLUSHED,NewFlushed, - State2#{mode => ?set_mode(ModeTab,async), - last_qlen => 0, - last_log_ts => T1})}. - -%% this clause is called to write to file -write(Name, Mode, T1, Bin, _CallOrCast, - State = #{module := Module, - handler_state := HandlerState, - mode_tab := ModeTab, - ctrl_sync_count := CtrlSync, - last_qlen := LastQLen, - last_log_ts := T0}) -> - %% check if we need to limit the number of writes - %% during a burst of log events - {DoWrite,State1} = limit_burst(State), - - %% only log synhrounously every ?CONTROLLER_SYNC_INTERVAL time, to - %% give the handler time between writes so it can keep up with - %% incoming messages - {Result,LastQLen1,HandlerState1} = - if DoWrite, CtrlSync == 0 -> - ?observe(Name,{_CallOrCast,1}), - {_,HS1} = Module:write(Name, sync, Bin, HandlerState), - {ok,element(2, process_info(self(), message_queue_len)),HS1}; - DoWrite -> - ?observe(Name,{_CallOrCast,1}), - {_,HS1} = Module:write(Name, async, Bin, HandlerState), - {ok,LastQLen,HS1}; - not DoWrite -> - ?observe(Name,{flushed,1}), - {dropped,LastQLen,HandlerState} - end, - - %% Check if the time since the previous log event is long enough - - %% and the queue length small enough - to assume the mailbox has - %% been emptied, and if so, do filesync operation and reset mode to - %% async. Note that this is the best we can do to detect an idle - %% handler without setting a timer after each log call/cast. If the - %% time between two consecutive log events is fast and no new - %% event comes in after the last one, idle state won't be detected! - Time = ?diff_time(T1,T0), - State2 = - if (LastQLen1 < ?FILESYNC_OK_QLEN) andalso - (Time > ?IDLE_DETECT_TIME_USEC) -> - {_,HS2} = Module:filesync(Name,async,HandlerState), - State1#{mode => ?change_mode(ModeTab, Mode, async), - burst_msg_count => 0, - handler_state => HS2}; - true -> - State1#{mode => Mode, handler_state => HandlerState1} - end, - State3 = ?update_calls_or_casts(_CallOrCast,1,State2), - State4 = ?update_max_qlen(LastQLen1,State3), - State5 = - ?update_max_time(Time, - State4#{last_qlen := LastQLen1, - last_log_ts => T1, - last_op => write, - ctrl_sync_count => - if CtrlSync==0 -> ?CONTROLLER_SYNC_INTERVAL; - true -> CtrlSync-1 - end}), - {Result,State5}. + [Name,Flushed], State); +notify(restart,#{id:=Name}=State) -> + log_handler_info(Name, "Handler ~p restarted", [Name], State); +notify(idle,#{id:=Name,module:=Module,handler_state:=HandlerState}=State) -> + {_,HS} = Module:filesync(Name,async,HandlerState), + State#{handler_state=>HS, last_op=>sync}. log_handler_info(Name, Format, Args, #{module:=Module, - formatter:=Formatter, - handler_state:=HandlerState}) -> - Config = #{formatter=>Formatter}, + handler_state:=HandlerState}=State) -> + Config = + case logger:get_handler_config(Name) of + {ok,Conf} -> Conf; + _ -> #{formatter=>{?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}} + end, Meta = #{time=>erlang:system_time(microsecond)}, Bin = log_to_binary(#{level => notice, msg => {Format,Args}, meta => Meta}, Config), - _ = Module:write(Name, async, Bin, HandlerState), - ok. + {_,HS} = Module:write(Name, async, Bin, HandlerState), + State#{handler_state=>HS, last_op=>write}. %%%----------------------------------------------------------------- %%% Convert log data on any form to binary @@ -540,42 +416,8 @@ string_to_binary(String) -> %%%----------------------------------------------------------------- %%% Check that the configuration term is valid check_config(Config) when is_map(Config) -> - case check_common_config(maps:to_list(Config)) of - ok -> - case overload_levels_ok(Config) of - true -> - ok; - false -> - Faulty = maps:with([sync_mode_qlen, - drop_mode_qlen, - flush_qlen],Config), - {error,{invalid_levels,Faulty}} - end; - Error -> - Error - end. + check_common_config(maps:to_list(Config)). -check_common_config([{sync_mode_qlen,N}|Config]) when is_integer(N) -> - check_common_config(Config); -check_common_config([{drop_mode_qlen,N}|Config]) when is_integer(N) -> - check_common_config(Config); -check_common_config([{flush_qlen,N}|Config]) when is_integer(N) -> - check_common_config(Config); -check_common_config([{burst_limit_enable,Bool}|Config]) when is_boolean(Bool) -> - check_common_config(Config); -check_common_config([{burst_limit_max_count,N}|Config]) when is_integer(N) -> - check_common_config(Config); -check_common_config([{burst_limit_window_time,N}|Config]) when is_integer(N) -> - check_common_config(Config); -check_common_config([{overload_kill_enable,Bool}|Config]) when is_boolean(Bool) -> - check_common_config(Config); -check_common_config([{overload_kill_qlen,N}|Config]) when is_integer(N) -> - check_common_config(Config); -check_common_config([{overload_kill_mem_size,N}|Config]) when is_integer(N) -> - check_common_config(Config); -check_common_config([{overload_kill_restart_after,NorA}|Config]) - when is_integer(NorA); NorA == infinity -> - check_common_config(Config); check_common_config([{filesync_repeat_interval,NorA}|Config]) when is_integer(NorA); NorA == no_repeat -> check_common_config(Config); @@ -585,156 +427,7 @@ check_common_config([]) -> ok. get_default_config() -> - #{sync_mode_qlen => ?SYNC_MODE_QLEN, - drop_mode_qlen => ?DROP_MODE_QLEN, - flush_qlen => ?FLUSH_QLEN, - burst_limit_enable => ?BURST_LIMIT_ENABLE, - burst_limit_max_count => ?BURST_LIMIT_MAX_COUNT, - burst_limit_window_time => ?BURST_LIMIT_WINDOW_TIME, - overload_kill_enable => ?OVERLOAD_KILL_ENABLE, - overload_kill_qlen => ?OVERLOAD_KILL_QLEN, - overload_kill_mem_size => ?OVERLOAD_KILL_MEM_SIZE, - overload_kill_restart_after => ?OVERLOAD_KILL_RESTART_AFTER, - filesync_repeat_interval => ?FILESYNC_REPEAT_INTERVAL}. - -%%%----------------------------------------------------------------- -%%% Overload Protection -call_cast_or_drop(_Name, HandlerPid, ModeTab, Bin) -> - %% If the handler process is getting overloaded, the log event - %% will be synchronous instead of asynchronous (slows down the - %% logging tempo of a process doing lots of logging. If the - %% handler is choked, drop mode is set and no event will be sent. - try ?get_mode(ModeTab) of - async -> - gen_server:cast(HandlerPid, {log,Bin}); - sync -> - case call(HandlerPid, {log,Bin}) of - ok -> - ok; - _Other -> - %% dropped or {error,handler_busy} - ?observe(_Name,{dropped,1}), - ok - end; - drop -> - ?observe(_Name,{dropped,1}) - catch - %% if the ETS table doesn't exist (maybe because of a - %% handler restart), we can only drop the event - _:_ -> ?observe(_Name,{dropped,1}) - end, - ok. - -set_restart_flag(#{id := Name, module := Module} = State) -> - log_handler_info(Name, "Handler ~p overloaded and stopping", [Name], State), - Flag = list_to_atom(lists:concat([Module,"_",Name,"_restarting"])), - spawn(fun() -> - register(Flag, self()), - timer:sleep(infinity) - end), - ok. - -unset_restart_flag(#{id := Name, module := Module} = State) -> - Flag = list_to_atom(lists:concat([Module,"_",Name,"_restarting"])), - case whereis(Flag) of - undefined -> - ok; - Pid -> - exit(Pid, kill), - log_handler_info(Name, "Handler ~p restarted", [Name], State) - end. - -check_load(State = #{id:=_Name, mode_tab := ModeTab, mode := Mode, - sync_mode_qlen := SyncModeQLen, - drop_mode_qlen := DropModeQLen, - flush_qlen := FlushQLen}) -> - {_,Mem} = process_info(self(), memory), - ?observe(_Name,{max_mem,Mem}), - {_,QLen} = process_info(self(), message_queue_len), - ?observe(_Name,{max_qlen,QLen}), - %% When the handler process gets scheduled in, it's impossible - %% to predict the QLen. We could jump "up" arbitrarily from say - %% async to sync, async to drop, sync to flush, etc. However, when - %% the handler process manages the log events (without flushing), - %% one after the other, we will move "down" from drop to sync and - %% from sync to async. This way we don't risk getting stuck in - %% drop or sync mode with an empty mailbox. - {Mode1,_NewDrops,_NewFlushes} = - if - QLen >= FlushQLen -> - {flush, 0,1}; - QLen >= DropModeQLen -> - %% Note that drop mode will force log events to - %% be dropped on the client side (never sent get to - %% the handler). - IncDrops = if Mode == drop -> 0; true -> 1 end, - {?change_mode(ModeTab, Mode, drop), IncDrops,0}; - QLen >= SyncModeQLen -> - {?change_mode(ModeTab, Mode, sync), 0,0}; - true -> - {?change_mode(ModeTab, Mode, async), 0,0} - end, - State1 = ?update_other(drops,DROPS,_NewDrops,State), - {Mode1, QLen, Mem, - ?update_other(flushes,FLUSHES,_NewFlushes, - State1#{last_qlen => QLen})}. - -limit_burst(#{burst_limit_enable := false}=State) -> - {true,State}; -limit_burst(#{burst_win_ts := BurstWinT0, - burst_msg_count := BurstMsgCount, - burst_limit_window_time := BurstLimitWinTime, - burst_limit_max_count := BurstLimitMaxCnt} = State) -> - if (BurstMsgCount >= BurstLimitMaxCnt) -> - %% the limit for allowed messages has been reached - BurstWinT1 = ?timestamp(), - case ?diff_time(BurstWinT1,BurstWinT0) of - BurstCheckTime when BurstCheckTime < (BurstLimitWinTime*1000) -> - %% we're still within the burst time frame - {false,?update_other(burst_drops,BURSTS,1,State)}; - _BurstCheckTime -> - %% burst time frame passed, reset counters - {true,State#{burst_win_ts => BurstWinT1, - burst_msg_count => 0}} - end; - true -> - %% the limit for allowed messages not yet reached - {true,State#{burst_win_ts => BurstWinT0, - burst_msg_count => BurstMsgCount+1}} - end. - -kill_if_choked(Name, QLen, Mem, State = #{overload_kill_enable := KillIfOL, - overload_kill_qlen := OLKillQLen, - overload_kill_mem_size := OLKillMem}) -> - if KillIfOL andalso - ((QLen > OLKillQLen) orelse (Mem > OLKillMem)) -> - set_restart_flag(State), - exit({shutdown,{overloaded,Name,QLen,Mem}}); - true -> - ok - end. - -flush_log_events(Limit) -> - process_flag(priority, high), - Flushed = flush_log_events(0, Limit), - process_flag(priority, normal), - Flushed. - -flush_log_events(Limit, Limit) -> - Limit; -flush_log_events(N, Limit) -> - %% flush log events but leave other events, such as - %% filesync, info and change_config, so that these - %% have a chance to be processed even under heavy load - receive - {'$gen_cast',{log,_}} -> - flush_log_events(N+1, Limit); - {'$gen_call',{Pid,MRef},{log,_}} -> - Pid ! {MRef, dropped}, - flush_log_events(N+1, Limit) - after - 0 -> N - end. + #{filesync_repeat_interval => ?FILESYNC_REPEAT_INTERVAL}. set_repeated_filesync(#{filesync_repeat_interval:=FSyncInt} = State) when is_integer(FSyncInt) -> @@ -752,51 +445,12 @@ cancel_repeated_filesync(State) -> error -> State end. - -stop_or_restart(Name, {shutdown,Reason={overloaded,_Name,_QLen,_Mem}}, - #{overload_kill_restart_after := RestartAfter}) -> - %% If we're terminating because of an overload situation (see - %% kill_if_choked/4), we need to remove the handler and set a - %% restart timer. A separate process must perform this in order to - %% avoid deadlock. - HandlerPid = self(), - ConfigResult = logger:get_handler_config(Name), - RemoveAndRestart = - fun() -> - MRef = erlang:monitor(process, HandlerPid), - receive - {'DOWN',MRef,_,_,_} -> - ok - after 30000 -> - error_notify(Reason), - exit(HandlerPid, kill) - end, - case ConfigResult of - {ok,#{module:=HMod}=HConfig0} when is_integer(RestartAfter) -> - _ = logger:remove_handler(Name), - HConfig = try HMod:filter_config(HConfig0) - catch _:_ -> HConfig0 - end, - _ = timer:apply_after(RestartAfter, logger, add_handler, - [Name,HMod,HConfig]); - {ok,_} -> - _ = logger:remove_handler(Name); - {error,CfgReason} when is_integer(RestartAfter) -> - error_notify({Name,restart_impossible,CfgReason}); - {error,_} -> - ok - end - end, - spawn(RemoveAndRestart), - ok; -stop_or_restart(_Name, _Reason, _State) -> - ok. - -overload_levels_ok(HandlerConfig) -> - SMQL = maps:get(sync_mode_qlen, HandlerConfig, ?SYNC_MODE_QLEN), - DMQL = maps:get(drop_mode_qlen, HandlerConfig, ?DROP_MODE_QLEN), - FQL = maps:get(flush_qlen, HandlerConfig, ?FLUSH_QLEN), - (DMQL > 1) andalso (SMQL =< DMQL) andalso (DMQL =< FQL). - error_notify(Term) -> ?internal_log(error, Term). + +maybe_set_repeated_filesync(_Olp, + #{filesync_repeat_interval:=FSyncInt}, + #{filesync_repeat_interval:=FSyncInt}) -> + ok; +maybe_set_repeated_filesync(Olp,_,#{filesync_repeat_interval:=FSyncInt}) -> + logger_olp:cast(Olp,{set_repeated_filesync,FSyncInt}). diff --git a/lib/kernel/src/logger_h_common.hrl b/lib/kernel/src/logger_h_common.hrl index 261b0a6246..004a61d9d9 100644 --- a/lib/kernel/src/logger_h_common.hrl +++ b/lib/kernel/src/logger_h_common.hrl @@ -1,50 +1,22 @@ - -%%%----------------------------------------------------------------- -%%% Overload protection configuration - -%%! *** NOTE *** -%%! It's important that: -%%! SYNC_MODE_QLEN =< DROP_MODE_QLEN =< FLUSH_QLEN -%%! and that DROP_MODE_QLEN >= 2. -%%! Otherwise the handler could end up in drop mode with no new -%%! log requests to process. This would cause all future requests -%%! to be dropped (no switch to async mode would ever take place). - -%% This specifies the message_queue_len value where the log -%% requests switch from asynchronous casts to synchronous calls. --define(SYNC_MODE_QLEN, 10). -%% Above this message_queue_len, log requests will be dropped, -%% i.e. no log requests get sent to the handler process. --define(DROP_MODE_QLEN, 200). -%% Above this message_queue_len, the handler process will flush -%% its mailbox and only leave this number of messages in it. --define(FLUSH_QLEN, 1000). - -%% Never flush more than this number of messages in one go, -%% or the handler will be unresponsive for seconds (keep this -%% number as large as possible or the mailbox could grow large). --define(FLUSH_MAX_N, 5000). - -%% BURST_LIMIT_MAX_COUNT is the max number of log requests allowed -%% to be written within a BURST_LIMIT_WINDOW_TIME time frame. --define(BURST_LIMIT_ENABLE, true). --define(BURST_LIMIT_MAX_COUNT, 500). --define(BURST_LIMIT_WINDOW_TIME, 1000). - -%% This enables/disables the feature to automatically get the -%% handler terminated if it gets too loaded (and can't keep up). --define(OVERLOAD_KILL_ENABLE, false). -%% If the message_queue_len goes above this size even after -%% flushing has been performed, the handler is terminated. --define(OVERLOAD_KILL_QLEN, 20000). -%% If the memory usage exceeds this level --define(OVERLOAD_KILL_MEM_SIZE, 3000000). - -%% This is the default time that the handler will wait before -%% restarting and accepting new requests. The value 'infinity' -%% disables restarts. --define(OVERLOAD_KILL_RESTART_AFTER, 5000). -%%-define(OVERLOAD_KILL_RESTART_AFTER, infinity). +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2015. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% %% The handler sends asynchronous write requests to the process %% controlling the i/o device, but every once in this interval @@ -65,12 +37,6 @@ -define(FILESYNC_REPEAT_INTERVAL, 5000). %%-define(FILESYNC_REPEAT_INTERVAL, no_repeat). -%% This is the time after last message received that we think/hope -%% that the handler has an empty mailbox (no new log request has -%% come in). --define(IDLE_DETECT_TIME_MSEC, 100). --define(IDLE_DETECT_TIME_USEC, 100000). - %% Default disk log option values -define(DISK_LOG_TYPE, wrap). -define(DISK_LOG_MAX_NO_FILES, 10). @@ -83,43 +49,6 @@ list_to_atom(lists:concat([MODULE,"_",Name]))). %%%----------------------------------------------------------------- -%%% Overload protection macros - --define(timestamp(), erlang:monotonic_time(microsecond)). - --define(get_mode(Tid), - case ets:lookup(Tid, mode) of - [{mode,M}] -> M; - _ -> async - end). - --define(set_mode(Tid, M), - begin ets:insert(Tid, {mode,M}), M end). - --define(change_mode(Tid, M0, M1), - if M0 == M1 -> - M0; - true -> - ets:insert(Tid, {mode,M1}), - M1 - end). - --define(min(X1, X2), - if X2 == undefined -> X1; - X2 < X1 -> X2; - true -> X1 - end). - --define(max(X1, X2), - if - X2 == undefined -> X1; - X2 > X1 -> X2; - true -> X1 - end). - --define(diff_time(OS_T1, OS_T0), OS_T1-OS_T0). - -%%%----------------------------------------------------------------- %%% The test hook macros make it possible to observe and manipulate %%% internal handler functionality. When enabled, these macros will %%% slow down execution and therefore should not be include in code @@ -183,7 +112,6 @@ [{_,ERROR}] -> ERROR catch _:_ -> disk_log:sync(LOG) end). - -define(DEFAULT_CALL_TIMEOUT, 5000). -else. % DEFAULTS! -define(TEST_HOOKS_TAB, undefined). @@ -196,68 +124,4 @@ -define(file_datasync(DEVICE), file:datasync(DEVICE)). -define(disk_log_write(LOG, MODE, DATA), disk_log_write(LOG, MODE, DATA)). -define(disk_log_sync(LOG), disk_log:sync(LOG)). - -define(DEFAULT_CALL_TIMEOUT, 10000). --endif. - -%%%----------------------------------------------------------------- -%%% These macros enable statistics counters in the state of the -%%% handler which is useful for analysing the overload protection -%%% behaviour. These counters should not be included in code to be -%%% officially released (as some counters will grow very large -%%% over time). - -%%-define(SAVE_STATS, true). --ifdef(SAVE_STATS). - -define(merge_with_stats(STATE), - STATE#{flushes => 0, flushed => 0, drops => 0, - burst_drops => 0, casts => 0, calls => 0, - max_qlen => 0, max_time => 0}). - - -define(update_max_qlen(QLEN, STATE), - begin #{max_qlen := QLEN0} = STATE, - STATE#{max_qlen => ?max(QLEN0,QLEN)} end). - - -define(update_calls_or_casts(CALL_OR_CAST, INC, STATE), - case CALL_OR_CAST of - cast -> - #{casts := CASTS0} = STATE, - STATE#{casts => CASTS0+INC}; - call -> - #{calls := CALLS0} = STATE, - STATE#{calls => CALLS0+INC} - end). - - -define(update_max_time(TIME, STATE), - begin #{max_time := TIME0} = STATE, - STATE#{max_time => ?max(TIME0,TIME)} end). - - -define(update_other(OTHER, VAR, INCVAL, STATE), - begin #{OTHER := VAR} = STATE, - STATE#{OTHER => VAR+INCVAL} end). - --else. % DEFAULT! - -define(merge_with_stats(STATE), STATE). - -define(update_max_qlen(_QLEN, STATE), STATE). - -define(update_calls_or_casts(_CALL_OR_CAST, _INC, STATE), STATE). - -define(update_max_time(_TIME, STATE), STATE). - -define(update_other(_OTHER, _VAR, _INCVAL, STATE), STATE). --endif. - -%%%----------------------------------------------------------------- -%%% These macros enable callbacks that make it possible to analyse -%%% the overload protection behaviour from outside the handler -%%% process (including dropped requests on the client side). -%%% An external callback module (?OBSERVER_MOD) is required which -%%% is not part of the kernel application. For this reason, these -%%% callbacks should not be included in code to be officially released. - -%%-define(OBSERVER_MOD, logger_test). --ifdef(OBSERVER_MOD). - -define(start_observation(NAME), ?OBSERVER:start_observation(NAME)). - -define(observe(NAME,EVENT), ?OBSERVER:observe(NAME,EVENT)). - --else. % DEFAULT! - -define(start_observation(_NAME), ok). - -define(observe(_NAME,_EVENT), ok). -endif. -%%! <--- diff --git a/lib/kernel/src/logger_internal.hrl b/lib/kernel/src/logger_internal.hrl index d96a4ac78b..e53922e5d3 100644 --- a/lib/kernel/src/logger_internal.hrl +++ b/lib/kernel/src/logger_internal.hrl @@ -19,6 +19,7 @@ %% -include_lib("kernel/include/logger.hrl"). -define(LOGGER_TABLE,logger). +-define(PROXY_KEY,'$proxy_config$'). -define(PRIMARY_KEY,'$primary_config$'). -define(HANDLER_KEY,'$handler_config$'). -define(LOGGER_META_KEY,'$logger_metadata$'). @@ -40,12 +41,14 @@ -define(DEFAULT_LOGGER_CALL_TIMEOUT, infinity). --define(LOG_INTERNAL(Level,Report), +-define(LOG_INTERNAL(Level,Report),?DO_LOG_INTERNAL(Level,[Report])). +-define(LOG_INTERNAL(Level,Format,Args),?DO_LOG_INTERNAL(Level,[Format,Args])). +-define(DO_LOG_INTERNAL(Level,Data), case logger:allow(Level,?MODULE) of true -> %% Spawn this to avoid deadlocks - _ = spawn(logger,macro_log,[?LOCATION,Level,Report, - logger:add_default_metadata(#{})]), + _ = spawn(logger,macro_log,[?LOCATION,Level|Data]++ + [logger:add_default_metadata(#{})]), ok; false -> ok diff --git a/lib/kernel/src/logger_olp.erl b/lib/kernel/src/logger_olp.erl new file mode 100644 index 0000000000..009280a9c9 --- /dev/null +++ b/lib/kernel/src/logger_olp.erl @@ -0,0 +1,626 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% +-module(logger_olp). +-behaviour(gen_server). + +-include("logger_olp.hrl"). +-include("logger_internal.hrl"). + +%% API +-export([start_link/4, load/2, info/1, reset/1, stop/1, restart/1, + set_opts/2, get_opts/1, get_default_opts/0, get_pid/1, + call/2, cast/2, get_ref/0, get_ref/1]). + +%% gen_server and proc_lib callbacks +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, + terminate/2, code_change/3]). + +-define(OPT_KEYS,[sync_mode_qlen, + drop_mode_qlen, + flush_qlen, + burst_limit_enable, + burst_limit_max_count, + burst_limit_window_time, + overload_kill_enable, + overload_kill_qlen, + overload_kill_mem_size, + overload_kill_restart_after]). + +-export_type([olp_ref/0, options/0]). + +-opaque olp_ref() :: {atom(),pid(),ets:tid()}. + +-type options() :: logger:olp_config(). + +%%%----------------------------------------------------------------- +%%% API + +-spec start_link(Name,Module,Args,Options) -> {ok,Pid,Olp} | {error,Reason} when + Name :: atom(), + Module :: module(), + Args :: term(), + Options :: options(), + Pid :: pid(), + Olp :: olp_ref(), + Reason :: term(). +start_link(Name,Module,Args,Options0) when is_map(Options0) -> + Options = maps:merge(get_default_opts(),Options0), + case check_opts(Options) of + ok -> + proc_lib:start_link(?MODULE,init,[[Name,Module,Args,Options]]); + Error -> + Error + end. + +-spec load(Olp, Msg) -> ok when + Olp :: olp_ref(), + Msg :: term(). +load({_Name,Pid,ModeRef},Msg) -> + %% If the process is getting overloaded, the message will be + %% synchronous instead of asynchronous (slows down the tempo of a + %% process causing much load). If the process is choked, drop mode + %% is set and no message is sent. + try ?get_mode(ModeRef) of + async -> + gen_server:cast(Pid, {'$olp_load',Msg}); + sync -> + case call(Pid, {'$olp_load',Msg}) of + ok -> + ok; + _Other -> + %% dropped or {error,busy} + ?observe(_Name,{dropped,1}), + ok + end; + drop -> + ?observe(_Name,{dropped,1}) + catch + %% if the ETS table doesn't exist (maybe because of a + %% process restart), we can only drop the event + _:_ -> ?observe(_Name,{dropped,1}) + end, + ok. + +-spec info(Olp) -> map() | {error, busy} when + Olp :: atom() | pid() | olp_ref(). +info(Olp) -> + call(Olp, info). + +-spec reset(Olp) -> ok | {error, busy} when + Olp :: atom() | pid() | olp_ref(). +reset(Olp) -> + call(Olp, reset). + +-spec stop(Olp) -> ok when + Olp :: atom() | pid() | olp_ref(). +stop({_Name,Pid,_ModRef}) -> + stop(Pid); +stop(Pid) -> + _ = gen_server:call(Pid, stop), + ok. + +-spec set_opts(Olp, Opts) -> ok | {error,term()} | {error, busy} when + Olp :: atom() | pid() | olp_ref(), + Opts :: options(). +set_opts(Olp, Opts) -> + call(Olp, {set_opts,Opts}). + +-spec get_opts(Olp) -> options() | {error, busy} when + Olp :: atom() | pid() | olp_ref(). +get_opts(Olp) -> + call(Olp, get_opts). + +-spec get_default_opts() -> options(). +get_default_opts() -> + #{sync_mode_qlen => ?SYNC_MODE_QLEN, + drop_mode_qlen => ?DROP_MODE_QLEN, + flush_qlen => ?FLUSH_QLEN, + burst_limit_enable => ?BURST_LIMIT_ENABLE, + burst_limit_max_count => ?BURST_LIMIT_MAX_COUNT, + burst_limit_window_time => ?BURST_LIMIT_WINDOW_TIME, + overload_kill_enable => ?OVERLOAD_KILL_ENABLE, + overload_kill_qlen => ?OVERLOAD_KILL_QLEN, + overload_kill_mem_size => ?OVERLOAD_KILL_MEM_SIZE, + overload_kill_restart_after => ?OVERLOAD_KILL_RESTART_AFTER}. + +-spec restart(fun(() -> any())) -> ok. +restart(Fun) -> + Result = + try Fun() + catch C:R:S -> + {error,{restart_failed,Fun,C,R,S}} + end, + ?LOG_INTERNAL(debug,[{logger_olp,restart}, + {result,Result}]), + ok. + +-spec get_ref() -> olp_ref(). +get_ref() -> + get(olp_ref). + +-spec get_ref(PidOrName) -> olp_ref() | {error, busy} when + PidOrName :: pid() | atom(). +get_ref(PidOrName) -> + call(PidOrName,get_ref). + +-spec get_pid(olp_ref()) -> pid(). +get_pid({_Name,Pid,_ModeRef}) -> + Pid. + +%%%=================================================================== +%%% gen_server callbacks +%%%=================================================================== + +init([Name,Module,Args,Options]) -> + register(Name, self()), + process_flag(message_queue_data, off_heap), + + ?start_observation(Name), + + try ets:new(Name, [public]) of + ModeRef -> + OlpRef = {Name,self(),ModeRef}, + put(olp_ref,OlpRef), + try Module:init(Args) of + {ok,CBState} -> + ?set_mode(ModeRef, async), + T0 = ?timestamp(), + proc_lib:init_ack({ok,self(),OlpRef}), + %% Storing options in state to avoid copying + %% (sending) the option data with each message + State0 = ?merge_with_stats( + Options#{id => Name, + idle=> true, + module => Module, + mode_ref => ModeRef, + mode => async, + last_qlen => 0, + last_load_ts => T0, + burst_win_ts => T0, + burst_msg_count => 0, + cb_state => CBState}), + State = reset_restart_flag(State0), + gen_server:enter_loop(?MODULE, [], State); + Error -> + _ = ets:delete(ModeRef), + unregister(Name), + proc_lib:init_ack(Error) + catch + _:Error -> + _ = ets:delete(ModeRef), + unregister(Name), + proc_lib:init_ack(Error) + end + catch + _:Error -> + unregister(Name), + proc_lib:init_ack(Error) + end. + +%% This is the synchronous load event. +handle_call({'$olp_load', Msg}, _From, State) -> + {Result,State1} = do_load(Msg, call, State#{idle=>false}), + %% Result == ok | dropped + reply_return(Result,State1); + +handle_call(get_ref,_From,#{id:=Name,mode_ref:=ModeRef}=State) -> + reply_return({Name,self(),ModeRef},State); + +handle_call({set_opts,Opts0},_From,State) -> + Opts = maps:merge(maps:with(?OPT_KEYS,State),Opts0), + case check_opts(Opts) of + ok -> + reply_return(ok, maps:merge(State,Opts)); + Error -> + reply_return(Error, State) + end; + +handle_call(get_opts,_From,State) -> + reply_return(maps:with(?OPT_KEYS,State), State); + +handle_call(info, _From, State) -> + reply_return(State, State); + +handle_call(reset, _From, #{module:=Module,cb_state:=CBState}=State) -> + State1 = ?merge_with_stats(State), + CBState1 = try_callback_call(Module,reset_state,[CBState],CBState), + reply_return(ok, State1#{idle => true, + last_qlen => 0, + last_load_ts => ?timestamp(), + cb_state => CBState1}); + +handle_call(stop, _From, State) -> + {stop, {shutdown,stopped}, ok, State}; + +handle_call(Msg, From, #{module:=Module,cb_state:=CBState}=State) -> + case try_callback_call(Module,handle_call,[Msg, From, CBState]) of + {reply,Reply,CBState1} -> + reply_return(Reply,State#{cb_state=>CBState1}); + {noreply,CBState1} -> + noreply_return(State#{cb_state=>CBState1}); + {stop, Reason, Reply, CBState1} -> + {stop, Reason, Reply, State#{cb_state=>CBState1}}; + {stop, Reason, CBState1} -> + {stop, Reason, State#{cb_state=>CBState1}} + end. + +%% This is the asynchronous load event. +handle_cast({'$olp_load', Msg}, State) -> + {_Result,State1} = do_load(Msg, cast, State#{idle=>false}), + noreply_return(State1); + +handle_cast(Msg, #{module:=Module, cb_state:=CBState} = State) -> + case try_callback_call(Module,handle_cast,[Msg, CBState]) of + {noreply,CBState1} -> + noreply_return(State#{cb_state=>CBState1}); + {stop, Reason, CBState1} -> + {stop, Reason, State#{cb_state=>CBState1}} + end. + +handle_info(timeout, #{mode_ref:=_ModeRef, mode:=Mode} = State) -> + State1 = notify(idle,State), + State2 = maybe_notify_mode_change(async,State1), + {noreply, State2#{idle => true, + mode => ?change_mode(_ModeRef, Mode, async), + burst_msg_count => 0}}; +handle_info(Msg, #{module := Module, cb_state := CBState} = State) -> + case try_callback_call(Module,handle_info,[Msg, CBState]) of + {noreply,CBState1} -> + noreply_return(State#{cb_state=>CBState1}); + {stop, Reason, CBState1} -> + {stop, Reason, State#{cb_state=>CBState1}}; + {load,CBState1} -> + {_,State1} = do_load(Msg, cast, State#{idle=>false, + cb_state=>CBState1}), + noreply_return(State1) + end. + +terminate({shutdown,{overloaded,_QLen,_Mem}}, + #{id:=Name, module := Module, cb_state := CBState, + overload_kill_restart_after := RestartAfter} = State) -> + %% We're terminating because of an overload situation (see + %% kill_if_choked/3). + unregister(Name), %%!!!! to avoid error printout of callback crashed on stop + case try_callback_call(Module,terminate,[overloaded,CBState],ok) of + {ok,Fun} when is_function(Fun,0), is_integer(RestartAfter) -> + set_restart_flag(State), + _ = timer:apply_after(RestartAfter,?MODULE,restart,[Fun]), + ok; + _ -> + ok + end; +terminate(Reason, #{id:=Name, module:=Module, cb_state:=CBState}) -> + _ = try_callback_call(Module,terminate,[Reason,CBState],ok), + unregister(Name), + ok. + +code_change(_OldVsn, State, _Extra) -> + {ok, State}. + + +%%%----------------------------------------------------------------- +%%% Internal functions +-spec call(Olp, term()) -> term() | {error,busy} when + Olp :: atom() | pid() | olp_ref(). +call({_Name, Pid, _ModeRef},Msg) -> + call(Pid, Msg); +call(Server, Msg) -> + try + gen_server:call(Server, Msg) + catch + _:{timeout,_} -> {error,busy} + end. + +-spec cast(olp_ref(),term()) -> ok. +cast({_Name, Pid, _ModeRef},Msg) -> + gen_server:cast(Pid, Msg). + +%% check for overload between every event (and set Mode to async, +%% sync or drop accordingly), but never flush the whole mailbox +%% before LogWindowSize events have been handled +do_load(Msg, CallOrCast, State) -> + T1 = ?timestamp(), + State1 = ?update_time(T1,State), + + %% check if the process is getting overloaded, or if it's + %% recovering from overload (the check must be done for each + %% event to react quickly to large bursts of events and + %% to ensure that the handler can never end up in drop mode + %% with an empty mailbox, which would stop operation) + {Mode1,QLen,Mem,State2} = check_load(State1), + + %% kill the handler if it can't keep up with the load + kill_if_choked(QLen, Mem, State2), + + if Mode1 == flush -> + flush(T1, State2); + true -> + handle_load(Mode1, T1, Msg, CallOrCast, State2) + end. + +%% this function is called by do_load/3 after an overload check +%% has been performed, where QLen > FlushQLen +flush(T1, State=#{id := _Name, mode := Mode, last_load_ts := _T0, mode_ref := ModeRef}) -> + %% flush load messages in the mailbox (a limited number in order + %% to not cause long delays) + NewFlushed = flush_load(?FLUSH_MAX_N), + + %% write info in log about flushed messages + State1=notify({flushed,NewFlushed},State), + + %% because of the receive loop when flushing messages, the + %% handler will be scheduled out often and the mailbox could + %% grow very large, so we'd better check the queue again here + {_,QLen1} = process_info(self(), message_queue_len), + ?observe(_Name,{max_qlen,QLen1}), + + %% Add 1 for the current log event + ?observe(_Name,{flushed,NewFlushed+1}), + + State2 = ?update_max_time(?diff_time(T1,_T0),State1), + State3 = ?update_max_qlen(QLen1,State2), + State4 = maybe_notify_mode_change(async,State3), + {dropped,?update_other(flushed,FLUSHED,NewFlushed, + State4#{mode => ?change_mode(ModeRef,Mode,async), + last_qlen => QLen1, + last_load_ts => T1})}. + +%% this function is called to actually handle the message +handle_load(Mode, T1, Msg, _CallOrCast, + State = #{id := _Name, + module := Module, + cb_state := CBState, + last_qlen := LastQLen, + last_load_ts := _T0}) -> + %% check if we need to limit the number of writes + %% during a burst of log events + {DoWrite,State1} = limit_burst(State), + + {Result,LastQLen1,CBState1} = + if DoWrite -> + ?observe(_Name,{_CallOrCast,1}), + CBS = try_callback_call(Module,handle_load,[Msg,CBState]), + {ok,element(2, process_info(self(), message_queue_len)),CBS}; + true -> + ?observe(_Name,{flushed,1}), + {dropped,LastQLen,CBState} + end, + State2 = State1#{cb_state=>CBState1}, + + State3 = State2#{mode => Mode}, + State4 = ?update_calls_or_casts(_CallOrCast,1,State3), + State5 = ?update_max_qlen(LastQLen1,State4), + State6 = + ?update_max_time(?diff_time(T1,_T0), + State5#{last_qlen := LastQLen1, + last_load_ts => T1}), + State7 = case Result of + ok -> + S = ?update_freq(T1,State6), + ?update_other(writes,WRITES,1,S); + _ -> + State6 + end, + {Result,State7}. + + +%%%----------------------------------------------------------------- +%%% Check that the options are valid +check_opts(Options) when is_map(Options) -> + case do_check_opts(maps:to_list(Options)) of + ok -> + case overload_levels_ok(Options) of + true -> + ok; + false -> + Faulty = maps:with([sync_mode_qlen, + drop_mode_qlen, + flush_qlen],Options), + {error,{invalid_olp_levels,Faulty}} + end; + {error,Key,Value} -> + {error,{invalid_olp_config,#{Key=>Value}}} + end. + +do_check_opts([{sync_mode_qlen,N}|Options]) when is_integer(N) -> + do_check_opts(Options); +do_check_opts([{drop_mode_qlen,N}|Options]) when is_integer(N) -> + do_check_opts(Options); +do_check_opts([{flush_qlen,N}|Options]) when is_integer(N) -> + do_check_opts(Options); +do_check_opts([{burst_limit_enable,Bool}|Options]) when is_boolean(Bool) -> + do_check_opts(Options); +do_check_opts([{burst_limit_max_count,N}|Options]) when is_integer(N) -> + do_check_opts(Options); +do_check_opts([{burst_limit_window_time,N}|Options]) when is_integer(N) -> + do_check_opts(Options); +do_check_opts([{overload_kill_enable,Bool}|Options]) when is_boolean(Bool) -> + do_check_opts(Options); +do_check_opts([{overload_kill_qlen,N}|Options]) when is_integer(N) -> + do_check_opts(Options); +do_check_opts([{overload_kill_mem_size,N}|Options]) when is_integer(N) -> + do_check_opts(Options); +do_check_opts([{overload_kill_restart_after,NorA}|Options]) + when is_integer(NorA); NorA == infinity -> + do_check_opts(Options); +do_check_opts([{Key,Value}|_]) -> + {error,Key,Value}; +do_check_opts([]) -> + ok. + +set_restart_flag(#{id := Name, module := Module}) -> + Flag = list_to_atom(lists:concat([Module,"_",Name,"_restarting"])), + spawn(fun() -> + register(Flag, self()), + timer:sleep(infinity) + end), + ok. + +reset_restart_flag(#{id := Name, module := Module} = State) -> + Flag = list_to_atom(lists:concat([Module,"_",Name,"_restarting"])), + case whereis(Flag) of + undefined -> + State; + Pid -> + exit(Pid, kill), + notify(restart,State) + end. + +check_load(State = #{id:=_Name, mode_ref := ModeRef, mode := Mode, + sync_mode_qlen := SyncModeQLen, + drop_mode_qlen := DropModeQLen, + flush_qlen := FlushQLen}) -> + {_,Mem} = process_info(self(), memory), + ?observe(_Name,{max_mem,Mem}), + {_,QLen} = process_info(self(), message_queue_len), + ?observe(_Name,{max_qlen,QLen}), + %% When the handler process gets scheduled in, it's impossible + %% to predict the QLen. We could jump "up" arbitrarily from say + %% async to sync, async to drop, sync to flush, etc. However, when + %% the handler process manages the log events (without flushing), + %% one after the other, we will move "down" from drop to sync and + %% from sync to async. This way we don't risk getting stuck in + %% drop or sync mode with an empty mailbox. + {Mode1,_NewDrops,_NewFlushes} = + if + QLen >= FlushQLen -> + {flush, 0,1}; + QLen >= DropModeQLen -> + %% Note that drop mode will force load messages to + %% be dropped on the client side (never sent to + %% the olp process). + IncDrops = if Mode == drop -> 0; true -> 1 end, + {?change_mode(ModeRef, Mode, drop), IncDrops,0}; + QLen >= SyncModeQLen -> + {?change_mode(ModeRef, Mode, sync), 0,0}; + true -> + {?change_mode(ModeRef, Mode, async), 0,0} + end, + State1 = ?update_other(drops,DROPS,_NewDrops,State), + State2 = ?update_max_qlen(QLen,State1), + State3 = maybe_notify_mode_change(Mode1,State2), + {Mode1, QLen, Mem, + ?update_other(flushes,FLUSHES,_NewFlushes, + State3#{last_qlen => QLen})}. + +limit_burst(#{burst_limit_enable := false}=State) -> + {true,State}; +limit_burst(#{burst_win_ts := BurstWinT0, + burst_msg_count := BurstMsgCount, + burst_limit_window_time := BurstLimitWinTime, + burst_limit_max_count := BurstLimitMaxCnt} = State) -> + if (BurstMsgCount >= BurstLimitMaxCnt) -> + %% the limit for allowed messages has been reached + BurstWinT1 = ?timestamp(), + case ?diff_time(BurstWinT1,BurstWinT0) of + BurstCheckTime when BurstCheckTime < (BurstLimitWinTime*1000) -> + %% we're still within the burst time frame + {false,?update_other(burst_drops,BURSTS,1,State)}; + _BurstCheckTime -> + %% burst time frame passed, reset counters + {true,State#{burst_win_ts => BurstWinT1, + burst_msg_count => 0}} + end; + true -> + %% the limit for allowed messages not yet reached + {true,State#{burst_win_ts => BurstWinT0, + burst_msg_count => BurstMsgCount+1}} + end. + +kill_if_choked(QLen, Mem, #{overload_kill_enable := KillIfOL, + overload_kill_qlen := OLKillQLen, + overload_kill_mem_size := OLKillMem}) -> + if KillIfOL andalso + ((QLen > OLKillQLen) orelse (Mem > OLKillMem)) -> + exit({shutdown,{overloaded,QLen,Mem}}); + true -> + ok + end. + +flush_load(Limit) -> + process_flag(priority, high), + Flushed = flush_load(0, Limit), + process_flag(priority, normal), + Flushed. + +flush_load(Limit, Limit) -> + Limit; +flush_load(N, Limit) -> + %% flush log events but leave other events, such as info, reset + %% and stop, so that these have a chance to be processed even + %% under heavy load + receive + {'$gen_cast',{'$olp_load',_}} -> + flush_load(N+1, Limit); + {'$gen_call',{Pid,MRef},{'$olp_load',_}} -> + Pid ! {MRef, dropped}, + flush_load(N+1, Limit); + {log,_,_,_,_} -> + flush_load(N+1, Limit); + {log,_,_,_} -> + flush_load(N+1, Limit) + after + 0 -> N + end. + +overload_levels_ok(Options) -> + SMQL = maps:get(sync_mode_qlen, Options, ?SYNC_MODE_QLEN), + DMQL = maps:get(drop_mode_qlen, Options, ?DROP_MODE_QLEN), + FQL = maps:get(flush_qlen, Options, ?FLUSH_QLEN), + (DMQL > 1) andalso (SMQL =< DMQL) andalso (DMQL =< FQL). + +maybe_notify_mode_change(drop,#{mode:=Mode0}=State) + when Mode0=/=drop -> + notify({mode_change,Mode0,drop},State); +maybe_notify_mode_change(Mode1,#{mode:=drop}=State) + when Mode1==async; Mode1==sync -> + notify({mode_change,drop,Mode1},State); +maybe_notify_mode_change(_,State) -> + State. + +notify(Note,#{module:=Module,cb_state:=CBState}=State) -> + CBState1 = try_callback_call(Module,notify,[Note,CBState],CBState), + State#{cb_state=>CBState1}. + +try_callback_call(Module, Function, Args) -> + try_callback_call(Module, Function, Args, '$no_default_return'). + +try_callback_call(Module, Function, Args, DefRet) -> + try apply(Module, Function, Args) + catch + throw:R -> R; + error:undef:S when DefRet=/='$no_default_return' -> + case S of + [{Module,Function,Args,_}|_] -> + DefRet; + _ -> + erlang:raise(error,undef,S) + end + end. + +noreply_return(#{idle:=true}=State) -> + {noreply,State}; +noreply_return(#{idle:=false}=State) -> + {noreply,State,?IDLE_DETECT_TIME}. + +reply_return(Reply,#{idle:=true}=State) -> + {reply,Reply,State}; +reply_return(Reply,#{idle:=false}=State) -> + {reply,Reply,State,?IDLE_DETECT_TIME}. diff --git a/lib/kernel/src/logger_olp.hrl b/lib/kernel/src/logger_olp.hrl new file mode 100644 index 0000000000..9b4f5ebf27 --- /dev/null +++ b/lib/kernel/src/logger_olp.hrl @@ -0,0 +1,180 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2015. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +%%%----------------------------------------------------------------- +%%% Overload protection configuration + +%%! *** NOTE *** +%%! It's important that: +%%! SYNC_MODE_QLEN =< DROP_MODE_QLEN =< FLUSH_QLEN +%%! and that DROP_MODE_QLEN >= 2. +%%! Otherwise the process could end up in drop mode with no new +%%! log requests to process. This would cause all future requests +%%! to be dropped (no switch to async mode would ever take place). + +%% This specifies the message_queue_len value where the log +%% requests switch from asynchronous casts to synchronous calls. +-define(SYNC_MODE_QLEN, 10). +%% Above this message_queue_len, log requests will be dropped, +%% i.e. no log requests get sent to the process. +-define(DROP_MODE_QLEN, 200). +%% Above this message_queue_len, the process will flush its mailbox +%% and only leave this number of messages in it. +-define(FLUSH_QLEN, 1000). + +%% Never flush more than this number of messages in one go, or the +%% process will be unresponsive for seconds (keep this number as large +%% as possible or the mailbox could grow large). +-define(FLUSH_MAX_N, 5000). + +%% BURST_LIMIT_MAX_COUNT is the max number of log requests allowed +%% to be written within a BURST_LIMIT_WINDOW_TIME time frame. +-define(BURST_LIMIT_ENABLE, true). +-define(BURST_LIMIT_MAX_COUNT, 500). +-define(BURST_LIMIT_WINDOW_TIME, 1000). + +%% This enables/disables the feature to automatically terminate the +%% process if it gets too loaded (and can't keep up). +-define(OVERLOAD_KILL_ENABLE, false). +%% If the message_queue_len goes above this size even after +%% flushing has been performed, the process is terminated. +-define(OVERLOAD_KILL_QLEN, 20000). +%% If the memory usage exceeds this level, the process is terminated. +-define(OVERLOAD_KILL_MEM_SIZE, 3000000). + +%% This is the default time to wait before restarting and accepting +%% new requests. The value 'infinity' disables restarts. +-define(OVERLOAD_KILL_RESTART_AFTER, 5000). + +%% This is the time in milliseconds after last load message received +%% that we notify the callback about being idle. +-define(IDLE_DETECT_TIME, 100). + +%%%----------------------------------------------------------------- +%%% Overload protection macros + +-define(timestamp(), erlang:monotonic_time(microsecond)). + +-define(get_mode(Tid), + case ets:lookup(Tid, mode) of + [{mode,M}] -> M; + _ -> async + end). + +-define(set_mode(Tid, M), + begin ets:insert(Tid, {mode,M}), M end). + +-define(change_mode(Tid, M0, M1), + if M0 == M1 -> + M0; + true -> + ets:insert(Tid, {mode,M1}), + M1 + end). + +-define(max(X1, X2), + if + X2 == undefined -> X1; + X2 > X1 -> X2; + true -> X1 + end). + +-define(diff_time(OS_T1, OS_T0), OS_T1-OS_T0). + +%%%----------------------------------------------------------------- +%%% These macros enable statistics counters in the state of the +%%% process, which is useful for analysing the overload protection +%%% behaviour. These counters should not be included in code to be +%%% officially released (as some counters will grow very large over +%%% time). + +%% -define(SAVE_STATS, true). +-ifdef(SAVE_STATS). + -define(merge_with_stats(STATE), + begin + TIME = ?timestamp(), + STATE#{start => TIME, time => {TIME,0}, + flushes => 0, flushed => 0, drops => 0, + burst_drops => 0, casts => 0, calls => 0, + writes => 0, max_qlen => 0, max_time => 0, + freq => {TIME,0,0}} end). + + -define(update_max_qlen(QLEN, STATE), + begin #{max_qlen := QLEN0} = STATE, + STATE#{max_qlen => ?max(QLEN0,QLEN)} end). + + -define(update_calls_or_casts(CALL_OR_CAST, INC, STATE), + case CALL_OR_CAST of + cast -> + #{casts := CASTS0} = STATE, + STATE#{casts => CASTS0+INC}; + call -> + #{calls := CALLS0} = STATE, + STATE#{calls => CALLS0+INC} + end). + + -define(update_max_time(TIME, STATE), + begin #{max_time := TIME0} = STATE, + STATE#{max_time => ?max(TIME0,TIME)} end). + + -define(update_other(OTHER, VAR, INCVAL, STATE), + begin #{OTHER := VAR} = STATE, + STATE#{OTHER => VAR+INCVAL} end). + + -define(update_freq(TIME,STATE), + begin + case STATE of + #{freq := {START, 49, _}} -> + STATE#{freq => {TIME, 0, trunc(1000000*50/(?diff_time(TIME,START)))}}; + #{freq := {START, N, FREQ}} -> + STATE#{freq => {START, N+1, FREQ}} + end end). + + -define(update_time(TIME,STATE), + begin #{start := START} = STATE, + STATE#{time => {TIME,trunc((?diff_time(TIME,START))/1000000)}} end). + +-else. % DEFAULT! + -define(merge_with_stats(STATE), STATE). + -define(update_max_qlen(_QLEN, STATE), STATE). + -define(update_calls_or_casts(_CALL_OR_CAST, _INC, STATE), STATE). + -define(update_max_time(_TIME, STATE), STATE). + -define(update_other(_OTHER, _VAR, _INCVAL, STATE), STATE). + -define(update_freq(_TIME, STATE), STATE). + -define(update_time(_TIME, STATE), STATE). +-endif. + +%%%----------------------------------------------------------------- +%%% These macros enable callbacks that make it possible to analyse the +%%% overload protection behaviour from outside the process (including +%%% dropped requests on the client side). An external callback module +%%% (?OBSERVER_MOD) is required which is not part of the kernel +%%% application. For this reason, these callbacks should not be +%%% included in code to be officially released. + +%%-define(OBSERVER_MOD, logger_test). +-ifdef(OBSERVER_MOD). + -define(start_observation(NAME), ?OBSERVER:start_observation(NAME)). + -define(observe(NAME,EVENT), ?OBSERVER:observe(NAME,EVENT)). + +-else. % DEFAULT! + -define(start_observation(_NAME), ok). + -define(observe(_NAME,_EVENT), ok). +-endif. diff --git a/lib/kernel/src/logger_proxy.erl b/lib/kernel/src/logger_proxy.erl new file mode 100644 index 0000000000..24b293805c --- /dev/null +++ b/lib/kernel/src/logger_proxy.erl @@ -0,0 +1,165 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% +-module(logger_proxy). + +%% API +-export([start_link/0, restart/0, log/1, child_spec/0, get_default_config/0]). + +%% logger_olp callbacks +-export([init/1, handle_load/2, handle_info/2, terminate/2, + notify/2]). + +-include("logger_internal.hrl"). + +-define(SERVER,?MODULE). + +%%%----------------------------------------------------------------- +%%% API +-spec log(RemoteLog) -> ok when + RemoteLog :: {remote,node(),LogEvent}, + LogEvent :: {log,Level,Format,Args,Meta} | + {log,Level,StringOrReport,Meta}, + Level :: logger:level(), + Format :: io:format(), + Args :: list(term()), + StringOrReport :: unicode:chardata() | logger:report(), + Meta :: logger:metadata(). +log(RemoteLog) -> + Olp = persistent_term:get(?MODULE), + case logger_olp:get_pid(Olp) =:= self() of + true -> + %% This happens when the log event comes from the + %% emulator, and the group leader is on a remote node. + _ = handle_load(RemoteLog, no_state), + ok; + false -> + logger_olp:load(Olp, RemoteLog) + end. + +%% Called by supervisor +-spec start_link() -> {ok,pid(),logger_olp:olp_ref()} | {error,term()}. +start_link() -> + %% Notice that sync_mode is only used when logging to remote node, + %% i.e. when the log/2 API function is called. + %% + %% When receiving log events from the emulator or from a remote + %% node, the log event is sent as a message to this process, and + %% thus received directly in handle_info/2. This means that the + %% mode (async/sync/drop) is not read before the message is + %% sent. Thus sync mode is never entered, and drop mode is + %% implemented by setting the system_logger flag to undefined (see + %% notify/2) + %% + %% Burst limit is disabled, since this is only a proxy and we + %% don't want to limit bursts twice (here and in the handler). + logger_olp:start_link(?SERVER,?MODULE,[],logger:get_proxy_config()). + +%% Fun used for restarting this process after it has been killed due +%% to overload (must set overload_kill_enable=>true in opts) +restart() -> + case supervisor:start_child(logger_sup, child_spec()) of + {ok,_Pid,Olp} -> + {ok,Olp}; + {error,{Reason,Ch}} when is_tuple(Ch), element(1,Ch)==child -> + {error,Reason}; + Error -> + Error + end. + +%% Called internally and by logger_sup +child_spec() -> + Name = ?SERVER, + #{id => Name, + start => {?MODULE, start_link, []}, + restart => temporary, + shutdown => 2000, + type => worker, + modules => [?MODULE]}. + +get_default_config() -> + OlpDefault = logger_olp:get_default_opts(), + OlpDefault#{sync_mode_qlen=>500, + drop_mode_qlen=>1000, + flush_qlen=>5000, + burst_limit_enable=>false}. + +%%%=================================================================== +%%% gen_server callbacks +%%%=================================================================== + +init([]) -> + process_flag(trap_exit, true), + _ = erlang:system_flag(system_logger,self()), + persistent_term:put(?MODULE,logger_olp:get_ref()), + {ok,no_state}. + +%% Log event to send to the node where the group leader of it's client resides +handle_load({remote,Node,Log},State) -> + %% If the connection is overloaded (send_nosuspend returns false), + %% we drop the message. + _ = erlang:send_nosuspend({?SERVER,Node},Log), + State; +%% Log event to log on this node +handle_load({log,Level,Format,Args,Meta},State) -> + try_log([Level,Format,Args,Meta]), + State; +handle_load({log,Level,Report,Meta},State) -> + try_log([Level,Report,Meta]), + State. + +%% Log event sent to this process e.g. from the emulator - it is really load +handle_info(Log,State) when is_tuple(Log), element(1,Log)==log -> + {load,State}. + +terminate(overloaded, _State) -> + _ = erlang:system_flag(system_logger,undefined), + {ok,fun ?MODULE:restart/0}; +terminate(_Reason, _State) -> + _ = erlang:system_flag(system_logger,whereis(logger)), + ok. + +notify({mode_change,Mode0,Mode1},State) -> + _ = if Mode1=:=drop -> % entering drop mode + erlang:system_flag(system_logger,undefined); + Mode0=:=drop -> % leaving drop mode + erlang:system_flag(system_logger,self()); + true -> + ok + end, + ?LOG_INTERNAL(notice,"~w switched from ~w to ~w mode",[?MODULE,Mode0,Mode1]), + State; +notify({flushed,Flushed},State) -> + ?LOG_INTERNAL(notice, "~w flushed ~w log events",[?MODULE,Flushed]), + State; +notify(restart,State) -> + ?LOG_INTERNAL(notice, "~w restarted", [?MODULE]), + State; +notify(_Note,State) -> + State. + +%%%----------------------------------------------------------------- +%%% Internal functions +try_log(Args) -> + try apply(logger,log,Args) + catch C:R:S -> + ?LOG_INTERNAL(debug,[{?MODULE,log_failed}, + {log,Args}, + {reason,{C,R,S}}]) + end. diff --git a/lib/kernel/src/logger_server.erl b/lib/kernel/src/logger_server.erl index b7735dbcf7..722246e82c 100644 --- a/lib/kernel/src/logger_server.erl +++ b/lib/kernel/src/logger_server.erl @@ -22,8 +22,7 @@ -behaviour(gen_server). %% API --export([start_link/0, - add_handler/3, remove_handler/1, +-export([start_link/0, add_handler/3, remove_handler/1, add_filter/2, remove_filter/2, set_module_level/2, unset_module_level/0, unset_module_level/1, cache_module_level/1, @@ -43,7 +42,7 @@ -define(SERVER, logger). -define(LOGGER_SERVER_TAG, '$logger_cb_process'). --record(state, {tid, async_req, async_req_queue}). +-record(state, {tid, async_req, async_req_queue, remote_logger}). %%%=================================================================== %%% API @@ -155,6 +154,8 @@ init([]) -> process_flag(trap_exit, true), put(?LOGGER_SERVER_TAG,true), Tid = logger_config:new(?LOGGER_TABLE), + %% Store initial proxy config. logger_proxy reads config from here at startup. + logger_config:create(Tid,proxy,logger_proxy:get_default_config()), PrimaryConfig = maps:merge(default_config(primary), #{handlers=>[simple]}), logger_config:create(Tid,primary,PrimaryConfig), @@ -221,6 +222,24 @@ handle_call({add_filter,Id,Filter}, _From,#state{tid=Tid}=State) -> handle_call({remove_filter,Id,FilterId}, _From, #state{tid=Tid}=State) -> Reply = do_remove_filter(Tid,Id,FilterId), {reply,Reply,State}; +handle_call({change_config,SetOrUpd,proxy,Config0},_From,#state{tid=Tid}=State) -> + Default = + case SetOrUpd of + set -> + logger_proxy:get_default_config(); + update -> + {ok,OldConfig} = logger_config:get(Tid,proxy), + OldConfig + end, + Config = maps:merge(Default,Config0), + Reply = + case logger_olp:set_opts(logger_proxy,Config) of + ok -> + logger_config:set(Tid,proxy,Config); + Error -> + Error + end, + {reply,Reply,State}; handle_call({change_config,SetOrUpd,primary,Config0}, _From, #state{tid=Tid}=State) -> {ok,#{handlers:=Handlers}=OldConfig} = logger_config:get(Tid,primary), @@ -357,7 +376,7 @@ terminate(_Reason, _State) -> %%%=================================================================== %%% Internal functions %%%=================================================================== -call(Request) -> +call(Request) when is_tuple(Request) -> Action = element(1,Request), case get(?LOGGER_SERVER_TAG) of true when @@ -369,6 +388,7 @@ call(Request) -> gen_server:call(?SERVER,Request,?DEFAULT_LOGGER_CALL_TIMEOUT) end. + do_add_filter(Tid,Id,{FId,_} = Filter) -> case logger_config:get(Tid,Id) of {ok,Config} -> @@ -413,11 +433,13 @@ default_config(Id,Module) -> sanity_check(Owner,Key,Value) -> sanity_check_1(Owner,[{Key,Value}]). -sanity_check(HandlerId,Config) when is_map(Config) -> - sanity_check_1(HandlerId,maps:to_list(Config)); +sanity_check(Owner,Config) when is_map(Config) -> + sanity_check_1(Owner,maps:to_list(Config)); sanity_check(_,Config) -> {error,{invalid_config,Config}}. +sanity_check_1(proxy,_Config) -> + ok; % Details are checked by logger_olp:set_opts/2 sanity_check_1(Owner,Config) when is_list(Config) -> try Type = get_type(Owner), diff --git a/lib/kernel/src/logger_std_h.erl b/lib/kernel/src/logger_std_h.erl index 63d1dbaba2..65f5b3876e 100644 --- a/lib/kernel/src/logger_std_h.erl +++ b/lib/kernel/src/logger_std_h.erl @@ -26,7 +26,7 @@ -include_lib("kernel/include/file.hrl"). %% API --export([info/1, filesync/1, reset/1]). +-export([filesync/1]). %% logger_h_common callbacks -export([init/2, check_config/4, reset_state/2, @@ -36,6 +36,8 @@ -export([log/2, adding_handler/1, removing_handler/1, changing_config/3, filter_config/1]). +-define(DEFAULT_CALL_TIMEOUT, 5000). + %%%=================================================================== %%% API %%%=================================================================== @@ -49,25 +51,6 @@ filesync(Name) -> logger_h_common:filesync(?MODULE,Name). -%%%----------------------------------------------------------------- -%%% --spec info(Name) -> Info | {error,Reason} when - Name :: atom(), - Info :: term(), - Reason :: handler_busy | {badarg,term()}. - -info(Name) -> - logger_h_common:info(?MODULE,Name). - -%%%----------------------------------------------------------------- -%%% --spec reset(Name) -> ok | {error,Reason} when - Name :: atom(), - Reason :: handler_busy | {badarg,term()}. - -reset(Name) -> - logger_h_common:reset(?MODULE,Name). - %%%=================================================================== %%% logger callbacks - just forward to logger_h_common %%%=================================================================== @@ -234,17 +217,24 @@ open_log_file(HandlerName, FileInfo) -> Error -> Error end. -do_open_log_file({file,File}) -> - do_open_log_file({file,File,[raw,append,delayed_write]}); +do_open_log_file({file,FileName}) -> + do_open_log_file({file,FileName,[raw,append,delayed_write]}); -do_open_log_file({file,File,[]}) -> - do_open_log_file({file,File,[raw,append,delayed_write]}); +do_open_log_file({file,FileName,[]}) -> + do_open_log_file({file,FileName,[raw,append,delayed_write]}); -do_open_log_file({file,File,Modes}) -> +do_open_log_file({file,FileName,Modes}) -> try - case filelib:ensure_dir(File) of + case filelib:ensure_dir(FileName) of ok -> - file:open(File, Modes); + case file:open(FileName, Modes) of + {ok, Fd} -> + {ok,#file_info{inode=INode}} = + file:read_file_info(FileName), + {ok, {Fd, INode}}; + Error -> + Error + end; Error -> Error end @@ -254,7 +244,7 @@ do_open_log_file({file,File,Modes}) -> close_log_file(Std) when Std == standard_io; Std == standard_error -> ok; -close_log_file(Fd) -> +close_log_file({Fd,_}) -> _ = file:datasync(Fd), _ = file:close(Fd). @@ -313,9 +303,9 @@ file_ctrl_init(HandlerName, FileInfo, Starter) when is_tuple(FileInfo) -> process_flag(message_queue_data, off_heap), FileName = element(2, FileInfo), case do_open_log_file(FileInfo) of - {ok,Fd} -> + {ok,File} -> Starter ! {self(),ok}, - file_ctrl_loop(Fd, FileName, false, ok, ok, HandlerName); + file_ctrl_loop(File, FileName, false, ok, ok, HandlerName); {error,Reason} -> Starter ! {self(),{error,{open_failed,FileName,Reason}}} end; @@ -323,39 +313,43 @@ file_ctrl_init(HandlerName, StdDev, Starter) -> Starter ! {self(),ok}, file_ctrl_loop(StdDev, StdDev, false, ok, ok, HandlerName). -file_ctrl_loop(Fd, DevName, Synced, +file_ctrl_loop(File, DevName, Synced, PrevWriteResult, PrevSyncResult, HandlerName) -> receive %% asynchronous event {log,Bin} -> - Fd1 = ensure(Fd, DevName), - Result = write_to_dev(Fd1, Bin, DevName, PrevWriteResult, HandlerName), - file_ctrl_loop(Fd1, DevName, false, + File1 = ensure(File, DevName), + Result = write_to_dev(File1, Bin, DevName, + PrevWriteResult, HandlerName), + file_ctrl_loop(File1, DevName, false, Result, PrevSyncResult, HandlerName); %% synchronous event {{log,Bin},{From,MRef}} -> - Fd1 = ensure(Fd, DevName), - Result = write_to_dev(Fd1, Bin, DevName, PrevWriteResult, HandlerName), + File1 = ensure(File, DevName), + Result = write_to_dev(File1, Bin, DevName, + PrevWriteResult, HandlerName), From ! {MRef,ok}, - file_ctrl_loop(Fd1, DevName, false, + file_ctrl_loop(File1, DevName, false, Result, PrevSyncResult, HandlerName); filesync -> - Fd1 = ensure(Fd, DevName), - Result = sync_dev(Fd1, DevName, Synced, PrevSyncResult, HandlerName), - file_ctrl_loop(Fd1, DevName, true, + File1 = ensure(File, DevName), + Result = sync_dev(File1, DevName, Synced, + PrevSyncResult, HandlerName), + file_ctrl_loop(File1, DevName, true, PrevWriteResult, Result, HandlerName); {filesync,{From,MRef}} -> - Fd1 = ensure(Fd, DevName), - Result = sync_dev(Fd1, DevName, Synced, PrevSyncResult, HandlerName), + File1 = ensure(File, DevName), + Result = sync_dev(File1, DevName, Synced, + PrevSyncResult, HandlerName), From ! {MRef,ok}, - file_ctrl_loop(Fd1, DevName, true, + file_ctrl_loop(File1, DevName, true, PrevWriteResult, Result, HandlerName); stop -> - _ = close_log_file(Fd), + _ = close_log_file(File), stopped end. @@ -364,16 +358,16 @@ file_ctrl_loop(Fd, DevName, Synced, %% logrotate) ensure(Fd,DevName) when is_atom(DevName) -> Fd; -ensure(Fd,FileName) -> +ensure({Fd,INode},FileName) -> case file:read_file_info(FileName) of - {ok,_} -> - Fd; + {ok,#file_info{inode=INode}} -> + {Fd,INode}; _ -> _ = file:close(Fd), _ = file:close(Fd), % delayed_write cause close not to close case do_open_log_file({file,FileName}) of - {ok,Fd1} -> - Fd1; + {ok,File} -> + File; Error -> exit({could_not_reopen_file,Error}) end @@ -382,13 +376,13 @@ ensure(Fd,FileName) -> write_to_dev(DevName, Bin, _DevName, _PrevWriteResult, _HandlerName) when is_atom(DevName) -> io:put_chars(DevName, Bin); -write_to_dev(Fd, Bin, FileName, PrevWriteResult, HandlerName) -> +write_to_dev({Fd,_}, Bin, FileName, PrevWriteResult, HandlerName) -> Result = ?file_write(Fd, Bin), maybe_notify_error(write,Result,PrevWriteResult,FileName,HandlerName). -sync_dev(_Fd, _FileName, true, PrevSyncResult, _HandlerName) -> +sync_dev(_, _FileName, true, PrevSyncResult, _HandlerName) -> PrevSyncResult; -sync_dev(Fd, FileName, false, PrevSyncResult, HandlerName) -> +sync_dev({Fd,_}, FileName, false, PrevSyncResult, HandlerName) -> Result = ?file_datasync(Fd), maybe_notify_error(filesync,Result,PrevSyncResult,FileName,HandlerName). diff --git a/lib/kernel/src/logger_sup.erl b/lib/kernel/src/logger_sup.erl index 3d6f482e20..9ea8558a16 100644 --- a/lib/kernel/src/logger_sup.erl +++ b/lib/kernel/src/logger_sup.erl @@ -50,7 +50,9 @@ init([]) -> start => {logger_handler_watcher, start_link, []}, shutdown => brutal_kill}, - {ok, {SupFlags, [Watcher]}}. + Proxy = logger_proxy:child_spec(), + + {ok, {SupFlags, [Watcher,Proxy]}}. %%%=================================================================== %%% Internal functions diff --git a/lib/kernel/src/standard_error.erl b/lib/kernel/src/standard_error.erl index 5d649e5f94..ef5b532960 100644 --- a/lib/kernel/src/standard_error.erl +++ b/lib/kernel/src/standard_error.erl @@ -27,7 +27,8 @@ -define(PROCNAME_SUP, standard_error_sup). %% Defines for control ops --define(CTRL_OP_GET_WINSIZE,100). +-define(ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER, 16#018b0900). +-define(CTRL_OP_GET_WINSIZE, (100 + ?ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER)). %% %% The basic server and start-up. diff --git a/lib/kernel/src/user.erl b/lib/kernel/src/user.erl index 872e63ab53..0c9e1ea303 100644 --- a/lib/kernel/src/user.erl +++ b/lib/kernel/src/user.erl @@ -28,7 +28,8 @@ -define(NAME, user). %% Defines for control ops --define(CTRL_OP_GET_WINSIZE,100). +-define(ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER, 16#018b0900). +-define(CTRL_OP_GET_WINSIZE, (100 + ?ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER)). %% %% The basic server and start-up. diff --git a/lib/kernel/src/user_drv.erl b/lib/kernel/src/user_drv.erl index 9f914aa222..08286dd476 100644 --- a/lib/kernel/src/user_drv.erl +++ b/lib/kernel/src/user_drv.erl @@ -32,9 +32,10 @@ -define(OP_BEEP,4). -define(OP_PUTC_SYNC,5). % Control op --define(CTRL_OP_GET_WINSIZE,100). --define(CTRL_OP_GET_UNICODE_STATE,101). --define(CTRL_OP_SET_UNICODE_STATE,102). +-define(ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER, 16#018b0900). +-define(CTRL_OP_GET_WINSIZE, (100 + ?ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER)). +-define(CTRL_OP_GET_UNICODE_STATE, (101 + ?ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER)). +-define(CTRL_OP_SET_UNICODE_STATE, (102 + ?ERTS_TTYSL_DRV_CONTROL_MAGIC_NUMBER)). %% start() %% start(ArgumentList) diff --git a/lib/kernel/test/Makefile b/lib/kernel/test/Makefile index 4a86265a4a..8a6ffe7e72 100644 --- a/lib/kernel/test/Makefile +++ b/lib/kernel/test/Makefile @@ -76,8 +76,11 @@ MODULES= \ logger_filters_SUITE \ logger_formatter_SUITE \ logger_legacy_SUITE \ + logger_olp_SUITE \ + logger_proxy_SUITE \ logger_simple_h_SUITE \ logger_std_h_SUITE \ + logger_stress_SUITE \ logger_test_lib \ os_SUITE \ pg2_SUITE \ diff --git a/lib/kernel/test/gen_tcp_misc_SUITE.erl b/lib/kernel/test/gen_tcp_misc_SUITE.erl index 244bd7e2a0..52edfaee29 100644 --- a/lib/kernel/test/gen_tcp_misc_SUITE.erl +++ b/lib/kernel/test/gen_tcp_misc_SUITE.erl @@ -53,7 +53,7 @@ active_once_closed/1, send_timeout/1, send_timeout_active/1, otp_7731/1, zombie_sockets/1, otp_7816/1, otp_8102/1, wrapping_oct/0, wrapping_oct/1, otp_9389/1, otp_13939/1, - otp_12242/1]). + otp_12242/1, delay_send_error/1]). %% Internal exports. -export([sender/3, not_owner/1, passive_sockets_server/2, priority_server/1, @@ -97,7 +97,7 @@ all() -> active_once_closed, send_timeout, send_timeout_active, otp_7731, wrapping_oct, zombie_sockets, otp_7816, otp_8102, otp_9389, - otp_12242]. + otp_12242, delay_send_error]. groups() -> []. @@ -3427,3 +3427,32 @@ otp_12242(Addr) when tuple_size(Addr) =:= 4 -> wait(Mref) -> receive {'DOWN',Mref,_,_,Reason} -> Reason end. + +%% OTP-15536 +%% Test that send error works correctly for delay_send +delay_send_error(Config) -> + {ok, LS} = gen_tcp:listen(0, [{reuseaddr, true}, {packet, 1}, {active, false}]), + {ok,{{0,0,0,0},PortNum}}=inet:sockname(LS), + P = spawn_link( + fun() -> + {ok, S} = gen_tcp:accept(LS), + receive die -> gen_tcp:close(S) end + end), + erlang:monitor(process, P), + {ok, S} = gen_tcp:connect("localhost", PortNum, + [{packet, 1}, {active, false}, {delay_send, true}]), + + %% Do a couple of sends first to see that it works + ok = gen_tcp:send(S, "hello"), + ok = gen_tcp:send(S, "hello"), + ok = gen_tcp:send(S, "hello"), + + %% Make the receiver close + P ! die, + receive _Down -> ok end, + + ok = gen_tcp:send(S, "hello"), + timer:sleep(500), %% Sleep in order for delay_send to have time to trigger + + %% This used to result in a double free + {error, closed} = gen_tcp:send(S, "hello"). diff --git a/lib/kernel/test/init_SUITE.erl b/lib/kernel/test/init_SUITE.erl index 6a006cdc01..4f90260f98 100644 --- a/lib/kernel/test/init_SUITE.erl +++ b/lib/kernel/test/init_SUITE.erl @@ -295,7 +295,7 @@ is_real_system(KernelVsn, StdlibVsn) -> %% before restart. %% ------------------------------------------------ many_restarts() -> - [{timetrap,{minutes,8}}]. + [{timetrap,{minutes,16}}]. many_restarts(Config) when is_list(Config) -> {ok, Node} = loose_node:start(init_test, "", ?DEFAULT_TIMEOUT_SEC), @@ -315,7 +315,7 @@ loop_restart(N,Node,EHPid) -> loose_node:stop(Node), ct:fail(not_stopping) end, - ok = wait_for(30, Node, EHPid), + ok = wait_for(60, Node, EHPid), loop_restart(N-1,Node,rpc:call(Node,erlang,whereis,[logger])). wait_for(0,Node,_) -> @@ -367,7 +367,8 @@ restart(Config) when is_list(Config) -> SysProcs0 = rpc:call(Node, ?MODULE, find_system_processes, []), io:format("SysProcs0=~p~n", [SysProcs0]), [InitPid, PurgerPid, LitCollectorPid, - DirtySigNPid, DirtySigHPid, DirtySigMPid] = SysProcs0, + DirtySigNPid, DirtySigHPid, DirtySigMPid, + PrimFilePid] = SysProcs0, InitPid = rpc:call(Node, erlang, whereis, [init]), PurgerPid = rpc:call(Node, erlang, whereis, [erts_code_purger]), Procs = rpc:call(Node, erlang, processes, []), @@ -385,7 +386,8 @@ restart(Config) when is_list(Config) -> SysProcs1 = rpc:call(Node, ?MODULE, find_system_processes, []), io:format("SysProcs1=~p~n", [SysProcs1]), [InitPid1, PurgerPid1, LitCollectorPid1, - DirtySigNPid1, DirtySigHPid1, DirtySigMPid1] = SysProcs1, + DirtySigNPid1, DirtySigHPid1, DirtySigMPid1, + PrimFilePid1] = SysProcs1, %% Still the same init process! InitPid1 = rpc:call(Node, erlang, whereis, [init]), @@ -411,6 +413,10 @@ restart(Config) when is_list(Config) -> DirtySigMP = pid_to_list(DirtySigMPid), DirtySigMP = pid_to_list(DirtySigMPid1), + %% and same prim_file helper process! + PrimFileP = pid_to_list(PrimFilePid), + PrimFileP = pid_to_list(PrimFilePid1), + NewProcs0 = rpc:call(Node, erlang, processes, []), NewProcs = NewProcs0 -- SysProcs1, case check_processes(NewProcs, MaxPid) of @@ -437,7 +443,8 @@ restart(Config) when is_list(Config) -> literal_collector, dirty_sig_handler_normal, dirty_sig_handler_high, - dirty_sig_handler_max}). + dirty_sig_handler_max, + prim_file}). find_system_processes() -> find_system_procs(processes(), #sys_procs{}). @@ -448,7 +455,8 @@ find_system_procs([], SysProcs) -> SysProcs#sys_procs.literal_collector, SysProcs#sys_procs.dirty_sig_handler_normal, SysProcs#sys_procs.dirty_sig_handler_high, - SysProcs#sys_procs.dirty_sig_handler_max]; + SysProcs#sys_procs.dirty_sig_handler_max, + SysProcs#sys_procs.prim_file]; find_system_procs([P|Ps], SysProcs) -> case process_info(P, [initial_call, priority]) of [{initial_call,{otp_ring0,start,2}},_] -> @@ -472,6 +480,9 @@ find_system_procs([P|Ps], SysProcs) -> {priority,max}] -> undefined = SysProcs#sys_procs.dirty_sig_handler_max, find_system_procs(Ps, SysProcs#sys_procs{dirty_sig_handler_max = P}); + [{initial_call,{prim_file,start,0}},_] -> + undefined = SysProcs#sys_procs.prim_file, + find_system_procs(Ps, SysProcs#sys_procs{prim_file = P}); _ -> find_system_procs(Ps, SysProcs) end. diff --git a/lib/kernel/test/kernel_bench.spec b/lib/kernel/test/kernel_bench.spec index 4de133f21b..898ceb59e0 100644 --- a/lib/kernel/test/kernel_bench.spec +++ b/lib/kernel/test/kernel_bench.spec @@ -1,2 +1,3 @@ {groups,"../kernel_test",zlib_SUITE,[bench]}. {groups,"../kernel_test",file_SUITE,[bench]}. +{suites,"../kernel_test",[logger_stress_SUITE]}. diff --git a/lib/kernel/test/logger.cover b/lib/kernel/test/logger.cover index 960bc0abff..9691aa295e 100644 --- a/lib/kernel/test/logger.cover +++ b/lib/kernel/test/logger.cover @@ -4,9 +4,12 @@ logger_backend, logger_config, logger_disk_log_h, - logger_h_common, logger_filters, logger_formatter, + logger_handler_watcher, + logger_h_common, + logger_olp, + logger_proxy, logger_server, logger_simple_h, logger_std_h, diff --git a/lib/kernel/test/logger.spec b/lib/kernel/test/logger.spec index 1ab90b3e93..3aec37951d 100644 --- a/lib/kernel/test/logger.spec +++ b/lib/kernel/test/logger.spec @@ -7,5 +7,7 @@ logger_filters_SUITE, logger_formatter_SUITE, logger_legacy_SUITE, + logger_olp_SUITE, + logger_proxy_SUITE, logger_simple_h_SUITE, logger_std_h_SUITE]}. diff --git a/lib/kernel/test/logger_SUITE.erl b/lib/kernel/test/logger_SUITE.erl index d831d0d108..2dad651f9c 100644 --- a/lib/kernel/test/logger_SUITE.erl +++ b/lib/kernel/test/logger_SUITE.erl @@ -101,7 +101,8 @@ all() -> compare_levels, process_metadata, app_config, - kernel_config]. + kernel_config, + pretty_print]. start_stop(_Config) -> S = whereis(logger), @@ -1141,6 +1142,61 @@ kernel_config(Config) -> ok. +pretty_print(Config) -> + ok = logger:add_handler(?FUNCTION_NAME,logger_std_h,#{}), + ok = logger:set_module_level([module1,module2],debug), + + ct:capture_start(), + logger:i(), + ct:capture_stop(), + I0 = ct:capture_get(), + + ct:capture_start(), + logger:i(primary), + ct:capture_stop(), + IPrim = ct:capture_get(), + + ct:capture_start(), + logger:i(handlers), + ct:capture_stop(), + IHs = ct:capture_get(), + + ct:capture_start(), + logger:i(proxy), + ct:capture_stop(), + IProxy = ct:capture_get(), + + ct:capture_start(), + logger:i(modules), + ct:capture_stop(), + IMs = ct:capture_get(), + + I02 = lists:append([IPrim,IHs,IProxy,IMs]), + %% ct:log("~p~n",[I0]), + %% ct:log("~p~n",[I02]), + I0 = I02, + + ct:capture_start(), + logger:i(handlers), + ct:capture_stop(), + IHs = ct:capture_get(), + + Ids = logger:get_handler_ids(), + IHs2 = + lists:append( + [begin + ct:capture_start(), + logger:i(Id), + ct:capture_stop(), + [_|IH] = ct:capture_get(), + IH + end || Id <- Ids]), + + %% ct:log("~p~n",[IHs]), + %% ct:log("~p~n",[["Handler configuration: \n"|IHs2]]), + IHs = ["Handler configuration: \n"|IHs2], + ok. + %%%----------------------------------------------------------------- %%% Internal check_logged(Level,Format,Args,Meta) -> diff --git a/lib/kernel/test/logger_disk_log_h_SUITE.erl b/lib/kernel/test/logger_disk_log_h_SUITE.erl index 87b8250781..9bbec42de8 100644 --- a/lib/kernel/test/logger_disk_log_h_SUITE.erl +++ b/lib/kernel/test/logger_disk_log_h_SUITE.erl @@ -24,6 +24,7 @@ -include_lib("common_test/include/ct.hrl"). -include_lib("kernel/include/logger.hrl"). -include_lib("kernel/src/logger_internal.hrl"). +-include_lib("kernel/src/logger_olp.hrl"). -include_lib("kernel/src/logger_h_common.hrl"). -include_lib("stdlib/include/ms_transform.hrl"). -include_lib("kernel/include/file.hrl"). @@ -97,7 +98,6 @@ all() -> formatter_fail, config_fail, bad_input, - info_and_reset, reconfig, sync, disk_log_full, @@ -306,9 +306,9 @@ logging(cleanup, _Config) -> filter_config(_Config) -> ok = logger:add_handler(?MODULE,logger_disk_log_h,#{}), {ok,#{config:=HConfig}=Config} = logger:get_handler_config(?MODULE), - HConfig = maps:without([handler_pid,mode_tab],HConfig), + HConfig = maps:without([olp],HConfig), - FakeFullHConfig = HConfig#{handler_pid=>self(),mode_tab=>erlang:make_ref()}, + FakeFullHConfig = HConfig#{olp=>{regname,self(),erlang:make_ref()}}, #{config:=HConfig} = logger_disk_log_h:filter_config(Config#{config=>FakeFullHConfig}), ok. @@ -351,9 +351,7 @@ errors(Config) -> %% Read-only fields may (accidentially) be included in the change, %% but it won't take effect {ok,C} = logger:get_handler_config(Name1), - ok = logger:set_handler_config(Name1,config, - #{handler_pid=>self(), - mode_tab=>erlang:make_ref()}), + ok = logger:set_handler_config(Name1,config,#{olp=>dummyvalue}), {ok,C} = logger:get_handler_config(Name1), @@ -419,19 +417,16 @@ config_fail(_Config) -> filter_default=>log, formatter=>{?MODULE,self()}}), - {error,{handler_not_added,{invalid_config,logger_disk_log_h, - {invalid_levels,#{drop_mode_qlen:=1}}}}} = + {error,{handler_not_added,{invalid_olp_levels,#{drop_mode_qlen:=1}}}} = logger:add_handler(?MODULE,logger_disk_log_h, #{config => #{drop_mode_qlen=>1}}), - {error,{handler_not_added,{invalid_config,logger_disk_log_h, - {invalid_levels,#{sync_mode_qlen:=43, - drop_mode_qlen:=42}}}}} = + {error,{handler_not_added,{invalid_olp_levels,#{sync_mode_qlen:=43, + drop_mode_qlen:=42}}}} = logger:add_handler(?MODULE,logger_disk_log_h, #{config => #{sync_mode_qlen=>43, drop_mode_qlen=>42}}), - {error,{handler_not_added,{invalid_config,logger_disk_log_h, - {invalid_levels,#{drop_mode_qlen:=43, - flush_qlen:=42}}}}} = + {error,{handler_not_added,{invalid_olp_levels,#{drop_mode_qlen:=43, + flush_qlen:=42}}}} = logger:add_handler(?MODULE,logger_disk_log_h, #{config => #{drop_mode_qlen=>43, flush_qlen=>42}}), @@ -445,7 +440,7 @@ config_fail(_Config) -> #{max_no_files=>2}), %% incorrect values of OP params {ok,#{config := HConfig}} = logger:get_handler_config(?MODULE), - {error,{invalid_config,logger_disk_log_h,{invalid_levels,_}}} = + {error,{invalid_olp_levels,_}} = logger:update_handler_config(?MODULE,config, HConfig#{sync_mode_qlen=>100, flush_qlen=>99}), @@ -459,18 +454,7 @@ config_fail(cleanup,_Config) -> bad_input(_Config) -> {error,{badarg,{filesync,["BadType"]}}} = - logger_disk_log_h:filesync("BadType"), - {error,{badarg,{info,["BadType"]}}} = logger_disk_log_h:info("BadType"), - {error,{badarg,{reset,["BadType"]}}} = logger_disk_log_h:reset("BadType"). - -info_and_reset(_Config) -> - ok = logger:add_handler(?MODULE,logger_disk_log_h, - #{filter_default=>log, - formatter=>{?MODULE,self()}}), - #{id := ?MODULE} = logger_disk_log_h:info(?MODULE), - ok = logger_disk_log_h:reset(?MODULE). -info_and_reset(cleanup,_Config) -> - logger:remove_handler(?MODULE). + logger_disk_log_h:filesync("BadType"). reconfig(Config) -> Dir = ?config(priv_dir,Config), @@ -479,7 +463,7 @@ reconfig(Config) -> #{filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}), - #{id := ?MODULE, + #{%id := ?MODULE, sync_mode_qlen := ?SYNC_MODE_QLEN, drop_mode_qlen := ?DROP_MODE_QLEN, flush_qlen := ?FLUSH_QLEN, @@ -490,13 +474,14 @@ reconfig(Config) -> overload_kill_qlen := ?OVERLOAD_KILL_QLEN, overload_kill_mem_size := ?OVERLOAD_KILL_MEM_SIZE, overload_kill_restart_after := ?OVERLOAD_KILL_RESTART_AFTER, - filesync_repeat_interval := ?FILESYNC_REPEAT_INTERVAL, - handler_state := - #{log_opts := #{type := ?DISK_LOG_TYPE, - max_no_files := ?DISK_LOG_MAX_NO_FILES, - max_no_bytes := ?DISK_LOG_MAX_NO_BYTES, - file := DiskLogFile}}} = - logger_disk_log_h:info(?MODULE), + cb_state := + #{handler_state := + #{log_opts := #{type := ?DISK_LOG_TYPE, + max_no_files := ?DISK_LOG_MAX_NO_FILES, + max_no_bytes := ?DISK_LOG_MAX_NO_BYTES, + file := DiskLogFile}}, + filesync_repeat_interval := ?FILESYNC_REPEAT_INTERVAL}} = + logger_olp:info(h_proc_name()), {ok,#{config := #{sync_mode_qlen := ?SYNC_MODE_QLEN, drop_mode_qlen := ?DROP_MODE_QLEN, @@ -527,7 +512,7 @@ reconfig(Config) -> overload_kill_restart_after => infinity, filesync_repeat_interval => no_repeat}, ok = logger:set_handler_config(?MODULE, config, HConfig1), - #{id := ?MODULE, + #{%id := ?MODULE, sync_mode_qlen := 1, drop_mode_qlen := 2, flush_qlen := 3, @@ -538,8 +523,8 @@ reconfig(Config) -> overload_kill_qlen := 100000, overload_kill_mem_size := 10000000, overload_kill_restart_after := infinity, - filesync_repeat_interval := no_repeat} = - logger_disk_log_h:info(?MODULE), + cb_state := #{filesync_repeat_interval := no_repeat}} = + logger_olp:info(h_proc_name()), {ok,#{config:=HConfig1}} = logger:get_handler_config(?MODULE), ok = logger:update_handler_config(?MODULE, config, @@ -577,12 +562,13 @@ reconfig(Config) -> max_no_files => 1, max_no_bytes => 1024, file => File}}), - #{handler_state := - #{log_opts := #{type := halt, - max_no_files := 1, - max_no_bytes := 1024, - file := File}}} = - logger_disk_log_h:info(?MODULE), + #{cb_state := + #{handler_state := + #{log_opts := #{type := halt, + max_no_files := 1, + max_no_bytes := 1024, + file := File}}}} = + logger_olp:info(h_proc_name()), {ok,#{config := #{type := halt, max_no_files := 1, @@ -650,13 +636,8 @@ sync(Config) -> {ok,#{config := HConfig}} = logger:get_handler_config(?MODULE), HConfig1 = HConfig#{filesync_repeat_interval => no_repeat}, ok = logger:update_handler_config(?MODULE, config, HConfig1), - no_repeat = maps:get(filesync_repeat_interval, - logger_disk_log_h:info(?MODULE)), - %% The following timer is to make sure the time from last log - %% ("first") to next ("second") is long enough, so the a flush is - %% triggered by the idle timeout between "fourth" and "fifth". - timer:sleep(?IDLE_DETECT_TIME_MSEC*2), + maps:get(cb_state,logger_olp:info(h_proc_name()))), start_tracer([{logger_disk_log_h,disk_log_write,3}, {disk_log,sync,1}], @@ -666,10 +647,10 @@ sync(Config) -> {disk_log,sync}]), logger:notice("second", ?domain), - timer:sleep(?IDLE_DETECT_TIME_MSEC*2), + timer:sleep(?IDLE_DETECT_TIME*2), logger:notice("third", ?domain), %% wait for automatic disk_log_sync - check_tracer(?IDLE_DETECT_TIME_MSEC*2), + check_tracer(?IDLE_DETECT_TIME*2), try_read_file(Log, {ok,<<"first\nsecond\nthird\n">>}, 1000), @@ -678,14 +659,15 @@ sync(Config) -> WaitT = 4500, OneSync = {logger_h_common,handle_cast,repeated_filesync}, %% receive 1 repeated_filesync per sec - start_tracer([{logger_h_common,handle_cast,2}], + start_tracer([{{logger_h_common,handle_cast,2}, + [{[repeated_filesync,'_'],[],[{message,{caller}}]}]}], [OneSync || _ <- lists:seq(1, trunc(WaitT/SyncInt))]), HConfig2 = HConfig#{filesync_repeat_interval => SyncInt}, ok = logger:update_handler_config(?MODULE, config, HConfig2), SyncInt = maps:get(filesync_repeat_interval, - logger_disk_log_h:info(?MODULE)), + maps:get(cb_state,logger_olp:info(h_proc_name()))), timer:sleep(WaitT), HConfig3 = HConfig#{filesync_repeat_interval => no_repeat}, ok = logger:update_handler_config(?MODULE, config, HConfig3), @@ -803,7 +785,7 @@ disk_log_full(cleanup, _Config) -> dbg:stop_clear(), logger:remove_handler(?MODULE). -disk_log_events(Config) -> +disk_log_events(_Config) -> Node = node(), Log = ?MODULE, ok = logger:add_handler(?MODULE, @@ -860,10 +842,12 @@ write_failure(Config) -> rpc:call(Node, ets, insert, [?TEST_HOOKS_TAB,{tester,self()}]), rpc:call(Node, ?MODULE, set_internal_log, [?MODULE,internal_log]), rpc:call(Node, ?MODULE, set_result, [disk_log_write,ok]), - HState = rpc:call(Node, logger_disk_log_h, info, [?STANDARD_HANDLER]), - ct:pal("LogOpts = ~p", [LogOpts = maps:get(log_opts, - maps:get(handler_state,HState))]), - + HState = rpc:call(Node, logger_olp, info, [h_proc_name(?STANDARD_HANDLER)]), + LogOpts = maps:get(log_opts, + maps:get(handler_state, + maps:get(cb_state,HState))), + ct:pal("LogOpts = ~p", [LogOpts]), + %% ?check and ?check_no_log in this test only check for internal log events ok = log_on_remote_node(Node, "Logged1"), rpc:call(Node, logger_disk_log_h, filesync, [?STANDARD_HANDLER]), @@ -914,15 +898,16 @@ sync_failure(Config) -> rpc:call(Node, ets, insert, [?TEST_HOOKS_TAB,{tester,self()}]), rpc:call(Node, ?MODULE, set_internal_log, [?MODULE,internal_log]), rpc:call(Node, ?MODULE, set_result, [disk_log_sync,ok]), - HState = rpc:call(Node, logger_disk_log_h, info, [?STANDARD_HANDLER]), - LogOpts = maps:get(log_opts, maps:get(handler_state,HState)), + HState = rpc:call(Node, logger_olp, info, [h_proc_name(?STANDARD_HANDLER)]), + LogOpts = maps:get(log_opts, maps:get(handler_state, + maps:get(cb_state,HState))), SyncInt = 500, ok = rpc:call(Node, logger, update_handler_config, [?STANDARD_HANDLER, config, #{filesync_repeat_interval => SyncInt}]), - Info = rpc:call(Node, logger_disk_log_h, info, [?STANDARD_HANDLER]), - SyncInt = maps:get(filesync_repeat_interval, Info), + Info = rpc:call(Node, logger_olp, info, [h_proc_name(?STANDARD_HANDLER)]), + SyncInt = maps:get(filesync_repeat_interval, maps:get(cb_state, Info)), ok = log_on_remote_node(Node, "Logged1"), ?check_no_log, @@ -1198,7 +1183,7 @@ qlen_kill_new(Config) -> receive {'DOWN', MRef, _, _, Info} -> case Info of - {shutdown,{overloaded,?MODULE,QLen,Mem}} -> + {shutdown,{overloaded,QLen,Mem}} -> ct:pal("Terminated with qlen = ~w, mem = ~w", [QLen,Mem]); killed -> ct:pal("Slow shutdown, handler process was killed!", []) @@ -1208,7 +1193,7 @@ qlen_kill_new(Config) -> ok after 5000 -> - Info = logger_disk_log_h:info(?MODULE), + Info = logger_olp:info(h_proc_name()), ct:pal("Handler state = ~p", [Info]), ct:fail("Handler not dead! It should not have survived this!") end. @@ -1235,7 +1220,7 @@ mem_kill_new(Config) -> receive {'DOWN', MRef, _, _, Info} -> case Info of - {shutdown,{overloaded,?MODULE,QLen,Mem}} -> + {shutdown,{overloaded,QLen,Mem}} -> ct:pal("Terminated with qlen = ~w, mem = ~w", [QLen,Mem]); killed -> ct:pal("Slow shutdown, handler process was killed!", []) @@ -1245,7 +1230,7 @@ mem_kill_new(Config) -> ok after 5000 -> - Info = logger_disk_log_h:info(?MODULE), + Info = logger_olp:info(h_proc_name()), ct:pal("Handler state = ~p", [Info]), ct:fail("Handler not dead! It should not have survived this!") end. @@ -1271,7 +1256,7 @@ restart_after(Config) -> ok after 5000 -> - Info1 = logger_std_h:info(?MODULE), + Info1 = logger_olp:info(h_proc_name()), ct:pal("Handler state = ~p", [Info1]), ct:fail("Handler not dead! It should not have survived this!") end, @@ -1295,7 +1280,7 @@ restart_after(Config) -> ok after 5000 -> - Info2 = logger_std_h:info(?MODULE), + Info2 = logger_olp:info(h_proc_name()), ct:pal("Handler state = ~p", [Info2]), ct:fail("Handler not dead! It should not have survived this!") end, @@ -1316,11 +1301,15 @@ handler_requests_under_load(Config) -> flush_qlen => 2000, burst_limit_enable => false}}, ok = logger:update_handler_config(?MODULE, NewHConfig), - Pid = spawn_link(fun() -> send_requests(?MODULE, 1, [{filesync,[]}, - {info,[]}, - {reset,[]}, - {change_config,[]}]) - end), + Pid = spawn_link( + fun() -> send_requests(1,[{logger_disk_log_h,filesync,[?MODULE],[]}, + {logger_olp,info,[h_proc_name()],[]}, + {logger_olp,reset,[h_proc_name()],[]}, + {logger,update_handler_config, + [?MODULE, config, + #{overload_kill_enable => false}], + []}]) + end), Procs = 100, Sent = Procs * send_burst({n,5000}, {spawn,Procs,10}, {chars,79}, notice), Pid ! {self(),finish}, @@ -1332,29 +1321,22 @@ handler_requests_under_load(Config) -> [E || E <- Res, is_tuple(E) andalso (element(1,E) == error)] end, - Errors = [{Req,FindError(Res)} || {Req,Res} <- ReqResult], - NoOfReqs = lists:foldl(fun({_,Res}, N) -> N + length(Res) end, 0, ReqResult), + Errors = [{Func,FindError(Res)} || {_,Func,_,Res} <- ReqResult], + NoOfReqs = lists:foldl(fun({_,_,_,Res}, N) -> N + length(Res) end, + 0, ReqResult), ct:pal("~w requests made. Errors: ~n~p", [NoOfReqs,Errors]), ok = file_delete(Log). handler_requests_under_load(cleanup, _Config) -> ok = stop_handler(?MODULE). -send_requests(HName, TO, Reqs = [{Req,Res}|Rs]) -> +send_requests(TO, Reqs = [{Mod,Func,Args,Res}|Rs]) -> receive {From,finish} -> From ! {self(),Reqs} after TO -> - Result = - case Req of - change_config -> - logger:update_handler_config(HName, logger_disk_log_h, - #{overload_kill_enable => - false}); - Func -> - logger_disk_log_h:Func(HName) - end, - send_requests(HName, TO, Rs ++ [{Req,[Result|Res]}]) + Result = apply(Mod,Func,Args), + send_requests(TO, Rs ++ [{Mod,Func,Args,[Result|Res]}]) end. %%%----------------------------------------------------------------- @@ -1472,15 +1454,6 @@ format(Msg,Tag) -> erlang:display(Error), exit(Error). -remove(Handler, LogName) -> - logger_disk_log_h:remove(Handler, LogName), - HState = #{log_names := Logs} = logger_disk_log_h:info(), - false = maps:is_key(LogName, HState), - false = lists:member(LogName, Logs), - false = logger_config:exist(?LOGGER_TABLE, LogName), - {error,no_such_log} = disk_log:info(LogName), - ok. - start_and_add(Name, Config, LogOpts) -> HConfig = maps:get(config, Config, #{}), HConfig1 = maps:merge(HConfig, LogOpts), @@ -1607,7 +1580,9 @@ start_tracer(Trace,Expected) -> ok. tpl([{M,F,A}|Trace]) -> - {ok,Match} = dbg:tpl(M,F,A,c), + tpl([{{M,F,A},c}|Trace]); +tpl([{{M,F,A},MS}|Trace]) -> + {ok,Match} = dbg:tpl(M,F,A,MS), case lists:keyfind(matched,1,Match) of {_,_,1} -> ok; diff --git a/lib/kernel/test/logger_env_var_SUITE.erl b/lib/kernel/test/logger_env_var_SUITE.erl index e8d1a313dc..9d2ad11be8 100644 --- a/lib/kernel/test/logger_env_var_SUITE.erl +++ b/lib/kernel/test/logger_env_var_SUITE.erl @@ -59,7 +59,8 @@ groups() -> logger_undefined, logger_many_handlers_default_first, logger_many_handlers_default_last, - logger_many_handlers_default_last_broken_filter + logger_many_handlers_default_last_broken_filter, + logger_proxy ]}, {bad,[],[bad_error_logger, bad_level, @@ -541,6 +542,19 @@ logger_many_handlers(Config, Env, LogErr, LogInfo, NumProgress) -> ok. +logger_proxy(Config) -> + %% assume current node runs with default settings + DefOpts = logger_olp:get_opts(logger_proxy), + {ok,_,Node} = setup(Config, + [{logger,[{proxy,#{sync_mode_qlen=>0, + drop_mode_qlen=>2}}]}]), + Expected = DefOpts#{sync_mode_qlen:=0, + drop_mode_qlen:=2}, + Expected = rpc:call(Node,logger_olp,get_opts,[logger_proxy]), + Expected = rpc:call(Node,logger,get_proxy_config,[]), + + ok. + sasl_compatible_false(Config) -> Log = file(Config,?FUNCTION_NAME), {ok,_,Node} = setup(Config, diff --git a/lib/kernel/test/logger_olp_SUITE.erl b/lib/kernel/test/logger_olp_SUITE.erl new file mode 100644 index 0000000000..ea3eec89f5 --- /dev/null +++ b/lib/kernel/test/logger_olp_SUITE.erl @@ -0,0 +1,90 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% +-module(logger_olp_SUITE). + +-compile(export_all). + +-include_lib("kernel/src/logger_olp.hrl"). + +suite() -> + [{timetrap,{seconds,30}}]. + +init_per_suite(Config) -> + Config. + +end_per_suite(_Config) -> + ok. + +init_per_group(_Group, Config) -> + Config. + +end_per_group(_Group, _Config) -> + ok. + +init_per_testcase(_TestCase, Config) -> + Config. + +end_per_testcase(Case, Config) -> + try apply(?MODULE,Case,[cleanup,Config]) + catch error:undef -> ok + end, + ok. + +groups() -> + []. + +all() -> + [idle_timer]. + +%%%----------------------------------------------------------------- +%%% Test cases +idle_timer(_Config) -> + {ok,_Pid,Olp} = logger_olp:start_link(?MODULE,?MODULE,self(),#{}), + [logger_olp:load(Olp,{msg,N}) || N<-lists:seq(1,3)], + timer:sleep(?IDLE_DETECT_TIME*2), + [{load,{msg,1}}, + {load,{msg,2}}, + {load,{msg,3}}, + {notify,idle}] = test_server:messages_get(), + logger_olp:cast(Olp,hello), + timer:sleep(?IDLE_DETECT_TIME*2), + [{cast,hello}] = test_server:messages_get(), + ok. +idle_timer(cleanup,_Config) -> + unlink(whereis(?MODULE)), + logger_olp:stop(?MODULE), + ok. + +%%%----------------------------------------------------------------- +%%% Olp callbacks +init(P) -> + {ok,P}. + +handle_load(M,P) -> + P ! {load,M}, + P. + +handle_cast(M,P) -> + P ! {cast,M}, + {noreply,P}. + +notify(N,P) -> + P ! {notify,N}, + P. diff --git a/lib/kernel/test/logger_proxy_SUITE.erl b/lib/kernel/test/logger_proxy_SUITE.erl new file mode 100644 index 0000000000..777531e4ed --- /dev/null +++ b/lib/kernel/test/logger_proxy_SUITE.erl @@ -0,0 +1,274 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% +-module(logger_proxy_SUITE). + +-compile(export_all). + +%% -include_lib("common_test/include/ct.hrl"). +%% -include_lib("kernel/include/logger.hrl"). +%% -include_lib("kernel/src/logger_internal.hrl"). + +%% -define(str,"Log from "++atom_to_list(?FUNCTION_NAME)++ +%% ":"++integer_to_list(?LINE)). +%% -define(map_rep,#{function=>?FUNCTION_NAME, line=>?LINE}). +%% -define(keyval_rep,[{function,?FUNCTION_NAME}, {line,?LINE}]). + +%% -define(MY_LOC(N),#{mfa=>{?MODULE,?FUNCTION_NAME,?FUNCTION_ARITY}, +%% file=>?FILE, line=>?LINE-N}). + +%% -define(TRY(X), my_try(fun() -> X end)). + + +-define(HNAME,list_to_atom(lists:concat([?MODULE,"_",?FUNCTION_NAME]))). +-define(LOC,#{mfa=>{?MODULE,?FUNCTION_NAME,?FUNCTION_ARITY},line=>?LINE}). +-define(ENSURE_TIME,5000). + +suite() -> + [{timetrap,{seconds,30}}, + {ct_hooks,[logger_test_lib]}]. + +init_per_suite(Config) -> + Config. + +end_per_suite(_Config) -> + ok. + +init_per_group(_Group, Config) -> + Config. + +end_per_group(_Group, _Config) -> + ok. + +init_per_testcase(_TestCase, Config) -> + Config. + +end_per_testcase(Case, Config) -> + try apply(?MODULE,Case,[cleanup,Config]) + catch error:undef -> ok + end, + ok. + +groups() -> + []. + +all() -> + [basic, + emulator, + remote, + remote_emulator, + config, + restart_after, + terminate]. + +%%%----------------------------------------------------------------- +%%% Test cases +basic(_Config) -> + ok = logger:add_handler(?HNAME,?MODULE,#{config=>self()}), + logger_proxy ! {log,notice,"Log from: ~p; ~p",[?FUNCTION_NAME,?LINE],L1=?LOC}, + ok = ensure(L1), + logger_proxy ! {log,notice,[{test_case,?FUNCTION_NAME},{line,?LINE}],L2=?LOC}, + ok = ensure(L2), + logger_proxy:log({remote,node(),{log,notice, + "Log from: ~p; ~p", + [?FUNCTION_NAME,?LINE], + L3=?LOC}}), + ok = ensure(L3), + logger_proxy:log({remote,node(),{log,notice, + [{test_case,?FUNCTION_NAME}, + {line,?LINE}], + L4=?LOC}}), + ok = ensure(L4), + ok. +basic(cleanup,_Config) -> + ok = logger:remove_handler(?HNAME). + +emulator(_Config) -> + ok = logger:add_handler(?HNAME,?MODULE,#{config=>self()}), + Pid = spawn(fun() -> erlang:error(some_reason) end), + ok = ensure(#{pid=>Pid}), + ok. +emulator(cleanup,_Config) -> + ok = logger:remove_handler(?HNAME). + +remote(Config) -> + {ok,_,Node} = logger_test_lib:setup(Config,[{logger,[{proxy,#{}}]}]), + ok = logger:add_handler(?HNAME,?MODULE,#{config=>self()}), + L1 = ?LOC, spawn(Node,fun() -> logger:notice("Log from ~p; ~p",[?FUNCTION_NAME,?LINE],L1) end), + ok = ensure(L1), + L2 = ?LOC, spawn(Node,fun() -> logger:notice([{test_case,?FUNCTION_NAME},{line,?LINE}],L2) end), + ok = ensure(L2), + ok. +remote(cleanup,_Config) -> + ok = logger:remove_handler(?HNAME). + +remote_emulator(Config) -> + {ok,_,Node} = logger_test_lib:setup(Config,[{logger,[{proxy,#{}}]}]), + ok = logger:add_handler(?HNAME,?MODULE,#{config=>self()}), + Pid = spawn(Node,fun() -> erlang:error(some_reason) end), + ok = ensure(#{pid=>Pid}), + ok. +remote_emulator(cleanup,_Config) -> + ok = logger:remove_handler(?HNAME). + +config(_Config) -> + C1 = #{sync_mode_qlen:=SQ, + drop_mode_qlen:=DQ} = logger:get_proxy_config(), + C1 = logger_olp:get_opts(logger_proxy), + + %% Update the existing config with these two values + SQ1 = SQ+1, + DQ1 = DQ+1, + ok = logger:update_proxy_config(#{sync_mode_qlen=>SQ1, + drop_mode_qlen=>DQ1}), + C2 = logger:get_proxy_config(), % reads from ets table + C2 = logger_olp:get_opts(logger_proxy), % ensure consistency with process opts + C2 = C1#{sync_mode_qlen:=SQ1, + drop_mode_qlen:=DQ1}, + + %% Update the existing again with only one value + SQ2 = SQ+2, + ok = logger:update_proxy_config(#{sync_mode_qlen=>SQ2}), + C3 = logger:get_proxy_config(), + C3 = logger_olp:get_opts(logger_proxy), + C3 = C2#{sync_mode_qlen:=SQ2}, + + %% Set the config, i.e. merge with defaults + ok = logger:set_proxy_config(#{sync_mode_qlen=>SQ1}), + C4 = logger:get_proxy_config(), + C4 = logger_olp:get_opts(logger_proxy), + C4 = C1#{sync_mode_qlen:=SQ1}, + + %% Reset to default + ok = logger:set_proxy_config(#{}), + C5 = logger:get_proxy_config(), + C5 = logger_olp:get_opts(logger_proxy), + C5 = logger_proxy:get_default_config(), + + %% Errors + {error,{invalid_olp_config,_}} = + logger:set_proxy_config(#{faulty_key=>1}), + {error,{invalid_olp_config,_}} = + logger:set_proxy_config(#{sync_mode_qlen=>infinity}), + {error,{invalid_config,[]}} = logger:set_proxy_config([]), + + {error,{invalid_olp_config,_}} = + logger:update_proxy_config(#{faulty_key=>1}), + {error,{invalid_olp_config,_}} = + logger:update_proxy_config(#{sync_mode_qlen=>infinity}), + {error,{invalid_config,[]}} = logger:update_proxy_config([]), + + C5 = logger:get_proxy_config(), + C5 = logger_olp:get_opts(logger_proxy), + + ok. +config(cleanup,_Config) -> + _ = logger:set_logger_proxy(logger_proxy:get_default_config()), + ok. + +restart_after(_Config) -> + Restart = 3000, + ok = logger:update_proxy_config(#{overload_kill_enable => true, + overload_kill_qlen => 10, + overload_kill_restart_after => Restart}), + Proxy = whereis(logger_proxy), + Proxy = erlang:system_info(system_logger), + ProxyConfig = logger:get_proxy_config(), + ProxyConfig = logger_olp:get_opts(logger_proxy), + + Ref = erlang:monitor(process,Proxy), + spawn(fun() -> + [logger_proxy ! {log,debug, + [{test_case,?FUNCTION_NAME}, + {line,?LINE}], + ?LOC} || _ <- lists:seq(1,100)] + end), + receive + {'DOWN',Ref,_,_,_Reason} -> + undefined = erlang:system_info(system_logger), + timer:sleep(Restart), + poll_restarted(10) + after 5000 -> + ct:fail(proxy_not_terminated) + end, + + Proxy1 = whereis(logger_proxy), + Proxy1 = erlang:system_info(system_logger), + ProxyConfig = logger:get_proxy_config(), + ProxyConfig = logger_olp:get_opts(logger_proxy), + + ok. +restart_after(cleanup,_Config) -> + _ = logger:set_logger_proxy(logger_proxy:get_default_config()), + ok. + +%% Test that system_logger flag is set to logger process if +%% logger_proxy terminates for other reason than overloaded. +terminate(_Config) -> + Logger = whereis(logger), + Proxy = whereis(logger_proxy), + Proxy = erlang:system_info(system_logger), + ProxyConfig = logger:get_proxy_config(), + ProxyConfig = logger_olp:get_opts(logger_proxy), + + Ref = erlang:monitor(process,Proxy), + ok = logger_olp:stop(Proxy), + receive + {'DOWN',Ref,_,_,_Reason} -> + Logger = erlang:system_info(system_logger), + logger_proxy:restart(), + poll_restarted(10) + after 5000 -> + ct:fail(proxy_not_terminated) + end, + + Proxy1 = whereis(logger_proxy), + Proxy1 = erlang:system_info(system_logger), + ProxyConfig = logger:get_proxy_config(), + ProxyConfig = logger_olp:get_opts(logger_proxy), + + ok. + +%%%----------------------------------------------------------------- +%%% Internal functions + +poll_restarted(0) -> + ct:fail(proxy_not_restarted); +poll_restarted(N) -> + timer:sleep(1000), + case whereis(logger_proxy) of + undefined -> + poll_restarted(N-1); + _Pid -> + ok + end. + +%% Logger handler callback +log(#{meta:=Meta},#{config:=Pid}) -> + Pid ! {logged,Meta}. + +%% Check that the log from the logger callback function log/2 is received +ensure(Match) -> + receive {logged,Meta} -> + case maps:with(maps:keys(Match),Meta) of + Match -> ok; + _NoMatch -> {error,Match,Meta,test_server:messages_get()} + end + after ?ENSURE_TIME -> {error,Match,test_server:messages_get()} + end. diff --git a/lib/kernel/test/logger_std_h_SUITE.erl b/lib/kernel/test/logger_std_h_SUITE.erl index eb17a6d857..b2c2c8ba67 100644 --- a/lib/kernel/test/logger_std_h_SUITE.erl +++ b/lib/kernel/test/logger_std_h_SUITE.erl @@ -25,10 +25,15 @@ -include_lib("kernel/include/logger.hrl"). -include_lib("kernel/src/logger_internal.hrl"). -include_lib("kernel/src/logger_h_common.hrl"). +-include_lib("kernel/src/logger_olp.hrl"). -include_lib("stdlib/include/ms_transform.hrl"). -include_lib("kernel/include/file.hrl"). --define(check_no_log, [] = test_server:messages_get()). +-define(check_no_log, + begin + timer:sleep(?IDLE_DETECT_TIME*2), + [] = test_server:messages_get() + end). -define(check(Expected), receive {log,Expected} -> @@ -115,7 +120,6 @@ all() -> crash_std_h_to_file, crash_std_h_to_disk_log, bad_input, - info_and_reset, reconfig, file_opts, sync, @@ -137,7 +141,8 @@ all() -> mem_kill_std, restart_after, handler_requests_under_load, - recreate_deleted_log + recreate_deleted_log, + reopen_changed_log ]. add_remove_instance_tty(_Config) -> @@ -209,9 +214,9 @@ default_formatter(_Config) -> filter_config(_Config) -> ok = logger:add_handler(?MODULE,logger_std_h,#{}), {ok,#{config:=HConfig}=Config} = logger:get_handler_config(?MODULE), - HConfig = maps:without([handler_pid,mode_tab],HConfig), + HConfig = maps:without([olp],HConfig), - FakeFullHConfig = HConfig#{handler_pid=>self(),mode_tab=>erlang:make_ref()}, + FakeFullHConfig = HConfig#{olp=>{regname,self(),erlang:make_ref()}}, #{config:=HConfig} = logger_std_h:filter_config(Config#{config=>FakeFullHConfig}), ok. @@ -246,13 +251,13 @@ errors(Config) -> _ -> NoDir = lists:concat(["/",?MODULE,"_dir"]), {error, - {handler_not_added,{{open_failed,NoDir,eacces},_}}} = + {handler_not_added,{open_failed,NoDir,eacces}}} = logger:add_handler(myh2,logger_std_h, #{config=>#{type=>{file,NoDir}}}) end, {error, - {handler_not_added,{{open_failed,Log,_},_}}} = + {handler_not_added,{open_failed,Log,_}}} = logger:add_handler(myh3,logger_std_h, #{config=>#{type=>{file,Log,[bad_file_opt]}}}), @@ -320,19 +325,16 @@ config_fail(_Config) -> #{config => #{restart_type => bad}, filter_default=>log, formatter=>{?MODULE,self()}}), - {error,{handler_not_added,{invalid_config,logger_std_h, - {invalid_levels,#{drop_mode_qlen:=1}}}}} = + {error,{handler_not_added,{invalid_olp_levels,#{drop_mode_qlen:=1}}}} = logger:add_handler(?MODULE,logger_std_h, #{config => #{drop_mode_qlen=>1}}), - {error,{handler_not_added,{invalid_config,logger_std_h, - {invalid_levels,#{sync_mode_qlen:=43, - drop_mode_qlen:=42}}}}} = + {error,{handler_not_added,{invalid_olp_levels,#{sync_mode_qlen:=43, + drop_mode_qlen:=42}}}} = logger:add_handler(?MODULE,logger_std_h, #{config => #{sync_mode_qlen=>43, drop_mode_qlen=>42}}), - {error,{handler_not_added,{invalid_config,logger_std_h, - {invalid_levels,#{drop_mode_qlen:=43, - flush_qlen:=42}}}}} = + {error,{handler_not_added,{invalid_olp_levels,#{drop_mode_qlen:=43, + flush_qlen:=42}}}} = logger:add_handler(?MODULE,logger_std_h, #{config => #{drop_mode_qlen=>43, flush_qlen=>42}}), @@ -344,7 +346,7 @@ config_fail(_Config) -> logger:set_handler_config(?MODULE,config, #{type=>{file,"file"}}), - {error,{invalid_config,logger_std_h,{invalid_levels,_}}} = + {error,{invalid_olp_levels,_}} = logger:set_handler_config(?MODULE,config, #{sync_mode_qlen=>100, flush_qlen=>99}), @@ -355,9 +357,7 @@ config_fail(_Config) -> %% Read-only fields may (accidentially) be included in the change, %% but it won't take effect {ok,C} = logger:get_handler_config(?MODULE), - ok = logger:set_handler_config(?MODULE,config, - #{handler_pid=>self(), - mode_tab=>erlang:make_ref()}), + ok = logger:set_handler_config(?MODULE,config,#{olp=>dummyvalue}), {ok,C} = logger:get_handler_config(?MODULE), ok. @@ -425,10 +425,13 @@ crash_std_h(Config,Func,Var,Type,Log) -> %% logger would send the log event to the logger process here instead %% of logging it itself. log_on_remote_node(Node,Msg) -> + Pid = self(), _ = spawn_link(Node, fun() -> erlang:group_leader(whereis(user),self()), - logger:notice(Msg) + logger:notice(Msg), + Pid ! done end), + receive done -> ok end, ok. @@ -456,14 +459,7 @@ sync_and_read(Node,file,Log) -> end. bad_input(_Config) -> - {error,{badarg,{filesync,["BadType"]}}} = logger_std_h:filesync("BadType"), - {error,{badarg,{info,["BadType"]}}} = logger_std_h:info("BadType"), - {error,{badarg,{reset,["BadType"]}}} = logger_std_h:reset("BadType"). - - -info_and_reset(_Config) -> - #{id := ?STANDARD_HANDLER} = logger_std_h:info(?STANDARD_HANDLER), - ok = logger_std_h:reset(?STANDARD_HANDLER). + {error,{badarg,{filesync,["BadType"]}}} = logger_std_h:filesync("BadType"). reconfig(Config) -> Dir = ?config(priv_dir,Config), @@ -473,9 +469,10 @@ reconfig(Config) -> filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}), - #{id := ?MODULE, - handler_state := #{type := standard_io, - file_ctrl_pid := FileCtrlPid}, + #{%id := ?MODULE, + cb_state:=#{handler_state := #{type := standard_io, + file_ctrl_pid := FileCtrlPid}, + filesync_repeat_interval := no_repeat}, sync_mode_qlen := ?SYNC_MODE_QLEN, drop_mode_qlen := ?DROP_MODE_QLEN, flush_qlen := ?FLUSH_QLEN, @@ -485,9 +482,8 @@ reconfig(Config) -> overload_kill_enable := ?OVERLOAD_KILL_ENABLE, overload_kill_qlen := ?OVERLOAD_KILL_QLEN, overload_kill_mem_size := ?OVERLOAD_KILL_MEM_SIZE, - overload_kill_restart_after := ?OVERLOAD_KILL_RESTART_AFTER, - filesync_repeat_interval := no_repeat} = DefaultInfo = - logger_std_h:info(?MODULE), + overload_kill_restart_after := ?OVERLOAD_KILL_RESTART_AFTER} = + logger_olp:info(h_proc_name()), {ok, #{config:= @@ -518,9 +514,10 @@ reconfig(Config) -> overload_kill_mem_size => 10000000, overload_kill_restart_after => infinity, filesync_repeat_interval => 5000}), - #{id := ?MODULE, - handler_state := #{type := standard_io, - file_ctrl_pid := FileCtrlPid}, + #{%id := ?MODULE, + cb_state := #{handler_state := #{type := standard_io, + file_ctrl_pid := FileCtrlPid}, + filesync_repeat_interval := no_repeat}, sync_mode_qlen := 1, drop_mode_qlen := 2, flush_qlen := 3, @@ -530,8 +527,7 @@ reconfig(Config) -> overload_kill_enable := true, overload_kill_qlen := 100000, overload_kill_mem_size := 10000000, - overload_kill_restart_after := infinity, - filesync_repeat_interval := no_repeat} = Info = logger_std_h:info(?MODULE), + overload_kill_restart_after := infinity} = logger_olp:info(h_proc_name()), {ok,#{config := #{type := standard_io, @@ -613,7 +609,7 @@ file_opts(Config) -> Log = filename:join(Dir, lists:concat([?FUNCTION_NAME,".log"])), BadFileOpts = [raw], BadType = {file,Log,BadFileOpts}, - {error,{handler_not_added,{{open_failed,Log,enoent},_}}} = + {error,{handler_not_added,{open_failed,Log,enoent}}} = logger:add_handler(?MODULE, logger_std_h, #{config => #{type => BadType}}), @@ -626,7 +622,9 @@ file_opts(Config) -> filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}), - #{handler_state := #{type := OkType}} = logger_std_h:info(?MODULE), + #{cb_state := #{handler_state := #{type := OkType}}} = + logger_olp:info(h_proc_name()), + {ok,#{config := #{type := OkType}}} = logger:get_handler_config(?MODULE), logger:notice(M1=?msg,?domain), ?check(M1), B1 = ?bin(M1), @@ -675,11 +673,8 @@ sync(Config) -> %% a filesync is still performed when handler goes idle ok = logger:update_handler_config(?MODULE, config, #{filesync_repeat_interval => no_repeat}), - no_repeat = maps:get(filesync_repeat_interval, logger_std_h:info(?MODULE)), - %% The following timer is to make sure the time from last log - %% ("second") to next ("third") is long enough, so the a flush is - %% triggered by the idle timeout between "thrid" and "fourth". - timer:sleep(?IDLE_DETECT_TIME_MSEC*2), + no_repeat = maps:get(filesync_repeat_interval, + maps:get(cb_state, logger_olp:info(h_proc_name()))), start_tracer([{logger_std_h, write_to_dev, 5}, {file, datasync, 1}], [{logger_std_h, write_to_dev, <<"third\n">>}, @@ -688,22 +683,24 @@ sync(Config) -> {file,datasync}]), logger:notice("third", ?domain), %% wait for automatic filesync - timer:sleep(?IDLE_DETECT_TIME_MSEC*2), + timer:sleep(?IDLE_DETECT_TIME*2), logger:notice("fourth", ?domain), %% wait for automatic filesync - check_tracer(?IDLE_DETECT_TIME_MSEC*2), + check_tracer(?IDLE_DETECT_TIME*2), %% switch repeated filesync on and verify that the looping works SyncInt = 1000, WaitT = 4500, OneSync = {logger_h_common,handle_cast,repeated_filesync}, %% receive 1 repeated_filesync per sec - start_tracer([{logger_h_common,handle_cast,2}], + start_tracer([{{logger_h_common,handle_cast,2}, + [{[repeated_filesync,'_'],[],[]}]}], [OneSync || _ <- lists:seq(1, trunc(WaitT/SyncInt))]), ok = logger:update_handler_config(?MODULE, config, #{filesync_repeat_interval => SyncInt}), - SyncInt = maps:get(filesync_repeat_interval, logger_std_h:info(?MODULE)), + SyncInt = maps:get(filesync_repeat_interval, + maps:get(cb_state,logger_olp:info(h_proc_name()))), timer:sleep(WaitT), ok = logger:update_handler_config(?MODULE, config, #{filesync_repeat_interval => no_repeat}), @@ -764,8 +761,6 @@ sync_failure(Config) -> ok = rpc:call(Node, logger, update_handler_config, [?STANDARD_HANDLER, config, #{filesync_repeat_interval => SyncInt}]), - Info = rpc:call(Node, logger_std_h, info, [?STANDARD_HANDLER]), - SyncInt = maps:get(filesync_repeat_interval, Info), ok = log_on_remote_node(Node, "Logged1"), ?check_no_log, @@ -1095,7 +1090,7 @@ qlen_kill_new(Config) -> receive {'DOWN', MRef, _, _, Info} -> case Info of - {shutdown,{overloaded,?MODULE,QLen,Mem}} -> + {shutdown,{overloaded,QLen,Mem}} -> ct:pal("Terminated with qlen = ~w, mem = ~w", [QLen,Mem]); killed -> ct:pal("Slow shutdown, handler process was killed!", []) @@ -1105,7 +1100,7 @@ qlen_kill_new(Config) -> ok after 5000 -> - Info = logger_std_h:info(?MODULE), + Info = logger_olp:info(h_proc_name()), ct:pal("Handler state = ~p", [Info]), ct:fail("Handler not dead! It should not have survived this!") end. @@ -1146,7 +1141,7 @@ mem_kill_new(Config) -> receive {'DOWN', MRef, _, _, Info} -> case Info of - {shutdown,{overloaded,?MODULE,QLen,Mem}} -> + {shutdown,{overloaded,QLen,Mem}} -> ct:pal("Terminated with qlen = ~w, mem = ~w", [QLen,Mem]); killed -> ct:pal("Slow shutdown, handler process was killed!", []) @@ -1156,7 +1151,7 @@ mem_kill_new(Config) -> ok after 5000 -> - Info = logger_std_h:info(?MODULE), + Info = logger_olp:info(h_proc_name()), ct:pal("Handler state = ~p", [Info]), ct:fail("Handler not dead! It should not have survived this!") end. @@ -1187,7 +1182,7 @@ restart_after(Config) -> ok after 5000 -> - Info1 = logger_std_h:info(?MODULE), + Info1 = logger_olp:info(h_proc_name()), ct:pal("Handler state = ~p", [Info1]), ct:fail("Handler not dead! It should not have survived this!") end, @@ -1212,7 +1207,7 @@ restart_after(Config) -> ok after 5000 -> - Info2 = logger_std_h:info(?MODULE), + Info2 = logger_olp:info(h_proc_name()), ct:pal("Handler state = ~p", [Info2]), ct:fail("Handler not dead! It should not have survived this!") end, @@ -1234,11 +1229,15 @@ handler_requests_under_load(Config) -> flush_qlen => 2000, burst_limit_enable => false}}, ok = logger:update_handler_config(?MODULE, NewHConfig), - Pid = spawn_link(fun() -> send_requests(?MODULE, 1, [{filesync,[]}, - {info,[]}, - {reset,[]}, - {change_config,[]}]) - end), + Pid = spawn_link( + fun() -> send_requests(1,[{logger_std_h,filesync,[?MODULE],[]}, + {logger_olp,info,[h_proc_name()],[]}, + {logger_olp,reset,[h_proc_name()],[]}, + {logger,update_handler_config, + [?MODULE, config, + #{overload_kill_enable => false}], + []}]) + end), Sent = send_burst({t,10000}, seq, {chars,79}, notice), Pid ! {self(),finish}, ReqResult = receive {Pid,Result} -> Result end, @@ -1249,8 +1248,9 @@ handler_requests_under_load(Config) -> [E || E <- Res, is_tuple(E) andalso (element(1,E) == error)] end, - Errors = [{Req,FindError(Res)} || {Req,Res} <- ReqResult], - NoOfReqs = lists:foldl(fun({_,Res}, N) -> N + length(Res) end, 0, ReqResult), + Errors = [{Func,FindError(Res)} || {_,Func,_,Res} <- ReqResult], + NoOfReqs = lists:foldl(fun({_,_,_,Res}, N) -> N + length(Res) end, + 0, ReqResult), ct:pal("~w requests made. Errors: ~n~p", [NoOfReqs,Errors]), ok = file_delete(Log). handler_requests_under_load(cleanup, _Config) -> @@ -1270,24 +1270,31 @@ recreate_deleted_log(Config) -> recreate_deleted_log(cleanup, _Config) -> ok = stop_handler(?MODULE). +reopen_changed_log(Config) -> + {Log,_HConfig,_StdHConfig} = + start_handler(?MODULE, ?FUNCTION_NAME, Config), + logger:notice("first",?domain), + logger_std_h:filesync(?MODULE), + ok = file:rename(Log,Log++".old"), + ok = file:write_file(Log,""), + logger:notice("second",?domain), + logger_std_h:filesync(?MODULE), + {ok,<<"first\n">>} = file:read_file(Log++".old"), + {ok,<<"second\n">>} = file:read_file(Log), + ok. +reopen_changed_log(cleanup, _Config) -> + ok = stop_handler(?MODULE). + %%%----------------------------------------------------------------- %%% -send_requests(HName, TO, Reqs = [{Req,Res}|Rs]) -> +send_requests(TO, Reqs = [{Mod,Func,Args,Res}|Rs]) -> receive {From,finish} -> From ! {self(),Reqs} after TO -> - Result = - case Req of - change_config -> - logger:update_handler_config(HName, config, - #{overload_kill_enable => - false}); - Func -> - logger_std_h:Func(HName) - end, - send_requests(HName, TO, Rs ++ [{Req,[Result|Res]}]) + Result = apply(Mod,Func,Args), + send_requests(TO, Rs ++ [{Mod,Func,Args,[Result|Res]}]) end. @@ -1624,7 +1631,8 @@ start_tracer(Trace,Expected) -> Pid = self(), FileCtrlPid = maps:get(file_ctrl_pid, maps:get(handler_state, - logger_std_h:info(?MODULE))), + maps:get(cb_state, + logger_olp:info(h_proc_name())))), dbg:tracer(process,{fun tracer/2,{Pid,Expected}}), dbg:p(whereis(h_proc_name()),[c]), dbg:p(FileCtrlPid,[c]), @@ -1632,7 +1640,9 @@ start_tracer(Trace,Expected) -> ok. tpl([{M,F,A}|Trace]) -> - {ok,Match} = dbg:tpl(M,F,A,[]), + tpl([{{M,F,A},[]}|Trace]); +tpl([{{M,F,A},MS}|Trace]) -> + {ok,Match} = dbg:tpl(M,F,A,MS), case lists:keyfind(matched,1,Match) of {_,_,1} -> ok; diff --git a/lib/kernel/test/logger_stress_SUITE.erl b/lib/kernel/test/logger_stress_SUITE.erl new file mode 100644 index 0000000000..4072e8c86a --- /dev/null +++ b/lib/kernel/test/logger_stress_SUITE.erl @@ -0,0 +1,456 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% +-module(logger_stress_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct_event.hrl"). +-include_lib("kernel/include/logger.hrl"). +-include_lib("kernel/src/logger_h_common.hrl"). + +-ifdef(SAVE_STATS). + -define(COLLECT_STATS(_All_,_Procs_), + ct:pal("~p",[stats(_All_,_Procs_)])). +-else. + -define(COLLECT_STATS(_All_,_Procs__), ok). +-endif. + +-define(TEST_DURATION,120). % seconds + +suite() -> + [{timetrap,{minutes,3}}, + {ct_hooks,[logger_test_lib]}]. + +init_per_suite(Config) -> + Config. + +end_per_suite(_Config) -> + ok. + +init_per_group(_Group, Config) -> + Config. + +end_per_group(_Group, _Config) -> + ok. + +init_per_testcase(_TestCase, Config) -> + Config. + +end_per_testcase(Case, Config) -> + try apply(?MODULE,Case,[cleanup,Config]) + catch error:undef -> ok + end, + ok. + +groups() -> + []. + +all() -> + [allow_events, + reject_events, + std_handler, + disk_log_handler, + emulator_events, + remote_events, + remote_to_disk_log, + remote_emulator_events, + remote_emulator_to_disk_log]. + +%%%----------------------------------------------------------------- +%%% Test cases +%%%----------------------------------------------------------------- +%% Time from log macro call to handler callback +allow_events(Config) -> + {ok,_,Node} = + logger_test_lib:setup(Config, + [{logger, + [{handler,default,?MODULE,#{}}]}, + {logger_level,notice}]), + N = 100000, + {T,_} = timer:tc(fun() -> rpc:call(Node,?MODULE,nlogs,[N]) end), + IOPS = N * 1000/T, % log events allowed per millisecond + ct_event:notify(#event{name = benchmark_data, + data = [{value,IOPS}]}), + {comment,io_lib:format("~.2f accepted events pr millisecond", + [IOPS])}. + +%% Time from log macro call to reject (log level) +reject_events(Config) -> + {ok,_,Node} = + logger_test_lib:setup(Config, + [{logger, + [{handler,default,?MODULE,#{}}]}, + {logger_level,error}]), + N = 1000000, + {T,_} = timer:tc(fun() -> rpc:call(Node,?MODULE,nlogs,[N]) end), + IOPS = N * 1000/T, % log events rejected per millisecond + ct_event:notify(#event{name = benchmark_data, + data = [{value,IOPS}]}), + {comment,io_lib:format("~.2f rejected events pr millisecond", + [IOPS])}. + +%% Cascading failure that produce gen_server and proc_lib reports - +%% how many of the produced log events are actually written to a log +%% with logger_std_h file handler. +std_handler(Config) -> + {ok,_,Node} = + logger_test_lib:setup(Config, + [{logger, + [{handler,default,logger_std_h, + #{config=>#{type=>{file,"default.log"}}}}]}]), + + cascade({Node,{logger_backend,log_allowed,2},[]}, + {Node,{logger_std_h,write,4},[{default,logger_std_h_default}]}, + fun otp_cascading/0). +std_handler(cleanup,_Config) -> + _ = file:delete("default.log"), + ok. + +%% Cascading failure that produce gen_server and proc_lib reports - +%% how many of the produced log events are actually written to a log +%% with logger_disk_log_h wrap file handler. +disk_log_handler(Config) -> + {ok,_,Node} = + logger_test_lib:setup(Config, + [{logger, + [{handler,default,logger_disk_log_h,#{}}]}]), + cascade({Node,{logger_backend,log_allowed,2},[]}, + {Node,{logger_disk_log_h,write,4}, + [{default,logger_disk_log_h_default}]}, + fun otp_cascading/0). +disk_log_handler(cleanup,_Config) -> + Files = filelib:wildcard("default.log.*"), + [_ = file:delete(F) || F <- Files], + ok. + +%% Cascading failure that produce log events from the emulator - how +%% many of the produced log events pass through the proxy. +emulator_events(Config) -> + {ok,_,Node} = + logger_test_lib:setup(Config, + [{logger, + [{handler,default,?MODULE,#{}}]}]), + cascade({Node,{?MODULE,producer,0},[]}, + {Node,{?MODULE,log,2},[{proxy,logger_proxy}]}, + fun em_cascading/0). + +%% Cascading failure that produce gen_server and proc_lib reports on +%% remote node - how many of the produced log events pass through the +%% proxy. +remote_events(Config) -> + {ok,_,Node1} = + logger_test_lib:setup([{postfix,1}|Config], + [{logger, + [{handler,default,?MODULE,#{}}]}]), + {ok,_,Node2} = + logger_test_lib:setup([{postfix,2}|Config],[]), + cascade({Node2,{logger_backend,log_allowed,2},[{remote_proxy,logger_proxy}]}, + {Node1,{?MODULE,log,2},[{local_proxy,logger_proxy}]}, + fun otp_cascading/0). + +%% Cascading failure that produce gen_server and proc_lib reports on +%% remote node - how many of the produced log events are actually +%% written to a log with logger_disk_log_h wrap file handler. +remote_to_disk_log(Config) -> + {ok,_,Node1} = + logger_test_lib:setup([{postfix,1}|Config], + [{logger, + [{handler,default,logger_disk_log_h,#{}}]}]), + {ok,_,Node2} = + logger_test_lib:setup([{postfix,2}|Config],[]), + cascade({Node2,{logger_backend,log_allowed,2},[{remote_proxy,logger_proxy}]}, + {Node1,{logger_disk_log_h,write,4}, + [{local_proxy,logger_proxy}, + {local_default,logger_disk_log_h_default}]}, + fun otp_cascading/0). +remote_to_disk_log(cleanup,_Config) -> + Files = filelib:wildcard("default.log.*"), + [_ = file:delete(F) || F <- Files], + ok. + +%% Cascading failure that produce log events from the emulator on +%% remote node - how many of the produced log events pass through the +%% proxy. +remote_emulator_events(Config) -> + {ok,_,Node1} = + logger_test_lib:setup([{postfix,1}|Config], + [{logger, + [{handler,default,?MODULE,#{}}]}]), + {ok,_,Node2} = + logger_test_lib:setup([{postfix,2}|Config],[]), + cascade({Node2,{?MODULE,producer,0},[{remote_proxy,logger_proxy}]}, + {Node1,{?MODULE,log,2},[{local_proxy,logger_proxy}]}, + fun em_cascading/0). + +%% Cascading failure that produce log events from the emulator on +%% remote node - how many of the produced log events are actually +%% written to a log with logger_disk_log_h wrap file handler. +remote_emulator_to_disk_log(Config) -> + {ok,_,Node1} = + logger_test_lib:setup([{postfix,1}|Config], + [{logger, + [{handler,default,logger_disk_log_h,#{}}]}]), + {ok,_,Node2} = + logger_test_lib:setup([{postfix,2}|Config],[]), + cascade({Node2,{?MODULE,producer,0},[{remote_proxy,logger_proxy}]}, + {Node1,{logger_disk_log_h,write,4}, + [{local_proxy,logger_proxy}, + {local_default,logger_disk_log_h_default}]}, + fun em_cascading/0). +remote_emulator_to_disk_log(cleanup,_Config) -> + Files = filelib:wildcard("default.log.*"), + [_ = file:delete(F) || F <- Files], + ok. + +%%%----------------------------------------------------------------- +%%% Internal functions +nlogs(N) -> + group_leader(whereis(user),self()), + Str = "\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "[\\]^_`abcdefghijklmnopqr", + [?LOG_NOTICE(Str) || _ <- lists:seq(1,N)], + ok. + +%% cascade(ProducerInfo,ConsumerInfo,TestFun) +cascade({PNode,PMFA,_PStatProcs},{CNode,CMFA,_CStatProcs},TestFun) -> + Tab = ets:new(counter,[set,public]), + ets:insert(Tab,{producer,0}), + ets:insert(Tab,{consumer,0}), + dbg:tracer(process,{fun tracer/2,{Tab,PNode,CNode}}), + dbg:n(PNode), + dbg:n(CNode), + dbg:cn(node()), + dbg:p(all,[call,arity]), + dbg:tpl(PMFA,[]), + dbg:tpl(CMFA,[]), + + Pid = rpc:call(CNode,?MODULE,wrap_test,[PNode,TestFun]), + MRef = erlang:monitor(process,Pid), + TO = ?TEST_DURATION*1000, + receive {'DOWN',MRef,_,_,Reason} -> + ct:fail({remote_pid_down,Reason}) + after TO -> + All = ets:lookup_element(Tab,producer,2), + Written = ets:lookup_element(Tab,consumer,2), + dbg:stop_clear(), + ?COLLECT_STATS(All, + [{PNode,P,Id} || {Id,P} <- _PStatProcs] ++ + [{CNode,P,Id} || {Id,P} <- _CStatProcs]), + Ratio = Written/All * 100, + ct_event:notify(#event{name = benchmark_data, + data = [{value,Ratio}]}), + {comment,io_lib:format("~p % (~p written, ~p produced)", + [round(Ratio),Written,All])} + end. + +wrap_test(Fun) -> + wrap_test(node(),Fun). +wrap_test(Node,Fun) -> + reset(), + group_leader(whereis(user),self()), + rpc:call(Node,?MODULE,do_fun,[Fun]). + +do_fun(Fun) -> + reset(), + Fun(). + +reset() -> + reset([logger_std_h_default, logger_disk_log_h_default, logger_proxy]). +reset([P|Ps]) -> + is_pid(whereis(P)) andalso logger_olp:reset(P), + reset(Ps); +reset([]) -> + ok. + + +tracer({trace,_,call,{?MODULE,producer,_}},{Tab,_PNode,_CNode}=S) -> + ets:update_counter(Tab,producer,1), + S; +tracer({trace,Pid,call,{logger_backend,log_allowed,_}},{Tab,PNode,_CNode}=S) when node(Pid)=:=PNode -> + ets:update_counter(Tab,producer,1), + S; +tracer({trace,_,call,{?MODULE,log,_}},{Tab,_PNode,_CNode}=S) -> + ets:update_counter(Tab,consumer,1), + S; +tracer({trace,_,call,{_,write,_}},{Tab,_PNode,_CNode}=S) -> + ets:update_counter(Tab,consumer,1), + S; +tracer(_,S) -> + S. + + +%%%----------------------------------------------------------------- +%%% Collect statistics +-define(STAT_KEYS, + [burst_drops, + calls, + casts, + drops, + flushed, + flushes, + freq, + last_qlen, + max_qlen, + time, + writes]). +-define(EVENT_KEYS, + [calls,casts,flushed]). + +stats(All,Procs) -> + NI = [{Id,rpc:call(N,logger_olp,info,[P])} || {N,P,Id}<-Procs], + [{all,All}|[stats(Id,I,All) || {Id,I} <- NI]]. + +stats(Id,Info,All) -> + S = maps:with(?STAT_KEYS,Info), + AllOnProc = lists:sum(maps:values(maps:with(?EVENT_KEYS,S))), + if All>0 -> + Writes = maps:get(writes,S), + {_,ActiveTime} = maps:get(time,S), + Rate = round(100*Writes/All), + RateOnProc = + if AllOnProc>0 -> + round(100*Writes/AllOnProc); + true -> + 0 + end, + AvFreq = + if ActiveTime>0 -> + round(Writes/ActiveTime); + true -> + 0 + end, + {Id, + {stats,S}, + {rate,Rate}, + {rate_on_proc,RateOnProc}, + {av_freq,AvFreq}}; + true -> + {Id,none} + end. + +%%%----------------------------------------------------------------- +%%% Spawn a lot of processes that crash repeatedly, causing a lot of +%%% error reports from the emulator. +em_cascading() -> + spawn(fun() -> super() end). + +super() -> + process_flag(trap_exit,true), + spawn_link(fun server/0), + [spawn_link(fun client/0) || _<-lists:seq(1,10000)], + super_loop(). + +super_loop() -> + receive + {'EXIT',_,server} -> + spawn_link(fun server/0), + super_loop(); + {'EXIT',_,_} -> + _L = lists:sum(lists:seq(1,10000)), + spawn_link(fun client/0), + super_loop() + end. + +client() -> + receive + after 1 -> + case whereis(server) of + Pid when is_pid(Pid) -> + ok; + undefined -> + producer(), + erlang:error(some_exception) + end + end, + client(). + +server() -> + register(server,self()), + receive + after 3000 -> + exit(server) + end. + + +%%%----------------------------------------------------------------- +%%% Create a supervisor tree with processes that crash repeatedly, +%%% causing a lot of supervisor reports and crashreports +otp_cascading() -> + {ok,Pid} = supervisor:start_link({local,otp_super}, ?MODULE, [otp_super]), + unlink(Pid), + Pid. + +otp_server_sup() -> + supervisor:start_link({local,otp_server_sup},?MODULE,[otp_server_sup]). + +otp_client_sup(N) -> + supervisor:start_link({local,otp_client_sup},?MODULE,[otp_client_sup,N]). + +otp_server() -> + gen_server:start_link({local,otp_server},?MODULE,[otp_server],[]). + +otp_client() -> + gen_server:start_link(?MODULE,[otp_client],[]). + +init([otp_super]) -> + {ok, {{one_for_one, 200, 10}, + [{client_sup, + {?MODULE, otp_client_sup, [10000]}, + permanent, 1000, supervisor, [?MODULE]}, + {server_sup, + {?MODULE, otp_server_sup, []}, + permanent, 1000, supervisor, [?MODULE]} + ]}}; +init([otp_server_sup]) -> + {ok, {{one_for_one, 2, 10}, + [{server, + {?MODULE, otp_server, []}, + permanent, 1000, worker, [?MODULE]} + ]}}; +init([otp_client_sup,N]) -> + spawn(fun() -> + [supervisor:start_child(otp_client_sup,[]) + || _ <- lists:seq(1,N)] + end), + {ok, {{simple_one_for_one, N*10, 1}, + [{client, + {?MODULE, otp_client, []}, + permanent, 1000, worker, [?MODULE]} + ]}}; +init([otp_server]) -> + {ok, server, 10000}; +init([otp_client]) -> + {ok, client,1}. + +handle_info(timeout, client) -> + true = is_pid(whereis(otp_server)), + {noreply,client,1}; +handle_info(timeout, server) -> + exit(self(), some_error). + +%%%----------------------------------------------------------------- +%%% Logger callbacks +log(_LogEvent,_Config) -> + ok. + +%%%----------------------------------------------------------------- +%%% Function to trace on for counting produced emulator messages +producer() -> + ok. diff --git a/lib/kernel/test/logger_test_lib.erl b/lib/kernel/test/logger_test_lib.erl index 81eb9ce5eb..be4bc427fb 100644 --- a/lib/kernel/test/logger_test_lib.erl +++ b/lib/kernel/test/logger_test_lib.erl @@ -28,11 +28,17 @@ post_end_per_testcase/5, post_end_per_suite/3]). setup(Config,Vars) -> + Postfix = case proplists:get_value(postfix, Config) of + undefined -> ""; + P -> ["_",P] + end, FuncStr = lists:concat([proplists:get_value(suite, Config), "_", - proplists:get_value(tc, Config)]), + proplists:get_value(tc, Config)| + Postfix]), ConfigFileName = filename:join(proplists:get_value(priv_dir, Config), FuncStr), file:write_file(ConfigFileName ++ ".config", io_lib:format("[{kernel, ~p}].",[Vars])), - case test_server:start_node(proplists:get_value(tc, Config), slave, + Sname = lists:concat([proplists:get_value(tc,Config)|Postfix]), + case test_server:start_node(Sname, slave, [{args, ["-pa ",filename:dirname(code:which(?MODULE)), " -boot start_sasl -kernel start_timer true " "-config ",ConfigFileName]}]) of diff --git a/lib/mnesia/doc/src/mnesia.xml b/lib/mnesia/doc/src/mnesia.xml index 94f1af34bf..11b0b8e987 100644 --- a/lib/mnesia/doc/src/mnesia.xml +++ b/lib/mnesia/doc/src/mnesia.xml @@ -2077,6 +2077,13 @@ mnesia:create_table(employee, <fsummary>Starts a local Mnesia system.</fsummary> <desc> <marker id="start"></marker> + <p>Mnesia startup is asynchronous. The function call + <c>mnesia:start()</c> returns the atom <c>ok</c> and then + starts to initialize the different tables. Depending on the + size of the database, this can take some time, and the + application programmer must wait for the tables that the + application needs before they can be used. This is achieved + by using the function <c>mnesia:wait_for_tables/2</c>.</p> <p>The startup procedure for a set of Mnesia nodes is a fairly complicated operation. A Mnesia system consists of a set of nodes, with Mnesia started locally on all diff --git a/lib/mnesia/src/mnesia_dumper.erl b/lib/mnesia/src/mnesia_dumper.erl index a2880d6cf4..cb2ee504f9 100644 --- a/lib/mnesia/src/mnesia_dumper.erl +++ b/lib/mnesia/src/mnesia_dumper.erl @@ -67,10 +67,10 @@ get_log_writes() -> incr_log_writes() -> Left = mnesia_lib:incr_counter(trans_log_writes_left, -1), if - Left > 0 -> - ignore; + Left =:= 0 -> + adjust_log_writes(true); true -> - adjust_log_writes(true) + ignore end. adjust_log_writes(DoCast) -> diff --git a/lib/observer/src/cdv_detail_wx.erl b/lib/observer/src/cdv_detail_wx.erl index 4b1984c394..5e1137511a 100644 --- a/lib/observer/src/cdv_detail_wx.erl +++ b/lib/observer/src/cdv_detail_wx.erl @@ -84,8 +84,9 @@ destroy_progress(_) -> ok. init(Id,ParentFrame,Callback,App,Parent,{Title,Info,TW}) -> + Scale = observer_wx:get_scale(), Frame=wxFrame:new(ParentFrame, ?wxID_ANY, [Title], - [{style, ?wxDEFAULT_FRAME_STYLE}, {size, {850,600}}]), + [{style, ?wxDEFAULT_FRAME_STYLE}, {size, {Scale*850,Scale*600}}]), MenuBar = wxMenuBar:new(), create_menus(MenuBar), wxFrame:setMenuBar(Frame, MenuBar), diff --git a/lib/observer/src/cdv_table_wx.erl b/lib/observer/src/cdv_table_wx.erl index 0f28a51017..0cad272262 100644 --- a/lib/observer/src/cdv_table_wx.erl +++ b/lib/observer/src/cdv_table_wx.erl @@ -50,11 +50,12 @@ init([ParentWin, {ColumnSpec,Info,TW}]) -> end, Grid = wxListCtrl:new(ParentWin, [{style, Style}]), Li = wxListItem:new(), + Scale = observer_wx:get_scale(), AddListEntry = fun({Name, Align, DefSize}, Col) -> wxListItem:setText(Li, Name), wxListItem:setAlign(Li, Align), wxListCtrl:insertColumn(Grid, Col, Li), - wxListCtrl:setColumnWidth(Grid, Col, DefSize), + wxListCtrl:setColumnWidth(Grid, Col, DefSize*Scale), Col + 1 end, lists:foldl(AddListEntry, 0, ColumnSpec), diff --git a/lib/observer/src/cdv_virtual_list_wx.erl b/lib/observer/src/cdv_virtual_list_wx.erl index 2702301021..14877b7eab 100644 --- a/lib/observer/src/cdv_virtual_list_wx.erl +++ b/lib/observer/src/cdv_virtual_list_wx.erl @@ -132,11 +132,12 @@ create_list_box(Panel, Holder, Callback, Owner) -> end} ]), Li = wxListItem:new(), + Scale = observer_wx:get_scale(), AddListEntry = fun({Name, Align, DefSize}, Col) -> wxListItem:setText(Li, Name), wxListItem:setAlign(Li, Align), wxListCtrl:insertColumn(ListCtrl, Col, Li), - wxListCtrl:setColumnWidth(ListCtrl, Col, DefSize), + wxListCtrl:setColumnWidth(ListCtrl, Col, DefSize*Scale), Col + 1 end, ListItems = Callback:col_spec(), diff --git a/lib/observer/src/cdv_wx.erl b/lib/observer/src/cdv_wx.erl index 1e9cef8952..811c767e66 100644 --- a/lib/observer/src/cdv_wx.erl +++ b/lib/observer/src/cdv_wx.erl @@ -101,8 +101,9 @@ init(File0) -> {ok,CdvServer} = crashdump_viewer:start_link(), catch wxSystemOptions:setOption("mac.listctrl.always_use_generic", 1), + Scale = observer_wx:get_scale(), Frame = wxFrame:new(wx:null(), ?wxID_ANY, "Crashdump Viewer", - [{size, {850, 600}}, {style, ?wxDEFAULT_FRAME_STYLE}]), + [{size, {Scale*850, Scale*600}}, {style, ?wxDEFAULT_FRAME_STYLE}]), IconFile = filename:join(code:priv_dir(observer), "erlang_observer.png"), Icon = wxIcon:new(IconFile, [{type,?wxBITMAP_TYPE_PNG}]), wxFrame:setIcon(Frame, Icon), diff --git a/lib/observer/src/observer_alloc_wx.erl b/lib/observer/src/observer_alloc_wx.erl index 54e246f247..da47a30fb1 100644 --- a/lib/observer/src/observer_alloc_wx.erl +++ b/lib/observer/src/observer_alloc_wx.erl @@ -282,11 +282,12 @@ create_mem_info(Parent) -> Grid = wxListCtrl:new(Parent, [{style, Style}]), Li = wxListItem:new(), + Scale = observer_wx:get_scale(), AddListEntry = fun({Name, Align, DefSize}, Col) -> wxListItem:setText(Li, Name), wxListItem:setAlign(Li, Align), wxListCtrl:insertColumn(Grid, Col, Li), - wxListCtrl:setColumnWidth(Grid, Col, DefSize), + wxListCtrl:setColumnWidth(Grid, Col, DefSize*Scale), Col + 1 end, ListItems = [{"Allocator Type", ?wxLIST_FORMAT_LEFT, 200}, diff --git a/lib/observer/src/observer_app_wx.erl b/lib/observer/src/observer_app_wx.erl index 2a481966da..8c3eef5411 100644 --- a/lib/observer/src/observer_app_wx.erl +++ b/lib/observer/src/observer_app_wx.erl @@ -117,16 +117,19 @@ init([Notebook, Parent, _Config]) -> UseGC = haveGC(), Version28 = ?wxMAJOR_VERSION =:= 2 andalso ?wxMINOR_VERSION =:= 8, + Scale = observer_wx:get_scale(), Font = case os:type() of {unix,_} when UseGC, Version28 -> - wxFont:new(12,?wxFONTFAMILY_DECORATIVE,?wxFONTSTYLE_NORMAL,?wxFONTWEIGHT_NORMAL); + wxFont:new(Scale * 12,?wxFONTFAMILY_DECORATIVE,?wxFONTSTYLE_NORMAL,?wxFONTWEIGHT_NORMAL); _ -> - wxSystemSettings:getFont(?wxSYS_DEFAULT_GUI_FONT) + Font0 = wxSystemSettings:getFont(?wxSYS_DEFAULT_GUI_FONT), + wxFont:setPointSize(Font0, Scale * wxFont:getPointSize(Font0)), + Font0 end, SelCol = wxSystemSettings:getColour(?wxSYS_COLOUR_HIGHLIGHT), GreyBrush = wxBrush:new({230,230,240}), SelBrush = wxBrush:new(SelCol), - LinkPen = wxPen:new(SelCol, [{width, 2}]), + LinkPen = wxPen:new(SelCol, [{width, Scale * 2}]), process_flag(trap_exit, true), {Panel, #state{parent=Parent, panel =Panel, @@ -134,7 +137,7 @@ init([Notebook, Parent, _Config]) -> app_w =DrawingArea, usegc = UseGC, paint=#paint{font = Font, - pen = wxPen:new({80,80,80}, [{width, 2}]), + pen = wxPen:new({80,80,80}, [{width, Scale * 2}]), brush= GreyBrush, sel = SelBrush, links= LinkPen diff --git a/lib/observer/src/observer_perf_wx.erl b/lib/observer/src/observer_perf_wx.erl index 21c6d26f49..79271addf2 100644 --- a/lib/observer/src/observer_perf_wx.erl +++ b/lib/observer/src/observer_perf_wx.erl @@ -110,25 +110,26 @@ setup_graph_drawing(Panels) -> _ = [Do(Panel) || Panel <- Panels], UseGC = haveGC(), Version28 = ?wxMAJOR_VERSION =:= 2 andalso ?wxMINOR_VERSION =:= 8, + Scale = observer_wx:get_scale(), {Font, SmallFont} = if UseGC, Version28 -> %% Def font is really small when using Graphics contexts in 2.8 %% Hardcode it - F = wxFont:new(12,?wxFONTFAMILY_DECORATIVE,?wxFONTSTYLE_NORMAL,?wxFONTWEIGHT_BOLD), - SF = wxFont:new(10, ?wxFONTFAMILY_DECORATIVE, ?wxFONTSTYLE_NORMAL, ?wxFONTWEIGHT_NORMAL), + F = wxFont:new(Scale * 12,?wxFONTFAMILY_DECORATIVE,?wxFONTSTYLE_NORMAL,?wxFONTWEIGHT_BOLD), + SF = wxFont:new(Scale * 10, ?wxFONTFAMILY_DECORATIVE, ?wxFONTSTYLE_NORMAL, ?wxFONTWEIGHT_NORMAL), {F, SF}; true -> DefFont = wxSystemSettings:getFont(?wxSYS_DEFAULT_GUI_FONT), DefSize = wxFont:getPointSize(DefFont), DefFamily = wxFont:getFamily(DefFont), - F = wxFont:new(DefSize-1, DefFamily, ?wxFONTSTYLE_NORMAL, ?wxFONTWEIGHT_BOLD), - SF = wxFont:new(DefSize-2, DefFamily, ?wxFONTSTYLE_NORMAL, ?wxFONTWEIGHT_NORMAL), + F = wxFont:new(Scale * (DefSize-1), DefFamily, ?wxFONTSTYLE_NORMAL, ?wxFONTWEIGHT_BOLD), + SF = wxFont:new(Scale * (DefSize-2), DefFamily, ?wxFONTSTYLE_NORMAL, ?wxFONTWEIGHT_NORMAL), {F, SF} end, - BlackPen = wxPen:new({0,0,0}, [{width, 1}]), - Pens = [wxPen:new(Col, [{width, 1}, {style, ?wxSOLID}]) + BlackPen = wxPen:new({0,0,0}, [{width, Scale}]), + Pens = [wxPen:new(Col, [{width, Scale}, {style, ?wxSOLID}]) || Col <- tuple_to_list(colors())], - DotPens = [wxPen:new(Col, [{width, 1}, {style, ?wxDOT}]) + DotPens = [wxPen:new(Col, [{width, Scale}, {style, ?wxDOT}]) || Col <- tuple_to_list(colors())], #paint{usegc = UseGC, font = Font, diff --git a/lib/observer/src/observer_port_wx.erl b/lib/observer/src/observer_port_wx.erl index 445f3dd6b1..00cf1b5fba 100644 --- a/lib/observer/src/observer_port_wx.erl +++ b/lib/observer/src/observer_port_wx.erl @@ -96,11 +96,12 @@ init([Notebook, Parent, Config]) -> wxListCtrl:setColumnWidth(Grid, Col, DefSize), Col + 1 end, - ListItems = [{"Id", ?wxLIST_FORMAT_LEFT, 150}, - {"Connected", ?wxLIST_FORMAT_LEFT, 150}, - {"Name", ?wxLIST_FORMAT_LEFT, 150}, - {"Controls", ?wxLIST_FORMAT_LEFT, 200}, - {"Slot", ?wxLIST_FORMAT_RIGHT, 50}], + Scale = observer_wx:get_scale(), + ListItems = [{"Id", ?wxLIST_FORMAT_LEFT, Scale*150}, + {"Connected", ?wxLIST_FORMAT_LEFT, Scale*150}, + {"Name", ?wxLIST_FORMAT_LEFT, Scale*150}, + {"Controls", ?wxLIST_FORMAT_LEFT, Scale*200}, + {"Slot", ?wxLIST_FORMAT_RIGHT, Scale*50}], lists:foldl(AddListEntry, 0, ListItems), wxListItem:destroy(Li), @@ -461,10 +462,11 @@ display_port_info(Parent, PortRec, Opened) -> do_display_port_info(Parent0, PortRec) -> Parent = observer_lib:get_wx_parent(Parent0), Title = "Port Info: " ++ PortRec#port.id_str, + Scale = observer_wx:get_scale(), Frame = wxMiniFrame:new(Parent, ?wxID_ANY, Title, [{style, ?wxSYSTEM_MENU bor ?wxCAPTION bor ?wxCLOSE_BOX bor ?wxRESIZE_BORDER}, - {size,{600,400}}]), + {size,{Scale * 600, Scale * 400}}]), ScrolledWin = wxScrolledWindow:new(Frame,[{style,?wxHSCROLL bor ?wxVSCROLL}]), wxScrolledWindow:enableScrolling(ScrolledWin,true,true), wxScrolledWindow:setScrollbars(ScrolledWin,20,20,0,0), diff --git a/lib/observer/src/observer_pro_wx.erl b/lib/observer/src/observer_pro_wx.erl index 04e654a37e..4ab4a78462 100644 --- a/lib/observer/src/observer_pro_wx.erl +++ b/lib/observer/src/observer_pro_wx.erl @@ -163,13 +163,14 @@ create_list_box(Panel, Holder) -> wxListCtrl:setColumnWidth(ListCtrl, Col, DefSize), Col + 1 end, - ListItems = [{"Pid", ?wxLIST_FORMAT_CENTRE, 120}, - {"Name or Initial Func", ?wxLIST_FORMAT_LEFT, 200}, -%% {"Time", ?wxLIST_FORMAT_CENTRE, 50}, - {"Reds", ?wxLIST_FORMAT_RIGHT, 100}, - {"Memory", ?wxLIST_FORMAT_RIGHT, 100}, - {"MsgQ", ?wxLIST_FORMAT_RIGHT, 50}, - {"Current Function", ?wxLIST_FORMAT_LEFT, 200}], + Scale = observer_wx:get_scale(), + ListItems = [{"Pid", ?wxLIST_FORMAT_CENTRE, Scale*120}, + {"Name or Initial Func", ?wxLIST_FORMAT_LEFT, Scale*200}, +%% {"Time", ?wxLIST_FORMAT_CENTRE, Scale*50}, + {"Reds", ?wxLIST_FORMAT_RIGHT, Scale*100}, + {"Memory", ?wxLIST_FORMAT_RIGHT, Scale*100}, + {"MsgQ", ?wxLIST_FORMAT_RIGHT, Scale*50}, + {"Current Function", ?wxLIST_FORMAT_LEFT, Scale*200}], lists:foldl(AddListEntry, 0, ListItems), wxListItem:destroy(Li), diff --git a/lib/observer/src/observer_procinfo.erl b/lib/observer/src/observer_procinfo.erl index f436886735..bd5fed0951 100644 --- a/lib/observer/src/observer_procinfo.erl +++ b/lib/observer/src/observer_procinfo.erl @@ -59,8 +59,9 @@ init([Pid, ParentFrame, Parent]) -> {registered_name, Registered} -> io_lib:format("~tp (~p)",[Registered, Pid]); undefined -> throw(process_undefined) end, + Scale = observer_wx:get_scale(), Frame=wxFrame:new(ParentFrame, ?wxID_ANY, [atom_to_list(node(Pid)), $:, Title], - [{style, ?wxDEFAULT_FRAME_STYLE}, {size, {850,600}}]), + [{style, ?wxDEFAULT_FRAME_STYLE}, {size, {Scale * 850, Scale * 600}}]), MenuBar = wxMenuBar:new(), create_menus(MenuBar), wxFrame:setMenuBar(Frame, MenuBar), @@ -245,12 +246,13 @@ init_dict_page(Parent, Pid, Table) -> init_stack_page(Parent, Pid) -> LCtrl = wxListCtrl:new(Parent, [{style, ?wxLC_REPORT bor ?wxLC_HRULES}]), Li = wxListItem:new(), + Scale = observer_wx:get_scale(), wxListItem:setText(Li, "Module:Function/Arg"), wxListCtrl:insertColumn(LCtrl, 0, Li), - wxListCtrl:setColumnWidth(LCtrl, 0, 300), + wxListCtrl:setColumnWidth(LCtrl, 0, Scale * 300), wxListItem:setText(Li, "File:LineNumber"), wxListCtrl:insertColumn(LCtrl, 1, Li), - wxListCtrl:setColumnWidth(LCtrl, 1, 300), + wxListCtrl:setColumnWidth(LCtrl, 1, Scale * 300), wxListItem:destroy(Li), Update = fun() -> case observer_wx:try_rpc(node(Pid), erlang, process_info, diff --git a/lib/observer/src/observer_trace_wx.erl b/lib/observer/src/observer_trace_wx.erl index 2c3b46a3a1..f458c8c34a 100644 --- a/lib/observer/src/observer_trace_wx.erl +++ b/lib/observer/src/observer_trace_wx.erl @@ -188,8 +188,9 @@ create_proc_port_view(Parent) -> wxListCtrl:setColumnWidth(Procs, Col, DefSize), Col + 1 end, - ProcListItems = [{"Process Id", ?wxLIST_FORMAT_CENTER, 120}, - {"Trace Options", ?wxLIST_FORMAT_LEFT, 300}], + Scale = observer_wx:get_scale(), + ProcListItems = [{"Process Id", ?wxLIST_FORMAT_CENTER, Scale*120}, + {"Trace Options", ?wxLIST_FORMAT_LEFT, Scale*300}], lists:foldl(AddProc, 0, ProcListItems), AddPort = fun({Name, Align, DefSize}, Col) -> @@ -199,8 +200,8 @@ create_proc_port_view(Parent) -> wxListCtrl:setColumnWidth(Ports, Col, DefSize), Col + 1 end, - PortListItems = [{"Port Id", ?wxLIST_FORMAT_CENTER, 120}, - {"Trace Options", ?wxLIST_FORMAT_LEFT, 300}], + PortListItems = [{"Port Id", ?wxLIST_FORMAT_CENTER, Scale*120}, + {"Trace Options", ?wxLIST_FORMAT_LEFT, Scale*300}], lists:foldl(AddPort, 0, PortListItems), wxListItem:destroy(Li), @@ -242,14 +243,15 @@ create_matchspec_view(Parent) -> Funcs = wxListCtrl:new(Splitter, [{winid, ?FUNCS_WIN}, {style, Style}]), Li = wxListItem:new(), + Scale = observer_wx:get_scale(), wxListItem:setText(Li, "Modules"), wxListCtrl:insertColumn(Modules, 0, Li), wxListItem:setText(Li, "Functions"), wxListCtrl:insertColumn(Funcs, 0, Li), - wxListCtrl:setColumnWidth(Funcs, 0, 150), + wxListCtrl:setColumnWidth(Funcs, 0, Scale*150), wxListItem:setText(Li, "Match Spec"), wxListCtrl:insertColumn(Funcs, 1, Li), - wxListCtrl:setColumnWidth(Funcs, 1, 300), + wxListCtrl:setColumnWidth(Funcs, 1, Scale*300), wxListItem:destroy(Li), wxSplitterWindow:setSashGravity(Splitter, 0.0), @@ -969,7 +971,8 @@ output_file(true, true, Opts) -> create_logwindow(_Parent, false) -> {false, false}; create_logwindow(Parent, true) -> - LogWin = wxFrame:new(Parent, ?LOG_WIN, "Trace Log", [{size, {750, 800}}]), + Scale = observer_wx:get_scale(), + LogWin = wxFrame:new(Parent, ?LOG_WIN, "Trace Log", [{size, {750*Scale, 800*Scale}}]), MB = wxMenuBar:new(), File = wxMenu:new(), wxMenu:append(File, ?LOG_CLEAR, "Clear Log\tCtrl-C"), diff --git a/lib/observer/src/observer_traceoptions_wx.erl b/lib/observer/src/observer_traceoptions_wx.erl index ea292b92af..514d55ff24 100644 --- a/lib/observer/src/observer_traceoptions_wx.erl +++ b/lib/observer/src/observer_traceoptions_wx.erl @@ -167,9 +167,10 @@ select_nodes(Parent, Nodes) -> check_selector(Parent, Choices). module_selector(Parent, Node) -> + Scale = observer_wx:get_scale(), Dialog = wxDialog:new(Parent, ?wxID_ANY, "Select Module or Event", [{style, ?wxDEFAULT_DIALOG_STYLE bor ?wxRESIZE_BORDER}, - {size, {400, 400}}]), + {size, {400*Scale, 400*Scale}}]), Panel = wxPanel:new(Dialog), PanelSz = wxBoxSizer:new(?wxVERTICAL), MainSz = wxBoxSizer:new(?wxVERTICAL), @@ -237,9 +238,10 @@ function_selector(Parent, Node, Module) -> end. check_selector(Parent, ParsedChoices) -> + Scale = observer_wx:get_scale(), Dialog = wxDialog:new(Parent, ?wxID_ANY, "Trace Functions", [{style, ?wxDEFAULT_DIALOG_STYLE bor ?wxRESIZE_BORDER}, - {size, {400, 400}}]), + {size, {400*Scale, 400*Scale}}]), Panel = wxPanel:new(Dialog), PanelSz = wxBoxSizer:new(?wxVERTICAL), @@ -331,9 +333,10 @@ select_matchspec(Pid, Parent, AllMatchSpecs, Key) -> {value,{Key,MSs0},Rest} -> {MSs0,Rest}; false -> {[],AllMatchSpecs} end, + Scale = observer_wx:get_scale(), Dialog = wxDialog:new(Parent, ?wxID_ANY, "Trace Match Specifications", [{style, ?wxDEFAULT_DIALOG_STYLE bor ?wxRESIZE_BORDER}, - {size, {400, 400}}]), + {size, {400*Scale, 400*Scale}}]), Panel = wxPanel:new(Dialog), PanelSz = wxBoxSizer:new(?wxVERTICAL), diff --git a/lib/observer/src/observer_tv_table.erl b/lib/observer/src/observer_tv_table.erl index d6dcee2cda..7bd67a0f0b 100644 --- a/lib/observer/src/observer_tv_table.erl +++ b/lib/observer/src/observer_tv_table.erl @@ -99,7 +99,8 @@ init([Parent, Opts]) -> ets -> "TV Ets: " ++ Title0; mnesia -> "TV Mnesia: " ++ Title0 end, - Frame = wxFrame:new(Parent, ?wxID_ANY, Title, [{size, {800, 600}}]), + Scale = observer_wx:get_scale(), + Frame = wxFrame:new(Parent, ?wxID_ANY, Title, [{size, {Scale * 800, Scale * 600}}]), IconFile = filename:join(code:priv_dir(observer), "erlang_observer.png"), Icon = wxIcon:new(IconFile, [{type,?wxBITMAP_TYPE_PNG}]), wxFrame:setIcon(Frame, Icon), diff --git a/lib/observer/src/observer_tv_wx.erl b/lib/observer/src/observer_tv_wx.erl index 814f3a1260..247b3e869f 100644 --- a/lib/observer/src/observer_tv_wx.erl +++ b/lib/observer/src/observer_tv_wx.erl @@ -87,12 +87,13 @@ init([Notebook, Parent, Config]) -> wxListCtrl:setColumnWidth(Grid, Col, DefSize), Col + 1 end, - ListItems = [{"Table Name", ?wxLIST_FORMAT_LEFT, 200}, - {"Objects", ?wxLIST_FORMAT_RIGHT, 100}, - {"Size (kB)", ?wxLIST_FORMAT_RIGHT, 100}, - {"Owner Pid", ?wxLIST_FORMAT_CENTER, 150}, - {"Owner Name", ?wxLIST_FORMAT_LEFT, 200}, - {"Table Id", ?wxLIST_FORMAT_LEFT, 250} + Scale = observer_wx:get_scale(), + ListItems = [{"Table Name", ?wxLIST_FORMAT_LEFT, Scale*200}, + {"Objects", ?wxLIST_FORMAT_RIGHT, Scale*100}, + {"Size (kB)", ?wxLIST_FORMAT_RIGHT, Scale*100}, + {"Owner Pid", ?wxLIST_FORMAT_CENTER, Scale*150}, + {"Owner Name", ?wxLIST_FORMAT_LEFT, Scale*200}, + {"Table Id", ?wxLIST_FORMAT_LEFT, Scale*250} ], lists:foldl(AddListEntry, 0, ListItems), wxListItem:destroy(Li), diff --git a/lib/observer/src/observer_wx.erl b/lib/observer/src/observer_wx.erl index 453e3bdc2d..71db586845 100644 --- a/lib/observer/src/observer_wx.erl +++ b/lib/observer/src/observer_wx.erl @@ -22,7 +22,7 @@ -export([start/0, stop/0]). -export([create_menus/2, get_attrib/1, get_tracer/0, get_active_node/0, get_menubar/0, - set_status/1, create_txt_dialog/4, try_rpc/4, return_to_localnode/2]). + get_scale/0, set_status/1, create_txt_dialog/4, try_rpc/4, return_to_localnode/2]). -export([init/1, handle_event/2, handle_cast/2, terminate/2, code_change/3, handle_call/3, handle_info/2, check_page_title/1]). @@ -91,14 +91,24 @@ get_active_node() -> get_menubar() -> wx_object:call(observer, get_menubar). +get_scale() -> + ScaleStr = os:getenv("OBSERVER_SCALE", "1"), + try list_to_integer(ScaleStr) of + Scale when Scale < 1 -> 1; + Scale -> Scale + catch _:_ -> + 1 + end. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% init(_Args) -> register(observer, self()), wx:new(), catch wxSystemOptions:setOption("mac.listctrl.always_use_generic", 1), + Scale = get_scale(), Frame = wxFrame:new(wx:null(), ?wxID_ANY, "Observer", - [{size, {850, 600}}, {style, ?wxDEFAULT_FRAME_STYLE}]), + [{size, {Scale * 850, Scale * 600}}, {style, ?wxDEFAULT_FRAME_STYLE}]), IconFile = filename:join(code:priv_dir(observer), "erlang_observer.png"), Icon = wxIcon:new(IconFile, [{type,?wxBITMAP_TYPE_PNG}]), wxFrame:setIcon(Frame, Icon), @@ -771,7 +781,11 @@ ensure_sasl_started(Node) -> ensure_mf_h_handler_used(Node) -> %% is log_mf_h used ? - Handlers = rpc:block_call(Node, gen_event, which_handlers, [error_logger]), + Handlers = + case rpc:block_call(Node, gen_event, which_handlers, [error_logger]) of + {badrpc,{'EXIT',noproc}} -> []; % OTP-21+ and no event handler exists + Hs -> Hs + end, case lists:any(fun(L)-> L == log_mf_h end, Handlers) of false -> throw("Error: log_mf_h handler not used in sasl."), error; diff --git a/lib/observer/test/crashdump_helper.erl b/lib/observer/test/crashdump_helper.erl index d5d3649525..d6b5eff9b5 100644 --- a/lib/observer/test/crashdump_helper.erl +++ b/lib/observer/test/crashdump_helper.erl @@ -204,5 +204,4 @@ dump_persistent_terms() -> create_persistent_terms() -> persistent_term:put({?MODULE,first}, {pid,42.0}), persistent_term:put({?MODULE,second}, [1,2,3]), - persistent_term:get(). - + {persistent_term:get({?MODULE,first}),persistent_term:get({?MODULE,second})}. diff --git a/lib/observer/test/crashdump_viewer_SUITE.erl b/lib/observer/test/crashdump_viewer_SUITE.erl index 8c5e618f4a..31cf7011d4 100644 --- a/lib/observer/test/crashdump_viewer_SUITE.erl +++ b/lib/observer/test/crashdump_viewer_SUITE.erl @@ -615,9 +615,8 @@ special(File,Procs) -> #proc{dict=Dict} = ProcDetails, %% io:format("~p\n", [Dict]), - Pts1 = crashdump_helper:create_persistent_terms(), - Pts2 = proplists:get_value(pts,Dict), - true = lists:sort(Pts1) =:= lists:sort(Pts2), + Pts = crashdump_helper:create_persistent_terms(), + Pts = proplists:get_value(pts,Dict), io:format(" persistent terms ok",[]), ok; _ -> diff --git a/lib/odbc/c_src/odbcserver.c b/lib/odbc/c_src/odbcserver.c index 8c799f6ff1..fb4f61417e 100644 --- a/lib/odbc/c_src/odbcserver.c +++ b/lib/odbc/c_src/odbcserver.c @@ -2749,6 +2749,11 @@ static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle, Boolean ext errmsg_buffer_size = errmsg_buffer_size - errmsg_size; acc_errmsg_size = acc_errmsg_size + errmsg_size; current_errmsg_pos = current_errmsg_pos + errmsg_size; + } else if(result == SQL_SUCCESS_WITH_INFO && errmsg_size >= errmsg_buffer_size) { + memcpy(diagnos.sqlState, current_sql_state, SQL_STATE_SIZE); + diagnos.nativeError = nativeError; + acc_errmsg_size = errmsg_buffer_size; + break; } else { break; } diff --git a/lib/public_key/doc/src/public_key.xml b/lib/public_key/doc/src/public_key.xml index 76dbe008ef..b7589f6653 100644 --- a/lib/public_key/doc/src/public_key.xml +++ b/lib/public_key/doc/src/public_key.xml @@ -644,7 +644,7 @@ fun(#'DistributionPoint'{}, #'CertificateList'{}, <v>conf_opt() = {cert, der_encoded()} | {key, PrivateKey} |{cacerts, [der_encoded()]}</v> <d> This is a subset of the type - <seealso marker="ssl:ssl#type-ssloption"> ssl:ssl_option()</seealso>. + <seealso marker="ssl:ssl#type-tls_option"> ssl:tls_option()</seealso>. <c>PrivateKey</c> is what <seealso marker="#generate_key-1">generate_key/1</seealso> returns. diff --git a/lib/public_key/src/public_key.erl b/lib/public_key/src/public_key.erl index 75d40d2e8a..47c5dbb95a 100644 --- a/lib/public_key/src/public_key.erl +++ b/lib/public_key/src/public_key.erl @@ -66,7 +66,7 @@ -export_type([public_key/0, private_key/0, pem_entry/0, pki_asn1_type/0, asn1_type/0, ssh_file/0, der_encoded/0, - key_params/0, digest_type/0]). + key_params/0, digest_type/0, issuer_name/0, oid/0]). -type public_key() :: rsa_public_key() | dsa_public_key() | ec_public_key() | ed_public_key() . -type private_key() :: rsa_private_key() | dsa_private_key() | ec_private_key() | ed_private_key() . diff --git a/lib/runtime_tools/examples/dist.systemtap b/lib/runtime_tools/examples/dist.systemtap index bb20d617e1..4102a5243c 100644 --- a/lib/runtime_tools/examples/dist.systemtap +++ b/lib/runtime_tools/examples/dist.systemtap @@ -19,18 +19,18 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ -probe process("beam").mark("dist-monitor") +probe process("beam.smp").mark("dist-monitor") { printf("monitor: pid %d, who %s, what %s, node %s, type %s, reason %s\n", pid(), @@ -38,38 +38,38 @@ probe process("beam").mark("dist-monitor") user_string($arg5)); } -probe process("beam").mark("dist-port_busy") +probe process("beam.smp").mark("dist-port_busy") { printf("dist port_busy: node %s, port %s, remote_node %s, blocked pid %s\n", user_string($arg1), user_string($arg2), user_string($arg3), user_string($arg4)); - blocked_procs[user_string($arg4)] = timestamp; + blocked_procs[user_string($arg4)] = local_clock_ns(); } -probe process("beam").mark("dist-port_busy") +probe process("beam.smp").mark("dist-port_busy") { printf("dist port_busy: node %s, port %s, remote_node %s, blocked pid %s\n", user_string($arg1), user_string($arg2), user_string($arg3), user_string($arg4)); - blocked_procs[user_string($arg4)] = timestamp; + blocked_procs[user_string($arg4)] = local_clock_ns(); } -probe process("beam").mark("dist-output") +probe process("beam.smp").mark("dist-output") { printf("dist output: node %s, port %s, remote_node %s bytes %d\n", user_string($arg1), user_string($arg2), user_string($arg3), $arg4); } -probe process("beam").mark("dist-outputv") +probe process("beam.smp").mark("dist-outputv") { printf("port outputv: node %s, port %s, remote_node %s bytes %d\n", user_string($arg1), user_string($arg2), user_string($arg3), $arg4); } -probe process("beam").mark("process-scheduled") +probe process("beam.smp").mark("process-scheduled") { pidstr = user_string($arg1); if (pidstr in blocked_procs) { printf("blocked pid %s scheduled now, waited %d microseconds\n", - pidstr, (timestamp - blocked_procs[pidstr]) / 1000); + pidstr, (local_clock_ns() - blocked_procs[pidstr]) / 1000); delete blocked_procs[pidstr]; } } diff --git a/lib/runtime_tools/examples/driver1.systemtap b/lib/runtime_tools/examples/driver1.systemtap index e1ee8ecffc..f5bc28b42d 100644 --- a/lib/runtime_tools/examples/driver1.systemtap +++ b/lib/runtime_tools/examples/driver1.systemtap @@ -19,108 +19,102 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ -probe process("beam").mark("driver-init") +probe process("beam.smp").mark("driver__init") { printf("driver init name %s major %d minor %d flags %d\n", user_string($arg1), $arg2, $arg3, $arg4); } -probe process("beam").mark("driver-start") +probe process("beam.smp").mark("driver__start") { printf("driver start pid %s driver name %s port %s\n", user_string($arg1), user_string($arg2), user_string($arg3)); } -probe process("beam").mark("driver-stop") +probe process("beam.smp").mark("driver__stop") { printf("driver stop pid %s driver name %s port %s\n", user_string($arg1), user_string($arg2), user_string($arg3)); } -probe process("beam").mark("driver-finish") +probe process("beam.smp").mark("driver__finish") { printf("driver finish driver name %s\n", user_string($arg1)); } -probe process("beam").mark("driver-flush") +probe process("beam.smp").mark("driver__flush") { printf("driver flush pid %s port %s port name %s\n", user_string($arg1), user_string($arg2), user_string($arg3)); } -probe process("beam").mark("driver-output") +probe process("beam.smp").mark("driver__output") { printf("driver output pid %s port %s port name %s bytes %d\n", user_string($arg1), user_string($arg2), user_string($arg3), $arg4); } -probe process("beam").mark("driver-outputv") +probe process("beam.smp").mark("driver__outputv") { printf("driver outputv pid %s port %s port name %s bytes %d\n", user_string($arg1), user_string($arg2), user_string($arg3), $arg4); } -probe process("beam").mark("driver-control") +probe process("beam.smp").mark("driver__control") { printf("driver control pid %s port %s port name %s command %d bytes %d\n", user_string($arg1), user_string($arg2), user_string($arg3), $arg4, $arg5); } -probe process("beam").mark("driver-call") +probe process("beam.smp").mark("driver__call") { printf("driver call pid %s port %s port name %s command %d bytes %d\n", user_string($arg1), user_string($arg2), user_string($arg3), $arg4, $arg5); } -probe process("beam").mark("driver-event") -{ - printf("driver event pid %s port %s port name %s\n", - user_string($arg1), user_string($arg2), user_string($arg3)); -} - -probe process("beam").mark("driver-ready_input") +probe process("beam.smp").mark("driver__ready_input") { printf("driver ready_input pid %s port %s port name %s\n", user_string($arg1), user_string($arg2), user_string($arg3)); } -probe process("beam").mark("driver-ready_output") +probe process("beam.smp").mark("driver__ready_output") { printf("driver ready_output pid %s port %s port name %s\n", user_string($arg1), user_string($arg2), user_string($arg3)); } -probe process("beam").mark("driver-timeout") +probe process("beam.smp").mark("driver__timeout") { printf("driver timeout pid %s port %s port name %s\n", user_string($arg1), user_string($arg2), user_string($arg3)); } -probe process("beam").mark("driver-ready_async") +probe process("beam.smp").mark("driver__ready_async") { printf("driver ready_async pid %s port %s port name %s\n", user_string($arg1), user_string($arg2), user_string($arg3)); } -probe process("beam").mark("driver-process_exit") +probe process("beam.smp").mark("driver__process_exit") { printf("driver process_exit pid %s port %s port name %s\n", user_string($arg1), user_string($arg2), user_string($arg3)); } -probe process("beam").mark("driver-stop_select") +probe process("beam.smp").mark("driver__stop_select") { printf("driver stop_select driver name %s\n", user_string($arg1)); } diff --git a/lib/runtime_tools/examples/function-calls.systemtap b/lib/runtime_tools/examples/function-calls.systemtap index 9c44b2d014..6bb173b3ec 100644 --- a/lib/runtime_tools/examples/function-calls.systemtap +++ b/lib/runtime_tools/examples/function-calls.systemtap @@ -18,51 +18,51 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ -probe process("beam").mark("local-function-entry") +probe process("beam.smp").mark("local-function-entry") { printf("pid %s enter (local) %s depth %d\n", user_string($arg1), user_string($arg2), $arg3); } -probe process("beam").mark("global-function-entry") +probe process("beam.smp").mark("global-function-entry") { printf("pid %s enter (global) %s depth %d\n", user_string($arg1), user_string($arg2), $arg3); } -probe process("beam").mark("function-return") +probe process("beam.smp").mark("function-return") { printf("pid %s return %s depth %d\n", user_string($arg1), user_string($arg2), $arg3); } -probe process("beam").mark("bif-entry") +probe process("beam.smp").mark("bif-entry") { printf("pid %s BIF entry mfa %s\n", user_string($arg1), user_string($arg2)); } -probe process("beam").mark("bif-return") +probe process("beam.smp").mark("bif-return") { printf("pid %s BIF return mfa %s\n", user_string($arg1), user_string($arg2)); } -probe process("beam").mark("nif-entry") +probe process("beam.smp").mark("nif-entry") { printf("pid %s NIF entry mfa %s\n", user_string($arg1), user_string($arg2)); } -probe process("beam").mark("nif-return") +probe process("beam.smp").mark("nif-return") { printf("pid %s NIF return mfa %s\n", user_string($arg1), user_string($arg2)); } diff --git a/lib/runtime_tools/examples/garbage-collection.systemtap b/lib/runtime_tools/examples/garbage-collection.systemtap index e414eea821..14f0d6851c 100644 --- a/lib/runtime_tools/examples/garbage-collection.systemtap +++ b/lib/runtime_tools/examples/garbage-collection.systemtap @@ -18,33 +18,33 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ -probe process("beam").mark("gc_major-start") +probe process("beam.smp").mark("gc_major-start") { printf("GC major start pid %s need %d words\n", user_string($arg1), $arg2); } -probe process("beam").mark("gc_minor-start") +probe process("beam.smp").mark("gc_minor-start") { printf("GC minor start pid %s need %d words\n", user_string($arg1), $arg2); } -probe process("beam").mark("gc_major-end") +probe process("beam.smp").mark("gc_major-end") { printf("GC major end pid %s reclaimed %d words\n", user_string($arg1), $arg2); } -probe process("beam").mark("gc_minor-start") +probe process("beam.smp").mark("gc_minor-start") { printf("GC minor end pid %s reclaimed %d words\n", user_string($arg1), $arg2); } diff --git a/lib/runtime_tools/examples/memory1.systemtap b/lib/runtime_tools/examples/memory1.systemtap index 04df4d64c4..2fdc5a796c 100644 --- a/lib/runtime_tools/examples/memory1.systemtap +++ b/lib/runtime_tools/examples/memory1.systemtap @@ -18,34 +18,34 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ -probe process("beam").mark("copy-struct") +probe process("beam.smp").mark("copy-struct") { printf("copy_struct %d bytes\n", $arg1); } -probe process("beam").mark("copy-object") +probe process("beam.smp").mark("copy-object") { printf("copy_object pid %s %d bytes\n", user_string($arg1), $arg2); } -probe process("beam").mark("process-heap_grow") +probe process("beam.smp").mark("process-heap_grow") { printf("proc heap grow pid %s %d -> %d bytes\n", user_string($arg1), $arg2, $arg3); } -probe process("beam").mark("process-heap_shrink") +probe process("beam.smp").mark("process-heap_shrink") { printf("proc heap shrink pid %s %d -> %d bytes\n", user_string($arg1), $arg2, $arg3); diff --git a/lib/runtime_tools/examples/messages.systemtap b/lib/runtime_tools/examples/messages.systemtap index f2ef56a22b..49b7f46d69 100644 --- a/lib/runtime_tools/examples/messages.systemtap +++ b/lib/runtime_tools/examples/messages.systemtap @@ -18,15 +18,15 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ probe begin @@ -38,7 +38,7 @@ probe begin printf("\n"); } -probe process("beam").mark("message-send") +probe process("beam.smp").mark("message-send") { if ($arg4 == 0 && $arg5 == 0 && $arg6 == 0) { printf("send: %s -> %s: %d words\n", @@ -51,7 +51,7 @@ probe process("beam").mark("message-send") } } -probe process("beam").mark("message-send-remote") +probe process("beam.smp").mark("message-send-remote") { if ($arg5 == 0 && $arg6 == 0 && $arg7 == 0) { printf("send : %s -> %s %s: %d words\n", @@ -64,7 +64,7 @@ probe process("beam").mark("message-send-remote") } } -probe process("beam").mark("message-queued") +probe process("beam.smp").mark("message-queued") { if ($arg4 == 0 && $arg5 == 0 && $arg6 == 0) { printf("queued: %s: %d words, queue len %d\n", user_string($arg1), $arg2, $arg3); @@ -75,7 +75,7 @@ probe process("beam").mark("message-queued") } } -probe process("beam").mark("message-receive") +probe process("beam.smp").mark("message-receive") { if ($arg4 == 0 && $arg5 == 0 && $arg6 == 0) { printf("receive: %s: %d words, queue len %d\n", diff --git a/lib/runtime_tools/examples/port1.systemtap b/lib/runtime_tools/examples/port1.systemtap index f7ce03a65e..235581b0b1 100644 --- a/lib/runtime_tools/examples/port1.systemtap +++ b/lib/runtime_tools/examples/port1.systemtap @@ -18,15 +18,15 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ probe begin @@ -96,19 +96,19 @@ probe begin driver_map["udp_inet", 62] = "BINDX"; } -probe process("beam").mark("port-open") +probe process("beam.smp").mark("port-open") { printf("port open pid %s port name %s port %s\n", user_string($arg1), user_string($arg2), user_string($arg3)); } -probe process("beam").mark("port-command") +probe process("beam.smp").mark("port-command") { printf("port command pid %s port %s port name %s command type %s\n", user_string($arg1), user_string($arg2), user_string($arg3), user_string($arg4)); } -probe process("beam").mark("port-control") +probe process("beam.smp").mark("port-control") { cmd = driver_map[user_string($arg3), $arg4]; cmd_str = (cmd == "") ? "unknown" : cmd; @@ -118,36 +118,36 @@ probe process("beam").mark("port-control") /* port-exit is fired as a result of port_close() or exit signal */ -probe process("beam").mark("port-exit") +probe process("beam.smp").mark("port-exit") { printf("port exit pid %s port %s port name %s reason %s\n", user_string($arg1), user_string($arg2), user_string($arg3), user_string($arg4)); } -probe process("beam").mark("port-connect") +probe process("beam.smp").mark("port-connect") { printf("port connect pid %s port %s port name %s new pid %s\n", user_string($arg1), user_string($arg2), user_string($arg3), user_string($arg4)); } -probe process("beam").mark("port-busy") +probe process("beam.smp").mark("port-busy") { printf("port busy %s\n", user_string($arg1)); } -probe process("beam").mark("port-not_busy") +probe process("beam.smp").mark("port-not_busy") { printf("port not busy %s\n", user_string($arg1)); } -probe process("beam").mark("aio_pool-add") +probe process("beam.smp").mark("aio_pool-add") { printf("async I/O pool add thread %d queue len %d\n", $arg1, $arg2); } -probe process("beam").mark("aio_pool-get") +probe process("beam.smp").mark("aio_pool-get") { printf("async I/O pool get thread %d queue len %d\n", $arg1, $arg2); } -global driver_map;
\ No newline at end of file +global driver_map; diff --git a/lib/runtime_tools/examples/process-scheduling.systemtap b/lib/runtime_tools/examples/process-scheduling.systemtap index b0b74257b3..231c589f64 100644 --- a/lib/runtime_tools/examples/process-scheduling.systemtap +++ b/lib/runtime_tools/examples/process-scheduling.systemtap @@ -18,28 +18,28 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ -probe process("beam").mark("process-scheduled") +probe process("beam.smp").mark("process-scheduled") { printf(" Schedule pid %s mfa %s\n", user_string($arg1), user_string($arg2)); } -probe process("beam").mark("process-unscheduled") +probe process("beam.smp").mark("process-unscheduled") { printf("Unschedule pid %s\n", user_string($arg1)); } -probe process("beam").mark("process-hibernate") +probe process("beam.smp").mark("process-hibernate") { printf(" Hibernate pid %s resume mfa %s\n", user_string($arg1), user_string($arg2)); diff --git a/lib/runtime_tools/examples/spawn-exit.systemtap b/lib/runtime_tools/examples/spawn-exit.systemtap index 89bca14496..a7b4a0a3ea 100644 --- a/lib/runtime_tools/examples/spawn-exit.systemtap +++ b/lib/runtime_tools/examples/spawn-exit.systemtap @@ -18,34 +18,34 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ -probe process("beam").mark("process-spawn") +probe process("beam.smp").mark("process-spawn") { printf("pid %s mfa %s\n", user_string($arg1), user_string($arg2)); } -probe process("beam").mark("process-exit") +probe process("beam.smp").mark("process-exit") { printf("pid %s reason %s\n", user_string($arg1), user_string($arg2)); } -probe process("beam").mark("process-exit_signal") +probe process("beam.smp").mark("process-exit_signal") { printf("sender %s -> pid %s reason %s\n", user_string($arg1), user_string($arg2), user_string($arg3)); } -probe process("beam").mark("process-exit_signal-remote") +probe process("beam.smp").mark("process-exit_signal-remote") { printf("sender %s -> node %s pid %s reason %s\n", user_string($arg1), user_string($arg2), user_string($arg3), user_string($arg4)); diff --git a/lib/runtime_tools/examples/user-probe-n.systemtap b/lib/runtime_tools/examples/user-probe-n.systemtap index 25f7503283..8a0a89c931 100644 --- a/lib/runtime_tools/examples/user-probe-n.systemtap +++ b/lib/runtime_tools/examples/user-probe-n.systemtap @@ -18,18 +18,19 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ -probe process("beam").mark("user_trace-n0") + +probe process("beam.smp").mark("user_trace-n0") { printf("probe n0: %s %s %d %d %d %d '%s' '%s' '%s' '%s'\n", user_string($arg1), @@ -41,7 +42,7 @@ probe process("beam").mark("user_trace-n0") $arg9 == NULL ? "" : user_string($arg9)); } -probe process("beam").mark("user_trace-n1") +probe process("beam.smp").mark("user_trace-n1") { printf("probe n1: %s %s %d %d %d %d '%s' '%s' '%s' '%s'\n", user_string($arg1), diff --git a/lib/runtime_tools/examples/user-probe.systemtap b/lib/runtime_tools/examples/user-probe.systemtap index 1777476e54..ce9dde30f8 100644 --- a/lib/runtime_tools/examples/user-probe.systemtap +++ b/lib/runtime_tools/examples/user-probe.systemtap @@ -18,23 +18,23 @@ * %CopyrightEnd% */ /* - * Note: This file assumes that you're using the non-SMP-enabled Erlang - * virtual machine, "beam". The SMP-enabled VM is called "beam.smp". + * Note: This file assumes that you're using the SMP-enabled Erlang + * virtual machine, "beam.smp". * Note that other variations of the virtual machine also have * different names, e.g. the debug build of the SMP-enabled VM * is "beam.debug.smp". * * To use a different virtual machine, replace each instance of - * "beam" with "beam.smp" or the VM name appropriate to your - * environment. + * "beam.smp" with "beam.debug.smp" or the VM name appropriate + * to your environment. */ -probe process("beam").mark("user_trace-s1") +probe process("beam.smp").mark("user_trace-s1") { printf("%s\n", user_string($arg1)); } -probe process("beam").mark("user_trace-i4s4") +probe process("beam.smp").mark("user_trace-i4s4") { printf("%s %s %d %d %d %d '%s' '%s' '%s' '%s'\n", user_string($arg1), diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml index e8988ce487..2e1b946ebb 100644 --- a/lib/ssh/doc/src/notes.xml +++ b/lib/ssh/doc/src/notes.xml @@ -308,6 +308,21 @@ </section> </section> +<section><title>Ssh 4.6.9.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed port leakage if a ssh:daemon call failed.</p> + <p> + Own Id: OTP-15397 Aux Id: ERL-801 </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 4.6.9.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssh/src/ssh_sftpd.erl b/lib/ssh/src/ssh_sftpd.erl index 278f6a9780..aa9ba0f9bb 100644 --- a/lib/ssh/src/ssh_sftpd.erl +++ b/lib/ssh/src/ssh_sftpd.erl @@ -508,11 +508,8 @@ close_our_file({_,Fd}, FileMod, FS0) -> FS1. %%% stat: do the stat -stat(Vsn, ReqId, Data, State, F) when Vsn =< 3-> - <<?UINT32(BLen), BPath:BLen/binary>> = Data, - stat(ReqId, unicode:characters_to_list(BPath), State, F); -stat(Vsn, ReqId, Data, State, F) when Vsn >= 4-> - <<?UINT32(BLen), BPath:BLen/binary, ?UINT32(_Flags)>> = Data, +stat(Vsn, ReqId, Data, State, F) -> + <<?UINT32(BLen), BPath:BLen/binary, _/binary>> = Data, stat(ReqId, unicode:characters_to_list(BPath), State, F). fstat(Vsn, ReqId, Data, State) when Vsn =< 3-> diff --git a/lib/ssh/test/ssh_bench_SUITE.erl b/lib/ssh/test/ssh_bench_SUITE.erl index 764c52b624..2ac4e5636a 100644 --- a/lib/ssh/test/ssh_bench_SUITE.erl +++ b/lib/ssh/test/ssh_bench_SUITE.erl @@ -109,11 +109,10 @@ connect(Config) -> lists:foreach( fun(KexAlg) -> PrefAlgs = preferred_algorithms(KexAlg), - report([{value, measure_connect(Config, - [{preferred_algorithms,PrefAlgs}])}, - {suite, ?MODULE}, - {name, mk_name(["Connect erlc erld ",KexAlg," [µs]"])} - ]) + TimeMicroSec = measure_connect(Config, + [{preferred_algorithms,PrefAlgs}]), + report(["Connect erlc erld ",KexAlg," [connects per sec]"], + 1000000 / TimeMicroSec) end, KexAlgs). @@ -130,7 +129,7 @@ measure_connect(Config, Opts) -> [begin {Time, {ok,Pid}} = timer:tc(ssh,connect,["localhost", Port, ConnectOptions]), ssh:close(Pid), - Time + Time % in µs end || _ <- lists:seq(1,?Nruns)]). %%%---------------------------------------------------------------- @@ -178,10 +177,6 @@ gen_data(DataSz) -> <<Data0/binary, Data1/binary>>. -%% connect_measure(Port, Cipher, Mac, Data, Options) -> -%% report([{value, 1}, -%% {suite, ?MODULE}, -%% {name, mk_name(["Transfer 1M bytes ",Cipher,"/",Mac," [µs]"])}]); connect_measure(Port, Cipher, Mac, Data, Options) -> AES_GCM = {cipher, []}, @@ -220,10 +215,8 @@ connect_measure(Port, Cipher, Mac, Data, Options) -> ssh:close(C), Time end || _ <- lists:seq(1,?Nruns)], - - report([{value, median(Times)}, - {suite, ?MODULE}, - {name, mk_name(["Transfer 1M bytes ",Cipher,"/",Mac," [µs]"])}]). + report(["Transfer ",Cipher,"/",Mac," [Mbyte per sec]"], + 1000000 / median(Times)). send_wait_acc(C, Ch, Data) -> ssh_connection:send(C, Ch, Data), @@ -238,12 +231,6 @@ send_wait_acc(C, Ch, Data) -> %%% %%%---------------------------------------------------------------- -mk_name(Name) -> [char(C) || C <- lists:concat(Name)]. - -char($-) -> $_; -char(C) -> C. - -%%%---------------------------------------------------------------- preferred_algorithms(KexAlg) -> [{kex, [KexAlg]}, {public_key, ['ssh-rsa']}, @@ -265,11 +252,22 @@ median(Data) when is_list(Data) -> 1 -> lists:nth(N div 2 + 1, SortedData) end, - ct:log("median(~p) = ~p",[SortedData,Median]), + ct:pal("median(~p) = ~p",[SortedData,Median]), Median. +%%%---------------------------------------------------------------- +report(LabelList, Value) -> + Label = report_chars(lists:concat(LabelList)), + ct:pal("ct_event:notify ~p: ~p", [Label, Value]), + ct_event:notify( + #event{name = benchmark_data, + data = [{suite, ?MODULE}, + {name, Label}, + {value, Value}]}). + +report_chars(Cs) -> + [case C of + $- -> $_; + _ -> C + end || C <- Cs]. -report(Data) -> - ct:log("EventData = ~p",[Data]), - ct_event:notify(#event{name = benchmark_data, - data = Data}). diff --git a/lib/ssh/test/ssh_compat_SUITE.erl b/lib/ssh/test/ssh_compat_SUITE.erl index f4eef2dc77..8e82527c6e 100644 --- a/lib/ssh/test/ssh_compat_SUITE.erl +++ b/lib/ssh/test/ssh_compat_SUITE.erl @@ -1126,7 +1126,24 @@ prepare_local_directory(ServerRootDir) -> "chmod 222 unreadable_file", "exit"]. + check_local_directory(ServerRootDir) -> + TimesToTry = 3, % sleep 0.5, 1, 2 and then 4 secs (7.5s in total) + check_local_directory(ServerRootDir, 500, TimesToTry-1). + +check_local_directory(ServerRootDir, SleepTime, N) -> + case do_check_local_directory(ServerRootDir) of + {error,Error} when N>0 -> + %% Could be that the erlang side is faster and the docker's operations + %% are not yet finalized. + %% Sleep for a while and retry a few times: + timer:sleep(SleepTime), + check_local_directory(ServerRootDir, 2*SleepTime, N-1); + Other -> + Other + end. + +do_check_local_directory(ServerRootDir) -> case lists:sort(ok(file:list_dir(ServerRootDir)) -- [".",".."]) of ["ex_tst1","mydir","tst2"] -> {ok,Expect} = file:read_file(filename:join(ServerRootDir,"ex_tst1")), @@ -1161,6 +1178,7 @@ check_local_directory(ServerRootDir) -> {error,{bad_dir_contents,"/"}} end. + call_sftp_in_docker(Config, ServerIP, ServerPort, Cmnds, UserDir) -> {DockerIP,DockerPort} = ip_port(Config), {ok,C} = ssh:connect(DockerIP, DockerPort, diff --git a/lib/ssh/test/ssh_trpt_test_lib.erl b/lib/ssh/test/ssh_trpt_test_lib.erl index 8de550af15..f2c9892f95 100644 --- a/lib/ssh/test/ssh_trpt_test_lib.erl +++ b/lib/ssh/test/ssh_trpt_test_lib.erl @@ -41,15 +41,20 @@ opts = [], timeout = 5000, % ms seen_hello = false, - enc = <<>>, ssh = #ssh{}, % #ssh{} alg_neg = {undefined,undefined}, % {own_kexinit, peer_kexinit} alg, % #alg{} vars = dict:new(), reply = [], % Some repy msgs are generated hidden in ssh_transport :[ prints = [], - return_value - }). + return_value, + + %% Packet retrival and decryption + decrypted_data_buffer = <<>>, + encrypted_data_buffer = <<>>, + aead_data = <<>>, + undecrypted_packet_length + }). -define(role(S), ((S#s.ssh)#ssh.role) ). @@ -475,11 +480,11 @@ recv(S0 = #s{}) -> %%%================================================================ try_find_crlf(Seen, S0) -> - case erlang:decode_packet(line,S0#s.enc,[]) of + case erlang:decode_packet(line,S0#s.encrypted_data_buffer,[]) of {more,_} -> - Line = <<Seen/binary,(S0#s.enc)/binary>>, + Line = <<Seen/binary,(S0#s.encrypted_data_buffer)/binary>>, S0#s{seen_hello = {more,Line}, - enc = <<>>, % didn't find a complete line + encrypted_data_buffer = <<>>, % didn't find a complete line % -> no more characters to test return_value = {more,Line} }; @@ -490,13 +495,13 @@ try_find_crlf(Seen, S0) -> S = opt(print_messages, S0, fun(X) when X==true;X==detail -> {"Recv info~n~p~n",[Line]} end), S#s{seen_hello = false, - enc = Rest, + encrypted_data_buffer = Rest, return_value = {info,Line}}; S1=#s{} -> S = opt(print_messages, S1, fun(X) when X==true;X==detail -> {"Recv hello~n~p~n",[Line]} end), S#s{seen_hello = true, - enc = Rest, + encrypted_data_buffer = Rest, return_value = {hello,Line}} end end. @@ -511,19 +516,73 @@ handle_hello(Bin, S=#s{ssh=C}) -> {{Vp,Vs}, server} -> S#s{ssh = C#ssh{c_vsn=Vp, c_version=Vs}} end. -receive_binary_msg(S0=#s{ssh=C0=#ssh{decrypt_block_size = BlockSize, +receive_binary_msg(S0=#s{}) -> + case ssh_transport:handle_packet_part( + S0#s.decrypted_data_buffer, + S0#s.encrypted_data_buffer, + S0#s.aead_data, + S0#s.undecrypted_packet_length, + S0#s.ssh) + of + {packet_decrypted, DecryptedBytes, EncryptedDataRest, Ssh1} -> + S1 = S0#s{ssh = Ssh1#ssh{recv_sequence = ssh_transport:next_seqnum(Ssh1#ssh.recv_sequence)}, + decrypted_data_buffer = <<>>, + undecrypted_packet_length = undefined, + aead_data = <<>>, + encrypted_data_buffer = EncryptedDataRest}, + case + catch ssh_message:decode(set_prefix_if_trouble(DecryptedBytes,S1)) + of + {'EXIT',_} -> fail(decode_failed,S1); + + Msg -> + Ssh2 = case Msg of + #ssh_msg_kexinit{} -> + ssh_transport:key_init(opposite_role(Ssh1), Ssh1, DecryptedBytes); + _ -> + Ssh1 + end, + S2 = opt(print_messages, S1, + fun(X) when X==true;X==detail -> {"Recv~n~s~n",[format_msg(Msg)]} end), + S3 = opt(print_messages, S2, + fun(detail) -> {"decrypted bytes ~p~n",[DecryptedBytes]} end), + S3#s{ssh = inc_recv_seq_num(Ssh2), + return_value = Msg + } + end; + + {get_more, DecryptedBytes, EncryptedDataRest, AeadData, TotalNeeded, Ssh1} -> + %% Here we know that there are not enough bytes in + %% EncryptedDataRest to use. We must wait for more. + Remaining = case TotalNeeded of + undefined -> 8; + _ -> TotalNeeded - size(DecryptedBytes) - size(EncryptedDataRest) + end, + receive_binary_msg( + receive_wait(Remaining, + S0#s{encrypted_data_buffer = EncryptedDataRest, + decrypted_data_buffer = DecryptedBytes, + undecrypted_packet_length = TotalNeeded, + aead_data = AeadData, + ssh = Ssh1} + )) + end. + + + +old_receive_binary_msg(S0=#s{ssh=C0=#ssh{decrypt_block_size = BlockSize, recv_mac_size = MacSize } }) -> - case size(S0#s.enc) >= max(8,BlockSize) of + case size(S0#s.encrypted_data_buffer) >= max(8,BlockSize) of false -> %% Need more bytes to decode the packet_length field - Remaining = max(8,BlockSize) - size(S0#s.enc), + Remaining = max(8,BlockSize) - size(S0#s.encrypted_data_buffer), receive_binary_msg( receive_wait(Remaining, S0) ); true -> %% Has enough bytes to decode the packet_length field {_, <<?UINT32(PacketLen), _/binary>>, _} = - ssh_transport:decrypt_blocks(S0#s.enc, BlockSize, C0), % FIXME: BlockSize should be at least 4 + ssh_transport:decrypt_blocks(S0#s.encrypted_data_buffer, BlockSize, C0), % FIXME: BlockSize should be at least 4 %% FIXME: Check that ((4+PacketLen) rem BlockSize) == 0 ? @@ -534,19 +593,19 @@ receive_binary_msg(S0=#s{ssh=C0=#ssh{decrypt_block_size = BlockSize, ((4+PacketLen) rem BlockSize) =/= 0 -> fail(bad_packet_length_modulo, S0); % FIXME: disconnect - size(S0#s.enc) >= (4 + PacketLen + MacSize) -> + size(S0#s.encrypted_data_buffer) >= (4 + PacketLen + MacSize) -> %% has the whole packet S0; true -> %% need more bytes to get have the whole packet - Remaining = (4 + PacketLen + MacSize) - size(S0#s.enc), + Remaining = (4 + PacketLen + MacSize) - size(S0#s.encrypted_data_buffer), receive_wait(Remaining, S0) end, %% Decrypt all, including the packet_length part (re-use the initial #ssh{}) {C1, SshPacket = <<?UINT32(_),?BYTE(PadLen),Tail/binary>>, EncRest} = - ssh_transport:decrypt_blocks(S1#s.enc, PacketLen+4, C0), + ssh_transport:decrypt_blocks(S1#s.encrypted_data_buffer, PacketLen+4, C0), PayloadLen = PacketLen - 1 - PadLen, <<CompressedPayload:PayloadLen/binary, _Padding:PadLen/binary>> = Tail, @@ -573,7 +632,7 @@ receive_binary_msg(S0=#s{ssh=C0=#ssh{decrypt_block_size = BlockSize, S3 = opt(print_messages, S2, fun(detail) -> {"decrypted bytes ~p~n",[SshPacket]} end), S3#s{ssh = inc_recv_seq_num(C3), - enc = Rest, + encrypted_data_buffer = Rest, return_value = Msg } end @@ -602,7 +661,7 @@ receive_poll(S=#s{socket=Sock}) -> inet:setopts(Sock, [{active,once}]), receive {tcp,Sock,Data} -> - receive_poll( S#s{enc = <<(S#s.enc)/binary,Data/binary>>} ); + receive_poll( S#s{encrypted_data_buffer = <<(S#s.encrypted_data_buffer)/binary,Data/binary>>} ); {tcp_closed,Sock} -> throw({tcp,tcp_closed}); {tcp_error, Sock, Reason} -> @@ -616,7 +675,7 @@ receive_wait(S=#s{socket=Sock, inet:setopts(Sock, [{active,once}]), receive {tcp,Sock,Data} -> - S#s{enc = <<(S#s.enc)/binary,Data/binary>>}; + S#s{encrypted_data_buffer = <<(S#s.encrypted_data_buffer)/binary,Data/binary>>}; {tcp_closed,Sock} -> throw({tcp,tcp_closed}); {tcp_error, Sock, Reason} -> @@ -627,11 +686,11 @@ receive_wait(S=#s{socket=Sock, receive_wait(N, S=#s{socket=Sock, timeout=Timeout, - enc=Enc0}) when N>0 -> + encrypted_data_buffer=Enc0}) when N>0 -> inet:setopts(Sock, [{active,once}]), receive {tcp,Sock,Data} -> - receive_wait(N-size(Data), S#s{enc = <<Enc0/binary,Data/binary>>}); + receive_wait(N-size(Data), S#s{encrypted_data_buffer = <<Enc0/binary,Data/binary>>}); {tcp_closed,Sock} -> throw({tcp,tcp_closed}); {tcp_error, Sock, Reason} -> diff --git a/lib/ssl/doc/specs/.gitignore b/lib/ssl/doc/specs/.gitignore new file mode 100644 index 0000000000..322eebcb06 --- /dev/null +++ b/lib/ssl/doc/specs/.gitignore @@ -0,0 +1 @@ +specs_*.xml diff --git a/lib/ssl/doc/src/Makefile b/lib/ssl/doc/src/Makefile index c72b6d6cc4..7cf251d8f9 100644 --- a/lib/ssl/doc/src/Makefile +++ b/lib/ssl/doc/src/Makefile @@ -80,11 +80,16 @@ HTML_REF_MAN_FILE = $(HTMLDIR)/index.html TOP_PDF_FILE = $(PDFDIR)/$(APPLICATION)-$(VSN).pdf +SPECS_FILES = $(XML_REF3_FILES:%.xml=$(SPECDIR)/specs_%.xml) + +TOP_SPECS_FILE = specs.xml + # ---------------------------------------------------- # FLAGS # ---------------------------------------------------- XML_FLAGS += DVIPS_FLAGS += +SPECS_FLAGS = -I../../../public_key/include -I../../../public_key/src -I../../.. # ---------------------------------------------------- # Targets @@ -92,7 +97,7 @@ DVIPS_FLAGS += $(HTMLDIR)/%.gif: %.gif $(INSTALL_DATA) $< $@ -docs: pdf html man +docs: html pdf man $(TOP_PDF_FILE): $(XML_FILES) @@ -105,6 +110,7 @@ clean clean_docs: rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) + rm -f $(SPECS_FILES) rm -f errs core *~ man: $(MAN3_FILES) $(MAN6_FILES) diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 854ab31883..674e38b054 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -27,6 +27,37 @@ </header> <p>This document describes the changes made to the SSL application.</p> +<section><title>SSL 9.1.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix encoding of the SRP extension length field in ssl. + The old encoding of the SRP extension length could cause + interoperability problems with third party SSL + implementations when SRP was used.</p> + <p> + Own Id: OTP-15477 Aux Id: ERL-790 </p> + </item> + <item> + <p> + Guarantee active once data delivery, handling TCP stream + properly.</p> + <p> + Own Id: OTP-15504 Aux Id: ERL-371 </p> + </item> + <item> + <p> + Correct gen_statem returns for some error cases</p> + <p> + Own Id: OTP-15505</p> + </item> + </list> + </section> + +</section> + <section><title>SSL 9.1.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssl/doc/src/specs.xml b/lib/ssl/doc/src/specs.xml new file mode 100644 index 0000000000..50e9428fec --- /dev/null +++ b/lib/ssl/doc/src/specs.xml @@ -0,0 +1,9 @@ +<?xml version="1.0" encoding="utf-8" ?> +<specs xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="../specs/specs_ssl_crl_cache_api.xml"/> + <xi:include href="../specs/specs_ssl_crl_cache.xml"/> + <xi:include href="../specs/specs_ssl_session_cache_api.xml"/> + <xi:include href="../specs/specs_ssl.xml"/> +</specs> + + diff --git a/lib/ssl/doc/src/ssl.xml b/lib/ssl/doc/src/ssl.xml index b4aa8746f9..67ec93809f 100644 --- a/lib/ssl/doc/src/ssl.xml +++ b/lib/ssl/doc/src/ssl.xml @@ -37,277 +37,334 @@ <seealso marker="ssl_app">ssl(6)</seealso>. </p> </description> - - <section> - <title>DATA TYPES</title> - <p>The following data types are used in the functions for SSL/TLS/DTLS:</p> - - <taglist> - - <tag><c>boolean() =</c></tag> - <item><p><c>true | false</c></p></item> - - <tag><c>option() =</c></tag> - <item><p><c>socketoption() | ssl_option() | transport_option()</c></p> - </item> - - <tag><c>socketoption() =</c></tag> - <item><p><c>proplists:property()</c></p> - <p>The default socket options are - <c>[{mode,list},{packet, 0},{header, 0},{active, true}]</c>.</p> - <p>For valid options, see the - <seealso marker="kernel:inet">inet(3)</seealso>, - <seealso marker="kernel:gen_tcp">gen_tcp(3)</seealso> and - <seealso marker="kernel:gen_tcp">gen_udp(3)</seealso> - manual pages - in Kernel. Note that stream oriented options such as packet are only relevant for SSL/TLS and not DTLS</p></item> - - <tag><marker id="type-ssloption"/><c>ssl_option() =</c></tag> - <item> - <p><c>{verify, verify_type()}</c></p> - <p><c>| {verify_fun, {fun(), term()}}</c></p> - <p><c>| {fail_if_no_peer_cert, boolean()}</c></p> - <p><c>| {depth, integer()}</c></p> - <p><c>| {cert, public_key:der_encoded()}</c></p> - <p><c>| {certfile, path()}</c></p> - <p><c>| {key, {'RSAPrivateKey'| 'DSAPrivateKey' | 'ECPrivateKey' - | 'PrivateKeyInfo', public_key:der_encoded()} | - #{algorithm := rsa | dss | ecdsa, - engine := crypto:engine_ref(), key_id := crypto:key_id(), password => crypto:password()}</c></p> - <p><c>| {keyfile, path()}</c></p> - <p><c>| {password, string()}</c></p> - <p><c>| {cacerts, [public_key:der_encoded()]}</c></p> - <p><c>| {cacertfile, path()}</c></p> - <p><c>| {dh, public_key:der_encoded()}</c></p> - <p><c>| {dhfile, path()}</c></p> - <p><c>| {ciphers, ciphers()}</c></p> - <p><c>| {user_lookup_fun, {fun(), term()}}, {psk_identity, string()}, - {srp_identity, {string(), string()}}</c></p> - <p><c>| {reuse_sessions, boolean()}</c></p> - <p><c>| {reuse_session, fun()} {next_protocols_advertised, [binary()]}</c></p> - <p><c>| {client_preferred_next_protocols, {client | server, - [binary()]} | {client | server, [binary()], binary()}}</c></p> - <p><c>| {log_alert, boolean()}</c></p> - <p><c>| {server_name_indication, hostname() | disable}</c></p> - <p><c>| {customize_hostname_check, list()}</c></p> - <p><c>| {sni_hosts, [{hostname(), [ssl_option()]}]}</c></p> - <p><c>| {sni_fun, SNIfun::fun()}</c></p> - </item> - - <tag><c>transport_option() =</c></tag> - <item><p><c>{cb_info, {CallbackModule::atom(), DataTag::atom(), - - ClosedTag::atom(), ErrTag:atom()}}</c></p> - <p>Defaults to <c>{gen_tcp, tcp, tcp_closed, tcp_error}</c> for TLS - and <c>{gen_udp, udp, udp_closed, udp_error}</c> for DTLS. Can be used - to customize the transport layer. For TLS the callback module must implement a - reliable transport protocol, behave as <c>gen_tcp</c>, and have functions - corresponding to <c>inet:setopts/2</c>, <c>inet:getopts/2</c>, - <c>inet:peername/1</c>, <c>inet:sockname/1</c>, and <c>inet:port/1</c>. - The callback <c>gen_tcp</c> is treated specially and calls <c>inet</c> - directly. For DTLS this feature must be considered exprimental.</p> - <taglist> - <tag><c>CallbackModule =</c></tag> - <item><p><c>atom()</c></p></item> - <tag><c>DataTag =</c></tag> - <item><p><c>atom()</c></p> - <p>Used in socket data message.</p></item> - <tag><c>ClosedTag =</c></tag> - <item><p><c>atom()</c></p> - <p>Used in socket close message.</p></item> - </taglist> - </item> - - <tag><c>verify_type() =</c></tag> - <item><p><c>verify_none | verify_peer</c></p></item> - - <tag><c>path() =</c></tag> - <item><p><c>string()</c></p> - <p>Represents a file path.</p></item> - <tag><c>public_key:der_encoded() =</c></tag> - <item><p><c>binary()</c></p> - <p>ASN.1 DER-encoded entity as an Erlang binary.</p></item> + <!-- + ================================================================ + = Data types = + ================================================================ + --> - <tag><c>host() =</c></tag> - <item><p><c>hostname() | ipaddress()</c></p></item> + <datatypes> + <datatype_title>Types used in SSL/TLS/DTLS</datatype_title> - <tag><c>hostname() =</c></tag> - <item><p><c>string() - DNS hostname</c></p></item> - - <tag><c>ip_address() =</c></tag> - <item><p><c>{N1,N2,N3,N4} % IPv4 | {K1,K2,K3,K4,K5,K6,K7,K8} % IPv6 - </c></p></item> + + <datatype> + <name name="socket"/> + </datatype> + + <datatype> + <name name="sslsocket"/> + <desc> + <p>An opaque reference to the TLS/DTLS connection, may be used for equality matching.</p> + </desc> + </datatype> + + <datatype> + <name name="tls_option"/> + </datatype> + + <datatype> + <name name="tls_client_option"/> + </datatype> + + <datatype> + <name name="tls_server_option"/> + </datatype> + + + <datatype> + <name name="socket_option"/> + <desc> + <p>The default socket options are + <c>[{mode,list},{packet, 0},{header, 0},{active, true}]</c>.</p> + <p>For valid options, see the + <seealso marker="kernel:inet">inet(3)</seealso>, + <seealso marker="kernel:gen_tcp">gen_tcp(3)</seealso> and + <seealso marker="kernel:gen_tcp">gen_udp(3)</seealso> + manual pages in Kernel. Note that stream oriented options such as packet + are only relevant for SSL/TLS and not DTLS</p> + </desc> + </datatype> - <tag><c>sslsocket() =</c></tag> - <item><p>opaque()</p></item> - - <tag><marker id="type-protocol"/><c> protocol_version() =</c></tag> - <item><p><c> ssl_tls_protocol() | dtls_protocol() </c></p></item> - - <item><p><c>sslv3 | tlsv1 | 'tlsv1.1' | 'tlsv1.2'</c></p></item> - - <tag><marker id="type-protocol"/><c> dtls_protocol() =</c></tag> - <item><p><c>'dtlsv1' | 'dtlsv1.2'</c></p></item> - - <tag><c>ciphers() =</c></tag> - <item><p><c>= [ciphersuite()]</c></p> - <p>Tuples and string formats accepted by versions - before ssl-8.2.4 will be converted for backwards compatibility</p></item> - - <tag><c>ciphersuite() =</c></tag> - <item><p><c> - #{key_exchange := key_exchange(), - cipher := cipher(), - mac := MAC::hash() | aead, - prf := PRF::hash() | default_prf} </c></p></item> - - <tag><c>key_exchange()=</c></tag> - <item><p><c>rsa | dhe_dss | dhe_rsa | dh_anon | psk | dhe_psk - | rsa_psk | srp_anon | srp_dss | srp_rsa | ecdh_anon | ecdh_ecdsa - | ecdhe_ecdsa | ecdh_rsa | ecdhe_rsa</c></p></item> - - <tag><c>cipher() =</c></tag> - <item><p><c>rc4_128 | des_cbc | '3des_ede_cbc' - | aes_128_cbc | aes_256_cbc | aes_128_gcm | aes_256_gcm | chacha20_poly1305</c></p></item> - - <tag><c>hash() =</c></tag> - <item><p><c>md5 | sha | sha224 | sha256 | sha348 | sha512</c></p></item> - - <tag><c>prf_random() =</c></tag> - <item><p><c>client_random | server_random</c></p></item> - - <tag><c>cipher_filters() =</c></tag> - <item><p><c> [{key_exchange | cipher | mac | prf, algo_filter()}])</c></p></item> - - <tag><c>algo_filter() =</c></tag> - <item><p>fun(key_exchange() | cipher() | hash() | aead | default_prf) -> true | false </p></item> - - <tag><c>srp_param_type() =</c></tag> - <item><p><c>srp_1024 | srp_1536 | srp_2048 | srp_3072 - | srp_4096 | srp_6144 | srp_8192</c></p></item> - - <tag><c>SNIfun::fun()</c></tag> - <item><p><c>= fun(ServerName :: string()) -> [ssl_option()]</c></p></item> - - <tag><c>named_curve() =</c></tag> - <item><p><c>sect571r1 | sect571k1 | secp521r1 | brainpoolP512r1 - | sect409k1 | sect409r1 | brainpoolP384r1 | secp384r1 - | sect283k1 | sect283r1 | brainpoolP256r1 | secp256k1 | secp256r1 - | sect239k1 | sect233k1 | sect233r1 | secp224k1 | secp224r1 - | sect193r1 | sect193r2 | secp192k1 | secp192r1 | sect163k1 - | sect163r1 | sect163r2 | secp160k1 | secp160r1 | secp160r2</c></p></item> - - <tag><c>hello_extensions() =</c></tag> - <item><p><c>#{renegotiation_info => binary() | undefined, - signature_algs => [{hash(), ecsda| rsa| dsa}] | undefined - alpn => binary() | undefined, - next_protocol_negotiation => binary() | undefined, - srp => string() | undefined, - ec_point_formats => list() | undefined, - elliptic_curves => [oid] | undefined, - sni => string() | undefined} - }</c></p></item> + <datatype> + <name name="active_msgs"/> + <desc> + <p>When an TLS/DTLS socket is in active mode (the default), data from the + socket is delivered to the owner of the socket in the form of + messages as described above.</p> + </desc> + </datatype> - - </taglist> - </section> - - <section> - <title>TLS/DTLS OPTION DESCRIPTIONS - COMMON for SERVER and CLIENT</title> - - <p>The following options have the same meaning in the client and - the server:</p> + <datatype> + <name name="transport_option"/> + <desc> + <p>Defaults to <c>{gen_tcp, tcp, tcp_closed, tcp_error}</c> + for TLS and <c>{gen_udp, udp, udp_closed, udp_error}</c> for + DTLS. Can be used to customize the transport layer. The tag + values should be the values used by the underlying transport + in its active mode messages. For TLS the callback module must implement a + reliable transport protocol, behave as <c>gen_tcp</c>, and have functions + corresponding to <c>inet:setopts/2</c>, <c>inet:getopts/2</c>, + <c>inet:peername/1</c>, <c>inet:sockname/1</c>, and <c>inet:port/1</c>. + The callback <c>gen_tcp</c> is treated specially and calls <c>inet</c> + directly. For DTLS this feature must be considered exprimental. + </p> + </desc> + </datatype> + + <datatype> + <name name="host"/> + </datatype> + + <datatype> + <name name="hostname"/> + </datatype> + + <datatype> + <name name="ip_address"/> + </datatype> + + <datatype> + <name name="protocol_version"/> + </datatype> + + <datatype> + <name name="tls_version"/> + </datatype> + + <datatype> + <name name="dtls_version"/> + </datatype> + + <datatype> + <name name="legacy_version"/> + </datatype> + + <datatype> + <name name="prf_random"/> + </datatype> + + <datatype> + <name name="verify_type"/> + </datatype> + + <datatype> + <name name="ciphers"/> + </datatype> + + <datatype> + <name name="erl_cipher_suite"/> + </datatype> + + <datatype> + <name name="cipher"/> + </datatype> - <taglist> - - <tag><c>{protocol, tls | dtls}</c></tag> - <item><p>Choose TLS or DTLS protocol for the transport layer security. - Defaults to <c>tls</c> Introduced in OTP 20, DTLS support is considered - experimental in this release. Other transports than UDP are not yet supported.</p></item> - - <tag><c>{handshake, hello | full}</c></tag> - <item><p> Defaults to <c>full</c>. If hello is specified the handshake will - pause after the hello message and give the user a possibility make decisions - based on hello extensions before continuing or aborting the handshake by calling - <seealso marker="#handshake_continue-3"> handshake_continue/3</seealso> or - <seealso marker="#handshake_cancel-1"> handshake_cancel/1</seealso> - </p></item> - - <tag><c>{cert, public_key:der_encoded()}</c></tag> - <item><p>The DER-encoded users certificate. If this option - is supplied, it overrides option <c>certfile</c>.</p></item> - - <tag><c>{certfile, path()}</c></tag> - <item><p>Path to a file containing the user certificate.</p></item> - - <tag> - <marker id="key_option_def"/> - <c>{key, {'RSAPrivateKey'| 'DSAPrivateKey' | 'ECPrivateKey' - |'PrivateKeyInfo', public_key:der_encoded()} | #{algorithm := rsa | dss | ecdsa, - engine := crypto:engine_ref(), key_id := crypto:key_id(), password => crypto:password()}</c></tag> - <item><p>The DER-encoded user's private key or a map refering to a crypto - engine and its key reference that optionally can be password protected, - seealso <seealso marker="crypto:crypto#engine_load-4"> crypto:engine_load/4 - </seealso> and <seealso marker="crypto:engine_load"> Crypto's Users Guide</seealso>. If this option - is supplied, it overrides option <c>keyfile</c>.</p></item> - - <tag><c>{keyfile, path()}</c></tag> - <item><p>Path to the file containing the user's - private PEM-encoded key. As PEM-files can contain several - entries, this option defaults to the same file as given by - option <c>certfile</c>.</p></item> - - <tag><c>{password, string()}</c></tag> - <item><p>String containing the user's password. Only used if the - private keyfile is password-protected.</p></item> - - <tag><c>{ciphers, ciphers()}</c></tag> - <item><p>Supported cipher suites. The function - <c>cipher_suites/0</c> can be used to find all ciphers that are - supported by default. <c>cipher_suites(all)</c> can be called - to find all available cipher suites. Pre-Shared Key - (<url href="http://www.ietf.org/rfc/rfc4279.txt">RFC 4279</url> and - <url href="http://www.ietf.org/rfc/rfc5487.txt">RFC 5487</url>), - Secure Remote Password - (<url href="http://www.ietf.org/rfc/rfc5054.txt">RFC 5054</url>), RC4 cipher suites, - and anonymous cipher suites only work if explicitly enabled by - this option; they are supported/enabled by the peer also. - Anonymous cipher suites are supported for testing purposes - only and are not be used when security matters.</p></item> - - <tag><c>{eccs, [named_curve()]}</c></tag> - <item><p> Allows to specify the order of preference for named curves - and to restrict their usage when using a cipher suite supporting them. - </p></item> - - <tag><c>{secure_renegotiate, boolean()}</c></tag> - <item><p>Specifies if to reject renegotiation attempt that does - not live up to - <url href="http://www.ietf.org/rfc/rfc5746.txt">RFC 5746</url>. - By default <c>secure_renegotiate</c> is set to <c>true</c>, - that is, secure renegotiation is enforced. If set to <c>false</c> secure renegotiation - will still be used if possible, - but it falls back to insecure renegotiation if the peer - does not support - <url href="http://www.ietf.org/rfc/rfc5746.txt">RFC 5746</url>.</p> - </item> - - <tag><c>{depth, integer()}</c></tag> - <item><p>Maximum number of non-self-issued + <datatype> + <name name="legacy_cipher"/> + </datatype> + + <datatype> + <name name="cipher_filters"/> + </datatype> + + <datatype> + <name name="hash"/> + </datatype> + + <datatype> + <name name="sha2"/> + </datatype> + + <datatype> + <name name="legacy_hash"/> + </datatype> + + <datatype> + <name name="old_cipher_suite"/> + </datatype> + + <datatype> + <name name="signature_algs"/> + </datatype> + + <datatype> + <name name="sign_algo"/> + </datatype> + + <datatype> + <name name="kex_algo"/> + </datatype> + + <datatype> + <name name="algo_filter"/> + </datatype> + + <datatype> + <name name="eccs"/> + </datatype> + + <datatype> + <name name="named_curve"/> + </datatype> + + <datatype> + <name name="psk_identity"/> + </datatype> + + <datatype> + <name name="srp_identity"/> + </datatype> + + <datatype> + <name name="srp_param_type"/> + </datatype> + + <datatype> + <name name="app_level_protocol"/> + </datatype> + + <datatype> + <name name="protocol_extensions"/> + </datatype> + + <datatype> + <name name="error_alert"/> + </datatype> + + <datatype> + <name name="tls_alert"/> + </datatype> + + <datatype_title>TLS/DTLS OPTION DESCRIPTIONS - COMMON for SERVER and CLIENT</datatype_title> + + <datatype> + <name name="common_option"/> + </datatype> + + <datatype> + <name since="OTP 20" name="protocol"/> + <desc> + <p>Choose TLS or DTLS protocol for the transport layer security. + Defaults to <c>tls</c>. For DTLS other transports than UDP are not yet supported.</p> + </desc> + </datatype> + + <datatype> + <name name="handshake_completion"/> + <desc> + <p>Defaults to <c>full</c>. If hello is specified the handshake will + pause after the hello message and give the user a possibility make decisions + based on hello extensions before continuing or aborting the handshake by calling + <seealso marker="#handshake_continue-3"> handshake_continue/3</seealso> or + <seealso marker="#handshake_cancel-1"> handshake_cancel/1</seealso></p> + </desc> + </datatype> + + <datatype> + <name name="cert"/> + <desc> + <p>The DER-encoded users certificate. If this option + is supplied, it overrides option <c>certfile</c>.</p> + </desc> + </datatype> + + <datatype> + <name name="cert_pem"/> + <desc> + <p>Path to a file containing the user certificate on PEM format.</p> + </desc> + </datatype> + + <datatype> + <name name="key"/> + <desc> + <p>The DER-encoded user's private key or a map refering to a crypto + engine and its key reference that optionally can be password protected, + seealso <seealso marker="crypto:crypto#engine_load-4"> crypto:engine_load/4 + </seealso> and <seealso marker="crypto:engine_load"> Crypto's Users Guide</seealso>. If this option + is supplied, it overrides option <c>keyfile</c>.</p> + </desc> + </datatype> + + <datatype> + <name name="key_pem"/> + <desc> + <p>Path to the file containing the user's + private PEM-encoded key. As PEM-files can contain several + entries, this option defaults to the same file as given by + option <c>certfile</c>.</p> + </desc> + </datatype> + + <datatype> + <name name="key_password"/> + <desc> + <p>String containing the user's password. Only used if the + private keyfile is password-protected.</p> + </desc> + </datatype> + + <datatype> + <name name="cipher_suites"/> + <desc> + <p>Supported cipher suites. The function + <c>cipher_suites/2</c> can be used to find all ciphers that + are supported by default. <c>cipher_suites(all, 'tlsv1.2')</c> can be + called to find all available cipher suites. Pre-Shared Key + (<url href="http://www.ietf.org/rfc/rfc4279.txt">RFC + 4279</url> and <url + href="http://www.ietf.org/rfc/rfc5487.txt">RFC 5487</url>), + Secure Remote Password (<url + href="http://www.ietf.org/rfc/rfc5054.txt">RFC 5054</url>), + RC4, 3DES, DES cipher suites, and anonymous cipher suites only work if + explicitly enabled by this option; they are supported/enabled + by the peer also. Anonymous cipher suites are supported for + testing purposes only and are not be used when security + matters.</p> + </desc> + </datatype> + + <datatype> + <name name="eccs"/> + <desc><p> Allows to specify the order of preference for named curves + and to restrict their usage when using a cipher suite supporting them.</p> + </desc> + </datatype> + + <datatype> + <name name="secure_renegotiation"/> + <desc><p>Specifies if to reject renegotiation attempt that does + not live up to <url + href="http://www.ietf.org/rfc/rfc5746.txt">RFC 5746</url>. By + default <c>secure_renegotiate</c> is set to <c>true</c>, that + is, secure renegotiation is enforced. If set to <c>false</c> + secure renegotiation will still be used if possible, but it + falls back to insecure renegotiation if the peer does not + support <url href="http://www.ietf.org/rfc/rfc5746.txt">RFC + 5746</url>.</p> + </desc> + </datatype> + + <datatype> + <name name="allowed_cert_chain_length"/> + <desc><p>Maximum number of non-self-issued intermediate certificates that can follow the peer certificate in a valid certification path. So, if depth is 0 the PEER must be signed by the trusted ROOT-CA directly; if 1 the path can be PEER, CA, ROOT-CA; if 2 the path can be PEER, CA, CA, - ROOT-CA, and so on. The default value is 1.</p></item> - - <tag><marker id="verify_fun"/><c>{verify_fun, {Verifyfun :: fun(), InitialUserState :: - term()}}</c></tag> - <item><p>The verification fun is to be defined as follows:</p> + ROOT-CA, and so on. The default value is 1.</p> + </desc> + </datatype> + + <datatype> + <name name="custom_verify"/> + <desc> + <p>The verification fun is to be defined as follows:</p> <code> -fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom() | {revoked, -atom()}} | +fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom() | + {revoked, atom()}} | {extension, #'Extension'{}}, InitialUserState :: term()) -> {valid, UserState :: term()} | {valid_peer, UserState :: term()} | {fail, Reason :: term()} | {unknown, UserState :: term()}. @@ -315,20 +372,21 @@ atom()}} | <p>The verification fun is called during the X509-path validation when an error or an extension unknown to the SSL - application is encountered. It is also called - when a certificate is considered valid by the path validation - to allow access to each certificate in the path to the user - application. It differentiates between the peer - certificate and the CA certificates by using <c>valid_peer</c> or - <c>valid</c> as second argument to the verification fun. See the - <seealso marker="public_key:public_key_records">public_key User's - Guide</seealso> for definition of <c>#'OTPCertificate'{}</c> and - <c>#'Extension'{}</c>.</p> + application is encountered. It is also called when a + certificate is considered valid by the path validation to + allow access to each certificate in the path to the user + application. It differentiates between the peer certificate + and the CA certificates by using <c>valid_peer</c> or + <c>valid</c> as second argument to the verification fun. See + the <seealso marker="public_key:public_key_records">public_key + User's Guide</seealso> for definition of + <c>#'OTPCertificate'{}</c> and <c>#'Extension'{}</c>.</p> <list type="bulleted"> - <item><p>If the verify callback fun returns <c>{fail, Reason}</c>, - the verification process is immediately stopped, an alert is - sent to the peer, and the TLS/DTLS handshake terminates.</p></item> + <item><p>If the verify callback fun returns <c>{fail, + Reason}</c>, the verification process is immediately + stopped, an alert is sent to the peer, and the TLS/DTLS + handshake terminates.</p></item> <item><p>If the verify callback fun returns <c>{valid, UserState}</c>, the verification process continues.</p></item> <item><p>If the verify callback fun always returns @@ -378,10 +436,12 @@ atom()}} | <taglist> <tag><c>unknown_ca</c></tag> - <item><p>No trusted CA was found in the trusted store. The trusted CA is - normally a so called ROOT CA, which is a self-signed certificate. Trust can - be claimed for an intermediate CA (trusted anchor does not have to be - self-signed according to X-509) by using option <c>partial_chain</c>.</p> + <item><p>No trusted CA was found in the trusted store. The + trusted CA is normally a so called ROOT CA, which is a + self-signed certificate. Trust can be claimed for an + intermediate CA (trusted anchor does not have to be + self-signed according to X-509) by using option + <c>partial_chain</c>.</p> </item> <tag><c>selfsigned_peer</c></tag> @@ -392,15 +452,17 @@ atom()}} | marker="public_key:public_key#pkix_path_validation-3">public_key:pkix_path_validation/3</seealso> </p></item> </taglist> - </item> - - <tag><c>{crl_check, boolean() | peer | best_effort }</c></tag> - <item> + </desc> + </datatype> + + <datatype> + <name name="crl_check"/> + <desc> <p>Perform CRL (Certificate Revocation List) verification <seealso marker="public_key:public_key#pkix_crls_validate-3"> - (public_key:pkix_crls_validate/3)</seealso> on all the certificates during the path validation - <seealso - marker="public_key:public_key#pkix_path_validation-3">(public_key:pkix_path_validation/3) + (public_key:pkix_crls_validate/3)</seealso> on all the + certificates during the path validation <seealso + marker="public_key:public_key#pkix_path_validation-3">(public_key:pkix_path_validation/3) </seealso> of the certificate chain. Defaults to <c>false</c>.</p> @@ -412,112 +474,111 @@ marker="public_key:public_key#pkix_path_validation-3">public_key:pkix_path_valid <item>if certificate revocation status can not be determined it will be accepted as valid.</item> </taglist> - + <p>The CA certificates specified for the connection will be used to construct the certificate chain validating the CRLs.</p> <p>The CRLs will be fetched from a local or external cache. See <seealso marker="ssl:ssl_crl_cache_api">ssl_crl_cache_api(3)</seealso>.</p> - </item> - - <tag><c>{crl_cache, {Module :: atom(), {DbHandle :: internal | term(), Args :: list()}}}</c></tag> - <item> - <p>Specify how to perform lookup and caching of certificate revocation lists. - <c>Module</c> defaults to <seealso marker="ssl:ssl_crl_cache">ssl_crl_cache</seealso> - with <c> DbHandle </c> being <c>internal</c> and an - empty argument list.</p> - - <p>There are two implementations available:</p> - - <taglist> - <tag><c>ssl_crl_cache</c></tag> - <item> - <p>This module maintains a cache of CRLs. CRLs can be - added to the cache using the function <seealso - marker="ssl:ssl_crl_cache#insert-1">ssl_crl_cache:insert/1</seealso>, - and optionally automatically fetched through HTTP if the - following argument is specified:</p> - - <taglist> - <tag><c>{http, timeout()}</c></tag> - <item><p> - Enables fetching of CRLs specified as http URIs in<seealso - marker="public_key:public_key_records">X509 certificate extensions</seealso>. - Requires the OTP inets application.</p> - </item> - </taglist> - </item> - - <tag><c>ssl_crl_hash_dir</c></tag> - <item> - <p>This module makes use of a directory where CRLs are - stored in files named by the hash of the issuer name.</p> - - <p>The file names consist of eight hexadecimal digits - followed by <c>.rN</c>, where <c>N</c> is an integer, - e.g. <c>1a2b3c4d.r0</c>. For the first version of the - CRL, <c>N</c> starts at zero, and for each new version, - <c>N</c> is incremented by one. The OpenSSL utility - <c>c_rehash</c> creates symlinks according to this - pattern.</p> - - <p>For a given hash value, this module finds all - consecutive <c>.r*</c> files starting from zero, and those - files taken together make up the revocation list. CRL - files whose <c>nextUpdate</c> fields are in the past, or - that are issued by a different CA that happens to have the - same name hash, are excluded.</p> - - <p>The following argument is required:</p> - - <taglist> - <tag><c>{dir, string()}</c></tag> - <item><p>Specifies the directory in which the CRLs can be found.</p></item> - </taglist> - - </item> - - <tag><c>max_handshake_size</c></tag> - <item> - <p>Integer (24 bits unsigned). Used to limit the size of - valid TLS handshake packets to avoid DoS attacks. - Defaults to 256*1024.</p> - </item> - - </taglist> - - </item> + </desc> + </datatype> - <tag><c>{partial_chain, fun(Chain::[DerCert]) -> {trusted_ca, DerCert} | - unknown_ca }</c></tag> - <item><p>Claim an intermediate CA in the chain as trusted. TLS then - performs <seealso - marker="public_key:public_key#pkix_path_validation-3">public_key:pkix_path_validation/3</seealso> - with the selected CA as trusted anchor and the rest of the chain.</p></item> + <datatype> + <name name="crl_cache_opts"/> + <desc> + <p>Specify how to perform lookup and caching of certificate revocation lists. + <c>Module</c> defaults to <seealso marker="ssl:ssl_crl_cache">ssl_crl_cache</seealso> + with <c> DbHandle </c> being <c>internal</c> and an + empty argument list.</p> + + <p>There are two implementations available:</p> + + <taglist> + <tag><c>ssl_crl_cache</c></tag> + <item> + <p>This module maintains a cache of CRLs. CRLs can be + added to the cache using the function <seealso + marker="ssl:ssl_crl_cache#insert-1">ssl_crl_cache:insert/1</seealso>, + and optionally automatically fetched through HTTP if the + following argument is specified:</p> + + <taglist> + <tag><c>{http, timeout()}</c></tag> + <item><p> + Enables fetching of CRLs specified as http URIs in<seealso + marker="public_key:public_key_records">X509 certificate extensions</seealso>. + Requires the OTP inets application.</p> + </item> + </taglist> + </item> + + <tag><c>ssl_crl_hash_dir</c></tag> + <item> + <p>This module makes use of a directory where CRLs are + stored in files named by the hash of the issuer name.</p> + + <p>The file names consist of eight hexadecimal digits + followed by <c>.rN</c>, where <c>N</c> is an integer, + e.g. <c>1a2b3c4d.r0</c>. For the first version of the + CRL, <c>N</c> starts at zero, and for each new version, + <c>N</c> is incremented by one. The OpenSSL utility + <c>c_rehash</c> creates symlinks according to this + pattern.</p> + + <p>For a given hash value, this module finds all + consecutive <c>.r*</c> files starting from zero, and those + files taken together make up the revocation list. CRL + files whose <c>nextUpdate</c> fields are in the past, or + that are issued by a different CA that happens to have the + same name hash, are excluded.</p> + + <p>The following argument is required:</p> + + <taglist> + <tag><c>{dir, string()}</c></tag> + <item><p>Specifies the directory in which the CRLs can be found.</p></item> + </taglist> + </item> + </taglist> + </desc> + </datatype> + + <datatype> + <name name="root_fun"/> + <desc> + <code> +fun(Chain::[public_key:der_encoded()]) -> + {trusted_ca, DerCert::public_key:der_encoded()} | unknown_ca} + </code> + <p>Claim an intermediate CA in the chain as trusted. TLS then + performs <seealso + marker="public_key:public_key#pkix_path_validation-3">public_key:pkix_path_validation/3</seealso> + with the selected CA as trusted anchor and the rest of the chain.</p> + </desc> + </datatype> - <tag><c>{versions, [protocol_version()]}</c></tag> - <item><p>TLS protocol versions supported by started clients and servers. + <datatype> + <name name="protocol_versions"/> + <desc><p>TLS protocol versions supported by started clients and servers. This option overrides the application environment option <c>protocol_version</c> and <c>dtls_protocol_version</c>. If the environment option is not set, it defaults to all versions, except SSL-3.0, supported by the SSL application. - See also <seealso marker="ssl:ssl_app">ssl(6).</seealso></p></item> + See also <seealso marker="ssl:ssl_app">ssl(6).</seealso></p> + </desc> + </datatype> - <tag><c>{hibernate_after, integer()|undefined}</c></tag> - <item><p>When an integer-value is specified, <c>TLS/DTLS-connection</c> - goes into hibernation after the specified number of milliseconds - of inactivity, thus reducing its memory footprint. When - <c>undefined</c> is specified (this is the default), the process - never goes into hibernation.</p></item> - <tag><c>{user_lookup_fun, {Lookupfun :: fun(), UserState :: term()}}</c></tag> - <item><p>The lookup fun is to defined as follows:</p> + <datatype> + <name name="custom_user_lookup"/> + <desc><p>The lookup fun is to defined as follows:</p> <code> fun(psk, PSKIdentity ::string(), UserState :: term()) -> {ok, SharedSecret :: binary()} | error; fun(srp, Username :: string(), UserState :: term()) -> - {ok, {SRPParams :: srp_param_type(), Salt :: binary(), DerivedKey :: binary()}} | error. + {ok, {SRPParams :: srp_param_type(), Salt :: binary(), + DerivedKey :: binary()}} | error. </code> <p>For Pre-Shared Key (PSK) cipher suites, the lookup fun is @@ -533,20 +594,54 @@ fun(srp, Username :: string(), UserState :: term()) -> <url href="http://tools.ietf.org/html/rfc5054#section-2.4"> RFC 5054</url>: <c>crypto:sha([Salt, crypto:sha([Username, <<$:>>, Password])])</c> </p> - </item> + </desc> + </datatype> - <tag><c>{padding_check, boolean()}</c></tag> - <item><p>Affects TLS-1.0 connections only. + <datatype> + <name name="session_id"/> + <desc> + <p>Identifies a TLS session.</p> + </desc> + </datatype> + + <datatype> + <name name="log_alert"/> + <desc><p>If set to <c>false</c>, error reports are not displayed.</p> + </desc> + </datatype> + + <datatype> + <name name="hibernate_after"/> + <desc><p>When an integer-value is specified, <c>TLS/DTLS-connection</c> + goes into hibernation after the specified number of milliseconds + of inactivity, thus reducing its memory footprint. When + <c>undefined</c> is specified (this is the default), the process + never goes into hibernation.</p> + </desc> + </datatype> + + <datatype> + <name name="handshake_size"/> + <desc> + <p>Integer (24 bits unsigned). Used to limit the size of + valid TLS handshake packets to avoid DoS attacks. + Defaults to 256*1024.</p> + </desc> + </datatype> + + <datatype> + <name name="padding_check"/> + <desc><p>Affects TLS-1.0 connections only. If set to <c>false</c>, it disables the block cipher padding check to be able to interoperate with legacy software.</p> <warning><p>Using <c>{padding_check, boolean()}</c> makes TLS vulnerable to the Poodle attack.</p></warning> - </item> - - + </desc> + </datatype> - <tag><c>{beast_mitigation, one_n_minus_one | zero_n | disabled}</c></tag> - <item><p>Affects SSL-3.0 and TLS-1.0 connections only. Used to change the BEAST + <datatype> + <name name="beast_mitigation"/> + <desc><p>Affects SSL-3.0 and TLS-1.0 connections only. Used to change the BEAST mitigation strategy to interoperate with legacy software. Defaults to <c>one_n_minus_one</c>.</p> @@ -556,127 +651,170 @@ fun(srp, Username :: string(), UserState :: term()) -> <p><c>disabled</c> - Disable BEAST mitigation.</p> - <warning><p>Using <c>{beast_mitigation, disabled}</c> makes SSL or TLS + <warning><p>Using <c>{beast_mitigation, disabled}</c> makes SSL-3.0 or TLS-1.0 vulnerable to the BEAST attack.</p></warning> - </item> - </taglist> - - </section> - - <section> - <title>TLS/DTLS OPTION DESCRIPTIONS - CLIENT SIDE</title> - - <p>The following options are client-specific or have a slightly different - meaning in the client than in the server:</p> - - <taglist> + </desc> + </datatype> + + <datatype> + <name name="ssl_imp"/> + <desc><p>Deprecated since OTP-17, has no affect.</p></desc> + </datatype> + + <datatype_title>TLS/DTLS OPTION DESCRIPTIONS - CLIENT</datatype_title> + + <datatype> + <name name="client_option"/> + </datatype> + + <datatype> + <name name="client_verify_type"/> + <desc><p>In mode <c>verify_none</c> the default behavior is to allow + all x509-path validation errors. See also option <seealso marker="#type-custom_verify">verify_fun</seealso>.</p> + </desc> + </datatype> - <tag><c>{verify, verify_type()}</c></tag> - <item><p>In mode <c>verify_none</c> the default behavior is to allow - all x509-path validation errors. See also option <c>verify_fun</c>.</p> - </item> + <datatype> + <name name="client_reuse_session"/> + <desc> + <p>Reuses a specific session earlier saved with the option + <c>{reuse_sessions, save} since OTP-21.3 </c> + </p> + </desc> + </datatype> - <tag><c>{reuse_sessions, boolean()}</c></tag> - <item><p>Specifies if the client is to try to reuse sessions - when possible.</p></item> + <datatype> + <name name="client_reuse_sessions"/> + <desc> + <p>When <c>save</c> is specified a new connection will be negotiated + and saved for later reuse. The session ID can be fetched with + <seealso marker="#connection_information-2">connection_information/2</seealso> + and used with the client option <seealso marker="#type-client_reuse_session">reuse_session</seealso> + The boolean value true specifies that if possible, automatized session reuse will + be performed. If a new session is created, and is unique in regard + to previous stored sessions, it will be saved for possible later reuse. Since OTP-21.3</p> + </desc> + </datatype> - <tag><c>{cacerts, [public_key:der_encoded()]}</c></tag> - <item><p>The DER-encoded trusted certificates. If this option - is supplied it overrides option <c>cacertfile</c>.</p></item> - - <tag><c>{cacertfile, path()}</c></tag> - <item><p>Path to a file containing PEM-encoded CA certificates. The CA + <datatype> + <name name="client_cacerts"/> + <desc> + <p>The DER-encoded trusted certificates. If this option + is supplied it overrides option <c>cacertfile</c>.</p> + </desc> + </datatype> + + <datatype> + <name name="client_cafile"/> + <desc> + <p>Path to a file containing PEM-encoded CA certificates. The CA certificates are used during server authentication and when building the client certificate chain.</p> - </item> - - <tag><c>{alpn_advertised_protocols, [binary()]}</c></tag> - <item> - <p>The list of protocols supported by the client to be sent to the - server to be used for an Application-Layer Protocol Negotiation (ALPN). - If the server supports ALPN then it will choose a protocol from this - list; otherwise it will fail the connection with a "no_application_protocol" - alert. A server that does not support ALPN will ignore this value.</p> - - <p>The list of protocols must not contain an empty binary.</p> - - <p>The negotiated protocol can be retrieved using the <c>negotiated_protocol/1</c> function.</p> - </item> - - <tag><c>{client_preferred_next_protocols, {Precedence :: server | client, ClientPrefs :: [binary()]}}</c><br/> - <c>{client_preferred_next_protocols, {Precedence :: server | client, ClientPrefs :: [binary()], Default :: binary()}}</c></tag> - <item> - <p>Indicates that the client is to try to perform Next Protocol - Negotiation.</p> - - <p>If precedence is server, the negotiated protocol is the - first protocol to be shown on the server advertised list, which is - also on the client preference list.</p> - - <p>If precedence is client, the negotiated protocol is the - first protocol to be shown on the client preference list, which is - also on the server advertised list.</p> - - <p>If the client does not support any of the server advertised - protocols or the server does not advertise any protocols, the - client falls back to the first protocol in its list or to the - default protocol (if a default is supplied). If the - server does not support Next Protocol Negotiation, the - connection terminates if no default protocol is supplied.</p> - </item> - - <tag><c>{psk_identity, string()}</c></tag> - <item><p>Specifies the identity the client presents to the server. - The matching secret is found by calling <c>user_lookup_fun</c>.</p> - </item> - - <tag><c>{srp_identity, {Username :: string(), Password :: string()} - </c></tag> - <item><p>Specifies the username and password to use to authenticate - to the server.</p></item> - - <tag><c>{server_name_indication, HostName :: hostname()}</c></tag> - <item><p>Specify the hostname to be used in TLS Server Name Indication extension. - If not specified it will default to the <c>Host</c> argument of <seealso marker="#connect-3">connect/[3,4]</seealso> - unless it is of type inet:ipaddress().</p> - <p> - The <c>HostName</c> will also be used in the hostname verification of the peer certificate using - <seealso marker="public_key:public_key#pkix_verify_hostname-2">public_key:pkix_verify_hostname/2</seealso>. - </p> - </item> - <tag><c>{server_name_indication, disable}</c></tag> - <item> - <p> Prevents the Server Name Indication extension from being sent and - disables the hostname verification check - <seealso marker="public_key:public_key#pkix_verify_hostname-2">public_key:pkix_verify_hostname/2</seealso> </p> - </item> - - <tag><c>{customize_hostname_check, Options::list()}</c></tag> - <item> - <p> Customizes the hostname verification of the peer certificate, as different protocols that use + </desc> + </datatype> + + <datatype> + <name name="client_alpn"/> + <desc> + <p>The list of protocols supported by the client to be sent to the + server to be used for an Application-Layer Protocol Negotiation (ALPN). + If the server supports ALPN then it will choose a protocol from this + list; otherwise it will fail the connection with a "no_application_protocol" + alert. A server that does not support ALPN will ignore this value.</p> + + <p>The list of protocols must not contain an empty binary.</p> + + <p>The negotiated protocol can be retrieved using the <c>negotiated_protocol/1</c> function.</p> + </desc> + </datatype> + + <datatype> + <name name="client_preferred_next_protocols"/> + <desc> + <p>Indicates that the client is to try to perform Next Protocol + Negotiation.</p> + + <p>If precedence is server, the negotiated protocol is the + first protocol to be shown on the server advertised list, which is + also on the client preference list.</p> + + <p>If precedence is client, the negotiated protocol is the + first protocol to be shown on the client preference list, which is + also on the server advertised list.</p> + + <p>If the client does not support any of the server advertised + protocols or the server does not advertise any protocols, the + client falls back to the first protocol in its list or to the + default protocol (if a default is supplied). If the + server does not support Next Protocol Negotiation, the + connection terminates if no default protocol is supplied.</p> + </desc> + </datatype> + + <datatype> + <name name="client_psk_identity"/> + <desc> + <p>Specifies the identity the client presents to the server. + The matching secret is found by calling <c>user_lookup_fun</c></p> + </desc> + </datatype> + + <datatype> + <name name="client_srp_identity"/> + <desc> + <p>Specifies the username and password to use to authenticate + to the server.</p> + </desc> + </datatype> + + <datatype> + <name name="sni"/> + <desc> + <p>Specify the hostname to be used in TLS Server Name Indication extension. + If not specified it will default to the <c>Host</c> argument of <seealso marker="#connect-3">connect/[3,4]</seealso> + unless it is of type inet:ipaddress().</p> + <p> + The <c>HostName</c> will also be used in the hostname verification of the peer certificate using + <seealso marker="public_key:public_key#pkix_verify_hostname-2">public_key:pkix_verify_hostname/2</seealso>. + </p> + <p> The special value <c>disable</c> prevents the Server Name Indication extension from being sent and + disables the hostname verification check + <seealso marker="public_key:public_key#pkix_verify_hostname-2">public_key:pkix_verify_hostname/2</seealso> </p> + </desc> + </datatype> + + <datatype> + <name name="customize_hostname_check"/> + <desc> + <p> Customizes the hostname verification of the peer certificate, as different protocols that use TLS such as HTTP or LDAP may want to do it differently, for possible options see <seealso marker="public_key:public_key#pkix_verify_hostname-3">public_key:pkix_verify_hostname/3</seealso> </p> - </item> - - <tag><c>{fallback, boolean()}</c></tag> - <item> - <p> Send special cipher suite TLS_FALLBACK_SCSV to avoid undesired TLS version downgrade. - Defaults to false</p> - <warning><p>Note this option is not needed in normal TLS usage and should not be used - to implement new clients. But legacy clients that retries connections in the following manner</p> - - <p><c> ssl:connect(Host, Port, [...{versions, ['tlsv2', 'tlsv1.1', 'tlsv1', 'sslv3']}])</c></p> - <p><c> ssl:connect(Host, Port, [...{versions, [tlsv1.1', 'tlsv1', 'sslv3']}, {fallback, true}])</c></p> - <p><c> ssl:connect(Host, Port, [...{versions, ['tlsv1', 'sslv3']}, {fallback, true}]) </c></p> - <p><c> ssl:connect(Host, Port, [...{versions, ['sslv3']}, {fallback, true}]) </c></p> - - <p>may use it to avoid undesired TLS version downgrade. Note that TLS_FALLBACK_SCSV must also - be supported by the server for the prevention to work. - </p></warning> - </item> - <tag><marker id="client_signature_algs"/><c>{signature_algs, [{hash(), ecdsa | rsa | dsa}]}</c></tag> - <item> - <p>In addition to the algorithms negotiated by the cipher + </desc> + </datatype> + + <datatype> + <name name="fallback"/> + <desc> + <p> Send special cipher suite TLS_FALLBACK_SCSV to avoid undesired TLS version downgrade. + Defaults to false</p> + <warning><p>Note this option is not needed in normal TLS usage and should not be used + to implement new clients. But legacy clients that retries connections in the following manner</p> + + <p><c> ssl:connect(Host, Port, [...{versions, ['tlsv2', 'tlsv1.1', 'tlsv1', 'sslv3']}])</c></p> + <p><c> ssl:connect(Host, Port, [...{versions, [tlsv1.1', 'tlsv1', 'sslv3']}, {fallback, true}])</c></p> + <p><c> ssl:connect(Host, Port, [...{versions, ['tlsv1', 'sslv3']}, {fallback, true}]) </c></p> + <p><c> ssl:connect(Host, Port, [...{versions, ['sslv3']}, {fallback, true}]) </c></p> + + <p>may use it to avoid undesired TLS version downgrade. Note that TLS_FALLBACK_SCSV must also + be supported by the server for the prevention to work. + </p></warning> + </desc> + </datatype> + + <datatype> + <name name="client_signature_algs"/> + <desc> + <p>In addition to the algorithms negotiated by the cipher suite used for key exchange, payload encryption, message authentication and pseudo random calculation, the TLS signature algorithm extension <url @@ -707,177 +845,230 @@ fun(srp, Username :: string(), UserState :: term()) -> Selected signature algorithm can restrict which hash functions that may be selected. Default support for {md5, rsa} removed in ssl-8.0 </p> - </item> - </taglist> - </section> - - <section> - <title>TLS/DTLS OPTION DESCRIPTIONS - SERVER SIDE</title> + </desc> + </datatype> + - <p>The following options are server-specific or have a slightly different - meaning in the server than in the client:</p> + <datatype_title>TLS/DTLS OPTION DESCRIPTIONS - SERVER </datatype_title> - <taglist> - <tag><c>{cacerts, [public_key:der_encoded()]}</c></tag> - <item><p>The DER-encoded trusted certificates. If this option - is supplied it overrides option <c>cacertfile</c>.</p></item> + <datatype> + <name name="server_option"/> + </datatype> + + <datatype> + <name name="server_cacerts"/> + <desc><p>The DER-encoded trusted certificates. If this option + is supplied it overrides option <c>cacertfile</c>.</p> + </desc> + </datatype> - <tag><c>{cacertfile, path()}</c></tag> - <item><p>Path to a file containing PEM-encoded CA - certificates. The CA certificates are used to build the server - certificate chain and for client authentication. The CAs are - also used in the list of acceptable client CAs passed to the - client when a certificate is requested. Can be omitted if there - is no need to verify the client and if there are no - intermediate CAs for the server certificate.</p></item> - - <tag><c>{dh, public_key:der_encoded()}</c></tag> - <item><p>The DER-encoded Diffie-Hellman parameters. If specified, - it overrides option <c>dhfile</c>.</p></item> - - <tag><c>{dhfile, path()}</c></tag> - <item><p>Path to a file containing PEM-encoded Diffie Hellman parameters - to be used by the server if a cipher suite using Diffie Hellman key - exchange is negotiated. If not specified, default parameters are used. - </p></item> - - <tag><c>{verify, verify_type()}</c></tag> - <item><p>A server only does x509-path validation in mode <c>verify_peer</c>, - as it then sends a certificate request to the client - (this message is not sent if the verify option is <c>verify_none</c>). - You can then also want to specify option <c>fail_if_no_peer_cert</c>. - </p></item> - - <tag><c>{fail_if_no_peer_cert, boolean()}</c></tag> - <item><p>Used together with <c>{verify, verify_peer}</c> by an TLS/DTLS server. - If set to <c>true</c>, the server fails if the client does not have - a certificate to send, that is, sends an empty certificate. If set to - <c>false</c>, it fails only if the client sends an invalid - certificate (an empty certificate is considered valid). Defaults to false.</p> - </item> - - <tag><c>{reuse_sessions, boolean()}</c></tag> - <item><p>Specifies if the server is to agree to reuse sessions - when requested by the clients. See also option <c>reuse_session</c>. - </p></item> - - <tag><c>{reuse_session, fun(SuggestedSessionId, - PeerCert, Compression, CipherSuite) -> boolean()}</c></tag> - <item><p>Enables the TLS/DTLS server to have a local policy - for deciding if a session is to be reused or not. - Meaningful only if <c>reuse_sessions</c> is set to <c>true</c>. - <c>SuggestedSessionId</c> is a <c>binary()</c>, <c>PeerCert</c> is - a DER-encoded certificate, <c>Compression</c> is an enumeration integer, - and <c>CipherSuite</c> is of type <c>ciphersuite()</c>.</p></item> - - <tag><c>{alpn_preferred_protocols, [binary()]}</c></tag> - <item> - <p>Indicates the server will try to perform Application-Layer - Protocol Negotiation (ALPN).</p> - - <p>The list of protocols is in order of preference. The protocol - negotiated will be the first in the list that matches one of the - protocols advertised by the client. If no protocol matches, the - server will fail the connection with a "no_application_protocol" alert.</p> - - <p>The negotiated protocol can be retrieved using the <c>negotiated_protocol/1</c> function.</p> - </item> - - <tag><c>{next_protocols_advertised, Protocols :: [binary()]}</c></tag> - <item><p>List of protocols to send to the client if the client indicates that - it supports the Next Protocol extension. The client can select a protocol - that is not on this list. The list of protocols must not contain an empty - binary. If the server negotiates a Next Protocol, it can be accessed - using the <c>negotiated_next_protocol/1</c> method.</p></item> - - <tag><c>{psk_identity, string()}</c></tag> - <item><p>Specifies the server identity hint, which the server presents to - the client.</p></item> - - <tag><c>{log_alert, boolean()}</c></tag> - <item><p>If set to <c>false</c>, error reports are not displayed.</p></item> - - <tag><c>{honor_cipher_order, boolean()}</c></tag> - <item><p>If set to <c>true</c>, use the server preference for cipher - selection. If set to <c>false</c> (the default), use the client - preference.</p></item> - - <tag><c>{sni_hosts, [{hostname(), [ssl_option()]}]}</c></tag> - <item><p>If the server receives a SNI (Server Name Indication) from the client - matching a host listed in the <c>sni_hosts</c> option, the specific options for - that host will override previously specified options. - - The option <c>sni_fun</c>, and <c>sni_hosts</c> are mutually exclusive.</p></item> - - <tag><c>{sni_fun, SNIfun::fun()}</c></tag> - <item><p>If the server receives a SNI (Server Name Indication) from the client, - the given function will be called to retrieve <c>[ssl_option()]</c> for the indicated server. - These options will be merged into predefined <c>[ssl_option()]</c>. - - The function should be defined as: - <c>fun(ServerName :: string()) -> [ssl_option()]</c> - and can be specified as a fun or as named <c>fun module:function/1</c> - - The option <c>sni_fun</c>, and <c>sni_hosts</c> are mutually exclusive.</p></item> - - <tag><c>{client_renegotiation, boolean()}</c></tag> - <item>In protocols that support client-initiated renegotiation, the cost - of resources of such an operation is higher for the server than the - client. This can act as a vector for denial of service attacks. The SSL - application already takes measures to counter-act such attempts, - but client-initiated renegotiation can be strictly disabled by setting - this option to <c>false</c>. The default value is <c>true</c>. - Note that disabling renegotiation can result in long-lived connections - becoming unusable due to limits on the number of messages the underlying - cipher suite can encipher. - </item> - - <tag><c>{honor_cipher_order, boolean()}</c></tag> - <item>If true, use the server's preference for cipher selection. If false - (the default), use the client's preference. - </item> - <tag><c>{honor_ecc_order, boolean()}</c></tag> - <item>If true, use the server's preference for ECC curve selection. If false - (the default), use the client's preference. - </item> - - <tag><c>{signature_algs, [{hash(), ecdsa | rsa | dsa}]}</c></tag> - <item><p> The algorithms specified by - this option will be the ones accepted by the server in a signature algorithm - negotiation, introduced in TLS-1.2. The algorithms will also be offered to the client if a - client certificate is requested. For more details see the <seealso marker="#client_signature_algs">corresponding client option</seealso>. - </p> </item> - - </taglist> - </section> - - <section> - <title>General</title> + <datatype> + <name name="server_cafile"/> + <desc><p>Path to a file containing PEM-encoded CA + certificates. The CA certificates are used to build the server + certificate chain and for client authentication. The CAs are + also used in the list of acceptable client CAs passed to the + client when a certificate is requested. Can be omitted if + there is no need to verify the client and if there are no + intermediate CAs for the server certificate.</p> + </desc> + </datatype> + + <datatype> + <name name="dh_der"/> + <desc><p>The DER-encoded Diffie-Hellman parameters. If + specified, it overrides option <c>dhfile</c>.</p> + </desc> + </datatype> + + <datatype> + <name name="dh_file"/> + <desc><p>Path to a file containing PEM-encoded Diffie Hellman + parameters to be used by the server if a cipher suite using + Diffie Hellman key exchange is negotiated. If not specified, + default parameters are used.</p> + </desc> + </datatype> - <p>When an TLS/DTLS socket is in active mode (the default), data from the - socket is delivered to the owner of the socket in the form of - messages:</p> - <list type="bulleted"> - <item><p><c>{ssl, Socket, Data}</c></p></item> - <item><p><c>{ssl_closed, Socket}</c></p></item> - <item><p><c>{ssl_error, Socket, Reason}</c></p></item> - </list> + <datatype> + <name name="server_verify_type"/> + <desc><p>A server only does x509-path validation in mode + <c>verify_peer</c>, as it then sends a certificate request to + the client (this message is not sent if the verify option is + <c>verify_none</c>). You can then also want to specify option + <c>fail_if_no_peer_cert</c>. </p> + </desc> + </datatype> + + <datatype> + <name name="fail_if_no_peer_cert"/> + <desc><p>Used together with <c>{verify, verify_peer}</c> by an + TLS/DTLS server. If set to <c>true</c>, the server fails if + the client does not have a certificate to send, that is, sends + an empty certificate. If set to <c>false</c>, it fails only if + the client sends an invalid certificate (an empty certificate + is considered valid). Defaults to false.</p> + </desc> + </datatype> + + <datatype> + <name name="server_reuse_sessions"/> + <desc><p>The boolean value true specifies that the server will + agree to reuse sessions. Setting it to false will result in an empty + session table, that is no sessions will be reused. + See also option <seealso marker="#type-server_reuse_session">reuse_session</seealso> + </p> + </desc> + </datatype> + + <datatype> + <name name="server_reuse_session"/> + <desc><p>Enables the TLS/DTLS server to have a local policy + for deciding if a session is to be reused or not. Meaningful + only if <c>reuse_sessions</c> is set to <c>true</c>. + <c>SuggestedSessionId</c> is a <c>binary()</c>, + <c>PeerCert</c> is a DER-encoded certificate, + <c>Compression</c> is an enumeration integer, and + <c>CipherSuite</c> is of type <c>ciphersuite()</c>.</p> + </desc> + </datatype> + + <datatype> + <name name="server_alpn"/> + <desc> + <p>Indicates the server will try to perform + Application-Layer Protocol Negotiation (ALPN).</p> + + <p>The list of protocols is in order of preference. The + protocol negotiated will be the first in the list that + matches one of the protocols advertised by the client. If no + protocol matches, the server will fail the connection with a + "no_application_protocol" alert.</p> + + <p>The negotiated protocol can be retrieved using the + <c>negotiated_protocol/1</c> function.</p> + </desc> + </datatype> + + <datatype> + <name name="server_next_protocol"/> + <desc><p>List of protocols to send to the client if the client + indicates that it supports the Next Protocol extension. The + client can select a protocol that is not on this list. The + list of protocols must not contain an empty binary. If the + server negotiates a Next Protocol, it can be accessed using + the <c>negotiated_next_protocol/1</c> method.</p> + </desc> + </datatype> + + <datatype> + <name name="server_psk_identity"/> + <desc> + <p>Specifies the server identity hint, which the server presents to + the client.</p> + </desc> + </datatype> + + <datatype> + <name name="honor_cipher_order"/> + <desc> + <p>If set to <c>true</c>, use the server preference for cipher + selection. If set to <c>false</c> (the default), use the client + preference.</p> + </desc> + </datatype> + + <datatype> + <name name="sni_hosts"/> + <desc><p>If the server receives a SNI (Server Name Indication) from the client + matching a host listed in the <c>sni_hosts</c> option, the specific options for + that host will override previously specified options. + + The option <c>sni_fun</c>, and <c>sni_hosts</c> are mutually exclusive.</p> + </desc> + </datatype> + + <datatype> + <name name="sni_fun"/> + <desc> + <p>If the server receives a SNI (Server Name Indication) + from the client, the given function will be called to + retrieve <seealso marker="#type-server_option">[server_option()] </seealso> for the indicated server. + These options will be merged into predefined + <seealso marker="#type-server_option">[server_option()] </seealso> list. + + The function should be defined as: + fun(ServerName :: string()) -> <seealso marker="#type-server_option">[server_option()] </seealso> + and can be specified as a fun or as named <c>fun module:function/1</c> + + The option <c>sni_fun</c>, and <c>sni_hosts</c> are mutually exclusive.</p> + </desc> + </datatype> + + <datatype> + <name name="client_renegotiation"/> + <desc><p>In protocols that support client-initiated + renegotiation, the cost of resources of such an operation is + higher for the server than the client. This can act as a + vector for denial of service attacks. The SSL application + already takes measures to counter-act such attempts, but + client-initiated renegotiation can be strictly disabled by + setting this option to <c>false</c>. The default value is + <c>true</c>. Note that disabling renegotiation can result in + long-lived connections becoming unusable due to limits on the + number of messages the underlying cipher suite can + encipher.</p> + </desc> + </datatype> + + <datatype> + <name name="honor_cipher_order"/> + <desc><p>If true, use the server's preference for cipher + selection. If false (the default), use the client's + preference.</p> + </desc> + </datatype> + + <datatype> + <name name="honor_ecc_order"/> + <desc><p>If true, use the server's preference for ECC curve + selection. If false (the default), use the client's + preference.</p> + </desc> + </datatype> + + <datatype> + <name name="server_signature_algs"/> + <desc><p> The algorithms specified by this option will be the + ones accepted by the server in a signature algorithm + negotiation, introduced in TLS-1.2. The algorithms will also + be offered to the client if a client certificate is + requested. For more details see the <seealso + marker="#type-client_signature_algs">corresponding client + option</seealso>. + </p> + </desc> + </datatype> + </datatypes> + +<!-- + ================================================================ + = Function definitions = + ================================================================ +--> - <p>A <c>Timeout</c> argument specifies a time-out in milliseconds. The - default value for argument <c>Timeout</c> is <c>infinity</c>.</p> - </section> - <funcs> <func> <name since="OTP 20.3">append_cipher_suites(Deferred, Suites) -> ciphers() </name> <fsummary></fsummary> <type> - <v>Deferred = ciphers() | cipher_filters() </v> - <v>Suites = ciphers() </v> + <v>Deferred = <seealso marker="#type-ciphers">ciphers()</seealso> | + <seealso marker="#type-cipher_filters">cipher_filters()</seealso></v> + <v>Suites = <seealso marker="#type-ciphers">ciphers()</seealso></v> </type> <desc><p>Make <c>Deferred</c> suites become the least preferred suites, that is put them at the end of the cipher suite list @@ -890,7 +1081,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <func> <name since="OTP R14B">cipher_suites() -></name> - <name since="OTP R14B">cipher_suites(Type) -> old_ciphers()</name> + <name since="OTP R14B">cipher_suites(Type) -> [old_cipher_suite()]</name> <fsummary>Returns a list of supported cipher suites.</fsummary> <type> <v>Type = erlang | openssl | all</v> @@ -906,7 +1097,7 @@ fun(srp, Username :: string(), UserState :: term()) -> all supported cipher suites.</fsummary> <type> <v> Supported = default | all | anonymous </v> - <v> Version = protocol_version() </v> + <v> Version = <seealso marker="#type-protocol_version">protocol_version() </seealso></v> </type> <desc><p>Returns all default or all supported (except anonymous), or all anonymous cipher suites for a @@ -916,9 +1107,15 @@ fun(srp, Username :: string(), UserState :: term()) -> <func> <name since="OTP 19.2">eccs() -></name> - <name since="OTP 19.2">eccs(protocol_version()) -> [named_curve()]</name> + <name since="OTP 19.2">eccs(Version) -> NamedCurves</name> <fsummary>Returns a list of supported ECCs.</fsummary> + <type> + <v> Version = <seealso marker="#type-protocol_version">protocol_version() </seealso></v> + <v> NamedCurves = <seealso marker="#type-named_curve">[named_curve()] </seealso></v> + + </type> + <desc><p>Returns a list of supported ECCs. <c>eccs()</c> is equivalent to calling <c>eccs(Protocol)</c> with all supported protocols and then deduplicating the output.</p> @@ -938,39 +1135,46 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name since="OTP R14B">connect(Socket, SslOptions) -> </name> - <name since="">connect(Socket, SslOptions, Timeout) -> {ok, SslSocket} | {ok, SslSocket, Ext} + <name since="OTP R14B">connect(Socket, Options) -> </name> + <name since="">connect(Socket, Options, Timeout) -> {ok, SslSocket} | {ok, SslSocket, Ext} | {error, Reason}</name> <fsummary>Upgrades a <c>gen_tcp</c>, or equivalent, connected socket to an TLS socket.</fsummary> <type> - <v>Socket = socket()</v> - <v>SslOptions = [{handshake, hello| full} | ssl_option()]</v> - <v>Timeout = integer() | infinity</v> - <v>SslSocket = sslsocket()</v> - <v>Ext = hello_extensions()</v> - <v>Reason = term()</v> + <v>Socket = <seealso marker="#type-socket"> socket() </seealso></v> + <v>Options = <seealso marker="#type-tls_client_option"> [tls_client_option()] </seealso></v> + <v>Timeout = timeout()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> + <v>Ext = <seealso marker="#type-protocol_extensions">protocol_extensions()</seealso></v> + <v>Reason = closed | timeout | <seealso marker="#type-error_alert"> error_alert() </seealso></v> </type> <desc><p>Upgrades a <c>gen_tcp</c>, or equivalent, connected socket to an TLS socket, that is, performs the client-side TLS handshake.</p> - <note><p>If the option <c>verify</c> is set to <c>verify_peer</c> - the option <c>server_name_indication</c> shall also be specified, - if it is not no Server Name Indication extension will be sent, - and <seealso marker="public_key:public_key#pkix_verify_hostname-2">public_key:pkix_verify_hostname/2</seealso> - will be called with the IP-address of the connection as <c>ReferenceID</c>, which is proably not what you want.</p> + <note><p>If the option <c>verify</c> is set to + <c>verify_peer</c> the option <c>server_name_indication</c> + shall also be specified, if it is not no Server Name + Indication extension will be sent, and <seealso + marker="public_key:public_key#pkix_verify_hostname-2">public_key:pkix_verify_hostname/2</seealso> + will be called with the IP-address of the connection as + <c>ReferenceID</c>, which is proably not what you want.</p> </note> <p> If the option <c>{handshake, hello}</c> is used the handshake is paused after receiving the server hello message and the success response is <c>{ok, SslSocket, Ext}</c> - instead of <c>{ok, SslSocket}</c>. Thereafter the handshake is continued or - canceled by calling <seealso marker="#handshake_continue-3"> + instead of <c>{ok, SslSocket}</c>. Thereafter the handshake + is continued or canceled by calling <seealso + marker="#handshake_continue-3"> <c>handshake_continue/3</c></seealso> or <seealso - marker="#handshake_cancel-1"><c>handshake_cancel/1</c></seealso>. + marker="#handshake_cancel-1"><c>handshake_cancel/1</c></seealso>. </p> + <p> If the option <c>active</c> is set to <c>once</c> or <c>true</c> the + process owning the sslsocket will receive messages of type + <seealso marker="#type-active_msgs"> active_msgs() </seealso> + </p> </desc> </func> @@ -980,19 +1184,19 @@ fun(srp, Username :: string(), UserState :: term()) -> {ok, SslSocket}| {ok, SslSocket, Ext} | {error, Reason}</name> <fsummary>Opens an TLS/DTLS connection to <c>Host</c>, <c>Port</c>.</fsummary> <type> - <v>Host = host()</v> - <v>Port = integer()</v> - <v>Options = [option()]</v> - <v>Timeout = integer() | infinity</v> - <v>SslSocket = sslsocket()</v> - <v>Reason = term()</v> + <v>Host =<seealso marker="#type-host"> host() </seealso> </v> + <v>Port = <seealso marker="kernel:inet#type-port_number">inet:port_number()</seealso></v> + <v>Options = <seealso marker="#type-tls_client_option"> [tls_client_option()]</seealso></v> + <v>Timeout = timeout()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> + <v>Reason = closed | timeout | <seealso marker="#type-error_alert"> error_alert() </seealso></v> </type> <desc><p>Opens an TLS/DTLS connection to <c>Host</c>, <c>Port</c>.</p> <p> When the option <c>verify</c> is set to <c>verify_peer</c> the check <seealso marker="public_key:public_key#pkix_verify_hostname-2">public_key:pkix_verify_hostname/2</seealso> will be performed in addition to the usual x509-path validation checks. If the check fails the error {bad_cert, hostname_check_failed} will - be propagated to the path validation fun <seealso marker="#verify_fun">verify_fun</seealso>, where it is possible to do customized + be propagated to the path validation fun <seealso marker="#type-custom_verify">verify_fun</seealso>, where it is possible to do customized checks by using the full possibilities of the <seealso marker="public_key:public_key#pkix_verify_hostname-3">public_key:pkix_verify_hostname/3</seealso> API. When the option <c>server_name_indication</c> is provided, its value (the DNS name) will be used as <c>ReferenceID</c> @@ -1014,6 +1218,11 @@ fun(srp, Username :: string(), UserState :: term()) -> <c>handshake_continue/3</c></seealso> or <seealso marker="#handshake_cancel-1"><c>handshake_cancel/1</c></seealso>. </p> + + <p> If the option <c>active</c> is set to <c>once</c> or <c>true</c> the + process owning the sslsocket will receive messages of type + <seealso marker="#type-active_msgs"> active_msgs() </seealso> + </p> </desc> </func> @@ -1021,7 +1230,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="">close(SslSocket) -> ok | {error, Reason}</name> <fsummary>Closes an TLS/DTLS connection.</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>Reason = term()</v> </type> <desc><p>Closes an TLS/DTLS connection.</p> @@ -1032,7 +1241,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="OTP 18.1">close(SslSocket, How) -> ok | {ok, port()} | {error, Reason}</name> <fsummary>Closes an TLS connection.</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>How = timeout() | {NewController::pid(), timeout()} </v> <v>Reason = term()</v> </type> @@ -1049,7 +1258,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <fsummary>Assigns a new controlling process to the TLS/DTLS socket.</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>NewOwner = pid()</v> <v>Reason = term()</v> </type> @@ -1065,7 +1274,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <fsummary>Returns all the connection information. </fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>Item = protocol | selected_cipher_suite | sni_hostname | ecc | session_id | atom()</v> <d>Meaningful atoms, not specified above, are the ssl option names.</d> <v>Result = [{Item::atom(), Value::term()}]</v> @@ -1086,7 +1295,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <fsummary>Returns the requested connection information. </fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>Items = [Item]</v> <v>Item = protocol | cipher_suite | sni_hostname | ecc | session_id | client_random | server_random | master_secret | atom()</v> @@ -1106,8 +1315,8 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="OTP 20.3">filter_cipher_suites(Suites, Filters) -> ciphers()</name> <fsummary></fsummary> <type> - <v> Suites = ciphers()</v> - <v> Filters = cipher_filters()</v> + <v> Suites = <seealso marker="#type-ciphers"> ciphers() </seealso></v> + <v> Filters = <seealso marker="#type-cipher_filters"> cipher_filters() </seealso></v> </type> <desc><p>Removes cipher suites if any of the filter functions returns false for any part of the cipher suite. This function @@ -1133,7 +1342,7 @@ fun(srp, Username :: string(), UserState :: term()) -> {ok, [socketoption()]} | {error, Reason}</name> <fsummary>Gets the values of the specified options.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>Socket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>OptionNames = [atom()]</v> </type> <desc> @@ -1149,7 +1358,7 @@ fun(srp, Username :: string(), UserState :: term()) -> {ok, OptionValues} | {error, inet:posix()}</name> <fsummary>Get one or more statistic options for a socket</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>OptionNames = [atom()]</v> <v>OptionValues = [{inet:stat_option(), integer()}]</v> </type> @@ -1164,27 +1373,32 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="OTP 21.0">handshake(HsSocket, Timeout) -> {ok, SslSocket} | {error, Reason}</name> <fsummary>Performs server-side SSL/TLS handshake.</fsummary> <type> - <v>HsSocket = SslSocket = sslsocket()</v> - <v>Timeout = integer()</v> - <v>Reason = term()</v> + <v>HsSocket = SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> + <v>Timeout = timeout()</v> + <v>Reason = closed | timeout | <seealso marker="#type-error_alert"> error_alert() </seealso></v> </type> <desc> <p>Performs the SSL/TLS/DTLS server-side handshake.</p> <p>Returns a new TLS/DTLS socket if the handshake is successful.</p> + + <p> If the option <c>active</c> is set to <c>once</c> or <c>true</c> the + process owning the sslsocket will receive messages of type + <seealso marker="#type-active_msgs"> active_msgs() </seealso> + </p> </desc> </func> <func> - <name since="OTP 21.0">handshake(Socket, SslOptions) -> </name> - <name since="OTP 21.0">handshake(Socket, SslOptions, Timeout) -> {ok, SslSocket} | {ok, SslSocket, Ext} | {error, Reason}</name> + <name since="OTP 21.0">handshake(Socket, Options) -> </name> + <name since="OTP 21.0">handshake(Socket, Options, Timeout) -> {ok, SslSocket} | {ok, SslSocket, Ext} | {error, Reason}</name> <fsummary>Performs server-side SSL/TLS/DTLS handshake.</fsummary> <type> - <v>Socket = socket() | sslsocket() </v> - <v>SslSocket = sslsocket() </v> - <v>Ext = hello_extensions()</v> - <v>SslOptions = [{handshake, hello| full} | ssl_option()]</v> - <v>Timeout = integer()</v> - <v>Reason = term()</v> + <v>Socket = socket() | <seealso marker="#type-sslsocket"> socket() </seealso> </v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso> </v> + <v>Ext = <seealso marker="#type-protocol_extensions">protocol_extensions()</seealso></v> + <v>Options = <seealso marker="#type-tls_server_option"> [server_option()] </seealso> </v> + <v>Timeout = timeout()</v> + <v>Reason = closed | timeout | <seealso marker="#type-error_alert"> error_alert() </seealso></v> </type> <desc> <p>If <c>Socket</c> is a ordinary <c>socket()</c>: upgrades a <c>gen_tcp</c>, @@ -1196,7 +1410,8 @@ fun(srp, Username :: string(), UserState :: term()) -> is undefined. </p></warning> - <p>If <c>Socket</c> is an <c>sslsocket()</c>: provides extra SSL/TLS/DTLS + <p>If <c>Socket</c> is an + <seealso marker="#type-sslsocket"> sslsocket() </seealso>: provides extra SSL/TLS/DTLS options to those specified in <seealso marker="#listen-2">listen/2 </seealso> and then performs the SSL/TLS/DTLS handshake. Returns a new TLS/DTLS socket if the handshake is successful.</p> @@ -1210,6 +1425,12 @@ fun(srp, Username :: string(), UserState :: term()) -> <c>handshake_continue/3</c></seealso> or <seealso marker="#handshake_cancel-1"><c>handshake_cancel/1</c></seealso>. </p> + + <p> If the option <c>active</c> is set to <c>once</c> or <c>true</c> the + process owning the sslsocket will receive messages of type + <seealso marker="#type-active_msgs"> active_msgs() </seealso> + </p> + </desc> </func> @@ -1217,7 +1438,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="OTP 21.0">handshake_cancel(SslSocket) -> ok </name> <fsummary>Cancel handshake with a fatal alert</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> </type> <desc> <p>Cancel the handshake with a fatal <c>USER_CANCELED</c> alert.</p> @@ -1225,14 +1446,14 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name since="OTP 21.0">handshake_continue(HsSocket, SSLOptions) -> {ok, SslSocket} | {error, Reason}</name> - <name since="OTP 21.0">handshake_continue(HsSocket, SSLOptions, Timeout) -> {ok, SslSocket} | {error, Reason}</name> + <name since="OTP 21.0">handshake_continue(HsSocket, Options) -> {ok, SslSocket} | {error, Reason}</name> + <name since="OTP 21.0">handshake_continue(HsSocket, Options, Timeout) -> {ok, SslSocket} | {error, Reason}</name> <fsummary>Continue the SSL/TLS handshake.</fsummary> <type> - <v>HsSocket = SslSocket = sslsocket()</v> - <v>SslOptions = [ssl_option()]</v> - <v>Timeout = integer()</v> - <v>Reason = term()</v> + <v>HsSocket = SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> + <v>Options = <seealso marker="#type-tls_option"> tls_option() </seealso> </v> + <v>Timeout = timeout()</v> + <v>Reason = closed | timeout | <seealso marker="#type-error_alert"> error_alert() </seealso></v> </type> <desc> <p>Continue the SSL/TLS handshake possiby with new, additional or changed options.</p> @@ -1244,9 +1465,9 @@ fun(srp, Username :: string(), UserState :: term()) -> {ok, ListenSocket} | {error, Reason}</name> <fsummary>Creates an SSL listen socket.</fsummary> <type> - <v>Port = integer()</v> - <v>Options = options()</v> - <v>ListenSocket = sslsocket()</v> + <v>Port = <seealso marker="kernel:inet#type-port_number">inet:port_number()</seealso></v> + <v>Options = <seealso marker="#type-tls_server_option"> [server_option()] </seealso></v> + <v>ListenSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> </type> <desc> <p>Creates an SSL listen socket.</p> @@ -1257,7 +1478,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="OTP 18.0">negotiated_protocol(SslSocket) -> {ok, Protocol} | {error, protocol_not_negotiated}</name> <fsummary>Returns the protocol negotiated through ALPN or NPN extensions.</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>Protocol = binary()</v> </type> <desc> @@ -1271,7 +1492,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="">peercert(SslSocket) -> {ok, Cert} | {error, Reason}</name> <fsummary>Returns the peer certificate.</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>Cert = binary()</v> </type> <desc> @@ -1287,9 +1508,9 @@ fun(srp, Username :: string(), UserState :: term()) -> {error, Reason}</name> <fsummary>Returns the peer address and port.</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>Address = ipaddress()</v> - <v>Port = integer()</v> + <v>Port = <seealso marker="kernel:inet#type-port_number">inet:port_number()</seealso></v> </type> <desc> <p>Returns the address and port number of the peer.</p> @@ -1300,8 +1521,9 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="OTP 20.3">prepend_cipher_suites(Preferred, Suites) -> ciphers()</name> <fsummary></fsummary> <type> - <v>Preferred = ciphers() | cipher_filters() </v> - <v>Suites = ciphers() </v> + <v>Preferred = <seealso marker="#type-ciphers">ciphers()</seealso> | + <seealso marker="#type-cipher_filters">cipher_filters()</seealso></v> + <v>Suites = <seealso marker="#type-ciphers">ciphers()</seealso></v> </type> <desc><p>Make <c>Preferred</c> suites become the most preferred suites that is put them at the head of the cipher suite list @@ -1316,10 +1538,10 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="OTP R15B01">prf(Socket, Secret, Label, Seed, WantedLength) -> {ok, binary()} | {error, reason()}</name> <fsummary>Uses a session Pseudo-Random Function to generate key material.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>Socket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>Secret = binary() | master_secret</v> <v>Label = binary()</v> - <v>Seed = [binary() | prf_random()]</v> + <v>Seed = [binary() | <seealso marker="#type-prf_random"> prf_random()</seealso>]</v> <v>WantedLength = non_neg_integer()</v> </type> <desc> @@ -1338,9 +1560,9 @@ fun(srp, Username :: string(), UserState :: term()) -> Reason}</name> <fsummary>Receives data on a socket.</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>Length = integer()</v> - <v>Timeout = integer()</v> + <v>Timeout = timeout()</v> <v>Data = [char()] | binary()</v> </type> <desc> @@ -1363,7 +1585,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="OTP R14B">renegotiate(SslSocket) -> ok | {error, Reason}</name> <fsummary>Initiates a new handshake.</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> </type> <desc><p>Initiates a new handshake. A notable return value is <c>{error, renegotiation_rejected}</c> indicating that the peer @@ -1376,7 +1598,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="">send(SslSocket, Data) -> ok | {error, Reason}</name> <fsummary>Writes data to a socket.</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>Data = iodata()</v> </type> <desc> @@ -1390,8 +1612,8 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="">setopts(SslSocket, Options) -> ok | {error, Reason}</name> <fsummary>Sets socket options.</fsummary> <type> - <v>SslSocket = sslsocket()</v> - <v>Options = [socketoption]()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> + <v>Options = <seealso marker="#type-socket_option"> [socket_option()] </seealso></v> </type> <desc> <p>Sets options according to <c>Options</c> for socket @@ -1403,7 +1625,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="OTP R14B">shutdown(SslSocket, How) -> ok | {error, Reason}</name> <fsummary>Immediately closes a socket.</fsummary> <type> - <v>SslSocket = sslsocket()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> <v>How = read | write | read_write</v> <v>Reason = reason()</v> </type> @@ -1422,9 +1644,9 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="">ssl_accept(SslSocket, Timeout) -> ok | {error, Reason}</name> <fsummary>Performs server-side SSL/TLS handshake.</fsummary> <type> - <v>SslSocket = sslsocket()</v> - <v>Timeout = integer()</v> - <v>Reason = term()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> + <v>Timeout = timeout()</v> + <v>Reason = closed | timeout | <seealso marker="#type-error_alert"> error_alert() </seealso></v> </type> <desc> <p>Deprecated in OTP 21, use <seealso marker="#handshake-1">handshake/[1,2]</seealso> instead.</p> @@ -1433,14 +1655,14 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name since="">ssl_accept(Socket, SslOptions) -> </name> - <name since="OTP R14B">ssl_accept(Socket, SslOptions, Timeout) -> {ok, Socket} | ok | {error, Reason}</name> + <name since="">ssl_accept(Socket, Options) -> </name> + <name since="OTP R14B">ssl_accept(Socket, Options, Timeout) -> {ok, Socket} | ok | {error, Reason}</name> <fsummary>Performs server-side SSL/TLS/DTLS handshake.</fsummary> <type> - <v>Socket = socket() | sslsocket() </v> - <v>SslOptions = [ssl_option()]</v> - <v>Timeout = integer()</v> - <v>Reason = term()</v> + <v>Socket = socket() | <seealso marker="#type-sslsocket"> sslsocket() </seealso> </v> + <v>Options = <seealso marker="#type-tls_server_option"> [server_option()] </seealso> </v> + <v>Timeout = timeout()</v> + <v>Reason = closed | timeout | <seealso marker="#type-error_alert"> error_alert() </seealso></v> </type> <desc> <p>Deprecated in OTP 21, use <seealso marker="#handshake-3">handshake/[2,3]</seealso> instead.</p> @@ -1453,9 +1675,9 @@ fun(srp, Username :: string(), UserState :: term()) -> {error, Reason}</name> <fsummary>Returns the local address and port.</fsummary> <type> - <v>SslSocket = sslsocket()</v> - <v>Address = ipaddress()</v> - <v>Port = integer()</v> + <v>SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> + <v>Address = <seealso marker="#type-ip_address">ip_address()</seealso></v> + <v>Port = <seealso marker="kernel:inet#type-port_number">inet:port_number()</seealso></v> </type> <desc> <p>Returns the local address and port number of socket @@ -1488,7 +1710,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <name since="OTP 21.0">suite_to_str(CipherSuite) -> String</name> <fsummary>Returns the string representation of a cipher suite.</fsummary> <type> - <v>CipherSuite = erl_cipher_suite()</v> + <v>CipherSuite = <seealso marker="#type-erl_cipher_suite"> erl_cipher_suite() </seealso></v> <v>String = string()</v> </type> <desc> @@ -1503,8 +1725,8 @@ fun(srp, Username :: string(), UserState :: term()) -> <fsummary>Accepts an incoming connection and prepares for <c>ssl_accept</c>.</fsummary> <type> - <v>ListenSocket = SslSocket = sslsocket()</v> - <v>Timeout = integer()</v> + <v>ListenSocket = SslSocket = <seealso marker="#type-sslsocket"> sslsocket() </seealso></v> + <v>Timeout = timeout()</v> <v>Reason = reason()</v> </type> <desc> diff --git a/lib/ssl/doc/src/ssl_crl_cache.xml b/lib/ssl/doc/src/ssl_crl_cache.xml index b766cfd2d9..a33aec62a7 100644 --- a/lib/ssl/doc/src/ssl_crl_cache.xml +++ b/lib/ssl/doc/src/ssl_crl_cache.xml @@ -34,15 +34,27 @@ the following functions are available. </p> </description> + + <datatypes> + <datatype_title>DATA TYPES</datatype_title> + + <datatype> + <name name="crl_src"/> + </datatype> + + <datatype> + <name name="uri"/> + </datatype> + + </datatypes> <funcs> <func> <name since="OTP 18.0">delete(Entries) -> ok | {error, Reason} </name> <fsummary> </fsummary> <type> - <v> Entries = <seealso marker="stdlib:uri_string">uri_string:uri_string()</seealso> | {file, string()} | {der, [<seealso - marker="public_key:public_key"> public_key:der_encoded() </seealso>]}</v> - <v> Reason = term()</v> + <v> Entries = <seealso marker="#type-crl_src">crl_src()</seealso>]}</v> + <v> Reason = crl_reason()</v> </type> <desc> <p>Delete CRLs from the ssl applications local cache. </p> @@ -53,13 +65,12 @@ <name since="OTP 18.0">insert(URI, CRLSrc) -> ok | {error, Reason}</name> <fsummary> </fsummary> <type> - <v> CRLSrc = {file, string()} | {der, [ <seealso - marker="public_key:public_key"> public_key:der_encoded() </seealso> ]}</v> - <v> URI = <seealso marker="stdlib:uri_string">uri_string:uri_string() </seealso> </v> + <v> CRLSrc = <seealso marker="#type-crl_src">crl_src()</seealso>]}</v> + <v> URI = <seealso marker="#type-uri">uri()</seealso> </v> <v> Reason = term()</v> </type> <desc> - <p>Insert CRLs into the ssl applications local cache. </p> + <p>Insert CRLs, available to fetch on DER format from <c>URI</c>, into the ssl applications local cache. </p> </desc> </func> </funcs> diff --git a/lib/ssl/doc/src/ssl_crl_cache_api.xml b/lib/ssl/doc/src/ssl_crl_cache_api.xml index c7e501867f..4cba4e1de1 100644 --- a/lib/ssl/doc/src/ssl_crl_cache_api.xml +++ b/lib/ssl/doc/src/ssl_crl_cache_api.xml @@ -39,35 +39,44 @@ a CRL cache. </p> </description> - - <section> - <title>DATA TYPES</title> - - <p>The following data types are used in the functions below: - </p> - - <taglist> - - <tag><c>cache_ref() =</c></tag> - <item>opaque()</item> - <tag><c>dist_point() =</c></tag> - <item><p>#'DistributionPoint'{} see <seealso - marker="public_key:public_key_records"> X509 certificates records</seealso></p></item> - - </taglist> + + + <!-- + ================================================================ + = Data types = + ================================================================ + --> + + <datatypes> - </section> + <datatype> + <name name="crl_cache_ref"/> + <desc> + <p>Reference to the CRL cache.</p> + </desc> + </datatype> + + + <datatype> + <name name="dist_point"/> + <desc> + <p>For description see <seealso + marker="public_key:public_key_records"> X509 certificates records</seealso></p> + </desc> + </datatype> + </datatypes> + <funcs> <func> <name since="OTP 18.0">fresh_crl(DistributionPoint, CRL) -> FreshCRL</name> <fsummary> <c>fun fresh_crl/2 </c> will be used as input option <c>update_crl</c> to public_key:pkix_crls_validate/3 </fsummary> <type> - <v> DistributionPoint = dist_point() </v> + <v> DistributionPoint = <seealso marker="#type-dist_point"> dist_point() </seealso> </v> <v> CRL = [<seealso - marker="public_key:public_key">public_key:der_encoded()</seealso>] </v> + marker="public_key:public_key#type-der_encoded">public_key:der_encoded()</seealso>] </v> <v> FreshCRL = [<seealso - marker="public_key:public_key">public_key:der_encoded()</seealso>] </v> + marker="public_key:public_key#type-der_encoded">public_key:der_encoded()</seealso>] </v> </type> <desc> <p> <c>fun fresh_crl/2 </c> will be used as input option <c>update_crl</c> to @@ -80,12 +89,12 @@ <name since="OTP 18.0">lookup(DistributionPoint, DbHandle) -> not_available | CRLs </name> <fsummary> </fsummary> <type> - <v> DistributionPoint = dist_point() </v> + <v> DistributionPoint = <seealso marker="#type-dist_point"> dist_point() </seealso> </v> <v> Issuer = <seealso - marker="public_key:public_key">public_key:issuer_name()</seealso> </v> - <v> DbHandle = cache_ref() </v> + marker="public_key:public_key#type-issuer_name">public_key:issuer_name()</seealso> </v> + <v> DbHandle = <seealso marker="#type-crl_cache_ref"> crl_cache_ref() </seealso></v> <v> CRLs = [<seealso - marker="public_key:public_key">public_key:der_encoded()</seealso>] </v> + marker="public_key:public_key#type-der_encoded">public_key:der_encoded()</seealso>] </v> </type> <desc> <p>Lookup the CRLs belonging to the distribution point <c> Distributionpoint</c>. This function may choose to only look in the cache or to follow distribution point @@ -110,8 +119,8 @@ <fsummary>Select the CRLs in the cache that are issued by <c>Issuer</c></fsummary> <type> <v> Issuer = <seealso - marker="public_key:public_key">public_key:issuer_name()</seealso></v> - <v> DbHandle = cache_ref() </v> + marker="public_key:public_key#type-issuer_name">public_key:issuer_name()</seealso></v> + <v> DbHandle = <seealso marker="#type-crl_cache_ref"> cache_ref() </seealso></v> </type> <desc> <p>Select the CRLs in the cache that are issued by <c>Issuer</c> </p> diff --git a/lib/ssl/doc/src/ssl_session_cache_api.xml b/lib/ssl/doc/src/ssl_session_cache_api.xml index 463cf15309..e841729e57 100644 --- a/lib/ssl/doc/src/ssl_session_cache_api.xml +++ b/lib/ssl/doc/src/ssl_session_cache_api.xml @@ -38,30 +38,41 @@ defining a new callback module implementing this API. </p> </description> - <section> - <title>DATA TYPES</title> - <p>The following data types are used in the functions for - <c>ssl_session_cache_api</c>:</p> - - <taglist> - <tag><c>cache_ref() =</c></tag> - <item><p><c>opaque()</c></p></item> - - <tag><c>key() =</c></tag> - <item><p><c>{partialkey(), session_id()}</c></p></item> - - <tag><c>partialkey() =</c></tag> - <item><p><c>opaque()</c></p></item> - - <tag><c>session_id() =</c></tag> - <item><p><c>binary()</c></p></item> - - <tag><c>session()</c> =</tag> - <item><p><c>opaque()</c></p></item> - </taglist> - - </section> + <!-- + ================================================================ + = Data types = + ================================================================ + --> + + <datatypes> + + <datatype> + <name name="session_cache_ref"/> + </datatype> + + <datatype> + <name name="session_cache_key"/> + <desc> + <p>A key to an entry in the session cache.</p> + </desc> + </datatype> + + <datatype> + <name name="partial_key"/> + <desc> + <p>The opaque part of the key. Does not need to be handled + by the callback.</p> + </desc> + </datatype> + + <datatype> + <name name="session"/> + <desc> + <p>The session data that is stored for each session.</p> + </desc> + </datatype> + </datatypes> <funcs> @@ -69,8 +80,8 @@ <name since="OTP R14B">delete(Cache, Key) -> _</name> <fsummary>Deletes a cache entry.</fsummary> <type> - <v>Cache = cache_ref()</v> - <v>Key = key()</v> + <v>Cache = <seealso marker="#type-session_cache_ref"> session_cache_ref() </seealso></v> + <v>Key = <seealso marker="#type-session_cache_key">session_cache_key() </seealso> </v> </type> <desc> <p>Deletes a cache entry. Is only called from the cache @@ -83,7 +94,9 @@ <name since="OTP R14B">foldl(Fun, Acc0, Cache) -> Acc</name> <fsummary></fsummary> <type> - <v></v> + <v>Fun = fun()</v> + <v>Acc0 = Acc = term()</v> + <v>Cache = <seealso marker="#type-session_cache_ref"> session_cache_ref() </seealso></v> </type> <desc> <p>Calls <c>Fun(Elem, AccIn)</c> on successive elements of the @@ -96,10 +109,11 @@ </func> <func> - <name since="OTP 18.0">init(Args) -> opaque() </name> + <name since="OTP 18.0">init(Args) -> Cache </name> <fsummary>Returns cache reference.</fsummary> <type> - <v>Args = proplists:proplist()</v> + <v>Cache = <seealso marker="#type-session_cache_ref"> session_cache_ref() </seealso></v> + <v>Args = <seealso marker="stdlib:proplists#type-proplist">proplists:proplist()</seealso></v> </type> <desc> <p>Includes property <c>{role, client | server}</c>. @@ -124,9 +138,9 @@ <name since="OTP R14B">lookup(Cache, Key) -> Entry</name> <fsummary>Looks up a cache entry.</fsummary> <type> - <v>Cache = cache_ref()</v> - <v>Key = key()</v> - <v>Entry = session() | undefined</v> + <v>Cache = <seealso marker="#type-session_cache_ref"> session_cache_ref() </seealso></v> + <v>Key = <seealso marker="#type-session_cache_key">session_cache_key()</seealso> </v> + <v>Session = <seealso marker="#type-session">session()</seealso> | undefined</v> </type> <desc> <p>Looks up a cache entry. Is to be callable from any @@ -136,12 +150,12 @@ </func> <func> - <name since="OTP R14B">select_session(Cache, PartialKey) -> [session()]</name> + <name since="OTP R14B">select_session(Cache, PartialKey) -> [Session]</name> <fsummary>Selects sessions that can be reused.</fsummary> <type> - <v>Cache = cache_ref()</v> - <v>PartialKey = partialkey()</v> - <v>Session = session()</v> + <v>Cache = <seealso marker="#type-session_cache_ref"> session_cache_ref() </seealso></v> + <v>PartialKey = <seealso marker="#type-partial_key"> partial_key() </seealso></v> + <v>Session = <seealso marker="#type-session">session()</seealso></v> </type> <desc> <p>Selects sessions that can be reused. Is to be callable @@ -154,7 +168,7 @@ <name since="OTP 19.3">size(Cache) -> integer()</name> <fsummary>Returns the number of sessions in the cache.</fsummary> <type> - <v>Cache = cache_ref()</v> + <v>Cache = <seealso marker="#type-session_cache_ref"> session_cache_ref() </seealso></v> </type> <desc> <p>Returns the number of sessions in the cache. If size @@ -170,7 +184,8 @@ <fsummary>Called by the process that handles the cache when it is about to terminate.</fsummary> <type> - <v>Cache = term() - as returned by init/0</v> + <v>Cache = <seealso marker="#type-session_cache_ref"> session_cache_ref() </seealso></v> + <d>As returned by init/0</d> </type> <desc> <p>Takes care of possible cleanup that is needed when the @@ -183,9 +198,9 @@ <name since="OTP R14B">update(Cache, Key, Session) -> _</name> <fsummary>Caches a new session or updates an already cached one.</fsummary> <type> - <v>Cache = cache_ref()</v> - <v>Key = key()</v> - <v>Session = session()</v> + <v>Cache = <seealso marker="#type-session_cache_ref"> session_cache_ref() </seealso></v> + <v>Key = <seealso marker="#type-session_cache_key">session_cache_key()</seealso> </v> + <v>Session = <seealso marker="#type-session">session()</seealso></v> </type> <desc> <p>Caches a new session or updates an already cached one. Is diff --git a/lib/ssl/src/dtls_connection.erl b/lib/ssl/src/dtls_connection.erl index 2583667fa2..2c6b71c97a 100644 --- a/lib/ssl/src/dtls_connection.erl +++ b/lib/ssl/src/dtls_connection.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2018. All Rights Reserved. +%% Copyright Ericsson AB 2013-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -50,8 +50,7 @@ -export([encode_alert/3, send_alert/2, send_alert_in_connection/2, close/5, protocol_name/0]). %% Data handling --export([encode_data/3, next_record/1, - send/3, socket/5, setopts/3, getopts/3]). +-export([next_record/1, socket/4, setopts/3, getopts/3]). %% gen_statem state functions -export([init/3, error/3, downgrade/3, %% Initiation and take down states @@ -80,7 +79,7 @@ start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = false},_, Tracker} end. %%-------------------------------------------------------------------- --spec start_link(atom(), host(), inet:port_number(), port(), list(), pid(), tuple()) -> +-spec start_link(atom(), ssl:host(), inet:port_number(), port(), list(), pid(), tuple()) -> {ok, pid()} | ignore | {error, reason()}. %% %% Description: Creates a gen_statem process which calls Module:init/1 to @@ -107,9 +106,11 @@ pids(_) -> %%==================================================================== %% State transition handling %%==================================================================== -next_record(#state{unprocessed_handshake_events = N} = State) when N > 0 -> - {no_record, State#state{unprocessed_handshake_events = N-1}}; - +next_record(#state{handshake_env = + #handshake_env{unprocessed_handshake_events = N} = HsEnv} + = State) when N > 0 -> + {no_record, State#state{handshake_env = + HsEnv#handshake_env{unprocessed_handshake_events = N-1}}}; next_record(#state{protocol_buffers = #protocol_buffers{dtls_cipher_texts = [#ssl_tls{epoch = Epoch} = CT | Rest]} = Buffers, @@ -249,26 +250,29 @@ handle_protocol_record(#ssl_tls{type = ?HANDSHAKE, fragment = Data}, StateName, #state{protocol_buffers = Buffers0, - negotiated_version = Version} = State0) -> + connection_env = #connection_env{negotiated_version = Version}} = State) -> try case dtls_handshake:get_dtls_handshake(Version, Data, Buffers0) of {[], Buffers} -> - next_event(StateName, no_record, State0#state{protocol_buffers = Buffers}); + next_event(StateName, no_record, State#state{protocol_buffers = Buffers}); {Packets, Buffers} -> - State = State0#state{protocol_buffers = Buffers}, + HsEnv = State#state.handshake_env, Events = dtls_handshake_events(Packets), {next_state, StateName, - State#state{unprocessed_handshake_events = unprocessed_events(Events)}, Events} + State#state{protocol_buffers = Buffers, + handshake_env = + HsEnv#handshake_env{unprocessed_handshake_events + = unprocessed_events(Events)}}, Events} end catch throw:#alert{} = Alert -> - handle_own_alert(Alert, Version, StateName, State0) + handle_own_alert(Alert, Version, StateName, State) end; %%% DTLS record protocol level change cipher messages handle_protocol_record(#ssl_tls{type = ?CHANGE_CIPHER_SPEC, fragment = Data}, StateName, State) -> {next_state, StateName, State, [{next_event, internal, #change_cipher_spec{type = Data}}]}; %%% DTLS record protocol level Alert messages handle_protocol_record(#ssl_tls{type = ?ALERT, fragment = EncAlerts}, StateName, - #state{negotiated_version = Version} = State) -> + #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> case decode_alerts(EncAlerts) of Alerts = [_|_] -> handle_alerts(Alerts, {next_state, StateName, State}); @@ -299,8 +303,8 @@ send_handshake(Handshake, #state{connection_states = ConnectionStates} = State) #{epoch := Epoch} = ssl_record:current_connection_state(ConnectionStates, write), send_handshake_flight(queue_handshake(Handshake, State), Epoch). -queue_handshake(Handshake0, #state{tls_handshake_history = Hist0, - negotiated_version = Version, +queue_handshake(Handshake0, #state{handshake_env = #handshake_env{tls_handshake_history = Hist0} = HsEnv, + connection_env = #connection_env{negotiated_version = Version}, flight_buffer = #{handshakes := HsBuffer0, change_cipher_spec := undefined, next_sequence := Seq} = Flight0} = State) -> @@ -308,17 +312,17 @@ queue_handshake(Handshake0, #state{tls_handshake_history = Hist0, Hist = update_handshake_history(Handshake0, Handshake, Hist0), State#state{flight_buffer = Flight0#{handshakes => [Handshake | HsBuffer0], next_sequence => Seq +1}, - tls_handshake_history = Hist}; + handshake_env = HsEnv#handshake_env{tls_handshake_history = Hist}}; -queue_handshake(Handshake0, #state{tls_handshake_history = Hist0, - negotiated_version = Version, +queue_handshake(Handshake0, #state{handshake_env = #handshake_env{tls_handshake_history = Hist0} = HsEnv, + connection_env = #connection_env{negotiated_version = Version}, flight_buffer = #{handshakes_after_change_cipher_spec := Buffer0, next_sequence := Seq} = Flight0} = State) -> Handshake = dtls_handshake:encode_handshake(Handshake0, Version, Seq), Hist = update_handshake_history(Handshake0, Handshake, Hist0), State#state{flight_buffer = Flight0#{handshakes_after_change_cipher_spec => [Handshake | Buffer0], next_sequence => Seq +1}, - tls_handshake_history = Hist}. + handshake_env = HsEnv#handshake_env{tls_handshake_history = Hist}}. queue_change_cipher(ChangeCipher, #state{flight_buffer = Flight, connection_states = ConnectionStates0} = State) -> @@ -330,11 +334,14 @@ queue_change_cipher(ChangeCipher, #state{flight_buffer = Flight, reinit(State) -> %% To be API compatible with TLS NOOP here reinit_handshake_data(State). -reinit_handshake_data(#state{protocol_buffers = Buffers} = State) -> - State#state{premaster_secret = undefined, - public_key_info = undefined, - tls_handshake_history = ssl_handshake:init_handshake_history(), - flight_state = {retransmit, ?INITIAL_RETRANSMIT_TIMEOUT}, +reinit_handshake_data(#state{static_env = #static_env{data_tag = DataTag}, + protocol_buffers = Buffers, + protocol_specific = PS, + handshake_env = HsEnv} = State) -> + State#state{handshake_env = HsEnv#handshake_env{tls_handshake_history = ssl_handshake:init_handshake_history(), + public_key_info = undefined, + premaster_secret = undefined}, + protocol_specific = PS#{flight_state => initial_flight_state(DataTag)}, flight_buffer = new_flight(), protocol_buffers = Buffers#protocol_buffers{ @@ -358,9 +365,9 @@ empty_connection_state(ConnectionEnd, BeastMitigation) -> encode_alert(#alert{} = Alert, Version, ConnectionStates) -> dtls_record:encode_alert_record(Alert, Version, ConnectionStates). -send_alert(Alert, #state{negotiated_version = Version, - static_env = #static_env{socket = Socket, +send_alert(Alert, #state{static_env = #static_env{socket = Socket, transport_cb = Transport}, + connection_env = #connection_env{negotiated_version = Version}, connection_states = ConnectionStates0} = State0) -> {BinMsg, ConnectionStates} = encode_alert(Alert, Version, ConnectionStates0), @@ -384,16 +391,13 @@ protocol_name() -> %% Data handling %%==================================================================== -encode_data(Data, Version, ConnectionStates0)-> - dtls_record:encode_data(Data, Version, ConnectionStates0). - -send(Transport, {_, {{_,_}, _} = Socket}, Data) -> - send(Transport, Socket, Data); -send(Transport, Socket, Data) -> - dtls_socket:send(Transport, Socket, Data). +send(Transport, {Listener, Socket}, Data) when is_pid(Listener) -> % Server socket + dtls_socket:send(Transport, Socket, Data); +send(Transport, Socket, Data) -> % Client socket + dtls_socket:send(Transport, Socket, Data). -socket(Pid, Transport, Socket, Connection, _) -> - dtls_socket:socket(Pid, Transport, Socket, Connection). +socket(Pid, Transport, Socket, _Tracker) -> + dtls_socket:socket(Pid, Transport, Socket, ?MODULE). setopts(Transport, Socket, Other) -> dtls_socket:setopts(Transport, Socket, Other). @@ -417,41 +421,34 @@ init({call, From}, {start, Timeout}, role = client, session_cache = Cache, session_cache_cb = CacheCb}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, + connection_env = CEnv, ssl_options = SslOpts, session = #session{own_certificate = Cert} = Session0, - connection_states = ConnectionStates0, - renegotiation = {Renegotiation, _} + connection_states = ConnectionStates0 } = State0) -> - Timer = ssl_connection:start_or_recv_cancel_timer(Timeout, From), Hello = dtls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, Cache, CacheCb, Renegotiation, Cert), Version = Hello#client_hello.client_version, HelloVersion = dtls_record:hello_version(Version, SslOpts#ssl_options.versions), - State1 = prepare_flight(State0#state{negotiated_version = Version}), - {State2, Actions} = send_handshake(Hello, State1#state{negotiated_version = HelloVersion}), - State3 = State2#state{negotiated_version = Version, %% Requested version + State1 = prepare_flight(State0#state{connection_env = CEnv#connection_env{negotiated_version = Version}}), + {State2, Actions} = send_handshake(Hello, State1#state{connection_env = CEnv#connection_env{negotiated_version = HelloVersion}}), + State3 = State2#state{connection_env = CEnv#connection_env{negotiated_version = Version}, %% RequestedVersion session = Session0#session{session_id = Hello#client_hello.session_id}, - start_or_recv_from = From, - timer = Timer, - flight_state = {retransmit, ?INITIAL_RETRANSMIT_TIMEOUT} - }, + start_or_recv_from = From}, {Record, State} = next_record(State3), - next_event(hello, Record, State, Actions); -init({call, _} = Type, Event, #state{static_env = #static_env{role = server, - data_tag = udp}} = State) -> + next_event(hello, Record, State, [{{timeout, handshake}, Timeout, close} | Actions]); +init({call, _} = Type, Event, #state{static_env = #static_env{role = server}, + protocol_specific = PS} = State) -> Result = gen_handshake(?FUNCTION_NAME, Type, Event, - State#state{flight_state = {retransmit, ?INITIAL_RETRANSMIT_TIMEOUT}, - protocol_specific = #{current_cookie_secret => dtls_v1:cookie_secret(), - previous_cookie_secret => <<>>, - ignored_alerts => 0, - max_ignored_alerts => 10}}), + State#state{protocol_specific = PS#{current_cookie_secret => dtls_v1:cookie_secret(), + previous_cookie_secret => <<>>, + ignored_alerts => 0, + max_ignored_alerts => 10}}), erlang:send_after(dtls_v1:cookie_timeout(), self(), new_cookie_secret), Result; -init({call, _} = Type, Event, #state{static_env = #static_env{role = server}} = State) -> - %% I.E. DTLS over sctp - gen_handshake(?FUNCTION_NAME, Type, Event, State#state{flight_state = reliable}); init(Type, Event, State) -> gen_handshake(?FUNCTION_NAME, Type, Event, State). @@ -487,6 +484,8 @@ hello(internal, #client_hello{cookie = <<>>, #state{static_env = #static_env{role = server, transport_cb = Transport, socket = Socket}, + handshake_env = HsEnv, + connection_env = CEnv, protocol_specific = #{current_cookie_secret := Secret}} = State0) -> {ok, {IP, Port}} = dtls_socket:peername(Transport, Socket), Cookie = dtls_handshake:cookie(Secret, IP, Port, Hello), @@ -497,45 +496,54 @@ hello(internal, #client_hello{cookie = <<>>, %% version 1.0 regardless of the version of TLS that is expected to be %% negotiated. VerifyRequest = dtls_handshake:hello_verify_request(Cookie, ?HELLO_VERIFY_REQUEST_VERSION), - State1 = prepare_flight(State0#state{negotiated_version = Version}), + State1 = prepare_flight(State0#state{connection_env = CEnv#connection_env{negotiated_version = Version}}), {State2, Actions} = send_handshake(VerifyRequest, State1), {Record, State} = next_record(State2), - next_event(?FUNCTION_NAME, Record, State#state{tls_handshake_history = ssl_handshake:init_handshake_history()}, Actions); + next_event(?FUNCTION_NAME, Record, + State#state{handshake_env = HsEnv#handshake_env{ + tls_handshake_history = + ssl_handshake:init_handshake_history()}}, + Actions); hello(internal, #hello_verify_request{cookie = Cookie}, #state{static_env = #static_env{role = client, host = Host, port = Port, session_cache = Cache, session_cache_cb = CacheCb}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}} = HsEnv, + connection_env = CEnv, ssl_options = SslOpts, session = #session{own_certificate = OwnCert} = Session0, - connection_states = ConnectionStates0, - renegotiation = {Renegotiation, _} + connection_states = ConnectionStates0 } = State0) -> Hello = dtls_handshake:client_hello(Host, Port, Cookie, ConnectionStates0, SslOpts, Cache, CacheCb, Renegotiation, OwnCert), Version = Hello#client_hello.client_version, - State1 = prepare_flight(State0#state{tls_handshake_history = ssl_handshake:init_handshake_history()}), + State1 = prepare_flight(State0#state{handshake_env = + HsEnv#handshake_env{tls_handshake_history + = ssl_handshake:init_handshake_history()}}), {State2, Actions} = send_handshake(Hello, State1), - State = State2#state{negotiated_version = Version, %% Requested version + State = State2#state{connection_env = CEnv#connection_env{negotiated_version = Version}, %% Requested version session = Session0#session{session_id = Hello#client_hello.session_id}}, next_event(?FUNCTION_NAME, no_record, State, Actions); hello(internal, #client_hello{extensions = Extensions} = Hello, #state{ssl_options = #ssl_options{handshake = hello}, + handshake_env = HsEnv, start_or_recv_from = From} = State) -> {next_state, user_hello, State#state{start_or_recv_from = undefined, - hello = Hello}, + handshake_env = HsEnv#handshake_env{hello = Hello}}, [{reply, From, {ok, ssl_connection:map_extensions(Extensions)}}]}; hello(internal, #server_hello{extensions = Extensions} = Hello, #state{ssl_options = #ssl_options{handshake = hello}, + handshake_env = HsEnv, start_or_recv_from = From} = State) -> {next_state, user_hello, State#state{start_or_recv_from = undefined, - hello = Hello}, + handshake_env = HsEnv#handshake_env{hello = Hello}}, [{reply, From, {ok, ssl_connection:map_extensions(Extensions)}}]}; hello(internal, #client_hello{cookie = Cookie} = Hello, #state{static_env = #static_env{role = server, @@ -560,9 +568,9 @@ hello(internal, #client_hello{cookie = Cookie} = Hello, #state{static_env = #sta hello(internal, #server_hello{} = Hello, #state{ static_env = #static_env{role = client}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, + connection_env = #connection_env{negotiated_version = ReqVersion}, connection_states = ConnectionStates0, - negotiated_version = ReqVersion, - renegotiation = {Renegotiation, _}, ssl_options = SslOptions} = State) -> case dtls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation) of #alert{} = Alert -> @@ -609,10 +617,11 @@ abbreviated(internal = Type, ConnectionStates1 = dtls_record:save_current_connection_state(ConnectionStates0, read), ConnectionStates = dtls_record:next_epoch(ConnectionStates1, read), gen_handshake(?FUNCTION_NAME, Type, Event, State#state{connection_states = ConnectionStates}); -abbreviated(internal = Type, #finished{} = Event, #state{connection_states = ConnectionStates} = State) -> +abbreviated(internal = Type, #finished{} = Event, #state{connection_states = ConnectionStates, + protocol_specific = PS} = State) -> gen_handshake(?FUNCTION_NAME, Type, Event, prepare_flight(State#state{connection_states = ConnectionStates, - flight_state = connection})); + protocol_specific = PS#{flight_state => connection}})); abbreviated(state_timeout, Event, State) -> handle_state_timeout(Event, ?FUNCTION_NAME, State); abbreviated(Type, Event, State) -> @@ -652,10 +661,11 @@ cipher(internal = Type, #change_cipher_spec{type = <<1>>} = Event, ConnectionStates1 = dtls_record:save_current_connection_state(ConnectionStates0, read), ConnectionStates = dtls_record:next_epoch(ConnectionStates1, read), ssl_connection:?FUNCTION_NAME(Type, Event, State#state{connection_states = ConnectionStates}, ?MODULE); -cipher(internal = Type, #finished{} = Event, #state{connection_states = ConnectionStates} = State) -> +cipher(internal = Type, #finished{} = Event, #state{connection_states = ConnectionStates, + protocol_specific = PS} = State) -> ssl_connection:?FUNCTION_NAME(Type, Event, prepare_flight(State#state{connection_states = ConnectionStates, - flight_state = connection}), + protocol_specific = PS#{flight_state => connection}}), ?MODULE); cipher(state_timeout, Event, State) -> handle_state_timeout(Event, ?FUNCTION_NAME, State); @@ -673,39 +683,43 @@ connection(info, Event, State) -> gen_info(Event, ?FUNCTION_NAME, State); connection(internal, #hello_request{}, #state{static_env = #static_env{host = Host, port = Port, + data_tag = DataTag, session_cache = Cache, session_cache_cb = CacheCb }, + handshake_env = #handshake_env{ renegotiation = {Renegotiation, _}}, + connection_env = CEnv, session = #session{own_certificate = Cert} = Session0, - ssl_options = SslOpts, connection_states = ConnectionStates0, - renegotiation = {Renegotiation, _}} = State0) -> + protocol_specific = PS + } = State0) -> Hello = dtls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, Cache, CacheCb, Renegotiation, Cert), Version = Hello#client_hello.client_version, HelloVersion = dtls_record:hello_version(Version, SslOpts#ssl_options.versions), State1 = prepare_flight(State0), - {State2, Actions} = send_handshake(Hello, State1#state{negotiated_version = HelloVersion}), + {State2, Actions} = send_handshake(Hello, State1#state{connection_env = CEnv#connection_env{negotiated_version = HelloVersion}}), {Record, State} = next_record( - State2#state{flight_state = {retransmit, ?INITIAL_RETRANSMIT_TIMEOUT}, + State2#state{protocol_specific = PS#{flight_state => initial_flight_state(DataTag)}, session = Session0#session{session_id - = Hello#client_hello.session_id}}), + = Hello#client_hello.session_id}}), next_event(hello, Record, State, Actions); connection(internal, #client_hello{} = Hello, #state{static_env = #static_env{role = server}, - allow_renegotiate = true} = State) -> + handshake_env = #handshake_env{allow_renegotiate = true} = HsEnv} = State) -> %% Mitigate Computational DoS attack %% http://www.educatedguesswork.org/2011/10/ssltls_and_computational_dos.html %% http://www.thc.org/thc-ssl-dos/ Rather than disabling client %% initiated renegotiation we will disallow many client initiated %% renegotiations immediately after each other. erlang:send_after(?WAIT_TO_ALLOW_RENEGOTIATION, self(), allow_renegotiate), - {next_state, hello, State#state{allow_renegotiate = false, renegotiation = {true, peer}}, + {next_state, hello, State#state{handshake_env = HsEnv#handshake_env{renegotiation = {true, peer}, + allow_renegotiate = false}}, [{next_event, internal, Hello}]}; connection(internal, #client_hello{}, #state{static_env = #static_env{role = server}, - allow_renegotiate = false} = State0) -> + handshake_env = #handshake_env{allow_renegotiate = false}} = State0) -> Alert = ?ALERT_REC(?WARNING, ?NO_RENEGOTIATION), State1 = send_alert(Alert, State0), {Record, State} = ssl_connection:prepare_connection(State1, ?MODULE), @@ -774,6 +788,12 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions, _}, User, }, #state{static_env = InitStatEnv, + handshake_env = #handshake_env{ + tls_handshake_history = ssl_handshake:init_handshake_history(), + renegotiation = {false, first}, + allow_renegotiate = SSLOptions#ssl_options.client_renegotiation + }, + connection_env = #connection_env{user_application = {Monitor, User}}, socket_options = SocketOptions, %% We do not want to save the password in the state so that %% could be written in the clear into error logs. @@ -781,15 +801,17 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions, _}, User, session = #session{is_resumable = new}, connection_states = ConnectionStates, protocol_buffers = #protocol_buffers{}, - user_application = {Monitor, User}, - user_data_buffer = <<>>, - renegotiation = {false, first}, - allow_renegotiate = SSLOptions#ssl_options.client_renegotiation, + user_data_buffer = {[],0,[]}, start_or_recv_from = undefined, flight_buffer = new_flight(), - flight_state = {retransmit, ?INITIAL_RETRANSMIT_TIMEOUT} + protocol_specific = #{flight_state => initial_flight_state(DataTag)} }. +initial_flight_state(udp)-> + {retransmit, ?INITIAL_RETRANSMIT_TIMEOUT}; +initial_flight_state(_) -> + reliable. + next_dtls_record(Data, StateName, #state{protocol_buffers = #protocol_buffers{ dtls_record_buffer = Buf0, dtls_cipher_texts = CT0} = Buffers} = State0) -> @@ -807,7 +829,7 @@ next_dtls_record(Data, StateName, #state{protocol_buffers = #protocol_buffers{ acceptable_record_versions(hello, _) -> [dtls_record:protocol_version(Vsn) || Vsn <- ?ALL_DATAGRAM_SUPPORTED_VERSIONS]; -acceptable_record_versions(_, #state{negotiated_version = Version}) -> +acceptable_record_versions(_, #state{connection_env = #connection_env{negotiated_version = Version}}) -> [Version]. dtls_handshake_events(Packets) -> @@ -826,8 +848,9 @@ decode_cipher_text(#state{protocol_buffers = #protocol_buffers{dtls_cipher_texts {Alert, State} end. -dtls_version(hello, Version, #state{static_env = #static_env{role = server}} = State) -> - State#state{negotiated_version = Version}; %%Inital version +dtls_version(hello, Version, #state{static_env = #static_env{role = server}, + connection_env = CEnv} = State) -> + State#state{connection_env = CEnv#connection_env{negotiated_version = Version}}; %%Inital version dtls_version(_,_, State) -> State. @@ -836,11 +859,11 @@ handle_client_hello(#client_hello{client_version = ClientVersion} = Hello, static_env = #static_env{port = Port, session_cache = Cache, session_cache_cb = CacheCb}, + handshake_env = #handshake_env{kex_algorithm = KeyExAlg, + renegotiation = {Renegotiation, _}, + negotiated_protocol = CurrentProtocol} = HsEnv, + connection_env = CEnv, session = #session{own_certificate = Cert} = Session0, - renegotiation = {Renegotiation, _}, - - negotiated_protocol = CurrentProtocol, - key_algorithm = KeyExAlg, ssl_options = SslOpts} = State0) -> case dtls_handshake:hello(Hello, SslOpts, {Port, Session0, Cache, CacheCb, @@ -855,11 +878,12 @@ handle_client_hello(#client_hello{client_version = ClientVersion} = Hello, end, State = prepare_flight(State0#state{connection_states = ConnectionStates, - negotiated_version = Version, - hashsign_algorithm = HashSign, - client_hello_version = ClientVersion, - session = Session, - negotiated_protocol = Protocol}), + connection_env = CEnv#connection_env{negotiated_version = Version}, + handshake_env = HsEnv#handshake_env{ + hashsign_algorithm = HashSign, + client_hello_version = ClientVersion, + negotiated_protocol = Protocol}, + session = Session}), ssl_connection:hello(internal, {common_client_hello, Type, ServerHelloExt}, State, ?MODULE) @@ -879,9 +903,9 @@ handle_info({Protocol, _, _, _, Data}, StateName, handle_info({CloseTag, Socket}, StateName, #state{static_env = #static_env{socket = Socket, close_tag = CloseTag}, + connection_env = #connection_env{negotiated_version = Version}, socket_options = #socket_options{active = Active}, - protocol_buffers = #protocol_buffers{dtls_cipher_texts = CTs}, - negotiated_version = Version} = State) -> + protocol_buffers = #protocol_buffers{dtls_cipher_texts = CTs}} = State) -> %% Note that as of DTLS 1.2 (TLS 1.1), %% failure to properly close a connection no longer requires that a %% session not be resumed. This is a change from DTLS 1.0 to conform @@ -917,9 +941,10 @@ handle_info(Msg, StateName, State) -> ssl_connection:StateName(info, Msg, State, ?MODULE). handle_state_timeout(flight_retransmission_timeout, StateName, - #state{flight_state = {retransmit, NextTimeout}} = State0) -> - {State1, Actions0} = send_handshake_flight(State0#state{flight_state = {retransmit, NextTimeout}}, - retransmit_epoch(StateName, State0)), + #state{protocol_specific = + #{flight_state := {retransmit, _NextTimeout}}} = State0) -> + {State1, Actions0} = send_handshake_flight(State0, + retransmit_epoch(StateName, State0)), {next_state, StateName, State, Actions} = next_event(StateName, no_record, State1, Actions0), %% This will reset the retransmission timer by repeating the enter state event {repeat_state, State, Actions}. @@ -959,7 +984,7 @@ decode_alerts(Bin) -> ssl_alert:decode(Bin). gen_handshake(StateName, Type, Event, - #state{negotiated_version = Version} = State) -> + #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> try ssl_connection:StateName(Type, Event, State, ?MODULE) of Result -> Result @@ -970,7 +995,7 @@ gen_handshake(StateName, Type, Event, Version, StateName, State) end. -gen_info(Event, connection = StateName, #state{negotiated_version = Version} = State) -> +gen_info(Event, connection = StateName, #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> try handle_info(Event, StateName, State) of Result -> Result @@ -981,7 +1006,7 @@ gen_info(Event, connection = StateName, #state{negotiated_version = Version} = Version, StateName, State) end; -gen_info(Event, StateName, #state{negotiated_version = Version} = State) -> +gen_info(Event, StateName, #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> try handle_info(Event, StateName, State) of Result -> Result @@ -1025,17 +1050,17 @@ next_flight(Flight) -> handshakes_after_change_cipher_spec => []}. handle_flight_timer(#state{static_env = #static_env{data_tag = udp}, - flight_state = {retransmit, Timeout}} = State) -> + protocol_specific = #{flight_state := {retransmit, Timeout}}} = State) -> start_retransmision_timer(Timeout, State); handle_flight_timer(#state{static_env = #static_env{data_tag = udp}, - flight_state = connection} = State) -> + protocol_specific = #{flight_state := connection}} = State) -> {State, []}; -handle_flight_timer(State) -> +handle_flight_timer(#state{protocol_specific = #{flight_state := reliable}} = State) -> %% No retransmision needed i.e DTLS over SCTP - {State#state{flight_state = reliable}, []}. + {State, []}. -start_retransmision_timer(Timeout, State) -> - {State#state{flight_state = {retransmit, new_timeout(Timeout)}}, +start_retransmision_timer(Timeout, #state{protocol_specific = PS} = State) -> + {State#state{protocol_specific = PS#{flight_state => {retransmit, new_timeout(Timeout)}}}, [{state_timeout, Timeout, flight_retransmission_timeout}]}. new_timeout(N) when N =< 30 -> @@ -1045,9 +1070,9 @@ new_timeout(_) -> send_handshake_flight(#state{static_env = #static_env{socket = Socket, transport_cb = Transport}, - flight_buffer = #{handshakes := Flight, + connection_env = #connection_env{negotiated_version = Version}, + flight_buffer = #{handshakes := Flight, change_cipher_spec := undefined}, - negotiated_version = Version, connection_states = ConnectionStates0} = State0, Epoch) -> %% TODO remove hardcoded Max size {Encoded, ConnectionStates} = @@ -1057,10 +1082,10 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, send_handshake_flight(#state{static_env = #static_env{socket = Socket, transport_cb = Transport}, + connection_env = #connection_env{negotiated_version = Version}, flight_buffer = #{handshakes := [_|_] = Flight0, change_cipher_spec := ChangeCipher, handshakes_after_change_cipher_spec := []}, - negotiated_version = Version, connection_states = ConnectionStates0} = State0, Epoch) -> {HsBefore, ConnectionStates1} = encode_handshake_flight(lists:reverse(Flight0), Version, 1400, Epoch, ConnectionStates0), @@ -1071,10 +1096,10 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, send_handshake_flight(#state{static_env = #static_env{socket = Socket, transport_cb = Transport}, + connection_env = #connection_env{negotiated_version = Version}, flight_buffer = #{handshakes := [_|_] = Flight0, change_cipher_spec := ChangeCipher, handshakes_after_change_cipher_spec := Flight1}, - negotiated_version = Version, connection_states = ConnectionStates0} = State0, Epoch) -> {HsBefore, ConnectionStates1} = encode_handshake_flight(lists:reverse(Flight0), Version, 1400, Epoch-1, ConnectionStates0), @@ -1087,10 +1112,10 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, send_handshake_flight(#state{static_env = #static_env{socket = Socket, transport_cb = Transport}, + connection_env = #connection_env{negotiated_version = Version}, flight_buffer = #{handshakes := [], change_cipher_spec := ChangeCipher, handshakes_after_change_cipher_spec := Flight1}, - negotiated_version = Version, connection_states = ConnectionStates0} = State0, Epoch) -> {EncChangeCipher, ConnectionStates1} = encode_change_cipher(ChangeCipher, Version, Epoch-1, ConnectionStates0), @@ -1144,21 +1169,21 @@ log_ignore_alert(false, _, _,_) -> send_application_data(Data, From, _StateName, #state{static_env = #static_env{socket = Socket, - protocol_cb = Connection, transport_cb = Transport}, - negotiated_version = Version, + connection_env = #connection_env{negotiated_version = Version}, + handshake_env = HsEnv, connection_states = ConnectionStates0, ssl_options = #ssl_options{renegotiate_at = RenegotiateAt}} = State0) -> case time_to_renegotiate(Data, ConnectionStates0, RenegotiateAt) of true -> - renegotiate(State0#state{renegotiation = {true, internal}}, + renegotiate(State0#state{handshake_env = HsEnv#handshake_env{renegotiation = {true, internal}}}, [{next_event, {call, From}, {application_data, Data}}]); false -> {Msgs, ConnectionStates} = - Connection:encode_data(Data, Version, ConnectionStates0), + dtls_record:encode_data(Data, Version, ConnectionStates0), State = State0#state{connection_states = ConnectionStates}, - case Connection:send(Transport, Socket, Msgs) of + case send(Transport, Socket, Msgs) of ok -> ssl_connection:hibernate_after(connection, State, [{reply, From, ok}]); Result -> @@ -1180,3 +1205,4 @@ is_time_to_renegotiate(N, M) when N < M-> false; is_time_to_renegotiate(_,_) -> true. + diff --git a/lib/ssl/src/dtls_handshake.erl b/lib/ssl/src/dtls_handshake.erl index 3f70eaec8a..6e9bf99e52 100644 --- a/lib/ssl/src/dtls_handshake.erl +++ b/lib/ssl/src/dtls_handshake.erl @@ -46,7 +46,7 @@ %% Handshake handling %%==================================================================== %%-------------------------------------------------------------------- --spec client_hello(host(), inet:port_number(), ssl_record:connection_states(), +-spec client_hello(ssl:host(), inet:port_number(), ssl_record:connection_states(), #ssl_options{}, integer(), atom(), boolean(), der_cert()) -> #client_hello{}. %% @@ -59,7 +59,7 @@ client_hello(Host, Port, ConnectionStates, SslOpts, Cache, CacheCb, Renegotiation, OwnCert). %%-------------------------------------------------------------------- --spec client_hello(host(), inet:port_number(), term(), ssl_record:connection_states(), +-spec client_hello(ssl:host(), inet:port_number(), term(), ssl_record:connection_states(), #ssl_options{}, integer(), atom(), boolean(), der_cert()) -> #client_hello{}. %% @@ -123,7 +123,7 @@ cookie(Key, Address, Port, #client_hello{client_version = {Major, Minor}, Random, SessionId, CipherSuites, CompressionMethods], crypto:hmac(sha, Key, CookieData). %%-------------------------------------------------------------------- --spec hello_verify_request(binary(), dtls_record:dtls_version()) -> #hello_verify_request{}. +-spec hello_verify_request(binary(), ssl_record:ssl_version()) -> #hello_verify_request{}. %% %% Description: Creates a hello verify request message sent by server to %% verify client @@ -151,7 +151,7 @@ encode_handshake(Handshake, Version, Seq) -> %%-------------------------------------------------------------------- %%-------------------------------------------------------------------- --spec get_dtls_handshake(dtls_record:dtls_version(), binary(), #protocol_buffers{}) -> +-spec get_dtls_handshake(ssl_record:ssl_version(), binary(), #protocol_buffers{}) -> {[dtls_handshake()], #protocol_buffers{}}. %% %% Description: Given buffered and new data from dtls_record, collects @@ -215,8 +215,6 @@ handle_client_hello_extensions(Version, Type, Random, CipherSuites, HelloExt, dtls_v1:corresponding_tls_version(Version), SslOpts, Session0, ConnectionStates0, Renegotiation) of - #alert{} = Alert -> - Alert; {Session, ConnectionStates, Protocol, ServerHelloExt} -> {Version, {Type, Session}, ConnectionStates, Protocol, ServerHelloExt, HashSign} catch throw:Alert -> @@ -225,17 +223,16 @@ handle_client_hello_extensions(Version, Type, Random, CipherSuites, handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation) -> - case ssl_handshake:handle_server_hello_extensions(dtls_record, Random, CipherSuite, - Compression, HelloExt, - dtls_v1:corresponding_tls_version(Version), - SslOpt, ConnectionStates0, Renegotiation) of - #alert{} = Alert -> - Alert; + try ssl_handshake:handle_server_hello_extensions(dtls_record, Random, CipherSuite, + Compression, HelloExt, + dtls_v1:corresponding_tls_version(Version), + SslOpt, ConnectionStates0, Renegotiation) of {ConnectionStates, ProtoExt, Protocol} -> {Version, SessionId, ConnectionStates, ProtoExt, Protocol} + catch throw:Alert -> + Alert end. - %%-------------------------------------------------------------------- enc_handshake(#hello_verify_request{protocol_version = {Major, Minor}, diff --git a/lib/ssl/src/dtls_handshake.hrl b/lib/ssl/src/dtls_handshake.hrl index 50e92027d2..41da8e5c8c 100644 --- a/lib/ssl/src/dtls_handshake.hrl +++ b/lib/ssl/src/dtls_handshake.hrl @@ -27,6 +27,7 @@ -define(dtls_handshake, true). -include("ssl_handshake.hrl"). %% Common TLS and DTLS records and Constantes +-include("ssl_api.hrl"). -define(HELLO_VERIFY_REQUEST, 3). -define(HELLO_VERIFY_REQUEST_VERSION, {254, 255}). diff --git a/lib/ssl/src/dtls_packet_demux.erl b/lib/ssl/src/dtls_packet_demux.erl index 1497c77cf3..dccc22a448 100644 --- a/lib/ssl/src/dtls_packet_demux.erl +++ b/lib/ssl/src/dtls_packet_demux.erl @@ -144,11 +144,11 @@ handle_info({Transport, Socket, IP, InPortNo, _} = Msg, #state{listener = Socket %% UDP socket does not have a connection and should not receive an econnreset %% This does however happens on some windows versions. Just ignoring it %% appears to make things work as expected! -handle_info({Error, Socket, econnreset = Error}, #state{listener = Socket, transport = {_,_,_, udp_error}} = State) -> +handle_info({udp_error, Socket, econnreset = Error}, #state{listener = Socket, transport = {_,_,_, udp_error}} = State) -> Report = io_lib:format("Ignore SSL UDP Listener: Socket error: ~p ~n", [Error]), error_logger:info_report(Report), {noreply, State}; -handle_info({Error, Socket, Error}, #state{listener = Socket, transport = {_,_,_, Error}} = State) -> +handle_info({ErrorTag, Socket, Error}, #state{listener = Socket, transport = {_,_,_, ErrorTag}} = State) -> Report = io_lib:format("SSL Packet muliplxer shutdown: Socket error: ~p ~n", [Error]), error_logger:info_report(Report), {noreply, State#state{close=true}}; diff --git a/lib/ssl/src/dtls_record.erl b/lib/ssl/src/dtls_record.erl index b7346d3ec8..2fe875da31 100644 --- a/lib/ssl/src/dtls_record.erl +++ b/lib/ssl/src/dtls_record.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2018. All Rights Reserved. +%% Copyright Ericsson AB 2013-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -49,9 +49,8 @@ is_acceptable_version/2, hello_version/2]). --export_type([dtls_version/0, dtls_atom_version/0]). +-export_type([dtls_atom_version/0]). --type dtls_version() :: ssl_record:ssl_version(). -type dtls_atom_version() :: dtlsv1 | 'dtlsv1.2'. -define(REPLAY_WINDOW_SIZE, 64). @@ -135,7 +134,7 @@ set_connection_state_by_epoch(ReadState, Epoch, #{saved_read := #{epoch := Epoch States#{saved_read := ReadState}. %%-------------------------------------------------------------------- --spec init_connection_state_seq(dtls_version(), ssl_record:connection_states()) -> +-spec init_connection_state_seq(ssl_record:ssl_version(), ssl_record:connection_states()) -> ssl_record:connection_state(). %% %% Description: Copy the read sequence number to the write sequence number @@ -163,7 +162,7 @@ current_connection_state_epoch(#{current_write := #{epoch := Epoch}}, Epoch. %%-------------------------------------------------------------------- --spec get_dtls_records(binary(), [dtls_version()], binary()) -> {[binary()], binary()} | #alert{}. +-spec get_dtls_records(binary(), [ssl_record:ssl_version()], binary()) -> {[binary()], binary()} | #alert{}. %% %% Description: Given old buffer and new data from UDP/SCTP, packs up a records %% and returns it as a list of tls_compressed binaries also returns leftover @@ -188,7 +187,7 @@ get_dtls_records(Data, Versions, Buffer) -> %%==================================================================== %%-------------------------------------------------------------------- --spec encode_handshake(iolist(), dtls_version(), integer(), ssl_record:connection_states()) -> +-spec encode_handshake(iolist(), ssl_record:ssl_version(), integer(), ssl_record:connection_states()) -> {iolist(), ssl_record:connection_states()}. % %% Description: Encodes a handshake message to send on the ssl-socket. @@ -198,7 +197,7 @@ encode_handshake(Frag, Version, Epoch, ConnectionStates) -> %%-------------------------------------------------------------------- --spec encode_alert_record(#alert{}, dtls_version(), ssl_record:connection_states()) -> +-spec encode_alert_record(#alert{}, ssl_record:ssl_version(), ssl_record:connection_states()) -> {iolist(), ssl_record:connection_states()}. %% %% Description: Encodes an alert message to send on the ssl-socket. @@ -210,7 +209,7 @@ encode_alert_record(#alert{level = Level, description = Description}, ConnectionStates). %%-------------------------------------------------------------------- --spec encode_change_cipher_spec(dtls_version(), integer(), ssl_record:connection_states()) -> +-spec encode_change_cipher_spec(ssl_record:ssl_version(), integer(), ssl_record:connection_states()) -> {iolist(), ssl_record:connection_states()}. %% %% Description: Encodes a change_cipher_spec-message to send on the ssl socket. @@ -219,7 +218,7 @@ encode_change_cipher_spec(Version, Epoch, ConnectionStates) -> encode_plain_text(?CHANGE_CIPHER_SPEC, Version, Epoch, ?byte(?CHANGE_CIPHER_SPEC_PROTO), ConnectionStates). %%-------------------------------------------------------------------- --spec encode_data(binary(), dtls_version(), ssl_record:connection_states()) -> +-spec encode_data(binary(), ssl_record:ssl_version(), ssl_record:connection_states()) -> {iolist(),ssl_record:connection_states()}. %% %% Description: Encodes data to send on the ssl-socket. @@ -248,8 +247,8 @@ decode_cipher_text(#ssl_tls{epoch = Epoch} = CipherText, ConnnectionStates0) -> %%==================================================================== %%-------------------------------------------------------------------- --spec protocol_version(dtls_atom_version() | dtls_version()) -> - dtls_version() | dtls_atom_version(). +-spec protocol_version(dtls_atom_version() | ssl_record:ssl_version()) -> + ssl_record:ssl_version() | dtls_atom_version(). %% %% Description: Creates a protocol version record from a version atom %% or vice versa. @@ -263,7 +262,7 @@ protocol_version({254, 253}) -> protocol_version({254, 255}) -> dtlsv1. %%-------------------------------------------------------------------- --spec lowest_protocol_version(dtls_version(), dtls_version()) -> dtls_version(). +-spec lowest_protocol_version(ssl_record:ssl_version(), ssl_record:ssl_version()) -> ssl_record:ssl_version(). %% %% Description: Lowes protocol version of two given versions %%-------------------------------------------------------------------- @@ -277,7 +276,7 @@ lowest_protocol_version(_,Version) -> Version. %%-------------------------------------------------------------------- --spec lowest_protocol_version([dtls_version()]) -> dtls_version(). +-spec lowest_protocol_version([ssl_record:ssl_version()]) -> ssl_record:ssl_version(). %% %% Description: Lowest protocol version present in a list %%-------------------------------------------------------------------- @@ -288,7 +287,7 @@ lowest_protocol_version(Versions) -> lowest_list_protocol_version(Ver, Vers). %%-------------------------------------------------------------------- --spec highest_protocol_version([dtls_version()]) -> dtls_version(). +-spec highest_protocol_version([ssl_record:ssl_version()]) -> ssl_record:ssl_version(). %% %% Description: Highest protocol version present in a list %%-------------------------------------------------------------------- @@ -299,7 +298,7 @@ highest_protocol_version(Versions) -> highest_list_protocol_version(Ver, Vers). %%-------------------------------------------------------------------- --spec highest_protocol_version(dtls_version(), dtls_version()) -> dtls_version(). +-spec highest_protocol_version(ssl_record:ssl_version(), ssl_record:ssl_version()) -> ssl_record:ssl_version(). %% %% Description: Highest protocol version of two given versions %%-------------------------------------------------------------------- @@ -315,7 +314,7 @@ highest_protocol_version(_,Version) -> Version. %%-------------------------------------------------------------------- --spec is_higher(V1 :: dtls_version(), V2::dtls_version()) -> boolean(). +-spec is_higher(V1 :: ssl_record:ssl_version(), V2::ssl_record:ssl_version()) -> boolean(). %% %% Description: Is V1 > V2 %%-------------------------------------------------------------------- @@ -327,7 +326,7 @@ is_higher(_, _) -> false. %%-------------------------------------------------------------------- --spec supported_protocol_versions() -> [dtls_version()]. +-spec supported_protocol_versions() -> [ssl_record:ssl_version()]. %% %% Description: Protocol versions supported %%-------------------------------------------------------------------- @@ -370,7 +369,7 @@ supported_protocol_versions([_|_] = Vsns) -> end. %%-------------------------------------------------------------------- --spec is_acceptable_version(dtls_version(), Supported :: [dtls_version()]) -> boolean(). +-spec is_acceptable_version(ssl_record:ssl_version(), Supported :: [ssl_record:ssl_version()]) -> boolean(). %% %% Description: ssl version 2 is not acceptable security risks are too big. %% @@ -378,7 +377,7 @@ supported_protocol_versions([_|_] = Vsns) -> is_acceptable_version(Version, Versions) -> lists:member(Version, Versions). --spec hello_version(dtls_version(), [dtls_version()]) -> dtls_version(). +-spec hello_version(ssl_record:ssl_version(), [ssl_record:ssl_version()]) -> ssl_record:ssl_version(). hello_version(Version, Versions) -> case dtls_v1:corresponding_tls_version(Version) of TLSVersion when TLSVersion >= {3, 3} -> @@ -547,15 +546,15 @@ decode_cipher_text(#ssl_tls{type = Type, version = Version, compression_algorithm = CompAlg}} = ReadState0, ConnnectionStates0) -> AAD = start_additional_data(Type, Version, Epoch, Seq), - CipherS1 = ssl_record:nonce_seed(BulkCipherAlgo, <<?UINT16(Epoch), ?UINT48(Seq)>>, CipherS0), + CipherS = ssl_record:nonce_seed(BulkCipherAlgo, <<?UINT16(Epoch), ?UINT48(Seq)>>, CipherS0), TLSVersion = dtls_v1:corresponding_tls_version(Version), - case ssl_record:decipher_aead(BulkCipherAlgo, CipherS1, AAD, CipherFragment, TLSVersion) of - {PlainFragment, CipherState} -> - {Plain, CompressionS1} = ssl_record:uncompress(CompAlg, + case ssl_record:decipher_aead(BulkCipherAlgo, CipherS, AAD, CipherFragment, TLSVersion) of + PlainFragment when is_binary(PlainFragment) -> + {Plain, CompressionS} = ssl_record:uncompress(CompAlg, PlainFragment, CompressionS0), - ReadState0 = ReadState0#{compression_state => CompressionS1, - cipher_state => CipherState}, - ReadState = update_replay_window(Seq, ReadState0), + ReadState1 = ReadState0#{compression_state := CompressionS, + cipher_state := CipherS}, + ReadState = update_replay_window(Seq, ReadState1), ConnnectionStates = set_connection_state_by_epoch(ReadState, Epoch, ConnnectionStates0, read), {CipherText#ssl_tls{fragment = Plain}, ConnnectionStates}; #alert{} = Alert -> diff --git a/lib/ssl/src/inet_tls_dist.erl b/lib/ssl/src/inet_tls_dist.erl index a4f8bb7562..5cab35fd4b 100644 --- a/lib/ssl/src/inet_tls_dist.erl +++ b/lib/ssl/src/inet_tls_dist.erl @@ -480,22 +480,25 @@ allowed_nodes(PeerCert, Allowed, PeerIP, Node, Host) -> allowed_nodes(PeerCert, Allowed, PeerIP) end. - - setup(Node, Type, MyNode, LongOrShortNames, SetupTime) -> gen_setup(inet_tcp, Node, Type, MyNode, LongOrShortNames, SetupTime). gen_setup(Driver, Node, Type, MyNode, LongOrShortNames, SetupTime) -> Kernel = self(), monitor_pid( - spawn_opt( - fun() -> - do_setup( - Driver, Kernel, Node, Type, - MyNode, LongOrShortNames, SetupTime) - end, - [link, {priority, max}])). + spawn_opt(setup_fun(Driver, Kernel, Node, Type, MyNode, LongOrShortNames, SetupTime), + [link, {priority, max}])). + +-spec setup_fun(_,_,_,_,_,_,_) -> fun(() -> no_return()). +setup_fun(Driver, Kernel, Node, Type, MyNode, LongOrShortNames, SetupTime) -> + fun() -> + do_setup( + Driver, Kernel, Node, Type, + MyNode, LongOrShortNames, SetupTime) + end. + +-spec do_setup(_,_,_,_,_,_,_) -> no_return(). do_setup(Driver, Kernel, Node, Type, MyNode, LongOrShortNames, SetupTime) -> {Name, Address} = split_node(Driver, Node, LongOrShortNames), ErlEpmd = net_kernel:epmd_module(), @@ -520,6 +523,8 @@ do_setup(Driver, Kernel, Node, Type, MyNode, LongOrShortNames, SetupTime) -> trace({getaddr_failed, Driver, Address, Other})) end. +-spec do_setup_connect(_,_,_,_,_,_,_,_,_,_) -> no_return(). + do_setup_connect(Driver, Kernel, Node, Address, Ip, TcpPort, Version, Type, MyNode, Timer) -> Opts = trace(connect_options(get_ssl_options(client))), dist_util:reset_timer(Timer), @@ -564,7 +569,7 @@ gen_close(Driver, Socket) -> %% Determine if EPMD module supports address resolving. Default %% is to use inet_tcp:getaddr/2. %% ------------------------------------------------------------ -get_address_resolver(EpmdModule, Driver) -> +get_address_resolver(EpmdModule, _Driver) -> case erlang:function_exported(EpmdModule, address_please, 3) of true -> {EpmdModule, address_please}; _ -> {erl_epmd, address_please} diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index 03a1e40bfc..50e74d5eb7 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2018. All Rights Reserved. +%% Copyright Ericsson AB 1999-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -61,16 +61,317 @@ -deprecated({ssl_accept, 2, eventually}). -deprecated({ssl_accept, 3, eventually}). +-export_type([socket/0, + sslsocket/0, + socket_option/0, + active_msgs/0, + host/0, + tls_option/0, + tls_client_option/0, + tls_server_option/0, + erl_cipher_suite/0, + old_cipher_suite/0, + ciphers/0, + cipher/0, + hash/0, + kex_algo/0, + prf_random/0, + cipher_filters/0, + sign_algo/0, + protocol_version/0, + protocol_extensions/0, + session_id/0, + error_alert/0, + srp_param_type/0]). + +%% ------------------------------------------------------------------------------------------------------- +-type socket() :: gen_tcp:socket(). +-type socket_option() :: gen_tcp:connect_option() | gen_tcp:listen_option() | gen_udp:option(). +-type sslsocket() :: any(). +-type tls_option() :: tls_client_option() | tls_server_option(). +-type tls_client_option() :: client_option() | common_option() | socket_option() | transport_option(). +-type tls_server_option() :: server_option() | common_option() | socket_option() | transport_option(). +-type active_msgs() :: {ssl, sslsocket(), Data::binary() | list()} | {ssl_closed, sslsocket()} | + {ssl_error, sslsocket(), Reason::term()}. +-type transport_option() :: {cb_info, {CallbackModule::atom(), DataTag::atom(), + ClosedTag::atom(), ErrTag::atom()}}. +-type host() :: hostname() | ip_address(). +-type hostname() :: string(). +-type ip_address() :: inet:ip_address(). +-type session_id() :: binary(). +-type protocol_version() :: tls_version() | dtls_version(). +-type tls_version() :: tlsv1 | 'tlsv1.1' | 'tlsv1.2' | 'tlsv1.3' | legacy_version(). +-type dtls_version() :: 'dtlsv1' | 'dtlsv1.2'. +-type legacy_version() :: sslv3. +-type verify_type() :: verify_none | verify_peer. +-type cipher() :: aes_128_cbc | + aes_256_cbc | + aes_128_gcm | + aes_256_gcm | + chacha20_poly1305 | + legacy_cipher(). +-type legacy_cipher() :: rc4_128 | + des_cbc | + '3des_ede_cbc'. + +-type hash() :: sha | + sha2() | + legacy_hash(). + +-type sha2() :: sha224 | + sha256 | + sha384 | + sha512. + +-type legacy_hash() :: md5. + +-type sign_algo() :: rsa | dsa | ecdsa. +-type kex_algo() :: rsa | + dhe_rsa | dhe_dss | + ecdhe_ecdsa | ecdh_ecdsa | ecdh_rsa | + srp_rsa| srp_dss | + psk | dhe_psk | rsa_psk | + dh_anon | ecdh_anon | srp_anon | + any. %% TLS 1.3 +-type erl_cipher_suite() :: #{key_exchange := kex_algo(), + cipher := cipher(), + mac := hash() | aead, + prf := hash() | default_prf %% Old cipher suites, version dependent + }. + +-type old_cipher_suite() :: {kex_algo(), cipher(), hash()} % Pre TLS 1.2 + %% TLS 1.2, internally PRE TLS 1.2 will use default_prf + | {kex_algo(), cipher(), hash() | aead, hash()}. + +-type named_curve() :: sect571r1 | + sect571k1 | + secp521r1 | + brainpoolP512r1 | + sect409k1 | + sect409r1 | + brainpoolP384r1 | + secp384r1 | + sect283k1 | + sect283r1 | + brainpoolP256r1 | + secp256k1 | + secp256r1 | + sect239k1 | + sect233k1 | + sect233r1 | + secp224k1 | + secp224r1 | + sect193r1 | + sect193r2 | + secp192k1 | + secp192r1 | + sect163k1 | + sect163r1 | + sect163r2 | + secp160k1 | + secp160r1 | + secp160r2. + +-type srp_param_type() :: srp_1024 | + srp_1536 | + srp_2048 | + srp_3072 | + srp_4096 | + srp_6144 | + srp_8192. + +-type error_alert() :: {tls_alert, {tls_alert(), Description::string()}}. + +-type tls_alert() :: close_notify | + unexpected_message | + bad_record_mac | + record_overflow | + handshake_failure | + bad_certificate | + unsupported_certificate | + certificate_revoked | + certificate_expired | + certificate_unknown | + illegal_parameter | + unknown_ca | + access_denied | + decode_error | + decrypt_error | + export_restriction| + protocol_version | + insufficient_security | + internal_error | + inappropriate_fallback | + user_canceled | + no_renegotiation | + unsupported_extension | + certificate_unobtainable | + unrecognized_name | + bad_certificate_status_response | + bad_certificate_hash_value | + unknown_psk_identity | + no_application_protocol. +%% ------------------------------------------------------------------------------------------------------- +-type common_option() :: {protocol, protocol()} | + {handshake, handshake_completion()} | + {cert, cert()} | + {certfile, cert_pem()} | + {key, key()} | + {keyfile, key_pem()} | + {password, key_password()} | + {ciphers, cipher_suites()} | + {eccs, eccs()} | + {secure_renegotiate, secure_renegotiation()} | + {depth, allowed_cert_chain_length()} | + {verify_fun, custom_verify()} | + {crl_check, crl_check()} | + {crl_cache, crl_cache_opts()} | + {max_handshake_size, handshake_size()} | + {partial_chain, root_fun()} | + {versions, protocol_versions()} | + {user_lookup_fun, custom_user_lookup()} | + {log_alert, log_alert()} | + {hibernate_after, hibernate_after()} | + {padding_check, padding_check()} | + {beast_mitigation, beast_mitigation()} | + {ssl_imp, ssl_imp()}. + +-type protocol() :: tls | dtls. +-type handshake_completion() :: hello | full. +-type cert() :: public_key:der_encoded(). +-type cert_pem() :: file:filename(). +-type key() :: {'RSAPrivateKey'| 'DSAPrivateKey' | 'ECPrivateKey' |'PrivateKeyInfo', + public_key:der_encoded()} | + #{algorithm := rsa | dss | ecdsa, + engine := crypto:engine_ref(), + key_id := crypto:key_id(), + password => crypto:password()}. +-type key_pem() :: file:filename(). +-type key_password() :: string(). +-type cipher_suites() :: ciphers(). +-type ciphers() :: [erl_cipher_suite()] | + string(). % (according to old API) +-type cipher_filters() :: list({key_exchange | cipher | mac | prf, + algo_filter()}). +-type algo_filter() :: fun((kex_algo()|cipher()|hash()|aead|default_prf) -> true | false). +-type eccs() :: [named_curve()]. +-type secure_renegotiation() :: boolean(). +-type allowed_cert_chain_length() :: integer(). +-type custom_verify() :: {Verifyfun :: fun(), InitialUserState :: term()}. +-type crl_check() :: boolean() | peer | best_effort. +-type crl_cache_opts() :: [term()]. +-type handshake_size() :: integer(). +-type hibernate_after() :: timeout(). +-type root_fun() :: fun(). +-type protocol_versions() :: [protocol_version()]. +-type signature_algs() :: [{hash(), sign_algo()}]. +-type custom_user_lookup() :: {Lookupfun :: fun(), UserState :: term()}. +-type padding_check() :: boolean(). +-type beast_mitigation() :: one_n_minus_one | zero_n | disabled. +-type srp_identity() :: {Username :: string(), Password :: string()}. +-type psk_identity() :: string(). +-type log_alert() :: boolean(). + +%% ------------------------------------------------------------------------------------------------------- + +-type client_option() :: {verify, client_verify_type()} | + {reuse_session, client_reuse_session()} | + {reuse_sessions, client_reuse_sessions()} | + {cacerts, client_cacerts()} | + {cacertfile, client_cafile()} | + {alpn_advertised_protocols, client_alpn()} | + {client_preferred_next_protocols, client_preferred_next_protocols()} | + {psk_identity, client_psk_identity()} | + {srp_identity, client_srp_identity()} | + {server_name_indication, sni()} | + {customize_hostname_check, customize_hostname_check()} | + {signature_algs, client_signature_algs()} | + {fallback, fallback()}. + +-type client_verify_type() :: verify_type(). +-type client_reuse_session() :: session_id(). +-type client_reuse_sessions() :: boolean() | save. +-type client_cacerts() :: [public_key:der_encoded()]. +-type client_cafile() :: file:filename(). +-type app_level_protocol() :: binary(). +-type client_alpn() :: [app_level_protocol()]. +-type client_preferred_next_protocols() :: {Precedence :: server | client, + ClientPrefs :: [app_level_protocol()]} | + {Precedence :: server | client, + ClientPrefs :: [app_level_protocol()], + Default::app_level_protocol()}. +-type client_psk_identity() :: psk_identity(). +-type client_srp_identity() :: srp_identity(). +-type customize_hostname_check() :: list(). +-type sni() :: HostName :: hostname() | disable. +-type client_signature_algs() :: signature_algs(). +-type fallback() :: boolean(). +-type ssl_imp() :: new | old. + +%% ------------------------------------------------------------------------------------------------------- + +-type server_option() :: {cacerts, server_cacerts()} | + {cacertfile, server_cafile()} | + {dh, dh_der()} | + {dhfile, dh_file()} | + {verify, server_verify_type()} | + {fail_if_no_peer_cert, fail_if_no_peer_cert()} | + {reuse_sessions, server_reuse_sessions()} | + {reuse_session, server_reuse_session()} | + {alpn_preferred_protocols, server_alpn()} | + {next_protocols_advertised, server_next_protocol()} | + {psk_identity, server_psk_identity()} | + {honor_cipher_order, boolean()} | + {sni_hosts, sni_hosts()} | + {sni_fun, sni_fun()} | + {honor_cipher_order, honor_cipher_order()} | + {honor_ecc_order, honor_ecc_order()} | + {client_renegotiation, client_renegotiation()}| + {signature_algs, server_signature_algs()}. + +-type server_cacerts() :: [public_key:der_encoded()]. +-type server_cafile() :: file:filename(). +-type server_alpn() :: [app_level_protocol()]. +-type server_next_protocol() :: [app_level_protocol()]. +-type server_psk_identity() :: psk_identity(). +-type dh_der() :: binary(). +-type dh_file() :: file:filename(). +-type server_verify_type() :: verify_type(). +-type fail_if_no_peer_cert() :: boolean(). +-type server_signature_algs() :: signature_algs(). +-type server_reuse_session() :: fun(). +-type server_reuse_sessions() :: boolean(). +-type sni_hosts() :: [{hostname(), [server_option() | common_option()]}]. +-type sni_fun() :: fun(). +-type honor_cipher_order() :: boolean(). +-type honor_ecc_order() :: boolean(). +-type client_renegotiation() :: boolean(). +%% ------------------------------------------------------------------------------------------------------- +-type prf_random() :: client_random | server_random. +-type protocol_extensions() :: #{renegotiation_info => binary(), + signature_algs => signature_algs(), + alpn => app_level_protocol(), + srp => binary(), + next_protocol => app_level_protocol(), + ec_point_formats => [0..2], + elliptic_curves => [public_key:oid()], + sni => hostname()}. +%% ------------------------------------------------------------------------------------------------------- + +%%%-------------------------------------------------------------------- +%%% API +%%%-------------------------------------------------------------------- + %%-------------------------------------------------------------------- --spec start() -> ok | {error, reason()}. --spec start(permanent | transient | temporary) -> ok | {error, reason()}. %% %% Description: Utility function that starts the ssl and applications %% that it depends on. %% see application(3) %%-------------------------------------------------------------------- +-spec start() -> ok | {error, reason()}. start() -> start(temporary). +-spec start(permanent | transient | temporary) -> ok | {error, reason()}. start(Type) -> case application:ensure_all_started(ssl, Type) of {ok, _} -> @@ -87,20 +388,17 @@ stop() -> application:stop(ssl). %%-------------------------------------------------------------------- --spec connect(host() | port(), [connect_option()]) -> {ok, #sslsocket{}} | - {error, reason()}. --spec connect(host() | port(), [connect_option()] | inet:port_number(), - timeout() | list()) -> - {ok, #sslsocket{}} | {error, reason()}. --spec connect(host() | port(), inet:port_number(), list(), timeout()) -> - {ok, #sslsocket{}} | {error, reason()}. - %% %% Description: Connect to an ssl server. %%-------------------------------------------------------------------- +-spec connect(host() | port(), [client_option()]) -> {ok, #sslsocket{}} | + {error, reason()}. connect(Socket, SslOptions) when is_port(Socket) -> connect(Socket, SslOptions, infinity). +-spec connect(host() | port(), [client_option()] | inet:port_number(), + timeout() | list()) -> + {ok, #sslsocket{}} | {error, reason()}. connect(Socket, SslOptions0, Timeout) when is_port(Socket), (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity) -> {Transport,_,_,_} = proplists:get_value(cb_info, SslOptions0, @@ -117,6 +415,8 @@ connect(Socket, SslOptions0, Timeout) when is_port(Socket), connect(Host, Port, Options) -> connect(Host, Port, Options, infinity). +-spec connect(host() | port(), inet:port_number(), [client_option()], timeout()) -> + {ok, #sslsocket{}} | {error, reason()}. connect(Host, Port, Options, Timeout) when (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity) -> try {ok, Config} = handle_options(Options, client, Host), @@ -132,7 +432,7 @@ connect(Host, Port, Options, Timeout) when (is_integer(Timeout) andalso Timeout end. %%-------------------------------------------------------------------- --spec listen(inet:port_number(), [listen_option()]) ->{ok, #sslsocket{}} | {error, reason()}. +-spec listen(inet:port_number(), [tls_server_option()]) ->{ok, #sslsocket{}} | {error, reason()}. %% %% Description: Creates an ssl listen socket. @@ -148,16 +448,16 @@ listen(Port, Options0) -> Error end. %%-------------------------------------------------------------------- --spec transport_accept(#sslsocket{}) -> {ok, #sslsocket{}} | - {error, reason()}. --spec transport_accept(#sslsocket{}, timeout()) -> {ok, #sslsocket{}} | - {error, reason()}. %% %% Description: Performs transport accept on an ssl listen socket %%-------------------------------------------------------------------- +-spec transport_accept(#sslsocket{}) -> {ok, #sslsocket{}} | + {error, reason()}. transport_accept(ListenSocket) -> transport_accept(ListenSocket, infinity). +-spec transport_accept(#sslsocket{}, timeout()) -> {ok, #sslsocket{}} | + {error, reason()}. transport_accept(#sslsocket{pid = {ListenSocket, #config{connection_cb = ConnectionCb} = Config}}, Timeout) when (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity) -> @@ -169,25 +469,25 @@ transport_accept(#sslsocket{pid = {ListenSocket, end. %%-------------------------------------------------------------------- --spec ssl_accept(#sslsocket{}) -> ok | {error, reason()}. --spec ssl_accept(#sslsocket{} | port(), timeout()| [ssl_option() - | transport_option()]) -> - ok | {ok, #sslsocket{}} | {error, reason()}. - --spec ssl_accept(#sslsocket{} | port(), [ssl_option()] | [ssl_option()| transport_option()], timeout()) -> - ok | {ok, #sslsocket{}} | {error, reason()}. %% %% Description: Performs accept on an ssl listen socket. e.i. performs %% ssl handshake. %%-------------------------------------------------------------------- +-spec ssl_accept(#sslsocket{}) -> ok | {error, timeout | closed | {options, any()}| error_alert()}. ssl_accept(ListenSocket) -> ssl_accept(ListenSocket, [], infinity). + +-spec ssl_accept(#sslsocket{} | port(), timeout()| [tls_server_option()]) -> + ok | {ok, #sslsocket{}} | {error, timeout | closed | {options, any()}| error_alert()}. ssl_accept(Socket, Timeout) when (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity) -> ssl_accept(Socket, [], Timeout); ssl_accept(ListenSocket, SslOptions) when is_port(ListenSocket) -> ssl_accept(ListenSocket, SslOptions, infinity); ssl_accept(Socket, Timeout) -> ssl_accept(Socket, [], Timeout). + +-spec ssl_accept(#sslsocket{} | port(), [tls_server_option()], timeout()) -> + ok | {ok, #sslsocket{}} | {error, timeout | closed | {options, any()}| error_alert()}. ssl_accept(Socket, SslOptions, Timeout) when is_port(Socket) -> handshake(Socket, SslOptions, Timeout); ssl_accept(Socket, SslOptions, Timeout) -> @@ -198,20 +498,18 @@ ssl_accept(Socket, SslOptions, Timeout) -> Error end. %%-------------------------------------------------------------------- --spec handshake(#sslsocket{}) -> {ok, #sslsocket{}} | {error, reason()}. --spec handshake(#sslsocket{} | port(), timeout()| [ssl_option() - | transport_option()]) -> - {ok, #sslsocket{}} | {error, reason()}. - --spec handshake(#sslsocket{} | port(), [ssl_option()] | [ssl_option()| transport_option()], timeout()) -> - {ok, #sslsocket{}} | {error, reason()}. %% %% Description: Performs accept on an ssl listen socket. e.i. performs %% ssl handshake. %%-------------------------------------------------------------------- + +%% Performs the SSL/TLS/DTLS server-side handshake. +-spec handshake(#sslsocket{}) -> {ok, #sslsocket{}} | {error, timeout | closed | {options, any()} | error_alert()}. handshake(ListenSocket) -> handshake(ListenSocket, infinity). +-spec handshake(#sslsocket{} | port(), timeout()| [tls_server_option()]) -> + {ok, #sslsocket{}} | {error, timeout | closed | {options, any()} | error_alert()}. handshake(#sslsocket{} = Socket, Timeout) when (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity) -> ssl_connection:handshake(Socket, Timeout); @@ -219,6 +517,8 @@ handshake(#sslsocket{} = Socket, Timeout) when (is_integer(Timeout) andalso Tim handshake(ListenSocket, SslOptions) when is_port(ListenSocket) -> handshake(ListenSocket, SslOptions, infinity). +-spec handshake(#sslsocket{} | port(), [tls_server_option()], timeout()) -> + {ok, #sslsocket{}} | {error, timeout | closed | {options, any()} | error_alert()}. handshake(#sslsocket{} = Socket, [], Timeout) when (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity)-> handshake(Socket, Timeout); @@ -261,7 +561,7 @@ handshake(Socket, SslOptions, Timeout) when is_port(Socket), %%-------------------------------------------------------------------- --spec handshake_continue(#sslsocket{}, [ssl_option()]) -> +-spec handshake_continue(#sslsocket{}, [tls_client_option() | tls_server_option()]) -> {ok, #sslsocket{}} | {error, reason()}. %% %% @@ -270,7 +570,7 @@ handshake(Socket, SslOptions, Timeout) when is_port(Socket), handshake_continue(Socket, SSLOptions) -> handshake_continue(Socket, SSLOptions, infinity). %%-------------------------------------------------------------------- --spec handshake_continue(#sslsocket{}, [ssl_option()], timeout()) -> +-spec handshake_continue(#sslsocket{}, [tls_client_option() | tls_server_option()], timeout()) -> {ok, #sslsocket{}} | {error, reason()}. %% %% @@ -322,7 +622,7 @@ close(#sslsocket{pid = {ListenSocket, #config{transport_info={Transport,_, _, _} send(#sslsocket{pid = [Pid]}, Data) when is_pid(Pid) -> ssl_connection:send(Pid, Data); send(#sslsocket{pid = [_, Pid]}, Data) when is_pid(Pid) -> - tls_sender:send_data(Pid, erlang:iolist_to_binary(Data)); + tls_sender:send_data(Pid, erlang:iolist_to_iovec(Data)); send(#sslsocket{pid = {_, #config{transport_info={_, udp, _, _}}}}, _) -> {error,enotconn}; %% Emulate connection behaviour send(#sslsocket{pid = {dtls,_}}, _) -> @@ -331,13 +631,14 @@ send(#sslsocket{pid = {ListenSocket, #config{transport_info={Transport, _, _, _} Transport:send(ListenSocket, Data). %% {error,enotconn} %%-------------------------------------------------------------------- --spec recv(#sslsocket{}, integer()) -> {ok, binary()| list()} | {error, reason()}. --spec recv(#sslsocket{}, integer(), timeout()) -> {ok, binary()| list()} | {error, reason()}. %% %% Description: Receives data when active = false %%-------------------------------------------------------------------- +-spec recv(#sslsocket{}, integer()) -> {ok, binary()| list()} | {error, reason()}. recv(Socket, Length) -> recv(Socket, Length, infinity). + +-spec recv(#sslsocket{}, integer(), timeout()) -> {ok, binary()| list()} | {error, reason()}. recv(#sslsocket{pid = [Pid|_]}, Length, Timeout) when is_pid(Pid), (is_integer(Timeout) andalso Timeout >= 0) or (Timeout == infinity)-> ssl_connection:recv(Pid, Length, Timeout); @@ -440,13 +741,13 @@ negotiated_protocol(#sslsocket{pid = [Pid|_]}) when is_pid(Pid) -> ssl_connection:negotiated_protocol(Pid). %%-------------------------------------------------------------------- --spec cipher_suites() -> [ssl_cipher_format:old_erl_cipher_suite()] | [string()]. +-spec cipher_suites() -> [old_cipher_suite()] | [string()]. %%-------------------------------------------------------------------- cipher_suites() -> cipher_suites(erlang). %%-------------------------------------------------------------------- -spec cipher_suites(erlang | openssl | all) -> - [ssl_cipher_format:old_erl_cipher_suite() | string()]. + [old_cipher_suite() | string()]. %% Description: Returns all supported cipher suites. %%-------------------------------------------------------------------- cipher_suites(erlang) -> @@ -460,9 +761,9 @@ cipher_suites(all) -> [ssl_cipher_format:erl_suite_definition(Suite) || Suite <- available_suites(all)]. %%-------------------------------------------------------------------- --spec cipher_suites(default | all | anonymous, tls_record:tls_version() | dtls_record:dtls_version() | +-spec cipher_suites(default | all | anonymous, ssl_record:ssl_version() | tls_record:tls_atom_version() | dtls_record:dtls_atom_version()) -> - [ssl_cipher_format:erl_cipher_suite()]. + [erl_cipher_suite()]. %% Description: Returns all default and all supported cipher suites for a %% TLS/DTLS version %%-------------------------------------------------------------------- @@ -478,9 +779,9 @@ cipher_suites(Base, Version) -> [ssl_cipher_format:suite_definition(Suite) || Suite <- supported_suites(Base, Version)]. %%-------------------------------------------------------------------- --spec filter_cipher_suites([ssl_cipher_format:erl_cipher_suite()], +-spec filter_cipher_suites([erl_cipher_suite()], [{key_exchange | cipher | mac | prf, fun()}] | []) -> - [ssl_cipher_format:erl_cipher_suite()]. + [erl_cipher_suite()]. %% Description: Removes cipher suites if any of the filter functions returns false %% for any part of the cipher suite. This function also calls default filter functions %% to make sure the cipher suite are supported by crypto. @@ -497,10 +798,10 @@ filter_cipher_suites(Suites, Filters0) -> prf_filters => add_filter(proplists:get_value(prf, Filters0), PrfF)}, ssl_cipher:filter_suites(Suites, Filters). %%-------------------------------------------------------------------- --spec prepend_cipher_suites([ssl_cipher_format:erl_cipher_suite()] | +-spec prepend_cipher_suites([erl_cipher_suite()] | [{key_exchange | cipher | mac | prf, fun()}], - [ssl_cipher_format:erl_cipher_suite()]) -> - [ssl_cipher_format:erl_cipher_suite()]. + [erl_cipher_suite()]) -> + [erl_cipher_suite()]. %% Description: Make <Preferred> suites become the most prefered %% suites that is put them at the head of the cipher suite list %% and remove them from <Suites> if present. <Preferred> may be a @@ -515,10 +816,10 @@ prepend_cipher_suites(Filters, Suites) -> Preferred = filter_cipher_suites(Suites, Filters), Preferred ++ (Suites -- Preferred). %%-------------------------------------------------------------------- --spec append_cipher_suites(Deferred :: [ssl_cipher_format:erl_cipher_suite()] | +-spec append_cipher_suites(Deferred :: [erl_cipher_suite()] | [{key_exchange | cipher | mac | prf, fun()}], - [ssl_cipher_format:erl_cipher_suite()]) -> - [ssl_cipher_format:erl_cipher_suite()]. + [erl_cipher_suite()]) -> + [erl_cipher_suite()]. %% Description: Make <Deferred> suites suites become the %% least prefered suites that is put them at the end of the cipher suite list %% and removed them from <Suites> if present. @@ -540,8 +841,8 @@ eccs() -> eccs_filter_supported(Curves). %%-------------------------------------------------------------------- --spec eccs(tls_record:tls_version() | tls_record:tls_atom_version() | - dtls_record:dtls_version() | dtls_record:dtls_atom_version()) -> +-spec eccs(tls_record:tls_atom_version() | + ssl_record:ssl_version() | dtls_record:dtls_atom_version()) -> tls_v1:curves(). %% Description: returns the curves supported for a given version of %% ssl/tls. @@ -723,7 +1024,7 @@ versions() -> SupportedDTLSVsns = [dtls_record:protocol_version(Vsn) || Vsn <- DTLSVsns], AvailableTLSVsns = ?ALL_AVAILABLE_VERSIONS, AvailableDTLSVsns = ?ALL_AVAILABLE_DATAGRAM_VERSIONS, - [{ssl_app, ?VSN}, {supported, SupportedTLSVsns}, + [{ssl_app, "9.2"}, {supported, SupportedTLSVsns}, {supported_dtls, SupportedDTLSVsns}, {available, AvailableTLSVsns}, {available_dtls, AvailableDTLSVsns}]. @@ -783,8 +1084,8 @@ format_error(Reason) when is_list(Reason) -> Reason; format_error(closed) -> "TLS connection is closed"; -format_error({tls_alert, Description}) -> - "TLS Alert: " ++ Description; +format_error({tls_alert, {_, Description}}) -> + Description; format_error({options,{FileType, File, Reason}}) when FileType == cacertfile; FileType == certfile; FileType == keyfile; @@ -813,7 +1114,7 @@ tls_version({254, _} = Version) -> %%-------------------------------------------------------------------- --spec suite_to_str(ssl_cipher_format:erl_cipher_suite()) -> string(). +-spec suite_to_str(erl_cipher_suite()) -> string(). %% %% Description: Return the string representation of a cipher suite. %%-------------------------------------------------------------------- @@ -891,15 +1192,12 @@ handle_options(Opts0, Role, Host) -> {list, [{mode, list}]}], Opts0), assert_proplist(Opts), RecordCb = record_cb(Opts), - - ReuseSessionFun = fun(_, _, _, _) -> true end, CaCerts = handle_option(cacerts, Opts, undefined), {Verify, FailIfNoPeerCert, CaCertDefault, VerifyFun, PartialChainHanlder, VerifyClientOnce} = handle_verify_options(Opts, CaCerts), CertFile = handle_option(certfile, Opts, <<>>), - RecordCb = record_cb(Opts), Versions = case handle_option(versions, Opts, []) of [] -> @@ -945,9 +1243,8 @@ handle_options(Opts0, Role, Host) -> default_option_role(server, tls_v1:default_signature_algs(Versions), Role)), tls_version(RecordCb:highest_protocol_version(Versions))), - %% Server side option - reuse_session = handle_option(reuse_session, Opts, ReuseSessionFun), - reuse_sessions = handle_option(reuse_sessions, Opts, true), + reuse_sessions = handle_reuse_sessions_option(reuse_sessions, Opts, Role), + reuse_session = handle_reuse_session_option(reuse_session, Opts, Role), secure_renegotiate = handle_option(secure_renegotiate, Opts, true), client_renegotiation = handle_option(client_renegotiation, Opts, default_option_role(server, true, Role), @@ -1003,8 +1300,8 @@ handle_options(Opts0, Role, Host) -> alpn_preferred_protocols, next_protocols_advertised, client_preferred_next_protocols, log_alert, server_name_indication, honor_cipher_order, padding_check, crl_check, crl_cache, - fallback, signature_algs, eccs, honor_ecc_order, beast_mitigation, - max_handshake_size, handshake, customize_hostname_check], + fallback, signature_algs, eccs, honor_ecc_order, + beast_mitigation, max_handshake_size, handshake, customize_hostname_check], SockOpts = lists:foldl(fun(Key, PropList) -> proplists:delete(Key, PropList) end, Opts, SslOptions), @@ -1138,11 +1435,16 @@ validate_option(srp_identity, {Username, Password}) {unicode:characters_to_binary(Username), unicode:characters_to_binary(Password)}; +validate_option(reuse_session, undefined) -> + undefined; validate_option(reuse_session, Value) when is_function(Value) -> Value; +validate_option(reuse_session, Value) when is_binary(Value) -> + Value; validate_option(reuse_sessions, Value) when is_boolean(Value) -> Value; - +validate_option(reuse_sessions, save = Value) -> + Value; validate_option(secure_renegotiate, Value) when is_boolean(Value) -> Value; validate_option(client_renegotiation, Value) when is_boolean(Value) -> @@ -1265,6 +1567,26 @@ handle_hashsigns_option(_, Version) when Version >= {3, 3} -> handle_hashsigns_option(_, _Version) -> undefined. +handle_reuse_sessions_option(Key, Opts, client) -> + Value = proplists:get_value(Key, Opts, true), + validate_option(Key, Value), + Value; +handle_reuse_sessions_option(Key, Opts0, server) -> + Opts = proplists:delete({Key, save}, Opts0), + Value = proplists:get_value(Key, Opts, true), + validate_option(Key, Value), + Value. + +handle_reuse_session_option(Key, Opts, client) -> + Value = proplists:get_value(Key, Opts, undefined), + validate_option(Key, Value), + Value; +handle_reuse_session_option(Key, Opts, server) -> + ReuseSessionFun = fun(_, _, _, _) -> true end, + Value = proplists:get_value(Key, Opts, ReuseSessionFun), + validate_option(Key, Value), + Value. + validate_options([]) -> []; validate_options([{Opt, Value} | Tail]) -> diff --git a/lib/ssl/src/ssl_alert.erl b/lib/ssl/src/ssl_alert.erl index 34e9797f1f..2a20d13cd5 100644 --- a/lib/ssl/src/ssl_alert.erl +++ b/lib/ssl/src/ssl_alert.erl @@ -48,8 +48,8 @@ decode(Bin) -> decode(Bin, [], 0). %%-------------------------------------------------------------------- --spec reason_code(#alert{}, client | server) -> - closed | {tls_alert, unicode:chardata()}. +%% -spec reason_code(#alert{}, client | server) -> +%% {tls_alert, unicode:chardata()} | closed. %-spec reason_code(#alert{}, client | server) -> closed | {essl, string()}. %% %% Description: Returns the error reason that will be returned to the @@ -58,8 +58,10 @@ decode(Bin) -> reason_code(#alert{description = ?CLOSE_NOTIFY}, _) -> closed; -reason_code(#alert{description = Description}, _) -> - {tls_alert, string:casefold(description_txt(Description))}. +reason_code(#alert{description = Description, role = Role} = Alert, Role) -> + {tls_alert, {description_atom(Description), own_alert_txt(Alert)}}; +reason_code(#alert{description = Description} = Alert, Role) -> + {tls_alert, {description_atom(Description), alert_txt(Alert#alert{role = Role})}}. %%-------------------------------------------------------------------- -spec own_alert_txt(#alert{}) -> string(). @@ -181,3 +183,70 @@ description_txt(?NO_APPLICATION_PROTOCOL) -> "No application protocol"; description_txt(Enum) -> lists:flatten(io_lib:format("unsupported/unknown alert: ~p", [Enum])). + +description_atom(?CLOSE_NOTIFY) -> + close_notify; +description_atom(?UNEXPECTED_MESSAGE) -> + unexpected_message; +description_atom(?BAD_RECORD_MAC) -> + bad_record_mac; +description_atom(?DECRYPTION_FAILED_RESERVED) -> + decryption_failed_reserved; +description_atom(?RECORD_OVERFLOW) -> + record_overflow; +description_atom(?DECOMPRESSION_FAILURE) -> + decompression_failure; +description_atom(?HANDSHAKE_FAILURE) -> + handshake_failure; +description_atom(?NO_CERTIFICATE_RESERVED) -> + no_certificate_reserved; +description_atom(?BAD_CERTIFICATE) -> + bad_certificate; +description_atom(?UNSUPPORTED_CERTIFICATE) -> + unsupported_certificate; +description_atom(?CERTIFICATE_REVOKED) -> + certificate_revoked; +description_atom(?CERTIFICATE_EXPIRED) -> + certificate_expired; +description_atom(?CERTIFICATE_UNKNOWN) -> + certificate_unknown; +description_atom(?ILLEGAL_PARAMETER) -> + illegal_parameter; +description_atom(?UNKNOWN_CA) -> + unknown_ca; +description_atom(?ACCESS_DENIED) -> + access_denied; +description_atom(?DECODE_ERROR) -> + decode_error; +description_atom(?DECRYPT_ERROR) -> + decrypt_error; +description_atom(?EXPORT_RESTRICTION) -> + export_restriction; +description_atom(?PROTOCOL_VERSION) -> + protocol_version; +description_atom(?INSUFFICIENT_SECURITY) -> + insufficient_security; +description_atom(?INTERNAL_ERROR) -> + internal_error; +description_atom(?USER_CANCELED) -> + user_canceled; +description_atom(?NO_RENEGOTIATION) -> + no_renegotiation; +description_atom(?UNSUPPORTED_EXTENSION) -> + unsupported_extension; +description_atom(?CERTIFICATE_UNOBTAINABLE) -> + certificate_unobtainable; +description_atom(?UNRECOGNISED_NAME) -> + unrecognised_name; +description_atom(?BAD_CERTIFICATE_STATUS_RESPONSE) -> + bad_certificate_status_response; +description_atom(?BAD_CERTIFICATE_HASH_VALUE) -> + bad_certificate_hash_value; +description_atom(?UNKNOWN_PSK_IDENTITY) -> + unknown_psk_identity; +description_atom(?INAPPROPRIATE_FALLBACK) -> + inappropriate_fallback; +description_atom(?NO_APPLICATION_PROTOCOL) -> + no_application_protocol; +description_atom(_) -> + 'unsupported/unkonwn_alert'. diff --git a/lib/ssl/src/ssl_api.hrl b/lib/ssl/src/ssl_api.hrl index 7b7b1cbcd9..f4594912bd 100644 --- a/lib/ssl/src/ssl_api.hrl +++ b/lib/ssl/src/ssl_api.hrl @@ -21,56 +21,7 @@ -ifndef(ssl_api). -define(ssl_api, true). --include("ssl_cipher.hrl"). - -%% Visible in API --export_type([connect_option/0, listen_option/0, ssl_option/0, transport_option/0, - prf_random/0, sslsocket/0]). - - %% Looks like it does for backwards compatibility reasons -record(sslsocket, {fd = nil, pid = nil}). - --type sslsocket() :: #sslsocket{}. --type connect_option() :: socket_connect_option() | ssl_option() | transport_option(). --type socket_connect_option() :: gen_tcp:connect_option(). --type listen_option() :: socket_listen_option() | ssl_option() | transport_option(). --type socket_listen_option() :: gen_tcp:listen_option(). - --type ssl_option() :: {versions, ssl_record:ssl_atom_version()} | - {verify, verify_type()} | - {verify_fun, {fun(), InitialUserState::term()}} | - {fail_if_no_peer_cert, boolean()} | {depth, integer()} | - {cert, Der::binary()} | {certfile, path()} | - {key, {private_key_type(), Der::binary()}} | - {keyfile, path()} | {password, string()} | {cacerts, [Der::binary()]} | - {cacertfile, path()} | {dh, Der::binary()} | {dhfile, path()} | - {user_lookup_fun, {fun(), InitialUserState::term()}} | - {psk_identity, string()} | - {srp_identity, {string(), string()}} | - {ciphers, ciphers()} | {ssl_imp, ssl_imp()} | {reuse_sessions, boolean()} | - {reuse_session, fun()} | {hibernate_after, integer()|undefined} | - {alpn_advertised_protocols, [binary()]} | - {alpn_preferred_protocols, [binary()]} | - {next_protocols_advertised, list(binary())} | - {client_preferred_next_protocols, binary(), client | server, list(binary())}. - --type verify_type() :: verify_none | verify_peer. --type path() :: string(). --type ciphers() :: [ssl_cipher_format:erl_cipher_suite()] | - string(). % (according to old API) --type ssl_imp() :: new | old. - --type transport_option() :: {cb_info, {CallbackModule::atom(), DataTag::atom(), - ClosedTag::atom(), ErrTag::atom()}}. --type prf_random() :: client_random | server_random. - --type private_key_type() :: rsa | %% Backwards compatibility - dsa | %% Backwards compatibility - 'RSAPrivateKey' | - 'DSAPrivateKey' | - 'ECPrivateKey' | - 'PrivateKeyInfo'. - -endif. % -ifdef(ssl_api). diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_cipher.erl index 66a00c60f1..fce48d1678 100644 --- a/lib/ssl/src/ssl_cipher.erl +++ b/lib/ssl/src/ssl_cipher.erl @@ -1,7 +1,7 @@ % %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2018. All Rights Reserved. +%% Copyright Ericsson AB 2007-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -41,7 +41,7 @@ rc4_suites/1, des_suites/1, rsa_suites/1, filter/3, filter_suites/1, filter_suites/2, hash_algorithm/1, sign_algorithm/1, is_acceptable_hash/2, is_fallback/1, - random_bytes/1, calc_mac_hash/4, + random_bytes/1, calc_mac_hash/4, calc_mac_hash/6, is_stream_ciphersuite/1]). -compile(inline). @@ -97,7 +97,8 @@ cipher_init(?AES_GCM, IV, Key) -> cipher_init(?CHACHA20_POLY1305, IV, Key) -> #cipher_state{iv = IV, key = Key, tag_len = 16}; cipher_init(_BCA, IV, Key) -> - #cipher_state{iv = IV, key = Key}. + %% Initialize random IV cache, not used for aead ciphers + #cipher_state{iv = IV, key = Key, state = <<>>}. nonce_seed(Seed, CipherState) -> CipherState#cipher_state{nonce = Seed}. @@ -112,12 +113,11 @@ nonce_seed(Seed, CipherState) -> %% data is calculated and the data plus the HMAC is ecncrypted. %%------------------------------------------------------------------- cipher(?NULL, CipherState, <<>>, Fragment, _Version) -> - GenStreamCipherList = [Fragment, <<>>], - {GenStreamCipherList, CipherState}; + {iolist_to_binary(Fragment), CipherState}; cipher(?RC4, CipherState = #cipher_state{state = State0}, Mac, Fragment, _Version) -> GenStreamCipherList = [Fragment, Mac], {State1, T} = crypto:stream_encrypt(State0, GenStreamCipherList), - {T, CipherState#cipher_state{state = State1}}; + {iolist_to_binary(T), CipherState#cipher_state{state = State1}}; cipher(?DES, CipherState, Mac, Fragment, Version) -> block_cipher(fun(Key, IV, T) -> crypto:block_encrypt(des_cbc, Key, IV, T) @@ -146,8 +146,7 @@ aead_type(?CHACHA20_POLY1305) -> build_cipher_block(BlockSz, Mac, Fragment) -> TotSz = byte_size(Mac) + erlang:iolist_size(Fragment) + 1, - {PaddingLength, Padding} = get_padding(TotSz, BlockSz), - [Fragment, Mac, PaddingLength, Padding]. + [Fragment, Mac, padding_with_len(TotSz, BlockSz)]. block_cipher(Fun, BlockSz, #cipher_state{key=Key, iv=IV} = CS0, Mac, Fragment, {3, N}) @@ -157,14 +156,21 @@ block_cipher(Fun, BlockSz, #cipher_state{key=Key, iv=IV} = CS0, NextIV = next_iv(T, IV), {T, CS0#cipher_state{iv=NextIV}}; -block_cipher(Fun, BlockSz, #cipher_state{key=Key, iv=IV} = CS0, +block_cipher(Fun, BlockSz, #cipher_state{key=Key, iv=IV, state = IV_Cache0} = CS0, Mac, Fragment, {3, N}) when N == 2; N == 3 -> - NextIV = random_iv(IV), + IV_Size = byte_size(IV), + <<NextIV:IV_Size/binary, IV_Cache/binary>> = + case IV_Cache0 of + <<>> -> + random_bytes(IV_Size bsl 5); % 32 IVs + _ -> + IV_Cache0 + end, L0 = build_cipher_block(BlockSz, Mac, Fragment), L = [NextIV|L0], T = Fun(Key, IV, L), - {T, CS0#cipher_state{iv=NextIV}}. + {T, CS0#cipher_state{iv=NextIV, state = IV_Cache}}. %%-------------------------------------------------------------------- -spec decipher(cipher_enum(), integer(), #cipher_state{}, binary(), @@ -481,8 +487,8 @@ filter(DerCert, Ciphers0, Version) -> filter_suites_signature(Sign, Ciphers, Version). %%-------------------------------------------------------------------- --spec filter_suites([ssl_cipher_format:erl_cipher_suite()] | [ssl_cipher_format:cipher_suite()], map()) -> - [ssl_cipher_format:erl_cipher_suite()] | [ssl_cipher_format:cipher_suite()]. +-spec filter_suites([ssl:erl_cipher_suite()] | [ssl_cipher_format:cipher_suite()], map()) -> + [ssl:erl_cipher_suite()] | [ssl_cipher_format:cipher_suite()]. %% %% Description: Filter suites using supplied filter funs %%------------------------------------------------------------------- @@ -508,8 +514,8 @@ filter_suite(Suite, Filters) -> filter_suite(ssl_cipher_format:suite_definition(Suite), Filters). %%-------------------------------------------------------------------- --spec filter_suites([ssl_cipher_format:erl_cipher_suite()] | [ssl_cipher_format:cipher_suite()]) -> - [ssl_cipher_format:erl_cipher_suite()] | [ssl_cipher_format:cipher_suite()]. +-spec filter_suites([ssl:erl_cipher_suite()] | [ssl_cipher_format:cipher_suite()]) -> + [ssl:erl_cipher_suite()] | [ssl_cipher_format:cipher_suite()]. %% %% Description: Filter suites for algorithms supported by crypto. %%------------------------------------------------------------------- @@ -593,7 +599,7 @@ is_acceptable_cipher(rc4_128, Algos) -> is_acceptable_cipher(des_cbc, Algos) -> proplists:get_bool(des_cbc, Algos); is_acceptable_cipher('3des_ede_cbc', Algos) -> - proplists:get_bool(des3_cbc, Algos); + proplists:get_bool(des_ede3, Algos); is_acceptable_cipher(aes_128_cbc, Algos) -> proplists:get_bool(aes_cbc128, Algos); is_acceptable_cipher(aes_256_cbc, Algos) -> @@ -633,12 +639,13 @@ random_bytes(N) -> calc_mac_hash(Type, Version, PlainFragment, #{sequence_number := SeqNo, mac_secret := MacSecret, - security_parameters:= - SecPars}) -> + security_parameters := + #security_parameters{mac_algorithm = MacAlgorithm}}) -> + calc_mac_hash(Type, Version, PlainFragment, MacAlgorithm, MacSecret, SeqNo). +%% +calc_mac_hash(Type, Version, PlainFragment, MacAlgorithm, MacSecret, SeqNo) -> Length = erlang:iolist_size(PlainFragment), - mac_hash(Version, SecPars#security_parameters.mac_algorithm, - MacSecret, SeqNo, Type, - Length, PlainFragment). + mac_hash(Version, MacAlgorithm, MacSecret, SeqNo, Type, Length, PlainFragment). is_stream_ciphersuite(#{cipher := rc4_128}) -> true; @@ -722,7 +729,6 @@ expanded_key_material(Cipher) when Cipher == aes_128_cbc; Cipher == chacha20_poly1305 -> unknown. - effective_key_bits(null) -> 0; effective_key_bits(des_cbc) -> @@ -742,18 +748,15 @@ iv_size(Cipher) when Cipher == null; Cipher == rc4_128; Cipher == chacha20_poly1305-> 0; - iv_size(Cipher) when Cipher == aes_128_gcm; Cipher == aes_256_gcm -> 4; - iv_size(Cipher) -> block_size(Cipher). block_size(Cipher) when Cipher == des_cbc; Cipher == '3des_ede_cbc' -> 8; - block_size(Cipher) when Cipher == aes_128_cbc; Cipher == aes_256_cbc; Cipher == aes_128_gcm; @@ -888,21 +891,51 @@ is_correct_padding(GenBlockCipher, {3, 1}, false) -> %% Padding must be checked in TLS 1.1 and after is_correct_padding(#generic_block_cipher{padding_length = Len, padding = Padding}, _, _) -> - Len == byte_size(Padding) andalso - binary:copy(?byte(Len), Len) == Padding. - -get_padding(Length, BlockSize) -> - get_padding_aux(BlockSize, Length rem BlockSize). - -get_padding_aux(_, 0) -> - {0, <<>>}; -get_padding_aux(BlockSize, PadLength) -> - N = BlockSize - PadLength, - {N, binary:copy(?byte(N), N)}. + (Len == byte_size(Padding)) andalso (padding(Len) == Padding). + +padding(PadLen) -> + case PadLen of + 0 -> <<>>; + 1 -> <<1>>; + 2 -> <<2,2>>; + 3 -> <<3,3,3>>; + 4 -> <<4,4,4,4>>; + 5 -> <<5,5,5,5,5>>; + 6 -> <<6,6,6,6,6,6>>; + 7 -> <<7,7,7,7,7,7,7>>; + 8 -> <<8,8,8,8,8,8,8,8>>; + 9 -> <<9,9,9,9,9,9,9,9,9>>; + 10 -> <<10,10,10,10,10,10,10,10,10,10>>; + 11 -> <<11,11,11,11,11,11,11,11,11,11,11>>; + 12 -> <<12,12,12,12,12,12,12,12,12,12,12,12>>; + 13 -> <<13,13,13,13,13,13,13,13,13,13,13,13,13>>; + 14 -> <<14,14,14,14,14,14,14,14,14,14,14,14,14,14>>; + 15 -> <<15,15,15,15,15,15,15,15,15,15,15,15,15,15,15>>; + _ -> + binary:copy(<<PadLen>>, PadLen) + end. -random_iv(IV) -> - IVSz = byte_size(IV), - random_bytes(IVSz). +padding_with_len(TextLen, BlockSize) -> + case BlockSize - (TextLen rem BlockSize) of + 0 -> <<0>>; + 1 -> <<1,1>>; + 2 -> <<2,2,2>>; + 3 -> <<3,3,3,3>>; + 4 -> <<4,4,4,4,4>>; + 5 -> <<5,5,5,5,5,5>>; + 6 -> <<6,6,6,6,6,6,6>>; + 7 -> <<7,7,7,7,7,7,7,7>>; + 8 -> <<8,8,8,8,8,8,8,8,8>>; + 9 -> <<9,9,9,9,9,9,9,9,9,9>>; + 10 -> <<10,10,10,10,10,10,10,10,10,10,10>>; + 11 -> <<11,11,11,11,11,11,11,11,11,11,11,11>>; + 12 -> <<12,12,12,12,12,12,12,12,12,12,12,12,12>>; + 13 -> <<13,13,13,13,13,13,13,13,13,13,13,13,13,13>>; + 14 -> <<14,14,14,14,14,14,14,14,14,14,14,14,14,14,14>>; + 15 -> <<15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15>>; + PadLen -> + binary:copy(<<PadLen>>, PadLen + 1) + end. next_iv(Bin, IV) -> BinSz = byte_size(Bin), diff --git a/lib/ssl/src/ssl_cipher_format.erl b/lib/ssl/src/ssl_cipher_format.erl index c311c0d097..1d28e1e3b4 100644 --- a/lib/ssl/src/ssl_cipher_format.erl +++ b/lib/ssl/src/ssl_cipher_format.erl @@ -25,26 +25,25 @@ %%---------------------------------------------------------------------- -module(ssl_cipher_format). +-include("ssl_api.hrl"). -include("ssl_cipher.hrl"). -include("ssl_internal.hrl"). -include_lib("public_key/include/public_key.hrl"). --export_type([cipher_suite/0, - erl_cipher_suite/0, old_erl_cipher_suite/0, openssl_cipher_suite/0, - hash/0, key_algo/0, sign_algo/0]). +-export_type([old_erl_cipher_suite/0, openssl_cipher_suite/0, cipher_suite/0]). --type cipher() :: null |rc4_128 | des_cbc | '3des_ede_cbc' | aes_128_cbc | aes_256_cbc | aes_128_gcm | aes_256_gcm | chacha20_poly1305. --type hash() :: null | md5 | sha | sha224 | sha256 | sha384 | sha512. --type sign_algo() :: rsa | dsa | ecdsa. --type key_algo() :: null | rsa | dhe_rsa | dhe_dss | ecdhe_ecdsa| ecdh_ecdsa | ecdh_rsa| srp_rsa| srp_dss | psk | dhe_psk | rsa_psk | dh_anon | ecdh_anon | srp_anon. --type erl_cipher_suite() :: #{key_exchange := key_algo(), - cipher := cipher(), - mac := hash() | aead, - prf := hash() | default_prf %% Old cipher suites, version dependent +-type internal_cipher() :: null | ssl:cipher(). +-type internal_hash() :: null | ssl:hash(). +-type internal_kex_algo() :: null | ssl:kex_algo(). +-type internal_erl_cipher_suite() :: #{key_exchange := internal_kex_algo(), + cipher := internal_cipher(), + mac := internal_hash() | aead, + prf := internal_hash() | default_prf %% Old cipher suites, version dependent }. --type old_erl_cipher_suite() :: {key_algo(), cipher(), hash()} % Pre TLS 1.2 +-type old_erl_cipher_suite() :: {ssl:kex_algo(), internal_cipher(), internal_hash()} % Pre TLS 1.2 %% TLS 1.2, internally PRE TLS 1.2 will use default_prf - | {key_algo(), cipher(), hash(), hash() | default_prf}. + | {ssl:kex_algo(), internal_cipher(), internal_hash(), + internal_hash() | default_prf}. -type cipher_suite() :: binary(). -type openssl_cipher_suite() :: string(). @@ -53,7 +52,7 @@ openssl_suite/1, openssl_suite_name/1]). %%-------------------------------------------------------------------- --spec suite_to_str(erl_cipher_suite()) -> string(). +-spec suite_to_str(internal_erl_cipher_suite()) -> string(). %% %% Description: Return the string representation of a cipher suite. %%-------------------------------------------------------------------- @@ -77,7 +76,7 @@ suite_to_str(#{key_exchange := Kex, "_" ++ string:to_upper(atom_to_list(Mac)). %%-------------------------------------------------------------------- --spec suite_definition(cipher_suite()) -> erl_cipher_suite(). +-spec suite_definition(cipher_suite()) -> internal_erl_cipher_suite(). %% %% Description: Return erlang cipher suite definition. %% Note: Currently not supported suites are commented away. @@ -805,7 +804,7 @@ suite_definition(?TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256) -> prf => sha256}. %%-------------------------------------------------------------------- --spec erl_suite_definition(cipher_suite() | erl_cipher_suite()) -> old_erl_cipher_suite(). +-spec erl_suite_definition(cipher_suite() | internal_erl_cipher_suite()) -> old_erl_cipher_suite(). %% %% Description: Return erlang cipher suite definition. Filters last value %% for now (compatibility reasons). @@ -822,7 +821,7 @@ erl_suite_definition(#{key_exchange := KeyExchange, cipher := Cipher, end. %%-------------------------------------------------------------------- --spec suite(erl_cipher_suite()) -> cipher_suite(). +-spec suite(internal_erl_cipher_suite()) -> cipher_suite(). %% %% Description: Return TLS cipher suite definition. %%-------------------------------------------------------------------- @@ -1585,7 +1584,7 @@ openssl_suite("ECDH-RSA-AES256-GCM-SHA384") -> ?TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384. %%-------------------------------------------------------------------- --spec openssl_suite_name(cipher_suite()) -> openssl_cipher_suite() | erl_cipher_suite(). +-spec openssl_suite_name(cipher_suite()) -> openssl_cipher_suite() | internal_erl_cipher_suite(). %% %% Description: Return openssl cipher suite name if possible %%------------------------------------------------------------------- diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index 41d853977e..9e037313bb 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2018. All Rights Reserved. +%% Copyright Ericsson AB 2013-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -40,7 +40,7 @@ -export([connect/8, handshake/7, handshake/2, handshake/3, handshake_continue/3, handshake_cancel/1, - socket_control/4, socket_control/5, start_or_recv_cancel_timer/2]). + socket_control/4, socket_control/5]). %% User Events -export([send/2, recv/3, close/2, shutdown/2, @@ -70,14 +70,14 @@ -export([terminate/3, format_status/2]). %% Erlang Distribution export --export([get_sslsocket/1, dist_handshake_complete/2]). +-export([dist_handshake_complete/2]). %%==================================================================== %% Setup %%==================================================================== %%-------------------------------------------------------------------- -spec connect(tls_connection | dtls_connection, - host(), inet:port_number(), + ssl:host(), inet:port_number(), port() | {tuple(), port()}, %% TLS | DTLS {#ssl_options{}, #socket_options{}, %% Tracker only needed on server side @@ -143,7 +143,7 @@ handshake(#sslsocket{pid = [Pid|_]} = Socket, SslOptions, Timeout) -> end. %%-------------------------------------------------------------------- --spec handshake_continue(#sslsocket{}, [ssl_option()], +-spec handshake_continue(#sslsocket{}, [ssl:tls_server_option()], timeout()) -> {ok, #sslsocket{}}| {error, reason()}. %% %% Description: Continues handshake with new options @@ -182,27 +182,23 @@ socket_control(Connection, Socket, Pid, Transport) -> %%-------------------------------------------------------------------- socket_control(Connection, Socket, Pids, Transport, udp_listener) -> %% dtls listener process must have the socket control - {ok, Connection:socket(Pids, Transport, Socket, Connection, undefined)}; + {ok, Connection:socket(Pids, Transport, Socket, undefined)}; socket_control(tls_connection = Connection, Socket, [Pid|_] = Pids, Transport, ListenTracker) -> case Transport:controlling_process(Socket, Pid) of ok -> - {ok, Connection:socket(Pids, Transport, Socket, Connection, ListenTracker)}; + {ok, Connection:socket(Pids, Transport, Socket, ListenTracker)}; {error, Reason} -> {error, Reason} end; socket_control(dtls_connection = Connection, {_, Socket}, [Pid|_] = Pids, Transport, ListenTracker) -> case Transport:controlling_process(Socket, Pid) of ok -> - {ok, Connection:socket(Pids, Transport, Socket, Connection, ListenTracker)}; + {ok, Connection:socket(Pids, Transport, Socket, ListenTracker)}; {error, Reason} -> {error, Reason} end. -start_or_recv_cancel_timer(infinity, _RecvFrom) -> - undefined; -start_or_recv_cancel_timer(Timeout, RecvFrom) -> - erlang:send_after(Timeout, self(), {cancel_start_or_recv, RecvFrom}). %%==================================================================== %% User events @@ -215,9 +211,9 @@ start_or_recv_cancel_timer(Timeout, RecvFrom) -> %%-------------------------------------------------------------------- send(Pid, Data) -> call(Pid, {application_data, - %% iolist_to_binary should really - %% be called iodata_to_binary() - erlang:iolist_to_binary(Data)}). + %% iolist_to_iovec should really + %% be called iodata_to_iovec() + erlang:iolist_to_iovec(Data)}). %%-------------------------------------------------------------------- -spec recv(pid(), integer(), timeout()) -> @@ -315,9 +311,6 @@ renegotiation(ConnectionPid) -> internal_renegotiation(ConnectionPid, #{current_write := WriteState}) -> gen_statem:cast(ConnectionPid, {internal_renegotiate, WriteState}). -get_sslsocket(ConnectionPid) -> - call(ConnectionPid, get_sslsocket). - dist_handshake_complete(ConnectionPid, DHandle) -> gen_statem:cast(ConnectionPid, {dist_handshake_complete, DHandle}). @@ -356,8 +349,8 @@ handle_normal_shutdown(Alert, _, #state{static_env = #static_env{role = Role, transport_cb = Transport, protocol_cb = Connection, tracker = Tracker}, - start_or_recv_from = StartFrom, - renegotiation = {false, first}} = State) -> + handshake_env = #handshake_env{renegotiation = {false, first}}, + start_or_recv_from = StartFrom} = State) -> Pids = Connection:pids(State), alert_user(Pids, Transport, Tracker,Socket, StartFrom, Alert, Role, Connection); @@ -366,8 +359,8 @@ handle_normal_shutdown(Alert, StateName, #state{static_env = #static_env{role = transport_cb = Transport, protocol_cb = Connection, tracker = Tracker}, - socket_options = Opts, - user_application = {_Mon, Pid}, + connection_env = #connection_env{user_application = {_Mon, Pid}}, + socket_options = Opts, start_or_recv_from = RecvFrom} = State) -> Pids = Connection:pids(State), alert_user(Pids, Transport, Tracker, Socket, StateName, Opts, Pid, RecvFrom, Alert, Role, Connection). @@ -380,9 +373,10 @@ handle_alert(#alert{level = ?FATAL} = Alert, StateName, tracker = Tracker, transport_cb = Transport, protocol_cb = Connection}, + connection_env = #connection_env{user_application = {_Mon, Pid}}, ssl_options = SslOpts, start_or_recv_from = From, - session = Session, user_application = {_Mon, Pid}, + session = Session, socket_options = Opts} = State) -> invalidate_session(Role, Host, Port, Session), log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), @@ -401,8 +395,8 @@ handle_alert(#alert{level = ?WARNING, description = ?CLOSE_NOTIFY} = Alert, handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, StateName, #state{static_env = #static_env{role = Role, protocol_cb = Connection}, - ssl_options = SslOpts, - renegotiation = {true, internal}} = State) -> + handshake_env = #handshake_env{renegotiation = {true, internal}}, + ssl_options = SslOpts} = State) -> log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), handle_normal_shutdown(Alert, StateName, State), @@ -411,26 +405,26 @@ handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, connection = StateName, #state{static_env = #static_env{role = Role, protocol_cb = Connection}, - ssl_options = SslOpts, - renegotiation = {true, From} + handshake_env = #handshake_env{renegotiation = {true, From}} = HsEnv, + ssl_options = SslOpts } = State0) -> log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), gen_statem:reply(From, {error, renegotiation_rejected}), State = Connection:reinit_handshake_data(State0), - Connection:next_event(connection, no_record, State#state{renegotiation = undefined}); + Connection:next_event(connection, no_record, State#state{handshake_env = HsEnv#handshake_env{renegotiation = undefined}}); handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, StateName, #state{static_env = #static_env{role = Role, protocol_cb = Connection}, - ssl_options = SslOpts, - renegotiation = {true, From} + handshake_env = #handshake_env{renegotiation = {true, From}} = HsEnv, + ssl_options = SslOpts } = State0) -> log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), gen_statem:reply(From, {error, renegotiation_rejected}), %% Go back to connection! - State = Connection:reinit(State0#state{renegotiation = undefined}), + State = Connection:reinit(State0#state{handshake_env = HsEnv#handshake_env{renegotiation = undefined}}), Connection:next_event(connection, no_record, State); %% Gracefully log and ignore all other warning alerts @@ -445,106 +439,113 @@ handle_alert(#alert{level = ?WARNING} = Alert, StateName, %%==================================================================== %% Data handling %%==================================================================== -passive_receive(State0 = #state{user_data_buffer = Buffer}, StateName, Connection) -> - case Buffer of - <<>> -> +passive_receive(State0 = #state{user_data_buffer = {_,BufferSize,_}}, StateName, Connection, StartTimerAction) -> + case BufferSize of + 0 -> {Record, State} = Connection:next_record(State0), - Connection:next_event(StateName, Record, State); + Connection:next_event(StateName, Record, State, StartTimerAction); _ -> case read_application_data(<<>>, State0) of {stop, _, _} = ShutdownError -> ShutdownError; {Record, State} -> - Connection:next_event(StateName, Record, State) + case State#state.start_or_recv_from of + undefined -> + %% Cancel recv timeout as data has been delivered + Connection:next_event(StateName, Record, State, + [{{timeout, recv}, infinity, timeout}]); + _ -> + Connection:next_event(StateName, Record, State, StartTimerAction) + end end end. read_application_data( Data, #state{ - user_data_buffer = Buffer0, - erl_dist_handle = DHandle} = State) -> + user_data_buffer = {Front0,BufferSize0,Rear0}, + connection_env = #connection_env{erl_dist_handle = DHandle}} = State) -> %% - Buffer = bincat(Buffer0, Data), + Front = Front0, + BufferSize = BufferSize0 + byte_size(Data), + Rear = [Data|Rear0], case DHandle of undefined -> - #state{ - socket_options = SocketOpts, - bytes_to_read = BytesToRead, - start_or_recv_from = RecvFrom, - timer = Timer} = State, - read_application_data( - Buffer, State, SocketOpts, RecvFrom, Timer, BytesToRead); + read_application_data(State, Front, BufferSize, Rear); _ -> - try read_application_dist_data(Buffer, State, DHandle) + try read_application_dist_data(DHandle, Front, BufferSize, Rear) of + Buffer -> + {no_record, State#state{user_data_buffer = Buffer}} catch error:_ -> {stop,disconnect, - State#state{ - user_data_buffer = Buffer, - bytes_to_read = undefined}} + State#state{user_data_buffer = {Front,BufferSize,Rear}}} end end. -read_application_dist_data(Buffer, State, DHandle) -> - case Buffer of - <<Size:32,Data:Size/binary>> -> - erlang:dist_ctrl_put_data(DHandle, Data), - {no_record, - State#state{ - user_data_buffer = <<>>, - bytes_to_read = undefined}}; - <<Size:32,Data:Size/binary,Rest/binary>> -> - erlang:dist_ctrl_put_data(DHandle, Data), - read_application_dist_data(Rest, State, DHandle); - _ -> - {no_record, - State#state{ - user_data_buffer = Buffer, - bytes_to_read = undefined}} - end. -read_application_data( - Buffer0, State, SocketOpts0, RecvFrom, Timer, BytesToRead) -> - %% - case get_data(SocketOpts0, BytesToRead, Buffer0) of - {ok, ClientData, Buffer} -> % Send data - #state{ - static_env = - #static_env{ - socket = Socket, - protocol_cb = Connection, - transport_cb = Transport, - tracker = Tracker}, - user_application = {_Mon, Pid}} = State, - SocketOpts = - deliver_app_data( - Connection:pids(State), - Transport, Socket, SocketOpts0, - ClientData, Pid, RecvFrom, Tracker, Connection), - cancel_timer(Timer), +read_application_data(#state{ + socket_options = SocketOpts, + bytes_to_read = BytesToRead, + start_or_recv_from = RecvFrom} = State, Front, BufferSize, Rear) -> + read_application_data(State, Front, BufferSize, Rear, SocketOpts, RecvFrom, BytesToRead). + +%% Pick binary from queue front, if empty wait for more data +read_application_data(State, [Bin|Front], BufferSize, Rear, SocketOpts, RecvFrom, BytesToRead) -> + read_application_data_bin(State, Front, BufferSize, Rear, SocketOpts, RecvFrom, BytesToRead, Bin); +read_application_data(State, [] = Front, BufferSize, [] = Rear, SocketOpts, RecvFrom, BytesToRead) -> + 0 = BufferSize, % Assert + {no_record, State#state{socket_options = SocketOpts, + bytes_to_read = BytesToRead, + start_or_recv_from = RecvFrom, + user_data_buffer = {Front,BufferSize,Rear}}}; +read_application_data(State, [], BufferSize, Rear, SocketOpts, RecvFrom, BytesToRead) -> + [Bin|Front] = lists:reverse(Rear), + read_application_data_bin(State, Front, BufferSize, [], SocketOpts, RecvFrom, BytesToRead, Bin). + +read_application_data_bin(State, Front, BufferSize, Rear, SocketOpts, RecvFrom, BytesToRead, <<>>) -> + %% Done with this binary - get next + read_application_data(State, Front, BufferSize, Rear, SocketOpts, RecvFrom, BytesToRead); +read_application_data_bin(State, Front0, BufferSize0, Rear0, SocketOpts0, RecvFrom, BytesToRead, Bin0) -> + %% Decode one packet from a binary + case get_data(SocketOpts0, BytesToRead, Bin0) of + {ok, Data, Bin} -> % Send data + BufferSize = BufferSize0 - (byte_size(Bin0) - byte_size(Bin)), + read_application_data_deliver( + State, [Bin|Front0], BufferSize, Rear0, SocketOpts0, RecvFrom, Data); + {more, undefined} -> + %% We need more data, do not know how much if - SocketOpts#socket_options.active =:= false; - Buffer =:= <<>> -> - %% Passive mode, wait for active once or recv - %% Active and empty, get more data - {no_record, - State#state{ - user_data_buffer = Buffer, - start_or_recv_from = undefined, - timer = undefined, - bytes_to_read = undefined, - socket_options = SocketOpts - }}; - true -> %% We have more data - read_application_data( - Buffer, State, SocketOpts, - undefined, undefined, undefined) + byte_size(Bin0) < BufferSize0 -> + %% We have more data in the buffer besides the first binary - concatenate all and retry + Bin = iolist_to_binary([Bin0,Front0|lists:reverse(Rear0)]), + read_application_data_bin( + State, [], BufferSize0, [], SocketOpts0, RecvFrom, BytesToRead, Bin); + true -> + %% All data is in the first binary, no use to retry - wait for more + {no_record, State#state{socket_options = SocketOpts0, + bytes_to_read = BytesToRead, + start_or_recv_from = RecvFrom, + user_data_buffer = {[Bin0|Front0],BufferSize0,Rear0}}} end; - {more, Buffer} -> % no reply, we need more data - {no_record, State#state{user_data_buffer = Buffer}}; - {passive, Buffer} -> - {no_record, State#state{user_data_buffer = Buffer}}; - {error,_Reason} -> %% Invalid packet in packet mode + {more, Size} when Size =< BufferSize0 -> + %% We have a packet in the buffer - collect it in a binary and decode + {Data,Front,Rear} = iovec_from_front(Size - byte_size(Bin0), Front0, Rear0, [Bin0]), + Bin = iolist_to_binary(Data), + read_application_data_bin( + State, Front, BufferSize0, Rear, SocketOpts0, RecvFrom, BytesToRead, Bin); + {more, _Size} -> + %% We do not have a packet in the buffer - wait for more + {no_record, State#state{socket_options = SocketOpts0, + bytes_to_read = BytesToRead, + start_or_recv_from = RecvFrom, + user_data_buffer = {[Bin0|Front0],BufferSize0,Rear0}}}; + passive -> + {no_record, State#state{socket_options = SocketOpts0, + bytes_to_read = BytesToRead, + start_or_recv_from = RecvFrom, + user_data_buffer = {[Bin0|Front0],BufferSize0,Rear0}}}; + {error,_Reason} -> + %% Invalid packet in packet mode #state{ static_env = #static_env{ @@ -552,13 +553,137 @@ read_application_data( protocol_cb = Connection, transport_cb = Transport, tracker = Tracker}, - user_application = {_Mon, Pid}} = State, + connection_env = + #connection_env{user_application = {_Mon, Pid}}} = State, + Buffer = iolist_to_binary([Bin0,Front0|lists:reverse(Rear0)]), deliver_packet_error( Connection:pids(State), Transport, Socket, SocketOpts0, - Buffer0, Pid, RecvFrom, Tracker, Connection), - {stop, {shutdown, normal}, State} + Buffer, Pid, RecvFrom, Tracker, Connection), + {stop, {shutdown, normal}, State#state{socket_options = SocketOpts0, + bytes_to_read = BytesToRead, + start_or_recv_from = RecvFrom, + user_data_buffer = {[Buffer],BufferSize0,[]}}} + end. + +read_application_data_deliver(State, Front, BufferSize, Rear, SocketOpts0, RecvFrom, Data) -> + #state{ + static_env = + #static_env{ + socket = Socket, + protocol_cb = Connection, + transport_cb = Transport, + tracker = Tracker}, + connection_env = + #connection_env{user_application = {_Mon, Pid}}} = State, + SocketOpts = + deliver_app_data( + Connection:pids(State), Transport, Socket, SocketOpts0, Data, Pid, RecvFrom, Tracker, Connection), + if + SocketOpts#socket_options.active =:= false -> + %% Passive mode, wait for active once or recv + {no_record, + State#state{ + user_data_buffer = {Front,BufferSize,Rear}, + start_or_recv_from = undefined, + bytes_to_read = undefined, + socket_options = SocketOpts + }}; + true -> %% Try to deliver more data + read_application_data(State, Front, BufferSize, Rear, SocketOpts, undefined, undefined) + end. + + +read_application_dist_data(DHandle, [Bin|Front], BufferSize, Rear) -> + read_application_dist_data(DHandle, Front, BufferSize, Rear, Bin); +read_application_dist_data(_DHandle, [] = Front, BufferSize, [] = Rear) -> + BufferSize = 0, + {Front,BufferSize,Rear}; +read_application_dist_data(DHandle, [], BufferSize, Rear) -> + [Bin|Front] = lists:reverse(Rear), + read_application_dist_data(DHandle, Front, BufferSize, [], Bin). +%% +read_application_dist_data(DHandle, Front0, BufferSize, Rear0, Bin0) -> + case Bin0 of + %% + %% START Optimization + %% It is cheaper to match out several packets in one match operation than to loop for each + <<SizeA:32, DataA:SizeA/binary, + SizeB:32, DataB:SizeB/binary, + SizeC:32, DataC:SizeC/binary, + SizeD:32, DataD:SizeD/binary, Rest/binary>> -> + %% We have 4 complete packets in the first binary + erlang:dist_ctrl_put_data(DHandle, DataA), + erlang:dist_ctrl_put_data(DHandle, DataB), + erlang:dist_ctrl_put_data(DHandle, DataC), + erlang:dist_ctrl_put_data(DHandle, DataD), + read_application_dist_data( + DHandle, Front0, BufferSize - (4*4+SizeA+SizeB+SizeC+SizeD), Rear0, Rest); + <<SizeA:32, DataA:SizeA/binary, + SizeB:32, DataB:SizeB/binary, + SizeC:32, DataC:SizeC/binary, Rest/binary>> -> + %% We have 3 complete packets in the first binary + erlang:dist_ctrl_put_data(DHandle, DataA), + erlang:dist_ctrl_put_data(DHandle, DataB), + erlang:dist_ctrl_put_data(DHandle, DataC), + read_application_dist_data( + DHandle, Front0, BufferSize - (3*4+SizeA+SizeB+SizeC), Rear0, Rest); + <<SizeA:32, DataA:SizeA/binary, + SizeB:32, DataB:SizeB/binary, Rest/binary>> -> + %% We have 2 complete packets in the first binary + erlang:dist_ctrl_put_data(DHandle, DataA), + erlang:dist_ctrl_put_data(DHandle, DataB), + read_application_dist_data( + DHandle, Front0, BufferSize - (2*4+SizeA+SizeB), Rear0, Rest); + %% END Optimization + %% + %% Basic one packet code path + <<Size:32, Data:Size/binary, Rest/binary>> -> + %% We have a complete packet in the first binary + erlang:dist_ctrl_put_data(DHandle, Data), + read_application_dist_data(DHandle, Front0, BufferSize - (4+Size), Rear0, Rest); + <<Size:32, FirstData/binary>> when 4+Size =< BufferSize -> + %% We have a complete packet in the buffer + %% - fetch the missing content from the buffer front + {Data,Front,Rear} = iovec_from_front(Size - byte_size(FirstData), Front0, Rear0, [FirstData]), + erlang:dist_ctrl_put_data(DHandle, Data), + read_application_dist_data(DHandle, Front, BufferSize - (4+Size), Rear); + <<Bin/binary>> -> + %% In OTP-21 the match context reuse optimization fails if we use Bin0 in recursion, so here we + %% match out the whole binary which will trick the optimization into keeping the match context + %% for the first binary contains complete packet code above + case Bin of + <<_Size:32, _InsufficientData/binary>> -> + %% We have a length field in the first binary but there is not enough data + %% in the buffer to form a complete packet - await more data + {[Bin|Front0],BufferSize,Rear0}; + <<IncompleteLengthField/binary>> when 4 < BufferSize -> + %% We do not have a length field in the first binary but the buffer + %% contains enough data to maybe form a packet + %% - fetch a tiny binary from the buffer front to complete the length field + {LengthField,Front,Rear} = + iovec_from_front(4 - byte_size(IncompleteLengthField), Front0, Rear0, [IncompleteLengthField]), + LengthBin = iolist_to_binary(LengthField), + read_application_dist_data(DHandle, Front, BufferSize, Rear, LengthBin); + <<IncompleteLengthField/binary>> -> + %% We do not have enough data in the buffer to even form a length field - await more data + {[IncompleteLengthField|Front0],BufferSize,Rear0} + end end. +iovec_from_front(Size, [], Rear, Acc) -> + iovec_from_front(Size, lists:reverse(Rear), [], Acc); +iovec_from_front(Size, [Bin|Front], Rear, Acc) -> + case Bin of + <<Last:Size/binary>> -> % Just enough + {lists:reverse(Acc, [Last]),Front,Rear}; + <<Last:Size/binary, Rest/binary>> -> % More than enough, split here + {lists:reverse(Acc, [Last]),[Rest|Front],Rear}; + <<_/binary>> -> % Not enough + BinSize = byte_size(Bin), + iovec_from_front(Size - BinSize, Front, Rear, [Bin|Acc]) + end. + + %%==================================================================== %% Help functions for tls|dtls_connection.erl %%==================================================================== @@ -571,8 +696,8 @@ handle_session(#server_hello{cipher_suite = CipherSuite, compression_method = Compression}, Version, NewId, ConnectionStates, ProtoExt, Protocol0, #state{session = #session{session_id = OldId}, - negotiated_version = ReqVersion, - negotiated_protocol = CurrentProtocol} = State0) -> + handshake_env = #handshake_env{negotiated_protocol = CurrentProtocol} = HsEnv, + connection_env = #connection_env{negotiated_version = ReqVersion} = CEnv} = State0) -> #{key_exchange := KeyAlgorithm} = ssl_cipher_format:suite_definition(CipherSuite), @@ -585,12 +710,12 @@ handle_session(#server_hello{cipher_suite = CipherSuite, {ProtoExt =:= npn, Protocol0} end, - State = State0#state{key_algorithm = KeyAlgorithm, - negotiated_version = Version, - connection_states = ConnectionStates, - premaster_secret = PremasterSecret, - expecting_next_protocol_negotiation = ExpectNPN, - negotiated_protocol = Protocol}, + State = State0#state{connection_states = ConnectionStates, + handshake_env = HsEnv#handshake_env{kex_algorithm = KeyAlgorithm, + premaster_secret = PremasterSecret, + expecting_next_protocol_negotiation = ExpectNPN, + negotiated_protocol = Protocol}, + connection_env = CEnv#connection_env{negotiated_version = Version}}, case ssl_session:is_new(OldId, NewId) of true -> @@ -604,10 +729,9 @@ handle_session(#server_hello{cipher_suite = CipherSuite, %%-------------------------------------------------------------------- -spec ssl_config(#ssl_options{}, client | server, #state{}) -> #state{}. %%-------------------------------------------------------------------- -ssl_config(Opts, Role, State) -> - ssl_config(Opts, Role, State, new). - -ssl_config(Opts, Role, #state{static_env = InitStatEnv0} =State0, Type) -> +ssl_config(Opts, Role, #state{static_env = InitStatEnv0, + handshake_env = HsEnv, + connection_env = CEnv} = State0) -> {ok, #{cert_db_ref := Ref, cert_db_handle := CertDbHandle, fileref_db_handle := FileRefHandle, @@ -619,27 +743,19 @@ ssl_config(Opts, Role, #state{static_env = InitStatEnv0} =State0, Type) -> ssl_config:init(Opts, Role), TimeStamp = erlang:monotonic_time(), Session = State0#state.session, - - State = State0#state{session = Session#session{own_certificate = OwnCert, - time_stamp = TimeStamp}, - static_env = InitStatEnv0#static_env{ - file_ref_db = FileRefHandle, - cert_db_ref = Ref, - cert_db = CertDbHandle, - crl_db = CRLDbHandle, - session_cache = CacheHandle - }, - private_key = Key, - diffie_hellman_params = DHParams, - ssl_options = Opts}, - case Type of - new -> - Handshake = ssl_handshake:init_handshake_history(), - State#state{tls_handshake_history = Handshake}; - continue -> - State - end. - + + State0#state{session = Session#session{own_certificate = OwnCert, + time_stamp = TimeStamp}, + static_env = InitStatEnv0#static_env{ + file_ref_db = FileRefHandle, + cert_db_ref = Ref, + cert_db = CertDbHandle, + crl_db = CRLDbHandle, + session_cache = CacheHandle + }, + handshake_env = HsEnv#handshake_env{diffie_hellman_params = DHParams}, + connection_env = CEnv#connection_env{private_key = Key}, + ssl_options = Opts}. %%==================================================================== %% gen_statem general state functions with connection cb argument @@ -652,8 +768,8 @@ ssl_config(Opts, Role, #state{static_env = InitStatEnv0} =State0, Type) -> %%-------------------------------------------------------------------- init({call, From}, {start, Timeout}, State0, Connection) -> - Timer = start_or_recv_cancel_timer(Timeout, From), - Connection:next_event(hello, no_record, State0#state{start_or_recv_from = From, timer = Timer}); + Connection:next_event(hello, no_record, State0#state{start_or_recv_from = From}, + [{{timeout, handshake}, Timeout, close}]); init({call, From}, {start, {Opts, EmOpts}, Timeout}, #state{static_env = #static_env{role = Role}, ssl_options = OrigSSLOptions, @@ -700,21 +816,18 @@ hello(info, Msg, State, _) -> hello(Type, Msg, State, Connection) -> handle_common_event(Type, Msg, ?FUNCTION_NAME, State, Connection). -user_hello({call, From}, cancel, #state{negotiated_version = Version} = State, _) -> +user_hello({call, From}, cancel, #state{connection_env = #connection_env{negotiated_version = Version}} = State, _) -> gen_statem:reply(From, ok), handle_own_alert(?ALERT_REC(?FATAL, ?USER_CANCELED, user_canceled), Version, ?FUNCTION_NAME, State); user_hello({call, From}, {handshake_continue, NewOptions, Timeout}, - #state{hello = Hello, - static_env = #static_env{role = Role}, - start_or_recv_from = RecvFrom, + #state{static_env = #static_env{role = Role}, + handshake_env = #handshake_env{hello = Hello}, ssl_options = Options0} = State0, _Connection) -> - Timer = start_or_recv_cancel_timer(Timeout, RecvFrom), Options = ssl:handle_options(NewOptions, Options0#ssl_options{handshake = full}), - State = ssl_config(Options, Role, State0, continue), - {next_state, hello, State#state{start_or_recv_from = From, - timer = Timer}, - [{next_event, internal, Hello}]}; + State = ssl_config(Options, Role, State0), + {next_state, hello, State#state{start_or_recv_from = From}, + [{next_event, internal, Hello}, {{timeout, handshake}, Timeout, close}]}; user_hello(_, _, _, _) -> {keep_state_and_data, [postpone]}. @@ -728,41 +841,41 @@ abbreviated({call, From}, Msg, State, Connection) -> handle_call(Msg, From, ?FUNCTION_NAME, State, Connection); abbreviated(internal, #finished{verify_data = Data} = Finished, #state{static_env = #static_env{role = server}, - negotiated_version = Version, - expecting_finished = true, - tls_handshake_history = Handshake, + handshake_env = #handshake_env{tls_handshake_history = Hist, + expecting_finished = true} = HsEnv, + connection_env = #connection_env{negotiated_version = Version}, session = #session{master_secret = MasterSecret}, connection_states = ConnectionStates0} = State0, Connection) -> case ssl_handshake:verify_connection(ssl:tls_version(Version), Finished, client, get_current_prf(ConnectionStates0, write), - MasterSecret, Handshake) of + MasterSecret, Hist) of verified -> ConnectionStates = ssl_record:set_client_verify_data(current_both, Data, ConnectionStates0), {Record, State} = prepare_connection(State0#state{connection_states = ConnectionStates, - expecting_finished = false}, Connection), - Connection:next_event(connection, Record, State); + handshake_env = HsEnv#handshake_env{expecting_finished = false}}, Connection), + Connection:next_event(connection, Record, State, [{{timeout, handshake}, infinity, close}]); #alert{} = Alert -> handle_own_alert(Alert, Version, ?FUNCTION_NAME, State0) end; abbreviated(internal, #finished{verify_data = Data} = Finished, #state{static_env = #static_env{role = client}, - tls_handshake_history = Handshake0, + handshake_env = #handshake_env{tls_handshake_history = Hist0}, + connection_env = #connection_env{negotiated_version = Version}, session = #session{master_secret = MasterSecret}, - negotiated_version = Version, connection_states = ConnectionStates0} = State0, Connection) -> case ssl_handshake:verify_connection(ssl:tls_version(Version), Finished, server, get_pending_prf(ConnectionStates0, write), - MasterSecret, Handshake0) of + MasterSecret, Hist0) of verified -> ConnectionStates1 = ssl_record:set_server_verify_data(current_read, Data, ConnectionStates0), - {State1, Actions} = + {#state{handshake_env = HsEnv} = State1, Actions} = finalize_handshake(State0#state{connection_states = ConnectionStates1}, ?FUNCTION_NAME, Connection), - {Record, State} = prepare_connection(State1#state{expecting_finished = false}, Connection), - Connection:next_event(connection, Record, State, Actions); + {Record, State} = prepare_connection(State1#state{handshake_env = HsEnv#handshake_env{expecting_finished = false}}, Connection), + Connection:next_event(connection, Record, State, [{{timeout, handshake}, infinity, close} | Actions]); #alert{} = Alert -> handle_own_alert(Alert, Version, ?FUNCTION_NAME, State0) end; @@ -770,19 +883,20 @@ abbreviated(internal, #finished{verify_data = Data} = Finished, %% & before finished message and it is not allowed during renegotiation abbreviated(internal, #next_protocol{selected_protocol = SelectedProtocol}, #state{static_env = #static_env{role = server}, - expecting_next_protocol_negotiation = true} = State, + handshake_env = #handshake_env{expecting_next_protocol_negotiation = true} = HsEnv} = State, Connection) -> Connection:next_event(?FUNCTION_NAME, no_record, - State#state{negotiated_protocol = SelectedProtocol, - expecting_next_protocol_negotiation = false}); + State#state{handshake_env = HsEnv#handshake_env{negotiated_protocol = SelectedProtocol, + expecting_next_protocol_negotiation = false}}); abbreviated(internal, #change_cipher_spec{type = <<1>>}, - #state{connection_states = ConnectionStates0} = State, Connection) -> + #state{connection_states = ConnectionStates0, + handshake_env = HsEnv} = State, Connection) -> ConnectionStates1 = ssl_record:activate_pending_connection_state(ConnectionStates0, read, Connection), Connection:next_event(?FUNCTION_NAME, no_record, State#state{connection_states = ConnectionStates1, - expecting_finished = true}); + handshake_env = HsEnv#handshake_env{expecting_finished = true}}); abbreviated(info, Msg, State, _) -> handle_info(Msg, ?FUNCTION_NAME, State); abbreviated(Type, Msg, State, Connection) -> @@ -801,7 +915,7 @@ certify(info, Msg, State, _) -> handle_info(Msg, ?FUNCTION_NAME, State); certify(internal, #certificate{asn1_certificates = []}, #state{static_env = #static_env{role = server}, - negotiated_version = Version, + connection_env = #connection_env{negotiated_version = Version}, ssl_options = #ssl_options{verify = verify_peer, fail_if_no_peer_cert = true}} = State, _) -> @@ -815,7 +929,7 @@ certify(internal, #certificate{asn1_certificates = []}, Connection:next_event(?FUNCTION_NAME, no_record, State0#state{client_certificate_requested = false}); certify(internal, #certificate{}, #state{static_env = #static_env{role = server}, - negotiated_version = Version, + connection_env = #connection_env{negotiated_version = Version}, ssl_options = #ssl_options{verify = verify_none}} = State, _) -> Alert = ?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE, unrequested_certificate), @@ -827,7 +941,7 @@ certify(internal, #certificate{} = Cert, cert_db = CertDbHandle, cert_db_ref = CertDbRef, crl_db = CRLDbInfo}, - negotiated_version = Version, + connection_env = #connection_env{negotiated_version = Version}, ssl_options = Opts} = State, Connection) -> case ssl_handshake:certify(Cert, CertDbHandle, CertDbRef, Opts, CRLDbInfo, Role, Host) of @@ -839,34 +953,42 @@ certify(internal, #certificate{} = Cert, end; certify(internal, #server_key_exchange{exchange_keys = Keys}, #state{static_env = #static_env{role = client}, - negotiated_version = Version, - key_algorithm = Alg, - public_key_info = PubKeyInfo, + handshake_env = #handshake_env{kex_algorithm = KexAlg, + public_key_info = PubKeyInfo} = HsEnv, + connection_env = #connection_env{negotiated_version = Version}, session = Session, connection_states = ConnectionStates} = State, Connection) - when Alg == dhe_dss; Alg == dhe_rsa; - Alg == ecdhe_rsa; Alg == ecdhe_ecdsa; - Alg == dh_anon; Alg == ecdh_anon; - Alg == psk; Alg == dhe_psk; Alg == ecdhe_psk; Alg == rsa_psk; - Alg == srp_dss; Alg == srp_rsa; Alg == srp_anon -> - - Params = ssl_handshake:decode_server_key(Keys, Alg, ssl:tls_version(Version)), + when KexAlg == dhe_dss; + KexAlg == dhe_rsa; + KexAlg == ecdhe_rsa; + KexAlg == ecdhe_ecdsa; + KexAlg == dh_anon; + KexAlg == ecdh_anon; + KexAlg == psk; + KexAlg == dhe_psk; + KexAlg == ecdhe_psk; + KexAlg == rsa_psk; + KexAlg == srp_dss; + KexAlg == srp_rsa; + KexAlg == srp_anon -> + + Params = ssl_handshake:decode_server_key(Keys, KexAlg, ssl:tls_version(Version)), %% Use negotiated value if TLS-1.2 otherwhise return default - HashSign = negotiated_hashsign(Params#server_key_params.hashsign, Alg, PubKeyInfo, ssl:tls_version(Version)), + HashSign = negotiated_hashsign(Params#server_key_params.hashsign, KexAlg, PubKeyInfo, ssl:tls_version(Version)), - case is_anonymous(Alg) of + case is_anonymous(KexAlg) of true -> calculate_secret(Params#server_key_params.params, - State#state{hashsign_algorithm = HashSign}, Connection); + State#state{handshake_env = HsEnv#handshake_env{hashsign_algorithm = HashSign}}, Connection); false -> case ssl_handshake:verify_server_key(Params, HashSign, ConnectionStates, ssl:tls_version(Version), PubKeyInfo) of true -> calculate_secret(Params#server_key_params.params, - State#state{hashsign_algorithm = HashSign, - session = session_handle_params(Params#server_key_params.params, Session)}, - Connection); + State#state{handshake_env = HsEnv#handshake_env{hashsign_algorithm = HashSign}, + session = session_handle_params(Params#server_key_params.params, Session)}, + Connection); false -> handle_own_alert(?ALERT_REC(?FATAL, ?DECRYPT_ERROR), Version, ?FUNCTION_NAME, State) @@ -874,11 +996,17 @@ certify(internal, #server_key_exchange{exchange_keys = Keys}, end; certify(internal, #certificate_request{}, #state{static_env = #static_env{role = client}, - negotiated_version = Version, - key_algorithm = Alg} = State, _) - when Alg == dh_anon; Alg == ecdh_anon; - Alg == psk; Alg == dhe_psk; Alg == ecdhe_psk; Alg == rsa_psk; - Alg == srp_dss; Alg == srp_rsa; Alg == srp_anon -> + handshake_env = #handshake_env{kex_algorithm = KexAlg}, + connection_env = #connection_env{negotiated_version = Version}} = State, _) + when KexAlg == dh_anon; + KexAlg == ecdh_anon; + KexAlg == psk; + KexAlg == dhe_psk; + KexAlg == ecdhe_psk; + KexAlg == rsa_psk; + KexAlg == srp_dss; + KexAlg == srp_rsa; + KexAlg == srp_anon -> handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE), Version, ?FUNCTION_NAME, State); certify(internal, #certificate_request{}, @@ -890,62 +1018,65 @@ certify(internal, #certificate_request{}, Connection:next_event(?FUNCTION_NAME, no_record, State#state{client_certificate_requested = true}); certify(internal, #certificate_request{} = CertRequest, #state{static_env = #static_env{role = client}, + handshake_env = HsEnv, + connection_env = #connection_env{negotiated_version = Version}, session = #session{own_certificate = Cert}, - ssl_options = #ssl_options{signature_algs = SupportedHashSigns}, - negotiated_version = Version} = State, Connection) -> + ssl_options = #ssl_options{signature_algs = SupportedHashSigns}} = State, Connection) -> case ssl_handshake:select_hashsign(CertRequest, Cert, SupportedHashSigns, ssl:tls_version(Version)) of #alert {} = Alert -> handle_own_alert(Alert, Version, ?FUNCTION_NAME, State); NegotiatedHashSign -> Connection:next_event(?FUNCTION_NAME, no_record, State#state{client_certificate_requested = true, - cert_hashsign_algorithm = NegotiatedHashSign}) + handshake_env = HsEnv#handshake_env{cert_hashsign_algorithm = NegotiatedHashSign}}) end; %% PSK and RSA_PSK might bypass the Server-Key-Exchange certify(internal, #server_hello_done{}, #state{static_env = #static_env{role = client}, session = #session{master_secret = undefined}, - negotiated_version = Version, - psk_identity = PSKIdentity, - ssl_options = #ssl_options{user_lookup_fun = PSKLookup}, - premaster_secret = undefined, - key_algorithm = Alg} = State0, Connection) - when Alg == psk -> - case ssl_handshake:premaster_secret({Alg, PSKIdentity}, PSKLookup) of + connection_env = #connection_env{negotiated_version = Version}, + handshake_env = #handshake_env{kex_algorithm = KexAlg, + premaster_secret = undefined, + server_psk_identity = PSKIdentity} = HsEnv, + ssl_options = #ssl_options{user_lookup_fun = PSKLookup}} = State0, Connection) + when KexAlg == psk -> + case ssl_handshake:premaster_secret({KexAlg, PSKIdentity}, PSKLookup) of #alert{} = Alert -> handle_own_alert(Alert, Version, ?FUNCTION_NAME, State0); PremasterSecret -> State = master_secret(PremasterSecret, - State0#state{premaster_secret = PremasterSecret}), - client_certify_and_key_exchange(State, Connection) + State0#state{handshake_env = + HsEnv#handshake_env{premaster_secret = PremasterSecret}}), + client_certify_and_key_exchange(State, Connection) end; certify(internal, #server_hello_done{}, #state{static_env = #static_env{role = client}, + connection_env = #connection_env{negotiated_version = {Major, Minor}} = Version, + handshake_env = #handshake_env{kex_algorithm = KexAlg, + premaster_secret = undefined, + server_psk_identity = PSKIdentity} = HsEnv, session = #session{master_secret = undefined}, - ssl_options = #ssl_options{user_lookup_fun = PSKLookup}, - negotiated_version = {Major, Minor} = Version, - psk_identity = PSKIdentity, - premaster_secret = undefined, - key_algorithm = Alg} = State0, Connection) - when Alg == rsa_psk -> + ssl_options = #ssl_options{user_lookup_fun = PSKLookup}} = State0, Connection) + when KexAlg == rsa_psk -> Rand = ssl_cipher:random_bytes(?NUM_OF_PREMASTERSECRET_BYTES-2), RSAPremasterSecret = <<?BYTE(Major), ?BYTE(Minor), Rand/binary>>, - case ssl_handshake:premaster_secret({Alg, PSKIdentity}, PSKLookup, + case ssl_handshake:premaster_secret({KexAlg, PSKIdentity}, PSKLookup, RSAPremasterSecret) of #alert{} = Alert -> handle_own_alert(Alert, Version, ?FUNCTION_NAME, State0); PremasterSecret -> State = master_secret(PremasterSecret, - State0#state{premaster_secret = RSAPremasterSecret}), + State0#state{handshake_env = + HsEnv#handshake_env{premaster_secret = RSAPremasterSecret}}), client_certify_and_key_exchange(State, Connection) end; %% Master secret was determined with help of server-key exchange msg certify(internal, #server_hello_done{}, #state{static_env = #static_env{role = client}, - session = #session{master_secret = MasterSecret} = Session, - connection_states = ConnectionStates0, - negotiated_version = Version, - premaster_secret = undefined} = State0, Connection) -> + connection_env = #connection_env{negotiated_version = Version}, + handshake_env = #handshake_env{premaster_secret = undefined}, + session = #session{master_secret = MasterSecret} = Session, + connection_states = ConnectionStates0} = State0, Connection) -> case ssl_handshake:master_secret(ssl:tls_version(Version), Session, ConnectionStates0, client) of {MasterSecret, ConnectionStates} -> @@ -957,10 +1088,10 @@ certify(internal, #server_hello_done{}, %% Master secret is calculated from premaster_secret certify(internal, #server_hello_done{}, #state{static_env = #static_env{role = client}, + connection_env = #connection_env{negotiated_version = Version}, + handshake_env = #handshake_env{premaster_secret = PremasterSecret}, session = Session0, - connection_states = ConnectionStates0, - negotiated_version = Version, - premaster_secret = PremasterSecret} = State0, Connection) -> + connection_states = ConnectionStates0} = State0, Connection) -> case ssl_handshake:master_secret(ssl:tls_version(Version), PremasterSecret, ConnectionStates0, client) of {MasterSecret, ConnectionStates} -> @@ -979,7 +1110,8 @@ certify(internal = Type, #client_key_exchange{} = Msg, %% We expect a certificate here handle_common_event(Type, Msg, ?FUNCTION_NAME, State, Connection); certify(internal, #client_key_exchange{exchange_keys = Keys}, - State = #state{key_algorithm = KeyAlg, negotiated_version = Version}, Connection) -> + State = #state{handshake_env = #handshake_env{kex_algorithm = KeyAlg}, + connection_env = #connection_env{negotiated_version = Version}}, Connection) -> try certify_client_key_exchange(ssl_handshake:decode_client_key(Keys, KeyAlg, ssl:tls_version(Version)), State, Connection) @@ -1003,50 +1135,51 @@ cipher(info, Msg, State, _) -> cipher(internal, #certificate_verify{signature = Signature, hashsign_algorithm = CertHashSign}, #state{static_env = #static_env{role = server}, - key_algorithm = KexAlg, - public_key_info = PublicKeyInfo, - negotiated_version = Version, - session = #session{master_secret = MasterSecret}, - tls_handshake_history = Handshake + handshake_env = #handshake_env{tls_handshake_history = Hist, + kex_algorithm = KexAlg, + public_key_info = PubKeyInfo} = HsEnv, + connection_env = #connection_env{negotiated_version = Version}, + session = #session{master_secret = MasterSecret} } = State, Connection) -> TLSVersion = ssl:tls_version(Version), %% Use negotiated value if TLS-1.2 otherwhise return default - HashSign = negotiated_hashsign(CertHashSign, KexAlg, PublicKeyInfo, TLSVersion), - case ssl_handshake:certificate_verify(Signature, PublicKeyInfo, - TLSVersion, HashSign, MasterSecret, Handshake) of + HashSign = negotiated_hashsign(CertHashSign, KexAlg, PubKeyInfo, TLSVersion), + case ssl_handshake:certificate_verify(Signature, PubKeyInfo, + TLSVersion, HashSign, MasterSecret, Hist) of valid -> Connection:next_event(?FUNCTION_NAME, no_record, - State#state{cert_hashsign_algorithm = HashSign}); + State#state{handshake_env = HsEnv#handshake_env{cert_hashsign_algorithm = HashSign}}); #alert{} = Alert -> handle_own_alert(Alert, Version, ?FUNCTION_NAME, State) end; %% client must send a next protocol message if we are expecting it cipher(internal, #finished{}, #state{static_env = #static_env{role = server}, - expecting_next_protocol_negotiation = true, - negotiated_protocol = undefined, negotiated_version = Version} = State0, + handshake_env = #handshake_env{expecting_next_protocol_negotiation = true, + negotiated_protocol = undefined}, + connection_env = #connection_env{negotiated_version = Version}} = State0, _Connection) -> handle_own_alert(?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE), Version, ?FUNCTION_NAME, State0); cipher(internal, #finished{verify_data = Data} = Finished, #state{static_env = #static_env{role = Role, host = Host, port = Port}, - negotiated_version = Version, - expecting_finished = true, + handshake_env = #handshake_env{tls_handshake_history = Hist, + expecting_finished = true} = HsEnv, + connection_env = #connection_env{negotiated_version = Version}, session = #session{master_secret = MasterSecret} = Session0, ssl_options = SslOpts, - connection_states = ConnectionStates0, - tls_handshake_history = Handshake0} = State, Connection) -> + connection_states = ConnectionStates0} = State, Connection) -> case ssl_handshake:verify_connection(ssl:tls_version(Version), Finished, opposite_role(Role), get_current_prf(ConnectionStates0, read), - MasterSecret, Handshake0) of + MasterSecret, Hist) of verified -> - Session = register_session(Role, host_id(Role, Host, SslOpts), Port, Session0), + Session = handle_session(Role, SslOpts, Host, Port, Session0), cipher_role(Role, Data, Session, - State#state{expecting_finished = false}, Connection); + State#state{handshake_env = HsEnv#handshake_env{expecting_finished = false}}, Connection); #alert{} = Alert -> handle_own_alert(Alert, Version, ?FUNCTION_NAME, State) end; @@ -1054,19 +1187,19 @@ cipher(internal, #finished{verify_data = Data} = Finished, %% & before finished message and it is not allowed during renegotiation cipher(internal, #next_protocol{selected_protocol = SelectedProtocol}, #state{static_env = #static_env{role = server}, - expecting_next_protocol_negotiation = true, - expecting_finished = true} = State0, Connection) -> + handshake_env = #handshake_env{expecting_finished = true, + expecting_next_protocol_negotiation = true} = HsEnv} = State0, Connection) -> {Record, State} = - Connection:next_record(State0#state{negotiated_protocol = SelectedProtocol}), + Connection:next_record(State0), Connection:next_event(?FUNCTION_NAME, Record, - State#state{expecting_next_protocol_negotiation = false}); -cipher(internal, #change_cipher_spec{type = <<1>>}, #state{connection_states = ConnectionStates0} = + State#state{handshake_env = HsEnv#handshake_env{negotiated_protocol = SelectedProtocol, + expecting_next_protocol_negotiation = false}}); +cipher(internal, #change_cipher_spec{type = <<1>>}, #state{handshake_env = HsEnv, connection_states = ConnectionStates0} = State, Connection) -> ConnectionStates = ssl_record:activate_pending_connection_state(ConnectionStates0, read, Connection), - Connection:next_event(?FUNCTION_NAME, no_record, State#state{connection_states = - ConnectionStates, - expecting_finished = true}); + Connection:next_event(?FUNCTION_NAME, no_record, State#state{handshake_env = HsEnv#handshake_env{expecting_finished = true}, + connection_states = ConnectionStates}); cipher(Type, Msg, State, Connection) -> handle_common_event(Type, Msg, ?FUNCTION_NAME, State, Connection). @@ -1079,14 +1212,14 @@ connection({call, RecvFrom}, {recv, N, Timeout}, #state{static_env = #static_env{protocol_cb = Connection}, socket_options = #socket_options{active = false}} = State0, Connection) -> - Timer = start_or_recv_cancel_timer(Timeout, RecvFrom), passive_receive(State0#state{bytes_to_read = N, - start_or_recv_from = RecvFrom, - timer = Timer}, ?FUNCTION_NAME, Connection); + start_or_recv_from = RecvFrom}, ?FUNCTION_NAME, Connection, + [{{timeout, recv}, Timeout, timeout}]); -connection({call, From}, renegotiate, #state{static_env = #static_env{protocol_cb = Connection}} = State, +connection({call, From}, renegotiate, #state{static_env = #static_env{protocol_cb = Connection}, + handshake_env = HsEnv} = State, Connection) -> - Connection:renegotiate(State#state{renegotiation = {true, From}}, []); + Connection:renegotiate(State#state{handshake_env = HsEnv#handshake_env{renegotiation = {true, From}}}, []); connection({call, From}, peer_certificate, #state{session = #session{peer_certificate = Cert}} = State, _) -> hibernate_after(?FUNCTION_NAME, State, [{reply, From, {ok, Cert}}]); @@ -1097,34 +1230,36 @@ connection({call, From}, {connection_information, false}, State, _) -> Info = connection_info(State), hibernate_after(?FUNCTION_NAME, State, [{reply, From, {ok, Info}}]); connection({call, From}, negotiated_protocol, - #state{negotiated_protocol = undefined} = State, _) -> + #state{handshake_env = #handshake_env{negotiated_protocol = undefined}} = State, _) -> hibernate_after(?FUNCTION_NAME, State, [{reply, From, {error, protocol_not_negotiated}}]); connection({call, From}, negotiated_protocol, - #state{negotiated_protocol = SelectedProtocol} = State, _) -> + #state{handshake_env = #handshake_env{negotiated_protocol = SelectedProtocol}} = State, _) -> hibernate_after(?FUNCTION_NAME, State, [{reply, From, {ok, SelectedProtocol}}]); connection({call, From}, Msg, State, Connection) -> handle_call(Msg, From, ?FUNCTION_NAME, State, Connection); connection(cast, {internal_renegotiate, WriteState}, #state{static_env = #static_env{protocol_cb = Connection}, + handshake_env = HsEnv, connection_states = ConnectionStates} = State, Connection) -> - Connection:renegotiate(State#state{renegotiation = {true, internal}, + Connection:renegotiate(State#state{handshake_env = HsEnv#handshake_env{renegotiation = {true, internal}}, connection_states = ConnectionStates#{current_write => WriteState}}, []); connection(cast, {dist_handshake_complete, DHandle}, #state{ssl_options = #ssl_options{erl_dist = true}, + connection_env = CEnv, socket_options = SockOpts} = State0, Connection) -> process_flag(priority, normal), State1 = State0#state{ socket_options = SockOpts#socket_options{active = true}, - erl_dist_handle = DHandle, + connection_env = CEnv#connection_env{erl_dist_handle = DHandle}, bytes_to_read = undefined}, {Record, State} = read_application_data(<<>>, State1), Connection:next_event(connection, Record, State); connection(info, Msg, State, _) -> handle_info(Msg, ?FUNCTION_NAME, State); -connection(internal, {recv, _}, State, Connection) -> - passive_receive(State, ?FUNCTION_NAME, Connection); +connection(internal, {recv, Timeout}, State, Connection) -> + passive_receive(State, ?FUNCTION_NAME, Connection, [{{timeout, recv}, Timeout, timeout}]); connection(Type, Msg, State, Connection) -> handle_common_event(Type, Msg, ?FUNCTION_NAME, State, Connection). @@ -1141,34 +1276,45 @@ downgrade(Type, Event, State, Connection) -> %% common or unexpected events for the state. %%-------------------------------------------------------------------- handle_common_event(internal, {handshake, {#hello_request{} = Handshake, _}}, connection = StateName, - #state{static_env = #static_env{role = client}} = State, _) -> + #state{static_env = #static_env{role = client}, + handshake_env = HsEnv} = State, _) -> %% Should not be included in handshake history - {next_state, StateName, State#state{renegotiation = {true, peer}}, [{next_event, internal, Handshake}]}; + {next_state, StateName, State#state{handshake_env = HsEnv#handshake_env{renegotiation = {true, peer}}}, + [{next_event, internal, Handshake}]}; handle_common_event(internal, {handshake, {#hello_request{}, _}}, StateName, #state{static_env = #static_env{role = client}}, _) when StateName =/= connection -> keep_state_and_data; handle_common_event(internal, {handshake, {Handshake, Raw}}, StateName, - #state{tls_handshake_history = Hs0} = State0, + #state{handshake_env = #handshake_env{tls_handshake_history = Hist0}} = State0, Connection) -> PossibleSNI = Connection:select_sni_extension(Handshake), %% This function handles client SNI hello extension when Handshake is %% a client_hello, which needs to be determined by the connection callback. %% In other cases this is a noop - State = handle_sni_extension(PossibleSNI, State0), - HsHist = ssl_handshake:update_handshake_history(Hs0, iolist_to_binary(Raw)), - {next_state, StateName, State#state{tls_handshake_history = HsHist}, + State = #state{handshake_env = HsEnv} = handle_sni_extension(PossibleSNI, State0), + + Hist = ssl_handshake:update_handshake_history(Hist0, Raw), + {next_state, StateName, State#state{handshake_env = HsEnv#handshake_env{tls_handshake_history = Hist}}, [{next_event, internal, Handshake}]}; handle_common_event(internal, {protocol_record, TLSorDTLSRecord}, StateName, State, Connection) -> Connection:handle_protocol_record(TLSorDTLSRecord, StateName, State); handle_common_event(timeout, hibernate, _, _, _) -> {keep_state_and_data, [hibernate]}; handle_common_event(internal, #change_cipher_spec{type = <<1>>}, StateName, - #state{negotiated_version = Version} = State, _) -> + #state{connection_env = #connection_env{negotiated_version = Version}} = State, _) -> handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE), Version, StateName, State); -handle_common_event(_Type, Msg, StateName, #state{negotiated_version = Version} = State, +handle_common_event({timeout, handshake}, close, _StateName, #state{start_or_recv_from = StartFrom} = State, _) -> + {stop_and_reply, + {shutdown, user_timeout}, + {reply, StartFrom, {error, timeout}}, State#state{start_or_recv_from = undefined}}; +handle_common_event({timeout, recv}, timeout, StateName, #state{start_or_recv_from = RecvFrom} = State, _) -> + {next_state, StateName, State#state{start_or_recv_from = undefined, + bytes_to_read = undefined}, [{reply, RecvFrom, {error, timeout}}]}; +handle_common_event(_Type, Msg, StateName, #state{connection_env = + #connection_env{negotiated_version = Version}} = State, _) -> Alert = ?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE, {unexpected_msg, Msg}), handle_own_alert(Alert, Version, StateName, State). @@ -1176,24 +1322,28 @@ handle_common_event(_Type, Msg, StateName, #state{negotiated_version = Version} handle_call({application_data, _Data}, _, _, _, _) -> %% In renegotiation priorities handshake, send data when handshake is finished {keep_state_and_data, [postpone]}; -handle_call({close, _} = Close, From, StateName, State, _Connection) -> +handle_call({close, _} = Close, From, StateName, #state{connection_env = CEnv} = State, _Connection) -> %% Run terminate before returning so that the reuseaddr %% inet-option works properly Result = terminate(Close, StateName, State), {stop_and_reply, {shutdown, normal}, - {reply, From, Result}, State#state{terminated = true}}; + {reply, From, Result}, State#state{connection_env = CEnv#connection_env{terminated = true}}}; handle_call({shutdown, read_write = How}, From, StateName, #state{static_env = #static_env{transport_cb = Transport, - socket = Socket}} = State, _) -> + socket = Socket}, + connection_env = CEnv} = State, _) -> try send_alert(?ALERT_REC(?WARNING, ?CLOSE_NOTIFY), StateName, State) of _ -> case Transport:shutdown(Socket, How) of ok -> - {next_state, StateName, State#state{terminated = true}, [{reply, From, ok}]}; + {next_state, StateName, State#state{connection_env = + CEnv#connection_env{terminated = true}}, + [{reply, From, ok}]}; Error -> - {stop, StateName, State#state{terminated = true}, [{reply, From, Error}]} + {stop_and_reply, {shutdown, normal}, {reply, From, Error}, + State#state{connection_env = CEnv#connection_env{terminated = true}}} end catch throw:Return -> @@ -1206,7 +1356,7 @@ handle_call({shutdown, How0}, From, StateName, ok -> {next_state, StateName, State, [{reply, From, ok}]}; Error -> - {stop, StateName, State, [{reply, From, Error}]} + {stop_and_reply, {shutdown, normal}, {reply, From, Error}, State} end; handle_call({recv, _N, _Timeout}, From, _, #state{socket_options = @@ -1215,15 +1365,13 @@ handle_call({recv, _N, _Timeout}, From, _, handle_call({recv, N, Timeout}, RecvFrom, StateName, State, _) -> %% Doing renegotiate wait with handling request until renegotiate is %% finished. - Timer = start_or_recv_cancel_timer(Timeout, RecvFrom), - {next_state, StateName, State#state{bytes_to_read = N, start_or_recv_from = RecvFrom, - timer = Timer}, - [{next_event, internal, {recv, RecvFrom}}]}; + {next_state, StateName, State#state{bytes_to_read = N, start_or_recv_from = RecvFrom}, + [{next_event, internal, {recv, RecvFrom}} , {{timeout, recv}, Timeout, timeout}]}; handle_call({new_user, User}, From, StateName, - State =#state{user_application = {OldMon, _}}, _) -> + State = #state{connection_env = #connection_env{user_application = {OldMon, _}} = CEnv}, _) -> NewMon = erlang:monitor(process, User), erlang:demonitor(OldMon, [flush]), - {next_state, StateName, State#state{user_application = {NewMon,User}}, + {next_state, StateName, State#state{connection_env = CEnv#connection_env{user_application = {NewMon, User}}}, [{reply, From, ok}]}; handle_call({get_opts, OptTags}, From, _, #state{static_env = #static_env{socket = Socket, @@ -1243,13 +1391,9 @@ handle_call({set_opts, Opts0}, From, StateName, handle_call(renegotiate, From, StateName, _, _) when StateName =/= connection -> {keep_state_and_data, [{reply, From, {error, already_renegotiating}}]}; -handle_call(get_sslsocket, From, _StateName, State, Connection) -> - SslSocket = Connection:socket(State), - {keep_state_and_data, [{reply, From, SslSocket}]}; - handle_call({prf, Secret, Label, Seed, WantedLength}, From, _, #state{connection_states = ConnectionStates, - negotiated_version = Version}, _) -> + connection_env = #connection_env{negotiated_version = Version}}, _) -> #{security_parameters := SecParams} = ssl_record:current_connection_state(ConnectionStates, read), #security_parameters{master_secret = MasterSecret, @@ -1297,14 +1441,14 @@ handle_info({ErrorTag, Socket, Reason}, StateName, #state{static_env = #static_e {stop, {shutdown,normal}, State}; handle_info({'DOWN', MonitorRef, _, _, Reason}, _, - #state{user_application = {MonitorRef, _Pid}, + #state{connection_env = #connection_env{user_application = {MonitorRef, _Pid}}, ssl_options = #ssl_options{erl_dist = true}}) -> {stop, {shutdown, Reason}}; handle_info({'DOWN', MonitorRef, _, _, _}, _, - #state{user_application = {MonitorRef, _Pid}}) -> + #state{connection_env = #connection_env{user_application = {MonitorRef, _Pid}}}) -> {stop, {shutdown, normal}}; handle_info({'EXIT', Pid, _Reason}, StateName, - #state{user_application = {_MonitorRef, Pid}} = State) -> + #state{connection_env = #connection_env{user_application = {_MonitorRef, Pid}}} = State) -> %% It seems the user application has linked to us %% - ignore that and let the monitor handle this {next_state, StateName, State}; @@ -1317,22 +1461,8 @@ handle_info({'EXIT', Socket, normal}, _StateName, #state{static_env = #static_en handle_info({'EXIT', Socket, Reason}, _StateName, #state{static_env = #static_env{socket = Socket}} = State) -> {stop,{shutdown, Reason}, State}; -handle_info(allow_renegotiate, StateName, State) -> - {next_state, StateName, State#state{allow_renegotiate = true}}; - -handle_info({cancel_start_or_recv, StartFrom}, StateName, - #state{renegotiation = {false, first}} = State) when StateName =/= connection -> - {stop_and_reply, - {shutdown, user_timeout}, - {reply, StartFrom, {error, timeout}}, - State#state{timer = undefined}}; -handle_info({cancel_start_or_recv, RecvFrom}, StateName, - #state{start_or_recv_from = RecvFrom} = State) when RecvFrom =/= undefined -> - {next_state, StateName, State#state{start_or_recv_from = undefined, - bytes_to_read = undefined, - timer = undefined}, [{reply, RecvFrom, {error, timeout}}]}; -handle_info({cancel_start_or_recv, _RecvFrom}, StateName, State) -> - {next_state, StateName, State#state{timer = undefined}}; +handle_info(allow_renegotiate, StateName, #state{handshake_env = HsEnv} = State) -> + {next_state, StateName, State#state{handshake_env = HsEnv#handshake_env{allow_renegotiate = true}}}; handle_info(Msg, StateName, #state{static_env = #static_env{socket = Socket, error_tag = Tag}} = State) -> Report = io_lib:format("SSL: Got unexpected info: ~p ~n", [{Msg, Tag, Socket}]), @@ -1342,7 +1472,7 @@ handle_info(Msg, StateName, #state{static_env = #static_env{socket = Socket, err %%==================================================================== %% general gen_statem callbacks %%==================================================================== -terminate(_, _, #state{terminated = true}) -> +terminate(_, _, #state{connection_env = #connection_env{terminated = true}}) -> %% Happens when user closes the connection using ssl:close/1 %% we want to guarantee that Transport:close has been called %% when ssl:close/1 returns unless it is a downgrade where @@ -1406,14 +1536,9 @@ format_status(terminate, [_, StateName, State]) -> [{data, [{"State", {StateName, State#state{connection_states = ?SECRET_PRINTOUT, protocol_buffers = ?SECRET_PRINTOUT, user_data_buffer = ?SECRET_PRINTOUT, - tls_handshake_history = ?SECRET_PRINTOUT, + handshake_env = ?SECRET_PRINTOUT, + connection_env = ?SECRET_PRINTOUT, session = ?SECRET_PRINTOUT, - private_key = ?SECRET_PRINTOUT, - diffie_hellman_params = ?SECRET_PRINTOUT, - diffie_hellman_keys = ?SECRET_PRINTOUT, - srp_params = ?SECRET_PRINTOUT, - srp_keys = ?SECRET_PRINTOUT, - premaster_secret = ?SECRET_PRINTOUT, ssl_options = NewOptions, flight_buffer = ?SECRET_PRINTOUT} }}]}]. @@ -1427,10 +1552,10 @@ send_alert(Alert, _, #state{static_env = #static_env{protocol_cb = Connection}} Connection:send_alert(Alert, State). connection_info(#state{static_env = #static_env{protocol_cb = Connection}, - sni_hostname = SNIHostname, + handshake_env = #handshake_env{sni_hostname = SNIHostname}, session = #session{session_id = SessionId, cipher_suite = CipherSuite, ecc = ECCCurve}, - negotiated_version = {_,_} = Version, + connection_env = #connection_env{negotiated_version = {_,_} = Version}, ssl_options = Opts}) -> RecordCB = record_cb(Connection), CipherSuiteDef = #{key_exchange := KexAlg} = ssl_cipher_format:suite_definition(CipherSuite), @@ -1458,16 +1583,17 @@ security_info(#state{connection_states = ConnectionStates}) -> do_server_hello(Type, #hello_extensions{next_protocol_negotiation = NextProtocols} = ServerHelloExt, - #state{negotiated_version = Version, + #state{connection_env = #connection_env{negotiated_version = Version}, + handshake_env = HsEnv, session = #session{session_id = SessId}, connection_states = ConnectionStates0} = State0, Connection) when is_atom(Type) -> - + ServerHello = ssl_handshake:server_hello(SessId, ssl:tls_version(Version), ConnectionStates0, ServerHelloExt), State = server_hello(ServerHello, - State0#state{expecting_next_protocol_negotiation = - NextProtocols =/= undefined}, Connection), + State0#state{handshake_env = HsEnv#handshake_env{expecting_next_protocol_negotiation = + NextProtocols =/= undefined}}, Connection), case Type of new -> new_server_hello(ServerHello, State, Connection); @@ -1478,8 +1604,8 @@ do_server_hello(Type, #hello_extensions{next_protocol_negotiation = NextProtocol new_server_hello(#server_hello{cipher_suite = CipherSuite, compression_method = Compression, session_id = SessionId}, - #state{session = Session0, - negotiated_version = Version} = State0, Connection) -> + #state{session = Session0, + connection_env = #connection_env{negotiated_version = Version}} = State0, Connection) -> try server_certify_and_key_exchange(State0, Connection) of #state{} = State1 -> {State, Actions} = server_hello_done(State1, Connection), @@ -1495,7 +1621,7 @@ new_server_hello(#server_hello{cipher_suite = CipherSuite, resumed_server_hello(#state{session = Session, connection_states = ConnectionStates0, - negotiated_version = Version} = State0, Connection) -> + connection_env = #connection_env{negotiated_version = Version}} = State0, Connection) -> case ssl_handshake:master_secret(ssl:tls_version(Version), Session, ConnectionStates0, server) of @@ -1512,19 +1638,20 @@ resumed_server_hello(#state{session = Session, server_hello(ServerHello, State0, Connection) -> CipherSuite = ServerHello#server_hello.cipher_suite, #{key_exchange := KeyAlgorithm} = ssl_cipher_format:suite_definition(CipherSuite), - State = Connection:queue_handshake(ServerHello, State0), - State#state{key_algorithm = KeyAlgorithm}. + #state{handshake_env = HsEnv} = State = Connection:queue_handshake(ServerHello, State0), + State#state{handshake_env = HsEnv#handshake_env{kex_algorithm = KeyAlgorithm}}. server_hello_done(State, Connection) -> HelloDone = ssl_handshake:server_hello_done(), Connection:send_handshake(HelloDone, State). handle_peer_cert(Role, PeerCert, PublicKeyInfo, - #state{session = #session{cipher_suite = CipherSuite} = Session} = State0, + #state{handshake_env = HsEnv, + session = #session{cipher_suite = CipherSuite} = Session} = State0, Connection) -> - State1 = State0#state{session = - Session#session{peer_certificate = PeerCert}, - public_key_info = PublicKeyInfo}, + State1 = State0#state{handshake_env = HsEnv#handshake_env{public_key_info = PublicKeyInfo}, + session = + Session#session{peer_certificate = PeerCert}}, #{key_exchange := KeyAlgorithm} = ssl_cipher_format:suite_definition(CipherSuite), State = handle_peer_cert_key(Role, PeerCert, PublicKeyInfo, KeyAlgorithm, State1), Connection:next_event(certify, no_record, State). @@ -1532,21 +1659,13 @@ handle_peer_cert(Role, PeerCert, PublicKeyInfo, handle_peer_cert_key(client, _, {?'id-ecPublicKey', #'ECPoint'{point = _ECPoint} = PublicKey, PublicKeyParams}, - KeyAlg, #state{session = Session} = State) when KeyAlg == ecdh_rsa; + KeyAlg, #state{handshake_env = HsEnv, + session = Session} = State) when KeyAlg == ecdh_rsa; KeyAlg == ecdh_ecdsa -> ECDHKey = public_key:generate_key(PublicKeyParams), PremasterSecret = ssl_handshake:premaster_secret(PublicKey, ECDHKey), - master_secret(PremasterSecret, State#state{diffie_hellman_keys = ECDHKey, + master_secret(PremasterSecret, State#state{handshake_env = HsEnv#handshake_env{kex_keys = ECDHKey}, session = Session#session{ecc = PublicKeyParams}}); -%% We do currently not support cipher suites that use fixed DH. -%% If we want to implement that the following clause can be used -%% to extract DH parameters form cert. -%% handle_peer_cert_key(client, _PeerCert, {?dhpublicnumber, PublicKey, PublicKeyParams}, -%% {_,SignAlg}, -%% #state{diffie_hellman_keys = {_, MyPrivatKey}} = State) when -%% SignAlg == dh_rsa; -%% SignAlg == dh_dss -> -%% dh_master_secret(PublicKeyParams, PublicKey, MyPrivatKey, State); handle_peer_cert_key(_, _, _, _, State) -> State. @@ -1562,16 +1681,16 @@ certify_client(#state{client_certificate_requested = false} = State, _) -> State. verify_client_cert(#state{static_env = #static_env{role = client}, + handshake_env = #handshake_env{tls_handshake_history = Hist, + cert_hashsign_algorithm = HashSign}, + connection_env = #connection_env{negotiated_version = Version, + private_key = PrivateKey}, client_certificate_requested = true, - negotiated_version = Version, - private_key = PrivateKey, session = #session{master_secret = MasterSecret, - own_certificate = OwnCert}, - cert_hashsign_algorithm = HashSign, - tls_handshake_history = Handshake0} = State, Connection) -> + own_certificate = OwnCert}} = State, Connection) -> case ssl_handshake:client_certificate_verify(OwnCert, MasterSecret, - ssl:tls_version(Version), HashSign, PrivateKey, Handshake0) of + ssl:tls_version(Version), HashSign, PrivateKey, Hist) of #certificate_verify{} = Verified -> Connection:queue_handshake(Verified, State); ignore -> @@ -1582,7 +1701,7 @@ verify_client_cert(#state{static_env = #static_env{role = client}, verify_client_cert(#state{client_certificate_requested = false} = State, _) -> State. -client_certify_and_key_exchange(#state{negotiated_version = Version} = +client_certify_and_key_exchange(#state{connection_env = #connection_env{negotiated_version = Version}} = State0, Connection) -> try do_client_certify_and_key_exchange(State0, Connection) of State1 = #state{} -> @@ -1607,7 +1726,9 @@ server_certify_and_key_exchange(State0, Connection) -> request_client_cert(State2, Connection). certify_client_key_exchange(#encrypted_premaster_secret{premaster_secret= EncPMS}, - #state{private_key = Key, client_hello_version = {Major, Minor} = Version} = State, Connection) -> + #state{connection_env = #connection_env{private_key = Key}, + handshake_env = #handshake_env{client_hello_version = {Major, Minor} = Version}} + = State, Connection) -> FakeSecret = make_premaster_secret(Version, rsa), %% Countermeasure for Bleichenbacher attack always provide some kind of premaster secret %% and fail handshake later.RFC 5246 section 7.4.7.1. @@ -1628,14 +1749,15 @@ certify_client_key_exchange(#encrypted_premaster_secret{premaster_secret= EncPMS end, calculate_master_secret(PremasterSecret, State, Connection, certify, cipher); certify_client_key_exchange(#client_diffie_hellman_public{dh_public = ClientPublicDhKey}, - #state{diffie_hellman_params = #'DHParameter'{} = Params, - diffie_hellman_keys = {_, ServerDhPrivateKey}} = State, + #state{handshake_env = #handshake_env{diffie_hellman_params = #'DHParameter'{} = Params, + kex_keys = {_, ServerDhPrivateKey}} + } = State, Connection) -> PremasterSecret = ssl_handshake:premaster_secret(ClientPublicDhKey, ServerDhPrivateKey, Params), calculate_master_secret(PremasterSecret, State, Connection, certify, cipher); certify_client_key_exchange(#client_ec_diffie_hellman_public{dh_public = ClientPublicEcDhPoint}, - #state{diffie_hellman_keys = ECDHKey} = State, Connection) -> + #state{handshake_env = #handshake_env{kex_keys = ECDHKey}} = State, Connection) -> PremasterSecret = ssl_handshake:premaster_secret(#'ECPoint'{point = ClientPublicEcDhPoint}, ECDHKey), calculate_master_secret(PremasterSecret, State, Connection, certify, cipher); certify_client_key_exchange(#client_psk_identity{} = ClientKey, @@ -1645,8 +1767,8 @@ certify_client_key_exchange(#client_psk_identity{} = ClientKey, PremasterSecret = ssl_handshake:premaster_secret(ClientKey, PSKLookup), calculate_master_secret(PremasterSecret, State0, Connection, certify, cipher); certify_client_key_exchange(#client_dhe_psk_identity{} = ClientKey, - #state{diffie_hellman_params = #'DHParameter'{} = Params, - diffie_hellman_keys = {_, ServerDhPrivateKey}, + #state{handshake_env = #handshake_env{diffie_hellman_params = #'DHParameter'{} = Params, + kex_keys = {_, ServerDhPrivateKey}}, ssl_options = #ssl_options{user_lookup_fun = PSKLookup}} = State0, Connection) -> @@ -1654,7 +1776,7 @@ certify_client_key_exchange(#client_dhe_psk_identity{} = ClientKey, ssl_handshake:premaster_secret(ClientKey, ServerDhPrivateKey, Params, PSKLookup), calculate_master_secret(PremasterSecret, State0, Connection, certify, cipher); certify_client_key_exchange(#client_ecdhe_psk_identity{} = ClientKey, - #state{diffie_hellman_keys = ServerEcDhPrivateKey, + #state{handshake_env = #handshake_env{kex_keys = ServerEcDhPrivateKey}, ssl_options = #ssl_options{user_lookup_fun = PSKLookup}} = State, Connection) -> @@ -1662,25 +1784,26 @@ certify_client_key_exchange(#client_ecdhe_psk_identity{} = ClientKey, ssl_handshake:premaster_secret(ClientKey, ServerEcDhPrivateKey, PSKLookup), calculate_master_secret(PremasterSecret, State, Connection, certify, cipher); certify_client_key_exchange(#client_rsa_psk_identity{} = ClientKey, - #state{private_key = Key, + #state{connection_env = #connection_env{private_key = Key}, ssl_options = #ssl_options{user_lookup_fun = PSKLookup}} = State0, Connection) -> PremasterSecret = ssl_handshake:premaster_secret(ClientKey, Key, PSKLookup), calculate_master_secret(PremasterSecret, State0, Connection, certify, cipher); certify_client_key_exchange(#client_srp_public{} = ClientKey, - #state{srp_params = Params, - srp_keys = Key + #state{handshake_env = #handshake_env{srp_params = Params, + kex_keys = Key} } = State0, Connection) -> PremasterSecret = ssl_handshake:premaster_secret(ClientKey, Key, Params), calculate_master_secret(PremasterSecret, State0, Connection, certify, cipher). -certify_server(#state{key_algorithm = Algo} = State, _) when Algo == dh_anon; - Algo == ecdh_anon; - Algo == psk; - Algo == dhe_psk; - Algo == ecdhe_psk; - Algo == srp_anon -> +certify_server(#state{handshake_env = #handshake_env{kex_algorithm = KexAlg}} = + State, _) when KexAlg == dh_anon; + KexAlg == ecdh_anon; + KexAlg == psk; + KexAlg == dhe_psk; + KexAlg == ecdhe_psk; + KexAlg == srp_anon -> State; certify_server(#state{static_env = #static_env{cert_db = CertDbHandle, cert_db_ref = CertDbRef}, @@ -1692,18 +1815,19 @@ certify_server(#state{static_env = #static_env{cert_db = CertDbHandle, throw(Alert) end. -key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = rsa} = State,_) -> +key_exchange(#state{static_env = #static_env{role = server}, + handshake_env = #handshake_env{kex_algorithm = rsa}} = State,_) -> State; -key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = Algo, - hashsign_algorithm = HashSignAlgo, - diffie_hellman_params = #'DHParameter'{} = Params, - private_key = PrivateKey, - connection_states = ConnectionStates0, - negotiated_version = Version - } = State0, Connection) - when Algo == dhe_dss; - Algo == dhe_rsa; - Algo == dh_anon -> +key_exchange(#state{static_env = #static_env{role = server}, + handshake_env = #handshake_env{kex_algorithm = KexAlg, + diffie_hellman_params = #'DHParameter'{} = Params, + hashsign_algorithm = HashSignAlgo}, + connection_env = #connection_env{negotiated_version = Version, + private_key = PrivateKey}, + connection_states = ConnectionStates0} = State0, Connection) + when KexAlg == dhe_dss; + KexAlg == dhe_rsa; + KexAlg == dh_anon -> DHKeys = public_key:generate_key(Params), #{security_parameters := SecParams} = ssl_record:pending_connection_state(ConnectionStates0, read), @@ -1713,24 +1837,26 @@ key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = Alg HashSignAlgo, ClientRandom, ServerRandom, PrivateKey}), - State = Connection:queue_handshake(Msg, State0), - State#state{diffie_hellman_keys = DHKeys}; + #state{handshake_env = HsEnv} = State = Connection:queue_handshake(Msg, State0), + State#state{handshake_env = HsEnv#handshake_env{kex_keys = DHKeys}}; key_exchange(#state{static_env = #static_env{role = server}, - private_key = #'ECPrivateKey'{parameters = ECCurve} = Key, - key_algorithm = Algo, + handshake_env = #handshake_env{kex_algorithm = KexAlg} = HsEnv, + connection_env = #connection_env{private_key = #'ECPrivateKey'{parameters = ECCurve} = Key}, session = Session} = State, _) - when Algo == ecdh_ecdsa; Algo == ecdh_rsa -> - State#state{diffie_hellman_keys = Key, + when KexAlg == ecdh_ecdsa; + KexAlg == ecdh_rsa -> + State#state{handshake_env = HsEnv#handshake_env{kex_keys = Key}, session = Session#session{ecc = ECCurve}}; -key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = Algo, - hashsign_algorithm = HashSignAlgo, - private_key = PrivateKey, +key_exchange(#state{static_env = #static_env{role = server}, + handshake_env = #handshake_env{kex_algorithm = KexAlg, + hashsign_algorithm = HashSignAlgo}, + connection_env = #connection_env{negotiated_version = Version, + private_key = PrivateKey}, session = #session{ecc = ECCCurve}, - connection_states = ConnectionStates0, - negotiated_version = Version - } = State0, Connection) - when Algo == ecdhe_ecdsa; Algo == ecdhe_rsa; - Algo == ecdh_anon -> + connection_states = ConnectionStates0} = State0, Connection) + when KexAlg == ecdhe_ecdsa; + KexAlg == ecdhe_rsa; + KexAlg == ecdh_anon -> ECDHKeys = public_key:generate_key(ECCCurve), #{security_parameters := SecParams} = @@ -1742,18 +1868,19 @@ key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = Alg HashSignAlgo, ClientRandom, ServerRandom, PrivateKey}), - State = Connection:queue_handshake(Msg, State0), - State#state{diffie_hellman_keys = ECDHKeys}; -key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = psk, + #state{handshake_env = HsEnv} = State = Connection:queue_handshake(Msg, State0), + State#state{handshake_env = HsEnv#handshake_env{kex_keys = ECDHKeys}}; +key_exchange(#state{static_env = #static_env{role = server}, + handshake_env = #handshake_env{kex_algorithm = psk}, ssl_options = #ssl_options{psk_identity = undefined}} = State, _) -> State; -key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = psk, +key_exchange(#state{static_env = #static_env{role = server}, ssl_options = #ssl_options{psk_identity = PskIdentityHint}, - hashsign_algorithm = HashSignAlgo, - private_key = PrivateKey, - connection_states = ConnectionStates0, - negotiated_version = Version - } = State0, Connection) -> + handshake_env = #handshake_env{kex_algorithm = psk, + hashsign_algorithm = HashSignAlgo}, + connection_env = #connection_env{negotiated_version = Version, + private_key = PrivateKey}, + connection_states = ConnectionStates0} = State0, Connection) -> #{security_parameters := SecParams} = ssl_record:pending_connection_state(ConnectionStates0, read), #security_parameters{client_random = ClientRandom, @@ -1762,15 +1889,16 @@ key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = psk {psk, PskIdentityHint, HashSignAlgo, ClientRandom, ServerRandom, - PrivateKey}), + PrivateKey}), Connection:queue_handshake(Msg, State0); -key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = dhe_psk, +key_exchange(#state{static_env = #static_env{role = server}, ssl_options = #ssl_options{psk_identity = PskIdentityHint}, - hashsign_algorithm = HashSignAlgo, - diffie_hellman_params = #'DHParameter'{} = Params, - private_key = PrivateKey, - connection_states = ConnectionStates0, - negotiated_version = Version + handshake_env = #handshake_env{kex_algorithm = dhe_psk, + diffie_hellman_params = #'DHParameter'{} = Params, + hashsign_algorithm = HashSignAlgo}, + connection_env = #connection_env{negotiated_version = Version, + private_key = PrivateKey}, + connection_states = ConnectionStates0 } = State0, Connection) -> DHKeys = public_key:generate_key(Params), #{security_parameters := SecParams} = @@ -1783,15 +1911,16 @@ key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = dhe HashSignAlgo, ClientRandom, ServerRandom, PrivateKey}), - State = Connection:queue_handshake(Msg, State0), - State#state{diffie_hellman_keys = DHKeys}; -key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = ecdhe_psk, + #state{handshake_env = HsEnv} = State = Connection:queue_handshake(Msg, State0), + State#state{handshake_env = HsEnv#handshake_env{kex_keys = DHKeys}}; +key_exchange(#state{static_env = #static_env{role = server}, ssl_options = #ssl_options{psk_identity = PskIdentityHint}, - hashsign_algorithm = HashSignAlgo, - private_key = PrivateKey, + handshake_env = #handshake_env{kex_algorithm = ecdhe_psk, + hashsign_algorithm = HashSignAlgo}, + connection_env = #connection_env{negotiated_version = Version, + private_key = PrivateKey}, session = #session{ecc = ECCCurve}, - connection_states = ConnectionStates0, - negotiated_version = Version + connection_states = ConnectionStates0 } = State0, Connection) -> ECDHKeys = public_key:generate_key(ECCCurve), #{security_parameters := SecParams} = @@ -1804,17 +1933,19 @@ key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = ecd HashSignAlgo, ClientRandom, ServerRandom, PrivateKey}), - State = Connection:queue_handshake(Msg, State0), - State#state{diffie_hellman_keys = ECDHKeys}; -key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = rsa_psk, + #state{handshake_env = HsEnv} = State = Connection:queue_handshake(Msg, State0), + State#state{handshake_env = HsEnv#handshake_env{kex_keys = ECDHKeys}}; +key_exchange(#state{static_env = #static_env{role = server}, + handshake_env = #handshake_env{kex_algorithm = rsa_psk}, ssl_options = #ssl_options{psk_identity = undefined}} = State, _) -> State; -key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = rsa_psk, +key_exchange(#state{static_env = #static_env{role = server}, ssl_options = #ssl_options{psk_identity = PskIdentityHint}, - hashsign_algorithm = HashSignAlgo, - private_key = PrivateKey, - connection_states = ConnectionStates0, - negotiated_version = Version + handshake_env = #handshake_env{kex_algorithm = rsa_psk, + hashsign_algorithm = HashSignAlgo}, + connection_env = #connection_env{negotiated_version = Version, + private_key = PrivateKey}, + connection_states = ConnectionStates0 } = State0, Connection) -> #{security_parameters := SecParams} = ssl_record:pending_connection_state(ConnectionStates0, read), @@ -1826,17 +1957,18 @@ key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = rsa ServerRandom, PrivateKey}), Connection:queue_handshake(Msg, State0); -key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = Algo, +key_exchange(#state{static_env = #static_env{role = server}, ssl_options = #ssl_options{user_lookup_fun = LookupFun}, - hashsign_algorithm = HashSignAlgo, + handshake_env = #handshake_env{kex_algorithm = KexAlg, + hashsign_algorithm = HashSignAlgo}, + connection_env = #connection_env{negotiated_version = Version, + private_key = PrivateKey}, session = #session{srp_username = Username}, - private_key = PrivateKey, - connection_states = ConnectionStates0, - negotiated_version = Version + connection_states = ConnectionStates0 } = State0, Connection) - when Algo == srp_dss; - Algo == srp_rsa; - Algo == srp_anon -> + when KexAlg == srp_dss; + KexAlg == srp_rsa; + KexAlg == srp_anon -> SrpParams = handle_srp_identity(Username, LookupFun), Keys = case generate_srp_server_keys(SrpParams, 0) of Alert = #alert{} -> @@ -1853,82 +1985,86 @@ key_exchange(#state{static_env = #static_env{role = server}, key_algorithm = Alg HashSignAlgo, ClientRandom, ServerRandom, PrivateKey}), - State = Connection:queue_handshake(Msg, State0), - State#state{srp_params = SrpParams, - srp_keys = Keys}; + #state{handshake_env = HsEnv} = State = Connection:queue_handshake(Msg, State0), + State#state{handshake_env = HsEnv#handshake_env{srp_params = SrpParams, + kex_keys = Keys}}; key_exchange(#state{static_env = #static_env{role = client}, - key_algorithm = rsa, - public_key_info = PublicKeyInfo, - negotiated_version = Version, - premaster_secret = PremasterSecret} = State0, Connection) -> + handshake_env = #handshake_env{kex_algorithm = rsa, + public_key_info = PublicKeyInfo, + premaster_secret = PremasterSecret}, + connection_env = #connection_env{negotiated_version = Version} + } = State0, Connection) -> Msg = rsa_key_exchange(ssl:tls_version(Version), PremasterSecret, PublicKeyInfo), Connection:queue_handshake(Msg, State0); key_exchange(#state{static_env = #static_env{role = client}, - key_algorithm = Algorithm, - negotiated_version = Version, - diffie_hellman_keys = {DhPubKey, _} - } = State0, Connection) - when Algorithm == dhe_dss; - Algorithm == dhe_rsa; - Algorithm == dh_anon -> + handshake_env = #handshake_env{kex_algorithm = KexAlg, + kex_keys = {DhPubKey, _}}, + connection_env = #connection_env{negotiated_version = Version} + } = State0, Connection) + when KexAlg == dhe_dss; + KexAlg == dhe_rsa; + KexAlg == dh_anon -> Msg = ssl_handshake:key_exchange(client, ssl:tls_version(Version), {dh, DhPubKey}), Connection:queue_handshake(Msg, State0); key_exchange(#state{static_env = #static_env{role = client}, - key_algorithm = Algorithm, - negotiated_version = Version, - session = Session, - diffie_hellman_keys = #'ECPrivateKey'{parameters = ECCurve} = Key} = State0, Connection) - when Algorithm == ecdhe_ecdsa; Algorithm == ecdhe_rsa; - Algorithm == ecdh_ecdsa; Algorithm == ecdh_rsa; - Algorithm == ecdh_anon -> + handshake_env = #handshake_env{kex_algorithm = KexAlg, + kex_keys = #'ECPrivateKey'{parameters = ECCurve} = Key}, + connection_env = #connection_env{negotiated_version = Version}, + session = Session + } = State0, Connection) + when KexAlg == ecdhe_ecdsa; + KexAlg == ecdhe_rsa; + KexAlg == ecdh_ecdsa; + KexAlg == ecdh_rsa; + KexAlg == ecdh_anon -> Msg = ssl_handshake:key_exchange(client, ssl:tls_version(Version), {ecdh, Key}), Connection:queue_handshake(Msg, State0#state{session = Session#session{ecc = ECCurve}}); key_exchange(#state{static_env = #static_env{role = client}, - ssl_options = SslOpts, - key_algorithm = psk, - negotiated_version = Version} = State0, Connection) -> + handshake_env = #handshake_env{kex_algorithm = psk}, + connection_env = #connection_env{negotiated_version = Version}, + ssl_options = SslOpts} = State0, Connection) -> Msg = ssl_handshake:key_exchange(client, ssl:tls_version(Version), {psk, SslOpts#ssl_options.psk_identity}), Connection:queue_handshake(Msg, State0); key_exchange(#state{static_env = #static_env{role = client}, - ssl_options = SslOpts, - key_algorithm = dhe_psk, - negotiated_version = Version, - diffie_hellman_keys = {DhPubKey, _}} = State0, Connection) -> + handshake_env = #handshake_env{kex_algorithm = dhe_psk, + kex_keys = {DhPubKey, _}}, + connection_env = #connection_env{negotiated_version = Version}, + ssl_options = SslOpts} = State0, Connection) -> Msg = ssl_handshake:key_exchange(client, ssl:tls_version(Version), {dhe_psk, SslOpts#ssl_options.psk_identity, DhPubKey}), Connection:queue_handshake(Msg, State0); key_exchange(#state{static_env = #static_env{role = client}, - ssl_options = SslOpts, - key_algorithm = ecdhe_psk, - negotiated_version = Version, - diffie_hellman_keys = ECDHKeys} = State0, Connection) -> + handshake_env = #handshake_env{kex_algorithm = ecdhe_psk, + kex_keys = ECDHKeys}, + connection_env = #connection_env{negotiated_version = Version}, + ssl_options = SslOpts} = State0, Connection) -> Msg = ssl_handshake:key_exchange(client, ssl:tls_version(Version), {ecdhe_psk, SslOpts#ssl_options.psk_identity, ECDHKeys}), Connection:queue_handshake(Msg, State0); key_exchange(#state{static_env = #static_env{role = client}, - ssl_options = SslOpts, - key_algorithm = rsa_psk, - public_key_info = PublicKeyInfo, - negotiated_version = Version, - premaster_secret = PremasterSecret} + handshake_env = #handshake_env{kex_algorithm = rsa_psk, + public_key_info = PublicKeyInfo, + premaster_secret = PremasterSecret}, + connection_env = #connection_env{negotiated_version = Version}, + ssl_options = SslOpts} = State0, Connection) -> Msg = rsa_psk_key_exchange(ssl:tls_version(Version), SslOpts#ssl_options.psk_identity, PremasterSecret, PublicKeyInfo), Connection:queue_handshake(Msg, State0); key_exchange(#state{static_env = #static_env{role = client}, - key_algorithm = Algorithm, - negotiated_version = Version, - srp_keys = {ClientPubKey, _}} + handshake_env = #handshake_env{kex_algorithm = KexAlg, + kex_keys = {ClientPubKey, _}}, + connection_env = #connection_env{negotiated_version = Version}} = State0, Connection) - when Algorithm == srp_dss; - Algorithm == srp_rsa; - Algorithm == srp_anon -> + when KexAlg == srp_dss; + KexAlg == srp_rsa; + KexAlg == srp_anon -> Msg = ssl_handshake:key_exchange(client, ssl:tls_version(Version), {srp, ClientPubKey}), Connection:queue_handshake(Msg, State0). @@ -1965,18 +2101,24 @@ rsa_psk_key_exchange(Version, PskIdentity, PremasterSecret, rsa_psk_key_exchange(_, _, _, _) -> throw (?ALERT_REC(?FATAL,?HANDSHAKE_FAILURE, pub_key_is_not_rsa)). -request_client_cert(#state{key_algorithm = Alg} = State, _) - when Alg == dh_anon; Alg == ecdh_anon; - Alg == psk; Alg == dhe_psk; Alg == ecdhe_psk; Alg == rsa_psk; - Alg == srp_dss; Alg == srp_rsa; Alg == srp_anon -> +request_client_cert(#state{handshake_env = #handshake_env{kex_algorithm = Alg}} = State, _) + when Alg == dh_anon; + Alg == ecdh_anon; + Alg == psk; + Alg == dhe_psk; + Alg == ecdhe_psk; + Alg == rsa_psk; + Alg == srp_dss; + Alg == srp_rsa; + Alg == srp_anon -> State; request_client_cert(#state{static_env = #static_env{cert_db = CertDbHandle, cert_db_ref = CertDbRef}, + connection_env = #connection_env{negotiated_version = Version}, ssl_options = #ssl_options{verify = verify_peer, signature_algs = SupportedHashSigns}, - connection_states = ConnectionStates0, - negotiated_version = Version} = State0, Connection) -> + connection_states = ConnectionStates0} = State0, Connection) -> #{security_parameters := #security_parameters{cipher_suite = CipherSuite}} = ssl_record:pending_connection_state(ConnectionStates0, read), @@ -1993,7 +2135,7 @@ request_client_cert(#state{ssl_options = #ssl_options{verify = verify_none}} = State. calculate_master_secret(PremasterSecret, - #state{negotiated_version = Version, + #state{connection_env = #connection_env{negotiated_version = Version}, connection_states = ConnectionStates0, session = Session0} = State0, Connection, _Current, Next) -> @@ -2022,11 +2164,11 @@ finalize_handshake(State0, StateName, Connection) -> next_protocol(#state{static_env = #static_env{role = server}} = State, _) -> State; -next_protocol(#state{negotiated_protocol = undefined} = State, _) -> +next_protocol(#state{handshake_env = #handshake_env{negotiated_protocol = undefined}} = State, _) -> State; -next_protocol(#state{expecting_next_protocol_negotiation = false} = State, _) -> +next_protocol(#state{handshake_env = #handshake_env{expecting_next_protocol_negotiation = false}} = State, _) -> State; -next_protocol(#state{negotiated_protocol = NextProtocol} = State0, Connection) -> +next_protocol(#state{handshake_env = #handshake_env{negotiated_protocol = NextProtocol}} = State0, Connection) -> NextProtocolMessage = ssl_handshake:next_protocol(NextProtocol), Connection:queue_handshake(NextProtocolMessage, State0). @@ -2034,14 +2176,15 @@ cipher_protocol(State, Connection) -> Connection:queue_change_cipher(#change_cipher_spec{}, State). finished(#state{static_env = #static_env{role = Role}, - negotiated_version = Version, + handshake_env = #handshake_env{tls_handshake_history = Hist}, + connection_env = #connection_env{negotiated_version = Version}, session = Session, - connection_states = ConnectionStates0, - tls_handshake_history = Handshake0} = State0, StateName, Connection) -> + connection_states = ConnectionStates0} = State0, + StateName, Connection) -> MasterSecret = Session#session.master_secret, Finished = ssl_handshake:finished(ssl:tls_version(Version), Role, get_current_prf(ConnectionStates0, write), - MasterSecret, Handshake0), + MasterSecret, Hist), ConnectionStates = save_verify_data(Role, Finished, ConnectionStates0, StateName), Connection:send_handshake(Finished, State0#state{connection_states = ConnectionStates}). @@ -2057,65 +2200,71 @@ save_verify_data(server, #finished{verify_data = Data}, ConnectionStates, abbrev calculate_secret(#server_dh_params{dh_p = Prime, dh_g = Base, dh_y = ServerPublicDhKey} = Params, - State, Connection) -> + #state{handshake_env = HsEnv} = State, Connection) -> Keys = {_, PrivateDhKey} = crypto:generate_key(dh, [Prime, Base]), PremasterSecret = ssl_handshake:premaster_secret(ServerPublicDhKey, PrivateDhKey, Params), calculate_master_secret(PremasterSecret, - State#state{diffie_hellman_keys = Keys}, + State#state{handshake_env = HsEnv#handshake_env{kex_keys = Keys}}, Connection, certify, certify); calculate_secret(#server_ecdh_params{curve = ECCurve, public = ECServerPubKey}, - State=#state{session=Session}, Connection) -> + #state{handshake_env = HsEnv, + session = Session} = State, Connection) -> ECDHKeys = public_key:generate_key(ECCurve), PremasterSecret = ssl_handshake:premaster_secret(#'ECPoint'{point = ECServerPubKey}, ECDHKeys), calculate_master_secret(PremasterSecret, - State#state{diffie_hellman_keys = ECDHKeys, + State#state{handshake_env = HsEnv#handshake_env{kex_keys = ECDHKeys}, session = Session#session{ecc = ECCurve}}, Connection, certify, certify); calculate_secret(#server_psk_params{ hint = IdentityHint}, - State, Connection) -> + #state{handshake_env = HsEnv} = State, Connection) -> %% store for later use - Connection:next_event(certify, no_record, State#state{psk_identity = IdentityHint}); + Connection:next_event(certify, no_record, + State#state{handshake_env = + HsEnv#handshake_env{server_psk_identity = IdentityHint}}); calculate_secret(#server_dhe_psk_params{ dh_params = #server_dh_params{dh_p = Prime, dh_g = Base}} = ServerKey, - #state{ssl_options = #ssl_options{user_lookup_fun = PSKLookup}} = + #state{handshake_env = HsEnv, + ssl_options = #ssl_options{user_lookup_fun = PSKLookup}} = State, Connection) -> Keys = {_, PrivateDhKey} = crypto:generate_key(dh, [Prime, Base]), PremasterSecret = ssl_handshake:premaster_secret(ServerKey, PrivateDhKey, PSKLookup), - calculate_master_secret(PremasterSecret, State#state{diffie_hellman_keys = Keys}, + calculate_master_secret(PremasterSecret, State#state{handshake_env = HsEnv#handshake_env{kex_keys = Keys}}, Connection, certify, certify); calculate_secret(#server_ecdhe_psk_params{ dh_params = #server_ecdh_params{curve = ECCurve}} = ServerKey, #state{ssl_options = #ssl_options{user_lookup_fun = PSKLookup}} = - State=#state{session=Session}, Connection) -> + #state{handshake_env = HsEnv, + session = Session} = State, Connection) -> ECDHKeys = public_key:generate_key(ECCurve), PremasterSecret = ssl_handshake:premaster_secret(ServerKey, ECDHKeys, PSKLookup), calculate_master_secret(PremasterSecret, - State#state{diffie_hellman_keys = ECDHKeys, + State#state{handshake_env = HsEnv#handshake_env{kex_keys = ECDHKeys}, session = Session#session{ecc = ECCurve}}, Connection, certify, certify); calculate_secret(#server_srp_params{srp_n = Prime, srp_g = Generator} = ServerKey, - #state{ssl_options = #ssl_options{srp_identity = SRPId}} = State, + #state{handshake_env = HsEnv, + ssl_options = #ssl_options{srp_identity = SRPId}} = State, Connection) -> Keys = generate_srp_client_keys(Generator, Prime, 0), PremasterSecret = ssl_handshake:premaster_secret(ServerKey, Keys, SRPId), - calculate_master_secret(PremasterSecret, State#state{srp_keys = Keys}, Connection, + calculate_master_secret(PremasterSecret, State#state{handshake_env = HsEnv#handshake_env{kex_keys = Keys}}, Connection, certify, certify). master_secret(#alert{} = Alert, _) -> Alert; master_secret(PremasterSecret, #state{static_env = #static_env{role = Role}, + connection_env = #connection_env{negotiated_version = Version}, session = Session, - negotiated_version = Version, connection_states = ConnectionStates0} = State) -> case ssl_handshake:master_secret(ssl:tls_version(Version), PremasterSecret, ConnectionStates0, Role) of @@ -2175,7 +2324,7 @@ cipher_role(client, Data, Session, #state{connection_states = ConnectionStates0} {Record, State} = prepare_connection(State0#state{session = Session, connection_states = ConnectionStates}, Connection), - Connection:next_event(connection, Record, State); + Connection:next_event(connection, Record, State, [{{timeout, handshake}, infinity, close}]); cipher_role(server, Data, Session, #state{connection_states = ConnectionStates0} = State0, Connection) -> ConnectionStates1 = ssl_record:set_client_verify_data(current_read, Data, @@ -2184,15 +2333,15 @@ cipher_role(server, Data, Session, #state{connection_states = ConnectionStates0 finalize_handshake(State0#state{connection_states = ConnectionStates1, session = Session}, cipher, Connection), {Record, State} = prepare_connection(State1, Connection), - Connection:next_event(connection, Record, State, Actions). - -is_anonymous(Algo) when Algo == dh_anon; - Algo == ecdh_anon; - Algo == psk; - Algo == dhe_psk; - Algo == ecdhe_psk; - Algo == rsa_psk; - Algo == srp_anon -> + Connection:next_event(connection, Record, State, [{{timeout, handshake}, infinity, close} | Actions]). + +is_anonymous(KexAlg) when KexAlg == dh_anon; + KexAlg == ecdh_anon; + KexAlg == psk; + KexAlg == dhe_psk; + KexAlg == ecdhe_psk; + KexAlg == rsa_psk; + KexAlg == srp_anon -> true; is_anonymous(_) -> false. @@ -2369,7 +2518,7 @@ handle_trusted_certs_db(#state{static_env = #static_env{cert_db_ref = Ref, ok end. -prepare_connection(#state{renegotiation = Renegotiate, +prepare_connection(#state{handshake_env = #handshake_env{renegotiation = Renegotiate}, start_or_recv_from = RecvFrom} = State0, Connection) when Renegotiate =/= {false, first}, RecvFrom =/= undefined -> @@ -2379,42 +2528,54 @@ prepare_connection(State0, Connection) -> State = Connection:reinit(State0), {no_record, ack_connection(State)}. -ack_connection(#state{renegotiation = {true, Initiater}} = State) when Initiater == peer; - Initiater == internal -> - State#state{renegotiation = undefined}; -ack_connection(#state{renegotiation = {true, From}} = State) -> +ack_connection(#state{handshake_env = #handshake_env{renegotiation = {true, Initiater}} = HsEnv} = State) when Initiater == peer; + Initiater == internal -> + State#state{handshake_env = HsEnv#handshake_env{renegotiation = undefined}}; +ack_connection(#state{handshake_env = #handshake_env{renegotiation = {true, From}} = HsEnv} = State) -> gen_statem:reply(From, ok), - State#state{renegotiation = undefined}; -ack_connection(#state{renegotiation = {false, first}, - start_or_recv_from = StartFrom, - timer = Timer} = State) when StartFrom =/= undefined -> + State#state{handshake_env = HsEnv#handshake_env{renegotiation = undefined}}; +ack_connection(#state{handshake_env = #handshake_env{renegotiation = {false, first}} = HsEnv, + start_or_recv_from = StartFrom} = State) when StartFrom =/= undefined -> gen_statem:reply(StartFrom, connected), - cancel_timer(Timer), - State#state{renegotiation = undefined, - start_or_recv_from = undefined, timer = undefined}; + State#state{handshake_env = HsEnv#handshake_env{renegotiation = undefined}, + start_or_recv_from = undefined}; ack_connection(State) -> State. -cancel_timer(undefined) -> - ok; -cancel_timer(Timer) -> - erlang:cancel_timer(Timer), - ok. - session_handle_params(#server_ecdh_params{curve = ECCurve}, Session) -> Session#session{ecc = ECCurve}; session_handle_params(_, Session) -> Session. -register_session(client, Host, Port, #session{is_resumable = new} = Session0) -> +handle_session(Role = server, #ssl_options{reuse_sessions = true} = SslOpts, + Host, Port, Session0) -> + register_session(Role, host_id(Role, Host, SslOpts), Port, Session0, true); +handle_session(Role = client, #ssl_options{verify = verify_peer, + reuse_sessions = Reuse} = SslOpts, + Host, Port, Session0) when Reuse =/= false -> + register_session(Role, host_id(Role, Host, SslOpts), Port, Session0, reg_type(Reuse)); +handle_session(server, _, Host, Port, Session) -> + %% Remove "session of type new" entry from session DB + ssl_manager:invalidate_session(Host, Port, Session), + Session; +handle_session(client, _,_,_, Session) -> + %% In client case there is no entry yet, so nothing to remove + Session. + +reg_type(save) -> + true; +reg_type(true) -> + unique. + +register_session(client, Host, Port, #session{is_resumable = new} = Session0, Save) -> Session = Session0#session{is_resumable = true}, - ssl_manager:register_session(Host, Port, Session), + ssl_manager:register_session(Host, Port, Session, Save), Session; -register_session(server, _, Port, #session{is_resumable = new} = Session0) -> +register_session(server, _, Port, #session{is_resumable = new} = Session0, _) -> Session = Session0#session{is_resumable = true}, ssl_manager:register_session(Port, Session), Session; -register_session(_, _, _, Session) -> +register_session(_, _, _, Session, _) -> Session. %% Already registered host_id(client, _Host, #ssl_options{server_name_indication = Hostname}) when is_list(Hostname) -> @@ -2436,9 +2597,8 @@ handle_resumed_session(SessId, #state{static_env = #static_env{host = Host, protocol_cb = Connection, session_cache = Cache, session_cache_cb = CacheCb}, - connection_states = ConnectionStates0, - negotiated_version = Version - } = State) -> + connection_env = #connection_env{negotiated_version = Version}, + connection_states = ConnectionStates0} = State) -> Session = CacheCb:lookup(Cache, {{Host, Port}, SessId}), case ssl_handshake:master_secret(ssl:tls_version(Version), Session, ConnectionStates0, client) of @@ -2496,7 +2656,7 @@ handle_active_option(false, connection = StateName, To, Reply, State) -> hibernate_after(StateName, State, [{reply, To, Reply}]); handle_active_option(_, connection = StateName0, To, Reply, #state{static_env = #static_env{protocol_cb = Connection}, - user_data_buffer = <<>>} = State0) -> + user_data_buffer = {_,0,_}} = State0) -> case Connection:next_event(StateName0, no_record, State0) of {next_state, StateName, State} -> hibernate_after(StateName, State, [{reply, To, Reply}]); @@ -2505,11 +2665,11 @@ handle_active_option(_, connection = StateName0, To, Reply, #state{static_env = {stop, _, _} = Stop -> Stop end; -handle_active_option(_, StateName, To, Reply, #state{user_data_buffer = <<>>} = State) -> +handle_active_option(_, StateName, To, Reply, #state{user_data_buffer = {_,0,_}} = State) -> %% Active once already set {next_state, StateName, State, [{reply, To, Reply}]}; -%% user_data_buffer =/= <<>> +%% user_data_buffer nonempty handle_active_option(_, StateName0, To, Reply, #state{static_env = #static_env{protocol_cb = Connection}} = State0) -> case read_application_data(<<>>, State0) of @@ -2529,33 +2689,25 @@ handle_active_option(_, StateName0, To, Reply, %% Picks ClientData -get_data(_, _, <<>>) -> - {more, <<>>}; -%% Recv timed out save buffer data until next recv -get_data(#socket_options{active=false}, undefined, Buffer) -> - {passive, Buffer}; -get_data(#socket_options{active=Active, packet=Raw}, BytesToRead, Buffer) +get_data(#socket_options{active=false}, undefined, _Bin) -> + %% Recv timed out save buffer data until next recv + passive; +get_data(#socket_options{active=Active, packet=Raw}, BytesToRead, Bin) when Raw =:= raw; Raw =:= 0 -> %% Raw Mode - if - Active =/= false orelse BytesToRead =:= 0 -> + case Bin of + <<_/binary>> when Active =/= false orelse BytesToRead =:= 0 -> %% Active true or once, or passive mode recv(0) - {ok, Buffer, <<>>}; - byte_size(Buffer) >= BytesToRead -> + {ok, Bin, <<>>}; + <<Data:BytesToRead/binary, Rest/binary>> -> %% Passive Mode, recv(Bytes) - <<Data:BytesToRead/binary, Rest/binary>> = Buffer, - {ok, Data, Rest}; - true -> + {ok, Data, Rest}; + <<_/binary>> -> %% Passive Mode not enough data - {more, Buffer} + {more, BytesToRead} end; -get_data(#socket_options{packet=Type, packet_size=Size}, _, Buffer) -> +get_data(#socket_options{packet=Type, packet_size=Size}, _, Bin) -> PacketOpts = [{packet_size, Size}], - case decode_packet(Type, Buffer, PacketOpts) of - {more, _} -> - {more, Buffer}; - Decoded -> - Decoded - end. + decode_packet(Type, Bin, PacketOpts). decode_packet({http, headers}, Buffer, PacketOpts) -> decode_packet(httph, Buffer, PacketOpts); @@ -2607,7 +2759,7 @@ format_reply(_, _, _,#socket_options{active = false, mode = Mode, packet = Packe {ok, do_format_reply(Mode, Packet, Header, Data)}; format_reply(CPids, Transport, Socket, #socket_options{active = _, mode = Mode, packet = Packet, header = Header}, Data, Tracker, Connection) -> - {ssl, Connection:socket(CPids, Transport, Socket, Connection, Tracker), + {ssl, Connection:socket(CPids, Transport, Socket, Tracker), do_format_reply(Mode, Packet, Header, Data)}. deliver_packet_error(CPids, Transport, Socket, @@ -2619,7 +2771,7 @@ format_packet_error(_, _, _,#socket_options{active = false, mode = Mode}, Data, {error, {invalid_packet, do_format_reply(Mode, raw, 0, Data)}}; format_packet_error(CPids, Transport, Socket, #socket_options{active = _, mode = Mode}, Data, Tracker, Connection) -> - {ssl_error, Connection:socket(CPids, Transport, Socket, Connection, Tracker), + {ssl_error, Connection:socket(CPids, Transport, Socket, Tracker), {invalid_packet, do_format_reply(Mode, raw, 0, Data)}}. do_format_reply(binary, _, N, Data) when N > 0 -> % Header mode @@ -2675,12 +2827,10 @@ alert_user(Pids, Transport, Tracker, Socket, Active, Pid, From, Alert, Role, Con case ssl_alert:reason_code(Alert, Role) of closed -> send_or_reply(Active, Pid, From, - {ssl_closed, Connection:socket(Pids, - Transport, Socket, Connection, Tracker)}); + {ssl_closed, Connection:socket(Pids, Transport, Socket, Tracker)}); ReasonCode -> send_or_reply(Active, Pid, From, - {ssl_error, Connection:socket(Pids, - Transport, Socket, Connection, Tracker), ReasonCode}) + {ssl_error, Connection:socket(Pids, Transport, Socket, Tracker), ReasonCode}) end. log_alert(true, Role, ProtocolName, StateName, #alert{role = Role} = Alert) -> @@ -2699,7 +2849,9 @@ invalidate_session(server, _, Port, Session) -> handle_sni_extension(undefined, State) -> State; -handle_sni_extension(#sni{hostname = Hostname}, #state{static_env = #static_env{role = Role} = InitStatEnv0} = State0) -> +handle_sni_extension(#sni{hostname = Hostname}, #state{static_env = #static_env{role = Role} = InitStatEnv0, + handshake_env = HsEnv, + connection_env = CEnv} = State0) -> NewOptions = update_ssl_options_from_sni(State0#state.ssl_options, Hostname), case NewOptions of undefined -> @@ -2722,12 +2874,12 @@ handle_sni_extension(#sni{hostname = Hostname}, #state{static_env = #static_env{ cert_db = CertDbHandle, crl_db = CRLDbHandle, session_cache = CacheHandle - }, - private_key = Key, - diffie_hellman_params = DHParams, - ssl_options = NewOptions, - sni_hostname = Hostname - } + }, + connection_env = CEnv#connection_env{private_key = Key}, + ssl_options = NewOptions, + handshake_env = HsEnv#handshake_env{sni_hostname = Hostname, + diffie_hellman_params = DHParams} + } end. update_ssl_options_from_sni(OrigSSLOptions, SNIHostname) -> @@ -2750,11 +2902,3 @@ new_emulated([], EmOpts) -> EmOpts; new_emulated(NewEmOpts, _) -> NewEmOpts. - --compile({inline, [bincat/2]}). -bincat(<<>>, B) -> - B; -bincat(A, <<>>) -> - A; -bincat(A, B) -> - <<A/binary, B/binary>>. diff --git a/lib/ssl/src/ssl_connection.hrl b/lib/ssl/src/ssl_connection.hrl index dc8aa7619b..9efd65b2d2 100644 --- a/lib/ssl/src/ssl_connection.hrl +++ b/lib/ssl/src/ssl_connection.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2018. All Rights Reserved. +%% Copyright Ericsson AB 2013-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -51,58 +51,69 @@ cert_db_ref :: certdb_ref() | 'undefined', tracker :: pid() | 'undefined' %% Tracker process for listen socket }). + +-record(handshake_env, { + client_hello_version :: ssl_record:ssl_version() | 'undefined', + unprocessed_handshake_events = 0 :: integer(), + tls_handshake_history :: ssl_handshake:ssl_handshake_history() | secret_printout() + | 'undefined', + expecting_finished = false ::boolean(), + renegotiation :: undefined | {boolean(), From::term() | internal | peer}, + allow_renegotiate = true ::boolean(), + %% Ext handling + hello, %%:: #client_hello{} | #server_hello{} + sni_hostname = undefined, + expecting_next_protocol_negotiation = false ::boolean(), + next_protocol = undefined :: undefined | binary(), + negotiated_protocol, + hashsign_algorithm = {undefined, undefined}, + cert_hashsign_algorithm = {undefined, undefined}, + %% key exchange + kex_algorithm :: ssl:kex_algo(), + kex_keys :: {PublicKey :: binary(), PrivateKey :: binary()} | #'ECPrivateKey'{} | undefined | secret_printout(), + diffie_hellman_params:: #'DHParameter'{} | undefined | secret_printout(), + srp_params :: #srp_user{} | secret_printout() | 'undefined', + public_key_info :: ssl_handshake:public_key_info() | 'undefined', + premaster_secret :: binary() | secret_printout() | 'undefined', + server_psk_identity :: binary() | 'undefined' % server psk identity hint + }). + +-record(connection_env, { + user_application :: {Monitor::reference(), User::pid()}, + downgrade, + terminated = false ::boolean() | closed, + negotiated_version :: ssl_record:ssl_version() | 'undefined', + erl_dist_handle = undefined :: erlang:dist_handle() | 'undefined', + private_key :: public_key:private_key() | secret_printout() | 'undefined' + }). + -record(state, { static_env :: #static_env{}, - %% Change seldome - user_application :: {Monitor::reference(), User::pid()}, + connection_env :: #connection_env{} | secret_printout(), ssl_options :: #ssl_options{}, socket_options :: #socket_options{}, - session :: #session{} | secret_printout(), - allow_renegotiate = true ::boolean(), - terminated = false ::boolean() | closed, - negotiated_version :: ssl_record:ssl_version() | 'undefined', - bytes_to_read :: undefined | integer(), %% bytes to read in passive mode - downgrade, - %% Changed often + %% Hanshake %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + handshake_env :: #handshake_env{} | secret_printout(), + %% Buffer of TLS/DTLS records, used during the TLS + %% handshake to when possible pack more than one TLS + %% record into the underlaying packet + %% format. Introduced by DTLS - RFC 4347. The + %% mecahnism is also usefull in TLS although we do not + %% need to worry about packet loss in TLS. In DTLS we + %% need to track DTLS handshake seqnr + flight_buffer = [] :: list() | map(), + client_certificate_requested = false :: boolean(), + protocol_specific = #{} :: map(), + session :: #session{} | secret_printout(), + %% Data shuffling %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% connection_states :: ssl_record:connection_states() | secret_printout(), protocol_buffers :: term() | secret_printout() , %% #protocol_buffers{} from tls_record.hrl or dtls_recor.hr - user_data_buffer :: undefined | binary() | secret_printout(), + user_data_buffer :: undefined | {[binary()],non_neg_integer(),[binary()]} | secret_printout(), + bytes_to_read :: undefined | integer(), %% bytes to read in passive mode - %% Used only in HS - unprocessed_handshake_events = 0 :: integer(), - tls_handshake_history :: ssl_handshake:ssl_handshake_history() | secret_printout() - | 'undefined', - client_hello_version :: ssl_record:ssl_version() | 'undefined', - client_certificate_requested = false :: boolean(), - key_algorithm :: ssl_cipher_format:key_algo(), - hashsign_algorithm = {undefined, undefined}, - cert_hashsign_algorithm = {undefined, undefined}, - public_key_info :: ssl_handshake:public_key_info() | 'undefined', - private_key :: public_key:private_key() | secret_printout() | 'undefined', - diffie_hellman_params:: #'DHParameter'{} | undefined | secret_printout(), - diffie_hellman_keys :: {PublicKey :: binary(), PrivateKey :: binary()} | #'ECPrivateKey'{} | undefined | secret_printout(), - psk_identity :: binary() | 'undefined', % server psk identity hint - srp_params :: #srp_user{} | secret_printout() | 'undefined', - srp_keys ::{PublicKey :: binary(), PrivateKey :: binary()} | secret_printout() | 'undefined', - premaster_secret :: binary() | secret_printout() | 'undefined', - renegotiation :: undefined | {boolean(), From::term() | internal | peer}, - start_or_recv_from :: term(), - timer :: undefined | reference(), % start_or_recive_timer - hello, %%:: #client_hello{} | #server_hello{}, - expecting_next_protocol_negotiation = false ::boolean(), - expecting_finished = false ::boolean(), - next_protocol = undefined :: undefined | binary(), - negotiated_protocol, - sni_hostname = undefined, - flight_buffer = [] :: list() | map(), %% Buffer of TLS/DTLS records, used during the TLS handshake - %% to when possible pack more than one TLS record into the - %% underlaying packet format. Introduced by DTLS - RFC 4347. - %% The mecahnism is also usefull in TLS although we do not - %% need to worry about packet loss in TLS. In DTLS we need to track DTLS handshake seqnr - flight_state = reliable, %% reliable | {retransmit, integer()}| {waiting, ref(), integer()} - last two is used in DTLS over udp. - erl_dist_handle = undefined :: erlang:dist_handle() | undefined, - protocol_specific = #{} :: map() + %% recv and start handling + start_or_recv_from :: term() }). diff --git a/lib/ssl/src/ssl_crl_cache.erl b/lib/ssl/src/ssl_crl_cache.erl index 9c1af86eeb..841620ce57 100644 --- a/lib/ssl/src/ssl_crl_cache.erl +++ b/lib/ssl/src/ssl_crl_cache.erl @@ -28,6 +28,10 @@ -behaviour(ssl_crl_cache_api). +-export_type([crl_src/0, uri/0]). +-type crl_src() :: {file, file:filename()} | {der, public_key:der_encoded()}. +-type uri() :: uri_string:uri_string(). + -export([lookup/3, select/2, fresh_crl/2]). -export([insert/1, insert/2, delete/1]). diff --git a/lib/ssl/src/ssl_crl_cache_api.erl b/lib/ssl/src/ssl_crl_cache_api.erl index d5380583e7..8a750b3929 100644 --- a/lib/ssl/src/ssl_crl_cache_api.erl +++ b/lib/ssl/src/ssl_crl_cache_api.erl @@ -21,12 +21,15 @@ %% -module(ssl_crl_cache_api). - -include_lib("public_key/include/public_key.hrl"). --type db_handle() :: term(). --type issuer_name() :: {rdnSequence, [#'AttributeTypeAndValue'{}]}. +-export_type([dist_point/0, crl_cache_ref/0]). + +-type crl_cache_ref() :: any(). +-type issuer_name() :: {rdnSequence,[#'AttributeTypeAndValue'{}]}. +-type dist_point() :: #'DistributionPoint'{}. --callback lookup(#'DistributionPoint'{}, issuer_name(), db_handle()) -> not_available | [public_key:der_encoded()]. --callback select(issuer_name(), db_handle()) -> [public_key:der_encoded()]. --callback fresh_crl(#'DistributionPoint'{}, public_key:der_encoded()) -> public_key:der_encoded(). + +-callback lookup(dist_point(), issuer_name(), crl_cache_ref()) -> not_available | [public_key:der_encoded()]. +-callback select(issuer_name(), crl_cache_ref()) -> [public_key:der_encoded()]. +-callback fresh_crl(dist_point(), public_key:der_encoded()) -> public_key:der_encoded(). diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index 14df1d2e02..9ba62b3a12 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -610,7 +610,7 @@ encode_hello_extensions([#ec_point_formats{ec_point_format_list = ECPointFormats ?UINT16(Len), ?BYTE(ListLen), ECPointFormatList/binary, Acc/binary>>); encode_hello_extensions([#srp{username = UserName} | Rest], Acc) -> SRPLen = byte_size(UserName), - Len = SRPLen + 2, + Len = SRPLen + 1, encode_hello_extensions(Rest, <<?UINT16(?SRP_EXT), ?UINT16(Len), ?BYTE(SRPLen), UserName/binary, Acc/binary>>); encode_hello_extensions([#hash_sign_algos{hash_sign_algos = HashSignAlgos} | Rest], Acc) -> @@ -729,7 +729,7 @@ decode_hello_extensions(Extensions) -> dec_hello_extensions(Extensions, #hello_extensions{}). %%-------------------------------------------------------------------- --spec decode_server_key(binary(), ssl_cipher_format:key_algo(), ssl_record:ssl_version()) -> +-spec decode_server_key(binary(), ssl:kex_algo(), ssl_record:ssl_version()) -> #server_key_params{}. %% %% Description: Decode server_key data and return appropriate type @@ -738,7 +738,7 @@ decode_server_key(ServerKey, Type, Version) -> dec_server_key(ServerKey, key_exchange_alg(Type), Version). %%-------------------------------------------------------------------- --spec decode_client_key(binary(), ssl_cipher_format:key_algo(), ssl_record:ssl_version()) -> +-spec decode_client_key(binary(), ssl:kex_algo(), ssl_record:ssl_version()) -> #encrypted_premaster_secret{} | #client_diffie_hellman_public{} | #client_ec_diffie_hellman_public{} @@ -972,34 +972,30 @@ handle_client_hello_extensions(RecordCB, Random, ClientCipherSuites, #session{cipher_suite = NegotiatedCipherSuite, compression_method = Compression} = Session0, ConnectionStates0, Renegotiation) -> - Session = handle_srp_extension(SRP, Session0), - ConnectionStates = handle_renegotiation_extension(server, RecordCB, Version, Info, - Random, NegotiatedCipherSuite, + Session = handle_srp_extension(SRP, Session0), + ConnectionStates = handle_renegotiation_extension(server, RecordCB, Version, Info, + Random, NegotiatedCipherSuite, ClientCipherSuites, Compression, - ConnectionStates0, Renegotiation, SecureRenegotation), - - ServerHelloExtensions = #hello_extensions{ - renegotiation_info = renegotiation_info(RecordCB, server, - ConnectionStates, Renegotiation), - ec_point_formats = server_ecc_extension(Version, ECCFormat) - }, - + ConnectionStates0, Renegotiation, SecureRenegotation), + + ServerHelloExtensions = #hello_extensions{ + renegotiation_info = renegotiation_info(RecordCB, server, + ConnectionStates, Renegotiation), + ec_point_formats = server_ecc_extension(Version, ECCFormat) + }, + %% If we receive an ALPN extension and have ALPN configured for this connection, %% we handle it. Otherwise we check for the NPN extension. if ALPN =/= undefined, ALPNPreferredProtocols =/= undefined -> - case handle_alpn_extension(ALPNPreferredProtocols, decode_alpn(ALPN)) of - #alert{} = Alert -> - Alert; - Protocol -> - {Session, ConnectionStates, Protocol, - ServerHelloExtensions#hello_extensions{alpn=encode_alpn([Protocol], Renegotiation)}} - end; + Protocol = handle_alpn_extension(ALPNPreferredProtocols, decode_alpn(ALPN)), + {Session, ConnectionStates, Protocol, + ServerHelloExtensions#hello_extensions{alpn=encode_alpn([Protocol], Renegotiation)}}; true -> - ProtocolsToAdvertise = handle_next_protocol_extension(NextProtocolNegotiation, Renegotiation, Opts), + ProtocolsToAdvertise = handle_next_protocol_extension(NextProtocolNegotiation, Renegotiation, Opts), {Session, ConnectionStates, undefined, - ServerHelloExtensions#hello_extensions{next_protocol_negotiation= - encode_protocols_advertised_on_server(ProtocolsToAdvertise)}} + ServerHelloExtensions#hello_extensions{next_protocol_negotiation= + encode_protocols_advertised_on_server(ProtocolsToAdvertise)}} end. handle_server_hello_extensions(RecordCB, Random, CipherSuite, Compression, @@ -1022,12 +1018,8 @@ handle_server_hello_extensions(RecordCB, Random, CipherSuite, Compression, [Protocol] when not Renegotiation -> {ConnectionStates, alpn, Protocol}; undefined -> - case handle_next_protocol(NextProtocolNegotiation, NextProtoSelector, Renegotiation) of - #alert{} = Alert -> - Alert; - Protocol -> - {ConnectionStates, npn, Protocol} - end; + Protocol = handle_next_protocol(NextProtocolNegotiation, NextProtoSelector, Renegotiation), + {ConnectionStates, npn, Protocol}; {error, Reason} -> ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, Reason); [] -> @@ -1941,7 +1933,7 @@ dec_hello_extensions(<<?UINT16(?RENEGOTIATION_EXT), ?UINT16(Len), Info:Len/binar RenegotiateInfo}}); dec_hello_extensions(<<?UINT16(?SRP_EXT), ?UINT16(Len), ?BYTE(SRPLen), SRP:SRPLen/binary, Rest/binary>>, Acc) - when Len == SRPLen + 2 -> + when Len == SRPLen + 1 -> dec_hello_extensions(Rest, Acc#hello_extensions{srp = #srp{username = SRP}}); dec_hello_extensions(<<?UINT16(?SIGNATURE_ALGORITHMS_EXT), ?UINT16(Len), @@ -2189,30 +2181,26 @@ filter_unavailable_ecc_suites(_, Suites) -> handle_renegotiation_extension(Role, RecordCB, Version, Info, Random, NegotiatedCipherSuite, ClientCipherSuites, Compression, ConnectionStates0, Renegotiation, SecureRenegotation) -> - case handle_renegotiation_info(RecordCB, Role, Info, ConnectionStates0, - Renegotiation, SecureRenegotation, - ClientCipherSuites) of - {ok, ConnectionStates} -> - hello_pending_connection_states(RecordCB, Role, - Version, - NegotiatedCipherSuite, - Random, - Compression, - ConnectionStates); - #alert{} = Alert -> - throw(Alert) - end. + {ok, ConnectionStates} = handle_renegotiation_info(RecordCB, Role, Info, ConnectionStates0, + Renegotiation, SecureRenegotation, + ClientCipherSuites), + hello_pending_connection_states(RecordCB, Role, + Version, + NegotiatedCipherSuite, + Random, + Compression, + ConnectionStates). %% Receive protocols, choose one from the list, return it. handle_alpn_extension(_, {error, Reason}) -> - ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, Reason); + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, Reason)); handle_alpn_extension([], _) -> - ?ALERT_REC(?FATAL, ?NO_APPLICATION_PROTOCOL); + throw(?ALERT_REC(?FATAL, ?NO_APPLICATION_PROTOCOL)); handle_alpn_extension([ServerProtocol|Tail], ClientProtocols) -> - case lists:member(ServerProtocol, ClientProtocols) of - true -> ServerProtocol; - false -> handle_alpn_extension(Tail, ClientProtocols) - end. + case lists:member(ServerProtocol, ClientProtocols) of + true -> ServerProtocol; + false -> handle_alpn_extension(Tail, ClientProtocols) + end. handle_next_protocol(undefined, _NextProtocolSelector, _Renegotiating) -> @@ -2225,14 +2213,14 @@ handle_next_protocol(#next_protocol_negotiation{} = NextProtocols, true -> select_next_protocol(decode_next_protocols(NextProtocols), NextProtocolSelector); false -> - ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, unexpected_next_protocol_extension) + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, unexpected_next_protocol_extension)) end. handle_next_protocol_extension(NextProtocolNegotiation, Renegotiation, SslOpts)-> case handle_next_protocol_on_server(NextProtocolNegotiation, Renegotiation, SslOpts) of #alert{} = Alert -> - Alert; + throw(Alert); ProtocolsToAdvertise -> ProtocolsToAdvertise end. @@ -2428,14 +2416,14 @@ handle_renegotiation_info(_RecordCB, client, #renegotiation_info{renegotiated_co true -> {ok, ConnectionStates}; false -> - ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, client_renegotiation) + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, client_renegotiation)) end; handle_renegotiation_info(_RecordCB, server, #renegotiation_info{renegotiated_connection = ClientVerify}, ConnectionStates, true, _, CipherSuites) -> case is_member(?TLS_EMPTY_RENEGOTIATION_INFO_SCSV, CipherSuites) of true -> - ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, {server_renegotiation, empty_renegotiation_info_scsv}); + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, {server_renegotiation, empty_renegotiation_info_scsv})); false -> ConnectionState = ssl_record:current_connection_state(ConnectionStates, read), Data = maps:get(client_verify_data, ConnectionState), @@ -2443,7 +2431,7 @@ handle_renegotiation_info(_RecordCB, server, #renegotiation_info{renegotiated_co true -> {ok, ConnectionStates}; false -> - ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, server_renegotiation) + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, server_renegotiation)) end end; @@ -2453,7 +2441,7 @@ handle_renegotiation_info(RecordCB, client, undefined, ConnectionStates, true, S handle_renegotiation_info(RecordCB, server, undefined, ConnectionStates, true, SecureRenegotation, CipherSuites) -> case is_member(?TLS_EMPTY_RENEGOTIATION_INFO_SCSV, CipherSuites) of true -> - ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, {server_renegotiation, empty_renegotiation_info_scsv}); + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, {server_renegotiation, empty_renegotiation_info_scsv})); false -> handle_renegotiation_info(RecordCB, ConnectionStates, SecureRenegotation) end. @@ -2462,9 +2450,9 @@ handle_renegotiation_info(_RecordCB, ConnectionStates, SecureRenegotation) -> ConnectionState = ssl_record:current_connection_state(ConnectionStates, read), case {SecureRenegotation, maps:get(secure_renegotiation, ConnectionState)} of {_, true} -> - ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, already_secure); + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, already_secure)); {true, false} -> - ?ALERT_REC(?FATAL, ?NO_RENEGOTIATION); + throw(?ALERT_REC(?FATAL, ?NO_RENEGOTIATION)); {false, false} -> {ok, ConnectionStates} end. diff --git a/lib/ssl/src/ssl_internal.hrl b/lib/ssl/src/ssl_internal.hrl index 63e751440a..57c72aa122 100644 --- a/lib/ssl/src/ssl_internal.hrl +++ b/lib/ssl/src/ssl_internal.hrl @@ -31,8 +31,6 @@ -type reply() :: term(). -type msg() :: term(). -type from() :: term(). --type host() :: inet:ip_address() | inet:hostname(). --type session_id() :: 0 | binary(). -type certdb_ref() :: reference(). -type db_handle() :: term(). -type der_cert() :: binary(). @@ -111,10 +109,10 @@ %% Local policy for the server if it want's to reuse the session %% or not. Defaluts to allways returning true. %% fun(SessionId, PeerCert, Compression, CipherSuite) -> boolean() - reuse_session, + reuse_session :: fun() | binary() | undefined, %% Server side is a fun() %% If false sessions will never be reused, if true they %% will be reused if possible. - reuse_sessions :: boolean(), + reuse_sessions :: boolean() | save, %% Only client side can use value save renegotiate_at, secure_renegotiate, client_renegotiation, @@ -148,6 +146,8 @@ max_handshake_size :: integer(), handshake, customize_hostname_check + %% , + %% save_session :: boolean() }). -record(socket_options, diff --git a/lib/ssl/src/ssl_manager.erl b/lib/ssl/src/ssl_manager.erl index 4b735b2400..c56675b691 100644 --- a/lib/ssl/src/ssl_manager.erl +++ b/lib/ssl/src/ssl_manager.erl @@ -30,7 +30,7 @@ connection_init/3, cache_pem_file/2, lookup_trusted_cert/4, new_session_id/1, clean_cert_db/2, - register_session/2, register_session/3, invalidate_session/2, + register_session/2, register_session/4, invalidate_session/2, insert_crls/2, insert_crls/3, delete_crls/1, delete_crls/2, invalidate_session/3, name/1]). @@ -42,6 +42,8 @@ -include("ssl_handshake.hrl"). -include("ssl_internal.hrl"). +-include("ssl_api.hrl"). + -include_lib("kernel/include/file.hrl"). -record(state, { @@ -148,7 +150,7 @@ lookup_trusted_cert(DbHandle, Ref, SerialNumber, Issuer) -> ssl_pkix_db:lookup_trusted_cert(DbHandle, Ref, SerialNumber, Issuer). %%-------------------------------------------------------------------- --spec new_session_id(integer()) -> session_id(). +-spec new_session_id(integer()) -> ssl:session_id(). %% %% Description: Creates a session id for the server. %%-------------------------------------------------------------------- @@ -170,9 +172,11 @@ clean_cert_db(Ref, File) -> %% %% Description: Make the session available for reuse. %%-------------------------------------------------------------------- --spec register_session(host(), inet:port_number(), #session{}) -> ok. -register_session(Host, Port, Session) -> - cast({register_session, Host, Port, Session}). +-spec register_session(ssl:host(), inet:port_number(), #session{}, unique | true) -> ok. +register_session(Host, Port, Session, true) -> + call({register_session, Host, Port, Session}); +register_session(Host, Port, Session, unique = Save) -> + cast({register_session, Host, Port, Session, Save}). -spec register_session(inet:port_number(), #session{}) -> ok. register_session(Port, Session) -> @@ -183,7 +187,7 @@ register_session(Port, Session) -> %% a the session has been marked "is_resumable = false" for some while %% it will be safe to remove the data from the session database. %%-------------------------------------------------------------------- --spec invalidate_session(host(), inet:port_number(), #session{}) -> ok. +-spec invalidate_session(ssl:host(), inet:port_number(), #session{}) -> ok. invalidate_session(Host, Port, Session) -> load_mitigation(), cast({invalidate_session, Host, Port, Session}). @@ -301,7 +305,10 @@ handle_call({{new_session_id, Port}, _}, _, #state{session_cache_cb = CacheCb, session_cache_server = Cache} = State) -> Id = new_id(Port, ?GEN_UNIQUE_ID_MAX_TRIES, Cache, CacheCb), - {reply, Id, State}. + {reply, Id, State}; +handle_call({{register_session, Host, Port, Session},_}, _, State0) -> + State = client_register_session(Host, Port, Session, State0), + {reply, ok, State}. %%-------------------------------------------------------------------- -spec handle_cast(msg(), #state{}) -> {noreply, #state{}}. @@ -311,8 +318,12 @@ handle_call({{new_session_id, Port}, _}, %% %% Description: Handling cast messages %%-------------------------------------------------------------------- -handle_cast({register_session, Host, Port, Session}, State0) -> - State = ssl_client_register_session(Host, Port, Session, State0), +handle_cast({register_session, Host, Port, Session, unique}, State0) -> + State = client_register_unique_session(Host, Port, Session, State0), + {noreply, State}; + +handle_cast({register_session, Host, Port, Session, true}, State0) -> + State = client_register_session(Host, Port, Session, State0), {noreply, State}; handle_cast({register_session, Port, Session}, State0) -> @@ -540,10 +551,10 @@ clean_cert_db(Ref, CertDb, RefDb, FileMapDb, File) -> ok end. -ssl_client_register_session(Host, Port, Session, #state{session_cache_client = Cache, - session_cache_cb = CacheCb, - session_cache_client_max = Max, - session_client_invalidator = Pid0} = State) -> +client_register_unique_session(Host, Port, Session, #state{session_cache_client = Cache, + session_cache_cb = CacheCb, + session_cache_client_max = Max, + session_client_invalidator = Pid0} = State) -> TimeStamp = erlang:monotonic_time(), NewSession = Session#session{time_stamp = TimeStamp}, @@ -557,6 +568,17 @@ ssl_client_register_session(Host, Port, Session, #state{session_cache_client = C register_unique_session(Sessions, NewSession, {Host, Port}, State) end. +client_register_session(Host, Port, Session, #state{session_cache_client = Cache, + session_cache_cb = CacheCb, + session_cache_client_max = Max, + session_client_invalidator = Pid0} = State) -> + TimeStamp = erlang:monotonic_time(), + NewSession = Session#session{time_stamp = TimeStamp}, + Pid = do_register_session({{Host, Port}, + NewSession#session.session_id}, + NewSession, Max, Pid0, Cache, CacheCb), + State#state{session_client_invalidator = Pid}. + server_register_session(Port, Session, #state{session_cache_server_max = Max, session_cache_server = Cache, session_cache_cb = CacheCb, diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl index b9d1320ef3..1a36b2dba8 100644 --- a/lib/ssl/src/ssl_record.erl +++ b/lib/ssl/src/ssl_record.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2018. All Rights Reserved. +%% Copyright Ericsson AB 2013-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -45,14 +45,16 @@ -export([compress/3, uncompress/3, compressions/0]). %% Payload encryption/decryption --export([cipher/4, decipher/4, cipher_aead/4, decipher_aead/5, is_correct_mac/2, nonce_seed/3]). +-export([cipher/4, cipher/5, decipher/4, + cipher_aead/4, cipher_aead/5, decipher_aead/5, + is_correct_mac/2, nonce_seed/3]). -export_type([ssl_version/0, ssl_atom_version/0, connection_states/0, connection_state/0]). -type ssl_version() :: {integer(), integer()}. -type ssl_atom_version() :: tls_record:tls_atom_version(). --type connection_states() :: term(). %% Map --type connection_state() :: term(). %% Map +-type connection_states() :: map(). %% Map +-type connection_state() :: map(). %% Map %%==================================================================== %% Connection state handling @@ -302,27 +304,49 @@ cipher(Version, Fragment, #security_parameters{bulk_cipher_algorithm = BulkCipherAlgo} } = WriteState0, MacHash) -> - + %% {CipherFragment, CipherS1} = ssl_cipher:cipher(BulkCipherAlgo, CipherS0, MacHash, Fragment, Version), {CipherFragment, WriteState0#{cipher_state => CipherS1}}. + +%%-------------------------------------------------------------------- +-spec cipher(ssl_version(), iodata(), #cipher_state{}, MacHash::binary(), #security_parameters{}) -> + {CipherFragment::binary(), #cipher_state{}}. +%% +%% Description: Payload encryption +%%-------------------------------------------------------------------- +cipher(Version, Fragment, CipherS0, MacHash, + #security_parameters{bulk_cipher_algorithm = BulkCipherAlgo}) -> + %% + ssl_cipher:cipher(BulkCipherAlgo, CipherS0, MacHash, Fragment, Version). + %%-------------------------------------------------------------------- -spec cipher_aead(ssl_version(), iodata(), connection_state(), AAD::binary()) -> {CipherFragment::binary(), connection_state()}. %% Description: Payload encryption %% %%-------------------------------------------------------------------- -cipher_aead(Version, Fragment, +cipher_aead(_Version, Fragment, #{cipher_state := CipherS0, security_parameters := #security_parameters{bulk_cipher_algorithm = BulkCipherAlgo} } = WriteState0, AAD) -> {CipherFragment, CipherS1} = - cipher_aead(BulkCipherAlgo, CipherS0, AAD, Fragment, Version), + do_cipher_aead(BulkCipherAlgo, Fragment, CipherS0, AAD), {CipherFragment, WriteState0#{cipher_state => CipherS1}}. %%-------------------------------------------------------------------- +-spec cipher_aead(ssl_version(), iodata(), #cipher_state{}, AAD::binary(), #security_parameters{}) -> + {CipherFragment::binary(), #cipher_state{}}. + +%% Description: Payload encryption +%% %%-------------------------------------------------------------------- +cipher_aead(_Version, Fragment, CipherS, AAD, + #security_parameters{bulk_cipher_algorithm = BulkCipherAlgo}) -> + do_cipher_aead(BulkCipherAlgo, Fragment, CipherS, AAD). + +%%-------------------------------------------------------------------- -spec decipher(ssl_version(), binary(), connection_state(), boolean()) -> {binary(), binary(), connection_state()} | #alert{}. %% @@ -343,9 +367,8 @@ decipher(Version, CipherFragment, Alert end. %%-------------------------------------------------------------------- --spec decipher_aead(ssl_cipher:cipher_enum(), #cipher_state{}, - binary(), binary(), ssl_record:ssl_version()) -> - {binary(), #cipher_state{}} | #alert{}. +-spec decipher_aead(ssl_cipher:cipher_enum(), #cipher_state{}, binary(), binary(), ssl_record:ssl_version()) -> + binary() | #alert{}. %% %% Description: Decrypts the data and checks the associated data (AAD) MAC using %% cipher described by cipher_enum() and updating the cipher state. @@ -357,7 +380,7 @@ decipher_aead(Type, #cipher_state{key = Key} = CipherState, AAD0, CipherFragment {AAD, CipherText, CipherTag} = aead_ciphertext_split(Type, CipherState, CipherFragment, AAD0), case ssl_cipher:aead_decrypt(Type, Key, Nonce, CipherText, CipherTag, AAD) of Content when is_binary(Content) -> - {Content, CipherState}; + Content; _ -> ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC, decryption_failed) end @@ -399,11 +422,13 @@ random() -> Random_28_bytes = ssl_cipher:random_bytes(28), <<?UINT32(Secs_since_1970), Random_28_bytes/binary>>. +-compile({inline, [is_correct_mac/2]}). is_correct_mac(Mac, Mac) -> true; is_correct_mac(_M,_H) -> false. +-compile({inline, [record_protocol_role/1]}). record_protocol_role(client) -> ?CLIENT; record_protocol_role(server) -> @@ -427,13 +452,15 @@ initial_security_params(ConnectionEnd) -> compression_algorithm = ?NULL}, ssl_cipher:security_parameters(?TLS_NULL_WITH_NULL_NULL, SecParams). -cipher_aead(?CHACHA20_POLY1305 = Type, #cipher_state{key=Key} = CipherState, AAD0, Fragment, _Version) -> - AAD = end_additional_data(AAD0, erlang:iolist_size(Fragment)), +-define(end_additional_data(AAD, Len), << (begin(AAD)end)/binary, ?UINT16(begin(Len)end) >>). + +do_cipher_aead(?CHACHA20_POLY1305 = Type, Fragment, #cipher_state{key=Key} = CipherState, AAD0) -> + AAD = ?end_additional_data(AAD0, erlang:iolist_size(Fragment)), Nonce = encrypt_nonce(Type, CipherState), {Content, CipherTag} = ssl_cipher:aead_encrypt(Type, Key, Nonce, Fragment, AAD), {<<Content/binary, CipherTag/binary>>, CipherState}; -cipher_aead(Type, #cipher_state{key=Key, nonce = ExplicitNonce} = CipherState, AAD0, Fragment, _Version) -> - AAD = end_additional_data(AAD0, erlang:iolist_size(Fragment)), +do_cipher_aead(Type, Fragment, #cipher_state{key=Key, nonce = ExplicitNonce} = CipherState, AAD0) -> + AAD = ?end_additional_data(AAD0, erlang:iolist_size(Fragment)), Nonce = encrypt_nonce(Type, CipherState), {Content, CipherTag} = ssl_cipher:aead_encrypt(Type, Key, Nonce, Fragment, AAD), {<<ExplicitNonce:64/integer, Content/binary, CipherTag/binary>>, CipherState#cipher_state{nonce = ExplicitNonce + 1}}. @@ -449,15 +476,12 @@ decrypt_nonce(?CHACHA20_POLY1305, #cipher_state{nonce = Nonce, iv = IV}, _) -> decrypt_nonce(?AES_GCM, #cipher_state{iv = <<Salt:4/bytes, _/binary>>}, <<ExplicitNonce:8/bytes, _/binary>>) -> <<Salt/binary, ExplicitNonce/binary>>. +-compile({inline, [aead_ciphertext_split/4]}). aead_ciphertext_split(?CHACHA20_POLY1305, #cipher_state{tag_len = Len}, CipherTextFragment, AAD) -> - CipherLen = size(CipherTextFragment) - Len, + CipherLen = byte_size(CipherTextFragment) - Len, <<CipherText:CipherLen/bytes, CipherTag:Len/bytes>> = CipherTextFragment, - {end_additional_data(AAD, CipherLen), CipherText, CipherTag}; + {?end_additional_data(AAD, CipherLen), CipherText, CipherTag}; aead_ciphertext_split(?AES_GCM, #cipher_state{tag_len = Len}, CipherTextFragment, AAD) -> - CipherLen = size(CipherTextFragment) - (Len + 8), %% 8 is length of explicit Nonce + CipherLen = byte_size(CipherTextFragment) - (Len + 8), %% 8 is length of explicit Nonce << _:8/bytes, CipherText:CipherLen/bytes, CipherTag:Len/bytes>> = CipherTextFragment, - {end_additional_data(AAD, CipherLen), CipherText, CipherTag}. - -end_additional_data(AAD, Len) -> - <<AAD/binary, ?UINT16(Len)>>. - + {?end_additional_data(AAD, CipherLen), CipherText, CipherTag}. diff --git a/lib/ssl/src/ssl_record.hrl b/lib/ssl/src/ssl_record.hrl index ed007f58d7..a927fba0de 100644 --- a/lib/ssl/src/ssl_record.hrl +++ b/lib/ssl/src/ssl_record.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -140,6 +140,8 @@ -define(ALERT, 21). -define(HANDSHAKE, 22). -define(APPLICATION_DATA, 23). +-define(KNOWN_RECORD_TYPE(Type), + (is_integer(Type) andalso (20 =< (Type)) andalso ((Type) =< 23))). -define(MAX_PLAIN_TEXT_LENGTH, 16384). -define(MAX_COMPRESSED_LENGTH, (?MAX_PLAIN_TEXT_LENGTH+1024)). -define(MAX_CIPHER_TEXT_LENGTH, (?MAX_PLAIN_TEXT_LENGTH+2048)). diff --git a/lib/ssl/src/ssl_session.erl b/lib/ssl/src/ssl_session.erl index c9607489e9..44305c65fe 100644 --- a/lib/ssl/src/ssl_session.erl +++ b/lib/ssl/src/ssl_session.erl @@ -27,6 +27,7 @@ -include("ssl_handshake.hrl"). -include("ssl_internal.hrl"). +-include("ssl_api.hrl"). %% Internal application API -export([is_new/2, client_id/4, server_id/6, valid_session/2]). @@ -34,7 +35,7 @@ -type seconds() :: integer(). %%-------------------------------------------------------------------- --spec is_new(session_id(), session_id()) -> boolean(). +-spec is_new(ssl:session_id(), ssl:session_id()) -> boolean(). %% %% Description: Checks if the session id decided by the server is a %% new or resumed sesion id. @@ -47,12 +48,19 @@ is_new(_ClientSuggestion, _ServerDecision) -> true. %%-------------------------------------------------------------------- --spec client_id({host(), inet:port_number(), #ssl_options{}}, db_handle(), atom(), +-spec client_id({ssl:host(), inet:port_number(), #ssl_options{}}, db_handle(), atom(), undefined | binary()) -> binary(). %% %% Description: Should be called by the client side to get an id %% for the client hello message. %%-------------------------------------------------------------------- +client_id({Host, Port, #ssl_options{reuse_session = SessionId}}, Cache, CacheCb, _) when is_binary(SessionId)-> + case CacheCb:lookup(Cache, {{Host, Port}, SessionId}) of + undefined -> + <<>>; + #session{} -> + SessionId + end; client_id(ClientInfo, Cache, CacheCb, OwnCert) -> case select_session(ClientInfo, Cache, CacheCb, OwnCert) of no_session -> @@ -91,7 +99,8 @@ server_id(Port, SuggestedId, Options, Cert, Cache, CacheCb) -> %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- -select_session({_, _, #ssl_options{reuse_sessions=false}}, _Cache, _CacheCb, _OwnCert) -> +select_session({_, _, #ssl_options{reuse_sessions = Reuse}}, _Cache, _CacheCb, _OwnCert) when Reuse =/= true -> + %% If reuse_sessions == true | save a new session should be created no_session; select_session({HostIP, Port, SslOpts}, Cache, CacheCb, OwnCert) -> Sessions = CacheCb:select_session(Cache, {HostIP, Port}), @@ -132,7 +141,7 @@ is_resumable(SuggestedSessionId, Port, #ssl_options{reuse_session = ReuseFun} = false -> {false, undefined} end; undefined -> - {false, undefined} + {false, undefined} end. resumable(new) -> diff --git a/lib/ssl/src/ssl_session_cache_api.erl b/lib/ssl/src/ssl_session_cache_api.erl index b68c75a09b..5f96f905b1 100644 --- a/lib/ssl/src/ssl_session_cache_api.erl +++ b/lib/ssl/src/ssl_session_cache_api.erl @@ -23,14 +23,20 @@ -module(ssl_session_cache_api). -include("ssl_handshake.hrl"). -include("ssl_internal.hrl"). +-include("ssl_api.hrl"). --type key() :: {{host(), inet:port_number()}, session_id()} | {inet:port_number(), session_id()}. +-export_type([session_cache_key/0, session/0, partial_key/0, session_cache_ref/0]). --callback init(list()) -> db_handle(). --callback terminate(db_handle()) -> any(). --callback lookup(db_handle(), key()) -> #session{} | undefined. --callback update(db_handle(), key(), #session{}) -> any(). --callback delete(db_handle(), key()) -> any(). --callback foldl(fun(), term(), db_handle()) -> term(). --callback select_session(db_handle(), {host(), inet:port_number()} | inet:port_number()) -> [#session{}]. --callback size(db_handle()) -> integer(). +-type session_cache_ref() :: any(). +-type session_cache_key() :: {partial_key(), ssl:session_id()}. +-opaque session() :: #session{}. +-opaque partial_key() :: {ssl:host(), inet:port_number()} | inet:port_number(). + +-callback init(list()) -> session_cache_ref(). +-callback terminate(session_cache_ref()) -> any(). +-callback lookup(session_cache_ref(), session_cache_key()) -> #session{} | undefined. +-callback update(session_cache_ref(), session_cache_key(), #session{}) -> any(). +-callback delete(session_cache_ref(), session_cache_key()) -> any(). +-callback foldl(fun(), term(), session_cache_ref()) -> term(). +-callback select_session(session_cache_ref(), {ssl:host(), inet:port_number()} | inet:port_number()) -> [#session{}]. +-callback size(session_cache_ref()) -> integer(). diff --git a/lib/ssl/src/tls_connection.erl b/lib/ssl/src/tls_connection.erl index 9edf48fdef..3229004c9d 100644 --- a/lib/ssl/src/tls_connection.erl +++ b/lib/ssl/src/tls_connection.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2018. All Rights Reserved. +%% Copyright Ericsson AB 2007-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -57,11 +57,10 @@ %% Alert and close handling -export([send_alert/2, send_alert_in_connection/2, send_sync_alert/2, - encode_alert/3, close/5, protocol_name/0]). + close/5, protocol_name/0]). %% Data handling --export([encode_data/3, next_record/1, - send/3, socket/5, setopts/3, getopts/3]). +-export([next_record/1, socket/4, setopts/3, getopts/3]). %% gen_statem state functions -export([init/3, error/3, downgrade/3, %% Initiation and take down states @@ -108,7 +107,7 @@ start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = true},_, Tracker} = end. %%-------------------------------------------------------------------- --spec start_link(atom(), pid(), host(), inet:port_number(), port(), list(), pid(), tuple()) -> +-spec start_link(atom(), pid(), ssl:host(), inet:port_number(), port(), list(), pid(), tuple()) -> {ok, pid()} | ignore | {error, reason()}. %% %% Description: Creates a gen_statem process which calls Module:init/1 to @@ -143,23 +142,17 @@ pids(#state{protocol_specific = #{sender := Sender}}) -> %%==================================================================== %% State transition handling %%==================================================================== -next_record(#state{unprocessed_handshake_events = N} = State) when N > 0 -> - {no_record, State#state{unprocessed_handshake_events = N-1}}; - +next_record(#state{handshake_env = + #handshake_env{unprocessed_handshake_events = N} = HsEnv} + = State) when N > 0 -> + {no_record, State#state{handshake_env = + HsEnv#handshake_env{unprocessed_handshake_events = N-1}}}; next_record(#state{protocol_buffers = - #protocol_buffers{tls_packets = [], tls_cipher_texts = [CT | Rest]} - = Buffers, - connection_states = ConnStates0, - ssl_options = #ssl_options{padding_check = Check}} = State) -> - case tls_record:decode_cipher_text(CT, ConnStates0, Check) of - {Plain, ConnStates} -> - {Plain, State#state{protocol_buffers = - Buffers#protocol_buffers{tls_cipher_texts = Rest}, - connection_states = ConnStates}}; - #alert{} = Alert -> - {Alert, State} - end; -next_record(#state{protocol_buffers = #protocol_buffers{tls_packets = [], tls_cipher_texts = []}, + #protocol_buffers{tls_cipher_texts = [_|_] = CipherTexts}, + connection_states = ConnectionStates, + ssl_options = #ssl_options{padding_check = Check}} = State) -> + next_record(State, CipherTexts, ConnectionStates, Check); +next_record(#state{protocol_buffers = #protocol_buffers{tls_cipher_texts = []}, protocol_specific = #{active_n_toggle := true, active_n := N} = ProtocolSpec, static_env = #static_env{socket = Socket, close_tag = CloseTag, @@ -175,16 +168,48 @@ next_record(#state{protocol_buffers = #protocol_buffers{tls_packets = [], tls_ci next_record(State) -> {no_record, State}. +%% Decipher next record and concatenate consecutive ?APPLICATION_DATA records into one +%% +next_record(State, CipherTexts, ConnectionStates, Check) -> + next_record(State, CipherTexts, ConnectionStates, Check, []). +%% +next_record(State, [#ssl_tls{type = ?APPLICATION_DATA} = CT|CipherTexts], ConnectionStates0, Check, Acc) -> + case tls_record:decode_cipher_text(CT, ConnectionStates0, Check) of + {#ssl_tls{fragment = Fragment}, ConnectionStates} -> + next_record(State, CipherTexts, ConnectionStates, Check, [Fragment|Acc]); + #alert{} = Alert -> + Alert + end; +next_record(State, [CT|CipherTexts], ConnectionStates0, Check, []) -> + case tls_record:decode_cipher_text(CT, ConnectionStates0, Check) of + {Record, ConnectionStates} -> + next_record_done(State, CipherTexts, ConnectionStates, Record); + #alert{} = Alert -> + Alert + end; +next_record(State, CipherTexts, ConnectionStates, _Check, Acc) -> + %% Not ?APPLICATION_DATA but we have a nonempty Acc + %% -> build an ?APPLICATION_DATA record with the accumulated fragments + next_record_done(State, CipherTexts, ConnectionStates, + #ssl_tls{type = ?APPLICATION_DATA, fragment = iolist_to_binary(lists:reverse(Acc))}). + +next_record_done(#state{protocol_buffers = Buffers} = State, CipherTexts, ConnectionStates, Record) -> + {Record, + State#state{protocol_buffers = Buffers#protocol_buffers{tls_cipher_texts = CipherTexts}, + connection_states = ConnectionStates}}. + + next_event(StateName, Record, State) -> next_event(StateName, Record, State, []). +%% next_event(StateName, no_record, State0, Actions) -> case next_record(State0) of {no_record, State} -> {next_state, StateName, State, Actions}; {#ssl_tls{} = Record, State} -> {next_state, StateName, State, [{next_event, internal, {protocol_record, Record}} | Actions]}; - {#alert{} = Alert, State} -> - {next_state, StateName, State, [{next_event, internal, Alert} | Actions]} + #alert{} = Alert -> + {next_state, StateName, State0, [{next_event, internal, Alert} | Actions]} end; next_event(StateName, Record, State, Actions) -> case Record of @@ -196,6 +221,7 @@ next_event(StateName, Record, State, Actions) -> {next_state, StateName, State, [{next_event, internal, Alert} | Actions]} end. + %%% TLS record protocol level application data messages handle_protocol_record(#ssl_tls{type = ?APPLICATION_DATA, fragment = Data}, StateName, State0) -> @@ -210,7 +236,7 @@ handle_protocol_record(#ssl_tls{type = ?APPLICATION_DATA, fragment = Data}, Stat handle_protocol_record(#ssl_tls{type = ?HANDSHAKE, fragment = Data}, StateName, #state{protocol_buffers = #protocol_buffers{tls_handshake_buffer = Buf0} = Buffers, - negotiated_version = Version, + connection_env = #connection_env{negotiated_version = Version}, ssl_options = Options} = State0) -> try {Packets, Buf} = tls_handshake:get_tls_handshake(Version,Data,Buf0, Options), @@ -227,8 +253,12 @@ handle_protocol_record(#ssl_tls{type = ?HANDSHAKE, fragment = Data}, connection -> ssl_connection:hibernate_after(StateName, State, Events); _ -> + HsEnv = State#state.handshake_env, {next_state, StateName, - State#state{unprocessed_handshake_events = unprocessed_events(Events)}, Events} + State#state{protocol_buffers = Buffers, + handshake_env = + HsEnv#handshake_env{unprocessed_handshake_events + = unprocessed_events(Events)}}, Events} end end catch throw:#alert{} = Alert -> @@ -239,7 +269,7 @@ handle_protocol_record(#ssl_tls{type = ?CHANGE_CIPHER_SPEC, fragment = Data}, St {next_state, StateName, State, [{next_event, internal, #change_cipher_spec{type = Data}}]}; %%% TLS record protocol level Alert messages handle_protocol_record(#ssl_tls{type = ?ALERT, fragment = EncAlerts}, StateName, - #state{negotiated_version = Version} = State) -> + #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> try decode_alerts(EncAlerts) of Alerts = [_|_] -> handle_alerts(Alerts, {next_state, StateName, State}); @@ -263,69 +293,71 @@ handle_protocol_record(#ssl_tls{type = _Unknown}, StateName, State) -> renegotiation(Pid, WriteState) -> gen_statem:call(Pid, {user_renegotiate, WriteState}). -renegotiate(#state{static_env = #static_env{role = client}} = State, Actions) -> +renegotiate(#state{static_env = #static_env{role = client}, + handshake_env = HsEnv} = State, Actions) -> %% Handle same way as if server requested %% the renegotiation Hs0 = ssl_handshake:init_handshake_history(), - {next_state, connection, State#state{tls_handshake_history = Hs0}, + {next_state, connection, State#state{handshake_env = HsEnv#handshake_env{tls_handshake_history = Hs0}}, [{next_event, internal, #hello_request{}} | Actions]}; renegotiate(#state{static_env = #static_env{role = server, socket = Socket, transport_cb = Transport}, - negotiated_version = Version, + handshake_env = HsEnv, + connection_env = #connection_env{negotiated_version = Version}, connection_states = ConnectionStates0} = State0, Actions) -> HelloRequest = ssl_handshake:hello_request(), Frag = tls_handshake:encode_handshake(HelloRequest, Version), Hs0 = ssl_handshake:init_handshake_history(), {BinMsg, ConnectionStates} = tls_record:encode_handshake(Frag, Version, ConnectionStates0), - send(Transport, Socket, BinMsg), + tls_socket:send(Transport, Socket, BinMsg), State = State0#state{connection_states = ConnectionStates, - tls_handshake_history = Hs0}, + handshake_env = HsEnv#handshake_env{tls_handshake_history = Hs0}}, next_event(hello, no_record, State, Actions). send_handshake(Handshake, State) -> send_handshake_flight(queue_handshake(Handshake, State)). -queue_handshake(Handshake, #state{negotiated_version = Version, - tls_handshake_history = Hist0, - flight_buffer = Flight0, +queue_handshake(Handshake, #state{handshake_env = #handshake_env{tls_handshake_history = Hist0} = HsEnv, + connection_env = #connection_env{negotiated_version = Version}, + flight_buffer = Flight0, connection_states = ConnectionStates0} = State0) -> {BinHandshake, ConnectionStates, Hist} = encode_handshake(Handshake, Version, ConnectionStates0, Hist0), State0#state{connection_states = ConnectionStates, - tls_handshake_history = Hist, + handshake_env = HsEnv#handshake_env{tls_handshake_history = Hist}, flight_buffer = Flight0 ++ [BinHandshake]}. send_handshake_flight(#state{static_env = #static_env{socket = Socket, transport_cb = Transport}, flight_buffer = Flight} = State0) -> - send(Transport, Socket, Flight), + tls_socket:send(Transport, Socket, Flight), {State0#state{flight_buffer = []}, []}. -queue_change_cipher(Msg, #state{negotiated_version = Version, - flight_buffer = Flight0, - connection_states = ConnectionStates0} = State0) -> +queue_change_cipher(Msg, #state{connection_env = #connection_env{negotiated_version = Version}, + flight_buffer = Flight0, + connection_states = ConnectionStates0} = State0) -> {BinChangeCipher, ConnectionStates} = encode_change_cipher(Msg, Version, ConnectionStates0), State0#state{connection_states = ConnectionStates, flight_buffer = Flight0 ++ [BinChangeCipher]}. reinit(#state{protocol_specific = #{sender := Sender}, - negotiated_version = Version, + connection_env = #connection_env{negotiated_version = Version}, connection_states = #{current_write := Write}} = State) -> tls_sender:update_connection_state(Sender, Write, Version), reinit_handshake_data(State). -reinit_handshake_data(State) -> +reinit_handshake_data(#state{handshake_env = HsEnv} =State) -> %% premaster_secret, public_key_info and tls_handshake_info %% are only needed during the handshake phase. %% To reduce memory foot print of a connection reinitialize them. State#state{ - premaster_secret = undefined, - public_key_info = undefined, - tls_handshake_history = ssl_handshake:init_handshake_history() + handshake_env = HsEnv#handshake_env{tls_handshake_history = ssl_handshake:init_handshake_history(), + public_key_info = undefined, + premaster_secret = undefined} }. select_sni_extension(#client_hello{extensions = HelloExtensions}) -> @@ -348,13 +380,13 @@ empty_connection_state(ConnectionEnd, BeastMitigation) -> encode_alert(#alert{} = Alert, Version, ConnectionStates) -> tls_record:encode_alert_record(Alert, Version, ConnectionStates). -send_alert(Alert, #state{negotiated_version = Version, - static_env = #static_env{socket = Socket, +send_alert(Alert, #state{static_env = #static_env{socket = Socket, transport_cb = Transport}, + connection_env = #connection_env{negotiated_version = Version}, connection_states = ConnectionStates0} = StateData0) -> {BinMsg, ConnectionStates} = encode_alert(Alert, Version, ConnectionStates0), - send(Transport, Socket, BinMsg), + tls_socket:send(Transport, Socket, BinMsg), StateData0#state{connection_states = ConnectionStates}. %% If an ALERT sent in the connection state, should cause the TLS @@ -408,14 +440,9 @@ protocol_name() -> %%==================================================================== %% Data handling %%==================================================================== -encode_data(Data, Version, ConnectionStates0)-> - tls_record:encode_data(Data, Version, ConnectionStates0). - -send(Transport, Socket, Data) -> - tls_socket:send(Transport, Socket, Data). -socket(Pids, Transport, Socket, Connection, Tracker) -> - tls_socket:socket(Pids, Transport, Socket, Connection, Tracker). +socket(Pids, Transport, Socket, Tracker) -> + tls_socket:socket(Pids, Transport, Socket, ?MODULE, Tracker). setopts(Transport, Socket, Other) -> tls_socket:setopts(Transport, Socket, Other). @@ -440,12 +467,12 @@ init({call, From}, {start, Timeout}, socket = Socket, session_cache = Cache, session_cache_cb = CacheCb}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}} = HsEnv, + connection_env = CEnv, ssl_options = SslOpts, session = #session{own_certificate = Cert} = Session0, - connection_states = ConnectionStates0, - renegotiation = {Renegotiation, _} + connection_states = ConnectionStates0 } = State0) -> - Timer = ssl_connection:start_or_recv_cancel_timer(Timeout, From), Hello = tls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, Cache, CacheCb, Renegotiation, Cert), @@ -454,15 +481,14 @@ init({call, From}, {start, Timeout}, Handshake0 = ssl_handshake:init_handshake_history(), {BinMsg, ConnectionStates, Handshake} = encode_handshake(Hello, HelloVersion, ConnectionStates0, Handshake0), - send(Transport, Socket, BinMsg), + tls_socket:send(Transport, Socket, BinMsg), State = State0#state{connection_states = ConnectionStates, - negotiated_version = Version, %% Requested version + connection_env = CEnv#connection_env{negotiated_version = Version}, %% Requested version session = Session0#session{session_id = Hello#client_hello.session_id}, - tls_handshake_history = Handshake, - start_or_recv_from = From, - timer = Timer}, - next_event(hello, no_record, State); + handshake_env = HsEnv#handshake_env{tls_handshake_history = Handshake}, + start_or_recv_from = From}, + next_event(hello, no_record, State, [{{timeout, handshake}, Timeout, close}]); init(Type, Event, State) -> gen_handshake(?FUNCTION_NAME, Type, Event, State). @@ -489,15 +515,17 @@ error(_, _, _) -> %%-------------------------------------------------------------------- hello(internal, #client_hello{extensions = Extensions} = Hello, #state{ssl_options = #ssl_options{handshake = hello}, + handshake_env = HsEnv, start_or_recv_from = From} = State) -> {next_state, user_hello, State#state{start_or_recv_from = undefined, - hello = Hello}, + handshake_env = HsEnv#handshake_env{hello = Hello}}, [{reply, From, {ok, ssl_connection:map_extensions(Extensions)}}]}; hello(internal, #server_hello{extensions = Extensions} = Hello, #state{ssl_options = #ssl_options{handshake = hello}, + handshake_env = HsEnv, start_or_recv_from = From} = State) -> {next_state, user_hello, State#state{start_or_recv_from = undefined, - hello = Hello}, + handshake_env = HsEnv#handshake_env{hello = Hello}}, [{reply, From, {ok, ssl_connection:map_extensions(Extensions)}}]}; hello(internal, #client_hello{client_version = ClientVersion} = Hello, #state{connection_states = ConnectionStates0, @@ -505,17 +533,18 @@ hello(internal, #client_hello{client_version = ClientVersion} = Hello, port = Port, session_cache = Cache, session_cache_cb = CacheCb}, + handshake_env = #handshake_env{kex_algorithm = KeyExAlg, + renegotiation = {Renegotiation, _}, + negotiated_protocol = CurrentProtocol} = HsEnv, + connection_env = CEnv, session = #session{own_certificate = Cert} = Session0, - renegotiation = {Renegotiation, _}, - negotiated_protocol = CurrentProtocol, - key_algorithm = KeyExAlg, ssl_options = SslOpts} = State) -> case tls_handshake:hello(Hello, SslOpts, {Port, Session0, Cache, CacheCb, ConnectionStates0, Cert, KeyExAlg}, Renegotiation) of #alert{} = Alert -> ssl_connection:handle_own_alert(Alert, ClientVersion, hello, - State#state{negotiated_version - = ClientVersion}); + State#state{connection_env = + CEnv#connection_env{negotiated_version = ClientVersion}}); {Version, {Type, Session}, ConnectionStates, Protocol0, ServerHelloExt, HashSign} -> Protocol = case Protocol0 of @@ -524,22 +553,24 @@ hello(internal, #client_hello{client_version = ClientVersion} = Hello, end, gen_handshake(?FUNCTION_NAME, internal, {common_client_hello, Type, ServerHelloExt}, State#state{connection_states = ConnectionStates, - negotiated_version = Version, - hashsign_algorithm = HashSign, - client_hello_version = ClientVersion, - session = Session, - negotiated_protocol = Protocol}) + connection_env = CEnv#connection_env{negotiated_version = Version}, + handshake_env = HsEnv#handshake_env{ + hashsign_algorithm = HashSign, + client_hello_version = ClientVersion, + negotiated_protocol = Protocol}, + session = Session + }) end; hello(internal, #server_hello{} = Hello, #state{connection_states = ConnectionStates0, - negotiated_version = ReqVersion, + connection_env = #connection_env{negotiated_version = ReqVersion} = CEnv, static_env = #static_env{role = client}, - renegotiation = {Renegotiation, _}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, ssl_options = SslOptions} = State) -> case tls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation) of - #alert{} = Alert -> + #alert{} = Alert -> %%TODO ssl_connection:handle_own_alert(Alert, ReqVersion, hello, - State#state{negotiated_version = ReqVersion}); + State#state{connection_env = CEnv#connection_env{negotiated_version = ReqVersion}}); {Version, NewId, ConnectionStates, ProtoExt, Protocol} -> ssl_connection:handle_session(Hello, Version, NewId, ConnectionStates, ProtoExt, Protocol, State) @@ -592,13 +623,16 @@ connection({call, From}, {user_renegotiate, WriteState}, [{next_event,{call, From}, renegotiate}]}; connection({call, From}, {close, {Pid, _Timeout}}, - #state{terminated = closed} = State) -> - {next_state, downgrade, State#state{terminated = true, downgrade = {Pid, From}}, + #state{connection_env = #connection_env{terminated = closed} =CEnv} = State) -> + {next_state, downgrade, State#state{connection_env = + CEnv#connection_env{terminated = true, + downgrade = {Pid, From}}}, [{next_event, internal, ?ALERT_REC(?WARNING, ?CLOSE_NOTIFY)}]}; connection({call, From}, {close,{Pid, Timeout}}, #state{connection_states = ConnectionStates, - protocol_specific = #{sender := Sender} + protocol_specific = #{sender := Sender}, + connection_env = CEnv } = State0) -> case tls_sender:downgrade(Sender, Timeout) of {ok, Write} -> @@ -609,8 +643,10 @@ connection({call, From}, State = send_alert(?ALERT_REC(?WARNING, ?CLOSE_NOTIFY), State0#state{connection_states = ConnectionStates#{current_write => Write}}), - {next_state, downgrade, State#state{downgrade = {Pid, From}, - terminated = true}, [{timeout, Timeout, downgrade}]}; + {next_state, downgrade, State#state{connection_env = + CEnv#connection_env{downgrade = {Pid, From}, + terminated = true}}, + [{timeout, Timeout, downgrade}]}; {error, timeout} -> {stop_and_reply, {shutdown, downgrade_fail}, [{reply, From, {error, timeout}}]} end; @@ -620,7 +656,7 @@ connection(internal, #hello_request{}, port = Port, session_cache = Cache, session_cache_cb = CacheCb}, - renegotiation = {Renegotiation, peer}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, peer}}, session = #session{own_certificate = Cert} = Session0, ssl_options = SslOpts, protocol_specific = #{sender := Pid}, @@ -642,7 +678,7 @@ connection(internal, #hello_request{}, port = Port, session_cache = Cache, session_cache_cb = CacheCb}, - renegotiation = {Renegotiation, _}, + handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, session = #session{own_certificate = Cert} = Session0, ssl_options = SslOpts, connection_states = ConnectionStates} = State0) -> @@ -653,7 +689,7 @@ connection(internal, #hello_request{}, = Hello#client_hello.session_id}}, Actions); connection(internal, #client_hello{} = Hello, #state{static_env = #static_env{role = server}, - allow_renegotiate = true, + handshake_env = #handshake_env{allow_renegotiate = true}= HsEnv, connection_states = CS, protocol_specific = #{sender := Sender} } = State) -> @@ -665,17 +701,16 @@ connection(internal, #client_hello{} = Hello, erlang:send_after(?WAIT_TO_ALLOW_RENEGOTIATION, self(), allow_renegotiate), {ok, Write} = tls_sender:renegotiate(Sender), next_event(hello, no_record, State#state{connection_states = CS#{current_write => Write}, - allow_renegotiate = false, - renegotiation = {true, peer} + handshake_env = HsEnv#handshake_env{renegotiation = {true, peer}, + allow_renegotiate = false} }, [{next_event, internal, Hello}]); connection(internal, #client_hello{}, - #state{static_env = #static_env{role = server, - protocol_cb = Connection}, - allow_renegotiate = false} = State0) -> + #state{static_env = #static_env{role = server}, + handshake_env = #handshake_env{allow_renegotiate = false}} = State0) -> Alert = ?ALERT_REC(?WARNING, ?NO_RENEGOTIATION), send_alert_in_connection(Alert, State0), - State = Connection:reinit_handshake_data(State0), + State = reinit_handshake_data(State0), next_event(?FUNCTION_NAME, no_record, State); connection(Type, Event, State) -> @@ -688,15 +723,16 @@ connection(Type, Event, State) -> downgrade(internal, #alert{description = ?CLOSE_NOTIFY}, #state{static_env = #static_env{transport_cb = Transport, socket = Socket}, - downgrade = {Pid, From}} = State) -> + connection_env = #connection_env{downgrade = {Pid, From}}} = State) -> tls_socket:setopts(Transport, Socket, [{active, false}, {packet, 0}, {mode, binary}]), Transport:controlling_process(Socket, Pid), {stop_and_reply, {shutdown, downgrade},[{reply, From, {ok, Socket}}], State}; -downgrade(timeout, downgrade, #state{downgrade = {_, From}} = State) -> +downgrade(timeout, downgrade, #state{ connection_env = #connection_env{downgrade = {_, From}}} = State) -> {stop_and_reply, {shutdown, normal},[{reply, From, {error, timeout}}], State}; downgrade(info, {CloseTag, Socket}, #state{static_env = #static_env{socket = Socket, - close_tag = CloseTag}, downgrade = {_, From}} = + close_tag = CloseTag}, + connection_env = #connection_env{downgrade = {_, From}}} = State) -> {stop_and_reply, {shutdown, normal},[{reply, From, {error, CloseTag}}], State}; downgrade(info, Info, State) -> @@ -762,15 +798,18 @@ initial_state(Role, Sender, Host, Port, Socket, {SSLOptions, SocketOptions, Trac }, #state{ static_env = InitStatEnv, + handshake_env = #handshake_env{ + tls_handshake_history = ssl_handshake:init_handshake_history(), + renegotiation = {false, first}, + allow_renegotiate = SSLOptions#ssl_options.client_renegotiation + }, + connection_env = #connection_env{user_application = {UserMonitor, User}}, socket_options = SocketOptions, ssl_options = SSLOptions, session = #session{is_resumable = new}, connection_states = ConnectionStates, protocol_buffers = #protocol_buffers{}, - user_application = {UserMonitor, User}, - user_data_buffer = <<>>, - renegotiation = {false, first}, - allow_renegotiate = SSLOptions#ssl_options.client_renegotiation, + user_data_buffer = {[],0,[]}, start_or_recv_from = undefined, flight_buffer = [], protocol_specific = #{sender => Sender, @@ -782,12 +821,11 @@ initial_state(Role, Sender, Host, Port, Socket, {SSLOptions, SocketOptions, Trac initialize_tls_sender(#state{static_env = #static_env{ role = Role, transport_cb = Transport, - protocol_cb = Connection, socket = Socket, tracker = Tracker }, - socket_options = SockOpts, - negotiated_version = Version, + connection_env = #connection_env{negotiated_version = Version}, + socket_options = SockOpts, ssl_options = #ssl_options{renegotiate_at = RenegotiateAt}, connection_states = #{current_write := ConnectionWriteState}, protocol_specific = #{sender := Sender}}) -> @@ -796,19 +834,23 @@ initialize_tls_sender(#state{static_env = #static_env{ socket => Socket, socket_options => SockOpts, tracker => Tracker, - protocol_cb => Connection, transport_cb => Transport, negotiated_version => Version, renegotiate_at => RenegotiateAt}, tls_sender:initialize(Sender, Init). - -next_tls_record(Data, StateName, #state{protocol_buffers = - #protocol_buffers{tls_record_buffer = Buf0, - tls_cipher_texts = CT0} = Buffers} - = State0) -> - case tls_record:get_tls_records(Data, - acceptable_record_versions(StateName, State0), - Buf0) of + +next_tls_record(Data, StateName, + #state{protocol_buffers = + #protocol_buffers{tls_record_buffer = Buf0, + tls_cipher_texts = CT0} = Buffers} = State0) -> + Versions = + case StateName of + hello -> + [tls_record:protocol_version(Vsn) || Vsn <- ?ALL_AVAILABLE_VERSIONS]; + _ -> + State0#state.connection_env#connection_env.negotiated_version + end, + case tls_record:get_tls_records(Data, Versions, Buf0) of {Records, Buf1} -> CT1 = CT0 ++ Records, next_record(State0#state{protocol_buffers = @@ -819,11 +861,6 @@ next_tls_record(Data, StateName, #state{protocol_buffers = end. -acceptable_record_versions(StateName, #state{negotiated_version = Version}) when StateName =/= hello-> - Version; -acceptable_record_versions(hello, _) -> - [tls_record:protocol_version(Vsn) || Vsn <- ?ALL_AVAILABLE_VERSIONS]. - handle_record_alert(Alert, _) -> Alert. @@ -850,18 +887,18 @@ handle_info({tcp_passive, Socket}, StateName, State#state{protocol_specific = PS#{active_n_toggle => true}}); handle_info({CloseTag, Socket}, StateName, #state{static_env = #static_env{socket = Socket, close_tag = CloseTag}, + connection_env = #connection_env{negotiated_version = Version}, socket_options = #socket_options{active = Active}, protocol_buffers = #protocol_buffers{tls_cipher_texts = CTs}, - user_data_buffer = Buffer, - protocol_specific = PS, - negotiated_version = Version} = State) -> + user_data_buffer = {_,BufferSize,_}, + protocol_specific = PS} = State) -> %% Note that as of TLS 1.1, %% failure to properly close a connection no longer requires that a %% session not be resumed. This is a change from TLS 1.0 to conform %% with widespread implementation practice. - case (Active == false) andalso ((CTs =/= []) or (Buffer =/= <<>>)) of + case (Active == false) andalso ((CTs =/= []) or (BufferSize =/= 0)) of false -> case Version of {1, N} when N >= 1 -> @@ -893,6 +930,14 @@ handle_alerts([], Result) -> Result; handle_alerts(_, {stop, _, _} = Stop) -> Stop; +handle_alerts([#alert{level = ?WARNING, description = ?CLOSE_NOTIFY} | _Alerts], + {next_state, connection = StateName, #state{connection_env = CEnv, + socket_options = #socket_options{active = false}, + user_data_buffer = {_,BufferSize,_}, + protocol_buffers = #protocol_buffers{tls_cipher_texts = CTs}} = + State}) when (BufferSize =/= 0) orelse + (CTs =/= []) -> + {next_state, StateName, State#state{connection_env = CEnv#connection_env{terminated = true}}}; handle_alerts([Alert | Alerts], {next_state, StateName, State}) -> handle_alerts(Alerts, ssl_connection:handle_alert(Alert, StateName, State)); handle_alerts([Alert | Alerts], {next_state, StateName, State, _Actions}) -> @@ -912,7 +957,7 @@ decode_alerts(Bin) -> ssl_alert:decode(Bin). gen_handshake(StateName, Type, Event, - #state{negotiated_version = Version} = State) -> + #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> try ssl_connection:StateName(Type, Event, State, ?MODULE) of Result -> Result @@ -923,7 +968,7 @@ gen_handshake(StateName, Type, Event, Version, StateName, State) end. -gen_info(Event, connection = StateName, #state{negotiated_version = Version} = State) -> +gen_info(Event, connection = StateName, #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> try handle_info(Event, StateName, State) of Result -> Result @@ -934,7 +979,7 @@ gen_info(Event, connection = StateName, #state{negotiated_version = Version} = Version, StateName, State) end; -gen_info(Event, StateName, #state{negotiated_version = Version} = State) -> +gen_info(Event, StateName, #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> try handle_info(Event, StateName, State) of Result -> Result diff --git a/lib/ssl/src/tls_connection.hrl b/lib/ssl/src/tls_connection.hrl index 0af2258932..9063b1b736 100644 --- a/lib/ssl/src/tls_connection.hrl +++ b/lib/ssl/src/tls_connection.hrl @@ -30,7 +30,6 @@ -include("tls_record.hrl"). -record(protocol_buffers, { - tls_packets = [], %% :: [#ssl_tls{}], % Not yet handled decode SSL/TLS packets. tls_record_buffer = <<>>, %% :: binary(), % Buffer of incomplete records tls_handshake_buffer = <<>>, %% :: binary(), % Buffer of incomplete handshakes tls_cipher_texts = [] %%:: [binary()] diff --git a/lib/ssl/src/tls_handshake.erl b/lib/ssl/src/tls_handshake.erl index 19a5eb0348..0f0de5936a 100644 --- a/lib/ssl/src/tls_handshake.erl +++ b/lib/ssl/src/tls_handshake.erl @@ -30,6 +30,7 @@ -include("ssl_alert.hrl"). -include("ssl_internal.hrl"). -include("ssl_cipher.hrl"). +-include("ssl_api.hrl"). -include_lib("public_key/include/public_key.hrl"). %% Handshake handling @@ -47,7 +48,7 @@ %% Handshake handling %%==================================================================== %%-------------------------------------------------------------------- --spec client_hello(host(), inet:port_number(), ssl_record:connection_states(), +-spec client_hello(ssl:host(), inet:port_number(), ssl_record:connection_states(), #ssl_options{}, integer(), atom(), boolean(), der_cert()) -> #client_hello{}. %% @@ -81,13 +82,13 @@ client_hello(Host, Port, ConnectionStates, -spec hello(#server_hello{} | #client_hello{}, #ssl_options{}, ssl_record:connection_states() | {inet:port_number(), #session{}, db_handle(), atom(), ssl_record:connection_states(), - binary() | undefined, ssl_cipher_format:key_algo()}, + binary() | undefined, ssl:kex_algo()}, boolean()) -> - {tls_record:tls_version(), session_id(), + {tls_record:tls_version(), ssl:session_id(), ssl_record:connection_states(), alpn | npn, binary() | undefined}| {tls_record:tls_version(), {resumed | new, #session{}}, ssl_record:connection_states(), binary() | undefined, - #hello_extensions{}, {ssl_cipher_format:hash(), ssl_cipher_format:sign_algo()} | + #hello_extensions{}, {ssl:hash(), ssl:sign_algo()} | undefined} | #alert{}. %% %% Description: Handles a received hello message @@ -223,8 +224,6 @@ handle_client_hello_extensions(Version, Type, Random, CipherSuites, HelloExt, Version, SslOpts, Session0, ConnectionStates0, Renegotiation) of - #alert{} = Alert -> - Alert; {Session, ConnectionStates, Protocol, ServerHelloExt} -> {Version, {Type, Session}, ConnectionStates, Protocol, ServerHelloExt, HashSign} @@ -235,14 +234,14 @@ handle_client_hello_extensions(Version, Type, Random, CipherSuites, handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation) -> - case ssl_handshake:handle_server_hello_extensions(tls_record, Random, CipherSuite, + try ssl_handshake:handle_server_hello_extensions(tls_record, Random, CipherSuite, Compression, HelloExt, Version, SslOpt, ConnectionStates0, - Renegotiation) of - #alert{} = Alert -> - Alert; + Renegotiation) of {ConnectionStates, ProtoExt, Protocol} -> {Version, SessionId, ConnectionStates, ProtoExt, Protocol} + catch throw:Alert -> + Alert end. %%-------------------------------------------------------------------- enc_handshake(#hello_request{}, _Version) -> diff --git a/lib/ssl/src/tls_record.erl b/lib/ssl/src/tls_record.erl index 1776ec2627..b456197398 100644 --- a/lib/ssl/src/tls_record.erl +++ b/lib/ssl/src/tls_record.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2018. All Rights Reserved. +%% Copyright Ericsson AB 2007-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -75,15 +75,23 @@ init_connection_states(Role, BeastMitigation) -> pending_write => Pending}. %%-------------------------------------------------------------------- --spec get_tls_records(binary(), [tls_version()] | tls_version(), binary()) -> {[binary()], binary()} | #alert{}. +-spec get_tls_records( + binary(), [tls_version()] | tls_version(), + Buffer0 :: binary() | {'undefined' | #ssl_tls{}, {[binary()],non_neg_integer(),[binary()]}}) -> + {Records :: [#ssl_tls{}], + Buffer :: {'undefined' | #ssl_tls{}, {[binary()],non_neg_integer(),[binary()]}}} | + #alert{}. %% %% and returns it as a list of tls_compressed binaries also returns leftover %% Description: Given old buffer and new data from TCP, packs up a records %% data %%-------------------------------------------------------------------- -get_tls_records(Data, Version, Buffer) -> - get_tls_records_aux(Version, <<Buffer/binary, Data/binary>>, []). - + +get_tls_records(Data, Versions, Buffer) when is_binary(Buffer) -> + parse_tls_records(Versions, {[Data],byte_size(Data),[]}, undefined); +get_tls_records(Data, Versions, {Hdr, {Front,Size,Rear}}) -> + parse_tls_records(Versions, {Front,Size + byte_size(Data),[Data|Rear]}, Hdr). + %%==================================================================== %% Encoding %%==================================================================== @@ -102,8 +110,8 @@ encode_handshake(Frag, Version, ConnectionStates) -> case iolist_size(Frag) of N when N > ?MAX_PLAIN_TEXT_LENGTH -> - Data = split_bin(iolist_to_binary(Frag), Version, BCA, BeastMitigation), - encode_iolist(?HANDSHAKE, Data, Version, ConnectionStates); + Data = split_iovec(erlang:iolist_to_iovec(Frag), Version, BCA, BeastMitigation), + encode_fragments(?HANDSHAKE, Version, Data, ConnectionStates); _ -> encode_plain_text(?HANDSHAKE, Version, Frag, ConnectionStates) end. @@ -129,18 +137,18 @@ encode_change_cipher_spec(Version, ConnectionStates) -> encode_plain_text(?CHANGE_CIPHER_SPEC, Version, ?byte(?CHANGE_CIPHER_SPEC_PROTO), ConnectionStates). %%-------------------------------------------------------------------- --spec encode_data(binary(), tls_version(), ssl_record:connection_states()) -> - {iolist(), ssl_record:connection_states()}. +-spec encode_data([binary()], tls_version(), ssl_record:connection_states()) -> + {[[binary()]], ssl_record:connection_states()}. %% %% Description: Encodes data to send on the ssl-socket. %%-------------------------------------------------------------------- -encode_data(Frag, Version, +encode_data(Data, Version, #{current_write := #{beast_mitigation := BeastMitigation, security_parameters := #security_parameters{bulk_cipher_algorithm = BCA}}} = ConnectionStates) -> - Data = split_bin(Frag, Version, BCA, BeastMitigation), - encode_iolist(?APPLICATION_DATA, Data, Version, ConnectionStates). + Fragments = split_iovec(Data, Version, BCA, BeastMitigation), + encode_fragments(?APPLICATION_DATA, Version, Fragments, ConnectionStates). %%==================================================================== %% Decoding @@ -152,57 +160,59 @@ encode_data(Frag, Version, %% %% Description: Decode cipher text %%-------------------------------------------------------------------- -decode_cipher_text(#ssl_tls{type = Type, version = Version, - fragment = CipherFragment} = CipherText, +decode_cipher_text(CipherText, #{current_read := - #{compression_state := CompressionS0, - sequence_number := Seq, - cipher_state := CipherS0, + #{sequence_number := Seq, security_parameters := - #security_parameters{ - cipher_type = ?AEAD, - bulk_cipher_algorithm = - BulkCipherAlgo, - compression_algorithm = CompAlg} - } = ReadState0} = ConnnectionStates0, _) -> - AAD = start_additional_data(Type, Version, ReadState0), - CipherS1 = ssl_record:nonce_seed(BulkCipherAlgo, <<?UINT64(Seq)>>, CipherS0), - case ssl_record:decipher_aead(BulkCipherAlgo, CipherS1, AAD, CipherFragment, Version) of - {PlainFragment, CipherState} -> - {Plain, CompressionS1} = ssl_record:uncompress(CompAlg, - PlainFragment, CompressionS0), - ConnnectionStates = ConnnectionStates0#{ + #security_parameters{cipher_type = ?AEAD, + bulk_cipher_algorithm = BulkCipherAlgo}, + cipher_state := CipherS0 + } + } = ConnectionStates0, _) -> + SeqBin = <<?UINT64(Seq)>>, + #ssl_tls{type = Type, version = {MajVer,MinVer} = Version, fragment = Fragment} = CipherText, + StartAdditionalData = <<SeqBin/binary, ?BYTE(Type), ?BYTE(MajVer), ?BYTE(MinVer)>>, + CipherS = ssl_record:nonce_seed(BulkCipherAlgo, SeqBin, CipherS0), + case ssl_record:decipher_aead( + BulkCipherAlgo, CipherS, StartAdditionalData, Fragment, Version) + of + PlainFragment when is_binary(PlainFragment) -> + #{current_read := + #{security_parameters := SecParams, + compression_state := CompressionS0} = ReadState0} = ConnectionStates0, + {Plain, CompressionS} = ssl_record:uncompress(SecParams#security_parameters.compression_algorithm, + PlainFragment, CompressionS0), + ConnectionStates = ConnectionStates0#{ current_read => ReadState0#{ - cipher_state => CipherState, + cipher_state => CipherS, sequence_number => Seq + 1, - compression_state => CompressionS1}}, - {CipherText#ssl_tls{fragment = Plain}, ConnnectionStates}; + compression_state => CompressionS}}, + {CipherText#ssl_tls{fragment = Plain}, ConnectionStates}; #alert{} = Alert -> Alert end; -decode_cipher_text(#ssl_tls{type = Type, version = Version, +decode_cipher_text(#ssl_tls{version = Version, fragment = CipherFragment} = CipherText, - #{current_read := - #{compression_state := CompressionS0, - sequence_number := Seq, - security_parameters := - #security_parameters{compression_algorithm = CompAlg} - } = ReadState0} = ConnnectionStates0, PaddingCheck) -> + #{current_read := ReadState0} = ConnnectionStates0, PaddingCheck) -> case ssl_record:decipher(Version, CipherFragment, ReadState0, PaddingCheck) of {PlainFragment, Mac, ReadState1} -> - MacHash = ssl_cipher:calc_mac_hash(Type, Version, PlainFragment, ReadState1), + MacHash = ssl_cipher:calc_mac_hash(CipherText#ssl_tls.type, Version, PlainFragment, ReadState1), case ssl_record:is_correct_mac(Mac, MacHash) of true -> + #{sequence_number := Seq, + compression_state := CompressionS0, + security_parameters := + #security_parameters{compression_algorithm = CompAlg}} = ReadState0, {Plain, CompressionS1} = ssl_record:uncompress(CompAlg, PlainFragment, CompressionS0), - ConnnectionStates = ConnnectionStates0#{ - current_read => ReadState1#{ - sequence_number => Seq + 1, - compression_state => CompressionS1}}, + ConnnectionStates = + ConnnectionStates0#{current_read => + ReadState1#{sequence_number => Seq + 1, + compression_state => CompressionS1}}, {CipherText#ssl_tls{fragment = Plain}, ConnnectionStates}; false -> - ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) + ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) end; #alert{} = Alert -> Alert @@ -384,124 +394,222 @@ initial_connection_state(ConnectionEnd, BeastMitigation) -> server_verify_data => undefined }. -get_tls_records_aux({MajVer, MinVer} = Version, <<?BYTE(Type),?BYTE(MajVer),?BYTE(MinVer), - ?UINT16(Length), Data:Length/binary, Rest/binary>>, - Acc) when Type == ?APPLICATION_DATA; - Type == ?HANDSHAKE; - Type == ?ALERT; - Type == ?CHANGE_CIPHER_SPEC -> - get_tls_records_aux(Version, Rest, [#ssl_tls{type = Type, - version = Version, - fragment = Data} | Acc]); -get_tls_records_aux(Versions, <<?BYTE(Type),?BYTE(MajVer),?BYTE(MinVer), - ?UINT16(Length), Data:Length/binary, Rest/binary>>, - Acc) when is_list(Versions) andalso - ((Type == ?APPLICATION_DATA) - orelse - (Type == ?HANDSHAKE) - orelse - (Type == ?ALERT) - orelse - (Type == ?CHANGE_CIPHER_SPEC)) -> - case is_acceptable_version({MajVer, MinVer}, Versions) of + +parse_tls_records(Versions, Q, undefined) -> + decode_tls_records(Versions, Q, [], undefined, undefined, undefined); +parse_tls_records(Versions, Q, #ssl_tls{type = Type, version = Version, fragment = Length}) -> + decode_tls_records(Versions, Q, [], Type, Version, Length). + +%% Generic code path +decode_tls_records(Versions, {_,Size,_} = Q0, Acc, undefined, _Version, _Length) -> + if + 5 =< Size -> + {<<?BYTE(Type),?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Length)>>, Q} = binary_from_front(5, Q0), + validate_tls_records_type(Versions, Q, Acc, Type, {MajVer,MinVer}, Length); + 3 =< Size -> + {<<?BYTE(Type),?BYTE(MajVer),?BYTE(MinVer)>>, Q} = binary_from_front(3, Q0), + validate_tls_records_type(Versions, Q, Acc, Type, {MajVer,MinVer}, undefined); + 1 =< Size -> + {<<?BYTE(Type)>>, Q} = binary_from_front(1, Q0), + validate_tls_records_type(Versions, Q, Acc, Type, undefined, undefined); + true -> + validate_tls_records_type(Versions, Q0, Acc, undefined, undefined, undefined) + end; +decode_tls_records(Versions, {_,Size,_} = Q0, Acc, Type, undefined, _Length) -> + if + 4 =< Size -> + {<<?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Length)>>, Q} = binary_from_front(4, Q0), + validate_tls_record_version(Versions, Q, Acc, Type, {MajVer,MinVer}, Length); + 2 =< Size -> + {<<?BYTE(MajVer),?BYTE(MinVer)>>, Q} = binary_from_front(2, Q0), + validate_tls_record_version(Versions, Q, Acc, Type, {MajVer,MinVer}, undefined); + true -> + validate_tls_record_version(Versions, Q0, Acc, Type, undefined, undefined) + end; +decode_tls_records(Versions, {_,Size,_} = Q0, Acc, Type, Version, undefined) -> + if + 2 =< Size -> + {<<?UINT16(Length)>>, Q} = binary_from_front(2, Q0), + validate_tls_record_length(Versions, Q, Acc, Type, Version, Length); + true -> + validate_tls_record_length(Versions, Q0, Acc, Type, Version, undefined) + end; +decode_tls_records(Versions, Q, Acc, Type, Version, Length) -> + validate_tls_record_length(Versions, Q, Acc, Type, Version, Length). + +validate_tls_records_type(_Versions, Q, Acc, undefined, _Version, _Length) -> + {lists:reverse(Acc), + {undefined, Q}}; +validate_tls_records_type(Versions, Q, Acc, Type, Version, Length) -> + if + ?KNOWN_RECORD_TYPE(Type) -> + validate_tls_record_version(Versions, Q, Acc, Type, Version, Length); + true -> + %% Not ?KNOWN_RECORD_TYPE(Type) + ?ALERT_REC(?FATAL, ?UNEXPECTED_MESSAGE) + end. + +validate_tls_record_version(_Versions, Q, Acc, Type, undefined, _Length) -> + {lists:reverse(Acc), + {#ssl_tls{type = Type, version = undefined, fragment = undefined}, Q}}; +validate_tls_record_version(Versions, Q, Acc, Type, Version, Length) -> + if + is_list(Versions) -> + case is_acceptable_version(Version, Versions) of + true -> + validate_tls_record_length(Versions, Q, Acc, Type, Version, Length); + false -> + ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) + end; + Version =:= Versions -> + %% Exact version match + validate_tls_record_length(Versions, Q, Acc, Type, Version, Length); true -> - get_tls_records_aux(Versions, Rest, [#ssl_tls{type = Type, - version = {MajVer, MinVer}, - fragment = Data} | Acc]); - false -> ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) + end. + +validate_tls_record_length(_Versions, Q, Acc, Type, Version, undefined) -> + {lists:reverse(Acc), + {#ssl_tls{type = Type, version = Version, fragment = undefined}, Q}}; +validate_tls_record_length(Versions, {_,Size0,_} = Q0, Acc, Type, Version, Length) -> + if + Length =< ?MAX_CIPHER_TEXT_LENGTH -> + if + Length =< Size0 -> + %% Complete record + {Fragment, Q} = binary_from_front(Length, Q0), + Record = #ssl_tls{type = Type, version = Version, fragment = Fragment}, + decode_tls_records(Versions, Q, [Record|Acc], undefined, undefined, undefined); + true -> + {lists:reverse(Acc), + {#ssl_tls{type = Type, version = Version, fragment = Length}, Q0}} + end; + true -> + ?ALERT_REC(?FATAL, ?RECORD_OVERFLOW) + end. + + +binary_from_front(SplitSize, {Front,Size,Rear}) -> + binary_from_front(SplitSize, Front, Size, Rear, []). +%% +binary_from_front(SplitSize, [], Size, [_] = Rear, Acc) -> + %% Optimize a simple case + binary_from_front(SplitSize, Rear, Size, [], Acc); +binary_from_front(SplitSize, [], Size, Rear, Acc) -> + binary_from_front(SplitSize, lists:reverse(Rear), Size, [], Acc); +binary_from_front(SplitSize, [Bin|Front], Size, Rear, []) -> + %% Optimize a frequent case + BinSize = byte_size(Bin), + if + SplitSize < BinSize -> + {RetBin, Rest} = erlang:split_binary(Bin, SplitSize), + {RetBin, {[Rest|Front],Size - SplitSize,Rear}}; + BinSize < SplitSize -> + binary_from_front(SplitSize - BinSize, Front, Size, Rear, [Bin]); + true -> % Perfect fit + {Bin, {Front,Size - SplitSize,Rear}} end; -get_tls_records_aux(_, <<?BYTE(Type),?BYTE(_MajVer),?BYTE(_MinVer), - ?UINT16(Length), _:Length/binary, _Rest/binary>>, - _) when Type == ?APPLICATION_DATA; - Type == ?HANDSHAKE; - Type == ?ALERT; - Type == ?CHANGE_CIPHER_SPEC -> - ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC); -get_tls_records_aux(_, <<0:1, _CT:7, ?BYTE(_MajVer), ?BYTE(_MinVer), - ?UINT16(Length), _/binary>>, - _Acc) when Length > ?MAX_CIPHER_TEXT_LENGTH -> - ?ALERT_REC(?FATAL, ?RECORD_OVERFLOW); -get_tls_records_aux(_, Data, Acc) -> - case size(Data) =< ?MAX_CIPHER_TEXT_LENGTH + ?INITIAL_BYTES of - true -> - {lists:reverse(Acc), Data}; - false -> - ?ALERT_REC(?FATAL, ?UNEXPECTED_MESSAGE) +binary_from_front(SplitSize, [Bin|Front], Size, Rear, Acc) -> + BinSize = byte_size(Bin), + if + SplitSize < BinSize -> + {Last, Rest} = erlang:split_binary(Bin, SplitSize), + RetBin = iolist_to_binary(lists:reverse(Acc, [Last])), + {RetBin, {[Rest|Front],Size - byte_size(RetBin),Rear}}; + BinSize < SplitSize -> + binary_from_front(SplitSize - BinSize, Front, Size, Rear, [Bin|Acc]); + true -> % Perfect fit + RetBin = iolist_to_binary(lists:reverse(Acc, [Bin])), + {RetBin, {Front,Size - byte_size(RetBin),Rear}} end. + +%%-------------------------------------------------------------------- +encode_plain_text(Type, Version, Data, ConnectionStates0) -> + {[CipherText],ConnectionStates} = encode_fragments(Type, Version, [Data], ConnectionStates0), + {CipherText,ConnectionStates}. %%-------------------------------------------------------------------- -encode_plain_text(Type, Version, Data, #{current_write := Write0} = ConnectionStates) -> - {CipherFragment, Write1} = do_encode_plain_text(Type, Version, Data, Write0), - {CipherText, Write} = encode_tls_cipher_text(Type, Version, CipherFragment, Write1), - {CipherText, ConnectionStates#{current_write => Write}}. - -encode_tls_cipher_text(Type, {MajVer, MinVer}, Fragment, #{sequence_number := Seq} = Write) -> - Length = erlang:iolist_size(Fragment), - {[<<?BYTE(Type), ?BYTE(MajVer), ?BYTE(MinVer), ?UINT16(Length)>>, Fragment], - Write#{sequence_number => Seq +1}}. - -encode_iolist(Type, Data, Version, ConnectionStates0) -> - {ConnectionStates, EncodedMsg} = - lists:foldl(fun(Text, {CS0, Encoded}) -> - {Enc, CS1} = - encode_plain_text(Type, Version, Text, CS0), - {CS1, [Enc | Encoded]} - end, {ConnectionStates0, []}, Data), - {lists:reverse(EncodedMsg), ConnectionStates}. -%%-------------------------------------------------------------------- -do_encode_plain_text(Type, Version, Data, #{compression_state := CompS0, - cipher_state := CipherS0, - sequence_number := Seq, - security_parameters := - #security_parameters{ - cipher_type = ?AEAD, - bulk_cipher_algorithm = BCAlg, - compression_algorithm = CompAlg} - } = WriteState0) -> - {Comp, CompS1} = ssl_record:compress(CompAlg, Data, CompS0), - CipherS = ssl_record:nonce_seed(BCAlg, <<?UINT64(Seq)>>, CipherS0), - WriteState = WriteState0#{compression_state => CompS1, - cipher_state => CipherS}, - AAD = start_additional_data(Type, Version, WriteState), - ssl_record:cipher_aead(Version, Comp, WriteState, AAD); -do_encode_plain_text(Type, Version, Data, #{compression_state := CompS0, - security_parameters := - #security_parameters{compression_algorithm = CompAlg} - }= WriteState0) -> - {Comp, CompS1} = ssl_record:compress(CompAlg, Data, CompS0), - WriteState1 = WriteState0#{compression_state => CompS1}, - MacHash = ssl_cipher:calc_mac_hash(Type, Version, Comp, WriteState1), - ssl_record:cipher(Version, Comp, WriteState1, MacHash); -do_encode_plain_text(_,_,_,CS) -> +encode_fragments(Type, Version, Data, + #{current_write := #{compression_state := CompS, + cipher_state := CipherS, + sequence_number := Seq}} = ConnectionStates) -> + encode_fragments(Type, Version, Data, ConnectionStates, CompS, CipherS, Seq, []). +%% +encode_fragments(_Type, _Version, [], #{current_write := WriteS} = CS, + CompS, CipherS, Seq, CipherFragments) -> + {lists:reverse(CipherFragments), + CS#{current_write := WriteS#{compression_state := CompS, + cipher_state := CipherS, + sequence_number := Seq}}}; +encode_fragments(Type, Version, [Text|Data], + #{current_write := #{security_parameters := + #security_parameters{cipher_type = ?AEAD, + bulk_cipher_algorithm = BCAlg, + compression_algorithm = CompAlg} = SecPars}} = CS, + CompS0, CipherS0, Seq, CipherFragments) -> + {CompText, CompS} = ssl_record:compress(CompAlg, Text, CompS0), + SeqBin = <<?UINT64(Seq)>>, + CipherS1 = ssl_record:nonce_seed(BCAlg, SeqBin, CipherS0), + {MajVer, MinVer} = Version, + VersionBin = <<?BYTE(MajVer), ?BYTE(MinVer)>>, + StartAdditionalData = <<SeqBin/binary, ?BYTE(Type), VersionBin/binary>>, + {CipherFragment,CipherS} = ssl_record:cipher_aead(Version, CompText, CipherS1, StartAdditionalData, SecPars), + Length = byte_size(CipherFragment), + CipherHeader = <<?BYTE(Type), VersionBin/binary, ?UINT16(Length)>>, + encode_fragments(Type, Version, Data, CS, CompS, CipherS, Seq + 1, + [[CipherHeader, CipherFragment] | CipherFragments]); +encode_fragments(Type, Version, [Text|Data], + #{current_write := #{security_parameters := + #security_parameters{compression_algorithm = CompAlg, + mac_algorithm = MacAlgorithm} = SecPars, + mac_secret := MacSecret}} = CS, + CompS0, CipherS0, Seq, CipherFragments) -> + {CompText, CompS} = ssl_record:compress(CompAlg, Text, CompS0), + MacHash = ssl_cipher:calc_mac_hash(Type, Version, CompText, MacAlgorithm, MacSecret, Seq), + {CipherFragment,CipherS} = ssl_record:cipher(Version, CompText, CipherS0, MacHash, SecPars), + Length = byte_size(CipherFragment), + {MajVer, MinVer} = Version, + CipherHeader = <<?BYTE(Type), ?BYTE(MajVer), ?BYTE(MinVer), ?UINT16(Length)>>, + encode_fragments(Type, Version, Data, CS, CompS, CipherS, Seq + 1, + [[CipherHeader, CipherFragment] | CipherFragments]); +encode_fragments(_Type, _Version, _Data, CS, _CompS, _CipherS, _Seq, _CipherFragments) -> exit({cs, CS}). %%-------------------------------------------------------------------- -start_additional_data(Type, {MajVer, MinVer}, - #{sequence_number := SeqNo}) -> - <<?UINT64(SeqNo), ?BYTE(Type), ?BYTE(MajVer), ?BYTE(MinVer)>>. %% 1/n-1 splitting countermeasure Rizzo/Duong-Beast, RC4 chiphers are %% not vulnerable to this attack. -split_bin(<<FirstByte:8, Rest/binary>>, Version, BCA, one_n_minus_one) when - BCA =/= ?RC4 andalso ({3, 1} == Version orelse - {3, 0} == Version) -> - [[FirstByte]|do_split_bin(Rest)]; +split_iovec([<<FirstByte:8, Rest/binary>>|Data], Version, BCA, one_n_minus_one) + when (BCA =/= ?RC4) andalso ({3, 1} == Version orelse + {3, 0} == Version) -> + [[FirstByte]|split_iovec([Rest|Data])]; %% 0/n splitting countermeasure for clients that are incompatible with 1/n-1 %% splitting. -split_bin(Bin, Version, BCA, zero_n) when - BCA =/= ?RC4 andalso ({3, 1} == Version orelse - {3, 0} == Version) -> - [<<>>|do_split_bin(Bin)]; -split_bin(Bin, _, _, _) -> - do_split_bin(Bin). - -do_split_bin(<<>>) -> []; -do_split_bin(Bin) -> - case Bin of - <<Chunk:?MAX_PLAIN_TEXT_LENGTH/binary, Rest/binary>> -> - [Chunk|do_split_bin(Rest)]; - _ -> - [Bin] - end. +split_iovec(Data, Version, BCA, zero_n) + when (BCA =/= ?RC4) andalso ({3, 1} == Version orelse + {3, 0} == Version) -> + [<<>>|split_iovec(Data)]; +split_iovec(Data, _Version, _BCA, _BeatMitigation) -> + split_iovec(Data). + +split_iovec([]) -> + []; +split_iovec(Data) -> + {Part,Rest} = split_iovec(Data, ?MAX_PLAIN_TEXT_LENGTH, []), + [Part|split_iovec(Rest)]. +%% +split_iovec([Bin|Data], SplitSize, Acc) -> + BinSize = byte_size(Bin), + if + SplitSize < BinSize -> + {Last, Rest} = erlang:split_binary(Bin, SplitSize), + {lists:reverse(Acc, [Last]), [Rest|Data]}; + BinSize < SplitSize -> + split_iovec(Data, SplitSize - BinSize, [Bin|Acc]); + true -> % Perfect match + {lists:reverse(Acc, [Bin]), Data} + end; +split_iovec([], _SplitSize, Acc) -> + {lists:reverse(Acc),[]}. + %%-------------------------------------------------------------------- lowest_list_protocol_version(Ver, []) -> Ver; diff --git a/lib/ssl/src/tls_sender.erl b/lib/ssl/src/tls_sender.erl index 11fcc6def0..c07b7f49cd 100644 --- a/lib/ssl/src/tls_sender.erl +++ b/lib/ssl/src/tls_sender.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2018-2018. All Rights Reserved. +%% Copyright Ericsson AB 2018-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -38,19 +38,23 @@ -define(SERVER, ?MODULE). --record(data, {connection_pid, - connection_states = #{}, - role, - socket, - socket_options, - tracker, - protocol_cb, - transport_cb, - negotiated_version, - renegotiate_at, - connection_monitor, - dist_handle - }). +-record(static, + {connection_pid, + role, + socket, + socket_options, + tracker, + transport_cb, + negotiated_version, + renegotiate_at, + connection_monitor, + dist_handle + }). + +-record(data, + {static = #static{}, + connection_states = #{} + }). %%%=================================================================== %%% API @@ -171,6 +175,10 @@ dist_tls_socket(Pid) -> callback_mode() -> state_functions. + +-define(HANDLE_COMMON, + ?FUNCTION_NAME(Type, Msg, StateData) -> + handle_common(Type, Msg, StateData)). %%-------------------------------------------------------------------- -spec init(Args :: term()) -> gen_statem:init_result(atom()). @@ -192,39 +200,35 @@ init({call, From}, {Pid, #{current_write := WriteState, socket := Socket, socket_options := SockOpts, tracker := Tracker, - protocol_cb := Connection, transport_cb := Transport, negotiated_version := Version, renegotiate_at := RenegotiateAt}}, - #data{connection_states = ConnectionStates} = StateData0) -> + #data{connection_states = ConnectionStates, static = Static0} = StateData0) -> Monitor = erlang:monitor(process, Pid), StateData = - StateData0#data{connection_pid = Pid, - connection_monitor = Monitor, - connection_states = - ConnectionStates#{current_write => WriteState}, - role = Role, - socket = Socket, - socket_options = SockOpts, - tracker = Tracker, - protocol_cb = Connection, - transport_cb = Transport, - negotiated_version = Version, - renegotiate_at = RenegotiateAt}, + StateData0#data{connection_states = ConnectionStates#{current_write => WriteState}, + static = Static0#static{connection_pid = Pid, + connection_monitor = Monitor, + role = Role, + socket = Socket, + socket_options = SockOpts, + tracker = Tracker, + transport_cb = Transport, + negotiated_version = Version, + renegotiate_at = RenegotiateAt}}, {next_state, handshake, StateData, [{reply, From, ok}]}; -init(info, Msg, StateData) -> - handle_info(Msg, ?FUNCTION_NAME, StateData). +init(_, _, _) -> + %% Just in case anything else sneeks through + {keep_state_and_data, [postpone]}. + %%-------------------------------------------------------------------- -spec connection(gen_statem:event_type(), Msg :: term(), StateData :: term()) -> gen_statem:event_handler_result(atom()). %%-------------------------------------------------------------------- -connection({call, From}, renegotiate, - #data{connection_states = #{current_write := Write}} = StateData) -> - {next_state, handshake, StateData, [{reply, From, {ok, Write}}]}; connection({call, From}, {application_data, AppData}, - #data{socket_options = #socket_options{packet = Packet}} = + #data{static = #static{socket_options = #socket_options{packet = Packet}}} = StateData) -> case encode_packet(Packet, AppData) of {error, _} = Error -> @@ -232,40 +236,40 @@ connection({call, From}, {application_data, AppData}, Data -> send_application_data(Data, From, ?FUNCTION_NAME, StateData) end; -connection({call, From}, {set_opts, _} = Call, StateData) -> - handle_call(From, Call, ?FUNCTION_NAME, StateData); +connection({call, From}, {ack_alert, #alert{} = Alert}, StateData0) -> + StateData = send_tls_alert(Alert, StateData0), + {next_state, ?FUNCTION_NAME, StateData, + [{reply,From,ok}]}; +connection({call, From}, renegotiate, + #data{connection_states = #{current_write := Write}} = StateData) -> + {next_state, handshake, StateData, [{reply, From, {ok, Write}}]}; +connection({call, From}, downgrade, #data{connection_states = + #{current_write := Write}} = StateData) -> + {next_state, death_row, StateData, [{reply,From, {ok, Write}}]}; +connection({call, From}, {set_opts, Opts}, StateData) -> + handle_set_opts(From, Opts, StateData); connection({call, From}, dist_get_tls_socket, - #data{protocol_cb = Connection, - transport_cb = Transport, - socket = Socket, - connection_pid = Pid, - tracker = Tracker} = StateData) -> - TLSSocket = Connection:socket([Pid, self()], Transport, Socket, Connection, Tracker), + #data{static = #static{transport_cb = Transport, + socket = Socket, + connection_pid = Pid, + tracker = Tracker}} = StateData) -> + TLSSocket = tls_connection:socket([Pid, self()], Transport, Socket, Tracker), {next_state, ?FUNCTION_NAME, StateData, [{reply, From, {ok, TLSSocket}}]}; connection({call, From}, {dist_handshake_complete, _Node, DHandle}, - #data{connection_pid = Pid, - socket_options = #socket_options{packet = Packet}} = - StateData) -> + #data{static = #static{connection_pid = Pid} = Static} = StateData) -> ok = erlang:dist_ctrl_input_handler(DHandle, Pid), ok = ssl_connection:dist_handshake_complete(Pid, DHandle), %% From now on we execute on normal priority process_flag(priority, normal), - {next_state, ?FUNCTION_NAME, StateData#data{dist_handle = DHandle}, - [{reply, From, ok} - | case dist_data(DHandle, Packet) of - [] -> - []; - Data -> - [{next_event, internal, - {application_packets,{self(),undefined},Data}}] - end]}; -connection({call, From}, {ack_alert, #alert{} = Alert}, StateData0) -> - StateData = send_tls_alert(Alert, StateData0), - {next_state, ?FUNCTION_NAME, StateData, - [{reply,From,ok}]}; -connection({call, From}, downgrade, #data{connection_states = - #{current_write := Write}} = StateData) -> - {next_state, death_row, StateData, [{reply,From, {ok, Write}}]}; + {keep_state, StateData#data{static = Static#static{dist_handle = DHandle}}, + [{reply,From,ok}| + case dist_data(DHandle) of + [] -> + []; + Data -> + [{next_event, internal, + {application_packets,{self(),undefined},erlang:iolist_to_iovec(Data)}}] + end]}; connection(internal, {application_packets, From, Data}, StateData) -> send_application_data(Data, From, ?FUNCTION_NAME, StateData); %% @@ -273,29 +277,26 @@ connection(cast, #alert{} = Alert, StateData0) -> StateData = send_tls_alert(Alert, StateData0), {next_state, ?FUNCTION_NAME, StateData}; connection(cast, {new_write, WritesState, Version}, - #data{connection_states = ConnectionStates0} = StateData) -> + #data{connection_states = ConnectionStates, static = Static} = StateData) -> {next_state, connection, StateData#data{connection_states = - ConnectionStates0#{current_write => WritesState}, - negotiated_version = Version}}; + ConnectionStates#{current_write => WritesState}, + static = Static#static{negotiated_version = Version}}}; %% -connection(info, dist_data, - #data{dist_handle = DHandle, - socket_options = #socket_options{packet = Packet}} = - StateData) -> - {next_state, ?FUNCTION_NAME, StateData, - case dist_data(DHandle, Packet) of +connection(info, dist_data, #data{static = #static{dist_handle = DHandle}}) -> + {keep_state_and_data, + case dist_data(DHandle) of [] -> []; Data -> [{next_event, internal, - {application_packets,{self(),undefined},Data}}] + {application_packets,{self(),undefined},erlang:iolist_to_iovec(Data)}}] end}; connection(info, tick, StateData) -> consume_ticks(), - {next_state, ?FUNCTION_NAME, StateData, - [{next_event, {call, {self(), undefined}}, - {application_data, <<>>}}]}; + Data = [<<0:32>>], % encode_packet(4, <<>>) + From = {self(), undefined}, + send_application_data(Data, From, ?FUNCTION_NAME, StateData); connection(info, {send, From, Ref, Data}, _StateData) -> %% This is for testing only! %% @@ -304,29 +305,37 @@ connection(info, {send, From, Ref, Data}, _StateData) -> From ! {Ref, ok}, {keep_state_and_data, [{next_event, {call, {self(), undefined}}, - {application_data, iolist_to_binary(Data)}}]}; -connection(info, Msg, StateData) -> - handle_info(Msg, ?FUNCTION_NAME, StateData). + {application_data, erlang:iolist_to_iovec(Data)}}]}; +?HANDLE_COMMON. + %%-------------------------------------------------------------------- -spec handshake(gen_statem:event_type(), Msg :: term(), StateData :: term()) -> gen_statem:event_handler_result(atom()). %%-------------------------------------------------------------------- -handshake({call, From}, {set_opts, _} = Call, StateData) -> - handle_call(From, Call, ?FUNCTION_NAME, StateData); +handshake({call, From}, {set_opts, Opts}, StateData) -> + handle_set_opts(From, Opts, StateData); handshake({call, _}, _, _) -> + %% Postpone all calls to the connection state + {keep_state_and_data, [postpone]}; +handshake(internal, {application_packets,_,_}, _) -> {keep_state_and_data, [postpone]}; handshake(cast, {new_write, WritesState, Version}, - #data{connection_states = ConnectionStates0} = StateData) -> + #data{connection_states = ConnectionStates, static = Static} = StateData) -> {next_state, connection, - StateData#data{connection_states = - ConnectionStates0#{current_write => WritesState}, - negotiated_version = Version}}; -handshake(internal, {application_packets,_,_}, _) -> + StateData#data{connection_states = ConnectionStates#{current_write => WritesState}, + static = Static#static{negotiated_version = Version}}}; +handshake(info, dist_data, _) -> {keep_state_and_data, [postpone]}; -handshake(info, Msg, StateData) -> - handle_info(Msg, ?FUNCTION_NAME, StateData). +handshake(info, tick, _) -> + %% Ignore - data is sent anyway during handshake + consume_ticks(), + keep_state_and_data; +handshake(info, {send, _, _, _}, _) -> + %% Testing only, OTP distribution test suites... + {keep_state_and_data, [postpone]}; +?HANDLE_COMMON. %%-------------------------------------------------------------------- -spec death_row(gen_statem:event_type(), @@ -361,49 +370,66 @@ code_change(_OldVsn, State, Data, _Extra) -> %%%=================================================================== %%% Internal functions %%%=================================================================== -handle_call(From, {set_opts, Opts}, StateName, #data{socket_options = SockOpts} = StateData) -> - {next_state, StateName, StateData#data{socket_options = set_opts(SockOpts, Opts)}, [{reply, From, ok}]}. - -handle_info({'DOWN', Monitor, _, _, Reason}, _, - #data{connection_monitor = Monitor, - dist_handle = Handle} = StateData) when Handle =/= undefined-> - {next_state, death_row, StateData, [{state_timeout, 5000, Reason}]}; -handle_info({'DOWN', Monitor, _, _, _}, _, - #data{connection_monitor = Monitor} = StateData) -> + +handle_set_opts( + From, Opts, #data{static = #static{socket_options = SockOpts} = Static} = StateData) -> + {keep_state, StateData#data{static = Static#static{socket_options = set_opts(SockOpts, Opts)}}, + [{reply, From, ok}]}. + +handle_common( + {call, From}, {set_opts, Opts}, + #data{static = #static{socket_options = SockOpts} = Static} = StateData) -> + {keep_state, StateData#data{static = Static#static{socket_options = set_opts(SockOpts, Opts)}}, + [{reply, From, ok}]}; +handle_common( + info, {'DOWN', Monitor, _, _, Reason}, + #data{static = #static{connection_monitor = Monitor, + dist_handle = Handle}} = StateData) when Handle =/= undefined -> + {next_state, death_row, StateData, + [{state_timeout, 5000, Reason}]}; +handle_common( + info, {'DOWN', Monitor, _, _, _}, + #data{static = #static{connection_monitor = Monitor}} = StateData) -> {stop, normal, StateData}; -handle_info(_,_,_) -> +handle_common(info, Msg, _) -> + Report = + io_lib:format("TLS sender: Got unexpected info: ~p ~n", [Msg]), + error_logger:info_report(Report), + keep_state_and_data; +handle_common(Type, Msg, _) -> + Report = + io_lib:format( + "TLS sender: Got unexpected event: ~p ~n", [{Type,Msg}]), + error_logger:error_report(Report), keep_state_and_data. -send_tls_alert(Alert, #data{negotiated_version = Version, - socket = Socket, - protocol_cb = Connection, - transport_cb = Transport, - connection_states = ConnectionStates0} = StateData0) -> +send_tls_alert(#alert{} = Alert, + #data{static = #static{negotiated_version = Version, + socket = Socket, + transport_cb = Transport}, + connection_states = ConnectionStates0} = StateData0) -> {BinMsg, ConnectionStates} = - Connection:encode_alert(Alert, Version, ConnectionStates0), - Connection:send(Transport, Socket, BinMsg), + tls_record:encode_alert_record(Alert, Version, ConnectionStates0), + tls_socket:send(Transport, Socket, BinMsg), StateData0#data{connection_states = ConnectionStates}. send_application_data(Data, From, StateName, - #data{connection_pid = Pid, - socket = Socket, - dist_handle = DistHandle, - negotiated_version = Version, - protocol_cb = Connection, - transport_cb = Transport, - connection_states = ConnectionStates0, - renegotiate_at = RenegotiateAt} = StateData0) -> + #data{static = #static{connection_pid = Pid, + socket = Socket, + dist_handle = DistHandle, + negotiated_version = Version, + transport_cb = Transport, + renegotiate_at = RenegotiateAt}, + connection_states = ConnectionStates0} = StateData0) -> case time_to_renegotiate(Data, ConnectionStates0, RenegotiateAt) of true -> ssl_connection:internal_renegotiation(Pid, ConnectionStates0), {next_state, handshake, StateData0, [{next_event, internal, {application_packets, From, Data}}]}; false -> - {Msgs, ConnectionStates} = - Connection:encode_data( - iolist_to_binary(Data), Version, ConnectionStates0), + {Msgs, ConnectionStates} = tls_record:encode_data(Data, Version, ConnectionStates0), StateData = StateData0#data{connection_states = ConnectionStates}, - case Connection:send(Transport, Socket, Msgs) of + case tls_socket:send(Transport, Socket, Msgs) of ok when DistHandle =/= undefined -> {next_state, StateName, StateData, []}; Reason when DistHandle =/= undefined -> @@ -419,9 +445,9 @@ send_application_data(Data, From, StateName, encode_packet(Packet, Data) -> Len = iolist_size(Data), case Packet of - 1 when Len < (1 bsl 8) -> [<<Len:8>>,Data]; - 2 when Len < (1 bsl 16) -> [<<Len:16>>,Data]; - 4 when Len < (1 bsl 32) -> [<<Len:32>>,Data]; + 1 when Len < (1 bsl 8) -> [<<Len:8>>|Data]; + 2 when Len < (1 bsl 16) -> [<<Len:16>>|Data]; + 4 when Len < (1 bsl 32) -> [<<Len:32>>|Data]; N when N =:= 1; N =:= 2; N =:= 4 -> {error, {badarg, {packet_to_large, Len, (1 bsl (Packet bsl 3)) - 1}}}; @@ -458,22 +484,30 @@ call(FsmPid, Event) -> {error, closed} end. -%%---------------Erlang distribution -------------------------------------- +%%-------------- Erlang distribution helpers ------------------------------ -dist_data(DHandle, Packet) -> +dist_data(DHandle) -> case erlang:dist_ctrl_get_data(DHandle) of none -> erlang:dist_ctrl_get_data_notification(DHandle), []; - Data -> - %% This is encode_packet(4, Data) without Len check - %% since the emulator will always deliver a Data - %% smaller than 4 GB, and the distribution will - %% therefore always have to use {packet,4} + %% This is encode_packet(4, Data) without Len check + %% since the emulator will always deliver a Data + %% smaller than 4 GB, and the distribution will + %% therefore always have to use {packet,4} + Data when is_binary(Data) -> + Len = byte_size(Data), + [[<<Len:32>>,Data]|dist_data(DHandle)]; + [BA,BB] = Data -> + Len = byte_size(BA) + byte_size(BB), + [[<<Len:32>>|Data]|dist_data(DHandle)]; + Data when is_list(Data) -> Len = iolist_size(Data), - [<<Len:32>>,Data|dist_data(DHandle, Packet)] + [[<<Len:32>>|Data]|dist_data(DHandle)] end. + +%% Empty the inbox from distribution ticks - do not let them accumulate consume_ticks() -> receive tick -> consume_ticks() diff --git a/lib/ssl/test/Makefile b/lib/ssl/test/Makefile index 9dfb2eba53..a10f71a3de 100644 --- a/lib/ssl/test/Makefile +++ b/lib/ssl/test/Makefile @@ -29,7 +29,7 @@ include $(ERL_TOP)/make/$(TARGET)/otp.mk # Application version # ---------------------------------------------------- include ../vsn.mk -VSN=$(GS_VSN) +VSN=$(SSL_VSN) # ---------------------------------------------------- # Target Specs diff --git a/lib/ssl/test/make_certs.erl b/lib/ssl/test/make_certs.erl index 8fe7c54549..7f3371da9a 100644 --- a/lib/ssl/test/make_certs.erl +++ b/lib/ssl/test/make_certs.erl @@ -189,6 +189,18 @@ gencrl(Root, CA, C, CrlHours) -> Env = [{"ROOTDIR", filename:absname(Root)}], cmd(Cmd, Env). +%% This function sets the number of seconds until the next CRL is due. +gencrl_sec(Root, CA, C, CrlSecs) -> + CACnfFile = filename:join([Root, CA, "ca.cnf"]), + CACRLFile = filename:join([Root, CA, "crl.pem"]), + Cmd = [C#config.openssl_cmd, " ca" + " -gencrl ", + " -crlsec ", integer_to_list(CrlSecs), + " -out ", CACRLFile, + " -config ", CACnfFile], + Env = [{"ROOTDIR", filename:absname(Root)}], + cmd(Cmd, Env). + can_generate_expired_crls(C) -> %% OpenSSL can generate CRLs with an expiration date in the past, %% if we pass a negative number for -crlhours. However, LibreSSL diff --git a/lib/ssl/test/ssl_ECC_SUITE.erl b/lib/ssl/test/ssl_ECC_SUITE.erl index a5309e866b..ca8d0ec70c 100644 --- a/lib/ssl/test/ssl_ECC_SUITE.erl +++ b/lib/ssl/test/ssl_ECC_SUITE.erl @@ -212,53 +212,61 @@ client_ecdsa_server_ecdsa_with_raw_key(Config) when is_list(Config) -> ecc_default_order(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], - ecdhe_ecdsa, ecdhe_ecdsa, Config), + ecdhe_ecdsa, ecdhe_ecdsa, + Config, DefaultCurve), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), ECCOpts = [], - case ssl_test_lib:supported_eccs([{eccs, [sect571r1]}]) of - true -> ssl_test_lib:ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); + case ssl_test_lib:supported_eccs([{eccs, [DefaultCurve]}]) of + true -> ssl_test_lib:ecc_test(DefaultCurve, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. ecc_default_order_custom_curves(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], - ecdhe_ecdsa, ecdhe_ecdsa, Config), + ecdhe_ecdsa, ecdhe_ecdsa, + Config, DefaultCurve), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{eccs, [secp256r1, sect571r1]}], + ECCOpts = [{eccs, [secp256r1, DefaultCurve]}], case ssl_test_lib:supported_eccs(ECCOpts) of - true -> ssl_test_lib:ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); + true -> ssl_test_lib:ecc_test(DefaultCurve, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. ecc_client_order(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], - ecdhe_ecdsa, ecdhe_ecdsa, Config), + ecdhe_ecdsa, ecdhe_ecdsa, + Config, DefaultCurve), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), ECCOpts = [{honor_ecc_order, false}], - case ssl_test_lib:supported_eccs([{eccs, [sect571r1]}]) of - true -> ssl_test_lib:ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); + case ssl_test_lib:supported_eccs([{eccs, [DefaultCurve]}]) of + true -> ssl_test_lib:ecc_test(DefaultCurve, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. ecc_client_order_custom_curves(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], - ecdhe_ecdsa, ecdhe_ecdsa, Config), + ecdhe_ecdsa, ecdhe_ecdsa, + Config, DefaultCurve), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{honor_ecc_order, false}, {eccs, [secp256r1, sect571r1]}], + ECCOpts = [{honor_ecc_order, false}, {eccs, [secp256r1, DefaultCurve]}], case ssl_test_lib:supported_eccs(ECCOpts) of - true -> ssl_test_lib:ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); + true -> ssl_test_lib:ecc_test(DefaultCurve, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. @@ -274,12 +282,13 @@ ecc_unknown_curve(Config) -> client_ecdh_rsa_server_ecdhe_ecdsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], ecdh_rsa, ecdhe_ecdsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], + ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, DefaultCurve]}], case ssl_test_lib:supported_eccs(ECCOpts) of true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} @@ -287,12 +296,13 @@ client_ecdh_rsa_server_ecdhe_ecdsa_server_custom(Config) -> client_ecdh_rsa_server_ecdhe_rsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], ecdh_rsa, ecdhe_rsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], + ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, DefaultCurve]}], case ssl_test_lib:supported_eccs(ECCOpts) of true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); @@ -301,12 +311,13 @@ client_ecdh_rsa_server_ecdhe_rsa_server_custom(Config) -> client_ecdhe_rsa_server_ecdhe_ecdsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], ecdhe_rsa, ecdhe_ecdsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], + ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, DefaultCurve]}], case ssl_test_lib:supported_eccs(ECCOpts) of true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} @@ -314,19 +325,21 @@ client_ecdhe_rsa_server_ecdhe_ecdsa_server_custom(Config) -> client_ecdhe_rsa_server_ecdhe_rsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], ecdhe_rsa, ecdhe_rsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], + ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, DefaultCurve]}], case ssl_test_lib:supported_eccs(ECCOpts) of true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. client_ecdhe_rsa_server_ecdh_rsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), Ext = x509_test:extensions([{key_usage, [keyEncipherment]}]), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, [[], [], [{extensions, Ext}]]}, {client_chain, Default}], @@ -334,8 +347,8 @@ client_ecdhe_rsa_server_ecdh_rsa_server_custom(Config) -> COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], - Expected = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), %% The certificate curve + ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, DefaultCurve]}], + Expected = secp256r1, %% The certificate curve case ssl_test_lib:supported_eccs(ECCOpts) of true -> ssl_test_lib:ecc_test(Expected, COpts, SOpts, [], ECCOpts, Config); @@ -344,12 +357,13 @@ client_ecdhe_rsa_server_ecdh_rsa_server_custom(Config) -> client_ecdhe_ecdsa_server_ecdhe_ecdsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], ecdhe_ecdsa, ecdhe_ecdsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], + ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, DefaultCurve]}], case ssl_test_lib:supported_eccs(ECCOpts) of true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} @@ -357,12 +371,13 @@ client_ecdhe_ecdsa_server_ecdhe_ecdsa_server_custom(Config) -> client_ecdhe_ecdsa_server_ecdhe_rsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], ecdhe_ecdsa, ecdhe_rsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], + ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, DefaultCurve]}], case ssl_test_lib:supported_eccs(ECCOpts) of true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} @@ -370,12 +385,13 @@ client_ecdhe_ecdsa_server_ecdhe_rsa_server_custom(Config) -> client_ecdhe_ecdsa_server_ecdhe_ecdsa_client_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], ecdhe_ecdsa, ecdhe_ecdsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{eccs, [secp256r1, sect571r1]}], + ECCOpts = [{eccs, [secp256r1, DefaultCurve]}], case ssl_test_lib:supported_eccs(ECCOpts) of true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, ECCOpts, [], Config); false -> {skip, "unsupported named curves"} @@ -383,12 +399,13 @@ client_ecdhe_ecdsa_server_ecdhe_ecdsa_client_custom(Config) -> client_ecdhe_rsa_server_ecdhe_ecdsa_client_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), + DefaultCurve = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], ecdhe_rsa, ecdhe_ecdsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), - ECCOpts = [{eccs, [secp256r1, sect571r1]}], + ECCOpts = [{eccs, [secp256r1, DefaultCurve]}], case ssl_test_lib:supported_eccs(ECCOpts) of true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, ECCOpts, [], Config); false -> {skip, "unsupported named curves"} diff --git a/lib/ssl/test/ssl_alpn_handshake_SUITE.erl b/lib/ssl/test/ssl_alpn_handshake_SUITE.erl index 04c4b257d9..dfc780479e 100644 --- a/lib/ssl/test/ssl_alpn_handshake_SUITE.erl +++ b/lib/ssl/test/ssl_alpn_handshake_SUITE.erl @@ -153,41 +153,41 @@ protocols_must_be_a_binary_list(Config) when is_list(Config) -> empty_client(Config) when is_list(Config) -> run_failing_handshake(Config, - [{alpn_advertised_protocols, []}], - [{alpn_preferred_protocols, [<<"spdy/2">>, <<"spdy/3">>, <<"http/2">>]}], - {error,{tls_alert,"no application protocol"}}). + [{alpn_advertised_protocols, []}], + [{alpn_preferred_protocols, [<<"spdy/2">>, <<"spdy/3">>, <<"http/2">>]}], + no_application_protocol). %-------------------------------------------------------------------------------- empty_server(Config) when is_list(Config) -> run_failing_handshake(Config, - [{alpn_advertised_protocols, [<<"http/1.0">>, <<"http/1.1">>]}], - [{alpn_preferred_protocols, []}], - {error,{tls_alert,"no application protocol"}}). + [{alpn_advertised_protocols, [<<"http/1.0">>, <<"http/1.1">>]}], + [{alpn_preferred_protocols, []}], + no_application_protocol). %-------------------------------------------------------------------------------- empty_client_empty_server(Config) when is_list(Config) -> run_failing_handshake(Config, - [{alpn_advertised_protocols, []}], - [{alpn_preferred_protocols, []}], - {error,{tls_alert,"no application protocol"}}). + [{alpn_advertised_protocols, []}], + [{alpn_preferred_protocols, []}], + no_application_protocol). %-------------------------------------------------------------------------------- no_matching_protocol(Config) when is_list(Config) -> run_failing_handshake(Config, - [{alpn_advertised_protocols, [<<"http/1.0">>, <<"http/1.1">>]}], - [{alpn_preferred_protocols, [<<"spdy/2">>, <<"spdy/3">>, <<"http/2">>]}], - {error,{tls_alert,"no application protocol"}}). + [{alpn_advertised_protocols, [<<"http/1.0">>, <<"http/1.1">>]}], + [{alpn_preferred_protocols, [<<"spdy/2">>, <<"spdy/3">>, <<"http/2">>]}], + no_application_protocol). %-------------------------------------------------------------------------------- client_alpn_and_server_alpn(Config) when is_list(Config) -> run_handshake(Config, - [{alpn_advertised_protocols, [<<"http/1.0">>, <<"http/1.1">>]}], - [{alpn_preferred_protocols, [<<"spdy/2">>, <<"http/1.1">>, <<"http/1.0">>]}], - {ok, <<"http/1.1">>}). + [{alpn_advertised_protocols, [<<"http/1.0">>, <<"http/1.1">>]}], + [{alpn_preferred_protocols, [<<"spdy/2">>, <<"http/1.1">>, <<"http/1.0">>]}], + {ok, <<"http/1.1">>}). %-------------------------------------------------------------------------------- @@ -262,52 +262,12 @@ client_renegotiate(Config) when is_list(Config) -> %-------------------------------------------------------------------------------- session_reused(Config) when is_list(Config)-> - ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), ClientOpts = [{alpn_advertised_protocols, [<<"http/1.0">>]}] ++ ClientOpts0, ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), ServerOpts = [{alpn_preferred_protocols, [<<"spdy/2">>, <<"http/1.1">>, <<"http/1.0">>]}] ++ ServerOpts0, - {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), - Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, - {from, self()}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {options, ServerOpts}]), - - Port = ssl_test_lib:inet_port(Server), - Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, - {host, Hostname}, - {from, self()}, - {mfa, {ssl_test_lib, no_result_msg, []}}, - {options, ClientOpts}]), - - SessionInfo = - receive - {Server, Info} -> - Info - end, - - Server ! {listen, {mfa, {ssl_test_lib, no_result, []}}}, - - %% Make sure session is registered - ct:sleep(?SLEEP), - - Client1 = - ssl_test_lib:start_client([{node, ClientNode}, - {port, Port}, {host, Hostname}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {from, self()}, {options, ClientOpts}]), - - receive - {Client1, SessionInfo} -> - ok; - {Client1, Other} -> - ct:fail(Other) - end, - - ssl_test_lib:close(Server), - ssl_test_lib:close(Client), - ssl_test_lib:close(Client1). - + ssl_test_lib:reuse_session(ClientOpts, ServerOpts, Config). %-------------------------------------------------------------------------------- alpn_not_supported_client(Config) when is_list(Config) -> @@ -337,7 +297,7 @@ alpn_not_supported_server(Config) when is_list(Config)-> %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- -run_failing_handshake(Config, ClientExtraOpts, ServerExtraOpts, ExpectedResult) -> +run_failing_handshake(Config, ClientExtraOpts, ServerExtraOpts, ExpectedAlert) -> ClientOpts = ClientExtraOpts ++ ssl_test_lib:ssl_options(client_rsa_opts, Config), ServerOpts = ServerExtraOpts ++ ssl_test_lib:ssl_options(server_rsa_opts, Config), @@ -353,8 +313,7 @@ run_failing_handshake(Config, ClientExtraOpts, ServerExtraOpts, ExpectedResult) {from, self()}, {mfa, {?MODULE, placeholder, []}}, {options, ClientOpts}]), - ssl_test_lib:check_result(Server, ExpectedResult, - Client, ExpectedResult). + ssl_test_lib:check_client_alert(Server, Client, ExpectedAlert). run_handshake(Config, ClientExtraOpts, ServerExtraOpts, ExpectedProtocol) -> Data = "hello world", diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index 90fcde609f..0a66dfa0b1 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2018. All Rights Reserved. +%% Copyright Ericsson AB 2007-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -166,6 +166,7 @@ api_tests() -> socket_options, cipher_suites, handshake_continue, + handshake_continue_timeout, hello_client_cancel, hello_server_cancel ]. @@ -654,8 +655,8 @@ new_options_in_accept(Config) when is_list(Config) -> handshake_continue() -> [{doc, "Test API function ssl:handshake_continue/3"}]. handshake_continue(Config) when is_list(Config) -> - ClientOpts = ssl_test_lib:ssl_options(client_verification_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_verification_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, @@ -681,6 +682,34 @@ handshake_continue(Config) when is_list(Config) -> ssl_test_lib:close(Server), ssl_test_lib:close(Client). +%%------------------------------------------------------------------ +handshake_continue_timeout() -> + [{doc, "Test API function ssl:handshake_continue/3 with short timeout"}]. +handshake_continue_timeout(Config) when is_list(Config) -> + ClientOpts = ssl_test_lib:ssl_options(client_verification_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_verification_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {timeout, 1}, + {options, ssl_test_lib:ssl_options([{reuseaddr, true}, {handshake, hello}], + Config)}, + {continue_options, proplists:delete(reuseaddr, ServerOpts)} + ]), + + Port = ssl_test_lib:inet_port(Server), + + + {connect_failed, _} = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {options, ClientOpts}]), + + ssl_test_lib:check_result(Server, {error,timeout}), + ssl_test_lib:close(Server). + + %%-------------------------------------------------------------------- hello_client_cancel() -> [{doc, "Test API function ssl:handshake_cancel/1 on the client side"}]. @@ -702,19 +731,12 @@ hello_client_cancel(Config) when is_list(Config) -> {from, self()}, {options, ssl_test_lib:ssl_options([{handshake, hello}], Config)}, {continue_options, cancel}]), - receive - {Server, {error, {tls_alert, "user canceled"}}} -> - ok; - {Server, {error, closed}} -> - ct:pal("Did not receive the ALERT"), - ok - end. - + ssl_test_lib:check_server_alert(Server, user_canceled). %%-------------------------------------------------------------------- hello_server_cancel() -> [{doc, "Test API function ssl:handshake_cancel/1 on the server side"}]. hello_server_cancel(Config) when is_list(Config) -> - ClientOpts = ssl_test_lib:ssl_options(client_verification_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, @@ -756,8 +778,8 @@ prf(Config) when is_list(Config) -> secret_connection_info() -> [{doc,"Test the API function ssl:connection_information/2"}]. secret_connection_info(Config) when is_list(Config) -> - ClientOpts = ssl_test_lib:ssl_options(client_verification_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_verification_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, @@ -1183,9 +1205,8 @@ fallback(Config) when is_list(Config) -> [{fallback, true}, {versions, ['tlsv1']} | ClientOpts]}]), - - ssl_test_lib:check_result(Server, {error,{tls_alert,"inappropriate fallback"}}, - Client, {error,{tls_alert,"inappropriate fallback"}}). + ssl_test_lib:check_server_alert(Server, Client, inappropriate_fallback). + %%-------------------------------------------------------------------- cipher_format() -> @@ -1446,8 +1467,8 @@ cipher_suites_mix() -> cipher_suites_mix(Config) when is_list(Config) -> CipherSuites = [{dhe_rsa,aes_128_cbc,sha256,sha256}, {dhe_rsa,aes_128_cbc,sha}], - ClientOpts = ssl_test_lib:ssl_options(client_verification_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_verification_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), @@ -2358,8 +2379,8 @@ invalid_options() -> [{doc,"Test what happens when we give invalid options"}]. invalid_options(Config) when is_list(Config) -> - ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Check = fun(Client, Server, {versions, [sslv2, sslv3]} = Option) -> @@ -2374,27 +2395,28 @@ invalid_options(Config) when is_list(Config) -> {error, {options, Option}}) end, - TestOpts = [{versions, [sslv2, sslv3]}, - {verify, 4}, - {verify_fun, function}, - {fail_if_no_peer_cert, 0}, - {verify_client_once, 1}, - {depth, four}, - {certfile, 'cert.pem'}, - {keyfile,'key.pem' }, - {password, foo}, - {cacertfile, ""}, - {dhfile,'dh.pem' }, - {ciphers, [{foo, bar, sha, ignore}]}, - {reuse_session, foo}, - {reuse_sessions, 0}, - {renegotiate_at, "10"}, - {mode, depech}, - {packet, 8.0}, - {packet_size, "2"}, - {header, a}, - {active, trice}, - {key, 'key.pem' }], + TestOpts = + [{versions, [sslv2, sslv3]}, + {verify, 4}, + {verify_fun, function}, + {fail_if_no_peer_cert, 0}, + {verify_client_once, 1}, + {depth, four}, + {certfile, 'cert.pem'}, + {keyfile,'key.pem' }, + {password, foo}, + {cacertfile, ""}, + {dhfile,'dh.pem' }, + {ciphers, [{foo, bar, sha, ignore}]}, + {reuse_session, foo}, + {reuse_sessions, 0}, + {renegotiate_at, "10"}, + {mode, depech}, + {packet, 8.0}, + {packet_size, "2"}, + {header, a}, + {active, trice}, + {key, 'key.pem' }], [begin Server = @@ -2650,8 +2672,7 @@ default_reject_anonymous(Config) when is_list(Config) -> [{ciphers,[CipherSuite]} | ClientOpts]}]), - ssl_test_lib:check_result(Server, {error, {tls_alert, "insufficient security"}}, - Client, {error, {tls_alert, "insufficient security"}}). + ssl_test_lib:check_server_alert(Server, Client, insufficient_security). %%-------------------------------------------------------------------- ciphers_ecdsa_signed_certs() -> @@ -2687,175 +2708,69 @@ ciphers_ecdh_rsa_signed_certs_openssl_names(Config) when is_list(Config) -> reuse_session() -> [{doc,"Test reuse of sessions (short handshake)"}]. reuse_session(Config) when is_list(Config) -> - ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_opts, Config), - {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), - - Server = - ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, - {from, self()}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {options, ServerOpts}]), - Port = ssl_test_lib:inet_port(Server), - Client0 = - ssl_test_lib:start_client([{node, ClientNode}, - {port, Port}, {host, Hostname}, - {mfa, {ssl_test_lib, no_result, []}}, - {from, self()}, {options, ClientOpts}]), - SessionInfo = - receive - {Server, Info} -> - Info - end, - - Server ! {listen, {mfa, {ssl_test_lib, no_result, []}}}, - - %% Make sure session is registered - ct:sleep(?SLEEP), - - Client1 = - ssl_test_lib:start_client([{node, ClientNode}, - {port, Port}, {host, Hostname}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {from, self()}, {options, ClientOpts}]), - receive - {Client1, SessionInfo} -> - ok; - {Client1, Other} -> - ct:log("Expected: ~p, Unexpected: ~p~n", - [SessionInfo, Other]), - ct:fail(session_not_reused) - end, - - Server ! {listen, {mfa, {ssl_test_lib, no_result, []}}}, - - Client2 = - ssl_test_lib:start_client([{node, ClientNode}, - {port, Port}, {host, Hostname}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {from, self()}, {options, [{reuse_sessions, false} - | ClientOpts]}]), - receive - {Client2, SessionInfo} -> - ct:fail( - session_reused_when_session_reuse_disabled_by_client); - {Client2, _} -> - ok - end, - - ssl_test_lib:close(Server), - - Server1 = - ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, - {from, self()}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {options, [{reuse_sessions, false} | ServerOpts]}]), - - Port1 = ssl_test_lib:inet_port(Server1), - Client3 = - ssl_test_lib:start_client([{node, ClientNode}, - {port, Port1}, {host, Hostname}, - {mfa, {ssl_test_lib, no_result, []}}, - {from, self()}, {options, ClientOpts}]), - - SessionInfo1 = - receive - {Server1, Info1} -> - Info1 - end, - - Server1 ! {listen, {mfa, {ssl_test_lib, no_result, []}}}, + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), - %% Make sure session is registered - ct:sleep(?SLEEP), - - Client4 = - ssl_test_lib:start_client([{node, ClientNode}, - {port, Port1}, {host, Hostname}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {from, self()}, {options, ClientOpts}]), - - receive - {Client4, SessionInfo1} -> - ct:fail( - session_reused_when_session_reuse_disabled_by_server); - {Client4, _Other} -> - ct:log("OTHER: ~p ~n", [_Other]), - ok - end, - - ssl_test_lib:close(Server1), - ssl_test_lib:close(Client0), - ssl_test_lib:close(Client1), - ssl_test_lib:close(Client2), - ssl_test_lib:close(Client3), - ssl_test_lib:close(Client4). - + ssl_test_lib:reuse_session(ClientOpts, ServerOpts, Config). %%-------------------------------------------------------------------- reuse_session_expired() -> [{doc,"Test sessions is not reused when it has expired"}]. reuse_session_expired(Config) when is_list(Config) -> - ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), - - Server = - ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + + Server0 = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {options, ServerOpts}]), - Port = ssl_test_lib:inet_port(Server), - Client0 = - ssl_test_lib:start_client([{node, ClientNode}, - {port, Port}, {host, Hostname}, - {mfa, {ssl_test_lib, no_result, []}}, - {from, self()}, {options, ClientOpts}]), - SessionInfo = - receive - {Server, Info} -> - Info - end, - - Server ! {listen, {mfa, {ssl_test_lib, no_result, []}}}, + {mfa, {ssl_test_lib, no_result, []}}, + {tcp_options, [{active, false}]}, + {options, ServerOpts}]), + Port0 = ssl_test_lib:inet_port(Server0), - %% Make sure session is registered - ct:sleep(?SLEEP), - - Client1 = - ssl_test_lib:start_client([{node, ClientNode}, - {port, Port}, {host, Hostname}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {from, self()}, {options, ClientOpts}]), + Client0 = ssl_test_lib:start_client([{node, ClientNode}, + {port, Port0}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, [{reuse_sessions, save} | ClientOpts]}]), + Server0 ! listen, + + Client1 = ssl_test_lib:start_client([{node, ClientNode}, + {port, Port0}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, ClientOpts}]), + + SID = receive + {Client0, Id0} -> + Id0 + end, + receive - {Client1, SessionInfo} -> - ok; - {Client1, Other} -> - ct:log("Expected: ~p, Unexpected: ~p~n", - [SessionInfo, Other]), - ct:fail(session_not_reused) + {Client1, SID} -> + ok + after ?SLEEP -> + ct:fail(session_not_reused) end, - Server ! listen, - + Server0 ! listen, + %% Make sure session is unregistered due to expiration - ct:sleep((?EXPIRE+1)), - [{session_id, Id} |_] = SessionInfo, + ct:sleep((?EXPIRE*2)), - make_sure_expired(Hostname, Port, Id), + make_sure_expired(Hostname, Port0, SID), Client2 = ssl_test_lib:start_client([{node, ClientNode}, - {port, Port}, {host, Hostname}, - {mfa, {ssl_test_lib, session_info_result, []}}, + {port, Port0}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, {from, self()}, {options, ClientOpts}]), receive - {Client2, SessionInfo} -> + {Client2, SID} -> ct:fail(session_reused_when_session_expired); {Client2, _} -> ok end, process_flag(trap_exit, false), - ssl_test_lib:close(Server), + ssl_test_lib:close(Server0), ssl_test_lib:close(Client0), ssl_test_lib:close(Client1), ssl_test_lib:close(Client2). @@ -2864,16 +2779,16 @@ make_sure_expired(Host, Port, Id) -> {status, _, _, StatusInfo} = sys:get_status(whereis(ssl_manager)), [_, _,_, _, Prop] = StatusInfo, State = ssl_test_lib:state(Prop), - Cache = element(2, State), + ClientCache = element(2, State), - case ssl_session_cache:lookup(Cache, {{Host, Port}, Id}) of + case ssl_session_cache:lookup(ClientCache, {{Host, Port}, Id}) of undefined -> - ok; + ok; #session{is_resumable = false} -> - ok; + ok; _ -> ct:sleep(?SLEEP), - make_sure_expired(Host, Port, Id) + make_sure_expired(Host, Port, Id) end. %%-------------------------------------------------------------------- @@ -3609,8 +3524,7 @@ no_common_signature_algs(Config) when is_list(Config) -> {options, [{signature_algs, [{sha384, rsa}]} | ClientOpts]}]), - ssl_test_lib:check_result(Server, {error, {tls_alert, "insufficient security"}}, - Client, {error, {tls_alert, "insufficient security"}}). + ssl_test_lib:check_server_alert(Server, Client, insufficient_security). %%-------------------------------------------------------------------- @@ -3641,7 +3555,7 @@ tls_dont_crash_on_handshake_garbage(Config) -> <<22, 3,3, 5:16, 92,64,37,228,209>> % garbage ]), % Send unexpected change_cipher_spec - ok = gen_tcp:send(Socket, <<20, 0,0,12, 111,40,244,7,137,224,16,109,197,110,249,152>>), + ok = gen_tcp:send(Socket, <<20, 3,3, 12:16, 111,40,244,7,137,224,16,109,197,110,249,152>>), % Ensure we receive an alert, not sudden disconnect {ok, <<21, _/binary>>} = drop_handshakes(Socket, 1000). @@ -4169,6 +4083,9 @@ rizzo_one_n_minus_one(Config) when is_list(Config) -> {cipher, fun(rc4_128) -> false; + %% TODO: remove this clause when chacha is fixed! + (chacha20_poly1305) -> + false; (_) -> true end}]), @@ -4310,8 +4227,7 @@ tls_versions_option(Config) when is_list(Config) -> {Server, _} -> ok end, - - ssl_test_lib:check_result(ErrClient, {error, {tls_alert, "protocol version"}}). + ssl_test_lib:check_client_alert(ErrClient, protocol_version). %%-------------------------------------------------------------------- @@ -4486,8 +4402,8 @@ tcp_send_recv_result(Socket) -> ok. basic_verify_test_no_close(Config) -> - ClientOpts = ssl_test_lib:ssl_options(client_verification_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_verification_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), @@ -4962,16 +4878,16 @@ run_suites(Ciphers, Config, Type) -> {ClientOpts, ServerOpts} = case Type of rsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), [{ciphers, Ciphers} | - ssl_test_lib:ssl_options(server_verification_opts, Config)]}; + ssl_test_lib:ssl_options(server_rsa_opts, Config)]}; dsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_dsa_verify_opts, Config), [{ciphers, Ciphers} | ssl_test_lib:ssl_options(server_dsa_opts, Config)]}; anonymous -> %% No certs in opts! - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), [{ciphers, Ciphers} | ssl_test_lib:ssl_options([], Config)]}; psk -> @@ -4993,46 +4909,50 @@ run_suites(Ciphers, Config, Type) -> ssl_test_lib:ssl_options(server_psk_anon_hint, Config)]}; srp -> {ssl_test_lib:ssl_options(client_srp, Config), - ssl_test_lib:ssl_options(server_srp, Config)}; + [{ciphers, Ciphers} | + ssl_test_lib:ssl_options(server_srp, Config)]}; srp_anon -> {ssl_test_lib:ssl_options(client_srp, Config), - ssl_test_lib:ssl_options(server_srp_anon, Config)}; + [{ciphers, Ciphers} | + ssl_test_lib:ssl_options(server_srp_anon, Config)]}; srp_dsa -> {ssl_test_lib:ssl_options(client_srp_dsa, Config), - ssl_test_lib:ssl_options(server_srp_dsa, Config)}; + [{ciphers, Ciphers} | + ssl_test_lib:ssl_options(server_srp_dsa, Config)]}; ecdsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_ecdsa_opts, Config), [{ciphers, Ciphers} | ssl_test_lib:ssl_options(server_ecdsa_opts, Config)]}; ecdh_rsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), - ssl_test_lib:ssl_options(server_ecdh_rsa_opts, Config)}; + {ssl_test_lib:ssl_options(client_ecdh_rsa_opts, Config), + [{ciphers, Ciphers} | + ssl_test_lib:ssl_options(server_ecdh_rsa_opts, Config)]}; rc4_rsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), [{ciphers, Ciphers} | - ssl_test_lib:ssl_options(server_verification_opts, Config)]}; + ssl_test_lib:ssl_options(server_rsa_verify_opts, Config)]}; rc4_ecdh_rsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_ecdh_rsa_opts, Config), [{ciphers, Ciphers} | ssl_test_lib:ssl_options(server_ecdh_rsa_opts, Config)]}; rc4_ecdsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), [{ciphers, Ciphers} | ssl_test_lib:ssl_options(server_ecdsa_opts, Config)]}; des_dhe_rsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), [{ciphers, Ciphers} | ssl_test_lib:ssl_options(server_verification_opts, Config)]}; des_rsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), [{ciphers, Ciphers} | - ssl_test_lib:ssl_options(server_verification_opts, Config)]}; + ssl_test_lib:ssl_options(server_rsa_verify_opts, Config)]}; chacha_rsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), [{ciphers, Ciphers} | - ssl_test_lib:ssl_options(server_verification_opts, Config)]}; + ssl_test_lib:ssl_options(server_rsa_verify_opts, Config)]}; chacha_ecdsa -> - {ssl_test_lib:ssl_options(client_verification_opts, Config), + {ssl_test_lib:ssl_options(client_ecdsa_opts, Config), [{ciphers, Ciphers} | ssl_test_lib:ssl_options(server_ecdsa_opts, Config)]} end, diff --git a/lib/ssl/test/ssl_certificate_verify_SUITE.erl b/lib/ssl/test/ssl_certificate_verify_SUITE.erl index 588ca153a9..c0a5367a57 100644 --- a/lib/ssl/test/ssl_certificate_verify_SUITE.erl +++ b/lib/ssl/test/ssl_certificate_verify_SUITE.erl @@ -298,15 +298,8 @@ server_require_peer_cert_fail(Config) when is_list(Config) -> {host, Hostname}, {from, self()}, {options, [{active, Active} | BadClientOpts]}]), - receive - {Server, {error, {tls_alert, "handshake failure"}}} -> - receive - {Client, {error, {tls_alert, "handshake failure"}}} -> - ok; - {Client, {error, closed}} -> - ok - end - end. + + ssl_test_lib:check_server_alert(Server, Client, handshake_failure). %%-------------------------------------------------------------------- server_require_peer_cert_empty_ok() -> @@ -365,15 +358,8 @@ server_require_peer_cert_partial_chain(Config) when is_list(Config) -> {options, [{active, Active}, {cacerts, [RootCA]} | proplists:delete(cacertfile, ClientOpts)]}]), - receive - {Server, {error, {tls_alert, "unknown ca"}}} -> - receive - {Client, {error, {tls_alert, "unknown ca"}}} -> - ok; - {Client, {error, closed}} -> - ok - end - end. + ssl_test_lib:check_server_alert(Server, Client, unknown_ca). + %%-------------------------------------------------------------------- server_require_peer_cert_allow_partial_chain() -> [{doc, "Server trusts intermediat CA and accepts a partial chain. (partial_chain option)"}]. @@ -446,17 +432,7 @@ server_require_peer_cert_do_not_allow_partial_chain(Config) when is_list(Config) {from, self()}, {mfa, {ssl_test_lib, no_result, []}}, {options, ClientOpts}]), - - receive - {Server, {error, {tls_alert, "unknown ca"}}} -> - receive - {Client, {error, {tls_alert, "unknown ca"}}} -> - ok; - {Client, {error, closed}} -> - ok - end - end. - + ssl_test_lib:check_server_alert(Server, Client, unknown_ca). %%-------------------------------------------------------------------- server_require_peer_cert_partial_chain_fun_fail() -> [{doc, "If parial_chain fun crashes, treat it as if it returned unkown_ca"}]. @@ -487,16 +463,7 @@ server_require_peer_cert_partial_chain_fun_fail(Config) when is_list(Config) -> {from, self()}, {mfa, {ssl_test_lib, no_result, []}}, {options, ClientOpts}]), - - receive - {Server, {error, {tls_alert, "unknown ca"}}} -> - receive - {Client, {error, {tls_alert, "unknown ca"}}} -> - ok; - {Client, {error, closed}} -> - ok - end - end. + ssl_test_lib:check_server_alert(Server, Client, unknown_ca). %%-------------------------------------------------------------------- verify_fun_always_run_client() -> @@ -535,14 +502,8 @@ verify_fun_always_run_client(Config) when is_list(Config) -> [{verify, verify_peer}, {verify_fun, FunAndState} | ClientOpts]}]), - %% Server error may be {tls_alert,"handshake failure"} or closed depending on timing - %% this is not a bug it is a circumstance of how tcp works! - receive - {Server, ServerError} -> - ct:log("Server Error ~p~n", [ServerError]) - end, - ssl_test_lib:check_result(Client, {error, {tls_alert, "handshake failure"}}). + ssl_test_lib:check_client_alert(Server, Client, handshake_failure). %%-------------------------------------------------------------------- verify_fun_always_run_server() -> @@ -581,16 +542,8 @@ verify_fun_always_run_server(Config) when is_list(Config) -> {mfa, {ssl_test_lib, no_result, []}}, {options, ClientOpts}]), - - %% Client error may be {tls_alert, "handshake failure" } or closed depending on timing - %% this is not a bug it is a circumstance of how tcp works! - receive - {Client, ClientError} -> - ct:log("Client Error ~p~n", [ClientError]) - end, - - ssl_test_lib:check_result(Server, {error, {tls_alert, "handshake failure"}}). - + + ssl_test_lib:check_client_alert(Server, Client, handshake_failure). %%-------------------------------------------------------------------- cert_expired() -> @@ -620,8 +573,7 @@ cert_expired(Config) when is_list(Config) -> {from, self()}, {options, [{verify, verify_peer}, {active, Active} | ClientOpts]}]), - ssl_test_lib:check_result(Server, {error, {tls_alert, "certificate expired"}}, - Client, {error, {tls_alert, "certificate expired"}}). + ssl_test_lib:check_client_alert(Server, Client, certificate_expired). two_digits_str(N) when N < 10 -> lists:flatten(io_lib:format("0~p", [N])); @@ -727,12 +679,8 @@ critical_extension_verify_server(Config) when is_list(Config) -> {options, [{verify, verify_none}, {active, Active} | ClientOpts]}]), %% This certificate has a critical extension that we don't - %% understand. Therefore, verification should fail. - - ssl_test_lib:check_result(Server, {error, {tls_alert, "unsupported certificate"}}, - Client, {error, {tls_alert, "unsupported certificate"}}), - - ssl_test_lib:close(Server). + %% understand. Therefore, verification should fail. + ssl_test_lib:check_server_alert(Server, Client, unsupported_certificate). %%-------------------------------------------------------------------- critical_extension_verify_client() -> @@ -763,12 +711,7 @@ critical_extension_verify_client(Config) when is_list(Config) -> {mfa, {ssl_test_lib, ReceiveFunction, []}}, {options, [{verify, verify_peer}, {active, Active} | ClientOpts]}]), - %% This certificate has a critical extension that we don't - %% understand. Therefore, verification should fail. - ssl_test_lib:check_result(Server, {error, {tls_alert, "unsupported certificate"}}, - Client, {error, {tls_alert, "unsupported certificate"}}), - - ssl_test_lib:close(Server). + ssl_test_lib:check_client_alert(Server, Client, unsupported_certificate). %%-------------------------------------------------------------------- critical_extension_verify_none() -> @@ -908,10 +851,7 @@ invalid_signature_server(Config) when is_list(Config) -> {host, Hostname}, {from, self()}, {options, [{verify, verify_peer} | ClientOpts]}]), - - ssl_test_lib:check_result(Server, {error, {tls_alert, "unknown ca"}}, - Client, {error, {tls_alert, "unknown ca"}}). - + ssl_test_lib:check_server_alert(Server, Client, unknown_ca). %%-------------------------------------------------------------------- invalid_signature_client() -> @@ -946,9 +886,7 @@ invalid_signature_client(Config) when is_list(Config) -> {from, self()}, {options, NewClientOpts}]), - ssl_test_lib:check_result(Server, {error, {tls_alert, "unknown ca"}}, - Client, {error, {tls_alert, "unknown ca"}}). - + ssl_test_lib:check_client_alert(Server, Client, unknown_ca). %%-------------------------------------------------------------------- @@ -1034,16 +972,7 @@ unknown_server_ca_fail(Config) when is_list(Config) -> [{verify, verify_peer}, {verify_fun, FunAndState} | ClientOpts]}]), - receive - {Client, {error, {tls_alert, "unknown ca"}}} -> - receive - {Server, {error, {tls_alert, "unknown ca"}}} -> - ok; - {Server, {error, closed}} -> - ok - end - end. - + ssl_test_lib:check_client_alert(Server, Client, unknown_ca). %%-------------------------------------------------------------------- unknown_server_ca_accept_verify_none() -> @@ -1193,11 +1122,7 @@ customize_hostname_check(Config) when is_list(Config) -> {mfa, {ssl_test_lib, no_result, []}}, {options, ClientOpts} ]), - ssl_test_lib:check_result(Client1, {error, {tls_alert, "handshake failure"}}, - Server, {error, {tls_alert, "handshake failure"}}), - - ssl_test_lib:close(Server), - ssl_test_lib:close(Client). + ssl_test_lib:check_client_alert(Server, Client1, handshake_failure). incomplete_chain() -> [{doc,"Test option verify_peer"}]. diff --git a/lib/ssl/test/ssl_crl_SUITE.erl b/lib/ssl/test/ssl_crl_SUITE.erl index 23c5eaf84d..b2fd3874a8 100644 --- a/lib/ssl/test/ssl_crl_SUITE.erl +++ b/lib/ssl/test/ssl_crl_SUITE.erl @@ -238,7 +238,7 @@ crl_verify_revoked(Config) when is_list(Config) -> end, crl_verify_error(Hostname, ServerNode, ServerOpts, ClientNode, ClientOpts, - "certificate revoked"). + certificate_revoked). crl_verify_no_crl() -> [{doc,"Verify a simple CRL chain when the CRL is missing"}]. @@ -277,10 +277,10 @@ crl_verify_no_crl(Config) when is_list(Config) -> %% The error "revocation status undetermined" gets turned %% into "bad certificate". crl_verify_error(Hostname, ServerNode, ServerOpts, ClientNode, ClientOpts, - "bad certificate"); + bad_certificate); peer -> crl_verify_error(Hostname, ServerNode, ServerOpts, ClientNode, ClientOpts, - "bad certificate"); + bad_certificate); best_effort -> %% In "best effort" mode, we consider the certificate not %% to be revoked if we can't find the appropriate CRL. @@ -341,7 +341,7 @@ crl_hash_dir_collision(Config) when is_list(Config) -> %% First certificate revoked; first fails, second succeeds. crl_verify_error(Hostname, ServerNode, ServerOpts1, ClientNode, ClientOpts, - "certificate revoked"), + certificate_revoked), crl_verify_valid(Hostname, ServerNode, ServerOpts2, ClientNode, ClientOpts), make_certs:revoke(PrivDir, CA2, "collision-client-2", CertsConfig), @@ -352,9 +352,9 @@ crl_hash_dir_collision(Config) when is_list(Config) -> %% Second certificate revoked; both fail. crl_verify_error(Hostname, ServerNode, ServerOpts1, ClientNode, ClientOpts, - "certificate revoked"), + certificate_revoked), crl_verify_error(Hostname, ServerNode, ServerOpts2, ClientNode, ClientOpts, - "certificate revoked"), + certificate_revoked), ok. @@ -383,8 +383,11 @@ crl_hash_dir_expired(Config) when is_list(Config) -> {verify, verify_peer}], {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), - %% First make a CRL that expired yesterday. - make_certs:gencrl(PrivDir, CA, CertsConfig, -24), + %% First make a CRL that will expire in one second. + make_certs:gencrl_sec(PrivDir, CA, CertsConfig, 1), + %% Sleep until the next CRL is due + ct:sleep({seconds, 1}), + CrlDir = filename:join(PrivDir, "crls"), populate_crl_hash_dir(PrivDir, CrlDir, [{CA, "1627b4b0"}], @@ -397,10 +400,10 @@ crl_hash_dir_expired(Config) when is_list(Config) -> %% The error "revocation status undetermined" gets turned %% into "bad certificate". crl_verify_error(Hostname, ServerNode, ServerOpts, ClientNode, ClientOpts, - "bad certificate"); + bad_certificate); peer -> crl_verify_error(Hostname, ServerNode, ServerOpts, ClientNode, ClientOpts, - "bad certificate"); + bad_certificate); best_effort -> %% In "best effort" mode, we consider the certificate not %% to be revoked if we can't find the appropriate CRL. @@ -448,11 +451,8 @@ crl_verify_error(Hostname, ServerNode, ServerOpts, ClientNode, ClientOpts, Expec {host, Hostname}, {from, self()}, {options, ClientOpts}]), - receive - {Server, AlertOrClose} -> - ct:pal("Server Alert or Close ~p", [AlertOrClose]) - end, - ssl_test_lib:check_result(Client, {error, {tls_alert, ExpectedAlert}}). + + ssl_test_lib:check_client_alert(Server, Client, ExpectedAlert). %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ diff --git a/lib/ssl/test/ssl_dist_bench_SUITE.erl b/lib/ssl/test/ssl_dist_bench_SUITE.erl index 7409b69639..618ad0789e 100644 --- a/lib/ssl/test/ssl_dist_bench_SUITE.erl +++ b/lib/ssl/test/ssl_dist_bench_SUITE.erl @@ -1,7 +1,7 @@ %%%------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017-2018. All Rights Reserved. +%% Copyright Ericsson AB 2017-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -42,7 +42,7 @@ throughput_1048576/1]). %% Debug --export([payload/1]). +-export([payload/1, roundtrip_runner/3, setup_runner/3, throughput_runner/4]). %%%------------------------------------------------------------------- @@ -407,17 +407,19 @@ throughput(A, B, Prefix, HA, HB, Packets, Size) -> [] = ssl_apply(HA, erlang, nodes, []), [] = ssl_apply(HB, erlang, nodes, []), #{time := Time, - dist_stats := DistStats, + client_dist_stats := ClientDistStats, client_msacc_stats := ClientMsaccStats, client_prof := ClientProf, server_msacc_stats := ServerMsaccStats, - server_prof := ServerProf} = + server_prof := ServerProf, + server_gc_before := Server_GC_Before, + server_gc_after := Server_GC_After} = ssl_apply(HA, fun () -> throughput_runner(A, B, Packets, Size) end), [B] = ssl_apply(HA, erlang, nodes, []), [A] = ssl_apply(HB, erlang, nodes, []), ClientMsaccStats =:= undefined orelse msacc:print(ClientMsaccStats), - io:format("DistStats: ~p~n", [DistStats]), + io:format("ClientDistStats: ~p~n", [ClientDistStats]), Overhead = 50 % Distribution protocol headers (empirical) (TLS+=54) + byte_size(erlang:term_to_binary([0|<<>>])), % Benchmark overhead @@ -436,6 +438,8 @@ throughput(A, B, Prefix, HA, HB, Packets, Size) -> end, io:format("******* ClientProf:~n", []), prof_print(ClientProf), io:format("******* ServerProf:~n", []), prof_print(ServerProf), + io:format("******* Server GC Before:~n~p~n", [Server_GC_Before]), + io:format("******* Server GC After:~n~p~n", [Server_GC_After]), Speed = round((Bytes * 1000000) / (1024 * Time)), report(Prefix++" Throughput_"++integer_to_list(Size), Speed, "kB/s"). @@ -457,10 +461,10 @@ throughput_runner(A, B, Rounds, Size) -> ok end, prof_start(), - {Time,ServerMsaccStats,ServerProf} = + #{time := Time} = Result = throughput_client(ServerPid, ServerMon, Payload, Rounds), prof_stop(), - ClientMsaccStats = + MsaccStats = case msacc:available() of true -> MStats = msacc:stats(), @@ -469,15 +473,13 @@ throughput_runner(A, B, Rounds, Size) -> false -> undefined end, - ClientProf = prof_end(), + Prof = prof_end(), [{_Node,Socket}] = dig_dist_node_sockets(), DistStats = inet:getstat(Socket), - #{time => microseconds(Time), - dist_stats => DistStats, - client_msacc_stats => ClientMsaccStats, - client_prof => ClientProf, - server_msacc_stats => ServerMsaccStats, - server_prof => ServerProf}. + Result#{time := microseconds(Time), + client_dist_stats => DistStats, + client_msacc_stats => MsaccStats, + client_prof => Prof}. dig_dist_node_sockets() -> [case DistCtrl of @@ -500,6 +502,9 @@ dig_dist_node_sockets() -> throughput_server(Pid, N) -> + GC_Before = get_server_gc_info(), + %% dbg:tracer(port, dbg:trace_port(file, "throughput_server_gc.log")), + %% dbg:p(TLSDistReceiver, garbage_collection), msacc:available() andalso begin msacc:stop(), @@ -508,9 +513,9 @@ throughput_server(Pid, N) -> ok end, prof_start(), - throughput_server_loop(Pid, N). + throughput_server_loop(Pid, GC_Before, N). -throughput_server_loop(_Pid, 0) -> +throughput_server_loop(_Pid, GC_Before, 0) -> prof_stop(), MsaccStats = case msacc:available() of @@ -523,11 +528,26 @@ throughput_server_loop(_Pid, 0) -> undefined end, Prof = prof_end(), - exit({ok,MsaccStats,Prof}); -throughput_server_loop(Pid, N) -> + %% dbg:flush_trace_port(), + exit(#{server_msacc_stats => MsaccStats, + server_prof => Prof, + server_gc_before => GC_Before, + server_gc_after => get_server_gc_info()}); +throughput_server_loop(Pid, GC_Before, N) -> receive {Pid, N, _} -> - throughput_server_loop(Pid, N-1) + throughput_server_loop(Pid, GC_Before, N-1) + end. + +get_server_gc_info() -> + case whereis(ssl_connection_sup_dist) of + undefined -> + undefined; + SupPid -> + [{_Id,TLSDistReceiver,_Type,_Modules}|_] = + supervisor:which_children(SupPid), + erlang:process_info( + TLSDistReceiver, [garbage_collection,garbage_collection_info]) end. throughput_client(Pid, Mon, Payload, N) -> @@ -535,8 +555,8 @@ throughput_client(Pid, Mon, Payload, N) -> throughput_client_loop(_Pid, Mon, _Payload, 0, StartTime) -> receive - {'DOWN', Mon, _, _, {ok,MsaccStats,Prof}} -> - {elapsed_time(StartTime),MsaccStats,Prof}; + {'DOWN', Mon, _, _, #{} = Result} -> + Result#{time => elapsed_time(StartTime)}; {'DOWN', Mon, _, _, Other} -> exit(Other) end; @@ -554,6 +574,7 @@ prof_start() -> ok. -elif(?prof =:= eprof). prof_start() -> + catch eprof:stop(), {ok,_} = eprof:start(), profiling = eprof:start_profiling(processes()), ok. diff --git a/lib/ssl/test/ssl_handshake_SUITE.erl b/lib/ssl/test/ssl_handshake_SUITE.erl index b8b9989d30..1fa6029963 100644 --- a/lib/ssl/test/ssl_handshake_SUITE.erl +++ b/lib/ssl/test/ssl_handshake_SUITE.erl @@ -25,6 +25,7 @@ -compile(export_all). -include_lib("common_test/include/ct.hrl"). +-include("ssl_handshake.hrl"). -include("ssl_internal.hrl"). -include("tls_handshake.hrl"). -include_lib("public_key/include/public_key.hrl"). @@ -41,7 +42,8 @@ all() -> [decode_hello_handshake, decode_empty_server_sni_correctly, select_proper_tls_1_2_rsa_default_hashsign, ignore_hassign_extension_pre_tls_1_2, - unorded_chain]. + unorded_chain, + encode_decode_srp]. %%-------------------------------------------------------------------- init_per_suite(Config) -> @@ -192,6 +194,31 @@ unorded_chain(Config) when is_list(Config) -> {ok, _, OrderedChain} = ssl_certificate:certificate_chain(PeerCert, ets:new(foo, []), ExtractedCerts, UnordedChain). +encode_decode_srp(_Config) -> + Exts = #hello_extensions{ + srp = #srp{username = <<"foo">>}, + sni = #sni{hostname = "bar"}, + renegotiation_info = undefined, + signature_algs = undefined, + alpn = undefined, + next_protocol_negotiation = undefined, + ec_point_formats = undefined, + elliptic_curves = undefined + }, + EncodedExts = <<0,20, % Length + 0,0, % SNI extension + 0,8, % Length + 0,6, % ServerNameLength + 0, % NameType (host_name) + 0,3, % HostNameLength + 98,97,114, % hostname = "bar" + 0,12, % SRP extension + 0,4, % Length + 3, % srp_I length + 102,111,111>>, % username = "foo" + EncodedExts = ssl_handshake:encode_hello_extensions(Exts), + Exts = ssl_handshake:decode_hello_extensions({client, EncodedExts}). + %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ diff --git a/lib/ssl/test/ssl_npn_handshake_SUITE.erl b/lib/ssl/test/ssl_npn_handshake_SUITE.erl index 1c7d6b5f9f..878e983bb9 100644 --- a/lib/ssl/test/ssl_npn_handshake_SUITE.erl +++ b/lib/ssl/test/ssl_npn_handshake_SUITE.erl @@ -64,13 +64,12 @@ next_protocol_not_supported() -> npn_not_supported_server ]. -init_per_suite(Config) -> +init_per_suite(Config0) -> catch crypto:stop(), try crypto:start() of ok -> ssl_test_lib:clean_start(), - {ok, _} = make_certs:all(proplists:get_value(data_dir, Config), - proplists:get_value(priv_dir, Config)), + Config = ssl_test_lib:make_rsa_cert(Config0), ssl_test_lib:cert_options(Config) catch _:_ -> {skip, "Crypto did not start"} @@ -196,10 +195,10 @@ client_negotiate_server_does_not_support(Config) when is_list(Config) -> renegotiate_from_client_after_npn_handshake(Config) when is_list(Config) -> Data = "hello world", - ClientOpts0 = ssl_test_lib:ssl_options(client_opts, Config), + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), ClientOpts = [{client_preferred_next_protocols, {client, [<<"http/1.0">>], <<"http/1.1">>}}] ++ ClientOpts0, - ServerOpts0 = ssl_test_lib:ssl_options(server_opts, Config), + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), ServerOpts = [{next_protocols_advertised, [<<"spdy/2">>, <<"http/1.1">>, <<"http/1.0">>]}] ++ ServerOpts0, ExpectedProtocol = {ok, <<"http/1.0">>}, @@ -221,7 +220,7 @@ renegotiate_from_client_after_npn_handshake(Config) when is_list(Config) -> %-------------------------------------------------------------------------------- npn_not_supported_client(Config) when is_list(Config) -> - ClientOpts0 = ssl_test_lib:ssl_options(client_opts, Config), + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), PrefProtocols = {client_preferred_next_protocols, {client, [<<"http/1.0">>], <<"http/1.1">>}}, ClientOpts = [PrefProtocols] ++ ClientOpts0, @@ -236,7 +235,7 @@ npn_not_supported_client(Config) when is_list(Config) -> %-------------------------------------------------------------------------------- npn_not_supported_server(Config) when is_list(Config)-> - ServerOpts0 = ssl_test_lib:ssl_options(server_opts, Config), + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), AdvProtocols = {next_protocols_advertised, [<<"spdy/2">>, <<"http/1.1">>, <<"http/1.0">>]}, ServerOpts = [AdvProtocols] ++ ServerOpts0, @@ -244,63 +243,24 @@ npn_not_supported_server(Config) when is_list(Config)-> %-------------------------------------------------------------------------------- npn_handshake_session_reused(Config) when is_list(Config)-> - ClientOpts0 = ssl_test_lib:ssl_options(client_opts, Config), + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), ClientOpts = [{client_preferred_next_protocols, {client, [<<"http/1.0">>], <<"http/1.1">>}}] ++ ClientOpts0, - ServerOpts0 = ssl_test_lib:ssl_options(server_opts, Config), + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), ServerOpts =[{next_protocols_advertised, [<<"spdy/2">>, <<"http/1.1">>, <<"http/1.0">>]}] ++ ServerOpts0, - {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), - Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, - {from, self()}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {options, ServerOpts}]), - - Port = ssl_test_lib:inet_port(Server), - Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, - {host, Hostname}, - {from, self()}, - {mfa, {ssl_test_lib, no_result_msg, []}}, - {options, ClientOpts}]), - - SessionInfo = - receive - {Server, Info} -> - Info - end, - - Server ! {listen, {mfa, {ssl_test_lib, no_result, []}}}, - - %% Make sure session is registered - ct:sleep(?SLEEP), - - Client1 = - ssl_test_lib:start_client([{node, ClientNode}, - {port, Port}, {host, Hostname}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {from, self()}, {options, ClientOpts}]), - - receive - {Client1, SessionInfo} -> - ok; - {Client1, Other} -> - ct:fail(Other) - end, + ssl_test_lib:reuse_session(ClientOpts, ServerOpts, Config). - ssl_test_lib:close(Server), - ssl_test_lib:close(Client), - ssl_test_lib:close(Client1). - %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- run_npn_handshake(Config, ClientExtraOpts, ServerExtraOpts, ExpectedProtocol) -> Data = "hello world", - ClientOpts0 = ssl_test_lib:ssl_options(client_opts, Config), + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), ClientOpts = ClientExtraOpts ++ ClientOpts0, - ServerOpts0 = ssl_test_lib:ssl_options(server_opts, Config), + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), ServerOpts = ServerExtraOpts ++ ServerOpts0, {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), diff --git a/lib/ssl/test/ssl_payload_SUITE.erl b/lib/ssl/test/ssl_payload_SUITE.erl index 1f9b6a5772..27b9c258a0 100644 --- a/lib/ssl/test/ssl_payload_SUITE.erl +++ b/lib/ssl/test/ssl_payload_SUITE.erl @@ -64,7 +64,8 @@ payload_tests() -> server_echos_active_huge, client_echos_passive_huge, client_echos_active_once_huge, - client_echos_active_huge]. + client_echos_active_huge, + client_active_once_server_close]. init_per_suite(Config) -> catch crypto:stop(), @@ -397,6 +398,23 @@ client_echos_active_huge(Config) when is_list(Config) -> client_echos_active( Data, ClientOpts, ServerOpts, ClientNode, ServerNode, Hostname). + +%%-------------------------------------------------------------------- +client_active_once_server_close() -> + [{doc, "Server sends 500000 bytes and immediately after closes the connection" + "Make sure client recives all data if possible"}]. + +client_active_once_server_close(Config) when is_list(Config) -> + ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + %% + Data = binary:copy(<<"1234567890">>, 50000), + client_active_once_server_close( + Data, ClientOpts, ServerOpts, ClientNode, ServerNode, Hostname). + + + %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- @@ -541,42 +559,57 @@ client_echos_active( ssl_test_lib:close(Server), ssl_test_lib:close(Client). +client_active_once_server_close( + Data, ClientOpts, ServerOpts, ClientNode, ServerNode, Hostname) -> + Length = byte_size(Data), + Server = + ssl_test_lib:start_server( + [{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, send_close, [Data]}}, + {options, [{active, once}, {mode, binary} | ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + Client = + ssl_test_lib:start_client( + [{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, active_once_recv, [Length]}}, + {options,[{active, once}, {mode, binary} | ClientOpts]}]), + %% + ssl_test_lib:check_result(Server, ok, Client, ok). + +send(_Socket, _Data, 0, _) -> + ok; +send(Socket, Data, Count, RecvEcho) -> + ok = ssl:send(Socket, Data), + RecvEcho(), + send(Socket, Data, Count - 1, RecvEcho). -send(Socket, Data, Count, Verify) -> - send(Socket, Data, Count, <<>>, Verify). -%% -send(_Socket, _Data, 0, Acc, _Verify) -> - Acc; -send(Socket, Data, Count, Acc, Verify) -> +send_close(Socket, Data) -> ok = ssl:send(Socket, Data), - NewAcc = Verify(Acc), - send(Socket, Data, Count - 1, NewAcc, Verify). + ssl:close(Socket). - sender(Socket, Data) -> ct:log("Sender recv: ~p~n", [ssl:getopts(Socket, [active])]), - <<>> = - send( - Socket, Data, 100, - fun(Acc) -> verify_recv(Socket, Data, Acc) end), - ok. + send(Socket, Data, 100, + fun() -> + ssl_test_lib:recv_disregard(Socket, byte_size(Data)) + end). sender_active_once(Socket, Data) -> ct:log("Sender active once: ~p~n", [ssl:getopts(Socket, [active])]), - <<>> = - send( - Socket, Data, 100, - fun(Acc) -> verify_active_once(Socket, Data, Acc) end), - ok. + send(Socket, Data, 100, + fun() -> + ssl_test_lib:active_once_disregard(Socket, byte_size(Data)) + end). sender_active(Socket, Data) -> ct:log("Sender active: ~p~n", [ssl:getopts(Socket, [active])]), - <<>> = - send( - Socket, Data, 100, - fun(Acc) -> verify_active(Socket, Data, Acc) end), - ok. - + send(Socket, Data, 100, + fun() -> + ssl_test_lib:active_disregard(Socket, byte_size(Data)) + end). echoer(Socket, Size) -> ct:log("Echoer recv: ~p~n", [ssl:getopts(Socket, [active])]), @@ -592,99 +625,32 @@ echoer_active(Socket, Size) -> %% Receive Size bytes +echo_recv(_Socket, 0) -> + ok; echo_recv(Socket, Size) -> {ok, Data} = ssl:recv(Socket, 0), ok = ssl:send(Socket, Data), - NewSize = Size - byte_size(Data), - if - 0 < NewSize -> - echo_recv(Socket, NewSize); - 0 == NewSize -> - ok - end. - -%% Verify that received data is SentData, return any superflous data -verify_recv(Socket, SentData, Acc) -> - {ok, NewData} = ssl:recv(Socket, 0), - SentSize = byte_size(SentData), - NewAcc = <<Acc/binary, NewData/binary>>, - NewSize = byte_size(NewAcc), - if - SentSize < NewSize -> - {SentData,Rest} = split_binary(NewAcc, SentSize), - Rest; - NewSize < SentSize -> - verify_recv(Socket, SentData, NewAcc); - true -> - SentData = NewAcc, - <<>> - end. + echo_recv(Socket, Size - byte_size(Data)). %% Receive Size bytes +echo_active_once(_Socket, 0) -> + ok; echo_active_once(Socket, Size) -> receive {ssl, Socket, Data} -> ok = ssl:send(Socket, Data), NewSize = Size - byte_size(Data), ssl:setopts(Socket, [{active, once}]), - if - 0 < NewSize -> - echo_active_once(Socket, NewSize); - 0 == NewSize -> - ok - end + echo_active_once(Socket, NewSize) end. -%% Verify that received data is SentData, return any superflous data -verify_active_once(Socket, SentData, Acc) -> - receive - {ssl, Socket, Data} -> - SentSize = byte_size(SentData), - NewAcc = <<Acc/binary, Data/binary>>, - NewSize = byte_size(NewAcc), - ssl:setopts(Socket, [{active, once}]), - if - SentSize < NewSize -> - {SentData,Rest} = split_binary(NewAcc, SentSize), - Rest; - NewSize < SentSize -> - verify_active_once(Socket, SentData, NewAcc); - true -> - SentData = NewAcc, - <<>> - end - end. - - %% Receive Size bytes +echo_active(_Socket, 0) -> + ok; echo_active(Socket, Size) -> receive {ssl, Socket, Data} -> ok = ssl:send(Socket, Data), - NewSize = Size - byte_size(Data), - if - 0 < NewSize -> - echo_active(Socket, NewSize); - 0 == NewSize -> - ok - end - end. - -%% Verify that received data is SentData, return any superflous data -verify_active(Socket, SentData, Acc) -> - receive - {ssl, Socket, Data} -> - SentSize = byte_size(SentData), - NewAcc = <<Acc/binary, Data/binary>>, - NewSize = byte_size(NewAcc), - if - SentSize < NewSize -> - {SentData,Rest} = split_binary(NewAcc, SentSize), - Rest; - NewSize < SentSize -> - verify_active(Socket, SentData, NewAcc); - true -> - SentData = NewAcc, - <<>> - end - end. + echo_active(Socket, Size - byte_size(Data)) + end. + diff --git a/lib/ssl/test/ssl_pem_cache_SUITE.erl b/lib/ssl/test/ssl_pem_cache_SUITE.erl index 25d2cb300d..6f11e2bbe8 100644 --- a/lib/ssl/test/ssl_pem_cache_SUITE.erl +++ b/lib/ssl/test/ssl_pem_cache_SUITE.erl @@ -44,11 +44,8 @@ init_per_suite(Config0) -> try crypto:start() of ok -> ssl_test_lib:clean_start(), - %% make rsa certs using oppenssl - {ok, _} = make_certs:all(proplists:get_value(data_dir, Config0), - proplists:get_value(priv_dir, Config0)), - Config1 = ssl_test_lib:make_dsa_cert(Config0), - ssl_test_lib:cert_options(Config1) + %% make rsa certs + ssl_test_lib:make_rsa_cert(Config0) catch _:_ -> {skip, "Crypto did not start"} end. @@ -86,8 +83,8 @@ pem_cleanup() -> [{doc, "Test pem cache invalidate mechanism"}]. pem_cleanup(Config)when is_list(Config) -> process_flag(trap_exit, true), - ClientOpts = proplists:get_value(client_verification_opts, Config), - ServerOpts = proplists:get_value(server_verification_opts, Config), + ClientOpts = proplists:get_value(client_rsa_verify_opts, Config), + ServerOpts = proplists:get_value(server_rsa_verify_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = @@ -118,8 +115,8 @@ invalid_insert() -> invalid_insert(Config)when is_list(Config) -> process_flag(trap_exit, true), - ClientOpts = proplists:get_value(client_verification_opts, Config), - ServerOpts = proplists:get_value(server_verification_opts, Config), + ClientOpts = proplists:get_value(client_rsa_verify_opts, Config), + ServerOpts = proplists:get_value(server_rsa_verify_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), BadClientOpts = [{cacertfile, "tmp/does_not_exist.pem"} | proplists:delete(cacertfile, ClientOpts)], Server = diff --git a/lib/ssl/test/ssl_session_cache_SUITE.erl b/lib/ssl/test/ssl_session_cache_SUITE.erl index a0fab58b9d..7f33fe3204 100644 --- a/lib/ssl/test/ssl_session_cache_SUITE.erl +++ b/lib/ssl/test/ssl_session_cache_SUITE.erl @@ -48,7 +48,8 @@ all() -> session_cache_process_list, session_cache_process_mnesia, client_unique_session, - max_table_size + max_table_size, + save_specific_session ]. groups() -> @@ -60,10 +61,7 @@ init_per_suite(Config0) -> ok -> ssl_test_lib:clean_start(), %% make rsa certs using - {ok, _} = make_certs:all(proplists:get_value(data_dir, Config0), - proplists:get_value(priv_dir, Config0)), - Config = ssl_test_lib:make_dsa_cert(Config0), - ssl_test_lib:cert_options(Config) + ssl_test_lib:make_rsa_cert(Config0) catch _:_ -> {skip, "Crypto did not start"} end. @@ -97,7 +95,10 @@ init_per_testcase(session_cleanup, Config) -> init_per_testcase(client_unique_session, Config) -> ct:timetrap({seconds, 40}), Config; - +init_per_testcase(save_specific_session, Config) -> + ssl_test_lib:clean_start(), + ct:timetrap({seconds, 5}), + Config; init_per_testcase(max_table_size, Config) -> ssl:stop(), application:load(ssl), @@ -141,7 +142,7 @@ end_per_testcase(max_table_size, Config) -> end_per_testcase(default_action, Config); end_per_testcase(Case, Config) when Case == session_cache_process_list; Case == session_cache_process_mnesia -> - ets:delete(ssl_test), + catch ets:delete(ssl_test), Config; end_per_testcase(_, Config) -> Config. @@ -154,8 +155,8 @@ client_unique_session() -> "sets up many connections"}]. client_unique_session(Config) when is_list(Config) -> process_flag(trap_exit, true), - ClientOpts = proplists:get_value(client_opts, Config), - ServerOpts = proplists:get_value(server_opts, Config), + ClientOpts = proplists:get_value(client_rsa_verify_opts, Config), + ServerOpts = proplists:get_value(server_rsa_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, @@ -164,8 +165,7 @@ client_unique_session(Config) when is_list(Config) -> {tcp_options, [{active, false}]}, {options, ServerOpts}]), Port = ssl_test_lib:inet_port(Server), - LastClient = clients_start(Server, - ClientNode, Hostname, Port, ClientOpts, client_unique_session, 20), + LastClient = clients_start(Server, ClientNode, Hostname, Port, ClientOpts, 20), receive {LastClient, {ok, _}} -> ok @@ -185,8 +185,8 @@ session_cleanup() -> "does not grow and grow ..."}]. session_cleanup(Config) when is_list(Config) -> process_flag(trap_exit, true), - ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = @@ -254,13 +254,75 @@ session_cache_process_mnesia(Config) when is_list(Config) -> session_cache_process(mnesia,Config). %%-------------------------------------------------------------------- +save_specific_session() -> + [{doc, "Test that we can save a specific client session" + }]. +save_specific_session(Config) when is_list(Config) -> + process_flag(trap_exit, true), + ClientOpts = proplists:get_value(client_rsa_verify_opts, Config), + ServerOpts = proplists:get_value(server_rsa_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {tcp_options, [{active, false}]}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + + Client1 = ssl_test_lib:start_client([{node, ClientNode}, + {port, Port}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, ClientOpts}]), + Server ! listen, + + Client2 = ssl_test_lib:start_client([{node, ClientNode}, + {port, Port}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, [{reuse_sessions, save} | ClientOpts]}]), + SessionID1 = + receive + {Client1, S1} -> + S1 + end, + + SessionID2 = + receive + {Client2, S2} -> + S2 + end, + + true = SessionID1 =/= SessionID2, + + {status, _, _, StatusInfo} = sys:get_status(whereis(ssl_manager)), + [_, _,_, _, Prop] = StatusInfo, + State = ssl_test_lib:state(Prop), + ClientCache = element(2, State), + 2 = ssl_session_cache:size(ClientCache), + + Server ! listen, + + Client3 = ssl_test_lib:start_client([{node, ClientNode}, + {port, Port}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, [{reuse_session, SessionID2} | ClientOpts]}]), + receive + {Client3, SessionID2} -> + ok; + {Client3, SessionID3}-> + ct:fail({got, SessionID3, expected, SessionID2}); + Other -> + ct:fail({got,Other}) + end. + +%%-------------------------------------------------------------------- max_table_size() -> [{doc,"Test max limit on session table"}]. max_table_size(Config) when is_list(Config) -> process_flag(trap_exit, true), - ClientOpts = proplists:get_value(client_verification_opts, Config), - ServerOpts = proplists:get_value(server_verification_opts, Config), + ClientOpts = proplists:get_value(client_rsa_verify_opts, Config), + ServerOpts = proplists:get_value(server_rsa_verify_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, @@ -270,7 +332,7 @@ max_table_size(Config) when is_list(Config) -> {options, ServerOpts}]), Port = ssl_test_lib:inet_port(Server), LastClient = clients_start(Server, - ClientNode, Hostname, Port, ClientOpts, max_table_size, 20), + ClientNode, Hostname, Port, ClientOpts, 20), receive {LastClient, {ok, _}} -> ok @@ -426,25 +488,27 @@ session_loop(Sess) -> %%-------------------------------------------------------------------- session_cache_process(_Type,Config) when is_list(Config) -> - ssl_basic_SUITE:reuse_session(Config). + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), + ssl_test_lib:reuse_session(ClientOpts, ServerOpts, Config). -clients_start(_Server, ClientNode, Hostname, Port, ClientOpts, Test, 0) -> +clients_start(_Server, ClientNode, Hostname, Port, ClientOpts, 0) -> %% Make sure session is registered ct:sleep(?SLEEP * 2), ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, {host, Hostname}, {mfa, {?MODULE, connection_info_result, []}}, - {from, self()}, {options, test_copts(Test, 0, ClientOpts)}]); -clients_start(Server, ClientNode, Hostname, Port, ClientOpts, Test, N) -> + {from, self()}, {options, ClientOpts}]); +clients_start(Server, ClientNode, Hostname, Port, ClientOpts, N) -> spawn_link(ssl_test_lib, start_client, [[{node, ClientNode}, {port, Port}, {host, Hostname}, {mfa, {ssl_test_lib, no_result, []}}, - {from, self()}, {options, test_copts(Test, N, ClientOpts)}]]), + {from, self()}, {options, ClientOpts}]]), Server ! listen, wait_for_server(), - clients_start(Server, ClientNode, Hostname, Port, ClientOpts, Test, N-1). + clients_start(Server, ClientNode, Hostname, Port, ClientOpts, N-1). connection_info_result(Socket) -> ssl:connection_information(Socket, [protocol, cipher_suite]). @@ -481,21 +545,3 @@ get_delay_timers() -> wait_for_server() -> ct:sleep(100). - - -test_copts(_, 0, ClientOpts) -> - ClientOpts; -test_copts(max_table_size, N, ClientOpts) -> - Version = tls_record:highest_protocol_version([]), - CipherSuites = %%lists:map(fun(X) -> ssl_cipher_format:suite_definition(X) end, ssl_cipher:filter_suites(ssl_cipher:suites(Version))), -[ Y|| Y = {Alg,_, _, _} <- lists:map(fun(X) -> ssl_cipher_format:suite_definition(X) end, ssl_cipher:filter_suites(ssl_cipher:suites(Version))), Alg =/= ecdhe_ecdsa, Alg =/= ecdh_ecdsa, Alg =/= ecdh_rsa, Alg =/= ecdhe_rsa, Alg =/= dhe_dss, Alg =/= dss], - case length(CipherSuites) of - M when M >= N -> - Cipher = lists:nth(N, CipherSuites), - ct:pal("~p",[Cipher]), - [{ciphers, [Cipher]} | ClientOpts]; - _ -> - ClientOpts - end; -test_copts(_, _, ClientOpts) -> - ClientOpts. diff --git a/lib/ssl/test/ssl_sni_SUITE.erl b/lib/ssl/test/ssl_sni_SUITE.erl index 251b6a2639..7629d75100 100644 --- a/lib/ssl/test/ssl_sni_SUITE.erl +++ b/lib/ssl/test/ssl_sni_SUITE.erl @@ -236,8 +236,8 @@ dns_name_reuse(Config) -> {mfa, {ssl_test_lib, session_info_result, []}}, {from, self()}, {options, [{verify, verify_peer} | ClientConf]}]), - ssl_test_lib:check_result(Client1, {error, {tls_alert, "handshake failure"}}), - ssl_test_lib:close(Client0). + ssl_test_lib:check_client_alert(Client1, handshake_failure). + %%-------------------------------------------------------------------- %% Internal Functions ------------------------------------------------ %%-------------------------------------------------------------------- @@ -370,8 +370,8 @@ unsuccessfull_connect(ServerOptions, ClientOptions, Hostname0, Config) -> {from, self()}, {options, ClientOptions}]), - ssl_test_lib:check_result(Server, {error, {tls_alert, "handshake failure"}}, - Client, {error, {tls_alert, "handshake failure"}}). + ssl_test_lib:check_server_alert(Server, Client, handshake_failure). + host_name(undefined, Hostname) -> Hostname; host_name(Hostname, _) -> diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 7767d76a0d..c6a4a45dce 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -30,6 +30,7 @@ -record(sslsocket, { fd = nil, pid = nil}). -define(SLEEP, 1000). +-define(DEFAULT_CURVE, secp256r1). %% For now always run locally run_where(_) -> @@ -437,6 +438,37 @@ check_result(Pid, Msg) -> {got, Unexpected}}, ct:fail(Reason) end. +check_server_alert(Pid, Alert) -> + receive + {Pid, {error, {tls_alert, {Alert, _}}}} -> + ok + end. +check_server_alert(Server, Client, Alert) -> + receive + {Server, {error, {tls_alert, {Alert, _}}}} -> + receive + {Client, {error, {tls_alert, {Alert, _}}}} -> + ok; + {Client, {error, closed}} -> + ok + end + end. +check_client_alert(Pid, Alert) -> + receive + {Pid, {error, {tls_alert, {Alert, _}}}} -> + ok + end. +check_client_alert(Server, Client, Alert) -> + receive + {Client, {error, {tls_alert, {Alert, _}}}} -> + receive + {Server, {error, {tls_alert, {Alert, _}}}} -> + ok; + {Server, {error, closed}} -> + ok + end + end. + wait_for_result(Server, ServerMsg, Client, ClientMsg) -> receive @@ -523,7 +555,7 @@ cert_options(Config) -> {client_verification_opts, [{cacertfile, ServerCaCertFile}, {certfile, ClientCertFile}, {keyfile, ClientKeyFile}, - {ssl_imp, new}]}, + {verify, verify_peer}]}, {client_verification_opts_digital_signature_only, [{cacertfile, ServerCaCertFile}, {certfile, ClientCertFileDigitalSignatureOnly}, {keyfile, ClientKeyFile}, @@ -618,9 +650,12 @@ make_rsa_cert_chains(UserConf, Config, Suffix) -> }. make_ec_cert_chains(UserConf, ClientChainType, ServerChainType, Config) -> + make_ec_cert_chains(UserConf, ClientChainType, ServerChainType, Config, ?DEFAULT_CURVE). +%% +make_ec_cert_chains(UserConf, ClientChainType, ServerChainType, Config, Curve) -> ClientChain = proplists:get_value(client_chain, UserConf, default_cert_chain_conf()), ServerChain = proplists:get_value(server_chain, UserConf, default_cert_chain_conf()), - CertChainConf = gen_conf(ClientChainType, ServerChainType, ClientChain, ServerChain), + CertChainConf = gen_conf(ClientChainType, ServerChainType, ClientChain, ServerChain, Curve), ClientFileBase = filename:join([proplists:get_value(priv_dir, Config), atom_to_list(ClientChainType)]), ServerFileBase = filename:join([proplists:get_value(priv_dir, Config), atom_to_list(ServerChainType)]), GenCertData = public_key:pkix_test_data(CertChainConf), @@ -635,7 +670,11 @@ default_cert_chain_conf() -> %% Use only default options [[],[],[]]. -gen_conf(mix, mix, UserClient, UserServer) -> + +gen_conf(ClientChainType, ServerChainType, UserClient, UserServer) -> + gen_conf(ClientChainType, ServerChainType, UserClient, UserServer, ?DEFAULT_CURVE). +%% +gen_conf(mix, mix, UserClient, UserServer, _) -> ClientTag = conf_tag("client"), ServerTag = conf_tag("server"), @@ -646,12 +685,12 @@ gen_conf(mix, mix, UserClient, UserServer) -> ServerConf = merge_chain_spec(UserServer, DefaultServer, []), new_format([{ClientTag, ClientConf}, {ServerTag, ServerConf}]); -gen_conf(ClientChainType, ServerChainType, UserClient, UserServer) -> +gen_conf(ClientChainType, ServerChainType, UserClient, UserServer, Curve) -> ClientTag = conf_tag("client"), ServerTag = conf_tag("server"), - DefaultClient = chain_spec(client, ClientChainType), - DefaultServer = chain_spec(server, ServerChainType), + DefaultClient = chain_spec(client, ClientChainType, Curve), + DefaultServer = chain_spec(server, ServerChainType, Curve), ClientConf = merge_chain_spec(UserClient, DefaultClient, []), ServerConf = merge_chain_spec(UserServer, DefaultServer, []), @@ -673,43 +712,43 @@ proplist_to_map([Head | Rest]) -> conf_tag(Role) -> list_to_atom(Role ++ "_chain"). -chain_spec(_Role, ecdh_rsa) -> +chain_spec(_Role, ecdh_rsa, Curve) -> Digest = {digest, appropriate_sha(crypto:supports())}, - CurveOid = hd(tls_v1:ecc_curves(0)), + CurveOid = pubkey_cert_records:namedCurves(Curve), [[Digest, {key, {namedCurve, CurveOid}}], [Digest, {key, hardcode_rsa_key(1)}], [Digest, {key, {namedCurve, CurveOid}}]]; -chain_spec(_Role, ecdhe_ecdsa) -> +chain_spec(_Role, ecdhe_ecdsa, Curve) -> Digest = {digest, appropriate_sha(crypto:supports())}, - CurveOid = hd(tls_v1:ecc_curves(0)), + CurveOid = pubkey_cert_records:namedCurves(Curve), [[Digest, {key, {namedCurve, CurveOid}}], [Digest, {key, {namedCurve, CurveOid}}], [Digest, {key, {namedCurve, CurveOid}}]]; -chain_spec(_Role, ecdh_ecdsa) -> +chain_spec(_Role, ecdh_ecdsa, Curve) -> Digest = {digest, appropriate_sha(crypto:supports())}, - CurveOid = hd(tls_v1:ecc_curves(0)), + CurveOid = pubkey_cert_records:namedCurves(Curve), [[Digest, {key, {namedCurve, CurveOid}}], [Digest, {key, {namedCurve, CurveOid}}], [Digest, {key, {namedCurve, CurveOid}}]]; -chain_spec(_Role, ecdhe_rsa) -> +chain_spec(_Role, ecdhe_rsa, _) -> Digest = {digest, appropriate_sha(crypto:supports())}, [[Digest, {key, hardcode_rsa_key(1)}], [Digest, {key, hardcode_rsa_key(2)}], [Digest, {key, hardcode_rsa_key(3)}]]; -chain_spec(_Role, ecdsa) -> +chain_spec(_Role, ecdsa, Curve) -> Digest = {digest, appropriate_sha(crypto:supports())}, - CurveOid = hd(tls_v1:ecc_curves(0)), + CurveOid = pubkey_cert_records:namedCurves(Curve), [[Digest, {key, {namedCurve, CurveOid}}], [Digest, {key, {namedCurve, CurveOid}}], [Digest, {key, {namedCurve, CurveOid}}]]; -chain_spec(_Role, rsa) -> +chain_spec(_Role, rsa, _) -> Digest = {digest, appropriate_sha(crypto:supports())}, [[Digest, {key, hardcode_rsa_key(1)}], [Digest, {key, hardcode_rsa_key(2)}], [Digest, {key, hardcode_rsa_key(3)}]]; -chain_spec(_Role, dsa) -> +chain_spec(_Role, dsa, _) -> Digest = {digest, appropriate_sha(crypto:supports())}, [[Digest, {key, hardcode_dsa_key(1)}], [Digest, {key, hardcode_dsa_key(2)}], @@ -742,7 +781,7 @@ merge_spec(User, Default, [Conf | Rest], Acc) -> make_mix_cert(Config) -> Ext = x509_test:extensions([{key_usage, [digitalSignature]}]), Digest = {digest, appropriate_sha(crypto:supports())}, - CurveOid = hd(tls_v1:ecc_curves(0)), + CurveOid = pubkey_cert_records:namedCurves(?DEFAULT_CURVE), Mix = proplists:get_value(mix, Config, peer_ecc), ClientChainType =ServerChainType = mix, {ClientChain, ServerChain} = mix(Mix, Digest, CurveOid, Ext), @@ -825,7 +864,8 @@ make_rsa_cert(Config) -> Config end. appropriate_sha(CryptoSupport) -> - case proplists:get_bool(sha256, CryptoSupport) of + Hashes = proplists:get_value(hashs, CryptoSupport), + case lists:member(sha256, Hashes) of true -> sha256; false -> @@ -1064,8 +1104,7 @@ ecc_test(Expect, COpts, SOpts, CECCOpts, SECCOpts, Config) -> ecc_test_error(COpts, SOpts, CECCOpts, SECCOpts, Config) -> {Server, Port} = start_server_ecc_error(erlang, SOpts, SECCOpts, Config), Client = start_client_ecc_error(erlang, Port, COpts, CECCOpts, Config), - Error = {error, {tls_alert, "insufficient security"}}, - check_result(Server, Error, Client, Error). + check_server_alert(Server, Client, insufficient_security). start_client(openssl, Port, ClientOpts, Config) -> Cert = proplists:get_value(certfile, ClientOpts), @@ -1073,11 +1112,11 @@ start_client(openssl, Port, ClientOpts, Config) -> CA = proplists:get_value(cacertfile, ClientOpts), Version = ssl_test_lib:protocol_version(Config), Exe = "openssl", - Args = ["s_client", "-verify", "2", "-port", integer_to_list(Port), + Args0 = ["s_client", "-verify", "2", "-port", integer_to_list(Port), ssl_test_lib:version_flag(Version), "-cert", Cert, "-CAfile", CA, "-key", Key, "-host","localhost", "-msg", "-debug"], - + Args = maybe_force_ipv4(Args0), OpenSslPort = ssl_test_lib:portable_open_port(Exe, Args), true = port_command(OpenSslPort, "Hello world"), OpenSslPort; @@ -1091,6 +1130,18 @@ start_client(erlang, Port, ClientOpts, Config) -> {mfa, {ssl_test_lib, check_key_exchange_send_active, [KeyEx]}}, {options, [{verify, verify_peer} | ClientOpts]}]). +%% Workaround for running tests on machines where openssl +%% s_client would use an IPv6 address with localhost. As +%% this test suite and the ssl application is not prepared +%% for that we have to force s_client to use IPv4 if +%% OpenSSL supports IPv6. +maybe_force_ipv4(Args0) -> + case is_ipv6_supported() of + true -> + Args0 ++ ["-4"]; + false -> + Args0 + end. start_client_ecc(erlang, Port, ClientOpts, Expect, ECCOpts, Config) -> {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), @@ -1583,35 +1634,22 @@ v_1_2_check(ecdh_rsa, ecdh_ecdsa) -> v_1_2_check(_, _) -> false. -send_recv_result_active(Socket) -> - ssl:send(Socket, "Hello world"), - receive - {ssl, Socket, "H"} -> - receive - {ssl, Socket, "ello world"} -> - ok - end; - {ssl, Socket, "Hello world"} -> - ok - end. - send_recv_result(Socket) -> - ssl:send(Socket, "Hello world"), - {ok,"Hello world"} = ssl:recv(Socket, 11), + Data = "Hello world", + ssl:send(Socket, Data), + {ok, Data} = ssl:recv(Socket, length(Data)), + ok. + +send_recv_result_active(Socket) -> + Data = "Hello world", + ssl:send(Socket, Data), + Data = active_recv(Socket, length(Data)), ok. send_recv_result_active_once(Socket) -> - ssl:send(Socket, "Hello world"), - receive - {ssl, Socket, "H"} -> - ssl:setopts(Socket, [{active, once}]), - receive - {ssl, Socket, "ello world"} -> - ok - end; - {ssl, Socket, "Hello world"} -> - ok - end. + Data = "Hello world", + ssl:send(Socket, Data), + active_once_recv_list(Socket, length(Data)). active_recv(Socket, N) -> active_recv(Socket, N, []). @@ -1624,6 +1662,55 @@ active_recv(Socket, N, Acc) -> active_recv(Socket, N-length(Bytes), Acc ++ Bytes) end. +active_once_recv(_Socket, 0) -> + ok; +active_once_recv(Socket, N) -> + receive + {ssl, Socket, Bytes} -> + ssl:setopts(Socket, [{active, once}]), + active_once_recv(Socket, N-byte_size(Bytes)) + end. + +active_once_recv_list(_Socket, 0) -> + ok; +active_once_recv_list(Socket, N) -> + receive + {ssl, Socket, Bytes} -> + ssl:setopts(Socket, [{active, once}]), + active_once_recv_list(Socket, N-length(Bytes)) + end. +recv_disregard(_Socket, 0) -> + ok; +recv_disregard(Socket, N) -> + {ok, Bytes} = ssl:recv(Socket, 0), + recv_disregard(Socket, N-byte_size(Bytes)). + +active_disregard(_Socket, 0) -> + ok; +active_disregard(Socket, N) -> + receive + {ssl, Socket, Bytes} -> + active_disregard(Socket, N-byte_size(Bytes)) + end. +active_once_disregard(_Socket, 0) -> + ok; +active_once_disregard(Socket, N) -> + receive + {ssl, Socket, Bytes} -> + ssl:setopts(Socket, [{active, once}]), + active_once_disregard(Socket, N-byte_size(Bytes)) + end. + +is_ipv6_supported() -> + case os:cmd("openssl version") of + "OpenSSL 0.9.8" ++ _ -> % Does not support IPv6 + false; + "OpenSSL 1.0" ++ _ -> % Does not support IPv6 + false; + _ -> + true + end. + is_sane_ecc(openssl) -> case os:cmd("openssl version") of "OpenSSL 1.0.0a" ++ _ -> % Known bug in openssl @@ -2161,3 +2248,98 @@ server_msg(Server, ServerMsg) -> Unexpected -> ct:fail(Unexpected) end. + +session_id(Socket) -> + {ok, [{session_id, ID}]} = ssl:connection_information(Socket, [session_id]), + ID. + +reuse_session(ClientOpts, ServerOpts, Config) -> + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server0 = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {tcp_options, [{active, false}]}, + {options, ServerOpts}]), + Port0 = ssl_test_lib:inet_port(Server0), + + Client0 = ssl_test_lib:start_client([{node, ClientNode}, + {port, Port0}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, [{reuse_sessions, save} | ClientOpts]}]), + Server0 ! listen, + + Client1 = ssl_test_lib:start_client([{node, ClientNode}, + {port, Port0}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, ClientOpts}]), + + SID = receive + {Client0, Id0} -> + Id0 + end, + + receive + {Client1, SID} -> + ok + after ?SLEEP -> + ct:fail(session_not_reused) + end, + + Server0 ! listen, + + Client2 = + ssl_test_lib:start_client([{node, ClientNode}, + {port, Port0}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, [{reuse_sessions, false} + | ClientOpts]}]), + receive + {Client2, SID} -> + ct:fail(session_reused_when_session_reuse_disabled_by_client); + {Client2, _} -> + ok + end, + + ssl_test_lib:close(Server0), + ssl_test_lib:close(Client0), + ssl_test_lib:close(Client1), + ssl_test_lib:close(Client2), + + Server1 = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {tcp_options, [{active, false}]}, + {options, [{reuse_sessions, false} |ServerOpts]}]), + Port1 = ssl_test_lib:inet_port(Server1), + + Client3 = ssl_test_lib:start_client([{node, ClientNode}, + {port, Port1}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, [{reuse_sessions, save} | ClientOpts]}]), + SID1 = receive + {Client3, Id3} -> + Id3 + end, + + Server1 ! listen, + + Client4 = + ssl_test_lib:start_client([{node, ClientNode}, + {port, Port1}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, ClientOpts}]), + + receive + {Client4, SID1} -> + ct:fail(session_reused_when_session_reuse_disabled_by_server); + {Client4, _} -> + ok + end, + + ssl_test_lib:close(Server1), + ssl_test_lib:close(Client3), + ssl_test_lib:close(Client4). + diff --git a/lib/ssl/test/ssl_to_openssl_SUITE.erl b/lib/ssl/test/ssl_to_openssl_SUITE.erl index 3c8b25b912..df84411b6d 100644 --- a/lib/ssl/test/ssl_to_openssl_SUITE.erl +++ b/lib/ssl/test/ssl_to_openssl_SUITE.erl @@ -260,8 +260,9 @@ special_init(TestCase, Config) when Config; special_init(TestCase, Config) when TestCase == erlang_client_openssl_server_renegotiate; - TestCase == erlang_client_openssl_server_nowrap_seqnum; - TestCase == erlang_server_openssl_client_nowrap_seqnum + TestCase == erlang_client_openssl_server_nowrap_seqnum; + TestCase == erlang_server_openssl_client_nowrap_seqnum; + TestCase == erlang_client_openssl_server_renegotiate_after_client_data -> {ok, Version} = application:get_env(ssl, protocol_version), check_sane_openssl_renegotaite(Config, Version); @@ -761,8 +762,8 @@ erlang_client_openssl_server_renegotiate() -> [{doc,"Test erlang client when openssl server issuses a renegotiate"}]. erlang_client_openssl_server_renegotiate(Config) when is_list(Config) -> process_flag(trap_exit, true), - ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), - ClientOpts = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), @@ -771,12 +772,14 @@ erlang_client_openssl_server_renegotiate(Config) when is_list(Config) -> Port = ssl_test_lib:inet_port(node()), CertFile = proplists:get_value(certfile, ServerOpts), + CaCertFile = proplists:get_value(cacertfile, ServerOpts), KeyFile = proplists:get_value(keyfile, ServerOpts), Version = ssl_test_lib:protocol_version(Config), Exe = "openssl", Args = ["s_server", "-accept", integer_to_list(Port), ssl_test_lib:version_flag(Version), + "-CAfile", CaCertFile, "-cert", CertFile, "-key", KeyFile, "-msg"], OpensslPort = ssl_test_lib:portable_open_port(Exe, Args), @@ -806,8 +809,8 @@ erlang_client_openssl_server_renegotiate_after_client_data() -> [{doc,"Test erlang client when openssl server issuses a renegotiate after reading client data"}]. erlang_client_openssl_server_renegotiate_after_client_data(Config) when is_list(Config) -> process_flag(trap_exit, true), - ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), - ClientOpts = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), @@ -815,6 +818,7 @@ erlang_client_openssl_server_renegotiate_after_client_data(Config) when is_list( OpenSslData = "From openssl to erlang", Port = ssl_test_lib:inet_port(node()), + CaCertFile = proplists:get_value(cacertfile, ServerOpts), CertFile = proplists:get_value(certfile, ServerOpts), KeyFile = proplists:get_value(keyfile, ServerOpts), Version = ssl_test_lib:protocol_version(Config), @@ -822,6 +826,7 @@ erlang_client_openssl_server_renegotiate_after_client_data(Config) when is_list( Exe = "openssl", Args = ["s_server", "-accept", integer_to_list(Port), ssl_test_lib:version_flag(Version), + "-CAfile", CaCertFile, "-cert", CertFile, "-key", KeyFile, "-msg"], OpensslPort = ssl_test_lib:portable_open_port(Exe, Args), @@ -856,7 +861,7 @@ erlang_client_openssl_server_nowrap_seqnum() -> " to lower treashold substantially."}]. erlang_client_openssl_server_nowrap_seqnum(Config) when is_list(Config) -> process_flag(trap_exit, true), - ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), ClientOpts = ssl_test_lib:ssl_options(client_rsa_opts, Config), {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), @@ -865,12 +870,14 @@ erlang_client_openssl_server_nowrap_seqnum(Config) when is_list(Config) -> N = 10, Port = ssl_test_lib:inet_port(node()), + CaCertFile = proplists:get_value(cacertfile, ServerOpts), CertFile = proplists:get_value(certfile, ServerOpts), KeyFile = proplists:get_value(keyfile, ServerOpts), Version = ssl_test_lib:protocol_version(Config), Exe = "openssl", Args = ["s_server", "-accept", integer_to_list(Port), ssl_test_lib:version_flag(Version), + "-CAfile", CaCertFile, "-cert", CertFile, "-key", KeyFile, "-msg"], OpensslPort = ssl_test_lib:portable_open_port(Exe, Args), @@ -899,7 +906,7 @@ erlang_server_openssl_client_nowrap_seqnum() -> " to lower treashold substantially."}]. erlang_server_openssl_client_nowrap_seqnum(Config) when is_list(Config) -> process_flag(trap_exit, true), - ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), {_, ServerNode, Hostname} = ssl_test_lib:run_where(Config), @@ -1242,7 +1249,7 @@ ssl2_erlang_server_openssl_client(Config) when is_list(Config) -> ct:log("Ports ~p~n", [[erlang:port_info(P) || P <- erlang:ports()]]), ssl_test_lib:consume_port_exit(OpenSslPort), - ssl_test_lib:check_result(Server, {error, {tls_alert, "bad record mac"}}), + ssl_test_lib:check_server_alert(Server, bad_record_mac), process_flag(trap_exit, false). %%-------------------------------------------------------------------- @@ -1648,8 +1655,8 @@ cipher(CipherSuite, Version, Config, ClientOpts, ServerOpts) -> start_erlang_client_and_openssl_server_with_opts(Config, ErlangClientOpts, OpensslServerOpts, Data, Callback) -> process_flag(trap_exit, true), - ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), - ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), ClientOpts = ErlangClientOpts ++ ClientOpts0, {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), @@ -1657,6 +1664,7 @@ start_erlang_client_and_openssl_server_with_opts(Config, ErlangClientOpts, Opens Data = "From openssl to erlang", Port = ssl_test_lib:inet_port(node()), + CaCertFile = proplists:get_value(cacertfile, ServerOpts), CertFile = proplists:get_value(certfile, ServerOpts), KeyFile = proplists:get_value(keyfile, ServerOpts), Version = ssl_test_lib:protocol_version(Config), @@ -1666,10 +1674,12 @@ start_erlang_client_and_openssl_server_with_opts(Config, ErlangClientOpts, Opens [] -> ["s_server", "-accept", integer_to_list(Port), ssl_test_lib:version_flag(Version), + "-CAfile", CaCertFile, "-cert", CertFile,"-key", KeyFile]; [Opt, Value] -> ["s_server", Opt, Value, "-accept", integer_to_list(Port), ssl_test_lib:version_flag(Version), + "-CAfile", CaCertFile, "-cert", CertFile,"-key", KeyFile] end, @@ -1694,8 +1704,8 @@ start_erlang_client_and_openssl_server_with_opts(Config, ErlangClientOpts, Opens start_erlang_client_and_openssl_server_for_alpn_negotiation(Config, Data, Callback) -> process_flag(trap_exit, true), - ServerOpts = proplists:get_value(server_rsa_opts, Config), - ClientOpts0 = proplists:get_value(client_rsa_opts, Config), + ServerOpts = proplists:get_value(server_rsa_verify_opts, Config), + ClientOpts0 = proplists:get_value(client_rsa_verify_opts, Config), ClientOpts = [{alpn_advertised_protocols, [<<"spdy/2">>]} | ClientOpts0], {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), @@ -1703,12 +1713,14 @@ start_erlang_client_and_openssl_server_for_alpn_negotiation(Config, Data, Callba Data = "From openssl to erlang", Port = ssl_test_lib:inet_port(node()), + CaCertFile = proplists:get_value(cacertfile, ServerOpts), CertFile = proplists:get_value(certfile, ServerOpts), KeyFile = proplists:get_value(keyfile, ServerOpts), Version = ssl_test_lib:protocol_version(Config), Exe = "openssl", Args = ["s_server", "-msg", "-alpn", "http/1.1,spdy/2", "-accept", integer_to_list(Port), ssl_test_lib:version_flag(Version), + "-CAfile", CaCertFile, "-cert", CertFile, "-key", KeyFile], OpensslPort = ssl_test_lib:portable_open_port(Exe, Args), ssl_test_lib:wait_for_openssl_server(Port, proplists:get_value(protocol, Config)), @@ -1826,8 +1838,8 @@ start_erlang_server_and_openssl_client_for_alpn_npn_negotiation(Config, Data, Ca start_erlang_client_and_openssl_server_for_npn_negotiation(Config, Data, Callback) -> process_flag(trap_exit, true), - ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), - ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), ClientOpts = [{client_preferred_next_protocols, {client, [<<"spdy/2">>], <<"http/1.1">>}} | ClientOpts0], {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), @@ -1835,6 +1847,7 @@ start_erlang_client_and_openssl_server_for_npn_negotiation(Config, Data, Callbac Data = "From openssl to erlang", Port = ssl_test_lib:inet_port(node()), + CaCertFile = proplists:get_value(cacertfile, ServerOpts), CertFile = proplists:get_value(certfile, ServerOpts), KeyFile = proplists:get_value(keyfile, ServerOpts), Version = ssl_test_lib:protocol_version(Config), @@ -1842,6 +1855,7 @@ start_erlang_client_and_openssl_server_for_npn_negotiation(Config, Data, Callbac Exe = "openssl", Args = ["s_server", "-msg", "-nextprotoneg", "http/1.1,spdy/2", "-accept", integer_to_list(Port), ssl_test_lib:version_flag(Version), + "-CAfile", CaCertFile, "-cert", CertFile, "-key", KeyFile], OpensslPort = ssl_test_lib:portable_open_port(Exe, Args), @@ -1932,6 +1946,11 @@ erlang_ssl_receive(Socket, Data) -> ct:log("Connection info: ~p~n", [ssl:connection_information(Socket)]), receive + {ssl, Socket, "R\n"} -> + %% Swallow s_client renegotiation command. + %% openssl s_client connected commands can appear on + %% server side with some openssl versions. + erlang_ssl_receive(Socket,Data); {ssl, Socket, Data} -> io:format("Received ~p~n",[Data]), %% open_ssl server sometimes hangs waiting in blocking read diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index 3501622f5a..3527062a8a 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1 +1 @@ -SSL_VSN = 9.1.1 +SSL_VSN = 9.1.2 diff --git a/lib/stdlib/doc/src/ets.xml b/lib/stdlib/doc/src/ets.xml index 70d1aaa74d..b9ceaa1d69 100644 --- a/lib/stdlib/doc/src/ets.xml +++ b/lib/stdlib/doc/src/ets.xml @@ -138,23 +138,71 @@ operation. In database terms the isolation level can be seen as "serializable", as if all isolated operations are carried out serially, one after the other in a strict order.</p> + </section> - <p>No other support is available within this module that would guarantee - consistency between objects. However, function - <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso> - can be used to guarantee that a sequence of - <seealso marker="#first/1"><c>first/1</c></seealso> and - <seealso marker="#next/2"><c>next/2</c></seealso> calls traverse the - table without errors and that each existing object in the table is - visited exactly once, even if another (or the same) process - simultaneously deletes or inserts objects into the table. - Nothing else is guaranteed; in particular objects that are inserted - or deleted during such a traversal can be visited once or not at all. - Functions that internally traverse over a table, like - <seealso marker="#select/1"><c>select</c></seealso> and - <seealso marker="#match/1"><c>match</c></seealso>, - give the same guarantee as - <seealso marker="#safe_fixtable/2"><c>safe_fixtable</c></seealso>.</p> + <section><marker id="traversal"></marker> + <title>Table traversal</title> + <p>There are different ways to traverse through the objects of a table.</p> + <list type="bulleted"> + <item><p><em>Single-step</em> traversal one key at at time, using + <seealso marker="#first/1"><c>first/1</c></seealso>, + <seealso marker="#next/2"><c>next/2</c></seealso>, + <seealso marker="#last/1"><c>last/1</c></seealso> and + <seealso marker="#prev/2"><c>prev/2</c></seealso>.</p> + </item> + <item><p>Search with simple <em>match patterns</em>, using + <seealso marker="#match/1"><c>match/1/2/3</c></seealso>, + <seealso marker="#match_delete/2"><c>match_delete/2</c></seealso> and + <seealso marker="#match_object/1"><c>match_object/1/2/3</c></seealso>.</p> + </item> + <item><p>Search with more powerful <em>match specifications</em>, using + <seealso marker="#select/1"><c>select/1/2/3</c></seealso>, + <seealso marker="#select_count/2"><c>select_count/2</c></seealso>, + <seealso marker="#select_delete/2"><c>select_delete/2</c></seealso>, + <seealso marker="#select_replace/2"><c>select_replace/2</c></seealso> and + <seealso marker="#select_reverse/1"><c>select_reverse/1/2/3</c></seealso>.</p> + </item> + <item><p><em>Table conversions</em>, using + <seealso marker="#tab2file/2"><c>tab2file/2/3</c></seealso> and + <seealso marker="#tab2list/1"><c>tab2list/1</c></seealso>.</p> + </item> + </list> + <p>None of these ways of table traversal will guarantee a consistent table snapshot + if the table is also updated during the traversal. Moreover, traversals not + done in a <em>safe</em> way, on tables where keys are inserted or deleted + during the traversal, may yield the following undesired effects:</p> + <list type="bulleted"> + <item><p>Any key may be missed.</p></item> + <item><p>Any key may be found more than once.</p></item> + <item><p>The traversal may fail with <c>badarg</c> exception if keys are deleted.</p> + </item> + </list> + <p>A table traversal is <em>safe</em> if either</p> + <list type="bulleted"> + <item><p>the table is of type <c>ordered_set</c>.</p> + </item> + <item><p>the entire table traversal is done within one ETS function + call.</p> + </item> + <item><p>function <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso> + is used to keep the table fixated during the entire traversal.</p> + </item> + </list> + <p>Traversals using <c>match</c> and <c>select</c> functions may not need to + scan the entire table depending on how the key is specified. A match + pattern with a <em>fully bound key</em> (without any match variables) will + optimize the operation to a single key lookup without any table traversal + at all. For <c>ordered_set</c> a <em>partially bound key</em> will limit the + traversal to only scan a subset of the table based on term order. A + partially bound key is either a list or a tuple with a prefix that is fully + bound. Example:</p> +<pre> +1> <input>T = ets:new(t,[ordered_set]), ets:insert(T, {"555-1234", "John Smith"}).</input> +true +2> <input>%% Efficient search of all with area code 555</input> +2> <input>ets:match(T,{[$5,$5,$5,$- |'$1'],'$2'}).</input> +[["1234","John Smith"]] +</pre> </section> <section> @@ -871,6 +919,9 @@ ets:is_compiled_ms(Broken).</code> <seealso marker="#first/1"><c>first/1</c></seealso> and <seealso marker="#next/2"><c>next/2</c></seealso>.</p> <p>If the table is empty, <c>'$end_of_table'</c> is returned.</p> + <p>Use <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso> + to guarantee <seealso marker="#traversal">safe traversal</seealso> + for subsequent calls to <seealso marker="#match/1"><c>match/1</c></seealso>.</p> </desc> </func> @@ -936,6 +987,10 @@ ets:is_compiled_ms(Broken).</code> <seealso marker="#first/1"><c>first/1</c></seealso> and <seealso marker="#next/2"><c>next/2</c></seealso>.</p> <p>If the table is empty, <c>'$end_of_table'</c> is returned.</p> + <p>Use <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso> + to guarantee <seealso marker="#traversal">safe traversal</seealso> + for subsequent calls to <seealso marker="#match_object/1"> + <c>match_object/1</c></seealso>.</p> </desc> </func> @@ -1192,12 +1247,13 @@ ets:select(Table, MatchSpec),</code> <p>To find the first key in the table, use <seealso marker="#first/1"><c>first/1</c></seealso>.</p> <p>Unless a table of type <c>set</c>, <c>bag</c>, or - <c>duplicate_bag</c> is protected using + <c>duplicate_bag</c> is fixated using <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso>, - a traversal can fail if - concurrent updates are made to the table. For table - type <c>ordered_set</c>, the function returns the next key in - order, even if the object does no longer exist.</p> + a call to <c>next/2</c> will fail if <c><anno>Key1</anno></c> no longer + exists in the table. For table type <c>ordered_set</c>, the function + always returns the next key after <c><anno>Key1</anno></c> in term + order, regardless whether <c><anno>Key1</anno></c> ever existed in the + table.</p> </desc> </func> @@ -1212,7 +1268,7 @@ ets:select(Table, MatchSpec),</code> table types, the function is synonymous to <seealso marker="#next/2"><c>next/2</c></seealso>. If no previous key exists, <c>'$end_of_table'</c> is returned.</p> - <p>To find the last key in the table, use + <p>To find the last key in an <c>ordered_set</c> table, use <seealso marker="#last/1"><c>last/1</c></seealso>.</p> </desc> </func> @@ -1287,7 +1343,16 @@ ets:select(ets:repair_continuation(Broken,MS)).</code> <fsummary>Fix an ETS table for safe traversal.</fsummary> <desc> <p>Fixes a table of type <c>set</c>, <c>bag</c>, or - <c>duplicate_bag</c> for safe traversal.</p> + <c>duplicate_bag</c> for <seealso marker="#traversal"> + safe traversal</seealso> using + <seealso marker="#first/1"><c>first/1</c></seealso> & + <seealso marker="#next/2"><c>next/2</c></seealso>, + <seealso marker="#match/3"><c>match/3</c></seealso> & + <seealso marker="#match/1"><c>match/1</c></seealso>, + <seealso marker="#match_object/3"><c>match_object/3</c></seealso> & + <seealso marker="#match_object/1"><c>match_object/1</c></seealso>, or + <seealso marker="#select/3"><c>select/3</c></seealso> & + <seealso marker="#select/1"><c>select/1</c></seealso>.</p> <p>A process fixes a table by calling <c>safe_fixtable(<anno>Tab</anno>, true)</c>. The table remains fixed until the process releases it by calling @@ -1300,11 +1365,11 @@ ets:select(ets:repair_continuation(Broken,MS)).</code> <p>When a table is fixed, a sequence of <seealso marker="#first/1"><c>first/1</c></seealso> and <seealso marker="#next/2"><c>next/2</c></seealso> calls are - guaranteed to succeed, and each object in - the table is returned only once, even if objects - are removed or inserted during the traversal. The keys for new - objects inserted during the traversal <em>can</em> be returned by - <c>next/2</c> (it depends on the internal ordering of the keys).</p> + guaranteed to succeed even if keys are removed during the + traversal. The keys for objects inserted or deleted during a + traversal may or may not be returned by <c>next/2</c> depending on + the ordering of keys within the table and if the key exists at the time + <c>next/2</c> is called.</p> <p><em>Example:</em></p> <code type="none"> clean_all_with_value(Tab,X) -> @@ -1322,7 +1387,7 @@ clean_all_with_value(Tab,X,Key) -> true end, clean_all_with_value(Tab,X,ets:next(Tab,Key)).</code> - <p>Notice that no deleted objects are removed from a + <p>Notice that deleted objects are not freed from a fixed table until it has been released. If a process fixes a table but never releases it, the memory used by the deleted objects is never freed. The performance of operations on @@ -1332,9 +1397,9 @@ clean_all_with_value(Tab,X,Key) -> <c>info(Tab, safe_fixed_monotonic_time)</c></seealso>. A system with many processes fixing tables can need a monitor that sends alarms when tables have been fixed for too long.</p> - <p>Notice that for table type <c>ordered_set</c>, - <c>safe_fixtable/2</c> is not necessary, as calls to - <c>first/1</c> and <c>next/2</c> always succeed.</p> + <p>Notice that <c>safe_fixtable/2</c> is not necessary for table type + <c>ordered_set</c> and for traversals done by a single ETS function call, + like <seealso marker="#select/2"><c>select/2</c></seealso>.</p> </desc> </func> @@ -1462,7 +1527,10 @@ is_integer(X), is_integer(Y), X + Y < 4711]]></code> table, which is still faster than traversing the table object by object using <seealso marker="#first/1"><c>first/1</c></seealso> and <seealso marker="#next/2"><c>next/2</c></seealso>.</p> - <p>If the table is empty, <c>'$end_of_table'</c> is returned.</p> + <p>If the table is empty, <c>'$end_of_table'</c> is returned.</p> + <p>Use <seealso marker="#safe_fixtable/2"><c>safe_fixtable/2</c></seealso> + to guarantee <seealso marker="#traversal">safe traversal</seealso> + for subsequent calls to <seealso marker="#select/1"><c>select/1</c></seealso>.</p> </desc> </func> @@ -1519,7 +1587,7 @@ is_integer(X), is_integer(Y), X + Y < 4711]]></code> the match specification result.</p> <p>The match-and-replace operation for each individual object is guaranteed to be <seealso marker="#concurrency">atomic and isolated</seealso>. The - <c>select_replace</c> table iteration as a whole, like all other select functions, + <c>select_replace</c> table traversal as a whole, like all other select functions, does not give such guarantees.</p> <p>The match specifiction must be guaranteed to <em>retain the key</em> of any matched object. If not, <c>select_replace</c> will fail with <c>badarg</c> @@ -1940,9 +2008,8 @@ true</pre> <p>This function provides an efficient way to update one or more counters, without the trouble of having to look up an object, update the object by incrementing an element, and insert the resulting - object into the table again. (The update is done atomically, - that is, no process - can access the ETS table in the middle of the operation.)</p> + object into the table again. The operation is guaranteed to be + <seealso marker="#concurrency">atomic and isolated</seealso>.</p> <p>This function destructively update the object with key <c><anno>Key</anno></c> in table <c><anno>Tab</anno></c> by adding <c><anno>Incr</anno></c> to the element at position diff --git a/lib/stdlib/doc/src/io_lib.xml b/lib/stdlib/doc/src/io_lib.xml index cd4ca0a3a7..4d527f8ed3 100644 --- a/lib/stdlib/doc/src/io_lib.xml +++ b/lib/stdlib/doc/src/io_lib.xml @@ -385,7 +385,7 @@ <func> <name name="write" arity="1" since=""/> <name name="write" arity="2" clause_i="1" since=""/> - <name name="write" arity="2" clause_i="2" since=""/> + <name name="write" arity="2" clause_i="2" since="OTP 20.0"/> <fsummary>Write a term.</fsummary> <desc> <p>Returns a character list that represents <c><anno>Term</anno></c>. diff --git a/lib/stdlib/doc/src/lists.xml b/lib/stdlib/doc/src/lists.xml index 66146e9258..2755fb3dce 100644 --- a/lib/stdlib/doc/src/lists.xml +++ b/lib/stdlib/doc/src/lists.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2018</year> + <year>1996</year><year>2019</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -689,6 +689,18 @@ c</pre> </func> <func> + <name name="search" arity="2" since="OTP 21.0"/> + <fsummary>Find the first element that satisfies a predicate.</fsummary> + <desc> + <p>If there is a <c><anno>Value</anno></c> in <c><anno>List</anno></c> + such that <c><anno>Pred</anno>(<anno>Value</anno>)</c> returns + <c>true</c>, returns <c>{value, <anno>Value</anno>}</c> + for the first such <c><anno>Value</anno></c>, + otherwise returns <c>false</c>.</p> + </desc> + </func> + + <func> <name name="seq" arity="2" since=""/> <name name="seq" arity="3" since=""/> <fsummary>Generate a sequence of integers.</fsummary> @@ -771,18 +783,6 @@ length(lists:seq(From, To, Incr)) =:= (To - From + Incr) div Incr</code> </func> <func> - <name name="search" arity="2" since="OTP 21.0"/> - <fsummary>Find the first element that satisfies a predicate.</fsummary> - <desc> - <p>If there is a <c><anno>Value</anno></c> in <c><anno>List</anno></c> - such that <c><anno>Pred</anno>(<anno>Value</anno>)</c> returns - <c>true</c>, returns <c>{value, <anno>Value</anno>}</c> - for the first such <c><anno>Value</anno></c>, - otherwise returns <c>false</c>.</p> - </desc> - </func> - - <func> <name name="splitwith" arity="2" since=""/> <fsummary>Split a list into two lists based on a predicate.</fsummary> <desc> diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index 7ba19a98ea..993945b9c7 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -31,6 +31,21 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 3.7.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Optimize pretty printing of terms. The slower + behaviour was introduced in Erlang/OTP 20. </p> + <p> + Own Id: OTP-15573 Aux Id: ERIERL-306 </p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 3.7</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/doc/src/proplists.xml b/lib/stdlib/doc/src/proplists.xml index 9d7eb55a7e..4465103469 100644 --- a/lib/stdlib/doc/src/proplists.xml +++ b/lib/stdlib/doc/src/proplists.xml @@ -57,6 +57,11 @@ <datatype> <name name="property"/> </datatype> + + <datatype> + <name name="proplist"/> + </datatype> + </datatypes> <funcs> diff --git a/lib/stdlib/src/calendar.erl b/lib/stdlib/src/calendar.erl index bb5d450cd6..3a083d9fda 100644 --- a/lib/stdlib/src/calendar.erl +++ b/lib/stdlib/src/calendar.erl @@ -529,24 +529,41 @@ valid_date({Y, M, D}) -> %% day_to_year(DayOfEpoch) = {Year, DayOfYear} %% -%% The idea here is to first guess a year, and then adjust. Although -%% the implementation is recursive, at most 1 or 2 recursive steps +%% The idea here is to first set the upper and lower bounds for a year, +%% and then adjust a range by interpolation search. Although complexity +%% of the algorithm is log(log(n)), at most 1 or 2 recursive steps %% are taken. -%% If DayOfEpoch is very large, we need far more than 1 or 2 iterations, -%% since we just subtract a yearful of days at a time until we're there. %% -spec day_to_year(non_neg_integer()) -> {year(), day_of_year()}. day_to_year(DayOfEpoch) when DayOfEpoch >= 0 -> - Y0 = DayOfEpoch div ?DAYS_PER_YEAR, - {Y1, D1} = dty(Y0, DayOfEpoch, dy(Y0)), + YMax = DayOfEpoch div ?DAYS_PER_YEAR, + YMin = DayOfEpoch div ?DAYS_PER_LEAP_YEAR, + {Y1, D1} = dty(YMin, YMax, DayOfEpoch, dy(YMin), dy(YMax)), {Y1, DayOfEpoch - D1}. --spec dty(year(), non_neg_integer(), non_neg_integer()) -> +-spec dty(year(), year(), non_neg_integer(), non_neg_integer(), + non_neg_integer()) -> {year(), non_neg_integer()}. -dty(Y, D1, D2) when D1 < D2 -> - dty(Y-1, D1, dy(Y-1)); -dty(Y, _D1, D2) -> - {Y, D2}. +dty(Min, Max, _D1, DMin, _DMax) when Min == Max -> + {Min, DMin}; +dty(Min, Max, D1, DMin, DMax) -> + Diff = Max - Min, + Mid = Min + (Diff * (D1 - DMin)) div (DMax - DMin), + MidLength = + case is_leap_year(Mid) of + true -> ?DAYS_PER_LEAP_YEAR; + false -> ?DAYS_PER_YEAR + end, + case dy(Mid) of + D2 when D1 < D2 -> + NewMax = Mid - 1, + dty(Min, NewMax, D1, DMin, dy(NewMax)); + D2 when D1 - D2 >= MidLength -> + NewMin = Mid + 1, + dty(NewMin, Max, D1, dy(NewMin), DMax); + D2 -> + {Mid, D2} + end. %% %% The Gregorian days of the iso week 01 day 1 for a given year. diff --git a/lib/stdlib/src/erl_parse.yrl b/lib/stdlib/src/erl_parse.yrl index 9602f0bcd9..5fa9c4f75c 100644 --- a/lib/stdlib/src/erl_parse.yrl +++ b/lib/stdlib/src/erl_parse.yrl @@ -872,7 +872,7 @@ Erlang code. -type af_fun_type() :: {'type', anno(), 'fun', []} | {'type', anno(), 'fun', [{'type', anno(), 'any'} | abstract_type()]} - | {'type', anno(), 'fun', af_function_type()}. + | af_function_type(). -type af_integer_range_type() :: {'type', anno(), 'range', [af_singleton_integer_type()]}. @@ -924,10 +924,11 @@ Erlang code. -type af_function_constraint() :: [af_constraint()]. -type af_constraint() :: {'type', anno(), 'constraint', - af_lit_atom('is_subtype'), - [af_type_variable() | abstract_type()]}. % [V, T] + [af_lit_atom('is_subtype') | + [af_type_variable() | abstract_type()]]}. % [IsSubtype, [V, T]] -type af_singleton_integer_type() :: af_integer() + | af_character() | af_unary_op(af_singleton_integer_type()) | af_binary_op(af_singleton_integer_type()). diff --git a/lib/stdlib/src/erl_pp.erl b/lib/stdlib/src/erl_pp.erl index dd302a2880..ada3ff5de3 100644 --- a/lib/stdlib/src/erl_pp.erl +++ b/lib/stdlib/src/erl_pp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2018. All Rights Reserved. +%% Copyright Ericsson AB 1996-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -697,6 +697,8 @@ fun_info(Extra) -> %% BITS: +bit_grp([], _Opts) -> + leaf("<<>>"); bit_grp(Fs, Opts) -> append([['<<'], [bit_elems(Fs, Opts)], ['>>']]). diff --git a/lib/stdlib/src/io_lib.erl b/lib/stdlib/src/io_lib.erl index 8223a52873..2b5a374cf2 100644 --- a/lib/stdlib/src/io_lib.erl +++ b/lib/stdlib/src/io_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2018. All Rights Reserved. +%% Copyright Ericsson AB 1996-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -87,6 +87,8 @@ -export([limit_term/2]). +-export([chars_length/1]). + -export_type([chars/0, latin1_string/0, continuation/0, fread_error/0, fread_item/0, format_spec/0, chars_limit/0]). @@ -1131,3 +1133,17 @@ test_limit_map_assoc(K, V, D) -> test_limit(V, D - 1). test_limit_bitstring(_, _) -> ok. + +-spec chars_length(chars()) -> non_neg_integer(). +%% Optimized for deep lists S such that deep_latin1_char_list(S) is +%% true. No binaries allowed! It is assumed that $\r is never followed +%% by $\n if S is an iolist() (string:length() assigns such a +%% sub-sequence length 1). +chars_length(S) -> + try + %% true = deep_latin1_char_list(S), + iolist_size(S) + catch + _:_ -> + string:length(S) + end. diff --git a/lib/stdlib/src/io_lib_format.erl b/lib/stdlib/src/io_lib_format.erl index ab9031573b..d1aa4cd157 100644 --- a/lib/stdlib/src/io_lib_format.erl +++ b/lib/stdlib/src/io_lib_format.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2018. All Rights Reserved. +%% Copyright Ericsson AB 1996-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -248,7 +248,7 @@ count_small([#{control_char := $s}|Cs], #{w := W} = Cnts) -> count_small(Cs, Cnts#{w := W + 1}); count_small([S|Cs], #{other := Other} = Cnts) when is_list(S); is_binary(S) -> - count_small(Cs, Cnts#{other := Other + string:length(S)}); + count_small(Cs, Cnts#{other := Other + io_lib:chars_length(S)}); count_small([C|Cs], #{other := Other} = Cnts) when is_integer(C) -> count_small(Cs, Cnts#{other := Other + 1}); count_small([], #{p := P, s := S, w := W, other := Other}) -> @@ -280,10 +280,15 @@ build_limited([#{control_char := C, args := As, width := F, adjust := Ad, true -> MaxLen0 div Count0 end, S = control_limited(C, As, F, Ad, P, Pad, Enc, Str, MaxChars, I), - Len = string:length(S), NumOfPs = decr_pc(C, NumOfPs0), Count = Count0 - 1, - MaxLen = sub(MaxLen0, Len), + MaxLen = if + MaxLen0 < 0 -> % optimization + MaxLen0; + true -> + Len = io_lib:chars_length(S), + sub(MaxLen0, Len) + end, if NumOfPs > 0 -> [S|build_limited(Cs, NumOfPs, Count, MaxLen, indentation(S, I))]; @@ -406,7 +411,7 @@ base(B) when is_integer(B) -> term(T, none, _Adj, none, _Pad) -> T; term(T, none, Adj, P, Pad) -> term(T, P, Adj, P, Pad); term(T, F, Adj, P0, Pad) -> - L = string:length(T), + L = io_lib:chars_length(T), P = erlang:min(L, case P0 of none -> F; _ -> min(P0, F) end), if L > P -> @@ -713,7 +718,7 @@ fwrite_g(Fl, F, Adj, P, Pad) when P >= 1 -> end. -%% iolist_to_chars(iolist()) -> deep_char_list() +%% iolist_to_chars(iolist()) -> io_lib:chars() iolist_to_chars([C|Cs]) when is_integer(C), C >= $\000, C =< $\377 -> [C | iolist_to_chars(Cs)]; @@ -729,7 +734,7 @@ iolist_to_chars(B) when is_binary(B) -> %% cbinary() | nil()) %% cbinary() :: unicode:unicode_binary() | unicode:latin1_binary() -%% cdata_to_chars(cdata()) -> io_lib:deep_char_list() +%% cdata_to_chars(cdata()) -> io_lib:chars() cdata_to_chars([C|Cs]) when is_integer(C), C >= $\000 -> [C | cdata_to_chars(Cs)]; @@ -745,7 +750,7 @@ cdata_to_chars(B) when is_binary(B) -> limit_string(S, F, CharsLimit) when CharsLimit < 0; CharsLimit >= F -> S; limit_string(S, _F, CharsLimit) -> - case string:length(S) =< CharsLimit of + case io_lib:chars_length(S) =< CharsLimit of true -> S; false -> [string:slice(S, 0, sub(CharsLimit, 3)), "..."] end. @@ -759,11 +764,11 @@ limit_field(F, CharsLimit) -> string(S, none, _Adj, none, _Pad) -> S; string(S, F, Adj, none, Pad) -> - string_field(S, F, Adj, string:length(S), Pad); + string_field(S, F, Adj, io_lib:chars_length(S), Pad); string(S, none, _Adj, P, Pad) -> - string_field(S, P, left, string:length(S), Pad); + string_field(S, P, left, io_lib:chars_length(S), Pad); string(S, F, Adj, P, Pad) when F >= P -> - N = string:length(S), + N = io_lib:chars_length(S), if F > P -> if N > P -> adjust(flat_trunc(S, P), chars(Pad, F-P), Adj); diff --git a/lib/stdlib/src/io_lib_pretty.erl b/lib/stdlib/src/io_lib_pretty.erl index ba9d9e8434..5483ea87b5 100644 --- a/lib/stdlib/src/io_lib_pretty.erl +++ b/lib/stdlib/src/io_lib_pretty.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2018. All Rights Reserved. +%% Copyright Ericsson AB 1996-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -507,20 +507,20 @@ print_length(#{}=M, _D, _T, _RF, _Enc, _Str) when map_size(M) =:= 0 -> {"#{}", 3, 0, no_more}; print_length(Atom, _D, _T, _RF, Enc, _Str) when is_atom(Atom) -> S = write_atom(Atom, Enc), - {S, string:length(S), 0, no_more}; + {S, io_lib:chars_length(S), 0, no_more}; print_length(List, D, T, RF, Enc, Str) when is_list(List) -> %% only flat lists are "printable" case Str andalso printable_list(List, D, T, Enc) of true -> %% print as string, escaping double-quotes in the list S = write_string(List, Enc), - {S, string:length(S), 0, no_more}; + {S, io_lib:chars_length(S), 0, no_more}; {true, Prefix} -> %% Truncated lists when T < 0 could break some existing code. S = write_string(Prefix, Enc), %% NumOfDots = 0 to avoid looping--increasing the depth %% does not make Prefix longer. - {[S | "..."], 3 + string:length(S), 0, no_more}; + {[S | "..."], 3 + io_lib:chars_length(S), 0, no_more}; false -> case print_length_list(List, D, T, RF, Enc, Str) of {What, Len, Dots, _More} when Dots > 0 -> @@ -564,7 +564,7 @@ print_length(<<_/bitstring>> = Bin, D, T, RF, Enc, Str) -> {[$<,$<,S,$>,$>], 4 + length(S), 0, no_more}; {false, List} when is_list(List) -> S = io_lib:write_string(List, $"), %" - {[$<,$<,S,"/utf8>>"], 9 + string:length(S), 0, no_more}; + {[$<,$<,S,"/utf8>>"], 9 + io_lib:chars_length(S), 0, no_more}; {true, true, Prefix} -> S = io_lib:write_string(Prefix, $"), %" More = fun(T1, Dd) -> @@ -576,7 +576,7 @@ print_length(<<_/bitstring>> = Bin, D, T, RF, Enc, Str) -> More = fun(T1, Dd) -> ?FUNCTION_NAME(Bin, D+Dd, T1, RF, Enc, Str) end, - {[$<,$<,S|"/utf8...>>"], 12 + string:length(S), 3, More}; + {[$<,$<,S|"/utf8...>>"], 12 + io_lib:chars_length(S), 3, More}; false -> case io_lib:write_binary(Bin, D, T) of {S, <<>>} -> @@ -591,7 +591,7 @@ print_length(<<_/bitstring>> = Bin, D, T, RF, Enc, Str) -> print_length(Term, _D, _T, _RF, _Enc, _Str) -> S = io_lib:write(Term), %% S can contain unicode, so iolist_size(S) cannot be used here - {S, string:length(S), 0, no_more}. + {S, io_lib:chars_length(S), 0, no_more}. print_length_map(Map, 1, _T, RF, Enc, Str) -> More = fun(T1, Dd) -> ?FUNCTION_NAME(Map, 1+Dd, T1, RF, Enc, Str) end, @@ -651,7 +651,7 @@ print_length_record(Tuple, 1, _T, RF, RDefs, Enc, Str) -> {"{...}", 5, 3, More}; print_length_record(Tuple, D, T, RF, RDefs, Enc, Str) -> Name = [$# | write_atom(element(1, Tuple), Enc)], - NameL = string:length(Name), + NameL = io_lib:chars_length(Name), T1 = tsub(T, NameL+2), L = print_length_fields(RDefs, D - 1, T1, Tuple, 2, RF, Enc, Str), {Len, Dots} = list_length(L, NameL + 2, 0), @@ -677,7 +677,7 @@ print_length_fields([Def | Defs], D, T, Tuple, I, RF, Enc, Str) -> print_length_field(Def, D, T, E, RF, Enc, Str) -> Name = write_atom(Def, Enc), - NameL = string:length(Name) + 3, + NameL = io_lib:chars_length(Name) + 3, {_, Len, Dots, _} = Field = print_length(E, D, tsub(T, NameL), RF, Enc, Str), {{field, Name, NameL, Field}, NameL + Len, Dots, no_more}. @@ -738,7 +738,7 @@ printable_list(L, _D, T, _Uni) when T < 0-> io_lib:printable_list(L). slice(L, N) -> - try string:length(L) =< N of + try io_lib:chars_length(L) =< N of true -> all; false -> diff --git a/lib/stdlib/src/ms_transform.erl b/lib/stdlib/src/ms_transform.erl index 6d243e1bec..97ec785c62 100644 --- a/lib/stdlib/src/ms_transform.erl +++ b/lib/stdlib/src/ms_transform.erl @@ -556,8 +556,8 @@ tg({call, Line, {remote,_,{atom,_,erlang},{atom, Line2, FunName}},ParaList}, FunName,length(ParaList)}}) end; tg({call, Line, {remote,_,{atom,_,ModuleName}, - {atom, _, FunName}},_ParaList},B) -> - throw({error,Line,{?ERR_GENREMOTECALL+B#tgd.eb,ModuleName,FunName}}); + {atom, _, FunName}},ParaList},B) -> + throw({error,Line,{?ERR_GENREMOTECALL+B#tgd.eb,ModuleName,FunName,length(ParaList)}}); tg({cons,Line, H, T},B) -> {cons, Line, tg(H,B), tg(T,B)}; tg({nil, Line},_B) -> diff --git a/lib/stdlib/src/stdlib.appup.src b/lib/stdlib/src/stdlib.appup.src index 2a324aef82..9e5d6a3bd8 100644 --- a/lib/stdlib/src/stdlib.appup.src +++ b/lib/stdlib/src/stdlib.appup.src @@ -38,7 +38,9 @@ {<<"^3\\.5\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.5\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.6$">>,[restart_new_emulator]}, - {<<"^3\\.6\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}], + {<<"^3\\.6\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^3\\.7$">>,[restart_new_emulator]}, + {<<"^3\\.7\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}], [{<<"^3\\.4$">>,[restart_new_emulator]}, {<<"^3\\.4\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.4\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, @@ -50,4 +52,6 @@ {<<"^3\\.5\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.5\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.6$">>,[restart_new_emulator]}, - {<<"^3\\.6\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}]}. + {<<"^3\\.6\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^3\\.7$">>,[restart_new_emulator]}, + {<<"^3\\.7\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}]}. diff --git a/lib/stdlib/test/binary_module_SUITE.erl b/lib/stdlib/test/binary_module_SUITE.erl index c5cfea5e9e..e0811f19cf 100644 --- a/lib/stdlib/test/binary_module_SUITE.erl +++ b/lib/stdlib/test/binary_module_SUITE.erl @@ -22,7 +22,8 @@ -export([all/0, suite/0, interesting/1,scope_return/1,random_ref_comp/1,random_ref_sr_comp/1, random_ref_fla_comp/1,parts/1, bin_to_list/1, list_to_bin/1, - copy/1, referenced/1,guard/1,encode_decode/1,badargs/1,longest_common_trap/1]). + copy/1, referenced/1,guard/1,encode_decode/1,badargs/1,longest_common_trap/1, + check_no_invalid_read_bug/1]). -export([random_number/1, make_unaligned/1]). @@ -36,7 +37,7 @@ all() -> [scope_return,interesting, random_ref_fla_comp, random_ref_sr_comp, random_ref_comp, parts, bin_to_list, list_to_bin, copy, referenced, guard, encode_decode, badargs, - longest_common_trap]. + longest_common_trap, check_no_invalid_read_bug]. -define(MASK_ERROR(EXPR),mask_error((catch (EXPR)))). @@ -1361,3 +1362,13 @@ make_unaligned2(Bin0) when is_binary(Bin0) -> Bin. id(I) -> I. + +check_no_invalid_read_bug(Config) when is_list(Config) -> + check_no_invalid_read_bug(24); +check_no_invalid_read_bug(60) -> + ok; +check_no_invalid_read_bug(I) -> + N = 1 bsl I, + binary:encode_unsigned(N+N), + binary:encode_unsigned(N+N, little), + check_no_invalid_read_bug(I+1). diff --git a/lib/stdlib/test/calendar_SUITE.erl b/lib/stdlib/test/calendar_SUITE.erl index df62c0921d..c6d9dbca4a 100644 --- a/lib/stdlib/test/calendar_SUITE.erl +++ b/lib/stdlib/test/calendar_SUITE.erl @@ -24,6 +24,7 @@ -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, init_per_group/2,end_per_group/2, gregorian_days/1, + big_gregorian_days/1, gregorian_seconds/1, day_of_the_week/1, day_of_the_week_calibrate/1, @@ -36,13 +37,16 @@ -define(START_YEAR, 1947). -define(END_YEAR, 2012). +-define(BIG_START_YEAR, 20000000). +-define(BIG_END_YEAR, 20000020). + suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> [gregorian_days, gregorian_seconds, day_of_the_week, day_of_the_week_calibrate, leap_years, last_day_of_the_month, local_time_to_universal_time_dst, - iso_week_number, system_time, rfc3339]. + iso_week_number, system_time, rfc3339, big_gregorian_days]. groups() -> []. @@ -67,6 +71,14 @@ gregorian_days(Config) when is_list(Config) -> MaxDays = calendar:date_to_gregorian_days({?END_YEAR, 1, 1}), check_gregorian_days(Days, MaxDays). +%% Tests that date_to_gregorian_days and gregorian_days_to_date +%% are each others inverses from ?BIG_START_YEAR-01-01 up to ?BIG_END_YEAR-01-01. +%% At the same time valid_date is tested. +big_gregorian_days(Config) when is_list(Config) -> + Days = calendar:date_to_gregorian_days({?BIG_START_YEAR, 1, 1}), + MaxDays = calendar:date_to_gregorian_days({?BIG_END_YEAR, 1, 1}), + check_gregorian_days(Days, MaxDays). + %% Tests that datetime_to_gregorian_seconds and %% gregorian_seconds_to_date are each others inverses for a sampled %% number of seconds from ?START_YEAR-01-01 up to ?END_YEAR-01-01: We check diff --git a/lib/stdlib/test/erl_pp_SUITE.erl b/lib/stdlib/test/erl_pp_SUITE.erl index dda8d0a12e..f5d80e7e68 100644 --- a/lib/stdlib/test/erl_pp_SUITE.erl +++ b/lib/stdlib/test/erl_pp_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2018. All Rights Reserved. +%% Copyright Ericsson AB 2006-2019. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -51,7 +51,7 @@ otp_6321/1, otp_6911/1, otp_6914/1, otp_8150/1, otp_8238/1, otp_8473/1, otp_8522/1, otp_8567/1, otp_8664/1, otp_9147/1, otp_10302/1, otp_10820/1, otp_11100/1, otp_11861/1, pr_1014/1, - otp_13662/1, otp_14285/1]). + otp_13662/1, otp_14285/1, otp_15592/1]). %% Internal export. -export([ehook/6]). @@ -81,7 +81,7 @@ groups() -> [otp_6321, otp_6911, otp_6914, otp_8150, otp_8238, otp_8473, otp_8522, otp_8567, otp_8664, otp_9147, otp_10302, otp_10820, otp_11100, otp_11861, pr_1014, otp_13662, - otp_14285]}]. + otp_14285, otp_15592]}]. init_per_suite(Config) -> Config. @@ -1167,6 +1167,11 @@ otp_14285(_Config) -> [{encoding,latin1}])), ok. +otp_15592(_Config) -> + ok = pp_expr(<<"long12345678901234567890123456789012345678901234" + "56789012345678901234:f(<<>>)">>), + ok. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% compile(Config, Tests) -> diff --git a/lib/stdlib/vsn.mk b/lib/stdlib/vsn.mk index e0217418fe..d46173497b 100644 --- a/lib/stdlib/vsn.mk +++ b/lib/stdlib/vsn.mk @@ -1 +1 @@ -STDLIB_VSN = 3.7 +STDLIB_VSN = 3.7.1 diff --git a/lib/syntax_tools/src/erl_prettypr.erl b/lib/syntax_tools/src/erl_prettypr.erl index 6906ef1553..6ad9bec2e6 100644 --- a/lib/syntax_tools/src/erl_prettypr.erl +++ b/lib/syntax_tools/src/erl_prettypr.erl @@ -1101,8 +1101,9 @@ lay_2(Node, Ctxt) -> Ctxt1 = reset_prec(Ctxt), D1 = lay(erl_syntax:constrained_function_type_body(Node), Ctxt1), + Ctxt2 = Ctxt1#ctxt{clause = undefined}, D2 = lay(erl_syntax:constrained_function_type_argument(Node), - Ctxt1), + Ctxt2), beside(D1, beside(floating(text(" when ")), D2)); @@ -1113,7 +1114,7 @@ lay_2(Node, Ctxt) -> _ -> {"fun(", ")"} end, - Ctxt1 = reset_prec(Ctxt), + Ctxt1 = (reset_prec(Ctxt))#ctxt{clause = undefined}, D1 = case erl_syntax:function_type_arguments(Node) of any_arity -> text("(...)"); diff --git a/lib/syntax_tools/test/syntax_tools_SUITE.erl b/lib/syntax_tools/test/syntax_tools_SUITE.erl index 9dbd0e302a..6b42f7a0a1 100644 --- a/lib/syntax_tools/test/syntax_tools_SUITE.erl +++ b/lib/syntax_tools/test/syntax_tools_SUITE.erl @@ -26,14 +26,14 @@ -export([app_test/1,appup_test/1,smoke_test/1,revert/1,revert_map/1, revert_map_type/1, t_abstract_type/1,t_erl_parse_type/1,t_type/1, t_epp_dodger/1, - t_comment_scan/1,t_igor/1,t_erl_tidy/1]). + t_comment_scan/1,t_igor/1,t_erl_tidy/1,t_prettypr/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> [app_test,appup_test,smoke_test,revert,revert_map,revert_map_type, t_abstract_type,t_erl_parse_type,t_type,t_epp_dodger, - t_comment_scan,t_igor,t_erl_tidy]. + t_comment_scan,t_igor,t_erl_tidy,t_prettypr]. groups() -> []. @@ -300,6 +300,14 @@ t_comment_scan(Config) when is_list(Config) -> ok = test_comment_scan(Filenames,DataDir), ok. +t_prettypr(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + PrivDir = ?config(priv_dir, Config), + Filenames = ["type_specs.erl", + "specs_and_funs.erl"], + ok = test_prettypr(Filenames,DataDir,PrivDir), + ok. + test_files(Config) -> DataDir = ?config(data_dir, Config), [ filename:join(DataDir,Filename) || Filename <- test_files() ]. @@ -307,7 +315,8 @@ test_files(Config) -> test_files() -> ["syntax_tools_SUITE_test_module.erl", "syntax_tools_test.erl", - "type_specs.erl"]. + "type_specs.erl", + "specs_and_funs.erl"]. t_igor(Config) when is_list(Config) -> DataDir = ?config(data_dir, Config), @@ -359,6 +368,27 @@ test_comment_scan([File|Files],DataDir) -> test_comment_scan(Files,DataDir). +test_prettypr([],_,_) -> ok; +test_prettypr([File|Files],DataDir,PrivDir) -> + Filename = filename:join(DataDir,File), + io:format("Parsing ~p~n", [Filename]), + {ok, Fs0} = epp:parse_file(Filename, [], []), + Fs = erl_syntax:form_list(Fs0), + PP = erl_prettypr:format(Fs, [{paper, 120}, {ribbon, 110}]), + io:put_chars(PP), + OutFile = filename:join(PrivDir, File), + ok = file:write_file(OutFile,iolist_to_binary(PP)), + io:format("Parsing OutFile: ~s~n", [OutFile]), + {ok, Fs2} = epp:parse_file(OutFile, [], []), + case [Error || {error, _} = Error <- Fs2] of + [] -> + ok; + Errors -> + ?t:fail(Errors) + end, + test_prettypr(Files,DataDir,PrivDir). + + test_epp_dodger([], _, _) -> ok; test_epp_dodger([Filename|Files],DataDir,PrivDir) -> io:format("Parsing ~p~n", [Filename]), diff --git a/lib/syntax_tools/test/syntax_tools_SUITE_data/specs_and_funs.erl b/lib/syntax_tools/test/syntax_tools_SUITE_data/specs_and_funs.erl new file mode 100644 index 0000000000..8dfeaf5a6b --- /dev/null +++ b/lib/syntax_tools/test/syntax_tools_SUITE_data/specs_and_funs.erl @@ -0,0 +1,18 @@ +-module(specs_and_funs). + +-export([my_apply/3, two/1]). + +%% OTP-15519, ERL-815 + +-spec my_apply(Fun, Arg, fun((A) -> A)) -> Result when + Fun :: fun((Arg) -> Result), + Arg :: any(), + Result :: any(). + +my_apply(Fun, Arg, _) -> + Fun(Arg). + +-spec two(fun((A) -> A)) -> fun((B) -> B). + +two(F) -> + F(fun(X) -> X end). diff --git a/lib/tools/priv/styles.css b/lib/tools/priv/styles.css index e10e94e3ad..84f00be9fd 100644 --- a/lib/tools/priv/styles.css +++ b/lib/tools/priv/styles.css @@ -53,21 +53,25 @@ table thead { display: none; } table td.line, +table td.line a, table td.hits { width: 20px; background: #eaeaea; text-align: center; + text-decoration: none; font-size: 11px; padding: 0 10px; color: #949494; } table td.hits { width: 10px; + text-align: right; padding: 2px 5px; color: rgba(0, 0, 0, 0.6); background-color: #f0f0f0; } tr.miss td.line, +tr.miss td.line a, tr.miss td.hits { background-color: #ffdce0; border-color: #fdaeb7; @@ -76,6 +80,7 @@ tr.miss td { background-color: #ffeef0; } tr.hit td.line, +tr.hit td.line a, tr.hit td.hits { background-color: #cdffd8; border-color: #bef5cb; diff --git a/lib/tools/src/cover.erl b/lib/tools/src/cover.erl index d7269e3f27..8d4561ca9e 100644 --- a/lib/tools/src/cover.erl +++ b/lib/tools/src/cover.erl @@ -2563,11 +2563,13 @@ table_row(Line, L) -> table_data(Line, L, N) -> LineNoNL = Line -- "\n", ["<td class=\"line\" id=\"L",integer_to_list(L),"\">", + "<a href=\"#L",integer_to_list(L),"\">", integer_to_list(L), - "</td>\n", + "</a></td>\n", "<td class=\"hits\">",maybe_integer_to_list(N),"</td>\n", "<td class=\"source\"><code>",LineNoNL,"</code></td>\n</tr>\n"]. +maybe_integer_to_list(0) -> "<pre style=\"display: inline;\">:-(</pre>"; maybe_integer_to_list(N) when is_integer(N) -> integer_to_list(N); maybe_integer_to_list(_) -> "". diff --git a/lib/wx/api_gen/wx_extra/added_func.h b/lib/wx/api_gen/wx_extra/added_func.h index bffe391140..28fecbf454 100644 --- a/lib/wx/api_gen/wx_extra/added_func.h +++ b/lib/wx/api_gen/wx_extra/added_func.h @@ -44,3 +44,9 @@ class wxWindowGTK { public: double GetContentScaleFactor(); }; + +class wxDisplay { + public: + // get the resolution of this monitor in pixels per inch + wxSize GetPPI() const; +}; diff --git a/lib/wx/api_gen/wx_gen.erl b/lib/wx/api_gen/wx_gen.erl index cec6ac9ccf..8a00498319 100644 --- a/lib/wx/api_gen/wx_gen.erl +++ b/lib/wx/api_gen/wx_gen.erl @@ -701,8 +701,13 @@ parse_type2(["wxe_cb"|R],Info,Opts, T) -> parse_type2(R,Info,Opts,T#type{name=int,base=wxe_cb}); parse_type2([const|R],Info,Opts,T=#type{mod=Mod}) -> parse_type2(R,Info,Opts,T#type{mod=[const|Mod]}); -parse_type2(["unsigned"|R],Info,Opts,T=#type{mod=Mod}) -> - parse_type2(R,Info,Opts,T#type{mod=[unsigned|Mod]}); +parse_type2(["unsigned"|R],Info,Opts,T=#type{mod=Mod}) -> + case T#type.base of + undefined -> + parse_type2(R,Info,Opts,T#type{name=int, base=int, mod=[unsigned|Mod]}); + _ -> + parse_type2(R,Info,Opts,T#type{mod=[unsigned|Mod]}) + end; parse_type2(["int"|R],Info,Opts, T) -> parse_type2(R,Info,Opts,T#type{name=int,base=int}); parse_type2(["wxByte"|R],Info,Opts, T) -> diff --git a/lib/wx/api_gen/wx_gen_cpp.erl b/lib/wx/api_gen/wx_gen_cpp.erl index f13d5873a0..c6f2534380 100644 --- a/lib/wx/api_gen/wx_gen_cpp.erl +++ b/lib/wx/api_gen/wx_gen_cpp.erl @@ -1165,6 +1165,7 @@ gen_macros() -> w("#include <wx/fontdlg.h>~n"), w("#include <wx/progdlg.h>~n"), w("#include <wx/printdlg.h>~n"), + w("#include <wx/display.h>~n"), w("#include <wx/dcbuffer.h>~n"), w("#include <wx/dcmirror.h>~n"), w("#include <wx/glcanvas.h>~n"), @@ -1176,6 +1177,7 @@ gen_macros() -> w("#include <wx/sashwin.h>~n"), w("#include <wx/laywin.h>~n"), w("#include <wx/graphics.h>~n"), + w("#include <wx/dcgraph.h>~n"), w("#include <wx/aui/aui.h>~n"), w("#include <wx/datectrl.h>~n"), w("#include <wx/filepicker.h>~n"), @@ -1330,8 +1332,10 @@ encode_events(Evs) -> w(" } else {~n"), w(" send_res = rt.send();~n"), w(" if(cb->skip) event->Skip();~n"), - #class{id=MouseId} = lists:keyfind("wxMouseEvent", #class.name, Evs), - w(" if(app->recurse_level < 1 && Etype->cID != ~p) {~n", [MouseId]), + #class{id=SizeId} = lists:keyfind("wxSizeEvent", #class.name, Evs), + #class{id=MoveId} = lists:keyfind("wxMoveEvent", #class.name, Evs), + w(" if(app->recurse_level < 1 && (Etype->cID == ~w || Etype->cID == ~w)) {~n", + [SizeId, MoveId]), w(" app->recurse_level++;~n"), w(" app->dispatch_cmds();~n"), w(" app->recurse_level--;~n"), diff --git a/lib/wx/api_gen/wxapi.conf b/lib/wx/api_gen/wxapi.conf index c1b55b6875..9707fedf67 100644 --- a/lib/wx/api_gen/wxapi.conf +++ b/lib/wx/api_gen/wxapi.conf @@ -27,7 +27,7 @@ {not_const, [wxHAS_INT64,wxBYTE_ORDER,wxRETAINED, wxFONTENCODING_UTF32,wxFONTENCODING_UTF16, wxDEFAULT_CONTROL_BORDER,wxMOD_CMD, - wxMAJOR_VERSION, wxMINOR_VERSION, + wxMAJOR_VERSION, wxMINOR_VERSION, wxRELEASE_NUMBER,wxSUBRELEASE_NUMBER,wxBETA_NUMBER, %% wxALWAYS_NATIVE_DOUBLE_BUFFER, @@ -37,16 +37,30 @@ wxCURSOR_DEFAULT, wxCURSOR_ARROWWAIT, wxCURSOR_MAX, - wxLanguage + wxLanguage, + wxFONTWEIGHT_NORMAL, + wxFONTWEIGHT_LIGHT, + wxFONTWEIGHT_BOLD, + wxFONTWEIGHT_MAX ]}. -{gvars, +{gvars, [ {wxITALIC_FONT, wxFont}, {wxNORMAL_FONT, wxFont}, {wxSMALL_FONT, wxFont}, {wxSWISS_FONT, wxFont}, - + + %% Added (enum) values in 3.1.2 + {wxFONTWEIGHT_INVALID, {test_if, "wxCHECK_VERSION(3,1,2)"}}, + {wxFONTWEIGHT_THIN, {test_if, "wxCHECK_VERSION(3,1,2)"}}, + {wxFONTWEIGHT_EXTRALIGHT, {test_if, "wxCHECK_VERSION(3,1,2)"}}, + {wxFONTWEIGHT_MEDIUM, {test_if, "wxCHECK_VERSION(3,1,2)"}}, + {wxFONTWEIGHT_SEMIBOLD, {test_if, "wxCHECK_VERSION(3,1,2)"}}, + {wxFONTWEIGHT_EXTRABOLD, {test_if, "wxCHECK_VERSION(3,1,2)"}}, + {wxFONTWEIGHT_HEAVY, {test_if, "wxCHECK_VERSION(3,1,2)"}}, + {wxFONTWEIGHT_EXTRAHEAVY, {test_if, "wxCHECK_VERSION(3,1,2)"}}, + {wxBLACK_DASHED_PEN, wxPen}, {wxBLACK_PEN, wxPen}, {wxCYAN_PEN, wxPen}, @@ -2016,3 +2030,17 @@ ['GetPosition', 'GetNumberOfFiles', {'GetFiles', [{return, [{single, {list, 'm_noFiles'}}]}]} ]}. + + +{class, wxDisplay, root, [{ifdef, wxUSE_DISPLAY}], + ['wxDisplay', '~wxDisplay', + 'IsOk', + {'GetClientArea', [{test_if, "wxCHECK_VERSION(2,8,12)"}]}, + 'GetGeometry', 'GetName', 'IsPrimary', + 'GetCount', 'GetFromPoint', 'GetFromWindow', + {'GetPPI', [{test_if, "wxCHECK_VERSION(3,1,2)"}]} + ]}. + +{class, wxGCDC, wxDC, [{ifdef, wxUSE_GRAPHICS_CONTEXT}], + ['wxGCDC', '~wxGCDC', 'GetGraphicsContext', 'SetGraphicsContext' + ]}. diff --git a/lib/wx/c_src/Makefile.in b/lib/wx/c_src/Makefile.in index daa8afce83..8ec64bea7e 100644 --- a/lib/wx/c_src/Makefile.in +++ b/lib/wx/c_src/Makefile.in @@ -181,6 +181,7 @@ release_spec: opt $(INSTALL_DIR) "$(RELSYSDIR)/priv" $(INSTALL_DATA) ../priv/erlang-logo32.png "$(RELSYSDIR)/priv/" $(INSTALL_DATA) ../priv/erlang-logo64.png "$(RELSYSDIR)/priv/" + $(INSTALL_DATA) ../priv/erlang-logo128.png "$(RELSYSDIR)/priv/" $(INSTALL_PROGRAM) $(TARGET_DIR)/wxe_driver$(SO_EXT) "$(RELSYSDIR)/priv/" $(INSTALL_PROGRAM) $(TARGET_DIR)/erl_gl$(SO_EXT) "$(RELSYSDIR)/priv/" diff --git a/lib/wx/c_src/gen/wxe_derived_dest.h b/lib/wx/c_src/gen/wxe_derived_dest.h index fc0ae0d9fc..a7114eb188 100644 --- a/lib/wx/c_src/gen/wxe_derived_dest.h +++ b/lib/wx/c_src/gen/wxe_derived_dest.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2016. All Rights Reserved. + * Copyright Ericsson AB 2008-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -799,3 +799,11 @@ class EwxDCOverlay : public wxDCOverlay { EwxDCOverlay(wxOverlay& overlay,wxWindowDC * dc) : wxDCOverlay(overlay,dc) {}; }; +#if wxUSE_GRAPHICS_CONTEXT +class EwxGCDC : public wxGCDC { + public: ~EwxGCDC() {((WxeApp *)wxTheApp)->clearPtr(this);}; + EwxGCDC(const wxWindowDC& dc) : wxGCDC(dc) {}; + EwxGCDC() : wxGCDC() {}; +}; +#endif // wxUSE_GRAPHICS_CONTEXT + diff --git a/lib/wx/c_src/gen/wxe_events.cpp b/lib/wx/c_src/gen/wxe_events.cpp index 01787c8a64..8c3283a670 100644 --- a/lib/wx/c_src/gen/wxe_events.cpp +++ b/lib/wx/c_src/gen/wxe_events.cpp @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2016. All Rights Reserved. + * Copyright Ericsson AB 2008-2019. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -910,7 +910,7 @@ case 238: {// wxDropFilesEvent } else { send_res = rt.send(); if(cb->skip) event->Skip(); - if(app->recurse_level < 1 && Etype->cID != 168) { + if(app->recurse_level < 1 && (Etype->cID == 171 || Etype->cID == 172)) { app->recurse_level++; app->dispatch_cmds(); app->recurse_level--; diff --git a/lib/wx/c_src/gen/wxe_funcs.cpp b/lib/wx/c_src/gen/wxe_funcs.cpp index 74961b2e5e..32e4bf855b 100644 --- a/lib/wx/c_src/gen/wxe_funcs.cpp +++ b/lib/wx/c_src/gen/wxe_funcs.cpp @@ -32113,6 +32113,120 @@ case wxDropFilesEvent_GetFiles: { // wxDropFilesEvent::GetFiles rt.add(tmpArrayStr); break; } +#if wxUSE_DISPLAY +case wxDisplay_new: { // wxDisplay::wxDisplay + int n=0; + while( * (int*) bp) { switch (* (int*) bp) { + case 1: {bp += 4; + n = (int)*(unsigned int *) bp; bp += 4; + } break; + }}; + wxDisplay * Result = new wxDisplay(n); + newPtr((void *) Result, 239, memenv); + rt.addRef(getRef((void *)Result,memenv), "wxDisplay"); + break; +} +case wxDisplay_destruct: { // wxDisplay::~wxDisplay + wxDisplay *This = (wxDisplay *) getPtr(bp,memenv); bp += 4; + if(This) { ((WxeApp *) wxTheApp)->clearPtr((void *) This); + delete This;} + break; +} +case wxDisplay_IsOk: { // wxDisplay::IsOk + wxDisplay *This = (wxDisplay *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + bool Result = This->IsOk(); + rt.addBool(Result); + break; +} +#if wxCHECK_VERSION(2,8,12) +case wxDisplay_GetClientArea: { // wxDisplay::GetClientArea + wxDisplay *This = (wxDisplay *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + wxRect Result = This->GetClientArea(); + rt.add(Result); + break; +} +#endif +case wxDisplay_GetGeometry: { // wxDisplay::GetGeometry + wxDisplay *This = (wxDisplay *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + wxRect Result = This->GetGeometry(); + rt.add(Result); + break; +} +case wxDisplay_GetName: { // wxDisplay::GetName + wxDisplay *This = (wxDisplay *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + wxString Result = This->GetName(); + rt.add(Result); + break; +} +case wxDisplay_IsPrimary: { // wxDisplay::IsPrimary + wxDisplay *This = (wxDisplay *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + bool Result = This->IsPrimary(); + rt.addBool(Result); + break; +} +case wxDisplay_GetCount: { // wxDisplay::GetCount + int Result = wxDisplay::GetCount(); + rt.addUint(Result); + break; +} +case wxDisplay_GetFromPoint: { // wxDisplay::GetFromPoint + int * ptX = (int *) bp; bp += 4; + int * ptY = (int *) bp; bp += 4; + wxPoint pt = wxPoint(*ptX,*ptY); + int Result = wxDisplay::GetFromPoint(pt); + rt.addInt(Result); + break; +} +case wxDisplay_GetFromWindow: { // wxDisplay::GetFromWindow + wxWindow *window = (wxWindow *) getPtr(bp,memenv); bp += 4; + int Result = wxDisplay::GetFromWindow(window); + rt.addInt(Result); + break; +} +#if wxCHECK_VERSION(3,1,2) +case wxDisplay_GetPPI: { // wxDisplay::GetPPI + wxDisplay *This = (wxDisplay *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + wxSize Result = This->GetPPI(); + rt.add(Result); + break; +} +#endif +#endif // wxUSE_DISPLAY +#if wxUSE_GRAPHICS_CONTEXT +case wxGCDC_new_1: { // wxGCDC::wxGCDC + wxWindowDC *dc = (wxWindowDC *) getPtr(bp,memenv); bp += 4; + wxGCDC * Result = new EwxGCDC(*dc); + newPtr((void *) Result, 8, memenv); + rt.addRef(getRef((void *)Result,memenv), "wxGCDC"); + break; +} +case wxGCDC_new_0: { // wxGCDC::wxGCDC + wxGCDC * Result = new EwxGCDC(); + newPtr((void *) Result, 8, memenv); + rt.addRef(getRef((void *)Result,memenv), "wxGCDC"); + break; +} +case wxGCDC_GetGraphicsContext: { // wxGCDC::GetGraphicsContext + wxGCDC *This = (wxGCDC *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + wxGraphicsContext * Result = (wxGraphicsContext*)This->GetGraphicsContext(); + rt.addRef(getRef((void *)Result,memenv,8), "wxGraphicsContext"); + break; +} +case wxGCDC_SetGraphicsContext: { // wxGCDC::SetGraphicsContext + wxGCDC *This = (wxGCDC *) getPtr(bp,memenv); bp += 4; + wxGraphicsContext *ctx = (wxGraphicsContext *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + This->SetGraphicsContext(ctx); + break; +} +#endif // wxUSE_GRAPHICS_CONTEXT default: { wxeReturn error = wxeReturn(WXE_DRV_PORT, Ecmd.caller, false); error.addAtom("_wxe_error_"); error.addInt((int) op); @@ -32174,6 +32288,7 @@ bool WxeApp::delete_object(void *ptr, wxeRefData *refd) { case 231: delete (EwxLocale *) ptr; return false; case 236: delete (wxOverlay *) ptr; break; case 237: delete (EwxDCOverlay *) ptr; return false; + case 239: delete (wxDisplay *) ptr; break; default: delete (wxObject *) ptr; return false; } return true; diff --git a/lib/wx/c_src/gen/wxe_init.cpp b/lib/wx/c_src/gen/wxe_init.cpp index 6ce33a5449..5a52d69003 100644 --- a/lib/wx/c_src/gen/wxe_init.cpp +++ b/lib/wx/c_src/gen/wxe_init.cpp @@ -55,6 +55,14 @@ void WxeApp::init_nonconsts(wxeMemEnv *memenv, ErlDrvTermData caller) { rt.addTupleCount(2); rt.addAtom("wxFONTENCODING_UTF32"); rt.addInt(wxFONTENCODING_UTF32); rt.addTupleCount(2); + rt.addAtom("wxFONTWEIGHT_BOLD"); rt.addInt(wxFONTWEIGHT_BOLD); + rt.addTupleCount(2); + rt.addAtom("wxFONTWEIGHT_LIGHT"); rt.addInt(wxFONTWEIGHT_LIGHT); + rt.addTupleCount(2); + rt.addAtom("wxFONTWEIGHT_MAX"); rt.addInt(wxFONTWEIGHT_MAX); + rt.addTupleCount(2); + rt.addAtom("wxFONTWEIGHT_NORMAL"); rt.addInt(wxFONTWEIGHT_NORMAL); + rt.addTupleCount(2); rt.addAtom("wxMOD_CMD"); rt.addInt(wxMOD_CMD); rt.addTupleCount(2); rt.addAtom("wxLANGUAGE_ABKHAZIAN"); rt.addInt(wxLANGUAGE_ABKHAZIAN); @@ -654,6 +662,62 @@ void WxeApp::init_nonconsts(wxeMemEnv *memenv, ErlDrvTermData caller) { rt.addTupleCount(2); rt.addAtom("wxCYAN_PEN"); rt.addRef(getRef((void *)wxCYAN_PEN,memenv),"wxPen"); rt.addTupleCount(2); +#if wxCHECK_VERSION(3,1,2) + rt.addAtom("wxFONTWEIGHT_EXTRABOLD"); rt.addInt(wxFONTWEIGHT_EXTRABOLD); + rt.addTupleCount(2); +#else + rt.addAtom("wxFONTWEIGHT_EXTRABOLD"); rt.addAtom("undefined"); + rt.addTupleCount(2); +#endif +#if wxCHECK_VERSION(3,1,2) + rt.addAtom("wxFONTWEIGHT_EXTRAHEAVY"); rt.addInt(wxFONTWEIGHT_EXTRAHEAVY); + rt.addTupleCount(2); +#else + rt.addAtom("wxFONTWEIGHT_EXTRAHEAVY"); rt.addAtom("undefined"); + rt.addTupleCount(2); +#endif +#if wxCHECK_VERSION(3,1,2) + rt.addAtom("wxFONTWEIGHT_EXTRALIGHT"); rt.addInt(wxFONTWEIGHT_EXTRALIGHT); + rt.addTupleCount(2); +#else + rt.addAtom("wxFONTWEIGHT_EXTRALIGHT"); rt.addAtom("undefined"); + rt.addTupleCount(2); +#endif +#if wxCHECK_VERSION(3,1,2) + rt.addAtom("wxFONTWEIGHT_HEAVY"); rt.addInt(wxFONTWEIGHT_HEAVY); + rt.addTupleCount(2); +#else + rt.addAtom("wxFONTWEIGHT_HEAVY"); rt.addAtom("undefined"); + rt.addTupleCount(2); +#endif +#if wxCHECK_VERSION(3,1,2) + rt.addAtom("wxFONTWEIGHT_INVALID"); rt.addInt(wxFONTWEIGHT_INVALID); + rt.addTupleCount(2); +#else + rt.addAtom("wxFONTWEIGHT_INVALID"); rt.addAtom("undefined"); + rt.addTupleCount(2); +#endif +#if wxCHECK_VERSION(3,1,2) + rt.addAtom("wxFONTWEIGHT_MEDIUM"); rt.addInt(wxFONTWEIGHT_MEDIUM); + rt.addTupleCount(2); +#else + rt.addAtom("wxFONTWEIGHT_MEDIUM"); rt.addAtom("undefined"); + rt.addTupleCount(2); +#endif +#if wxCHECK_VERSION(3,1,2) + rt.addAtom("wxFONTWEIGHT_SEMIBOLD"); rt.addInt(wxFONTWEIGHT_SEMIBOLD); + rt.addTupleCount(2); +#else + rt.addAtom("wxFONTWEIGHT_SEMIBOLD"); rt.addAtom("undefined"); + rt.addTupleCount(2); +#endif +#if wxCHECK_VERSION(3,1,2) + rt.addAtom("wxFONTWEIGHT_THIN"); rt.addInt(wxFONTWEIGHT_THIN); + rt.addTupleCount(2); +#else + rt.addAtom("wxFONTWEIGHT_THIN"); rt.addAtom("undefined"); + rt.addTupleCount(2); +#endif rt.addAtom("wxGREEN"); rt.add(*(wxGREEN)); rt.addTupleCount(2); rt.addAtom("wxGREEN_BRUSH"); rt.addRef(getRef((void *)wxGREEN_BRUSH,memenv),"wxBrush"); @@ -723,7 +787,7 @@ void WxeApp::init_nonconsts(wxeMemEnv *memenv, ErlDrvTermData caller) { rt.addAtom("wx_GL_COMPAT_PROFILE"); rt.addAtom("undefined"); rt.addTupleCount(2); #endif - rt.endList(309); + rt.endList(321); rt.addTupleCount(2); rt.send(); } diff --git a/lib/wx/c_src/gen/wxe_macros.h b/lib/wx/c_src/gen/wxe_macros.h index 4c8e52def2..c23e8a83bd 100644 --- a/lib/wx/c_src/gen/wxe_macros.h +++ b/lib/wx/c_src/gen/wxe_macros.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2017. All Rights Reserved. + * Copyright Ericsson AB 2008-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -35,6 +35,7 @@ #include <wx/fontdlg.h> #include <wx/progdlg.h> #include <wx/printdlg.h> +#include <wx/display.h> #include <wx/dcbuffer.h> #include <wx/dcmirror.h> #include <wx/glcanvas.h> @@ -46,6 +47,7 @@ #include <wx/sashwin.h> #include <wx/laywin.h> #include <wx/graphics.h> +#include <wx/dcgraph.h> #include <wx/aui/aui.h> #include <wx/datectrl.h> #include <wx/filepicker.h> @@ -3426,5 +3428,21 @@ #define wxDropFilesEvent_GetPosition 3597 #define wxDropFilesEvent_GetNumberOfFiles 3598 #define wxDropFilesEvent_GetFiles 3599 +#define wxDisplay_new 3600 +#define wxDisplay_destruct 3601 +#define wxDisplay_IsOk 3602 +#define wxDisplay_GetClientArea 3603 +#define wxDisplay_GetGeometry 3604 +#define wxDisplay_GetName 3605 +#define wxDisplay_IsPrimary 3606 +#define wxDisplay_GetCount 3607 +#define wxDisplay_GetFromPoint 3608 +#define wxDisplay_GetFromWindow 3609 +#define wxDisplay_GetPPI 3610 +#define wxGCDC_new_1 3611 +#define wxGCDC_new_0 3612 +#define wxGCDC_destruct 3613 +#define wxGCDC_GetGraphicsContext 3614 +#define wxGCDC_SetGraphicsContext 3615 diff --git a/lib/wx/c_src/wxe_impl.cpp b/lib/wx/c_src/wxe_impl.cpp index bd22502d00..43b5476073 100644 --- a/lib/wx/c_src/wxe_impl.cpp +++ b/lib/wx/c_src/wxe_impl.cpp @@ -267,7 +267,7 @@ int WxeApp::dispatch_cmds() return more; } -#define BREAK_BATCH 10000 +#define CHECK_EVENTS 10000 int WxeApp::dispatch(wxeFifo * batch) { @@ -278,13 +278,14 @@ int WxeApp::dispatch(wxeFifo * batch) erl_drv_mutex_lock(wxe_batch_locker_m); while(true) { while((event = batch->Get()) != NULL) { + wait += 1; erl_drv_mutex_unlock(wxe_batch_locker_m); switch(event->op) { case WXE_BATCH_END: if(blevel>0) { blevel--; if(blevel==0) - wait += BREAK_BATCH/4; + wait += CHECK_EVENTS/4; } break; case WXE_BATCH_BEGIN: @@ -314,21 +315,18 @@ int WxeApp::dispatch(wxeFifo * batch) break; } event->Delete(); + if(wait > CHECK_EVENTS) + return 1; // Let wx check for events erl_drv_mutex_lock(wxe_batch_locker_m); batch->Cleanup(); } - if(blevel <= 0 || wait >= BREAK_BATCH) { + if(blevel <= 0) { erl_drv_mutex_unlock(wxe_batch_locker_m); - if(blevel > 0) { - return 1; // We are still in a batch but we can let wx check for events - } else { - return 0; - } + return 0; } // sleep until something happens // fprintf(stderr, "%s:%d sleep %d %d %d\r\n", __FILE__, __LINE__, batch->m_n, blevel, wait);fflush(stderr); wxe_needs_signal = 1; - wait += 1; while(batch->m_n == 0) { erl_drv_cond_wait(wxe_batch_locker_c, wxe_batch_locker_m); } diff --git a/lib/wx/c_src/wxe_ps_init.c b/lib/wx/c_src/wxe_ps_init.c index 4b3b47a80b..62c7c51c13 100644 --- a/lib/wx/c_src/wxe_ps_init.c +++ b/lib/wx/c_src/wxe_ps_init.c @@ -64,6 +64,10 @@ void * wxe_ps_init2() { size_t app_len = 127; char app_title_buf[128]; char * app_title; + size_t app_icon_len = 1023; + char app_icon_buf[1024]; + char * app_icon; + // Setup and enable gui pool = [[NSAutoreleasePool alloc] init]; @@ -78,9 +82,15 @@ void * wxe_ps_init2() { if(!GetCurrentProcess(&psn)) { CPSSetProcessName(&psn, app_title?app_title:"Erlang"); } - // Load and set icon + // Enable setting custom application icon for Mac OS X + res = erl_drv_getenv("WX_APP_ICON", app_icon_buf, &app_icon_len); NSMutableString *file = [[NSMutableString alloc] init]; - [file appendFormat:@"%s/%s", erl_wx_privdir, "erlang-logo64.png"]; + if (res >= 0) { + [file appendFormat:@"%s", app_icon_buf]; + } else { + [file appendFormat:@"%s/%s", erl_wx_privdir, "erlang-logo128.png"]; + } + // Load and set icon NSImage *icon = [[NSImage alloc] initWithContentsOfFile: file]; [NSApp setApplicationIconImage: icon]; }; diff --git a/lib/wx/include/wx.hrl b/lib/wx/include/wx.hrl index 23f3b95403..2c145595ee 100644 --- a/lib/wx/include/wx.hrl +++ b/lib/wx/include/wx.hrl @@ -398,6 +398,14 @@ -define(wxCYAN, wxe_util:get_const(wxCYAN)). -define(wxCYAN_BRUSH, wxe_util:get_const(wxCYAN_BRUSH)). -define(wxCYAN_PEN, wxe_util:get_const(wxCYAN_PEN)). +-define(wxFONTWEIGHT_EXTRABOLD, wxe_util:get_const(wxFONTWEIGHT_EXTRABOLD)). +-define(wxFONTWEIGHT_EXTRAHEAVY, wxe_util:get_const(wxFONTWEIGHT_EXTRAHEAVY)). +-define(wxFONTWEIGHT_EXTRALIGHT, wxe_util:get_const(wxFONTWEIGHT_EXTRALIGHT)). +-define(wxFONTWEIGHT_HEAVY, wxe_util:get_const(wxFONTWEIGHT_HEAVY)). +-define(wxFONTWEIGHT_INVALID, wxe_util:get_const(wxFONTWEIGHT_INVALID)). +-define(wxFONTWEIGHT_MEDIUM, wxe_util:get_const(wxFONTWEIGHT_MEDIUM)). +-define(wxFONTWEIGHT_SEMIBOLD, wxe_util:get_const(wxFONTWEIGHT_SEMIBOLD)). +-define(wxFONTWEIGHT_THIN, wxe_util:get_const(wxFONTWEIGHT_THIN)). -define(wxGREEN, wxe_util:get_const(wxGREEN)). -define(wxGREEN_BRUSH, wxe_util:get_const(wxGREEN_BRUSH)). -define(wxGREEN_PEN, wxe_util:get_const(wxGREEN_PEN)). @@ -1685,10 +1693,10 @@ -define(wxFONTSTYLE_SLANT, ?wxSLANT). -define(wxFONTSTYLE_MAX, (?wxSLANT+1)). % From "font.h": wxFontWeight --define(wxFONTWEIGHT_NORMAL, ?wxNORMAL). --define(wxFONTWEIGHT_LIGHT, ?wxLIGHT). --define(wxFONTWEIGHT_BOLD, ?wxBOLD). --define(wxFONTWEIGHT_MAX, (?wxBOLD+1)). +-define(wxFONTWEIGHT_NORMAL, wxe_util:get_const(wxFONTWEIGHT_NORMAL)). +-define(wxFONTWEIGHT_LIGHT, wxe_util:get_const(wxFONTWEIGHT_LIGHT)). +-define(wxFONTWEIGHT_BOLD, wxe_util:get_const(wxFONTWEIGHT_BOLD)). +-define(wxFONTWEIGHT_MAX, wxe_util:get_const(wxFONTWEIGHT_MAX)). % From "fontenc.h": wxFontEncoding -define(wxFONTENCODING_SYSTEM, -1). -define(wxFONTENCODING_DEFAULT, 0). diff --git a/lib/wx/src/gen/wxDisplay.erl b/lib/wx/src/gen/wxDisplay.erl new file mode 100644 index 0000000000..b6a2bf22ac --- /dev/null +++ b/lib/wx/src/gen/wxDisplay.erl @@ -0,0 +1,131 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% This file is generated DO NOT EDIT + +%% @doc See external documentation: <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html">wxDisplay</a>. +%% @type wxDisplay(). An object reference, The representation is internal +%% and can be changed without notice. It can't be used for comparsion +%% stored on disc or distributed for use on other nodes. + +-module(wxDisplay). +-include("wxe.hrl"). +-export([destroy/1,getClientArea/1,getCount/0,getFromPoint/1,getFromWindow/1, + getGeometry/1,getName/1,getPPI/1,isOk/1,isPrimary/1,new/0,new/1]). + +%% inherited exports +-export([parent_class/1]). + +-export_type([wxDisplay/0]). +%% @hidden +parent_class(_Class) -> erlang:error({badtype, ?MODULE}). + +-type wxDisplay() :: wx:wx_object(). +%% @equiv new([]) +-spec new() -> wxDisplay(). + +new() -> + new([]). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html#wxdisplaywxdisplay">external documentation</a>. +-spec new([Option]) -> wxDisplay() when + Option :: {'n', integer()}. +new(Options) + when is_list(Options) -> + MOpts = fun({n, N}, Acc) -> [<<1:32/?UI,N:32/?UI>>|Acc]; + (BadOpt, _) -> erlang:error({badoption, BadOpt}) end, + BinOpt = list_to_binary(lists:foldl(MOpts, [<<0:32>>], Options)), + wxe_util:construct(?wxDisplay_new, + <<BinOpt/binary>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html#wxdisplayisok">external documentation</a>. +-spec isOk(This) -> boolean() when + This::wxDisplay(). +isOk(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxDisplay), + wxe_util:call(?wxDisplay_IsOk, + <<ThisRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html#wxdisplaygetclientarea">external documentation</a>. +-spec getClientArea(This) -> {X::integer(), Y::integer(), W::integer(), H::integer()} when + This::wxDisplay(). +getClientArea(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxDisplay), + wxe_util:call(?wxDisplay_GetClientArea, + <<ThisRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html#wxdisplaygetgeometry">external documentation</a>. +-spec getGeometry(This) -> {X::integer(), Y::integer(), W::integer(), H::integer()} when + This::wxDisplay(). +getGeometry(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxDisplay), + wxe_util:call(?wxDisplay_GetGeometry, + <<ThisRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html#wxdisplaygetname">external documentation</a>. +-spec getName(This) -> unicode:charlist() when + This::wxDisplay(). +getName(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxDisplay), + wxe_util:call(?wxDisplay_GetName, + <<ThisRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html#wxdisplayisprimary">external documentation</a>. +-spec isPrimary(This) -> boolean() when + This::wxDisplay(). +isPrimary(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxDisplay), + wxe_util:call(?wxDisplay_IsPrimary, + <<ThisRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html#wxdisplaygetcount">external documentation</a>. +-spec getCount() -> integer(). +getCount() -> + wxe_util:call(?wxDisplay_GetCount, + <<>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html#wxdisplaygetfrompoint">external documentation</a>. +-spec getFromPoint(Pt) -> integer() when + Pt::{X::integer(), Y::integer()}. +getFromPoint({PtX,PtY}) + when is_integer(PtX),is_integer(PtY) -> + wxe_util:call(?wxDisplay_GetFromPoint, + <<PtX:32/?UI,PtY:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html#wxdisplaygetfromwindow">external documentation</a>. +-spec getFromWindow(Window) -> integer() when + Window::wxWindow:wxWindow(). +getFromWindow(#wx_ref{type=WindowT,ref=WindowRef}) -> + ?CLASS(WindowT,wxWindow), + wxe_util:call(?wxDisplay_GetFromWindow, + <<WindowRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxdisplay.html#wxdisplaygetppi">external documentation</a>. +-spec getPPI(This) -> {W::integer(), H::integer()} when + This::wxDisplay(). +getPPI(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxDisplay), + wxe_util:call(?wxDisplay_GetPPI, + <<ThisRef:32/?UI>>). + +%% @doc Destroys this object, do not use object again +-spec destroy(This::wxDisplay()) -> 'ok'. +destroy(Obj=#wx_ref{type=Type}) -> + ?CLASS(Type,wxDisplay), + wxe_util:destroy(?wxDisplay_destruct,Obj), + ok. diff --git a/lib/wx/src/gen/wxGCDC.erl b/lib/wx/src/gen/wxGCDC.erl new file mode 100644 index 0000000000..467013b14e --- /dev/null +++ b/lib/wx/src/gen/wxGCDC.erl @@ -0,0 +1,287 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% This file is generated DO NOT EDIT + +%% @doc See external documentation: <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxgcdc.html">wxGCDC</a>. +%% <p>This class is derived (and can use functions) from: +%% <br />{@link wxDC} +%% </p> +%% @type wxGCDC(). An object reference, The representation is internal +%% and can be changed without notice. It can't be used for comparsion +%% stored on disc or distributed for use on other nodes. + +-module(wxGCDC). +-include("wxe.hrl"). +-export([destroy/1,getGraphicsContext/1,new/0,new/1,setGraphicsContext/2]). + +%% inherited exports +-export([blit/5,blit/6,calcBoundingBox/3,clear/1,computeScaleAndOrigin/1,crossHair/2, + destroyClippingRegion/1,deviceToLogicalX/2,deviceToLogicalXRel/2, + deviceToLogicalY/2,deviceToLogicalYRel/2,drawArc/4,drawBitmap/3,drawBitmap/4, + drawCheckMark/2,drawCircle/3,drawEllipse/2,drawEllipse/3,drawEllipticArc/5, + drawIcon/3,drawLabel/3,drawLabel/4,drawLine/3,drawLines/2,drawLines/3, + drawPoint/2,drawPolygon/2,drawPolygon/3,drawRectangle/2,drawRectangle/3, + drawRotatedText/4,drawRoundedRectangle/3,drawRoundedRectangle/4, + drawText/3,endDoc/1,endPage/1,floodFill/3,floodFill/4,getBackground/1, + getBackgroundMode/1,getBrush/1,getCharHeight/1,getCharWidth/1,getClippingBox/1, + getFont/1,getLayoutDirection/1,getLogicalFunction/1,getMapMode/1, + getMultiLineTextExtent/2,getMultiLineTextExtent/3,getPPI/1,getPartialTextExtents/2, + getPen/1,getPixel/2,getSize/1,getSizeMM/1,getTextBackground/1,getTextExtent/2, + getTextExtent/3,getTextForeground/1,getUserScale/1,gradientFillConcentric/4, + gradientFillConcentric/5,gradientFillLinear/4,gradientFillLinear/5, + isOk/1,logicalToDeviceX/2,logicalToDeviceXRel/2,logicalToDeviceY/2, + logicalToDeviceYRel/2,maxX/1,maxY/1,minX/1,minY/1,parent_class/1,resetBoundingBox/1, + setAxisOrientation/3,setBackground/2,setBackgroundMode/2,setBrush/2, + setClippingRegion/2,setClippingRegion/3,setDeviceOrigin/3,setFont/2, + setLayoutDirection/2,setLogicalFunction/2,setMapMode/2,setPalette/2, + setPen/2,setTextBackground/2,setTextForeground/2,setUserScale/3,startDoc/2, + startPage/1]). + +-export_type([wxGCDC/0]). +-compile([{nowarn_deprecated_function, {wxDC,computeScaleAndOrigin,1}}]). + +%% @hidden +parent_class(wxDC) -> true; +parent_class(_Class) -> erlang:error({badtype, ?MODULE}). + +-type wxGCDC() :: wx:wx_object(). +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxgcdc.html#wxgcdcwxgcdc">external documentation</a>. +-spec new() -> wxGCDC(). +new() -> + wxe_util:construct(?wxGCDC_new_0, + <<>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxgcdc.html#wxgcdcwxgcdc">external documentation</a>. +-spec new(Dc) -> wxGCDC() when + Dc::wxWindowDC:wxWindowDC(). +new(#wx_ref{type=DcT,ref=DcRef}) -> + ?CLASS(DcT,wxWindowDC), + wxe_util:construct(?wxGCDC_new_1, + <<DcRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxgcdc.html#wxgcdcgetgraphicscontext">external documentation</a>. +-spec getGraphicsContext(This) -> wxGraphicsContext:wxGraphicsContext() when + This::wxGCDC(). +getGraphicsContext(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxGCDC), + wxe_util:call(?wxGCDC_GetGraphicsContext, + <<ThisRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxgcdc.html#wxgcdcsetgraphicscontext">external documentation</a>. +-spec setGraphicsContext(This, Ctx) -> 'ok' when + This::wxGCDC(), Ctx::wxGraphicsContext:wxGraphicsContext(). +setGraphicsContext(#wx_ref{type=ThisT,ref=ThisRef},#wx_ref{type=CtxT,ref=CtxRef}) -> + ?CLASS(ThisT,wxGCDC), + ?CLASS(CtxT,wxGraphicsContext), + wxe_util:cast(?wxGCDC_SetGraphicsContext, + <<ThisRef:32/?UI,CtxRef:32/?UI>>). + +%% @doc Destroys this object, do not use object again +-spec destroy(This::wxGCDC()) -> 'ok'. +destroy(Obj=#wx_ref{type=Type}) -> + ?CLASS(Type,wxGCDC), + wxe_util:destroy(?DESTROY_OBJECT,Obj), + ok. + %% From wxDC +%% @hidden +startPage(This) -> wxDC:startPage(This). +%% @hidden +startDoc(This,Message) -> wxDC:startDoc(This,Message). +%% @hidden +setUserScale(This,X,Y) -> wxDC:setUserScale(This,X,Y). +%% @hidden +setTextForeground(This,Colour) -> wxDC:setTextForeground(This,Colour). +%% @hidden +setTextBackground(This,Colour) -> wxDC:setTextBackground(This,Colour). +%% @hidden +setPen(This,Pen) -> wxDC:setPen(This,Pen). +%% @hidden +setPalette(This,Palette) -> wxDC:setPalette(This,Palette). +%% @hidden +setMapMode(This,Mode) -> wxDC:setMapMode(This,Mode). +%% @hidden +setLogicalFunction(This,Function) -> wxDC:setLogicalFunction(This,Function). +%% @hidden +setLayoutDirection(This,Dir) -> wxDC:setLayoutDirection(This,Dir). +%% @hidden +setFont(This,Font) -> wxDC:setFont(This,Font). +%% @hidden +setDeviceOrigin(This,X,Y) -> wxDC:setDeviceOrigin(This,X,Y). +%% @hidden +setClippingRegion(This,Pt,Sz) -> wxDC:setClippingRegion(This,Pt,Sz). +%% @hidden +setClippingRegion(This,Region) -> wxDC:setClippingRegion(This,Region). +%% @hidden +setBrush(This,Brush) -> wxDC:setBrush(This,Brush). +%% @hidden +setBackgroundMode(This,Mode) -> wxDC:setBackgroundMode(This,Mode). +%% @hidden +setBackground(This,Brush) -> wxDC:setBackground(This,Brush). +%% @hidden +setAxisOrientation(This,XLeftRight,YBottomUp) -> wxDC:setAxisOrientation(This,XLeftRight,YBottomUp). +%% @hidden +resetBoundingBox(This) -> wxDC:resetBoundingBox(This). +%% @hidden +isOk(This) -> wxDC:isOk(This). +%% @hidden +minY(This) -> wxDC:minY(This). +%% @hidden +minX(This) -> wxDC:minX(This). +%% @hidden +maxY(This) -> wxDC:maxY(This). +%% @hidden +maxX(This) -> wxDC:maxX(This). +%% @hidden +logicalToDeviceYRel(This,Y) -> wxDC:logicalToDeviceYRel(This,Y). +%% @hidden +logicalToDeviceY(This,Y) -> wxDC:logicalToDeviceY(This,Y). +%% @hidden +logicalToDeviceXRel(This,X) -> wxDC:logicalToDeviceXRel(This,X). +%% @hidden +logicalToDeviceX(This,X) -> wxDC:logicalToDeviceX(This,X). +%% @hidden +gradientFillLinear(This,Rect,InitialColour,DestColour, Options) -> wxDC:gradientFillLinear(This,Rect,InitialColour,DestColour, Options). +%% @hidden +gradientFillLinear(This,Rect,InitialColour,DestColour) -> wxDC:gradientFillLinear(This,Rect,InitialColour,DestColour). +%% @hidden +gradientFillConcentric(This,Rect,InitialColour,DestColour,CircleCenter) -> wxDC:gradientFillConcentric(This,Rect,InitialColour,DestColour,CircleCenter). +%% @hidden +gradientFillConcentric(This,Rect,InitialColour,DestColour) -> wxDC:gradientFillConcentric(This,Rect,InitialColour,DestColour). +%% @hidden +getUserScale(This) -> wxDC:getUserScale(This). +%% @hidden +getTextForeground(This) -> wxDC:getTextForeground(This). +%% @hidden +getTextExtent(This,String, Options) -> wxDC:getTextExtent(This,String, Options). +%% @hidden +getTextExtent(This,String) -> wxDC:getTextExtent(This,String). +%% @hidden +getTextBackground(This) -> wxDC:getTextBackground(This). +%% @hidden +getSizeMM(This) -> wxDC:getSizeMM(This). +%% @hidden +getSize(This) -> wxDC:getSize(This). +%% @hidden +getPPI(This) -> wxDC:getPPI(This). +%% @hidden +getPixel(This,Pt) -> wxDC:getPixel(This,Pt). +%% @hidden +getPen(This) -> wxDC:getPen(This). +%% @hidden +getPartialTextExtents(This,Text) -> wxDC:getPartialTextExtents(This,Text). +%% @hidden +getMultiLineTextExtent(This,String, Options) -> wxDC:getMultiLineTextExtent(This,String, Options). +%% @hidden +getMultiLineTextExtent(This,String) -> wxDC:getMultiLineTextExtent(This,String). +%% @hidden +getMapMode(This) -> wxDC:getMapMode(This). +%% @hidden +getLogicalFunction(This) -> wxDC:getLogicalFunction(This). +%% @hidden +getLayoutDirection(This) -> wxDC:getLayoutDirection(This). +%% @hidden +getFont(This) -> wxDC:getFont(This). +%% @hidden +getClippingBox(This) -> wxDC:getClippingBox(This). +%% @hidden +getCharWidth(This) -> wxDC:getCharWidth(This). +%% @hidden +getCharHeight(This) -> wxDC:getCharHeight(This). +%% @hidden +getBrush(This) -> wxDC:getBrush(This). +%% @hidden +getBackgroundMode(This) -> wxDC:getBackgroundMode(This). +%% @hidden +getBackground(This) -> wxDC:getBackground(This). +%% @hidden +floodFill(This,Pt,Col, Options) -> wxDC:floodFill(This,Pt,Col, Options). +%% @hidden +floodFill(This,Pt,Col) -> wxDC:floodFill(This,Pt,Col). +%% @hidden +endPage(This) -> wxDC:endPage(This). +%% @hidden +endDoc(This) -> wxDC:endDoc(This). +%% @hidden +drawText(This,Text,Pt) -> wxDC:drawText(This,Text,Pt). +%% @hidden +drawRoundedRectangle(This,Pt,Sz,Radius) -> wxDC:drawRoundedRectangle(This,Pt,Sz,Radius). +%% @hidden +drawRoundedRectangle(This,R,Radius) -> wxDC:drawRoundedRectangle(This,R,Radius). +%% @hidden +drawRotatedText(This,Text,Pt,Angle) -> wxDC:drawRotatedText(This,Text,Pt,Angle). +%% @hidden +drawRectangle(This,Pt,Sz) -> wxDC:drawRectangle(This,Pt,Sz). +%% @hidden +drawRectangle(This,Rect) -> wxDC:drawRectangle(This,Rect). +%% @hidden +drawPoint(This,Pt) -> wxDC:drawPoint(This,Pt). +%% @hidden +drawPolygon(This,Points, Options) -> wxDC:drawPolygon(This,Points, Options). +%% @hidden +drawPolygon(This,Points) -> wxDC:drawPolygon(This,Points). +%% @hidden +drawLines(This,Points, Options) -> wxDC:drawLines(This,Points, Options). +%% @hidden +drawLines(This,Points) -> wxDC:drawLines(This,Points). +%% @hidden +drawLine(This,Pt1,Pt2) -> wxDC:drawLine(This,Pt1,Pt2). +%% @hidden +drawLabel(This,Text,Rect, Options) -> wxDC:drawLabel(This,Text,Rect, Options). +%% @hidden +drawLabel(This,Text,Rect) -> wxDC:drawLabel(This,Text,Rect). +%% @hidden +drawIcon(This,Icon,Pt) -> wxDC:drawIcon(This,Icon,Pt). +%% @hidden +drawEllipticArc(This,Pt,Sz,Sa,Ea) -> wxDC:drawEllipticArc(This,Pt,Sz,Sa,Ea). +%% @hidden +drawEllipse(This,Pt,Sz) -> wxDC:drawEllipse(This,Pt,Sz). +%% @hidden +drawEllipse(This,Rect) -> wxDC:drawEllipse(This,Rect). +%% @hidden +drawCircle(This,Pt,Radius) -> wxDC:drawCircle(This,Pt,Radius). +%% @hidden +drawCheckMark(This,Rect) -> wxDC:drawCheckMark(This,Rect). +%% @hidden +drawBitmap(This,Bmp,Pt, Options) -> wxDC:drawBitmap(This,Bmp,Pt, Options). +%% @hidden +drawBitmap(This,Bmp,Pt) -> wxDC:drawBitmap(This,Bmp,Pt). +%% @hidden +drawArc(This,Pt1,Pt2,Centre) -> wxDC:drawArc(This,Pt1,Pt2,Centre). +%% @hidden +deviceToLogicalYRel(This,Y) -> wxDC:deviceToLogicalYRel(This,Y). +%% @hidden +deviceToLogicalY(This,Y) -> wxDC:deviceToLogicalY(This,Y). +%% @hidden +deviceToLogicalXRel(This,X) -> wxDC:deviceToLogicalXRel(This,X). +%% @hidden +deviceToLogicalX(This,X) -> wxDC:deviceToLogicalX(This,X). +%% @hidden +destroyClippingRegion(This) -> wxDC:destroyClippingRegion(This). +%% @hidden +crossHair(This,Pt) -> wxDC:crossHair(This,Pt). +%% @hidden +computeScaleAndOrigin(This) -> wxDC:computeScaleAndOrigin(This). +%% @hidden +clear(This) -> wxDC:clear(This). +%% @hidden +calcBoundingBox(This,X,Y) -> wxDC:calcBoundingBox(This,X,Y). +%% @hidden +blit(This,DestPt,Sz,Source,SrcPt, Options) -> wxDC:blit(This,DestPt,Sz,Source,SrcPt, Options). +%% @hidden +blit(This,DestPt,Sz,Source,SrcPt) -> wxDC:blit(This,DestPt,Sz,Source,SrcPt). diff --git a/lib/wx/src/gen/wxe_debug.hrl b/lib/wx/src/gen/wxe_debug.hrl index 533f9f2df0..b64a1b4c61 100644 --- a/lib/wx/src/gen/wxe_debug.hrl +++ b/lib/wx/src/gen/wxe_debug.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -3377,6 +3377,22 @@ wxdebug_table() -> {3597, {wxDropFilesEvent, getPosition, 0}}, {3598, {wxDropFilesEvent, getNumberOfFiles, 0}}, {3599, {wxDropFilesEvent, getFiles, 0}}, + {3600, {wxDisplay, new, 1}}, + {3601, {wxDisplay, destruct, 0}}, + {3602, {wxDisplay, isOk, 0}}, + {3603, {wxDisplay, getClientArea, 0}}, + {3604, {wxDisplay, getGeometry, 0}}, + {3605, {wxDisplay, getName, 0}}, + {3606, {wxDisplay, isPrimary, 0}}, + {3607, {wxDisplay, getCount, 0}}, + {3608, {wxDisplay, getFromPoint, 1}}, + {3609, {wxDisplay, getFromWindow, 1}}, + {3610, {wxDisplay, getPPI, 0}}, + {3611, {wxGCDC, new_1, 1}}, + {3612, {wxGCDC, new_0, 0}}, + {3613, {wxGCDC, destruct, 0}}, + {3614, {wxGCDC, getGraphicsContext, 0}}, + {3615, {wxGCDC, setGraphicsContext, 1}}, {-1, {mod, func, -1}} ]. diff --git a/lib/wx/src/gen/wxe_funcs.hrl b/lib/wx/src/gen/wxe_funcs.hrl index 14b5545676..030f7f117d 100644 --- a/lib/wx/src/gen/wxe_funcs.hrl +++ b/lib/wx/src/gen/wxe_funcs.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -3374,3 +3374,19 @@ -define(wxDropFilesEvent_GetPosition, 3597). -define(wxDropFilesEvent_GetNumberOfFiles, 3598). -define(wxDropFilesEvent_GetFiles, 3599). +-define(wxDisplay_new, 3600). +-define(wxDisplay_destruct, 3601). +-define(wxDisplay_IsOk, 3602). +-define(wxDisplay_GetClientArea, 3603). +-define(wxDisplay_GetGeometry, 3604). +-define(wxDisplay_GetName, 3605). +-define(wxDisplay_IsPrimary, 3606). +-define(wxDisplay_GetCount, 3607). +-define(wxDisplay_GetFromPoint, 3608). +-define(wxDisplay_GetFromWindow, 3609). +-define(wxDisplay_GetPPI, 3610). +-define(wxGCDC_new_1, 3611). +-define(wxGCDC_new_0, 3612). +-define(wxGCDC_destruct, 3613). +-define(wxGCDC_GetGraphicsContext, 3614). +-define(wxGCDC_SetGraphicsContext, 3615). diff --git a/lib/xmerl/doc/src/notes.xml b/lib/xmerl/doc/src/notes.xml index a97036127e..7f6874e36b 100644 --- a/lib/xmerl/doc/src/notes.xml +++ b/lib/xmerl/doc/src/notes.xml @@ -32,6 +32,21 @@ <p>This document describes the changes made to the Xmerl application.</p> +<section><title>Xmerl 1.3.19</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>The charset detection parsing crash in some cases when + the XML directive is not syntactic correct.</p> + <p> + Own Id: OTP-15492 Aux Id: ERIERL-283 </p> + </item> + </list> + </section> + +</section> + <section><title>Xmerl 1.3.18</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -62,6 +77,21 @@ </section> +<section><title>Xmerl 1.3.16.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>The charset detection parsing crash in some cases when + the XML directive is not syntactic correct.</p> + <p> + Own Id: OTP-15492 Aux Id: ERIERL-283 </p> + </item> + </list> + </section> + +</section> + <section><title>Xmerl 1.3.16</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -1412,4 +1442,3 @@ </section> </section> </chapter> - diff --git a/lib/xmerl/src/xmerl_sax_parser.erl b/lib/xmerl/src/xmerl_sax_parser.erl index e383c4c349..fe836fd8cd 100644 --- a/lib/xmerl/src/xmerl_sax_parser.erl +++ b/lib/xmerl/src/xmerl_sax_parser.erl @@ -1,8 +1,8 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. +%% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. %% You may obtain a copy of the License at @@ -14,13 +14,13 @@ %% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. %% See the License for the specific language governing permissions and %% limitations under the License. -%% +%% %% %CopyrightEnd% %%---------------------------------------------------------------------- %% File : xmerl_sax_parser.erl %% Description : XML SAX parse API module. %% -%% Created : 4 Jun 2008 +%% Created : 4 Jun 2008 %%---------------------------------------------------------------------- -module(xmerl_sax_parser). @@ -72,9 +72,9 @@ file(Name,Options) -> CL = filename:absname(Dir), File = filename:basename(Name), ContinuationFun = fun default_continuation_cb/1, - Res = stream(<<>>, + Res = stream(<<>>, [{continuation_fun, ContinuationFun}, - {continuation_state, FD}, + {continuation_state, FD}, {current_location, CL}, {entity, File} |Options], @@ -101,39 +101,39 @@ stream(Xml, Options, InputType) when is_list(Xml), is_list(Options) -> State = parse_options(Options, initial_state()), case State#xmerl_sax_parser_state.file_type of dtd -> - xmerl_sax_parser_list:parse_dtd(Xml, + xmerl_sax_parser_list:parse_dtd(Xml, State#xmerl_sax_parser_state{encoding = list, input_type = InputType}); normal -> - xmerl_sax_parser_list:parse(Xml, + xmerl_sax_parser_list:parse(Xml, State#xmerl_sax_parser_state{encoding = list, input_type = InputType}) end; stream(Xml, Options, InputType) when is_binary(Xml), is_list(Options) -> - case parse_options(Options, initial_state()) of + case parse_options(Options, initial_state()) of {error, Reason} -> {error, Reason}; State -> - ParseFunction = + ParseFunction = case State#xmerl_sax_parser_state.file_type of dtd -> parse_dtd; normal -> parse end, - try + try {Xml1, State1} = detect_charset(Xml, State), parse_binary(Xml1, State1#xmerl_sax_parser_state{input_type = InputType}, ParseFunction) catch throw:{fatal_error, {State2, Reason}} -> - {fatal_error, + {fatal_error, { State2#xmerl_sax_parser_state.current_location, - State2#xmerl_sax_parser_state.entity, + State2#xmerl_sax_parser_state.entity, 1 }, - Reason, [], + Reason, [], State2#xmerl_sax_parser_state.event_state} end end. @@ -157,7 +157,7 @@ parse_binary(Xml, #xmerl_sax_parser_state{encoding={utf16,big}}=State, F) -> xmerl_sax_parser_utf16be:F(Xml, State); parse_binary(Xml, #xmerl_sax_parser_state{encoding=latin1}=State, F) -> xmerl_sax_parser_latin1:F(Xml, State); -parse_binary(_, #xmerl_sax_parser_state{encoding=Enc}, State) -> +parse_binary(_, #xmerl_sax_parser_state{encoding=Enc}, State) -> ?fatal_error(State, lists:flatten(io_lib:format("Charcter set ~p not supported", [Enc]))). %%---------------------------------------------------------------------- @@ -177,9 +177,9 @@ initial_state() -> %%---------------------------------------------------------------------- %% Function: parse_options(Options, State) %% Input: Options = [Option] -%% Option = {event_state, term()} | {event_fun, fun()} | +%% Option = {event_state, term()} | {event_fun, fun()} | %% {continuation_state, term()} | {continuation_fun, fun()} | -%% {encoding, Encoding} | {file_type, FT} +%% {encoding, Encoding} | {file_type, FT} %% FT = normal | dtd %% Encoding = utf8 | utf16le | utf16be | list | iso8859 %% State = #xmerl_sax_parser_state{} @@ -200,7 +200,7 @@ parse_options([{file_type, FT} |Options], State) when FT==normal; FT==dtd -> parse_options(Options, State#xmerl_sax_parser_state{file_type = FT}); parse_options([{encoding, E} |Options], State) -> case check_encoding_option(E) of - {error, Reason} -> + {error, Reason} -> {error, Reason}; Enc -> parse_options(Options, State#xmerl_sax_parser_state{encoding = Enc}) @@ -231,7 +231,7 @@ check_encoding_option(E) -> %% Description: Detects which character set is used in a binary stream. %%---------------------------------------------------------------------- detect_charset(<<>>, #xmerl_sax_parser_state{continuation_fun = undefined} = State) -> - ?fatal_error(State, "Can't detect character encoding due to lack of indata"); + ?fatal_error(State, "Can't detect character encoding due to lack of indata"); detect_charset(<<>>, State) -> cf(<<>>, State, fun detect_charset/2); detect_charset(Bytes, State) -> @@ -269,22 +269,14 @@ detect_charset_1(<<16#3C, 16#3F, 16#78, 16#6D>> = Xml, State) -> cf(Xml, State, fun detect_charset_1/2); detect_charset_1(<<16#3C, 16#3F, 16#78, 16#6D, 16#6C, Xml2/binary>>, State) -> {Xml3, State1} = read_until_end_of_xml_directive(Xml2, State), - case parse_xml_directive(Xml3) of - {error, Reason} -> - ?fatal_error(State, Reason); - AttrList -> - case lists:keysearch("encoding", 1, AttrList) of - {value, {_, E}} -> - case convert_encoding(E) of - {error, Reason} -> - ?fatal_error(State, Reason); - Enc -> - {<<16#3C, 16#3F, 16#78, 16#6D, 16#6C, Xml3/binary>>, - State1#xmerl_sax_parser_state{encoding=Enc}} - end; - _ -> - {<<16#3C, 16#3F, 16#78, 16#6D, 16#6C, Xml3/binary>>, State1} - end + AttrList = parse_xml_directive(Xml3, State), + case lists:keysearch("encoding", 1, AttrList) of + {value, {_, E}} -> + Enc = convert_encoding(E, State), + {<<16#3C, 16#3F, 16#78, 16#6D, 16#6C, Xml3/binary>>, + State1#xmerl_sax_parser_state{encoding=Enc}}; + _ -> + {<<16#3C, 16#3F, 16#78, 16#6D, 16#6C, Xml3/binary>>, State1} end; detect_charset_1(Xml, State) -> {Xml, State}. @@ -295,7 +287,7 @@ detect_charset_1(Xml, State) -> %% Output: utf8 | iso8859 %% Description: Converting 7,8 bit and utf8 encoding strings to internal format. %%---------------------------------------------------------------------- -convert_encoding(Enc) -> %% Just for 7,8 bit + utf8 +convert_encoding(Enc, State) -> %% Just for 7,8 bit + utf8 case string:to_lower(Enc) of "utf-8" -> utf8; "us-ascii" -> utf8; @@ -309,19 +301,19 @@ convert_encoding(Enc) -> %% Just for 7,8 bit + utf8 "iso-8859-7" -> latin1; "iso-8859-8" -> latin1; "iso-8859-9" -> latin1; - _ -> {error, "Unknown encoding: " ++ Enc} + _ -> ?fatal_error(State, "Unknown encoding: " ++ Enc) end. %%---------------------------------------------------------------------- %% Function: parse_xml_directive(Xml) %% Input: Xml = binary() %% Acc = list() -%% Output: +%% Output: %% Description: Parsing the xml declaration from the input stream. %%---------------------------------------------------------------------- -parse_xml_directive(<<C, Rest/binary>>) when ?is_whitespace(C) -> - parse_xml_directive_1(Rest, []). - +parse_xml_directive(<<C, Rest/binary>>, State) when ?is_whitespace(C) -> + parse_xml_directive_1(Rest, [], State). + %%---------------------------------------------------------------------- %% Function: parse_xml_directive_1(Xml, Acc) -> [{Name, Value}] %% Input: Xml = binary() @@ -331,20 +323,20 @@ parse_xml_directive(<<C, Rest/binary>>) when ?is_whitespace(C) -> %% Output: see above %% Description: Parsing the xml declaration from the input stream. %%---------------------------------------------------------------------- -parse_xml_directive_1(<<C, Rest/binary>>, Acc) when ?is_whitespace(C) -> - parse_xml_directive_1(Rest, Acc); -parse_xml_directive_1(<<"?>", _/binary>>, Acc) -> +parse_xml_directive_1(<<C, Rest/binary>>, Acc, State) when ?is_whitespace(C) -> + parse_xml_directive_1(Rest, Acc, State); +parse_xml_directive_1(<<"?>", _/binary>>, Acc, _State) -> Acc; -parse_xml_directive_1(<<C, Rest/binary>>, Acc) when 97 =< C, C =< 122 -> +parse_xml_directive_1(<<C, Rest/binary>>, Acc, State) when 97 =< C, C =< 122 -> {Name, Rest1} = parse_name(Rest, [C]), - Rest2 = parse_eq(Rest1), - {Value, Rest3} = parse_value(Rest2), - parse_xml_directive_1(Rest3, [{Name, Value} |Acc]); -parse_xml_directive_1(_, _) -> - {error, "Unknown attribute in xml directive"}. + Rest2 = parse_eq(Rest1, State), + {Value, Rest3} = parse_value(Rest2, State), + parse_xml_directive_1(Rest3, [{Name, Value} |Acc], State); +parse_xml_directive_1(_, _, State) -> + ?fatal_error(State, "Unknown attribute in xml directive"). %%---------------------------------------------------------------------- -%% Function: parse_xml_directive_1(Xml, Acc) -> Name +%% Function: parse_name(Xml, Acc) -> Name %% Input: Xml = binary() %% Acc = string() %% Output: Name = string() @@ -361,10 +353,12 @@ parse_name(Rest, Acc) -> %% Output: Rest = binary() %% Description: Reads an '=' from the stream. %%---------------------------------------------------------------------- -parse_eq(<<C, Rest/binary>>) when ?is_whitespace(C) -> - parse_eq(Rest); -parse_eq(<<"=", Rest/binary>>) -> - Rest. +parse_eq(<<C, Rest/binary>>, State) when ?is_whitespace(C) -> + parse_eq(Rest, State); +parse_eq(<<"=", Rest/binary>>, _State) -> + Rest; +parse_eq(_, State) -> + ?fatal_error(State, "expecting = or whitespace"). %%---------------------------------------------------------------------- %% Function: parse_value(Xml) -> {Value, Rest} @@ -373,10 +367,12 @@ parse_eq(<<"=", Rest/binary>>) -> %% Rest = binary() %% Description: Parsing an attribute value from the stream. %%---------------------------------------------------------------------- -parse_value(<<C, Rest/binary>>) when ?is_whitespace(C) -> - parse_value(Rest); -parse_value(<<C, Rest/binary>>) when C == $'; C == $" -> - parse_value_1(Rest, C, []). +parse_value(<<C, Rest/binary>>, State) when ?is_whitespace(C) -> + parse_value(Rest, State); +parse_value(<<C, Rest/binary>>, _State) when C == $'; C == $" -> + parse_value_1(Rest, C, []); +parse_value(_, State) -> + ?fatal_error(State, "\', \" or whitespace expected"). %%---------------------------------------------------------------------- %% Function: parse_value_1(Xml, Stop, Acc) -> {Value, Rest} @@ -431,7 +427,7 @@ read_until_end_of_xml_directive(Rest, State) -> nomatch -> case cf(Rest, State) of {<<>>, _} -> - ?fatal_error(State, "Can't detect character encoding due to lack of indata"); + ?fatal_error(State, "Can't detect character encoding due to lack of indata"); {NewBytes, NewState} -> read_until_end_of_xml_directive(NewBytes, NewState) end; @@ -450,9 +446,9 @@ read_until_end_of_xml_directive(Rest, State) -> %% input stream and calls the fun in NextCall. %%---------------------------------------------------------------------- cf(_Rest, #xmerl_sax_parser_state{continuation_fun = undefined} = State) -> - ?fatal_error(State, "Continuation function undefined"); + ?fatal_error(State, "Continuation function undefined"); cf(Rest, #xmerl_sax_parser_state{continuation_fun = CFun, continuation_state = CState} = State) -> - Result = + Result = try CFun(CState) catch @@ -463,9 +459,9 @@ cf(Rest, #xmerl_sax_parser_state{continuation_fun = CFun, continuation_state = C end, case Result of {<<>>, _} -> - ?fatal_error(State, "Can't detect character encoding due to lack of indata"); + ?fatal_error(State, "Can't detect character encoding due to lack of indata"); {NewBytes, NewContState} -> - {<<Rest/binary, NewBytes/binary>>, + {<<Rest/binary, NewBytes/binary>>, State#xmerl_sax_parser_state{continuation_state = NewContState}} end. @@ -479,10 +475,10 @@ cf(Rest, #xmerl_sax_parser_state{continuation_fun = CFun, continuation_state = C %% input stream and calls the fun in NextCall. %%---------------------------------------------------------------------- cf(_Rest, #xmerl_sax_parser_state{continuation_fun = undefined} = State, _) -> - ?fatal_error(State, "Continuation function undefined"); -cf(Rest, #xmerl_sax_parser_state{continuation_fun = CFun, continuation_state = CState} = State, + ?fatal_error(State, "Continuation function undefined"); +cf(Rest, #xmerl_sax_parser_state{continuation_fun = CFun, continuation_state = CState} = State, NextCall) -> - Result = + Result = try CFun(CState) catch @@ -493,8 +489,8 @@ cf(Rest, #xmerl_sax_parser_state{continuation_fun = CFun, continuation_state = C end, case Result of {<<>>, _} -> - ?fatal_error(State, "Can't detect character encoding due to lack of indata"); + ?fatal_error(State, "Can't detect character encoding due to lack of indata"); {NewBytes, NewContState} -> - NextCall(<<Rest/binary, NewBytes/binary>>, + NextCall(<<Rest/binary, NewBytes/binary>>, State#xmerl_sax_parser_state{continuation_state = NewContState}) end. diff --git a/lib/xmerl/vsn.mk b/lib/xmerl/vsn.mk index 3a266a56bd..b6486681c2 100644 --- a/lib/xmerl/vsn.mk +++ b/lib/xmerl/vsn.mk @@ -1 +1 @@ -XMERL_VSN = 1.3.18 +XMERL_VSN = 1.3.19 diff --git a/make/otp_patch_solve_forward_merge_version b/make/otp_patch_solve_forward_merge_version index 7ed6ff82de..7f8f011eb7 100644 --- a/make/otp_patch_solve_forward_merge_version +++ b/make/otp_patch_solve_forward_merge_version @@ -1 +1 @@ -5 +7 diff --git a/make/otp_version_tickets_in_merge b/make/otp_version_tickets_in_merge new file mode 100644 index 0000000000..32e6923ce9 --- /dev/null +++ b/make/otp_version_tickets_in_merge @@ -0,0 +1,2 @@ +OTP-14728 +OTP-15573 diff --git a/otp_versions.table b/otp_versions.table index 11586467c1..aea1e8d5cd 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -1,3 +1,7 @@ +OTP-21.2.6 : erts-10.2.4 stdlib-3.7.1 # asn1-5.0.8 common_test-1.16.1 compiler-7.3.1 crypto-4.4 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 et-1.6.4 eunit-2.3.7 ftp-1.0.1 hipe-3.18.2 inets-7.0.5 jinterface-1.9.1 kernel-6.2 megaco-3.18.4 mnesia-4.15.5 observer-2.8.2 odbc-2.12.2 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.4 reltool-0.7.8 runtime_tools-1.13.1 sasl-3.3 snmp-5.2.12 ssh-4.7.3 ssl-9.1.2 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.2 wx-1.8.6 xmerl-1.3.19 : +OTP-21.2.5 : inets-7.0.5 # asn1-5.0.8 common_test-1.16.1 compiler-7.3.1 crypto-4.4 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 erts-10.2.3 et-1.6.4 eunit-2.3.7 ftp-1.0.1 hipe-3.18.2 jinterface-1.9.1 kernel-6.2 megaco-3.18.4 mnesia-4.15.5 observer-2.8.2 odbc-2.12.2 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.4 reltool-0.7.8 runtime_tools-1.13.1 sasl-3.3 snmp-5.2.12 ssh-4.7.3 ssl-9.1.2 stdlib-3.7 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.2 wx-1.8.6 xmerl-1.3.19 : +OTP-21.2.4 : erts-10.2.3 inets-7.0.4 # asn1-5.0.8 common_test-1.16.1 compiler-7.3.1 crypto-4.4 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 et-1.6.4 eunit-2.3.7 ftp-1.0.1 hipe-3.18.2 jinterface-1.9.1 kernel-6.2 megaco-3.18.4 mnesia-4.15.5 observer-2.8.2 odbc-2.12.2 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.4 reltool-0.7.8 runtime_tools-1.13.1 sasl-3.3 snmp-5.2.12 ssh-4.7.3 ssl-9.1.2 stdlib-3.7 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.2 wx-1.8.6 xmerl-1.3.19 : +OTP-21.2.3 : compiler-7.3.1 erts-10.2.2 ssl-9.1.2 xmerl-1.3.19 # asn1-5.0.8 common_test-1.16.1 crypto-4.4 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 et-1.6.4 eunit-2.3.7 ftp-1.0.1 hipe-3.18.2 inets-7.0.3 jinterface-1.9.1 kernel-6.2 megaco-3.18.4 mnesia-4.15.5 observer-2.8.2 odbc-2.12.2 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.4 reltool-0.7.8 runtime_tools-1.13.1 sasl-3.3 snmp-5.2.12 ssh-4.7.3 stdlib-3.7 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.2 wx-1.8.6 : OTP-21.2.2 : ssh-4.7.3 # asn1-5.0.8 common_test-1.16.1 compiler-7.3 crypto-4.4 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 erts-10.2.1 et-1.6.4 eunit-2.3.7 ftp-1.0.1 hipe-3.18.2 inets-7.0.3 jinterface-1.9.1 kernel-6.2 megaco-3.18.4 mnesia-4.15.5 observer-2.8.2 odbc-2.12.2 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.4 reltool-0.7.8 runtime_tools-1.13.1 sasl-3.3 snmp-5.2.12 ssl-9.1.1 stdlib-3.7 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.2 wx-1.8.6 xmerl-1.3.18 : OTP-21.2.1 : erts-10.2.1 ssl-9.1.1 # asn1-5.0.8 common_test-1.16.1 compiler-7.3 crypto-4.4 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 et-1.6.4 eunit-2.3.7 ftp-1.0.1 hipe-3.18.2 inets-7.0.3 jinterface-1.9.1 kernel-6.2 megaco-3.18.4 mnesia-4.15.5 observer-2.8.2 odbc-2.12.2 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.4 reltool-0.7.8 runtime_tools-1.13.1 sasl-3.3 snmp-5.2.12 ssh-4.7.2 stdlib-3.7 syntax_tools-2.1.6 tftp-1.0.1 tools-3.0.2 wx-1.8.6 xmerl-1.3.18 : OTP-21.2 : asn1-5.0.8 compiler-7.3 crypto-4.4 erts-10.2 et-1.6.4 hipe-3.18.2 inets-7.0.3 kernel-6.2 observer-2.8.2 os_mon-2.4.7 public_key-1.6.4 reltool-0.7.8 sasl-3.3 ssh-4.7.2 ssl-9.1 stdlib-3.7 tools-3.0.2 wx-1.8.6 # common_test-1.16.1 debugger-4.2.6 dialyzer-3.3.1 diameter-2.1.6 edoc-0.9.4 eldap-1.2.6 erl_docgen-0.8.1 erl_interface-3.10.4 eunit-2.3.7 ftp-1.0.1 jinterface-1.9.1 megaco-3.18.4 mnesia-4.15.5 odbc-2.12.2 otp_mibs-1.2.1 parsetools-2.1.8 runtime_tools-1.13.1 snmp-5.2.12 syntax_tools-2.1.6 tftp-1.0.1 xmerl-1.3.18 : @@ -16,6 +20,11 @@ OTP-21.0.3 : erts-10.0.3 # asn1-5.0.6 common_test-1.16 compiler-7.2.2 crypto-4.3 OTP-21.0.2 : compiler-7.2.2 erts-10.0.2 public_key-1.6.1 stdlib-3.5.1 # asn1-5.0.6 common_test-1.16 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0.1 : compiler-7.2.1 erts-10.0.1 # asn1-5.0.6 common_test-1.16 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 stdlib-3.5 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : OTP-21.0 : asn1-5.0.6 common_test-1.16 compiler-7.2 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 erts-10.0 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 mnesia-4.15.4 observer-2.8 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 ssh-4.7 ssl-9.0 stdlib-3.5 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 # megaco-3.18.3 odbc-2.12.1 snmp-5.2.11 : +OTP-20.3.8.20 : common_test-1.15.4.1 # asn1-5.0.5.2 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4.1 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.9 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.3 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16.1 : +OTP-20.3.8.19 : diameter-2.1.4.1 erts-9.3.3.9 # asn1-5.0.5.2 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.3 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16.1 : +OTP-20.3.8.18 : erts-9.3.3.8 # asn1-5.0.5.2 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.3 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16.1 : +OTP-20.3.8.17 : xmerl-1.3.16.1 # asn1-5.0.5.2 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.7 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.3 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 : +OTP-20.3.8.16 : erts-9.3.3.7 ssh-4.6.9.3 # asn1-5.0.5.2 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.8.15 : asn1-5.0.5.2 # common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.6 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.2 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.8.14 : ssh-4.6.9.2 # asn1-5.0.5.1 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.6 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssl-8.2.6.4 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.8.13 : ssl-8.2.6.4 # asn1-5.0.5.1 common_test-1.15.4 compiler-7.1.5.2 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3.1 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.6 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.4 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3.2 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 stdlib-3.4.5.1 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : @@ -37,6 +46,7 @@ OTP-20.3.6 : crypto-4.2.2 ssh-4.6.9 # asn1-5.0.5 common_test-1.15.4 compiler-7.1 OTP-20.3.5 : erts-9.3.1 ssl-8.2.6 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.1 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.2 erl_interface-3.10.2 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4 inets-6.5.1 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.10 ssh-4.6.8 stdlib-3.4.5 syntax_tools-2.1.4 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.4 : erl_interface-3.10.2 ic-4.4.4 inets-6.5.1 ssh-4.6.8 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.1 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.2 erts-9.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.10 ssl-8.2.5 stdlib-3.4.5 syntax_tools-2.1.4 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.3 : sasl-3.1.2 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.1 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.2 erl_interface-3.10.1 erts-9.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.3 inets-6.5 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 snmp-5.2.10 ssh-4.6.7 ssl-8.2.5 stdlib-3.4.5 syntax_tools-2.1.4 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : +OTP-20.3.2.1 : common_test-1.15.4.0.1 # asn1-5.0.5 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.1 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.2 erl_interface-3.10.1 erts-9.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.3 inets-6.5 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.1 snmp-5.2.10 ssh-4.6.7 ssl-8.2.5 stdlib-3.4.5 syntax_tools-2.1.4 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.2 : ssh-4.6.7 stdlib-3.4.5 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.1 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.2 erl_interface-3.10.1 erts-9.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.3 inets-6.5 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.1 snmp-5.2.10 ssl-8.2.5 syntax_tools-2.1.4 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.1 : ssl-8.2.5 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.1 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.2 erl_interface-3.10.1 erts-9.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.3 inets-6.5 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.1 snmp-5.2.10 ssh-4.6.6 stdlib-3.4.4 syntax_tools-2.1.4 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3 : asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 crypto-4.2.1 dialyzer-3.2.4 diameter-2.1.4 erts-9.3 hipe-3.17.1 inets-6.5 kernel-5.4.3 observer-2.7 runtime_tools-1.12.5 snmp-5.2.10 ssh-4.6.6 ssl-8.2.4 stdlib-3.4.4 tools-2.11.2 # cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 debugger-4.2.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.2 erl_interface-3.10.1 et-1.6.1 eunit-2.3.5 ic-4.4.3 jinterface-1.8.1 megaco-3.18.3 mnesia-4.15.3 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 sasl-3.1.1 syntax_tools-2.1.4 wx-1.8.3 xmerl-1.3.16 : @@ -61,6 +71,7 @@ OTP-20.0.3 : asn1-5.0.2 compiler-7.1.1 erts-9.0.3 ssh-4.5.1 # common_test-1.15.1 OTP-20.0.2 : asn1-5.0.1 erts-9.0.2 kernel-5.3.1 # common_test-1.15.1 compiler-7.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.2 cosTime-1.2.2 cosTransactions-1.3.2 crypto-4.0 debugger-4.2.2 dialyzer-3.2 diameter-2.0 edoc-0.9 eldap-1.2.2 erl_docgen-0.7 erl_interface-3.10 et-1.6 eunit-2.3.3 hipe-3.16 ic-4.4.2 inets-6.4 jinterface-1.8 megaco-3.18.2 mnesia-4.15 observer-2.4 odbc-2.12 orber-3.8.3 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.5 public_key-1.4.1 reltool-0.7.4 runtime_tools-1.12.1 sasl-3.0.4 snmp-5.2.6 ssh-4.5 ssl-8.2 stdlib-3.4.1 syntax_tools-2.1.2 tools-2.10.1 wx-1.8.1 xmerl-1.3.15 : OTP-20.0.1 : common_test-1.15.1 erts-9.0.1 runtime_tools-1.12.1 stdlib-3.4.1 tools-2.10.1 # asn1-5.0 compiler-7.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.2 cosTime-1.2.2 cosTransactions-1.3.2 crypto-4.0 debugger-4.2.2 dialyzer-3.2 diameter-2.0 edoc-0.9 eldap-1.2.2 erl_docgen-0.7 erl_interface-3.10 et-1.6 eunit-2.3.3 hipe-3.16 ic-4.4.2 inets-6.4 jinterface-1.8 kernel-5.3 megaco-3.18.2 mnesia-4.15 observer-2.4 odbc-2.12 orber-3.8.3 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.5 public_key-1.4.1 reltool-0.7.4 sasl-3.0.4 snmp-5.2.6 ssh-4.5 ssl-8.2 syntax_tools-2.1.2 wx-1.8.1 xmerl-1.3.15 : OTP-20.0 : asn1-5.0 common_test-1.15 compiler-7.1 cosProperty-1.2.2 crypto-4.0 debugger-4.2.2 dialyzer-3.2 diameter-2.0 edoc-0.9 erl_docgen-0.7 erl_interface-3.10 erts-9.0 eunit-2.3.3 hipe-3.16 inets-6.4 jinterface-1.8 kernel-5.3 megaco-3.18.2 mnesia-4.15 observer-2.4 orber-3.8.3 parsetools-2.1.5 public_key-1.4.1 reltool-0.7.4 runtime_tools-1.12 sasl-3.0.4 snmp-5.2.6 ssh-4.5 ssl-8.2 stdlib-3.4 syntax_tools-2.1.2 tools-2.10 wx-1.8.1 xmerl-1.3.15 # cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosTime-1.2.2 cosTransactions-1.3.2 eldap-1.2.2 et-1.6 ic-4.4.2 odbc-2.12 os_mon-2.4.2 otp_mibs-1.1.1 : +OTP-19.3.6.13 : erts-8.3.5.7 # asn1-4.0.4 common_test-1.14 compiler-7.0.4.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7.4 debugger-4.2.1 dialyzer-3.1.1 diameter-1.12.2 edoc-0.8.1 eldap-1.2.2.1 erl_docgen-0.6.1 erl_interface-3.9.3 et-1.6 eunit-2.3.2 gs-1.6.2 hipe-3.15.4 ic-4.4.2 inets-6.3.9 jinterface-1.7.1 kernel-5.2.0.1 megaco-3.18.1 mnesia-4.14.3.1 observer-2.3.1 odbc-2.12 orber-3.8.2 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.4 percept-0.9 public_key-1.4 reltool-0.7.3 runtime_tools-1.11.1 sasl-3.0.3 snmp-5.2.5 ssh-4.4.2.4 ssl-8.1.3.1.1 stdlib-3.3 syntax_tools-2.1.1.1 tools-2.9.1 typer-0.9.12 wx-1.8 xmerl-1.3.14 : OTP-19.3.6.12 : eldap-1.2.2.1 # asn1-4.0.4 common_test-1.14 compiler-7.0.4.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7.4 debugger-4.2.1 dialyzer-3.1.1 diameter-1.12.2 edoc-0.8.1 erl_docgen-0.6.1 erl_interface-3.9.3 erts-8.3.5.6 et-1.6 eunit-2.3.2 gs-1.6.2 hipe-3.15.4 ic-4.4.2 inets-6.3.9 jinterface-1.7.1 kernel-5.2.0.1 megaco-3.18.1 mnesia-4.14.3.1 observer-2.3.1 odbc-2.12 orber-3.8.2 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.4 percept-0.9 public_key-1.4 reltool-0.7.3 runtime_tools-1.11.1 sasl-3.0.3 snmp-5.2.5 ssh-4.4.2.4 ssl-8.1.3.1.1 stdlib-3.3 syntax_tools-2.1.1.1 tools-2.9.1 typer-0.9.12 wx-1.8 xmerl-1.3.14 : OTP-19.3.6.11 : erts-8.3.5.6 # asn1-4.0.4 common_test-1.14 compiler-7.0.4.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7.4 debugger-4.2.1 dialyzer-3.1.1 diameter-1.12.2 edoc-0.8.1 eldap-1.2.2 erl_docgen-0.6.1 erl_interface-3.9.3 et-1.6 eunit-2.3.2 gs-1.6.2 hipe-3.15.4 ic-4.4.2 inets-6.3.9 jinterface-1.7.1 kernel-5.2.0.1 megaco-3.18.1 mnesia-4.14.3.1 observer-2.3.1 odbc-2.12 orber-3.8.2 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.4 percept-0.9 public_key-1.4 reltool-0.7.3 runtime_tools-1.11.1 sasl-3.0.3 snmp-5.2.5 ssh-4.4.2.4 ssl-8.1.3.1.1 stdlib-3.3 syntax_tools-2.1.1.1 tools-2.9.1 typer-0.9.12 wx-1.8 xmerl-1.3.14 : OTP-19.3.6.10 : erts-8.3.5.5 syntax_tools-2.1.1.1 # asn1-4.0.4 common_test-1.14 compiler-7.0.4.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7.4 debugger-4.2.1 dialyzer-3.1.1 diameter-1.12.2 edoc-0.8.1 eldap-1.2.2 erl_docgen-0.6.1 erl_interface-3.9.3 et-1.6 eunit-2.3.2 gs-1.6.2 hipe-3.15.4 ic-4.4.2 inets-6.3.9 jinterface-1.7.1 kernel-5.2.0.1 megaco-3.18.1 mnesia-4.14.3.1 observer-2.3.1 odbc-2.12 orber-3.8.2 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.4 percept-0.9 public_key-1.4 reltool-0.7.3 runtime_tools-1.11.1 sasl-3.0.3 snmp-5.2.5 ssh-4.4.2.4 ssl-8.1.3.1.1 stdlib-3.3 tools-2.9.1 typer-0.9.12 wx-1.8 xmerl-1.3.14 : |