diff options
66 files changed, 1415 insertions, 529 deletions
diff --git a/bootstrap/lib/kernel/ebin/error_logger.beam b/bootstrap/lib/kernel/ebin/error_logger.beam Binary files differindex 83692bd7a9..0de0bc3ab3 100644 --- a/bootstrap/lib/kernel/ebin/error_logger.beam +++ b/bootstrap/lib/kernel/ebin/error_logger.beam diff --git a/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam b/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam Binary files differindex 901e741f4b..63e350df89 100644 --- a/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam +++ b/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam diff --git a/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam b/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam Binary files differindex deb851fde3..77a5216e39 100644 --- a/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam +++ b/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_server.beam b/bootstrap/lib/stdlib/ebin/gen_server.beam Binary files differindex c0908e1680..cc30bcbbb8 100644 --- a/bootstrap/lib/stdlib/ebin/gen_server.beam +++ b/bootstrap/lib/stdlib/ebin/gen_server.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_statem.beam b/bootstrap/lib/stdlib/ebin/gen_statem.beam Binary files differindex 2512b7be49..847042c84a 100644 --- a/bootstrap/lib/stdlib/ebin/gen_statem.beam +++ b/bootstrap/lib/stdlib/ebin/gen_statem.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib.beam b/bootstrap/lib/stdlib/ebin/io_lib.beam Binary files differindex a86d1356b8..be6c16582a 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/proc_lib.beam b/bootstrap/lib/stdlib/ebin/proc_lib.beam Binary files differindex 4658c863c1..934c3d96a1 100644 --- a/bootstrap/lib/stdlib/ebin/proc_lib.beam +++ b/bootstrap/lib/stdlib/ebin/proc_lib.beam diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index e1aa5ce76e..8b152b83f5 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -379,6 +379,16 @@ <c><![CDATA[Host]]></c> is the fully qualified host name of the current host. For short names, use flag <c><![CDATA[-sname]]></c> instead.</p> + <warning> + <p> + Starting a distributed node without also specifying + <seealso marker="#proto_dist"><c>-proto_dist inet_tls</c></seealso> + will expose the node to attacks that may give the attacker + complete access to the node and in extension the cluster. + When using un-secure distributed nodes, make sure that the + network is configured to keep potential attackers out. + </p> + </warning> </item> <tag><c><![CDATA[-noinput]]></c></tag> <item> @@ -428,12 +438,17 @@ </item> <tag><c><![CDATA[-proto_dist Proto]]></c></tag> <item> + <marker id="proto_dist"/> <p>Specifies a protocol for Erlang distribution:</p> <taglist> <tag><c>inet_tcp</c></tag> <item>TCP over IPv4 (the default)</item> <tag><c>inet_tls</c></tag> - <item>Distribution over TLS/SSL</item> + <item>Distribution over TLS/SSL, See the + <seealso marker="ssl:ssl_distribution"> + Using SSL for Erlang Distribution</seealso> User's Guide + for details on how to setup a secure distributed node. + </item> <tag><c>inet6_tcp</c></tag> <item>TCP over IPv6</item> </taglist> @@ -497,6 +512,16 @@ exist between nodes running with flag <c><![CDATA[-sname]]></c> and those running with flag <c><![CDATA[-name]]></c>, as node names must be unique in distributed Erlang systems.</p> + <warning> + <p> + Starting a distributed node without also specifying + <seealso marker="#proto_dist"><c>-proto_dist inet_tls</c></seealso> + will expose the node to attacks that may give the attacker + complete access to the node and in extension the cluster. + When using un-secure distributed nodes, make sure that the + network is configured to keep potential attackers out. + </p> + </warning> </item> <tag><marker id="start_epmd"/><c>-start_epmd true | false</c></tag> <item> diff --git a/erts/doc/src/erl_dist_protocol.xml b/erts/doc/src/erl_dist_protocol.xml index ee74983730..8391408a2e 100644 --- a/erts/doc/src/erl_dist_protocol.xml +++ b/erts/doc/src/erl_dist_protocol.xml @@ -70,6 +70,17 @@ <p>The integers in all multibyte fields are in big-endian order.</p> + <warning> + <p> + The Erlang Distribution protocol is not by itself secure and does not + aim to be so. In order to get secure distribution the distributed nodes + should be configured to use distribution over tls. + See the <seealso marker="ssl:ssl_distribution"> + Using SSL for Erlang Distribution</seealso> User's Guide + for details on how to setup a secure distributed node. + </p> + </warning> + <section> <title>EPMD Protocol</title> <p>The requests served by the EPMD are summarized in the following diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml index 51ba70994a..3eb3e04f33 100644 --- a/erts/doc/src/erl_nif.xml +++ b/erts/doc/src/erl_nif.xml @@ -714,6 +714,7 @@ typedef struct { typedef struct { ErlNifResourceDtor* dtor; ErlNifResourceStop* stop; + ErlNifResourceDown* down; } ErlNifResourceTypeInit;</code> <p>Initialization structure read by <seealso marker="#enif_open_resource_type_x"> enif_open_resource_type_x</seealso>.</p> @@ -1395,13 +1396,12 @@ typedef enum { <p>Returns <c>true</c> if a pending exception is associated with the environment <c>env</c>. If <c>reason</c> is a <c>NULL</c> pointer, ignore it. Otherwise, if a pending exception associated with - <c>env</c> exists, set <c>ERL_NIF_TERM</c> to which <c>reason</c> - points to the value of the exception's term. For example, if - <seealso marker="#enif_make_badarg"> + <c>env</c> exists, set <c>*reason</c> to the value of the exception + term. For example, if <seealso marker="#enif_make_badarg"> <c>enif_make_badarg</c></seealso> is called to set a pending <c>badarg</c> exception, a later call to <c>enif_has_pending_exception(env, &reason)</c> sets - <c>reason</c> to the atom <c>badarg</c>, then return <c>true</c>.</p> + <c>*reason</c> to the atom <c>badarg</c>, then return <c>true</c>.</p> <p>See also <seealso marker="#enif_make_badarg"> <c>enif_make_badarg</c></seealso> and <seealso marker="#enif_raise_exception"> @@ -2384,20 +2384,23 @@ enif_map_iterator_destroy(env, &iter);</code> called in the two callbacks <seealso marker="#load"><c>load</c></seealso> and <seealso marker="#upgrade"><c>upgrade</c></seealso>.</p> + <p>See also <seealso marker="#enif_open_resource_type_x"> + <c>enif_open_resource_type_x</c></seealso>.</p> </desc> </func> <func> <name><ret>ErlNifResourceType *</ret> <nametext>enif_open_resource_type_x(ErlNifEnv* env, const char* name, - ErlNifResourceTypeInit* init, + const ErlNifResourceTypeInit* init, ErlNifResourceFlags flags, ErlNifResourceFlags* tried)</nametext> </name> <fsummary>Create or takeover a resource type.</fsummary> <desc> <p>Same as <seealso marker="#enif_open_resource_type"><c>enif_open_resource_type</c></seealso> - except is also accept a <c>stop</c> callback for resource types that are - used together with <seealso marker="#enif_select"><c>enif_select</c></seealso>.</p> + except it accepts additional callback functions for resource types that are + used together with <seealso marker="#enif_select"><c>enif_select</c></seealso> + and <seealso marker="#enif_monitor_process"><c>enif_monitor_process</c></seealso>.</p> <p>Argument <c>init</c> is a pointer to an <seealso marker="#ErlNifResourceTypeInit"><c>ErlNifResourceTypeInit</c></seealso> structure that contains the function pointers for destructor, down and stop callbacks @@ -2695,6 +2698,21 @@ enif_map_iterator_destroy(env, &iter);</code> the event object. This safe way of closing event objects must be used even if all notifications have been received and no further calls to <c>enif_select</c> have been made.</p> + <p>The first call to <c>enif_select</c> for a specific OS <c>event</c> will establish + a relation between the event object and the containing resource. All subsequent calls + for an <c>event</c> must pass its containing resource as argument + <c>obj</c>. The relation is dissolved when <c>enif_select</c> has + been called with <c>mode</c> as <c>ERL_NIF_SELECT_STOP</c> and the + corresponding <c>stop</c> callback has returned. A resource can contain + several event objects but one event object can only be contained within + one resource. A resource will not be destructed until all its contained relations + have been dissolved.</p> + <note> + <p>Use <seealso marker="#enif_monitor_process"><c>enif_monitor_process</c></seealso> + together with <c>enif_select</c> to detect failing Erlang + processes and prevent them from causing permanent leakage of resources + and their contained OS event objects.</p> + </note> <p>Returns a non-negative value on success where the following bits can be set:</p> <taglist> <tag><c>ERL_NIF_SELECT_STOP_CALLED</c></tag> diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 6d165e9eff..d9cc5ef936 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -6148,28 +6148,60 @@ true</pre> <fsummary>Information about active processes and ports.</fsummary> <desc> <marker id="statistics_active_tasks"></marker> + <p>Returns the same as + <seealso marker="#statistics_active_tasks_all"> + <c>statistics(active_tasks_all)</c></seealso> + with the exception that no information about the dirty + IO run queue and its associated schedulers is part of + the result. That is, only tasks that are expected to be + CPU bound are part of the result.</p> + </desc> + </func> + + <func> + <name name="statistics" arity="1" clause_i="2"/> + <fsummary>Information about active processes and ports.</fsummary> + <desc> + <marker id="statistics_active_tasks_all"></marker> <p>Returns a list where each element represents the amount of active processes and ports on each run queue and its - associated scheduler. That is, the number of processes and - ports that are ready to run, or are currently running. The - element location in the list corresponds to the scheduler - and its run queue. The first element corresponds to scheduler - number 1 and so on. The information is <em>not</em> gathered - atomically. That is, the result is not necessarily a - consistent snapshot of the state, but instead quite - efficiently gathered.</p> + associated schedulers. That is, the number of processes and + ports that are ready to run, or are currently running. + Values for normal run queues and their associated schedulers + are located first in the resulting list. The first element + corresponds to scheduler number 1 and so on. If support for + dirty schedulers exist, an element with the value for the + dirty CPU run queue and its associated dirty CPU schedulers + follow and then as last element the value for the the dirty + IO run queue and its associated dirty IO schedulers follow. + The information is <em>not</em> gathered atomically. That is, + the result is not necessarily a consistent snapshot of the + state, but instead quite efficiently gathered.</p> + <note><p>Each normal scheduler has one run queue that it + manages. If dirty schedulers schedulers are supported, all + dirty CPU schedulers share one run queue, and all dirty IO + schedulers share one run queue. That is, we have multiple + normal run queues, one dirty CPU run queue and one dirty + IO run queue. Work can <em>not</em> migrate between the + different types of run queues. Only work in normal run + queues can migrate to other normal run queues. This has + to be taken into account when evaluating the result.</p></note> <p>See also <seealso marker="#statistics_total_active_tasks"> <c>statistics(total_active_tasks)</c></seealso>, <seealso marker="#statistics_run_queue_lengths"> - <c>statistics(run_queue_lengths)</c></seealso>, and + <c>statistics(run_queue_lengths)</c></seealso>, + <seealso marker="#statistics_run_queue_lengths_all"> + <c>statistics(run_queue_lengths_all)</c></seealso>, <seealso marker="#statistics_total_run_queue_lengths"> - <c>statistics(total_run_queue_lengths)</c></seealso>.</p> + <c>statistics(total_run_queue_lengths)</c></seealso>, and + <seealso marker="#statistics_total_run_queue_lengths_all"> + <c>statistics(total_run_queue_lengths_all)</c></seealso>.</p> </desc> </func> <func> - <name name="statistics" arity="1" clause_i="2"/> + <name name="statistics" arity="1" clause_i="3"/> <fsummary>Information about context switches.</fsummary> <desc> <p>Returns the total number of context switches since the @@ -6178,7 +6210,7 @@ true</pre> </func> <func> - <name name="statistics" arity="1" clause_i="3"/> + <name name="statistics" arity="1" clause_i="4"/> <fsummary>Information about exact reductions.</fsummary> <desc> <marker id="statistics_exact_reductions"></marker> @@ -6194,7 +6226,7 @@ true</pre> </func> <func> - <name name="statistics" arity="1" clause_i="4"/> + <name name="statistics" arity="1" clause_i="5"/> <fsummary>Information about garbage collection.</fsummary> <desc> <p>Returns information about garbage collection, for example:</p> @@ -6206,7 +6238,7 @@ true</pre> </func> <func> - <name name="statistics" arity="1" clause_i="5"/> + <name name="statistics" arity="1" clause_i="6"/> <fsummary>Information about I/O.</fsummary> <desc> <p>Returns <c><anno>Input</anno></c>, @@ -6217,7 +6249,7 @@ true</pre> </func> <func> - <name name="statistics" arity="1" clause_i="6"/> + <name name="statistics" arity="1" clause_i="7"/> <fsummary>Information about microstate accounting.</fsummary> <desc> <marker id="statistics_microstate_accounting"></marker> @@ -6353,7 +6385,7 @@ lists:map( </func> <func> - <name name="statistics" arity="1" clause_i="7"/> + <name name="statistics" arity="1" clause_i="8"/> <fsummary>Information about reductions.</fsummary> <desc> <marker id="statistics_reductions"></marker> @@ -6372,12 +6404,13 @@ lists:map( </func> <func> - <name name="statistics" arity="1" clause_i="8"/> + <name name="statistics" arity="1" clause_i="9"/> <fsummary>Information about the run-queues.</fsummary> <desc><marker id="statistics_run_queue"></marker> - <p>Returns the total length of the run-queues. That is, the number + <p>Returns the total length of all normal run-queues. That is, the number of processes and ports that are ready to run on all available - run-queues. The information is gathered atomically. That + normal run-queues. Dirty run queues are not part of the + result. The information is gathered atomically. That is, the result is a consistent snapshot of the state, but this operation is much more expensive compared to <seealso marker="#statistics_total_run_queue_lengths"> @@ -6387,29 +6420,63 @@ lists:map( </func> <func> - <name name="statistics" arity="1" clause_i="9"/> + <name name="statistics" arity="1" clause_i="10"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc><marker id="statistics_run_queue_lengths"></marker> + <p>Returns the same as + <seealso marker="#statistics_run_queue_lengths_all"> + <c>statistics(run_queue_lengths_all)</c></seealso> + with the exception that no information about the dirty + IO run queue is part of the result. That is, only + run queues with work that is expected to be CPU bound + is part of the result.</p> + </desc> + </func> + + <func> + <name name="statistics" arity="1" clause_i="11"/> + <fsummary>Information about the run-queue lengths.</fsummary> + <desc><marker id="statistics_run_queue_lengths_all"></marker> <p>Returns a list where each element represents the amount - of processes and ports ready to run for each run queue. The - element location in the list corresponds to the run queue - of a scheduler. The first element corresponds to the run - queue of scheduler number 1 and so on. The information is - <em>not</em> gathered atomically. That is, the result is - not necessarily a consistent snapshot of the state, but - instead quite efficiently gathered.</p> + of processes and ports ready to run for each run queue. + Values for normal run queues are located first in the + resulting list. The first element corresponds to the + normal run queue of scheduler number 1 and so on. If + support for dirty schedulers exist, values for the dirty + CPU run queue and the dirty IO run queue follow (in that + order) at the end. The information is <em>not</em> + gathered atomically. That is, the result is not + necessarily a consistent snapshot of the state, but + instead quite efficiently gathered.</p> + <note><p>Each normal scheduler has one run queue that it + manages. If dirty schedulers schedulers are supported, all + dirty CPU schedulers share one run queue, and all dirty IO + schedulers share one run queue. That is, we have multiple + normal run queues, one dirty CPU run queue and one dirty + IO run queue. Work can <em>not</em> migrate between the + different types of run queues. Only work in normal run + queues can migrate to other normal run queues. This has + to be taken into account when evaluating the result.</p></note> <p>See also + <seealso marker="#statistics_run_queue_lengths"> + <c>statistics(run_queue_lengths)</c></seealso>, + <seealso marker="#statistics_total_run_queue_lengths_all"> + <c>statistics(total_run_queue_lengths_all)</c></seealso>, <seealso marker="#statistics_total_run_queue_lengths"> <c>statistics(total_run_queue_lengths)</c></seealso>, <seealso marker="#statistics_active_tasks"> - <c>statistics(active_tasks)</c></seealso>, and + <c>statistics(active_tasks)</c></seealso>, + <seealso marker="#statistics_active_tasks_all"> + <c>statistics(active_tasks_all)</c></seealso>, and <seealso marker="#statistics_total_active_tasks"> - <c>statistics(total_active_tasks)</c></seealso>.</p> + <c>statistics(total_active_tasks)</c></seealso>, + <seealso marker="#statistics_total_active_tasks_all"> + <c>statistics(total_active_tasks_all)</c></seealso>.</p> </desc> </func> <func> - <name name="statistics" arity="1" clause_i="10"/> + <name name="statistics" arity="1" clause_i="12"/> <fsummary>Information about runtime.</fsummary> <desc> <p>Returns information about runtime, in milliseconds.</p> @@ -6424,7 +6491,7 @@ lists:map( </func> <func> - <name name="statistics" arity="1" clause_i="11"/> + <name name="statistics" arity="1" clause_i="13"/> <fsummary>Information about each schedulers work time.</fsummary> <desc> <marker id="statistics_scheduler_wall_time"></marker> @@ -6545,7 +6612,7 @@ ok </func> <func> - <name name="statistics" arity="1" clause_i="12"/> + <name name="statistics" arity="1" clause_i="14"/> <fsummary>Information about each schedulers work time.</fsummary> <desc> <marker id="statistics_scheduler_wall_time_all"></marker> @@ -6570,47 +6637,47 @@ ok </desc> </func> <func> - <name name="statistics" arity="1" clause_i="13"/> + <name name="statistics" arity="1" clause_i="15"/> <fsummary>Information about active processes and ports.</fsummary> <desc><marker id="statistics_total_active_tasks"></marker> - <p>Returns the total amount of active processes and ports in - the system. That is, the number of processes and ports that - are ready to run, or are currently running. The information - is <em>not</em> gathered atomically. That is, the result - is not necessarily a consistent snapshot of the state, but - instead quite efficiently gathered.</p> - <p>See also - <seealso marker="#statistics_active_tasks"> - <c>statistics(active_tasks)</c></seealso>, - <seealso marker="#statistics_run_queue_lengths"> - <c>statistics(run_queue_lengths)</c></seealso>, and - <seealso marker="#statistics_total_run_queue_lengths"> - <c>statistics(total_run_queue_lengths)</c></seealso>.</p> + <p>The same as calling + <c>lists:sum(</c><seealso marker="#statistics_active_tasks"><c>statistics(active_tasks)</c></seealso><c>)</c>, + but more efficient.</p> </desc> </func> <func> - <name name="statistics" arity="1" clause_i="14"/> + <name name="statistics" arity="1" clause_i="16"/> + <fsummary>Information about active processes and ports.</fsummary> + <desc><marker id="statistics_total_active_tasks_all"></marker> + <p>The same as calling + <c>lists:sum(</c><seealso marker="#statistics_active_tasks_all"><c>statistics(active_tasks_all)</c></seealso><c>)</c>, + but more efficient.</p> + </desc> + </func> + + <func> + <name name="statistics" arity="1" clause_i="17"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc><marker id="statistics_total_run_queue_lengths"></marker> - <p>Returns the total length of the run queues. That is, the number - of processes and ports that are ready to run on all available - run queues. The information is <em>not</em> gathered atomically. - That is, the result is not necessarily a consistent snapshot of - the state, but much more efficiently gathered compared to - <seealso marker="#statistics_run_queue"> - <c>statistics(run_queue)</c></seealso>.</p> - <p>See also <seealso marker="#statistics_run_queue_lengths"> - <c>statistics(run_queue_lengths)</c></seealso>, - <seealso marker="#statistics_total_active_tasks"> - <c>statistics(total_active_tasks)</c></seealso>, and - <seealso marker="#statistics_active_tasks"> - <c>statistics(active_tasks)</c></seealso>.</p> + <p>The same as calling + <c>lists:sum(</c><seealso marker="#statistics_run_queue_lengths"><c>statistics(run_queue_lengths)</c></seealso><c>)</c>, + but more efficient.</p> </desc> </func> <func> - <name name="statistics" arity="1" clause_i="15"/> + <name name="statistics" arity="1" clause_i="18"/> + <fsummary>Information about the run-queue lengths.</fsummary> + <desc><marker id="statistics_total_run_queue_lengths_all"></marker> + <p>The same as calling + <c>lists:sum(</c><seealso marker="#statistics_run_queue_lengths_all"><c>statistics(run_queue_lengths_all)</c></seealso><c>)</c>, + but more efficient.</p> + </desc> + </func> + + <func> + <name name="statistics" arity="1" clause_i="19"/> <fsummary>Information about wall clock.</fsummary> <desc> <p>Returns information about wall clock. <c>wall_clock</c> can diff --git a/erts/emulator/beam/atom.names b/erts/emulator/beam/atom.names index 477a7676d6..a44d23b181 100644 --- a/erts/emulator/beam/atom.names +++ b/erts/emulator/beam/atom.names @@ -76,6 +76,7 @@ atom ac atom accessor atom active atom active_tasks +atom active_tasks_all atom alive atom all atom all_but_first @@ -567,6 +568,7 @@ atom return_to atom return_trace atom run_queue atom run_queue_lengths +atom run_queue_lengths_all atom runnable atom runnable_ports atom runnable_procs @@ -656,8 +658,10 @@ atom Times='*' atom timestamp atom total atom total_active_tasks +atom total_active_tasks_all atom total_heap_size atom total_run_queue_lengths +atom total_run_queue_lengths_all atom tpkt atom trace trace_ts traced atom trace_control_word diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 5fc70dfc02..e2773475b0 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -3450,9 +3450,15 @@ BIF_RETTYPE statistics_1(BIF_ALIST_1) if (is_non_value(res)) BIF_RET(am_undefined); BIF_TRAP1(gather_sched_wall_time_res_trap, BIF_P, res); - } else if (BIF_ARG_1 == am_total_active_tasks - || BIF_ARG_1 == am_total_run_queue_lengths) { - Uint no = erts_run_queues_len(NULL, 0, BIF_ARG_1 == am_total_active_tasks); + } else if ((BIF_ARG_1 == am_total_active_tasks) + | (BIF_ARG_1 == am_total_run_queue_lengths) + | (BIF_ARG_1 == am_total_active_tasks_all) + | (BIF_ARG_1 == am_total_run_queue_lengths_all)) { + Uint no = erts_run_queues_len(NULL, 0, + ((BIF_ARG_1 == am_total_active_tasks) + | (BIF_ARG_1 == am_total_active_tasks_all)), + ((BIF_ARG_1 == am_total_active_tasks_all) + | (BIF_ARG_1 == am_total_run_queue_lengths_all))); if (IS_USMALL(0, no)) res = make_small(no); else { @@ -3460,13 +3466,21 @@ BIF_RETTYPE statistics_1(BIF_ALIST_1) res = uint_to_big(no, hp); } BIF_RET(res); - } else if (BIF_ARG_1 == am_active_tasks - || BIF_ARG_1 == am_run_queue_lengths) { + } else if ((BIF_ARG_1 == am_active_tasks) + | (BIF_ARG_1 == am_run_queue_lengths) + | (BIF_ARG_1 == am_active_tasks_all) + | (BIF_ARG_1 == am_run_queue_lengths_all)) { Eterm res, *hp, **hpp; Uint sz, *szp; - int no_qs = erts_no_run_queues; + int incl_dirty_io = ((BIF_ARG_1 == am_active_tasks_all) + | (BIF_ARG_1 == am_run_queue_lengths_all)); + int no_qs = (erts_no_run_queues + ERTS_NUM_DIRTY_CPU_RUNQS + + (incl_dirty_io ? ERTS_NUM_DIRTY_IO_RUNQS : 0)); Uint *qszs = erts_alloc(ERTS_ALC_T_TMP,sizeof(Uint)*no_qs*2); - (void) erts_run_queues_len(qszs, 0, BIF_ARG_1 == am_active_tasks); + (void) erts_run_queues_len(qszs, 0, + ((BIF_ARG_1 == am_active_tasks) + | (BIF_ARG_1 == am_active_tasks_all)), + incl_dirty_io); sz = 0; szp = &sz; hpp = NULL; @@ -3539,7 +3553,7 @@ BIF_RETTYPE statistics_1(BIF_ALIST_1) res = TUPLE2(hp, b1, b2); BIF_RET(res); } else if (BIF_ARG_1 == am_run_queue) { - res = erts_run_queues_len(NULL, 1, 0); + res = erts_run_queues_len(NULL, 1, 0, 0); BIF_RET(make_small(res)); } else if (BIF_ARG_1 == am_wall_clock) { UWord w1, w2; @@ -3557,9 +3571,9 @@ BIF_RETTYPE statistics_1(BIF_ALIST_1) else if (ERTS_IS_ATOM_STR("run_queues", BIF_ARG_1)) { Eterm res, *hp, **hpp; Uint sz, *szp; - int no_qs = erts_no_run_queues; + int no_qs = erts_no_run_queues + ERTS_NUM_DIRTY_RUNQS; Uint *qszs = erts_alloc(ERTS_ALC_T_TMP,sizeof(Uint)*no_qs*2); - (void) erts_run_queues_len(qszs, 0, 0); + (void) erts_run_queues_len(qszs, 0, 0, 1); sz = 0; szp = &sz; hpp = NULL; diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index da27c7e7c6..7952e3031d 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -405,13 +405,59 @@ static erts_atomic_t runq_supervisor_sleeping; ErtsSchedulerData *erts_scheduler_data; #endif -ErtsAlignedRunQueue *erts_aligned_run_queues; -Uint erts_no_run_queues; +ErtsAlignedRunQueue * ERTS_WRITE_UNLIKELY(erts_aligned_run_queues); +Uint ERTS_WRITE_UNLIKELY(erts_no_run_queues); -ErtsAlignedSchedulerData *erts_aligned_scheduler_data; #ifdef ERTS_DIRTY_SCHEDULERS -ErtsAlignedSchedulerData *erts_aligned_dirty_cpu_scheduler_data; -ErtsAlignedSchedulerData *erts_aligned_dirty_io_scheduler_data; + +struct { + union { + erts_smp_atomic32_t active; + char align__[ERTS_CACHE_LINE_SIZE]; + } cpu; + union { + erts_smp_atomic32_t active; + char align__[ERTS_CACHE_LINE_SIZE]; + } io; +} dirty_count erts_align_attribute(ERTS_CACHE_LINE_SIZE); + +#endif + +static ERTS_INLINE void +dirty_active(ErtsSchedulerData *esdp, erts_aint32_t add) +{ +#ifdef ERTS_DIRTY_SCHEDULERS + erts_aint32_t val; + erts_smp_atomic32_t *ap; + switch (esdp->type) { + case ERTS_SCHED_DIRTY_CPU: + ap = &dirty_count.cpu.active; + break; + case ERTS_SCHED_DIRTY_IO: + ap = &dirty_count.io.active; + break; + default: + ap = NULL; + ERTS_INTERNAL_ERROR("Not a dirty scheduler"); + break; + } + + /* + * All updates done under run-queue lock, so + * no inc or dec needed... + */ + ERTS_SMP_ASSERT(erts_smp_lc_runq_is_locked(esdp->run_queue)); + + val = erts_smp_atomic32_read_nob(ap); + val += add; + erts_smp_atomic32_set_nob(ap, val); +#endif +} + +ErtsAlignedSchedulerData * ERTS_WRITE_UNLIKELY(erts_aligned_scheduler_data); +#ifdef ERTS_DIRTY_SCHEDULERS +ErtsAlignedSchedulerData * ERTS_WRITE_UNLIKELY(erts_aligned_dirty_cpu_scheduler_data); +ErtsAlignedSchedulerData * ERTS_WRITE_UNLIKELY(erts_aligned_dirty_io_scheduler_data); typedef union { Process dsp; char align[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(Process))]; @@ -539,22 +585,28 @@ do { \ } \ } while (0) -#define ERTS_ATOMIC_FOREACH_RUNQ_X(RQVAR, DO, DOX) \ +#define ERTS_ATOMIC_FOREACH_RUNQ_X(RQVAR, NRQS, DO, DOX) \ do { \ ErtsRunQueue *RQVAR; \ + int nrqs = (NRQS); \ int ix__; \ - for (ix__ = 0; ix__ < erts_no_run_queues; ix__++) { \ + for (ix__ = 0; ix__ < nrqs; ix__++) { \ RQVAR = ERTS_RUNQ_IX(ix__); \ erts_smp_runq_lock(RQVAR); \ { DO; } \ } \ { DOX; } \ - for (ix__ = 0; ix__ < erts_no_run_queues; ix__++) \ + for (ix__ = 0; ix__ < nrqs; ix__++) \ erts_smp_runq_unlock(ERTS_RUNQ_IX(ix__)); \ } while (0) -#define ERTS_ATOMIC_FOREACH_RUNQ(RQVAR, DO) \ - ERTS_ATOMIC_FOREACH_RUNQ_X(RQVAR, DO, ) +#define ERTS_ATOMIC_FOREACH_RUNQ(RQVAR, DO) \ + ERTS_ATOMIC_FOREACH_RUNQ_X(RQVAR, erts_no_run_queues + ERTS_NUM_DIRTY_RUNQS, DO, ) + +#define ERTS_ATOMIC_FOREACH_NORMAL_RUNQ(RQVAR, DO) \ + ERTS_ATOMIC_FOREACH_RUNQ_X(RQVAR, erts_no_run_queues, DO, ) + + /* * Local functions. */ @@ -2971,7 +3023,7 @@ erts_active_schedulers(void) { Uint as = erts_no_schedulers; - ERTS_ATOMIC_FOREACH_RUNQ(rq, as -= abs(rq->waiting)); + ERTS_ATOMIC_FOREACH_NORMAL_RUNQ(rq, as -= abs(rq->waiting)); return as; } @@ -3383,6 +3435,7 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) rq->sleepers.list->prev = ssi; rq->sleepers.list = ssi; erts_smp_spin_unlock(&rq->sleepers.lock); + dirty_active(esdp, -1); } #endif @@ -3724,6 +3777,9 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) sched_active_sys(esdp->no, rq); } + if (ERTS_SCHEDULER_IS_DIRTY(esdp)) + dirty_active(esdp, 1); + ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(rq)); } @@ -6123,7 +6179,7 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online #endif ) { - int ix, n, no_ssi; + int ix, n, no_ssi, tot_rqs; char *daww_ptr; size_t daww_sz; size_t size_runqs; @@ -6156,26 +6212,19 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online /* Create and initialize run queues */ n = no_schedulers; - size_runqs = sizeof(ErtsAlignedRunQueue) * (n + ERTS_NUM_DIRTY_RUNQS); + tot_rqs = (n + ERTS_NUM_DIRTY_RUNQS); + size_runqs = sizeof(ErtsAlignedRunQueue) * tot_rqs; erts_aligned_run_queues = erts_alloc_permanent_cache_aligned(ERTS_ALC_T_RUNQS, size_runqs); #ifdef ERTS_SMP -#ifdef ERTS_DIRTY_SCHEDULERS - erts_aligned_run_queues += ERTS_NUM_DIRTY_RUNQS; -#endif erts_smp_atomic32_init_nob(&no_empty_run_queues, 0); #endif erts_no_run_queues = n; - for (ix = -(ERTS_NUM_DIRTY_RUNQS); ix < n; ix++) { + for (ix = 0; ix < tot_rqs; ix++) { int pix, rix; -#ifdef ERTS_DIRTY_SCHEDULERS - ErtsRunQueue *rq = ERTS_RUNQ_IX_IS_DIRTY(ix) ? - ERTS_DIRTY_RUNQ_IX(ix) : ERTS_RUNQ_IX(ix); -#else ErtsRunQueue *rq = ERTS_RUNQ_IX(ix); -#endif rq->ix = ix; @@ -6448,6 +6497,11 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online ERTS_SCHED_DIRTY_IO, no_dirty_io_schedulers); + erts_smp_atomic32_init_nob(&dirty_count.cpu.active, + (erts_aint32_t) no_dirty_cpu_schedulers); + erts_smp_atomic32_init_nob(&dirty_count.io.active, + (erts_aint32_t) no_dirty_io_schedulers); + #endif if (set_schdlr_sspnd_change_flags) @@ -7822,6 +7876,7 @@ suspend_scheduler(ErtsSchedulerData *esdp) #endif if (sched_type != ERTS_SCHED_NORMAL) { + dirty_active(esdp, -1); erts_smp_runq_unlock(esdp->run_queue); dirty_sched_wall_time_change(esdp, 0); } @@ -8130,7 +8185,9 @@ suspend_scheduler(ErtsSchedulerData *esdp) erts_smp_runq_lock(esdp->run_queue); non_empty_runq(esdp->run_queue); - if (sched_type == ERTS_SCHED_NORMAL) { + if (sched_type != ERTS_SCHED_NORMAL) + dirty_active(esdp, 1); + else { schedule_bound_processes(esdp->run_queue, &sbp); erts_sched_check_cpu_bind_post_suspend(esdp); @@ -9764,38 +9821,69 @@ erts_internal_is_process_executing_dirty_1(BIF_ALIST_1) BIF_RET(am_false); } +static ERTS_INLINE void +run_queues_len_aux(ErtsRunQueue *rq, Uint *tot_len, Uint *qlen, int *ip, int incl_active_sched, int locked) +{ + Sint rq_len; + + if (locked) + rq_len = (Sint) erts_smp_atomic32_read_dirty(&rq->len); + else + rq_len = (Sint) erts_smp_atomic32_read_nob(&rq->len); + ASSERT(rq_len >= 0); + + if (incl_active_sched) { +#ifdef ERTS_DIRTY_SCHEDULERS + if (ERTS_RUNQ_IX_IS_DIRTY(rq->ix)) { + erts_aint32_t dcnt; + if (ERTS_RUNQ_IS_DIRTY_CPU_RUNQ(rq)) { + dcnt = erts_smp_atomic32_read_nob(&dirty_count.cpu.active); + ASSERT(0 <= dcnt && dcnt <= erts_no_dirty_cpu_schedulers); + } + else { + ASSERT(ERTS_RUNQ_IS_DIRTY_IO_RUNQ(rq)); + dcnt = erts_smp_atomic32_read_nob(&dirty_count.io.active); + ASSERT(0 <= dcnt && dcnt <= erts_no_dirty_io_schedulers); + } + rq_len += (Sint) dcnt; + } + else +#endif + { + if (ERTS_RUNQ_FLGS_GET_NOB(rq) & ERTS_RUNQ_FLG_EXEC) + rq_len++; + } + } + if (qlen) + qlen[(*ip)++] = rq_len; + *tot_len += (Uint) rq_len; +} Uint -erts_run_queues_len(Uint *qlen, int atomic_queues_read, int incl_active_sched) +erts_run_queues_len(Uint *qlen, int atomic_queues_read, int incl_active_sched, + int incl_dirty_io) { - int i = 0; + int i = 0, j = 0; Uint len = 0; - if (atomic_queues_read) - ERTS_ATOMIC_FOREACH_RUNQ(rq, - { - Sint rq_len = (Sint) erts_smp_atomic32_read_dirty(&rq->len); - ASSERT(rq_len >= 0); - if (incl_active_sched - && (ERTS_RUNQ_FLGS_GET_NOB(rq) & ERTS_RUNQ_FLG_EXEC)) { - rq_len++; - } - if (qlen) - qlen[i++] = rq_len; - len += (Uint) rq_len; - } - ); + int no_rqs = erts_no_run_queues; + +#ifdef ERTS_DIRTY_SCHEDULERS + if (incl_dirty_io) + no_rqs += ERTS_NUM_DIRTY_RUNQS; + else + no_rqs += ERTS_NUM_DIRTY_CPU_RUNQS; +#endif + + if (atomic_queues_read) { + ERTS_ATOMIC_FOREACH_RUNQ_X(rq, no_rqs, + run_queues_len_aux(rq, &len, qlen, &j, + incl_active_sched, 1), + /* Nothing... */); + } else { - for (i = 0; i < erts_no_run_queues; i++) { + for (i = 0; i < no_rqs; i++) { ErtsRunQueue *rq = ERTS_RUNQ_IX(i); - Sint rq_len = (Sint) erts_smp_atomic32_read_nob(&rq->len); - ASSERT(rq_len >= 0); - if (incl_active_sched - && (ERTS_RUNQ_FLGS_GET_NOB(rq) & ERTS_RUNQ_FLG_EXEC)) { - rq_len++; - } - if (qlen) - qlen[i] = rq_len; - len += (Uint) rq_len; + run_queues_len_aux(rq, &len, qlen, &j, incl_active_sched, 0); } } @@ -12097,6 +12185,8 @@ erts_get_total_reductions(Uint *redsp, Uint *diffp) Uint reds = 0; ERTS_ATOMIC_FOREACH_RUNQ_X(rq, + erts_no_run_queues + ERTS_NUM_DIRTY_RUNQS, + reds += rq->procs.reductions, if (redsp) *redsp = reds; diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index 2b169bb9ce..d44e8c252d 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -1541,24 +1541,29 @@ extern int erts_system_profile_ts_type; } while (0) #if defined(ERTS_DIRTY_SCHEDULERS) && defined(ERTS_SMP) -#define ERTS_NUM_DIRTY_RUNQS 2 +#define ERTS_NUM_DIRTY_CPU_RUNQS 1 +#define ERTS_NUM_DIRTY_IO_RUNQS 1 #else -#define ERTS_NUM_DIRTY_RUNQS 0 +#define ERTS_NUM_DIRTY_CPU_RUNQS 0 +#define ERTS_NUM_DIRTY_IO_RUNQS 0 #endif +#define ERTS_NUM_DIRTY_RUNQS (ERTS_NUM_DIRTY_CPU_RUNQS+ERTS_NUM_DIRTY_IO_RUNQS) + #define ERTS_RUNQ_IX(IX) \ - (ASSERT(0 <= (IX) && (IX) < erts_no_run_queues), \ + (ASSERT(0 <= (IX) && (IX) < erts_no_run_queues+ERTS_NUM_DIRTY_RUNQS), \ &erts_aligned_run_queues[(IX)].runq) #ifdef ERTS_DIRTY_SCHEDULERS #define ERTS_RUNQ_IX_IS_DIRTY(IX) \ - (-(ERTS_NUM_DIRTY_RUNQS) <= (IX) && (IX) < 0) + (ASSERT(0 <= (IX) && (IX) < erts_no_run_queues+ERTS_NUM_DIRTY_RUNQS), \ + (erts_no_run_queues <= (IX))) #define ERTS_DIRTY_RUNQ_IX(IX) \ (ASSERT(ERTS_RUNQ_IX_IS_DIRTY(IX)), \ &erts_aligned_run_queues[(IX)].runq) -#define ERTS_DIRTY_CPU_RUNQ (&erts_aligned_run_queues[-1].runq) -#define ERTS_DIRTY_IO_RUNQ (&erts_aligned_run_queues[-2].runq) -#define ERTS_RUNQ_IS_DIRTY_CPU_RUNQ(RQ) ((RQ)->ix == -1) -#define ERTS_RUNQ_IS_DIRTY_IO_RUNQ(RQ) ((RQ)->ix == -2) +#define ERTS_DIRTY_CPU_RUNQ (&erts_aligned_run_queues[erts_no_run_queues].runq) +#define ERTS_DIRTY_IO_RUNQ (&erts_aligned_run_queues[erts_no_run_queues+1].runq) +#define ERTS_RUNQ_IS_DIRTY_CPU_RUNQ(RQ) ((RQ) == ERTS_DIRTY_CPU_RUNQ) +#define ERTS_RUNQ_IS_DIRTY_IO_RUNQ(RQ) ((RQ) == ERTS_DIRTY_IO_RUNQ) #else #define ERTS_RUNQ_IX_IS_DIRTY(IX) 0 #endif @@ -1836,7 +1841,7 @@ Uint erts_active_schedulers(void); void erts_init_process(int, int, int); Eterm erts_process_state2status(erts_aint32_t); Eterm erts_process_status(Process *, Eterm); -Uint erts_run_queues_len(Uint *, int, int); +Uint erts_run_queues_len(Uint *, int, int, int); void erts_add_to_runq(Process *); Eterm erts_bound_schedulers_term(Process *c_p); Eterm erts_get_cpu_topology_term(Process *c_p, Eterm which); diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 2de908ffb3..bfebff5706 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -604,18 +604,6 @@ static size_t my_strnlen(const char *s, size_t maxlen) #endif -#if defined(HAVE_SYS_UN_H) - -/* Check that some character in the buffer != '\0' */ -static int is_nonzero(const char *s, size_t n) -{ - size_t i; - for (i = 0; i < n; i++) if (s[i] != '\0') return !0; - return 0; -} - -#endif - #ifdef VALGRIND # include <valgrind/memcheck.h> #else @@ -4025,13 +4013,30 @@ static char* inet_set_address(int family, inet_address* dst, int n; if (*len == 0) return str_einval; n = *((unsigned char*)(*src)); /* Length field */ - if ((*len < 1+n) || (sizeof(dst->sal.sun_path) < n+1)) { + if (*len < 1+n) return str_einval; + if (n + +#ifdef __linux__ + /* Make sure the address gets zero terminated + * except when the first byte is \0 because then it is + * sort of zero terminated although the zero termination + * comes before the address... + * This fix handles Linux's nonportable + * abstract socket address extension. + */ + ((*len) > 1 && (*src)[1] == '\0' ? 0 : 1) +#else + 1 +#endif + > sizeof(dst->sal.sun_path)) { return str_einval; } sys_memzero((char*)dst, sizeof(struct sockaddr_un)); dst->sal.sun_family = family; sys_memcpy(dst->sal.sun_path, (*src)+1, n); *len = offsetof(struct sockaddr_un, sun_path) + n; +#ifndef NO_SA_LEN + dst->sal.sun_len = *len; +#endif *src += 1 + n; return NULL; } @@ -4176,15 +4181,16 @@ static int inet_get_address(char* dst, inet_address* src, unsigned int* len) if (*len < offsetof(struct sockaddr_un, sun_path)) return -1; n = *len - offsetof(struct sockaddr_un, sun_path); if (255 < n) return -1; - /* Portability fix: Assume that the address is a zero terminated - * string, except when the first byte is \0 i.e the - * string length is 0. Then use the reported length instead. - * This fix handles Linux's abstract socket address - * nonportable extension. - */ m = my_strnlen(src->sal.sun_path, n); - if ((m == 0) && is_nonzero(src->sal.sun_path, n)) - m = n; +#ifdef __linux__ + /* Assume that the address is a zero terminated string, + * except when the first byte is \0 i.e the string length is 0, + * then use the reported length instead. + * This fix handles Linux's nonportable + * abstract socket address extension. + */ + if (m == 0) m = n; +#endif dst[0] = INET_AF_LOCAL; dst[1] = (char) ((unsigned char) m); sys_memcpy(dst+2, src->sal.sun_path, m); @@ -4241,15 +4247,16 @@ inet_address_to_erlang(char *dst, inet_address **src, SOCKLEN_T sz) { if (sz < offsetof(struct sockaddr_un, sun_path)) return -1; n = sz - offsetof(struct sockaddr_un, sun_path); if (255 < n) return -1; - /* Portability fix: Assume that the address is a zero terminated - * string, except when the first byte is \0 i.e the - * string length is 0. Then use the reported length instead. - * This fix handles Linux's abstract socket address - * nonportable extension. - */ m = my_strnlen((*src)->sal.sun_path, n); - if ((m == 0) && is_nonzero((*src)->sal.sun_path, n)) - m = n; +#ifdef __linux__ + /* Assume that the address is a zero terminated string, + * except when the first byte is \0 i.e the string length is 0, + * Then use the reported length instead. + * This fix handles Linux's nonportable + * abstract socket address extension. + */ + if (m == 0) m = n; +#endif if (dst) { dst[0] = INET_AF_LOCAL; dst[1] = (char) ((unsigned char) m); @@ -8680,6 +8687,7 @@ static ErlDrvSSizeT inet_ctl(inet_descriptor* desc, int cmd, char* buf, else { ptr = &peer; sz = sizeof(peer); + sys_memzero((char *) &peer, sz); if (IS_SOCKET_ERROR (sock_peer (desc->s, (struct sockaddr*)ptr, &sz))) @@ -10830,10 +10838,11 @@ static int tcp_inet_output(tcp_descriptor* desc, HANDLE event) #ifndef SO_ERROR { - int sz = sizeof(desc->inet.remote); - int code = sock_peer(desc->inet.s, - (struct sockaddr*) &desc->inet.remote, &sz); - + int sz, code; + sz = sizeof(desc->inet.remote); + sys_memzero((char *) &desc->inet.remote, sz); + code = sock_peer(desc->inet.s, + (struct sockaddr*) &desc->inet.remote, &sz); if (IS_SOCKET_ERROR(code)) { desc->inet.state = INET_STATE_OPEN; /* restore state */ ret = async_error(INETP(desc), sock_errno()); diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c index 0079912b10..b1bea3a960 100644 --- a/erts/emulator/sys/unix/sys.c +++ b/erts/emulator/sys/unix/sys.c @@ -885,7 +885,6 @@ void erts_replace_intr(void) { void init_break_handler(void) { sys_signal(SIGINT, request_break); - sys_signal(SIGHUP, generic_signal_handler); #ifndef ETHR_UNUSABLE_SIGUSRX sys_signal(SIGUSR1, generic_signal_handler); #endif /* #ifndef ETHR_UNUSABLE_SIGUSRX */ diff --git a/erts/emulator/test/nif_SUITE.erl b/erts/emulator/test/nif_SUITE.erl index 1eb58699b2..f9ee96c9ca 100644 --- a/erts/emulator/test/nif_SUITE.erl +++ b/erts/emulator/test/nif_SUITE.erl @@ -488,7 +488,7 @@ select(Config) when is_list(Config) -> %% Wait for read eagain = read_nif(R, 3), 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,null,Ref), - [] = flush(), + [] = flush(0), ok = write_nif(W, <<"hej">>), [{select, R, Ref, ready_input}] = flush(), 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,self(),Ref2), @@ -505,7 +505,7 @@ select(Config) when is_list(Config) -> %% Wait for write Written = write_full(W, $a), 0 = select_nif(W,?ERL_NIF_SELECT_WRITE,W,self(),Ref), - [] = flush(), + [] = flush(0), Written = read_nif(R,byte_size(Written)), [{select, W, Ref, ready_output}] = flush(), @@ -515,7 +515,7 @@ select(Config) when is_list(Config) -> [{fd_resource_stop, W_ptr, _}] = flush(), {1, {W_ptr,_}} = last_fd_stop_call(), true = is_closed_nif(W), - [] = flush(), + [] = flush(0), 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,self(),Ref), [{select, R, Ref, ready_input}] = flush(), eof = read_nif(R,1), @@ -540,7 +540,7 @@ select_2(Config) -> 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,null,Ref1), 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,self(),Ref2), - [] = flush(), + [] = flush(0), ok = write_nif(W, <<"hej">>), [{select, R, Ref2, ready_input}] = flush(), <<"hej">> = read_nif(R, 3), @@ -551,7 +551,7 @@ select_2(Config) -> Papa = self(), spawn_link(fun() -> 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,null,Ref1), - [] = flush(), + [] = flush(0), Papa ! sync, [{select, R, Ref1, ready_input}] = flush(), <<"hej">> = read_nif(R, 3), @@ -560,7 +560,7 @@ select_2(Config) -> sync = receive_any(), ok = write_nif(W, <<"hej">>), done = receive_any(), - [] = flush(), + [] = flush(0), check_stop_ret(select_nif(R,?ERL_NIF_SELECT_STOP,R,null,Ref1)), [{fd_resource_stop, R_ptr, _}] = flush(), @@ -629,6 +629,15 @@ monitor_process_a(Config) -> monitor_process_b(Config) -> ensure_lib_loaded(Config), + monitor_process_b_do(false), + case erlang:system_info(threads) of + true -> monitor_process_b_do(true); + false -> ok + end, + ok. + + +monitor_process_b_do(FromThread) -> Pid = spawn_link(fun() -> receive return -> ok @@ -637,8 +646,11 @@ monitor_process_b(Config) -> R_ptr = alloc_monitor_resource_nif(), {0,_} = monitor_process_nif(R_ptr, Pid, true, self()), [R_ptr] = monitored_by(Pid), - ok = release_resource(R_ptr), - [] = flush(), + case FromThread of + false -> ok = release_resource(R_ptr); + true -> ok = release_resource_from_thread(R_ptr) + end, + [] = flush(0), {R_ptr, _, 1} = last_resource_dtor_call(), [] = monitored_by(Pid), Pid ! return, @@ -660,7 +672,7 @@ monitor_process_c(Config) -> exit end), [{Pid, done, R_ptr, Mon1}, - {monitor_resource_down, R_ptr, Pid, Mon2}] = flush(), + {monitor_resource_down, R_ptr, Pid, Mon2}] = flush(2), compare_monitors_nif(Mon1, Mon2), {R_ptr, _, 1} = last_resource_dtor_call(), ok. @@ -708,7 +720,7 @@ demonitor_process(Config) -> 1 = demonitor_process_nif(R_ptr, MonBin2), ok = release_resource(R_ptr), - [] = flush(), + [] = flush(0), {R_ptr, _, 1} = last_resource_dtor_call(), [] = monitored_by(Pid), Pid ! return, @@ -2307,10 +2319,16 @@ receive_any(Timeout) -> after Timeout -> timeout end. flush() -> - flush(10). -flush(Timeout) -> + flush(1). + +flush(0) -> + flush(0, 10); % don't waste too much time waiting for nothing +flush(N) -> + flush(N, 1000). + +flush(N, Timeout) -> receive M -> - [M | flush(Timeout)] + [M | flush(N-1)] after Timeout -> [] end. @@ -2789,6 +2807,7 @@ alloc_resource(_,_) -> ?nif_stub. make_resource(_) -> ?nif_stub. get_resource(_,_) -> ?nif_stub. release_resource(_) -> ?nif_stub. +release_resource_from_thread(_) -> ?nif_stub. last_resource_dtor_call() -> ?nif_stub. make_new_resource(_,_) -> ?nif_stub. check_is(_,_,_,_,_,_,_,_,_,_,_) -> ?nif_stub. diff --git a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c index 3747291e7e..15d31162ed 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c +++ b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c @@ -972,6 +972,30 @@ static ERL_NIF_TERM release_resource(ErlNifEnv* env, int argc, const ERL_NIF_TER return enif_make_atom(env,"ok"); } +static void* threaded_release_resource(void* resource) +{ + enif_release_resource(resource); +} + +static ERL_NIF_TERM release_resource_from_thread(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + void* resource; + ErlNifTid tid; + int err; + + if (!get_pointer(env, argv[0], &resource)) { + return enif_make_badarg(env); + } + if (enif_thread_create("nif_SUITE:release_resource_from_thread", &tid, + threaded_release_resource, resource, NULL) != 0) { + return enif_make_badarg(env); + } + err = enif_thread_join(tid, NULL); + assert(err == 0); + return atom_ok; +} + + /* * argv[0] an atom * argv[1] a binary @@ -2537,6 +2561,7 @@ static ERL_NIF_TERM monitor_frenzy_nif(ErlNifEnv* env, int argc, const ERL_NIF_T static unsigned long spawn_cnt = 0; static unsigned long kill_cnt = 0; static unsigned long proc_histogram[FRENZY_PROCS_MAX]; + static int initialized = 0; static const unsigned int primes[] = {7, 13, 17, 19}; @@ -2556,7 +2581,7 @@ static ERL_NIF_TERM monitor_frenzy_nif(ErlNifEnv* env, int argc, const ERL_NIF_T if (enif_is_atom(env, Op)) { if (Op == atom_init) { - if (procs_lock || !enif_get_uint(env, Rnd, &frenzy_rand_bits_max)) + if (initialized || !enif_get_uint(env, Rnd, &frenzy_rand_bits_max)) return enif_make_badarg(env); procs_lock = enif_mutex_create("nif_SUITE:monitor_frenzy.procs"); @@ -2583,6 +2608,7 @@ static ERL_NIF_TERM monitor_frenzy_nif(ErlNifEnv* env, int argc, const ERL_NIF_T spawn_cnt = 1; kill_cnt = 0; + initialized = 1; return enif_make_uint(env, 0); /* SelfPix */ } else if (Op == atom_stats) { @@ -2613,7 +2639,7 @@ static ERL_NIF_TERM monitor_frenzy_nif(ErlNifEnv* env, int argc, const ERL_NIF_T enif_make_ulong(env, res_dtor_cnt))); } - else if (Op == atom_stop && procs_lock) { /* stop all */ + else if (Op == atom_stop && initialized) { /* stop all */ /* Release all resources */ for (rix = 0; rix < FRENZY_RESOURCES_MAX; rix++) { @@ -2903,6 +2929,7 @@ static ErlNifFunc nif_funcs[] = {"make_resource", 1, make_resource}, {"get_resource", 2, get_resource}, {"release_resource", 1, release_resource}, + {"release_resource_from_thread", 1, release_resource_from_thread}, {"last_resource_dtor_call", 0, last_resource_dtor_call}, {"make_new_resource", 2, make_new_resource}, {"check_is", 11, check_is}, diff --git a/erts/emulator/test/statistics_SUITE.erl b/erts/emulator/test/statistics_SUITE.erl index 0cea941687..7690557fda 100644 --- a/erts/emulator/test/statistics_SUITE.erl +++ b/erts/emulator/test/statistics_SUITE.erl @@ -502,20 +502,37 @@ run_queues_lengths_active_tasks(_Config) -> end, lists:seq(1,10)), + + TRQLs0 = statistics(total_run_queue_lengths), + TRQLAs0 = statistics(total_run_queue_lengths_all), TATs0 = statistics(total_active_tasks), + TATAs0 = statistics(total_active_tasks_all), true = is_integer(TRQLs0), true = is_integer(TATs0), true = TRQLs0 >= 0, + true = TRQLAs0 >= 0, true = TATs0 >= 11, + true = TATAs0 >= 11, NoScheds = erlang:system_info(schedulers), + {DefRqs, + AllRqs} = case erlang:system_info(dirty_cpu_schedulers) of + 0 -> {NoScheds, NoScheds}; + _ -> {NoScheds+1, NoScheds+2} + end, RQLs0 = statistics(run_queue_lengths), + RQLAs0 = statistics(run_queue_lengths_all), ATs0 = statistics(active_tasks), - NoScheds = length(RQLs0), - NoScheds = length(ATs0), + ATAs0 = statistics(active_tasks_all), + DefRqs = length(RQLs0), + AllRqs = length(RQLAs0), + DefRqs = length(ATs0), + AllRqs = length(ATAs0), true = lists:sum(RQLs0) >= 0, + true = lists:sum(RQLAs0) >= 0, true = lists:sum(ATs0) >= 11, + true = lists:sum(ATAs0) >= 11, SO = erlang:system_flag(schedulers_online, 1), @@ -531,8 +548,8 @@ run_queues_lengths_active_tasks(_Config) -> RQLs1 = statistics(run_queue_lengths), ATs1 = statistics(active_tasks), - NoScheds = length(RQLs1), - NoScheds = length(ATs1), + DefRqs = length(RQLs1), + DefRqs = length(ATs1), TRQLs2 = lists:sum(RQLs1), TATs2 = lists:sum(ATs1), true = TRQLs2 >= 10, diff --git a/erts/etc/unix/etp-commands.in b/erts/etc/unix/etp-commands.in index b7b3a2ae99..8f70f879d5 100644 --- a/erts/etc/unix/etp-commands.in +++ b/erts/etc/unix/etp-commands.in @@ -1,3 +1,4 @@ +# -*- gdb-script -*- # # %CopyrightBegin% # @@ -2153,13 +2154,22 @@ define etp-processes printf "No processes, since system isn't initialized!\n" else set $proc_ix = 0 - while $proc_ix < erts_proc.r.o.max - set $proc = (Process *) *((UWord *) &erts_proc.r.o.tab[$proc_ix]) - if ($proc != ((Process *) 0) && $proc != &erts_invalid_process) + set $proc_max_ix = erts_proc.r.o.max + set $proc_tab = erts_proc.r.o.tab + set $invalid_proc = &erts_invalid_process + set $proc_decentile = $proc_max_ix / 10 + set $proc_printile = $proc_decentile + while $proc_ix < $proc_max_ix + set $proc = (Process *) *((UWord *) ($proc_tab + $proc_ix)) + if ($proc != ((Process *) 0) && $proc != $invalid_proc) printf "---\n" printf " Pix: %d\n", $proc_ix etp-process-info $proc end + if $proc_ix == $proc_printile + printf "--- %d%% (%d / %d) searched\n", $proc_printile / $proc_decentile * 10, $proc_ix, $proc_max_ix + set $proc_printile += $proc_decentile + end set $proc_ix++ end printf "---\n", @@ -2479,15 +2489,19 @@ document etp-port-info %--------------------------------------------------------------------------- end - define etp-ports if (!erts_initialized) printf "No ports, since system isn't initialized!\n" else set $port_ix = 0 - while $port_ix < erts_port.r.o.max - set $port = (Port *) *((UWord *) &erts_port.r.o.tab[$port_ix]) - if ($port != ((Port *) 0) && $port != &erts_invalid_port) + set $port_max_ix = erts_port.r.o.max + set $port_tab = erts_port.r.o.tab + set $invalid_port = &erts_invalid_port + set $port_decentile = $port_max_ix / 10 + set $port_printile = $port_decentile + while $port_ix < $port_max_ix + set $port = (Port *) *((UWord *) ($port_tab + $port_ix)) + if ($port != ((Port *) 0) && $port != $invalid_port) if (*(((Uint32 *) &(((Port *) $port)->state))) & 0x100) == 0 # I.e, not free printf "---\n" @@ -2495,6 +2509,10 @@ define etp-ports etp-port-info $port end end + if $port_ix == $port_printile + printf "--- %d%% (%d / %d) searched\n", $port_printile / $port_decentile * 10, $port_ix, $port_max_ix + set $port_printile += $port_decentile + end set $port_ix++ end printf "---\n", @@ -2810,10 +2828,10 @@ define etp-run-queue-info-internal else if ($sched_type == 1) printf "\n--- Dirty CPU Run Queue ---\n" - set $runq = &erts_aligned_run_queues[-1].runq + set $runq = &erts_aligned_run_queues[erts_no_run_queues].runq else printf "\n--- Dirty I/O Run Queue ---\n" - set $runq = &erts_aligned_run_queues[-2].runq + set $runq = &erts_aligned_run_queues[erts_no_run_queues+1].runq end end printf " Length: total=%d", *((Uint32 *) &($runq->len)) diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex 980df873ca..63518ed6e1 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index fcb27ef575..72dd804412 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -2306,6 +2306,8 @@ spawn_opt(_Tuple) -> -spec statistics(active_tasks) -> [ActiveTasks] when ActiveTasks :: non_neg_integer(); + (active_tasks_all) -> [ActiveTasks] when + ActiveTasks :: non_neg_integer(); (context_switches) -> {ContextSwitches,0} when ContextSwitches :: non_neg_integer(); (exact_reductions) -> {Total_Exact_Reductions, @@ -2335,6 +2337,8 @@ spawn_opt(_Tuple) -> (run_queue) -> non_neg_integer(); (run_queue_lengths) -> [RunQueueLength] when RunQueueLength :: non_neg_integer(); + (run_queue_lengths_all) -> [RunQueueLength] when + RunQueueLength :: non_neg_integer(); (runtime) -> {Total_Run_Time, Time_Since_Last_Call} when Total_Run_Time :: non_neg_integer(), Time_Since_Last_Call :: non_neg_integer(); @@ -2347,9 +2351,13 @@ spawn_opt(_Tuple) -> ActiveTime :: non_neg_integer(), TotalTime :: non_neg_integer(); (total_active_tasks) -> ActiveTasks when + ActiveTasks :: non_neg_integer(); + (total_active_tasks_all) -> ActiveTasks when ActiveTasks :: non_neg_integer(); (total_run_queue_lengths) -> TotalRunQueueLengths when TotalRunQueueLengths :: non_neg_integer(); + (total_run_queue_lengths_all) -> TotalRunQueueLengths when + TotalRunQueueLengths :: non_neg_integer(); (wall_clock) -> {Total_Wallclock_Time, Wallclock_Time_Since_Last_Call} when Total_Wallclock_Time :: non_neg_integer(), diff --git a/lib/common_test/doc/src/ct_ssh.xml b/lib/common_test/doc/src/ct_ssh.xml index 137e4c3f1d..63627d8840 100644 --- a/lib/common_test/doc/src/ct_ssh.xml +++ b/lib/common_test/doc/src/ct_ssh.xml @@ -1034,6 +1034,33 @@ ChannelId, 0, Data, End, Timeout)</c></seealso>.</p> </func> <func> + <name>shell(SSH, ChannelId) -> ok | {error, Reason}</name> + <fsummary>Equivalent to shell(SSH, ChannelId, DefaultTimeout).</fsummary> + <desc><marker id="shell-2"/> + <p>Equivalent to + <seealso marker="#shell-3"><c>ct_ssh:shell(SSH, ChannelId, + DefaultTimeout)</c></seealso>.</p> + </desc> + </func> + + <func> + <name>shell(SSH, ChannelId, Timeout) -> ok | {error, Reason}</name> + <fsummary>Requests that the user default shell is executed at the + server end.</fsummary> + <type> + <v>SSH = connection()</v> + <v>ChannelId = integer()</v> + <v>Timeout = integer()</v> + <v>Reason = term()</v> + </type> + <desc><marker id="shell-3"/> + <p>Requests that the user default shell (typically defined in + <c>/etc/passwd</c> in Unix systems) is executed at the + server end.</p> + </desc> + </func> + + <func> <name>subsystem(SSH, ChannelId, Subsystem) -> Status | {error, Reason}</name> <fsummary>Equivalent to subsystem(SSH, ChannelId, Subsystem, DefaultTimeout).</fsummary> diff --git a/lib/common_test/src/ct_ssh.erl b/lib/common_test/src/ct_ssh.erl index 6ab3bf036c..5ac91f396b 100644 --- a/lib/common_test/src/ct_ssh.erl +++ b/lib/common_test/src/ct_ssh.erl @@ -68,7 +68,8 @@ send_and_receive/3, send_and_receive/4, send_and_receive/5, send_and_receive/6, exec/2, exec/3, exec/4, - subsystem/3, subsystem/4]). + subsystem/3, subsystem/4, + shell/2, shell/3]). %% STFP Functions -export([sftp_connect/1, @@ -94,6 +95,7 @@ -record(state, {ssh_ref, conn_type, target}). +-type handle() :: pid(). %%%----------------------------------------------------------------- %%%------------------------ SSH COMMANDS --------------------------- @@ -490,6 +492,22 @@ subsystem(SSH, ChannelId, Subsystem, Timeout) -> call(SSH, {subsystem,ChannelId,Subsystem,Timeout}). +-spec shell(SSH, ChannelId) -> Result when + SSH :: handle() | ct:target_name(), + ChannelId :: ssh:ssh_channel_id(), + Result :: ok | {error,term()}. +shell(SSH, ChannelId) -> + shell(SSH, ChannelId, ?DEFAULT_TIMEOUT). + +-spec shell(SSH, ChannelId, Timeout) -> Result when + SSH :: handle() | ct:target_name(), + ChannelId :: ssh:ssh_channel_id(), + Timeout :: timeout(), + Result :: ok | {error,term()}. +shell(SSH, ChannelId, Timeout) -> + call(SSH, {shell,ChannelId,Timeout}). + + %%%----------------------------------------------------------------- %%%------------------------ SFTP COMMANDS -------------------------- @@ -1067,6 +1085,14 @@ handle_msg({subsystem,Chn,Subsystem,TO}, State) -> Result = ssh_connection:subsystem(SSHRef, Chn, Subsystem, TO), {Result,State}; +handle_msg({shell,Chn,TO}, State) -> + #state{ssh_ref=SSHRef, target=Target} = State, + try_log(heading(shell,Target), + "SSH Ref: ~p, Chn: ~p, Timeout: ~p", + [SSHRef,Chn,TO]), + Result = ssh_connection:shell(SSHRef, Chn), + {Result,State}; + %% --- SFTP Commands --- handle_msg({read_file,Srv,File}=Cmd, S=#state{ssh_ref=SSHRef}) -> diff --git a/lib/hipe/cerl/erl_types.erl b/lib/hipe/cerl/erl_types.erl index ea8cc1677d..4cfa80f153 100644 --- a/lib/hipe/cerl/erl_types.erl +++ b/lib/hipe/cerl/erl_types.erl @@ -5171,9 +5171,9 @@ cache_put(Key, Type, DeltaL, #cache{types = Types} = Cache) -> NewTypes = maps:put(Key, {Type, DeltaL}, Types), Cache#cache{types = NewTypes}. --spec t_var_names([erl_type()]) -> [atom()]. +-spec t_var_names([parse_form()]) -> [atom()]. -t_var_names([{var, _, Name}|L]) when L =/= '_' -> +t_var_names([{var, _, Name}|L]) when Name =/= '_' -> [Name|t_var_names(L)]; t_var_names([]) -> []. diff --git a/lib/kernel/doc/src/error_logger.xml b/lib/kernel/doc/src/error_logger.xml index 814e8eac46..27db00819f 100644 --- a/lib/kernel/doc/src/error_logger.xml +++ b/lib/kernel/doc/src/error_logger.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2017</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -163,6 +163,19 @@ ok</pre> </desc> </func> <func> + <name name="get_format_depth" arity="0"/> + <fsummary>Get the value of the Kernel application variable + <c>error_logger_format_depth</c>.</fsummary> + <desc> + <p>Returns <c>max(10, Depth)</c>, where <c>Depth</c> is the + value of + <seealso marker="kernel:kernel_app#error_logger_format_depth"> + error_logger_format_depth</seealso> + in the Kernel application, if Depth is an integer. Otherwise, + <c>unlimited</c> is returned.</p> + </desc> + </func> + <func> <name name="info_msg" arity="1"/> <name name="info_msg" arity="2"/> <fsummary>Send a standard information event to the error logger.</fsummary> diff --git a/lib/kernel/doc/src/inet.xml b/lib/kernel/doc/src/inet.xml index b7c904ff45..b71e8a1e5d 100644 --- a/lib/kernel/doc/src/inet.xml +++ b/lib/kernel/doc/src/inet.xml @@ -75,8 +75,8 @@ Address ip_address() ------- ------------ ::1 {0,0,0,0,0,0,0,1} ::192.168.42.2 {0,0,0,0,0,0,(192 bsl 8) bor 168,(42 bsl 8) bor 2} -FFFF::192.168.42.2 - {16#FFFF,0,0,0,0,0,(192 bsl 8) bor 168,(42 bsl 8) bor 2} +::FFFF:192.168.42.2 + {0,0,0,0,0,16#FFFF,(192 bsl 8) bor 168,(42 bsl 8) bor 2} 3ffe:b80:1f8d:2:204:acff:fe17:bf38 {16#3ffe,16#b80,16#1f8d,16#2,16#204,16#acff,16#fe17,16#bf38} fe80::204:acff:fe17:bf38 @@ -87,8 +87,8 @@ fe80::204:acff:fe17:bf38 <pre> 1> <input>inet:parse_address("192.168.42.2").</input> {ok,{192,168,42,2}} -2> <input>inet:parse_address("FFFF::192.168.42.2").</input> -{ok,{65535,0,0,0,0,0,49320,10754}}</pre> +2> <input>inet:parse_address("::FFFF:192.168.42.2").</input> +{ok,{0,0,0,0,0,65535,49320,10754}}</pre> </description> <datatypes> diff --git a/lib/kernel/doc/src/kernel_app.xml b/lib/kernel/doc/src/kernel_app.xml index 9fccb4c7ac..75e1e18d86 100644 --- a/lib/kernel/doc/src/kernel_app.xml +++ b/lib/kernel/doc/src/kernel_app.xml @@ -58,6 +58,7 @@ </section> <section> + <marker id="erl_signal_server"/> <title>OS Signal Event Handler</title> <p>Asynchronous OS signals may be subscribed to via the Kernel applications event manager (see <seealso marker="doc/design_principles:des_princ">OTP Design Principles</seealso> and diff --git a/lib/kernel/doc/src/net_kernel.xml b/lib/kernel/doc/src/net_kernel.xml index 4e2b0c69db..7ddb849824 100644 --- a/lib/kernel/doc/src/net_kernel.xml +++ b/lib/kernel/doc/src/net_kernel.xml @@ -64,6 +64,19 @@ $ <input>erl -sname foobar</input></pre> by the magic cookie system, see section <seealso marker="doc/reference_manual:distributed">Distributed Erlang</seealso> in the Erlang Reference Manual.</p> + <warning> + <p> + Starting a distributed node without also specifying + <seealso marker="erts:erl#proto_dist"><c>-proto_dist inet_tls</c></seealso> + will expose the node to attacks that may give the attacker + complete access to the node and in extension the cluster. + When using un-secure distributed nodes, make sure that the + network is configured to keep potential attackers out. + See the <seealso marker="ssl:ssl_distribution"> + Using SSL for Erlang Distribution</seealso> User's Guide + for details on how to setup a secure distributed node. + </p> + </warning> </description> <funcs> diff --git a/lib/kernel/doc/src/os.xml b/lib/kernel/doc/src/os.xml index 64c5cbe571..0e9add4161 100644 --- a/lib/kernel/doc/src/os.xml +++ b/lib/kernel/doc/src/os.xml @@ -174,8 +174,9 @@ DirOut = os:cmd("dir"), % on Win32 platform</code> <tag><c>handle</c></tag> <item> - This signal will notify <c>erl_signal_server</c> when it is received by - the Erlang runtime system. + This signal will notify + <seealso marker="kernel_app#erl_signal_server"><c>erl_signal_server</c></seealso> + when it is received by the Erlang runtime system. </item> </taglist> </desc> diff --git a/lib/kernel/src/dist_util.erl b/lib/kernel/src/dist_util.erl index 1c326afca8..b3507e5d13 100644 --- a/lib/kernel/src/dist_util.erl +++ b/lib/kernel/src/dist_util.erl @@ -572,12 +572,25 @@ recv_name(#hs_data{socket = Socket, f_recv = Recv}) -> ?shutdown(no_node) end. -get_name([$n,VersionA, VersionB, Flag1, Flag2, Flag3, Flag4 | OtherNode]) -> - {?u32(Flag1, Flag2, Flag3, Flag4), list_to_atom(OtherNode), - ?u16(VersionA,VersionB)}; +get_name([$n,VersionA, VersionB, Flag1, Flag2, Flag3, Flag4 | OtherNode] = Data) -> + case is_valid_name(OtherNode) of + true -> + {?u32(Flag1, Flag2, Flag3, Flag4), list_to_atom(OtherNode), + ?u16(VersionA,VersionB)}; + false -> + ?shutdown(Data) + end; get_name(Data) -> ?shutdown(Data). +is_valid_name(OtherNodeName) -> + case string:lexemes(OtherNodeName,"@") of + [_OtherNodeName,_OtherNodeHost] -> + true; + _else -> + false + end. + publish_type(Flags) -> case Flags band ?DFLAG_PUBLISHED of 0 -> diff --git a/lib/kernel/src/error_logger.erl b/lib/kernel/src/error_logger.erl index 81f1bf8d97..9bf8547745 100644 --- a/lib/kernel/src/error_logger.erl +++ b/lib/kernel/src/error_logger.erl @@ -31,6 +31,8 @@ handle_event/2, handle_call/2, handle_info/2, terminate/2]). +-export([get_format_depth/0, limit_term/1]). + -define(buffer_size, 10). %%----------------------------------------------------------------- @@ -518,3 +520,21 @@ string_p1([H|T]) when is_list(H) -> end; string_p1([]) -> true; string_p1(_) -> false. + +-spec limit_term(term()) -> term(). + +limit_term(Term) -> + case get_format_depth() of + unlimited -> Term; + D -> io_lib:limit_term(Term, D) + end. + +-spec get_format_depth() -> 'unlimited' | pos_integer(). + +get_format_depth() -> + case application:get_env(kernel, error_logger_format_depth) of + {ok, Depth} when is_integer(Depth) -> + max(10, Depth); + undefined -> + unlimited + end. diff --git a/lib/kernel/src/global.erl b/lib/kernel/src/global.erl index 3d6415036c..a9e92b28b8 100644 --- a/lib/kernel/src/global.erl +++ b/lib/kernel/src/global.erl @@ -447,7 +447,8 @@ info() -> init([]) -> process_flag(trap_exit, true), _ = ets:new(global_locks, [set, named_table, protected]), - _ = ets:new(global_names, [set, named_table, protected]), + _ = ets:new(global_names, [set, named_table, protected, + {read_concurrency, true}]), _ = ets:new(global_names_ext, [set, named_table, protected]), _ = ets:new(global_pid_names, [bag, named_table, protected]), diff --git a/lib/mnesia/src/mnesia.erl b/lib/mnesia/src/mnesia.erl index dece995d39..3b771e8c5b 100644 --- a/lib/mnesia/src/mnesia.erl +++ b/lib/mnesia/src/mnesia.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -166,7 +166,7 @@ -type select_continuation() :: term(). -type snmp_struct() :: [{atom(), snmp_type() | tuple_of(snmp_type())}]. -type snmp_type() :: 'fix_string' | 'string' | 'integer'. --type tuple_of(_) :: tuple(). +-type tuple_of(_T) :: tuple(). -define(DEFAULT_ACCESS, ?MODULE). diff --git a/lib/observer/src/etop.erl b/lib/observer/src/etop.erl index 925f4456bb..2093e6a0d7 100644 --- a/lib/observer/src/etop.erl +++ b/lib/observer/src/etop.erl @@ -180,10 +180,16 @@ stop(Opts) -> end, unregister(etop_server). -update(#opts{store=Store,node=Node,tracing=Tracing}=Opts) -> +update(#opts{store=Store,node=Node,tracing=Tracing,intv=Interval}=Opts) -> Pid = spawn_link(Node,observer_backend,etop_collect,[self()]), Info = receive {Pid,I} -> I - after 1000 -> exit(connection_lost) + after Interval -> + %% Took more than the update interval to fetch + %% data. Either the connection is lost or the + %% fetching took too long... + io:format("Timeout when waiting for process info from " + "node ~p; exiting~n", [Node]), + exit(timeout) end, #etop_info{procinfo=ProcInfo} = Info, ProcInfo1 = diff --git a/lib/observer/src/observer_lib.erl b/lib/observer/src/observer_lib.erl index 4145a8d961..7352af936c 100644 --- a/lib/observer/src/observer_lib.erl +++ b/lib/observer/src/observer_lib.erl @@ -173,13 +173,13 @@ fill_info([{Str,Attrib,Key}|Rest], Data) when is_atom(Key); is_function(Key) -> Value -> [{Str,Attrib,Value} | fill_info(Rest, Data)] end; fill_info([{Str, {Format, Key}}|Rest], Data) - when is_atom(Key); is_function(Key), is_atom(Format) -> + when is_atom(Key); is_function(Key) -> case get_value(Key, Data) of undefined -> [undefined | fill_info(Rest, Data)]; Value -> [{Str, {Format, Value}} | fill_info(Rest, Data)] end; fill_info([{Str, Attrib, {Format, Key}}|Rest], Data) - when is_atom(Key); is_function(Key), is_atom(Format) -> + when is_atom(Key); is_function(Key) -> case get_value(Key, Data) of undefined -> [undefined | fill_info(Rest, Data)]; Value -> [{Str, Attrib, {Format, Value}} | fill_info(Rest, Data)] @@ -252,6 +252,8 @@ to_str({bytes, B}) -> KB > 0 -> integer_to_list(KB) ++ " kB"; true -> integer_to_list(B) ++ " B" end; +to_str({{words,WSz}, Sz}) -> + to_str({bytes, WSz*Sz}); to_str({time_ms, MS}) -> S = MS div 1000, Min = S div 60, diff --git a/lib/observer/src/observer_procinfo.erl b/lib/observer/src/observer_procinfo.erl index 8d19d77488..10decd8b62 100644 --- a/lib/observer/src/observer_procinfo.erl +++ b/lib/observer/src/observer_procinfo.erl @@ -198,10 +198,11 @@ code_change(_, _, State) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% init_process_page(Panel, Pid) -> - Fields0 = process_info_fields(Pid), + WSz = observer_wx:try_rpc(node(Pid), erlang, system_info,[wordsize]), + Fields0 = process_info_fields(Pid, WSz), {FPanel, _, UpFields} = observer_lib:display_info(Panel, Fields0), {FPanel, fun() -> - Fields = process_info_fields(Pid), + Fields = process_info_fields(Pid, WSz), observer_lib:update_info(UpFields, Fields) end}. @@ -359,7 +360,7 @@ create_menus(MenuBar) -> {"View", [#create_menu{id=?REFRESH, text="Refresh\tCtrl-R"}]}], observer_lib:create_menus(Menus, MenuBar, new_window). -process_info_fields(Pid) -> +process_info_fields(Pid, WSz) -> Struct = [{"Overview", [{"Initial Call", initial_call}, {"Current Function", current_function}, @@ -383,10 +384,10 @@ process_info_fields(Pid) -> {"Monitored by", {click, monitored_by}}]}, {"Memory and Garbage Collection", right, [{"Memory", {bytes, memory}}, - {"Stack and Heaps", {bytes, total_heap_size}}, - {"Heap Size", {bytes, heap_size}}, - {"Stack Size", {bytes, stack_size}}, - {"GC Min Heap Size", {bytes, get_gc_info(min_heap_size)}}, + {"Stack and Heaps", {{words,WSz}, total_heap_size}}, + {"Heap Size", {{words,WSz}, heap_size}}, + {"Stack Size", {{words,WSz}, stack_size}}, + {"GC Min Heap Size", {{words,WSz}, get_gc_info(min_heap_size)}}, {"GC FullSweep After", get_gc_info(fullsweep_after)} ]}], case observer_wx:try_rpc(node(Pid), erlang, process_info, [Pid, item_list()]) of diff --git a/lib/public_key/src/pubkey_ssh.erl b/lib/public_key/src/pubkey_ssh.erl index 6974afa992..9bda76d670 100644 --- a/lib/public_key/src/pubkey_ssh.erl +++ b/lib/public_key/src/pubkey_ssh.erl @@ -408,10 +408,11 @@ comma_list_encode([Option | Rest], Acc) -> comma_list_encode(Rest, Acc ++ "," ++ Option). -%% An experimental fix adding the signature algorithm name as the last element in a tuple... - ssh2_pubkey_encode(#'RSAPublicKey'{modulus = N, publicExponent = E}) -> ssh2_pubkey_encode({#'RSAPublicKey'{modulus = N, publicExponent = E}, 'ssh-rsa'}); + +ssh2_pubkey_encode({Key, 'rsa-sha2-256'}) -> ssh2_pubkey_encode({Key, 'ssh-rsa'}); +ssh2_pubkey_encode({Key, 'rsa-sha2-512'}) -> ssh2_pubkey_encode({Key, 'ssh-rsa'}); ssh2_pubkey_encode({#'RSAPublicKey'{modulus = N, publicExponent = E}, SignAlg}) -> SignAlgName = list_to_binary(atom_to_list(SignAlg)), StrLen = size(SignAlgName), @@ -448,16 +449,12 @@ ssh2_pubkey_encode(Key={#'ECPoint'{point = Q}, {namedCurve,OID}}) -> ssh2_pubkey_decode(Bin = <<?UINT32(Len), Type:Len/binary, _/binary>>) -> ssh2_pubkey_decode(Type, Bin). -%% An experimental fix with the Signature Algorithm Name -ssh2_pubkey_decode(SignAlgName, +ssh2_pubkey_decode(<<"rsa-sha2-256">>, Bin) -> ssh2_pubkey_decode(<<"ssh-rsa">>, Bin); +ssh2_pubkey_decode(<<"rsa-sha2-512">>, Bin) -> ssh2_pubkey_decode(<<"ssh-rsa">>, Bin); +ssh2_pubkey_decode(<<"ssh-rsa">>, <<?UINT32(Len), _:Len/binary, ?UINT32(SizeE), E:SizeE/binary, - ?UINT32(SizeN), N:SizeN/binary>>) - when SignAlgName == <<"ssh-rsa">> ; - SignAlgName == <<"rsa-sha2-256">> ; - SignAlgName == <<"rsa-sha2-384">> ; - SignAlgName == <<"rsa-sha2-512">> - -> + ?UINT32(SizeN), N:SizeN/binary>>) -> #'RSAPublicKey'{modulus = erlint(SizeN, N), publicExponent = erlint(SizeE, E)}; @@ -471,6 +468,7 @@ ssh2_pubkey_decode(<<"ssh-dss">>, #'Dss-Parms'{p = erlint(SizeP, P), q = erlint(SizeQ, Q), g = erlint(SizeG, G)}}; + ssh2_pubkey_decode(<<"ecdsa-sha2-",Id/binary>>, <<?UINT32(Len), ECDSA_SHA2_etc:Len/binary, ?UINT32(SizeId), Id:SizeId/binary, diff --git a/lib/sasl/src/sasl_report.erl b/lib/sasl/src/sasl_report.erl index 14702b0ad2..eb454155d5 100644 --- a/lib/sasl/src/sasl_report.erl +++ b/lib/sasl/src/sasl_report.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -68,12 +68,12 @@ write_report2(IO, Fd, Head, progress, Report) -> Format = format_key_val(Report), write_report_action(IO, Fd, Head, "~s", [Format]); write_report2(IO, Fd, Head, crash_report, Report) -> - Depth = get_depth(), + Depth = error_logger:get_format_depth(), Format = proc_lib:format(Report, latin1, Depth), write_report_action(IO, Fd, Head, "~s", [Format]). supervisor_format(Args0) -> - case get_depth() of + case error_logger:get_format_depth() of unlimited -> {" Supervisor: ~p~n" " Context: ~p~n" @@ -102,14 +102,6 @@ format_key_val([{Tag,Data}|Rep]) -> format_key_val(_) -> []. -get_depth() -> - case application:get_env(kernel, error_logger_format_depth) of - {ok, Depth} when is_integer(Depth) -> - max(10, Depth); - undefined -> - unlimited - end. - sup_get(Tag, Report) -> case lists:keysearch(Tag, 1, Report) of {value, {_, Value}} -> diff --git a/lib/ssh/src/ssh_connection_handler.erl b/lib/ssh/src/ssh_connection_handler.erl index 6a6b9896cb..f1ce337947 100644 --- a/lib/ssh/src/ssh_connection_handler.erl +++ b/lib/ssh/src/ssh_connection_handler.erl @@ -346,7 +346,7 @@ renegotiate_data(ConnectionHandler) -> | undefined, last_size_rekey = 0 :: non_neg_integer(), event_queue = [] :: list(), - opts :: ssh_options:options(), +% opts :: ssh_options:options(), inet_initial_recbuf_size :: pos_integer() | undefined }). @@ -398,8 +398,7 @@ init([Role,Socket,Opts]) -> transport_protocol = Protocol, transport_cb = Callback, transport_close_tag = CloseTag, - ssh_params = init_ssh_record(Role, Socket, PeerAddr, Opts), - opts = Opts + ssh_params = init_ssh_record(Role, Socket, PeerAddr, Opts) }, D = case Role of client -> @@ -1012,7 +1011,7 @@ handle_event(cast, renegotiate, _, _) -> handle_event(cast, data_size, {connected,Role}, D) -> {ok, [{send_oct,Sent0}]} = inet:getstat(D#data.socket, [send_oct]), Sent = Sent0 - D#data.last_size_rekey, - MaxSent = ?GET_OPT(rekey_limit, D#data.opts), + MaxSent = ?GET_OPT(rekey_limit, (D#data.ssh_params)#ssh.opts), timer:apply_after(?REKEY_DATA_TIMOUT, gen_statem, cast, [self(), data_size]), case Sent >= MaxSent of true -> @@ -1712,7 +1711,12 @@ ext_info({"server-sig-algs",SigAlgs}, D0 = #data{ssh_params=#ssh{role=client, ], CommonAlgs = [Alg || Alg <- ServerSigAlgs, lists:member(Alg, ClientSigAlgs)], - D0#data{ssh_params = Ssh0#ssh{userauth_pubkeys = CommonAlgs} }; + SelectedAlgs = + case CommonAlgs of + [] -> ClientSigAlgs; % server-sig-algs value is just an advice + _ -> CommonAlgs + end, + D0#data{ssh_params = Ssh0#ssh{userauth_pubkeys = SelectedAlgs} }; ext_info(_, D0) -> %% Not implemented @@ -1862,7 +1866,7 @@ get_repl(X, Acc) -> exit({get_repl,X,Acc}). %%%---------------------------------------------------------------- --define(CALL_FUN(Key,D), catch (?GET_OPT(Key, D#data.opts)) ). +-define(CALL_FUN(Key,D), catch (?GET_OPT(Key, (D#data.ssh_params)#ssh.opts)) ). disconnect_fun({disconnect,Msg}, D) -> ?CALL_FUN(disconnectfun,D)(Msg); disconnect_fun(Reason, D) -> ?CALL_FUN(disconnectfun,D)(Reason). @@ -1912,7 +1916,7 @@ retry_fun(User, Reason, #data{ssh_params = #ssh{opts = Opts, %%% channels open for a while. cache_init_idle_timer(D) -> - case ?GET_OPT(idle_time, D#data.opts) of + case ?GET_OPT(idle_time, (D#data.ssh_params)#ssh.opts) of infinity -> D#data{idle_timer_value = infinity, idle_timer_ref = infinity % A flag used later... diff --git a/lib/ssh/src/ssh_dbg.erl b/lib/ssh/src/ssh_dbg.erl index 7dfbfc3b4b..003b3856e6 100644 --- a/lib/ssh/src/ssh_dbg.erl +++ b/lib/ssh/src/ssh_dbg.erl @@ -22,9 +22,10 @@ -module(ssh_dbg). --export([messages/0, - messages/1, - messages/2, +-export([messages/0, messages/1, messages/2, + ct_messages/0, + auth/0, auth/1, auth/2, + ct_auth/0, stop/0 ]). @@ -43,75 +44,167 @@ messages() -> messages(fun(String,_D) -> io:format(String) end). +ct_messages() -> + messages(fun(String,_D) -> ct:log(String,[]) end). + messages(Write) when is_function(Write,2) -> messages(Write, fun(X) -> X end). messages(Write, MangleArg) when is_function(Write,2), is_function(MangleArg,1) -> - catch dbg:start(), - setup_tracer(Write, MangleArg), - dbg:p(new,[c,timestamp]), - dbg_ssh_messages(). + cond_start(msg, Write, MangleArg), + dbg_ssh_messages(), + dbg_ssh_auth(). + + +auth() -> + auth(fun(String,_D) -> io:format(String) end). + +ct_auth() -> + auth(fun(String,_D) -> ct:log(String,[]) end). + +auth(Write) when is_function(Write,2) -> + auth(Write, fun(X) -> X end). + +auth(Write, MangleArg) when is_function(Write,2), + is_function(MangleArg,1) -> + cond_start(auth, Write, MangleArg), + dbg_ssh_auth(). + dbg_ssh_messages() -> dbg:tp(ssh_message,encode,1, x), dbg:tp(ssh_message,decode,1, x), dbg:tpl(ssh_transport,select_algorithm,4, x), dbg:tp(ssh_transport,hello_version_msg,1, x), - dbg:tp(ssh_transport,handle_hello_version,1, x). + dbg:tp(ssh_transport,handle_hello_version,1, x), + dbg:tpl(ssh_connection_handler,ext_info,2, x). + +dbg_ssh_auth() -> + dbg:tp(ssh_transport,hello_version_msg,1, x), + dbg:tp(ssh_transport,handle_hello_version,1, x), + dbg:tp(ssh_message,encode,1, x), + dbg:tpl(ssh_transport,select_algorithm,4, x), + dbg:tpl(ssh_connection_handler,ext_info,2, x), + lists:foreach(fun(F) -> dbg:tp(ssh_auth, F, x) end, + [publickey_msg, password_msg, keyboard_interactive_msg]). %%%---------------------------------------------------------------- stop() -> dbg:stop(). %%%================================================================ -msg_formater({trace_ts,Pid,call,{ssh_message,encode,[Msg]},TS}, D) -> +cond_start(Type, Write, MangleArg) -> + try + dbg:start(), + setup_tracer(Type, Write, MangleArg), + dbg:p(new,[c,timestamp]) + catch + _:_ -> ok + end. + + +msg_formater(msg, {trace_ts,Pid,call,{ssh_message,encode,[Msg]},TS}, D) -> fmt("~n~s SEND ~p ~s~n", [ts(TS),Pid,wr_record(shrink_bin(Msg))], D); -msg_formater({trace_ts,_Pid,return_from,{ssh_message,encode,1},_Res,_TS}, D) -> +msg_formater(msg, {trace_ts,_Pid,return_from,{ssh_message,encode,1},_Res,_TS}, D) -> D; -msg_formater({trace_ts,_Pid,call,{ssh_message,decode,_},_TS}, D) -> +msg_formater(msg, {trace_ts,_Pid,call,{ssh_message,decode,_},_TS}, D) -> D; -msg_formater({trace_ts,Pid,return_from,{ssh_message,decode,1},Msg,TS}, D) -> +msg_formater(msg, {trace_ts,Pid,return_from,{ssh_message,decode,1},Msg,TS}, D) -> fmt("~n~s ~p RECV ~s~n", [ts(TS),Pid,wr_record(shrink_bin(Msg))], D); + +msg_formater(auth, {trace_ts,Pid,return_from,{ssh_message,decode,1},#ssh_msg_userauth_failure{authentications=As},TS}, D) -> + fmt("~n~s ~p Client login FAILURE. Try ~s~n", [ts(TS),Pid,As], D); + +msg_formater(auth, {trace_ts,Pid,return_from,{ssh_message,decode,1},#ssh_msg_userauth_success{},TS}, D) -> + fmt("~n~s ~p Client login SUCCESS~n", [ts(TS),Pid], D); + -msg_formater({trace_ts,_Pid,call,{ssh_transport,select_algorithm,_},_TS}, D) -> +msg_formater(_, {trace_ts,_Pid,call,{ssh_transport,select_algorithm,_},_TS}, D) -> D; -msg_formater({trace_ts,Pid,return_from,{ssh_transport,select_algorithm,_},{ok,Alg},TS}, D) -> +msg_formater(_, {trace_ts,Pid,return_from,{ssh_transport,select_algorithm,_},{ok,Alg},TS}, D) -> fmt("~n~s ~p ALGORITHMS~n~s~n", [ts(TS),Pid, wr_record(Alg)], D); -msg_formater({trace_ts,_Pid,call,{ssh_transport,hello_version_msg,_},_TS}, D) -> +msg_formater(_, {trace_ts,_Pid,call,{ssh_transport,hello_version_msg,_},_TS}, D) -> D; -msg_formater({trace_ts,Pid,return_from,{ssh_transport,hello_version_msg,1},Hello,TS}, D) -> +msg_formater(_, {trace_ts,Pid,return_from,{ssh_transport,hello_version_msg,1},Hello,TS}, D) -> fmt("~n~s ~p TCP SEND HELLO~n ~p~n", [ts(TS),Pid,lists:flatten(Hello)], D); -msg_formater({trace_ts,Pid,call,{ssh_transport,handle_hello_version,[Hello]},TS}, D) -> +msg_formater(_, {trace_ts,Pid,call,{ssh_transport,handle_hello_version,[Hello]},TS}, D) -> fmt("~n~s ~p RECV HELLO~n ~p~n", [ts(TS),Pid,lists:flatten(Hello)], D); -msg_formater({trace_ts,_Pid,return_from,{ssh_transport,handle_hello_version,1},_,_TS}, D) -> +msg_formater(_, {trace_ts,_Pid,return_from,{ssh_transport,handle_hello_version,1},_,_TS}, D) -> D; -msg_formater({trace_ts,Pid,send,{tcp,Sock,Bytes},Pid,TS}, D) -> +msg_formater(_, {trace_ts,Pid,call,{ssh_connection_handler,ext_info,[{"server-sig-algs",_SigAlgs},State]},TS}, D) -> + try lists:keyfind(ssh, 1, tuple_to_list(State)) of + false -> + D; + #ssh{userauth_pubkeys = PKs} -> + fmt("~n~s ~p Client got suggestion to use user public key sig-algs~n ~p~n", [ts(TS),Pid,PKs], D) + catch + _:_ -> + D + end; + +msg_formater(_, {trace_ts,Pid,return_from,{ssh_connection_handler,ext_info,2},State,TS}, D) -> + try lists:keyfind(ssh, 1, tuple_to_list(State)) of + false -> + D; + #ssh{userauth_pubkeys = PKs} -> + fmt("~n~s ~p Client will try user public key sig-algs~n ~p~n", [ts(TS),Pid,PKs], D) + catch + _:_ -> + D + end; + +msg_formater(_, {trace_ts,Pid,call,{ssh_auth,publickey_msg,[[SigAlg,#ssh{user=User}]]},TS}, D) -> + fmt("~n~s ~p Client will try to login user ~p with public key algorithm ~p~n", [ts(TS),Pid,User,SigAlg], D); +msg_formater(_, {trace_ts,Pid,return_from,{ssh_auth,publickey_msg,1},{not_ok,#ssh{user=User}},TS}, D) -> + fmt("~s ~p User ~p can't login with that kind of public key~n", [ts(TS),Pid,User], D); + +msg_formater(_, {trace_ts,Pid,call,{ssh_auth,password_msg,[[#ssh{user=User}]]},TS}, D) -> + fmt("~n~s ~p Client will try to login user ~p with password~n", [ts(TS),Pid,User], D); +msg_formater(_, {trace_ts,Pid,return_from,{ssh_auth,password_msg,1},{not_ok,#ssh{user=User}},TS}, D) -> + fmt("~s ~p User ~p can't login with password~n", [ts(TS),Pid,User], D); + +msg_formater(_, {trace_ts,Pid,call,{ssh_auth,keyboard_interactive_msg,[[#ssh{user=User}]]},TS}, D) -> + fmt("~n~s ~p Client will try to login user ~p with password~n", [ts(TS),Pid,User], D); +msg_formater(_, {trace_ts,Pid,return_from,{ssh_auth,keyboard_interactive_msg,1},{not_ok,#ssh{user=User}},TS}, D) -> + fmt("~s ~p User ~p can't login with keyboard_interactive password~n", [ts(TS),Pid,User], D); + +msg_formater(msg, {trace_ts,Pid,send,{tcp,Sock,Bytes},Pid,TS}, D) -> fmt("~n~s ~p TCP SEND on ~p~n ~p~n", [ts(TS),Pid,Sock, shrink_bin(Bytes)], D); -msg_formater({trace_ts,Pid,send,{tcp,Sock,Bytes},Dest,TS}, D) -> +msg_formater(msg, {trace_ts,Pid,send,{tcp,Sock,Bytes},Dest,TS}, D) -> fmt("~n~s ~p TCP SEND from ~p TO ~p~n ~p~n", [ts(TS),Pid,Sock,Dest, shrink_bin(Bytes)], D); -msg_formater({trace_ts,Pid,send,ErlangMsg,Dest,TS}, D) -> +msg_formater(msg, {trace_ts,Pid,send,ErlangMsg,Dest,TS}, D) -> fmt("~n~s ~p ERL MSG SEND TO ~p~n ~p~n", [ts(TS),Pid,Dest, shrink_bin(ErlangMsg)], D); -msg_formater({trace_ts,Pid,'receive',{tcp,Sock,Bytes},TS}, D) -> +msg_formater(msg, {trace_ts,Pid,'receive',{tcp,Sock,Bytes},TS}, D) -> fmt("~n~s ~p TCP RECEIVE on ~p~n ~p~n", [ts(TS),Pid,Sock,shrink_bin(Bytes)], D); -msg_formater({trace_ts,Pid,'receive',ErlangMsg,TS}, D) -> +msg_formater(msg, {trace_ts,Pid,'receive',ErlangMsg,TS}, D) -> fmt("~n~s ~p ERL MSG RECEIVE~n ~p~n", [ts(TS),Pid,shrink_bin(ErlangMsg)], D); -msg_formater(M, D) -> - fmt("~nDBG ~n~p~n", [shrink_bin(M)], D). +%% msg_formater(_, {trace_ts,_Pid,return_from,MFA,_Ret,_TS}=M, D) -> +%% case lists:member(MFA, [{ssh_auth,keyboard_interactive_msg,1}, +%% {ssh_auth,password_msg,1}, +%% {ssh_auth,publickey_msg,1}]) of +%% true -> +%% D; +%% false -> +%% fmt("~nDBG ~n~p~n", [shrink_bin(M)], D) +%% end; + +%% msg_formater(_, M, D) -> +%% fmt("~nDBG ~n~p~n", [shrink_bin(M)], D). -%% msg_formater(_, D) -> -%% D. +msg_formater(_, _, D) -> + D. fmt(Fmt, Args, D=#data{writer=Write,acc=Acc}) -> @@ -123,9 +216,9 @@ ts({_,_,Usec}=Now) -> ts(_) -> "-". %%%---------------------------------------------------------------- -setup_tracer(Write, MangleArg) -> +setup_tracer(Type, Write, MangleArg) -> Handler = fun(Arg, D) -> - msg_formater(MangleArg(Arg), D) + msg_formater(Type, MangleArg(Arg), D) end, InitialData = #data{writer = Write}, {ok,_} = dbg:tracer(process, {Handler, InitialData}), diff --git a/lib/ssh/src/ssh_message.erl b/lib/ssh/src/ssh_message.erl index 609040826f..b1fc05ae33 100644 --- a/lib/ssh/src/ssh_message.erl +++ b/lib/ssh/src/ssh_message.erl @@ -598,8 +598,8 @@ decode_kex_init(<<?DEC_BIN(Data,__0), Rest/binary>>, Acc, N) -> %%% Signature decode/encode %%% -decode_signature(<<?DEC_BIN(_Alg,__0), ?UINT32(_), Signature/binary>>) -> - Signature. +decode_signature(<<?DEC_BIN(Alg,__0), ?UINT32(_), Signature/binary>>) -> + {binary_to_list(Alg), Signature}. encode_signature({#'RSAPublicKey'{},Sign}, Signature) -> @@ -611,13 +611,3 @@ encode_signature({{#'ECPoint'{}, {namedCurve,OID}},_}, Signature) -> CurveName = public_key:oid2ssh_curvename(OID), <<?Ebinary(<<"ecdsa-sha2-",CurveName/binary>>), ?Ebinary(Signature)>>. -%% encode_signature(#'RSAPublicKey'{}, Signature) -> -%% SignName = <<"ssh-rsa">>, -%% <<?Ebinary(SignName), ?Ebinary(Signature)>>; -%% encode_signature({_, #'Dss-Parms'{}}, Signature) -> -%% <<?Ebinary(<<"ssh-dss">>), ?Ebinary(Signature)>>; -%% encode_signature({#'ECPoint'{}, {namedCurve,OID}}, Signature) -> -%% CurveName = public_key:oid2ssh_curvename(OID), -%% <<?Ebinary(<<"ecdsa-sha2-",CurveName/binary>>), ?Ebinary(Signature)>>. - - diff --git a/lib/ssh/src/ssh_options.erl b/lib/ssh/src/ssh_options.erl index 12c0190082..aebb5a7062 100644 --- a/lib/ssh/src/ssh_options.erl +++ b/lib/ssh/src/ssh_options.erl @@ -392,12 +392,6 @@ default(server) -> class => user_options }, - {preferred_algorithms, def} => - #{default => ssh:default_algorithms(), - chk => fun check_preferred_algorithms/1, - class => user_options - }, - %%%%% Undocumented {infofun, def} => #{default => fun(_,_,_) -> void end, @@ -436,26 +430,11 @@ default(client) -> }, {pref_public_key_algs, def} => - #{default => ssh_transport:default_algorithms(public_key) -- ['rsa-sha2-256', - 'rsa-sha2-512'], + #{default => ssh_transport:default_algorithms(public_key), chk => fun check_pref_public_key_algs/1, class => user_options }, - {preferred_algorithms, def} => - #{default => [{K,Vs} || {K,Vs0} <- ssh:default_algorithms(), - Vs <- [case K of - public_key -> - Vs0 -- ['rsa-sha2-256', - 'rsa-sha2-512']; - _ -> - Vs0 - end] - ], - chk => fun check_preferred_algorithms/1, - class => user_options - }, - {dh_gex_limits, def} => #{default => {1024, 6144, 8192}, % FIXME: Is this true nowadays? chk => fun({Min,I,Max}) -> @@ -521,6 +500,12 @@ default(common) -> class => user_options }, + {preferred_algorithms, def} => + #{default => ssh:default_algorithms(), + chk => fun check_preferred_algorithms/1, + class => user_options + }, + {id_string, def} => #{default => undefined, % FIXME: see ssh_transport:ssh_vsn/0 chk => fun(random) -> diff --git a/lib/ssh/src/ssh_transport.erl b/lib/ssh/src/ssh_transport.erl index 1a15798080..412f5de9de 100644 --- a/lib/ssh/src/ssh_transport.erl +++ b/lib/ssh/src/ssh_transport.erl @@ -776,16 +776,20 @@ extract_public_key(#'ECPrivateKey'{parameters = {namedCurve,OID}, {#'ECPoint'{point=Q}, {namedCurve,OID}}. -verify_host_key(#ssh{algorithms=Alg}=SSH, PublicKey, Digest, Signature) -> - case verify(Digest, sha(Alg#alg.hkey), Signature, PublicKey) of - false -> - {error, bad_signature}; - true -> - known_host_key(SSH, PublicKey, public_algo(PublicKey)) +verify_host_key(#ssh{algorithms=Alg}=SSH, PublicKey, Digest, {AlgStr,Signature}) -> + case atom_to_list(Alg#alg.hkey) of + AlgStr -> + case verify(Digest, sha(Alg#alg.hkey), Signature, PublicKey) of + false -> + {error, bad_signature}; + true -> + known_host_key(SSH, PublicKey, public_algo(PublicKey)) + end; + _ -> + {error, bad_signature_name} end. - accepted_host(Ssh, PeerName, Public, Opts) -> case ?GET_OPT(silently_accept_hosts, Opts) of diff --git a/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl b/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl index 0995182623..165274241c 100644 --- a/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl +++ b/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl @@ -280,21 +280,21 @@ msg_code(Num) -> Name -include_lib("ssh/src/ssh_transport.hrl"). %%% Encoding and decodeing is asymetric so out=binary in=string. Sometimes. :( +-define(fix_asym_Xdh_reply(S), + fix_asym(#S{public_host_key = Key, h_sig = {Alg,Sig}} = M) -> + M#S{public_host_key = {Key, list_to_atom(Alg)}, h_sig = Sig} +). + + fix_asym(#ssh_msg_global_request{name=N} = M) -> M#ssh_msg_global_request{name = binary_to_list(N)}; fix_asym(#ssh_msg_debug{message=D,language=L} = M) -> M#ssh_msg_debug{message = binary_to_list(D), language = binary_to_list(L)}; fix_asym(#ssh_msg_kexinit{cookie=C} = M) -> M#ssh_msg_kexinit{cookie = <<C:128>>}; - -fix_asym(#ssh_msg_kexdh_reply{public_host_key = Key} = M) -> M#ssh_msg_kexdh_reply{public_host_key = key_sigalg(Key)}; -fix_asym(#ssh_msg_kex_dh_gex_reply{public_host_key = Key} = M) -> M#ssh_msg_kex_dh_gex_reply{public_host_key = key_sigalg(Key)}; -fix_asym(#ssh_msg_kex_ecdh_reply{public_host_key = Key} = M) -> M#ssh_msg_kex_ecdh_reply{public_host_key = key_sigalg(Key)}; - +?fix_asym_Xdh_reply(ssh_msg_kexdh_reply); +?fix_asym_Xdh_reply(ssh_msg_kex_dh_gex_reply); +?fix_asym_Xdh_reply(ssh_msg_kex_ecdh_reply); fix_asym(M) -> M. -%%% Keys now contains an sig-algorithm name -key_sigalg(#'RSAPublicKey'{} = Key) -> {Key,'ssh-rsa'}; -key_sigalg({_, #'Dss-Parms'{}} = Key) -> {Key,'ssh-dss'}; -key_sigalg({#'ECPoint'{}, {namedCurve,OID}} = Key) -> {Key,"ecdsa-sha2-256"}. %%% Message codes 30 and 31 are overloaded depending on kex family so arrange the decoder %%% input as the test object does diff --git a/lib/ssh/test/ssh.spec b/lib/ssh/test/ssh.spec index 68268cb20d..b4e3d36072 100644 --- a/lib/ssh/test/ssh.spec +++ b/lib/ssh/test/ssh.spec @@ -1,6 +1,7 @@ {suites,"../ssh_test",all}. -{skip_suites, "../ssh_test", [ssh_bench_SUITE +{skip_suites, "../ssh_test", [ssh_bench_SUITE, + ssh_upgrade_SUITE ], "Benchmarks run separately"}. diff --git a/lib/ssh/test/ssh_algorithms_SUITE.erl b/lib/ssh/test/ssh_algorithms_SUITE.erl index 736461624d..0f69910e40 100644 --- a/lib/ssh/test/ssh_algorithms_SUITE.erl +++ b/lib/ssh/test/ssh_algorithms_SUITE.erl @@ -171,6 +171,7 @@ init_per_testcase(_, {public_key,Alg}, Config) -> Opts = pubkey_opts(Config), case {ssh_file:user_key(Alg,Opts), ssh_file:host_key(Alg,Opts)} of {{ok,_}, {ok,_}} -> + ssh_dbg:ct_auth(), start_pubkey_daemon([proplists:get_value(pref_algs,Config)], [{extra_daemon,true}|Config]); {{ok,_}, _} -> @@ -192,6 +193,7 @@ init_per_testcase(_, _, Config) -> end_per_testcase(_TC, Config) -> + catch ssh_dbg:stop(), case proplists:get_value(extra_daemon, Config, false) of true -> case proplists:get_value(srvr_pid,Config) of diff --git a/lib/ssl/src/dtls_udp_listener.erl b/lib/ssl/src/dtls_udp_listener.erl index f0ace2d887..29380e3008 100644 --- a/lib/ssl/src/dtls_udp_listener.erl +++ b/lib/ssl/src/dtls_udp_listener.erl @@ -121,6 +121,18 @@ handle_info({udp, Socket, IP, InPortNo, _} = Msg, #state{listner = Socket} = Sta next_datagram(Socket), {noreply, State}; +%% UDP socket does not have a connection and should not receive an econnreset +%% This does however happens on on some windows versions. Just ignoring it +%% appears to make things work as expected! +handle_info({udp_error, Socket, econnreset = Error}, #state{listner = Socket} = State) -> + Report = io_lib:format("Ignore SSL UDP Listener: Socket error: ~p ~n", [Error]), + error_logger:info_report(Report), + {noreply, State}; +handle_info({udp_error, Socket, Error}, #state{listner = Socket} = State) -> + Report = io_lib:format("SSL UDP Listener shutdown: Socket error: ~p ~n", [Error]), + error_logger:info_report(Report), + {noreply, State#state{close=true}}; + handle_info({'DOWN', _, process, Pid, _}, #state{clients = Clients, dtls_processes = Processes0, close = ListenClosed} = State) -> diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index 58db8449d6..3cf466e78f 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -2295,6 +2295,8 @@ is_acceptable_hash_sign({_, dsa} = Algos, dsa, _, srp_dss, SupportedHashSigns) - is_acceptable_hash_sign(Algos, SupportedHashSigns); is_acceptable_hash_sign({_, ecdsa} = Algos, ecdsa, _, dhe_ecdsa, SupportedHashSigns) -> is_acceptable_hash_sign(Algos, SupportedHashSigns); +is_acceptable_hash_sign({_, ecdsa} = Algos, ecdsa, ecdsa, ecdh_ecdsa, SupportedHashSigns) -> + is_acceptable_hash_sign(Algos, SupportedHashSigns); is_acceptable_hash_sign({_, ecdsa} = Algos, ecdsa, ecdsa, ecdhe_ecdsa, SupportedHashSigns) -> is_acceptable_hash_sign(Algos, SupportedHashSigns); is_acceptable_hash_sign(_, _, _, KeyExAlgo, _) when diff --git a/lib/ssl/test/ssl_ECC_SUITE.erl b/lib/ssl/test/ssl_ECC_SUITE.erl index 1149513417..0fbb0bb79a 100644 --- a/lib/ssl/test/ssl_ECC_SUITE.erl +++ b/lib/ssl/test/ssl_ECC_SUITE.erl @@ -44,27 +44,74 @@ groups() -> {'tlsv1.2', [], all_versions_groups()}, {'tlsv1.1', [], all_versions_groups()}, {'tlsv1', [], all_versions_groups()}, - {'erlang_server', [], key_cert_combinations()}, - {'erlang_client', [], key_cert_combinations()}, + {'erlang_server', [], openssl_key_cert_combinations()}, + %%{'erlang_client', [], openssl_key_cert_combinations()}, {'erlang', [], key_cert_combinations() ++ misc() ++ ecc_negotiation()} ]. all_versions_groups ()-> [{group, 'erlang_server'}, - {group, 'erlang_client'}, + %%{group, 'erlang_client'}, {group, 'erlang'} ]. + +openssl_key_cert_combinations() -> + ECDH_RSA = case ssl_test_lib:openssl_filter("ECDH-RSA") of + [] -> + []; + _ -> + server_ecdh_rsa() + end, + + ECDHE_RSA = case ssl_test_lib:openssl_filter("ECDHE-RSA") of + [] -> + []; + _ -> + server_ecdhe_rsa() + end, + ECDH_ECDSA = case ssl_test_lib:openssl_filter("ECDH-ECDSA") of + [] -> + []; + _ -> + server_ecdhe_ecdsa() + end, + + ECDHE_ECDSA = case ssl_test_lib:openssl_filter("ECDHE-ECDSA") of + [] -> + []; + _ -> + server_ecdhe_ecdsa() + end, + ECDH_RSA ++ ECDHE_RSA ++ ECDH_ECDSA ++ ECDHE_ECDSA. + key_cert_combinations() -> + server_ecdh_rsa() ++ + server_ecdhe_rsa() ++ + server_ecdh_ecdsa() ++ + server_ecdhe_ecdsa(). + +server_ecdh_rsa() -> [client_ecdh_rsa_server_ecdh_rsa, - client_ecdhe_rsa_server_ecdh_rsa, - client_ecdh_rsa_server_ecdhe_rsa, + client_ecdhe_rsa_server_ecdh_rsa, + client_ecdhe_ecdsa_server_ecdh_rsa]. + +server_ecdhe_rsa() -> + [client_ecdh_rsa_server_ecdhe_rsa, client_ecdhe_rsa_server_ecdhe_rsa, - client_ecdhe_ecdsa_server_ecdhe_rsa, - client_ecdhe_ecdsa_server_ecdhe_ecdsa, - client_ecdh_rsa_server_ecdhe_ecdsa - ]. + client_ecdhe_ecdsa_server_ecdhe_rsa]. + +server_ecdh_ecdsa() -> + [client_ecdh_ecdsa_server_ecdh_ecdsa, + client_ecdhe_rsa_server_ecdh_ecdsa, + client_ecdhe_ecdsa_server_ecdh_ecdsa]. + +server_ecdhe_ecdsa() -> + [client_ecdh_rsa_server_ecdhe_ecdsa, + client_ecdh_ecdsa_server_ecdhe_ecdsa, + client_ecdhe_ecdsa_server_ecdhe_ecdsa]. + misc()-> [client_ecdsa_server_ecdsa_with_raw_key]. @@ -175,37 +222,63 @@ end_per_testcase(_TestCase, Config) -> %% ECDH_RSA client_ecdh_rsa_server_ecdh_rsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_rsa, ecdh_rsa, Config), - basic_test(COpts, SOpts, Config). - + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], + ecdh_rsa, ecdh_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdh_rsa} | proplists:delete(check_keyex, Config)]). client_ecdhe_rsa_server_ecdh_rsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdh_rsa, Config), - basic_test(COpts, SOpts, Config). - + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_rsa, ecdh_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdh_rsa} | proplists:delete(check_keyex, Config)]). +client_ecdhe_ecdsa_server_ecdh_rsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdh_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdh_rsa} | proplists:delete(check_keyex, Config)]). + %% ECDHE_RSA client_ecdh_rsa_server_ecdhe_rsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_rsa, ecdhe_rsa, Config), - basic_test(COpts, SOpts, Config). - + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdh_rsa, ecdhe_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_rsa} | proplists:delete(check_keyex, Config)]). client_ecdhe_rsa_server_ecdhe_rsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdhe_rsa, Config), - basic_test(COpts, SOpts, Config). - + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_rsa, ecdhe_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_rsa} | proplists:delete(check_keyex, Config)]). client_ecdhe_ecdsa_server_ecdhe_rsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_ecdsa, ecdhe_rsa, Config), - basic_test(COpts, SOpts, Config). + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdh_ecdsa, ecdhe_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_rsa} | proplists:delete(check_keyex, Config)]). -%% ECDHE_ECDSA -client_ecdhe_ecdsa_server_ecdhe_ecdsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), - basic_test(COpts, SOpts, Config). +%% ECDH_ECDSA +client_ecdh_ecdsa_server_ecdh_ecdsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([{server_peer_opts, + [{extensions, [{key_usage, [keyEncipherment] + }]}]}], + ecdh_ecdsa, ecdh_ecdsa, Config), + basic_test(COpts, SOpts, + [{check_keyex, ecdh_ecdsa} | proplists:delete(check_keyex, Config)]). +client_ecdhe_rsa_server_ecdh_ecdsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([{server_peer_opts, + [{extensions, [{key_usage, [keyEncipherment] + }]}]}], + ecdhe_rsa, ecdh_ecdsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdh_ecdsa} | proplists:delete(check_keyex, Config)]). + +client_ecdhe_ecdsa_server_ecdh_ecdsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([{server_peer_opts, + [{extensions, [{key_usage, [keyEncipherment] + }]}]}], + ecdhe_ecdsa, ecdh_ecdsa, Config), + basic_test(COpts, SOpts, + [{check_keyex, ecdh_ecdsa} | proplists:delete(check_keyex, Config)]). +%% ECDHE_ECDSA client_ecdh_rsa_server_ecdhe_ecdsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_rsa, ecdhe_ecdsa, Config), - basic_test(COpts, SOpts, Config). + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdh_rsa, ecdhe_ecdsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_ecdsa} | proplists:delete(check_keyex, Config)]). +client_ecdh_ecdsa_server_ecdhe_ecdsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdh_ecdsa, ecdhe_ecdsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_ecdsa} | proplists:delete(check_keyex, Config)]). +client_ecdhe_ecdsa_server_ecdhe_ecdsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_ecdsa, ecdhe_ecdsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_ecdsa} | proplists:delete(check_keyex, Config)]). client_ecdsa_server_ecdsa_with_raw_key(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_ecdsa, ecdhe_ecdsa, Config), ServerKeyFile = proplists:get_value(keyfile, SOpts), {ok, PemBin} = file:read_file(ServerKeyFile), PemEntries = public_key:pem_decode(PemBin), @@ -221,7 +294,7 @@ client_ecdsa_server_ecdsa_with_raw_key(Config) when is_list(Config) -> close(Server, Client). ecc_default_order(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [], case supported_eccs([{eccs, [sect571r1]}]) of true -> ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); @@ -229,7 +302,7 @@ ecc_default_order(Config) -> end. ecc_default_order_custom_curves(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); @@ -237,7 +310,7 @@ ecc_default_order_custom_curves(Config) -> end. ecc_client_order(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{honor_ecc_order, false}], case supported_eccs([{eccs, [sect571r1]}]) of true -> ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); @@ -245,7 +318,7 @@ ecc_client_order(Config) -> end. ecc_client_order_custom_curves(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{honor_ecc_order, false}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); @@ -253,12 +326,12 @@ ecc_client_order_custom_curves(Config) -> end. ecc_unknown_curve(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{eccs, ['123_fake_curve']}], ecc_test_error(COpts, SOpts, [], ECCOpts, Config). client_ecdh_rsa_server_ecdhe_ecdsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_rsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdh_rsa, ecdhe_ecdsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); @@ -266,7 +339,7 @@ client_ecdh_rsa_server_ecdhe_ecdsa_server_custom(Config) -> end. client_ecdh_rsa_server_ecdhe_rsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_rsa, ecdhe_rsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdh_rsa, ecdhe_rsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); @@ -274,7 +347,7 @@ client_ecdh_rsa_server_ecdhe_rsa_server_custom(Config) -> end. client_ecdhe_rsa_server_ecdhe_ecdsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_rsa, ecdhe_ecdsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); @@ -282,14 +355,16 @@ client_ecdhe_rsa_server_ecdhe_ecdsa_server_custom(Config) -> end. client_ecdhe_rsa_server_ecdhe_rsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdhe_rsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_rsa, ecdhe_rsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. client_ecdhe_rsa_server_ecdh_rsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdh_rsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([{server_peer_opts, + [{extensions, [{key_usage, [keyEncipherment] + }]}]}], ecdhe_rsa, ecdh_rsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); @@ -297,7 +372,7 @@ client_ecdhe_rsa_server_ecdh_rsa_server_custom(Config) -> end. client_ecdhe_ecdsa_server_ecdhe_ecdsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); @@ -305,7 +380,7 @@ client_ecdhe_ecdsa_server_ecdhe_ecdsa_server_custom(Config) -> end. client_ecdhe_ecdsa_server_ecdhe_rsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_rsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_rsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); @@ -313,7 +388,7 @@ client_ecdhe_ecdsa_server_ecdhe_rsa_server_custom(Config) -> end. client_ecdhe_ecdsa_server_ecdhe_ecdsa_client_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(secp256r1, COpts, SOpts, ECCOpts, [], Config); @@ -321,7 +396,7 @@ client_ecdhe_ecdsa_server_ecdhe_ecdsa_client_custom(Config) -> end. client_ecdhe_rsa_server_ecdhe_ecdsa_client_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_rsa, ecdhe_ecdsa, Config), ECCOpts = [{eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(secp256r1, COpts, SOpts, ECCOpts, [], Config); @@ -370,10 +445,11 @@ start_client(openssl, Port, ClientOpts, _Config) -> start_client(erlang, Port, ClientOpts, Config) -> {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), + KeyEx = proplists:get_value(check_keyex, Config, false), ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, {host, Hostname}, {from, self()}, - {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {mfa, {ssl_test_lib, check_key_exchange_send_active, [KeyEx]}}, {options, [{verify, verify_peer} | ClientOpts]}]). @@ -412,11 +488,12 @@ start_server(openssl, ServerOpts, _Config) -> {OpenSslPort, Port}; start_server(erlang, ServerOpts, Config) -> {_, ServerNode, _} = ssl_test_lib:run_where(Config), + KeyEx = proplists:get_value(check_keyex, Config, false), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, {mfa, {ssl_test_lib, - send_recv_result_active, - []}}, + check_key_exchange_send_active, + [KeyEx]}}, {options, [{verify, verify_peer} | ServerOpts]}]), {Server, ssl_test_lib:inet_port(Server)}. diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 68a99c28dd..77c21d9b57 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -498,12 +498,12 @@ make_rsa_cert_chains(ChainConf, Config, Suffix) -> [{reuseaddr, true}, {verify, verify_peer} | ServerConf] }. -make_ec_cert_chains(ClientChainType, ServerChainType, Config) -> +make_ec_cert_chains(ChainConf, ClientChainType, ServerChainType, Config) -> CryptoSupport = crypto:supports(), KeyGenSpec = key_gen_info(ClientChainType, ServerChainType), ClientFileBase = filename:join([proplists:get_value(priv_dir, Config), atom_to_list(ClientChainType)]), ServerFileBase = filename:join([proplists:get_value(priv_dir, Config), atom_to_list(ServerChainType)]), - GenCertData = x509_test:gen_test_certs([{digest, appropriate_sha(CryptoSupport)} | KeyGenSpec]), + GenCertData = x509_test:gen_test_certs([{digest, appropriate_sha(CryptoSupport)} | KeyGenSpec] ++ ChainConf), [{server_config, ServerConf}, {client_config, ClientConf}] = x509_test:gen_pem_config_files(GenCertData, ClientFileBase, ServerFileBase), @@ -1009,6 +1009,12 @@ openssl_ecdh_rsa_suites() -> lists:filter(fun(Str) -> string_regex_filter(Str, "ECDH-RSA") end, Ciphers). +openssl_filter(FilterStr) -> + Ciphers = string:tokens(os:cmd("openssl ciphers"), ":"), + lists:filter(fun(Str) -> string_regex_filter(Str, FilterStr) + end, Ciphers). + + string_regex_filter(Str, Search) when is_list(Str) -> case re:run(Str, Search, []) of nomatch -> @@ -1174,6 +1180,21 @@ sufficient_crypto_support(Group) when Group == ciphers_ec; %% From ssl_basic sufficient_crypto_support(_) -> true. +check_key_exchange_send_active(Socket, false) -> + send_recv_result_active(Socket); +check_key_exchange_send_active(Socket, KeyEx) -> + {ok, [{cipher_suite, Suite}]} = ssl:connection_information(Socket, [cipher_suite]), + true = check_key_exchange(Suite, KeyEx), + send_recv_result_active(Socket). + +check_key_exchange({KeyEx,_, _}, KeyEx) -> + true; +check_key_exchange({KeyEx,_,_,_}, KeyEx) -> + true; +check_key_exchange(KeyEx1, KeyEx2) -> + ct:pal("Negotiated ~p Expected ~p", [KeyEx1, KeyEx2]), + false. + send_recv_result_active(Socket) -> ssl:send(Socket, "Hello world"), receive diff --git a/lib/stdlib/doc/src/gen_statem.xml b/lib/stdlib/doc/src/gen_statem.xml index 1aac88c308..ad7f2f2e95 100644 --- a/lib/stdlib/doc/src/gen_statem.xml +++ b/lib/stdlib/doc/src/gen_statem.xml @@ -346,7 +346,7 @@ ok <p> To compare styles, here follows the same example using <seealso marker="#type-callback_mode"><em>callback mode</em></seealso> - <c>state_functions</c>, or rather the code to replace + <c>handle_event_function</c>, or rather the code to replace after function <c>init/1</c> of the <c>pushbutton.erl</c> example file above: </p> diff --git a/lib/stdlib/src/erl_parse.yrl b/lib/stdlib/src/erl_parse.yrl index 2dcddeb8c2..733932e711 100644 --- a/lib/stdlib/src/erl_parse.yrl +++ b/lib/stdlib/src/erl_parse.yrl @@ -1052,6 +1052,9 @@ build_typed_attribute({atom,Aa,record}, build_typed_attribute({atom,Aa,Attr}, {type_def, {call,_,{atom,_,TypeName},Args}, Type}) when Attr =:= 'type' ; Attr =:= 'opaque' -> + lists:foreach(fun({var, A, '_'}) -> ret_err(A, "bad type variable"); + (_) -> ok + end, Args), case lists:all(fun({var, _, _}) -> true; (_) -> false end, Args) of diff --git a/lib/stdlib/src/error_logger_file_h.erl b/lib/stdlib/src/error_logger_file_h.erl index 0b262de3ab..76f89841b9 100644 --- a/lib/stdlib/src/error_logger_file_h.erl +++ b/lib/stdlib/src/error_logger_file_h.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -57,7 +57,7 @@ init(File, PrevHandler) -> process_flag(trap_exit, true), case file:open(File, [write]) of {ok,Fd} -> - Depth = get_depth(), + Depth = error_logger:get_format_depth(), State = #st{fd=Fd,filename=File,prev_handler=PrevHandler, depth=Depth}, {ok, State}; @@ -65,14 +65,6 @@ init(File, PrevHandler) -> Error end. -get_depth() -> - case application:get_env(kernel, error_logger_format_depth) of - {ok, Depth} when is_integer(Depth) -> - max(10, Depth); - undefined -> - unlimited - end. - handle_event({_Type, GL, _Msg}, State) when node(GL) =/= node() -> {ok, State}; handle_event(Event, State) -> diff --git a/lib/stdlib/src/error_logger_tty_h.erl b/lib/stdlib/src/error_logger_tty_h.erl index 2f2fd65252..8f0d7b0362 100644 --- a/lib/stdlib/src/error_logger_tty_h.erl +++ b/lib/stdlib/src/error_logger_tty_h.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -44,7 +44,7 @@ %% This one is used when we takeover from the simple error_logger. init({[], {error_logger, Buf}}) -> User = set_group_leader(), - Depth = get_depth(), + Depth = error_logger:get_format_depth(), State = #st{user=User,prev_handler=error_logger,depth=Depth}, write_events(State, Buf), {ok, State}; @@ -56,17 +56,9 @@ init({[], {error_logger_tty_h, PrevHandler}}) -> %% This one is used when we are started directly. init([]) -> User = set_group_leader(), - Depth = get_depth(), + Depth = error_logger:get_format_depth(), {ok, #st{user=User,prev_handler=[],depth=Depth}}. -get_depth() -> - case application:get_env(kernel, error_logger_format_depth) of - {ok, Depth} when is_integer(Depth) -> - max(10, Depth); - undefined -> - unlimited - end. - handle_event({_Type, GL, _Msg}, State) when node(GL) =/= node() -> {ok, State}; handle_event(Event, State) -> diff --git a/lib/stdlib/src/gen_server.erl b/lib/stdlib/src/gen_server.erl index ba0a7ae8e5..a3d53efd0d 100644 --- a/lib/stdlib/src/gen_server.erl +++ b/lib/stdlib/src/gen_server.erl @@ -107,7 +107,9 @@ %% Internal exports -export([init_it/6]). --import(error_logger, [format/2]). +-define( + STACKTRACE(), + try throw(ok) catch _ -> erlang:get_stacktrace() end). %%%========================================================================= %%% API @@ -326,15 +328,16 @@ init_it(Starter, self, Name, Mod, Args, Options) -> init_it(Starter, Parent, Name0, Mod, Args, Options) -> Name = gen:name(Name0), Debug = gen:debug_options(Name, Options), - HibernateAfterTimeout = gen:hibernate_after(Options), - case catch Mod:init(Args) of - {ok, State} -> + HibernateAfterTimeout = gen:hibernate_after(Options), + + case init_it(Mod, Args) of + {ok, {ok, State}} -> proc_lib:init_ack(Starter, {ok, self()}), loop(Parent, Name, State, Mod, infinity, HibernateAfterTimeout, Debug); - {ok, State, Timeout} -> + {ok, {ok, State, Timeout}} -> proc_lib:init_ack(Starter, {ok, self()}), loop(Parent, Name, State, Mod, Timeout, HibernateAfterTimeout, Debug); - {stop, Reason} -> + {ok, {stop, Reason}} -> %% For consistency, we must make sure that the %% registered name (if any) is unregistered before %% the parent process is notified about the failure. @@ -344,18 +347,25 @@ init_it(Starter, Parent, Name0, Mod, Args, Options) -> gen:unregister_name(Name0), proc_lib:init_ack(Starter, {error, Reason}), exit(Reason); - ignore -> + {ok, ignore} -> gen:unregister_name(Name0), proc_lib:init_ack(Starter, ignore), exit(normal); - {'EXIT', Reason} -> - gen:unregister_name(Name0), - proc_lib:init_ack(Starter, {error, Reason}), - exit(Reason); - Else -> + {ok, Else} -> Error = {bad_return_value, Else}, proc_lib:init_ack(Starter, {error, Error}), - exit(Error) + exit(Error); + {'EXIT', Class, Reason, Stacktrace} -> + gen:unregister_name(Name0), + proc_lib:init_ack(Starter, {error, terminate_reason(Class, Reason, Stacktrace)}), + erlang:raise(Class, Reason, Stacktrace) + end. +init_it(Mod, Args) -> + try + {ok, Mod:init(Args)} + catch + throw:R -> {ok, R}; + Class:R -> {'EXIT', Class, R, erlang:get_stacktrace()} end. %%%======================================================================== @@ -397,7 +407,7 @@ decode_msg(Msg, Parent, Name, State, Mod, Time, HibernateAfterTimeout, Debug, Hi sys:handle_system_msg(Req, From, Parent, ?MODULE, Debug, [Name, State, Mod, Time, HibernateAfterTimeout], Hib); {'EXIT', Parent, Reason} -> - terminate(Reason, Name, undefined, Msg, Mod, State, Debug); + terminate(Reason, ?STACKTRACE(), Name, undefined, Msg, Mod, State, Debug); _Msg when Debug =:= [] -> handle_msg(Msg, Parent, Name, State, Mod, HibernateAfterTimeout); _Msg -> @@ -589,17 +599,11 @@ start_monitor(Node, Name) when is_atom(Node), is_atom(Name) -> %% --------------------------------------------------- %% Helper functions for try-catch of callbacks. %% Returns the return value of the callback, or -%% {'EXIT', ExitReason, ReportReason} (if an exception occurs) -%% -%% ExitReason is the reason that shall be used when the process -%% terminates. +%% {'EXIT', Class, Reason, Stack} (if an exception occurs) %% -%% ReportReason is the reason that shall be printed in the error -%% report. -%% -%% These functions are introduced in order to add the stack trace in -%% the error report produced when a callback is terminated with -%% erlang:exit/1 (OTP-12263). +%% The Class, Reason and Stack are given to erlang:raise/3 +%% to make sure proc_lib receives the proper reasons and +%% stacktraces. %% --------------------------------------------------- try_dispatch({'$gen_cast', Msg}, Mod, State) -> @@ -621,15 +625,10 @@ try_dispatch(Mod, Func, Msg, State) -> [Mod, Msg]), {ok, {noreply, State}}; true -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', {R, Stacktrace}, {R, Stacktrace}} + {'EXIT', error, R, erlang:get_stacktrace()} end; - error:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', {R, Stacktrace}, {R, Stacktrace}}; - exit:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', R, {R, Stacktrace}} + Class:R -> + {'EXIT', Class, R, erlang:get_stacktrace()} end. try_handle_call(Mod, Msg, From, State) -> @@ -638,12 +637,8 @@ try_handle_call(Mod, Msg, From, State) -> catch throw:R -> {ok, R}; - error:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', {R, Stacktrace}, {R, Stacktrace}}; - exit:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', R, {R, Stacktrace}} + Class:R -> + {'EXIT', Class, R, erlang:get_stacktrace()} end. try_terminate(Mod, Reason, State) -> @@ -654,12 +649,8 @@ try_terminate(Mod, Reason, State) -> catch throw:R -> {ok, R}; - error:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', {R, Stacktrace}, {R, Stacktrace}}; - exit:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', R, {R, Stacktrace}} + Class:R -> + {'EXIT', Class, R, erlang:get_stacktrace()} end; false -> {ok, ok} @@ -684,10 +675,11 @@ handle_msg({'$gen_call', From, Msg}, Parent, Name, State, Mod, HibernateAfterTim {ok, {noreply, NState, Time1}} -> loop(Parent, Name, NState, Mod, Time1, HibernateAfterTimeout, []); {ok, {stop, Reason, Reply, NState}} -> - {'EXIT', R} = - (catch terminate(Reason, Name, From, Msg, Mod, NState, [])), - reply(From, Reply), - exit(R); + try + terminate(Reason, ?STACKTRACE(), Name, From, Msg, Mod, NState, []) + after + reply(From, Reply) + end; Other -> handle_common_reply(Other, Parent, Name, From, Msg, Mod, HibernateAfterTimeout, State) end; handle_msg(Msg, Parent, Name, State, Mod, HibernateAfterTimeout) -> @@ -712,10 +704,11 @@ handle_msg({'$gen_call', From, Msg}, Parent, Name, State, Mod, HibernateAfterTim {noreply, NState}), loop(Parent, Name, NState, Mod, Time1, HibernateAfterTimeout, Debug1); {ok, {stop, Reason, Reply, NState}} -> - {'EXIT', R} = - (catch terminate(Reason, Name, From, Msg, Mod, NState, Debug)), - _ = reply(Name, From, Reply, NState, Debug), - exit(R); + try + terminate(Reason, ?STACKTRACE(), Name, From, Msg, Mod, NState, Debug) + after + _ = reply(Name, From, Reply, NState, Debug) + end; Other -> handle_common_reply(Other, Parent, Name, From, Msg, Mod, HibernateAfterTimeout, State, Debug) end; @@ -730,11 +723,11 @@ handle_common_reply(Reply, Parent, Name, From, Msg, Mod, HibernateAfterTimeout, {ok, {noreply, NState, Time1}} -> loop(Parent, Name, NState, Mod, Time1, HibernateAfterTimeout, []); {ok, {stop, Reason, NState}} -> - terminate(Reason, Name, From, Msg, Mod, NState, []); - {'EXIT', ExitReason, ReportReason} -> - terminate(ExitReason, ReportReason, Name, From, Msg, Mod, State, []); + terminate(Reason, ?STACKTRACE(), Name, From, Msg, Mod, NState, []); + {'EXIT', Class, Reason, Stacktrace} -> + terminate(Class, Reason, Stacktrace, Name, From, Msg, Mod, State, []); {ok, BadReply} -> - terminate({bad_return_value, BadReply}, Name, From, Msg, Mod, State, []) + terminate({bad_return_value, BadReply}, ?STACKTRACE(), Name, From, Msg, Mod, State, []) end. handle_common_reply(Reply, Parent, Name, From, Msg, Mod, HibernateAfterTimeout, State, Debug) -> @@ -748,11 +741,11 @@ handle_common_reply(Reply, Parent, Name, From, Msg, Mod, HibernateAfterTimeout, {noreply, NState}), loop(Parent, Name, NState, Mod, Time1, HibernateAfterTimeout, Debug1); {ok, {stop, Reason, NState}} -> - terminate(Reason, Name, From, Msg, Mod, NState, Debug); - {'EXIT', ExitReason, ReportReason} -> - terminate(ExitReason, ReportReason, Name, From, Msg, Mod, State, Debug); + terminate(Reason, ?STACKTRACE(), Name, From, Msg, Mod, NState, Debug); + {'EXIT', Class, Reason, Stacktrace} -> + terminate(Class, Reason, Stacktrace, Name, From, Msg, Mod, State, Debug); {ok, BadReply} -> - terminate({bad_return_value, BadReply}, Name, From, Msg, Mod, State, Debug) + terminate({bad_return_value, BadReply}, ?STACKTRACE(), Name, From, Msg, Mod, State, Debug) end. reply(Name, {To, Tag}, Reply, State, Debug) -> @@ -770,7 +763,7 @@ system_continue(Parent, Debug, [Name, State, Mod, Time, HibernateAfterTimeout]) -spec system_terminate(_, _, _, [_]) -> no_return(). system_terminate(Reason, _Parent, Debug, [Name, State, Mod, _Time, _HibernateAfterTimeout]) -> - terminate(Reason, Name, undefined, [], Mod, State, Debug). + terminate(Reason, ?STACKTRACE(), Name, undefined, [], Mod, State, Debug). system_code_change([Name, State, Mod, Time, HibernateAfterTimeout], _Module, OldVsn, Extra) -> case catch Mod:code_change(OldVsn, State, Extra) of @@ -811,35 +804,58 @@ print_event(Dev, Event, Name) -> %%% --------------------------------------------------- %%% Terminate the server. +%%% +%%% terminate/8 is triggered by {stop, Reason} or bad +%%% return values. The stacktrace is generated via the +%%% ?STACKTRACE() macro and the ReportReason must not +%%% be wrapped in tuples. +%%% +%%% terminate/9 is triggered in case of error/exit in +%%% the user callback. In this case the report reason +%%% always includes the user stacktrace. +%%% +%%% The reason received in the terminate/2 callbacks +%%% always includes the stacktrace for errors and never +%%% for exits. %%% --------------------------------------------------- - --spec terminate(_, _, _, _, _, _, _) -> no_return(). -terminate(Reason, Name, From, Msg, Mod, State, Debug) -> - terminate(Reason, Reason, Name, From, Msg, Mod, State, Debug). -spec terminate(_, _, _, _, _, _, _, _) -> no_return(). -terminate(ExitReason, ReportReason, Name, From, Msg, Mod, State, Debug) -> - Reply = try_terminate(Mod, ExitReason, State), +terminate(Reason, Stacktrace, Name, From, Msg, Mod, State, Debug) -> + terminate(exit, Reason, Stacktrace, Reason, Name, From, Msg, Mod, State, Debug). + +-spec terminate(_, _, _, _, _, _, _, _, _) -> no_return(). +terminate(Class, Reason, Stacktrace, Name, From, Msg, Mod, State, Debug) -> + ReportReason = {Reason, Stacktrace}, + terminate(Class, Reason, Stacktrace, ReportReason, Name, From, Msg, Mod, State, Debug). + +-spec terminate(_, _, _, _, _, _, _, _, _, _) -> no_return(). +terminate(Class, Reason, Stacktrace, ReportReason, Name, From, Msg, Mod, State, Debug) -> + Reply = try_terminate(Mod, terminate_reason(Class, Reason, Stacktrace), State), case Reply of - {'EXIT', ExitReason1, ReportReason1} -> + {'EXIT', C, R, S} -> FmtState = format_status(terminate, Mod, get(), State), - error_info(ReportReason1, Name, From, Msg, FmtState, Debug), - exit(ExitReason1); + error_info({R, S}, Name, From, Msg, FmtState, Debug), + erlang:raise(C, R, S); _ -> - case ExitReason of - normal -> - exit(normal); - shutdown -> - exit(shutdown); - {shutdown,_}=Shutdown -> - exit(Shutdown); + case {Class, Reason} of + {exit, normal} -> ok; + {exit, shutdown} -> ok; + {exit, {shutdown,_}} -> ok; _ -> FmtState = format_status(terminate, Mod, get(), State), - error_info(ReportReason, Name, From, Msg, FmtState, Debug), - exit(ExitReason) + error_info(ReportReason, Name, From, Msg, FmtState, Debug) end + end, + case Stacktrace of + [] -> + erlang:Class(Reason); + _ -> + erlang:raise(Class, Reason, Stacktrace) end. +terminate_reason(error, Reason, Stacktrace) -> {Reason, Stacktrace}; +terminate_reason(exit, Reason, _Stacktrace) -> Reason. + error_info(_Reason, application_controller, _From, _Msg, _State, _Debug) -> %% OTP-5811 Don't send an error report if it's the system process %% application_controller which is terminating - let init take care @@ -861,14 +877,15 @@ error_info(Reason, Name, From, Msg, State, Debug) -> end end; _ -> - Reason + error_logger:limit_term(Reason) end, {ClientFmt, ClientArgs} = client_stacktrace(From), - format("** Generic server ~p terminating \n" - "** Last message in was ~p~n" - "** When Server state == ~p~n" - "** Reason for termination == ~n** ~p~n" ++ ClientFmt, - [Name, Msg, State, Reason1] ++ ClientArgs), + LimitedState = error_logger:limit_term(State), + error_logger:format("** Generic server ~p terminating \n" + "** Last message in was ~p~n" + "** When Server state == ~p~n" + "** Reason for termination == ~n** ~p~n" ++ ClientFmt, + [Name, Msg, LimitedState, Reason1] ++ ClientArgs), sys:print_log(Debug), ok. client_stacktrace(undefined) -> diff --git a/lib/stdlib/src/gen_statem.erl b/lib/stdlib/src/gen_statem.erl index 86109f04b4..b5e9da1e66 100644 --- a/lib/stdlib/src/gen_statem.erl +++ b/lib/stdlib/src/gen_statem.erl @@ -1722,6 +1722,8 @@ error_info( end; _ -> {Reason,Stacktrace} end, + [LimitedP, LimitedFmtData, LimitedFixedReason] = + [error_logger:limit_term(D) || D <- [P, FmtData, FixedReason]], CBMode = case StateEnter of true -> @@ -1755,8 +1757,8 @@ error_info( [] -> []; [Event|_] -> [Event] end] ++ - [FmtData, - Class,FixedReason, + [LimitedFmtData, + Class,LimitedFixedReason, CBMode] ++ case Q of [_|[_|_] = Events] -> [Events]; @@ -1764,7 +1766,7 @@ error_info( end ++ case P of [] -> []; - _ -> [P] + _ -> [LimitedP] end ++ case FixedStacktrace of [] -> []; diff --git a/lib/stdlib/src/io_lib.erl b/lib/stdlib/src/io_lib.erl index 5ed2f4d888..9d447418f8 100644 --- a/lib/stdlib/src/io_lib.erl +++ b/lib/stdlib/src/io_lib.erl @@ -84,6 +84,8 @@ -export([write_unicode_string/1, write_unicode_char/1, deep_unicode_char_list/1]). +-export([limit_term/2]). + -export_type([chars/0, latin1_string/0, continuation/0, fread_error/0, fread_item/0, format_spec/0]). @@ -911,3 +913,116 @@ binrev(L) -> binrev(L, T) -> list_to_binary(lists:reverse(L, T)). + +-spec limit_term(term(), non_neg_integer()) -> term(). + +%% The intention is to mimic the depth limitation of io_lib:write() +%% and io_lib_pretty:print(). The leaves ('...') should never be +%% seen when printed with the same depth. Bitstrings are never +%% truncated, which is OK as long as they are not sent to other nodes. +limit_term(Term, Depth) -> + try test_limit(Term, Depth) of + ok -> Term + catch + throw:limit -> + limit(Term, Depth) + end. + +limit(_, 0) -> '...'; +limit([H|T]=L, D) -> + if + D =:= 1 -> '...'; + true -> + case printable_list(L) of + true -> L; + false -> + [limit(H, D-1)|limit_tail(T, D-1)] + end + end; +limit(Term, D) when is_map(Term) -> + limit_map(Term, D); +limit({}=T, _D) -> T; +limit(T, D) when is_tuple(T) -> + if + D =:= 1 -> '...'; + true -> + list_to_tuple([limit(element(1, T), D-1)| + limit_tail(tl(tuple_to_list(T)), D-1)]) + end; +limit(<<_/bitstring>>=Term, D) -> limit_bitstring(Term, D); +limit(Term, _D) -> Term. + +limit_tail([], _D) -> []; +limit_tail(_, 1) -> ['...']; +limit_tail([H|T], D) -> + [limit(H, D-1)|limit_tail(T, D-1)]; +limit_tail(Other, D) -> + limit(Other, D-1). + +%% Cannot limit maps properly since there is no guarantee that +%% maps:from_list() creates a map with the same internal ordering of +%% the selected associations as in Map. +limit_map(Map, D) -> + maps:from_list(erts_internal:maps_to_list(Map, D)). +%% maps:from_list(limit_map_body(erts_internal:maps_to_list(Map, D), D)). + +%% limit_map_body(_, 0) -> [{'...', '...'}]; +%% limit_map_body([], _) -> []; +%% limit_map_body([{K,V}], D) -> [limit_map_assoc(K, V, D)]; +%% limit_map_body([{K,V}|KVs], D) -> +%% [limit_map_assoc(K, V, D) | limit_map_body(KVs, D-1)]. + +%% limit_map_assoc(K, V, D) -> +%% {limit(K, D-1), limit(V, D-1)}. + +limit_bitstring(B, _D) -> B. %% Keeps all printable binaries. + +test_limit(_, 0) -> throw(limit); +test_limit([H|T]=L, D) when is_integer(D) -> + if + D =:= 1 -> throw(limit); + true -> + case printable_list(L) of + true -> ok; + false -> + test_limit(H, D-1), + test_limit_tail(T, D-1) + end + end; +test_limit(Term, D) when is_map(Term) -> + test_limit_map(Term, D); +test_limit({}, _D) -> ok; +test_limit(T, D) when is_tuple(T) -> + test_limit_tuple(T, 1, tuple_size(T), D); +test_limit(<<_/bitstring>>=Term, D) -> test_limit_bitstring(Term, D); +test_limit(_Term, _D) -> ok. + +test_limit_tail([], _D) -> ok; +test_limit_tail(_, 1) -> throw(limit); +test_limit_tail([H|T], D) -> + test_limit(H, D-1), + test_limit_tail(T, D-1); +test_limit_tail(Other, D) -> + test_limit(Other, D-1). + +test_limit_tuple(_T, I, Sz, _D) when I > Sz -> ok; +test_limit_tuple(_, _, _, 1) -> throw(limit); +test_limit_tuple(T, I, Sz, D) -> + test_limit(element(I, T), D-1), + test_limit_tuple(T, I+1, Sz, D-1). + +test_limit_map(_Map, _D) -> ok. +%% test_limit_map_body(erts_internal:maps_to_list(Map, D), D). + +%% test_limit_map_body(_, 0) -> throw(limit); +%% test_limit_map_body([], _) -> ok; +%% test_limit_map_body([{K,V}], D) -> test_limit_map_assoc(K, V, D); +%% test_limit_map_body([{K,V}|KVs], D) -> +%% test_limit_map_assoc(K, V, D), +%% test_limit_map_body(KVs, D-1). + +%% test_limit_map_assoc(K, V, D) -> +%% test_limit(K, D-1), +%% test_limit(V, D-1). + +test_limit_bitstring(_, _) -> ok. diff --git a/lib/stdlib/src/proc_lib.erl b/lib/stdlib/src/proc_lib.erl index 2219467a8d..3fa54cd0d5 100644 --- a/lib/stdlib/src/proc_lib.erl +++ b/lib/stdlib/src/proc_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -519,7 +519,8 @@ my_info_1(Class, Reason, Stacktrace) -> get_process_info(self(), registered_name), {error_info, {Class,Reason,Stacktrace}}, get_ancestors(self()), - get_process_info(self(), messages), + get_process_info(self(), message_queue_len), + get_messages(self()), get_process_info(self(), links), get_cleaned_dictionary(self()), get_process_info(self(), trap_exit), @@ -539,12 +540,49 @@ get_ancestors(Pid) -> {ancestors,[]} end. +%% The messages and the dictionary are possibly limited too much if +%% some error handles output the messages or the dictionary using ~P +%% or ~W with depth greater than the depth used here (the depth of +%% control characters P and W takes precedence over the depth set by +%% application variable error_logger_format_depth). However, it is +%% assumed that all report handlers call proc_lib:format(). +get_messages(Pid) -> + Messages = get_process_messages(Pid), + {messages, error_logger:limit_term(Messages)}. + +get_process_messages(Pid) -> + Depth = error_logger:get_format_depth(), + case Pid =/= self() orelse Depth =:= unlimited of + true -> + {messages, Messages} = get_process_info(Pid, messages), + Messages; + false -> + %% If there are more messages than Depth, garbage + %% collection can sometimes be avoided by collecting just + %% enough messages for the crash report. It is assumed the + %% process is about to die anyway. + receive_messages(Depth) + end. + +receive_messages(0) -> []; +receive_messages(N) -> + receive + M -> + [M|receive_messages(N - 1)] + after 0 -> + [] + end. + get_cleaned_dictionary(Pid) -> case get_process_info(Pid,dictionary) of - {dictionary,Dict} -> {dictionary,clean_dict(Dict)}; + {dictionary,Dict} -> {dictionary,cleaned_dict(Dict)}; _ -> {dictionary,[]} end. +cleaned_dict(Dict) -> + CleanDict = clean_dict(Dict), + error_logger:limit_term(CleanDict). + clean_dict([{'$ancestors',_}|Dict]) -> clean_dict(Dict); clean_dict([{'$initial_call',_}|Dict]) -> @@ -582,20 +620,24 @@ make_neighbour_reports1([P|Ps]) -> make_neighbour_reports1([]) -> []. +%% Do not include messages or process dictionary, even if +%% error_logger_format_depth is unlimited. make_neighbour_report(Pid) -> [{pid, Pid}, get_process_info(Pid, registered_name), get_initial_call(Pid), get_process_info(Pid, current_function), get_ancestors(Pid), - get_process_info(Pid, messages), + get_process_info(Pid, message_queue_len), + %% get_messages(Pid), get_process_info(Pid, links), - get_cleaned_dictionary(Pid), + %% get_cleaned_dictionary(Pid), get_process_info(Pid, trap_exit), get_process_info(Pid, status), get_process_info(Pid, heap_size), get_process_info(Pid, stack_size), - get_process_info(Pid, reductions) + get_process_info(Pid, reductions), + get_process_info(Pid, current_stacktrace) ]. get_initial_call(Pid) -> @@ -722,24 +764,37 @@ format(CrashReport, Encoding) -> format([OwnReport,LinkReport], Encoding, Depth) -> Extra = {Encoding,Depth}, - OwnFormat = format_report(OwnReport, Extra), - LinkFormat = format_report(LinkReport, Extra), + MyIndent = " ", + OwnFormat = format_report(OwnReport, MyIndent, Extra), + LinkFormat = format_link_report(LinkReport, MyIndent, Extra), Str = io_lib:format(" crasher:~n~ts neighbours:~n~ts", [OwnFormat, LinkFormat]), lists:flatten(Str). -format_report(Rep, Extra) when is_list(Rep) -> - format_rep(Rep, Extra); -format_report(Rep, {Enc,_}) -> - io_lib:format("~"++modifier(Enc)++"p~n", [Rep]). - -format_rep([{initial_call,InitialCall}|Rep], {_Enc,Depth}=Extra) -> - [format_mfa(InitialCall, Depth)|format_rep(Rep, Extra)]; -format_rep([{error_info,{Class,Reason,StackTrace}}|Rep], Extra) -> - [format_exception(Class, Reason, StackTrace, Extra)|format_rep(Rep, Extra)]; -format_rep([{Tag,Data}|Rep], Extra) -> - [format_tag(Tag, Data, Extra)|format_rep(Rep, Extra)]; -format_rep(_, _Extra) -> +format_link_report([Link|Reps], Indent, Extra) -> + Rep = case Link of + {neighbour,Rep0} -> Rep0; + _ -> Link + end, + LinkIndent = [" ",Indent], + [Indent,"neighbour:\n",format_report(Rep, LinkIndent, Extra)| + format_link_report(Reps, Indent, Extra)]; +format_link_report([], _, _) -> + []. + +format_report(Rep, Indent, Extra) when is_list(Rep) -> + format_rep(Rep, Indent, Extra); +format_report(Rep, Indent, {Enc,Depth}) -> + io_lib:format("~s~"++modifier(Enc)++"P~n", [Indent, Rep, Depth]). + +format_rep([{initial_call,InitialCall}|Rep], Indent, Extra) -> + [format_mfa(Indent, InitialCall, Extra)|format_rep(Rep, Indent, Extra)]; +format_rep([{error_info,{Class,Reason,StackTrace}}|Rep], Indent, Extra) -> + [format_exception(Class, Reason, StackTrace, Extra)| + format_rep(Rep, Indent, Extra)]; +format_rep([{Tag,Data}|Rep], Indent, Extra) -> + [format_tag(Indent, Tag, Data, Extra)|format_rep(Rep, Indent, Extra)]; +format_rep(_, _, _Extra) -> []. format_exception(Class, Reason, StackTrace, {Enc,_}=Extra) -> @@ -750,14 +805,14 @@ format_exception(Class, Reason, StackTrace, {Enc,_}=Extra) -> [EI, lib:format_exception(1+length(EI), Class, Reason, StackTrace, StackFun, PF, Enc), "\n"]. -format_mfa({M,F,Args}=StartF, Depth) -> +format_mfa(Indent, {M,F,Args}=StartF, Extra) -> try A = length(Args), - [" initial call: ",atom_to_list(M),$:,atom_to_list(F),$/, + [Indent,"initial call: ",atom_to_list(M),$:,atom_to_list(F),$/, integer_to_list(A),"\n"] catch error:_ -> - format_tag(initial_call, StartF, Depth) + format_tag(Indent, initial_call, StartF, Extra) end. pp_fun({Enc,Depth}) -> @@ -770,12 +825,12 @@ pp_fun({Enc,Depth}) -> io_lib:format("~." ++ integer_to_list(I) ++ P, [Term|Tl]) end. -format_tag(Tag, Data, {_Enc,Depth}) -> +format_tag(Indent, Tag, Data, {_Enc,Depth}) -> case Depth of unlimited -> - io_lib:format(" ~p: ~80.18p~n", [Tag, Data]); + io_lib:format("~s~p: ~80.18p~n", [Indent, Tag, Data]); _ -> - io_lib:format(" ~p: ~80.18P~n", [Tag, Data, Depth]) + io_lib:format("~s~p: ~80.18P~n", [Indent, Tag, Data, Depth]) end. modifier(latin1) -> ""; diff --git a/lib/stdlib/test/erl_lint_SUITE.erl b/lib/stdlib/test/erl_lint_SUITE.erl index 02524679fa..cc3d605840 100644 --- a/lib/stdlib/test/erl_lint_SUITE.erl +++ b/lib/stdlib/test/erl_lint_SUITE.erl @@ -3978,7 +3978,11 @@ otp_14323(Config) -> {13,erl_lint,{undefined_function,{a,1}}}, {14,erl_lint,{bad_dialyzer_attribute, {nowarn_function,{a,-1}}}}], - []}}], + []}}, + {otp_14323_2, + <<"-type t(_) :: atom().">>, + [], + {errors,[{1,erl_parse,"bad type variable"}],[]}}], [] = run(Config, Ts), ok. diff --git a/lib/stdlib/test/gen_server_SUITE.erl b/lib/stdlib/test/gen_server_SUITE.erl index 7e3c71715e..2e9dc4d4fb 100644 --- a/lib/stdlib/test/gen_server_SUITE.erl +++ b/lib/stdlib/test/gen_server_SUITE.erl @@ -404,7 +404,7 @@ crash(Config) when is_list(Config) -> end, receive {error_report,_,{Pid4,crash_report,[List4|_]}} -> - {exit,crashed,_} = proplists:get_value(error_info, List4), + {exit,crashed,[{?MODULE, handle_call, 3, _}|_]} = proplists:get_value(error_info, List4), Pid4 = proplists:get_value(pid, List4); Other4 -> io:format("Unexpected: ~p", [Other4]), diff --git a/lib/stdlib/test/io_SUITE.erl b/lib/stdlib/test/io_SUITE.erl index ef3f0be5d7..e2c73371cd 100644 --- a/lib/stdlib/test/io_SUITE.erl +++ b/lib/stdlib/test/io_SUITE.erl @@ -31,7 +31,7 @@ otp_10836/1, io_lib_width_too_small/1, io_with_huge_message_queue/1, format_string/1, maps/1, coverage/1, otp_14178_unicode_atoms/1, otp_14175/1, - otp_14285/1]). + otp_14285/1, limit_term/1]). -export([pretty/2]). @@ -63,7 +63,7 @@ all() -> io_lib_print_binary_depth_one, otp_10302, otp_10755, otp_10836, io_lib_width_too_small, io_with_huge_message_queue, format_string, maps, coverage, otp_14178_unicode_atoms, otp_14175, - otp_14285]. + otp_14285, limit_term]. %% Error cases for output. error_1(Config) when is_list(Config) -> @@ -2373,3 +2373,58 @@ otp_14285(_Config) -> latin1_fmt(Fmt, Args) -> L = fmt(Fmt, Args), true = lists:all(fun is_latin1/1, L). + +limit_term(_Config) -> + {_, 2} = limt([a,b,c], 2), + {_, 2} = limt([a,b,c], 3), + {_, 2} = limt([a,b|c], 2), + {_, 2} = limt([a,b|c], 3), + {_, 2} = limt({a,b,c,[d,e]}, 2), + {_, 2} = limt({a,b,c,[d,e]}, 3), + {_, 2} = limt({a,b,c,[d,e]}, 4), + {_, 1} = limt(<<"foo">>, 18), + ok = blimt(<<"123456789012345678901234567890">>), + {_, 1} = limt(<<7:3>>, 2), + {_, 1} = limt(<<7:21>>, 2), + {_, 1} = limt([], 2), + {_, 1} = limt({}, 2), + {_, 1} = limt(#{}, 2), + {_, 1} = limt(#{[] => {}}, 2), + {_, 1} = limt(#{[] => {}}, 3), + T = #{[] => {},[a] => [b]}, + {_, 1} = limt(T, 2), + {_, 1} = limt(T, 3), + {_, 1} = limt(T, 4), + ok. + +blimt(Binary) -> + blimt(Binary, byte_size(Binary)). + +blimt(_B, 1) -> ok; +blimt(B, D) -> + {_, 1} = limt(B, D), + blimt(B, D - 1). + +limt(Term, Depth) when is_integer(Depth) -> + T1 = io_lib:limit_term(Term, Depth), + S = form(Term, Depth), + S1 = form(T1, Depth), + OK1 = S1 =:= S, + + T2 = io_lib:limit_term(Term, Depth+1), + S2 = form(T2, Depth), + OK2 = S2 =:= S, + + T3 = io_lib:limit_term(Term, Depth-1), + S3 = form(T3, Depth), + OK3 = S3 =/= S, + + R = case {OK1, OK2, OK3} of + {true, true, true} -> 2; + {true, true, false} -> 1; + _ -> 0 + end, + {{S, S1, S2}, R}. + +form(Term, Depth) -> + lists:flatten(io_lib:format("~W", [Term, Depth])). diff --git a/system/doc/design_principles/statem.xml b/system/doc/design_principles/statem.xml index 0667af7868..7febe31df3 100644 --- a/system/doc/design_principles/statem.xml +++ b/system/doc/design_principles/statem.xml @@ -1582,10 +1582,12 @@ format_status(Opt, [_PDict,State,Data]) -> for example, a complex state term like a tuple. </p> <p> - One reason to use this is when you have - a state item that affects the event handling, - in particular in combination with postponing events. - We complicate the previous example + One reason to use this is when you have a state item + that when changed should cancel the + <seealso marker="#State Time-Outs">state time-out</seealso>, + or one that affects the event handling + in combination with postponing events. + We will complicate the previous example by introducing a configurable lock button (this is the state item in question), which in the <c>open</c> state immediately locks the door, diff --git a/system/doc/getting_started/conc_prog.xml b/system/doc/getting_started/conc_prog.xml index f3136898ad..4b19095d95 100644 --- a/system/doc/getting_started/conc_prog.xml +++ b/system/doc/getting_started/conc_prog.xml @@ -355,8 +355,8 @@ pong ! {ping, self()},</code> <p>Let us rewrite the ping pong program with "ping" and "pong" on different computers. First a few things are needed to set up to get this to work. The distributed Erlang - implementation provides a basic security mechanism to prevent - unauthorized access to an Erlang system on another computer. + implementation provides a very basic authentication mechanism to prevent + unintentional access to an Erlang system on another computer. Erlang systems which talk to each other must have the same <em>magic cookie</em>. The easiest way to achieve this is by having a file called <c>.erlang.cookie</c> in your home diff --git a/system/doc/reference_manual/distributed.xml b/system/doc/reference_manual/distributed.xml index 0a4a323fe9..01d78436c5 100644 --- a/system/doc/reference_manual/distributed.xml +++ b/system/doc/reference_manual/distributed.xml @@ -42,6 +42,19 @@ <p>The distribution mechanism is implemented using TCP/IP sockets. How to implement an alternative carrier is described in the <seealso marker="erts:alt_dist">ERTS User's Guide</seealso>.</p> + <warning> + <p> + Starting a distributed node without also specifying + <seealso marker="erts:erl#proto_dist"><c>-proto_dist inet_tls</c></seealso> + will expose the node to attacks that may give the attacker + complete access to the node and in extension the cluster. + When using un-secure distributed nodes, make sure that the + network is configured to keep potential attackers out. + See the <seealso marker="ssl:ssl_distribution"> + Using SSL for Erlang Distribution</seealso> User's Guide + for details on how to setup a secure distributed node. + </p> + </warning> </section> <section> |