From f9459092940943876dff040ee997515b96fd5d50 Mon Sep 17 00:00:00 2001 From: Rickard Green Date: Wed, 4 Jan 2017 18:10:26 +0100 Subject: Scheduler wall time support for dirty schedulers --- erts/doc/src/erlang.xml | 97 +++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 85 insertions(+), 12 deletions(-) (limited to 'erts/doc') diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index b3fab3874b..1f64d7be86 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -6415,12 +6415,17 @@ lists:map( TotalTime is the total time duration since scheduler_wall_time - activation. The time unit is undefined and can be subject - to change between releases, OSs, and system restarts. - scheduler_wall_time is only to be used to - calculate relative values for scheduler-utilization. - ActiveTime can never exceed - TotalTime.

+ activation for the specific scheduler. Note that + activation time can differ significantly between + schedulers. Currently dirty schedulers are activated + at system start while normal schedulers are activated + some time after the scheduler_wall_time + functionality is enabled. The time unit is undefined + and can be subject to change between releases, OSs, + and system restarts. scheduler_wall_time is only + to be used to calculate relative values for scheduler + utilization. ActiveTime can never + exceed TotalTime.

The definition of a busy scheduler is when it is not idle and is not scheduling (selecting) a process or port, that is:

@@ -6438,15 +6443,37 @@ lists:map( scheduler_wall_time is turned off.

The list of scheduler information is unsorted and can appear in different order between calls.

+

As of ERTS version 9.0, also dirty CPU schedulers will + be included in the result. That is, all scheduler threads + that are expected to handle CPU bound work. If you also + want information about dirty I/O schedulers, use + statistics(scheduler_wall_time_all) + instead.

+ +

Normal schedulers will have scheduler identifiers in + the range 1 =< SchedulerId =< + erlang:system_info(schedulers). + Dirty CPU schedulers will have scheduler identifiers in + the range erlang:system_info(schedulers) < + SchedulerId =< erlang:system_info(schedulers) + + + erlang:system_info(dirty_cpu_schedulers). +

+

The different types of schedulers handle + specific types of jobs. Every job is assigned to a specific + scheduler type. Jobs can migrate between different schedulers + of the same type, but never between schedulers of different + types. This fact has to be taken under consideration when + evaluating the result returned.

Using scheduler_wall_time to calculate - scheduler-utilization:

+ scheduler utilization:

 > erlang:system_flag(scheduler_wall_time, true).
 false
 > Ts0 = lists:sort(erlang:statistics(scheduler_wall_time)), ok.
 ok

Some time later the user takes another snapshot and calculates - scheduler-utilization per scheduler, for example:

+ scheduler utilization per scheduler, for example:

 > Ts1 = lists:sort(erlang:statistics(scheduler_wall_time)), ok.
 ok
@@ -6461,11 +6488,32 @@ ok
  {7,0.973237033077876},
  {8,0.9741297293248656}]

Using the same snapshots to calculate a total - scheduler-utilization:

+ scheduler utilization:

 > {A, T} = lists:foldl(fun({{_, A0, T0}, {_, A1, T1}}, {Ai,Ti}) ->
-	{Ai + (A1 - A0), Ti + (T1 - T0)} end, {0, 0}, lists:zip(Ts0,Ts1)), A/T.
+	{Ai + (A1 - A0), Ti + (T1 - T0)} end, {0, 0}, lists:zip(Ts0,Ts1)),
+	TotalSchedulerUtilization = A/T.
+0.9769136803764825
+

Total scheduler utilization will equal 1.0 when + all schedulers have been active all the time between the + two measurements.

+

Another (probably more) useful value is to calculate + total scheduler utilization weighted against maximum amount + of available CPU time:

+ 
+> WeightedSchedulerUtilization = (TotalSchedulerUtilization
+                                  * (erlang:system_info(schedulers)
+                                     + erlang:system_info(dirty_cpu_schedulers)))
+                                 / erlang:system_info(logical_processors_available).
 0.9769136803764825
+

This weighted scheduler utilization will reach 1.0 + when schedulers are active the same amount of time as + maximum available CPU time. If more schedulers exist + than available logical processors, this value may + be greater than 1.0.

+

As of ERTS version 9.0, the Erlang runtime system + with SMP support will as default have more schedulers + than logical processors. This due to the dirty schedulers.

scheduler_wall_time is by default disabled. To enable it, use @@ -6476,6 +6524,31 @@ ok + Information about each schedulers work time. + + +

The same as + statistics(scheduler_wall_time), + except that it also include information about all dirty I/O + schedulers.

+

Dirty IO schedulers will have scheduler identifiers in + the range + erlang:system_info(schedulers) + + + erlang:system_info(dirty_cpu_schedulers) < + SchedulerId =< erlang:system_info(schedulers) + + erlang:system_info(dirty_cpu_schedulers) + + + erlang:system_info(dirty_io_schedulers).

+

Note that work executing on dirty I/O schedulers + are expected to mainly wait for I/O. That is, when you + get high scheduler utilization on dirty I/O schedulers, + CPU utilization is not expected to be high due to + this work.

 + + + + Information about active processes and ports.

Returns the total amount of active processes and ports in @@ -6495,7 +6568,7 @@ ok - + Information about the run-queue lengths.

Returns the total length of the run queues. That is, the number @@ -6515,7 +6588,7 @@ ok - + Information about wall clock.

Returns information about wall clock. wall_clock can -- cgit v1.2.3 From 0d20adbfbb0918c8d6bfc629e9e4ff89d2ef1a64 Mon Sep 17 00:00:00 2001 From: Rickard Green Date: Thu, 12 Jan 2017 14:55:53 +0100 Subject: Remove experimental disclaimer on dirty schedulers --- erts/doc/src/erl.xml | 12 +++--------- erts/doc/src/erl_nif.xml | 24 +++++++++--------------- erts/doc/src/erlang.xml | 14 -------------- 3 files changed, 12 insertions(+), 38 deletions(-) (limited to 'erts/doc') diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index 8da832ac37..4e32118405 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -946,9 +946,7 @@ schedulers was allowed to be unlimited, dirty CPU bound jobs would potentially starve normal jobs.

This option is ignored if the emulator does not have threading - support enabled. This option is experimental and - is supported only if the emulator was configured and built with - support for dirty schedulers enabled (it is disabled by default).

+ support enabled.

@@ -974,9 +972,7 @@ either order) results in 2 dirty CPU scheduler threads (50% of 4) and 1 dirty CPU scheduler thread online (25% of 4).

This option is ignored if the emulator does not have threading - support enabled. This option is experimental and - is supported only if the emulator was configured and built with - support for dirty schedulers enabled (it is disabled by default).

+ support enabled.

@@ -992,9 +988,7 @@ bound jobs on dirty I/O schedulers, these jobs might starve ordinary jobs executing on ordinary schedulers.

This option is ignored if the emulator does not have threading - support enabled. This option is experimental and - is supported only if the emulator was configured and built with - support for dirty schedulers enabled (it is disabled by default).

+ support enabled.

 diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml index 185ecd9ed9..51b095e6ef 100644 --- a/erts/doc/src/erl_nif.xml +++ b/erts/doc/src/erl_nif.xml @@ -402,14 +402,14 @@ return term; Dirty NIF -

The dirty NIF functionality described here - is experimental. Dirty NIF support is available only when - the emulator is configured with dirty schedulers enabled. This - feature is disabled by default. The Erlang runtime - without SMP support does not support dirty schedulers even when - the dirty scheduler support is enabled. To check at runtime for - the presence of dirty scheduler threads, code can use the - +

Dirty NIF support is available only when the emulator is + configured with dirty scheduler support. As of ERTS version + 9.0, dirty scheduler support is enabled by default on the + runtime system with SMP support. The Erlang runtime without + SMP support does not support dirty schedulers even + when the dirty scheduler support is explicitly enabled. To + check at runtime for the presence of dirty scheduler threads, + code can use the enif_system_info() API function.

A NIF that cannot be split and cannot execute in a millisecond @@ -642,9 +642,6 @@ typedef struct {

flags can be used to indicate that the NIF is a dirty NIF that is to be executed on a dirty scheduler thread.

-

The dirty NIF functionality described here is - experimental. You have to enable support for dirty - schedulers when building OTP to try out the functionality.

If the dirty NIF is expected to be CPU-bound, its flags field is to be set to ERL_NIF_DIRTY_JOB_CPU_BOUND or ERL_NIF_DIRTY_JOB_IO_BOUND.

@@ -2450,9 +2447,6 @@ enif_map_iterator_destroy(env, &iter); application to break up long-running work into multiple regular NIF calls or to schedule a dirty NIF to execute on a dirty scheduler thread.

-

The dirty NIF functionality described here is - experimental. You have to enable support for dirty - schedulers when building OTP to try out the functionality.

fun_name @@ -2463,7 +2457,7 @@ enif_map_iterator_destroy(env, &iter); flags

Must be set to 0 for a regular NIF. If the emulator was - built with the experimental dirty scheduler support enabled, + built with dirty scheduler support enabled, flags can be set to either ERL_NIF_DIRTY_JOB_CPU_BOUND if the job is expected to be CPU-bound, or ERL_NIF_DIRTY_JOB_IO_BOUND for diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 1f64d7be86..7815bfa510 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -6794,11 +6794,6 @@ ok down to 3. Similarly, the number of dirty CPU schedulers online increases proportionally to increases in the number of schedulers online.

- -

The dirty schedulers functionality is experimental. - Enable support for dirty schedulers when building OTP to - try out the functionality.

-

For more information, see erlang:system_info(dirty_cpu_schedulers) and @@ -7643,9 +7638,6 @@ ok +SDcpu or +SDPcpu in erl(1).

-

Notice that the dirty schedulers functionality is - experimental. Enable support for dirty schedulers when - building OTP to try out the functionality.

See also erlang:system_flag(dirty_cpu_schedulers_online, @@ -7675,9 +7667,6 @@ ok startup by passing command-line flag +SDcpu in erl(1).

-

Notice that the dirty schedulers functionality is - experimental. Enable support for dirty schedulers when - building OTP to try out the functionality.

For more information, see erlang:system_info(dirty_cpu_schedulers), @@ -7699,9 +7688,6 @@ ok

This value can be set at startup by passing command-line argument +SDio in erl(1).

-

Notice that the dirty schedulers functionality is - experimental. Enable support for dirty schedulers when - building OTP to try out the functionality.

For more information, see erlang:system_info(dirty_cpu_schedulers), -- cgit v1.2.3