aboutsummaryrefslogtreecommitdiffstats
path: root/erts/doc/src
diff options
context:
space:
mode:
authorRickard Green <[email protected]>2017-01-17 12:39:55 +0100
committerRickard Green <[email protected]>2017-01-17 12:39:55 +0100
commit8e77d78810c2b77a81e261234b351d30e374d629 (patch)
treedb5c8dc37898b5c8e8bcd9d91b1c1e35ca3ff4dc /erts/doc/src
parent894cceaa2191bbb59474c1a1bfc657493c5ea79e (diff)
parent0d20adbfbb0918c8d6bfc629e9e4ff89d2ef1a64 (diff)
downloadotp-8e77d78810c2b77a81e261234b351d30e374d629.tar.gz
otp-8e77d78810c2b77a81e261234b351d30e374d629.tar.bz2
otp-8e77d78810c2b77a81e261234b351d30e374d629.zip
Merge branch 'rickard/ds-features-20'
OTP-14152 * rickard/ds-features-20: Remove experimental disclaimer on dirty schedulers Scheduler wall time support for dirty schedulers Always return info from system_info(dirty_[cpu|io]_schedulers*) Improve etp-commands Switch between scheduler types when multi-scheduling is blocked Perform potentially long GC on dirty schedulers if available Return and exception trace for nif-export scheduled BIFs Optimize handling of BIF errors Support for dirty BIFs Reduction counting on non-tail return
Diffstat (limited to 'erts/doc/src')
-rw-r--r--erts/doc/src/erl.xml12
-rw-r--r--erts/doc/src/erl_nif.xml24
-rw-r--r--erts/doc/src/erlang.xml111
3 files changed, 97 insertions, 50 deletions
diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml
index 8da832ac37..4e32118405 100644
--- a/erts/doc/src/erl.xml
+++ b/erts/doc/src/erl.xml
@@ -946,9 +946,7 @@
schedulers was allowed to be unlimited, dirty CPU bound jobs would
potentially starve normal jobs.</p>
<p>This option is ignored if the emulator does not have threading
- support enabled. <em>This option is experimental</em> and
- is supported only if the emulator was configured and built with
- support for dirty schedulers enabled (it is disabled by default).</p>
+ support enabled.</p>
</item>
<tag><marker id="+SDPcpu"/><c><![CDATA[+SDPcpu
DirtyCPUSchedulersPercentage:DirtyCPUSchedulersOnlinePercentage]]></c></tag>
@@ -974,9 +972,7 @@
either order) results in 2 dirty CPU scheduler threads (50% of 4) and
1 dirty CPU scheduler thread online (25% of 4).</p>
<p>This option is ignored if the emulator does not have threading
- support enabled. <em>This option is experimental</em> and
- is supported only if the emulator was configured and built with
- support for dirty schedulers enabled (it is disabled by default).</p>
+ support enabled.</p>
</item>
<tag><marker id="+SDio"/><c><![CDATA[+SDio DirtyIOSchedulers]]></c></tag>
<item>
@@ -992,9 +988,7 @@
bound jobs on dirty I/O schedulers, these jobs might starve ordinary
jobs executing on ordinary schedulers.</p>
<p>This option is ignored if the emulator does not have threading
- support enabled. <em>This option is experimental</em> and
- is supported only if the emulator was configured and built with
- support for dirty schedulers enabled (it is disabled by default).</p>
+ support enabled.</p>
</item>
<tag><c><![CDATA[+sFlag Value]]></c></tag>
<item>
diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml
index 185ecd9ed9..51b095e6ef 100644
--- a/erts/doc/src/erl_nif.xml
+++ b/erts/doc/src/erl_nif.xml
@@ -402,14 +402,14 @@ return term;</code>
<tag><marker id="dirty_nifs"/>Dirty NIF</tag>
<item>
<note>
- <p><em>The dirty NIF functionality described here
- is experimental</em>. Dirty NIF support is available only when
- the emulator is configured with dirty schedulers enabled. This
- feature is disabled by default. The Erlang runtime
- without SMP support does not support dirty schedulers even when
- the dirty scheduler support is enabled. To check at runtime for
- the presence of dirty scheduler threads, code can use the
- <seealso marker="#enif_system_info">
+ <p>Dirty NIF support is available only when the emulator is
+ configured with dirty scheduler support. As of ERTS version
+ 9.0, dirty scheduler support is enabled by default on the
+ runtime system with SMP support. The Erlang runtime without
+ SMP support does <em>not</em> support dirty schedulers even
+ when the dirty scheduler support is explicitly enabled. To
+ check at runtime for the presence of dirty scheduler threads,
+ code can use the <seealso marker="#enif_system_info">
<c>enif_system_info()</c></seealso> API function.</p>
</note>
<p>A NIF that cannot be split and cannot execute in a millisecond
@@ -642,9 +642,6 @@ typedef struct {
<p><c>flags</c> can be used to indicate that the NIF is a
<seealso marker="#dirty_nifs">dirty NIF</seealso> that is to be
executed on a dirty scheduler thread.</p>
- <p><em>The dirty NIF functionality described here is
- experimental.</em> You have to enable support for dirty
- schedulers when building OTP to try out the functionality.</p>
<p>If the dirty NIF is expected to be CPU-bound, its <c>flags</c>
field is to be set to <c>ERL_NIF_DIRTY_JOB_CPU_BOUND</c> or
<c>ERL_NIF_DIRTY_JOB_IO_BOUND</c>.</p>
@@ -2450,9 +2447,6 @@ enif_map_iterator_destroy(env, &amp;iter);</code>
application to break up long-running work into multiple regular NIF
calls or to schedule a <seealso marker="#dirty_nifs">
dirty NIF</seealso> to execute on a dirty scheduler thread.</p>
- <p><em>The dirty NIF functionality described here is
- experimental.</em> You have to enable support for dirty
- schedulers when building OTP to try out the functionality.</p>
<taglist>
<tag><c>fun_name</c></tag>
<item>
@@ -2463,7 +2457,7 @@ enif_map_iterator_destroy(env, &amp;iter);</code>
<tag><c>flags</c></tag>
<item>
<p>Must be set to <c>0</c> for a regular NIF. If the emulator was
- built with the experimental dirty scheduler support enabled,
+ built with dirty scheduler support enabled,
<c>flags</c> can be set to either
<c>ERL_NIF_DIRTY_JOB_CPU_BOUND</c> if the job is expected to be
CPU-bound, or <c>ERL_NIF_DIRTY_JOB_IO_BOUND</c> for
diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml
index b3fab3874b..7815bfa510 100644
--- a/erts/doc/src/erlang.xml
+++ b/erts/doc/src/erlang.xml
@@ -6415,12 +6415,17 @@ lists:map(
<c><anno>TotalTime</anno></c> is the total time duration since
<seealso marker="#system_flag_scheduler_wall_time">
<c>scheduler_wall_time</c></seealso>
- activation. The time unit is undefined and can be subject
- to change between releases, OSs, and system restarts.
- <c>scheduler_wall_time</c> is only to be used to
- calculate relative values for scheduler-utilization.
- <c><anno>ActiveTime</anno></c> can never exceed
- <c><anno>TotalTime</anno></c>.</p>
+ activation for the specific scheduler. Note that
+ activation time can differ significantly between
+ schedulers. Currently dirty schedulers are activated
+ at system start while normal schedulers are activated
+ some time after the <c>scheduler_wall_time</c>
+ functionality is enabled. The time unit is undefined
+ and can be subject to change between releases, OSs,
+ and system restarts. <c>scheduler_wall_time</c> is only
+ to be used to calculate relative values for scheduler
+ utilization. <c><anno>ActiveTime</anno></c> can never
+ exceed <c><anno>TotalTime</anno></c>.</p>
<p>The definition of a busy scheduler is when it is not idle
and is not scheduling (selecting) a process or port,
that is:</p>
@@ -6438,15 +6443,37 @@ lists:map(
<c>scheduler_wall_time</c></seealso> is turned off.</p>
<p>The list of scheduler information is unsorted and can
appear in different order between calls.</p>
+ <p>As of ERTS version 9.0, also dirty CPU schedulers will
+ be included in the result. That is, all scheduler threads
+ that are expected to handle CPU bound work. If you also
+ want information about dirty I/O schedulers, use
+ <seealso marker="#statistics_scheduler_wall_time_all"><c>statistics(scheduler_wall_time_all)</c></seealso>
+ instead.</p>
+
+ <p>Normal schedulers will have scheduler identifiers in
+ the range <c>1 =&lt; <anno>SchedulerId</anno> =&lt;
+ </c><seealso marker="#system_info_schedulers"><c>erlang:system_info(schedulers)</c></seealso>.
+ Dirty CPU schedulers will have scheduler identifiers in
+ the range <c>erlang:system_info(schedulers) &lt;
+ <anno>SchedulerId</anno> =&lt; erlang:system_info(schedulers)
+ +
+ </c><seealso marker="#system_info_dirty_cpu_schedulers"><c>erlang:system_info(dirty_cpu_schedulers)</c></seealso>.
+ </p>
+ <note><p>The different types of schedulers handle
+ specific types of jobs. Every job is assigned to a specific
+ scheduler type. Jobs can migrate between different schedulers
+ of the same type, but never between schedulers of different
+ types. This fact has to be taken under consideration when
+ evaluating the result returned.</p></note>
<p>Using <c>scheduler_wall_time</c> to calculate
- scheduler-utilization:</p>
+ scheduler utilization:</p>
<pre>
> <input>erlang:system_flag(scheduler_wall_time, true).</input>
false
> <input>Ts0 = lists:sort(erlang:statistics(scheduler_wall_time)), ok.</input>
ok</pre>
<p>Some time later the user takes another snapshot and calculates
- scheduler-utilization per scheduler, for example:</p>
+ scheduler utilization per scheduler, for example:</p>
<pre>
> <input>Ts1 = lists:sort(erlang:statistics(scheduler_wall_time)), ok.</input>
ok
@@ -6461,11 +6488,32 @@ ok
{7,0.973237033077876},
{8,0.9741297293248656}]</pre>
<p>Using the same snapshots to calculate a total
- scheduler-utilization:</p>
+ scheduler utilization:</p>
<pre>
> <input>{A, T} = lists:foldl(fun({{_, A0, T0}, {_, A1, T1}}, {Ai,Ti}) ->
- {Ai + (A1 - A0), Ti + (T1 - T0)} end, {0, 0}, lists:zip(Ts0,Ts1)), A/T.</input>
+ {Ai + (A1 - A0), Ti + (T1 - T0)} end, {0, 0}, lists:zip(Ts0,Ts1)),
+ TotalSchedulerUtilization = A/T.</input>
+0.9769136803764825</pre>
+ <p>Total scheduler utilization will equal <c>1.0</c> when
+ all schedulers have been active all the time between the
+ two measurements.</p>
+ <p>Another (probably more) useful value is to calculate
+ total scheduler utilization weighted against maximum amount
+ of available CPU time:</p>
+ <pre>
+> <input>WeightedSchedulerUtilization = (TotalSchedulerUtilization
+ * (erlang:system_info(schedulers)
+ + erlang:system_info(dirty_cpu_schedulers)))
+ / erlang:system_info(logical_processors_available).</input>
0.9769136803764825</pre>
+ <p>This weighted scheduler utilization will reach <c>1.0</c>
+ when schedulers are active the same amount of time as
+ maximum available CPU time. If more schedulers exist
+ than available logical processors, this value may
+ be greater than <c>1.0</c>.</p>
+ <p>As of ERTS version 9.0, the Erlang runtime system
+ with SMP support will as default have more schedulers
+ than logical processors. This due to the dirty schedulers.</p>
<note>
<p><c>scheduler_wall_time</c> is by default disabled. To
enable it, use
@@ -6476,6 +6524,31 @@ ok
<func>
<name name="statistics" arity="1" clause_i="12"/>
+ <fsummary>Information about each schedulers work time.</fsummary>
+ <desc>
+ <marker id="statistics_scheduler_wall_time_all"></marker>
+ <p>The same as
+ <seealso marker="#statistics_scheduler_wall_time"><c>statistics(scheduler_wall_time)</c></seealso>,
+ except that it also include information about all dirty I/O
+ schedulers.</p>
+ <p>Dirty IO schedulers will have scheduler identifiers in
+ the range
+ <seealso marker="#system_info_schedulers"><c>erlang:system_info(schedulers)</c></seealso><c>
+ +
+ </c><seealso marker="#system_info_dirty_cpu_schedulers"><c>erlang:system_info(dirty_cpu_schedulers)</c></seealso><c> &lt;
+ <anno>SchedulerId</anno> =&lt; erlang:system_info(schedulers)
+ + erlang:system_info(dirty_cpu_schedulers)
+ +
+ </c><seealso marker="#system_info_dirty_io_schedulers"><c>erlang:system_info(dirty_io_schedulers)</c></seealso>.</p>
+ <note><p>Note that work executing on dirty I/O schedulers
+ are expected to mainly wait for I/O. That is, when you
+ get high scheduler utilization on dirty I/O schedulers,
+ CPU utilization is <em>not</em> expected to be high due to
+ this work.</p></note>
+ </desc>
+ </func>
+ <func>
+ <name name="statistics" arity="1" clause_i="13"/>
<fsummary>Information about active processes and ports.</fsummary>
<desc><marker id="statistics_total_active_tasks"></marker>
<p>Returns the total amount of active processes and ports in
@@ -6495,7 +6568,7 @@ ok
</func>
<func>
- <name name="statistics" arity="1" clause_i="13"/>
+ <name name="statistics" arity="1" clause_i="14"/>
<fsummary>Information about the run-queue lengths.</fsummary>
<desc><marker id="statistics_total_run_queue_lengths"></marker>
<p>Returns the total length of the run queues. That is, the number
@@ -6515,7 +6588,7 @@ ok
</func>
<func>
- <name name="statistics" arity="1" clause_i="14"/>
+ <name name="statistics" arity="1" clause_i="15"/>
<fsummary>Information about wall clock.</fsummary>
<desc>
<p>Returns information about wall clock. <c>wall_clock</c> can
@@ -6721,11 +6794,6 @@ ok
down to 3. Similarly, the number of dirty CPU schedulers
online increases proportionally to increases in the number of
schedulers online.</p>
- <note>
- <p>The dirty schedulers functionality is experimental.
- Enable support for dirty schedulers when building OTP to
- try out the functionality.</p>
- </note>
<p>For more information, see
<seealso marker="#system_info_dirty_cpu_schedulers">
<c>erlang:system_info(dirty_cpu_schedulers)</c></seealso> and
@@ -7570,9 +7638,6 @@ ok
<seealso marker="erts:erl#+SDcpu"><c>+SDcpu</c></seealso> or
<seealso marker="erts:erl#+SDPcpu"><c>+SDPcpu</c></seealso> in
<c>erl(1)</c>.</p>
- <p>Notice that the dirty schedulers functionality is
- experimental. Enable support for dirty schedulers when
- building OTP to try out the functionality.</p>
<p>See also
<seealso marker="#system_flag_dirty_cpu_schedulers_online">
<c>erlang:system_flag(dirty_cpu_schedulers_online,
@@ -7602,9 +7667,6 @@ ok
startup by passing command-line flag
<seealso marker="erts:erl#+SDcpu"><c>+SDcpu</c></seealso> in
<c>erl(1)</c>.</p>
- <p>Notice that the dirty schedulers functionality is
- experimental. Enable support for dirty schedulers when
- building OTP to try out the functionality.</p>
<p>For more information, see
<seealso marker="#system_info_dirty_cpu_schedulers">
<c>erlang:system_info(dirty_cpu_schedulers)</c></seealso>,
@@ -7626,9 +7688,6 @@ ok
<p>This value can be set at startup by passing command-line
argument <seealso marker="erts:erl#+SDio"><c>+SDio</c></seealso>
in <c>erl(1)</c>.</p>
- <p>Notice that the dirty schedulers functionality is
- experimental. Enable support for dirty schedulers when
- building OTP to try out the functionality.</p>
<p>For more information, see
<seealso marker="#system_info_dirty_cpu_schedulers">
<c>erlang:system_info(dirty_cpu_schedulers)</c></seealso>,