From 1e1c5905d7b0db874b2df2ba838e3ab163e8f02a Mon Sep 17 00:00:00 2001 From: Lukas Larsson Date: Thu, 19 Oct 2017 08:55:57 +0200 Subject: system: Add External OSS tools section to profiling guilde --- system/doc/efficiency_guide/profiling.xml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) (limited to 'system/doc') diff --git a/system/doc/efficiency_guide/profiling.xml b/system/doc/efficiency_guide/profiling.xml index bf50a03fa6..f661abf285 100644 --- a/system/doc/efficiency_guide/profiling.xml +++ b/system/doc/efficiency_guide/profiling.xml @@ -65,6 +65,20 @@

The tools are further described in Tools.

+ +

There are also several open source tools outside of Erlang/OTP + that can be used to help profiling. Some of them are:

+ + + erlgrind + can be used to visualize fprof data in kcachegrind. + eflame + is an alternative to fprof that displays the profiling output as a flamegraph. + recon + is a collection of Erlang profiling and debugging tools. + This tool comes with an accompanying E-book called + Erlang in Anger. +
-- cgit v1.2.3 From 6c25aaa1e447b12dcb640b0e3311a3722b501ddc Mon Sep 17 00:00:00 2001 From: Lukas Larsson Date: Tue, 24 Oct 2017 16:55:33 +0200 Subject: system: Add info to profiling effeciency guide --- system/doc/efficiency_guide/profiling.xml | 157 +++++++++++++++++++++--------- 1 file changed, 111 insertions(+), 46 deletions(-) (limited to 'system/doc') diff --git a/system/doc/efficiency_guide/profiling.xml b/system/doc/efficiency_guide/profiling.xml index f661abf285..f185456158 100644 --- a/system/doc/efficiency_guide/profiling.xml +++ b/system/doc/efficiency_guide/profiling.xml @@ -41,26 +41,33 @@

Erlang/OTP contains several tools to help finding bottlenecks:

- fprof provides the most detailed information about - where the program time is spent, but it significantly slows down the - program it profiles. - -

eprof provides time information of each function - used in the program. No call graph is produced, but eprof has - considerable less impact on the program it profiles.

-

If the program is too large to be profiled by fprof or - eprof, the cover and cprof tools can be used - to locate code parts that are to be more thoroughly profiled using - fprof or eprof.

 - - cover provides execution counts per line per - process, with less overhead than fprof. Execution counts - can, with some caution, be used to locate potential performance - bottlenecks. - - cprof is the most lightweight tool, but it only - provides execution counts on a function basis (for all processes, - not per process). +

fprof provides + the most detailed information about where the program time is spent, + but it significantly slows down the program it profiles.

 + +

eprof provides + time information of each function used in the program. No call graph is + produced, but eprof has considerable less impact on the program it + profiles.

+

If the program is too large to be profiled by fprof or + eprof, cprof can be used to locate code parts that + are to be more thoroughly profiled using fprof or eprof.

 + +

cprof is the + most lightweight tool, but it only provides execution counts on a + function basis (for all processes, not per process).

 + +

dbg is the + generic erlang tracing frontend. By using the timestamp or + cpu_timestamp options it can be used to time how long function + calls in a live system take.

 + +

lcnt is used + to find contention points in the Erlang Run-Time System's internal + locking mechanisms. It is useful when looking for bottlenecks in + interaction between process, port, ets tables and other entities + that can be run in parallel.

 +

The tools are further described in @@ -81,6 +88,42 @@

+
+ Memory profiling + 
eheap_alloc: Cannot allocate 1234567890 bytes of memory (of type "heap").
+

The above slogan is one of the more common reasons for Erlang to terminate. + For unknown reasons the Erlang Run-Time System failed to allocate memory to + use. When this happens a crash dump is generated that contains information + about the state of the system as it ran out of mmeory. Use the + crashdump_viewer to get a + view of the memory is being used. Look for processes with large heaps or + many messages, large ets tables, etc.

+

When looking at memory usage in a running system the most basic function + to get information from is + erlang:memory(). It returns the current memory usage + of the system. instrument(3) + can be used to get a more detailed breakdown of where memory is used.

+

Processes, ports and ets tables can then be inspecting using their + respective info functions, i.e. + erlang:process_info/2 + , + erlang:port_info/2 + and + ets:info/1. +

+

Sometimes the system can enter a state where the reported memory + from erlang:memory(total) is very different from the + memory reported by the OS. This can be because of internal + fragmentation within the Erlang Run-Time System. Data about + how memory is allocated can be retrieved using + + erlang:system_info(allocator). + The data you get from that function is very raw and not very plesant to read. + recon_alloc + can be used to extract useful information from system_info + statistics counters.

+
+
Large Systems

For a large system, it can be interesting to run profiling @@ -94,6 +137,22 @@ tools on the whole system. Instead you want to concentrate on central processes and modules, which contribute for a big part of the execution.

+ +

There are also some tools that can be used to get a view of the + whole system with more or less overhead.

+ + observer + is a GUI tool that can connect to remote nodes and display a + variety of information about the running system. + etop + is a command line tool that can connect to remote nodes and + display information similar to what the UNIX tool top shows. + msacc + allows the user to get a view of what the Erlang Run-Time system + is spending its time doing. Has a very low overhead, which makes it + useful to run in heavily loaded systems to get some idea of where + to start doing more granular profiling. +
@@ -142,7 +201,7 @@ performance impact. Using fprof is just a matter of calling a few library functions, see the fprof manual page in - Tools .fprof was introduced in R8.

+ Tools.

@@ -155,20 +214,6 @@ manual page in Tools.

-
- cover -

The primary use of cover is coverage analysis to verify - test cases, making sure that all relevant code is covered. - cover counts how many times each executable line of code - is executed when a program is run, on a per module basis.

-

Clearly, this information can be used to determine what - code is run very frequently and can therefore be subject for - optimization. Using cover is just a matter of calling a - few library functions, see the - cover manual page in - Tools.

-
-
cprof

cprof is something in between fprof and @@ -215,16 +260,6 @@ No No - - cover - Per module to screen/file - Small - Moderate slowdown - Yes, per line - No - No - No - cprof Per module to caller @@ -238,6 +273,37 @@ Tool Summary

+ +
+ dbg +

dbg is a generic Erlang trace tool. By using the + timestamp or cpu_timestamp options it can be used + as a precision instrument to profile how long time a function + call takes for a specific process. This can be very useful when + trying to understand where time is spent in a heavily loaded + system as it is possible to limit the scope of what is profiled + to be very small. + For more information, see the + dbg manual page in + Runtime Tools.

+
+ +
+ lcnt +

lcnt is used to profile interactions inbetween + entities that run in parallel. For example if you have + a process that all other processes in the system needs + to interact with (maybe it has some global configuration), + then lcnt can be used to figure out if the interaction + with that process is a problem.

+

In the Erlang Run-time System entities are only run in parallel + when there are multiple schedulers. Therefore lcnt will + show more contention points (and thus be more useful) on systems + using many schedulers on many cores.

+

For more information, see the + lcnt manual page in Tools.

+
+
@@ -296,4 +362,3 @@
- -- cgit v1.2.3