From 1e1c5905d7b0db874b2df2ba838e3ab163e8f02a Mon Sep 17 00:00:00 2001 From: Lukas Larsson Date: Thu, 19 Oct 2017 08:55:57 +0200 Subject: system: Add External OSS tools section to profiling guilde --- system/doc/efficiency_guide/profiling.xml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/system/doc/efficiency_guide/profiling.xml b/system/doc/efficiency_guide/profiling.xml index bf50a03fa6..f661abf285 100644 --- a/system/doc/efficiency_guide/profiling.xml +++ b/system/doc/efficiency_guide/profiling.xml @@ -65,6 +65,20 @@

The tools are further described in Tools.

+ +

There are also several open source tools outside of Erlang/OTP + that can be used to help profiling. Some of them are:

+ + + erlgrind + can be used to visualize fprof data in kcachegrind. + eflame + is an alternative to fprof that displays the profiling output as a flamegraph. + recon + is a collection of Erlang profiling and debugging tools. + This tool comes with an accompanying E-book called + Erlang in Anger. +
-- cgit v1.2.3 From c5b2a6161381951d273b124ff37a25597a27d2ef Mon Sep 17 00:00:00 2001 From: Lukas Larsson Date: Mon, 23 Oct 2017 10:15:01 +0200 Subject: kernel: Clarify gen_tcp:listen documentation --- lib/kernel/doc/src/gen_tcp.xml | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/lib/kernel/doc/src/gen_tcp.xml b/lib/kernel/doc/src/gen_tcp.xml index 070782e1f3..e6104b0c76 100644 --- a/lib/kernel/doc/src/gen_tcp.xml +++ b/lib/kernel/doc/src/gen_tcp.xml @@ -51,6 +51,7 @@ server() -> {ok, Sock} = gen_tcp:accept(LSock), {ok, Bin} = do_recv(Sock, []), ok = gen_tcp:close(Sock), + ok = gen_tcp:close(LSock), Bin. do_recv(Sock, Bs) -> @@ -309,9 +310,9 @@ do_recv(Sock, Bs) -> inet:setopts/2.

-

The returned socket ListenSocket can only be - used in calls to - accept/1,2.

+

The returned socket ListenSocket should be used + in calls to accept/1,2 to + accept incoming connection requests.

The default values for options specified to listen can be affected by the Kernel configuration parameter -- cgit v1.2.3 From 6ef6bbbcb18323b86eb4e654f72d2b1c6bad086f Mon Sep 17 00:00:00 2001 From: Lukas Larsson Date: Tue, 24 Oct 2017 10:46:12 +0200 Subject: Update message queue and pinfo binary docs --- erts/doc/src/erlang.xml | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 2465f49581..9d3ec406b4 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -4687,7 +4687,7 @@ RealSystem = system + MissedSystem

The default message_queue_data process flag is determined by command-line argument +hmqd in erl(1).

-

If the process potentially can get many messages, +

If the process potentially can get many messages in its queue, you are advised to set the flag to off_heap. This because a garbage collection with many messages placed on the heap can become extremely expensive and the process can @@ -4960,11 +4960,15 @@ RealSystem = system + MissedSystem {binary, BinInfo}

BinInfo is a list containing miscellaneous - information about binaries currently referred to by this - process. This InfoTuple can be changed or + information about binaries on the heap of this + process. + This InfoTuple can be changed or removed without prior notice. In the current implementation BinInfo is a list of tuples. The tuples contain; BinaryId, BinarySize, BinaryRefcCount.

+

The message queue is on the heap depending on the + process flag + message_queue_data.

{catchlevel, CatchLevel} -- cgit v1.2.3 From 6c25aaa1e447b12dcb640b0e3311a3722b501ddc Mon Sep 17 00:00:00 2001 From: Lukas Larsson Date: Tue, 24 Oct 2017 16:55:33 +0200 Subject: system: Add info to profiling effeciency guide --- erts/doc/src/erlang.xml | 5 +- system/doc/efficiency_guide/profiling.xml | 157 +++++++++++++++++++++--------- 2 files changed, 115 insertions(+), 47 deletions(-) diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 9d3ec406b4..7e5388b708 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -2955,7 +2955,10 @@ os_prompt%

The total amount of memory currently allocated for the emulator that is not directly related to any Erlang process. Memory presented as processes is not - included in this memory.

+ included in this memory. + instrument(3) can be used to + get a more detailed breakdown of what memory is part + of this type.

 atom diff --git a/system/doc/efficiency_guide/profiling.xml b/system/doc/efficiency_guide/profiling.xml index f661abf285..f185456158 100644 --- a/system/doc/efficiency_guide/profiling.xml +++ b/system/doc/efficiency_guide/profiling.xml @@ -41,26 +41,33 @@

Erlang/OTP contains several tools to help finding bottlenecks:

- fprof provides the most detailed information about - where the program time is spent, but it significantly slows down the - program it profiles. - -

eprof provides time information of each function - used in the program. No call graph is produced, but eprof has - considerable less impact on the program it profiles.

-

If the program is too large to be profiled by fprof or - eprof, the cover and cprof tools can be used - to locate code parts that are to be more thoroughly profiled using - fprof or eprof.

 - - cover provides execution counts per line per - process, with less overhead than fprof. Execution counts - can, with some caution, be used to locate potential performance - bottlenecks. - - cprof is the most lightweight tool, but it only - provides execution counts on a function basis (for all processes, - not per process). +

fprof provides + the most detailed information about where the program time is spent, + but it significantly slows down the program it profiles.

 + +

eprof provides + time information of each function used in the program. No call graph is + produced, but eprof has considerable less impact on the program it + profiles.

+

If the program is too large to be profiled by fprof or + eprof, cprof can be used to locate code parts that + are to be more thoroughly profiled using fprof or eprof.

 + +

cprof is the + most lightweight tool, but it only provides execution counts on a + function basis (for all processes, not per process).

 + +

dbg is the + generic erlang tracing frontend. By using the timestamp or + cpu_timestamp options it can be used to time how long function + calls in a live system take.

 + +

lcnt is used + to find contention points in the Erlang Run-Time System's internal + locking mechanisms. It is useful when looking for bottlenecks in + interaction between process, port, ets tables and other entities + that can be run in parallel.

 +

The tools are further described in @@ -81,6 +88,42 @@

+
+ Memory profiling + 
eheap_alloc: Cannot allocate 1234567890 bytes of memory (of type "heap").
+

The above slogan is one of the more common reasons for Erlang to terminate. + For unknown reasons the Erlang Run-Time System failed to allocate memory to + use. When this happens a crash dump is generated that contains information + about the state of the system as it ran out of mmeory. Use the + crashdump_viewer to get a + view of the memory is being used. Look for processes with large heaps or + many messages, large ets tables, etc.

+

When looking at memory usage in a running system the most basic function + to get information from is + erlang:memory(). It returns the current memory usage + of the system. instrument(3) + can be used to get a more detailed breakdown of where memory is used.

+

Processes, ports and ets tables can then be inspecting using their + respective info functions, i.e. + erlang:process_info/2 + , + erlang:port_info/2 + and + ets:info/1. +

+

Sometimes the system can enter a state where the reported memory + from erlang:memory(total) is very different from the + memory reported by the OS. This can be because of internal + fragmentation within the Erlang Run-Time System. Data about + how memory is allocated can be retrieved using + + erlang:system_info(allocator). + The data you get from that function is very raw and not very plesant to read. + recon_alloc + can be used to extract useful information from system_info + statistics counters.

+
+
Large Systems

For a large system, it can be interesting to run profiling @@ -94,6 +137,22 @@ tools on the whole system. Instead you want to concentrate on central processes and modules, which contribute for a big part of the execution.

+ +

There are also some tools that can be used to get a view of the + whole system with more or less overhead.

+ + observer + is a GUI tool that can connect to remote nodes and display a + variety of information about the running system. + etop + is a command line tool that can connect to remote nodes and + display information similar to what the UNIX tool top shows. + msacc + allows the user to get a view of what the Erlang Run-Time system + is spending its time doing. Has a very low overhead, which makes it + useful to run in heavily loaded systems to get some idea of where + to start doing more granular profiling. +
@@ -142,7 +201,7 @@ performance impact. Using fprof is just a matter of calling a few library functions, see the fprof manual page in - Tools .fprof was introduced in R8.

+ Tools.

@@ -155,20 +214,6 @@ manual page in Tools.

-
- cover -

The primary use of cover is coverage analysis to verify - test cases, making sure that all relevant code is covered. - cover counts how many times each executable line of code - is executed when a program is run, on a per module basis.

-

Clearly, this information can be used to determine what - code is run very frequently and can therefore be subject for - optimization. Using cover is just a matter of calling a - few library functions, see the - cover manual page in - Tools.

-
-
cprof

cprof is something in between fprof and @@ -215,16 +260,6 @@ No No - - cover - Per module to screen/file - Small - Moderate slowdown - Yes, per line - No - No - No - cprof Per module to caller @@ -238,6 +273,37 @@ Tool Summary

+ +
+ dbg +

dbg is a generic Erlang trace tool. By using the + timestamp or cpu_timestamp options it can be used + as a precision instrument to profile how long time a function + call takes for a specific process. This can be very useful when + trying to understand where time is spent in a heavily loaded + system as it is possible to limit the scope of what is profiled + to be very small. + For more information, see the + dbg manual page in + Runtime Tools.

+
+ +
+ lcnt +

lcnt is used to profile interactions inbetween + entities that run in parallel. For example if you have + a process that all other processes in the system needs + to interact with (maybe it has some global configuration), + then lcnt can be used to figure out if the interaction + with that process is a problem.

+

In the Erlang Run-time System entities are only run in parallel + when there are multiple schedulers. Therefore lcnt will + show more contention points (and thus be more useful) on systems + using many schedulers on many cores.

+

For more information, see the + lcnt manual page in Tools.

+
+
@@ -296,4 +362,3 @@
- -- cgit v1.2.3 From feee5399358d5eede6b1c45fbc59b3ec8d992b6e Mon Sep 17 00:00:00 2001 From: Lukas Larsson Date: Tue, 24 Oct 2017 16:55:50 +0200 Subject: Fix broken link in INSTALL howto --- HOWTO/INSTALL.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/HOWTO/INSTALL.md b/HOWTO/INSTALL.md index 36365799e3..987a714936 100644 --- a/HOWTO/INSTALL.md +++ b/HOWTO/INSTALL.md @@ -18,9 +18,6 @@ Required Utilities These are the tools you need in order to unpack and build Erlang/OTP. -> *WARNING*: Please have a look at the [Known platform issues][] chapter -> before you start. - ### Unpacking ### * GNU unzip, or a modern uncompress. @@ -794,7 +791,6 @@ Use `hipe:help_options/0` to print out the available options. [man pages]: http://www.erlang.org/download/otp_doc_man_%OTP-VSN%.tar.gz [the released source tar ball]: http://www.erlang.org/download/otp_src_%OTP-VSN%.tar.gz [System Principles]: ../system_principles/system_principles - [Known platform issues]: #Known-platform-issues [native build]: #How-to-Build-and-Install-ErlangOTP [cross build]: INSTALL-CROSS.md [Required Utilities]: #Required-Utilities -- cgit v1.2.3 From c4606d0ce55eac7c383520de765cffba2c235ace Mon Sep 17 00:00:00 2001 From: Lukas Larsson Date: Tue, 31 Oct 2017 10:37:39 +0100 Subject: erts: Add term2bin note about encoding guarantee --- erts/doc/src/erlang.xml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 7e5388b708..b04f2b008e 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -8876,6 +8876,10 @@ hello

See also binary_to_term/1.

+ +

There is no guarantee that this function will return + the same encoded representation for the same term.

+ -- cgit v1.2.3