Update of systemtap trace example scripts.

Own Id: OTP-15670

Optimize observer by using new system_info(ets_count) instead of more expensive length(ets:all()).

Own Id: OTP-15163 Aux Id: PR-1844

New utility module scheduler which makes it easier to measure scheduler utilization.

Own Id: OTP-14904

system_information:to_file/1 will now use slightly less memory.

Own Id: OTP-14816

New family of erts_alloc strategies: Age Order First Fit. Similar to "address order", but instead the oldest possible carrier is always chosen for allocation.

Own Id: OTP-14917 Aux Id: ERIERL-88

Removed all old unused files in the documentation.

Own Id: OTP-14475 Aux Id: ERL-409, PR-1493

General Unicode improvements.

Own Id: OTP-14462

A faulty encoding comment was added when saving trace patterns to file. This is now corrected.

Own Id: OTP-14479

Add compile option -compile(no_native) in modules with on_load directive which is not yet supported by HiPE.

Own Id: OTP-14316 Aux Id: PR-1390

Miscellaneous updates due to atoms containing arbitrary Unicode characters.

Own Id: OTP-14285

Sockets can now be bound to device (SO_BINDTODEVICE) on platforms where it is supported.

This has been implemented e.g to support VRF-Lite under Linux; see VRF , and GitHub pull request #1326.

Own Id: OTP-14357 Aux Id: PR-1326

etop erroneously reported the average scheduler utilization since the tool was first started instead of the scheduler utilization since last update. This is now corrected.

Own Id: OTP-14090 Aux Id: seq13232

Add option queue_size to ttb:tracer/2. This sets the maximum queue size for the IP trace driver which is used when tracing to shell and/or {local,File}.

The default value for queue_size is specified by dbg, and it is now changed from 50 to 200.

Own Id: OTP-13829 Aux Id: seq13171

The port information page is updated to show more information per port.

Own Id: OTP-13948 Aux Id: ERL-272

Correct some minor documentation issues.

Own Id: OTP-13891

Fix bug in dbg:trace_port/2 that could cause the trace ip driver to produce faulty error reports "...(re)selected before stop_select was called for driver trace_ip_drv".

Own Id: OTP-13576 Aux Id: ERL-119

Add microstate accounting

Microstate accounting is a way to track which state the different threads within ERTS are in. The main usage area is to pin point performance bottlenecks by checking which states the threads are in and then from there figuring out why and where to optimize.

Since checking whether microstate accounting is on or off is relatively expensive only a few of the states are enabled by default and more states can be enabled through configure.

There is a convenience module called msacc that has been added to runtime_tools that can assist in gathering and interpreting the data from Microstate accounting.

For more information see erlang:statistics(microstate_accounting, _) and the msacc module in runtime_tools.

Own Id: OTP-12345

Update observer GUI to support tracing on ports, and to set matchspecs for send/receive. This required some minor bugfixes in runtime_tools/dbg.

Own Id: OTP-13481

Update dbg and ttb to work with a tracer module as tracer and tracing on ports.

Own Id: OTP-13500

Updated dbg to accept the new trace options monotonic_timestamp and strict_monotonic_timestamp.

Own Id: OTP-13502

Introduce LTTng tracing via Erlang tracing.

For LTTng to be enabled OTP needs to be built with configure option --with-dynamic-trace=lttng.

The dynamic trace module dyntrace is now capable to be used as a LTTng sink for Erlang tracing. For a list of all tracepoints, see Runtime Tools User's Guide .

This feature also introduces an incompatible change in trace tags. The trace tags gc_start and gc_end has been split into gc_minor_start, gc_minor_end and gc_major_start, gc_major_end.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13532

dbg:trace_client() now uses a read buffer to speed up reading of trace files.

Own Id: OTP-13279

Clarified dbg:stop documentation

Own Id: OTP-13078

The trace_file_drv did not handle EINTR correct which caused it to fail when the runtime system received a signal.

Own Id: OTP-12890 Aux Id: seq12885

Change license text from Erlang Public License to Apache Public License v2

Own Id: OTP-12845

The trace process started by dbg would not always terminate when dbg:stop/0 was called.

Own Id: OTP-12517

Add nif_version to erlang:system_info/1 in order to get the NIF API version of the runtime system in a way similar to driver_version.

Own Id: OTP-12298

The documentation for the return value of dbg:{stop,stop_clear} functions are now correct (Thanks to Luca Favatella)

Own Id: OTP-11603

Fix DTrace build on Illumos. (Thanks to Ryan Zezeski.)

Own Id: OTP-11622

Do not turn off scheduler_wall_time, as it can interfere with other applications usage.

Own Id: OTP-11693 Aux Id: seq12528

Application upgrade (appup) files are corrected for the following applications:

asn1, common_test, compiler, crypto, debugger, dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, inets, observer, odbc, os_mon, otp_mibs, parsetools, percept, public_key, reltool, runtime_tools, ssh, syntax_tools, test_server, tools, typer, webtool, wx, xmerl

A new test utility for testing appup files is added to test_server. This is now used by most applications in OTP.

(Thanks to Tobias Schlager)

Own Id: OTP-11744

Allow install path to have unicode characters.

Own Id: OTP-10877

The erts_alloc_config tool has been updated to produce configurations that better fit todays SMP support in the VM.

Own Id: OTP-11662

The app-file key runtime_dependencies has been introduced.

Runtime dependencies have been added to all app-files in OTP. Note that these may not be completely correct during OTP 17, but this is actively being worked on.

The function system_information:sanity_check/0 will verify all declared runtime dependencies in the system when called.

Own Id: OTP-11773

Observer did not produce correct result when ERTS internal memory allocators had been disabled.

Own Id: OTP-11520

The process trace flag 'silent' is now allowed in call to dbg:p/2.

Own Id: OTP-11222

Introduced functionality for inspection of system and build configuration.

Own Id: OTP-11196

Some bugs related to calculation of CPU/scheduler utilization in observer are corrected.

Current function for a process is accepted to be 'undefined' when running hipe.

Own Id: OTP-10894

Erlang source files with non-ASCII characters are now encoded in UTF-8 (instead of latin1).

Own Id: OTP-11041 Aux Id: OTP-10907

Fix Table Viewer refresh crash on no more existing ets tables (Thanks to Peti Gömori)

Own Id: OTP-10635

User Guides for the dynamic tracing tools dtrace and systemtap have been added to the documentation.

Own Id: OTP-10155

Where necessary a comment stating encoding has been added to Erlang files. The comment is meant to be removed in Erlang/OTP R17B when UTF-8 becomes the default encoding.

Own Id: OTP-10630

Some examples overflowing the width of PDF pages have been corrected.

Own Id: OTP-10665

The backend module appmon_info.erl is moved from appmon application to runtime_tools. This allows appmon to be run from a remote erlang node towards a target node which does not have appmon (and its dependencies) installed, as long as runtime_tools is installed there.

Own Id: OTP-10786

Change the module-level docs to give complete step-by-step instructions for using the `dyntrace:p()` trace function. (Thanks to Scott Lystig Fritchie)

Own Id: OTP-10141

Add 1024 separate USDT probes to dyntrace.erl and dyntrace.c (Thanks to Scott Lystig Fritchie)

Own Id: OTP-10143

Relocate bodies of DTrace probes to the statically-linked VM.

Due to various operating systems (in both the DTrace and SystemTap worlds) not fully supporting DTrace probes (or SystemTap-compatibility mode probes) in shared libraries, we relocate those probes to the statically-linked virtual machine. This could be seen as pollution of the pristine VM by a (yet) experimental feature. However:

1. This code can be eliminated completely by the C preprocessor. 2. Leaving the probes in the dyntrace NIF shared library simply does not work correctly on too many platforms. *Many* thanks to Macneil Shonle at Basho for assisting when my RSI-injured fingers gave out. (note: Solaris 10 and FreeBSD 9.0-RELEASE can take a long time to compile)

Own Id: OTP-10189

The DTrace source patch from Scott Lystig Fritchie is integrated in the source tree. Using an emulator with dtrace probe is still not supported for production use, but may be a valuable debugging tool. Configure with --with-dynamic-trace=dtrace (or --with-dynamic-trace=systemtap) to create a build with dtrace probes enabled. See runtime_tools for documentation and examples.

Own Id: OTP-10017

Earlier dbg:stop only did erlang:trace_delivered and did not flush the trace file driver. Therefore there could still be trace messages that were delivered to the driver (guaranteed by erlang:trace_delivered) but not yet written to the file when dbg:stop returned. Flushing is now added on each node before the dbg process terminates.

Own Id: OTP-9651

File handles created by the trace_file_drv driver was inherited to child processes. This is now corrected.

Own Id: OTP-9658

Erlang/OTP can now be built using parallel make if you limit the number of jobs, for instance using 'make -j6' or 'make -j10'. 'make -j' does not work at the moment because of some missing dependencies.

Own Id: OTP-9451

Two new built-in trace pattern aliases have been added: caller_trace (c) and caller_exception_trace (cx). See the dbg:ltp/0 documentation for more info.

Own Id: OTP-9458

Two new built-in trace pattern aliases have been added: caller_trace (c) and caller_exception_trace (cx). See the dbg:ltp/0 documentation for more info.

Own Id: OTP-9458

When a big number of trace patterns are set by inviso the Erlang VM could get unresponsive for several seconds. This is now corrected.

Own Id: OTP-9048 Aux Id: seq11480

Minor corrections and removal of a temporary workaround.

Own Id: OTP-8755 Aux Id: seq-11628, seq-11639

Small fix in inviso_autostart_server.

Own Id: OTP-8783 Aux Id: seq11628

Miscellaneous updates.

Own Id: OTP-8705

Cross compilation improvements and other build system improvements.

Most notable:

(Thanks to Henrik Riomar for suggestions and testing)

(Thanks to Winston Smith for the AVR32-Linux cross configuration and testing)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8323

Cleanups suggested by tidier and modernization of types and specs.

Own Id: OTP-8455

The documentation is now built with open source tools (xsltproc and fop) that exists on most platforms. One visible change is that the frames are removed.

Own Id: OTP-8201

Makefile.in has been updated to use the LDFLAGS environment variable (if set). (Thanks to Davide Pesavento.)

Own Id: OTP-8157

etop would crash if the emulator's custom allocators had been turned off (e.g. using the +Meamin option).

Own Id: OTP-7519

The copyright notices have been updated.

Own Id: OTP-7851

Now, dbg:p/2 accepts {X,Y,Z} process specification as stated in the documentation. It also now accepts "<X.Y.Z>" like from erlang:pid_to_list/1.

There is now a pre-saved match spec in dbg that saves a lot of typing. Use dbg:ltp/0 to find out more...

Own Id: OTP-7867

Fixed a timestamp problem where some events could be sent out of order. Minor fixes to presentation of data.

Own Id: OTP-7544 Aux Id: otp-7442

etop now collects process information faster and more reliably than before (a race condition reported by Igor Goryachev has been eliminated).

Trace messages could be lost when ttb:stop/0 was called.

Own Id: OTP-7164

The documentation has been updated so as to reflect the last updates of the Erlang shell as well as the minor modifications of the control sequence p of the io_lib module.

Superfluous empty lines have been removed from code examples and from Erlang shell examples.

Own Id: OTP-6944 Aux Id: OTP-6554, OTP-6911

Memory management improvements especially for the runtime system with SMP support:

If you for some reason want the memory allocators to be configured as before, you can pass the +Mea r11b command-line argument to erl.

For more information see the erts_alloc(3), the erts_alloc_config(3), and the erlang(3) documentation.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7100

dbg could leave traced processes in a suspended state if the tracer process was killed with exit reason kill.

erlang:suspend_process/2 has been introduced which accepts an option list as second argument. For more information see the erlang(3) documentation.

Processes suspended via erlang:suspend_process/[1,2] will now be automatically resumed if the process that called erlang:suspend_process/[1,2] terminates.

Processes could previously be suspended by one process and resumed by another unless someone was tracing the suspendee. This is not possible anymore. The process resuming a process has to be the one that suspended it.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6946

The undocumented and unsupported function dbg:tracer/1 has been removed. The undocumented, unsupported, and broken function dbg:i/1 has been removed.

Own Id: OTP-6939

In this release the following has been fixed and enhanced: Autostart: It is now possible to configure modules that shall be loaded by the autostart mechanism. This because it is not certain that all application systems make use of the OTP boot script to set up paths to all Erlang modules. Runtime_tools/Inviso: A bug in the fetch_log functionality has been fixed. Further a bug that was (supposedly) fixed in a previous patch concerning meta-tracer write_ti has been fixed (again) in this patch. A bug in inviso_as_lib making remote autostart config file updates fail has been fixed. Inviso: inviso_tool has been given a flush API.

Own Id: OTP-6918

The following bugs/improvements have been done: Internal interworking between inviso_rt and inviso_rt_meta. The call function used by inviso_rt to call inviso_rt_meta is now protected with a monitor. Inviso_rt_meta now includes the timestamp of the incoming meta trace message when calling the call-callback. (Makes it possible to add a "better" timestamp to the ti-file.) Bug in inviso_tool making it not remove trace patterns when terminating. Bug in internal function h_start_session making inviso_tool crash if there were no active nodes to start the session on. The user-inviso_tool and inviso API-inviso control component request/response gen_server protocols had default time-out. Since many trace operations can be time consuming, a longer time-out is necessary. Improved overload protection. It is now possible to let the overload protection renew itself (e.g after an exit from an external overload protector). Inviso_rt_meta now fully uses the exception_trace match spec action term. Run Trace Case API (as in contrast to activate and deactivate trace case APIs) in inviso_tool. Flush trace-port API added to inviso. Get_session_data API added to inviso_tool. Improved inviso_tool:stop making it possible to name nodes which shall not have their trace patterns removed when inviso_tool terminates. Bug in handling of writing multiple ti-entries if returned from a call/return_from call-back in inviso_rt_meta Process trace flags are no longer explicitly removed by the inviso_tool when it terminates. Not necessary. Inviso_tool get_autostart_data adopted to standard autostarter.

*** INCOMPATIBILITY with Meta trace call-backs are called with different arguments now. ***

Own Id: OTP-6881

A bug in inviso_rt_meta caused an ETS table containing information on initiated (init_tpm) functions to be lost when suspending tracing. Further an enhancement to inviso_rt has been introduced making it possible to activate process trace flags based on globally registered names. It is then not an error to activate a global name on a node where the name does not reside. The process count in the return value will simply be set to zero (hence exactly one node in the NodeResult part of the return value will indicate one matching process found). A bug was found in fetch_log API. At the same time the fetch_log functionality was enhanced to also offer flow control making fetcher processes send chunks of transferred file data at a slower pace.

Own Id: OTP-6703

Minor Makefile changes.

Own Id: OTP-6689 Aux Id: OTP-6742

An experimental tool called erts_alloc_config has been added. erts_alloc_config is intended to be used to aid creation of an erts_alloc configuration that is suitable for a limited number of runtime scenarios. For more information see the erts_alloc_config(3) documentation.

Own Id: OTP-6700

Misc improvements.

Own Id: OTP-6576

This application has been updated to eliminate warnings by Dialyzer.

Own Id: OTP-6551

This ticket includes several improvements and bugfixes to both runtime_tools and inviso. The overload mechanism can now also react to incoming messages. This is useful if an external overload watch-dog is used. Some improvements of variable bindings has been done to the default autostart mechanism - inviso_autostart_server. Autostart "jobs" can now be done in parallel, allowing for some jobs to hang waiting for some parts of the traced system to become ready before proceeding. Previously when using named meta-match-specs (tpm_ms) ending up with zero match-specs still kept the meta trace pattern active. This caused zero match-specs to be equal to unlimited meta tracing on that particular function. If the internal database becomes empty of meta match specs, meta trace pattern is removed for that function. Standard public loop data in the inviso runtime meta tracer process is now extended to a 2-tuple. The functions ctp/1 and ctpl/1 are added making it possible to remove trace patterns for a list of functions rather than one by one. Inviso_rt_meta will now accept a list of binaries to be output into the trace information file, in additions to a single binary. Further it is also possible to make own output to the trace information file using the write_ti/1 function. An error was discovered in inviso_rt making the inviso_rt_meta remain rather than terminate if the inviso_rt terminated due to "running alone" (not allowed to run without a control component). A new tool, inviso_tool, has been added to the inviso application.

Own Id: OTP-6426

Several minor bugs and race conditions eliminated in the runtime_tools and observer applications.

Own Id: OTP-6265

There are new BIFs erlang:spawn_monitor/1,3, and the new option monitor for spawn_opt/2,3,4,5.

The observer_backend module has been updated to handle the new BIFs.

Own Id: OTP-6281

The dbg manual page has been updated with information about how to avoid deadlock when tracing.

Own Id: OTP-5373 Aux Id: seq9729

Linked in drivers in the Crypto, and Asn1 applications are now compiled with the -D_THREAD_SAFE and -D_REENTRANT switches on unix when the emulator has thread support enabled.

Linked in drivers on MacOSX are not compiled with the undocumented -lbundle1.o switch anymore. Thanks to Sean Hinde who sent us a patch.

Linked in driver in Crypto, and port programs in SSL, now compiles on OSF1.

Minor Makefile improvements in Runtime_Tools.

Own Id: OTP-5346