aboutsummaryrefslogtreecommitdiffstats
path: root/lib
AgeCommit message (Collapse)Author
2012-03-22Update dtrace for changes in R15Björn-Egil Dahlberg
2012-03-22Add DTrace support for OS X, Solaris, and Linux (via SystemTap), 4/4Scott Lystig Fritchie
Add probes to (mostly) the efile_drv.c driver and other file I/O-related source files.
2012-03-22Add DTrace support for OS X, Solaris, and Linux (via SystemTap), 2/4Scott Lystig Fritchie
Add D scripts in the `lib/dtrace/examples` directory. Add SystemTap README and sample scripts, contributed by Andreas Schultz <[email protected]>.
2012-03-22Add DTrace support for OS X, Solaris, and Linux (via SystemTap), 1/4Scott Lystig Fritchie
Since it's been quite a while since I've written C code, *and* I haven't done any significant hacking on the VM itself in years, it's quite likely that I haven't done things in 100% proper style. Or my co-collaborators Dustin Sallings (CouchBase) or Michal Ptaszek (Erlang Solutions). My intent for this patch is to start discussion and review of DTrace support for consideration for the R15 release. For additional background on the motivation for this work, please see the slides for the presentation at the Erlang User Conference 2011 in Stockholm: https://www.erlang-factory.com/upload/presentations/462/euc2011-draft2.pdf Changes relative to dtrace-review2 branch: * Fix errors in OTP test suite 'kernel' file_SUITE * Fix errors in OTP test suite 'kernel' prim_file_SUITE * Fix bad pointer bug in efile_drv.c flush_write() * Move the DTrace material from the top of `README.md` into a new file, `README.dtrace.md` Changes since last push to GitHub (relative to commit 5828a4fb28, which was the former `dtrace-review1` branch): * Rebased onto 14 Nov 2011's "master" branch * Recent changes to the async task queuing mechanism means that the async worker queue length is not available. A bogus value of -1 is hard-coded until there's a good way to peek into the new queue structure and find the queue length. * Small fixes based on review comments by Mikael Pettersson, Andrew Thompson, and Andreas Schultz. Add autoconf support: use "./configure --enable-dtrace" on all supported platforms: * OS X Snow Leopard or later * Solaris 10 or OpenSolaris * Linux, via SystemTap's DTrace compatibility packages * FreeBSD 9.0RC1. FreeBSD 8 and earlier do not have support for USDT, DTrace's User-land Statically Defined Tracing. See the file `erts/emulator/beam/erlang_dtrace.d` for the definition of all DTrace probes in the virtual machine so far. Example D scripts can be found in `lib/dtrace/examples`. Note that if you see the error message `{name of probe} does not match any probes`, then there is no Erlang VM process + DTrace probes running. To fix, start a DTrace-enabled VM or remove `-q` from the `dtrace` command line. The `lib/dtrace` directory contains a small code-only OTP application that contains code that allows Erlang code to trigger a DTrace probe. Dynamic creation & deletion of DTrace probes is not currently supported, so the `dtrace:p()` function is hacked to allow a variable number of arguments (up to four integers and up to four strings) to be used. See the comments at the top of `lib/dtrace/src/dtrace.c` for more detail. One feature that may be controversial is the notion I've introduced of a special process dictionary key that can be used by Erlang code to tag I/O operations for an application-specific purpose. Right now, that tag's name is `dtrace_utag`. The dictionary keys used by `sys` and other modules start with a dollar sign. Perhaps there is some convention (but not a dollar sign?) that this tag should use? The purpose of the process dictionary key is to allow the tag to be included in trace messages, e.g. for file I/O, without changing the API of the `file.erl` module's functions. For example, here's a use of the tag when calling the `file:rename/2` function: (bar@sbb2)1> put(dtrace_utag, "GGOOOAAALL!!!!!"). undefined (bar@sbb2)2> dtrace:init(). ok %% Now start both the `user-probe.d` and `efile_drv.d` D scripts %% found in the `lib/dtrace/examples` directory. (bar@sbb2)3> dtrace:p(7, 8, 9, "one", "four"). true %% The output from the `user-probe.d` script: <0.40.0> GGOOOAAALL!!!!! 7 8 9 0 'one' 'four' '' '' (bar@sbb2)4> file:rename("old-name", "new-name"). {error,enoent} %% The output from the `efile_drv.d` script: async I/O pool port #Port<0.59> queue len 1 async I/O pool port #Port<0.59> queue len 0 efile_drv enter tag={1,110} user tag GGOOOAAALL!!!!! | RENAME (12) | args: old-name new-name , 0 0 (port #Port<0.59>) async I/O worker tag={1,110} | RENAME (12) | efile_drv-int_entry async I/O worker tag={1,110} | RENAME (12) | efile_drv-int_return efile_drv return tag={1,110} user tag GGOOOAAALL!!!!! | RENAME (12) | errno 2 I'm not exactly happy with this choice of tagging, namely using `put(dtrace_utag, Tag::list())`. But this is an experiment, so we'll see how it goes. I can't imagine changing the API for all file.erl functions in order pass the tag explicitly. Some modules have some extensive (ab)use of the C preprocessor to reduce the amount of #ifdefs that clutter the code. In several places, I have not #ifdef'ed automatic variables because of clutter. For the same reason, there are a handful of cases where I added DTrace-related members to a struct definition without an #ifdef. I feel that the result is easier to read than earlier drafts where I did use many more `https://github.com/slfritchie/otp/tree/dtrace-experiment+michal2` if you're curious.) I expect there may be some debate about whether the bloat of the affected structs is worthwhile. I erred on adding stuff to structs, especially in the efile_drv.c driver, not having a full grasp on what was thread-safe and what was not ... so I erred on the side of caution. The efile_drv.c has a work-around for a crazy GCC optimization bug. Thank goodness for Google, I dunno how I would've found a work-around for this silly thing. Many thanks to Trond Norbye for writing clearly about the problem in a membase Git repo commit message. /* * A note on probe naming: if "__" appears in a provider probe * definition, then two things happen during compilation: * * 1. The "__" will turn into a hypen, "-", for the probe name. * 2. The "__" will turn into a single underscore, "_", for the * macro names and function definitions that the compiler and * C developers will see. * * We'll try to use the following naming convention. We're a bit * limited because, as a USDT probe, we can only specify the 4th part * of the probe name, e.g. erlang*:::mumble. The 2nd part of the * probe name is always going to be "beam" or "beam.smp", and the 3rd * part of the probe name will always be the name of the function * that's calling the probe. * * So, all probes will be have names defined in this file using the * convention category__name or category__sub_category__name. This * will translate to probe names of category-name or * category-sub_category-name. * * Each of "category", "sub_category", and "name" may have underscores * but may not have hyphens. */ Add tentative support for sequential tracing sending, queueing, and receiving a message. I don't believe I've fully covered all the major places where it would be useful to have the sequential trace token info in a probe -- guidance from the OTP team would be helpful, if there's time to do that kind of review. Add global variable `erts_this_node_sysname`.
2012-03-22Merge branch 'raimo/close-ports-on-halt/OTP-9985' into maintRaimo Niskanen
* raimo/close-ports-on-halt/OTP-9985: erts: Basic test of erlang:halt/0..2 erts: Document erlang:halt/2 and update erlang:halt/0,1 erts: Implement erlang:halt/2 stdlib: Stop working around erlang:halt not flushing erts: Make erlang:halt/0,1 close ports and flush async threads erts: Remove forgotten and unused function erl_exit0 erts: Implement erl_halt erts: Remove unused ntbuild.erl
2012-03-22Merge branch 'ia/public_key/prepare-for-release' into maintIngela Anderton Andin
* ia/public_key/prepare-for-release: Prepare for release
2012-03-22Merge branch 'ia/ssh/prepare-for-release' into maintIngela Anderton Andin
* ia/ssh/prepare-for-release: Prepare for release
2012-03-22Merge branch 'ia/ssl/prepare-for-release' into maintIngela Anderton Andin
* ia/ssl/prepare-for-release: Prepare for release
2012-03-21Merge branch 'siri/reltool/fresh-up' into maintSiri Hansen
OTP-9792 OTP-9794 OTP-9967 OTP-9968 OTP-9980 OTP-9984 * siri/reltool/fresh-up: (27 commits) [reltool] Improve title of dependency colum [reltool] Update status bar for all config changes and for generate target [reltool] Fix GUI problems on Windows [reltool] Remove skip statement for test that failed due to stdlib error [sasl] Sort applications used by other applications correctly in .script [sasl] Doc that included applications in .rel defaults to same as in .app [reltool] Fix problems with sorting of applications [reltool] Just warn if same module occurs twice in .app file [reltool] Always return warnings as flat strings [reltool] Fix type spec and doc for app() and mod() [reltool] Move tables out of common record [reltool] Display warnings continously in GUI [reltool] Cleanup test code [reltool] Unset ERL_FLAGS when running escript from reltool_server_SUITE [reltool] Improve error handling [reltool] Fix rollback and undo [reltool] Link together escript with inlined application [reltool] Fix badmatch when starting GUI [reltool] Update state and tables consistently for all types of config changes [reltool] Add test cases for handling of escripts ...
2012-03-21Merge branch 'lukas/common_test/add_surefire_hook/OTP-9995' into maintLukas Larsson
* lukas/common_test/add_surefire_hook/OTP-9995: Add the surefire ct hook
2012-03-21Merge branch 'bmk/inets/inets59_integration2' into maintMicael Karlberg
2012-03-21erts: Implement erlang:halt/2Raimo Niskanen
2012-03-21Align java test Makefile.src with c tests ditto in icPatrik Nyblom
2012-03-21Teach ei testsuites parallel makePatrik Nyblom
2012-03-21Teach edlin_expand_SUITE:init_per_suite independencePatrik Nyblom
2012-03-21Teach big_boot_embedded testcase about comtePatrik Nyblom
2012-03-21io_proto_SUITE: make sure temp files are removedPatrik Nyblom
2012-03-21Prepare for releaseIngela Anderton Andin
Updates for ssh that needs to restarted so no need for soft upgrade
2012-03-21Merge branch 'dgud/observer/fix-mem-graph' into maintDan Gudmundsson
* dgud/observer/fix-mem-graph: [observer] Remove system from memory graph
2012-03-21Merge branch 'dgud/wx/misc-improvements/OTP-9947' into maintDan Gudmundsson
* dgud/wx/misc-improvements/OTP-9947: [wx] Add more types [wx] Fix listctrl issues [wx] Fix bad defines [wx] Fix bad type references [wx] Fix missing export_types
2012-03-21Merge branch 'dgud/debugger/breakpoints_bug/OTP-9950' into maintDan Gudmundsson
* dgud/debugger/breakpoints_bug/OTP-9950: [debugger] Fixed disappearing breakpoints
2012-03-21stdlib: Stop working around erlang:halt not flushingRaimo Niskanen
2012-03-21Prepare for releaseIngela Anderton Andin
Supervisor tree changed so can not do soft upgrade
2012-03-21Prepare for releaseIngela Anderton Andin
Distribution fixes so no need to soft upgrade.
2012-03-21Merge branch 'hb/kernel/too_greedy_disk_log/OTP-9999' into maintHans Bolinder
* hb/kernel/too_greedy_disk_log/OTP-9999: Make sure disk_log does not write too much data
2012-03-21Merge branch 'hb/stdlib/fix_qlc_suite/OTP-10000' into maintHans Bolinder
* hb/stdlib/fix_qlc_suite/OTP-10000: Correct a minor bug in qlc's testsuite
2012-03-21Fix continuation errorLars Thorsen
2012-03-21Remove unused fileLars Thorsen
2012-03-20[observer] Remove system from memory graphDan Gudmundsson
It is confusing we have total and system is total - processes
2012-03-20Add the surefire ct hookLukas Larsson
The hook should work with modern versions of Jenkins CI to gather test results.
2012-03-20Correct a minor bug in qlc's testsuiteHans Bolinder
2012-03-20Make sure disk_log does not write too much dataHans Bolinder
While disk_log eagerly collects logged data for better performance, collecting too much data may choke the system and cause huge binaries to be written. The problem was addressed in OTP-9764, but the situation was not improved in all cases. Thanks to Richard Carlsson.
2012-03-20Merge branch 'gustav/ic/remove-warnings' into maintGustav Simonsson
* gustav/ic/remove-warnings: Bump version from 4.2.29 to 4.2.30 Add generation of Erlang callback functions to avoid compiler warnings of generated Erlang source code Seq12023 OTP-9998
2012-03-20Bump version from 4.2.29 to 4.2.30Gustav Simonsson
2012-03-20[reltool] Improve title of dependency columSiri Hansen
OTP-9792 The colum listing modules that uses the current application or module did earlier have the title "Modules used by others". This is now changed to "Modules using this".
2012-03-20[reltool] Update status bar for all config changes and for generate targetSiri Hansen
OTP-9792 For some configuration changes and during generation of target system, there was no indication in the status bar that reltool was working. This has been corrected - it now says "Processing libraries...".
2012-03-20Merge branch 'sverk/zlib_port_leak' into maintSverker Eriksson
* sverk/zlib_port_leak: Fix port leaks in zlib OTP-9981
2012-03-20Merge branch 'bmk/snmp/snmp422_integration/r15' into maintMicael Karlberg
2012-03-20Merge branch 'lukas/kernel/sendfile_fixes' into maintLukas Larsson
* lukas/kernel/sendfile_fixes: Fix reselecting bug on OS X Fix bug when sending long files using select Fix ifdef to check if we are on OS X Add test case for sending multiple small files on same connection Fix memory leak when sendfile process crashes Extend timeout for windows Skip sendfile suite if solaris 8
2012-03-20Fix reselecting bug on OS XLukas Larsson
Since stop_select is called at an arbitrary point in the future it would sometime not be alled before the tcp driver started selecting on the fd. So now ERL_DRV_USE_NO_CALLBACK is used so that the stop_select call is never made. This seems to only have happened OS X.
2012-03-20Fix bug when sending long files using selectLukas Larsson
The return value from efile_sendfile was not consistent inbetween platforms. The API should now be working as it was intended. OTP-9994
2012-03-20Add test case for sending multiple small files on same connectionLukas Larsson
2012-03-20Extend timeout for windowsLukas Larsson
2012-03-20Merge remote branch 'origin/peppe/common_test/otp-9880_9986_9989' into maintPeter Andersson
* origin/peppe/common_test/otp-9880_9986_9989: Fix error with group config deleted before post_end_per_group Introduce new test suite for CTHs Fix bug in handling ct_hooks start arguments Use calls to ct_framework:init/end_per_suite for configuration Change ct_init/end_per_group to init/end_per_group Run hooks for non-existing config functions OTP-9880 OTP-9986 OTP-9989 OTP-9992
2012-03-20Merge branch 'bjorn/kernel/erts_debug_size-fix/OTP-9991' into maintBjörn Gustavsson
* bjorn/kernel/erts_debug_size-fix/OTP-9991: Improve erts_debug_SUITE Fix calculation of fun size in erts_debug:size/1
2012-03-19Merge branch 'ph/fix-enfile/OTP-9990' into maintBjörn-Egil Dahlberg
* ph/fix-enfile/OTP-9990: Update preloaded prim_inet.beam gen_tcp: Use correct specs for accept and listen prim_inet: Catch system_limit in open_port Fix returned error from gen_tcp:accept/1,2 when running out of ports
2012-03-19Fix calculation of fun size in erts_debug:size/1Björn Gustavsson
Funs would be calculated using erts_debug:flat_size/1.
2012-03-19[reltool] Fix GUI problems on WindowsSiri Hansen
The new warning list did not work on Windows. It could not display tooltips for each warning and the popup window would always disappear behind the main system window. Also, column width did not occur well initially in list controls.
2012-03-19Merge branch 'bmk/megaco/megaco31601_integration/r15' into maintMicael Karlberg
2012-03-19Fix error with group config deleted before post_end_per_groupPeter Andersson
OTP-9989 Also improve ct_hooks test suite.