| Age | Commit message (Collapse) | Author |
|---|
|
Since it's been quite a while since I've written C code, *and* I
haven't done any significant hacking on the VM itself in years, it's
quite likely that I haven't done things in 100% proper style. Or
my co-collaborators Dustin Sallings (CouchBase) or Michal Ptaszek
(Erlang Solutions). My intent for this patch is to start discussion
and review of DTrace support for consideration for the R15 release.
For additional background on the motivation for this work, please
see the slides for the presentation at the Erlang User Conference 2011
in Stockholm:
https://www.erlang-factory.com/upload/presentations/462/euc2011-draft2.pdf
Changes relative to dtrace-review2 branch:
* Fix errors in OTP test suite 'kernel' file_SUITE
* Fix errors in OTP test suite 'kernel' prim_file_SUITE
* Fix bad pointer bug in efile_drv.c flush_write()
* Move the DTrace material from the top of `README.md` into a
new file, `README.dtrace.md`
Changes since last push to GitHub (relative to commit 5828a4fb28, which
was the former `dtrace-review1` branch):
* Rebased onto 14 Nov 2011's "master" branch
* Recent changes to the async task queuing mechanism means that
the async worker queue length is not available. A bogus value
of -1 is hard-coded until there's a good way to peek into the
new queue structure and find the queue length.
* Small fixes based on review comments by Mikael Pettersson,
Andrew Thompson, and Andreas Schultz.
Add autoconf support: use "./configure --enable-dtrace" on all supported
platforms:
* OS X Snow Leopard or later
* Solaris 10 or OpenSolaris
* Linux, via SystemTap's DTrace compatibility packages
* FreeBSD 9.0RC1. FreeBSD 8 and earlier do not have support
for USDT, DTrace's User-land Statically Defined Tracing.
See the file `erts/emulator/beam/erlang_dtrace.d` for the definition
of all DTrace probes in the virtual machine so far.
Example D scripts can be found in `lib/dtrace/examples`. Note that if
you see the error message `{name of probe} does not match any probes`,
then there is no Erlang VM process + DTrace probes running. To fix,
start a DTrace-enabled VM or remove `-q` from the `dtrace` command line.
The `lib/dtrace` directory contains a small code-only OTP application
that contains code that allows Erlang code to trigger a DTrace probe.
Dynamic creation & deletion of DTrace probes is not currently
supported, so the `dtrace:p()` function is hacked to allow a variable
number of arguments (up to four integers and up to four strings) to be
used. See the comments at the top of `lib/dtrace/src/dtrace.c` for
more detail.
One feature that may be controversial is the notion I've introduced
of a special process dictionary key that can be used by Erlang code to
tag I/O operations for an application-specific purpose. Right now,
that tag's name is `dtrace_utag`. The dictionary keys used by `sys`
and other modules start with a dollar sign. Perhaps there is some
convention (but not a dollar sign?) that this tag should use?
The purpose of the process dictionary key is to allow the tag to
be included in trace messages, e.g. for file I/O, without changing the
API of the `file.erl` module's functions. For example, here's a use
of the tag when calling the `file:rename/2` function:
(bar@sbb2)1> put(dtrace_utag, "GGOOOAAALL!!!!!").
undefined
(bar@sbb2)2> dtrace:init().
ok
%% Now start both the `user-probe.d` and `efile_drv.d` D scripts
%% found in the `lib/dtrace/examples` directory.
(bar@sbb2)3> dtrace:p(7, 8, 9, "one", "four").
true
%% The output from the `user-probe.d` script:
<0.40.0> GGOOOAAALL!!!!! 7 8 9 0 'one' 'four' '' ''
(bar@sbb2)4> file:rename("old-name", "new-name").
{error,enoent}
%% The output from the `efile_drv.d` script:
async I/O pool port #Port<0.59> queue len 1
async I/O pool port #Port<0.59> queue len 0
efile_drv enter tag={1,110} user tag GGOOOAAALL!!!!! | RENAME (12) | args: old-name new-name , 0 0 (port #Port<0.59>)
async I/O worker tag={1,110} | RENAME (12) | efile_drv-int_entry
async I/O worker tag={1,110} | RENAME (12) | efile_drv-int_return
efile_drv return tag={1,110} user tag GGOOOAAALL!!!!! | RENAME (12) | errno 2
I'm not exactly happy with this choice of tagging, namely using
`put(dtrace_utag, Tag::list())`. But this is an experiment, so
we'll see how it goes. I can't imagine changing the API for
all file.erl functions in order pass the tag explicitly.
Some modules have some extensive (ab)use of the C preprocessor to
reduce the amount of #ifdefs that clutter the code. In several places,
I have not #ifdef'ed automatic variables because of clutter. For the
same reason, there are a handful of cases where I added DTrace-related
members to a struct definition without an #ifdef. I feel that the
result is easier to read than earlier drafts where I did use many more
`https://github.com/slfritchie/otp/tree/dtrace-experiment+michal2` if
you're curious.) I expect there may be some debate about whether the
bloat of the affected structs is worthwhile. I erred on adding stuff
to structs, especially in the efile_drv.c driver, not having a full
grasp on what was thread-safe and what was not ... so I erred on the
side of caution.
The efile_drv.c has a work-around for a crazy GCC optimization bug.
Thank goodness for Google, I dunno how I would've found a work-around
for this silly thing. Many thanks to Trond Norbye for writing clearly
about the problem in a membase Git repo commit message.
/*
* A note on probe naming: if "__" appears in a provider probe
* definition, then two things happen during compilation:
*
* 1. The "__" will turn into a hypen, "-", for the probe name.
* 2. The "__" will turn into a single underscore, "_", for the
* macro names and function definitions that the compiler and
* C developers will see.
*
* We'll try to use the following naming convention. We're a bit
* limited because, as a USDT probe, we can only specify the 4th part
* of the probe name, e.g. erlang*:::mumble. The 2nd part of the
* probe name is always going to be "beam" or "beam.smp", and the 3rd
* part of the probe name will always be the name of the function
* that's calling the probe.
*
* So, all probes will be have names defined in this file using the
* convention category__name or category__sub_category__name. This
* will translate to probe names of category-name or
* category-sub_category-name.
*
* Each of "category", "sub_category", and "name" may have underscores
* but may not have hyphens.
*/
Add tentative support for sequential tracing sending, queueing, and
receiving a message. I don't believe I've fully covered all the major
places where it would be useful to have the sequential trace token info
in a probe -- guidance from the OTP team would be helpful, if there's
time to do that kind of review.
Add global variable `erts_this_node_sysname`.
|
|
* raimo/close-ports-on-halt/OTP-9985:
erts: Basic test of erlang:halt/0..2
erts: Document erlang:halt/2 and update erlang:halt/0,1
erts: Implement erlang:halt/2
stdlib: Stop working around erlang:halt not flushing
erts: Make erlang:halt/0,1 close ports and flush async threads
erts: Remove forgotten and unused function erl_exit0
erts: Implement erl_halt
erts: Remove unused ntbuild.erl
|
|
|
|
|
|
|
|
|
|
* ph/fix-enfile/OTP-9990:
Update preloaded prim_inet.beam
gen_tcp: Use correct specs for accept and listen
prim_inet: Catch system_limit in open_port
Fix returned error from gen_tcp:accept/1,2 when running out of ports
|
|
* rickard/barriers/OTP-9922:
Remove newly introduced excessive barrier in erts_thr_progress_later()
Fix implementation with ddrb barriers
|
|
|
|
|
|
A large 64-bit immediate number will be stored as SMALL_BIG_EXT by ETS
compressed format. When uncompressing, the SMALL_BIG_EXT was first
decoded as as bignum (by bytes_to_big) and then turned into a small
(by big_norm). This works for normal "binary_to_term" as
decoded_size() over-estimates the needed heap size. But for ETS no
over-estimation is done as the real term size is known and stored in
DbTerm.
Fixed by preventing bytes_to_big() from writing bignum digit when the
number is seen to fit in an immediate.
|
|
* rickard/barriers/OTP-9922:
Reduce thread progress read operations in handle_aux_work()
Misc memory barrier fixes
|
|
|
|
- Document barrier semantics
- Introduce ddrb suffix on atomic ops
- Barrier macros for both non-SMP and SMP case
- Make the thread progress API a bit more intuitive
|
|
* jz/erts-remove-unused-var:
erts: Remove unused variable
OTP-9926
|
|
* rc/spell-registered:
Correct spelling of "registered" in various places in the source code
OTP-9925
|
|
* jz/reduce-smp-locking-time-range:
erts: reduce smp locking time range in erts_garbage_collect
OTP-9912
|
|
* dgud/sched-work-time/OTP-9858:
emulator: Document and test scheduler_wall_time
Implement statistics(scheduler_wall_time)
|
|
The incorrect assertion would fire when '+Muramv true' has been
given, i.e. when a reallocation always forces a move.
Change the assertion to not call the lock checker if the mutex is
not used (because the allocator will only be accessed from a
specific scheduler and thus no lock is needed).
|
|
|
|
|
|
* egil/fix-profiler-pid:
erts: Do not profile profiler pid (non smp)
erts: Do not profile profiler pid (smp)
Revert "erts: Do not profile system profiler pid"
|
|
* raimo/64-bit-efile_drv/OTP-9820:
file_SUITE: Assume 64-bit windows has enough memory
erts: Badarg if port output overflows iov_len
prim_file_SUITE: large_file success depends on wordsize
file_SUITE: Increase timeout for large_file
file_SITE,prim_file_SUITE: Accept old OpenBSD pecularity
prim_file_SUITE: large_write - meaner test data and check result content
file_SUITE: large_write - check mem size before creating huge binary
erts: rewrite efile_writev to handle partial writes correctly
erts: Bugfix - driver_deq freed wrong length due to short type (int)
prim_file_SUITE: Add large_write/1
file_SUITE: Add large_write/1
file_SUITE: Refactor large_file/1
file_SUITE: Fix unix_free/1
|
|
|
|
|
|
This reverts commit 4a8ce05083b9c88b94560f400370dbc656893b6e.
* Above commit has a lock order violation
|
|
* sverk/hipe-hibernate-sparc-bug:
erts: Make GC tolerate hibernated process without hipe stack
|
|
|
|
* egil/fix-compiler-warnings:
erts: Use re instead of regexp in testsuite
erts: Remove unused code in testsuites
orber: Use modern list guard in testsuite
orber: Ignore unused variable in orber_tb.erl
xmerl: Ignore unused variable in xmerl_uri.erl
syntax_tool: Add missing type information
hipe: Suppress warnings for unused variables
hipe: Add type information to cfg_info record
tools: Use literal formatting in erl_memory.c
asn1: Remove unused variable in asn1_erl_nif.c
ei: Remove unused variable in ei_format.c
erts: Add missing prototype to DRIVER_INIT
runtime_tools: Fix signedness in trace_ip_drv.c
to_erl: Remove compiler warnings
heart: Suppress compiler warnings
erts: Remove compiler warnings in inet_gethost.c
erts: Remove dead code in inet_gethost.c
erts: Remove dead code
|
|
The {error, enfile} return value is badly misleading and confusing for
this case, since the Posix ENFILE errno value has a well-defined meaning
that has nothing to do with Erlang ports. The fix changes the return
value to {error, system_limit}, which is consistent with e.g. various
file(3) functions. inet:format_error/1 has also been updated to support
system_limit in the same manner as file:format_error/1.
|
|
* The DRIVER_INIT macro will now produce an prototype for
the driver_init() function in addition to previous
behaviour.
|
|
|
|
|
|
|
|
Remove unused variable erts_system_monitor_msg_queue_len
|
|
It is possible also in non-SMP case:
1. The process receives an exit signal and is set in status exiting
and inserted into the run queue.
2. The distribution port exits before the process has been selected
for execution and cannot remove the link half on the process since
it is in status exiting.
3. Process is selected for execution and when removing this link half
the distribution channel is gone!
|
|
|
|
get_now should be use out of process status locking, then
it can reduce smp locking time range in here.
|
|
Seen causing segv on sparc with hibernate_native_SUITE:basic:
nstack_walk_frame_ra (nsp=0x0, sdesc=0x6c8f2c) at hipe_risc_gc.h:105
gensweep_nstack (p=0x6a6930, ptr_old_htop=0xffbfea50, ptr_n_htop=0xffbfea4c) at hipe/hipe_gc.c:224
do_minor (p=0x6a6930, new_sz=233, objv=0x6a6984, nobj=3) at beam/erl_gc.c:949
minor_collection (p=0x6a6930, need=3, objv=0x6a6984, nobj=3, recl=0xffbfece4) at beam/erl_gc.c:811
erts_garbage_collect (p=0x6a6930, need=3, objv=0x6a6984, nobj=3) at beam/erl_gc.c:379
erts_send_message (sender=0x6a6048, receiver=0x6a6930, receiver_locks=0xffbfedf8, message=6132762, flags=0) at beam/erl_message.c:922
do_send (p=0x6a6048, to=515, msg=6132762, suspend=1) at beam/bif.c:2052
erl_send (p=0x6a6048, to=515, msg=6132762) at beam/bif.c:2151
send_2 (A__p=0x6a6048, BIF__ARGS=0x6a609c) at beam/bif.c:2146
nbif_send_2 ()
|
|
* rickard/alloc-opt/OTP-7775:
Fix thread progress debug code for has_reached_wakeup()
|
|
|
|
* pan/win64-fixes:
Update INSTALL-WIN32.md to reflect changes in R15B
Make erl_alloc.c use correct strtol variant on windows
Set absolute limit on number of threads in ethread_SUITE
OTP-9130
|
|
|
|
* raimo/64-bit-driver-api/OTP-9795: (22 commits)
driver_SUITE.erl: Fix sys info drivers
emulator test drivers: Conform to updated driver API
runtime_tools's drivers: Conform to updated driver API
ws's xwe_driver.c: Conform to updated driver API
megaco's flex scanner: Conform to updated driver API
seq_trace_SUITE_data/echo_drv.c: Conform to updated driver API
erl_interface tests: Conform port_call_drv.c updated driver API
erl_drv_thread_SUITE_data/testcase_driver.c: Conform to updated driver API
float_SUITE_data/fp_drv.c: Conform to updated driver API
port_SUITE_data/*_drv.c: Conform to updated driver API
port_bif_SUITE_data/control_drv.c: Conform to updated driver API
send_term_SUITE_data/send_term_drv.c: Conform to updated driver API
system_profile_SUITE_data/echo_drv.c: Conform to updated driver API
trace_port_SUITE_data/echo_drv.c: Conform to updated driver API
Remove support for old drivers without ERL_DRV_EXTENDED_MARKER
built-in drivers: Add ERL_DRV_EXTENDED_MARKER and version numbers
Bump driver version to 2.0
erl_driver.h: Enlarge type on return value from call
erl_driver.h: Enlarge types on driver callbacks output, control and call
erl_driver.h: Enlarge types in driver output functions
...
Conflicts:
erts/emulator/test/driver_SUITE_data/monitor_drv.c
erts/emulator/test/driver_SUITE_data/timer_drv.c
|
|
Because of the extensive changes in the driver API (and especially
the change of return types for callbacks such as 'control'), we
can no longer allow drivers without version numbers.
|
|
|
|
|
|
|
|
* rickard/alloc-opt/OTP-7775:
Fix erts_thr_progress_wakeup()
|
|
|
