| Age | Commit message (Collapse) | Author |
|---|
|
|
|
All of the Red-Black Tree _yielding functions have been
updated to work with reductions returned by the called
function instead of yielding on each element.
|
|
There seems to be an incorrect merge conflict resolution
in abde22933f5a that merged maint to master. Dumping of
references (and possibly other terms) were broken.
|
|
* maint:
Implement a tab for persistent terms in crashdump viewer
Add tests of persistent terms for crashdump_viewer
Add a persistent term storage
Refactor releasing of literals
Extend the sharing-preserving routines to optionally copy literals
Conflicts:
erts/emulator/Makefile.in
erts/emulator/beam/erl_process_dump.c
erts/preloaded/ebin/erts_internal.beam
erts/preloaded/ebin/init.beam
lib/sasl/src/systools_make.erl
|
|
Persistent terms are useful for storing Erlang terms that are never
or infrequently updated. They have the following advantages:
* Constant time access. A persistent term is not copied when it is
looked up. The constant factor is lower than for ETS, and no locks
are taken when looking up a term.
* Persistent terms are not copied in garbage collections.
* There is only ever one copy of a persistent term (until it is
deleted). That makes them useful for storing configuration data
that needs to be easily accessible by all processes.
Persistent terms have the following drawbacks:
* Updates are expensive. The hash table holding the keys for the
persistent terms are updated whenever a persistent term is added,
updated or deleted.
* Updating or deleting a persistent term triggers a "global GC", which
will schedule a heap scan of all processes to search the heap of all
processes for the deleted term. If a process still holds a reference
to the deleted term, the process will be garbage collected and the
term copied to the heap of the process. This global GC can make the
system less responsive for some time.
Three BIFs (implemented in C in the emulator) is the entire
interface to the persistent term functionality:
* put(Key, Value) to store a persistent term.
* get(Key) to look up a persistent term.
* erase(Key) to delete a persistent term.
There are also two additional BIFs to obtain information about
persistent terms:
* info() to return a map with information about persistent terms.
* get() to return a list of a {Key,Value} tuples for all persistent
terms. (The values are not copied.)
|
|
This introduces a similar optimisation for normal funs
to what was introduced for external funs in #1725.
It is possible to allocate the fun as a literal, if it does not capture
the environment (i.e. it does not close over any variables).
Unfortunately it's not possible to do this in the compiler due to
problems with representation of such functions in the `.beam` files.
Fortunately, we can do this in the loader.
Simple evaluation shows that functions that don't capture the
enviornment consistute over 60% of all funs in the source code of
Erlang/OTP itself.
The only downside is that we lose a meningful value in the `pid` field
of the fun. The goal of this field, beyond debugging, was to be
able to identify the original node of a function. To be able to still do
this, the functions that are created in the loader are assigned the init
pid as the creator.
To solve issues with staryp, initially set the `erts_init_process_id`
to `ERTS_INVALID_PID` and skip the described optimisation if the value
is still uninitialised.
|
|
* rickard/process_info/OTP-14966:
New process_info() implementation using signals
|
|
|
|
63e1c58d27ab (PR #1725) started to compile external funs
as literals.
This commit updates the dumping of literal areas to dump
external fun literals to the crash dump.
|
|
Communication between Erlang processes has conceptually always been
performed through asynchronous signaling. The runtime system
implementation has however previously preformed most operation
synchronously. In a system with only one true thread of execution, this
is not problematic (often the opposite). In a system with multiple threads
of execution (as current runtime system implementation with SMP support)
it becomes problematic. This since it often involves locking of structures
when updating them which in turn cause resource contention. Utilizing
true asynchronous communication often avoids these resource contention
issues.
The case that triggered this change was contention on the link lock due
to frequent updates of the monitor trees during communication with a
frequently used server. The signal order delivery guarantees of the
language makes it hard to change the implementation of only some signals
to use true asynchronous signaling. Therefore the implementations
of (almost) all signals have been changed.
Currently the following signals have been implemented as true
asynchronous signals:
- Message signals
- Exit signals
- Monitor signals
- Demonitor signals
- Monitor triggered signals (DOWN, CHANGE, etc)
- Link signals
- Unlink signals
- Group leader signals
All of the above already defined as asynchronous signals in the
language. The implementation of messages signals was quite
asynchronous to begin with, but had quite strict delivery constraints
due to the ordering guarantees of signals between a pair of processes.
The previously used message queue partitioned into two halves has been
replaced by a more general signal queue partitioned into three parts
that service all kinds of signals. More details regarding the signal
queue can be found in comments in the erl_proc_sig_queue.h file.
The monitor and link implementations have also been completely replaced
in order to fit the new asynchronous signaling implementation as good
as possible. More details regarding the new monitor and link
implementations can be found in the erl_monitor_link.h file.
|
|
|
|
|
|
|
|
by testing F_SENSITIVE only once.
|
|
Exclude garbing processes, EXCEPT if run by crash dumping thread
in which case we assume the heap is healthy
without any move markers yet/left.
Switched order between (allocating) setup_rootset()
and (move marking) collect_live_heap_frags().
|
|
* maint:
Use base64 encoding in crash dumps
Correct parsing of sub binaries
Generalize passing of options for decoding
|
|
This will reduce the size of crash dumps, especially if
there are large binaries.
|
|
* maint:
erl_process_dump: Don't assume that literals can be found
Remove one superfluous closing parenthesis in oam_intro.xml
|
|
Native code does not register its literals in the code header for the
loaded code. Therefore, a literal created by native code can not be
found by mark_literal(). Ignore literals that can't be found instead
of crashing (the crasdump_viewer will report such literals as
incomplete heap data, but will not crash).
|
|
* maint:
Bump version of crash dumps to 0.4
Verify that binaries of different sizes are dumped correctly
Don't dump literal areas that are not referenced at all
Dump literals separately to avoid incomplete heap data
Implement dumping of maps in crash dumps
Buffer writing of crash dumps
Conflicts:
erts/emulator/beam/erl_alloc.types
|
|
|
|
When a literal was used from several processes, the literal would
be dumped in only one of the processes. The other processes
that referenced the literals would have incomplete heap data.
|
|
Maps would be dumped as the atom 'undefined', which is
not very informative.
|
|
|
|
|
|
to replace macro ERTS_INTERNAL_BINARY_FIELDS
as header in Binary and friends.
|
|
* maint:
Atomic reference count of binaries also in non-SMP
Conflicts:
erts/emulator/beam/erl_fun.c
|
|
OTP-14202
* rickard/binary-refc:
Atomic reference count of binaries also in non-SMP
Conflicts:
erts/emulator/beam/beam_bp.c
|
|
NIF resources was not handled in a thread-safe manner in the runtime
system without SMP support.
As a consequence of this fix, the following driver functions are now
thread-safe also in the runtime system without SMP support:
- driver_free_binary()
- driver_realloc_binary()
- driver_binary_get_refc()
- driver_binary_inc_refc()
- driver_binary_dec_refc()
|
|
* maint:
Fix issues with abandoned heap
Conflicts:
erts/emulator/beam/beam_bif_load.c
|
|
|
|
|
|
Instead of passing around a file descriptor
use a function pointer to facilitate more advanced
backend write logic such as size limitation or compression.
|
|
|
|
This commit adds two new structs to be used to represent
erlang code in erts.
ErtsCodeInfo is used to describe the i_func_info header
that is part of all Export entries and the prelude of
each function. This replaces all the BeamInstr * that
were previously used to point to these locations.
After this change the code should never use BeamInstr *
with offsets to figure out different parts of the
func_info header.
ErtsCodeMFA is a struct that is used to descripe a
MFA in code. It is used within ErtsCodeInfo and also
in Process->current.
All function that previously took Eterm * or BeamInstr *
to identify a MFA now use the ErtsCodeMFA or ErtsCodeInfo
where appropriate.
The code has been tested to work when adding a new field to the
ErtsCodeInfo struct, but some updates are needed in ops.tab to
make it work.
|
|
- Large Maps could cause a stack overrun during crash dump generation.
- This is a simple workaround until a solution has been implemented.
- The error has no impact on a running system.
|
|
- Termination of a process...
- Modify trace flags of process...
- Process info on process...
- Register/unregister of name on process...
- Set group leader on process...
... while it is executing a dirty NIF.
|
|
* henrik/update-copyrightyear:
update copyright-year
|
|
|
|
|
|
* rickard/ds-fixes:
Fix unique_SUITE for dirty schedulers
Add dirty scheduler process termination test
Ensure that work is done on the correct type of schedulers
Conflicts:
erts/emulator/beam/erl_process.c
erts/emulator/beam/erl_process.h
erts/emulator/beam/erl_process_dump.c
|
|
Only the actual call to the dirty nif is allowed to execute on
dirty schedulers. The dirty nif is not allowed to execute on
normal schedulers if dirty schedulers are available.
Arrival of exit signals and system tasks, while a process was
scheduled for execution on a dirty scheduler, could mess up the
process internal state.
Preparation for dirty system task has been made, but is currently
unused.
|
|
* The youngest generation of the heap can now consist of multiple
blocks. Heap fragments and message fragments are added to the
youngest generation when needed without triggering a GC. After
a GC the youngest generation is contained in one single block.
* The off_heap_message_queue process flag has been added. When
enabled all message data in the queue is kept off heap. When
a message is selected from the queue, the message fragment (or
heap fragment) containing the actual message is attached to the
youngest generation. Messages stored off heap is not part of GC.
|
|
* Removed COMPRESS_POINTER and EXPAND_POINTER
|
|
|
|
This commit improves crash dumps in several ways:
* Suspends schedulers to get a current snapshot
* Dumps information about scheduler
* Dumps stack trace of current running process
(including Garbing processes)
|
|
When dumping the heaps and stacks in the erl_crash.dump file, only
the lower 32 bits of pointers would be printed. Ensure that all bits
are printed.
While at it, make sure that printing sub binaries with a debug compiled
run-time system will work.
|
|
|
|
|
|
rickard/r16/port-optimizations/OTP-10336
* rickard/port-optimizations/OTP-10336:
Change annotate level for emacs-22 in cerl
Update etp-commands
Add documentation on communication in Erlang
Add support for busy port message queue
Add driver callback epilogue
Implement true asynchronous signaling between processes and ports
Add erl_drv_[send|output]_term
Move busy port flag
Use rwlock for driver list
Optimize management of port tasks
Improve configuration of process and port tables
Remove R9 compatibility features
Use ptab functionality also for ports
Prepare for use of ptab functionality also for ports
Atomic port state
Generalize process table implementation
Implement functionality for delaying thread progress from unmanaged threads
Conflicts:
erts/doc/src/erl_driver.xml
erts/doc/src/erlang.xml
erts/emulator/beam/beam_bif_load.c
erts/emulator/beam/beam_bp.c
erts/emulator/beam/beam_emu.c
erts/emulator/beam/bif.c
erts/emulator/beam/copy.c
erts/emulator/beam/erl_alloc.c
erts/emulator/beam/erl_alloc.types
erts/emulator/beam/erl_bif_info.c
erts/emulator/beam/erl_bif_port.c
erts/emulator/beam/erl_bif_trace.c
erts/emulator/beam/erl_init.c
erts/emulator/beam/erl_message.c
erts/emulator/beam/erl_port_task.c
erts/emulator/beam/erl_process.c
erts/emulator/beam/erl_process.h
erts/emulator/beam/erl_process_lock.c
erts/emulator/beam/erl_trace.c
erts/emulator/beam/export.h
erts/emulator/beam/global.h
erts/emulator/beam/io.c
erts/emulator/sys/unix/sys.c
erts/emulator/sys/vxworks/sys.c
erts/emulator/test/port_SUITE.erl
erts/etc/unix/cerl.src
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/prim_inet.beam
erts/preloaded/src/prim_inet.erl
lib/hipe/cerl/erl_bif_types.erl
lib/kernel/doc/src/inet.xml
lib/kernel/src/inet.erl
|
