| Age | Commit message (Collapse) | Author |
|---|
|
A step toward better integration of hipe load and purge
Highlights:
* code_server no longer needs to call hipe_unified_loader:post_beam_load/1
Instead new internal function hipe_redirect_to_module()
is called by loading BIFs to patch native call sites if needed.
* hipe_purge_module() is called by erts_internal:purge_module/2
to purge any native code.
* struct hipe_mfa_info redesigned and only used for exported
functions that are called from or implemented by native code.
A list of native call sites (struct hipe_ref) are kept for each hipe_mfa_info.
* struct hipe_sdesc used by hipe_find_mfa_from_ra()
to build native stack traces.
|
|
* maint:
erts: Add nif_SUITE:t_on_load
erts: Improve nif_SUITE:upgrade test
Don't leak old code when loading a modules with an on_load function
Conflicts:
erts/preloaded/ebin/erts_code_purger.beam
erts/preloaded/ebin/erts_internal.beam
erts/preloaded/src/erts_code_purger.erl
|
|
Normally, calling code:delete/1 before re-loading the code for a
module is unnecessary but causes no problem.
But there will be be problems if the new code has an on_load function.
Code with an on_load function will always be loaded as old code
to allowed it to be easily purged if the on_load function would fail.
If the on_load function succeeds, the old and current code will be
swapped.
So in the scenario where code:delete/1 has been called explicitly,
there is old code but no current code. Loading code with an
on_load function will cause the reference to the old code to be
overwritten. That will at best cause a memory leak, and at worst
an emulator crash (especially if NIFs are involved).
To avoid that situation, we will put the code with the on_load
function in a special, third slot in Module.
ERL-240
|
|
* maint:
Introduce literal sweep of native stack in new purge strategy
|
|
|
|
* rickard/remove-old-purge-strategy/OTP-13844:
Avoid selective receive in code-purger process
Remove old purge strategy
Conflicts:
erts/emulator/beam/beam_bif_load.c
erts/emulator/beam/global.h
|
|
* maint:
Fix reclaim of literal areas
Conflicts:
erts/emulator/beam/global.h
|
|
* rickard/new-purge-strategy/OTP-13833:
Fix reclaim of literal areas
Conflicts:
erts/emulator/beam/beam_bif_load.c
|
|
|
|
|
|
* maint:
Fix tracing of processes executing dirty
Perform check_process_code while process is executing dirty
Fix purge of code
Reclaim literal area after purge has completed
Separate literal area from code
Conflicts:
erts/emulator/beam/global.h
|
|
* rickard/ds-purge-module/OTP-13808:
Perform check_process_code while process is executing dirty
Conflicts:
erts/doc/src/erl_nif.xml
|
|
'rickard/new-purge-strategy/OTP-13833' into maint
* rickard/fun-purge-bug/OTP-13809:
Fix purge of code
Reclaim literal area after purge has completed
Separate literal area from code
Conflicts:
erts/doc/src/erlang.xml
erts/emulator/beam/beam_bif_load.c
erts/emulator/beam/erl_init.c
erts/preloaded/ebin/init.beam
|
|
|
|
Ensure that we cannot get any dangling pointers into code that
has been purged. This is done by a two phase purge. At first
phase all fun entries pointing into the code to purge are marked
for purge. All processes trying to call these funs will be suspended
and by this we avoid getting new direct references into the code.
When all processes has been checked, these processes are resumed.
The new purge strategy now also completely ignore the existence of
indirect references to the code (funs). If such exist, they will
cause bad fun exceptions to the caller, but will not prevent a
soft purge or cause a kill of a process having such live references
during a hard purge. This since it is impossible to give any
guarantees that no processes in the system have such indirect
references. Even when the system is completely clean from such
references, new ones can appear via distribution and/or disk.
|
|
|
|
|
|
* egil/erts/dont-copy-literals/OTP-13529:
erts: Refactor literal purge area arguments
erts: No need to literal opt message token
erts: Don't copy literals in enif_send
erts: Fix literal size calculation in check_process_code
erts: Remove the need for copying of literals
|
|
We want to know the total size of literals of all heap fragmens.
|
|
Because check_process_code neglected checking the HiPE stack for
references to the literal area, such references would survive the purge
and subsequent deletion of a module and its literal area. These dangling
references would then cause incorrect behaviour or even hard crashes of
the VM.
By simply adding a scan of the HiPE stack to check_process_code and
erts_garbage_collect_literals, this problem is fixed.
In order to support full stack walks without deleting the graylimit
trap, a new stack walking interface function,
nstack_walk_init_sdesc_ignore_trap() was introduced.
|
|
|
|
During check process code, explicitly copy all referenced literals
in a message (in the private queue) to a heap fragment and attach
it to the message reference.
Not all types of message communication does an explicit copy of a
literal and this needs to be taken care of before a module is purged.
|
|
* bjorn/fix-on_load/OTP-12593:
Update documentation regarding improvements
Correctly handle multiple load attempts when on_load is pending
Avoid deadlock when an on_load function makes an external call to the module itself
code_server: Eliminate unnecessary Tag in handle_call/3
Reimplement -on_load()
Refactor erts_finish_loading() and insert_new_code()
code_SUITE: Make on_load_binary/1 clearer by using merl
|
|
|
|
Load the module as old code; swap old and new code if the
-on_load function succeeds. That way, a failed update attempt
for a module that has an -on_load function will preserve the
previous version of the code.
|
|
|
|
Add the possibility to use modules as trace data receivers. The functions
in the module have to be nifs as otherwise complex trace probes will be
very hard to handle (complex means trace probes for ports for example).
This commit changes the way that the ptab->tracer field works from always
being an immediate, to now be NIL if no tracer is present or else be
the tuple {TracerModule, TracerState} where TracerModule is an atom that
is later used to lookup the appropriate tracer callbacks to call and
TracerState is just passed to the tracer callback. The default process and
port tracers have been rewritten to use the new API.
This commit also changes the order which trace messages are delivered to the
potential tracer process. Any enif_send done in a tracer module may be delayed
indefinitely because of lock order issues. If a message is delayed any other
trace message send from that process is also delayed so that order is preserved
for each traced entity. This means that for some trace events (i.e. send/receive)
the events may come in an unintuitive order (receive before send) to the
trace receiver. Timestamps are taken when the trace message is generated so
trace messages from differented processes may arrive with the timestamp
out of order.
Both the erlang:trace and seq_trace:set_system_tracer accept the new tracer
module tracers and also the backwards compatible arguments.
OTP-10267
|
|
|
|
|
|
The BIFs prepare_loading/2 and finish_loading/1 have been
designed to allow fast loading in parallel of many modules.
Because of the complications with on_load functions,
the initial implementation of finish_loading/1 only allowed
a single element in the list of prepared modules.
finish_loading/1 does not suspend other processes, but it must wait
for all schedulers to pass a write barrier ("thread progress"). The
time for all schedulers to pass the write barrier is highly variable,
depending on what kind of code they are executing. Therefore, allowing
finish_loading/1 to finish the loading for more than one module before
passing the write barrier could potentially be much faster than
calling finish_loading/1 multiple times.
The test case many/1 run on my computer shows that with "heavy load",
finish loading of 100 modules in parallel is almost 50 times faster
than loading them sequentially. With "light load", the gain is still
almost 10 times.
Here follows an actual sample of the output from the test case on
my computer (an 2012 iMac):
Light load
==========
Sequential: 22361 µs
Parallel: 2586 µs
Ratio: 9
Heavy load
==========
Sequential: 254512 µs
Parallel: 5246 µs
Ratio: 49
|
|
This avoids potential integer arithmetic overflow for very large lists.
|
|
by ignoring literals.
erts_internal:check_process_code will be called again anyway
(with option {copy_literals, true}) before the module is actually purged.
No need to check literals twice.
|
|
* Same process must do enable-disable.
* System process will force it and never get 'aborted'
|
|
as it's not a public interface.
|
|
Problem: erlang:purge_module/1 is not safe in the sense
that very bad things may happen if the code to be purged
is still referred to by live processes.
Introduce erts_internal:purge_module which is the same as the old
erlang:purge_module BIF (except it returns false if no such old module).
Implement erlang:purge_module in Erlang and let it invoke
erts_code_purger for safe purging where all clogging processes
first are killed.
|
|
OTP-13167
* sverk/proc-dict-opt:
erts: Add new test case pdict_SUITE:mixed
erts: Add 'fill_heap' to erts_debug:state_internal_state
erts: Rename proc dict size to arraySize
erts: Refactor proc dict with 'usedSlots'
erts: Add sizeMask for faster proc dict indexing
erts: Remove ProcDict.used
erts: Add proc dict macros ERTS_PD_START/SIZE
erts: Optimize away function "array_put" in proc dict
erts: Optimize hashing in process dictionary
|
|
|
|
* rickard/ohmq-fixup/OTP-13047:
Fix check_process_code()
|
|
|
|
|
|
|
|
|
|
|
|
* The youngest generation of the heap can now consist of multiple
blocks. Heap fragments and message fragments are added to the
youngest generation when needed without triggering a GC. After
a GC the youngest generation is contained in one single block.
* The off_heap_message_queue process flag has been added. When
enabled all message data in the queue is kept off heap. When
a message is selected from the queue, the message fragment (or
heap fragment) containing the actual message is attached to the
youngest generation. Messages stored off heap is not part of GC.
|
|
to use a real C struct instead of array.
|
|
|
|
|
|
* Removed COMPRESS_POINTER and EXPAND_POINTER
|
|
|
|
* egil/fix-purge-literals/OTP-12821:
erts: Fix garbage collect literals in code purge
|
