aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam/erl_gc.c
AgeCommit message (Collapse)Author
2016-09-14Merge branch 'maint-19' into maintHenrik Nord
2016-09-14Ensure we dont use an invalid live heap fragment pointerRickard Green
2016-09-14Update vheap size when moving msgq to heap after GCRickard Green
2016-09-14Merge branch 'maint-19' into maintHenrik Nord
2016-09-13Fix crash due to heap fragments left after GCRickard Green
Do not decode distribution messages as part of the GC operation. Distribution messages containing maps may generate heap fragments.
2016-08-30Ensure correct mbuf_sz value in process structureRickard Green
2016-07-29Merge branch 'margnus1/erts/fix-hipe-literal-gc/PR-1122/OTP-13777' into maintBjörn-Egil Dahlberg
* margnus1/erts/fix-hipe-literal-gc/PR-1122/OTP-13777: check_process_code: Sweep HiPE stack for literals
2016-07-14erts: Fix msacc for dirty scheduler and heap_limitLukas Larsson
2016-07-06check_process_code: Sweep HiPE stack for literalsMagnus Lång
Because check_process_code neglected checking the HiPE stack for references to the literal area, such references would survive the purge and subsequent deletion of a module and its literal area. These dangling references would then cause incorrect behaviour or even hard crashes of the VM. By simply adding a scan of the HiPE stack to check_process_code and erts_garbage_collect_literals, this problem is fixed. In order to support full stack walks without deleting the graylimit trap, a new stack walking interface function, nstack_walk_init_sdesc_ignore_trap() was introduced.
2016-05-30erts: Clean up some goto spaghettiSverker Eriksson
and replace with a nice else-if chain.
2016-05-30erts: Fix ASSERT provoked by map_SUITE:t_rare_map_overflowSverker Eriksson
The same bug was fixed for OTP 18 in cb62c989e59f0ec8556f9f1d4e9a45b by provoking yet another GC. But now in 19 we are ok with heap fragments so just remove the asserts.
2016-05-26Fix message queue update on replacement and removal of messageRickard Green
2016-05-20erts: Refactor ERTS_MSG_COMBINED_HFRAG to heap fragmentBjörn-Egil Dahlberg
2016-05-11Merge branch 'rickard/ds-proc-exit/OTP-13123'Rickard Green
* rickard/ds-proc-exit/OTP-13123: Add dirty_heap_access test case Add dirty_call_while_terminated test case Move dirty nif test cases into dirty_nif_SUITE Add better support for communication with a process executing dirty NIF Remove conditional dirty schedulers API
2016-05-11Add better support for communication with a process executing dirty NIFRickard Green
- Termination of a process... - Modify trace flags of process... - Process info on process... - Register/unregister of name on process... - Set group leader on process... ... while it is executing a dirty NIF.
2016-05-11erts: Fix non-smp max_heap_size assertLukas Larsson
2016-05-10erts: Implement max_heap_size process flagLukas Larsson
The max_heap_size process flag can be used to limit the growth of a process heap by killing it before it becomes too large to handle. It is possible to set the maximum using the `erl +hmax` option, `system_flag(max_heap_size, ...)`, `spawn_opt(Fun, [{max_heap_size, ...}])` and `process_flag(max_heap_size, ...)`. It is possible to configure the behaviour of the process when the maximum heap size is reached. The process may be sent an untrappable exit signal with reason kill and/or send an error_logger message with details on the process state. A new trace event called gc_max_heap_size is also triggered for the garbage_collection trace flag when the heap grows larger than the configured size. If kill and error_logger are disabled, it is still possible to see that the maximum has been reached by doing garbage collection tracing on the process. The heap size is defined as the sum of the heap memory that the process is currently using. This includes all generational heaps, the stack, any messages that are considered to be part of the heap and any extra memory the garbage collector may need during collection. In the current implementation this means that when a process is set using on_heap message queue data mode, the messages that are in the internal message queue are counted towards this value. For off_heap, only matched messages count towards the size of the heap. For mixed, it depends on race conditions within the VM whether a message is part of the heap or not. Below is an example run of the new behaviour: Eshell V8.0 (abort with ^G) 1> f(P),P = spawn_opt(fun() -> receive ok -> ok end end, [{max_heap_size, 512}]). <0.60.0> 2> erlang:trace(P, true, [garbage_collection, procs]). 1 3> [P ! lists:duplicate(M,M) || M <- lists:seq(1,15)],ok. ok 4> =ERROR REPORT==== 26-Apr-2016::16:25:10 === Process: <0.60.0> Context: maximum heap size reached Max heap size: 512 Total heap size: 723 Kill: true Error Logger: true GC Info: [{old_heap_block_size,0}, {heap_block_size,609}, {mbuf_size,145}, {recent_size,0}, {stack_size,9}, {old_heap_size,0}, {heap_size,211}, {bin_vheap_size,0}, {bin_vheap_block_size,46422}, {bin_old_vheap_size,0}, {bin_old_vheap_block_size,46422}] flush(). Shell got {trace,<0.60.0>,gc_start, [{old_heap_block_size,0}, {heap_block_size,233}, {mbuf_size,145}, {recent_size,0}, {stack_size,9}, {old_heap_size,0}, {heap_size,211}, {bin_vheap_size,0}, {bin_vheap_block_size,46422}, {bin_old_vheap_size,0}, {bin_old_vheap_block_size,46422}]} Shell got {trace,<0.60.0>,gc_max_heap_size, [{old_heap_block_size,0}, {heap_block_size,609}, {mbuf_size,145}, {recent_size,0}, {stack_size,9}, {old_heap_size,0}, {heap_size,211}, {bin_vheap_size,0}, {bin_vheap_block_size,46422}, {bin_old_vheap_size,0}, {bin_old_vheap_block_size,46422}]} Shell got {trace,<0.60.0>,exit,killed}
2016-05-09Merge branch 'sverker/trace-send-receive-matchspec/OTP-13507'Sverker Eriksson
2016-05-04erts: Add Sender in 'receive' trace matchspecSverker Eriksson
All 'EXIT' and monitor messages are sent from 'system' Timeouts are "sent" from 'clock_service'
2016-05-04Merge branch 'rickard/reds-fix/master/OTP-13512'Rickard Green
* rickard/reds-fix/master/OTP-13512: Ensure correct reduction counting
2016-05-04Ensure correct reduction countingRickard Green
2016-05-03Merge branch 'lukas/erts/rename_xmqd_to_hmqd/OTP-13366'Lukas Larsson
* lukas/erts/rename_xmqd_to_hmqd/OTP-13366: erts: Rename erl flag +xmqd to +hmqd in erlexec Fix proc_lib message_queue_data spec erts: Fix total_heap_size calculation for on_heap erts: Rename erl flag +xmqd to +hmqd
2016-04-29Merge branch 'egil/erts/tracing-support-lttng/OTP-13532'Björn-Egil Dahlberg
* egil/erts/tracing-support-lttng/OTP-13532: (28 commits) runtime_tools: User's guide to LTTng and dyntrace runtime_tools: Fix Dtrace build erts: Fix gc messages in tracer_SUITE erts: Fix gc messages in sensitive_SUITE erts: Fix gc messages in trace_port_SUITE tools: Update fprof tests tools: Update fprof with new gc traces runtime_tools: Update dyntrace_lttng_SUITE tests runtime_tools: Add 'return_to' for call tracing erts: Fix return_to trace callback erts: Update erl_tracer documentation erts: Fix erl_tracer documentation typos Update preloaded erl_tracer.beam erts: Update erl_tracer type specs runtime_tools: Add lttng dyntrace tests runtime_tools: Extend 'trace' and 'enabled' tracer callbacks erts: Extend 'enabled' and 'trace' tracer callbacks runtime_tools: Extend 'enabled' tracer callbacks erts: Extend 'enabled' tracer callbacks runtime_tools: Update lttng garbage collection trace ...
2016-04-29erts: Extend garbage collection traceBjörn-Egil Dahlberg
Replace 'gc_start' and 'gc_end' with * 'gc_minor_start' * 'gc_minor_end' * 'gc_major_start' * 'gc_major_end'
2016-04-26erts: Fix total_heap_size calculation for on_heapLukas Larsson
2016-04-21erts: Fix copy share with onheap messagesBjörn-Egil Dahlberg
2016-04-15erts: Implement tracer modulesLukas Larsson
Add the possibility to use modules as trace data receivers. The functions in the module have to be nifs as otherwise complex trace probes will be very hard to handle (complex means trace probes for ports for example). This commit changes the way that the ptab->tracer field works from always being an immediate, to now be NIL if no tracer is present or else be the tuple {TracerModule, TracerState} where TracerModule is an atom that is later used to lookup the appropriate tracer callbacks to call and TracerState is just passed to the tracer callback. The default process and port tracers have been rewritten to use the new API. This commit also changes the order which trace messages are delivered to the potential tracer process. Any enif_send done in a tracer module may be delayed indefinitely because of lock order issues. If a message is delayed any other trace message send from that process is also delayed so that order is preserved for each traced entity. This means that for some trace events (i.e. send/receive) the events may come in an unintuitive order (receive before send) to the trace receiver. Timestamps are taken when the trace message is generated so trace messages from differented processes may arrive with the timestamp out of order. Both the erlang:trace and seq_trace:set_system_tracer accept the new tracer module tracers and also the backwards compatible arguments. OTP-10267
2016-03-15update copyright-yearHenrik Nord
2016-02-24Merge branch 'master' into sverk/master/halt-INT_MINSverker Eriksson
2016-02-24erts: Change erl_exit into erts_exitSverker Eriksson
This is mostly a pure refactoring. Except for the buggy cases when calling erlang:halt() with a positive integer in the range -(INT_MIN+2) to -INT_MIN that got confused with ERTS_ABORT_EXIT, ERTS_DUMP_EXIT and ERTS_INTR_EXIT. Outcome OLD erl_exit(n, ) NEW erts_exit(n, ) ------- ------------------- ------------------------------------------- exit(Status) n = -Status <= 0 n = Status >= 0 crashdump+abort n > 0, ignore n n = ERTS_ERROR_EXIT < 0 The outcome of the old ERTS_ABORT_EXIT, ERTS_INTR_EXIT and ERTS_DUMP_EXIT are the same as before (even though their values have changed).
2016-02-02erts: Add microstate accountingLukas Larsson
Microstate accounting is a way to track which state the different threads within ERTS are in. The main usage area is to pin point performance bottlenecks by checking which states the threads are in and then from there figuring out why and where to optimize. Since checking whether microstate accounting is on or off is relatively expensive if done in a short loop only a few of the states are enabled by default and more states can be enabled through configure. I've done some benchmarking and the overhead with it turned off is not noticible and with it on it is a fraction of a percent. If you enable the extra states, depending on the benchmark, the ovehead when turned off is about 1% and when turned on somewhere inbetween 5-15%. OTP-12345
2016-01-29Remove faulty assertsRickard Green
2016-01-26Merge branch 'lukas/erts/gc_info/OTP-13265'Lukas Larsson
* lukas/erts/gc_info/OTP-13265: erts: Add garbage_collection_info to process_info/2 Conflicts: erts/emulator/beam/erl_bif_info.c
2016-01-07Merge branch 'sverk/proc-dict-opt'Sverker Eriksson
OTP-13167 * sverk/proc-dict-opt: erts: Add new test case pdict_SUITE:mixed erts: Add 'fill_heap' to erts_debug:state_internal_state erts: Rename proc dict size to arraySize erts: Refactor proc dict with 'usedSlots' erts: Add sizeMask for faster proc dict indexing erts: Remove ProcDict.used erts: Add proc dict macros ERTS_PD_START/SIZE erts: Optimize away function "array_put" in proc dict erts: Optimize hashing in process dictionary
2015-12-18Merge branch 'rickard/ohmq-fixup/OTP-13047'Rickard Green
* rickard/ohmq-fixup/OTP-13047: Fix offset_mqueue
2015-12-16Fix offset_mqueueRickard Green
2015-12-09erts: Add proc dict macros ERTS_PD_START/SIZESverker Eriksson
2015-12-08Merge branch 'rickard/ohmq-fixup/OTP-13047'Rickard Green
* rickard/ohmq-fixup/OTP-13047: Replace off_heap_message_queue option with message_queue_data option Always use literal_alloc Distinguish between GC disabled by BIFs and other disabled GC Fix process_info(_, off_heap_message_queue) Off heap message queue test suite Remove unused variable Fix memory leaks
2015-12-08Replace off_heap_message_queue option with message_queue_data optionRickard Green
The message_queue_data option can have the values - off_heap - on_heap - mixed
2015-12-08Distinguish between GC disabled by BIFs and other disabled GCRickard Green
Processes remember heap fragments that are known to be fully live due to creation in a just called BIF that yields in the live_hf_end field. This field must not be used if we have not disabled GC in a BIF. F_DELAY_GC has been introduced in order to distinguish between to two different scenarios. - F_DISABLE_GC should *only* be used by BIFs. This when the BIF needs to yield while preventig a GC. - F_DELAY_GC should only be used when GC is temporarily disabled while the process is scheduled. A process must not be scheduled out while F_DELAY_GC is set.
2015-12-08Fix memory leaksRickard Green
2015-11-27erts: Fix rare case of faulty heap fragment deallocationSverker Eriksson
after major GC. Can only be caused by distributed messages containing large maps. Bad map hashing will increase the risk.
2015-11-17Add -debug +vc flag for debuging SHCOPYNikolaos S. Papaspyrou
This is very verbose, you have been warned. It should work with the copy-spy.py script, which may be a bit outdated.
2015-11-17Add the BIF size_shared/1 and debug cleanupNikolaos S. Papaspyrou
2015-11-17Add all the main machineryNikolaos S. Papaspyrou
Add functions size_shared, copy_shared_calculate and copy_shared_perform. Add the infrastructure for making these communicate with each other. Add debug information to other places in the VM, to watch interaction with the sharing-preserving copy. CAUTION: If you define the SHCOPY_DEBUG macro (after SHCOPY is actually used in the VM) and make the whole OTP, there will be a lot of debugging messages during make (it will also be enabled in erlc). You have been warned...
2015-11-16erts: Add garbage_collection_info to process_info/2Lukas Larsson
This info request returns greater details about the current gc state. This info is not included in the default process_info/1 as it would clutter the default printout with too much information.
2015-11-13Merge branch 'rickard/gc-bump-reds/OTP-13097'Rickard Green
* rickard/gc-bump-reds/OTP-13097: Bump reductions on GC
2015-11-12Merge branch 'rickard/ohmq/OTP-13047'Rickard Green
* rickard/ohmq/OTP-13047: Fragmented young heap generation and off_heap_message_queue option Refactor GC Introduce literal tag Conflicts: erts/doc/src/erlang.xml erts/emulator/beam/erl_gc.c
2015-11-12Merge branch 'sverk/literal-memory-range'Rickard Green
* sverk/literal-memory-range: erts: Refactor line table in loaded beam code erts: Refactor header of loaded beam code fix check_process_code for separate literal area erts: Add support for fast erts_is_literal() erts: Refactor erl_mmap to allow several mapper instances erts: Add new allocator LITERAL erts: Fix strangeness in treatment of MSEG_ALIGN_BITS erts: Cleanup main carrier creation erts: Remove unused erts_have_erts_mmap erts: Refactor config test for posix_memalign
2015-11-12Bump reductions on GCRickard Green