aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam/erl_process.h
AgeCommit message (Collapse)Author
2016-05-31Update process state flags in etp-commandsRickard Green
2016-05-31Move dirty reduction count to erts_dirty_process_main()Rickard Green
2016-05-17Merge branch 'mikpe/otp-19-erts-integer-truncation-bugs/PR-1045/OTP-13606'Lukas Larsson
* mikpe/otp-19-erts-integer-truncation-bugs/PR-1045/OTP-13606: erl_unicode.c: fix integer truncation problems do not limit heap fragments to 4 giga-words erts_new_mso_binary(): do not truncate len Conflicts: erts/emulator/beam/erl_nif.c
2016-05-17erts: Move max_heap_size field so that ErLLVM works againLukas Larsson
2016-05-11Merge branch 'rickard/ds-proc-exit/OTP-13123'Rickard Green
* rickard/ds-proc-exit/OTP-13123: Add dirty_heap_access test case Add dirty_call_while_terminated test case Move dirty nif test cases into dirty_nif_SUITE Add better support for communication with a process executing dirty NIF Remove conditional dirty schedulers API
2016-05-11Add better support for communication with a process executing dirty NIFRickard Green
- Termination of a process... - Modify trace flags of process... - Process info on process... - Register/unregister of name on process... - Set group leader on process... ... while it is executing a dirty NIF.
2016-05-10erts: Implement max_heap_size process flagLukas Larsson
The max_heap_size process flag can be used to limit the growth of a process heap by killing it before it becomes too large to handle. It is possible to set the maximum using the `erl +hmax` option, `system_flag(max_heap_size, ...)`, `spawn_opt(Fun, [{max_heap_size, ...}])` and `process_flag(max_heap_size, ...)`. It is possible to configure the behaviour of the process when the maximum heap size is reached. The process may be sent an untrappable exit signal with reason kill and/or send an error_logger message with details on the process state. A new trace event called gc_max_heap_size is also triggered for the garbage_collection trace flag when the heap grows larger than the configured size. If kill and error_logger are disabled, it is still possible to see that the maximum has been reached by doing garbage collection tracing on the process. The heap size is defined as the sum of the heap memory that the process is currently using. This includes all generational heaps, the stack, any messages that are considered to be part of the heap and any extra memory the garbage collector may need during collection. In the current implementation this means that when a process is set using on_heap message queue data mode, the messages that are in the internal message queue are counted towards this value. For off_heap, only matched messages count towards the size of the heap. For mixed, it depends on race conditions within the VM whether a message is part of the heap or not. Below is an example run of the new behaviour: Eshell V8.0 (abort with ^G) 1> f(P),P = spawn_opt(fun() -> receive ok -> ok end end, [{max_heap_size, 512}]). <0.60.0> 2> erlang:trace(P, true, [garbage_collection, procs]). 1 3> [P ! lists:duplicate(M,M) || M <- lists:seq(1,15)],ok. ok 4> =ERROR REPORT==== 26-Apr-2016::16:25:10 === Process: <0.60.0> Context: maximum heap size reached Max heap size: 512 Total heap size: 723 Kill: true Error Logger: true GC Info: [{old_heap_block_size,0}, {heap_block_size,609}, {mbuf_size,145}, {recent_size,0}, {stack_size,9}, {old_heap_size,0}, {heap_size,211}, {bin_vheap_size,0}, {bin_vheap_block_size,46422}, {bin_old_vheap_size,0}, {bin_old_vheap_block_size,46422}] flush(). Shell got {trace,<0.60.0>,gc_start, [{old_heap_block_size,0}, {heap_block_size,233}, {mbuf_size,145}, {recent_size,0}, {stack_size,9}, {old_heap_size,0}, {heap_size,211}, {bin_vheap_size,0}, {bin_vheap_block_size,46422}, {bin_old_vheap_size,0}, {bin_old_vheap_block_size,46422}]} Shell got {trace,<0.60.0>,gc_max_heap_size, [{old_heap_block_size,0}, {heap_block_size,609}, {mbuf_size,145}, {recent_size,0}, {stack_size,9}, {old_heap_size,0}, {heap_size,211}, {bin_vheap_size,0}, {bin_vheap_block_size,46422}, {bin_old_vheap_size,0}, {bin_old_vheap_block_size,46422}]} Shell got {trace,<0.60.0>,exit,killed}
2016-05-05do not limit heap fragments to 4 giga-wordsMikael Pettersson
- struct erl_heap_fragment: use Uint for sizes - erl_nif.c: use size_t for sizes - erts_heap_frag_shrink(): remove now redundant cast
2016-05-04Merge branch 'rickard/reds-fix/master/OTP-13512'Rickard Green
* rickard/reds-fix/master/OTP-13512: Ensure correct reduction counting
2016-05-04Ensure correct reduction countingRickard Green
2016-04-29erl_process: Restore R18 HiPE PCB offsetsMagnus Lång
It was recently observed that the LLVM backend for HiPE was broken, generating code that crashes the VM. It turns out that this was caused by LLVM hard-coding an offset into the Erlang PCB, which became incorrect as of commit 3ac08f9b, that introduced a new member into the PCB before the HiPE-specific state, changing its offsets. For now, until a proper fix is implemented, we move this new member to after the HiPE-specific state, allowing the LLVM backend to work once more.
2016-04-15erts: Extend process and port tracingLukas Larsson
This commit completes the tracing for processes so that all messages sent by a process (via nifs or otherwise) will be traced. The commit also adds tracing of all types of events from ports. When enabling tracing using erlang:trace, the 'all' flag now also enables tracing on all ports. OTP-13496
2016-04-15erts: Implement tracer modulesLukas Larsson
Add the possibility to use modules as trace data receivers. The functions in the module have to be nifs as otherwise complex trace probes will be very hard to handle (complex means trace probes for ports for example). This commit changes the way that the ptab->tracer field works from always being an immediate, to now be NIL if no tracer is present or else be the tuple {TracerModule, TracerState} where TracerModule is an atom that is later used to lookup the appropriate tracer callbacks to call and TracerState is just passed to the tracer callback. The default process and port tracers have been rewritten to use the new API. This commit also changes the order which trace messages are delivered to the potential tracer process. Any enif_send done in a tracer module may be delayed indefinitely because of lock order issues. If a message is delayed any other trace message send from that process is also delayed so that order is preserved for each traced entity. This means that for some trace events (i.e. send/receive) the events may come in an unintuitive order (receive before send) to the trace receiver. Timestamps are taken when the trace message is generated so trace messages from differented processes may arrive with the timestamp out of order. Both the erlang:trace and seq_trace:set_system_tracer accept the new tracer module tracers and also the backwards compatible arguments. OTP-10267
2016-04-13Merge branch 'henrik/update-copyrightyear'Henrik Nord
* henrik/update-copyrightyear: update copyright-year
2016-03-24Improve process/port specific data managementRickard Green
2016-03-15update copyright-yearHenrik Nord
2016-03-10Improved scheduler suspend functionalityRickard Green
- The calling process is now suspended while synchronizing scheduler suspends via erlang:system_flag(schedulers_online, _) and erlang:system_flag(multi_scheduling, _), instead of blocking the scheduler thread in the BIF call waiting for the operation to synchronize. Besides releasing the scheduler for other work (or immediate suspend) it also makes it possible to abort the operation by killing the process. - erlang:system_flag(schedulers_online, _) now only wait for normal schedulers to complete before it returns. This since it may take a very long time before all dirty schedulers suspends. - erlang:system_flag(multi_scheduling, block_normal|unblock_normal) which only operate on normal schedulers has been introduced. This since there are use cases where suspend of dirty schedulers are not of interest (hipe loader). - erlang:system_flag(multi_scheduling, block) still blocks all dirty schedulers as well as all normal schedulers except one since it is hard to redefine what multi scheduling block means. - The three operations: - changing amount of schedulers online - blocking/unblocking normal multi scheduling - blocking/unblocking full multi scheduling can now be done in parallel. This is important since otherwise a full multi scheduling block would potentially delay the other operations for a very long time.
2016-03-01Merge branch 'sverk/master/halt-INT_MIN'Sverker Eriksson
* sverk/master/halt-INT_MIN: erts: Make erlang:halt() accept bignums as Status erts: Change erl_exit into erts_exit kernel: Remove calls to erl_exit
2016-02-25Merge branch 'maint'Björn-Egil Dahlberg
Conflicts: erts/emulator/beam/erl_alloc.types erts/emulator/beam/erl_bif_info.c erts/emulator/beam/erl_process.c erts/preloaded/ebin/erts_internal.beam
2016-02-25Merge branch 'egil/heart-callback/OTP-13250' into maintBjörn-Egil Dahlberg
2016-02-24Merge branch 'master' into sverk/master/halt-INT_MINSverker Eriksson
2016-02-24erts: Change erl_exit into erts_exitSverker Eriksson
This is mostly a pure refactoring. Except for the buggy cases when calling erlang:halt() with a positive integer in the range -(INT_MIN+2) to -INT_MIN that got confused with ERTS_ABORT_EXIT, ERTS_DUMP_EXIT and ERTS_INTR_EXIT. Outcome OLD erl_exit(n, ) NEW erts_exit(n, ) ------- ------------------- ------------------------------------------- exit(Status) n = -Status <= 0 n = Status >= 0 crashdump+abort n > 0, ignore n n = ERTS_ERROR_EXIT < 0 The outcome of the old ERTS_ABORT_EXIT, ERTS_INTR_EXIT and ERTS_DUMP_EXIT are the same as before (even though their values have changed).
2016-02-19Merge branch 'rickard/rq-state-bug/OTP-13298' into maintRickard Green
* rickard/rq-state-bug/OTP-13298: Fix bug causing run-queue mask to become inconsistent
2016-02-16erts: Add BIF erts_internal:system_check/1Björn-Egil Dahlberg
This commit implements erts_internal:system_check(schedulers) with the intent of a basic responsiveness test check of the schedulers.
2016-02-15Merge branch 'rickard/ds-fixes' into rickard/ds-fixes-masterRickard Green
* rickard/ds-fixes: Fix unique_SUITE for dirty schedulers Add dirty scheduler process termination test Ensure that work is done on the correct type of schedulers Conflicts: erts/emulator/beam/erl_process.c erts/emulator/beam/erl_process.h erts/emulator/beam/erl_process_dump.c
2016-02-15Ensure that work is done on the correct type of schedulersRickard Green
Only the actual call to the dirty nif is allowed to execute on dirty schedulers. The dirty nif is not allowed to execute on normal schedulers if dirty schedulers are available. Arrival of exit signals and system tasks, while a process was scheduled for execution on a dirty scheduler, could mess up the process internal state. Preparation for dirty system task has been made, but is currently unused.
2016-02-04Fix bug causing run-queue mask to become inconsistentRickard Green
2016-02-02Merge branch 'maint'Rickard Green
* maint: Introduce time management in native APIs Introduce time warp safe replacement for safe_fixed option Introduce time warp safe trace timestamp formats Conflicts: erts/emulator/beam/erl_bif_trace.c erts/emulator/beam/erl_driver.h erts/emulator/beam/erl_nif.h erts/emulator/beam/erl_trace.c erts/preloaded/ebin/erlang.beam
2016-02-02Merge branch 'rickard/monotonic-time-improvements/OTP-13222' into maintRickard Green
* rickard/monotonic-time-improvements/OTP-13222: Introduce time management in native APIs Introduce time warp safe replacement for safe_fixed option Introduce time warp safe trace timestamp formats
2016-01-20Introduce time warp safe trace timestamp formatsRickard Green
New timestamp options for trace, sequential trace, and system profile: - monotonic_timestamp - strict_monotonic_timestamp
2016-01-14Merge branch 'maint'Rickard Green
* maint: Do not allow aux work on dirty schedulers
2016-01-14Merge branch 'vinoski/dirty-sched-no-aux-work/OTP-13236' into maintRickard Green
* vinoski/dirty-sched-no-aux-work/OTP-13236: Do not allow aux work on dirty schedulers
2015-12-30Merge branch 'maint'Rickard Green
* maint: Light weight statistics of run queue lengths Conflicts: erts/preloaded/ebin/erlang.beam
2015-12-30Light weight statistics of run queue lengthsRickard Green
- statistics(total_run_queue_lengths) - statistics(run_queue_lengths) - statistics(total_active_tasks) - statistics(active_tasks) Conflicts: erts/emulator/beam/erl_process.c
2015-12-21Do not allow aux work on dirty schedulersSteve Vinoski
The nature of aux work is such that dirty schedulers should not attempt to perform it. Modify the code to ensure that dirty schedulers avoid aux work. Also fix an incorrect assumption about the size of a Uint in the ErtsDirtySchedId type.
2015-12-15Merge branch 'lukas/erts/forker'Lukas Larsson
* lukas/erts/forker: (28 commits) erts: Never abort in the forked child erts: Mend ASSERT makro for erl_child_setup erts: Allow enomem failures in port_SUITE erts: iter_port sleep longer on freebsd erts: Allow one dangling fd if there is a gethost port erts: Only use forker StackAck on freebsd erts: It is not possible to exit the forker driver erts: Add forker StartAck for port start flowcontrol erts: Fix large open_port arg segfault for win32 erts: Fix memory leak at async open port kernel: Remove cmd server for unix os:cmd erts: Add testcase for huge port environment erts: Move os_pid to port hash to child setup erts: Handle all EINTR and EAGAIN cases in child setup erts: Make child_setup work with large environments erts: Fix forker driver ifdefs for win32 erts: Fix uds socket handling for os x erts: Fix dereferencing of unaligned integer for sparc erts: Flatten too long io vectors in uds write erts: Add fd count test for spawn_driver ... Conflicts: erts/emulator/beam/erl_node_tables.c erts/preloaded/src/erts_internal.erl
2015-12-15erts: Create forker process for spawn driverLukas Larsson
Instead of forking from the beam process, we create a separate process in which all forks are done. This has several advantages: 1) performance: * don't have to close all fd's in the world * fork only has to copy stuff from a small process * work is done in a completely seperate process * a 3x performance increase has been measured, can be made even greater (10x) if we cache the environment in child setup 2) stability * the exec is done in another process than beam, which means that if the file that we exec to is on an nfs that is not available right now we will not block a scheduler until the nfs returns. 3) simplicity * don't have to deal with SIGCHLD in the erts Unfortunately, this solution also implies some badness. 1) There will always be a seperate process running together with beam on unix. This could be confusing and undesirable. 2) We have to transfer the entire environment to child_setup for each command. OTP-13088
2015-12-08Merge branch 'rickard/ohmq-fixup/OTP-13047'Rickard Green
* rickard/ohmq-fixup/OTP-13047: Replace off_heap_message_queue option with message_queue_data option Always use literal_alloc Distinguish between GC disabled by BIFs and other disabled GC Fix process_info(_, off_heap_message_queue) Off heap message queue test suite Remove unused variable Fix memory leaks
2015-12-08Replace off_heap_message_queue option with message_queue_data optionRickard Green
The message_queue_data option can have the values - off_heap - on_heap - mixed
2015-12-08Distinguish between GC disabled by BIFs and other disabled GCRickard Green
Processes remember heap fragments that are known to be fully live due to creation in a just called BIF that yields in the live_hf_end field. This field must not be used if we have not disabled GC in a BIF. F_DELAY_GC has been introduced in order to distinguish between to two different scenarios. - F_DISABLE_GC should *only* be used by BIFs. This when the BIF needs to yield while preventig a GC. - F_DELAY_GC should only be used when GC is temporarily disabled while the process is scheduled. A process must not be scheduled out while F_DELAY_GC is set.
2015-12-08Remove unused variableRickard Green
2015-11-26Merge branch 'maint'Sverker Eriksson
2015-11-12Merge branch 'rickard/ohmq/OTP-13047'Rickard Green
* rickard/ohmq/OTP-13047: Fragmented young heap generation and off_heap_message_queue option Refactor GC Introduce literal tag Conflicts: erts/doc/src/erlang.xml erts/emulator/beam/erl_gc.c
2015-11-12Fragmented young heap generation and off_heap_message_queue optionRickard Green
* The youngest generation of the heap can now consist of multiple blocks. Heap fragments and message fragments are added to the youngest generation when needed without triggering a GC. After a GC the youngest generation is contained in one single block. * The off_heap_message_queue process flag has been added. When enabled all message data in the queue is kept off heap. When a message is selected from the queue, the message fragment (or heap fragment) containing the actual message is attached to the youngest generation. Messages stored off heap is not part of GC.
2015-11-02erts: Remove ERTS_PSD_DIST_ENTRYSverker Eriksson
Not needed as it is always set to erts_this_dist_entry (on net_kernel).
2015-10-09erts: Remove vheap mature from process control blockBjörn-Egil Dahlberg
Binary vheap mature is not necessary for binary gc.
2015-07-14Merge branch 'lukas/ose/remove/OTP-12573'Lukas Larsson
* lukas/ose/remove/OTP-12573: Remove OSE from erl_interface ose: Remove all code related to the OSE port
2015-07-10ose: Remove all code related to the OSE portLukas Larsson
The OSE port is no longer supported and this commit removed it and any changes related to it. The things that were general improvements have been left in the code.
2015-07-01erts: Remove halfword !HEAP_ON_C_STACKBjörn-Egil Dahlberg
2015-06-18Change license text to APLv2Bruce Yinhe