aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam/erl_message.h
AgeCommit message (Collapse)Author
2018-03-21Implementation of true asynchronous signaling between processesRickard Green
Communication between Erlang processes has conceptually always been performed through asynchronous signaling. The runtime system implementation has however previously preformed most operation synchronously. In a system with only one true thread of execution, this is not problematic (often the opposite). In a system with multiple threads of execution (as current runtime system implementation with SMP support) it becomes problematic. This since it often involves locking of structures when updating them which in turn cause resource contention. Utilizing true asynchronous communication often avoids these resource contention issues. The case that triggered this change was contention on the link lock due to frequent updates of the monitor trees during communication with a frequently used server. The signal order delivery guarantees of the language makes it hard to change the implementation of only some signals to use true asynchronous signaling. Therefore the implementations of (almost) all signals have been changed. Currently the following signals have been implemented as true asynchronous signals: - Message signals - Exit signals - Monitor signals - Demonitor signals - Monitor triggered signals (DOWN, CHANGE, etc) - Link signals - Unlink signals - Group leader signals All of the above already defined as asynchronous signals in the language. The implementation of messages signals was quite asynchronous to begin with, but had quite strict delivery constraints due to the ordering guarantees of signals between a pair of processes. The previously used message queue partitioned into two halves has been replaced by a more general signal queue partitioned into three parts that service all kinds of signals. More details regarding the signal queue can be found in comments in the erl_proc_sig_queue.h file. The monitor and link implementations have also been completely replaced in order to fit the new asynchronous signaling implementation as good as possible. More details regarding the new monitor and link implementations can be found in the erl_monitor_link.h file.
2017-11-14Fix broken receive optimizationBjörn Gustavsson
When a ref is created before performing a receive that will only receive message containing that ref, there is a compiler optimization to avoid scanning messages that can't possible contain the newly created ref. Magnus Lång pointed out that the implementation of the optimization is flawed. Exceptions or recursive calls could cause the receive operation to scan the receive queue from a position beyond the expected message (that is, the message containing the ref would never be matched out). See the receive_opt_exception/1 and receive_opt_recursion/1 test cases in receive_SUITE. It turns out that we can simplify the implementation of the optimization while fixing the bug (suggested by Magnus Lång). We actually don't need the c_p->msg.mark field. It is enough to have c_p->msg.saved_pos; if it is non-zero, it is a valid position in the message qeueue. All we need to do is to ensure that we clear c_p->msg.saved_pos when a receive is exited normally or abnormally. We can clear c_p->msg.saved_pos in JOIN_MESSAGE(), since it is called both when leaving a receive because a message matched and because there was a timeout and the 'after' clause was executed. In addition, we need to clear c_p->msg.saved_pos when an exception is caught. https://bugs.erlang.org/browse/ERL-511
2017-07-17erts: Replace usage of all erts_smp prefixes to just ertsLukas Larsson
2017-07-17erts: Remove ERTS_SMP and USE_THREAD definesLukas Larsson
This refactor was done using the unifdef tool like this: for file in $(find erts/ -name *.[ch]); do unifdef -t -f defile -o $file $file; done where defile contained: #define ERTS_SMP 1 #define USE_THREADS 1 #define DDLL_SMP 1 #define ERTS_HAVE_SMP_EMU 1 #define SMP 1 #define ERL_BITS_REENTRANT 1 #define ERTS_USE_ASYNC_READY_Q 1 #define FDBLOCK 1 #undef ERTS_POLL_NEED_ASYNC_INTERRUPT_SUPPORT #define ERTS_POLL_ASYNC_INTERRUPT_SUPPORT 0 #define ERTS_POLL_USE_WAKEUP_PIPE 1 #define ERTS_POLL_USE_UPDATE_REQUESTS_QUEUE 1 #undef ERTS_HAVE_PLAIN_EMU #undef ERTS_SIGNAL_STATE
2016-08-30Ensure correct mbuf_sz value in process structureRickard Green
2016-05-26Fix message queue update on replacement and removal of messageRickard Green
2016-05-24Merge branch 'egil/erts/fix-purge_code-literals'Björn-Egil Dahlberg
* egil/erts/fix-purge_code-literals: erts: Refactor ERTS_MSG_COMBINED_HFRAG to heap fragment erts: Copy literals in messages on module purge erts: Add testcase for purge of literals
2016-05-20erts: Refactor ERTS_MSG_COMBINED_HFRAG to heap fragmentBjörn-Egil Dahlberg
2016-05-17Merge branch 'mikpe/otp-19-erts-integer-truncation-bugs/PR-1045/OTP-13606'Lukas Larsson
* mikpe/otp-19-erts-integer-truncation-bugs/PR-1045/OTP-13606: erl_unicode.c: fix integer truncation problems do not limit heap fragments to 4 giga-words erts_new_mso_binary(): do not truncate len Conflicts: erts/emulator/beam/erl_nif.c
2016-05-05do not limit heap fragments to 4 giga-wordsMikael Pettersson
- struct erl_heap_fragment: use Uint for sizes - erl_nif.c: use size_t for sizes - erts_heap_frag_shrink(): remove now redundant cast
2016-05-04erts: Add Sender in 'receive' trace matchspecSverker Eriksson
All 'EXIT' and monitor messages are sent from 'system' Timeouts are "sent" from 'clock_service'
2016-04-15erts: Implement tracer modulesLukas Larsson
Add the possibility to use modules as trace data receivers. The functions in the module have to be nifs as otherwise complex trace probes will be very hard to handle (complex means trace probes for ports for example). This commit changes the way that the ptab->tracer field works from always being an immediate, to now be NIL if no tracer is present or else be the tuple {TracerModule, TracerState} where TracerModule is an atom that is later used to lookup the appropriate tracer callbacks to call and TracerState is just passed to the tracer callback. The default process and port tracers have been rewritten to use the new API. This commit also changes the order which trace messages are delivered to the potential tracer process. Any enif_send done in a tracer module may be delayed indefinitely because of lock order issues. If a message is delayed any other trace message send from that process is also delayed so that order is preserved for each traced entity. This means that for some trace events (i.e. send/receive) the events may come in an unintuitive order (receive before send) to the trace receiver. Timestamps are taken when the trace message is generated so trace messages from differented processes may arrive with the timestamp out of order. Both the erlang:trace and seq_trace:set_system_tracer accept the new tracer module tracers and also the backwards compatible arguments. OTP-10267
2016-03-15update copyright-yearHenrik Nord
2015-12-09Merge branch 'sverk/dist-ctrl-msg-overflow-master' into masterSverker Eriksson
* sverk/dist-ctrl-msg-overflow-master: erts: Tweak hashmap heap size estimation erts: Fix bug for remote control message containing fat maps erts: Add test for remote exit signal with fat map erts: Fix bug in heap_factory_undo for FACTORY_HEAP_FRAGS mode
2015-12-08Merge branch 'rickard/ohmq-fixup/OTP-13047'Rickard Green
* rickard/ohmq-fixup/OTP-13047: Replace off_heap_message_queue option with message_queue_data option Always use literal_alloc Distinguish between GC disabled by BIFs and other disabled GC Fix process_info(_, off_heap_message_queue) Off heap message queue test suite Remove unused variable Fix memory leaks
2015-12-08Merge branch 'sverk/dist-ctrl-msg-overflow' into ↵Sverker Eriksson
sverk/dist-ctrl-msg-overflow-master
2015-12-08Replace off_heap_message_queue option with message_queue_data optionRickard Green
The message_queue_data option can have the values - off_heap - on_heap - mixed
2015-12-07erts: Fix bug for remote control message containing fat mapsSverker Eriksson
that could cause the static factory to overflow Fix: Introduce a new factory mode FACTORY_TMP
2015-11-19Refactor have seq_trace token testBjörn-Egil Dahlberg
2015-11-18Refactor sharing preserved copy flagsBjörn-Egil Dahlberg
The TMPBUF option is no longer needed due to is_literal test and NONE was only used for initial debugging. So we remove the entire option.
2015-11-17Refactor copy sharingBjörn-Egil Dahlberg
2015-11-17Enable shcopy for sending messagesNikolaos S. Papaspyrou
2015-11-12Fragmented young heap generation and off_heap_message_queue optionRickard Green
* The youngest generation of the heap can now consist of multiple blocks. Heap fragments and message fragments are added to the youngest generation when needed without triggering a GC. After a GC the youngest generation is contained in one single block. * The off_heap_message_queue process flag has been added. When enabled all message data in the queue is kept off heap. When a message is selected from the queue, the message fragment (or heap fragment) containing the actual message is attached to the youngest generation. Messages stored off heap is not part of GC.
2015-06-24erts: Remove HALFWORD_HEAP definitionBjörn-Egil Dahlberg
2015-06-18Change license text to APLv2Bruce Yinhe
2015-06-15erts: Add erts_factory_trim_and_closeSverker Eriksson
2015-06-15erts: Remove hashmap probabilistic heap overestimationSverker Eriksson
by adding a dynamic heap factory. "binary_to_term" is now a hybrid solution with both a call to decoded_size() to calculate needed heap space AND possible dynamic allocation of more heap space if needed for big maps. The heap size returned from decoded_size() is guaranteed to be sufficient for all term heap data except for hashmap nodes. All hashmap nodes are created at the end of dec_term() by invoking the heap factory interface that may allocate more heap space on process heap or in fragments. With this commit it is no longer guaranteed that a message is confined to only one heap fragment.
2015-05-08Merge branch 'rickard/timer-optimization/OTP-12650'Rickard Green
* rickard/timer-optimization/OTP-12650: Optimized timer implementation Reusable red-black tree implementation Conflicts: erts/emulator/beam/erl_bif_timer.c
2015-05-08Optimized timer implementationRickard Green
2015-04-14erts: Refactor erts_queue_messageBjörn-Egil Dahlberg
2015-03-20Merge branch 'rickard/time_api/OTP-11997'Rickard Green
* rickard/time_api/OTP-11997: (22 commits) Update primary bootstrap inets: Suppress deprecated warning on erlang:now/0 inets: Cleanup of multiple copies of functions Add inets_lib with common functions used by multiple modules inets: Update comments Suppress deprecated warning on erlang:now/0 Use new time API and be back-compatible in inets Remove unused functions and removed redundant test asn1 test SUITE: Eliminate use of now/0 Disable deprecated warning on erlang:now/0 in diameter_lib Use new time API and be back-compatible in ssh Replace all calls to now/0 in CT with new time API functions test_server: Replace usage of erlang:now() with usage of new API Replace usage of erlang:now() with usage of new API Replace usage of erlang:now() with usage of new API Replace usage of erlang:now() with usage of new API Replace usage of erlang:now() with usage of new API otp_SUITE: Warn for calls to erlang:now/0 Replace usage of erlang:now() with usage of new API Multiple timer wheels Erlang based BIF timer implementation for scalability Implement ethread events with timeout ... Conflicts: bootstrap/bin/start.boot bootstrap/bin/start_clean.boot bootstrap/lib/compiler/ebin/beam_asm.beam bootstrap/lib/compiler/ebin/compile.beam bootstrap/lib/kernel/ebin/auth.beam bootstrap/lib/kernel/ebin/dist_util.beam bootstrap/lib/kernel/ebin/global.beam bootstrap/lib/kernel/ebin/hipe_unified_loader.beam bootstrap/lib/kernel/ebin/inet_db.beam bootstrap/lib/kernel/ebin/inet_dns.beam bootstrap/lib/kernel/ebin/inet_res.beam bootstrap/lib/kernel/ebin/os.beam bootstrap/lib/kernel/ebin/pg2.beam bootstrap/lib/stdlib/ebin/dets.beam bootstrap/lib/stdlib/ebin/dets_utils.beam bootstrap/lib/stdlib/ebin/erl_tar.beam bootstrap/lib/stdlib/ebin/escript.beam bootstrap/lib/stdlib/ebin/file_sorter.beam bootstrap/lib/stdlib/ebin/otp_internal.beam bootstrap/lib/stdlib/ebin/qlc.beam bootstrap/lib/stdlib/ebin/random.beam bootstrap/lib/stdlib/ebin/supervisor.beam bootstrap/lib/stdlib/ebin/timer.beam erts/aclocal.m4 erts/emulator/beam/bif.c erts/emulator/beam/erl_bif_info.c erts/emulator/beam/erl_db_hash.c erts/emulator/beam/erl_init.c erts/emulator/beam/erl_process.h erts/emulator/beam/erl_thr_progress.c erts/emulator/beam/utils.c erts/emulator/sys/unix/sys.c erts/preloaded/ebin/erlang.beam erts/preloaded/ebin/erts_internal.beam erts/preloaded/ebin/init.beam erts/preloaded/src/erts_internal.erl lib/common_test/test/ct_hooks_SUITE_data/cth/tests/empty_cth.erl lib/diameter/src/base/diameter_lib.erl lib/kernel/src/os.erl lib/ssh/test/ssh_basic_SUITE.erl system/doc/efficiency_guide/advanced.xml
2015-03-20Erlang based BIF timer implementation for scalabilityRickard Green
2015-03-19erts: Fix map bug in dec_term for 32-bit debug VMSverker Eriksson
Adding ERTS_SWORD_MAX to a pointer does not work as a way to disable a bound check. Remove the hp_end from ErtsHeapFactory as it isn't really used anyway.
2015-03-12First stab at binary_to_term for hamtSverker Eriksson
with over estimation of heap size.
2013-11-18Execution of system tasks in context of another processRickard Green
A process requesting a system task to be executed in the context of another process will be notified by a message when the task has executed. This message will be on the form: {RequestType, RequestId, Pid, Result}. A process requesting a system task to be executed can set priority on the system task. The requester typically set the same priority on the task as its own process priority, and by this avoiding priority inversion. A request for execution of a system task is made by calling the statically linked in NIF erts_internal:request_system_task(Pid, Prio, Request). This is an undocumented ERTS internal function that should remain so. It should *only* be called from BIF implementations. Currently defined system tasks are: * garbage_collect * check_process_code Further system tasks can and will be implemented in the future. The erlang:garbage_collect/[1,2] and erlang:check_process_code/[2,3] BIFs are now implemented using system tasks. Both the 'garbage_collect' and the 'check_process_code' operations perform or may perform garbage_collections. By doing these via the system task functionality all garbage collect operations in the system will be performed solely in the context of the process being garbage collected. This makes it possible to later implement functionality for disabling garbage collection of a process over context switches. Newly introduced BIFs: * erlang:garbage_collect/2 - The new second argument is an option list. Introduced option: * {async, RequestId} - making it possible for users to issue asynchronous garbage collect requests. * erlang:check_process_code/3 - The new third argument is an option list. Introduced options: * {async, RequestId} - making it possible for users to issue asynchronous check process code requests. * {allow_gc, boolean()} - making it possible to issue requests that aren't allowed to garbage collect (operation will abort if gc should be needed). These options have been introduced as a preparation for parallelization of check_process_code operations when the code_server is about to purge a module.
2012-12-03Generalize process table implementationRickard Green
2012-08-02Use thread progress instead of scheduling misc aux work were possibleRickard Green
Functionality for scheduling operations at thread progress later has been introduced. Deallocation of ETS table structures were previously done by scheduling misc aux work. Deallocation of process structures (not released yet) was also implemented this way. Instead of using the misc aux work functionality these implementation now use the newly introduced functionality for scheduling operations at thread progress later. By using this new functionaliy we reduce the amount of memory allocation/deallocation operations needed.
2012-08-02Read message queue lengths while having lockRickard Green
2012-03-30Update copyright yearsBjörn-Egil Dahlberg
2012-03-22Change to more specific configure options for dtracePatrik Nyblom
2012-03-22Add user tag spreading functionality to VM and use in filePatrik Nyblom
User tags in a dynamic trace enabled VM are spread throughout the system in the same way as seq_trace tokens. This is used by the file module and various other modules to get hold of the tag from the user process without changing the protocol.
2010-08-12Increase vheap counter to Uint64Björn-Egil Dahlberg
This will reduce the risk of integer wrapping in bin vheap counting. The vheap size series will now use the golden ratio instead of doubling and fibonacci sequences. OTP #8730
2010-07-22Mending halfword heap emulatorSverker Eriksson
2010-07-20One off-heap list, to eliminate two words per ETS object.Sverker Eriksson
Merging the three off-heap lists (binaries, funs and externals) into one list. This reduces memory consumption by two words (pointers) per ETS object.
2010-06-03OTP-8555 Send message from NIFSverker Eriksson
New NIF features: Send messages from a NIF, or from thread created by NIF, to any local process (enif_send) Store terms between NIF calls (enif_alloc_env, enif_make_copy) Create binary terms with user defined memory management (enif_make_resource_binary)
2010-05-11erts: Implement recv_mark/1 and recv_set/1 for realBjörn Gustavsson
The recv_mark/1 instruction will both save the current position in the message queue and a mark (the address of the loop_rec/2 instruction just following the recv_set/1 instruction). The recv_mark/1 instruction will only use the saved position if the mark is correct. The reason for saving and verifying the mark is that the compiler does not need to guarantee that no other receive instruction can be executed in between the recv_mark/1 and recv_set/1 instructions (the mark will be cleared by the remove_message/0 instruction when a message is removed from the message queue). That means that arbitrary function calls in between those instruction can be allowed.
2010-03-10Store pointers to heap data in 32-bit wordsPatrik Nyblom
Store Erlang terms in 32-bit entities on the heap, expanding the pointers to 64-bit when needed. This works because all terms are stored on addresses in the 32-bit address range (the 32 most significant bits of pointers to term data are always 0). Introduce a new datatype called UWord (along with its companion SWord), which is an integer having the exact same size as the machine word (a void *), but might be larger than Eterm/Uint. Store code as machine words, as the instructions are pointers to executable code which might reside outside the 32-bit address range. Continuation pointers are stored on the 32-bit stack and hence must point to addresses in the low range, which means that loaded beam code much be placed in the low 32-bit address range (but, as said earlier, the instructions themselves are full words). No Erlang term data can be stored on C stacks (enforced by an earlier commit). This version gives a prompt, but test cases still fail (and dump core). The loader (and emulator loop) has instruction packing disabled. The main issues has been in rewriting loader and actual virtual machine. Subsystems (like distribution) does not work yet.
2010-01-13OTP-8240 Improved GC performance after BIF/NIF call when a lot of heapSverker Eriksson
fragments was created. This will mainly benefit NIFs that return large compound terms.
2009-11-20The R13B03 release.OTP_R13B03Erlang/OTP