aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam/erl_nif.c
AgeCommit message (Collapse)Author
2018-03-21Implementation of true asynchronous signaling between processesRickard Green
Communication between Erlang processes has conceptually always been performed through asynchronous signaling. The runtime system implementation has however previously preformed most operation synchronously. In a system with only one true thread of execution, this is not problematic (often the opposite). In a system with multiple threads of execution (as current runtime system implementation with SMP support) it becomes problematic. This since it often involves locking of structures when updating them which in turn cause resource contention. Utilizing true asynchronous communication often avoids these resource contention issues. The case that triggered this change was contention on the link lock due to frequent updates of the monitor trees during communication with a frequently used server. The signal order delivery guarantees of the language makes it hard to change the implementation of only some signals to use true asynchronous signaling. Therefore the implementations of (almost) all signals have been changed. Currently the following signals have been implemented as true asynchronous signals: - Message signals - Exit signals - Monitor signals - Demonitor signals - Monitor triggered signals (DOWN, CHANGE, etc) - Link signals - Unlink signals - Group leader signals All of the above already defined as asynchronous signals in the language. The implementation of messages signals was quite asynchronous to begin with, but had quite strict delivery constraints due to the ordering guarantees of signals between a pair of processes. The previously used message queue partitioned into two halves has been replaced by a more general signal queue partitioned into three parts that service all kinds of signals. More details regarding the signal queue can be found in comments in the erl_proc_sig_queue.h file. The monitor and link implementations have also been completely replaced in order to fit the new asynchronous signaling implementation as good as possible. More details regarding the new monitor and link implementations can be found in the erl_monitor_link.h file.
2018-03-09Always use sys_memcpy/cmp/etc instead of plain memcpy/cmp/etcJohn Högberg
2018-02-26Make enif_make_binary return heap binaries if possibleJohn Högberg
This does not avoid allocating a ProcBin since we still need to release the binary somehow (and using the tmp obj list is slightly slower since it's an unavoidable off-heap allocation), but it does give us a hard limit on how long said ProcBin will live, which is a lot better than before. OTP-14845
2018-02-26Unconditionally transfer ownership to the created termJohn Högberg
This fixes two corner-cases: 1) We will no longer return an invalid term when a binary inspected on environment A is used in enif_make_binary on environment B 2) A double-free in this sequence of events: * enif_alloc_binary(size, &bin); * enif_ioq_enq_binary(ioq, &bin, skip); * enif_make_binary(env, &bin); * enif_make_binary(env, &bin); OTP-14931 OTP-14932
2018-02-26Tweak asserts and explain why the copy buf doesn't use the tmp obj listJohn Högberg
2018-02-26Free temporary iovecs through the tmp object listJohn Högberg
Attaching to a ProcBin is the fastest way to delay the release of an already allocated Binary, but alloc_tmp_obj is a lot faster when starting from scratch since it usually uses temp_alloc.
2018-02-23Add a helper function for attaching tmp objects to NIF environmentsJohn Högberg
2018-02-19Merge branch 'maint'John Högberg
2018-02-15Handle unaligned binaries in enif_inspect_iovecJohn Högberg
2018-01-16Merge branch 'sverker/build-proc-bin'Sverker Eriksson
2018-01-05erts: Refactor more usage of erts_build_proc_binSverker Eriksson
2017-12-06"Fix" tracing of dirty NIFs on debug buildsJohn Högberg
When a dirty NIF is executed a "schedule in" trace event is generated, which may in turn result in a generic system task being created, causing the process to be scheduled out as it can't run dirty with pending tasks. This is usually fine since said system task is seldom created, but ERTS_FORCE_ENIF_SEND_DELAY was de-facto always on for debug builds, causing the process to bounce between dirty and normal schedulers forever. This commit is not a complete fix and it can go off the rails even on normal builds; if there's a lot of dirty jobs lined up and the receiver's msgq lock happens to be busy at the wrong time, the additional trace messages generated through this will hammer the lock and keep everything bouncing.
2017-11-30Add enif_ioq_peek_headJohn Högberg
This introduces a way to retrieve erlang terms from NIF IO queues without having to resort to copying. OTP-14797
2017-11-21Merge branch 'maint'Sverker Eriksson
2017-11-15erts: Refactor connection_id in ErtsDistExternalSverker Eriksson
Break out from 'flags' into new dedicated 'connection_id' just for simplicity. Also changed flags to low bits and that affected enif_binary_to_term.
2017-11-15erts: Fix NIF debug readonly check of binariesSverker Eriksson
when done by enif_free_env or enif_clear_env. Do check before we free heap fragments.
2017-11-10Merge branch 'maint'Ingela Anderton Andin
* maint: Updated OTP version Update release notes Update version numbers inets: Prepare for release inets: Add missing guard Avoid WindowBits=8 as per the manual Fix deflateParams on zlib 1.2.11 Ignore empty binaries in enif_inspect_iovec Emasculate writable binaries on entering an iovec Only apply EOS behaviors if there's pending data Conflicts: OTP_VERSION
2017-11-07Ignore empty binaries in enif_inspect_iovecJohn Högberg
2017-11-07Emasculate writable binaries on entering an iovecJohn Högberg
The lack of this caused serious data corruption when a binary was altered after entering the queue. This went unnoticed because it was never used without erlang:iolist_to_iovec, which always emasculates binaries.
2017-10-01Refactor macros for accessing Beam instructionsBjörn Gustavsson
The BeamOp() macro in erl_vm.h is clumsy to use. All users cast the return value to BeamInstr. Define new macros that are easier to use. In the future, we might want to pack an operand into the same word as the pointer to the instruction, so we will define two macros. BeamIsOpCode() is used to rewrite code like this: if (Instr == (BeamInstr) BeamOp(op_i_func_info_IaaI) { ... } to: if (BeamIsOpCode(Instr, op_i_func_info_IaaI)) { ... } BeamOpCodeAddr(op_apply_bif) is used when we need the address for an instruction. Also elimiminate the global variables em_* in beam_emu.c. They are not really needed. Use the BeamOpCodeAddr() macro instead.
2017-09-11erts: Remove possibility to disable dirty schedulersLukas Larsson
2017-09-11Merge branch 'maint'John Högberg
2017-09-07Replace ad-hoc MIN/MAX macros with common onesJohn Högberg
Besides being noisy, they were already defined by a global Unix- specific header, causing the Windows build to fail if one forgot to define them.
2017-09-06Merge branch 'maint' into john/erts/merge-zlib-and-vector-qJohn Högberg
2017-09-06Merge branch 'lukas/erts/nif_vector_q/OTP-14598' into maintJohn Högberg
OTP-14520
2017-09-05erts: Add nif ioqLukas Larsson
2017-08-16Merge branch 'maint'Lukas Larsson
Conflicts: erts/emulator/beam/erl_nif.c erts/emulator/beam/erl_process.c
2017-08-16Merge branch 'lukas/erts/fix_dirty_trace_message_flush/OTP-14538' into maintLukas Larsson
* lukas/erts/fix_dirty_trace_message_flush/OTP-14538: erts: Must have main lock when flushing trace messages
2017-07-25Merge branch 'maint'Sverker Eriksson
2017-07-25Merge branch 'sverker/enif_whereis-bug/OTP-14523' into maintSverker Eriksson
* sverker/enif_whereis-bug: erts: Fix bug in enif_whereis_pid/port
2017-07-17erts: Replace usage of all erts_smp prefixes to just ertsLukas Larsson
2017-07-17erts: Remove ERTS_SMP and USE_THREAD definesLukas Larsson
This refactor was done using the unifdef tool like this: for file in $(find erts/ -name *.[ch]); do unifdef -t -f defile -o $file $file; done where defile contained: #define ERTS_SMP 1 #define USE_THREADS 1 #define DDLL_SMP 1 #define ERTS_HAVE_SMP_EMU 1 #define SMP 1 #define ERL_BITS_REENTRANT 1 #define ERTS_USE_ASYNC_READY_Q 1 #define FDBLOCK 1 #undef ERTS_POLL_NEED_ASYNC_INTERRUPT_SUPPORT #define ERTS_POLL_ASYNC_INTERRUPT_SUPPORT 0 #define ERTS_POLL_USE_WAKEUP_PIPE 1 #define ERTS_POLL_USE_UPDATE_REQUESTS_QUEUE 1 #undef ERTS_HAVE_PLAIN_EMU #undef ERTS_SIGNAL_STATE
2017-07-11erts: Fix bug in enif_whereis_pid/portSverker Eriksson
that could cause heap corruption if whereis lookup conflicts with other register updater AND other thread sends on-heap message while main lock is released. Also improved enif_whereis from dirty nifs by passing c_p as NULL.
2017-07-06Merge branch 'john/erts/runtime-lcnt' into maintJohn Högberg
* john/erts/runtime-lcnt: Document rt_mask and add warnings about copy_save Add an emulator test suite for lock counting Break erts_debug:lock_counters/1 into separate BIFs Allow toggling lock counting at runtime Move lock flags to a common header Enable register_SUITE for lcnt builds Enable lcnt smoke test on all builds that have lcnt enabled Make lock counter info independent of the locks being counted OTP-14412 OTP-13170 OTP-14413
2017-07-06Allow toggling lock counting at runtimeJohn Högberg
The implementation is still hidden behind ERTS_ENABLE_LOCK_COUNT, and all categories are still enabled by default, but the actual counting can be toggled at will. OTP-13170
2017-06-15erts: Must have main lock when flushing trace messagesLukas Larsson
If we don't have the main lock, multiple threads may come in and do the flush at the same time which will lead to all kinds of strang problems.
2017-05-22Add enif_whereis_...() functionsTed Burghart
#### Why do we need this new feature? There are cases when a NIF needs to send a message, using `enif_send()`, to a long-lived process with a registered name. A common use-case is logging, where asynchronous fire-and-forget messages are the norm. There can also be cases where a yielding or dirty NIF or background thread may request a callback from a service with additional information it needs to complete its operation, yielding or waiting (with suitable timeouts, etc) until its state has been updated through the NIF module's API. NIFs can only send messages to pids, and the lack of name resolution leaves a complicated dance between separate monitoring processes and the NIF as the only way to keep a NIF informed of the whereabouts of such long-lived processes. Providing a reliable, built-in facility for NIFs to resolve process (or port) names simplifies these use cases considerably. #### Risks or uncertain artifacts? Testing has not exposed any significant risk. The implementation behaves as expected on regular and dirty scheduler threads as well as non-scheduler threads. By constraining the `enif_whereis_...()` functions to their minimal scopes and using patterns consistent with related functions, the implementation, testing, and maintenance burden is low. The API and behavior of existing functions is unchanged. #### How did you solve it? While extending `enif_send()` to operate on a pid or an atom (as `erlang:send/2` does) was attractive, it would have entailed changing the type of its `to_pid` parameter and thereby breaking backward compatibility. The same consideration applies to `enif_port_command()`. That leaves a choice between 1, 2, or 3 new functions: 1. `enif_whereis()` 2. `enif_whereis_pid()` and `enif_whereis_port()` 3. All of the above. While option (1), directly mimicking the behavior of `erlang:whereis/1`, is appealing, it poses potential problems if `pid()` or `port()` are subsequently implemented as non-integral types that must be bound to an owning `ErlNifEnv` instance. Therefore, option (2) has been chosen to use `ErlNifPid`/`ErlNifPort` structures in the API to maintain proper term ownership semantics.
2017-05-04Update copyright yearRaimo Niskanen
2017-04-28Merge PR1413 from g-andrade/feature/phash2_nifSverker Eriksson
Support hashing terms from NIF code
2017-04-25Merge branch 'sverker/refactor'Sverker Eriksson
* sverker/refactor: erts: Introduce struct binary_internals erts: Introduce erts_bin_release erts: Init refc=1 in erts_bin_drv_alloc* erts: Init refc=1 in erts_bin_nrml_alloc erts: Remove deliberate leak of hipe fun entries erts: Remove hipe_bifs:remove_refs_from/1 Refactor hipe specific code to use ErtsCodeInfo erts: Refactor ErtsCodeInfo.native
2017-04-24erts: Discontinue salted use of enif_hash/phash2Guilherme Andrade
2017-04-22erts: Allow for easier future enif_hash expansionGuilherme Andrade
Allow for expanding support to 64-bit hashes without breaking the interface.
2017-04-22erts: Support custom salt in enif_hashGuilherme Andrade
2017-04-20erts: Refactor enif_phash2 into enif_hashGuilherme Andrade
A more generic hashing function which can also hash terms based on `make_internal_hash'.
2017-04-20erts: Remove enif_phash2_rangedGuilherme Andrade
2017-04-19erts: Add enif_phash2 and enif_phash2_rangedGuilherme Andrade
These allow one to hash VM terms from NIF code.
2017-04-12erts: Introduce struct binary_internalsSverker Eriksson
to replace macro ERTS_INTERNAL_BINARY_FIELDS as header in Binary and friends.
2017-04-11Merge branch 'rickard/sched_type_tests'Rickard Green
* rickard/sched_type_tests: Fix dirty scheduler type tests
2017-04-11erts: Introduce erts_bin_releaseSverker Eriksson
2017-04-11erts: Init refc=1 in erts_bin_drv_alloc*Sverker Eriksson