aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/Makefile.in
AgeCommit message (Collapse)Author
2018-11-15erts: Add new module 'counters'Sverker Eriksson
2018-11-15erts: Add new module 'atomics'Sverker Eriksson
2018-11-06Add a persistent term storageBjörn Gustavsson
Persistent terms are useful for storing Erlang terms that are never or infrequently updated. They have the following advantages: * Constant time access. A persistent term is not copied when it is looked up. The constant factor is lower than for ETS, and no locks are taken when looking up a term. * Persistent terms are not copied in garbage collections. * There is only ever one copy of a persistent term (until it is deleted). That makes them useful for storing configuration data that needs to be easily accessible by all processes. Persistent terms have the following drawbacks: * Updates are expensive. The hash table holding the keys for the persistent terms are updated whenever a persistent term is added, updated or deleted. * Updating or deleting a persistent term triggers a "global GC", which will schedule a heap scan of all processes to search the heap of all processes for the deleted term. If a process still holds a reference to the deleted term, the process will be garbage collected and the term copied to the heap of the process. This global GC can make the system less responsive for some time. Three BIFs (implemented in C in the emulator) is the entire interface to the persistent term functionality: * put(Key, Value) to store a persistent term. * get(Key) to look up a persistent term. * erase(Key) to delete a persistent term. There are also two additional BIFs to obtain information about persistent terms: * info() to return a map with information about persistent terms. * get() to return a list of a {Key,Value} tuples for all persistent terms. (The values are not copied.)
2018-05-23make: Don't call mkdir with empty argumentLukas Larsson
2018-05-09Disable the use of floating point exceptionsBjörn Gustavsson
Floating point exceptions has been disabled since 2011 on macOS (fa0f8d2c29b) and on Linux since 2014 (c7ddafbe6dbc) because there were unresolved stability issues. Floating point exceptions are not disabled by default on FreeBSD, and if OTP is compiled with gcc (as opposed to clang) floating point exceptions will be used. 81a6adab693a introduced a bug in erts/emulator/Makefile.in which would cause the building of OTP to fail if floating point exceptions were enabled. The bug was not noticed because it turns out that none of our daily build machines has floating point exceptions enabled. Since floating point exceptions is not tested, we should not expect them to work reliably on any platform. Therefore, turn off floating point exceptions unconditionally in erts/configure.in. For the moment we will keep the code in the runtime system that handles floating point exceptions. (This commit also fixes the bug in erts/emulator/Makefile.in, in case floating point exceptions ever become reliable and enabled.) https://bugs.erlang.org/browse/ERL-620
2018-04-23erts: Rewrite memory instrumentationJohn Högberg
This commit replaces the old memory instrumentation with a new implementation that scans carriers instead of wrapping erts_alloc/erts_free. The old implementation could not extract information without halting the emulator, had considerable runtime overhead, and the memory maps it produced were noisy and lacked critical information. Since the new implementation walks through existing data structures there's no longer a need to start the emulator with special flags to get information about carrier utilization/fragmentation. Memory fragmentation is also easier to diagnose as it's presented on a per-carrier basis which eliminates the need to account for "holes" between mmap segments. To help track allocations, each allocation can now be tagged with what it is and who allocated it at the cost of one extra word per allocation. This is controlled on a per-allocator basis with the +M<S>atags option, and is enabled by default for binary_alloc and driver_alloc (which is also used by NIFs).
2018-03-21Implementation of true asynchronous signaling between processesRickard Green
Communication between Erlang processes has conceptually always been performed through asynchronous signaling. The runtime system implementation has however previously preformed most operation synchronously. In a system with only one true thread of execution, this is not problematic (often the opposite). In a system with multiple threads of execution (as current runtime system implementation with SMP support) it becomes problematic. This since it often involves locking of structures when updating them which in turn cause resource contention. Utilizing true asynchronous communication often avoids these resource contention issues. The case that triggered this change was contention on the link lock due to frequent updates of the monitor trees during communication with a frequently used server. The signal order delivery guarantees of the language makes it hard to change the implementation of only some signals to use true asynchronous signaling. Therefore the implementations of (almost) all signals have been changed. Currently the following signals have been implemented as true asynchronous signals: - Message signals - Exit signals - Monitor signals - Demonitor signals - Monitor triggered signals (DOWN, CHANGE, etc) - Link signals - Unlink signals - Group leader signals All of the above already defined as asynchronous signals in the language. The implementation of messages signals was quite asynchronous to begin with, but had quite strict delivery constraints due to the ordering guarantees of signals between a pair of processes. The previously used message queue partitioned into two halves has been replaced by a more general signal queue partitioned into three parts that service all kinds of signals. More details regarding the signal queue can be found in comments in the erl_proc_sig_queue.h file. The monitor and link implementations have also been completely replaced in order to fit the new asynchronous signaling implementation as good as possible. More details regarding the new monitor and link implementations can be found in the erl_monitor_link.h file.
2018-01-03Replace the libc environment with a thread-safe emulationJohn Högberg
putenv(3) and friends aren't thread-safe regardless of how you slice it; a global lock around all environment operations (like before) keeps things safe as far as our own operations go, but we have absolutely no control over what libc or a library dragged in by a driver/NIF does -- they're free to call getenv(3) or putenv(3) without honoring our lock. This commit solves this by setting up an "emulated" environment which can't be touched without going through our interfaces. Third-party libraries can still shoot themselves in the foot but benign uses of os:putenv/2 will no longer risk crashing the emulator.
2017-11-30Reimplement efile_drv as a dirty NIFJohn Högberg
This improves the latency of file operations as dirty schedulers are a bit more eager to run jobs than async threads, and use a single global queue rather than per-thread queues, eliminating the risk of a job stalling behind a long-running job on the same thread while other async threads sit idle. There's no such thing as a free lunch though; the lowered latency comes at the cost of increased busy-waiting which may have an adverse effect on some applications. This behavior can be tweaked with the +sbwt flag, but unfortunately it affects all types of schedulers and not just dirty ones. We plan to add type-specific flags at a later stage. sendfile has been moved to inet_drv to lessen the effect of a nasty race; the cooperation between inet_drv and efile has never been airtight and the socket dying at the wrong time (Regardless of reason) could result in fd aliasing. Moving it to the inet driver makes it impossible to trigger this by closing the socket in the middle of a sendfile operation, while still allowing it to be aborted -- something that can't be done if it stays in the file driver. The race still occurs if the controlling process dies in the short window between dispatching the sendfile operation and the dup(2) call in the driver, but it's much less likely to happen now. A proper fix is in the works. -- Notable functional differences: * The use_threads option for file:sendfile/5 no longer has any effect. * The file-specific DTrace probes have been removed. The same effect can be achieved with normal tracing together with the nif__entry/nif__return probes to track scheduling. -- OTP-14256
2017-11-30Add a mutable binary buffer type (prim_buffer)John Högberg
2017-10-30Merge branch 'lukas/erts/pgo/OTP-14604'Lukas Larsson
* lukas/erts/pgo/OTP-14604: erts: Only do PGO if gcc supports -fprofile-correction
2017-10-25erts: Only do PGO if gcc supports -fprofile-correctionLukas Larsson
2017-10-12Merge branch 'lukas/erts/poll-thread/OTP-14346'Lukas Larsson
* lukas/erts/poll-thread/OTP-14346: erts: Make a copy of erl_poll.c to help debuggers
2017-10-09erts: Make a copy of erl_poll.c to help debuggersLukas Larsson
Especially lldb seems to get very confused when the same source file is used by two different object files.
2017-10-09Merge branch 'bjorn/erts/pack-with-opcode/OTP-14325'Björn Gustavsson
OTP-14327 OTP-14340 * bjorn/erts/pack-with-opcode/OTP-14325: Pack operands for combined instructions into the instruction word beam_makeops: Use named arguments for the code generation functions Optimize packing for "optional use" operands beam_makeops: Print the instruction name for fatal packing errors Introduce a syntax for marking operands as "optional use" beam_makeops: Refactor parsing of specific instructions Optimize instruction prefetch Pack operands into the instruction word Use 32-bits pointers to C code Move LD flags for hipe from Makefile.in to configure.in beam_disasm: Correct printing of y registers ops.tab: Slightly optimize badmatch on a Y register macros.tab: Fix assertion in SET_I_REL()
2017-10-05Merge branch 'lukas/erts/beam-emu-vars'Lukas Larsson
* lukas/erts/beam-emu-vars: erts: Add makefile target to check emu register allocation
2017-10-05Pack operands into the instruction wordBjörn Gustavsson
On 64-bit machines where the C code is always at address below 4Gb, pack one or more operands into the instruction word.
2017-10-05Move LD flags for hipe from Makefile.in to configure.inBjörn Gustavsson
We want the flags to be available for other tests in configure.in.
2017-10-02erts: Move all I/O polling to a seperate threadLukas Larsson
2017-09-27erts: Add makefile target to check emu register allocationLukas Larsson
2017-09-11erts: Remove possibility to disable dirty schedulersLukas Larsson
2017-09-11Merge branch 'lukas/erts/pgo/OTP-14604'Lukas Larsson
* lukas/erts/pgo/OTP-14604: Add support for building a pgo beam_emu
2017-09-08Add support for building a pgo beam_emuLukas Larsson
2017-09-06Merge branch 'maint' into john/erts/merge-zlib-and-vector-qJohn Högberg
2017-09-06Merge branch 'maint' into john/erts/merge-zlib-and-vector-qJohn Högberg
2017-09-05Replace the zlib driver with a NIFJohn Högberg
All operations will now yield appropriately, allowing them to be used freely in concurrent applications. This commit also deprecates the functions listed below, although they won't raise deprecation warnings until OTP 21: zlib:adler32 zlib:crc32 zlib:inflateChunk zlib:getBufSize zlib:setBufSize The behavior of throwing an error when a dictionary is required for decompression has also been deprecated.
2017-09-05erts: Add nif ioqLukas Larsson
2017-08-31Introduce '%warm' and beam_warm.hBjörn Gustavsson
The bit syntax instructions are mixed among other instructions in beam_hot.h and beam_cold.h. Introduce a new hotness level called '%warm' with is associated file beam_warm.h. Mark all bit syntax instructions as '%warm'.
2017-08-23Eliminate the beam_instrs.h fileBjörn Gustavsson
The beam_instrs.h file serves no useful purpose. Put the instructions in beam_hot.h instead.
2017-08-11Break out most instructions from beam_emu.cBjörn Gustavsson
2017-08-08Simplify specifying implementation of instructionsBjörn Gustavsson
Eliminate the need to write pre-processor macros for each instruction. Instead allow the implementation of instruction to be written in C directly in the .tab files. Rewrite all existing macros in this way and remove the %macro directive.
2017-07-17erts: Cleanup alloc.types after non-smp removalLukas Larsson
2017-07-17erts: Cleanup configure and makefiles after non-smp removalLukas Larsson
2017-07-06Merge branch 'john/erts/runtime-lcnt' into maintJohn Högberg
* john/erts/runtime-lcnt: Document rt_mask and add warnings about copy_save Add an emulator test suite for lock counting Break erts_debug:lock_counters/1 into separate BIFs Allow toggling lock counting at runtime Move lock flags to a common header Enable register_SUITE for lcnt builds Enable lcnt smoke test on all builds that have lcnt enabled Make lock counter info independent of the locks being counted OTP-14412 OTP-13170 OTP-14413
2017-07-06Move lock flags to a common headerJohn Högberg
2017-05-29hipe: Fix hipe_mkliterals make recipeLukas Larsson
If a link dependency is added in LDFLAGS it should be resolved in LIBS, so we have to use LIBS and not TYPE_LIBS.
2017-05-18Make lock counter info independent of the locks being countedJohn Högberg
This allows us to enable/disable lock counting at will, and greatly improves the performance of erts_debug:lock_counters/1 since we no longer have to worry about the lock counters "dying" while we're enumerating them. OTP-14412
2017-05-04Update copyright yearRaimo Niskanen
2017-04-21erts: Deprecate the non-smp emulatorsLukas Larsson
2017-03-20erts: Make generated files depend on MakefileSverker Eriksson
in order to regenerate them if config has changed. ERL-379 Maybe a better solution would have been to run 'clean' target or stop and ask user to run make clean.
2017-01-12Optimize handling of BIF errorsRickard Green
2017-01-12Support for dirty BIFsRickard Green
2016-11-11Merge branch 'maint'Sverker Eriksson
2016-11-11erts: Fix correct link flags for hipe_mkliteralsSverker Eriksson
and no need for $(INCLUDED).
2016-10-14Add a loader state for HiPE code loadingMagnus Lång
Just like the BEAM loader state (as returned by erlang:prepare_loading/2), the HiPE loader state is contained in a magic binary. Eventually, we will separate HiPE loading into a prepare and a finalise phase, like the BEAM loader, where the prepare phase will be implemented by hipe_unified_loader and the finalise phase be implemented in C by hipe_load.c and beam_load.c, making prepare side-effect free and finalise atomic. The finalise phase will be exposed through the erlang:finish_loading/1 API, just like the BEAM loader, as this will allow HiPE and BEAM modules to be mixed in the same atomic "commit". The usage of a loader state makes it easier to keep track of all resources allocated during loading, and will not only make it easy to prevent leaks when hipe_unified_loader crashes, but also paves the way for proper, leak-free, unloading of HiPE modules.
2016-08-31Remove old purge strategyRickard Green
2016-08-29Merge branch 'maint'Rickard Green
* maint: Fix tracing of processes executing dirty Perform check_process_code while process is executing dirty Fix purge of code Reclaim literal area after purge has completed Separate literal area from code Conflicts: erts/emulator/beam/global.h
2016-08-29Merge branch 'rickard/ds-purge-module/OTP-13808' into maintRickard Green
* rickard/ds-purge-module/OTP-13808: Perform check_process_code while process is executing dirty Conflicts: erts/doc/src/erl_nif.xml
2016-08-29Merge branch 'rickard/fun-purge-bug/OTP-13809' and ↵Rickard Green
'rickard/new-purge-strategy/OTP-13833' into maint * rickard/fun-purge-bug/OTP-13809: Fix purge of code Reclaim literal area after purge has completed Separate literal area from code Conflicts: erts/doc/src/erlang.xml erts/emulator/beam/beam_bif_load.c erts/emulator/beam/erl_init.c erts/preloaded/ebin/init.beam
2016-08-29Perform check_process_code while process is executing dirtyRickard Green