aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam/external.c
AgeCommit message (Collapse)Author
2017-04-25Merge branch 'sverker/refactor'Sverker Eriksson
* sverker/refactor: erts: Introduce struct binary_internals erts: Introduce erts_bin_release erts: Init refc=1 in erts_bin_drv_alloc* erts: Init refc=1 in erts_bin_nrml_alloc erts: Remove deliberate leak of hipe fun entries erts: Remove hipe_bifs:remove_refs_from/1 Refactor hipe specific code to use ErtsCodeInfo erts: Refactor ErtsCodeInfo.native
2017-04-12erts: Introduce struct binary_internalsSverker Eriksson
to replace macro ERTS_INTERNAL_BINARY_FIELDS as header in Binary and friends.
2017-04-11erts: Init refc=1 in erts_bin_nrml_allocSverker Eriksson
Only term_to_binary needed some extra attention as it used to initialize refc as 0 instead of 1.
2017-03-08erts: Do not generate atoms on old latin1 external formatSverker Eriksson
term_to_binary will always generate utf8 atoms ATOM_UTF8_EXT and SMALL_ATOM_UTF8_EXT. Old latin1 atoms, ATOM_EXT and SMALL_ATOM_EXT, are still decoded.
2017-03-08erts: Fix faulty ASSERT for failed dec_termSverker Eriksson
when a recursive call to dec_term() has already done erts_factory_undo and set factory->hp to NULL.
2017-02-14erts: Add deallocation veto for magic destructorsSverker Eriksson
A magic destructor can return 0 and thereby take control and prolong the lifetime of a magic binary.
2017-02-06Add binary overhead for magic ref/binariesRickard Green
2017-02-06Use magic refs binary_to_term/term_to_binary trap contextRickard Green
2017-02-06Implement magic referencesRickard Green
Magic references are *intentionally* indistinguishable from ordinary references for the Erlang software. Magic references do not change the language, and are intended as a pure runtime internal optimization. An ordinary reference is typically used as a key in some table. A magic reference has a direct pointer to a reference counted magic binary. This makes it possible to implement various things without having to do lookups in a table, but instead access the data directly. Besides very fast lookups this can also improve scalability by removing a potentially contended table. A couple of examples of planned future usage of magic references are ETS table identifiers, and BIF timer identifiers. Besides future optimizations using magic references it should also be possible to replace the exposed magic binary cludge with magic references. That is, magic binaries that are exposed as empty binaries to the Erlang software.
2017-02-06Merge branch 'maint'Rickard Green
* maint: Atomic reference count of binaries also in non-SMP Conflicts: erts/emulator/beam/erl_fun.c
2017-02-06Merge branch 'rickard/binary-refc' into maintRickard Green
OTP-14202 * rickard/binary-refc: Atomic reference count of binaries also in non-SMP Conflicts: erts/emulator/beam/beam_bp.c
2017-02-06Atomic reference count of binaries also in non-SMPRickard Green
NIF resources was not handled in a thread-safe manner in the runtime system without SMP support. As a consequence of this fix, the following driver functions are now thread-safe also in the runtime system without SMP support: - driver_free_binary() - driver_realloc_binary() - driver_binary_get_refc() - driver_binary_inc_refc() - driver_binary_dec_refc()
2017-02-01Merge branch 'maint'Sverker Eriksson
2017-02-01Merge branch 'sverker/bin2term-zlib-bug/ERL-340/OTP-14159' into maintSverker Eriksson
* sverker/bin2term-zlib-bug/ERL-340/OTP-14159: erts: Fix binary_to_term for compressed and zlib >= v1.2.9
2017-01-23erts: Fix binary_to_term for compressed and zlib >= v1.2.9Sverker Eriksson
Problem: z_stream was incorrectly copied with memcpy which just happened to work with zlib < v1.2.9. Solution: Avoid copying z_stream.
2016-11-29erts: Use secret atom as sysname for ETS compressedSverker Eriksson
2016-10-26erts: Remove unused argument to hipe_set_closure_stubSverker Eriksson
2016-10-14Merge branch 'master' into sverker/hipe-code-loadnpurgeSverker Eriksson
Conflicts: erts/emulator/beam/beam_bif_load.c erts/emulator/beam/beam_load.c and added macro DBG_TRACE_MFA_P in beam_load.h
2016-10-14hipe,erts: Remove cached fun 'native_address'Sverker Eriksson
Did not work with purge and made worse by new purge strategy. Did yield terrible performance when fun thing is created *before* fun code is loaded. Like when receiving not yet loaded fun from other node. The cached 'native_address' in ErlFunThing will not be updated leading to mode switch and error_handler being called for every call to the fun from native code.
2016-10-12erts: Refactor out func_info into structLukas Larsson
This commit adds two new structs to be used to represent erlang code in erts. ErtsCodeInfo is used to describe the i_func_info header that is part of all Export entries and the prelude of each function. This replaces all the BeamInstr * that were previously used to point to these locations. After this change the code should never use BeamInstr * with offsets to figure out different parts of the func_info header. ErtsCodeMFA is a struct that is used to descripe a MFA in code. It is used within ErtsCodeInfo and also in Process->current. All function that previously took Eterm * or BeamInstr * to identify a MFA now use the ErtsCodeMFA or ErtsCodeInfo where appropriate. The code has been tested to work when adding a new field to the ErtsCodeInfo struct, but some updates are needed in ops.tab to make it work.
2016-06-15erts: Change local sysname for ETS compressedSverker Eriksson
Yes this is an ugly workaround. One approach for a better solution could be to introduce an internal secret atom tagged as an atom with a unique index, but impossible to find by string hash lookup/insert.
2016-04-29Merge branch 'sverk/big-external-creations.OTP-13488'Sverker Eriksson
second merge of this branch with some bug fixing
2016-04-15erts: Silence harmless valgrind warning in dec_termSverker Eriksson
provoked by nif_SUITE:nif_binary_to_term. If we fail to decode an immediate (unsafe atom for example) with a dummy factory then hp and factory->hp will both be uninitialized and valgrind will complain about comparing them.
2016-04-13erts: Fix decoding of pids, ports and refs with big creationsSverker Eriksson
Must skip 3 extra bytes after node name atom.
2016-04-13Merge branch 'henrik/update-copyrightyear'Henrik Nord
* henrik/update-copyrightyear: update copyright-year
2016-04-07erts: Support 32-bit creation for external pid,port,refsSverker Eriksson
from future nodes.
2016-04-07erts: Refactor ETS compressed encoding of local nodeSverker Eriksson
Instead of INTERNAL_CREATION (255), use empty atom for node name to mean the local node (regardless of node name or creation). The purpose is to get rid of special value 255, for future expansion of creation to 32-bit.
2016-03-29erts: Remove printout when dec_term fails in DEBUGSverker Eriksson
2016-03-29erts: Improve enif_binary_to_termSverker Eriksson
* Accept a raw data buffer instead of ErlNifBinary * Accept option ERL_NIF_BIN2TERM_SAFE * Return number of read bytes
2016-03-15update copyright-yearHenrik Nord
2016-02-24Merge branch 'master' into sverk/master/halt-INT_MINSverker Eriksson
2016-02-24erts: Change erl_exit into erts_exitSverker Eriksson
This is mostly a pure refactoring. Except for the buggy cases when calling erlang:halt() with a positive integer in the range -(INT_MIN+2) to -INT_MIN that got confused with ERTS_ABORT_EXIT, ERTS_DUMP_EXIT and ERTS_INTR_EXIT. Outcome OLD erl_exit(n, ) NEW erts_exit(n, ) ------- ------------------- ------------------------------------------- exit(Status) n = -Status <= 0 n = Status >= 0 crashdump+abort n > 0, ignore n n = ERTS_ERROR_EXIT < 0 The outcome of the old ERTS_ABORT_EXIT, ERTS_INTR_EXIT and ERTS_DUMP_EXIT are the same as before (even though their values have changed).
2015-11-23Merge branch 'maint'Björn-Egil Dahlberg
Conflicts: erts/emulator/beam/external.c
2015-11-20erts: Fix maps decode in erlang:binary_to_term/1Björn-Egil Dahlberg
Decoding a term with a large (HAMT) map in an small (FLAT) map could cause a critical error if the external format was not produced by beam.
2015-06-24erts: Remove halfword pointer compressionBjörn-Egil Dahlberg
* Removed COMPRESS_POINTER and EXPAND_POINTER
2015-06-24erts: Remove HALFWORD_HEAP definitionBjörn-Egil Dahlberg
2015-06-18Change license text to APLv2Bruce Yinhe
2015-06-15erts: Remove hashmap probabilistic heap overestimationSverker Eriksson
by adding a dynamic heap factory. "binary_to_term" is now a hybrid solution with both a call to decoded_size() to calculate needed heap space AND possible dynamic allocation of more heap space if needed for big maps. The heap size returned from decoded_size() is guaranteed to be sufficient for all term heap data except for hashmap nodes. All hashmap nodes are created at the end of dec_term() by invoking the heap factory interface that may allocate more heap space on process heap or in fragments. With this commit it is no longer guaranteed that a message is confined to only one heap fragment.
2015-03-30erts: Refactor encode_size_struct_intSverker Eriksson
to handle the "start of list" case in one place and not seven. Note that this commit reverts (47d6fd3ccf35) back to using WSTACK and pushing raw pointers. We disable GC while yielding, so this should not be a problem.
2015-03-26Merge branch 'sverk/hamt-encode-size-bug/OTP-12585'Sverker Eriksson
* sverk/hamt-encode-size-bug/OTP-12585: erts: Fix bug in term_to_binary size estimation for hamt erts: Optimize term_to_binary size estimation
2015-03-25erts: Fix bug in term_to_binary size estimation for hamtSverker Eriksson
2015-03-25erts: Optimize term_to_binary size estimationSverker Eriksson
for tuples and maps containing ascii strings (lists).
2015-03-25erts: Combine flat and hash maps under one unifying tagBjörn-Egil Dahlberg
2015-03-24erts: Remove HAMT_SUBTAG_NODE_ARRAYSverker Eriksson
This will also fix a bug in term_to_binary treating full nodes as tuples and emiting LIST_EXT for leafs.
2015-03-24erts: Fix bug in binary_to_term for hamt when yieldingSverker Eriksson
Must save hamt_list in context.
2015-03-20Merge branch 'sverk/dec_term-bin-overhead/OTP-12554'Sverker Eriksson
* sverk/dec_term-bin-overhead/OTP-12554: erts: Add missing binary offheap overhead in binary_to_term
2015-03-19erts: Fix bug in binary_to_term with more than one big mapSverker Eriksson
2015-03-19erts: Fix map bug in dec_term for 32-bit debug VMSverker Eriksson
Adding ERTS_SWORD_MAX to a pointer does not work as a way to disable a bound check. Remove the hp_end from ErtsHeapFactory as it isn't really used anyway.
2015-03-12erts: Refactor maps naming conventionSverker Eriksson
flatmap: Small map hashmap: Large map map: flatmap or hashmap
2015-03-12erts: Tweak over estimation of hashmap size for binary_to_termSverker Eriksson
Strategy: Calculate an over estimation of heap size that will give such a low probability for overflow, that "it will not happen". Scary assumption 1: Uniformly distributed hash values. Scary assumption 2: Tree size is normally distributed (right?)