aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam/external.c
AgeCommit message (Collapse)Author
2016-04-15erts: Silence harmless valgrind warning in dec_termSverker Eriksson
provoked by nif_SUITE:nif_binary_to_term. If we fail to decode an immediate (unsafe atom for example) with a dummy factory then hp and factory->hp will both be uninitialized and valgrind will complain about comparing them.
2016-04-13Merge branch 'henrik/update-copyrightyear'Henrik Nord
* henrik/update-copyrightyear: update copyright-year
2016-04-07erts: Support 32-bit creation for external pid,port,refsSverker Eriksson
from future nodes.
2016-04-07erts: Refactor ETS compressed encoding of local nodeSverker Eriksson
Instead of INTERNAL_CREATION (255), use empty atom for node name to mean the local node (regardless of node name or creation). The purpose is to get rid of special value 255, for future expansion of creation to 32-bit.
2016-03-29erts: Remove printout when dec_term fails in DEBUGSverker Eriksson
2016-03-29erts: Improve enif_binary_to_termSverker Eriksson
* Accept a raw data buffer instead of ErlNifBinary * Accept option ERL_NIF_BIN2TERM_SAFE * Return number of read bytes
2016-03-15update copyright-yearHenrik Nord
2016-02-24Merge branch 'master' into sverk/master/halt-INT_MINSverker Eriksson
2016-02-24erts: Change erl_exit into erts_exitSverker Eriksson
This is mostly a pure refactoring. Except for the buggy cases when calling erlang:halt() with a positive integer in the range -(INT_MIN+2) to -INT_MIN that got confused with ERTS_ABORT_EXIT, ERTS_DUMP_EXIT and ERTS_INTR_EXIT. Outcome OLD erl_exit(n, ) NEW erts_exit(n, ) ------- ------------------- ------------------------------------------- exit(Status) n = -Status <= 0 n = Status >= 0 crashdump+abort n > 0, ignore n n = ERTS_ERROR_EXIT < 0 The outcome of the old ERTS_ABORT_EXIT, ERTS_INTR_EXIT and ERTS_DUMP_EXIT are the same as before (even though their values have changed).
2015-11-23Merge branch 'maint'Björn-Egil Dahlberg
Conflicts: erts/emulator/beam/external.c
2015-11-20erts: Fix maps decode in erlang:binary_to_term/1Björn-Egil Dahlberg
Decoding a term with a large (HAMT) map in an small (FLAT) map could cause a critical error if the external format was not produced by beam.
2015-06-24erts: Remove halfword pointer compressionBjörn-Egil Dahlberg
* Removed COMPRESS_POINTER and EXPAND_POINTER
2015-06-24erts: Remove HALFWORD_HEAP definitionBjörn-Egil Dahlberg
2015-06-18Change license text to APLv2Bruce Yinhe
2015-06-15erts: Remove hashmap probabilistic heap overestimationSverker Eriksson
by adding a dynamic heap factory. "binary_to_term" is now a hybrid solution with both a call to decoded_size() to calculate needed heap space AND possible dynamic allocation of more heap space if needed for big maps. The heap size returned from decoded_size() is guaranteed to be sufficient for all term heap data except for hashmap nodes. All hashmap nodes are created at the end of dec_term() by invoking the heap factory interface that may allocate more heap space on process heap or in fragments. With this commit it is no longer guaranteed that a message is confined to only one heap fragment.
2015-03-30erts: Refactor encode_size_struct_intSverker Eriksson
to handle the "start of list" case in one place and not seven. Note that this commit reverts (47d6fd3ccf35) back to using WSTACK and pushing raw pointers. We disable GC while yielding, so this should not be a problem.
2015-03-26Merge branch 'sverk/hamt-encode-size-bug/OTP-12585'Sverker Eriksson
* sverk/hamt-encode-size-bug/OTP-12585: erts: Fix bug in term_to_binary size estimation for hamt erts: Optimize term_to_binary size estimation
2015-03-25erts: Fix bug in term_to_binary size estimation for hamtSverker Eriksson
2015-03-25erts: Optimize term_to_binary size estimationSverker Eriksson
for tuples and maps containing ascii strings (lists).
2015-03-25erts: Combine flat and hash maps under one unifying tagBjörn-Egil Dahlberg
2015-03-24erts: Remove HAMT_SUBTAG_NODE_ARRAYSverker Eriksson
This will also fix a bug in term_to_binary treating full nodes as tuples and emiting LIST_EXT for leafs.
2015-03-24erts: Fix bug in binary_to_term for hamt when yieldingSverker Eriksson
Must save hamt_list in context.
2015-03-20Merge branch 'sverk/dec_term-bin-overhead/OTP-12554'Sverker Eriksson
* sverk/dec_term-bin-overhead/OTP-12554: erts: Add missing binary offheap overhead in binary_to_term
2015-03-19erts: Fix bug in binary_to_term with more than one big mapSverker Eriksson
2015-03-19erts: Fix map bug in dec_term for 32-bit debug VMSverker Eriksson
Adding ERTS_SWORD_MAX to a pointer does not work as a way to disable a bound check. Remove the hp_end from ErtsHeapFactory as it isn't really used anyway.
2015-03-12erts: Refactor maps naming conventionSverker Eriksson
flatmap: Small map hashmap: Large map map: flatmap or hashmap
2015-03-12erts: Tweak over estimation of hashmap size for binary_to_termSverker Eriksson
Strategy: Calculate an over estimation of heap size that will give such a low probability for overflow, that "it will not happen". Scary assumption 1: Uniformly distributed hash values. Scary assumption 2: Tree size is normally distributed (right?)
2015-03-12erts: Reject duplicate keys for hamt in binary_to_termSverker Eriksson
2015-03-12First stab at binary_to_term for hamtSverker Eriksson
with over estimation of heap size.
2015-03-12erts: Remove erl_hashmap.[ch] filesBjörn-Egil Dahlberg
2015-03-12erts: Refactor erl_hashmap header includesBjörn-Egil Dahlberg
2015-03-12erts: Add term_to_binary for hashmapSverker Eriksson
2015-03-12erts: Add missing binary offheap overhead in binary_to_termSverker Eriksson
Binary offheap overhead is used to trigger GC when a process is referring "too much" binary offheap data. Offheap binaries created from external format (binary_to_term, distributed messages or compacted ets tables) were not accounted for. Example: A process receiving a lot of binary data in distributed messages, while not building much terms on its heap, could cause an extensive memory consumption for garbage binaries.
2015-03-12Merge branch 'maint'Henrik Nord
Conflicts: erts/emulator/hipe/hipe_bif0.c
2015-02-04don't create oversize bignums in binary matchingMikael Pettersson
Bignums are artifically restricted in size. Arithmetic and logical operations check the sizes of resulting bignums, and turn oversize results into system_limit exceptions. However, this check is not performed when bignums are constructed by binary matching. The consequence is that such matchings can construct oversize bignums that satisfy is_integer/1 yet don't work. Performing arithmetic such as Term - 0 fails with a system_limit exception. Worse, performing a logical operation such as Term band Term results in []. The latter occurs because the size checking (e.g. in erts_band()) is a simple ASSERT(is_not_nil(...)) on the result of the bignum operation, which internally is [] (NIL) in the case of oversize results. However, ASSERT is a no-op in release builds, so the error goes unnoticed and [] is returned as the result of the band/2. This patch addresses this by preventing oversize bignums from entering the VM via binary matching: - the internal bytes_to_big() procedure is augmented to return NIL for oversize results, just like big_norm() - callers of bytes_to_big() are augmented to check for NIL returns and signal errors in those cases - erts_bs_get_integer_2() can only fail with badmatch, so that is the Erlang-level result of oversize bignums from binary matches - big_SUITE.erl is extended with a test case that fails without this fix (no error signalled) and passes with it (badmatch occurs) Credit goes to Nico Kruber for the initial bug report.
2014-11-03Merge branch 'sverk/yielding-distr-send/OTP-12232'Sverker Eriksson
* sverk/yielding-distr-send/OTP-12232: erts: Add constant TERM_TO_BINARY_MEMCPY_FACTOR erts: Optimize some repeated calls to {E,W}STACK_PUSH erts: Yield in term_to_binary when encoding big maps erts: Remove unnecessary goto for fun encoding erts: Yield in term_to_binary while copying large binaries erts: Implement yielding for distributed send of large messages
2014-11-03erts: Add constant TERM_TO_BINARY_MEMCPY_FACTORSverker Eriksson
and do not piggyback on B2T_MEMCPY_FACTOR
2014-10-24Merge branch 'sverk/hipe-inline-reserve-trap-frame'Sverker Eriksson
* sverk/hipe-inline-reserve-trap-frame: erts: Extend usage of ASM macro to avoid including asm macros in C code erts: Make hipe_{un}reserve_beam_trap_frame INLINE
2014-10-24erts: Make hipe_{un}reserve_beam_trap_frame INLINESverker Eriksson
2014-09-04erts: Refactor binary allocation interface to also initialize BinarySverker Eriksson
except the reference counter 'refc', as different callers have different strategies regarding the lifetime of the binary.
2014-09-03erts: Fix bug in term_to_binary that reallocates binary with wrong sizeSverker Eriksson
2014-08-29erts: Optimize some repeated calls to {E,W}STACK_PUSHSverker Eriksson
2014-08-29erts: Yield in term_to_binary when encoding big mapsSverker Eriksson
2014-08-29erts: Remove unnecessary goto for fun encodingSverker Eriksson
2014-08-29erts: Yield in term_to_binary while copying large binariesSverker Eriksson
Applies also to distributed send.
2014-08-29erts: Implement yielding for distributed send of large messagesSverker Eriksson
Use same mechanism as term_to_binary to yield while encoding large messages for distributed send.
2014-06-05Replace erlang:binary_to_term() Erlang wrappersRickard Green
Replace the 'erlang:binary_to_term/1' and 'erlang:binary_to_term/2' Erlang wrappers taking care of failure after yield with management of this in the hidden yield BIF.
2014-05-13HiPE wrappers for BIFs disabling GCSverker Eriksson
2014-03-14erts: Change external format for mapsSverker Eriksson
to be: 116,Arity, K1,V1,K2,V2,...,Kn,Vn instead of: 116,Arity, K1,K2,...,Kn, V1,V2,....,Vn We think this will be better for future internal map structures like HAMT. Would be bad if we need to iterate twice over HAMT in term_to_binary, one for keys and one for values.
2014-02-25Merge branch 'sverk/hipe-disable-gc-bug'Sverker Eriksson
* sverk/hipe-disable-gc-bug: erts: Fix heap overwrite by hipe "trap frames" when GC is disabled ASSERT that GC is not tried with "need" when GC is disabled