aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam/external.c
AgeCommit message (Collapse)Author
2013-07-11Merge branch 'jw/fix-float-middle-endian/OTP-11201' into maintFredrik Gustafsson
* jw/fix-float-middle-endian/OTP-11201: Fix binary construction on floating point middle-endian machines Fix binary matching on floating point middle-endian machines Fix erlang:phash2() on floating point middle-endian machines Fix external term format BIFs on floating point middle-endian machines
2013-06-14erts: Fix memory leak in term_to_binary/2 with compressed optionSverker Eriksson
2013-06-14Fix external term format BIFs on floating point middle-endian machinesJohannes Weißl
This complements 933e701 (OTP-10209). Simple error example: 1> <<131,70,63,240,0,0,0,0,0,0>> = term_to_binary(1.0, [{minor_version,1}]). ** exception error: no match of right hand side value <<131,70,0,0,0,0,63,240,0,0>> 2> 1.0 = binary_to_term(<<131,70,63,240,0,0,0,0,0,0>>). ** exception error: no match of right hand side value 5.299808824e-315 But roundtrip always works: 3> 1.0 = binary_to_term(term_to_binary(1.0, [{minor_version,1}])). 1.0
2013-06-10term_to_binary: Remove debug code and set production trap levelsPatrik Nyblom
2013-06-06Teach erl_gc:offset_rootset about extra_rootPatrik Nyblom
2013-06-05Teach external.c to handle reallocs before compressionPatrik Nyblom
2013-06-05Make all steps ofterm_to_binary work in chunks and yieldPatrik Nyblom
Rewrite and extend of Happi's initial work Extra_root to process structure to enable GC of state - Changed the process structure to point to a separate struct, the struct also contains a destructor function to allow for proper cleanup. Rewrote encode_size_struct and enc_term to have internal versions with reduction counters which will result in interrupt for later restart when the counter reaches zero - removed the EWA_STACK from Happis version and directly save the ESTACK's and WSTACK's in the above mentioned struct (or array thereof) that are pointed out from the process structure. The destructor will take care of the deallocation in case of process death. Added ESTACK and WSTACK macros to save and restore stack and to change allocator, which makes the previously mentioned stack-save work. Rewrote enc_term to not store pointers on the stack, and use one WSTACK for commands etc and another ESTACK for Eterms - Slightly different than Happis version to make halfword code simpler. Rewrote encode_size_struct2 so that it does not store pointers on the stack, also switched to ESTACK instead of WSTACK, this also handles halfword correctly. Added interfaces for chunkwise compression, that are used from term_to_binary/2 when the compressed option is given.
2013-06-04Make term_to_binary yield (trap).happi
2013-04-15erts: Remove unnecessary utf8 analysis step in external decodingSverker Eriksson
Faulty utf8 atoms are rejected anyway later when the atom is created.
2013-01-28Merge branch 'sverk/enc_atom-opt'Sverker Eriksson
* sverk/enc_atom-opt: erts: Optimize atom encoding to use memcpy for pure ascii erts: Refactor erts_atom_get to use ErtsAtomEncoding
2013-01-25Merge branch 'pan/fix-compiler-warnings-clang-and-new-gcc'Patrik Nyblom
* pan/fix-compiler-warnings-clang-and-new-gcc: Fix compiler warnings from GCC 4.7.1 on ARCH Linux Fix clang compiler warnings on FreeBSD in erts
2013-01-25Update copyright yearsBjörn-Egil Dahlberg
2013-01-25erts: Optimize atom encoding to use memcpy for pure asciiSverker Eriksson
2013-01-25erts: Refactor erts_atom_get to use ErtsAtomEncodingSverker Eriksson
instead of 'is_latin1' boolean argument.
2013-01-23Fix clang compiler warnings on FreeBSD in ertsPatrik Nyblom
The following are deliberately left, as I have only a list of compiler warnings and no system to test on: hipe/hipe_x86_signal.c:264:5: warning: no previous prototype for function '_sigaction' [-Wmissing-prototypes] int __SIGACTION(int signum, const struct sigaction *act, struct sigaction *oldact) ^ hipe/hipe_x86_signal.c:222:21: note: expanded from macro '__SIGACTION' ^ 1 warning generated. sys/unix/sys_float.c:835:16: warning: declaration of 'struct exception' will not be visible outside of this function [-Wvisibility] matherr(struct exception *exc) ^ sys/unix/sys_float.c:835:1: warning: no previous prototype for function 'matherr' [-Wmissing-prototypes] matherr(struct exception *exc) ^ 2 warnings generated. drivers/unix/unix_efile.c:1504:11: warning: implicit declaration of function 'sendfile' [-Wimplicit-function-declaration] retval = sendfile(in_fd, out_fd, *offset, SENDFILE_CHUNK_SIZE, ^ 1 warning generated.
2013-01-16UTF-8 support for distributionRickard Green
2013-01-08erts: Change internal representation of atoms to utf8Sverker Eriksson
2013-01-08erts: Refactor rename DFLAG(S)_INTERNAL_TAGS for conformitySverker Eriksson
2012-09-07Replace sprintf with erts_snprintf in beamBjörn-Egil Dahlberg
2012-08-15Merge branch 'maint'Sverker Eriksson
2012-08-14Fix corrupted binaries in compressed ETS tablesSverker Eriksson
2012-06-04Merge branch 'maint'Lukas Larsson
* maint: Remove stale code for hybrid heap and incremental GC Remove the hipe_bifs:show_message_area/0 BIF Remove support for erlang:system_info(global_heaps_size) Remove the erlang:garbage_collect_message_area/0 BIF Remove workarounds for hybrid and shared heaps in test suites Conflicts: erts/doc/src/erlang.xml erts/emulator/beam/erl_message.c erts/emulator/beam/erl_process.c erts/emulator/beam/erl_process.h erts/emulator/hipe/hipe_bif2.tab lib/hipe/cerl/erl_bif_types.erl
2012-05-10Remove stale code for hybrid heap and incremental GCBjörn Gustavsson
The hybrid heap emulator was last working in the non-SMP R11B run-time system. When the constant pools were introduced in R12B, the hybrid heap emulator was not updated to handle them. At this point, the harm from reduced readability of the code is greater than any potential usefulness of keeping the code.
2012-03-30Merge branch 'maint'Björn-Egil Dahlberg
2012-03-30Update copyright yearsBjörn-Egil Dahlberg
2012-03-09Merge branch 'sverk/threadsafe-code-loading'Sverker Eriksson
* sverk/threadsafe-code-loading: (59 commits) erts: Fix assert failure when code_server exits "during" commit erts: Fix memory leak in code loading erts: Adapt gdb etp-command for new beam_ranges erts: Set correct default tracing when loading code erts: Fix faulty assert in non-smp debug vm erts: Use correct macro for "yield-return" erts: Refactor code loading with renaming erts: Seize code_ix lock when updating trace settings erts: Switch order between code_ix lock and thread blocking erts: Fix race bug in finish_after_on_load erts: Refactor export staging lock erts: Activate staged code in a thread safe way erts: Suspend processes waiting for code_ix lock erts: Fix compiler warning in inet_drv erts: Fix single threaded fallback in new BIF finish_loading_1 erts: Fix type bug Break apart erlang:load_module/2 into two separate BIFs Use magic binaries in erts_prepare_loading() and erts_finish_loading() erts: Cleanup non-blocking load erts: Fix memory query for non-blocking module table ... OTP-9974
2012-02-21erts: Multiple export tab's using code_ixSverker Eriksson
Still blocking code loading
2012-02-21erts: Fail binary_to_term if bignum arity is too largeSverker Eriksson
2011-12-02Get cerl and distribution working in Win64Patrik Nyblom
Can still not setup -a, but cerl works.
2011-11-17external.c: Remove the option to create over-sized heap binariesBjörn Gustavsson
Since refc binaries are now supported in literal pools, there is no longer any need to allow the creation of over-sized heap binaries.
2011-10-26Use the proper macros in all BIFsBjörn Gustavsson
As a preparation for changing the calling convention for BIFs, make sure that all BIFs use the macros. Also, eliminate all calls from one BIF to another, since that also breaks the calling convention abstraction.
2011-08-27Add erlang:external_size/2 BIFFilipe David Manana
This BIF's second parameter is a list of options. Currently the only allowed option is {minor_version, Version} where version is either 0 (default) or 1.
2011-03-11Update copyright yearsBjörn-Egil Dahlberg
2011-03-09Fix 18 exabyte memory allocation failureBjörn Gustavsson
The new_binary() function takes a size argument that is an int. In the 64-bit emulator (sizeof(int) == 4, sizeof(Uint) == 8), any sizes >= 0x8000000 become 0xffffffff80000000 and above and triggers a memory allocation failure. Change the type of the size argument to Uint, and change any callers that cast the argument to an int. Correction-by: Jon Meredith
2010-11-22ETS 'compressed' option.Sverker Eriksson
The compressed format is using a slighty modified variant of the extern format (term_to_binary). To not worsen key lookup's too much, the top tuple itself and the key element are not compressed. Table objects with only immediate non-key elements will therefor not gain anything (but actually consume one extra word for "alloc_size").
2010-08-07Align trailing message data for all types of structuresRickard Green
Alignment of trailing data in messages has been adjusted. This in order to be able to pass data of any type as trailing data in the future.
2010-07-20One off-heap list, to eliminate two words per ETS object.Sverker Eriksson
Merging the three off-heap lists (binaries, funs and externals) into one list. This reduces memory consumption by two words (pointers) per ETS object.
2010-07-05term_to_binary use all 32 bits of INTEGER_EXTSverker Eriksson
Earlier, external format INTEGER_EXT was only produced for 28-bit signed integers. Now full 32-bit signed integers are produced as INTEGER_EXT to avoid the more costly SMALL_BIG_EXT format. Both old and new code can read 32-bit INTEGER_EXT. Also fixed integer encoding bugs in erl_interface erl_encode/erl_decode. (Thanks to Alexander Demidenko for reporting)
2010-03-10Fix further test-suite problemsPatrik Nyblom
Fix safe_mul in the loader, which caused failures in the bit syntax test cases. Fix yet another Uint in erl_alloc.h (ERTS_CACHE_LINE_SIZE) causing segmentation fault when we have many schedulers (why only in that situation?). Clean up erl_mseg (remove old code for the Linux 32-bit mmap flag). While at it, also remove compilation warnings.
2010-03-10Correct the VM to work for more test suitesPatrik Nyblom
The following test suites now work: send_term_SUITE trace_nif_SUITE binary_SUITE match_spec_SUITE node_container_SUITE beam_literals_SUITE Also add a testcases for system_info({wordsize,internal|external}).
2010-03-10Make tracing and distribution workPatrik Nyblom
Rewrite trace code and external coding. Also slightly correct the interface to the match-spec engine to make tracing work. That will make the test suites runnable.
2010-03-10Turn on instruction packing in the loader and virtual machinePatrik Nyblom
2010-03-10Store pointers to heap data in 32-bit wordsPatrik Nyblom
Store Erlang terms in 32-bit entities on the heap, expanding the pointers to 64-bit when needed. This works because all terms are stored on addresses in the 32-bit address range (the 32 most significant bits of pointers to term data are always 0). Introduce a new datatype called UWord (along with its companion SWord), which is an integer having the exact same size as the machine word (a void *), but might be larger than Eterm/Uint. Store code as machine words, as the instructions are pointers to executable code which might reside outside the 32-bit address range. Continuation pointers are stored on the 32-bit stack and hence must point to addresses in the low range, which means that loaded beam code much be placed in the low 32-bit address range (but, as said earlier, the instructions themselves are full words). No Erlang term data can be stored on C stacks (enforced by an earlier commit). This version gives a prompt, but test cases still fail (and dump core). The loader (and emulator loop) has instruction packing disabled. The main issues has been in rewriting loader and actual virtual machine. Subsystems (like distribution) does not work yet.
2010-03-10Fit all heap data into the 32-bit address rangePatrik Nyblom
This is the first step in the implementation of the half-word emulator, a 64-bit emulator where all pointers to heap data will be stored in 32-bit words. Code specific for this emulator variant is conditionally compiled when the HALFWORD_HEAP define has a non-zero value. First force all pointers to heap data to fall into a single 32-bit range, but still store them in 64-bit words. Temporary term data stored on C stack is moved into scheduler specific storage (allocated as heaps) and macros are added to make this happen only in emulators where this is needed. For a vanilla VM the temporary terms are still stored on the C stack.
2010-02-11OTP-8335 Even more NIF featuresSverker Eriksson
2010-02-02Add the +t emulator option to change the maximum number of atomsJulien Barbot
It is now possible to increase or decrease the maximum number of atoms the VM can handle. The default value is 1048576 (1024*1024).
2010-01-13OTP-8240 Improved GC performance after BIF/NIF call when a lot of heapSverker Eriksson
fragments was created. This will mainly benefit NIFs that return large compound terms.
2010-01-12Merge branch 'jv/binary_to_term-opts' into ccase/r13b04_devErlang/OTP
* jv/binary_to_term-opts: document ErtsExternalDist flags and CON_ID mask add options to binary_to_term OTP-8367 There is new erlang:binary_to_binary/2 BIF that takes an option list. The option safe can be used to prevent creation of resources that are not garbage collected (such as atoms). (Thanks to Jayson Vantuyl.)
2010-01-08add options to binary_to_termJayson Vantuyl
term_to_binary and binary_to_term are powerful tools that can be used easily in lieu of a custom binary network protocol. Unfortunately, carefully crafted data can be used to exhaust the memory in an Erlang node by merely attempting to decode binaries. This makes it unsafe to receive data from untrusted sources. This is possible because binary_to_term/1 will allocate new atoms and new external function references. These data structures are not garbage collected. This patch implements the new form of binary_to_term that takes a list of options, and a simple option called 'safe'. If specified, this option will cause decoding to fail with a badarg error if an atom or external function reference would be allocated. In the general case, it will happily decode any Erlang term other than those containing new atoms or new external function references. However, fun, pid, and ref data types can embed atoms. They might fail to decode if one of these embedded atoms is new to the node. This may be an issue if encoded binaries are transferred between nodes or persisted between invocations of Erlang.
2009-11-20The R13B03 release.OTP_R13B03Erlang/OTP