| Age | Commit message (Collapse) | Author |
|---|
|
|
|
* hamt_bin2term:
erts: Add erts_factory_trim_and_close
erts: Optimize driver_deliver_term
erts: Remove hashmap probabilistic heap overestimation
Conflicts:
erts/emulator/beam/beam_load.c
|
|
by adding a dynamic heap factory.
"binary_to_term" is now a hybrid solution with both
a call to decoded_size() to calculate needed heap space
AND possible dynamic allocation of more heap space
if needed for big maps.
The heap size returned from decoded_size() is guaranteed
to be sufficient for all term heap data except for hashmap
nodes. All hashmap nodes are created at the end of dec_term()
by invoking the heap factory interface that may allocate more
heap space on process heap or in fragments.
With this commit it is no longer guaranteed that a message
is confined to only one heap fragment.
|
|
Add a check to protect from segfault when erlang:get_module_info/1/2 is
called on a deleted module (i.e. with no current code). Also refactor
erts_module_info_0/1 to avoid repeated calls to erts_active_code_ix() and
remove some obsolete comments. Add test for module_info on deleted modules.
|
|
Use the md5 of the native code chunk instead of the Beam code md5.
|
|
|
|
|
|
A type error caused the optimization to never kick in.
|
|
See the previous commit for justification and use cases.
|
|
Let the loader pre-compute the hash value when a single, literal key
is matched as in:
#{<<"some_key">>:=V} = Map
In my measurements, this optimization resulted in a 30 percent
speedup for short binary keys.
Unfortunately, this optimizization makes no difference for small
maps with less than 32 keys, since the hash value is not used.
Still, there are the following use cases:
* A map used instead of a record with more than 32 entries. I have
seen some applications with huge records.
* Lookup in JSON dictionaries represented as maps.
The hash value will only be used when the map is a hash map
(currently, that means at least 32 entries).
|
|
The map instructions require that the keys in the instructions
are sorted (for flatmaps). But that is an implementation detail
that should not exposed outside of the BEAM virtual machine.
Therefore, make the sorting of the keys the responsibility of
the loader and not the compiler.
Also note that the sort order for maps with numeric keys or keys
with numeric components has changed in OTP 18. That means that
code compiled for OTP 17 that operated on maps with map keys
might not work in OTP 18 without the sorting in the loader
(although it is unlikely to be an issue in practice).
|
|
The has_map_fields instruction is infrequently used. Thus there
is no need to have the fastest possible implementation; it is
better to have an implementation that reduces the code size in
the already big process_main() function.
We can transform has_map_fields to a get_map_elements instruction,
targeting the same unused x[0] register for all keys. That
instruction will only be marginally slower than existing
implementation.
|
|
The compiler will only emit is_map/1 instructions with literal
argument if optimization is turned off. Therefore, the only
reason for this commit is cleanliness.
|
|
A put_map_assoc instruction with an empty source map should be
converted to a simpler new_map instruction. The transformation
didn't happen because an empty source map is no longer represented
as a NIL term (as it was in the beginning before map literals
were implemented).
|
|
|
|
Conflicts:
erts/emulator/hipe/hipe_bif0.c
|
|
Bignums are artifically restricted in size. Arithmetic and logical
operations check the sizes of resulting bignums, and turn oversize
results into system_limit exceptions.
However, this check is not performed when bignums are constructed by
binary matching. The consequence is that such matchings can construct
oversize bignums that satisfy is_integer/1 yet don't work. Performing
arithmetic such as Term - 0 fails with a system_limit exception. Worse,
performing a logical operation such as Term band Term results in [].
The latter occurs because the size checking (e.g. in erts_band()) is
a simple ASSERT(is_not_nil(...)) on the result of the bignum operation,
which internally is [] (NIL) in the case of oversize results. However,
ASSERT is a no-op in release builds, so the error goes unnoticed and []
is returned as the result of the band/2.
This patch addresses this by preventing oversize bignums from entering
the VM via binary matching:
- the internal bytes_to_big() procedure is augmented to return NIL for
oversize results, just like big_norm()
- callers of bytes_to_big() are augmented to check for NIL returns and
signal errors in those cases
- erts_bs_get_integer_2() can only fail with badmatch, so that is the
Erlang-level result of oversize bignums from binary matches
- big_SUITE.erl is extended with a test case that fails without this
fix (no error signalled) and passes with it (badmatch occurs)
Credit goes to Nico Kruber for the initial bug report.
|
|
For searching a key in an array we use linear search in arrays
up to 10 elements.
Selecting on tuple arity will always use linear search.
Instead of using two different instructions we assume selecting on
different tuple arities are always few in numbers.
|
|
* maint:
add enif_schedule_nif() to NIF API
|
|
In the #erlang IRC channel Anthony Ramine once mentioned the idea of
allowing a NIF to use an emulator trap, similar to a BIF trap, to schedule
another NIF for execution. This is exactly how dirty NIFs were implemented
for Erlang/OTP 17.0, so this commit refactors and generalizes that dirty
NIF code to support a new enif_schedule_nif() API function.
The enif_schedule_nif() function allows a long-running NIF to be broken
into separate NIF invocations. The NIF first executes part of the
long-running task, then calls enif_schedule_nif() to schedule a NIF for
later execution to continue the task. Any number of NIFs can be scheduled
in this manner, one after another. Since the emulator regains control
between invocations, this helps avoid problems caused by native code tying
up scheduler threads for too long.
The enif_schedule_nif() function also replaces the original experimental
dirty NIF API. The function takes a flags parameter that a caller can use
to indicate the NIF should be scheduled onto either a dirty CPU scheduler
thread, a dirty I/O scheduler thread, or scheduled as a regular NIF on a
regular scheduler thread. With this change, the original experimental
enif_schedule_dirty_nif(), enif_schedule_dirty_nif_finalizer() and
enif_dirty_nif_finalizer() API functions are no longer needed and have been
removed. Explicit scheduling of a dirty NIF finalization function is no
longer necessary; if an application wants similar functionality, it can
have a dirty NIF just invoke enif_schedule_nif() to schedule a non-dirty
NIF to complete its task.
Lift the restriction that dirty NIFs can't call enif_make_badarg() to raise
an exception. This was a problem with the original dirty NIF API because it
forced developers to get and check all incoming arguments in a regular NIF,
and then schedule the dirty NIF which then had to get all the arguments
again. Now, the argument checking can be done in the dirty NIF and it can
call enif_make_badarg() itself to flag incorrect arguments.
Extend the ErlNifFunc struct with a new flags field that allows NIFs to be
declared as dirty. The default value for this field is 0, indicating a
regular NIF, so it's backwards compatible with all existing statically
initialized ErlNifFunc struct instances, and so such instances require no
code changes. Defining the flags field with a value of
ERL_NIF_DIRTY_JOB_CPU_BOUND indicates that the NIF should execute on a
dirty CPU scheduler thread, or defining it with a value of
ERL_NIF_DIRTY_JOB_IO_BOUND indicates that the NIF should execute on a dirty
I/O scheduler thread. Any other flags field value causes a NIF library
loading error.
Extend the ErlNifEntry struct with a new options field that indicates
whether a NIF library was built with support for optional features such as
dirty NIFs. When a NIF library is loaded, the runtime checks the options
field to ensure compatibility. If a NIF library built with dirty NIF
support is loaded into a runtime that does not support dirty NIFs, and the
library defines one or more ErlNifFunc entries with non-zero flags fields
indicating dirty NIFs, a NIF library loading error results. There is no
error if a NIF library built with dirty NIF support is loaded into a
runtime that does not support dirty NIFs but the library does not have any
dirty NIFs. It is also not an error if a library without dirty NIF support
is loaded into a runtime built with dirty NIF support.
Add documentation and tests for enif_schedule_nif().
|
|
It previously always returned an empty list
and was documented to be removed.
|
|
as it exist in module_info/1.
|
|
|
|
* lukas/ose/master/OTP-11334: (71 commits)
erts: Fix unix efile assert
ose: Use -O2 when building
ose: Expand OSE docs
ose: Add dummy ttsl driver
ose: Cleanup cleanup of mutex selection defines
ose: Polish mmap configure checks
ose: Add ose specific x-compile flags
ose: Updating fd_driver and spawn_driver for OSE
ose: Updating event and signal API for OSE
ose: Cleanup of mutex selection defines
win32: Compile erl_log.exe
ose: Remove uneccesary define
ose: Fix ssl configure test for osx
erts: Fix sys_msg_dispatcher assert
ose: Fix broken doc links
ose: Thread priorities configurable from lmconf
ose: Yielding the cpu is done "the OSE" way
ose: Start using ppdata for tse key
ose: Do not use spinlocks on OSE
ose: Fix support for crypto
...
Conflicts:
lib/crypto/c_src/crypto.c
|
|
This port has support for both non-smp and smp.
It contains a new way to do io checking in which erts_poll_wait
receives the payload of the polled entity. This has implications
for all linked-in drivers.
|
|
* egil/compiler/maps-get_map_elements:
compiler: Strengthen Maps compile tests
compiler: Remove dead warning
erts: Fix erts_debug:disassemble/1
compiler: Transform list of Args to exact literal type
compiler: Test Maps aliasing
compiler: Use aliasing in map pair patterns
compiler: Check literal order in beam_validator
erts: Introduce new instructions for combined key fetches
compiler: Change map instructions for fetching values
|
|
|
|
|
|
|
|
It was not possible to preserve extra arguments in transformations.
The following (hypothetical) example will now work:
some_op Lit=c SizeArg Rest=* => move Lit x | some_op x SizeArg Rest
|
|
files as delimiters.
While working on a tool that processes Erlang code and testing it against this repo,
I found out about those little sneaky 0xff. I thought it may be of help to other
people build such tools to remove non-conforming-to-standard characters.
|
|
|
|
|
|
The BEAM loader will put floating point constants into the
literal pools for the module, but it will not check for duplicates.
We can do much better by having the compiler use the literal
pool for floating point constants.
|
|
* pan/fix-compiler-warnings-clang-and-new-gcc:
Fix compiler warnings from GCC 4.7.1 on ARCH Linux
Fix clang compiler warnings on FreeBSD in erts
|
|
|
|
The following are deliberately left, as I have only a list of compiler
warnings and no system to test on:
hipe/hipe_x86_signal.c:264:5: warning: no previous prototype for function '_sigaction' [-Wmissing-prototypes]
int __SIGACTION(int signum, const struct sigaction *act, struct sigaction *oldact)
^
hipe/hipe_x86_signal.c:222:21: note: expanded from macro '__SIGACTION'
^
1 warning generated.
sys/unix/sys_float.c:835:16: warning: declaration of 'struct exception' will not be visible outside of this function [-Wvisibility]
matherr(struct exception *exc)
^
sys/unix/sys_float.c:835:1: warning: no previous prototype for function 'matherr' [-Wmissing-prototypes]
matherr(struct exception *exc)
^
2 warnings generated.
drivers/unix/unix_efile.c:1504:11: warning: implicit declaration of function 'sendfile' [-Wimplicit-function-declaration]
retval = sendfile(in_fd, out_fd, *offset, SENDFILE_CHUNK_SIZE,
^
1 warning generated.
|
|
|
|
|
|
* sverk/code-load-refactor-later-op:
erts: Fix faulty lock check assert
erts: Refactor naming regarding code_write_permission
erts: Refactor tracing to use erts_schedule_thr_prgr_later_op
erts: Allow thr_prgr_later_op to reschedule
erts: Refactor code loading to use erts_schedule_thr_prgr_later_op
erts: Remove some compiler warnings
|
|
In the erl_crash.dump file, native-compiled modules did not have
any information about attributes and compilation.
The problem is that the code:make_stub_module/3 BIF (which is
internally used when native code is loaded) did not copy the size
field the attribute and compilation info chunks. Those size fields
are only used when writing crash dumps.
|
|
The concept of code_write_permission is used by tracing as well
and is not specific to code_ix.
|
|
|
|
Introduce two new BIFs, erlang:prepare_loading/2 and
erlang:finish_loading/1, and re-implement erlang:load_module/2 in
Erlang code.
We have two reasons for doing this:
* To facilitate suspending a process if another process
is already doing code loading.
* In the future, we can implement parallel and atomic loading
of several modules. Atomic loading works except for modules with
on_load handlers. Because of that issue, erlang:finish_loading/2
will currently only accept a list with a single magic binary.
|
|
|
|
Move implementation from beam_load into new file code_ix.c and module.c
and make some function inline.
|
|
The is a refactoring in preparation to add a counter in Module struct
for export entry tracing. It is nicer if the two are kept together.
|
|
|
|
Staging is a better and more general name as does not necessary need
to involve code loading (can be deletion, tracing, etc).
|
|
|
