Age | Commit message (Collapse) | Author |
|
* maint:
add enif_schedule_nif() to NIF API
|
|
In the #erlang IRC channel Anthony Ramine once mentioned the idea of
allowing a NIF to use an emulator trap, similar to a BIF trap, to schedule
another NIF for execution. This is exactly how dirty NIFs were implemented
for Erlang/OTP 17.0, so this commit refactors and generalizes that dirty
NIF code to support a new enif_schedule_nif() API function.
The enif_schedule_nif() function allows a long-running NIF to be broken
into separate NIF invocations. The NIF first executes part of the
long-running task, then calls enif_schedule_nif() to schedule a NIF for
later execution to continue the task. Any number of NIFs can be scheduled
in this manner, one after another. Since the emulator regains control
between invocations, this helps avoid problems caused by native code tying
up scheduler threads for too long.
The enif_schedule_nif() function also replaces the original experimental
dirty NIF API. The function takes a flags parameter that a caller can use
to indicate the NIF should be scheduled onto either a dirty CPU scheduler
thread, a dirty I/O scheduler thread, or scheduled as a regular NIF on a
regular scheduler thread. With this change, the original experimental
enif_schedule_dirty_nif(), enif_schedule_dirty_nif_finalizer() and
enif_dirty_nif_finalizer() API functions are no longer needed and have been
removed. Explicit scheduling of a dirty NIF finalization function is no
longer necessary; if an application wants similar functionality, it can
have a dirty NIF just invoke enif_schedule_nif() to schedule a non-dirty
NIF to complete its task.
Lift the restriction that dirty NIFs can't call enif_make_badarg() to raise
an exception. This was a problem with the original dirty NIF API because it
forced developers to get and check all incoming arguments in a regular NIF,
and then schedule the dirty NIF which then had to get all the arguments
again. Now, the argument checking can be done in the dirty NIF and it can
call enif_make_badarg() itself to flag incorrect arguments.
Extend the ErlNifFunc struct with a new flags field that allows NIFs to be
declared as dirty. The default value for this field is 0, indicating a
regular NIF, so it's backwards compatible with all existing statically
initialized ErlNifFunc struct instances, and so such instances require no
code changes. Defining the flags field with a value of
ERL_NIF_DIRTY_JOB_CPU_BOUND indicates that the NIF should execute on a
dirty CPU scheduler thread, or defining it with a value of
ERL_NIF_DIRTY_JOB_IO_BOUND indicates that the NIF should execute on a dirty
I/O scheduler thread. Any other flags field value causes a NIF library
loading error.
Extend the ErlNifEntry struct with a new options field that indicates
whether a NIF library was built with support for optional features such as
dirty NIFs. When a NIF library is loaded, the runtime checks the options
field to ensure compatibility. If a NIF library built with dirty NIF
support is loaded into a runtime that does not support dirty NIFs, and the
library defines one or more ErlNifFunc entries with non-zero flags fields
indicating dirty NIFs, a NIF library loading error results. There is no
error if a NIF library built with dirty NIF support is loaded into a
runtime that does not support dirty NIFs but the library does not have any
dirty NIFs. It is also not an error if a library without dirty NIF support
is loaded into a runtime built with dirty NIF support.
Add documentation and tests for enif_schedule_nif().
|
|
It previously always returned an empty list
and was documented to be removed.
|
|
as it exist in module_info/1.
|
|
|
|
* lukas/ose/master/OTP-11334: (71 commits)
erts: Fix unix efile assert
ose: Use -O2 when building
ose: Expand OSE docs
ose: Add dummy ttsl driver
ose: Cleanup cleanup of mutex selection defines
ose: Polish mmap configure checks
ose: Add ose specific x-compile flags
ose: Updating fd_driver and spawn_driver for OSE
ose: Updating event and signal API for OSE
ose: Cleanup of mutex selection defines
win32: Compile erl_log.exe
ose: Remove uneccesary define
ose: Fix ssl configure test for osx
erts: Fix sys_msg_dispatcher assert
ose: Fix broken doc links
ose: Thread priorities configurable from lmconf
ose: Yielding the cpu is done "the OSE" way
ose: Start using ppdata for tse key
ose: Do not use spinlocks on OSE
ose: Fix support for crypto
...
Conflicts:
lib/crypto/c_src/crypto.c
|
|
This port has support for both non-smp and smp.
It contains a new way to do io checking in which erts_poll_wait
receives the payload of the polled entity. This has implications
for all linked-in drivers.
|
|
* egil/compiler/maps-get_map_elements:
compiler: Strengthen Maps compile tests
compiler: Remove dead warning
erts: Fix erts_debug:disassemble/1
compiler: Transform list of Args to exact literal type
compiler: Test Maps aliasing
compiler: Use aliasing in map pair patterns
compiler: Check literal order in beam_validator
erts: Introduce new instructions for combined key fetches
compiler: Change map instructions for fetching values
|
|
|
|
|
|
|
|
It was not possible to preserve extra arguments in transformations.
The following (hypothetical) example will now work:
some_op Lit=c SizeArg Rest=* => move Lit x | some_op x SizeArg Rest
|
|
files as delimiters.
While working on a tool that processes Erlang code and testing it against this repo,
I found out about those little sneaky 0xff. I thought it may be of help to other
people build such tools to remove non-conforming-to-standard characters.
|
|
|
|
|
|
The BEAM loader will put floating point constants into the
literal pools for the module, but it will not check for duplicates.
We can do much better by having the compiler use the literal
pool for floating point constants.
|
|
* pan/fix-compiler-warnings-clang-and-new-gcc:
Fix compiler warnings from GCC 4.7.1 on ARCH Linux
Fix clang compiler warnings on FreeBSD in erts
|
|
|
|
The following are deliberately left, as I have only a list of compiler
warnings and no system to test on:
hipe/hipe_x86_signal.c:264:5: warning: no previous prototype for function '_sigaction' [-Wmissing-prototypes]
int __SIGACTION(int signum, const struct sigaction *act, struct sigaction *oldact)
^
hipe/hipe_x86_signal.c:222:21: note: expanded from macro '__SIGACTION'
^
1 warning generated.
sys/unix/sys_float.c:835:16: warning: declaration of 'struct exception' will not be visible outside of this function [-Wvisibility]
matherr(struct exception *exc)
^
sys/unix/sys_float.c:835:1: warning: no previous prototype for function 'matherr' [-Wmissing-prototypes]
matherr(struct exception *exc)
^
2 warnings generated.
drivers/unix/unix_efile.c:1504:11: warning: implicit declaration of function 'sendfile' [-Wimplicit-function-declaration]
retval = sendfile(in_fd, out_fd, *offset, SENDFILE_CHUNK_SIZE,
^
1 warning generated.
|
|
|
|
|
|
* sverk/code-load-refactor-later-op:
erts: Fix faulty lock check assert
erts: Refactor naming regarding code_write_permission
erts: Refactor tracing to use erts_schedule_thr_prgr_later_op
erts: Allow thr_prgr_later_op to reschedule
erts: Refactor code loading to use erts_schedule_thr_prgr_later_op
erts: Remove some compiler warnings
|
|
In the erl_crash.dump file, native-compiled modules did not have
any information about attributes and compilation.
The problem is that the code:make_stub_module/3 BIF (which is
internally used when native code is loaded) did not copy the size
field the attribute and compilation info chunks. Those size fields
are only used when writing crash dumps.
|
|
The concept of code_write_permission is used by tracing as well
and is not specific to code_ix.
|
|
|
|
Introduce two new BIFs, erlang:prepare_loading/2 and
erlang:finish_loading/1, and re-implement erlang:load_module/2 in
Erlang code.
We have two reasons for doing this:
* To facilitate suspending a process if another process
is already doing code loading.
* In the future, we can implement parallel and atomic loading
of several modules. Atomic loading works except for modules with
on_load handlers. Because of that issue, erlang:finish_loading/2
will currently only accept a list with a single magic binary.
|
|
|
|
Move implementation from beam_load into new file code_ix.c and module.c
and make some function inline.
|
|
The is a refactoring in preparation to add a counter in Module struct
for export entry tracing. It is nicer if the two are kept together.
|
|
|
|
Staging is a better and more general name as does not necessary need
to involve code loading (can be deletion, tracing, etc).
|
|
|
|
As it can only be used at initialization for preloading
|
|
Renamed merge_secondary_table and called by export_start_load
|
|
Implemented some code_ix locks
and commented calls to erts_smp_thr_progress_block()
|
|
|
|
Having the entire implementation of range handling (address table)
in one source file will help when we'll need to update the ranges
without stopping all schedulers in the next commit.
|
|
|
|
|
|
Still blocking code loading
|
|
Code loading still blocking
|
|
The default array was defined but not used.
|
|
|
|
Commit 64ccd8c9b7a782ca777ca4649dbb1f4a1ef00bce introduced BIF
stubs. The stub functions were not actually remove the loaded
code, but the name of the function in the func_info instruction
was changed to [] to mark it as invalid.
The actual code for module_info(native_addresses) did not need
to be updated (a BIF stub can never have a native address and
a function without a native address will never be included in
the list), but the assertion that the name is an atom is no
no longer correct.
|
|
We want to be able to write type specifications for BIFs in the same
way as for any other function. Currently, the type for BIFs need
to be described in erl_bif_types.
To avoid extending the compiler and Dialyzer with special directives
for providing specifications for BIFs, we have decided to let the
loader accept a local definition for a function which exists as a
BIF. As an example, here is how a stub for lists:reverse/2 can be
defined:
-export([reverse/2]).
-spec reverse([term()], term()) -> [term()].
reverse(_, _) ->
erlang:nif_error(undef).
Essentially, the loader will discard the local definition of reverse/2.
Other functions in the same module must *not* do local calls to a BIF
stub. If a local call to a BIF is found, the loader will refuse to load
the module. That is, the following call is not allowed:
reverse(List) ->
reverse(List, []).
but the following is:
reverse(List) ->
?MODULE:reverse(List, []).
A few words about the implementation.
It turns out to be too complicated to actually discard the BIF
stubs. Although it would be possibly with some jiggery pokery in
ops.tab, the code would be difficult to maintain and it could slow
down loading of modules that don't define BIFs (which are almost
all modules).
Therefore, the stub functions are kept in the loaded code, but
their names in the func_info instruction are invalidated so that
module_info(functions) can filter them out.
|
|
For errors that occur after reading the code chunking, saying
that the error occurred in the last function in the module and
in the instruction int_code_end/0 is just confusing.
|
|
The idea was probably to cause less fragmentation. Even if that would
be true, it is irrelevant because the short-lived allocator that is
used does not have any problems with fragmentation.
In CodeNeed() we will simply double the size of the area used for code
instead of using the next heap size.
|
|
It is wrongly assumed in the BEAM loader that apply/2 is a BIF
and must be treated specially. Also make it clearer in ops.tab
that apply/3 is a BIF, but apply/2 is not.
|
|
In commit b67d3e5447f4b2bca3ed92f3db84adb3f79f9b16 (which cleaned up
handling of error reasons), the test of the return value from
beam_make_current_old() in insert_new_code() was not updated, which
meant that it never was true and any number of versions of code could
be loaded for a module.
|
|
Almost all uses of the 'long' datatype is removed from VM and tests
Emulator test now runs w/o drivers crashing
Nasty abs bug fixed in VM as well as type errors in allocator debug functions
Still one allocator test that fails, domain knowledge is needed to fix that.
Fix type inconsistency in beam_load causing crashes
|