| Age | Commit message (Collapse) | Author |
|---|
|
See https://github.com/erszcz/docsh/issues/23 for an explanation
about how the files were found.
|
|
Currently HiPE amd64 assumes the runtime system code is loaded into
the low 2G of the address space. However, this is not the case when
PIE is enabled, it is loaded into a random location. So trampolines
are required to call BIFs, and also we have first to load the address
of sse2_fnegate_mask to a regisiter before xorpd in fchs.
|
|
particularly slow erlc when compiler is hipe compiled.
hipe_unified_loader:load did not patch external call sites
and instead caused a double hipe mode switch per call.
hipe_unified_loader:load is only used
for early modules first loaded as beam
and by code:atomic_load and friends.
|
|
|
|
|
|
The code of add_ref/6 contained an assertion that somehow appeared
twice. Also, a recent commit (2b2de61) suppressed an unmatched
return warning in a non-optimal way. By changing the return type
of hipe_bifs:add_ref/2 to be 'ok' instead of [] the code can be
cleaned up and simplified.
While at it, fixed the wrong indentation of some nearby code.
|
|
|
|
by introducing hipe_bifs:commit_patch_load/1
that creates the HipeModule.
|
|
|
|
Just like the BEAM loader state (as returned by
erlang:prepare_loading/2), the HiPE loader state is contained in a magic
binary.
Eventually, we will separate HiPE loading into a prepare and a finalise
phase, like the BEAM loader, where the prepare phase will be implemented
by hipe_unified_loader and the finalise phase be implemented in C by
hipe_load.c and beam_load.c, making prepare side-effect free and
finalise atomic. The finalise phase will be exposed through the
erlang:finish_loading/1 API, just like the BEAM loader, as this will
allow HiPE and BEAM modules to be mixed in the same atomic "commit".
The usage of a loader state makes it easier to keep track of all
resources allocated during loading, and will not only make it easy to
prevent leaks when hipe_unified_loader crashes, but also paves the way
for proper, leak-free, unloading of HiPE modules.
|
|
and hipe_bifs:update_code_size
|
|
A step toward better integration of hipe load and purge
Highlights:
* code_server no longer needs to call hipe_unified_loader:post_beam_load/1
Instead new internal function hipe_redirect_to_module()
is called by loading BIFs to patch native call sites if needed.
* hipe_purge_module() is called by erts_internal:purge_module/2
to purge any native code.
* struct hipe_mfa_info redesigned and only used for exported
functions that are called from or implemented by native code.
A list of native call sites (struct hipe_ref) are kept for each hipe_mfa_info.
* struct hipe_sdesc used by hipe_find_mfa_from_ra()
to build native stack traces.
|
|
as they are lists of #fundef records
|
|
- The calling process is now suspended while synchronizing
scheduler suspends via erlang:system_flag(schedulers_online, _)
and erlang:system_flag(multi_scheduling, _), instead of blocking
the scheduler thread in the BIF call waiting for the operation
to synchronize. Besides releasing the scheduler for other work
(or immediate suspend) it also makes it possible to abort the
operation by killing the process.
- erlang:system_flag(schedulers_online, _) now only wait for normal
schedulers to complete before it returns. This since it may take
a very long time before all dirty schedulers suspends.
- erlang:system_flag(multi_scheduling, block_normal|unblock_normal)
which only operate on normal schedulers has been introduced. This
since there are use cases where suspend of dirty schedulers are
not of interest (hipe loader).
- erlang:system_flag(multi_scheduling, block) still blocks all
dirty schedulers as well as all normal schedulers except one since
it is hard to redefine what multi scheduling block means.
- The three operations:
- changing amount of schedulers online
- blocking/unblocking normal multi scheduling
- blocking/unblocking full multi scheduling
can now be done in parallel. This is important since otherwise
a full multi scheduling block would potentially delay the other
operations for a very long time.
|
|
After loading a module without native code, it is still necessary
to call hipe_unified_loader:post_beam_load() to ensure that any
native calls to the module is done to the newly loaded module
(and not to a previous version of the module in native code).
Unfortunately, hipe_unified_loader:post_beam_load() can be slow
and most of the time it doesn't do anything because no previous
native code was loaded. Therefore, ad2962278f added a kludge using
the process dictionary to avoid calling post_beam_load() if no
native code at all has been loaded.
Remove the kludge by keeping track exactly of which modules that
have native code in the existing ets table. Also generalize
post_beam_load() to handle severals modules at once, since we
will soon need that functionality.
|
|
Make code_server be responsible for finding the architecture and deciding
whether to try to load native code, in order to avoid repeated calls to
system_info(hipe_architecture) and clumsy uses of try/catch to test if hipe
is enabled.
|
|
Make hipe_unified_loader not call system_info(hipe_architecture) repeatedly.
Also clean up some ugly architecture-dependent case switches.
|
|
Use the md5 of the native code chunk instead of the Beam code md5.
|
|
The hipe_bifs:make_native_stub/2 and hipe_bifs:get_emu_address/1
BIFs were originally used by hipe_unified_loader.erl, but the
code been obsolete and disabled for ages.
Remove the BIFs and all references to them.
In hipe_unified_loader.erl, remove the no-op emu_make_stubs/1
function.
|
|
Add flags to enable and use the LLVM backend:
* to_llvm: use the LLVM pipeline for compilation (default optimization level
is O3),
* llvm_save_temps: save the intermediate files in current directory in order
to be able to debug or optimize the LLVM assembly,
* {to_llvm, O}: set the optimization level of LLVM opt and llc tools.
Add some debug support to the loader; no semantic change intented.
|
|
The loader of native code tries to avoid race condition by bringing
down the system to a single scheduler. Unfortunately, that will not
completely avoid race conditions beacuse other processes may still
run while the code is being loaded. Therefore, we must still make
sure that native code has been fully fixed up before it can be
executed.
Under unlucky circumstances, it was possible for the global name
register process to call a fun in native code before the native
code for the fun had been fully fixed up. The run-time system
would crash when running the not-fully-fixed-up code.
Here is a way to make the bug 100% reproducible. First add this
function to the 'global' module:
silly_looper(List) ->
lists:foreach(fun(E) -> put(any_atom, E) end, List),
silly_looper(List).
Then insert the following code at the beginning of the init/1
function in the 'global' module:
spawn_link(fun() -> silly_looper(lists:seq(1, 100)) end),
The run-time system will crash during start up when the native
code for the 'global' module is being loaded. What will happen
is that lists:foreach/2 (which runs in native code) will call
the native code for the fun in silly_looper/1 before the reference
to 'any_atom' has been changed from a 0 to the proper tagged
atom value. The put/2 BIF will exit when attempting to calculate
the hash value for the illegal value 0 (it is not properly tagged).
To eliminate this race condition, we must delay patching the
native code addresses for a fun into the fun entry until the native
code has been fully fixed up. We will use the new BIFs introduced
in the previous commit.
While we are at it, we will also make sure that we erase any
temporary variables in the process dictionary used by the
hipe_unified_loader module.
|
|
This commit is a preparation for eliminating a race condition
loading the native code for modules whose BEAM code has already
been loaded. Here we introduce two new BIFs so that looking up
a fun entry is separate from setting the native address in the
fun entry.
|
|
Conflicts:
bootstrap/lib/kernel/ebin/hipe_unified_loader.beam
|
|
Consisting of three items:
- Eliminate uses of is_subtype/2 in specs
- Change module-local void functions to return 'ok' instead of []
- Make sure there are no dialyzer warnings with --Wunmatched_returns
|
|
|
|
In order to avoid an ummatched return warning in erts and make the code
more sane, the return value of the bif was changed from [] to 'ok'.
(Probably more hipe_bifs need such changes but they will have to wait.)
While at it, the code of various functions in hipe_unified_loader was
shortened by using lists:foreach/1 instead of explicit recursion.
|
|
Workaround for crash in code_server.
|
|
|
|
Strange that the effect of this bug is not more fatal.
Kudos to Fabian and Tom for spotting this one.
|
|
We want to be able to write type specifications for BIFs in the same
way as for any other function. Currently, the type for BIFs need
to be described in erl_bif_types.
To avoid extending the compiler and Dialyzer with special directives
for providing specifications for BIFs, we have decided to let the
loader accept a local definition for a function which exists as a
BIF. As an example, here is how a stub for lists:reverse/2 can be
defined:
-export([reverse/2]).
-spec reverse([term()], term()) -> [term()].
reverse(_, _) ->
erlang:nif_error(undef).
Essentially, the loader will discard the local definition of reverse/2.
Other functions in the same module must *not* do local calls to a BIF
stub. If a local call to a BIF is found, the loader will refuse to load
the module. That is, the following call is not allowed:
reverse(List) ->
reverse(List, []).
but the following is:
reverse(List) ->
?MODULE:reverse(List, []).
A few words about the implementation.
It turns out to be too complicated to actually discard the BIF
stubs. Although it would be possibly with some jiggery pokery in
ops.tab, the code would be difficult to maintain and it could slow
down loading of modules that don't define BIFs (which are almost
all modules).
Therefore, the stub functions are kept in the loaded code, but
their names in the func_info instruction are invalidated so that
module_info(functions) can filter them out.
|
|
Reorganize in a systematic way the code that loads the modules needed
by the code_server process. While at it, remove the useless
hipe_unified_loader:load_hipe_modules/0 function.
|
|
|
|
This avoids costly scheduling changes during module loading if native
code is disabled in erts, or not present in the module being loaded.
Signed-off-by: Mikael Pettersson <[email protected]>
|
|
Loading native code is a multi-step operation that needs to be
atomic, but the HiPE loader failed to ensure that.
This caused corruption of runtime system structure in some cases.
Fix by blocking multi_scheduling around main loader entry points.
Signed-off-by: Mikael Pettersson <[email protected]>
|
|
|
