aboutsummaryrefslogtreecommitdiffstats
path: root/lib/compiler/src/compile.erl
AgeCommit message (Collapse)Author
2018-11-28Share the code for semantically equivalent blocksBjörn Gustavsson
Share code for semantically equivalent blocks referred to to by `br` and `switch` instructions. A similar optimization is done in `beam_jump`, but doing it here as well is beneficial as it may enable other optimizations. Also, if there are many semantically equivalent clauses, this optimization can substanstially decrease compilation times.
2018-10-23Add a diffable option for the compilerBjörn Gustavsson
Add the `diffable` option to produce a more diff-friendly output in a .S file. In a diffable .S file, label numbers starts from 1 in each function and local function calls use function names instead of labels. scripts/diffable produces diff-friendly files but only for a hard-coded sub set of files in OTP. Having the option directly in the compiler makes is easier to diff the BEAM code for arbitrary source modules.
2018-10-08Merge branch 'maint'John Högberg
* maint: compiler: Forward +source flag to epp and fix bug in +deterministic epp: Allow user to set source name independently of input file name
2018-10-05compiler: Forward +source flag to epp and fix bug in +deterministicJohn Högberg
The source file path as given to `erlc` was included in an implicit file attribute inserted by epp, even when the +source flag was set to something else which was a bit surprising. It was also included when +deterministic was specified, breaking the flag's promise. This commit forwards the +source flag to epp so it inserts the right information, and if +deterministic is given it will be shaved to just the base name of the file, guaranteeing the same result regardless of how the input is reached.
2018-10-04Merge pull request #1973 from ↵John Högberg
jhogberg/john/compiler/improve-named-funs/OTP-15273/ERL-639 Optimize named funs and fun-wrapped macros
2018-10-03Optimize named funs and fun-wrapped macrosJohn Högberg
If a fun is defined locally and only used for calls, it can be replaced with direct calls to the relevant function. This greatly speeds up "named functions" (which rely on make_fun to recreate themselves) and macros that wrap their body in a fun.
2018-10-01Introduce the no_spawn_compiler_process optionJosé Valim
By default, all code is compiled in a separate process which is terminated at the end of compilation. However, some tools, like Dialyzer or compilers for other BEAM languages, may already manage their own worker processes and spawning an extra process may slow the compilation down. In such scenarios, you can pass this option to stop the compiler from spawning an additional process. This generalizes commit 657760e18087b0cdbaecc5e96e46f6f66bc9497a.
2018-09-28Rewrite BSM optimizations in the new SSA-based intermediate formatJohn Högberg
This commit improves the bit-syntax match optimization pass, leveraging the new SSA intermediate format to perform much more aggressive optimizations. Some highlights: * Watch contexts can be reused even after being passed to a function or being used in a try block. * Sub-binaries are no longer eagerly extracted, making it far easier to keep "happy paths" free from binary creation. * Trivial wrapper functions no longer disable context reuse.
2018-09-19Merge pull request #1955 from bjorng/bjorn/compiler/beam_ssa_deadBjörn Gustavsson
Replace beam_dead with beam_ssa_dead
2018-09-17Remove the beam_dead and beam_split passesBjörn Gustavsson
Most of the optimizations in beam_dead have been superseded by the optimizations in beam_ssa_dead. The forward/1 pass of beam_dead has been moved to beam_jump. The beam_split pass splits blocks that contain instructions with non-zero labels. Because there are no optimizations left that optimize instructions within blocks, beam_block never needs to put such instructions into blocks in the first place. beam_split also moved 'move' instructions out block to help beam_dead. That is no longer necessary since beam_dead no longer exists.
2018-09-17Add beam_ssa_dead.erlBjörn Gustavsson
Add beam_ssa_dead to perform the main optimizations done by beam_dead: * Shortcut branches that jump to another block with a branch. If it can be seen that the second branch will always branch to a specific block, replace the target of the first branch. * Combined nested sequences of '=:=' tests and switch instructions operating on the same variable to a single switch. Diffing the compiler output, it seems that beam_ssa_dead finds many more opportunities for optimizations than beam_dead, although it does not find all opportunities that beam_dead does. In total, beam_ssa_dead is such improvement over beam_dead that there is no reason to keep beam_dead as well as beam_ssa_dead. Note that beam_ssa_dead does not attempt to optimize away redundant bs_context_binary instructions, because that instruction will be superseded by new instructions in the near future.
2018-09-12Merge branch 'maint'Rickard Green
* maint: Updated OTP version Update release notes Update version numbers erts: Fix "Prevent inconsistent node lists" fix Fix include-path regression caused by dd0a39c Restore default SIGTERM behaviour for port programs
2018-09-10Fix include-path regression caused by dd0a39cJohn Högberg
Include paths don't actually affect code generation in any way, but it's reasonable for a build tool like rebar3 to recompile when the include paths change. This commit restores the old behavior without the +deterministic flag.
2018-09-03Introduce a put_tuple2 instructionBjörn Gustavsson
Sometimes when building a tuple, there is no way to avoid an extra `move` instruction. Consider this code: make_tuple(A) -> {ok,A}. The corresponding BEAM code looks like this: {test_heap,3,1}. {put_tuple,2,{x,1}}. {put,{atom,ok}}. {put,{x,0}}. {move,{x,1},{x,0}}. return. To avoid overwriting the source register `{x,0}`, a `move` instruction is necessary. The problem doesn't exist when building a list: %% build_list(A) -> [A]. {test_heap,2,1}. {put_list,{x,0},nil,{x,0}}. return. Introduce a new `put_tuple2` instruction that builds a tuple in a single instruction, so that the `move` instruction can be eliminated: %% make_tuple(A) -> {ok,A}. {test_heap,3,1}. {put_tuple2,{x,0},{list,[{atom,ok},{x,0}]}}. return. Note that the BEAM loader already combines `put_tuple` and `put` instructions into an internal instruction similar to `put_tuple2`. Therefore the introduction of the new instruction will not speed up execution of tuple building itself, but it will be less work for the loader to load the new instruction.
2018-08-24Introduce a new SSA-based intermediate formatBjörn Gustavsson
v3_codegen is replaced by three new passes: * beam_kernel_to_ssa which translates the Kernel Erlang format to a new SSA-based intermediate format. * beam_ssa_pre_codegen which prepares the SSA-based format for code generation, including register allocation. Registers are allocated using the linear scan algorithm. * beam_ssa_codegen which generates BEAM assembly code from the SSA-based format. It easier and more effective to optimize the SSA-based format before X and Y registers have been assigned. The current optimization passes constantly have to make sure no "holes" in the X register assignments are created (that is, that no X register becomes undefined that an allocation instruction depends on). This commit also introduces the following optimizations: * Replacing of tuple matching of records with the is_tagged_tuple instruction. (Replacing beam_record.) * Sinking of get_tuple_element instructions to just before the first use of the extracted values. As well as potentially avoiding extracting tuple elements when they are not actually used on all executions paths, this optimization could also reduce the number values that will need to be stored in Y registers. (Similar to beam_reorder, but more effective.) * Live optimizations, removing the definition of a variable that is not subsequently used (provided that the operation has no side effects), as well strength reduction of binary matching by replacing the extraction of value from a binary with a skip instruction. (Used to be done by beam_block, beam_utils, and v3_codegen.) * Removal of redundant bs_restore2 instructions. (Formerly done by beam_bs.) * Type-based optimizations across branches. More effective than the old beam_type pass that only did type-based optimizations in basic blocks. * Optimization of floating point instructions. (Formerly done by beam_type.) * Optimization of receive statements to introduce recv_mark and recv_set instructions. More effective with far fewer restrictions on what instructions are allowed between creating the reference and entering the receive statement. * Common subexpression elimination. (Formerly done by beam_block.)
2018-08-17Don't run unsafe compiler passesBjörn Gustavsson
As a preparation for replacing v3_codegen with a new code generator, remove unsafe optimization passes. Especially the older compiler passes have implicit assumptions about how the code is generated. Remove the optimizations in beam_block (keep the code that creates blocks) because they are unsafe. beam_block also calls beam_utils:live_opt/1, which is unsafe. Remove beam_type because it calls beam_utils:live_opt/1, and also because it recalculates the number of heaps words and number of live registers in allocation instructions, thus potentially hiding bugs in other passes. Remove beam_receive because it is unsafe. Remove beam_record because it is the only remaining user of beam_utils:anno_defs/1. Remove beam_reorder because it makes much more sense to run it as an early SSA-based optimization pass. Remove the now unused functions in beam_utils: anno_def/1 delete_annos/1 is_killed_block/2 live_opt/1 usage/3 Note that the following test cases will fail because of the removed optimizations: compile_SUITE:optimized_guards/1 compile_SUITE:bc_options/1 receive_SUITE:ref_opt/1
2018-08-10Merge branch ↵Erlang/OTP
'john/compiler/fix-deterministic-include-paths/OTP-15204/ERL-679' into maint-21 * john/compiler/fix-deterministic-include-paths/OTP-15204/ERL-679: Omit include path debug info for +deterministic builds # Conflicts: # lib/compiler/test/compile_SUITE.erl
2018-08-09Merge branch ↵Rickard Green
'john/compiler/fix-deterministic-include-paths/OTP-15204/ERL-679' into maint * john/compiler/fix-deterministic-include-paths/OTP-15204/ERL-679: Omit include path debug info for +deterministic builds
2018-08-09Omit include path debug info for +deterministic buildsJohn Högberg
Compiling the same file with different include paths resulted in different files with the `+deterministic` flag even if everything but the paths were identical. This was caused by the absolute path of each include directory being unconditionally included in a debug information chunk. This commit fixes this by only including this information in non-deterministic builds.
2018-06-18Update copyright yearHenrik Nord
2018-05-04Move error formatting to erl_error.erl and delete lib.erlRichard Carlsson
2018-01-26Eliminate get_list/3 internally in the compilerBjörn Gustavsson
Instructions that produce more than one result complicate optimizations. get_list/3 is one of two instructions that produce multiple results (get_map_elements/3 is the other). Introduce the get_hd/2 and get_tl/2 instructions that return the head and tail of a cons cell, respectively, and use it internally in all optimization passes. For efficiency, we still want to use get_list/3 if both head and tail are used, so we will translate matching pairs of get_hd and get_tl back to get_list instructions.
2018-01-11Run beam_block again after other optimizations have been runBjörn Gustavsson
Running beam_block again after the other optimizations have run will give it more opportunities for optimizations. In particular, more allocate_zero/2 instructions can be turned into allocate/2 instructions, and more get_tuple_element/3 instructions can store the retrieved value into the correct register at once. Out of a sample of about 700 modules in OTP, 64 modules were improved by this commit.
2017-12-14Merge pull request #1653 from tonyrog/makedep_side_effectBjörn Gustavsson
Add -MMD option to erlc OTP-14830
2017-12-08Use the new syntax for retrieving stack tracesBjörn Gustavsson
2017-12-06Add -MMD option to erlcTony Rogvall
The compile option makedep_side_effect, erlc -MMD, instructs the compiler to emit dependencies and continue to compile as normal.
2017-11-22Fix broken to_dis after merge from maint to masterBjörn Gustavsson
d8d07a7593d811 that added the to_dis option to the compiler no longer works when merged to master. That is because of 79f28cfd8df1b7 that removed some unused code in erts_debug. Fix this by adding a new function erts_debug:dis_to_file/2 and use it from compile module where we have access to the filename (in beam_listing we only have an opened file).
2017-11-20Merge branch 'maint'Lukas Larsson
Conflicts: lib/compiler/src/beam_listing.erl
2017-11-20compiler: Add +to_dis option that dumps loaded asmLukas Larsson
2017-11-13Merge branch 'maint'Björn Gustavsson
* maint: Recognize 'deterministic' when given in a -compile() attribute Conflicts: lib/compiler/src/beam_asm.erl
2017-10-27Eliminate the v3_life passBjörn Gustavsson
The v3_life pass does not do enough to be worth being its own pass. Essentially it does two things: * Calculates life-time information starting from the annotations that v3_kernel provides. That part can be moved into v3_codegen. * Rewrites the Kernel Erlang records to similar plain tuples (for example, #k_cons{hd=Hd,tl=Tl} is rewritten to {cons,Hd,Tl}). That rewriting is not needed and can be eliminated.
2017-09-14Add compile_info option to compileJosé Valim
This allows compilers built on top of the compile module to attach external compilation metadata to the compile_info chunk. For example, Erlang uses this chunk to store the compiler version. Elixir and LFE may augment this by also adding their own compiler versions, which can be useful when debugging. The deterministic option does not affect the user supplied compile_info. It is therefore the responsibility of external compilers to guarantee any added information does not violate the determinsitic option, if such option is supported. Finally, this code moves the building of the compile_info options to the compile module instead of beam_asm, moving all of the option mangling code to a single place.
2017-07-06Introduce a new core pass called sys_core_aliasJosé Valim
The goal of this pass is to find values that are built from patterns and generate aliases for those values to remove pressure from the GC. For example, this code: example({ok, Val}) -> {ok, Val}. shall become: example({ok, Val} = Tuple) -> Tuple. Currently this pass aliases tuple and cons nodes made of literals, variables and other cons. The tuple/cons may appear anywhere in the pattern and it will be aliased if used later on. Notice a tuple/cons made only of literals is not aliased as it may be part of the literal pool.
2017-06-12Merge branch 'hasse/unicode_atoms/OTP-14285'Hans Bolinder
* hasse/unicode_atoms/OTP-14285: compiler: Handle (bad) Unicode parse transform module names kernel: Improve handling of Unicode filenames stdlib: Handle Unicode atoms in ms_transform stdlib: Improve Unicode handling of the Erlang parser stdlib: Handle unknown compiler options with Unicode stdlib: Handle Unicode macro names stdlib: Correct Unicode handling in escript dialyzer: Improve handling of Unicode parsetools: Improve handling of Unicode atoms stdlib: Handle Unicode atoms when formatting stacktraces stdlib: Add more checks of module names to the linter stdlib: Handle Unicode atoms better in io_lib_format stdlib: Handle Unicode atoms in c.erl
2017-06-12compiler: Handle (bad) Unicode parse transform module namesHans Bolinder
2017-06-07Fix unsafe bit syntax matching optimizationBjörn Gustavsson
As part of sys_core_fold, variables involved in bit syntax matching would be annotated when it would be safe for a later pass to do the delayed sub-binary creation optimization. An implicit assumption regarding the annotation was that the code must not be further optimized. That assumption was broken in 05130e48555891, which introduced a fixpoint iteration (applying the optimizations until there were no more changes). That means that a variable could be annotated as safe for reusing the match context in one iteration, but a later iteration could rewrite the code in a way that would make the optimization unsafe. One way to fix this would be to clear all reuse_for_context annotations before each iteration. But that would be wasteful. Instead I chose to fix the problem by moving out the annotation code to a separate pass (sys_core_bsm) that is run later after all major optimizations of Core Erlang has been done.
2017-05-23Correct handling of module name in compile:forms/1,2Björn Gustavsson
compile:forms/1,2 is documented to return: {ok,ModuleName,BinaryOrCode} However, if one of the options 'from_core', 'from_asm', or 'from_beam' is given, ModuleName will be returned as []. A worse problem is that is that if one those options are combined with the 'native' option, compilation will crash. Correct compile:forms/1,2 to pick up the module name from the forms provided (either Core Erlang, Beam assembly code, or a Beam file). Reported here: https://bugs.erlang.org/browse/ERL-417
2017-05-04Update copyright yearRaimo Niskanen
2017-04-25warnings_as_errors and outdir do not affect code generationJosé Valim
By moving to effects_code_generation/1, there is no need to explicitly remove those options when storing compile information in the DebugInfo chunk.
2017-04-25Store abstract code in the Dbgi chunkJosé Valim
The new Dbgi chunk returns data in the following format: {debug_info_v1, Backend, Data} This allows compilers to store the debug info in different formats. In order to retrieve a particular format, for instance, Erlang Abstract Format, one may invoke: Backend:debug_info(erlang_v1, Module, Data, Opts) Besides introducing the chunk above, this commit also: * Changes beam_lib:chunk(Beam, [:abstract_code]) to read from the new Dbgi chunk while keeping backwards compatibility with old .beams * Adds the {debug_info, {Backend, Data}} option to compile:file/2 and friends that are stored in the Dbgi chunk. This allows the debug info encryption mechanism to work across compilers * Improves dialyzer to work directly on Core Erlang, allowing languages that do not have the Erlang Abstract Format to be dialyzer as long as they emit the new chunk and their backend implementation is available Backwards compatibility is kept across the board except for those calling beam_lib:chunk(Beam, ["Abst"]), as the old chunk is no longer available. Note however the "Abst" chunk has always been optional. Future OTP versions may remove parsing the "Abst" chunk altogether from beam_lib once Erlang 19 and earlier is no longer supported. The current Dialyzer implementation still supports earlier .beam files and such may also be removed in future versions.
2017-04-20compile: Remove the r12 through r15 optionsBjörn Gustavsson
The main purpose of these options is compatibility with old Erlang systems. Since it is no longer possible to communicate with R15B or earlier, we no longer need the r12 through r15 options.
2017-03-24compiler: Add is_tagged_tuple instructionBjörn-Egil Dahlberg
Rewrite the instruction stream on tagged tuple tests. Tagged tuples means a tuple of any arity with an atom as its first element. Typically records, ok-tuples and error-tuples. from: ... {test,is_tuple,Fail,[Src]}. {test,test_arity,Fail,[Src,Sz]}. ... {get_tuple_element,Src,0,Dst}. ... {test,is_eq_exact,Fail,[Dst,Atom]}. ... to: ... {test,is_tagged_tuple,Fail,[Src,Sz,Atom]}. ...
2017-02-12Add extra_chunks option to compileJosé Valim
This allow languages such as Elixir and LFE to attach extra chunks to the .beam file without having to parse the beam file after compilation. This commit also cleans up the interface to beam_asm, allowing chunks to be passed from the compiler without a need to change beam_asm API on every new chunk.
2017-01-30Add new AtU8 beam chunkJosé Valim
The new chunk stores atoms encoded in UTF-8. beam_lib has also been modified to handle the new 'utf8_atoms' attribute while the 'atoms' attribute may be a missing chunk from now on. The binary_to_atom/2 BIF can now encode any utf8 binary with up to 255 characters. The list_to_atom/1 BIF can now accept codepoints higher than 255 with up to 255 characters (thanks to Björn Gustavsson).
2017-01-12compile: Add missing types and specsBjörn Gustavsson
2017-01-12Merge branch 'maint'Hans Bolinder
* maint: dialyzer: Compact 'file' annotations in Core code dialyzer: Try to reduce memory usage dialyzer: Use less memory when translating contracts dialyzer: Use maps instaed of dict dialyzer: Use maps instead of dict for module contracts map dialyzer: Compress a few more ETS tables dialyzer: Optimize memory consumption dialyzer: Reduce memory consumption during 'remote' phase dialyzer: Update code for finding parallelism compiler: Do not spawn process when dialyzing dialyzer: Reduce ETS usage during the typesig phase dialyzer: Optimize graph condensation dialyzer: Do not send full PLTs as messages
2017-01-11compiler: Do not spawn process when dialyzingHans Bolinder
Memory consumption is reduced during the compilation phase by keeping the Core parse tree shared. In particular the file annotation takes a lot of memory when not shared.
2016-12-15compile: Reduce memory consumption during compilationBjörn Gustavsson
The compiler would keep the data structures for two compiler passes in memory. That could increase the maximum amount of memory that the compiler uses, and could also have a negative impact on performance (terms that would not be used again would be copied by a garbage collection). Here is an example that shows how the previous version of the code could get captured: a_compiler_pass(Mod, St) -> case Mod:module(St#compile.code, St#compile.options) of {ok,Code} -> {ok,St#compile{code=Code}}; ... The reference to the code from the previous pass will only be released when St is updated. We can avoid the problem by passing the current version of the code as a function argument: a_compiler_pass(Mod, Code0, St) -> case Mod:module(Code0, St#compile.options) of {ok,Code} -> {ok,Code,St}; ... In practice, this change does not seem to significantly speed up the compiler, but it does not do any harm either. It should help dialyzer in situations when dialyzer compiles several large modules at the same time.
2016-12-07Merge branch 'maint'Dan Gudmundsson
* maint: Update copyright-year Conflicts: lib/dialyzer/src/dialyzer.hrl lib/dialyzer/src/dialyzer_options.erl lib/dialyzer/test/opaque_SUITE_data/src/recrec/dialyzer.hrl lib/dialyzer/test/opaque_SUITE_data/src/recrec/dialyzer_races.erl lib/hipe/icode/hipe_icode.erl lib/hipe/main/hipe.erl lib/hipe/main/hipe.hrl.src lib/hipe/main/hipe_main.erl
2016-12-07Update copyright-yearErlang/OTP