| Age | Commit message (Collapse) | Author |
|---|
|
Before the beam_validator was added as compiler pass, it was a
standalone module that could analyse existing .beam files and .S
files.
Even though beam_validator has been part of the compiler for many
releases, it still supports the analysis of .beam and .S files.
To reduce the code bloat and to improve coverage of beam_validator,
remove the file/1 and files/1 functions and all associated help
functions. We'll need to update the test suite, since some of the
checked in .S files have errors that beam_validator ignores, but
that will not be accepted when running them throught the compiler
using the 'from_asm' option. In particular, we will need to export
all functions that should be validated (since the beam_clean pass
will remove any function that is not possible to call).
|
|
|
|
|
|
The assert_strict_literal_termorder/1 function is used to validate the
get_map_elements and has_map_fields instructions. In neither case is
it useful to allow an empty lists of fields, so we should no longer
allow an empty list.
The mmap/2 function is cute, but it is used in only one place, so it
is much simpler to write a special-purpose function to extract the
keys from the list of map pairs.
|
|
|
|
|
|
Introduce access functions to hide the low-level details of how
type information is implemented.
|
|
This module is the last that uses -include() instead of
include_lib(). With that change, a quick smoke test can
be done simply like this:
PATH=$ERL_TOP/bin:$PATH erlc -W0 *.erl
Without the change, you would also need to add
-I $ERL_TOP/lib/test_server/include
to the command line.
|
|
* bjorn/compiler/dup-bug-fix/OTP-12453:
Teach case_opt/3 to avoid unnecessary building
sys_core_fold: Optimize let statements more aggressively
Suppress warnings for expressions that are assigned to '_'
trace_bif_SUITE: Ensure that a call to time/0 is not removed
|
|
* bjorn/compiler/map-core-syntax/OTP-12454:
Make the syntax for map pairs readable
|
|
* maint:
Update primary bootstrap
Correct unsafe optimization of '==' and '/='
Conflicts:
bootstrap/lib/compiler/ebin/sys_core_fold.beam
|
|
Since '=:=' is cheaper than '==', the compiler tries to replace
'==' with '=:=' if the result of comparison will be the same.
As an example:
V == {a,b}
can be rewritten to:
V =:= {a,b}
since the literal on the right side contains no numeric values
that '==' would compare differently to '=:='.
With the introduction of maps, we will need to take them into
account. Since the comparison of maps is planned to change in 18.0,
we will be very conservative and only do the optimization if
both keys and values are non-numeric.
|
|
Given this code:
f(S) ->
F0 = F1 = {S,S},
[F0,F1].
case_opt/3 would "optimize" it like this:
f(S) ->
F1 = {S,S},
F0 = {S,S},
[F0,F1].
Similarly, this code:
g({a,_,_}=T) ->
{b,
[_,_] = [T,none],
x}.
would be rewritten to:
g({a,Tmp1,Tmp2}=T) ->
Tmp3 = {a,Tmp1,Tmp2},
{b,
[Tmp3,none],
x}.
where the tuple is rebuilt instead of using the T variable.
Rewrite case_opt/3 to be more careful while optimizing.
|
|
I originally decided that in 'value' context, rewriting a let statement
where the variables were not in the body to a sequence was not worth
it, because the variables would be unused in only one let in a
thousand lets (roughly).
I have reconsidered.
The main reason is that if we do the rewrite, core_lib:is_var_used/2
will be used much more frequently, which will help us to find bugs
in it sooner.
Another reason is that the way letify/2 is currently implemented
with its own calls to core_lib:is_var_used/2 is only safe as long
as all the bindings are independent of each other. We could make
letify/2 smarter, but if we introduce this new optimization there
is no need.
Measuring compilation speed, I have not seen any significant slowdown.
It seems that although core_lib:is_var_used/2 is called much more
frequently, most calls will be fast because is_var_used/2 will quickly
find a use of the variable.
Also add a test case to cover a line opt_guard_try/1 that was
no longer covered.
|
|
In c34ad2d5, the compiler learned to silence some warnings for
expressions that were explicitly assigned to the '_' variable,
as in this example:
_ = list_to_integer(S),
ok
That commit intentionally only made it possible to silence warnings
for BIFs that could cause an exception. Warnings would still be
produced for:
_ = date(),
ok
because date/0 can never fail and thus making the call completely
useless. The reasoning was that such warnings can always be
eliminated by eliminating the offending code.
While that is true, there is the question about rules and their
consistency. It is surprising that '_' can be used to silence
some warnings, but has no effect on other warnings.
Therefore, we will teach the compiler to silence warnings for
the following constructs:
* Calls to safe BIFs such as date/0
* Expressions that will cause an exception such as 'X/0'
* Terms that are built but not used, such as '{x,X}'
|
|
* maint:
Update primary bootstrap
Be more careful about map patterns when evalutating element/2
Do not convert map patterns to map expressions
Conflicts:
bootstrap/lib/compiler/ebin/sys_core_fold.beam
lib/compiler/test/match_SUITE.erl
|
|
We must not convert map patterns to map expressions.
|
|
In code such as:
case {a,Map} of
{a,#{}}=T ->
T
end
we must NOT rewrite a map pattern to a map expression like this:
case Map of
#{} ->
{a,#{}}
end
because the pattern '#{}' will match any map, but the expression
'#{}' will construct an empty map.
|
|
Use the same syntax for map pairs in Core Erlang code as in the
Erlang Code. This Erlang code:
M#{x:=42,y=>1}
will look like this in Core Erlang:
~{'x':=42,'y'=>1|M}~
|
|
* bjorn/compiler/map-fixes:
cerl: Remove a clause in fold_map_pairs/3 that will never be reached
Move grouping of map constructions from v3_core to v3_kernel
core_pp: Correct printing of map literals
Strengthen and modernize compile_SUITE
core_parse: Always fold literal conses
cerl: Make sure that we preserve the invariants for maps
cerl_clauses: Fix indentation
sys_core_fold: Strengthen optimization of letrecs in effect context
Fix handling of binary map keys in comprehensions
core_lib: Teach is_var_used/2 to handle keys in map patterns
warnings_SUITE: Eliminate compiler warning for a shadowed variable
lc_SUITE: Add shadow/1
Modernize lc_SUITE
|
|
A map key in a pattern would be incorrectly pretty-printed.
As an example, the pattern in:
x() ->
#{ #{ a => 3 } := 42 } = X.
would be pretty-printed as:
<~{~<~{~<'a',3>}~,42>}~
instead of:
<~{~<~{::<'a',3>}~,42>}~
When this problem has been corrected, the workaround for it in
cerl:ann_c_map/3 can be removed. The workaround was not harmless,
as it would cause the following map update to incorrectly succeed:
(#{})#{a:=1}
|
|
When we compile from Core Erlang, do it with and without
Core Erlang optimizations to ensure that we are not dependent
on the optimizations always being run.
|
|
I have spent too much time lately waiting for 'cover' to finish,
so now its time to optimize the running time of the tests suite
in coverage mode.
Basically, when 'cover' is running, the test suites would not
run any tests in parallel. The reason is that using too many
parallel processes when running 'cover' would be slower than
running them sequentially. But those measurements were made
several years ago, and many improvements have been made to
improve the parallelism of the run-time system.
Experimenting with the test_lib:p_run/2 function, I found that
increasing the number of parallel processes would speed up the
self_compile tests cases in compilation_SUITE. The difference
between using 3 processes or 4 processes was slight, though,
so it seems that we should not use more than 4 processes when
running 'cover'.
We don't want to change test_lib:parallel/0, because there is
no way to limit the number of test cases that will be run in
parallel by common_test. However, there as test suites (such as
andor_SUITE) that don't invoke the compiler at run-time. We can
run the cases in such test suites in parallel even if 'cover'
is running.
|
|
We used to evaluate the body of a 'letrec' in value context, even
if the 'letrec' was being evaluated in effect context. In most
cases, the context does not matter because the body is usually
just an 'apply' which will never be optimized away.
However, in the case of incorrect code described in the previous
commit, it does matter. We can find such bad code by evaluating
the body in effect context. For example, if we have the following
incorrect code:
letrec
f/1 = fun(A) -> ... <use of Var> ...
in let Var = <<2:301>>
in apply(Arg)
If the letrec is evaluated in effect context, the code will be
reduced to:
letrec
f/1 = fun(A) -> ... <use of Var> ...
in seq Var = <<2:301>> do apply(Arg)
Now Var will be unbound and a later compiler pass will crash to
ensure that the bad Core Erlang code is noticed.
Also add a test case to ensure that the compiler crashes if the
bug fixed in the previous commit re-surfaces.
|
|
is_var_used/2 did not notice that variable keys in map patterns
were used, which could cause sys_core_fold to do unsafe
optimizations.
|
|
|
|
|
|
Remove ?line macros. Run test cases in parallel.
|
|
* egil/fix-maps-compiler-coverage/OTP-12425:
compiler: Rename util function to adhere to name policy
compiler: Remove get_map_elements label check in blocks
compiler: Remove unnecassary guard for get_map_elements
compiler: Remove dead code in beam_flatten
compiler: Increase Maps code coverage
|
|
* bjorn/compiler/coverage:
map_SUITE: Ensure recompilation when running cover
Add beam_utils_SUITE to cover more lines in beam_utils
beam_utils: Remove unreachable clauses in live_opt/4
receive_SUITE: Cover handling of recv_mark & recv_set in beam_utils
beam_validator_SUITE: Mend the compiler_bug/1 test case
beam_clean: Remove handling of forgotten instructions
compile_SUITE: Test the 'dialyzer' option
|
|
* maint:
Update primary bootstrap
core_lib: Handle patterns in map values
|
|
|
|
|
|
While we are it, also remove ?line macros in ref_opt_1/1 and correct
the indentation in do_ref_opt/2.
|
|
The compiler_bug/1 test case succeeded for the wrong reason. The
'asm' option is no longer supported (was ignored) and the compiler
looked for a .erl file.
Make sure that we don't fall for this trick again by making sure
that the error is reported from beam_validator.
|
|
core_lib:is_var_used/2 would not consider a variable used in the
value of a map pattern such as:
case Map of
#{key := <<42:N>>} -> ok
end
Here the variable 'N' would not be considered used.
It was assumed that there was no need to check map patterns at
all, since maps currently don't support variables in keys.
|
|
* oliv3/math_log2/OTP-12411:
Add math:log2/1
|
|
Cover more code in v3_core.
|
|
* maint:
Update primary bootstrap
beam_bool: Correct live calculation for GC BIFs
beam_bool: Correct indentation for try...catch
sys_core_fold: Correct optimization of 'case'
Conflicts:
bootstrap/bin/start.boot
bootstrap/bin/start_clean.boot
bootstrap/lib/compiler/ebin/beam_asm.beam
bootstrap/lib/stdlib/ebin/io_lib_pretty.beam
|
|
* bjorn/compiler/map-in-record-bug/OTP-12402:
sys_core_fold: Correct optimization of 'case'
|
|
|
|
When optimizing boolean expressions, it is not always possible
to find a number of live registers for a GC BIF that both preserves
all source registers that will be tested and at the same time
does not include registers that are not initialized.
As currently implemented, we have incomplete information about
the register calculated from the free variables. Some registers
are marked as "reserved". Reserved registers means that we don't
know anything about them; they may or may not be initialized.
As a conservative correction (suitable for a maintenance release), we
will abort the optimization if we find any reserved registers when
calculating the number of live registers. We will not attempt to
improve the information about the registers in this commit.
By examining the coverage when running the existing compiler test
suite we find that the optimization is aborted 15 times (before
adding any new test cases). To put that in perspective, the
optimization is successfully applied 4927 times, and aborted for
other reasons 547 times.
Reported-by: Ulf Norell
Reported-by: Anthony Ramine
|
|
The optimization of a 'case' statement could lead to incorrect
code that would cause an exception at run-time.
Here is an example to show how the optimization went wrong. Start
with the following code:
f({r,#{key:=Val},X}=S) ->
case S of
{r,_,_} ->
setelement(3, Val, S)
end.
(The record operations have already been translated to the
corresponding tuple operations.) The first step in case_opt/3 is
to substitute S to obtain:
f({r,#{key:=Val},X}=S) ->
case {r,#{key:=Val},X} of
{r,_,_} ->
setelement(3, Val, S)
end.
After that substitution the 'case' can be simplified to:
f({r,#{key:=Val},_}=S) ->
case #{key:=Val} of
NewVar ->
setelement(3, Val, S)
end.
That is the result from case_opt/3. Now eval_case/2 notices
that since there is only one clause left in the 'case', the
'case' can eliminated:
f({r,#{key:=Val},_}=S) ->
NewVar = #{key:=Val},
setelement(3, Val, S).
Since the map construction may have a side effect, it was not
eliminated, but assigned to a variable that is never used.
The problem is that '#{key:=Val}' is fine as a pattern, but in a
construction of a new map, the '=>' operator must be used. So the
map construction will fail, generating an exception.
As a conservative correction for a maintenance release, we will
abort the 'case' optimization if the substitution into the 'case'
expression is anything but data items (tuples, conses, or
literals) or variables.
Reported-by: Dmitry Aleksandrov
|
|
This commit covers 'split_block_label_used' in the beam_bool pass for Maps.
|
|
While we are, clean up the comments and rearrange the code for
clarity. Also add a test to cover the last uncovered line in
beam_dead.erl.
|
|
Amend the test suite to call beam_dead as originally intended (and not
beam_block), and modify the input data so that the exception will
occur within the try ... catch block in function/2.
|
|
The BEAM compiler translates code such as:
is_hex_digit(D) when $0 =< D, D =< $9 -> true;
is_hex_digit(D) when $a =< D, D =< $z -> true;
is_hex_digit(D) when $A =< D, D =< $Z -> true;
is_hex_digit(_) -> false.
to something like this:
L0: test is_ge L1 {x,0} 48
test is_ge L1 57 {x,0}
move true {x,0}
return.
L1: test is_ge L2 {x,0} 97
test is_ge L2 122 {x,0}
move true {x,0}
return
L2: test is_ge L3 {x,0} 65
test is_ge L3 90 {x,0}
move true {x,0}
return
L3: move false {x,0}
return
We can see that tests will be repeated even if they cannot possibly
succeed. For instance, if we pass in {x,0} equal to 32, the first
test that {x,0} is greater than or equal to 48 at L0 will fail.
The control will transfer to L1, where it will be tested whether
{x,0} is greater than 97. That test will fail and control
will pass to L2, where again the test will fail.
The compiler can do better by short-circuiting repeating tests:
L0: test is_ge L3 {x,0} 48
test is_ge L1 57 {x,0}
move true {x,0}
return.
L1: test is_ge L2 {x,0} 97
test is_ge L3 122 {x,0}
move true {x,0}
return
L2: test is_ge L3 {x,0} 65
test is_ge L3 90 {x,0}
move true {x,0}
return
L3: move false {x,0}
return
|
|
* maint:
Fix miscompilation when module contains multiple named funs
Fix locations of shadowing warnings in ms_transform
|
|
A module containing two named funs bearing the same name and arity could be
miscompiled.
Reported-by: Sam Chapin
|
|
* tuncer/compiler/finalize-asm-deprecation/OTP-12100:
compiler: finalize 18.x 'asm' deprecation
|
