aboutsummaryrefslogtreecommitdiffstats
path: root/lib/compiler/test/bs_match_SUITE.erl
AgeCommit message (Collapse)Author
2017-10-01Merge branch 'maint'Björn Gustavsson
* maint: Fix incorrect internal consistency failure for binary matching code
2017-09-29Fix incorrect internal consistency failure for binary matching codeBjörn Gustavsson
4c31fd0b9665 made the merging of match contexts stricter; in fact, a little bit too strict. Two match contexts with different number of slots would be downgraded to the 'term' type. The correct way is to keep the match context but set the number of slots to the lowest number of slots of the two match contexts. https://bugs.erlang.org/browse/ERL-490
2017-08-31Eliminate unnecessary 'move' instructionsBjörn Gustavsson
The compiler could sometimes emit unnecessary 'move' instructions in the code for binary matching, for example for this function: escape(<<Byte, Rest/bits>>, Pos) when Byte >= 127 -> escape(Rest, Pos + 1); escape(<<Byte, Rest/bits>>, Pos) -> escape(Rest, Pos + Byte); escape(<<_Rest/bits>>, Pos) -> Pos. The generated code would look like this: {function, escape, 2, 2}. {label,1}. {line,[{location,"t.erl",17}]}. {func_info,{atom,t},{atom,escape},2}. {label,2}. {test,bs_start_match2,{f,1},2,[{x,0},0],{x,0}}. {test,bs_get_integer2, {f,4}, 2, [{x,0}, {integer,8}, 1, {field_flags,[{anno,[17,{file,"t.erl"}]},unsigned,big]}], {x,2}}. {'%',{bin_opt,[17,{file,"t.erl"}]}}. {move,{x,0},{x,3}}. %% UNECESSARY! {test,is_ge,{f,3},[{x,2},{integer,127}]}. {line,[{location,"t.erl",18}]}. {gc_bif,'+',{f,0},4,[{x,1},{integer,1}],{x,1}}. {move,{x,3},{x,0}}. %% UNECESSARY! {call_only,2,{f,2}}. {label,3}. {line,[{location,"t.erl",20}]}. {gc_bif,'+',{f,0},4,[{x,1},{x,2}],{x,1}}. {move,{x,3},{x,0}}. %% UNECESSARY! {call_only,2,{f,2}}. {label,4}. {move,{x,1},{x,0}}. return. The redundant 'move' instructions have been marked. To avoid the 'move' instructions, we can extend the existing function is_context_unused/1 in v3_codegen. If v3_codegen can know that the match context will not be used again, it can reuse the register for the match context and avoid the extra 'move' instructions. https://bugs.erlang.org/browse/ERL-444
2017-06-14Update copyright yearHans Nilsson
2017-06-07Fix unsafe bit syntax matching optimizationBjörn Gustavsson
As part of sys_core_fold, variables involved in bit syntax matching would be annotated when it would be safe for a later pass to do the delayed sub-binary creation optimization. An implicit assumption regarding the annotation was that the code must not be further optimized. That assumption was broken in 05130e48555891, which introduced a fixpoint iteration (applying the optimizations until there were no more changes). That means that a variable could be annotated as safe for reusing the match context in one iteration, but a later iteration could rewrite the code in a way that would make the optimization unsafe. One way to fix this would be to clear all reuse_for_context annotations before each iteration. But that would be wasteful. Instead I chose to fix the problem by moving out the annotation code to a separate pass (sys_core_bsm) that is run later after all major optimizations of Core Erlang has been done.
2016-11-18v3_kernel: Generate optimized code for guardsBjörn Gustavsson
The compiler produces poor code for complex guard expressions with andalso/orelse. Here is an example from the filename module: -define(IS_DRIVELETTER(Letter),(((Letter >= $A) andalso (Letter =< $Z)) orelse ((Letter >= $a) andalso (Letter =< $z)))). skip_prefix(Name, false) -> Name; skip_prefix([L, DrvSep|Name], DrvSep) when ?IS_DRIVELETTER(L) -> Name; skip_prefix(Name, _) -> Name. beam_bool fails to simplify the code for the guard, leaving several 'bif' instructions: {function, skip_prefix, 2, 49}. {label,48}. {line,[{location,"filename.erl",187}]}. {func_info,{atom,filename},{atom,skip_prefix},2}. {label,49}. {test,is_ne_exact,{f,52},[{x,1},{atom,false}]}. {test,is_nonempty_list,{f,52},[{x,0}]}. {get_list,{x,0},{x,2},{x,3}}. {test,is_nonempty_list,{f,52},[{x,3}]}. {get_list,{x,3},{x,4},{x,5}}. {bif,'=:=',{f,52},[{x,1},{x,4}],{x,6}}. {test,is_ge,{f,50},[{x,2},{integer,65}]}. {bif,'=<',{f,52},[{x,2},{integer,90}],{x,7}}. {test,is_eq_exact,{f,51},[{x,7},{atom,false}]}. {test,is_ge,{f,50},[{x,2},{integer,97}]}. {bif,'=<',{f,52},[{x,2},{integer,122}],{x,7}}. {jump,{f,51}}. {label,50}. {move,{atom,false},{x,7}}. {label,51}. {bif,'=:=',{f,52},[{x,7},{atom,true}],{x,7}}. {test,is_eq_exact,{f,52},[{x,6},{atom,true}]}. {test,is_eq_exact,{f,52},[{x,7},{atom,true}]}. {move,{x,5},{x,0}}. return. {label,52}. return. We can add optimizations of guard tests to v3_kernel to achive a better result: {function, skip_prefix, 2, 49}. {label,48}. {line,[{location,"filename.erl",187}]}. {func_info,{atom,filename},{atom,skip_prefix},2}. {label,49}. {test,is_ne_exact,{f,51},[{x,1},{atom,false}]}. {test,is_nonempty_list,{f,51},[{x,0}]}. {get_list,{x,0},{x,2},{x,3}}. {test,is_nonempty_list,{f,51},[{x,3}]}. {get_list,{x,3},{x,4},{x,5}}. {test,is_eq_exact,{f,51},[{x,1},{x,4}]}. {test,is_ge,{f,51},[{x,2},{integer,65}]}. {test,is_lt,{f,50},[{integer,90},{x,2}]}. {test,is_ge,{f,51},[{x,2},{integer,97}]}. {test,is_ge,{f,51},[{integer,122},{x,2}]}. {label,50}. {move,{x,5},{x,0}}. return. {label,51}. return. Looking at the STDLIB application, there were 112 lines of BIF calls in guards that beam_bool failed to convert to test instructions. This commit eliminates all those BIF calls. Here is how I counted the instructions: $ PATH=$ERL_TOP/bin:$PATH erlc -I ../include -I ../../kernel/include -S *.erl $ grep "bif,'[=<>]" *.S | grep -v f,0 dets.S: {bif,'=:=',{f,547},[{x,4},{atom,read_write}],{x,4}}. dets.S: {bif,'=:=',{f,547},[{x,5},{atom,saved}],{x,5}}. dets.S: {bif,'=:=',{f,589},[{x,5},{atom,read}],{x,5}}. . . . $ grep "bif,'[=<>]" *.S | grep -v f,0 | wc 112 224 6765 $
2016-10-07Merge branch 'maint'Björn Gustavsson
* maint: beam_bsm: Eliminate unsafe optimization
2016-10-06beam_bsm: Eliminate unsafe optimizationBjörn Gustavsson
The following code causes a compiler failure: first_after(Data, Offset) -> case byte_size(Data) > Offset of false -> {First, Rest} = {ok, ok}, ok; true -> <<_:Offset/binary, Rest/binary>> = Data, %% 'Rest' saved in y(0) before the call. {First, _} = match_first(Data, Rest), %% When beam_bsm sees the code, the following line %% which uses y(0) has been optimized away. {First, Rest} = {First, Rest}, First end. match_first(_, <<First:1/binary, Rest/binary>>) -> {First, Rest}. Here is the error message from beam_validator: t: function first_after/2+15: Internal consistency check failed - please report this bug. Instruction: {call,2,{f,7}} Error: {multiple_match_contexts,[{x,1},0]}: Basically, what happens is that at time of code generation, the variable 'Rest' is needed after the call to match_first/2 and is therefore saved in y(0). When beam_bsm (a late optimization pass) sees the code, the use of y(0) following the call to match_first/2 has been optimized away. beam_bsm therefore assumes that the delayed sub-binary creation is safe. (Actually, it is safe, but beam_validator does not realize it.) The bug was caused by two separate commits: e199e2471a reduced the number of special cases to handle in BEAM optimization passed by breaking apart the tail-recursive call instructions (call_only and call_last) into separate instructions. Unfortunately, the special handling for tail calls was lost, which resulted in worse code (i.e. the delayed sub-binary creation optimization could not be applied). e1aa422290 tried to compensate, but did so in a way that was not always safe. Teaching beam_validator that this kind of code is safe would be expensive. Instead, we will undo the damage caused by the two commits. Re-introduce the special handling of tail-recursive calls in beam_bsm that was lost in the first commit. (Effectively) revert the change in the second commit. ERL-268
2016-09-21bs_match_SUITE: Add a test case for beam_utilsBjörn Gustavsson
During development, a bug in beam_utils caused a compiler failure in xmerl. If the bug reappears, make sure that we catch it when compiling the compiler test suite.
2016-04-27Move bit syntax test cases from compilation_SUITE to bs_match_SUITEBjörn Gustavsson
We used to put code that would crash the compiler into compilation_SUITE_data. That way we would have a failing test case to remind us to fix a bug. Nowadays, we generally fix the bug and write the test case at the same time. Therefore it makes more sense to put the test code directly into a test suite. Move out bin_syntax_1 through bin_syntax_5 test cases. Scrap bin_syntax_6 because it does not longer seems to be relevant. While we are it, rename the fun_shadow/1 test to size_shadow/1. Also make sure that the code produces the correct result.
2016-03-15update copyright-yearHenrik Nord
2016-02-26Merge branch 'bjorn/compiler/modernize-tests'Björn Gustavsson
* bjorn/compiler/modernize-tests: Remove ?line macros Replace use of lists:keysearch/3 with lists:keyfind/3 Eliminate use of doc and suite clauses Replace ?t with test_server Replace use of test_server:format/2 with io:format/2 Eliminate use of test_server:fail/0,1 Eliminate use of ?config() macro Modernize use of timetraps Eliminate useless helper functions
2016-02-25Remove ?line macrosBjörn Gustavsson
2016-02-25Replace use of lists:keysearch/3 with lists:keyfind/3Björn Gustavsson
2016-02-25Eliminate use of test_server:fail/0,1Björn Gustavsson
2016-02-25Modernize use of timetrapsBjörn Gustavsson
Either rely on the default 30 minutes timetrap, or set the timeout using the supported methods in common_test.
2016-02-25Produce warnings for binary patterns that will never matchBjörn Gustavsson
Binary matching can be confusing. For example: 1> <<-1>> = <<-1>>. ** exception error: no match of right hand side value <<"ÿ">> 2> When constructing binaries, the value will be masked to fit in the binary segment. But no such masking happens when matching binaries. One solution that we considered was to do the same masking when matching. We have rejected that solution for several reasons: * Masking in construction is highly controversial and by some people considered a bad design decision. * While masking of unsigned numbers can be understood, masking of signed numbers it not easy to understand. * Then there is the question of backward compatibility. Adding masking to matching would mean that clauses that did not match earlier would start to match. That means that code that has never been tested will be executed. Code that has not been tested will usually not work. Therefore, we have decided to warn for binary patterns that cannot possibly match. While we are it, we will also warn for the following example where size for a binary segment is invalid: bad_size(Bin) -> BadSize = bad_size, <<42:BadSize>> = Bin. That example would crash the HiPE compiler because the BEAM compiler would generate a bs_get_integer2 instruction with an invalid size field. We can avoid that crash if sys_core_fold not only warns for bad binary pattern, but also removes the clauses that will not match. Reported-by: http://bugs.erlang.org/browse/ERL-44 Reported-by: Kostis Sagonas
2016-02-17Eliminate use of test_server.hrl and test_server_line.hrlBjörn Gustavsson
As a first step to removing the test_server application as as its own separate application, change the inclusion of test_server.hrl to an inclusion of ct.hrl and remove the inclusion of test_server_line.hrl.
2016-02-10Merge branch 'maint'Björn Gustavsson
* maint: Eliminate crash because of unsafe delaying of sub-binary creation
2016-02-08Eliminate crash because of unsafe delaying of sub-binary creationBjörn Gustavsson
The following code would fail to compile: decode(<<Code/integer, Bin/binary>>) -> <<C1/integer, B1/binary>> = Bin, case C1 of X when X =:= 1 orelse X =:= 2 -> Bin2 = <<>>; _ -> Bin2 = B1 end, case Code of 1 -> decode(Bin2); _ -> Bin2 end. The error message would be: t: function decode/1+28: Internal consistency check failed - please report this bug. Instruction: return Error: {match_context,{x,0}}: The beam_bsm pass would delay the creation of a sub-binary when it was unsafe to do so. The culprit was the btb_follow_branch/3 function that for performance reasons cached labels that had already been checked. The problem was the safety of a label also depends on the contents of the registers. Therefore, the key for caching needs to be both the label and the register contents. Reported-by: José Valim
2015-11-20Merge branch 'maint'Björn Gustavsson
* maint: Fix missing filename and line number in warning Conflicts: lib/compiler/test/bs_match_SUITE.erl
2015-11-20Fix missing filename and line number in warningBjörn Gustavsson
When the 'bin_opt_info' is given, warnings without filenames and line numbers could sometimes be produced: no_file: Warning: INFO: matching non-variables after a previous clause matching a variable will prevent delayed sub binary optimization The reason for the missing information is that #c_alias{} records lack location information. There are several ways to fix the problem. The easiest seems to be to get the location information from the code). Noticed-by: José Valim
2015-11-10sys_pre_expand: Cover coerce_to_float/2Björn Gustavsson
2015-09-28beam_type: Improve optimization by keeping track of integersBjörn Gustavsson
The ASN.1 compiler often generates code similar to: f(<<0:1,...>>) -> ...; f(<<1:1,...>>) -> .... Internally that will be rewritten to (conceptually): f(<<B:1,Tail/binary>>) -> case B of 0 -> case Tail of ... end; 1 -> case Tail of ... end; _ -> error(case_clause) end. Since B comes from a bit field of one bit, we know that the only possible values are 0 and 1. Therefore the error clause can be eliminated like this: f(<<B:1,Tail/binary>>) -> case B of 0 -> case Tail of ... end; _ -> case Tail of ... end end. Similarly, we can also a deduce the range for an integer from a 'band' operation with a literal integer. While we are at it, also add a test case to improve the coverage.
2015-06-18Change license text to APLv2Bruce Yinhe
2015-04-22beam_block: Optimize matching of binary literalsBjörn Gustavsson
When matching a binary literal as in: <<"abc">> = Bin the compiler will produce a sequence of three instructions (some details in the instructions removed for simplicity): bs_start_match2 Fail BinReg CtxtReg bs_match_string Fail CtxtReg "abc" bs_test_tail2 Fail CtxtReg 0 The sequence can be replaced with: is_eq_exact Fail BinReg "abc"
2015-04-22test suite: Always place .core files in data directoriesBjörn Gustavsson
For tidiness, always place .core files in data directories.
2015-01-26Speed up running of compiler test suites in coverage modeBjörn Gustavsson
I have spent too much time lately waiting for 'cover' to finish, so now its time to optimize the running time of the tests suite in coverage mode. Basically, when 'cover' is running, the test suites would not run any tests in parallel. The reason is that using too many parallel processes when running 'cover' would be slower than running them sequentially. But those measurements were made several years ago, and many improvements have been made to improve the parallelism of the run-time system. Experimenting with the test_lib:p_run/2 function, I found that increasing the number of parallel processes would speed up the self_compile tests cases in compilation_SUITE. The difference between using 3 processes or 4 processes was slight, though, so it seems that we should not use more than 4 processes when running 'cover'. We don't want to change test_lib:parallel/0, because there is no way to limit the number of test cases that will be run in parallel by common_test. However, there as test suites (such as andor_SUITE) that don't invoke the compiler at run-time. We can run the cases in such test suites in parallel even if 'cover' is running.
2015-01-21Merge branch 'maint'Björn Gustavsson
* maint: Update primary bootstrap core_lib: Handle patterns in map values
2015-01-19core_lib: Handle patterns in map valuesBjörn Gustavsson
core_lib:is_var_used/2 would not consider a variable used in the value of a map pattern such as: case Map of #{key := <<42:N>>} -> ok end Here the variable 'N' would not be considered used. It was assumed that there was no need to check map patterns at all, since maps currently don't support variables in keys.
2015-01-09Extend count_bits_matched/3 to handle the UTF instructionsBjörn Gustavsson
While we are, clean up the comments and rearrange the code for clarity. Also add a test to cover the last uncovered line in beam_dead.erl.
2014-02-05beam_bsm: Eliminate emulator crash when a binary is calledBjörn Gustavsson
We must not do the delayed binary creation optimization if the code attempts to call the matched out binary. Calling a matchstate will crash the run-time system. Reported-by: Loïc Hoguin
2013-01-25Update copyright yearsBjörn-Egil Dahlberg
2013-01-18compiler: Eliminate internal consistency failure in binary matchingBjörn Gustavsson
The following code: check(<<"string">>, a1) -> one; check(_, a2) -> two; check(undefined, a3) -> three. produces an internal consistency failure: check: function check/2+17: Internal consistency check failed - please report this bug. Instruction: {test,is_eq_exact,{f,7},[{x,0},{atom,undefined}]} Error: {match_context,{x,0}}: Actually, in the current implementation of the run-time system, comparing a match context to an atom is safe, so I briefly considered updating the beam_validator to let this code pass through. I abandoned that approach because not all terms would be safe to compare to a match context, and the implementation might change in the future. Therefore, fix this problem by not allowing any matching of non-variables (in the argument position for binary being matched) following binary matching. That solution is simple and safe, and since this kind of code seems to be rare in practice, there is no need to pursue any more compilicated solution. Reported-by: Viktor Sovietov
2012-11-26beam_bsm: Improve handling of bs_start_match2 instructionsBjörn Gustavsson
The handling of bs_start_match2 was both too conservative and too careless. It was too conservative in that would not do the optimization if the were copies of the match state in other registers. It was careless in that it did not consider the failure branch. Reorganize the code and fix both these issues. Add a test case to test that the failure branch is considered.
2012-11-23Merge branch 'maint'Björn-Egil Dahlberg
2012-11-23Update copyright yearsBjörn-Egil Dahlberg
2012-11-12Merge branch 'maint'Björn Gustavsson
* maint: Fix compiler crash for binary matching and a complicated guard
2012-11-06Fix compiler crash for binary matching and a complicated guardBjörn Gustavsson
The compiler would crash when attempting to compile a function head that did binary matching and had a complex expression using 'andalso' and 'not'. Noticed-by: José Valim
2012-10-23compiler: Run testcases in parallelBjörn Gustavsson
Run testcases in parallel will make the test suite run slightly faster. Another reason for this change is that we want more testing of parallel testcase support in common_test.
2012-10-09v3_kernel: Fix match code for matched out segment size in multiple clausesBjörn Gustavsson
When matched variable is used as a size field in multiple clauses, as in: foo(<<L:8,A:L>>) -> A; foo(<<L:8,A:L,B:8>>) -> {A,B}. the match tree would branch out before the segment that used the matched-out variable (in this example, the tree would branch out before the matching of A:L). That happens because the pattern matching compilator did not take variable substitutions into account when grouping clauses that match the same value. That is, the generated code would work similarly to this code: foo(<<L:8,T/binary>>) -> case T of <<A:L>> -> A; _ -> case T of <<A:L,B:8>> -> %% A matched out again! {A,B} end end. We would like the matching to work more like: foo(<<L,A:L,T/binary>>) -> case T of <<>> -> A; <<B:8>> -> {A,B} end. Fix the problem by taking the substitutions into account when grouping clauses that match out the same value.
2012-01-04bs_match_SUITE: Add a test case to cover bsm_ensure_no_partition_2/5Björn Gustavsson
Also correct the comment in bsm_ensure_no_partition_2/5, and while at it correct typos in the comment for bsm_nonempty/2.
2011-08-16emulator: Add a fourth element in exception stacktracesBjörn Gustavsson
This commit is a preparation for introducing location information (filename/line number) in stacktraces in exceptions. Currently a stack trace looks like: [{Mod1,Function1,Arity1}, . . . {ModN,FunctionN,ArityN}] Add a forth element to each tuple that can be used indication the filename and line number of the source file: [{Mod1,Function1,Arity1,Location1}, . . . {ModN,FunctionN,ArityN,LocationN}] In this commit, the fourth element will just be an empty list, and we will change all code that look at or manipulate stacktraces.
2011-04-12bs_match_SUITE: Improve coverage of beam_bsmBjörn Gustavsson
2011-04-12compiler tests: Reinstate ?MODULE macro in calls to test_lib:recompile/1Björn Gustavsson
In 3d0f4a3085f11389e5b22d10f96f0cbf08c9337f (an update to conform with common_test), in all test_lib:recompile(?MODULE) calls, ?MODULE was changed to the actual name of the module. That would cause test_lib:recompile/1 to compile the module with the incorrect compiler options in cloned modules such as record_no_opt_SUITE, causing worse coverage.
2011-03-23v3_core: Fix variable incorrectly unbound after binary matchBjörn Gustavsson
In the following code: m(<<Sz:8,_:Sz/binary>>) -> Sz = wrong. the Sz variable is supposed to be bound in the function header and the matching "Sz = wrong" should cause a badarg exception. But what happens is that the Sz variables seems to be unbound and the matching succeds and the m/1 function returns 'wrong'. If the Sz variable is used directly (not matched), it will have the expected value. Thus the following code: m(<<Sz:8,_:Sz/binary>>) -> Sz. will correctly return the value of Sz that was matched out from the binary. Reported-by: Bernard Duggan
2011-03-11Update copyright yearsBjörn-Egil Dahlberg
2011-02-17Rename Suite Callback to Common Test HookLukas Larsson
2011-02-17Fix formatting for compilerLukas Larsson
2011-02-17Add init_per_suite and end_per_suiteLukas Larsson