Merge branch 'pan/otp_8579_autoimport_override' into dev

* origin/pan/otp_8579_autoimport_override:
  Update preloaded modules
  Update primary bootstrap
  Remove outcommented code from erl_lint
  Make port_command/3 auto-imported
  Remove (harmless) warnings about min/max in core applications
  Autoimport min/2 and max/2
  Improve coverage of erl_int in testcases
  Change warning to error for nowarn_bif_clash compiler directive
  Add -compile({no_auto_import,[F/A]}) doc to compiler.xml
  Add some testcases to compiler to verify that overriding really happens
  Return nowarn_bif_clash functionality but with warning
  Teach erl_lint to better override BIFs with local functions and imports
  Teach compiler to override autoimport with import
  First prototype for local functions overriding autoimported

OTP-8579  Local functions should override auto-imported

Local and imported functions now override the autoimported
BIFs when the names clash. The pre R14 behaviour was that
autoimported BIFs would override local functions. To avoid
that old programs change behaviour, the following will
generate an error:

Doing a call without explicit module name to a local function
having a name clashing with the name of an autoimported BIF
that was present (and autoimported) before OTP R14A
Explicitly importing a function having a name clashing with
the name of an autoimported BIF that was present (and
autoimported) before OTP R14A Using any form of the old
compiler directive nowarn_bif_clash

If the BIF was added or autoimported in OTP R14A or later,
overriding it with an import or a local function will only
result in a warning,

To resolve clashes, you can either use the explicit module
name erlang to call the BIF, or you can remove the autoimport
of that specific BIF by using the new compiler directive
-compile({no_auto_import,[F/A]})., which makes all calls to
the local or imported function without explicit module name
pass without warnings or errors.

The change makes it possible to add autoimported BIFs without
breaking or silently changing old code in the future. However
some current code ingeniously utilizing the old behaviour or
the nowarn_bif_clash compiler directive, might need changing
to be accepted by the compiler.

9 files changed, 286 insertions, 57 deletions

diff --git a/lib/compiler/doc/src/compile.xml b/lib/compiler/doc/src/compile.xml
index bbd3f1043d..e1f24b602d 100644
--- a/lib/compiler/doc/src/compile.xml
+++ b/lib/compiler/doc/src/compile.xml
@@ -310,6 +310,23 @@
 	    (there will not even be a warning if there is a mismatch).</p>
           </item>
 
+          <tag><c>{no_auto_import,[F/A, ...]}</c></tag>
+          <item>
+	  <p>Makes the function <c>F/A</c> no longer beeing
+          auto-imported from the module <c>erlang</c>, which resolves
+          BIF name clashes. This option has to be used to resolve name
+          clashes with BIFs auto-imported before R14A, if one wants to
+          call the local function with the same name as an
+          auto-imported BIF without module prefix.</p>
+	  <note>
+	  <p>From R14A and forward, the compiler resolves calls
+	  without module prefix to local or imported functions before
+	  trying auto-imported BIFs. If the BIF is to be
+	  called, use the <c>erlang</c> module prefix in the call, not
+	  <c>{ no_auto_import,[F/A, ...]}</c></p>
+	  </note>
+	  </item>
+
         </taglist>
 
         <p>If warnings are turned on (the <c>report_warnings</c> option
@@ -338,31 +355,35 @@
 
           <tag><c>nowarn_bif_clash</c></tag>
           <item>
-            <p>By default, there will be a compilation error if a
-              module contains an exported function with the same name
-              as an auto-imported BIF (such as <c>size/1</c>) AND
-              there is a call to it without a qualifying module name.
-              The reason is that the BIF will be called, not
-	      the function in the same module. The recommended way to
-	      eliminate that warning is to use a call with a module
-	      name - either <c>erlang</c> to call the BIF or
-	      <c>?MODULE</c> to call the function in the same module.
-              The warning can also be turned off using this option,
-	      but that is not recommended.</p>
+            <p>This option is removed, it will generate a fatal error if used.</p>
+
+	    <warning>
+	    <p>Beginning with R14A, the compiler no longer calls the
+	    auto-imported BIF if the name clashes with a local or
+	    explicitly imported function and a call without explicit
+	    module name is issued. Instead the local or imported
+	    function is called. Still accepting <c>nowarn_bif_clash</c> would makes a
+	    module calling functions clashing with autoimported BIFs
+	    compile with both the old and new compilers, but with
+	    completely different semantics, why the option was removed.</p>
 
-	    <p><em>The use of this option is strongly discouraged,
-	    as code that uses it will probably break in a future
-	    major release (R14 or R15).</em></p>
+	    <p>The use of this option has always been strongly discouraged.
+	    From OTP R14A and forward it's an error to use it.</p>
+	    <p>To resolve BIF clashes, use explicit module names or the
+	    <c>{no_auto_import,[F/A]}</c> compiler directive.</p>
+	    </warning>
           </item>
 
           <tag><c>{nowarn_bif_clash, FAs}</c></tag>
           <item>
-            <p>Turns off warnings as <c>nowarn_bif_clash</c> but only
-	      for the mentioned local functions. <c>FAs</c> is a tuple
-	      <c>{Name,Arity}</c> or a list of such tuples.</p>
-	      <p><em>The use of this option is strongly discouraged,
-	      as code that uses it will probably break in a future
-	      major release (R14 or R15).</em></p>
+            <p>This option is removed, it will generate a fatal error if used.</p>
+
+	    <warning>
+	    <p>The use of this option has always been strongly discouraged.
+	    From OTP R14A and forward it's an error to use it.</p>
+	    <p>To resolve BIF clashes, use explicit module names or the
+	    <c>{no_auto_import,[F/A]}</c> compiler directive.</p>
+	    </warning>
           </item>
 
 	  <tag><c>warn_export_all</c></tag>
diff --git a/lib/compiler/src/beam_validator.erl b/lib/compiler/src/beam_validator.erl
index dc5a1068db..f3a2b01e04 100644
--- a/lib/compiler/src/beam_validator.erl
+++ b/lib/compiler/src/beam_validator.erl
@@ -18,6 +18,8 @@
 
 -module(beam_validator).
 
+-compile({no_auto_import,[min/2]}).
+
 -export([file/1, files/1]).
 
 %% Interface for compiler.
diff --git a/lib/compiler/src/cerl_inline.erl b/lib/compiler/src/cerl_inline.erl
index 6d7eca0113..c15103999f 100644
--- a/lib/compiler/src/cerl_inline.erl
+++ b/lib/compiler/src/cerl_inline.erl
@@ -65,7 +65,6 @@
 	       try_evars/1, try_handler/1, tuple_es/1, tuple_arity/1,
 	       type/1, values_es/1, var_name/1]).
 
--import(erlang, [max/2]).
 -import(lists, [foldl/3, foldr/3, mapfoldl/3, reverse/1]).
 
 %%
@@ -201,9 +200,9 @@ start(Reply, Tree, Ctxt, Opts) ->
         false ->
             ok
     end,
-    Size = max(1, proplists:get_value(inline_size, Opts)),
-    Effort = max(1, proplists:get_value(inline_effort, Opts)),
-    Unroll = max(1, proplists:get_value(inline_unroll, Opts)),
+    Size = erlang:max(1, proplists:get_value(inline_size, Opts)),
+    Effort = erlang:max(1, proplists:get_value(inline_effort, Opts)),
+    Unroll = erlang:max(1, proplists:get_value(inline_unroll, Opts)),
     case proplists:get_bool(verbose, Opts) of
 	true ->
 	    io:fwrite("Inlining: inline_size=~w inline_effort=~w\n",
diff --git a/lib/compiler/src/cerl_trees.erl b/lib/compiler/src/cerl_trees.erl
index 7a2057713e..1e3755025f 100644
--- a/lib/compiler/src/cerl_trees.erl
+++ b/lib/compiler/src/cerl_trees.erl
@@ -1,19 +1,19 @@
 %%
 %% %CopyrightBegin%
-%% 
-%% Copyright Ericsson AB 2001-2009. All Rights Reserved.
-%% 
+%%
+%% Copyright Ericsson AB 2001-2010. All Rights Reserved.
+%%
 %% The contents of this file are subject to the Erlang Public License,
 %% Version 1.1, (the "License"); you may not use this file except in
 %% compliance with the License. You should have received a copy of the
 %% Erlang Public License along with this software. If not, it can be
 %% retrieved online at http://www.erlang.org/.
-%% 
+%%
 %% Software distributed under the License is distributed on an "AS IS"
 %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
 %% the License for the specific language governing rights and limitations
 %% under the License.
-%% 
+%%
 %% %CopyrightEnd%
 
 %% @doc Basic functions on Core Erlang abstract syntax trees.
@@ -73,14 +73,12 @@ depth(T) ->
 	[] ->
 	    0;
 	Gs ->
-	    1 + lists:foldl(fun (G, A) -> max(depth_1(G), A) end, 0, Gs)
+	    1 + lists:foldl(fun (G, A) -> erlang:max(depth_1(G), A) end, 0, Gs)
     end.
 
 depth_1(Ts) ->
-    lists:foldl(fun (T, A) -> max(depth(T), A) end, 0, Ts).
+    lists:foldl(fun (T, A) -> erlang:max(depth(T), A) end, 0, Ts).
 
-max(X, Y) when X > Y -> X; 
-max(_, Y) -> Y.
 
 
 %% @spec size(Tree::cerl()) -> integer()
diff --git a/lib/compiler/src/sys_pre_expand.erl b/lib/compiler/src/sys_pre_expand.erl
index f80d03dfac..480954adac 100644
--- a/lib/compiler/src/sys_pre_expand.erl
+++ b/lib/compiler/src/sys_pre_expand.erl
@@ -403,16 +403,21 @@ expr({'fun',Line,Body}, St) ->
 expr({call,Line,{atom,La,N}=Atom,As0}, St0) ->
     {As,St1} = expr_list(As0, St0),
     Ar = length(As),
-    case erl_internal:bif(N, Ar) of
-        true ->
-            {{call,Line,{remote,La,{atom,La,erlang},Atom},As},St1};
-        false ->
-            case imported(N, Ar, St1) of
-                {yes,Mod} ->
-                    {{call,Line,{remote,La,{atom,La,Mod},Atom},As},St1};
-                no ->
-                    {{call,Line,Atom,As},St1}
-            end
+    case defined(N,Ar,St1) of
+	true ->
+	    {{call,Line,Atom,As},St1};
+	_ ->
+	    case imported(N, Ar, St1) of
+		{yes,Mod} ->
+		    {{call,Line,{remote,La,{atom,La,Mod},Atom},As},St1};
+		no ->
+		    case erl_internal:bif(N, Ar) of
+			true ->
+			    {{call,Line,{remote,La,{atom,La,erlang},Atom},As},St1};
+			false -> %% This should have been handled by erl_lint
+			    {{call,Line,Atom,As},St1}
+		    end
+	    end
     end;
 expr({call,Line,{record_field,_,_,_}=M,As0}, St0) ->
     expr({call,Line,expand_package(M, St0),As0}, St0);
@@ -685,3 +690,6 @@ imported(F, A, St) ->
         {ok,Mod} -> {yes,Mod};
         error -> no
     end.
+
+defined(F, A, St) ->
+    ordsets:is_element({F,A}, St#expand.defined).
diff --git a/lib/compiler/test/error_SUITE.erl b/lib/compiler/test/error_SUITE.erl
index 4530313bb0..0874225a62 100644
--- a/lib/compiler/test/error_SUITE.erl
+++ b/lib/compiler/test/error_SUITE.erl
@@ -21,11 +21,133 @@
 -include("test_server.hrl").
 
 -export([all/1,
-	 head_mismatch_line/1,warnings_as_errors/1]).
+	 head_mismatch_line/1,warnings_as_errors/1, bif_clashes/1]).
 
 all(suite) ->
     test_lib:recompile(?MODULE),
-    [head_mismatch_line,warnings_as_errors].
+    [head_mismatch_line,warnings_as_errors,bif_clashes].
+
+
+bif_clashes(Config) when is_list(Config) ->
+    Ts = [{bif_clashes1,
+           <<"
+              -export([t/0]).
+              t() ->
+                 length([a,b,c]).
+
+              length(X) ->
+               erlang:length(X).
+             ">>,
+           [return_warnings],
+	   {error,
+	    [{4, erl_lint,{call_to_redefined_old_bif,{length,1}}}], []} }],
+    ?line [] = run(Config, Ts),
+    Ts1 = [{bif_clashes2,
+           <<"
+              -export([t/0]).
+              -import(x,[length/1]).
+              t() ->
+                 length([a,b,c]).
+             ">>,
+           [return_warnings],
+	    {error,
+	     [{3, erl_lint,{redefine_old_bif_import,{length,1}}}], []} }],
+    ?line [] = run(Config, Ts1),
+    Ts00 = [{bif_clashes3,
+           <<"
+              -export([t/0]).
+              -compile({no_auto_import,[length/1]}).
+              t() ->
+                 length([a,b,c]).
+
+              length(X) ->
+               erlang:length(X).
+             ">>,
+           [return_warnings],
+	   []}],
+    ?line [] = run(Config, Ts00),
+    Ts11 = [{bif_clashes4,
+           <<"
+              -export([t/0]).
+              -compile({no_auto_import,[length/1]}).
+              -import(x,[length/1]).
+              t() ->
+                 length([a,b,c]).
+             ">>,
+           [return_warnings],
+	    []}],
+    ?line [] = run(Config, Ts11),
+    Ts000 = [{bif_clashes5,
+           <<"
+              -export([t/0]).
+              t() ->
+                 binary_part(<<1,2,3,4>>,1,2).
+
+              binary_part(X,Y,Z) ->
+               erlang:binary_part(X,Y,Z).
+             ">>,
+           [return_warnings],
+	   {warning,
+	    [{4, erl_lint,{call_to_redefined_bif,{binary_part,3}}}]} }],
+    ?line [] = run(Config, Ts000),
+    Ts111 = [{bif_clashes6,
+           <<"
+              -export([t/0]).
+              -import(x,[binary_part/3]).
+              t() ->
+                  binary_part(<<1,2,3,4>>,1,2).
+             ">>,
+           [return_warnings],
+	    {warning,
+	     [{3, erl_lint,{redefine_bif_import,{binary_part,3}}}]} }],
+    ?line [] = run(Config, Ts111),
+    Ts2 = [{bif_clashes7,
+           <<"
+              -export([t/0]).
+              -compile({no_auto_import,[length/1]}).
+              -import(x,[length/1]).
+              t() ->
+                 length([a,b,c]).
+              length(X) ->
+                 erlang:length(X).
+             ">>,
+           [],
+          {error,
+           [{7,erl_lint,{define_import,{length,1}}}],
+           []} }],
+    ?line [] = run2(Config, Ts2),
+    Ts3 = [{bif_clashes8,
+           <<"
+              -export([t/1]).
+              -compile({no_auto_import,[length/1]}).
+              t(X) when length(X) > 3 ->
+                 length([a,b,c]).
+              length(X) ->
+                 erlang:length(X).
+             ">>,
+           [],
+          {error,
+           [{4,erl_lint,illegal_guard_expr}],
+           []} }],
+    ?line [] = run2(Config, Ts3),
+    Ts4 = [{bif_clashes9,
+           <<"
+              -export([t/1]).
+              -compile({no_auto_import,[length/1]}).
+              -import(x,[length/1]).
+              t(X) when length(X) > 3 ->
+                 length([a,b,c]).
+             ">>,
+           [],
+          {error,
+           [{5,erl_lint,illegal_guard_expr}],
+           []} }],
+    ?line [] = run2(Config, Ts4),
+
+    ok.
+
+
+
 
 %% Tests that a head mismatch is reported on the correct line (OTP-2125).
 head_mismatch_line(Config) when is_list(Config) ->
@@ -49,7 +171,7 @@ warnings_as_errors(Config) when is_list(Config) ->
                  A = unused,
                  ok.
              ">>,
-           [warnings_as_errors],
+           [export_all,warnings_as_errors],
           {error,
            [],
            [{3,erl_lint,{unused_var,'A'}}]} }],
@@ -70,6 +192,24 @@ run(Config, Tests) ->
         end,
     lists:foldl(F, [], Tests).
 
+run2(Config, Tests) ->
+    F = fun({N,P,Ws,E}, BadL) ->
+                case catch filter(run_test(Config, P, Ws)) of
+                    E ->
+                        BadL;
+                    Bad ->
+                        ?t:format("~nTest ~p failed. Expected~n  ~p~n"
+                                  "but got~n  ~p~n", [N, E, Bad]),
+			fail()
+                end
+        end,
+    lists:foldl(F, [], Tests).
+
+filter({error,Es,_Ws}) ->
+    {error,Es,[]};
+filter(X) ->
+    X.
+
 
 %% Compiles a test module and returns the list of errors and warnings.
 
@@ -78,17 +218,29 @@ run_test(Conf, Test0, Warnings) ->
     ?line DataDir = ?config(priv_dir, Conf),
     ?line Test = ["-module(errors_test). ", Test0],
     ?line File = filename:join(DataDir, Filename),
-    ?line Opts = [binary,export_all,return|Warnings],
+    ?line Opts = [binary,return_errors|Warnings],
     ?line ok = file:write_file(File, Test),
 
     %% Compile once just to print all errors and warnings.
-    ?line compile:file(File, [binary,export_all,report|Warnings]),
+    ?line compile:file(File, [binary,report|Warnings]),
 
     %% Test result of compilation.
     ?line Res = case compile:file(File, Opts) of
-		    {error,[{_File,Es}],Ws} ->
+		    {ok,errors_test,_,[{_File,Ws}]} ->
+			%io:format("compile:file(~s,~p) ->~n~p~n",
+			%	  [File,Opts,Ws]),
+			{warning,Ws};
+		    {ok,errors_test,_,[]} ->
+			%io:format("compile:file(~s,~p) ->~n~p~n",
+			%	  [File,Opts,Ws]),
+			[];
+		    {error,[{XFile,Es}],Ws} = _ZZ when is_list(XFile) ->
+			%io:format("compile:file(~s,~p) ->~n~p~n",
+			%	  [File,Opts,_ZZ]),
 			{error,Es,Ws};
-		    {error,Es,[{_File,Ws}]} ->
+		    {error,Es,[{_File,Ws}]} = _ZZ->
+			%io:format("compile:file(~s,~p) ->~n~p~n",
+			%	  [File,Opts,_ZZ]),
 			{error,Es,Ws}
 		end,
     file:delete(File),
diff --git a/lib/compiler/test/guard_SUITE.erl b/lib/compiler/test/guard_SUITE.erl
index aa1b3b16dc..8f23bd2e5a 100644
--- a/lib/compiler/test/guard_SUITE.erl
+++ b/lib/compiler/test/guard_SUITE.erl
@@ -31,7 +31,7 @@
 	 t_is_boolean/1,is_function_2/1,
 	 tricky/1,rel_ops/1,literal_type_tests/1,
 	 basic_andalso_orelse/1,traverse_dcd/1,
-	 check_qlc_hrl/1,andalso_semi/1,tuple_size/1,binary_part/1]).
+	 check_qlc_hrl/1,andalso_semi/1,t_tuple_size/1,binary_part/1]).
 
 all(suite) ->
     test_lib:recompile(?MODULE),
@@ -43,7 +43,7 @@ all(suite) ->
      build_in_guard,old_guard_tests,gbif,
      t_is_boolean,is_function_2,tricky,rel_ops,literal_type_tests,
      basic_andalso_orelse,traverse_dcd,check_qlc_hrl,andalso_semi,
-     tuple_size,binary_part].
+     t_tuple_size,binary_part].
 
 misc(Config) when is_list(Config) ->
     ?line 42 = case id(42) of
@@ -1330,7 +1330,7 @@ andalso_semi_bar(Bar) when is_list(Bar) andalso length(Bar) =:= 3; Bar =:= 1 ->
    ok.
 
 
-tuple_size(Config) when is_list(Config) ->
+t_tuple_size(Config) when is_list(Config) ->
     ?line 10 = do_tuple_size({1,2,3,4}),
     ?line fc(catch do_tuple_size({1,2,3})),
     ?line fc(catch do_tuple_size(42)),
diff --git a/lib/compiler/test/misc_SUITE.erl b/lib/compiler/test/misc_SUITE.erl
index 126a679724..450a4e279d 100644
--- a/lib/compiler/test/misc_SUITE.erl
+++ b/lib/compiler/test/misc_SUITE.erl
@@ -20,10 +20,23 @@
 
 -export([all/1,init_per_testcase/2,fin_per_testcase/2,
 	 tobias/1,empty_string/1,md5/1,silly_coverage/1,
-	 confused_literals/1,integer_encoding/1]).
+	 confused_literals/1,integer_encoding/1,override_bif/1]).
 	 
 -include("test_server.hrl").
 
+%% For the override_bif testcase.
+%% NB, no other testcases in this testsuite can use these without erlang:prefix!
+-compile({no_auto_import,[abs/1]}).
+-compile({no_auto_import,[binary_part/3]}).
+-compile({no_auto_import,[binary_part/2]}).
+-import(test_lib,[binary_part/2]).
+
+%% This should do no harm (except for fun byte_size/1 which does not, by design, work with import
+-compile({no_auto_import,[byte_size/1]}).
+-import(erlang,[byte_size/1]).
+
+
+
 %% Include an opaque declaration to cover the stripping of
 %% opaque types from attributes in v3_kernel.
 -opaque misc_SUITE_test_cases() :: [atom()].
@@ -42,7 +55,39 @@ fin_per_testcase(Case, Config) when is_atom(Case), is_list(Config) ->
 all(suite) ->
     test_lib:recompile(?MODULE),
     [tobias,empty_string,md5,silly_coverage,confused_literals,
-     integer_encoding].
+     integer_encoding, override_bif].
+
+
+%%
+%% Functions that override new and old bif's
+%%
+abs(_N) ->
+    dummy_abs.
+
+binary_part(_,_,_) ->
+    dummy_bp.
+
+% Make sure that auto-imported BIF's are overridden correctly
+
+override_bif(suite) ->
+    [];
+override_bif(doc) ->
+    ["Test dat local functions and imports override auto-imported BIFs."];
+override_bif(Config) when is_list(Config) ->
+    ?line dummy_abs = abs(1),
+    ?line dummy_bp = binary_part(<<"hello">>,1,1),
+    ?line dummy = binary_part(<<"hello">>,{1,1}),
+    ?line 1 = erlang:abs(1),
+    ?line <<"e">> = erlang:binary_part(<<"hello">>,1,1),
+    ?line <<"e">> = erlang:binary_part(<<"hello">>,{1,1}),
+    F = fun(X) when byte_size(X) =:= 4 ->
+		four;
+	   (X) ->
+		byte_size(X)
+	end,
+    ?line four = F(<<1,2,3,4>>),
+    ?line 5 = F(<<1,2,3,4,5>>),
+    ok.
 
 %% A bug reported by Tobias Lindahl for a development version of R11B.
 
diff --git a/lib/compiler/test/test_lib.erl b/lib/compiler/test/test_lib.erl
index 05236ee010..d8799952a9 100644
--- a/lib/compiler/test/test_lib.erl
+++ b/lib/compiler/test/test_lib.erl
@@ -19,8 +19,8 @@
 -module(test_lib).
 
 -include("test_server.hrl").
-
--export([recompile/1,opt_opts/1,get_data_dir/1,smoke_disasm/1,p_run/2]).
+-compile({no_auto_import,[binary_part/2]}).
+-export([recompile/1,opt_opts/1,get_data_dir/1,smoke_disasm/1,p_run/2,binary_part/2]).
 
 recompile(Mod) when is_atom(Mod) ->
     case whereis(cover_server) of
@@ -104,3 +104,7 @@ p_run_loop(Test, List, N, Refs0, Errors0, Ws0) ->
 	    Refs = Refs0 -- [Ref],
 	    p_run_loop(Test, List, N, Refs, Errors, Ws)
     end.
+
+%% This is for the misc_SUITE:override_bif testcase
+binary_part(_A,_B) ->
+    dummy.