ts_run: Don't run make:all/1 without a good reason

ts:run() runs make:all/1 to facilitate interactive development
of a test suite - when you have changed your test suite, just
run ts:run() and it will rebuild the test suite for you.

Some test suites, such as for the public_key application,
are meant to be built once and for all by "make release_tests".
Re-building it later with make:all/1 will fail because
it references an internal .hrl file in the source directory
for public_key.

Running make:all/1 for the public_key test suite would not
cause any harm in most circumstances, because make:all/1 would
consider the *.beam files to be up-to-date and therefore not
actually try to re-build anything.

In 2c6d9b57ce4557a431bcf02565c3634a0ed7ca61, however, the
public_key test suite started to include "test_server.hrl"
using -include_lib() instead of -include(). That means that
"test_server.hrl" will be included from the Erlang/OTP
system being tested, and if that system has been built
*after* the test suite was built, the *.beam files in the
test suite will be out-of-date and make:all/1 will try to
re-build them (and fail).

To avoid this problem, modify ts:run() as follows:

* If the test suite has an Emakefile (such as the emulator test
  suite), it is clearly meant to be (re-)built by make:all/1,
  so we run make:all/1.

* If there is no Emakefile and no .beam files (such as in the
  erl_interface test suite), it also seems clear that the
  test suite is meant to be built by make:all/1. To make sure
  that the test suite will also be *re-built*, create an
  Emakefile that will re-build all *.erl files in the
  directory for the test suite.

* If there is no Emakefile and there are *.beam files (such
  as for the public_key test suite), we assume that the test
  suite are not meant to re-built. However, there may be
  generated *_SUITE_make.erl files (if there are *_SUITE_data
  directories containg Makefile.src files, as in the xmerl
  application). If there are any *_SUITE_make.erl files,
  generate an Emakefile for compiling only those files
  and run make:all/1. Otherwise, don't run make:all/1.

1 files changed, 49 insertions, 1 deletions

diff --git a/lib/test_server/src/ts_run.erl b/lib/test_server/src/ts_run.erl
index 3461e1383c..0a5e43fc50 100644
--- a/lib/test_server/src/ts_run.erl
+++ b/lib/test_server/src/ts_run.erl
@@ -392,7 +392,7 @@ make_test_suite(Vars, _Spec, State) ->
 
     {ok, Cwd} = file:get_cwd(),
     ok = file:set_cwd(TestDir),
-    Result = (catch make:all(Erl_flags)),
+    Result = (catch make_all(Erl_flags)),
     ok = file:set_cwd(Cwd),
     case Result of
 	up_to_date ->
@@ -743,4 +743,52 @@ split_one(Path) ->
 split_path(Path) ->
     string:tokens(Path,";").
 
+%%
+%% Run make:all/1 if the test suite seems to be designed
+%% to be built/re-built by ts.
+%%
+make_all(Flags) ->
+    case filelib:is_regular("Emakefile") of
+	false ->
+	    make_all_no_emakefile(Flags);
+	true ->
+	    make:all(Flags)
+    end.
 
+make_all_no_emakefile(Flags) ->
+    case filelib:wildcard("*.beam") of
+	[] ->
+	    %% Since there are no *.beam files, we will assume
+	    %% that this test suite was designed to be built and
+	    %% re-built by ts. Create an Emakefile so that
+	    %% make:all/1 will be run the next time too
+	    %% (in case a test suite is being interactively
+	    %% developed).
+	    create_emakefile(Flags, "*.erl");
+	[_|_] ->
+	    %% There is no Emakefile and there already are
+	    %% some *.beam files here. Assume that this test
+	    %% suite was not designed to be re-built by ts.
+	    %% Only create a Emakefile that will compile
+	    %% generated *_SUITE_make files (if any).
+	    create_emakefile(Flags, "*_SUITE_make.erl")
+    end.
+
+create_emakefile(Flags, Wc) ->
+    case filelib:wildcard(Wc) of
+	[] ->
+	    %% There are no files to be built (i.e. not even any
+	    %% generated *_SUITE_make.erl files). We must handle
+	    %% this case specially, because make:all/1 will crash
+	    %% on Emakefile with an empty list of modules.
+	    io:put_chars("No Emakefile found - not running make:all/1\n"),
+	    up_to_date;
+	[_|_]=Ms0 ->
+	    io:format("Creating an Emakefile for compiling files matching ~s\n",
+		      [Wc]),
+	    Ms = [list_to_atom(filename:rootname(M, ".erl")) || M <- Ms0],
+	    Make0 = {Ms,Flags},
+	    Make = io_lib:format("~p. \n", [Make0]),
+	    ok = file:write_file("Emakefile", Make),
+	    make:all(Flags)
+    end.