Don't allow null chars in various strings

Various places that now reject null chars inside strings
- Primitive file operations reject it in filenames.
- Primitive environment variable operations reject it in
  names and values.
- os:cmd() reject it in its input.

Also '=' characters are rejected by primitive environment
variable operations in environment variable names.

Documentation has been updated to document null characters
in these types of data as invalid. Currently these operations
accept null chars at the end of strings, but that will change
in the future.

6 files changed, 204 insertions, 29 deletions

diff --git a/lib/kernel/doc/src/file.xml b/lib/kernel/doc/src/file.xml
index b674b3ca93..2ab35b9b05 100644
--- a/lib/kernel/doc/src/file.xml
+++ b/lib/kernel/doc/src/file.xml
@@ -41,7 +41,7 @@
 
     <p>Regarding filename encoding, the Erlang VM can operate in
     two modes. The current mode can be queried using function
-    <seealso marker="#native_name_encoding"><c>native_name_encoding/0</c></seealso>.
+    <seealso marker="#native_name_encoding/0"><c>native_name_encoding/0</c></seealso>.
     It returns <c>latin1</c> or <c>utf8</c>.</p>
 
     <p>In <c>latin1</c> mode, the Erlang VM does not change the
@@ -59,7 +59,7 @@
     terminal supports UTF-8, otherwise <c>latin1</c>. The default can
     be overridden using <c>+fnl</c> (to force <c>latin1</c> mode)
     or <c>+fnu</c> (to force <c>utf8</c> mode) when starting
-    <seealso marker="erts:erl"><c>erts:erl</c></seealso>.</p>
+    <seealso marker="erts:erl"><c>erl</c></seealso>.</p>
 
     <p>On operating systems with transparent naming, files can be
     inconsistently named, for example, some files are encoded in UTF-8 while
@@ -81,6 +81,23 @@
 
     <p>See also section <seealso marker="stdlib:unicode_usage#notes-about-raw-filenames">Notes About Raw Filenames</seealso> in the STDLIB User's Guide.</p>
 
+    <note><p>
+      File operations used to accept filenames containing
+      null characters (integer value zero). This caused
+      the name to be truncated and in some cases arguments
+      to primitive operations to be mixed up. Filenames
+      containing null characters inside the filename
+      are now <em>rejected</em> and will cause primitive
+      file operations fail.
+    </p></note>
+    <warning><p>
+      Currently null characters at the end of the filename
+      will be accepted by primitive file operations. Such
+      filenames are however still documented as invalid. The
+      implementation will also change in the future and
+      reject such filenames.
+    </p></warning>
+
   </description>
 
   <datatypes>
@@ -96,9 +113,21 @@
     </datatype>
     <datatype>
       <name name="filename"/>
+      <desc>
+        <p>
+	  See also the documentation of the
+	  <seealso marker="#type-name_all"><c>name_all()</c></seealso> type.
+	</p>
+      </desc>
     </datatype>
     <datatype>
       <name name="filename_all"/>
+      <desc>
+        <p>
+	  See also the documentation of the
+	  <seealso marker="#type-name_all"><c>name_all()</c></seealso> type.
+	</p>
+      </desc>
     </datatype>
     <datatype>
       <name name="io_device"/>
@@ -112,21 +141,23 @@
       <name name="name"/>
       <desc>
         <p>If VM is in Unicode filename mode, <c>string()</c> and <c>char()</c>
-          are allowed to be &gt; 255.
+          are allowed to be &gt; 255. See also the documentation of the
+	  <seealso marker="#type-name_all"><c>name_all()</c></seealso> type.
         </p>
       </desc>
     </datatype>
     <datatype>
       <name name="name_all"/>
       <desc>
-        <p>If VM is in Unicode filename mode, <c>string()</c> and <c>char()</c>
+        <p>If VM is in Unicode filename mode, characters
           are allowed to be &gt; 255.
           <c><anno>RawFilename</anno></c> is a filename not subject to
           Unicode translation,
           meaning that it can contain characters not conforming to
           the Unicode encoding expected from the file system
           (that is, non-UTF-8 characters although the VM is started
-          in Unicode filename mode).
+          in Unicode filename mode). Null characters (integer value zero)
+	  are <em>not</em> allowed in filenames (not even at the end).
         </p>
       </desc>
     </datatype>
@@ -1825,7 +1856,7 @@ f.txt:  {person, "kalle", 25}.
         <p>The functions in the module <c>file</c> usually treat binaries
           as raw filenames, that is, they are passed "as is" even when the
 	  encoding of the binary does not agree with
-	  <seealso marker="#native_name_encoding"><c>native_name_encoding()</c></seealso>.
+	  <seealso marker="#native_name_encoding/0"><c>native_name_encoding()</c></seealso>.
           However, this function expects binaries to be encoded according to the
           value returned by <c>native_name_encoding()</c>.</p>
         <p>Typical error reasons are:</p>
diff --git a/lib/kernel/doc/src/os.xml b/lib/kernel/doc/src/os.xml
index 0e9add4161..0a08e2c78a 100644
--- a/lib/kernel/doc/src/os.xml
+++ b/lib/kernel/doc/src/os.xml
@@ -36,8 +36,99 @@
       only run on a specific platform. On the other hand, with careful
       use, these functions can be of help in enabling a program to run on
       most platforms.</p>
+
+    <note>
+      <p>
+	File operations used to accept filenames containing
+	null characters (integer value zero). This caused
+	the name to be truncated and in some cases arguments
+	to primitive operations to be mixed up. Filenames
+	containing null characters inside the filename
+	are now <em>rejected</em> and will cause primitive
+	file operations to fail.
+      </p>
+      <p>
+	Also environment variable operations used to accept
+	names and values of environment variables containing
+	null characters (integer value zero). This caused
+	operations to silently produce erroneous results.
+	Environment variable names and values containing
+	null characters inside the name or value are now
+	<em>rejected</em> and will cause environment variable
+	operations to fail.
+      </p>
+    </note>
+    <warning>
+      <p>
+	Currently null characters at the end of filenames,
+	environment variable names and values will be accepted
+	by the primitive operations. Such filenames, environment
+	variable names and values are however still documented as
+	invalid. The implementation will also change in the
+	future and reject such filenames, environment variable
+	names and values.
+      </p>
+    </warning>
   </description>
 
+  <datatypes>
+    <datatype>
+      <name name="env_var_name"/>
+      <desc>
+        <p>A string containing valid characters on the specific
+	OS for environment variable names using
+	<seealso marker="file#native_name_encoding/0"><c>file:native_name_encoding()</c></seealso>
+	encoding. Note that specifically null characters (integer
+	value zero) and <c>$=</c> characters are not allowed.
+	However, note that not all invalid characters necessarily
+	will cause the primitiv operations to fail, but may instead
+	produce invalid results.
+	</p>
+      </desc>
+    </datatype>
+    <datatype>
+      <name name="env_var_value"/>
+      <desc>
+        <p>A string containing valid characters on the specific
+	OS for environment variable values using
+	<seealso marker="file#native_name_encoding/0"><c>file:native_name_encoding()</c></seealso>
+	encoding. Note that specifically null characters (integer
+	value zero) are not allowed. However, note that not all
+	invalid characters necessarily will cause the primitiv
+	operations to fail, but may instead produce invalid results.
+	</p>
+      </desc>
+    </datatype>
+    <datatype>
+      <name name="env_var_name_value"/>
+      <desc>
+        <p>
+	  Assuming that environment variables has been correctly
+	  set, a strings containing valid characters on the specific
+	  OS for environment variable names and values using
+	  <seealso marker="file#native_name_encoding/0"><c>file:native_name_encoding()</c></seealso>
+	  encoding. The first <c>$=</c> characters appearing in
+	  the string separates environment variable name (on the
+	  left) from environment variable value (on the right).
+	</p>
+      </desc>
+    </datatype>
+    <datatype>
+      <name name="command_input"/>
+      <desc>
+        <p>All characters needs to be valid characters on the
+	specific OS using
+	<seealso marker="file#native_name_encoding/0"><c>file:native_name_encoding()</c></seealso>
+	encoding. Note that specifically null characters (integer
+	value zero) are not allowed. However, note that not all
+	invalid characters not necessarily will cause
+	<seealso marker="#cmd/1"><c>os:cmd/1</c></seealso>
+	to fail, but may instead produce invalid results.
+	</p>
+      </desc>
+    </datatype>
+  </datatypes>
+ 
   <funcs>
     <func>
       <name name="cmd" arity="1"/>
@@ -49,6 +140,15 @@
           result as a string. This function is a replacement of
           the previous function <c>unix:cmd/1</c>; they are equivalent on a
           Unix platform.</p>
+	  <warning><p>Previous implementation used to allow all characters
+	  as long as they were integer values greater than or equal to zero.
+	  This sometimes lead to unwanted results since null characters
+	  (integer value zero) often are interpreted as string termination.
+	  Current implementation still accepts null characters at the end
+	  of <c><anno>Command</anno></c> even though the documentation
+	  states that no null characters are allowed. This will however
+	  be changed in the future so that no null characters at all will
+	  be accepted.</p></warning>
         <p><em>Examples:</em></p>
         <code type="none">
 LsOut = os:cmd("ls"), % on unix platform
@@ -152,6 +252,15 @@ DirOut = os:cmd("dir"), % on Win32 platform</code>
 	<p>On Unix platforms, the environment is set using UTF-8 encoding
 	  if Unicode filename translation is in effect. On Windows, the
 	  environment is set using wide character interfaces.</p>
+	  <note>
+	    <p>
+	      <c><anno>VarName</anno></c> is not allowed to contain
+	      an <c>$=</c> character. Previous implementations used
+	      to just let the <c>$=</c> character through which
+	      silently caused erroneous results. Current implementation
+	      will instead throw a <c>badarg</c> exception.
+	    </p>
+	  </note>
       </desc>
     </func>
 
diff --git a/lib/kernel/src/kernel.app.src b/lib/kernel/src/kernel.app.src
index 2a88cc7e26..080b11fc4d 100644
--- a/lib/kernel/src/kernel.app.src
+++ b/lib/kernel/src/kernel.app.src
@@ -120,6 +120,6 @@
   {applications, []},
   {env, [{error_logger, tty}]},
   {mod, {kernel, []}},
-  {runtime_dependencies, ["erts-9.1", "stdlib-3.4", "sasl-3.0"]}
+  {runtime_dependencies, ["erts-10.0", "stdlib-3.5", "sasl-3.0"]}
  ]
 }.
diff --git a/lib/kernel/src/os.erl b/lib/kernel/src/os.erl
index 0250783632..3675e923a3 100644
--- a/lib/kernel/src/os.erl
+++ b/lib/kernel/src/os.erl
@@ -25,6 +25,8 @@
 
 -include("file.hrl").
 
+-export_type([env_var_name/0, env_var_value/0, env_var_name_value/0, command_input/0]).
+
 %%% BIFs
 
 -export([getenv/0, getenv/1, getenv/2, getpid/0,
@@ -32,21 +34,29 @@
          putenv/2, set_signal/2, system_time/0, system_time/1,
 	 timestamp/0, unsetenv/1]).
 
--spec getenv() -> [string()].
+-type env_var_name() :: nonempty_string().
+
+-type env_var_value() :: string().
+
+-type env_var_name_value() :: nonempty_string().
+
+-type command_input() :: atom() | io_lib:chars().
+
+-spec getenv() -> [env_var_name_value()].
 
 getenv() -> erlang:nif_error(undef).
 
 -spec getenv(VarName) -> Value | false when
-      VarName :: string(),
-      Value :: string().
+      VarName :: env_var_name(),
+      Value :: env_var_value().
 
 getenv(_) ->
     erlang:nif_error(undef).
 
 -spec getenv(VarName, DefaultValue) -> Value when
-      VarName :: string(),
-      DefaultValue :: string(),
-      Value :: string().
+      VarName :: env_var_name(),
+      DefaultValue :: env_var_value(),
+      Value :: env_var_value().
 
 getenv(VarName, DefaultValue) ->
     case os:getenv(VarName) of
@@ -75,8 +85,8 @@ perf_counter(Unit) ->
       erlang:convert_time_unit(os:perf_counter(), perf_counter, Unit).
 
 -spec putenv(VarName, Value) -> true when
-      VarName :: string(),
-      Value :: string().
+      VarName :: env_var_name(),
+      Value :: env_var_value().
 
 putenv(_, _) ->
     erlang:nif_error(undef).
@@ -99,7 +109,7 @@ timestamp() ->
     erlang:nif_error(undef).
 
 -spec unsetenv(VarName) -> true when
-      VarName :: string().
+      VarName :: env_var_name().
 
 unsetenv(_) ->
     erlang:nif_error(undef).
@@ -232,10 +242,9 @@ extensions() ->
 
 %% Executes the given command in the default shell for the operating system.
 -spec cmd(Command) -> string() when
-      Command :: atom() | io_lib:chars().
+      Command :: os:command_input().
 cmd(Cmd) ->
-    validate(Cmd),
-    {SpawnCmd, SpawnOpts, SpawnInput, Eot} = mk_cmd(os:type(), Cmd),
+    {SpawnCmd, SpawnOpts, SpawnInput, Eot} = mk_cmd(os:type(), validate(Cmd)),
     Port = open_port({spawn, SpawnCmd}, [binary, stderr_to_stdout,
                                          stream, in, hide | SpawnOpts]),
     MonRef = erlang:monitor(port, Port),
@@ -255,8 +264,6 @@ mk_cmd({win32,Wtype}, Cmd) ->
                   {Cspec,_} -> lists:concat([Cspec," /c",Cmd])
               end,
     {Command, [], [], <<>>};
-mk_cmd(OsType,Cmd) when is_atom(Cmd) ->
-    mk_cmd(OsType, atom_to_list(Cmd));
 mk_cmd(_,Cmd) ->
     %% Have to send command in like this in order to make sh commands like
     %% cd and ulimit available
@@ -279,17 +286,33 @@ mk_cmd(_,Cmd) ->
      <<$\^D>>}.
 
 validate(Atom) when is_atom(Atom) ->
-    ok;
+    validate(atom_to_list(Atom));
 validate(List) when is_list(List) ->
-    validate1(List).
+    case validate1(List) of
+        false ->
+            List;
+        true -> 
+            %% Had zeros at end; remove them...
+            string:trim(List, trailing, [0])
+    end.
 
-validate1([C|Rest]) when is_integer(C) ->
+validate1([0|Rest]) ->
+    validate2(Rest);
+validate1([C|Rest]) when is_integer(C), C > 0 ->
     validate1(Rest);
 validate1([List|Rest]) when is_list(List) ->
-    validate1(List),
-    validate1(Rest);
+    validate1(List) or validate1(Rest);
 validate1([]) ->
-    ok.
+    false.
+
+%% Ensure that the rest is zero only...
+validate2([]) ->
+    true;
+validate2([0|Rest]) ->
+    validate2(Rest);
+validate2([List|Rest]) when is_list(List) ->
+    validate2(List),
+    validate2(Rest).
 
 get_data(Port, MonRef, Eot, Sofar) ->
     receive
diff --git a/lib/kernel/test/file_name_SUITE.erl b/lib/kernel/test/file_name_SUITE.erl
index 899102c908..f23529fec9 100644
--- a/lib/kernel/test/file_name_SUITE.erl
+++ b/lib/kernel/test/file_name_SUITE.erl
@@ -302,7 +302,9 @@ check_normal(Mod) ->
 	      {ok, BC} = Mod:read(FD,1024),
 	      ok = file:close(FD)
 	  end || {regular,Name,Content} <- NormalDir ],
+	{error, badarg} = Mod:rename("fil1\0tmp_fil2","tmp_fil1"),
 	Mod:rename("fil1","tmp_fil1"),
+	{error, badarg} = Mod:read_file("tmp_fil1\0.txt"),
 	{ok, <<"fil1">>} = Mod:read_file("tmp_fil1"),
 	{error,enoent} = Mod:read_file("fil1"),
 	Mod:rename("tmp_fil1","fil1"),
diff --git a/lib/kernel/test/os_SUITE.erl b/lib/kernel/test/os_SUITE.erl
index 53a9e168ef..8056321448 100644
--- a/lib/kernel/test/os_SUITE.erl
+++ b/lib/kernel/test/os_SUITE.erl
@@ -22,7 +22,8 @@
 -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1,
 	 init_per_group/2,end_per_group/2,
 	 init_per_testcase/2,end_per_testcase/2]).
--export([space_in_cwd/1, quoting/1, cmd_unicode/1, space_in_name/1, bad_command/1,
+-export([space_in_cwd/1, quoting/1, cmd_unicode/1, 
+         null_in_command/1, space_in_name/1, bad_command/1,
 	 find_executable/1, unix_comment_in_command/1, deep_list_command/1,
          large_output_command/1, background_command/0, background_command/1,
          message_leak/1, close_stdin/0, close_stdin/1, perf_counter_api/1]).
@@ -34,7 +35,8 @@ suite() ->
      {timetrap,{minutes,1}}].
 
 all() ->
-    [space_in_cwd, quoting, cmd_unicode, space_in_name, bad_command,
+    [space_in_cwd, quoting, cmd_unicode, null_in_command,
+     space_in_name, bad_command,
      find_executable, unix_comment_in_command, deep_list_command,
      large_output_command, background_command, message_leak,
      close_stdin, perf_counter_api].
@@ -125,6 +127,14 @@ cmd_unicode(Config) when is_list(Config) ->
     [] = receive_all(),
     ok.
 
+null_in_command(Config) ->
+    {Ok, Error} = case os:type() of
+                      {win32,_} -> {"dir", "di\0r"};
+                      _ -> {"ls", "l\0s"}
+                  end,
+    true = is_list(try os:cmd(Ok) catch Class0:_ -> Class0 end),
+    error = try os:cmd(Error) catch Class1:_ -> Class1 end,
+    ok.
 
 %% Test that program with a space in its name can be executed.
 space_in_name(Config) when is_list(Config) ->