Merge branch 'siri/logger-fix'

* siri/logger-fix:
  Change logger callback removing_handler/1 to removing_handler/2
  Don't crash logger_server due to unexpected message
  Add logger:update_process_metadata/1
  Improve documentation of logger:set_*_config functions
  Change return type from logger_formatter:format/2
  Improve test of logger_disk_log_h and logger_std_h
  Update logger documentation
  Set single_line=true by default in logger_formatter

4 files changed, 208 insertions, 140 deletions

diff --git a/lib/kernel/doc/src/kernel_app.xml b/lib/kernel/doc/src/kernel_app.xml
index 554d675383..f96d946a5d 100644
--- a/lib/kernel/doc/src/kernel_app.xml
+++ b/lib/kernel/doc/src/kernel_app.xml
@@ -226,7 +226,7 @@
 	  <p>This configuration parameter is used both for the global
 	    logger level, and for the standard handler started by
 	    the Kernel application (see <c>logger_dest</c> variable above).</p>
-	  <p>The default value is <c>info</c></p>
+	  <p>The default value is <c>info</c>.</p>
       </item>
       <tag><marker id="disk_log_vars"/>
 	<c>logger_disk_log_type = halt | wrap</c></tag>
@@ -251,14 +251,14 @@ logger_disk_log_maxbytes = 1048576</code>
       <item>
 	<p>If this parameter is set to true, then the logger handler
 	  started by kernel will not log any progress-, crash-, or
-	  supervisor reports. If the SASL application is starated,
+	  supervisor reports. If the SASL application is started,
 	  these log events will be sent to a second handler instance
-	  named sasl_h, according to values of the SASL environment
-	  variables <c>sasl_error_logger</c>
+	  named <c>sasl_h</c>, according to values of the SASL
+	  environment variables <c>sasl_error_logger</c>
 	  and <c>sasl_errlog_type</c>, see
 	  <seealso marker="sasl:sasl_app#configuration">SASL(6)
 	</seealso></p>
-	<p>The default value is <c>false</c></p>
+	<p>The default value is <c>false</c>.</p>
 	<p>See chapter <seealso marker="logger_chapter#compatibility">Backwards
 	  compatibility with error_logger</seealso> for more
 	  information about handling of the so called SASL reports.</p>
@@ -271,7 +271,7 @@ logger_disk_log_maxbytes = 1048576</code>
 	  reports from <c>supervisor</c>
 	  and <c>application_controller</c> shall be logged or
 	  not.</p>
-	<p>If <c>logger_sasl_compatible = false</c>,
+	<p>If <c>logger_sasl_compatible = true</c>,
 	  then <c>logger_log_progress</c> is ignored.</p>
       </item>
       <tag><marker id="logger_format_depth"/>
@@ -280,14 +280,6 @@ logger_disk_log_maxbytes = 1048576</code>
 	<p>Can be used to limit the size of the
 	formatted output from the logger handlers.</p>
 
-	<note><p>This configuration parameter was introduced in OTP 18.1
-	and is experimental. Based on user feedback, it
-	can be changed or improved in future releases, for example,
-	to gain better control over how to limit the size of the
-	formatted output. We have no plans to remove this
-	new feature entirely, unless it turns out to be
-	useless.</p></note>
-
         <p><c>Depth</c> is a positive integer representing the maximum
         depth to which terms are printed by the logger
         handlers included in OTP. This
@@ -312,11 +304,11 @@ logger_disk_log_maxbytes = 1048576</code>
       </item>
       <tag><c>logger_max_size = integer() | unlimited</c></tag>
       <item>
-	<p>This parameter specifies the maximum size (bytes) each
-	  log event can have when printed by the standard logger
-	  handler. If the resulting string after formatting an event
-	  is bigger than this, it will be truncated before printed
-	  to the handler's destination.</p>
+	<p>This parameter specifies a hard maximum size limit (number
+	  of characters) each log event can have when printed by the
+	  default logger formatter. If the resulting string after
+	  formatting an event is bigger than this, it will be
+	  truncated before printed to the handler's destination.</p>
       </item>
       <tag><c>logger_utc = boolean()</c></tag>
       <item>
diff --git a/lib/kernel/doc/src/logger.xml b/lib/kernel/doc/src/logger.xml
index 66e6e5c689..2f7feb5eef 100644
--- a/lib/kernel/doc/src/logger.xml
+++ b/lib/kernel/doc/src/logger.xml
@@ -131,8 +131,8 @@
 
     <code>
 #{mfa=>{?MODULE,?FUNCTION_NAME,?FUNCTION_ARITY},
-        file=>?FILE,
-        line=>?LINE}
+  file=>?FILE,
+  line=>?LINE}
     </code>
 
     <p>The call is wrapped in a case statement and will be evaluated
@@ -297,7 +297,7 @@
 	  <tag>print</tag>
           <item>Pretty print all the current logger configuration to
             standard out. Example:
-            <code><![CDATA[1> logger:i().
+            <code><![CDATA[1> logger:i(print).
 Current logger configuration:
   Level: info
   FilterDefault: log
@@ -404,21 +404,55 @@ Current logger configuration:
 
     <func>
       <name name="set_logger_config" arity="1"/>
+      <fsummary>Set configuration data for the logger.</fsummary>
+      <desc>
+        <p>Set configuration data for the logger. This overwrites the
+          current logger configuration.</p>
+	<p>To modify the existing configuration,
+	  use <seealso marker="#set_logger_config-2"><c>set_logger_config/2</c>
+	  </seealso>, or read the current configuration
+	  with <seealso marker="#get_logger_config-0"><c>get_logger_config/0</c>
+	  </seealso>, then merge in your added or updated
+	  associations before writing it back.</p>
+      </desc>
+    </func>
+
+    <func>
       <name name="set_logger_config" arity="2"/>
       <fsummary>Add or update configuration data for the logger.</fsummary>
       <desc>
-        <p>Add or update configuration data for the logger.</p>
+        <p>Add or update configuration data for the logger. If the
+          given <c><anno>Key</anno></c> already exists, its associated
+          value will be set to <c><anno>Value</anno></c>. If it
+          doesn't exist, it will be added.</p>
       </desc>
     </func>
 
     <func>
       <name name="set_handler_config" arity="2"/>
+      <fsummary>Set configuration data for the specified handler.</fsummary>
+      <desc>
+        <p>Set configuration data for the specified handler. This
+          overwrites the current handler configuration.</p>
+	<p>To modify the existing configuration,
+	  use <seealso marker="#set_handler_config-3"><c>set_handler_config/3</c>
+	  </seealso>, or read the current configuration
+	  with <seealso marker="#get_handler_config-1"><c>get_handler_config/1</c>
+	  </seealso>, then merge in your added or updated
+	  associations before writing it back.</p>
+      </desc>
+    </func>
+
+    <func>
       <name name="set_handler_config" arity="3"/>
       <fsummary>Add or update configuration data for the specified
         handler.</fsummary>
       <desc>
         <p>Add or update configuration data for the specified
-          handler.</p>
+          handler. If the
+          given <c><anno>Key</anno></c> already exists, its associated
+          value will be set to <c><anno>Value</anno></c>. If it
+          doesn't exist, it will be added.</p>
       </desc>
     </func>
 
@@ -437,10 +471,9 @@ Current logger configuration:
       <name name="set_process_metadata" arity="1"/>
       <fsummary>Set metadata to use when logging from current process.</fsummary>
       <desc>
-        <p>Set metadata which <c>logger</c> automatically inserts it
-          in all log events produced on the current
-          process. Subsequent calls will overwrite previous data set
-          by this function.</p>
+        <p>Set metadata which <c>logger</c> automatically inserts in
+          all log events produced on the current process. Subsequent
+          calls will overwrite previous data set by this function.</p>
 	<p>When logging, location data produced by the log macros,
 	  and/or metadata given as argument to the log call (API
 	  function or macro), will be merged with the process
@@ -452,6 +485,21 @@ Current logger configuration:
     </func>
 
     <func>
+      <name name="update_process_metadata" arity="1"/>
+      <fsummary>Update metadata to use when logging from current process.</fsummary>
+      <desc>
+	<p>Update metadata to use when logging from current process</p>
+	<p>This function behaves as if it was implemented as follows:</p>
+	<code type="erl">
+logger:set_process_metadata(maps:merge(logger:get_process_metadata(),Meta))
+	</code>
+	<p>If no process metadata exists, the function behaves as
+	  <seealso marker="#set_process_metadata-1"><c>set_process_metadata/1</c>
+	  </seealso>.</p>
+      </desc>
+    </func>
+
+    <func>
       <name name="get_process_metadata" arity="0"/>
       <fsummary>Retrieve data set with set_process_metadata/1.</fsummary>
       <desc>
diff --git a/lib/kernel/doc/src/logger_chapter.xml b/lib/kernel/doc/src/logger_chapter.xml
index 88dcfbe8d9..519df2ba48 100644
--- a/lib/kernel/doc/src/logger_chapter.xml
+++ b/lib/kernel/doc/src/logger_chapter.xml
@@ -157,7 +157,7 @@
 	  <p>A formatter is defined as a module exporting the following
 	    function:</p>
 
-	  <code>format(Log,Extra) -> string()</code>
+	  <code>format(Log,Extra) -> unicode:chardata()</code>
 
 	  <p>The formatter plugin is called by each handler, and the
 	    returned string can be printed to the handler's destination
@@ -322,19 +322,6 @@
 	    return <c>ignore</c>.</p>
 	  <p>Default is <c>log</c>.</p>
 	</item>
-        <tag><c>depth = pos_integer() | unlimited</c></tag>
-	<item>
-	  <p>Specifies if the depth of terms in the log events shall
-	    be limited by using control characters <c>~P</c>
-	    and <c>~W</c> instead of <c>~p</c> and <c>~w</c>,
-	    respectively. See
-	    <seealso marker="stdlib:io#format-1"><c>io:format</c></seealso>.</p>
-	</item>
-        <tag><c>max_size = pos_integer() | unlimited</c></tag>
-	<item>
-	  <p>Specifies if the size of a log event shall be limited by
-	    truncating the formatted string.</p>
-	</item>
 	<tag><c>formatter = {Module::module(),Extra::term()}</c></tag>
 	<item>
 	  <p>See <seealso marker="#Formatter">Formatter</seealso> for more
@@ -347,10 +334,9 @@
 
       <p>Note that <c>level</c> and <c>filters</c> are obeyed by
 	Logger itself before forwarding the log events to each
-	handler, while <c>depth</c>, <c>max_size</c>
-	and <c>formatter</c> are left to the handler
-	implementation. All Logger's built-in handlers do apply these
-	configuration parameters before printing.</p>
+	handler, while <c>formatter</c> is left to the handler
+	implementation. All Logger's built-in handlers will call the
+	given formatter before printing.</p>
     </section>
 
   </section>
@@ -488,8 +474,9 @@ error_logger:add_report_handler/1,2.
   level => debug}
 2> <input>logger:add_handler(debug_handler,logger_std_h,Config).</input>
 ok</pre>
-    <p>By default, the handler receives all events, so we need to add a filter
-      to stop all non-debug events:</p>
+    <p>By default, the handler receives all events
+      (<c>filter_defalt=log</c>), so we need to add a filter to stop
+      all non-debug events:</p>
     <pre>
 3> <input>Fun = fun(#{level:=debug}=Log,_) -> Log; (_,_) -> stop end.</input>
 #Fun&lt;erl_eval.12.98642416>
@@ -516,7 +503,7 @@ ok</pre>
     <p>It may also implement the following callbacks:</p>
     <code>
 adding_handler(logger:handler_id(),logger:config()) -> {ok,logger:config()} | {error,term()}
-removing_handler(logger:handler_id()) -> ok
+removing_handler(logger:handler_id(),logger:config()) -> ok
 changing_config(logger:handler_id(),logger:config(),logger:config()) -> {ok,logger:config()} | {error,term()}
     </code>
     <p>When logger:add_handler(Id,Module,Config) is called, logger
@@ -526,7 +513,7 @@ changing_config(logger:handler_id(),logger:config(),logger:config()) -> {ok,logg
       events as calls to Module:log/2.</p>
     <p>A handler can be removed by calling
       logger:remove_handler(Id). logger will call
-      Module:removing_handler(Id), and then remove the handler's
+      Module:removing_handler(Id,Config), and then remove the handler's
       configuration from the configuration database.</p>
     <p>When logger:set_handler_config is called, logger calls
       Module:changing_config(Id,OldConfig,NewConfig). If this function
@@ -547,7 +534,7 @@ log(Log,#{formatter:={FModule,FConfig}) ->
       this:</p>
     <code>
 -module(myhandler).
--export([adding_handler/2, removing_handler/1, log/2]).
+-export([adding_handler/2, removing_handler/2, log/2]).
 -export([init/1, handle_call/3, handle_cast/2, terminate/2]).
 
 adding_handler(Id,Config) ->
@@ -576,7 +563,7 @@ log(Log,#{myhandler_fd:=Fd,formatter:={FModule,FConfig}}) ->
       single process.</p>
     <code>
 -module(myhandler).
--export([adding_handler/2, removing_handler/1, log/2]).
+-export([adding_handler/2, removing_handler/2, log/2]).
 -export([init/1, handle_call/3, handle_cast/2, terminate/2]).
 
 adding_handler(Id,Config) ->
diff --git a/lib/kernel/doc/src/logger_formatter.xml b/lib/kernel/doc/src/logger_formatter.xml
index 6a17e3641f..a0940100ee 100644
--- a/lib/kernel/doc/src/logger_formatter.xml
+++ b/lib/kernel/doc/src/logger_formatter.xml
@@ -37,116 +37,157 @@
 
   <description>
     <p>Default formatter for the Logger application.</p>
-  </description>
-
-  <datatypes>
-    <datatype>
-      <name name="template"/>
-      <desc>
-      </desc>
-    </datatype>
-  </datatypes>
-
-  <funcs>
-    <func>
-      <name name="format" arity="2"/>
-      <fsummary>Formats the given message.</fsummary>
-      <desc>
-        <p>Formats the given message.</p>
-	<p>The template is a list of atoms, tuples and strings. Atoms
-	  can be <c>level</c> or <c>msg</c>, which are placeholders
-	  for the severity level and the log message,
-	  repectively. Tuples are interpreted as placeholders for
-	  metadata. Each element in the tuple must be an atom which
-	  matches a key in the nested metadata map, e.g. the
-	  tuple <c>{key1,key2}</c> will be replaced by the value of
-	  the key2 field in this nested map (the value vill be
-	  converted to a string):</p>
-
-<code>
-#{key1=>#{key2=>my_value,
-          ...},
-  ...}</code>
-
-
-        <p> Strings are printed literally.</p>
-
-	<p><c>depth</c> is a positive integer representing the maximum
-	  depth to which terms shall be printed by this
-	  formatter. Format strings passed to this formatter are
-	  rewritten. The format controls ~p and ~w are replaced with
-	  ~P and ~W, respectively, and the value is used as the depth
-	  parameter. For details, see
-	  <seealso marker="stdlib:io#format-2">io:format/2,3</seealso>
-	  in STDLIB.</p>
-
-	<p><c>chars_limit</c> is a positive integer representing the
-	  value of the option with the same name to be used when calling
-	  <seealso marker="stdlib:io#format-3">io:format/3</seealso>. This
-	  value limits the total number of characters printed bu the
-	  formatter. Notes that this is a soft limit. For a hard
-	  truncation limit, see option <c>max_size</c>.</p>
-
-	<p><c>max_size</c> is a positive integer representing the
-	  maximum size a string returned from this formatter can
-	  have. If the formatted string is longer, after possibly
-	  being limited by <c>depth</c> and/or <c>chars_limit</c>, it
-	  will be truncated.</p>
-
-	<p><c>utc</c> is a boolean. If set to true, all dates are
-	  displayed in Universal Coordinated Time. Default
-	  is <c>false</c>.</p>
 
-	<p><c>report_cb</c> must be a function with arity 1,
-	  returning <c>{Format,Args}</c>. This function will replace
-	  any <c>report_cb</c> found in metadata.</p>
-
-	<p>If <c>single_line=true</c>, all newlines in the message are
-	  replaced with <c>", "</c>, and whitespaces following directly
-	  after newlines are removed. Note that newlines added by the
-	  formatter template are not replaced.</p>
+  </description>
 
-	<p>If <c>legacy_header=true</c> a header field is added to
+  <section>
+    <title>Configuration</title>
+    <p>The following configuration parameters can be set
+      for <c>logger_formatter</c>:</p>
+    <taglist>
+      <tag><c>single_line = boolean()</c></tag>
+      <item>
+	<p>If set to <c>true</c>, all newlines in the message are
+	  replaced with <c>", "</c>, and whitespaces following
+	  directly after newlines are removed. Note that newlines
+	  added by the formatter template are not replaced.</p>
+	<p>Default is <c>true</c>.</p>
+      </item>
+      <tag><c>legacy_header = boolen()</c></tag>
+      <item>
+	<p>If set to <c>true</c> a header field is added to
 	  logger_formatter's part of <c>Metadata</c>. The value of
 	  this field is a string similar to the header created by the
 	  old <c>error_logger</c> event handlers. It can be included
 	  in the log event by adding the
-	  tuple <c>{logger_formatter,header}</c> to the template.</p>
-
-	<p>The default template when <c>legacy_header=true</c> is</p>
-
-	<code>[{logger_formatter,header},"\n",msg,"\n"]</code>
+	  tuple <c>{logger_formatter,header}</c> to the
+	  template. See <seealso marker="#default_templates">Default
+	  Templates</seealso> for more information</p>
+	<p>Default is <c>false</c>.</p>
+      </item>
+      <tag><c>report_cb = fun((logger:report()) -> {io:format(),[term()]})</c></tag>
+      <item>
+	<p>A function with arity 1,
+	  returning <c>{Format,Args}</c>. This function will replace
+	  any <c>report_cb</c> found in metadata.</p>
+      </item>
+      <tag><c>depth = pos_integer() | unlimited</c></tag>
+      <item>
+	<p>A positive integer representing the maximum depth to
+	  which terms shall be printed by this formatter. Format
+	  strings passed to this formatter are rewritten. The format
+	  controls ~p and ~w are replaced with ~P and ~W,
+	  respectively, and the value is used as the depth
+	  parameter. For details, see
+	  <seealso marker="stdlib:io#format-2">io:format/2,3</seealso>
+	  in STDLIB.</p>
+	<p>Default is <c>unlimited</c>.</p>
+      </item>
+      <tag><c>chars_limit = pos_integer() | unlimited</c></tag>
+      <item>
+	<p>A positive integer representing the value of the option
+	  with the same name to be used when calling
+	  <seealso marker="stdlib:io_lib#format-3">io_lib:format/3</seealso>.
+	  This value limits the total number of characters printed
+	  for each log event. Note that this is a soft limit. For a
+	  hard truncation limit, see option <c>max_size</c>.</p>
+	<p>Default is <c>unlimited</c>.</p>
+      </item>
+      <tag><c>max_size = pos_integer() | unlimited</c></tag>
+      <item>
+	<p>A positive integer representing the absolute maximum size
+	  a string returned from this formatter can have. If the
+	  formatted string is longer, after possibly being limited
+	  by <c>depth</c> and/or <c>chars_limit</c>, it will be
+	  truncated.</p>
+	<p>Default is <c>unlimited</c>.</p>
+      </item>
+      <tag><c>template = </c><seealso marker="#type-template"><c>template()</c></seealso></tag>
+      <item>
+	<p>The template is a list of atoms, tuples and strings. The
+	  atoms <c>level</c> or <c>msg</c>, are treated as
+	  placeholders for the severity level and the log message,
+	  repectively. Other atoms or tuples are interpreted as
+	  placeholders for metadata, where atoms are expected to
+	  match top level keys, and tuples represent paths to sub
+	  keys in a nested map. For example the
+	  tuple <c>{key1,key2}</c> will be replaced by the value of
+	  the <c>key2</c> field in the nested map below. The
+	  atom <c>key1</c> on its own would be replaced by the
+	  complete value of the <c>key1</c> field. The values are
+	  converted to strings.</p>
 
-	<p>which will cause log entries like this:</p>
+<code>
+#{key1=>#{key2=>my_value,
+          ...}
+  ...}</code>
 
-	<code>=ERROR REPORT==== 29-Dec-2017::13:30:51.245123 ===
+        <p>Strings are printed literally.</p>
+	<p>The default template differs depending on the values
+	  of <c>legacy_header</c>
+	  and <c>single_line</c>. See <seealso marker="#default_templates">Default
+	  Templates</seealso> for more information</p>
+      </item>
+      <tag><c>utc = boolean()</c></tag>
+      <item>
+	<p>If set to <c>true</c>, all dates are displayed in Universal
+	  Coordinated Time. Default is <c>false</c>.</p>
+      </item>
+    </taglist>
+  </section>
+
+  <section>
+    <marker id="default_templates"/>
+    <title>Default templates</title>
+    <p>The default template when <c>legacy_header=true</c> is</p>
+
+    <code>[{logger_formatter,header},"\n",msg,"\n"]</code>
+
+    <p>which will cause log entries like this:</p>
+
+    <code>=ERROR REPORT==== 29-Dec-2017::13:30:51.245123 ===
     process: &lt;0.74.0&gt;
     exit_reason: "Something went wrong"</code>
 
-	<p>Note that all eight levels might occur here, not
-	  only <c>ERROR</c>, <c>WARNING</c> or <c>INFO</c>. And also
-	  that micro seconds are added at the end of the
-	  timestamp.</p>
+    <p>Note that all eight levels might occur here, not
+      only <c>ERROR</c>, <c>WARNING</c> or <c>INFO</c>. And also that
+      micro seconds are added at the end of the timestamp.</p>
 
-	<p>The default template when <c>single_line=true</c> is</p>
+    <p>The default template when <c>single_line=true</c> is</p>
 
-	<code>[time," ",level,": ",msg,"\n"]</code>
+    <code>[time," ",level,": ",msg,"\n"]</code>
 
-	<p>which will cause log entries like this:</p>
+    <p>which will cause log entries like this:</p>
 
-	<code>2017-12-29 13:31:49.640317 error: process: &lt;0.74.0&gt;, exit_reason: "Something went wrong"</code>
+    <code>2017-12-29 13:31:49.640317 error: process: &lt;0.74.0&gt;, exit_reason: "Something went wrong"</code>
 
-	<p>The default template when both <c>legacy_header</c> and
-	  <c>single_line</c> are set to false is:</p>
+    <p>The default template when both <c>legacy_header</c> and
+      <c>single_line</c> are set to false is:</p>
 
-	<code>[time," ",level,":\n",msg,"\n"]</code>
+    <code>[time," ",level,":\n",msg,"\n"]</code>
 
-	<p>which will cause log entries like this:</p>
+    <p>which will cause log entries like this:</p>
 
-	<code>2017-12-29 13:32:25.191925 error:
+    <code>2017-12-29 13:32:25.191925 error:
     process: &lt;0.74.0&gt;
     exit_reason: "Something went wrong"</code>
+  </section>
+
+  <datatypes>
+    <datatype>
+      <name name="template"/>
+      <desc>
+      </desc>
+    </datatype>
+  </datatypes>
 
+  <funcs>
+    <func>
+      <name name="format" arity="2"/>
+      <fsummary>Formats the given message.</fsummary>
+      <desc>
+	<p>This the callback function to be called from handlers. It
+	  formats the given messages.</p>
       </desc>
     </func>