<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE chapter SYSTEM "chapter.dtd">
<chapter>
<header>
<copyright>
<year>2017</year>
<holder>Ericsson AB. All Rights Reserved.</holder>
</copyright>
<legalnotice>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
</legalnotice>
<title>Logging</title>
<prepared></prepared>
<docno></docno>
<date></date>
<rev></rev>
<file>logger_chapter.xml</file>
</header>
<section>
<title>Overview</title>
<p>Erlang/OTP provides a standard API for logging. The backend of
this API can be used as is, or it can be customized to suite
specific needs.</p>
<p>It consists of two parts - the <em>logger</em> part and the
<em>handler</em> part. The logger will forward log events to one
or more handler(s).</p>
<image file="logger_arch.png">
<icaption>Conceptual overview</icaption>
</image>
<p><em>Filters</em> can be added to the logger and to each
handler. The filters decide if an event is to be forwarded or
not, and they can also modify all parts of the log event.</p>
<p>A <em>formatter</em> can be set for each handler. The formatter
does the final formatting of the log event, including the log
message itself, and possibly a timestamp, header and other
metadata.</p>
<p>In accordance with the Syslog protocol, RFC-5424, eight
severity levels can be specified:</p>
<table align="left">
<row>
<cell><strong>Level</strong></cell>
<cell align="center"><strong>Integer</strong></cell>
<cell><strong>Description</strong></cell>
</row>
<row>
<cell>emergency</cell>
<cell align="center">0</cell>
<cell>system is unusable</cell>
</row>
<row>
<cell>alert</cell>
<cell align="center">1</cell>
<cell>action must be taken immediately</cell>
</row>
<row>
<cell>critical</cell>
<cell align="center">2</cell>
<cell>critical contidions</cell>
</row>
<row>
<cell>error</cell>
<cell align="center">3</cell>
<cell>error conditions</cell>
</row>
<row>
<cell>warning</cell>
<cell align="center">4</cell>
<cell>warning conditions</cell>
</row>
<row>
<cell>notice</cell>
<cell align="center">5</cell>
<cell>normal but significant conditions</cell>
</row>
<row>
<cell>info</cell>
<cell align="center">6</cell>
<cell>informational messages</cell>
</row>
<row>
<cell>debug</cell>
<cell align="center">7</cell>
<cell>debug-level messages</cell>
</row>
<tcaption>Severity levels</tcaption>
</table>
<p>A log event is allowed by Logger if the integer value of
its <c>Level</c> is less than or equal to the currently
configured log level. The log level can be configured globally,
or to allow more verbose logging from a specific part of the
system, per module.</p>
<section>
<title>Customizable parts</title>
<taglist>
<tag><marker id="Handler"/>Handler</tag>
<item>
<p>A handler is defined as a module exporting the following
function:</p>
<code>log(Log, Config) -> ok</code>
<p>A handler is called by the logger backend after filtering on
logger level and on handler level for the handler which is
about to be called. The function call is done on the client
process, and it is up to the handler implementation if other
processes are to be involved or not.</p>
<p>Multiple instances of the same handler can be
added. Configuration is per instance.</p>
</item>
<tag><marker id="Filter"/>Filter</tag>
<item>
<p>Filters can be set on the logger or on a handler. Logger
filters are applied first, and if passed, the handler filters
for each handler are applied. The handler plugin is only
called if all handler filters for the handler in question also
pass.</p>
<p>A filter is specified as:</p>
<code>{fun((Log,Extra) -> Log | stop | ignore), Extra}</code>
<p>The configuration parameter <c>filter_default</c>
specifies the behavior if all filters return <c>ignore</c>.
<c>filter_default</c> is by default set to <c>log</c>.</p>
<p>The <c>Extra</c> parameter may contain any data that the
filter needs.</p>
</item>
<tag><marker id="Formatter"/>Formatter</tag>
<item>
<p>A formatter is defined as a module exporting the following
function:</p>
<code>format(Log,Extra) -> string()</code>
<p>The formatter plugin is called by each handler, and the
returned string can be printed to the handler's destination
(stdout, file, ...).</p>
</item>
</taglist>
</section>
<section>
<title>Built-in handlers</title>
<taglist>
<tag><c>logger_std_h</c></tag>
<item>
<p>This is the default handler used by OTP. Multiple instances
can be started, and each instance will write log events to a
given destination, console or file. Filters can be used for
selecting which event to send to which handler instance.</p>
</item>
<tag><c>logger_disk_log_h</c></tag>
<item>
<p>This handler behaves much like logger_std_h, except it uses
<seealso marker="disk_log"><c>disk_log</c></seealso> as its
destination.</p>
</item>
<tag><marker id="ErrorLoggerManager"/><c>error_logger</c></tag>
<item>
<p>This handler is to be used for backwards compatibility
only. It is not started by default, but will be automatically
started the first time an event handler is added
with <seealso marker="error_logger#add_report_handler-1">
<c>error_logger:add_report_handler/1,2</c></seealso>.</p>
<p>No built-in event handlers exist.</p>
</item>
</taglist>
</section>
<section>
<title>Built-in filters</title>
<taglist>
<tag><c>logger_filters:domain/2</c></tag>
<item>
<p>This filter provides a way of filtering log events based on a
<c>domain</c> field <c>Metadata</c>. See
<seealso marker="logger_filters#domain-2">
<c>logger_filters:domain/2</c></seealso></p>
</item>
<tag><c>logger_filters:level/2</c></tag>
<item>
<p>This filter provides a way of filtering log events based
on the log level. See <seealso marker="logger_filters#domain-2">
<c>logger_filters:domain/2</c></seealso></p>
</item>
<tag><c>logger_filters:progress/2</c></tag>
<item>
<p>This filter matches all progress reports
from <c>supervisor</c> and <c>application_controller</c>.
See <seealso marker="logger_filters#progress/2">
<c>logger_filters:progress/2</c></seealso></p>
</item>
<tag><c>logger_filters:remote_gl/2</c></tag>
<item>
<p>This filter matches all events originating from a process
that has its group leader on a remote node.
See <seealso marker="logger_filters#remote_gl/2">
<c>logger_filters:remote_gl/2</c></seealso></p>
</item>
</taglist>
</section>
<section>
<title>Default formatter</title>
<p>The default formatter is <c>logger_formatter</c>.
See <seealso marker="logger_formatter#format-2">
<c>logger_formatter:format/2</c></seealso>.</p>
</section>
</section>
<section>
<title>Configuration</title>
<section>
<title>Application environment variables</title>
<p>See <seealso marker="kernel_app#configuration">Kernel(6)</seealso> for
information about the application environment variables that can
be used for configuring logger.</p>
</section>
<section>
<title>Logger configuration</title>
<taglist>
<tag><c>level</c></tag>
<item>
<p>Specifies the severity level to log.</p>
</item>
<tag><c>filters</c></tag>
<item>
<p>Logger filters are added or removed with
<seealso marker="logger#add_logger_filter-2">
<c>logger:add_logger_filter/2</c></seealso> and
<seealso marker="logger#remove_logger_filter-1">
<c>logger:remove_logger_filter/1</c></seealso>,
respectively.</p>
<p>See <seealso marker="#Filter">Filter</seealso> for more
information.</p>
<p>By default, no filters exist.</p>
</item>
<tag><c>filter_default = log | stop</c></tag>
<item>
<p>Specifies what to do with an event if all filters
return <c>ignore</c>.</p>
<p>Default is <c>log</c>.</p>
</item>
<tag><c>handlers</c></tag>
<item>
<p>Handlers are added or removed with
<seealso marker="logger#add_handler-3">
<c>logger:add_handler/3</c></seealso> and
<seealso marker="logger#remove_handler-1">
<c>logger:remove_handler/1</c></seealso>,
respectively.</p>
<p>See <seealso marker="#Handler">Handler</seealso> for more
information.</p>
</item>
</taglist>
</section>
<section>
<title>Handler configuration</title>
<taglist>
<tag><c>level</c></tag>
<item>
<p>Specifies the severity level to log.</p>
</item>
<tag><c>filters</c></tag>
<item>
<p>Handler filters can be specified when adding the handler,
or added or removed later with
<seealso marker="logger#add_handler_filter-3">
<c>logger:add_handler_filter/3</c></seealso> and
<seealso marker="logger#remove_handler_filter-2">
<c>logger:remove_handler_filter/2</c></seealso>,
respectively.</p>
<p>See <seealso marker="#Filter">Filter</seealso> for more
information.</p>
<p>By default, no filters exist.</p>
</item>
<tag><c>filter_default = log | stop</c></tag>
<item>
<p>Specifies what to do with an event if all filters
return <c>ignore</c>.</p>
<p>Default is <c>log</c>.</p>
</item>
<tag><c>depth = pos_integer() | unlimited</c></tag>
<item>
<p>Specifies if the depth of terms in the log events shall
be limited by using control characters <c>~P</c>
and <c>~W</c> instead of <c>~p</c> and <c>~w</c>,
respectively. See
<seealso marker="stdlib:io#format-1"><c>io:format</c></seealso>.</p>
</item>
<tag><c>max_size = pos_integer() | unlimited</c></tag>
<item>
<p>Specifies if the size of a log event shall be limited by
truncating the formatted string.</p>
</item>
<tag><c>formatter = {Module::module(),Extra::term()}</c></tag>
<item>
<p>See <seealso marker="#Formatter">Formatter</seealso> for more
information.</p>
<p>The default module is <seealso marker="logger_formatter">
<c>logger_formatter</c></seealso>, and <c>Extra</c> is
it's configuration map.</p>
</item>
</taglist>
<p>Note that <c>level</c> and <c>filters</c> are obeyed by
Logger itself before forwarding the log events to each
handler, while <c>depth</c>, <c>max_size</c>
and <c>formatter</c> are left to the handler
implementation. All Logger's built-in handlers do apply these
configuration parameters before printing.</p>
</section>
</section>
<section>
<marker id="compatibility"/>
<title>Backwards compatibility with error_logger</title>
<p>Logger provides backwards compatibility with the old
<c>error_logger</c> in the following ways:</p>
<taglist>
<tag>Legacy event handlers</tag>
<item>
<p>To use event handlers written for <c>error_logger</c>, just
add your event handler with</p>
<code>
error_logger:add_report_handler/1,2.
</code>
<p>This will automatically start the <c>error_logger</c>
event manager, and add <c>error_logger</c> as a
handler to <c>logger</c>, with configuration</p>
<code>
#{level=>info,
filter_default=>log,
filters=>[]}.
</code>
<p>Note that this handler will ignore events that do not
originate from the old <c>error_logger</c> API, or from
within OTP. This means that if your code uses the logger API
for logging, then your log events will be discarded by this
handler.</p>
<p>Also note that <c>error_logger</c> is not overload
protected.</p>
</item>
<tag>Logger API</tag>
<item>
<p>The old <c>error_logger</c> API still exists, but should
only be used by legacy code. It will be removed in a later
release.</p>
</item>
<tag>Output format</tag>
<item>
<p>To get log events on the same format as produced
by <c>error_logger_tty_h</c> and <c>error_logger_file_h</c>,
use the default formatter, <c>logger_formatter</c>, with
configuration parameter <c>legacy_header=>true</c>. This is
also the default.</p>
</item>
<tag>Default format of log events from OTP</tag>
<item>
<p>By default, all log events originating from within OTP,
except the former so called "SASL reports", look the same as
before.</p>
</item>
<tag>SASL reports</tag>
<item>
<p>By SASL reports we mean supervisor reports, crash reports
and progress reports.</p>
<p>In earlier releases, these reports were only logged when
the SASL application was running, and they were printed
trough specific event handlers
named <c>sasl_report_tty_h</c>
and <c>sasl_report_file_h</c>.</p>
<p>The destination of these log events were configured by
environment variables for the SASL application.</p>
<p>Due to the specific event handlers, the output format
slightly differed from other log events.</p>
<p>As of OTP-21, the concept of SASL reports is removed,
meaning that the default behavior is as follows:</p>
<list>
<item>Supervisor reports, crash reports and progress reports
are no longer connected to the SASL application.</item>
<item>Supervisor reports and crash reports are logged by
default.</item>
<item>Progress reports are not logged by default, but can be
enabled with the kernel environment
variable <c>logger_log_progress</c>.</item>
<item>The output format is the same for all log
events.</item>
</list>
<p>If the old behavior is preferred, the kernel environment
variable <c>logger_sasl_compatible</c> can be set
to <c>true</c>. The old SASL environment variables can then
be used as before, and the SASL reports will only be printed
if the SASL application is running - through a second log
handler named <c>sasl_h</c>.</p>
<p>All SASL reports have a metadata
field <c>domain=>[beam,erlang,otp,sasl]</c>, which can be
used, for example, by filters to to stop or allow the
events.</p>
</item>
</taglist>
</section>
<section>
<title>Error handling</title>
<p>Log data is expected to be either a format string and
arguments, a string (unicode:chardata), or a report (map or
key-value list) which can be converted to a format string and
arguments by the handler. A default report callback should be
included in the log event's metadata, which can be used for
converting the report to a format string and arguments. The
handler might also do a custom conversion if the default format
is not desired.</p>
<p><c>logger</c> does, to a certain extent, check its input data
before forwarding a log event to the handlers, but it does not
evaluate conversion funs or check the validity of format strings
and arguments. This means that any filter or handler must be
careful when formatting the data of a log event, making sure
that it does not crash due to bad input data or faulty
callbacks.</p>
<p>If a filter or handler still crashes, logger will remove the
filter or handler in question from the configuration, and then
print a short error message on the console. A debug event
containing the crash reason and other details is also issued,
and can be seen if a handler is installed which logs on debug
level.</p>
</section>
<section>
<title>Example: add a handler to log debug events to file</title>
<p>When starting an erlang node, the default behavior is that all
log events with level info and above are logged to the
console. In order to also log debug events, you can either
change the global log level to <c>debug</c> or add a separate
handler to take care of this. In this example we will add a new
handler which prints the debug events to a separate file.</p>
<p>First, we add an instance of logger_std_h with
type <c>{file,File}</c>, and we set the handler's level
to <c>debug</c>:</p>
<pre>
1> <input>Config = #{level=>debug,logger_std_h=>#{type=>{file,"./debug.log"}}}.</input>
#{logger_std_h => #{type => {file,"./debug.log"}},
level => debug}
2> <input>logger:add_handler(debug_handler,logger_std_h,Config).</input>
ok</pre>
<p>By default, the handler receives all events, so we need to add a filter
to stop all non-debug events:</p>
<pre>
3> <input>Fun = fun(#{level:=debug}=Log,_) -> Log; (_,_) -> stop end.</input>
#Fun<erl_eval.12.98642416>
4> <input>logger:add_handler_filter(debug_handler,allow_debug,{Fun,[]}).</input>
ok</pre>
<p>And finally, we need to make sure that the logger itself allows
debug events. This can either be done by setting the global
logger level:</p>
<pre>
5> <input>logger:set_logger_config(level,debug).</input>
ok</pre>
<p>Or by allowing debug events from one or a few modules only:</p>
<pre>
6> <input>logger:set_module_level(mymodule,debug).</input>
ok</pre>
</section>
<section>
<title>Example: implement a handler</title>
<p>The only requirement that a handler MUST fulfill is to export
the following function:</p>
<code>log(logger:log(),logger:config()) ->ok</code>
<p>It may also implement the following callbacks:</p>
<code>
adding_handler(logger:handler_id(),logger:config()) -> {ok,logger:config()} | {error,term()}
removing_handler(logger:handler_id()) -> ok
changing_config(logger:handler_id(),logger:config(),logger:config()) -> {ok,logger:config()} | {error,term()}
</code>
<p>When logger:add_handler(Id,Module,Config) is called, logger
will first call Module:adding_handler(Id,Config), and if it
returns {ok,NewConfig} the NewConfig is written to the
configuration database. After this, the handler may receive log
events as calls to Module:log/2.</p>
<p>A handler can be removed by calling
logger:remove_handler(Id). logger will call
Module:removing_handler(Id), and then remove the handler's
configuration from the configuration database.</p>
<p>When logger:set_handler_config is called, logger calls
Module:changing_config(Id,OldConfig,NewConfig). If this function
returns ok, the NewConfig is written to the configuration
database.</p>
<p>A simple handler which prints to the console could be
implemented as follows:</p>
<code>
-module(myhandler).
-export([log/2]).
log(#{msg:={report,R}},_) ->
io:format("~p~n",[R]);
log(#{msg:={string,S}},_) ->
io:put_chars(S);
log(#{msg:={F,A}},_) ->
io:format(F,A).
</code>
<p>A simple handler which prints to file could be implemented like
this:</p>
<code>
-module(myhandler).
-export([adding_handler/2, removing_handler/1, log/2]).
-export([init/1, handle_call/3, handle_cast/2, terminate/2]).
adding_handler(Id,Config) ->
{ok,Fd} = file:open(File,[append,{encoding,utf8}]),
{ok,Config#{myhandler_fd=>Fd}}.
removing_handler(Id,#{myhandler_fd:=Fd}) ->
_ = file:close(Fd),
ok.
log(#{msg:={report,R}},#{myhandler_fd:=Fd}) ->
io:format(Fd,"~p~n",[R]);
log(#{msg:={string,S}},#{myhandler_fd:=Fd}) ->
io:put_chars(Fd,S);
log(#{msg:={F,A}},#{myhandler_fd:=Fd}) ->
io:format(Fd,F,A).
</code>
<p>Note that none of the above handlers have any overload
protection, and all log events are printed directly from the
client process. Neither do the handlers use the formatter or
in any way add time or other metadata to the printed events.</p>
<p>For examples of overload protection, please refer to the
implementation
of <seealso marker="logger_std_h"><c>logger_std_h</c></seealso>
and <!--<seealso marker="logger_disk_log_h"--><c>logger_disk_log_h</c>
<!--/seealso-->.</p>
<p>Below is a simpler example of a handler which logs through one
single process, and uses the default formatter to gain a common
look of the log events.</p>
<p>It also uses the metadata field <c>report_cb</c>, if it exists,
to print reports in the way the event issuer suggests. The
formatter will normally do this, but if the handler either has
an own default (as in this example) or if the
given <c>report_cb</c> should not be used at all, then the
handler must take care of this itself.</p>
<code>
-module(myhandler).
-export([adding_handler/2, removing_handler/1, log/2]).
-export([init/1, handle_call/3, handle_cast/2, terminate/2]).
adding_handler(Id,Config) ->
{ok,Pid} = gen_server:start(?MODULE,Config),
{ok,Config#{myhandler_pid=>Pid}}.
removing_handler(Id,#{myhandler_pid:=Pid}) ->
gen_server:stop(Pid).
log(Log,#{myhandler_pid:=Pid} = Config) ->
gen_server:cast(Pid,{log,Log,Config}).
init(#{myhandler_file:=File}) ->
{ok,Fd} = file:open(File,[append,{encoding,utf8}]),
{ok,#{file=>File,fd=>Fd}}.
handle_call(_,_,State) ->
{reply,{error,bad_request},State}.
handle_cast({log,Log,Config},#{fd:=Fd} = State) ->
do_log(Fd,Log,Config),
{noreply,State}.
terminate(Reason,#{fd:=Fd}) ->
_ = file:close(Fd),
ok.
do_log(Fd,#{msg:={report,R}} = Log, Config) ->
Fun = maps:get(report_cb,Config,fun my_report_cb/1,
{F,A} = Fun(R),
do_log(Fd,Log#{msg=>{F,A},Config);
do_log(Fd,Log,#{formatter:={FModule,FConfig}}) ->
String = FModule:format(Log,FConfig),
io:put_chars(Fd,String).
my_report_cb(R) ->
{"~p",[R]}.
</code>
</section>
<section>
<title>See Also</title>
<p><seealso marker="error_logger"><c>error_logger(3)</c></seealso>,
<seealso marker="sasl:sasl_app"><c>SASL(6)</c></seealso></p>
</section>
</chapter>
