diff options
Diffstat (limited to 'lib/kernel/doc/src/logger_chapter.xml')
| -rw-r--r-- | lib/kernel/doc/src/logger_chapter.xml | 641 |
1 files changed, 641 insertions, 0 deletions
diff --git a/lib/kernel/doc/src/logger_chapter.xml b/lib/kernel/doc/src/logger_chapter.xml new file mode 100644 index 0000000000..2a325453da --- /dev/null +++ b/lib/kernel/doc/src/logger_chapter.xml @@ -0,0 +1,641 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2017</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + </legalnotice> + + <title>Logging</title> + <prepared></prepared> + <docno></docno> + <date></date> + <rev></rev> + <file>logger_chapter.xml</file> + </header> + + <section> + <title>Overview</title> + <p>Erlang/OTP provides a standard API for logging. The backend of + this API can be used as is, or it can be customized to suite + specific needs.</p> + <p>It consists of two parts - the <em>logger</em> part and the + <em>handler</em> part. The logger will forward log events to one + or more handler(s).</p> + + <image file="logger_arch.png"> + <icaption>Conceptual overview</icaption> + </image> + + <p><em>Filters</em> can be added to the logger and to each + handler. The filters decide if an event is to be forwarded or + not, and they can also modify all parts of the log event.</p> + + <p>A <em>formatter</em> can be set for each handler. The formatter + does the final formatting of the log event, including the log + message itself, and possibly a timestamp, header and other + metadata.</p> + + <p>In accordance with the Syslog protocol, RFC-5424, eight + severity levels can be specified:</p> + + <table align="left"> + <row> + <cell><strong>Level</strong></cell> + <cell align="center"><strong>Integer</strong></cell> + <cell><strong>Description</strong></cell> + </row> + <row> + <cell>emergency</cell> + <cell align="center">0</cell> + <cell>system is unusable</cell> + </row> + <row> + <cell>alert</cell> + <cell align="center">1</cell> + <cell>action must be taken immediately</cell> + </row> + <row> + <cell>critical</cell> + <cell align="center">2</cell> + <cell>critical contidions</cell> + </row> + <row> + <cell>error</cell> + <cell align="center">3</cell> + <cell>error conditions</cell> + </row> + <row> + <cell>warning</cell> + <cell align="center">4</cell> + <cell>warning conditions</cell> + </row> + <row> + <cell>notice</cell> + <cell align="center">5</cell> + <cell>normal but significant conditions</cell> + </row> + <row> + <cell>info</cell> + <cell align="center">6</cell> + <cell>informational messages</cell> + </row> + <row> + <cell>debug</cell> + <cell align="center">7</cell> + <cell>debug-level messages</cell> + </row> + <tcaption>Severity levels</tcaption> + </table> + + <p>A log event is allowed by Logger if the integer value of + its <c>Level</c> is less than or equal to the currently + configured log level. The log level can be configured globally, + or to allow more verbose logging from a specific part of the + system, per module.</p> + + <section> + <title>Customizable parts</title> + + <taglist> + <tag><marker id="Handler"/>Handler</tag> + <item> + <p>A handler is defined as a module exporting the following + function:</p> + + <code>log(Log, Config) -> ok</code> + + <p>A handler is called by the logger backend after filtering on + logger level and on handler level for the handler which is + about to be called. The function call is done on the client + process, and it is up to the handler implementation if other + processes are to be involved or not.</p> + + <p>Multiple instances of the same handler can be + added. Configuration is per instance.</p> + + </item> + + <tag><marker id="Filter"/>Filter</tag> + <item> + <p>Filters can be set on the logger or on a handler. Logger + filters are applied first, and if passed, the handler filters + for each handler are applied. The handler plugin is only + called if all handler filters for the handler in question also + pass.</p> + + <p>A filter is specified as:</p> + + <code>{fun((Log,Extra) -> Log | stop | ignore), Extra}</code> + + <p>The configuration parameter <c>filter_default</c> + specifies the behavior if all filters return <c>ignore</c>. + <c>filter_default</c> is by default set to <c>log</c>.</p> + + <p>The <c>Extra</c> parameter may contain any data that the + filter needs.</p> + </item> + + <tag><marker id="Formatter"/>Formatter</tag> + <item> + <p>A formatter is defined as a module exporting the following + function:</p> + + <code>format(Log,Extra) -> string()</code> + + <p>The formatter plugin is called by each handler, and the + returned string can be printed to the handler's destination + (stdout, file, ...).</p> + </item> + + </taglist> + </section> + + <section> + <title>Built-in handlers</title> + + <taglist> + <tag><c>logger_std_h</c></tag> + <item> + <p>This is the default handler used by OTP. Multiple instances + can be started, and each instance will write log events to a + given destination, console or file. Filters can be used for + selecting which event to send to which handler instance.</p> + </item> + + <tag><c>logger_disk_log_h</c></tag> + <item> + <p>This handler behaves much like logger_std_h, except it uses + <seealso marker="disk_log"><c>disk_log</c></seealso> as its + destination.</p> + </item> + + <tag><marker id="ErrorLoggerManager"/><c>error_logger</c></tag> + <item> + <p>This handler is to be used for backwards compatibility + only. It is not started by default, but will be automatically + started the first time an event handler is added + with <seealso marker="error_logger#add_report_handler-1"> + <c>error_logger:add_report_handler/1,2</c></seealso>.</p> + + <p>No built-in event handlers exist.</p> + </item> + </taglist> + </section> + + <section> + <title>Built-in filters</title> + + <taglist> + <tag><c>logger_filters:domain/2</c></tag> + <item> + <p>This filter provides a way of filtering log events based on a + <c>domain</c> field <c>Metadata</c>. See + <seealso marker="logger_filters#domain-2"> + <c>logger_filters:domain/2</c></seealso></p> + </item> + + <tag><c>logger_filters:level/2</c></tag> + <item> + <p>This filter provides a way of filtering log events based + on the log level. See <seealso marker="logger_filters#domain-2"> + <c>logger_filters:domain/2</c></seealso></p> + </item> + + <tag><c>logger_filters:progress/2</c></tag> + <item> + <p>This filter matches all progress reports + from <c>supervisor</c> and <c>application_controller</c>. + See <seealso marker="logger_filters#progress/2"> + <c>logger_filters:progress/2</c></seealso></p> + </item> + + <tag><c>logger_filters:remote_gl/2</c></tag> + <item> + <p>This filter matches all events originating from a process + that has its group leader on a remote node. + See <seealso marker="logger_filters#remote_gl/2"> + <c>logger_filters:remote_gl/2</c></seealso></p> + </item> + </taglist> + </section> + + <section> + <title>Default formatter</title> + + <p>The default formatter is <c>logger_formatter</c>. + See <seealso marker="logger_formatter#format-2"> + <c>logger_formatter:format/2</c></seealso>.</p> + </section> + </section> + + <section> + <title>Configuration</title> + + <section> + <title>Application environment variables</title> + <p>See <seealso marker="kernel_app#configuration">Kernel(6)</seealso> for + information about the application environment variables that can + be used for configuring logger.</p> + </section> + + <section> + <title>Logger configuration</title> + + <taglist> + <tag><c>level</c></tag> + <item> + <p>Specifies the severity level to log.</p> + </item> + <tag><c>filters</c></tag> + <item> + <p>Logger filters are added or removed with + <seealso marker="logger#add_logger_filter-2"> + <c>logger:add_logger_filter/2</c></seealso> and + <seealso marker="logger#remove_logger_filter-1"> + <c>logger:remove_logger_filter/1</c></seealso>, + respectively.</p> + <p>See <seealso marker="#Filter">Filter</seealso> for more + information.</p> + <p>By default, no filters exist.</p> + </item> + <tag><c>filter_default = log | stop</c></tag> + <item> + <p>Specifies what to do with an event if all filters + return <c>ignore</c>.</p> + <p>Default is <c>log</c>.</p> + </item> + <tag><c>handlers</c></tag> + <item> + <p>Handlers are added or removed with + <seealso marker="logger#add_handler-3"> + <c>logger:add_handler/3</c></seealso> and + <seealso marker="logger#remove_handler-1"> + <c>logger:remove_handler/1</c></seealso>, + respectively.</p> + <p>See <seealso marker="#Handler">Handler</seealso> for more + information.</p> + </item> + </taglist> + </section> + + <section> + <title>Handler configuration</title> + <taglist> + <tag><c>level</c></tag> + <item> + <p>Specifies the severity level to log.</p> + </item> + <tag><c>filters</c></tag> + <item> + <p>Handler filters can be specified when adding the handler, + or added or removed later with + <seealso marker="logger#add_handler_filter-3"> + <c>logger:add_handler_filter/3</c></seealso> and + <seealso marker="logger#remove_handler_filter-2"> + <c>logger:remove_handler_filter/2</c></seealso>, + respectively.</p> + <p>See <seealso marker="#Filter">Filter</seealso> for more + information.</p> + <p>By default, no filters exist.</p> + </item> + <tag><c>filter_default = log | stop</c></tag> + <item> + <p>Specifies what to do with an event if all filters + return <c>ignore</c>.</p> + <p>Default is <c>log</c>.</p> + </item> + <tag><c>depth = pos_integer() | unlimited</c></tag> + <item> + <p>Specifies if the depth of terms in the log events shall + be limited by using control characters <c>~P</c> + and <c>~W</c> instead of <c>~p</c> and <c>~w</c>, + respectively. See + <seealso marker="stdlib:io#format-1"><c>io:format</c></seealso>.</p> + </item> + <tag><c>max_size = pos_integer() | unlimited</c></tag> + <item> + <p>Specifies if the size of a log event shall be limited by + truncating the formatted string.</p> + </item> + <tag><c>formatter = {Module::module(),Extra::term()}</c></tag> + <item> + <p>See <seealso marker="#Formatter">Formatter</seealso> for more + information.</p> + <p>The default module is <seealso marker="logger_formatter"> + <c>logger_formatter</c></seealso>, and <c>Extra</c> is + it's configuration map.</p> + </item> + </taglist> + + <p>Note that <c>level</c> and <c>filters</c> are obeyed by + Logger itself before forwarding the log events to each + handler, while <c>depth</c>, <c>max_size</c> + and <c>formatter</c> are left to the handler + implementation. All Logger's built-in handlers do apply these + configuration parameters before printing.</p> + </section> + + </section> + + <section> + <marker id="compatibility"/> + <title>Backwards compatibility with error_logger</title> + <p>Logger provides backwards compatibility with the old + <c>error_logger</c> in the following ways:</p> + + <taglist> + <tag>Legacy event handlers</tag> + <item> + <p>To use event handlers written for <c>error_logger</c>, just + add your event handler with</p> + <code> +error_logger:add_report_handler/1,2. + </code> + <p>This will automatically start the <c>error_logger</c> + event manager, and add <c>error_logger</c> as a + handler to <c>logger</c>, with configuration</p> +<code> +#{level=>info, + filter_default=>log, + filters=>[]}. +</code> + <p>Note that this handler will ignore events that do not + originate from the old <c>error_logger</c> API, or from + within OTP. This means that if your code uses the logger API + for logging, then your log events will be discarded by this + handler.</p> + <p>Also note that <c>error_logger</c> is not overload + protected.</p> + </item> + <tag>Logger API</tag> + <item> + <p>The old <c>error_logger</c> API still exists, but should + only be used by legacy code. It will be removed in a later + release.</p> + </item> + <tag>Output format</tag> + <item> + <p>To get log events on the same format as produced + by <c>error_logger_tty_h</c> and <c>error_logger_file_h</c>, + use the default formatter, <c>logger_formatter</c>, with + configuration parameter <c>legacy_header=>true</c>. This is + also the default.</p> + </item> + <tag>Default format of log events from OTP</tag> + <item> + <p>By default, all log events originating from within OTP, + except the former so called "SASL reports", look the same as + before.</p> + </item> + <tag>SASL reports</tag> + <item> + <p>By SASL reports we mean supervisor reports, crash reports + and progress reports.</p> + <p>In earlier releases, these reports were only logged when + the SASL application was running, and they were printed + trough specific event handlers + named <c>sasl_report_tty_h</c> + and <c>sasl_report_file_h</c>.</p> + <p>The destination of these log events were configured by + environment variables for the SASL application.</p> + <p>Due to the specific event handlers, the output format + slightly differed from other log events.</p> + <p>As of OTP-21, the concept of SASL reports is removed, + meaning that the default behavior is as follows:</p> + <list> + <item>Supervisor reports, crash reports and progress reports + are no longer connected to the SASL application.</item> + <item>Supervisor reports and crash reports are logged by + default.</item> + <item>Progress reports are not logged by default, but can be + enabled with the kernel environment + variable <c>logger_log_progress</c>.</item> + <item>The output format is the same for all log + events.</item> + </list> + <p>If the old behavior is preferred, the kernel environment + variable <c>logger_sasl_compatible</c> can be set + to <c>true</c>. The old SASL environment variables can then + be used as before, and the SASL reports will only be printed + if the SASL application is running - through a second log + handler named <c>sasl_h</c>.</p> + <p>All SASL reports have a metadata + field <c>domain=>[beam,erlang,otp,sasl]</c>, which can be + used, for example, by filters to to stop or allow the + events.</p> + </item> + </taglist> + </section> + + + <section> + <title>Error handling</title> + <p>Log data is expected to be either a format string and + arguments, a string (unicode:chardata), or a report (map or + key-value list) which can be converted to a format string and + arguments by the handler. A default report callback should be + included in the log event's metadata, which can be used for + converting the report to a format string and arguments. The + handler might also do a custom conversion if the default format + is not desired.</p> + <p><c>logger</c> does, to a certain extent, check its input data + before forwarding a log event to the handlers, but it does not + evaluate conversion funs or check the validity of format strings + and arguments. This means that any filter or handler must be + careful when formatting the data of a log event, making sure + that it does not crash due to bad input data or faulty + callbacks.</p> + <p>If a filter or handler still crashes, logger will remove the + filter or handler in question from the configuration, and then + print a short error message on the console. A debug event + containing the crash reason and other details is also issued, + and can be seen if a handler is installed which logs on debug + level.</p> + </section> + + <section> + <title>Example: add a handler to log debug events to file</title> + <p>When starting an erlang node, the default behavior is that all + log events with level info and above are logged to the + console. In order to also log debug events, you can either + change the global log level to <c>debug</c> or add a separate + handler to take care of this. In this example we will add a new + handler which prints the debug events to a separate file.</p> + <p>First, we add an instance of logger_std_h with + type <c>{file,File}</c>, and we set the handler's level + to <c>debug</c>:</p> + <pre> +1> <input>Config = #{level=>debug,logger_std_h=>#{type=>{file,"./debug.log"}}}.</input> +#{logger_std_h => #{type => {file,"./debug.log"}}, + level => debug} +2> <input>logger:add_handler(debug_handler,logger_std_h,Config).</input> +ok</pre> + <p>By default, the handler receives all events, so we need to add a filter + to stop all non-debug events:</p> + <pre> +3> <input>Fun = fun(#{level:=debug}=Log,_) -> Log; (_,_) -> stop end.</input> +#Fun<erl_eval.12.98642416> +4> <input>logger:add_handler_filter(debug_handler,allow_debug,{Fun,[]}).</input> +ok</pre> + <p>And finally, we need to make sure that the logger itself allows + debug events. This can either be done by setting the global + logger level:</p> + <pre> +5> <input>logger:set_logger_config(level,debug).</input> +ok</pre> + <p>Or by allowing debug events from one or a few modules only:</p> + <pre> +6> <input>logger:set_module_level(mymodule,debug).</input> +ok</pre> + + </section> + + <section> + <title>Example: implement a handler</title> + <p>The only requirement that a handler MUST fulfill is to export + the following function:</p> + <code>log(logger:log(),logger:config()) ->ok</code> + <p>It may also implement the following callbacks:</p> + <code> +adding_handler(logger:handler_id(),logger:config()) -> {ok,logger:config()} | {error,term()} +removing_handler(logger:handler_id()) -> ok +changing_config(logger:handler_id(),logger:config(),logger:config()) -> {ok,logger:config()} | {error,term()} + </code> + <p>When logger:add_handler(Id,Module,Config) is called, logger + will first call Module:adding_handler(Id,Config), and if it + returns {ok,NewConfig} the NewConfig is written to the + configuration database. After this, the handler may receive log + events as calls to Module:log/2.</p> + <p>A handler can be removed by calling + logger:remove_handler(Id). logger will call + Module:removing_handler(Id), and then remove the handler's + configuration from the configuration database.</p> + <p>When logger:set_handler_config is called, logger calls + Module:changing_config(Id,OldConfig,NewConfig). If this function + returns ok, the NewConfig is written to the configuration + database.</p> + + <p>A simple handler which prints to the console could be + implemented as follows:</p> + <code> +-module(myhandler). +-export([log/2]). + +log(#{msg:={report,R}},_) -> + io:format("~p~n",[R]); +log(#{msg:={string,S}},_) -> + io:put_chars(S); +log(#{msg:={F,A}},_) -> + io:format(F,A). + </code> + + <p>A simple handler which prints to file could be implemented like + this:</p> + <code> +-module(myhandler). +-export([adding_handler/2, removing_handler/1, log/2]). +-export([init/1, handle_call/3, handle_cast/2, terminate/2]). + +adding_handler(Id,Config) -> + {ok,Fd} = file:open(File,[append,{encoding,utf8}]), + {ok,Config#{myhandler_fd=>Fd}}. + +removing_handler(Id,#{myhandler_fd:=Fd}) -> + _ = file:close(Fd), + ok. + +log(#{msg:={report,R}},#{myhandler_fd:=Fd}) -> + io:format(Fd,"~p~n",[R]); +log(#{msg:={string,S}},#{myhandler_fd:=Fd}) -> + io:put_chars(Fd,S); +log(#{msg:={F,A}},#{myhandler_fd:=Fd}) -> + io:format(Fd,F,A). + </code> + + <p>Note that none of the above handlers have any overload + protection, and all log events are printed directly from the + client process. Neither do the handlers use the formatter or + in any way add time or other metadata to the printed events.</p> + + <p>For examples of overload protection, please refer to the + implementation + of <seealso marker="logger_std_h"><c>logger_std_h</c></seealso> + and <!--<seealso marker="logger_disk_log_h"--><c>logger_disk_log_h</c> + <!--/seealso-->.</p> + + <p>Below is a simpler example of a handler which logs through one + single process, and uses the default formatter to gain a common + look of the log events.</p> + <p>It also uses the metadata field <c>report_cb</c>, if it exists, + to print reports in the way the event issuer suggests. The + formatter will normally do this, but if the handler either has + an own default (as in this example) or if the + given <c>report_cb</c> should not be used at all, then the + handler must take care of this itself.</p> + <code> +-module(myhandler). +-export([adding_handler/2, removing_handler/1, log/2]). +-export([init/1, handle_call/3, handle_cast/2, terminate/2]). + +adding_handler(Id,Config) -> + {ok,Pid} = gen_server:start(?MODULE,Config), + {ok,Config#{myhandler_pid=>Pid}}. + +removing_handler(Id,#{myhandler_pid:=Pid}) -> + gen_server:stop(Pid). + +log(Log,#{myhandler_pid:=Pid} = Config) -> + gen_server:cast(Pid,{log,Log,Config}). + +init(#{myhandler_file:=File}) -> + {ok,Fd} = file:open(File,[append,{encoding,utf8}]), + {ok,#{file=>File,fd=>Fd}}. + +handle_call(_,_,State) -> + {reply,{error,bad_request},State}. + +handle_cast({log,Log,Config},#{fd:=Fd} = State) -> + do_log(Fd,Log,Config), + {noreply,State}. + +terminate(Reason,#{fd:=Fd}) -> + _ = file:close(Fd), + ok. + +do_log(Fd,#{msg:={report,R}} = Log, Config) -> + Fun = maps:get(report_cb,Config,fun my_report_cb/1, + {F,A} = Fun(R), + do_log(Fd,Log#{msg=>{F,A},Config); +do_log(Fd,Log,#{formatter:={FModule,FConfig}}) -> + String = FModule:format(Log,FConfig), + io:put_chars(Fd,String). + +my_report_cb(R) -> + {"~p",[R]}. + </code> + </section> + + + <section> + <title>See Also</title> + <p><seealso marker="error_logger"><c>error_logger(3)</c></seealso>, + <seealso marker="sasl:sasl_app"><c>SASL(6)</c></seealso></p> + </section> +</chapter> |