From 0deea4a8f369013ec00e231d0c2c37e4ab3f0ba1 Mon Sep 17 00:00:00 2001 From: Siri Hansen Date: Tue, 24 Apr 2018 14:26:31 +0200 Subject: Add logger --- lib/kernel/doc/src/Makefile | 21 +- lib/kernel/doc/src/book.xml | 3 + lib/kernel/doc/src/introduction_chapter.xml | 64 +++ lib/kernel/doc/src/logger.xml | 478 +++++++++++++++++++++ lib/kernel/doc/src/logger_arch.png | Bin 0 -> 31459 bytes lib/kernel/doc/src/logger_chapter.xml | 641 ++++++++++++++++++++++++++++ lib/kernel/doc/src/logger_filters.xml | 191 +++++++++ lib/kernel/doc/src/logger_formatter.xml | 149 +++++++ lib/kernel/doc/src/logger_std_h.xml | 81 ++++ lib/kernel/doc/src/part.xml | 40 ++ lib/kernel/doc/src/ref_man.xml | 4 + lib/kernel/doc/src/specs.xml | 4 + 12 files changed, 1671 insertions(+), 5 deletions(-) create mode 100644 lib/kernel/doc/src/introduction_chapter.xml create mode 100644 lib/kernel/doc/src/logger.xml create mode 100644 lib/kernel/doc/src/logger_arch.png create mode 100644 lib/kernel/doc/src/logger_chapter.xml create mode 100644 lib/kernel/doc/src/logger_filters.xml create mode 100644 lib/kernel/doc/src/logger_formatter.xml create mode 100644 lib/kernel/doc/src/logger_std_h.xml create mode 100644 lib/kernel/doc/src/part.xml (limited to 'lib/kernel/doc/src') diff --git a/lib/kernel/doc/src/Makefile b/lib/kernel/doc/src/Makefile index 2413541082..10bdf46119 100644 --- a/lib/kernel/doc/src/Makefile +++ b/lib/kernel/doc/src/Makefile @@ -56,6 +56,10 @@ XML_REF3_FILES = application.xml \ inet.xml \ inet_res.xml \ init_stub.xml \ + logger.xml \ + logger_std_h.xml \ + logger_filters.xml \ + logger_formatter.xml \ net_adm.xml \ net_kernel.xml \ os.xml \ @@ -70,11 +74,17 @@ XML_REF4_FILES = app.xml config.xml XML_REF6_FILES = kernel_app.xml -XML_PART_FILES = -XML_CHAPTER_FILES = notes.xml +XML_PART_FILES = part.xml +XML_CHAPTER_FILES = \ + notes.xml \ + introduction_chapter.xml \ + logger_chapter.xml BOOK_FILES = book.xml +IMAGE_FILES = \ + logger_arch.png + XML_FILES = \ $(BOOK_FILES) $(XML_CHAPTER_FILES) \ $(XML_PART_FILES) $(XML_REF3_FILES) $(XML_REF4_FILES)\ @@ -111,7 +121,7 @@ SPECS_FLAGS = -I../../include # ---------------------------------------------------- # Targets # ---------------------------------------------------- -$(HTMLDIR)/%.gif: %.gif +$(HTMLDIR)/%: % $(INSTALL_DATA) $< $@ docs: man pdf html @@ -120,11 +130,12 @@ $(TOP_PDF_FILE): $(XML_FILES) pdf: $(TOP_PDF_FILE) -html: gifs $(HTML_REF_MAN_FILE) +html: images $(HTML_REF_MAN_FILE) man: $(MAN3_FILES) $(MAN4_FILES) $(MAN6_FILES) -gifs: $(GIF_FILES:%=$(HTMLDIR)/%) +images: $(IMAGE_FILES:%=$(HTMLDIR)/%) + debug opt: clean clean_docs: diff --git a/lib/kernel/doc/src/book.xml b/lib/kernel/doc/src/book.xml index 81a87d126d..0b69b547e7 100644 --- a/lib/kernel/doc/src/book.xml +++ b/lib/kernel/doc/src/book.xml @@ -34,6 +34,9 @@ + + + diff --git a/lib/kernel/doc/src/introduction_chapter.xml b/lib/kernel/doc/src/introduction_chapter.xml new file mode 100644 index 0000000000..6e6990ddda --- /dev/null +++ b/lib/kernel/doc/src/introduction_chapter.xml @@ -0,0 +1,64 @@ + + + + +
+ + 2017 + Ericsson AB. All Rights Reserved. + + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + + + Introduction + + + + + + + + introduction.xml +
+ +
+ Scope +

The Kernel application has all the code necessary to run + the Erlang runtime system: file servers, code servers, + and so on.

+

The Kernel application is the first application started. It is + mandatory in the sense that the minimal system based on + Erlang/OTP consists of Kernel and STDLIB. Kernel + contains the following functional areas:

+ + Start, stop, supervision, configuration, and distribution of applications + Code loading + Logging + Error logging + Global name service + Supervision of Erlang/OTP + Communication with sockets + Operating system interface + +
+ +
+ Prerequisites +

It is assumed that the reader is familiar with the Erlang programming + language.

+
+ + + diff --git a/lib/kernel/doc/src/logger.xml b/lib/kernel/doc/src/logger.xml new file mode 100644 index 0000000000..66e6e5c689 --- /dev/null +++ b/lib/kernel/doc/src/logger.xml @@ -0,0 +1,478 @@ + + + + +
+ + 2017 + Ericsson AB. All Rights Reserved. + + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + + + logger + + + + + + + A + logger.xml +
+ logger + API module for the logger application. + + + + + + + + + +

The severity level for the message to be logged.

+ + + + + +

+ + + + + +

+ + + + + +

+ + + + + +

Metadata associated with the message to be logged.

+ + + + + +

+ + + + + +

+ + + + + +

+ + + + + +

+ + + + + +

+ + + + +
+ Macros +

The following macros are defined:

+ + + ?LOG_EMERGENCY(StringOrReport[,Metadata]) + ?LOG_EMERGENCY(FunOrFormat,Args[,Metadata]) + ?LOG_ALERT(StringOrReport[,Metadata]) + ?LOG_ALERT(FunOrFormat,Args[,Metadata]) + ?LOG_CRITICAL(StringOrReport[,Metadata]) + ?LOG_CRITICAL(FunOrFormat,Args[,Metadata]) + ?LOG_ERROR(StringOrReport[,Metadata]) + ?LOG_ERROR(FunOrFormat,Args[,Metadata]) + ?LOG_WARNING(StringOrReport[,Metadata]) + ?LOG_WARNING(FunOrFormat,Args[,Metadata]) + ?LOG_NOTICE(StringOrReport[,Metadata]) + ?LOG_NOTICE(FunOrFormat,Args[,Metadata]) + ?LOG_INFO(StringOrReport[,Metadata]) + ?LOG_INFO(FunOrFormat,Args[,Metadata]) + ?LOG_DEBUG(StringOrReport[,Metadata]) + ?LOG_DEBUG(FunOrFormat,Args[,Metadata]) + + +

All macros expand to a call to logger, where Level is + taken from the macro name, and the following metadata is added, + or merged with the given Metadata:

+ + +#{mfa=>{?MODULE,?FUNCTION_NAME,?FUNCTION_ARITY}, + file=>?FILE, + line=>?LINE} + + +

The call is wrapped in a case statement and will be evaluated + only if Level is equal to or below the configured log + level.

+
+ + + + emergency(StringOrReport[,Metadata]) + emergency(Format,Args[,Metadata]) + emergency(Fun,FunArgs[,Metadata]) + Logs the given message as level emergency. + +

Equivalent to + log(emergency,...).

+ + + + + alert(StringOrReport[,Metadata]) + alert(Format,Args[,Metadata]) + alert(Fun,FunArgs[,Metadata]) + Logs the given message as level alert. + +

Equivalent to + log(alert,...).

+ + + + + critical(StringOrReport[,Metadata]) + critical(Format,Args[,Metadata]) + critical(Fun,FunArgs[,Metadata]) + Logs the given message as level critical. + +

Equivalent to + log(critical,...).

+ + + + + error(StringOrReport[,Metadata]) + error(Format,Args[,Metadata]) + error(Fun,FunArgs[,Metadata]) + Logs the given message as level error. + +

Equivalent to + log(error,...).

+ + + + + warning(StringOrReport[,Metadata]) + warning(Format,Args[,Metadata]) + warning(Fun,FunArgs[,Metadata]) + Logs the given message as level warning. + +

Equivalent to + log(warning,...).

+ + + + + notice(StringOrReport[,Metadata]) + notice(Format,Args[,Metadata]) + notice(Fun,FunArgs[,Metadata]) + Logs the given message as level notice. + +

Equivalent to + log(notice,...).

+ + + + + info(StringOrReport[,Metadata]) + info(Format,Args[,Metadata]) + info(Fun,FunArgs[,Metadata]) + Logs the given message as level info. + +

Equivalent to + log(info,...).

+ + + + + debug(StringOrReport[,Metadata]) + debug(Format,Args[,Metadata]) + debug(Fun,FunArgs[,Metadata]) + Logs the given message as level debug. + +

Equivalent to + log(debug,...).

+ + + + + + + + + + + Logs the given message. + + + + + + + + +

Log the given message.

+ + + + + + Lookup the current configuration for logger. + +

Lookup the current configuration for logger.

+ + + + + + Lookup the current configuration for the given handler. + +

Lookup the current configuration for the given handler.

+ + + + + + Get information about all logger configurations + +

Same as logger:i(term)

+ + + + + + + + Get information about all logger configurations + +

The logger:i/1 function can be used to get all + current logger configuration. The way that the information + is returned depends on the Action

+ + string + Return the pretty printed current logger configuration + as iodata. + term + Return the current logger configuration as a term. The + format of this term may change inbetween releases. For a + stable format use + logger:get_handler_config/1 + and + logger:get_logger_config/0. + The same as calling logger:i(). + print + Pretty print all the current logger configuration to + standard out. Example: + logger:i(). +Current logger configuration: + Level: info + FilterDefault: log + Filters: + Handlers: + Id: logger_std_h + Module: logger_std_h + Level: info + Formatter: + Module: logger_formatter + Config: #{template => [{logger_formatter,header},"\n",msg,"\n"], + legacy_header => true} + Filter Default: stop + Filters: + Id: stop_progress + Fun: fun logger_filters:progress/2 + Config: stop + Id: remote_gl + Fun: fun logger_filters:remote_gl/2 + Config: stop + Id: domain + Fun: fun logger_filters:domain/2 + Config: {log,prefix_of,[beam,erlang,otp,sasl]} + Id: no_domain + Fun: fun logger_filters:domain/2 + Config: {log,no_domain,[]} + Handler Config: + logger_std_h: #{type => standard_io} + Level set per module: + Module: my_module + Level: debug]]> + + + + + + + + Add a filter to the logger. + +

Add a filter to the logger.

+ + + + + + Add a filter to the specified handler. + +

Add a filter to the specified handler.

+ + + + + + Remove a filter from the logger. + +

Remove the filter with the specified identity from the logger.

+ + + + + + Remove a filter from the specified handler. + +

Remove the filter with the specified identity from the given handler.

+ + + + + + Add a handler with the given configuration. + +

Add a handler with the given configuration.

+ + + + + + Remove the handler with the specified identity. + +

Remove the handler with the specified identity.

+ + + + + + Set the log level for the specified module. + +

Set the log level for the specified module.

+

To change the logging level globally, use + logger:set_logger_config(level, Level). +

+ + + + + + Remove a module specific log setting. + +

Remove a module specific log setting. After this, the + global log level is used for the specified module.

+ + + + + + + Add or update configuration data for the logger. + +

Add or update configuration data for the logger.

+ + + + + + + Add or update configuration data for the specified + handler. + +

Add or update configuration data for the specified + handler.

+ + + + + + Compare the severity of two log levels. + +

Compare the severity of two log levels. Returns gt + if Level1 is more severe than + Level2, lt if Level1 is less severe, + and eq if the levels are equal.

+ + + + + + Set metadata to use when logging from current process. + +

Set metadata which logger automatically inserts it + in all log events produced on the current + process. Subsequent calls will overwrite previous data set + by this function.

+

When logging, location data produced by the log macros, + and/or metadata given as argument to the log call (API + function or macro), will be merged with the process + metadata. If the same keys occur, values from the metadata + argument to the log call will overwrite values in the + process metadata, which in turn will overwrite values from + the location data.

+ + + + + + Retrieve data set with set_process_metadata/1. + +

Retrieve data set + with + set_process_metadata/1.

+ + + + + + Delete data set with set_process_metadata/1. + +

Delete data set + with + set_process_metadata/1.

+ + + + + + + + diff --git a/lib/kernel/doc/src/logger_arch.png b/lib/kernel/doc/src/logger_arch.png new file mode 100644 index 0000000000..727609a6ef Binary files /dev/null and b/lib/kernel/doc/src/logger_arch.png differ diff --git a/lib/kernel/doc/src/logger_chapter.xml b/lib/kernel/doc/src/logger_chapter.xml new file mode 100644 index 0000000000..2a325453da --- /dev/null +++ b/lib/kernel/doc/src/logger_chapter.xml @@ -0,0 +1,641 @@ + + + + +
+ + 2017 + Ericsson AB. All Rights Reserved. + + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + + + Logging + + + + + logger_chapter.xml +
+ +
+ Overview +

Erlang/OTP provides a standard API for logging. The backend of + this API can be used as is, or it can be customized to suite + specific needs.

+

It consists of two parts - the logger part and the + handler part. The logger will forward log events to one + or more handler(s).

+ + + Conceptual overview + + +

Filters can be added to the logger and to each + handler. The filters decide if an event is to be forwarded or + not, and they can also modify all parts of the log event.

+ +

A formatter can be set for each handler. The formatter + does the final formatting of the log event, including the log + message itself, and possibly a timestamp, header and other + metadata.

+ +

In accordance with the Syslog protocol, RFC-5424, eight + severity levels can be specified:

+ + + + Level + Integer + Description + + + emergency + 0 + system is unusable + + + alert + 1 + action must be taken immediately + + + critical + 2 + critical contidions + + + error + 3 + error conditions + + + warning + 4 + warning conditions + + + notice + 5 + normal but significant conditions + + + info + 6 + informational messages + + + debug + 7 + debug-level messages + + Severity levels +
+ +

A log event is allowed by Logger if the integer value of + its Level is less than or equal to the currently + configured log level. The log level can be configured globally, + or to allow more verbose logging from a specific part of the + system, per module.

+ +
+ Customizable parts + + + Handler + +

A handler is defined as a module exporting the following + function:

+ + log(Log, Config) -> ok + +

A handler is called by the logger backend after filtering on + logger level and on handler level for the handler which is + about to be called. The function call is done on the client + process, and it is up to the handler implementation if other + processes are to be involved or not.

+ +

Multiple instances of the same handler can be + added. Configuration is per instance.

+ + + + Filter + +

Filters can be set on the logger or on a handler. Logger + filters are applied first, and if passed, the handler filters + for each handler are applied. The handler plugin is only + called if all handler filters for the handler in question also + pass.

+ +

A filter is specified as:

+ + {fun((Log,Extra) -> Log | stop | ignore), Extra} + +

The configuration parameter filter_default + specifies the behavior if all filters return ignore. + filter_default is by default set to log.

+ +

The Extra parameter may contain any data that the + filter needs.

+ + + Formatter + +

A formatter is defined as a module exporting the following + function:

+ + format(Log,Extra) -> string() + +

The formatter plugin is called by each handler, and the + returned string can be printed to the handler's destination + (stdout, file, ...).

+ + + +
+ +
+ Built-in handlers + + + logger_std_h + +

This is the default handler used by OTP. Multiple instances + can be started, and each instance will write log events to a + given destination, console or file. Filters can be used for + selecting which event to send to which handler instance.

+ + + logger_disk_log_h + +

This handler behaves much like logger_std_h, except it uses + disk_log as its + destination.

+ + + error_logger + +

This handler is to be used for backwards compatibility + only. It is not started by default, but will be automatically + started the first time an event handler is added + with + error_logger:add_report_handler/1,2.

+ +

No built-in event handlers exist.

+ + +
+ +
+ Built-in filters + + + logger_filters:domain/2 + +

This filter provides a way of filtering log events based on a + domain field Metadata. See + + logger_filters:domain/2

+ + + logger_filters:level/2 + +

This filter provides a way of filtering log events based + on the log level. See + logger_filters:domain/2

+ + + logger_filters:progress/2 + +

This filter matches all progress reports + from supervisor and application_controller. + See + logger_filters:progress/2

+ + + logger_filters:remote_gl/2 + +

This filter matches all events originating from a process + that has its group leader on a remote node. + See + logger_filters:remote_gl/2

+ + +
+ +
+ Default formatter + +

The default formatter is logger_formatter. + See + logger_formatter:format/2.

+
+
+ +
+ Configuration + +
+ Application environment variables +

See Kernel(6) for + information about the application environment variables that can + be used for configuring logger.

+
+ +
+ Logger configuration + + + level + +

Specifies the severity level to log.

+ + filters + +

Logger filters are added or removed with + + logger:add_logger_filter/2 and + + logger:remove_logger_filter/1, + respectively.

+

See Filter for more + information.

+

By default, no filters exist.

+ + filter_default = log | stop + +

Specifies what to do with an event if all filters + return ignore.

+

Default is log.

+ + handlers + +

Handlers are added or removed with + + logger:add_handler/3 and + + logger:remove_handler/1, + respectively.

+

See Handler for more + information.

+ + +
+ +
+ Handler configuration + + level + +

Specifies the severity level to log.

+ + filters + +

Handler filters can be specified when adding the handler, + or added or removed later with + + logger:add_handler_filter/3 and + + logger:remove_handler_filter/2, + respectively.

+

See Filter for more + information.

+

By default, no filters exist.

+ + filter_default = log | stop + +

Specifies what to do with an event if all filters + return ignore.

+

Default is log.

+ + depth = pos_integer() | unlimited + +

Specifies if the depth of terms in the log events shall + be limited by using control characters ~P + and ~W instead of ~p and ~w, + respectively. See + io:format.

+ + max_size = pos_integer() | unlimited + +

Specifies if the size of a log event shall be limited by + truncating the formatted string.

+ + formatter = {Module::module(),Extra::term()} + +

See Formatter for more + information.

+

The default module is + logger_formatter, and Extra is + it's configuration map.

+ + + +

Note that level and filters are obeyed by + Logger itself before forwarding the log events to each + handler, while depth, max_size + and formatter are left to the handler + implementation. All Logger's built-in handlers do apply these + configuration parameters before printing.

+
+ +
+ +
+ + Backwards compatibility with error_logger +

Logger provides backwards compatibility with the old + error_logger in the following ways:

+ + + Legacy event handlers + +

To use event handlers written for error_logger, just + add your event handler with

+ +error_logger:add_report_handler/1,2. + +

This will automatically start the error_logger + event manager, and add error_logger as a + handler to logger, with configuration

+ +#{level=>info, + filter_default=>log, + filters=>[]}. + +

Note that this handler will ignore events that do not + originate from the old error_logger API, or from + within OTP. This means that if your code uses the logger API + for logging, then your log events will be discarded by this + handler.

+

Also note that error_logger is not overload + protected.

+ + Logger API + +

The old error_logger API still exists, but should + only be used by legacy code. It will be removed in a later + release.

+ + Output format + +

To get log events on the same format as produced + by error_logger_tty_h and error_logger_file_h, + use the default formatter, logger_formatter, with + configuration parameter legacy_header=>true. This is + also the default.

+ + Default format of log events from OTP + +

By default, all log events originating from within OTP, + except the former so called "SASL reports", look the same as + before.

+ + SASL reports + +

By SASL reports we mean supervisor reports, crash reports + and progress reports.

+

In earlier releases, these reports were only logged when + the SASL application was running, and they were printed + trough specific event handlers + named sasl_report_tty_h + and sasl_report_file_h.

+

The destination of these log events were configured by + environment variables for the SASL application.

+

Due to the specific event handlers, the output format + slightly differed from other log events.

+

As of OTP-21, the concept of SASL reports is removed, + meaning that the default behavior is as follows:

+ + Supervisor reports, crash reports and progress reports + are no longer connected to the SASL application. + Supervisor reports and crash reports are logged by + default. + Progress reports are not logged by default, but can be + enabled with the kernel environment + variable logger_log_progress. + The output format is the same for all log + events. + +

If the old behavior is preferred, the kernel environment + variable logger_sasl_compatible can be set + to true. The old SASL environment variables can then + be used as before, and the SASL reports will only be printed + if the SASL application is running - through a second log + handler named sasl_h.

+

All SASL reports have a metadata + field domain=>[beam,erlang,otp,sasl], which can be + used, for example, by filters to to stop or allow the + events.

+ + +
+ + +
+ Error handling +

Log data is expected to be either a format string and + arguments, a string (unicode:chardata), or a report (map or + key-value list) which can be converted to a format string and + arguments by the handler. A default report callback should be + included in the log event's metadata, which can be used for + converting the report to a format string and arguments. The + handler might also do a custom conversion if the default format + is not desired.

+

logger does, to a certain extent, check its input data + before forwarding a log event to the handlers, but it does not + evaluate conversion funs or check the validity of format strings + and arguments. This means that any filter or handler must be + careful when formatting the data of a log event, making sure + that it does not crash due to bad input data or faulty + callbacks.

+

If a filter or handler still crashes, logger will remove the + filter or handler in question from the configuration, and then + print a short error message on the console. A debug event + containing the crash reason and other details is also issued, + and can be seen if a handler is installed which logs on debug + level.

+
+ +
+ Example: add a handler to log debug events to file +

When starting an erlang node, the default behavior is that all + log events with level info and above are logged to the + console. In order to also log debug events, you can either + change the global log level to debug or add a separate + handler to take care of this. In this example we will add a new + handler which prints the debug events to a separate file.

+

First, we add an instance of logger_std_h with + type {file,File}, and we set the handler's level + to debug:

+ 
+1> Config = #{level=>debug,logger_std_h=>#{type=>{file,"./debug.log"}}}.
+#{logger_std_h => #{type => {file,"./debug.log"}},
+  level => debug}
+2> logger:add_handler(debug_handler,logger_std_h,Config).
+ok
+

By default, the handler receives all events, so we need to add a filter + to stop all non-debug events:

+ 
+3> Fun = fun(#{level:=debug}=Log,_) -> Log; (_,_) -> stop end.
+#Fun<erl_eval.12.98642416>
+4> logger:add_handler_filter(debug_handler,allow_debug,{Fun,[]}).
+ok
+

And finally, we need to make sure that the logger itself allows + debug events. This can either be done by setting the global + logger level:

+ 
+5> logger:set_logger_config(level,debug).
+ok
+

Or by allowing debug events from one or a few modules only:

+ 
+6> logger:set_module_level(mymodule,debug).
+ok
+ +
+ +
+ Example: implement a handler +

The only requirement that a handler MUST fulfill is to export + the following function:

+ log(logger:log(),logger:config()) ->ok +

It may also implement the following callbacks:

+ +adding_handler(logger:handler_id(),logger:config()) -> {ok,logger:config()} | {error,term()} +removing_handler(logger:handler_id()) -> ok +changing_config(logger:handler_id(),logger:config(),logger:config()) -> {ok,logger:config()} | {error,term()} + +

When logger:add_handler(Id,Module,Config) is called, logger + will first call Module:adding_handler(Id,Config), and if it + returns {ok,NewConfig} the NewConfig is written to the + configuration database. After this, the handler may receive log + events as calls to Module:log/2.

+

A handler can be removed by calling + logger:remove_handler(Id). logger will call + Module:removing_handler(Id), and then remove the handler's + configuration from the configuration database.

+

When logger:set_handler_config is called, logger calls + Module:changing_config(Id,OldConfig,NewConfig). If this function + returns ok, the NewConfig is written to the configuration + database.

+ +

A simple handler which prints to the console could be + implemented as follows:

+ +-module(myhandler). +-export([log/2]). + +log(#{msg:={report,R}},_) -> + io:format("~p~n",[R]); +log(#{msg:={string,S}},_) -> + io:put_chars(S); +log(#{msg:={F,A}},_) -> + io:format(F,A). + + +

A simple handler which prints to file could be implemented like + this:

+ +-module(myhandler). +-export([adding_handler/2, removing_handler/1, log/2]). +-export([init/1, handle_call/3, handle_cast/2, terminate/2]). + +adding_handler(Id,Config) -> + {ok,Fd} = file:open(File,[append,{encoding,utf8}]), + {ok,Config#{myhandler_fd=>Fd}}. + +removing_handler(Id,#{myhandler_fd:=Fd}) -> + _ = file:close(Fd), + ok. + +log(#{msg:={report,R}},#{myhandler_fd:=Fd}) -> + io:format(Fd,"~p~n",[R]); +log(#{msg:={string,S}},#{myhandler_fd:=Fd}) -> + io:put_chars(Fd,S); +log(#{msg:={F,A}},#{myhandler_fd:=Fd}) -> + io:format(Fd,F,A). + + +

Note that none of the above handlers have any overload + protection, and all log events are printed directly from the + client process. Neither do the handlers use the formatter or + in any way add time or other metadata to the printed events.

+ +

For examples of overload protection, please refer to the + implementation + of logger_std_h + and logger_disk_log_h + .

+ +

Below is a simpler example of a handler which logs through one + single process, and uses the default formatter to gain a common + look of the log events.

+

It also uses the metadata field report_cb, if it exists, + to print reports in the way the event issuer suggests. The + formatter will normally do this, but if the handler either has + an own default (as in this example) or if the + given report_cb should not be used at all, then the + handler must take care of this itself.

+ +-module(myhandler). +-export([adding_handler/2, removing_handler/1, log/2]). +-export([init/1, handle_call/3, handle_cast/2, terminate/2]). + +adding_handler(Id,Config) -> + {ok,Pid} = gen_server:start(?MODULE,Config), + {ok,Config#{myhandler_pid=>Pid}}. + +removing_handler(Id,#{myhandler_pid:=Pid}) -> + gen_server:stop(Pid). + +log(Log,#{myhandler_pid:=Pid} = Config) -> + gen_server:cast(Pid,{log,Log,Config}). + +init(#{myhandler_file:=File}) -> + {ok,Fd} = file:open(File,[append,{encoding,utf8}]), + {ok,#{file=>File,fd=>Fd}}. + +handle_call(_,_,State) -> + {reply,{error,bad_request},State}. + +handle_cast({log,Log,Config},#{fd:=Fd} = State) -> + do_log(Fd,Log,Config), + {noreply,State}. + +terminate(Reason,#{fd:=Fd}) -> + _ = file:close(Fd), + ok. + +do_log(Fd,#{msg:={report,R}} = Log, Config) -> + Fun = maps:get(report_cb,Config,fun my_report_cb/1, + {F,A} = Fun(R), + do_log(Fd,Log#{msg=>{F,A},Config); +do_log(Fd,Log,#{formatter:={FModule,FConfig}}) -> + String = FModule:format(Log,FConfig), + io:put_chars(Fd,String). + +my_report_cb(R) -> + {"~p",[R]}. + +
+ + +
+ See Also +

error_logger(3), + SASL(6)

+
+ diff --git a/lib/kernel/doc/src/logger_filters.xml b/lib/kernel/doc/src/logger_filters.xml new file mode 100644 index 0000000000..d742391e35 --- /dev/null +++ b/lib/kernel/doc/src/logger_filters.xml @@ -0,0 +1,191 @@ + + + + +
+ + 2018 + Ericsson AB. All Rights Reserved. + + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + + + logger_filters + + + + + + + A + logger_filters.xml +
+ logger_filters + Filters to use with logger. + + +

Filters to use with logger. All functions exported from this + module can be used as logger or handler + filters. See + logger:add_logger_filter/2 + and + logger:add_handler_filter/3 + for more information about how filters are added.

+ + + + + + Filter log events based on the domain field in metadata. + +

This filter provides a way of filtering log events based on a + domain field Metadata.

+ +

The Extra parameter is specified when + adding the filter + via + logger:add_logger_filter/2 + or + logger:add_handler_filter/3.

+ +

The filter compares the value of the domain field + in the log event's metadata (Domain) + to MatchDomain as follows:

+ + + Compare = starts_with +

The filter matches if MatchDomain is a prefix + of Domain.

 + Compare = prefix_of +

The filter matches if Domain is a prefix + of MatchDomain.

 + Compare = equals +

The filter matches if Domain is equal + to MatchDomain.

 + Compare = no_domain +

The filter matches if there is no domain field in + metadata. In this case MatchDomain shall + be [].

 + + +

If the filter matches and Action = + log, the log event is allowed. If the filter matches + and Action = stop, the log event is + stopped.

+ +

If the filter does not match, it returns ignore, + meaning that other filters, or the value of the + configuration parameter filter_default, will decide + if the event is allowed or not.

+ +

Log events that do not contain any domain field, will + only match when Compare = no_domain.

+ +

Example: stop all events with + domain [beam,erlang,otp,sasl|_]

+ + +logger:set_handler_config(h1,filter_default,log). % this is the default +Filter = {fun logger_filters:domain/2,{stop,starts_with,[beam,erlang,otp,sasl]}}. +logger:add_handler_filter(h1,no_sasl,Filter). +ok + + + + + + Filter log events based on the log level. + +

This filter provides a way of filtering log events based + on the log level. It matches log events by comparing the + log level with a predefined MatchLevel

+ +

The Extra parameter is specified when + adding the filter + via + logger:add_logger_filter/2 + or + logger:add_handler_filter/3.

+ +

The filter compares the value of the event's log level + (Level) to MatchLevel by + calling + logger:compare_levels(Level,MatchLevel) -> CmpRet. It + matches the event if:

+ + + CmpRet = eq and Operator = + eq | lteq | gteq + CmpRet = lt and Operator = + lt | lteq | neq + CmpRet = gt and Operator = + gt | gteq | neq + + +

If the filter matches and Action = + log, the log event is allowed. If the filter matches + and Action = stop, the log event is + stopped.

+ +

If the filter does not match, it returns ignore, + meaning that other filters, or the value of the + configuration parameter filter_default, will decide + if the event is allowed or not.

+ +

Example: only allow debug level log events

+ + +logger:set_handler_config(h1,filter_default,stop). +Filter = {fun logger_filters:level/2,{log,eq,debug}}. +logger:add_handler_filter(h1,debug_only,Filter). +ok + + + + + + Filter progress reports from supervisor and application_controller. + +

This filter matches all progress reports + from supervisor and application_controller.

+ +

If Extra = log, the progress reports + are allowed. If Extra = stop, the + progress reports are stopped.

+ +

The filter returns ignore for all other log events.

+ + + + + + Filter events with group leader on remote node. + +

This filter matches all events originating from a process + that has its group leader on a remote node.

+ +

If Extra = log, the matching events + are allowed. If Extra = stop, the + matching events are stopped.

+ +

The filter returns ignore for all other log events.

+ + + + + + + + diff --git a/lib/kernel/doc/src/logger_formatter.xml b/lib/kernel/doc/src/logger_formatter.xml new file mode 100644 index 0000000000..213a592e47 --- /dev/null +++ b/lib/kernel/doc/src/logger_formatter.xml @@ -0,0 +1,149 @@ + + + + +
+ + 2017 + Ericsson AB. All Rights Reserved. + + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + + + logger_formatter + + + + + + + A + logger_formatter.xml +
+ logger_formatter + Default formatter for the Logger application. + + +

Default formatter for the Logger application.

+ + + + + + + + + + + + + + Formats the given message. + +

Formats the given message.

+

The template is a list of atoms, tuples and strings. Atoms + can be level or msg, which are placeholders + for the severity level and the log message, + repectively. Tuples are interpreted as placeholders for + metadata. Each element in the tuple must be an atom which + matches a key in the nested metadata map, e.g. the + tuple {key1,key2} will be replaced by the value of + the key2 field in this nested map (the value vill be + converted to a string):

+ + +#{key1=>#{key2=>my_value, + ...}, + ...} + + +

Strings are printed literally.

+ +

depth is a positive integer representing the maximum + depth to which terms shall be printed by this + formatter. Format strings passed to this formatter are + rewritten. The format controls ~p and ~w are replaced with + ~P and ~W, respectively, and the value is used as the depth + parameter. For details, see + io:format/2 + in STDLIB.

+ +

max_size is a positive integer representing the + maximum size a string returned from this formatter can + have. If the formatted string is longer, it will be + truncated.

+ +

utc is a boolean. If set to true, all dates are + displayed in Universal Coordinated Time. Default + is false.

+ +

report_cb must be a function with arity 1, + returning {Format,Args}. This function will replace + any report_cb found in metadata.

+ +

If single_line=true, all newlines in the message are + replaced with ", ", and whitespaces following directly + after newlines are removed. Note that newlines added by the + formatter template are not replaced.

+ +

If legacy_header=true a header field is added to + logger_formatter's part of Metadata. The value of + this field is a string similar to the header created by the + old error_logger event handlers. It can be included + in the log event by adding the + tuple {logger_formatter,header} to the template.

+ +

The default template when legacy_header=true is

+ + [{logger_formatter,header},"\n",msg,"\n"] + +

which will cause log entries like this:

+ + =ERROR REPORT==== 29-Dec-2017::13:30:51.245123 === + process: <0.74.0> + exit_reason: "Something went wrong" + +

Note that all eight levels might occur here, not + only ERROR, WARNING or INFO. And also + that micro seconds are added at the end of the + timestamp.

+ +

The default template when single_line=true is

+ + [time," ",level,": ",msg,"\n"] + +

which will cause log entries like this:

+ + 2017-12-29 13:31:49.640317 error: process: <0.74.0>, exit_reason: "Something went wrong" + +

The default template when both legacy_header and + single_line are set to false is:

+ + [time," ",level,":\n",msg,"\n"] + +

which will cause log entries like this:

+ + 2017-12-29 13:32:25.191925 error: + process: <0.74.0> + exit_reason: "Something went wrong" + + + + + + + + + diff --git a/lib/kernel/doc/src/logger_std_h.xml b/lib/kernel/doc/src/logger_std_h.xml new file mode 100644 index 0000000000..2a368b16eb --- /dev/null +++ b/lib/kernel/doc/src/logger_std_h.xml @@ -0,0 +1,81 @@ + + + + +
+ + 2017 + Ericsson AB. All Rights Reserved. + + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + + + logger_std_h + + + + + + + A + logger_std_h.xml +
+ logger_std_h + Default handler for the Logger application. + + +

This is the default handler for the Logger + application. Multiple instances of this handler can be added to + logger, and each instance will print logs to standard_io, + standard_error or to a file.

+ +

To add a new instance, + use logger:add_handler/3 + .

+ +

The handler configuration may contain the following keys, + associated with values as described:

+ + filters + +

A list of {Id,{Fun,Args}}, each representing a filter + that may selct or modify log events to forward to this + handler.

 + filter_default + +

The atom log or stop, specifying what to + do with a log event if all filters + return ignore.

 + formatter + +

{Module,Extra}, + where Module:format(Log,Extra) will be called by + the handler to produce the string that will be printed to + the handler's destination.

 + level + +

The level of log events that logger shall forward to + this handler. Log events of the specified, or more severe + levels, are forwarded.

 + + + + + + + + + diff --git a/lib/kernel/doc/src/part.xml b/lib/kernel/doc/src/part.xml new file mode 100644 index 0000000000..68eb4530e2 --- /dev/null +++ b/lib/kernel/doc/src/part.xml @@ -0,0 +1,40 @@ + + + + +
+ + 19962017 + Ericsson AB. All Rights Reserved. + + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + + + Logger User's Guide + OTP Team + + 2017-12-01 + 0.1 + part.xml +
+ +

The System Architecture Support Libraries SASL application + provides support for alarm handling, release handling, and + related functions.

+ + + + + diff --git a/lib/kernel/doc/src/ref_man.xml b/lib/kernel/doc/src/ref_man.xml index 5cd77e0f6f..7f4d5b384f 100644 --- a/lib/kernel/doc/src/ref_man.xml +++ b/lib/kernel/doc/src/ref_man.xml @@ -52,6 +52,10 @@ + + + + diff --git a/lib/kernel/doc/src/specs.xml b/lib/kernel/doc/src/specs.xml index 29d52f23bb..a67e26ef16 100644 --- a/lib/kernel/doc/src/specs.xml +++ b/lib/kernel/doc/src/specs.xml @@ -20,6 +20,10 @@ + + + + -- cgit v1.2.3 From 05e1548e5b452d4bfe3bc6837a1a1b51a6367f72 Mon Sep 17 00:00:00 2001 From: Siri Hansen Date: Tue, 24 Apr 2018 14:55:29 +0200 Subject: Remove error_logger process and add logger process --- lib/kernel/doc/src/error_logger.xml | 19 ++++- lib/kernel/doc/src/kernel_app.xml | 154 ++++++++++++++++++++++++++++++------ 2 files changed, 147 insertions(+), 26 deletions(-) (limited to 'lib/kernel/doc/src') diff --git a/lib/kernel/doc/src/error_logger.xml b/lib/kernel/doc/src/error_logger.xml index 91bf57cb91..cb6165c73e 100644 --- a/lib/kernel/doc/src/error_logger.xml +++ b/lib/kernel/doc/src/error_logger.xml @@ -31,6 +31,16 @@ error_logger Erlang error logger. + + +

In OTP-21, a new API for logging was added to Erlang/OTP. The + old error_logger module can still be used by legacy + code, but new code should use the new API instead.

+

See logger(3) and + the Logging chapter + in the user's guide for more information.

+ +

The Erlang error logger is an event manager (see OTP Design Principles and gen_event(3)), @@ -171,14 +181,17 @@ ok Get the value of the Kernel application variable - error_logger_format_depth. + logger_format_depth.

Returns max(10, Depth), where Depth is the value of - - error_logger_format_depth + + logger_format_depth in the Kernel application, if Depth is an integer. Otherwise, unlimited is returned.

+

For backwards compatibility, the value + of error_logger_format_depth is used + if logger_format_depth is not set.

diff --git a/lib/kernel/doc/src/kernel_app.xml b/lib/kernel/doc/src/kernel_app.xml index 0762cebc94..554d675383 100644 --- a/lib/kernel/doc/src/kernel_app.xml +++ b/lib/kernel/doc/src/kernel_app.xml @@ -51,10 +51,13 @@
- Error Logger Event Handlers -

Two standard error logger event handlers are defined in - the Kernel application. These are described in - error_logger(3).

+ Logger Handlers +

Two standard logger handlers are defined in + the Kernel application. These are described in the + Kernel User's Guide, + and in logger_std_h(3) + and logger_disk_log_h(3) + .

@@ -113,6 +116,7 @@
+ Configuration

The following configuration parameters are defined for the Kernel application. For more information about configuration parameters, @@ -176,34 +180,105 @@

Permissions are described in application:permit/2.

- error_logger = Value + logger_dest = Value

Value is one of:

tty -

Installs the standard event handler, which prints error - reports to stdio. This is the default option.

 +

Installs the standard handler, + logger_std_h(3), with type set + to standard_io. This is the default + option.

 {file, FileName} -

Installs the standard event handler, which prints error - reports to file FileName, where FileName +

Installs the standard handler, + logger_std_h(3), with type set + to {file, FileName}, where FileName is a string. The file is opened with encoding UTF-8.

 + {disk_log, FileName} +

Installs the disk_log handler, + logger_disk_log_h(3), with file set + to FileName (a string), and possibly other disk_log + parameters set by the environment variables + logger_disk_log_type, logger_disk_log_maxfiles and + logger_disk_log_maxbytes, + see below. The + file is opened with encoding UTF-8.

 false -

No standard event handler is installed, but - the initial, primitive event handler is kept, printing +

No standard handler is installed, but + the initial, primitive handler is kept, printing raw event messages to tty.

 silent -

Error logging is turned off.

+

No standard handler is started, and the initial, + primitive handler is removed.

 - error_logger_format_depth = Depth + logger_level = Level + +

Value = emergency | alert | critical | error | warning | + notice | info | debug

+

This parameter specifies which log levels to log. The + specified level, and all levels that are more severe, will + be logged.

+

This configuration parameter is used both for the global + logger level, and for the standard handler started by + the Kernel application (see logger_dest variable above).

+

The default value is info

+ + + logger_disk_log_type = halt | wrap + + logger_disk_log_maxfiles = integer() + + logger_disk_log_maxbytes = integer() + +

If logger_dest is set to {disk_log,File}, then these + parameters specify the configuration to use when opening the + disk log file. They specify the type of disk log, the + maximum number of files (if the type is wrap) and the + maximum size of each file, respectively.

+

The default values are:

+ +logger_disk_log_type = wrap +logger_disk_log_maxfiles = 10 +logger_disk_log_maxbytes = 1048576 + + + logger_sasl_compatible = boolean() + +

If this parameter is set to true, then the logger handler + started by kernel will not log any progress-, crash-, or + supervisor reports. If the SASL application is starated, + these log events will be sent to a second handler instance + named sasl_h, according to values of the SASL environment + variables sasl_error_logger + and sasl_errlog_type, see + SASL(6) +

+

The default value is false

+

See chapter Backwards + compatibility with error_logger for more + information about handling of the so called SASL reports.

+ + + logger_log_progress = boolean() + +

If logger_sasl_compatible = false, + then logger_log_progress specifies if progress + reports from supervisor + and application_controller shall be logged or + not.

+

If logger_sasl_compatible = false, + then logger_log_progress is ignored.

+ + + logger_format_depth = Depth -

Can be used to limit the size of the - formatted output from the error logger event handlers.

+ formatted output from the logger handlers.

This configuration parameter was introduced in OTP 18.1 and is experimental. Based on user feedback, it @@ -214,16 +289,16 @@ useless.

Depth is a positive integer representing the maximum - depth to which terms are printed by the error logger event + depth to which terms are printed by the logger handlers included in OTP. This - configuration parameter is used by the two event handlers - defined by the Kernel application and the two event - handlers in the SASL application. - (If you have implemented your own error handlers, this configuration - parameter has no effect on them.)

+ configuration parameter is used by the default formatter, + logger_formatter(3), + unless the formatter's depth parameter is explicitly set. + (If you have implemented your own formatter, this configuration + parameter has no effect on that.)

Depth is used as follows: Format strings - passed to the event handlers are rewritten. + received by the formatter are rewritten. The format controls ~p and ~w are replaced with ~P and ~W, respectively, and Depth is used as the depth parameter. For details, see @@ -234,7 +309,20 @@ 30. We recommend to test crashing various processes in your application, examine the logs from the crashes, and then increase or decrease the value.

- + + logger_max_size = integer() | unlimited + +

This parameter specifies the maximum size (bytes) each + log event can have when printed by the standard logger + handler. If the resulting string after formatting an event + is bigger than this, it will be truncated before printed + to the handler's destination.

+ + logger_utc = boolean() + +

If set to true, the default formatter will display + all dates in Universal Coordinated Time.

+ global_groups = [GroupTuple] @@ -496,6 +584,26 @@ MaxT = TickTime + TickTime / 4
+
+ Deprecated Configuration Parameters +

In OTP-21, a new API for logging was added to Erlang/OTP. The + old error_logger event manager, and event handlers + running on this manager, will still work, but they are not used + by default.

+

The following application environment variables can still be + set, but they will only be used if the corresponding new logger + variables are not set.

+ + error_logger + Replaced by logger_dest + error_logger_format_depth + Replaced by logger_format_depth + +

See Backwards + compatibility with error_logger for more + information.

+
+
See Also

app(4), -- cgit v1.2.3 From 992b57ad0a52f7a3ba945cbfaea53ce3ae3c3f5d Mon Sep 17 00:00:00 2001 From: Siri Hansen Date: Wed, 25 Apr 2018 18:49:09 +0200 Subject: Add chars_limit option to logger_formatter --- lib/kernel/doc/src/logger_formatter.xml | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) (limited to 'lib/kernel/doc/src') diff --git a/lib/kernel/doc/src/logger_formatter.xml b/lib/kernel/doc/src/logger_formatter.xml index 213a592e47..6a17e3641f 100644 --- a/lib/kernel/doc/src/logger_formatter.xml +++ b/lib/kernel/doc/src/logger_formatter.xml @@ -77,13 +77,21 @@ rewritten. The format controls ~p and ~w are replaced with ~P and ~W, respectively, and the value is used as the depth parameter. For details, see - io:format/2 + io:format/2,3 in STDLIB.

+

chars_limit is a positive integer representing the + value of the option with the same name to be used when calling + io:format/3. This + value limits the total number of characters printed bu the + formatter. Notes that this is a soft limit. For a hard + truncation limit, see option max_size.

+

max_size is a positive integer representing the maximum size a string returned from this formatter can - have. If the formatted string is longer, it will be - truncated.

+ have. If the formatted string is longer, after possibly + being limited by depth and/or chars_limit, it + will be truncated.

utc is a boolean. If set to true, all dates are displayed in Universal Coordinated Time. Default -- cgit v1.2.3 From ee44197422710dabe43f2f4b80d1034aae9c916f Mon Sep 17 00:00:00 2001 From: Peter Andersson Date: Wed, 25 Apr 2018 21:18:08 +0200 Subject: Add documentation of the built-in logger handlers --- lib/kernel/doc/src/Makefile | 1 + lib/kernel/doc/src/logger_chapter.xml | 178 ++++++++++++++++++++++++++++++- lib/kernel/doc/src/logger_disk_log_h.xml | 146 +++++++++++++++++++++++++ lib/kernel/doc/src/logger_std_h.xml | 110 ++++++++++++++----- lib/kernel/doc/src/ref_man.xml | 1 + lib/kernel/doc/src/specs.xml | 1 + 6 files changed, 406 insertions(+), 31 deletions(-) create mode 100644 lib/kernel/doc/src/logger_disk_log_h.xml (limited to 'lib/kernel/doc/src') diff --git a/lib/kernel/doc/src/Makefile b/lib/kernel/doc/src/Makefile index 10bdf46119..82869d7b15 100644 --- a/lib/kernel/doc/src/Makefile +++ b/lib/kernel/doc/src/Makefile @@ -58,6 +58,7 @@ XML_REF3_FILES = application.xml \ init_stub.xml \ logger.xml \ logger_std_h.xml \ + logger_disk_log_h.xml \ logger_filters.xml \ logger_formatter.xml \ net_adm.xml \ diff --git a/lib/kernel/doc/src/logger_chapter.xml b/lib/kernel/doc/src/logger_chapter.xml index 2a325453da..0bc3b37476 100644 --- a/lib/kernel/doc/src/logger_chapter.xml +++ b/lib/kernel/doc/src/logger_chapter.xml @@ -296,6 +296,7 @@

+ Handler configuration level @@ -577,8 +578,8 @@ log(#{msg:={F,A}},#{myhandler_fd:=Fd}) ->

For examples of overload protection, please refer to the implementation of logger_std_h - and logger_disk_log_h - .

+ and logger_disk_log_h + .

Below is a simpler example of a handler which logs through one single process, and uses the default formatter to gain a common @@ -632,6 +633,179 @@ my_report_cb(R) ->

+
+ + Protecting the handler from overload +

In order for the built-in handlers to survive, and stay responsive, + during periods of high load (i.e. when huge numbers of incoming + log requests must be handled), a mechanism for overload protection + has been implemented in the + logger_std_h + and logger_disk_log_h + handler. The mechanism, used by both handlers, works + as follows:

+ +
+ Message queue length +

The handler process keeps track of the length of its message + queue and reacts in different ways depending on the current status. + The purpose is to keep the handler in, or (as quickly as possible), + get the handler into, a state where it can keep up with the pace + of incoming log requests. The memory usage of the handler must never + keep growing larger and larger, since that would eventually cause the + handler to crash. Three thresholds with associated actions have been + defined:

+ + + toggle_sync_qlen + +

The default value of this level is 10 messages, + and as long as the length of the message queue is lower, all log + requests are handled asynchronously. This simply means that the + process sending the log request (by calling a log function in the + logger API) does not wait for a response from the handler but + continues executing immediately after the request (i.e. it will not + be affected by the time it takes the handler to print to the log + device). If the message queue grows larger than this value, however, + the handler starts handling the log requests synchronously instead, + meaning the process sending the request will have to wait for a + response. When the handler manages to reduce the message queue to a + level below the toggle_sync_qlen threshold, asynchronous + operation is resumed. The switch from asynchronous to synchronous + mode will force the logging tempo of few busy senders to slow down, + but can not protect the handler sufficiently in situations of many + concurrent senders.

+ + drop_new_reqs_qlen + +

When the message queue has grown larger than this threshold, which + defaults to 200 messages, the handler switches to a mode in + which it drops any new requests being made. Dropping a message in + this state means that the log function never actually sends a message + to the handler. The log call simply returns without an action. When + the length of the message queue has been reduced to a level below this + threshold, synchronous or asynchronous request handling mode is + resumed.

+ + flush_reqs_qlen + +

Above this threshold, which defaults to 1000 messages, a + flush operation takes place, in which all messages buffered in the + process mailbox get deleted without any logging actually taking + place. (Processes waiting for a response from a synchronous log request + will receive a reply indicating that the request has been dropped).

+ + + +

For the overload protection algorithm to work properly, it is a + requirement that:

+ +

toggle_sync_qlen < drop_new_reqs_qlen < flush_reqs_qlen

+ +

During high load scenarios, the length of the handler message queue + rarely grows in a linear and predictable way. Instead, whenever the + handler process gets scheduled in, it can have an almost arbitrary number + of messages waiting in the mailbox. It's for this reason that the overload + protection mechanism is focused on acting quickly and quite drastically + (such as immediately dropping or flushing messages) as soon as a large + queue length is detected.

+ +

The thresholds listed above may be modified by the user if, e.g, a handler + shouldn't drop or flush messages unless the message queue length grows + extremely large. (The handler must be allowed to use large amounts of memory + under such circumstances however). Another example of when the user might want + to change the settings is if, for performance reasons, the logging processes must + never get blocked by synchronous log requests, while dropping or flushing requests + is perfectly acceptable (since it doesn't affect the performance of the + loggers).

+ +

A configuration example:

+ +logger:add_handler(my_standard_h, logger_std_h, + #{logger_std_h => + #{type => {file,"./system_info.log"}, + toggle_sync_qlen => 100, + drop_new_reqs_qlen => 1000, + flush_reqs_qlen => 2000}}). + +
+ +
+ Controlling bursts of log requests +

A potential problem with large bursts of log requests, is that log files + may get full or wrapped too quickly (in the latter case overwriting + previously logged data that could be of great importance). For this reason, + both built-in handlers offer the possibility to set a maximum level of how + many requests to process with a certain time frame. With this burst control + feature enabled, the handler will take care of bursts of log requests + without choking log files, or the console, with massive amounts of + printouts. These are the configuration parameters:

+ + + enable_burst_limit + +

This is set to true by default. The value false + disables the burst control feature.

+ + burst_limit_size + +

This is how many requests should be processed within the + burst_window_time time frame. After this maximum has been + reached, successive requests will be dropped until the end of the + time frame. The default value is 500 messages.

+ + burst_window_time + +

The default window is 1000 milliseconds long.

+ + + +

A configuration example:

+ +logger:add_handler(my_disk_log_h, logger_disk_log_h, + #{disk_log_opts => + #{file => "./my_disk_log"}, + logger_disk_log_h => + #{burst_limit_size => 10, + burst_window_time => 500}}). + +
+ +
+ Terminating a large handler +

A handler process may grow large even if it can manage peaks of high load + without crashing. The overload protection mechanism includes user configurable + levels for a maximum allowed message queue length and maximum allowed memory + usage. This feature is disabled by default, but can be switched on by means + of the following configuration parameters:

+ + + enable_kill_overloaded + +

This is set to false by default. The value true + enables the feature.

+ + handler_overloaded_qlen + +

This is the maximum allowed queue length. If the mailbox grows larger + than this, the handler process gets terminated.

+ + handler_overloaded_mem + +

This is the maximum allowed memory usage of the handler process. If + the handler grows any larger, the process gets terminated.

+ + handler_restart_after + +

If the handler gets terminated because of its queue length or + memory usage, it can get automatically restarted again after a + configurable delay time. The time is specified in milliseconds + and 5000 is the default value. The value never can + also be set, which prevents a restart.

+ + +
+
See Also diff --git a/lib/kernel/doc/src/logger_disk_log_h.xml b/lib/kernel/doc/src/logger_disk_log_h.xml new file mode 100644 index 0000000000..90cc4fec30 --- /dev/null +++ b/lib/kernel/doc/src/logger_disk_log_h.xml @@ -0,0 +1,146 @@ + + + + +
+ + 2017 + Ericsson AB. All Rights Reserved. + + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + + + logger_disk_log_h + + + + + + + A + logger_disk_log_h.xml +
+ logger_disk_log_h + A disk_log based handler for the Logger + application. + + +

This is a handler for the Logger application that offers circular + (wrapped) logs by using the disk_log application. Multiple instances + of this handler can be added to logger, and each instance will print to + its own disk_log file, created with the name and settings specified in + the handler configuration.

+

The default standard handler, + logger_std_h, can be + replaced by a disk_log handler at startup of the kernel application. + See an example of this below.

+

The handler has an overload protection mechanism that will keep the handler + process and the kernel application alive during a high load of log + requests. How this feature works, and how to modify the configuration, + is described in the + User's Guide + .

+

To add a new instance of the disk_log handler, use + logger:add_handler/3 + . The handler configuration argument is a map which may contain + general configuration parameters, as documented in the + User's Guide + , as well as handler specific parameters.

+

The settings for the disk_log log file should be specified with the + key disk_log_opts. These settings are a subset of the disk_log + datatype + dlog_option().

+

Parameters in the disk_log_opts map:

+ + file + This is the full name of the disk_log log file. + type + This is the disk_log type, wrap or halt. The + default value is wrap. + max_no_files + This is the maximum number of files that disk_log will use + for its circular logging. The default value is 10. (The setting + has no effect on a halt log). + max_no_bytes + This is the maximum number of bytes that will be written to + a log file before disk_log proceeds with the next file in order (or + generates an error in case of a full halt log). The default value for + a wrap log is 1048576 bytes, and infinity for a halt + log. + +

Specific configuration for the handler (represented as a sub map) + is specified with the key logger_disk_log_h. It may contain the + following parameter:

+ + filesync_repeat_interval + +

This value (in milliseconds) specifies how often the handler will + do a disk_log sync operation in order to make sure that buffered data + gets written to disk. The handler will repeatedly attempt this + operation, but only perform it if something has actually been logged + since the last sync. The default value is 5000 milliseconds. + If no_repeat is set as value, the repeated sync operation is + disabled. The user can also call the + disk_log_sync/1 + function to perform a disk_log sync.

 + +

There are a number of other configuration parameters available, that are + to be used for customizing the overload protection behaviour. The same + parameters are used both in the standard handler and the disk_log handler, + and are documented in the + User's Guide + .

+

Note that when changing the configuration of the handler in runtime, by + calling + logger:set_handler_config/2 + or logger:set_handler_config/3, the disk_log_opts + settings may not be modified.

+

Example of adding a disk_log handler:

+ +logger:add_handler(my_disk_log_h, logger_disk_log_h, + #{level => error, + filter_default => log, + disk_log_opts => + #{file => "./my_disk_log", + type => wrap, + max_no_files => 4, + max_no_bytes => 10000}, + logger_disk_log_h => + #{filesync_repeat_interval => 1000}}). + +

In order to use the disk_log handler instead of the default standard + handler when starting en Erlang node, use the kernel configuration parameter + logger_dest with + value {disk_log,FileName}. Example:

+ +erl -kernel logger_dest '{disk_log,"./system_disk_log"}' + + + + + + + + Writes buffered data to disk. + +

Write buffered data to disk.

+ + + + + + + + diff --git a/lib/kernel/doc/src/logger_std_h.xml b/lib/kernel/doc/src/logger_std_h.xml index 2a368b16eb..fe9b9ca5a9 100644 --- a/lib/kernel/doc/src/logger_std_h.xml +++ b/lib/kernel/doc/src/logger_std_h.xml @@ -39,42 +39,94 @@

This is the default handler for the Logger application. Multiple instances of this handler can be added to logger, and each instance will print logs to standard_io, - standard_error or to a file.

- -

To add a new instance, - use logger:add_handler/3 + standard_error or to file. The default instance that starts + with kernel is named logger_std_h - which is the name to be used + for reconfiguration.

+

The handler has an overload protection mechanism that will keep the handler + process and the kernel application alive during a high load of log + requests. How this feature works, and how to modify the configuration, + is described in the + User's Guide .

- -

The handler configuration may contain the following keys, - associated with values as described:

+

To add a new instance of the standard handler, use + logger:add_handler/3 + . The handler configuration argument is a map which may contain + general configuration parameters, as documented in the + User's Guide + , as well as handler specific parameters. The specific parameters + are stored in a sub map with the key logger_std_h. The following + keys and values may be specified:

- filters - -

A list of {Id,{Fun,Args}}, each representing a filter - that may selct or modify log events to forward to this - handler.

 - filter_default - -

The atom log or stop, specifying what to - do with a log event if all filters - return ignore.

 - formatter + type -

{Module,Extra}, - where Module:format(Log,Extra) will be called by - the handler to produce the string that will be printed to - the handler's destination.

 - level +

This will have the value standard_io, standard_error, + {file,LogFileName}, or {file,LogFileName,LogFileOpts}, + where standard_io is the default value for type. It's recommended + to not specify LogFileOpts if not absolutely necessary. The + default options used by the handler to open a file for logging are: + raw, append and delayed_write. The standard + handler does not have support for circular logging. Use the + logger_disk_log_h + handler for this.

+ filesync_repeat_interval -

The level of log events that logger shall forward to - this handler. Log events of the specified, or more severe - levels, are forwarded.

 +

This value (in milliseconds) specifies how often the handler will + do a file sync operation in order to make sure that buffered data gets + written to disk. The handler will repeatedly attempt this + operation, but only perform it if something has actually been logged + since the last sync. The default value is 5000 milliseconds. + If no_repeat is set as value, the repeated file sync operation + is disabled, and it will be the operating system settings that determine + how quickly or slowly data gets written to disk. The user can also call + the filesync/1 + function to perform a file sync.

 - +

There are a number of other configuration parameters available, that are + to be used for customizing the overload protection behaviour. The same + parameters are used both in the standard handler and the disk_log handler, + and are documented in the + User's Guide + .

+

Note that when changing the configuration of the handler in runtime, by + calling + logger:set_handler_config/2 + , or + logger:set_handler_config/3 + , + the type parameter may not be modified.

+

Example of adding a standard handler:

+ +logger:add_handler(my_standard_h, logger_std_h, + #{level => info, + filter_default => log, + logger_std_h => + #{type => {file,"./system_info.log"}, + filesync_repeat_interval => 1000}}). + +

In order to configure the default handler (that starts initially with + the kernel application) to log to file instead of standard_io, + use the kernel configuration parameter + logger_dest with + value {file,FileName}. Example:

+ +erl -kernel logger_dest '{file,"./erl.log"}' + +

An example of how to replace the standard handler with a disk_log handler + at startup can be found in the manual of + logger_disk_log_h.

- + + + + + Writes buffered data to disk. + +

Write buffered data to disk.

+ + + + diff --git a/lib/kernel/doc/src/ref_man.xml b/lib/kernel/doc/src/ref_man.xml index 7f4d5b384f..c06914d23d 100644 --- a/lib/kernel/doc/src/ref_man.xml +++ b/lib/kernel/doc/src/ref_man.xml @@ -56,6 +56,7 @@ + diff --git a/lib/kernel/doc/src/specs.xml b/lib/kernel/doc/src/specs.xml index a67e26ef16..bcc422930e 100644 --- a/lib/kernel/doc/src/specs.xml +++ b/lib/kernel/doc/src/specs.xml @@ -24,6 +24,7 @@ + -- cgit v1.2.3