From e03c2bd640e1fc1503e92dfd74991f4205973c61 Mon Sep 17 00:00:00 2001 From: Siri Hansen Date: Tue, 5 Mar 2019 11:25:00 +0100 Subject: [logger] Add better control of file modes in logger_std_h OTP-15602 It is allowed to set file modes for the handler to use when opening its log file. The given modes were earlier accepted without any checks, which could make the handler behave unexpectedly. This commit makes sure that * if none of write, append or exclusive is given, then append is added * if raw is not given, it is added * if delayed_write or {delayed_write,_,_} is not given, then delayed_write is added --- lib/kernel/doc/src/logger_std_h.xml | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) (limited to 'lib/kernel/doc/src') diff --git a/lib/kernel/doc/src/logger_std_h.xml b/lib/kernel/doc/src/logger_std_h.xml index fcd180abd6..8c5a476d86 100644 --- a/lib/kernel/doc/src/logger_std_h.xml +++ b/lib/kernel/doc/src/logger_std_h.xml @@ -59,14 +59,21 @@

This has the value standard_io, standard_error, {file,LogFileName}, or {file,LogFileName,LogFileOpts}.

-

If LogFileOpts is specified, it replaces the default - list of options used when opening the log file. The default - list is [raw,append,delayed_write]. One reason to do - so can be to change append to, for - example, write, ensuring that the old log is - truncated when a node is restarted. See the reference manual - for file:open/2 - for more information about file options.

+

LogFileOpts specify the file options used when + opening the log file, + see file:open/2. + If LogFileOpts is not specified, the default option + list used is [raw,append,delayed_write]. If LogFileOpts + is specified, it replaces the default options list with the + following adjustments:

+ + + If raw is not found in the list, it is added + + + It write, append or exclusice are not + found in the list, append is added +

Log files are always UTF-8 encoded. The encoding can not be changed by setting the option {encoding,Encoding} in LogFileOpts.

-- cgit v1.2.3 From 56cacefda4f18c8fdc45839b6bec1d07d0d2d78e Mon Sep 17 00:00:00 2001 From: Siri Hansen Date: Thu, 28 Feb 2019 20:49:08 +0100 Subject: [logger] Add log file rotation by options to logger_std_h OTP-15479 OTP-15662 New configuration map for logger_std_h: #{type => file, file => file:filename(), modes => [file:mode()], max_no_bytes => pos_integer() | infinity, max_no_files => non_neg_integer(), compress_on_rotate => boolean()} For backwards compatibility, the old variant for specifying the file name via the 'type' parameter is still supported, i.e. {file,FileName} and {file,FileName,Modes}, but it is no longer documented. Rotation scheme: The current log file always has the same name, and the archived files get extensions ".0", ".1", ... The newest archive has extension ".0", and the oldest archive has the highest number. If 'compress_on_rotate' is set to true, the archived files are gzipped and get the additional extension ".gz", e.g. error.log.0.gz. Rotation is turned off by setting 'max_no_bytes' to infinity. Setting 'max_no_files' to 0 does not turn off rotation, but only specifies that no archives are to be saved. --- lib/kernel/doc/src/kernel_app.xml | 10 ++-- lib/kernel/doc/src/logger.xml | 2 +- lib/kernel/doc/src/logger_chapter.xml | 14 ++--- lib/kernel/doc/src/logger_std_h.xml | 96 ++++++++++++++++++++++++++--------- 4 files changed, 87 insertions(+), 35 deletions(-) (limited to 'lib/kernel/doc/src') diff --git a/lib/kernel/doc/src/kernel_app.xml b/lib/kernel/doc/src/kernel_app.xml index 15dbdb47dc..dbd83e1a6e 100644 --- a/lib/kernel/doc/src/kernel_app.xml +++ b/lib/kernel/doc/src/kernel_app.xml @@ -510,11 +510,13 @@ MaxT = TickTime + TickTime / 4 parameters for Logger are not set.

error_logger - Replaced by setting the type of the default - logger_std_h - to the same value. Example: + Replaced by setting the type, and possibly + file and + modes + parameters of the default logger_std_h handler. Example: -erl -kernel logger '[{handler,default,logger_std_h,#{config=>#{type=>{file,"/tmp/erlang.log"}}}}]' +erl -kernel logger '[{handler,default,logger_std_h,#{config=>#{file=>"/tmp/erlang.log"}}}]' error_logger_format_depth diff --git a/lib/kernel/doc/src/logger.xml b/lib/kernel/doc/src/logger.xml index df2d081d76..50b1419783 100644 --- a/lib/kernel/doc/src/logger.xml +++ b/lib/kernel/doc/src/logger.xml @@ -66,7 +66,7 @@ logger:error("error happened because: ~p", [Reason]). % Without macro [{kernel, [{logger, [{handler, default, logger_std_h, - #{config => #{type => {file, "path/to/file.log"}}}}]}]}]. + #{config => #{file => "path/to/file.log"}}}]}]}].

For more information about: diff --git a/lib/kernel/doc/src/logger_chapter.xml b/lib/kernel/doc/src/logger_chapter.xml index 5ed7397135..bfd0acf634 100644 --- a/lib/kernel/doc/src/logger_chapter.xml +++ b/lib/kernel/doc/src/logger_chapter.xml @@ -801,7 +801,7 @@ logger:debug(#{got => connection_request, id => Id, state => State}, [{kernel, [{logger, [{handler, default, logger_std_h, % {handler, HandlerId, Module, - #{config => #{type => {file,"log/erlang.log"}}}} % Config} + #{config => #{file => "log/erlang.log"}}} % Config} ]}]}].

Modify the default handler to print each log event as a @@ -831,10 +831,10 @@ logger:debug(#{got => connection_request, id => Id, state => State}, [{logger, [{handler, default, logger_std_h, #{level => error, - config => #{type => {file, "log/erlang.log"}}}}, + config => #{file => "log/erlang.log"}}}, {handler, info, logger_std_h, #{level => debug, - config => #{type => {file, "log/debug.log"}}}} + config => #{file => "log/debug.log"}}} ]}]}]. @@ -1004,10 +1004,10 @@ ok

Then, add a new handler which prints to file. You can use the handler module logger_std_h, - and specify type {file,File}.:

+ and configure it to log to file:

-4> Config = #{config => #{type => {file,"./info.log"}}, level => info}.
-#{config => #{type => {file,"./info.log"}},level => info}
+4> Config = #{config => #{file => "./info.log"}, level => info}.
+#{config => #{file => "./info.log"},level => info}
 5> logger:add_handler(myhandler, logger_std_h, Config).
 ok

Since filter_default defaults to log, this @@ -1246,7 +1246,7 @@ do_log(Fd, LogEvent, #{formatter := {FModule, FConfig}}) ->

A configuration example:

logger:add_handler(my_standard_h, logger_std_h, - #{config => #{type => {file,"./system_info.log"}, + #{config => #{file => "./system_info.log", sync_mode_qlen => 100, drop_mode_qlen => 1000, flush_qlen => 2000}}). diff --git a/lib/kernel/doc/src/logger_std_h.xml b/lib/kernel/doc/src/logger_std_h.xml index 8c5a476d86..ce54a91342 100644 --- a/lib/kernel/doc/src/logger_std_h.xml +++ b/lib/kernel/doc/src/logger_std_h.xml @@ -55,37 +55,86 @@ is stored in a sub map with the key config, and can contain the following parameters:

- type + type = standard_io | standard_error | file -

This has the value standard_io, standard_error, - {file,LogFileName}, or {file,LogFileName,LogFileOpts}.

-

LogFileOpts specify the file options used when - opening the log file, +

Specifies the log destination.

+

The value is set when the handler is added, and it can not + be changed in runtime.

+

Defaults to standard_io, unless + parameter file is + given, in which case it defaults to file.

+ + file = file:filename() + +

This specifies the name of the log file when the handler is + of type file.

+

The value is set when the handler is added, and it can not + be changed in runtime.

+

Defaults to the same name as the handler identity, in the + current directory.

+ + modes = [file:mode()] + +

This specifies the file modes to use when opening the log + file, see file:open/2. - If LogFileOpts is not specified, the default option - list used is [raw,append,delayed_write]. If LogFileOpts - is specified, it replaces the default options list with the + If modes are not specified, the default list used + is [raw,append,delayed_write]. If modes are + specified, the list replaces the default modes list with the following adjustments:

- If raw is not found in the list, it is added + If raw is not found in the list, it is added. - It write, append or exclusice are not - found in the list, append is added + If none of write, append or exclusive is + found in the list, append is added. + If none of delayed_write + or {delayed_write,Size,Delay} is found in the + list, delayed_write is added.

Log files are always UTF-8 encoded. The encoding can not be - changed by setting the option {encoding,Encoding} - in LogFileOpts.

-

Notice that the standard handler does not have support for - circular logging. Use the disk_log handler, - logger_disk_log_h, - for this.

+ changed by setting the mode {encoding,Encoding}.

The value is set when the handler is added, and it can not be changed in runtime.

-

Defaults to standard_io.

+

Defaults to [raw,append,delayed_write].

+ + max_no_bytes = pos_integer() | infinity + +

This parameter specifies if the log file should be rotated + or not. The value infinity means the log file will + grow indefinitely, while an integer value specifies at which + file size (bytes) the file is rotated.

+

Defaults to infinity.

+ + max_no_files = non_neg_integer() + +

This parameter specifies the number of rotated log file + archives to keep. This has meaning only + if max_no_bytes + is set to an integer value.

+

The log archives are + named FileName.0, FileName.1, + ... FileName.N, where FileName is the name of + the current log file. FileName.0 is the newest of the + archives. The maximum value for N is the value + of max_no_files minus 1.

+

Notice that setting this value to 0 does not turn of + rotation. It only specifies that no archives are kept.

+

Defaults to 0.

+ + compress_on_rotate = boolean() + +

This parameter specifies if the rotated log file archives + shall be compressed or not. If set to true, all + archives are compressed with gzip, and renamed + to FileName.N.gz

+

compress_on_rotate has no meaning if max_no_bytes has the + value infinity.

+

Defaults to false.

 - filesync_repeat_interval + filesync_repeat_interval = pos_integer() | no_repeat

This value, in milliseconds, specifies how often the handler does a file sync operation to write buffered data to disk. The handler attempts @@ -104,12 +153,13 @@ standard handler and the disk_log handler, and are documented in the User's Guide .

-

Notice that if changing the configuration of the handler in runtime, - the type parameter must not be modified.

+

Notice that if changing the configuration of the handler in + runtime, the type, file, or modes parameters + must not be modified.

Example of adding a standard handler:

logger:add_handler(my_standard_h, logger_std_h, - #{config => #{type => {file,"./system_info.log"}, + #{config => #{file => "./system_info.log", filesync_repeat_interval => 1000}}).

To set the default handler, that starts initially with @@ -117,7 +167,7 @@ logger:add_handler(my_standard_h, logger_std_h, change the Kernel default logger configuration. Example:

erl -kernel logger '[{handler,default,logger_std_h, - #{config => #{type => {file,"./log.log"}}}}]' + #{config => #{file => "./log.log"}}}]'

An example of how to replace the standard handler with a disk_log handler at startup is found in the -- cgit v1.2.3 From e2af137524ac6a645689daf309871893dcc655a1 Mon Sep 17 00:00:00 2001 From: Siri Hansen Date: Mon, 4 Mar 2019 19:15:24 +0100 Subject: [logger] Add option file_check to logger_std_h OTP-15663 This option indicates how often the handler shall check if the log file still exists and if the inode is changed. --- lib/kernel/doc/src/logger_std_h.xml | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) (limited to 'lib/kernel/doc/src') diff --git a/lib/kernel/doc/src/logger_std_h.xml b/lib/kernel/doc/src/logger_std_h.xml index ce54a91342..5ed1a2f210 100644 --- a/lib/kernel/doc/src/logger_std_h.xml +++ b/lib/kernel/doc/src/logger_std_h.xml @@ -134,6 +134,25 @@ value infinity.

Defaults to false.

 + file_check = non_neg_integer() + +

When logger_std_h logs to a file, it reads the file + information of the log file prior to each write + operation. This is to make sure the file still exists and + has the same inode as when it was opened. This implies some + performance loss, but ensures that no log events are lost in + the case when the file has been removed or renamed by an + external actor.

+

In order to allow minimizing the performance loss, the + file_check parameter can be set to a positive integer + value, N. The handler will then skip reading the file + information prior to writing, as long as no more + than N milliseconds have passed since it was last + read.

+

Notice that the risk of loosing log events grows when + the file_check value grows.

+

Defaults to 0.

+ filesync_repeat_interval = pos_integer() | no_repeat

This value, in milliseconds, specifies how often the handler does -- cgit v1.2.3