| Age | Commit message (Collapse) | Author |
|---|
|
* dgud/kernel/refc_sched_wall_time/OTP-11694:
test: spawn scheduler_wall_time flag holder
Turn on scheduler_wall_time in an alive process
Redirect system_flag(scheduler_wall_time,_) to kernel_refc
kernel: add a resource reference counter
|
|
System resources/functionality may need to be reference counted
to be handled correctly when used or enabled/disabled from more
than one process or application.
It is easier to handle this in erlang code than in erts, so make a
process that deals with the housekeeping.
|
|
This improves the latency of file operations as dirty schedulers
are a bit more eager to run jobs than async threads, and use a
single global queue rather than per-thread queues, eliminating the
risk of a job stalling behind a long-running job on the same thread
while other async threads sit idle.
There's no such thing as a free lunch though; the lowered latency
comes at the cost of increased busy-waiting which may have an
adverse effect on some applications. This behavior can be tweaked
with the +sbwt flag, but unfortunately it affects all types of
schedulers and not just dirty ones. We plan to add type-specific
flags at a later stage.
sendfile has been moved to inet_drv to lessen the effect of a nasty
race; the cooperation between inet_drv and efile has never been
airtight and the socket dying at the wrong time (Regardless of
reason) could result in fd aliasing. Moving it to the inet driver
makes it impossible to trigger this by closing the socket in the
middle of a sendfile operation, while still allowing it to be
aborted -- something that can't be done if it stays in the file
driver.
The race still occurs if the controlling process dies in the short
window between dispatching the sendfile operation and the dup(2)
call in the driver, but it's much less likely to happen now.
A proper fix is in the works.
--
Notable functional differences:
* The use_threads option for file:sendfile/5 no longer has any
effect.
* The file-specific DTrace probes have been removed. The same
effect can be achieved with normal tracing together with the
nif__entry/nif__return probes to track scheduling.
--
OTP-14256
|
|
Various places that now reject null chars inside strings
- Primitive file operations reject it in filenames.
- Primitive environment variable operations reject it in
names and values.
- os:cmd() reject it in its input.
Also '=' characters are rejected by primitive environment
variable operations in environment variable names.
Documentation has been updated to document null characters
in these types of data as invalid. Currently these operations
accept null chars at the end of strings, but that will change
in the future.
|
|
~tw and new string functions are new since OTP-20 (stdlib-3.4)
|
|
* john/erts/runtime-lcnt:
Document rt_mask and add warnings about copy_save
Add an emulator test suite for lock counting
Break erts_debug:lock_counters/1 into separate BIFs
Allow toggling lock counting at runtime
Move lock flags to a common header
Enable register_SUITE for lcnt builds
Enable lcnt smoke test on all builds that have lcnt enabled
Make lock counter info independent of the locks being counted
OTP-14412
OTP-13170
OTP-14413
|
|
|
|
|
|
This patch adds a mechanism by which shell history gets stored
persistently on disk and gets loaded whenever a new shell session
begins.
The log files are added to the user's cache directory, with multiple
files in it, although the location is configurable.
All Erlang instances on a given host will share the same log file.
There are two hook points in group.erl that are used: when instantiating
the buffer, where we fetch from disk, and when adding a line to the
buffer (gets stored).
This allows all shell instances to use the same base set of lines when
loaded, but to keep their history separate after the fact. As lines are
added as you go, a new shell session (with ^G -> s -> c) will have
access to previous sessions' lines.
The implementation makes use of the disk_log library with a rotating log
in order to store data, and allow automated repairs.
By default, the feature is disabled and must be turned on through OTP
environment variable part of the kernel application. The options
include:
| Option | Accepted values | Default | Description |
| ------------------------ | ----------------- |---------- | -------------------- |
| shell_history | enabled, disabled | disabled | turn feature on/off |
| shell_history_path | any string | user cache| where to put files |
| shell_history_file_bytes | 51200..N bytes | 512kb | max data size (>50kb)|
| shell_history_drop | ["q().", ...] | [] | blacklisted lines |
A version header is added to the disk_log configuration, allowing
changes in the storage format to be enacted in the future (i.e. adding
segmentation by node name in the same file, or encoding) without
conflict.
Log rotation is used with multiple log files to ensure proper
enforcement of resizing without too much data loss. Because disk_log
does not allow to just flush bits of content on rewrite (it truncates
any full file), we instead use a wrap log and try to divide the
configured size into up to 10 log files so that every time we rotate a
log, we lose only 10% of the data. This remains true for corrupted files
that cannot fully be repaired.
This many-logs-based approach in turn forces the use of a minimal log
size for the `shell_history_file_bytes` configuration, since it has to
be divided by 10.
The shell history is not loaded for the `user` process which, despite
running the group.erl module, does not actually require history as it
mostly just forwards IO protocol information.
|
|
|
|
|
|
* saleyn/uds/PR-612/OTP-13572:
Rewrite inet* for address family 'local'
Rewrite inet_drv for AF_LOCAL
Assign externally open fd to gen_tcp (UDS support)
Conflicts:
erts/preloaded/ebin/prim_inet.beam
lib/kernel/doc/src/gen_tcp.xml
lib/kernel/doc/src/gen_udp.xml
lib/kernel/src/inet6_sctp.erl
lib/kernel/test/inet_SUITE.erl
|
|
|
|
|
|
|
|
- The calling process is now suspended while synchronizing
scheduler suspends via erlang:system_flag(schedulers_online, _)
and erlang:system_flag(multi_scheduling, _), instead of blocking
the scheduler thread in the BIF call waiting for the operation
to synchronize. Besides releasing the scheduler for other work
(or immediate suspend) it also makes it possible to abort the
operation by killing the process.
- erlang:system_flag(schedulers_online, _) now only wait for normal
schedulers to complete before it returns. This since it may take
a very long time before all dirty schedulers suspends.
- erlang:system_flag(multi_scheduling, block_normal|unblock_normal)
which only operate on normal schedulers has been introduced. This
since there are use cases where suspend of dirty schedulers are
not of interest (hipe loader).
- erlang:system_flag(multi_scheduling, block) still blocks all
dirty schedulers as well as all normal schedulers except one since
it is hard to redefine what multi scheduling block means.
- The three operations:
- changing amount of schedulers online
- blocking/unblocking normal multi scheduling
- blocking/unblocking full multi scheduling
can now be done in parallel. This is important since otherwise
a full multi scheduling block would potentially delay the other
operations for a very long time.
|
|
New timestamp options for trace, sequential trace, and
system profile:
- monotonic_timestamp
- strict_monotonic_timestamp
|
|
|
|
|
|
|
|
|
|
|
|
Most dependencies introduced are exactly the dependencies to other
applications found by xref. That is, there might be real dependencies
missing. There might also be pure debug dependencies listed that
probably should be removed. Each application has to be manually
inspected in order to ensure that all real dependencies are listed.
All dependencies introduced are to application versions used in
OTP 17.0. This since the previously used version scheme wasn't
designed for this, and in order to minimize the work of introducing
the dependencies.
|
|
|
|
|
|
|
