| Age | Commit message (Collapse) | Author |
|---|
|
* rickard/timer-optimization/OTP-12650:
Optimized timer implementation
Reusable red-black tree implementation
Conflicts:
erts/emulator/beam/erl_bif_timer.c
|
|
|
|
According to EEP-43 for maps, a 'badmap' exception should be
generated when an attempt is made to update non-map term such as:
<<>>#{a=>42}
That was not implemented in the OTP 17.
José Valim suggested that we should take the opportunity to
improve the errors coming from map operations:
http://erlang.org/pipermail/erlang-questions/2015-February/083588.html
This commit implement better errors from map operations similar
to his suggestion.
When a map update operation (Map#{...}) or a BIF that expects a map
is given a non-map term, the exception will be:
{badmap,Term}
This kind of exception is similar to the {badfun,Term} exception
from operations that expect a fun.
When a map operation requires a key that is not present in a map,
the following exception will be raised:
{badkey,Key}
José Valim suggested that the exception should be
{badkey,Key,Map}. We decided not to do that because the map
could potentially be huge and cause problems if the error
propagated through links to other processes.
For BIFs, it could be argued that the exceptions could be simply
'badmap' and 'badkey', because the bad map and bad key can be found in
the argument list for the BIF in the stack backtrace. However, for the
map update operation (Map#{...}), the bad map or bad key will not be
included in the stack backtrace, so that information must be included
in the exception reason itself. For consistency, the BIFs should raise
the same exceptions as update operation.
If more than one key is missing, it is undefined which of
keys that will be reported in the {badkey,Key} exception.
|
|
* rickard/time_api/OTP-11997: (22 commits)
Update primary bootstrap
inets: Suppress deprecated warning on erlang:now/0
inets: Cleanup of multiple copies of functions Add inets_lib with common functions used by multiple modules
inets: Update comments
Suppress deprecated warning on erlang:now/0
Use new time API and be back-compatible in inets Remove unused functions and removed redundant test
asn1 test SUITE: Eliminate use of now/0
Disable deprecated warning on erlang:now/0 in diameter_lib
Use new time API and be back-compatible in ssh
Replace all calls to now/0 in CT with new time API functions
test_server: Replace usage of erlang:now() with usage of new API
Replace usage of erlang:now() with usage of new API
Replace usage of erlang:now() with usage of new API
Replace usage of erlang:now() with usage of new API
Replace usage of erlang:now() with usage of new API
otp_SUITE: Warn for calls to erlang:now/0
Replace usage of erlang:now() with usage of new API
Multiple timer wheels
Erlang based BIF timer implementation for scalability
Implement ethread events with timeout
...
Conflicts:
bootstrap/bin/start.boot
bootstrap/bin/start_clean.boot
bootstrap/lib/compiler/ebin/beam_asm.beam
bootstrap/lib/compiler/ebin/compile.beam
bootstrap/lib/kernel/ebin/auth.beam
bootstrap/lib/kernel/ebin/dist_util.beam
bootstrap/lib/kernel/ebin/global.beam
bootstrap/lib/kernel/ebin/hipe_unified_loader.beam
bootstrap/lib/kernel/ebin/inet_db.beam
bootstrap/lib/kernel/ebin/inet_dns.beam
bootstrap/lib/kernel/ebin/inet_res.beam
bootstrap/lib/kernel/ebin/os.beam
bootstrap/lib/kernel/ebin/pg2.beam
bootstrap/lib/stdlib/ebin/dets.beam
bootstrap/lib/stdlib/ebin/dets_utils.beam
bootstrap/lib/stdlib/ebin/erl_tar.beam
bootstrap/lib/stdlib/ebin/escript.beam
bootstrap/lib/stdlib/ebin/file_sorter.beam
bootstrap/lib/stdlib/ebin/otp_internal.beam
bootstrap/lib/stdlib/ebin/qlc.beam
bootstrap/lib/stdlib/ebin/random.beam
bootstrap/lib/stdlib/ebin/supervisor.beam
bootstrap/lib/stdlib/ebin/timer.beam
erts/aclocal.m4
erts/emulator/beam/bif.c
erts/emulator/beam/erl_bif_info.c
erts/emulator/beam/erl_db_hash.c
erts/emulator/beam/erl_init.c
erts/emulator/beam/erl_process.h
erts/emulator/beam/erl_thr_progress.c
erts/emulator/beam/utils.c
erts/emulator/sys/unix/sys.c
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/erts_internal.beam
erts/preloaded/ebin/init.beam
erts/preloaded/src/erts_internal.erl
lib/common_test/test/ct_hooks_SUITE_data/cth/tests/empty_cth.erl
lib/diameter/src/base/diameter_lib.erl
lib/kernel/src/os.erl
lib/ssh/test/ssh_basic_SUITE.erl
system/doc/efficiency_guide/advanced.xml
|
|
|
|
The old time API is based on erlang:now/0. The major issue with
erlang:now/0 is that it was intended to be used for so many
unrelated things. This tied these unrelated operations together
and unnecessarily caused performance, scalability as well as
accuracy, and precision issues for operations that do not need
to have such issues. The new API spreads different functionality
over multiple functions in order to improve on this.
The new API consists of a number of new BIFs:
- erlang:convert_time_unit/3
- erlang:monotonic_time/0
- erlang:monotonic_time/1
- erlang:system_time/0
- erlang:system_time/1
- erlang:time_offset/0
- erlang:time_offset/1
- erlang:timestamp/0
- erlang:unique_integer/0
- erlang:unique_integer/1
- os:system_time/0
- os:system_time/1
and a number of extensions of existing BIFs:
- erlang:monitor(time_offset, clock_service)
- erlang:system_flag(time_offset, finalize)
- erlang:system_info(os_monotonic_time_source)
- erlang:system_info(time_offset)
- erlang:system_info(time_warp_mode)
- erlang:system_info(time_correction)
- erlang:system_info(start_time)
See the "Time and Time Correction in Erlang" chapter of the
ERTS User's Guide for more information.
|
|
* sverk/yielding-distr-send/OTP-12232:
erts: Add constant TERM_TO_BINARY_MEMCPY_FACTOR
erts: Optimize some repeated calls to {E,W}STACK_PUSH
erts: Yield in term_to_binary when encoding big maps
erts: Remove unnecessary goto for fun encoding
erts: Yield in term_to_binary while copying large binaries
erts: Implement yielding for distributed send of large messages
|
|
Use same mechanism as term_to_binary to yield
while encoding large messages for distributed send.
|
|
|
|
- erlang:list_to_binary/1
- erlang:iolist_to_binary/1
- erlang:list_to_bitstring/1
- binary:list_to_bin/1
|
|
- erlang:binary_to_list/1
- erlang:binary_to_list/3
- erlang:bitstring_to_list/1
|
|
It previously always returned an empty list
and was documented to be removed.
|
|
|
|
Add support for setting the number of dirty CPU schedulers online via
erlang:system_flag/2. Assuming the emulator is built with dirty schedulers
enabled, the number of dirty CPU schedulers online may not be less than 1,
nor greater than the number of dirty CPU schedulers available, nor greater
than the number of normal schedulers online. Dirty CPU scheduler threads
that are taken offline via system_flag/2 are suspended. The number of dirty
CPU schedulers online may be adjusted independently of the number of normal
schedulers online, but if system_flag/2 is used to set the number of normal
schedulers online to a value less than the current number of normal
schedulers online, the number of dirty CPU schedulers online is decreased
proportionally. Likewise, if the number of normal schedulers online is
increased, the number of dirty CPU schedulers online is increased
proportionally. For example, if 8 normal schedulers and 4 dirty CPU
schedulers are online, and system_flag/2 is called to set the number of
normal schedulers online to 4, the number of dirty CPU schedulers online is
also decreased by half, to 2. Subsequently setting the number of normal
schedulers online back to 8 also sets the number of dirty CPU schedulers
online back to 4. Augment the system_flag/2 documentation in the erlang man
page to explain this relationship between schedulers_online and
dirty_cpu_schedulers_online.
Also ensure that all dirty CPU and I/O schedulers are suspended when
multi-scheduling is blocked via system_flag/2, and brought back online when
multi-scheduling is unblocked.
Add Rickard Green's rewritten check_enqueue_in_prio_queue() function that
inspects process state more thoroughly to determine if to enqueue it and if
so on what queue, including dirty queues when appropriate.
Make sure dirty NIF jobs do not trigger erlang:system_monitor long_schedule
messages.
Add more dirty scheduler testing to the scheduler test suite.
Remove the erts_no_dirty_cpu_schedulers_online global variable, since it's
no longer needed.
Execute dirty NIFs on a normal scheduler thread while multi-scheduling
blocking is in effect. Evacuate any dirty jobs residing in the dirty run
queues over to a normal run queue when multi-scheduling is blocked.
Allow dirty schedulers to execute aux work.
Set the dirty run queues halt_in_progress flag when halting the normal
schedulers.
Change dirty scheduler numbers to a structure including both scheduler
number and type, either dirty CPU or dirty I/O. Add some assertions to
ensure that dirty CPU schedulers operate only on dirty CPU scheduler
process flags, and the same for dirty I/O schedulers.
|
|
Yielding binary_to_term.
OTP-11535
* tag 'OTP_R16B03_yielding_binary_to_term':
Increase versions for OTP_R16B03_yielding_binary_to_term
erts: Adjust term_to_binary reduction factors
erts: Yield after trapping term_to_binary if gc has been ordered
erts: Let term_to_binary disable gc while trapping
erts: Improve stress of binary_to_term in binary_SUITE
erts: Fix bug in binary_to_term for compressed on halfword
erts: Fix crash when binary_to_term throws badarg
erts: Trapping memcpy in binary_to_term
erts: Cleanup code for trapping binary_to_term
erts: Add erlang wrappers to binary_to_term
trapping uncompress
trapping size calculation
trapping binary_to_term/2
trapping STRING_EXT
trapping lists and tuples
trapping binary_to_term passing binary_SUITE
Conflicts:
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/erts_internal.beam
erts/vsn.mk
lib/kernel/vsn.mk
lib/stdlib/vsn.mk
|
|
* rickard/garbage_collect/OTP-11388:
Parallel check_process_code when code_server purge a module
Functionality for disabling garbage collection
Use asynchronous check_process_code in code_parallel_SUITE
Execution of system tasks in context of another process
Conflicts:
bootstrap/lib/kernel/ebin/hipe_unified_loader.beam
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/erts_internal.beam
|
|
|
|
A process requesting a system task to be executed in the context of
another process will be notified by a message when the task has
executed. This message will be on the form:
{RequestType, RequestId, Pid, Result}.
A process requesting a system task to be executed can set priority
on the system task. The requester typically set the same priority
on the task as its own process priority, and by this avoiding
priority inversion. A request for execution of a system task is
made by calling the statically linked in NIF
erts_internal:request_system_task(Pid, Prio, Request). This is an
undocumented ERTS internal function that should remain so. It
should *only* be called from BIF implementations.
Currently defined system tasks are:
* garbage_collect
* check_process_code
Further system tasks can and will be implemented in the future.
The erlang:garbage_collect/[1,2] and erlang:check_process_code/[2,3]
BIFs are now implemented using system tasks. Both the
'garbage_collect' and the 'check_process_code' operations perform
or may perform garbage_collections. By doing these via the
system task functionality all garbage collect operations in the
system will be performed solely in the context of the process
being garbage collected. This makes it possible to later implement
functionality for disabling garbage collection of a process over
context switches.
Newly introduced BIFs:
* erlang:garbage_collect/2 - The new second argument is an option
list. Introduced option:
* {async, RequestId} - making it possible for users to issue
asynchronous garbage collect requests.
* erlang:check_process_code/3 - The new third argument is an
option list. Introduced options:
* {async, RequestId} - making it possible for users to issue
asynchronous check process code requests.
* {allow_gc, boolean()} - making it possible to issue requests
that aren't allowed to garbage collect (operation will abort
if gc should be needed).
These options have been introduced as a preparation for
parallelization of check_process_code operations when the
code_server is about to purge a module.
|
|
* pan/update_pcre_8.33:
Workaround TR gnu/181328, GCC 4.2.1 20070831 on FreeBSD 9.1
Clarify relation between erts_iolist_{size|to_buf}
Fix backslash in titles of manpages
Correct UTF-8 in stdlib's notes.xml
Add more tests for corner error cases in erl_bif_re.c
Add documentation of report_errors and match_limit(_recursion)
Add match_limit and match_limit_recursion options
Add return_errors option to re:run/3
Add README for updating PCRE
Add documentation of extensions to re module
Add new options to Erlang re interface and mend dupnames
Update PCRE doc part of re.xml to PCRE 8.33 state
Integrate new PCRE test suites
Integrate patch for PCRE bug id 1370
Handle CRLF correctly in global regexp
Add erts_prefix to pcre_library and update erl_bif_re
Update to PCRE 8.33, w/o the erts_ prefix added
OTP-11204
OTP-11205
OTP-10285
|
|
|
|
Add notempty_atstart, no_start_optimize, ucp and
never_utf options from new PCRE version.
Use the new notempty_atstart in global matching.
Add inspect/2 function
Correctly handle dupnames when capturing a name, as
in Perl, get the leftmost matching occurence.
Also added all_names, to get all the names in the pattern
in alphabetical (name) order.
To be able to use this in global matching, an inspect
function that can dig out a namelist was added.
|
|
|
|
* pan/happi/yield_in_term_to_binary:
Add testcase to stress extra_root
term_to_binary: Remove debug code and set production trap levels
Teach erl_gc:offset_rootset about extra_root
Teach external.c to handle reallocs before compression
Make all steps ofterm_to_binary work in chunks and yield
Make term_to_binary yield (trap).
OTP-11163
|
|
* pan/r16b01/system_monitor_long_schedule/OTP-11067:
Minor spelling correction
Add system_monitor of long_schedule
|
|
Rewrite and extend of Happi's initial work
Extra_root to process structure to enable GC of state - Changed the
process structure to point to a separate struct, the struct also
contains a destructor function to allow for proper cleanup.
Rewrote encode_size_struct and enc_term to have internal versions with
reduction counters which will result in interrupt for later restart
when the counter reaches zero - removed the EWA_STACK from Happis
version and directly save the ESTACK's and WSTACK's in the above
mentioned struct (or array thereof) that are pointed out from the
process structure. The destructor will take care of the deallocation
in case of process death.
Added ESTACK and WSTACK macros to save and restore stack and to change
allocator, which makes the previously mentioned stack-save work.
Rewrote enc_term to not store pointers on the stack, and use one WSTACK
for commands etc and another ESTACK for Eterms - Slightly different than
Happis version to make halfword code simpler.
Rewrote encode_size_struct2 so that it does not store pointers on the
stack, also switched to ESTACK instead of WSTACK, this also handles
halfword correctly.
Added interfaces for chunkwise compression, that are
used from term_to_binary/2 when the compressed option is given.
|
|
|
|
|
|
|
|
|
|
The fix affects list_dir and read_link. Raw filenames are now
never produced, just consumed even if +fnu or +fna is used on
Linux etc.
This also adds the options to get error return or error handler
warning messages with +fn{u|a}{i|w|e} as an option to erl.
This is still not documented and there needs to be other versions
of read_dir and read_link to facilitate reading of all types
of filenames and links.
A check that we will not change to an invalid directory is also needed.
|
|
|
|
* sverk/r16/utf8-atoms:
erl_interface: Fix bug when transcoding atoms from and to UTF8
erl_interface: Changed erlang_char_encoding interface
erts: Testcase doing unicode atom printout with ~w
erl_interface: even more utf8 atom stuff
erts: Fix bug in analyze_utf8 causing faulty latin1 detection
Add UTF-8 node name support for epmd
workaround...
Fix merge conflict with hasse
UTF-8 atom documentation
test case
erl_interface: utf8 atoms continued
Add utf8 atom distribution test cases
atom fixes for NIFs and atom_to_binary
UTF-8 support for distribution
Implement UTF-8 atom support for jinterface
erl_interface: Enable decode of unicode atoms
stdlib: Fix printing of unicode atoms
erts: Change internal representation of atoms to utf8
erts: Refactor rename DFLAG(S)_INTERNAL_TAGS for conformity
Conflicts:
erts/emulator/beam/io.c
OTP-10753
|
|
This BIF solves a problem of float_to_list/1 that doesn't allow
specifying the number of digits after the decimal point when
formatting floats.
float_to_list(Float, Options) -> string()
Float = float()
Options = [Option]
Option = {decimals, Decimals::0..249} |
{scientific, Decimals::0..249} |
compact
Returns a string which corresponds to the text representation of
a `Float` formatted using given options.
When decimals option is specified the returned value will contain
at most `Decimals` number of digits past the decimal point.
When `compact` option is provided the trailing zeros at the end
of the list are truncated (this option is only meaningful together
with the `decimals` option). When `scientific` option is provided,
the float will be formatted using scientific notation with
`Decimals` digits of precision. If `Options` is `[]` the function
behaves like `float_to_list/1`. When using `decimals` option and
the number doesn't fit in the static internal buffer of 256 bytes
the function throws `badarg`.
|
|
|
|
rickard/r16/port-optimizations/OTP-10336
* rickard/port-optimizations/OTP-10336:
Change annotate level for emacs-22 in cerl
Update etp-commands
Add documentation on communication in Erlang
Add support for busy port message queue
Add driver callback epilogue
Implement true asynchronous signaling between processes and ports
Add erl_drv_[send|output]_term
Move busy port flag
Use rwlock for driver list
Optimize management of port tasks
Improve configuration of process and port tables
Remove R9 compatibility features
Use ptab functionality also for ports
Prepare for use of ptab functionality also for ports
Atomic port state
Generalize process table implementation
Implement functionality for delaying thread progress from unmanaged threads
Conflicts:
erts/doc/src/erl_driver.xml
erts/doc/src/erlang.xml
erts/emulator/beam/beam_bif_load.c
erts/emulator/beam/beam_bp.c
erts/emulator/beam/beam_emu.c
erts/emulator/beam/bif.c
erts/emulator/beam/copy.c
erts/emulator/beam/erl_alloc.c
erts/emulator/beam/erl_alloc.types
erts/emulator/beam/erl_bif_info.c
erts/emulator/beam/erl_bif_port.c
erts/emulator/beam/erl_bif_trace.c
erts/emulator/beam/erl_init.c
erts/emulator/beam/erl_message.c
erts/emulator/beam/erl_port_task.c
erts/emulator/beam/erl_process.c
erts/emulator/beam/erl_process.h
erts/emulator/beam/erl_process_lock.c
erts/emulator/beam/erl_trace.c
erts/emulator/beam/export.h
erts/emulator/beam/global.h
erts/emulator/beam/io.c
erts/emulator/sys/unix/sys.c
erts/emulator/sys/vxworks/sys.c
erts/emulator/test/port_SUITE.erl
erts/etc/unix/cerl.src
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/prim_inet.beam
erts/preloaded/src/prim_inet.erl
lib/hipe/cerl/erl_bif_types.erl
lib/kernel/doc/src/inet.xml
lib/kernel/src/inet.erl
|
|
|
|
|
|
|
|
When a crash dump is about to be written and we have
heartbeat enabled on a system. We need time to write it
before heart explicitly kills the beam.
|
|
The hybrid heap emulator was last working in the non-SMP R11B
run-time system. When the constant pools were introduced in R12B,
the hybrid heap emulator was not updated to handle them.
At this point, the harm from reduced readability of the code is
greater than any potential usefulness of keeping the code.
|
|
|
|
When spawning OS (unix) processes with erlang:open_port, store the
resulting unix pid so that it can be queried later on using
erlang:port_info/1,2.
|
|
|
|
User tags in a dynamic trace enabled VM are spread throughout the system
in the same way as seq_trace tokens. This is used by the file module
and various other modules to get hold of the tag from the user process
without changing the protocol.
|
|
|
|
* rickard/alloc-opt/OTP-7775:
Optimize memory allocation
Conflicts:
erts/aclocal.m4
erts/emulator/hipe/hipe_bif_list.m4
erts/preloaded/ebin/erl_prim_loader.beam
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/init.beam
erts/preloaded/ebin/otp_ring0.beam
erts/preloaded/ebin/prim_file.beam
erts/preloaded/ebin/prim_inet.beam
erts/preloaded/ebin/prim_zip.beam
erts/preloaded/ebin/zlib.beam
|
|
A number of memory allocation optimizations have been implemented. Most
optimizations reduce contention caused by synchronization between
threads during allocation and deallocation of memory. Most notably:
* Synchronization of memory management in scheduler specific allocator
instances has been rewritten to use lock-free synchronization.
* Synchronization of memory management in scheduler specific
pre-allocators has been rewritten to use lock-free synchronization.
* The 'mseg_alloc' memory segment allocator now use scheduler specific
instances instead of one instance. Apart from reducing contention
this also ensures that memory allocators always create memory
segments on the local NUMA node on a NUMA system.
|
|
|
|
|
|
|
