If you set {linger,{true,0}} on a gen_tcp listen socket, accept a connection on that socket, and then close the accepted socket, now the linger zero setting is transferred to the accepted socket. Before this correction that information was lost and the close behaviour on the accepted socket incorrect.

Own Id: OTP-15370 Aux Id: ERIERL-353

Fixed process_info(Pid,reductions) to not categorically increase reduction count of the measured process Pid. Repeated reduction measure of an idle process will most often (but not guaranteed) return the same value, like it behaved before OTP 21.3.8.

Own Id: OTP-15865 Aux Id: ERL-964

The runtime system disconnected a connection if it received an exit/2 signal where the recipient was a process on an old incarnation of the current node. That is, the receiving node had the same node name, but another "creation" number. The signal will now just be dropped since the receiving process no longer exists.

Own Id: OTP-15867 Aux Id: ERIERL-373

The possibility to send ancillary data, in particular the TOS field, has been added to gen_udp:send/4,5.

Own Id: OTP-15747 Aux Id: ERIERL-294

In nested use of try/catch, rethrowing an exception using erlang:raise/3 with a different class would not always be able to change the class of the exception.

Own Id: OTP-15834 Aux Id: ERIERL-367

Fixed bug in seq_trace:set_token(label,Term) which could cause VM crash if Term was heap allocated (not an atom, small integer, local pid or port). Bug exists since OTP 21.0 when terms other than small integers were first allowed as labels.

Own Id: OTP-15849 Aux Id: ERL-700

Fix seq_trace:print/2 not to raise badarg exception if label is not a small integer. Bug exists since OTP 21.0.

Own Id: OTP-15859 Aux Id: ERL-700

Fixed a buffer overflow when binary_to_existing_atom/2 and list_to_existing_atom/2 was used with the latin1 encoding.

Own Id: OTP-15819 Aux Id: ERL-944

Fixed more bugs in process_info(reductions) causing it to sometimes behave non-monotonic. That is, a subsequent call toward the same process could return a lower reduction value.

Own Id: OTP-15793 Aux Id: ERIERL-337, OTP-15709

Add missing documentation of new external tags NEW_PID, NEW_PORT and NEWER_REFERENCE introduced in OTP 19.

These new tags are planned to be "activated" in OTP 23 when distribution capability flag DFLAG_BIG_CREATION becomes mandatory. Older nodes (>= 19) are able to decode these new tags and send them back to the new node. Nodes older than OTP 23 will however never encode their own local pids, ports and references using the new tags.

Own Id: OTP-15766

Fix erlang:open_port/2 with the fd option to correctly cleanup the pollset when the port is closed. Before this fix there would be error printouts sent to logger when the same fd was reused in a new port.

Own Id: OTP-15753 Aux Id: ERL-900

Fixed a bug in seq_trace:reset_trace/0 that could crash the emulator.

Own Id: OTP-15704

Fixed bug in process_info(reductions) causing it to sometimes return invalid results.

Own Id: OTP-15709 Aux Id: ERIERL-337

If a suspend/resume signal pair was sent to a process while it was executing dirty, the receiving process could later end up in a suspended state indefinitely. This bug was introduced in ERTS version 10.0 (OTP 21.0).

Suspend/resume signals are sent from erlang:suspend_process()/erlang:resume_process(). The dbg trace tool utilize this functionality and could thus trigger this bug.

Own Id: OTP-15688

Fix a possible deadlock when terminating the ERTS caused by a dirty scheduler not releasing it's run-queue lock when terminating.

Own Id: OTP-15690 Aux Id: PR-2172

When multiplying a number by itself, a word beyond the number on the heap could be read (and ignored). This bug was extremely unlikely to actually cause a real problem.

Own Id: OTP-15484

Fix bug where doing seq_trace:reset_trace() while another process was doing a garbage collection could cause the run-time system to segfault.

Own Id: OTP-15490

Fix reading of ancillary data from packet oriented sockets on old Linux kernel versions. Without this fix, getting the data would cause the port to enter an infinite loop.

Own Id: OTP-15494

Fix bug where crash dumping or doing erlang:system_info(procs) while another process was doing a garbage collection could cause the run-time system to segfault.

Own Id: OTP-15527

Fix erlang:system_info(kernel_poll) to return correct value. Before this fix, the call always returned false.

Own Id: OTP-15556

Fix bug in enif_make_map_from_arrays that would produce broken maps when number of keys were 32. Bug exists since OTP 21.0.

Own Id: OTP-15567

Fix a bug in binary:encode_unsigned that may cause a read of uninitialized memory.

The bug existed since the function was added (OTP R16B02).

Own Id: OTP-15583 Aux Id: PR-2118

Fixed a bug that could cause heart to kill an exiting node before it had time to flush all buffered writes. If environment variable HEART_KILL_SIGNAL=SIGABRT was set a superfluous core dump could also be generated.

Own Id: OTP-15599 Aux Id: ERIERL-298

Fix enif_consume_timeslice to be a no-op on dirty scheduler and not crash debug compiled emulator.

Own Id: OTP-15604

Fixed macro redefinition warnings.

Own Id: OTP-15629

to_erl fixed to not garble terminal input beyond 7-bit ASCII.

Own Id: OTP-15650 Aux Id: ERL-854, PR-2161

Minor fixes for make clean.

Own Id: OTP-15657

Fixed a bug in all ets:select* and ets:match* functions that could in some rare cases lead to very poor performance.

Own Id: OTP-15660 Aux Id: ERL-869

Add erlang:system_flag(system_logger, Pid) and erlang:system_info(system_logger). This system_flag can be used to set the process that will receive the logging messages generated by ERTS.

Own Id: OTP-15375

integer_to_list/2 and integer_to_binary/2 are now implemented in C, improving their performance.

Own Id: OTP-15503 Aux Id: PR-2052

Improved term_to_binary to do more fair reduction count and yielding when encoding large byte lists (strings).

Own Id: OTP-15514 Aux Id: ERL-774

Made internal port drivers more robust against erlang:port_control with invalid arguments and added documentation warnings about such abuse.

Own Id: OTP-15555 Aux Id: ERIERL-231

Fix bug on NetBSD where the exit_status from a port program would never be sent.

Own Id: OTP-15558 Aux Id: ERL-725

There is a new function persistent:term(Key, Default) to allow specifying a default when looking up a persistent term.

Own Id: OTP-15576 Aux Id: ERL-843

A transitory emulator option '+ztma true' has been added to allow running existing BEAM code that relies on "tuple calls" (dispatch on parameterized modules) which has been compiled under OTP 20 or earlier. This option will be removed in OTP 22, so such modules should eventually be recompiled with the +tuple_calls option.

Own Id: OTP-15580 Aux Id: PR-2113

Fixes of install/release phase in build system.

Own Id: OTP-15551

Setting the recbuf size of an inet socket the buffer is also automatically increased. Fix a bug where the auto adjustment of inet buffer size would be triggered even if an explicit inet buffer size had already been set.

Own Id: OTP-15651 Aux Id: ERIERL-304

Reading from UDP using active true or active N mode has been optimized when more packets than specified by read_packets are available on the socket.

Own Id: OTP-15652 Aux Id: ERIERL-304

When using the {linger,{true,T}} option; gen_tcp:listen/2 used the full linger time before returning for example eaddrinuse. This bug has now been corrected.

Own Id: OTP-14728 Aux Id: ERIERL-303

Fix bug where doing a gen_tcp:send on a socket with delay_send set to true could cause a segfault if the other side closes the connection.

Bug was introduced in erts-10.2 (OTP-21.2).

Own Id: OTP-15536 Aux Id: ERL-827

Fix a race condition when a port program closes that could result in the next started port to hang during startup.

When this fault happens the following error is normally (but not always) logged:

=ERROR REPORT==== 14-Jan-2019::10:45:52.868246 ===
Bad input fd in erts_poll()! fd=11, port=#Port<0.505>, driver=spawn, name=/bin/sh -s unix:cmd

Bug was introduced in erts-10.0 (OTP-21.0).

Own Id: OTP-15537

Fix a bug where polling for external events could be delayed for a very long time if all active schedulers were 100% loaded.

Bug was introduced in erts-10.2 (OTP-21.2).

Own Id: OTP-15538 Aux Id: ERIERL-229

Fixed a crash when dangling files were closed after init:restart/0.

Own Id: OTP-15495 Aux Id: ERL-821

A bug that could cause dirty schedulers to become unresponsive has been fixed.

Own Id: OTP-15509 Aux Id: PR-2027, PR-2093

Fixed bug on big endian architectures when changing file permissions or ownership with file:change_mode, change_owner, change_group or write_file_info. Bug exists since OTP-21.0.

Own Id: OTP-15485

Fixed bug in atomics with option {signed,false} when returned values are (1 bsl 63) or larger. Could cause heap corruption leading to VM crash or other unpleasant symptoms. Bug exists since OTP-21.2 when module atomics was introduced.

Own Id: OTP-15486 Aux Id: PR-2061

Fixed bug in operator band of two negative operands causing erroneous result if the absolute value of one of the operands have the lowest N*W bits as zero and the other absolute value is not larger than N*W bits. N is an integer of 1 or larger and W is 32 or 64 depending on word size.

Own Id: OTP-15487 Aux Id: ERL-804

When a process was waiting for a TCP socket send operation to complete, and another process closed the socket during that send, the sending process could hang. This bug has now been corrected.

Own Id: OTP-12242 Aux Id: ERL-561

Document bit_size in match specifications and allow it in ets:fun2ms.

Own Id: OTP-15343 Aux Id: PR-1962

Fixed bug in ets:select_replace when called with a fully bound key could cause a following call to ets:next or ets:prev to crash the emulator or return invalid result.

Own Id: OTP-15346

When a module has been purged from memory, any literals belonging to that module will be copied to all processes that hold references to them. The max heap size limit would be ignored in the garbage collection initiated when copying literals to a process. If the max heap size was exceeded, the process would typically be terminated in the following garbage collection. Corrected to terminate the process directly if copying a literal would exceed the max heap size.

Own Id: OTP-15360

Fix compilation of run_erl on Solaris 11.4 and later.

Own Id: OTP-15389

Fixed a bug where lists:reverse/1-2 could use far too many reductions. This bug was introduced in OTP 21.1.

Own Id: OTP-15436

Fixed a bug where a dirty scheduler could stay awake forever if a distribution entry was removed as part of a dirty GC.

Own Id: OTP-15446 Aux Id: PR-2024

Fix microstate accounting handing in various places. Most importantly the GC states when the GC is run on a dirty scheduler are now managed correctly.

Own Id: OTP-15450 Aux Id: ERIERL-229

Fixed bug in file:sendfile when the send operation failed. For sockets in active modes it could cause emulator crash or a hanging call. For sockets with {active,false} an unexpected {inet_reply, _, _} message could be sent to the calling process. The bug exists since OTP-21.0.

Own Id: OTP-15461 Aux Id: ERL-784

The erts configure script has been updated to reject any CFLAGS that does not have -O. This in order to prevent the common mistake of forgetting to add -O2 to custom CFLAGS.

Own Id: OTP-15465

Fix reduction count in lists:member/2

Own Id: OTP-15474 Aux Id: ERIERL-229

New counters and atomics modules supplies access to highly efficient operations on mutable fixed word sized variables.

Own Id: OTP-13468

There is a new module persistent_term that implements a term storage suitable for terms that are frequently used but never or infrequently updated. Lookups are done in constant time without copying the terms.

Own Id: OTP-14669 Aux Id: PR-1989

A function inet:getifaddrs/1 that takes a list with a namespace option has been added, for platforms that support that feature, for example Linux (only?).

Own Id: OTP-15121 Aux Id: ERIERL-189, PR-1974

Added the nopush option for TCP sockets, which corresponds to TCP_NOPUSH on *BSD and TCP_CORK on Linux.

This is also used internally in file:sendfile to reduce latency on subsequent send operations.

Own Id: OTP-15357 Aux Id: ERL-698

List subtraction (The -- operator) will now yield properly on large inputs.

Own Id: OTP-15371

Optimize handling of send_delay for tcp sockes to better work with the new pollthread implementation introduced in OTP-21.

Own Id: OTP-15471 Aux Id: ERIERL-229

Optimize driver_set_timer with a zero timeout to short-circuit and not create any timer structure, but instead schedule the timer immediately.

Own Id: OTP-15472 Aux Id: ERIERL-229

Add erl_xcomp_code_model_small as a cross configure variable in order to let the emulator be build with the assumption that a small code model will be used on the target machine.

Own Id: OTP-15473 Aux Id: ERIERL-229

Add a new pollset that is made to handle sockets that use {active, true} or {active, N}. The new pollset will not be polled by a pollthread, but instead polled by a normal scheduler.

This change was made because of the overhead associated with constantly having to re-apply the ONESHOT mechanism on fds that all input events were interesting.

The new pollset is only active on platforms that support concurrent kernel poll updates, i.e. Linux and BSD.

Own Id: OTP-15475 Aux Id: ERIERL-229

Fix bug where emulator would segfault if a literal message was sent when sequence tracing was enabled.

Own Id: OTP-15478 Aux Id: ERL-741

Added an optional ./configure flag to compile the emulator with spectre mitigation: --with-spectre-mitigation

Note that this requires a recent version of GCC with support for spectre mitigation and the --mindirect-branch=thunk flag, such as 8.1.

Own Id: OTP-15430 Aux Id: ERIERL-237

Fixed a rare bug where files could be closed on a normal instead of an IO scheduler, resulting in system instability if the operation blocked.

Own Id: OTP-15421

A bug where the socket option 'pktoptions' caused a read of uninitialized memory has been fixed. Would cause malfunction on FreeBSD.

Own Id: OTP-14297 Aux Id: OTP-15141

Fixed a memory leak on errors when reading files.

Own Id: OTP-15318

File access through UNC paths works again on Windows. This regression was introduced in OTP 21.

Own Id: OTP-15333 Aux Id: ERL-737

Fix the seq_trace token to not be cleared when a process receives messages sent by erts. Some examples of when this could happen is all port BIFs, i.e. open_port, port_command etc etc.

Fix so that messages sent by nifs can be traced using normal and seq_trace tracing.

Own Id: OTP-15038 Aux Id: ERL-602

Fixed specs and documentation for process_info item monitored_by to include port identifiers and nif resources as possible types.

Own Id: OTP-15180 Aux Id: ERL-648

Fix bug in generation of erl_crash.dump, which could cause VM to crash.

Bug exist since erts-9.2 (OTP-20.2).

Own Id: OTP-15181

Fix bug where ctrl-break or ctrl-c would not trigger the break mode properly on Windows. This bug was introduced in erts-10.0 (OTP-21).

Own Id: OTP-15205

Fix a performance bug for reception of UDP packages, where a memory buffer would be reallocated when it should not have been.

Introduce a limit on the maximum automatic increase of the UDP user-space buffer to the theoretical max of the network PATH, i.e. 65535.

Own Id: OTP-15206

Fix alignment of erts allocator state internally in erts. With the improper alignment the emulator would refuse to start when compiled with clang on 32-bit systems.

Own Id: OTP-15208 Aux Id: PR-1897 ERL-677

Fix bug where too many concurrent calls to erlang:open_port({spawn,"cmd"},...) would result in the emulator terminating with the reason "Failed to write to erl_child_setup: ". After this fix the open_port call will throw an emfile exception instead.

Own Id: OTP-15210

Upgraded the ERTS internal PCRE library from version 8.41 to version 8.42. See http://pcre.org/original/changelog.txt for information about changes made to PCRE. This library implements major parts of the re regular expressions module.

Own Id: OTP-15217

Fix open_port({fd,X,Y}, ...) to release the file descriptors from the pollset when closing the port. Without this fix the same file descriptor number could not be reused when doing multiple open_port and port_close sequences.

Own Id: OTP-15236 Aux Id: ERL-692

Fixed bug in float_to_list/2 and float_to_binary/2 with options [{decimals,0},compact] causing totally wrong results. Bug exists since OTP-21.0.

Own Id: OTP-15276 Aux Id: PR-1920

Fixed bug in erlang:memory causing ets to report too much. This small false memory leak (16 bytes each time) can only happen when a specific race condition occurs between scheduler threads on a table with option write_concurrency.

Own Id: OTP-15278

Minor configure test fixes

Own Id: OTP-15282

Improved robustness of distribution connection setup. In OTP-21.0 a truly asynchronous connection setup was introduced. This is further improvement on that work to make the emulator more robust and also be able to recover in cases when involved Erlang processes misbehave.

Own Id: OTP-15297 Aux Id: OTP-15279, OTP-15280

The socket options recvtos, recvttl, recvtclass and pktoptions have been implemented in the socket modules. See the documentation for the gen_tcp, gen_udp and inet modules. Note that support for these in the runtime system is platform dependent. Especially for pktoptions which is very Linux specific and obsoleted by the RFCs that defined it.

Own Id: OTP-15145 Aux Id: ERIERL-187

As of ERTS version 10.0 (OTP 21.0) the erl_child_setup program, which creates port programs, ignores TERM signals. This setting was unintentionally inherited by port programs. Handling of TERM signals in port programs has now been restored to the default behavior. That is, terminate the process.

Own Id: OTP-15289 Aux Id: ERIERL-235, OTP-14943, ERL-576

The fix made for OTP-15279 in erts-10.07 (OTP-21.0.8) was not complete. It could cause a new connection attempt to be incorrectly aborted in certain cases. This fix will amend that flaw.

Own Id: OTP-15296 Aux Id: OTP-15279, ERIERL-226

A process could get stuck in an infinite rescheduling loop between normal and dirty schedulers. This bug was introduced in ERTS version 10.0.

Thanks to Maxim Fedorov for finding and fixing this issue.

Own Id: OTP-15275 Aux Id: PR-1943

Garbage collection of a distribution entry could cause an emulator crash if net_kernel had not brought previous connection attempts on it down properly.

Own Id: OTP-15279 Aux Id: ERIERL-226

A race between termination of a process and resume of the same process via erlang:resume_process/1 could cause the VM to crash. This bug was introduced in erts version 10.0 (OTP 21.0).

Own Id: OTP-15237

When tracing on running, in trace events could be lost when a process was rescheduled between a dirty and a normal scheduler.

Own Id: OTP-15269 Aux Id: ERL-713

Fixed a bug which caused an emulator crash when enif_send() was called by a NIF that executed on a dirty scheduler. The bug was either triggered when the NIF called enif_send() without a message environment, or when the process executing the NIF was send traced.

Own Id: OTP-15223

Fixed a bug causing some Erlang references to be inconsistently ordered. This could for example cause failure to look up certain elements with references as keys in search data structures. This bug was introduced in R13B02.

Thanks to Simon Cornish for finding the bug and supplying a fix.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-15225

Fixed a bug that prevented the noshell option from working correctly on Mac OS X and BSD.

Own Id: OTP-15169

Fixed a crash when matching directly against a literal map using a single key that had been saved on the stack.

Own Id: OTP-15184

Fix node crash when passing a bad time option to file:read_file_info/2.

Own Id: OTP-15196

Fixed a scheduler bug that caused normal schedulers to run dirty code.

Own Id: OTP-15154

Fixed a bug in erlang:trace_info/2 which caused the emulator to crash when a bad argument was passed. The bug was introduced in ERTS version 10.0.

Own Id: OTP-15183 Aux Id: ERL-670

Fixed a rare bug that could cause processes to be scheduled after they had been freed.

Own Id: OTP-15067 Aux Id: ERL-573

Fixed a race condition in the inet driver that could cause receive to hang when the emulator was compiled with gcc 8.

Own Id: OTP-15158 Aux Id: ERL-654

The keys used in os:getenv and os:putenv are case-insensitive again on Windows.

Own Id: OTP-15147 Aux Id: ERL-644

The type specifications for file:posix/0 and inet:posix/0 have been updated according to which errors file and socket operations should be able to return.

Own Id: OTP-14019 Aux Id: ERL-550

Fix error printout from run_erl and a bug that could cause unintended fds to be leaked into the started program.

Own Id: OTP-14537 Aux Id: PR1529

File operations used to accept filenames containing null characters (integer value zero). This caused the name to be truncated and in some cases arguments to primitive operations to be mixed up. Filenames containing null characters inside the filename are now rejected and will cause primitive file operations to fail.

Also environment variable operations used to accept names and values of environment variables containing null characters (integer value zero). This caused operations to silently produce erroneous results. Environment variable names and values containing null characters inside the name or value are now rejected and will cause environment variable operations to fail.

Primitive environment variable operations also used to accept the $= character in environment variable names causing various problems. $= characters in environment variable names are now also rejected.

Also os:cmd/1 now reject null characters inside its command.

erlang:open_port/2 will also reject null characters inside the port name from now on.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14543 Aux Id: ERL-370

Fix bugs related to the bookkeeping of microstate accounting states.

Own Id: OTP-14652

os:putenv and os:getenv no longer access the process environment directly and instead work on a thread-safe emulation. The only observable difference is that it's not kept in sync with libc getenv(3) / putenv(3), so those who relied on that behavior in drivers or NIFs will need to add manual synchronization.

On Windows this means that you can no longer resolve DLL dependencies by modifying the PATH just before loading the driver/NIF. To make this less of a problem, the emulator now adds the target DLL's folder to the DLL search path.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14666

Corrected erlang:is_builtin(erlang, M, F) to return true for apply/2 and yield/0.

Own Id: OTP-14713 Aux Id: ERL-500

Fixed a bug where the PATH environment variable wasn't updated correctly on a release downgrade, effectively keeping the PATH of the new release.

Own Id: OTP-14719

A receive optimization that avoids scanning the entire message queue when receiving a message containing a freshly created reference could in rare circumstances (involving recursive calls to the functions that does the receive) cause the receive to hang. This has been corrected.

Own Id: OTP-14782 Aux Id: ERL-511

Fix building of Erlang/OTP on platforms which have small data area with short addressing. For example the PowerPC/RTEMS platform.

Own Id: OTP-14909 Aux Id: PR-1692

Fixed a crash when enif_make_binary is called with a binary produced by enif_inspect_binary in a different environment.

Own Id: OTP-14931

Fixed a crash when enif_make_binary is called more than once with a binary that had previously been added to an enif_ioq.

Own Id: OTP-14932

The erl_child_setup program now ignores SIGTERM signals.

Own Id: OTP-14943 Aux Id: ERL-576

Force 64-bit alignment on pre-allocators on architectures which needs it.

Own Id: OTP-14977

Fixed a bug where dirty scheduler picked up non-dirty work.

Own Id: OTP-14978

Calls to gen_tcp:send/2 on closed sockets now returns {error, closed} instead of {error,enotconn}.

Own Id: OTP-15001

erlang:monotonic_time/1 failed with badarg when passing the perf_counter time unit as argument.

Own Id: OTP-15008

Fix bug where rapid init:restart() calls would sometimes crash because a code load request leaked in between the restarts.

Own Id: OTP-15013

Improve float_to_list(F, [{decimals,D}]) to closer conform with io_lib:format("~.*f", [D,F]).

There are however, still cases when float_to_list does not produce the exact same result as io_lib:format, especially for large values F and/or many decimals D.

Own Id: OTP-15015 Aux Id: OTP-14890

Fixed a deadlock that would occur on certain allocators when a reallocation failed with +ramv enabled.

Own Id: OTP-15024

Fix bug that made it impossible to use an erl_tracer as the seq_trace trace receiver.

Own Id: OTP-15029

Fix bug where a large (> 1 GB) emulator generated error logger message would cause the emulator to crash.

Own Id: OTP-15032

The emulator will no longer crash when reading the file information of an ordinary file that has an NTFS reparse point, such as files stored in a OneDrive-mapped folder.

Own Id: OTP-15062 Aux Id: ERL-615

Fixed bug in enif_binary_to_term which could cause memory corruption for immediate terms (atoms, small integers, pids, ports, empty lists).

Own Id: OTP-15080

Fixed bug in erlang:system_profile/2 that could cause superfluous {profile,_,active,_,_} messages for terminating processes.

Own Id: OTP-15085

On OSs with per thread CPU time support, change cpu_timestamp in erlang:trace/3 to use it instead of per process CPU time. This makes this option useable on such OSs when running multiple schedulers.

Own Id: OTP-15090

Fix segfault in abort_signal_task which could happen if a port terminated while there were outstanding port tasks that were not signals, for example a ready_input/ready_output event.

Own Id: OTP-15108 Aux Id: ERL-621

Fixed bug in ets that could cause VM crash if process A terminates after fixating a table and process B deletes the table at "the same time". The table fixation could be done with ets:safe_fixtable or if process A terminates in the middle of a long running select or match call.

Own Id: OTP-15109

Owner and group changes through file:write_file_info, file:change_owner, and file:change_group will no longer report success on permission errors.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-15118

Fix a bug error reporting from escripts on windows where the error message would get garbled.

Own Id: OTP-15119 Aux Id: PR-1826

Fix segfault when a process is interally re-scheduled while being traced for in out events. This bug was introduced in erts-8.0 (OTP-19.0).

Own Id: OTP-15125

It is now possible to open device files and FIFOs with file:open/2.

Own Id: OTP-11462

The erlang:system_flag(scheduler_wall_time,Bool) call is now reference counted and will be turned off if the (last) process that started the performance statistics dies. Thus it is no longer possible to start the statistics with rpc:call(Node, erlang, system_flag, [scheduler_wall_time, true]) since it will be turned off directly afterwards when the rpc process dies.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11694

A new logging API is added to Erlang/OTP, see the logger(3) manual page, and section Logging in the Kernel User's Guide.

Calls to error_logger are automatically redirected to the new API, and legacy error logger event handlers can still be used. It is, however, recommended to use the Logger API directly when writing new code.

Notice the following potential incompatibilities:

Since Logger is new in Erlang/OTP 21.0, we do reserve the right to introduce changes to the Logger API and functionality in patches following this release. These changes might or might not be backwards compatible with the initial version.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13295

gen_sctp:connect_init/4 or rather connect in inet_drv.c for SCTP has been fixed to not check the write file descriptor for writeability after a connect, since for SCTP (SOCK_SEQPACKET) that property does not seem to be any kind of indicator for when a connect has finished. This fixes connects that the OS returned as "in progress" that was misinterpreted by gen_sctp:connect_init as failed.

Own Id: OTP-13760 Aux Id: PR-1592

The file driver has been rewritten as a NIF, decreasing the latency of file operations. Notable incompatibilities are:

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14256

The I/O polling functionality of erts has been re-written to better make use of the OSs polling mechanisms. This change means that erts will now always prefer to use a kernel-polling mechanism if possible. Also all of the I/O polling has been moved to dedicated threads instead of being placed in the scheduler loops.

As a result of this, the erl options +K and +secio have been removed. It is still possible to disable kernel-poll, but it has to be done at compile time through the configure option --disable-kernel-poll.

The new erl options +IOt and +IOp can be used to change how many IO poll threads and poll sets that erts should use. See their respective documentation for more details.

Own Id: OTP-14346

Truly asynchronous auto-connect. Earlier, when erlang:send was aimed toward an unconnected node, the function would not return until the connection setup had completed (or failed). Now the function returns directly after the message has been enqueued and the connection setup started.

The same applies to all distributed operations that may trigger auto-connect, i.e. '!', send, link, monitor, monitor_node, exit/2 and group_leader.

The interface for all these functions are unchanged as they do not return connection failures. The only exception is erlang:monitor where a possible incompatibility is introduced: An attempt to monitor a process on a primitive node (such as erl_interface or jinterface), where remote process monitoring is not implemented, will no longer fail with badarg exception. Instead a monitor will be created, but it will only supervise the connection to the node.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14370

Changed the default behaviour of .erlang loading: .erlang is no longer loaded from the current directory. c:erlangrc(PathList) can be used to search and load an .erlang file from user specified directories.

escript, erlc, dialyzer and typer no longer load an .erlang at all.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14439

New functionality for implementation of alternative carriers for the Erlang distribution has been introduced. This mainly consists of support for usage of distribution controller processes (previously only ports could be used as distribution controllers). For more information see ERTS User's Guide ➜ How to implement an Alternative Carrier for the Erlang Distribution ➜ Distribution Module.

Own Id: OTP-14459

Add support for the lcc compiler and in extension the Elbrus 2000 platform.

Own Id: OTP-14492

Support for "tuple calls" have been removed from the run-time system. Tuple calls was an undocumented and unsupported feature which allowed the module argument for an apply operation to be a tuple: Var = dict:new(), Var:size(). This "feature" frequently caused confusion, especially when such call failed. The stacktrace would point out functions that don't exist in the source code.

For legacy code that need to use parameterized modules or tuple calls for some other reason, there is a new compiler option called tuple_calls. When this option is given, the compiler will generate extra code that emulates the old behavior for calls where the module is a variable.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14497

Creation of small maps with literal keys has been optimized to be faster and potentially use less memory. The keys are combined into a literal key tuple which is put into the literal pool. The key tuple can be shared between many instances of maps having the same keys.

Own Id: OTP-14502

When an exception is thrown, include the arguments of the call in the stacktrace for BIFs band, bor, bsl, bsr, bxor, div, rem and the operators +, -, * and /.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14508

The non-smp emulators have been removed. This means that the configure options --disable-threads and --enable-plain-emulator have been removed and configure will now refuse to build Erlang/OTP on platforms without thread support.

In order to achieve a similar setup as the non-smp emulator, it is possible to start Erlang/OTP with the +S 1 option.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14518

Modules that use floating point constants compiled with R15 or earlier will need to be re-compiled before they can be loaded.

Own Id: OTP-14575

Implementation of true asynchronous signaling between processes in order to improve scalability. Signals affected include exit, monitor, demonitor, monitor triggered, link, unlink, and group leader.

Own Id: OTP-14589

Added a PGO (profile guided optimization) pass to the build step of erts. This can be disabled by passing --disable-pgo to configure.

Own Id: OTP-14604

Improved the performance of binary:split and binary:match.

Own Id: OTP-14610 Aux Id: PR-1480

It is not longer possible to disable dirty schedulers when building erlang.

Own Id: OTP-14613

Loaded BEAM code in a 64-bit system requires less memory because of better packing of operands for instructions.

These memory savings were achieved by major improvements to the beam_makeops scripts used when building the run time system and BEAM compiler. There is also new for documentation for beam_makeops that describes how new BEAM instructions and loader transformations can be implemented. The documentation is found in here in a source directory or git repository: erts/emulator/internal_doc/beam_makeops.md. An online version can be found here: https://github.com/erlang/otp/blob/master/erts/emulator/internal_doc/beam_makeops.md

Own Id: OTP-14626

file:read_file has been changed to read the content of files that report a size of 0 even when data can be read from them. An example of such a file is /proc/cpuinfo on Linux.

Own Id: OTP-14637 Aux Id: ERL-327 PR-1524

It is no longer possible to disable the temp_alloc allocator. Disabling it caused serious performance degradations and was never what was wanted.

Own Id: OTP-14651

The reduction cost of sending messages is now constant. It will no longer scale according to the length of the receiving process' message queue.

Own Id: OTP-14667

Improved loading of modules with -on_load directive, to no longer block all schedulers when the load operation is completed.

Own Id: OTP-14680

On platforms with real-time signals available, SIGRTMIN+1 is now used as the internal scheduler suspend signal instead of SIGUSR2.

Own Id: OTP-14682

When the value returned from a 'catch' expression is ignored, no stacktrace will be built if an exception is caught. That will save time and produce less garbage. There are also some minor optimizations of 'try/catch' both in the compiler and run-time system.

Own Id: OTP-14683

The guarantees and non-guarantees of erlang:get_stacktrace/0 are now documented.

Own Id: OTP-14687

There is a new syntax in 'try/catch' for retrieving the stacktrace without calling 'erlang:get_stacktrace/0'. See the reference manual for a description of the new syntax. The 'erlang:get_stacktrace/0' BIF is now deprecated.

Own Id: OTP-14692

New 'used' option for binary_to_term/2 that will also return number of bytes actually read from the binary. This enables easy access to any extra data in the binary located directly after the returned term.

Own Id: OTP-14780

Added more statistics for erlang:system_info({allocator,A}) in the mbcs_pool section.

Own Id: OTP-14795 Aux Id: ERL-88

Added enif_ioq_peek_head to retrieve Erlang terms from NIF IO queues without having to resort to copying.

Own Id: OTP-14797

There is a new option 'makedep_side_effect' for the compiler and -MMD for 'erlc' that generates dependencies and continues to compile as normal.

Own Id: OTP-14830

Added ets:whereis/1 for retrieving the table identifier of a named table.

Own Id: OTP-14884

seq_trace labels may now be any erlang term.

Own Id: OTP-14899

Optimized the common case of monitor followed by send to the same local process. The monitor signal is now delayed in order to be piggybacked with the sent message and thereby only get one lock operation on the message queue of the receiver. A delayed monitor signal is flushed if no send has been done at the latest when the process is scheduled out.

Own Id: OTP-14901

Make hipe compiled code work on x86_64 (amd64) with OS security feature PIE, where executable code can be loaded into a random location. Old behavior, if hipe was enabled, was to disable PIE build options for the VM.

Own Id: OTP-14903

The number of driver async threads will now default to 1 as the standard drivers do not use them anymore. Users that changed this value to tweak the file driver should replace +A with +SDio since it now uses dirty IO schedulers instead of async threads.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14928

Optimize == and /= for binaries with different sizes to be constant in time instead of proportional to the size of their common prefix.

Own Id: OTP-14934 Aux Id: PR-1708

Refactorings making some internal process flags available for other usage.

Own Id: OTP-14948

Removed need for HiPE to allocate native executable memory in low 2GB address space on x86_64. Command line option +MXscs is thereby obsolete and ignored.

Own Id: OTP-14951

Added enif_make_map_from_arrays for creating a populated map, analogous to enif_make_list_from_array.

Own Id: OTP-14954

Added configuration switches for busy-wait and wake up thresholds for dirty schedulers, and changing these settings for normal schedulers will no longer affect dirty schedulers.

Refer to the documentation for details. The new switches are +sbwtdcpu, +sbwtdio, +swtdcpu, and +swtdio.

The default busy wait threshold for dirty scheduler threads has also been lowered to short.

Own Id: OTP-14959

The list of "taints" now also includes dynamic loaded drivers in addition to NIF libraries. Statically linked drivers and NIF libraries that are part of erts are not included. The "taints" are returned by system_info(taints) and printed in the header of erl_crash.dump files.

Own Id: OTP-14960

Added instrument:allocations and instrument:carriers for retrieving information about memory utilization and fragmentation.

The old instrument interface has been removed, as have the related options +Mim and +Mis.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14961

The process suspend functionality used by the erlang:suspend_process/2 BIF has been reimplemented using the newly introduced true asynchronous signaling between processes. This mainly to reduce memory usage in the process control block of all processes, but also in order to simplify the implementation.

Other ERTS internal functionality that used the previous process suspend functionality have also been reimplemented to use asynchronous signaling instead.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14964 Aux Id: OTP-14589

Added the nifs option to ?MODULE:module_info/1 for listing a module's installed NIF functions.

Own Id: OTP-14965

New implementation of erlang:process_info/[1,2].

In the general case when inspecting another process, the new implementation sends an asynchronous process-info request signal to the other process and waits for the result instead of locking the other process and reading the result directly. In some special cases where no conflicts occur, signal order wont be violated, and the amount of data requested is guaranteed to be small, the inspected process may be inspected directly.

Appropriate amount of reductions are now also bumped when inspecting a process.

Own Id: OTP-14966

Removed process start time from crash dump in order to save memory in process control block.

Own Id: OTP-14975 Aux Id: PR-1597

Optimize erlang:put/2 when updating existing key with a new immediate value (atom, small integer, pid, port).

Own Id: OTP-14976

erlang:process_info/1 has been changed to no longer include messages by default. Instead erlang:process_info/2 should be used.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14986 Aux Id: PR-1745

New erlang:system_info(ets_count) to get total number of ets tables existing at the local node.

Own Id: OTP-14987

New NIF functions: enif_mutex_name, enif_cond_name, enif_rwlock_name, enif_thread_name, enif_vfprintf, enif_vsnprintf.

Own Id: OTP-14994

When erlang:system_flag(backtrace_depth, 0) has been called, all exceptions will now contain the entry for one function (despite the zero). It used to be that a hand-made stack backtrace passed to erlang:raise/3 would be be truncated to an empty list.

Own Id: OTP-15026

Fixed bug for named ets tables which could cause unexpected results from matchspec iteration functions (ets:select* and ets:match*) if the table was deleted and recreated with the same name during the iteration. The iteration could incorrectly continue through the recreated table. The expected correct behavior is now for the iteration call to fail with a badarg exception if the table is deleted before the iteration has completed.

Own Id: OTP-15031

Two new guards BIFs operating on maps have been added: map_get/2 and is_map_key/2. They do the same as maps:get/2 and maps:is_key/2, respectively, except that they are allowed to be used in guards.

Own Id: OTP-15037 Aux Id: PR-1784, PR-1802

Release run-queue lock while cleaning up terminated dirty process.

Own Id: OTP-15081

The callback module passed as -epmd_module to erl has been expanded to be able to do name and port resolving.

Documentation has also been added in the erl_epmd reference manual and ERTS User's Guide How to Implement an Alternative Service Discovery for Erlang Distribution.

Own Id: OTP-15086 Aux Id: PR-1694

Added an optional ./configure flag to compile the emulator with spectre mitigation: --with-spectre-mitigation

Note that this requires a recent version of GCC with support for spectre mitigation and the --mindirect-branch=thunk flag, such as 8.1.

Own Id: OTP-15430 Aux Id: ERIERL-237

A bug that could cause dirty schedulers to become unresponsive has been fixed.

Own Id: OTP-15509 Aux Id: PR-2027, PR-2093

Fixed bug in operator band of two negative operands causing erroneous result if the absolute value of one of the operands have the lowest N*W bits as zero and the other absolute value is not larger than N*W bits. N is an integer of 1 or larger and W is 32 or 64 depending on word size.

Own Id: OTP-15487 Aux Id: ERL-804

List subtraction (The -- operator) will now yield properly on large inputs.

Own Id: OTP-15371

ERTS internal trees of monitor structures could get into an inconsistent state. This could cause 'DOWN' messages not to be delivered when they should, as well as delivery of 'DOWN' messages that should not be delivered.

This bug was introduced in ERTS version 9.0 (OTP 20.0) and was fixed in ERTS version 10.0 (OTP 21.0) due to a rewrite of the monitor code. That is, this bug only exist in the OTP 20 release.

Own Id: OTP-15399 Aux Id: ERL-751, ERIERL-262, OTP-14205

Fixed bug in ets:select_replace when called with a fully bound key could cause a following call to ets:next or ets:prev to crash the emulator or return invalid result.

Own Id: OTP-15346

Fixed a bug which caused an emulator crash when enif_send() was called by a NIF that executed on a dirty scheduler. The bug was either triggered when the NIF called enif_send() without a message environment, or when the process executing the NIF was send traced.

Own Id: OTP-15223

Fixed a bug causing some Erlang references to be inconsistently ordered. This could for example cause failure to look up certain elements with references as keys in search data structures. This bug was introduced in R13B02.

Thanks to Simon Cornish for finding the bug and supplying a fix.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-15225

Fixed a race condition in the inet driver that could cause receive to hang when the emulator was compiled with gcc 8.

Own Id: OTP-15158 Aux Id: ERL-654

Fix bug in generation of erl_crash.dump, which could cause VM to crash.

Bug exist since erts-9.2 (OTP-20.2).

Own Id: OTP-15181

Fixed a rare bug that could cause processes to be scheduled after they had been freed.

Own Id: OTP-15067 Aux Id: ERL-573

Fixed bug in ets that could cause VM crash if process A terminates after fixating a table and process B deletes the table at "the same time". The table fixation could be done with ets:safe_fixtable or if process A terminates in the middle of a long running select or match call.

Own Id: OTP-15109

Fixed bug in enif_binary_to_term which could cause memory corruption for immediate terms (atoms, small integers, pids, ports, empty lists).

Own Id: OTP-15080

Fixed bug in erlang:system_profile/2 that could cause superfluous {profile,_,active,_,_} messages for terminating processes.

Own Id: OTP-15085

Fixed a crash in heart:get_cmd/0 when the stored command was too long.

Own Id: OTP-15034

Fixed a configure test for libz internals that unintentionally caused various native code in OTP to link against libz. Under certain circumstances this caused the build of OTP to fail.

Own Id: OTP-14840 Aux Id: ERL-529

File names containing unicode codepoints larger than 255 were not correctly encoded in stack traces.

Own Id: OTP-14847 Aux Id: ERL-544

Fix HiPE bug for binary constructs like <<X/utf8>> which could in rare cases cause faulty results or VM crash.

This fix affects both the hipe compiler and erts runtime in an incompatible way. Old hipe compiled files need to be recompiled to load and run properly as native.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14850 Aux Id: PR-1664

Fix term_to_binary/2 spec for minor_version.

Own Id: OTP-14876 Aux Id: ERL-548

Fix bug in erlang:binary_to_integer/2 where invalid characters were not detected for bases larger then 10. e.g. binary_to_integer(<<":">>, 16) would return 3 and not badarg as it should.

Own Id: OTP-14879

Fixed bug in float_to_list/2 and float_to_binary/2 with option decimals that caused a faulty rounding up of the last decimal digit for about 6% of floats with a fraction part.

For example, float_to_list(0.145, [{decimals,1}]) returned "0.2" instead of "0.1".

Own Id: OTP-14890

Fix bug causing slow hipe execution in modules loaded early during boot or loaded by code:atomic_load or code:finish_loading.

Own Id: OTP-14891

Fixed a buffer overflow in an internal string formatting function that could be hit if specifying a long floating-point format specifier to erts_sprintf or similar.

Own Id: OTP-14920

erlang:iolist_to_iovec/1 and enif_inspect_iovec will no longer fail when provided with binaries that have been matched-out on a non-byte boundary.

Own Id: OTP-14921

iolist_to_binary/1 and erlang:iolist_to_iovec/1 will now badarg if supplied with a bitstring (without a list).

Own Id: OTP-14926

Reject loading modules with names containing directory separators ('/' or '\' on Windows).

Own Id: OTP-14933 Aux Id: ERL-564, PR-1716

Fix potential dead-lock when the tracer process dies while a the traced process is running on a dirty scheduler.

Own Id: OTP-14938

More crash dump info such as: process binary virtual heap stats, full info for process causing out-of-mem during GC, more port related info, and dirty scheduler info.

Own Id: OTP-14820

Improve search algorithm of abandoned memory carriers. Instead of limited linear search, each allocator instance maintain a balanced search tree of all its abandoned carriers for faster and more exhaustive search.

Own Id: OTP-14915 Aux Id: ERIERL-88

New erts_alloc command line options +M_acnl and +M_acfml to limit carrier abandonment.

Own Id: OTP-14916 Aux Id: ERIERL-88

New family of erts_alloc strategies: Age Order First Fit. Similar to "address order", but instead the oldest possible carrier is always chosen for allocation.

Own Id: OTP-14917 Aux Id: ERIERL-88

Add possibility to change allocator options at runtime with system_info(erts_alloc, ...). Only option sbct (single block carrier threshold) is currently supported via this interface.

Own Id: OTP-14918 Aux Id: ERIERL-88

Extra internal consistency checks wrt communication with erl_child_setup process.

Own Id: OTP-15488 Aux Id: ERIERL-231

Fix a bug in tracing where the {caller} match spec function would be set to undefined incorrectly when used in conjunction with return_to or return_trace on some functions.

The functions effected are: erlang:put/2, erlang:erase/1, erlang:process_info/1,2, erlang:nif_load/2, erts_internal:garbage_collection/1 and erts_internal:check_process_code/1.

Because of this bug, the analysis done by fprof could become incorrect when the functions above are the tail-call in a function.

Own Id: OTP-14677

Fix emulator deadlock that would happen if trap_exit was set to true and a process sends an exit signal to itself using exit(self(), Reason) while receive tracing was enabled for that process.

Own Id: OTP-14678 Aux Id: ERL-495

Writing of crash dumps is significantly faster.

Maps are now included in crash dumps.

Constants terms would only be shown in one process, while other processes referencing the same constant term would show a marker for incomplete heap.

Own Id: OTP-14685 Aux Id: OTP-14611, OTP-14603, OTP-14595

The fallback home directory for windows has been changed to be the PROFILE directory instead of the WINDOWS directory. The fallback is used when the environment variables HOMEDRIVE and HOMEPATH have not been set.

Own Id: OTP-14691

Fix bug for hipe compiled code using <<X/utf32>> binary construction that could cause faulty result or even VM crash.

On architectures other than x86_64, code need to be recompiled to benefit from this fix.

Own Id: OTP-14740

Fixed bug in erlang:garbage_collect/2 and erlang:check_process_code/3, when called with option {async,ReqestId}. Could cause VM crash or heap corruption if RequestId was an immediate term (like a pid, atom or small integer). Bug exists since OTP-17.0.

Own Id: OTP-14752

ERL_NIF_MINOR_VERSION wasn't bumped with the addition of enif_ioq_*.

Own Id: OTP-14779

Purging of loaded code that contained "fake literals" (for example the magic reference obtained from 'ets:new/2') would crash the runtime system. Corrected.

Own Id: OTP-14791

Setting the size of the atom table to a number near 2147483647 (using the '+t' option) would cause the emulator to exit with a failure to allocate a huge amount of memory. This has been corrected. Also the usage message for the '+t' option has been corrected to show the correct upper limit 2147483647 instead of 0.

Own Id: OTP-14796

Fixed a bug that prevented registered process names from being resolved in lcnt results.

Own Id: OTP-14803

Formatting bugs were fixed in several HiPE debug BIFs.

Own Id: OTP-14804

Binaries and some other data in crash dumps are now encoded in base64 (instead of in hex), which will reduce the size of crash dumps.

A few bugs in the handling of sub binaries in crashdump_viewer have been fixed.

Own Id: OTP-14686

Micro optimization for send operations of messages to other nodes. The local ack-message, which is otherwise sent back from TPC/IP port driver to sending client process, is now ignored earlier for distributed send operations.

Own Id: OTP-14689

Fixed a bug in file closure on Unix; close(2) was retried on EINTR which could cause a different (recently opened) file to be closed as well.

Own Id: OTP-14775

A race-condition when tearing down a connection with active node monitors could cause the runtime system to crash.

This bug was introduced in ERTS version 8.0 (OTP 19.0).

Own Id: OTP-14781 Aux Id: OTP-13047

Microstate accounting sometimes produced incorrect results for dirty schedulers.

Own Id: OTP-14707

Fixed a regression in zlib:gunzip/1 that prevented it from working when the decompressed size was a perfect multiple of 16384. This regression was introduced in 20.1.1

Own Id: OTP-14730 Aux Id: ERL-507

Fixed a memory corruption bug in enif_inspect_iovec; writable binaries stayed writable after entering the iovec.

Own Id: OTP-14745

Fixed a crash in enif_inspect_iovec on encountering empty binaries.

Own Id: OTP-14750

zlib:deflateParams/3 will no longer return buf_error when called after zlib:deflate/2 with zlib 1.2.11.

Own Id: OTP-14751

Added zlib:set_controlling_process/2 to move a zstream() between processes.

Own Id: OTP-14672 Aux Id: ERL-494

Fix so that schedulers are bound correctly when the first available cpu is not the first detected cpu. e.g. when using "taskset -c X..Y" when X is not equal to 0.

Own Id: OTP-14694

Fixed bug that could cause a VM crash when a corrupt message is received on distribution channel from other node.

Own Id: OTP-14661 Aux Id: ERIERL-80

The new zlib module returned a data_error when inflating concatenated streams, which was incompatible with the old module's behavior of returning the uncompressed data up to the end of the first stream.

Own Id: OTP-14648

zlib:gunzip/1 will no longer stop at the end of the first stream when decompressing concatenated gzip files.

Own Id: OTP-14649

Changed erlang:apply/2 to raise a badarg exception if the second argument is not a proper list. Previous behavior was a misleading undef exception.

Own Id: OTP-14490 Aux Id: ERL-432

On macOS, crypto would crash if observer had been started before crypto. On the beta for macOS 10.13 (High Sierra), crypto would crash. Both of those bugs have been fixed.

Own Id: OTP-14499 Aux Id: ERL-251 ERL-439

Fixed bug in enif_whereis_pid/port that could cause heap corruption in rare cases.

Own Id: OTP-14523

Fix so that trace messages generated when in a dirty nif are flushed correctly when the dirty nif is done executing.

Own Id: OTP-14538

Fix escape code handling when using ANSI color codes in the shell.

Own Id: OTP-14549 Aux Id: PR1536

Upgraded the ERTS internal PCRE library from version 8.40 to version 8.41. See http://pcre.org/original/changelog.txt for information about changes made to PCRE. This library implements major parts of the re regular expressions module.

Own Id: OTP-14574

Fixed a bug causing statistics(runtime) to produce negative values and a bug in statistics(wall_clock) causing it to produce values one second too long.

statistics(runtime) now also use getrusage() as source when available preventing the returned value from wrapping as frequent as before.

Own Id: OTP-14597 Aux Id: ERL-465

Fixed small memory leak that could occur when sending to a terminating port.

Own Id: OTP-14609

Fix bug causing VM crash when a module with -on_load directive is loaded while erlang:trace(on_load, ...) is enabled.

Own Id: OTP-14612

A warning that the compiler may optimize away atoms have been added to the documentation of list_to_existing_atom/1 and binary_to_existing_atom/2.

Own Id: OTP-14614 Aux Id: ERL-453

Lock counting can now be fully toggled at runtime in the lock counting emulator (-emu_type lcnt). Everything is enabled by default to match the old behavior, but specific categories can be toggled at will with minimal runtime overhead when disabled. Refer to the documentation on lcnt:rt_mask/1 for details.

Own Id: OTP-13170

The zlib module has been refactored and all its operations will now yield appropriately, allowing them to be used freely in concurrent applications.

The following functions have been deprecated, but will not produce compiler warnings until OTP 21: zlib:adler32, zlib:crc32, zlib:inflateChunk, zlib:getBufSize, zlib:setBufSize.

The behavior of throwing an error when a dictionary is required for decompression has also been deprecated. Refer to the documentation on inflateSetDictionary/2 for details.

Own Id: OTP-14185

lcnt:collect and lcnt:clear will no longer block all other threads in the runtime system.

Own Id: OTP-14412

Add erlang:iolist_to_iovec/1, which converts an iolist() to an erlang:iovec(), which suitable for use with enif_inspect_iovec.

Own Id: OTP-14520

When provided with bad arguments, the zlib module will now raise named exceptions instead of just badarg. For example, not_initialized when using zlib:inflate/2 with an uninitialized stream.

Own Id: OTP-14527

erlang:halt/2 allows any Unicode string as slogan for the crash dump.

Own Id: OTP-14553

Add new nif API functions for managing an I/O Queue. The added functions are:

Own Id: OTP-14598

Fixed bug in binary_to_term and binary_to_atom that could cause VM crash. Typically happens when the last character of an UTF8 string is in the range 128 to 255, but truncated to only one byte. Bug exists in binary_to_term since ERTS version 5.10.2 (OTP_R16B01) and binary_to_atom since ERTS version 9.0 (OTP-20.0).

Own Id: OTP-14590 Aux Id: ERL-474

A timer internal bit-field used for storing scheduler id was too small. As a result, VM internal timer data structures could become inconsistent when using 1024 schedulers on the system. Note that systems with less than 1024 schedulers are not effected by this bug.

This bug was introduced in ERTS version 7.0 (OTP 18.0).

Own Id: OTP-14548 Aux Id: OTP-11997, ERL-468

Automatic cleanup of a BIF timer, when the owner process terminated, could race with the timeout of the timer. This could cause the VM internal data structures to become inconsistent which very likely caused a VM crash.

This bug was introduced in ERTS version 9.0 (OTP 20.0).

Own Id: OTP-14554 Aux Id: OTP-14356, ERL-468

Binary append operations did not check for overflow, resulting in nonsensical results when huge binaries were appended.

Own Id: OTP-14524

Added missing release notes for OTP-14491 ("performance bug in pre-allocators") which was included in erts-9.0.1 (OTP-20.0.1).

Own Id: OTP-14494

Fixed a bug that prevented TCP sockets from being closed properly on send timeouts.

Own Id: OTP-14509

Fixed bug in operator bxor causing erroneuos result when one operand is a big negative integer with the lowest N*W bits as zero and the other operand not larger than N*W bits. N is an integer of 1 or larger and W is 32 or 64 depending on word size.

Own Id: OTP-14514

Fixed a bug in gen_tcp:send where it never returned when repeatedly called on a remotely closed TCP socket.

Own Id: OTP-13939 Aux Id: ERL-193

Fixed segfault that could happen during cleanup of aborted erlang:port_command/3 calls. A port_command is aborted if the port is closed at the same time as the port_command was issued. This bug was introduced in erts-8.0.

Own Id: OTP-14481

Fixed implementation of statistics(wall_clock) and statistics(runtime) so that values do not unnecessarily wrap due to the emulator. Note that the values returned by statistics(runtime) may still wrap due to limitations in the underlying functionality provided by the operating system.

Own Id: OTP-14484

Fix performance bug in pre-allocators that could cause them to permanently fall back on normal more expensive memory allocation. Pre-allocators are used for quick allocation of short lived meta data used by messages and other scheduled tasks. Bug exists since OTP_R15B02. [this release note was missing in erts-9.0.1]

Own Id: OTP-14491

Fix various bugs regarding loading, upgrade and purge of HiPE compiled code:

Own Id: OTP-13968

Hipe optional LLVM backend does require LLVM version 3.9 or later as older versions forced strong dependencies on erts internals structures.

Own Id: OTP-14238

When an exception such as 'throw(HugeTerm)' was caught, HugeTerm term would be kept in the process until the next exception occurred, potentially increasing the heap size for the process. That has been corrected.

Own Id: OTP-14255 Aux Id: OTP-14400, OTP-14401

Slogans in crash dumps have been extended to print more complex terms.

Own Id: OTP-14303

Fixed bug when using enif_inspect_binary in combination with enif_copy. In some circumstances the inspected binary could be reallocated by the enif_copy call when it shouldn't have been.

Own Id: OTP-14304

The address family local (AF_UNIX / AF_LOCAL) now does not ensure zero termination of Linux Abstract Addresses so they can use all bytes.

Own Id: OTP-14305

Use -fno-PIE for Gentoo Hardened and others that don't accept linker flag -no-pie.

Own Id: OTP-14307 Aux Id: PR-1379

Disable hipe for ppc64le architecture (little endian) as it is not, and has never been, supported. It was earlier equated with ppc64 (big endian) which lead to broken build without --disable-hipe.

Own Id: OTP-14314 Aux Id: ERL-369, PR-1394

Fix 'epmd -kill' to return a failure exit status code if epmd was not killed because of some error.

Own Id: OTP-14324

Fixed the following dirty scheduler related bugs:

Own Id: OTP-14335

Fix erlexec to handle mismatch in sysconf and proc fs when figuring out the cpu topology. This behaviour has been seen when using docker together with --cpuset-cpus.

Own Id: OTP-14352

Fixed memory segment cache used for multiblock carriers. Huge (> 2GB) memory segments could cause a VM crash. Creation of such huge memory segments used for multiblock carriers is however very uncommon.

Own Id: OTP-14360 Aux Id: ERL-401, PR-1417

Fix bug causing code:is_module_native to falsely return true when local call trace is enabled for the module.

Own Id: OTP-14390

Fix emulator crash when receive tracing on a trace_delivered message.

Own Id: OTP-14411

Fix file:sendfile error handling on SunOS when a connection is closed during transmission.

Own Id: OTP-14424

escript did not handle paths with spaces correct.

Own Id: OTP-14433

Fix erroneous lock check assertion when wx is run on MacOS X.

Own Id: OTP-14437 Aux Id: ERL-360

Active-mode TCP sockets are now cleaned up properly on send/shutdown errors.

Own Id: OTP-14441 Aux Id: ERL-430

Fix compilation of hipe_mkliterals when the LIBS configure variable had to be set.

Own Id: OTP-14447

Added erlang:garbage_collect/2 that takes an option list as the last argument that can be used to control whether a minor or a major garbage collection is to be done. Doing a minor collection only collects terms that have recently died, but is cheaper than a major collection.

Own Id: OTP-11695

Optimized test for tuples with an atom as first element.

Own Id: OTP-12148

Erlang literals are no longer copied during process to process messaging.

Own Id: OTP-13529

Add support in the erl_nif API for asynchronous message notifications when sockets or other file descriptors are ready to accept read or write operations. The following functions have been added:

Own Id: OTP-13684

There are two new guard BIFs 'floor/1' and 'ceil/1'. They both return integers. In the 'math' module, there are two new BIFs with the same names that return floating point values.

Own Id: OTP-13692

Remove deprecated erlang:hash/2.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13827

Replaced usage of deprecated symbolic time unit representations.

Own Id: OTP-13831 Aux Id: OTP-13735

Added support in zlib for extraction of the inflation dictionary.

Own Id: OTP-13842

The previously used purge strategy has been removed. The optional purge strategy introduced in ERTS version 8.1 is now the only strategy available.

The new purge strategy is slightly incompatible with the old strategy. Previously processes holding funs that referred to the module being purged either failed a soft purge, or was killed during a hard purge. The new strategy completely ignores funs. If funs referring to the code being purged exist, and are used after a purge, an exception will be raised upon usage. That is, the behavior will be exactly the same as the case when a fun is received by a process after the purge.

For more information see the documentation of erlang:check_process_code/3.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13844 Aux Id: OTP-13833

Dirty schedulers are now enabled by default when the runtime system is built with SMP support.

Own Id: OTP-13860

Improved ETS lookup/insert/delete speed for large set, bag and duplicate_bag by a significant reduction of the hash load factor. This speed improvement comes at the expense of less than one word per table entry. Tables with less than 256 entries are not affected at all.

Own Id: OTP-13903

The NIF library reload feature is not supported anymore. It has been marked as deprecated since OTP R15B. This means that you are only allowed to do one successful call to erlang:load_nif/2 for each module instance. A second call to erlang:load_nif/2 will return {error, {reload, _}} even if the NIF library implements the reload callback.

Runtime upgrade of a NIF library is still supported by using the Erlang module upgrade mechanics with a current and an old module instance existing at the same time with their corresponding NIF libraries.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13908

Add erlang:system_info(atom_count) and erlang:system_info(atom_limit) to provide a way to retrieve the current and maximum number of atoms.

Own Id: OTP-13976

The function fmod/2 has been added to the math module.

Own Id: OTP-14000

erlang:load_nif/2 returns new error type notsup when called for a HiPE compiled module, which is not supported.

Own Id: OTP-14002

Add driver and nif lock instrumentation to lcnt

Own Id: OTP-14069

Reduce memory pressure by converting sub-binaries to heap-binaries when possible. This is done during garbage collection.

Own Id: OTP-14149

Dirty schedulers are now enabled and supported on Erlang runtime systems with SMP support.

Besides support for dirty NIFs also support for dirty BIFs and dirty garbage collection have been introduced. All garbage collections that potentially will take a long time to complete are now performed on dirty schedulers if enabled.

erlang:statistics/1 with arguments inspecting scheduler and run queue states have been changed due to the dirty scheduler support. Code using this functionality may have to be rewritten taking these incompatibilities into consideration. Examples of such uses are calls to erlang:statistics(scheduler_wall_time), statistics(total_run_queue_lengths), statistics(total_active_tasks), etc.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14152

Atoms may now contain arbitrary Unicode characters.

Own Id: OTP-14178

Introduce an event manager in Erlang to handle OS signals. A subset of OS signals may be subscribed to and those are described in the Kernel application.

Own Id: OTP-14186

The escript program now handles symbolic links to escripts.

This is useful for standalone systems with escripts residing on a bin directory not included in the execution path (as it may cause their erl program(s) to override the desired one). Instead the escripts can be referred to via symbolic links from a bin directory in the path.

Own Id: OTP-14201 Aux Id: PR-1293

All uses of the magic binary kludge has been replaced by uses of erlang references.

A magic binary was presented as an empty binary, but actually referred other data internally in the Erlang VM. Since they were presented as empty binaries, different magic binaries compared as equal, and also lost their internal data when passed out of an erlang node.

The new usage of references has not got any of these strange semantic issues, and the usage of these references has been optimized to give the same performance benefits as well as memory usage benefits as magic binaries had.

A couple of examples of previous uses of magic binaries are match specifications and NIF resources.

Own Id: OTP-14205

The non-smp emulators have been deprecated and are scheduled for removal in OTP-21.

In preparation for this, the threaded non-smp emulator is no longer built by default and has to be enabled using the --enable-plain-emulator to configure.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14272

Allow HiPE to run on VM built with --enable-m32-build.

Own Id: OTP-14330 Aux Id: PR-1397

Upgraded the OTP internal PCRE library from version 8.33 to version 8.40. This library is used for implementation of the re regular expressions module.

Besides various bug fixes, the new version allows for better stack protection. In order to utilize this feature, the stack size of normal scheduler threads is now by default set to 128 kilo words on all platforms. The stack size of normal scheduler threads can be set upon system start by passing the +sss command line argument to the erl command.

See http://pcre.org/original/changelog.txt for information about changes made to PCRE between the versions 8.33 and 8.40.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14331 Aux Id: ERL-208

Remove generation of atoms in old latin1 external format in the distribution between erlang nodes, erl_interface, and jinterface. The new utf8 format for atoms was introduced in OTP R16. An OTP 20 node can therefore not connect to nodes older than R16.

Atoms that can be encoded using latin1 are still encoded by term_to_binary() using latin1 encoding. Note that all atoms will by default be encoded using utf8 in a future Erlang/OTP release. For more information see the documentation of erlang:term_to_binary/2.

Own Id: OTP-14337

Added function re:version/0 which returns information about the OTP internal PCRE version used for implementation of the re module.

Own Id: OTP-14347 Aux Id: PR-1412

Added new debug bif erlang:list_to_port/1.

Own Id: OTP-14348

Various improvements of timer management internally in the VM. These improvements both reduced memory consumption of timer wheels as well as reduce the amount of work that has to be performed in order to handle timers.

Own Id: OTP-14356

Sockets can now be bound to device (SO_BINDTODEVICE) on platforms where it is supported.

This has been implemented e.g to support VRF-Lite under Linux; see VRF , and GitHub pull request #1326.

Own Id: OTP-14357 Aux Id: PR-1326

Added the following erl command line arguments with which you can set suggested stack for dirty schedulers:

The default suggested stack size for dirty schedulers is 40 kilo words.

Own Id: OTP-14380

Changed erts startup program name, argv 0, to use the environment variable ESCRIPT_NAME so that erlc, dialyzer, typer, ct_run, or the escript name can be seen with external programs, such as ps and htop (depending on options), on unix.

Own Id: OTP-14381

Improvements of escript documentation.

Own Id: OTP-14384 Aux Id: OTP-14201

Add function enif_hash for NIFs to calculate hash values of arbitrary terms.

Own Id: OTP-14385 Aux Id: PR-1413

'./configure --enable-lock-counter' will enabling building of an additional emulator that has support for lock counting. (The option previously existed, but would turn on lock counting in the default emulator being built.) To start the lock-counting emulator, use 'erl -emu_type lcnt'.

On Windows, erl recognized the undocumented option -debug for starting a debug-compiled emulator. That option has been removed. Use 'erl -emu_type debug' instead.

Own Id: OTP-14407

Warnings have been added to the relevant documentation about not using un-secure distributed nodes in exposed environments.

Own Id: OTP-14425

Improvement of the documentation of the environment variable ERL_CRASH_DUMP_SECONDS as well as the default behavior when it is not set.

Own Id: OTP-14434

Enabled off-heap message queue for some system processes that might receive large amounts of messages.

Own Id: OTP-14438

ETS lock indexes have been replaced with the table name in LCNT results.

Own Id: OTP-14442 Aux Id: ERIERL-22

Introduced the new functions enif_whereis_pid() and enif_whereis_port().

Own Id: OTP-14453 Aux Id: PR-1400

Fixed bug in operator band of two negative operands causing erroneous result if the absolute value of one of the operands have the lowest N*W bits as zero and the other absolute value is not larger than N*W bits. N is an integer of 1 or larger and W is 32 or 64 depending on word size.

Own Id: OTP-15487 Aux Id: ERL-804

Added an optional ./configure flag to compile the emulator with spectre mitigation: --with-spectre-mitigation

Note that this requires a recent version of GCC with support for spectre mitigation and the --mindirect-branch=thunk flag, such as 8.1.

Own Id: OTP-15430 Aux Id: ERIERL-237

Fixed small memory leak that could occur when sending to a terminating port.

Own Id: OTP-14609 Aux Id: ERIERL-238

Fixed a race condition in the inet driver that could cause receive to hang when the emulator was compiled with gcc 8.

Own Id: OTP-15158 Aux Id: ERL-654

Fixed a bug causing some Erlang references to be inconsistently ordered. This could for example cause failure to look up certain elements with references as keys in search data structures. This bug was introduced in R13B02.

Thanks to Simon Cornish for finding the bug and supplying a fix.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-15225

Fixed a bug in file closure on Unix; close(2) was retried on EINTR which could cause a different (recently opened) file to be closed as well.

Own Id: OTP-14775

A race-condition when tearing down a connection with active node monitors could cause the runtime system to crash.

This bug was introduced in ERTS version 8.0 (OTP 19.0).

Own Id: OTP-14781 Aux Id: OTP-13047

A timer internal bit-field used for storing scheduler id was too small. As a result, VM internal timer data structures could become inconsistent when using 1024 schedulers on the system. Note that systems with less than 1024 schedulers are not effected by this bug.

This bug was introduced in ERTS version 7.0 (OTP 18.0).

Own Id: OTP-14548 Aux Id: OTP-11997, ERL-468

Fixed bug in binary_to_term and binary_to_atom that could cause VM crash. Typically happens when the last character of an UTF8 string is in the range 128 to 255, but truncated to only one byte. Bug exists in binary_to_term since ERTS version 5.10.2 (OTP_R16B01) and binary_to_atom since ERTS version 9.0 (OTP-20.0).

Own Id: OTP-14590 Aux Id: ERL-474

Fix bug causing VM crash when a module with -on_load directive is loaded while erlang:trace(on_load, ...) is enabled.

Own Id: OTP-14612

Fixed bug that could cause a VM crash when a corrupt message is received on distribution channel from other node.

Own Id: OTP-14661 Aux Id: ERIERL-80

Fix performance bug in pre-allocators that could cause them to permanently fall back on normal more expensive memory allocation. Pre-allocators are used for quick allocation of short lived meta data used by messages and other scheduled tasks. Bug exists since OTP_R15B02.

Own Id: OTP-14491

Fixed a bug that prevented TCP sockets from being closed properly on send timeouts.

Own Id: OTP-14509

Fixed bug in operator bxor causing erroneuos result when one operand is a big negative integer with the lowest N*W bits as zero and the other operand not larger than N*W bits. N is an integer of 1 or larger and W is 32 or 64 depending on word size.

Own Id: OTP-14514

Fixed a bug in gen_tcp:send where it never returned when repeatedly called on a remotely closed TCP socket.

Own Id: OTP-13939 Aux Id: ERL-193

Fixed segfault that could happen during cleanup of aborted erlang:port_command/3 calls. A port_command is aborted if the port is closed at the same time as the port_command was issued. This bug was introduced in erts-8.0.

Own Id: OTP-14481

Fixed implementation of statistics(wall_clock) and statistics(runtime) so that values do not unnecessarily wrap due to the emulator. Note that the values returned by statistics(runtime) may still wrap due to limitations in the underlying functionality provided by the operating system.

Own Id: OTP-14484

Active-mode TCP sockets are now cleaned up properly on send/shutdown errors.

Own Id: OTP-14441 Aux Id: ERL-430

A code purge operation could under certain circumstances expand the size of hibernated processes.

Own Id: OTP-14444 Aux Id: ERIERL-24

Fix so that the ERL_ZZ_SIGTERM_KILL introduced in erts-8.3.4 works.

Own Id: OTP-14451

Add option to make SIGTERM trigger the OS default behaviour instead of doing a gracefull shutdown. To activate this bahviour the environment variable ERL_ZZ_SIGTERM_KILL should be set to "true". This option only works in OTP 19 as OTP 20 will have a different way to deal with SIGTERM.

Own Id: OTP-14418 Aux Id: ERIERL-15

Fixed memory segment cache used for multiblock carriers. Huge (> 2GB) memory segments could cause a VM crash. Creation of such huge memory segments used for multiblock carriers is however very uncommon.

Own Id: OTP-14360 Aux Id: ERL-401, PR-1417

Fix release note for OTP-14290 in ERTS version 8.3.1. It was erroneously placed under "Known Bugs and Problems".

Own Id: OTP-14363 Aux Id: OTP-14290

The +Bi command line argument of erl erroneously caused SIGTERM to be ignored by the VM as well as of all its child processes. This bug was introduced in erts version 8.3.

Own Id: OTP-14358 Aux Id: OTP-14085

Invoking init:stop/0 via the SIGTERM signal, in a non-SMP BEAM, could cause BEAM to terminate with fatal error. This has now been fixed and the BEAM will terminate normally when SIGTERM is received.

Own Id: OTP-14290

Trying to open a directory with file:read_file/1 on Unix leaked a file descriptor. This bug has now been fixed.

Own Id: OTP-14308 Aux Id: ERL-383

Fixed a number of bugs that caused faulty stack-traces to be generated. The faulty stack traces were generated either when applying the following functions or tracing the following functions:

Own Id: OTP-14055

Corrected documentation about memory footprint for maps.

Own Id: OTP-14118

Fix process_info(Pid, current_stacktrace) to use stack depth limit set by system_flag(backtrace_depth). The old behavior was a hard coded depth limit of 8.

Own Id: OTP-14119 Aux Id: PR-1263

A process calling erlang:system_flag(multi_scheduling, block) could end up hanging forever in the call.

Own Id: OTP-14121

Dirty scheduler bug fixes:

Own Id: OTP-14122

Fixed crash that occurred when writing timer data to a crash dump.

Own Id: OTP-14133

A literal area could be removed while still referred from processes.

Own Id: OTP-14134

Fixed a bug in the garbage collector that could crash the runtime system.

Own Id: OTP-14135

Fixed a bug in call-time trace for NIFs which caused tracing to erroneously be started multiple times for one call.

Own Id: OTP-14136

Remove a debug printout and an unnecessary garbage collection when handling exceptions in hipe compiled code.

Own Id: OTP-14153

Fix bug in tracing of garbage collection that could cause VM crash. Bug exists since OTP 19.0.

Own Id: OTP-14154

Fix bug in binary_to_term for binaries created by term_to_binary with option compressed. The bug can cause badarg exception for a valid binary when Erlang VM is linked against a zlib library of version 1.2.9 or newer. Bug exists since OTP 17.0.

Own Id: OTP-14159 Aux Id: ERL-340

Fix suspension of schedulers when generating a crashdump.

Own Id: OTP-14164

NIF resources was not handled in a thread-safe manner in the runtime system without SMP support.

As a consequence of this fix, the following driver functions are now thread-safe also in the runtime system without SMP support:

Own Id: OTP-14202

Fix erlang:round/1 for large floating point numbers with an odd absolute value between (1 bsl 52) and (1 bsl 53). The result was falsely calculated as the next higher even number even though all integer values up to (1 bsl 53) can be represented as floats with full precision.

Own Id: OTP-14227

Add size of literals to module code size in crash dump and (l)oaded command in break menu like it used to be before OTP-19.0.

Own Id: OTP-14228

Fix potential bug in enif_send when called without a process context and with argument msg_env as NULL.

Own Id: OTP-14229

Fix bug where passing an appendable binary to erlang:port_control() could crash the emulator.

Own Id: OTP-14231

Receive expressions with timeout in the Erlang shell could cause a VM crash.

Own Id: OTP-14241 Aux Id: ERL-365

A received SIGTERM signal to beam will generate a 'stop' message to the init process and terminate the Erlang VM nicely. This is equivalent to calling init:stop/0.

Own Id: OTP-14085

Workaround for buggy Android implementation of PTHREAD_STACK_MIN causing build of runtime system to crash on undeclared PAGE_SIZE.

Own Id: OTP-14165 Aux Id: ERL-319

Add configure option --without-thread-names that removes the naming of individual emulator threads.

Own Id: OTP-14234

Add warning in documentation of zlib:deflateInit/6 about option WindowsBits values 8 and -8.

Own Id: OTP-14254 Aux Id: ERL-362

Fix bug in binary_to_term for binaries created by term_to_binary with option compressed. The bug can cause badarg exception for a valid binary when Erlang VM is linked against a zlib library of version 1.2.9 or newer. Bug exists since OTP 17.0.

Own Id: OTP-14159 Aux Id: ERL-340

The driver efile_drv when opening a file now use fstat() on the open file instead of stat() before opening, if fstat() exists. This avoids a race when the file happens to change between stat() and open().

Own Id: OTP-14184 Aux Id: seq-13266

Fix a quite rare bug causing VM crash during code loading and the use of export funs (fun M:F/A) of not yet loaded modules. Requires a very specfic timing of concurrent scheduler threads. Has been seen on ARM but can probably also occure on other architectures. Bug has existed since OTP R16.

Own Id: OTP-14144 Aux Id: seq13242

Fixed configure failures on MacOSX. Most important clock_gettime() was detected when building for MacOSX - El Capitan using XCode 8 despite it is not available until MacOSX - Sierra.

Own Id: OTP-13904 Aux Id: ERL-256

code:add_pathsa/1 and command line option -pa both revert the given list of directories when adding it at the beginning of the code path. This is now documented.

Own Id: OTP-13920 Aux Id: ERL-267

Fix a compilation error of erts in OpenBSD related to the usage of the __errno variable.

Own Id: OTP-13927

Fixed so that when enabling tracing on a process that had an invalid tracer associated with it, the new tracer overwrites the old tracer. Before this fix, calling erlang:trace/3 would behave as if the tracer was still alive and not apply the new trace.

This fault was introduced in ERTS 8.0.

Own Id: OTP-13928

Fix parsing of -profile_boot 'true' | 'false'

Own Id: OTP-13955 Aux Id: ERL-280

A slight improvement of erlang:get_stacktrace/0 for exceptions raised in hipe compiled code. Beam compiled functions in such stack trace was earlier replaced by some unrelated function. They are now instead omitted. This is an attempt to reduce the confusion in the absence of a complete and correct stack trace for mixed beam and hipe functions.

Own Id: OTP-13992

Correct type declaration of match specification head.

Own Id: OTP-13996

HiPE code loading failed for x86_64 if gcc was configured with --enable-default-pie. Fixed by disabling PIE, if needed for HiPE, when building the VM.

Own Id: OTP-14031 Aux Id: ERL-294, PR-1239

Faulty arguments could be presented on exception from a NIF that had rescheduled itself using enif_schedule_nif().

Own Id: OTP-14048

The runtime system could crash if a garbage collection on a process was performed immediately after a NIF had been rescheduled using enif_schedule_nif().

Own Id: OTP-14049

A reference to purged code could be left undetected by the purge operation if a process just had rescheduled a NIF call using enif_schedule_nif() when the process was checked. This could cause a runtime system crash.

Own Id: OTP-14050

Fixed a number of dirty scheduler related bugs:

Own Id: OTP-14051

Fix minor soft purge race bug that could incorrectly trigger code_server to load new code for the module if the soft purge failed and no current version of the module existed.

Own Id: OTP-13925

To ease troubleshooting, erlang:load_nif/2 now includes the return value from a failed call to load/reload/upgrade in the text part of the error tuple. The crypto NIF makes use of this feature by returning the source line where/if the initialization fails.

Own Id: OTP-13951

New environment variable ERL_CRASH_DUMP_BYTES can be used to limit the size of crash dumps. If the limit is reached, crash dump generation is aborted and the generated file will be truncated.

Own Id: OTP-14046

A code purge operation could under certain circumstances expand the size of hibernated processes.

Own Id: OTP-14444 Aux Id: ERIERL-24

The emulator got a dynamic library dependency towards libsctp, which on Linux was not intended since the emulator there loads and resolves the needed sctp functions in runtime. This has been fixed and a configure switch --enable-sctp=lib has been added for those who want such a library dependency.

Own Id: OTP-13956 Aux Id: ERL-262, ERL-133

Fix SIGUSR1 crashdump generation

Do not generate a core when a crashdump is asked for.

Own Id: OTP-13997

The new functions in code that allows loading of many modules at once had a performance problem. While executing a helper function in the erl_prim_loader process, garbage messages were produced. The garbages messages were ignored and ultimately discarded, but there would be a negative impact on performance and memory usage. The number of garbage message depended on both the number of modules to be loaded and the length of the code path.

The functions affected of this problem were: atomic_load/1, ensure_modules_loaded/1, and prepare_loading/1.

Own Id: OTP-14009

Fix bug for calls from hipe code to BIFs that disable GC while yielding. Has been causing Dialyzer crashes on ARM (and presumably all other non-intel platforms).

Own Id: OTP-13724 Aux Id: PR-1116

Fix a bug where changing the current working directory of the VM would not change the current working directory of programs spawned using erlang:open_port({spawn,""}, ...).

Own Id: OTP-13733 Aux Id: ERL-175

Fix a bug where disabling tracing from a process that had return_to tracing enabled and was tracing on erlang:trace/3 would cause a segmentation fault.

Own Id: OTP-13734

Update all erts documentation to use simpler English, use consistent terminology and be easier to navigate.

Own Id: OTP-13740

Add dirty schedulers to the microstate accounting statistics.

Own Id: OTP-13744

Fixed dirty scheduler build support on 32-bit windows.

Own Id: OTP-13759

inet:getstat(Socket) on an SCTP socket returned 0 for send stats. This bug has now been corrected. Reported by systra as issue ERL-102 on bugs.erlang.org.

Own Id: OTP-13773 Aux Id: ERL-102

AF_UNSPEC and unknown address families were misread by UDP receive in prim_inet resulting in an exception. This bug has now been corrected.

Own Id: OTP-13775

Sweep HiPE stack for literals during code purge.

Own Id: OTP-13777 Aux Id: PR-1122

Fix bug in run_erl for OpenBSD that could cause it on rare occations to exit without starting the program (erl) at all.

Own Id: OTP-13795

Update build scripts to not make assumtions about where env, cp and perl are located.

Own Id: OTP-13800

Fixed a bug where init:stop could deadlock if a process with infinite shutdown timeout (e.g. a supervisor) attempted to load code while terminating.

Own Id: OTP-13802

Fixed a segmentation fault on sparc CPUs when free'ing a tracer module's state.

Own Id: OTP-13803

funs was not properly handled during purge of a module. This could cause a crash of the VM after a purge of a module.

Own Id: OTP-13809

Fixed a memory leak when the process monitoring a port crashed.

Own Id: OTP-13818

Fixed multiple dirty scheduler related tracing bugs.

Own Id: OTP-13822

Fix error handling in beam code runtime loader for a number of cases when index and size fields got corrupted (negative) values.

Own Id: OTP-13848 Aux Id: ERL-216

Minor fix of dirty scheduler implementation.

Own Id: OTP-13852

Calls to erl_drv_send_term() or erl_drv_output_term() from a non-scheduler thread while the corresponding port was invalid caused the emulator to enter an inconsistent state which eventually caused an emulator crash.

Own Id: OTP-13866

Fix a rare race condition in erlang:open_port({spawn, ""}, ...) that would result in the erl_child_setup program aborting and cause the emulator to exit.

Own Id: OTP-13868

Driver and NIF operations accessing processes or ports could cause an emulator crash when used from non-scheduler threads. Those operations are:

Own Id: OTP-13869

Fix start scripts generation dependency in Makefile

Own Id: OTP-13871 Aux Id: ERL-241

The VM could crash if erlang:get_stacktrace() was called after a rescheduled NIF had thrown an exception.

Own Id: OTP-13877

Calling code:delete/1 before a loading a module with an on_load function, the old code would be overwritten, causing a memory or a crash if NIFs were involved. (Thanks to vans163 for reporting this bug.)

Own Id: OTP-13893 Aux Id: ERL-240

Improved accuracy of timeouts on MacOS X. This by setting premature timeouts followed by a short actual timeout during scheduler wait.

Own Id: OTP-13698

Added the following symbolic time unit representations to the erlang:time_unit() type:

The following symbolic time unit representations are now deprecated, but still part of the erlang:time_unit() type:

Own Id: OTP-13735

Fix maps hashing entropy of maps with maps keys.

Own Id: OTP-13763 Aux Id: ERL-199

Improved dirty scheduler support. A purge of a module can now be performed without having to wait for completion of all ongoing dirty NIF calls.

Note that when enabling support for dirty schedulers, a new purge strategy will as of ERTS version 8.1 be enabled. This new strategy is not fully backwards compatible with the strategy used by default. For more information see the documentation of erlang:check_process_code/3.

Own Id: OTP-13808 Aux Id: OTP-13833

A new purge strategy has been introduced. The new strategy will by default be disabled during the OTP 19 release, but will be the only strategy available as of the OTP 20 release.

The new strategy is slightly incompatible with the strategy being used by default in OTP 19. Using the default strategy, processes holding funs that refer to the module being purged either fail a soft purge, or will be killed during a hard purge. The new strategy completely ignores funs. If funs referring to the code being purged exist, and are used after a purge, an exception will be raised upon usage. That is, the behavior will be exactly the same as the case when a fun is received by a process after the purge.

The new strategy can optionally be enabled when building OTP during OTP 19, and will automatically be enabled if the runtime system is built with support for dirty schedulers.

For more information see the documentation of erlang:check_process_code/3.

Own Id: OTP-13833

Fixed unnecessary overestimation of heap size need during garbage collection.

Own Id: OTP-13851

Fixed a VM crash that occurred in a garbage collection of a process when it had received binaries. This bug was introduced in ERTS version 8.0 (OTP 19.0).

Own Id: OTP-13890

Fixed a VM crash that occurred in garbage collection of a process when it had received maps over the distribution. This bug was introduced in ERTS version 8.0 (OTP 19.0).

Own Id: OTP-13889

Fixed a race that could cause a lost wakeup of a process that timed out in a receive ... after. This bug was introduced in ERTS version 7.0.

Own Id: OTP-13798 Aux Id: OTP-11997

Fixed segfault after writing an erl crash dump.

Own Id: OTP-13799

Fix scheduler deadlock bug in ets:update_counter/4 when key is not found and inserting the default object causes the table to grow.

Own Id: OTP-13731 Aux Id: ERL-188

Fix VM abort "Overrun stack and heap" in garbage collection triggered by a bsl operation and some very specific heap conditions.

Own Id: OTP-13732 Aux Id: seq13142

A memory allocation bug in group_leader/2 could cause an emulator crash when garbage collecting a process that had been assigned a remote group leader. This bug was introduced in ERTS version 8.0.

Own Id: OTP-13716

The handling of on_load functions has been improved. The major improvement is that if a code upgrade fails because the on_load function fails, the previous version of the module will now be retained.

Own Id: OTP-12593

is_builtin(erlang, apply, 3) will now return true.

Own Id: OTP-13034

Fix enif_get_list_length to return false if list is improper or have length larger than UINT_MAX (did return true and an incorrect length value).

Own Id: OTP-13288 Aux Id: PR913

Cleanup hipe signal handling code for x86 and make it more portable.

Own Id: OTP-13341 Aux Id: PR951

Make file:datasync use fsync instead of fdatasync on Mac OSX.

Own Id: OTP-13411

Make sure to create a crash dump when running out of memory. This was accidentally removed in the erts-7.3 release.

Own Id: OTP-13419

A bug has been fixed where if erlang was started +B on a unix platform it would be killed by a SIGUSR2 signal when creating a crash dump.

Own Id: OTP-13425

Fix race between process_flag(trap_exit,true) and a received exit signal.

A process could terminate due to exit signal even though process_flag(trap_exit,true) had returned. A very specific timing between call to process_flag/2 and exit signal from another scheduler was required for this to happen.

Own Id: OTP-13452

Don't search for non-existing Map keys twice

For maps:get/2,3 and maps:find/2, searching for an immediate key, e.g. an atom, in a small map, the search was performed twice if the key did not exist.

Own Id: OTP-13459

When an abnormally large distribution message is about to be sent, the VM has been changed to create a crash dump instead of a core dump.

Own Id: OTP-13474

Fix erlang:process_info/2 type specification

Own Id: OTP-13485 Aux Id: ERL-123

Fix bug in open_port/2 with option {args, List}. A vm crash could be caused by an improper List.

Own Id: OTP-13489 Aux Id: ERL-127

Fixed a race-condition bug where the emulator could crash when erlang:system_profile/1,2 was enabled and a process had to be re-scheduled during termination.

Own Id: OTP-13494 Aux Id: ERL-126

Fixed bugs where the reduction counter was not handled correct.

Own Id: OTP-13512

Fixed typo in description of the EPMD_DUMP_REQ response.

Own Id: OTP-13517

Fixed a bug where a process flagged as sensitive would sometimes record its save_calls when it shouldn't.

Own Id: OTP-13540

Update configure scripts to not use hard-coded path for /bin/pwd and /bin/rm.

Own Id: OTP-13562

When passing a larger binary than the outputv callback of a linked-in driver can handle in one io vector slot, the binary is now split into multiple slots in the io vector. This change only effects system where the max size of an io vector slot is smaller then the word size of the system (e.g. Windows).

This change means that it is now possible on Windows to send binaries that are larger than 4GB to port_command, which is what is used for file:write, gen_tcp:send etc.

Own Id: OTP-13628

Workaround of Maps output in crashdumps. Currently the atom 'undefined' is generated instead of Map data if a Map type is encountered during crash.

Own Id: OTP-13657

The tracing support has been extended to allow a tracer module to be the trace event handler instead of a process or port. The tracer module makes it possible for trace tools to filter or manipulate trace event data without the trace event first having to be copied from the traced process or port.

With the introduction of this feature, erlang:trace(all|existing, _, _) now also returns the tracer process as part of the number of processes on which tracing is enabled. The is incompatible with the previous releases.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10267

Introduce LTTng tracing of Erlang Runtime System

For LTTng to be enabled OTP needs to be built with configure option --with-dynamic-trace=lttng.

This feature introduces tracepoints for schedulers, drivers, memory carriers, memory and async thread pool.

For a list of all tracepoints, see Runtime Tools User's Guide .

Own Id: OTP-10282

Make it possible to monitor/demonitor ports using the erlang:monitor/2 API. The process and port information functions have also been updated to include information about monitors from processes to ports.

Own Id: OTP-11384

Add microstate accounting

Microstate accounting is a way to track which state the different threads within ERTS are in. The main usage area is to pin point performance bottlenecks by checking which states the threads are in and then from there figuring out why and where to optimize.

Since checking whether microstate accounting is on or off is relatively expensive only a few of the states are enabled by default and more states can be enabled through configure.

There is a convenience module called msacc that has been added to runtime_tools that can assist in gathering and interpreting the data from Microstate accounting.

For more information see erlang:statistics(microstate_accounting, _) and the msacc module in runtime_tools.

Own Id: OTP-12345

The port of Erlang/OTP to the real-time operating system OSE has been removed.

Own Id: OTP-12573

Sharing preserved copy for messages and exit signals

Enable sharing preserved copy with configure option --enable-sharing-preserving. This will preserve sharing, within the process, when communication with other processes in the Erlang node. There is a trade-off, the copy is more costly but this cost can be reclaimed if there is a lot of sharing in the message. In addition literals will not be copied in a send except during a purge phase of the module where the literals are located. This feature is considered experimental in 19.0.

Own Id: OTP-12590 Aux Id: OTP-10251

Halfword BEAM has been removed.

Own Id: OTP-12883

Added os:perf_counter/1.

The perf_counter is a very very cheap and high resolution timer that can be used to timestamp system events. It does not have monoticity guarantees, but should on most OS's expose a monotonous time.

Own Id: OTP-12908

Support for a fragmented young heap generation. That is, the young heap generation can consist of multiple non continuous memory areas. The main reason for this change is to avoid extra copying of messages that could not be allocated directly on the receivers heap.

Own Id: OTP-13047

Erlang linked-in driver can now force the call to open_port to block until a call to erl_drv_init_ack is made inside the driver. This is useful when you want to do some asynchronous initialization, for example getting configuration from a pipe, and you want the initial open_port call to fail if the configuration is incomplete or wrong. See the erl_driver documentation for more details on the API.

Own Id: OTP-13086

Erlang linked-in drivers can now set their own pids as seen in erlang:port_info/1 by using the erl_drv_set_pid function. For more details see the erl_driver documentation.

Own Id: OTP-13087

The functionality behind erlang:open_port/2 when called with spawn or spawn_executable has been redone so that the forking of the new program is done in a separate process called erl_child_setup. This allows for a much more robust implementation that uses less memory and does not block the entire emulator if the program to be started is on an un-accessible NFS. Benchmarks have shown this approach to be about 3-5 times as fast as the old approach where the fork/vfork was done by erts. This is a pure stability and performance fix, however some error messages may have changed, which is why it is marked as a backwards incompatible change.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13088

Improved yielding strategy in the implementation of the following native functions:

This in order to improve performance of these functions.

Own Id: OTP-13096

All garbage collections of processes now bump reductions. Also the amount of reductions bumped when garbage collecting has been adjusted. It now better corresponds to the amount of work performed. This in order to improve the real time characteristics of the system.

Own Id: OTP-13097

New functions that can load multiple modules at once have been added to the 'code' module. The functions are code:atomic_load/1, code:prepare_loading/1, code:finish_loading/1, and code:ensure_modules_loaded/1.

Own Id: OTP-13111

The -boot_var option for erl now only supports a single key and single value (as documented). The option used to allow multiple key/value pairs, but that behavior was undocumented.

The function erl_prim_loader:start/3 has been removed. Its documentation has also been removed.

The undocumented and unsupported function erl_prim_loader:get_files/2 has been removed.

Own Id: OTP-13112

Low level BIF erlang:purge_module/1 is made more robust against incorrect use. Lingering processes that still refer the old code are now killed before the module is purged to prevent fatal VM behavior.

Own Id: OTP-13122

Improved dirty scheduler implementation. For more information see the NIF documentation.

Own Id: OTP-13123

Various optimizations done to process dictionary access.

Own Id: OTP-13167

Added max_heap_size process flag. max_heap_size allows the user to limit the maximum heap used by a process. See erlang:process_flag for more details.

Own Id: OTP-13174

Allow dynamic drivers and NIF libraries to be built with gcc option -fvisibility=hidden for faster loading and more optimized code.

Own Id: OTP-13227

Add erlang:process_info(Pid, garbage_collection_info) which returns extended garbage_collection information. For more details see the documentation.

Own Id: OTP-13265

The functions erlang:list_to_integer/1 and string:to_integer/1 have been optimized for large inputs.

Own Id: OTP-13293

Improved memory allocation strategy for hipe native code on x86_64 (amd64) architectures by reserving enough low virtual address space needed for the HiPE/AMD64 small code model. The default virtual address area for hipe code is set to 512Mb, but can be changed with emulator flag +MXscs.

Own Id: OTP-13359

Introduction of configurable management of data referred to by the message queue of a process. Each process can be configured individually.

It is now possible to configure the message queue of a process, so that all data referred by it will be kept outside of the heap, and by this prevent this data from being part of garbage collections.

For more information see the documentation of process_flag(message_queue_data, MQD).

Own Id: OTP-13366 Aux Id: OTP-13047

Processes now yield when scanning large message queues and not finding a matching message. This in order to improve real time characteristics.

Own Id: OTP-13401

Optimized an erts internal function that is used to traverse erlang terms. The internal function was mainly used by term_to_binary and comparison of terms. Benchmarks have shown up to a 10% performance increase in those functions after the optimization.

Own Id: OTP-13440

Add the following NIF API functions:

For details of what each function does, see the erl_nif documentation.

Own Id: OTP-13442

Optimize '++' operator and lists:append/2 by using a single pass to build a new list while checking for properness.

Own Id: OTP-13487

Handle terms (pids,ports and refs) from nodes with a 'creation' value larger than 3. This is a preparation of the distribution protocol to allow OTP 19 nodes to correctly communicate with future nodes (20 or higher). The 'creation' value differentiates different incarnations of the same node (name).

Own Id: OTP-13488

Don't send unasked for systemd notifications in epmd

Own Id: OTP-13493 Aux Id: PR-999

The enif_send API has been extended to allow NULL to be used as the message environment. When used this way, a message environment is implicitly created and the given term is copied into that environment before sending. This can be an optimization if many small messages are being sent by the nif.

Own Id: OTP-13495

The tracing support has been extended to allow tracing on ports. Ports can be traced on using the 'ports', 'send' and 'receive' trace flags.

The first argument of erlang:trace/3 has been extended so that 'all', 'existing' and 'new' now include both processes and ports. New Tracee variants, 'all_processes', 'all_ports', 'existing_processes' etc have been added to specify only processes or ports.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13496

When the 'procs' trace flag is enabled, a 'spawned' trace event is now also generated by a newly created process. The previous event 'spawn' remains, but as it is generated by the process that did the spawn, it is not guaranteed that it is ordered with other trace events from the newly spawned process. So when tracking the lifetime of a process this new event should be used as the creation event.

This new trace event is marked as an incompatibility because tools that expect certain trace events when enabling 'procs' will have to updated.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13497

Add the erlang:match_spec_test/3 function. The functions allows the testing of match specifications for both tracing and ets tables. It can be used to test that a match specification does the expected filtering on specific data. It also returns more verbose error reasons for incorrectly constructed match specifications.

Own Id: OTP-13501

The erts internal tracing support has been changed to have much less overhead and be more scalable.

This rewrite does not break any backwards incompatibilities, but it does change the ordering of some trace messages when compared to previous releases. It should be noted that this only applies to trace messages sent to processes or ports, it does not apply to the new tracer module. However in future releases they may also be effected by this.

Trace messages are only guaranteed to be ordered from one traced process or port. In previous releases this was not visible as a 'send' trace message would always arrive before the corresponding 'receive' trace message that is no longer always the case. This also means that timestamped trace messages may seem to arrive out of order as the timestamp is taken when the event is triggered and not when it is put in the queue of the tracer.

Own Id: OTP-13503

Add possibility to filter send and receive trace with match specifications.

Own Id: OTP-13507

Add maps:update_with/3,4 and maps:take/2

Own Id: OTP-13522 Aux Id: PR-1025

Introduce LTTng tracing via Erlang tracing.

For LTTng to be enabled OTP needs to be built with configure option --with-dynamic-trace=lttng.

The dynamic trace module dyntrace is now capable to be used as a LTTng sink for Erlang tracing. For a list of all tracepoints, see Runtime Tools User's Guide .

This feature also introduces an incompatible change in trace tags. The trace tags gc_start and gc_end has been split into gc_minor_start, gc_minor_end and gc_major_start, gc_major_end.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13532

Print heap pointers for garbing processes during crashdump

Own Id: OTP-13541 Aux Id: PR-1026

Changed and improved low level memory statistics returned by erlang:system_info/1. The info for erts_mmap has been moved from mseg_alloc to its own section returned by {allocator, erts_mmap}.

Own Id: OTP-13560

Add enif_snprintf to the NIF API

The function enif_snprintf is similar to snprintf call but can handle formatting of Erlang terms via %T format specifier.

Own Id: OTP-13580

The warning in the documentation for erlang:raise/3 has been removed. It is now officially perfectly fine to use raise/3 in production code.

Own Id: OTP-13599

Fix bugs caused by the VM sometimes truncating object sizes or offsets to 32 bits on 64-bit hosts. These bugs were mainly found when working with large unicode strings and nifs environments.

Own Id: OTP-13606

Add -start_epmd command line option, this lets you disable automatic starting of epmd when starting a distributed node.

Add -epmd_module command line option, this lets you specify a module to register and look-up node names in. The default module is erl_epmd.

Own Id: OTP-13627

erlang:halt now truncates strings longer than 200 characters instead of failing with badarg.

Own Id: OTP-13630

Fix possible race in poller wake up on windows

Own Id: OTP-13634

List subtraction (The -- operator) will now yield properly on large inputs.

Own Id: OTP-15371

Fixed small memory leak that could occur when sending to a terminating port.

Own Id: OTP-14609 Aux Id: ERIERL-238

Fix performance bug in pre-allocators that could cause them to permanently fall back on normal more expensive memory allocation. Pre-allocators are used for quick allocation of short lived meta data used by messages and other scheduled tasks. Bug exists since OTP_R15B02.

Own Id: OTP-14491

Fixed bug in operator bxor causing erroneuos result when one operand is a big negative integer with the lowest N*W bits as zero and the other operand not larger than N*W bits. N is an integer of 1 or larger and W is 32 or 64 depending on word size.

Own Id: OTP-14514

A timer internal bit-field used for storing scheduler id was too small. As a result, VM internal timer data structures could become inconsistent when using 1024 schedulers on the system. Note that systems with less than 1024 schedulers are not effected by this bug.

This bug was introduced in ERTS version 7.0 (OTP 18.0).

Own Id: OTP-14548 Aux Id: OTP-11997, ERL-468

Fixed bug in binary_to_term and binary_to_atom that could cause VM crash. Typically happens when the last character of an UTF8 string is in the range 128 to 255, but truncated to only one byte. Bug exists in binary_to_term since ERTS version 5.10.2 (OTP_R16B01) and binary_to_atom since ERTS version 9.0 (OTP-20.0).

Own Id: OTP-14590 Aux Id: ERL-474

A bug has been fixed where if erlang was started +B on a unix platform it would be killed by a SIGUSR2 signal when creating a crash dump.

Own Id: OTP-13425 Aux Id: ERL-94

Calls to erl_drv_send_term() or erl_drv_output_term() from a non-scheduler thread while the corresponding port was invalid caused the emulator to enter an inconsistent state which eventually caused an emulator crash.

Own Id: OTP-13866

Driver and NIF operations accessing processes or ports could cause an emulator crash when used from non-scheduler threads. Those operations are:

Own Id: OTP-13869

Fix bug in binary_to_term for binaries created by term_to_binary with option compressed. The bug can cause badarg exception for a valid binary when Erlang VM is linked against a zlib library of version 1.2.9 or newer. Bug exists since OTP 17.0.

Own Id: OTP-14159 Aux Id: ERL-340

Fixed a race that could cause a lost wakeup of a process that timed out in a receive ... after. This bug was introduced in ERTS version 7.0.

Own Id: OTP-13798 Aux Id: OTP-11997

Fixed segfault after writing an erl crash dump.

Own Id: OTP-13799

Fix scheduler deadlock bug in ets:update_counter/4 when key is not found and inserting the default object causes the table to grow.

Own Id: OTP-13731 Aux Id: ERL-188

process_info(Pid, last_calls) did not work for Pid /= self().

Own Id: OTP-13418

Make sure to create a crash dump when running out of memory. This was accidentally removed in the erts-7.3 release.

Own Id: OTP-13419

Schedulers could be woken by a premature timeout on Linux. This premature wakeup was however harmless.

Own Id: OTP-13420

A process communicating with a port via one of the erlang:port_* BIFs could potentially end up in an inconsistent state if the port terminated during the communication. When this occurred the process could later block in a receive even though it had messages matching in its message queue.

This bug was introduced in erts version 5.10 (OTP R16A).

Own Id: OTP-13424 Aux Id: OTP-10336

The reference count of a process structure could under rare circumstances be erroneously managed. When this happened invalid memory accesses occurred.

Own Id: OTP-13446

Fix race between process_flag(trap_exit,true) and a received exit signal.

A process could terminate due to exit signal even though process_flag(trap_exit,true) had returned. A very specific timing between call to process_flag/2 and exit signal from another scheduler was required for this to happen.

Own Id: OTP-13452

The '-path' flag to 'erl' has been documented. This flag replaces the path specified in the boot script. It has always existed, but was earlier only documented in SASL (script).

Own Id: OTP-13060

The call_time tracing functionality internally used a time based on OS system time in order to measure call time which could cause erroneous results if OS system time was changed during tracing.

This functionality now use Erlang monotonic time in order to measure time. Besides fixing the erroneous results due to OS system time being used, the results are often also better since Erlang monotonic time often has better accuracy and precision.

Own Id: OTP-13216

Fix behaviour of -delay_write command line switch of epmd, which is used for debugging - in some cases epmd was sleeping twice the requested amount of time.

Own Id: OTP-13220

Fix race between timeout and exit signal that could cause a process to ignore the exit signal and continue execution. Bug exist since OTP 18.0.

Own Id: OTP-13245

Fix bug in erlang:halt/1,2 for large exit status values, causing either badarg (on 32-bit) or exit with a crash dump and/or core dump (on 64-bit). Make erlang:halt/1,2 tolerate any non negative integer as exit status and truncate high order bits if the OS does not support it.

Own Id: OTP-13251 Aux Id: ERL-49

gen_tcp:accept/2 was not time warp safe. This since it used the same time as returned by erlang:now/0 when calculating timeout. This has now been fixed.

Own Id: OTP-13254 Aux Id: OTP-11997, OTP-13222

Fix faulty error handling when writing to a compressed file.

Own Id: OTP-13270

Fix sendfile usage for large files on FreeBSD

Own Id: OTP-13271

Fix bug that could cause process_info(P,current_location) to crash emulator for hipe compiled modules.

Own Id: OTP-13282 Aux Id: ERL-79

Out of memory errors have been changed to cause an exit instead of abort.

Own Id: OTP-13292

When calling garbage_collect/[1,2] or check_process_code/[2,3] from a process with a higher priority than the priority of the process operated on, the run queues could end up in an inconsistent state. This bug has now been fixed.

Own Id: OTP-13298 Aux Id: OTP-11388

A workaround for an issue with older gcc versions (less than 5) and inline assembly on 32-bit x86 caused an emulator crash when it had been compiled with a newer gcc version. An improved configure test, run when building OTP, now detects whether the workaround should be used or not.

Own Id: OTP-13326 Aux Id: ERL-80

Introduced new statistics functionality in order to more efficiently retrieve information about run able and active processes and ports. For more information see:

Own Id: OTP-13201

Time warp safety improvements.

Introduced the options monotonic_timestamp, and strict_monotonic_timestamp to the trace, sequential trace, and system profile functionality. This since the already existing timestamp option is not time warp safe.

Introduced the option safe_fixed_monotonic_time to ets:info/2 and dets:info/2. This since the already existing safe_fixed option is not time warp safe.

Own Id: OTP-13222 Aux Id: OTP-11997

Fix a register race where down nodes goes undetected in epmd

Own Id: OTP-13301

Improved the gcc inline assembly implementing double word atomic compare and exchange on x86/x86_64 so that it also can be used when compiling with clang.

Own Id: OTP-13336

An optimization preventing a long wait for a scheduler thread looking up information about a process executing on another scheduler thread had unintentionally been lost in erts-5.10 (OTP R16A). This optimization has now been reintroduced.

Own Id: OTP-13365 Aux Id: OTP-9892

Introduced new statistics functionality in order to more efficiently retrieve information about run able and active processes and ports. For more information see:

Own Id: OTP-13201

Revert "Fix erroneous splitting of emulator path"

Own Id: OTP-13202

Fix HiPE enabled emulator for FreeBSD.

Own Id: OTP-13204 Aux Id: pr926

Small documentation fixes

Own Id: OTP-13017

Fix memory corruption bug caused by disabling distribution and then re-enable distribution with a node name that has previously been used by a remote node.

Own Id: OTP-13076 Aux Id: seq12959

Renamed variables with name bool as Visual Studio 2015 now treats this is a keyword.

Own Id: OTP-13079

erl_prim_loader has not supported custom loaders for several releases. In the documentation for erl_prim_loader, all references to custom loaders have now been removed.

Own Id: OTP-13102

Fixed compilation of erts together with libc versions that do not define __uint32_t.

Own Id: OTP-13105

erl -make now returns non-zero exit codes on failure

Own Id: OTP-13107

Fix crash on init:restart in embedded mode caused by on_load handler process not being relaunched leading to load failure for modules such as crypto and asn1rt_nif that need it to be present for correct NIF loading.

Own Id: OTP-13115

Fix maps decode in erlang:binary_to_term/1

Decoding a term with a large (HAMT) map in an small (FLAT) map could cause a critical error if the external format was not produced by beam.

Own Id: OTP-13125

Fix very rare bug in GC when big maps with a lot of hash collisions from a remote node are waiting in inner message queue.

Own Id: OTP-13146

Fixed a bug that could cause a crash dump to become almost empty.

Own Id: OTP-13150

Updated the xmllint target to just check the xml files with real documentation content.
Corrected some errors and added some missing target in the DTD's.

Own Id: OTP-13026

Add function enif_getenv to read OS environment variables in a portable way from NIFs.

Own Id: OTP-13147

Fix bug in ETS that could cause stray objects marked for deletion to occasionally be missed by the cleanup done by safe_fixtable(_,false).

Own Id: OTP-12870

Fixed VM crash that could occur if a trace port was linked to a process, and the trace port terminated abnormally while handling a trace message. This bug has always existed in the runtime system with SMP support.

Own Id: OTP-12901

Instead of aborting, the vm now creates a crash dump when a system process is terminated.

Own Id: OTP-12934

Fixed a rare emulator dead lock that occurred when erlang:process_flag(priority,...) was called by a process that was also scheduled for an internal system activity.

Own Id: OTP-12943

The runtime system on various posix platforms (except for Linux and Solaris) could crash when large amounts of file-descriptors were in use.

Own Id: OTP-12954

A beam file compiled by hipe for an incompatible runtime system was sometimes not rejected by the loader, which could lead to vm crash. This fix will also allow the same hipe compiler to be used by both normal and debug-built vm.

Own Id: OTP-12962

Fix bug in maps:merge/2 when called by hipe compiled code that could cause vm crash. Bug exists since erts-7.0 (OTP 18.0).

Own Id: OTP-12965

When tracing with process_dump option, the VM could abort if there was an ongoing binary match somewhere in the call stack of the traced process.

Own Id: OTP-12968

Fixed possible output deadlock in tty driver when hitting "CTRL-C" in a non-smp emulator shell on unix.

Own Id: OTP-12987 Aux Id: Seq12947

Fix binary_to_integer to throw badarg for "+" and "-" similar to list_to_integer.

Own Id: OTP-12988

Suppress warning of unused argument when using macro enif_make_pid.

Own Id: OTP-12989

Changed default clock source used for OS system time on MacOS X to gettimeofday() in order to improve performance. The system can be configured during build to use the previously used higher resolution clock source by passing the switch --with-clock-resolution=high when configuring the build.

Own Id: OTP-12945 Aux Id: OTP-12892

Added the configure option --disable-saved-compile-time which disables saving of compile date and time in the emulator binary.

Own Id: OTP-12971

Fixed a binary memory leak when printing to shell using the tty driver (i.e. not -oldshell).

Own Id: OTP-12941

Fix a bug where the standard error port sometimes crashes with eagain as the reason.

Own Id: OTP-12942

When tracing with process_dump option, the VM could abort if there was an ongoing binary match somewhere in the call stack of the traced process./

Own Id: OTP-12968

A process could end up in an inconsistent half exited state in the runtime system without SMP support. This could occur if the processes was traced by a port that it also was linked to, and the port terminated abnormally while handling a trace message for the process.

This bug has always existed in the runtime system without SMP support, but never in the runtime system with SMP support.

Own Id: OTP-12889 Aux Id: seq12885

Removed unnecessary copying of data when retrieving corrected Erlang monotonic time.

Own Id: OTP-12894

Changed default OS monotonic clock source chosen at build time. This in order to improve performance. The behavior will now on most systems be that (both OS and Erlang) monotonic time stops when the system is suspended.

If you prefer that monotonic time elapse during suspend of the machine, you can pass the command line argument --enable-prefer-elapsed-monotonic-time-during-suspend to configure when building Erlang/OTP. The configuration stage will try to find such a clock source, but might not be able to find it. Note that there might be a performance penalty associated with such a clock source.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12895

erlang:system_info(end_time) returned a faulty value on 32-bit architectures.

Own Id: OTP-12896

The configure command line argument --enable-gettimeofday-as-os-system-time has been added which force usage of gettimeofday() for OS system time. This will improve performance of os:system_time() and os:timestamp() on MacOS X, at the expense of worse accuracy, resolution and precision of Erlang monotonic time, Erlang system time, and OS system time.

Own Id: OTP-12892

Fix a rare hanging of the VM seen to happen just after emulator start. Bug exists since R14.

Own Id: OTP-12859 Aux Id: seq12882

Fix issuing with spaces and quoting in the arguments when using erlang:open_port spawn_executable on windows. The behavior now mimics how unix works. This change implies a backwards incompatibility for how spawn_executable works on windows.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11905

Fix global call trace when hipe compiled code call beam compiled functions. Tracing of beam functions should now alway work regardless who the caller is.

Own Id: OTP-11939

Correct cache alignment for ETS write_concurrency locks to improve performance by reduced false sharing. May increase memory footprint for tables with write_concurrency.

Own Id: OTP-11974

All possibly blocking operations in the fd/spawn and terminal driver have been converted to non-blocking operations. Before this fix it was possible for the VM to be blocked for a long time if the entity consuming stdout/stderr did not consume it fast enough.

Own Id: OTP-12239

Add missing overhead for offheap binaries created from external format. This fix can improve the garbage collection of large binaries originating from binary_to_term or messages from remote nodes.

Own Id: OTP-12554

Ensure hashing of zero is consistent

Erlang treats positive and negative zero as equal:

true = 0.0 =:= 0.0/-1

However, Erlangs hash functions: hash, phash and phash2 did not reflect this behaviour. The hash values produced by the different hash functions would not be identical for positive and negative zero.

This change ensures that hash value of positive zero is always produced regardless of the signedness of the zero float, i.e.,

true = erlang:phash2(0.0) =:= erlang:phash2(0.0/-1)

Own Id: OTP-12641

Ensure NIF term creation disallows illegal floating point values and too long atoms. Such values will cause a NIF to throw badarg exception when it returns.

Own Id: OTP-12655

Fixed building of Map results from match_specs

A faulty "box-value" entered into the heap which could cause a segmentation fault in the garbage collector if it was written on a heap fragment.

Own Id: OTP-12656

Fix hipe bug when matching a "writable" binary. The bug has been seen to sometimes cause a failed binary matching of a correct utf8 character, but other symptoms are also possible.

Own Id: OTP-12667

Keep dirty schedulers from waking other schedulers.

Own Id: OTP-12685

Disable floating point exceptions if the VM is compiled by clang/llvm. This is a known long-standing problem in clang/llvm.

Own Id: OTP-12717

Fix bug in file:sendfile for FreeBSD causing not the entire file to be sent.

Own Id: OTP-12720

Fix the broken Android support in erl_child_setup.c

Own Id: OTP-12751

Faulty statistics reported by the fix_alloc allocator.

Own Id: OTP-12766

Fix two erts_snprintf() calls to correct sizes.

- run_erl.c (ose): Use the size of the signal type, not its pointer. - erl_node_tables.c: Use the size of the _BUFFER in erts_snprintf() to make sure we can use the full space.

Own Id: OTP-12771

Delayed memory allocations could be delayed an unnecessarily long time.

Own Id: OTP-12812

Make sure that timeouts on a pool of acceptors are released in the correct order.

Own Id: OTP-12817

Fix segmentation fault in module_info for deleted modules

Own Id: OTP-12820

Fix garbage collection of literals in code purge

During code purging and check_process_code, the checking of the binary reference embedded in the match binary state was omitted for the tracing tests. This would cause the binary match state to reference deallocated memory.

Own Id: OTP-12821

A bug has been corrected for gen_tcp:close so when {linger,{true,0}} is in effect it does not wait for data in the driver queue to transfer out before closing the port. Bug fix by Rory Byrne.

Own Id: OTP-12840

The documentation of the driver callback start() erroneously stated that a return value of ERL_DRV_ERROR_ERRNO caused the error value to be passed via erl_errno when it should have been errno.

Own Id: OTP-12855

Add md5 and module entries to ?MODULE:module_info/0/1 and remove obsolete entry 'import'.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11940

Debug function erlang:display/1 shows content of binaries and bitstrings, not only the length.

Own Id: OTP-11941

The time functionality of Erlang has been extended. This both includes a new API for time, as well as time warp modes which alters the behavior of the system when system time changes. You are strongly encouraged to use the new API instead of the old API based on erlang:now/0. erlang:now/0 has been deprecated since it is and forever will be a scalability bottleneck. For more information see the Time and Time Correction chapter of the ERTS User's Guide.

Besides the API changes and time warp modes a lot of scalability and performance improvements regarding time management has been made internally in the runtime system. Examples of such improvements are scheduler specific timer wheels, scheduler specific BIF timer management, parallel retrieval of monotonic time and system time on systems with primitives that are not buggy.

Own Id: OTP-11997

erlang:function_exported(M, F, A) will now return true if M:F/A refers to a BIF.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12099

New BIF: erlang:get_keys/0, lists all keys associated with the process dictionary. Note: erlang:get_keys/0 is auto-imported.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12151 Aux Id: seq12521

Make distributed send of large messages yield to improve real-time characteristics.

Own Id: OTP-12232

Use high accuracy poll timeouts

Where available, use poll/select API's that can handle time resolutions less than 1ms. In the cases where such API's are not available the timeout is rounded up to the nearest ms.

Own Id: OTP-12236

The internal group to user_drv protocol has been changed to be synchronous in order to guarantee that output sent to a process implementing the user_drv protocol is printed before replying. This protocol is used by the standard_output device and the ssh application when acting as a client.

This change changes the previous unlimited buffer when printing to standard_io and other devices that end up in user_drv to 1KB.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12240

The previously introduced "eager check I/O" feature is now enabled by default.

Eager check I/O can be disabled using the erl command line argument: +secio false

Characteristics impact compared to previous default:

Own Id: OTP-12254 Aux Id: OTP-12117

Properly support maps in match_specs

Own Id: OTP-12270

The notice that a crashdump has been written has been moved to be printed before the crashdump is generated instead of afterwords. The wording of the notice has also been changed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12292

New function ets:take/2. Works the same as ets:delete/2 but also returns the deleted object(s).

Own Id: OTP-12309

Tracing with cpu_timestamp option has been enabled on Linux.

Own Id: OTP-12366

ets:info/1,2 now contains information about whether write_concurrency or read_concurrency is enabled.

Own Id: OTP-12376

Improved usage of gcc's builtins for atomic memory access. These are used when no other implementation of atomic memory operations is available. For example, when compiling for ARM when libatomic_ops is not available.

The largest improvement will be seen when compiling with a gcc with support for the __atomic_* builtins (using a gcc of at least version 4.7), but also when only the legacy __sync_* builtins are available (using a gcc of at least version 4.1) an improvement can be seen.

For more information see the "Atomic Memory Operations and the VM" section of $ERL_TOP/HOWTO/INSTALL.md.

Own Id: OTP-12383

Introduce math:log2/1 function to math module.

Own Id: OTP-12411

The documentation of the Abstract Format (in the ERTS User's Guide) has been updated with types and specification. (Thanks to Anthony Ramine.)

The explicit representation of parentheses used in types of the abstract format has been removed. Instead the new functions erl_parse:type_inop_prec() and erl_parse:type_preop_prec() can be used for inserting parentheses where needed.

Own Id: OTP-12492

Remove perfctr support

Development of perfctr in the linux kernel ceased in 2010. The perfctr support code in the Erlang VM is thus effectively dead code and therefor removed.

Own Id: OTP-12508

zlib:inflateChunk/2 has been added. It works like zlib:inflate/2, but decompresses no more data than will fit in the buffer configured by zlib:setBufSize/2.

Own Id: OTP-12548

Use linear search for small select_val arrays

Own Id: OTP-12555

New BIF ets:update_counter/4 with a default object as argument, which will be inserted in the table if the key was not found.

Own Id: OTP-12563

Export missing types from zlib module

Own Id: OTP-12584

Use persistent hashmaps for large Maps

Maps will use a persistent hashmap implementation when the number of pairs in a Map becomes sufficiently large. The change will occur when a Map reaches 33 pairs in size but this limit might change in the future.

The most significant impact for the user by this change is speed, and to a lesser degree memory consumption and introspection of Maps. Memory consumption size is probalistic but lesser than gb_trees or dict for instance. Any other impacts will be transparent for the user except for the following changes.

Semantics of Maps have changed in two incompatible ways compared to the experimental implementation in OTP 17:

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12585

Scalability improvement for erlang:make_ref/0, and other functionality that create references. Each scheduler now manage its own set of references. By this no communication at all is needed when creating references.

Previous implementation generated a strictly monotonically increasing sequence of references corresponding to creation time on the runtime system instance. This is not the case with current implementation. You can only expect reference to be unique. The Erlang/OTP documentation has never mentioned anything else but the uniqueness property, so this change is fully compatible. The only reason we've marked this as a potential incompatibility is since an early draft for an Erlang specification mentions strict monotonicity as a property.

If you need to create data with a strict monotonicity property use erlang:unique_integer([monotonic]). Do not use the deprecated erlang:now().

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12610

Enable different abort signal from heart

By using environment variable HEART_KILL_SIGNAL, heart can now use a different signal to kill the old running Erlang.

By default the signal is SIGKILL but SIGABRT may also be used by setting environment variable: HEART_KILL_SIGNAL=SIGABRT

Own Id: OTP-12613 Aux Id: seq12826

Update autconf to latest version 2015-03-04

Own Id: OTP-12646

Optimization of timers internally in the VM. This include process timers (receive ... after), port timers (driver_set_timer()) as well as BIF timers (erlang:send_after()/erlang:start_timer()).

Each scheduler thread now has its own lock-free timer service instead of one locked central service. This dramatically improves performance of timer management on systems with a large amount of schedulers and timers.

The timer service internal data structure has also been optimized to be able to handle more timers than before. That is, each timer service is by its self able to handle more timers without dramatic performance loss than the old centralized timer service.

The API of BIF timers has also been extended. Timeout values are for example no longer limited to 32-bit integers. For more information see the documentation of erlang:start_timer/4, erlang:send_after/4, erlang:cancel_timer/2, and erlang:read_timer/2.

Characteristics impact: Calls to the synchronous versions of erlang:cancel_timer(), and erlang:read_timer() may take substantially longer time to complete than before. This occur when the timer that is accessed is managed by a remote scheduler. You typically want to use the new asynchronous option in order to avoid blocking the calling process.

Own Id: OTP-12650 Aux Id: OTP-11997

Specialize instructions from common assembler patterns

Specialize common instructions of rem, band, minus and plus in the beam loader. This will reduce the number of fetches and thus lessen the instruction dispatch pressure during runtime and speed up those operations in some common cases.

Specialize move patterns from x-registers to the stack with a new move_window instruction. This change will reduce instruction dispatch pressure.

Own Id: OTP-12690

Fix cross compilation for Android.

Own Id: OTP-12693

Fix incorrect use of autoconf macro AC_EGREP_CPP, which could cause faulty configuration if run from a path containing the string 'yes'.

Own Id: OTP-12706

Minimal Java version is now 1.6

Own Id: OTP-12715

Send format and args on process exit to error_logger

Previously, the emulator would generate a whole string with values and call the error_logger passing "~s~n". This changes it to a format string containing ~p with the respective values as arguments.

Own Id: OTP-12735

Map error logger warnings to warning messages by default.

Own Id: OTP-12755

Configure architecture ppc64le architecture as a ppc64

Own Id: OTP-12761

Add function enif_raise_exception to allow a NIF to raise an error exception with any type of reason.

Own Id: OTP-12770

Optimized node table statistics retrieval.

Own Id: OTP-12777

Map beam error logger warnings to warning messages by default. Previously these messages were mapped to the error channel by default.

Own Id: OTP-12781

gen_tcp:shutdown/2 is now asynchronous

This solves the following problems with the old implementation:

It doesn't block when the TCP peer is idle or slow. This is the expected behaviour when shutdown() is called: the caller needs to be able to continue reading from the socket, not be prevented from doing so.

It doesn't truncate the output. The current version of gen_tcp:shutdown/2 will truncate any outbound data in the driver queue after about 10 seconds if the TCP peer is idle of slow. Worse yet, it doesn't even inform anyone that the data has been truncated: 'ok' is returned to the caller; and a FIN rather than an RST is sent to the TCP peer.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12797

Introduced delayed node table GC. This in order to avoid oscillation of entries in and out of the tables. The oscillation caused unnecessary lock contention on the table locks. The delay length can be set by passing the +zdntgc command line argument.

Characteristics impact: The tables can grow to very large sizes with unused entries if the node is get huge amounts of short lived connections from other nodes. This problem can be alleviated by shortening the length of the delay using the +zdntgc command line argument.

Own Id: OTP-12802

Improved implementation of erlang:statistics(io) in order to reduce contention between schedulers.

Characteristics impact: The actual call to erlang:statistics(io) takes longer time to complete, but the overall impact on the system is improved.

Own Id: OTP-12842

There are many cases where user code needs to be able to distinguish between a socket that was closed normally and one that was aborted. Setting the option {show_econnreset, true} enables the user to receive ECONNRESET errors on both active and passive sockets.

Own Id: OTP-12843

Do not preallocate too large event pool

A default pool size of 4000 is too excessive for the common case. This corresponds directly to the number of threads in the system. Change ERTS_TS_EV_ALLOC_DEFAULT_POOL_SIZE to 2048. Change ERTS_TS_EV_ALLOC_POOL_SIZE to 32.

Own Id: OTP-12849

A process communicating with a port via one of the erlang:port_* BIFs could potentially end up in an inconsistent state if the port terminated during the communication. When this occurred the process could later block in a receive even though it had messages matching in its message queue.

This bug was introduced in erts version 5.10 (OTP R16A).

Own Id: OTP-13424 Aux Id: OTP-10336

Calls to erl_drv_send_term() or erl_drv_output_term() from a non-scheduler thread while the corresponding port was invalid caused the emulator to enter an inconsistent state which eventually caused an emulator crash.

Own Id: OTP-13866

Driver and NIF operations accessing processes or ports could cause an emulator crash when used from non-scheduler threads. Those operations are:

Own Id: OTP-13869

Fix bug in binary_to_term for binaries created by term_to_binary with option compressed. The bug can cause badarg exception for a valid binary when Erlang VM is linked against a zlib library of version 1.2.9 or newer. Bug exists since OTP 17.0.

Own Id: OTP-14159 Aux Id: ERL-340

Fixed bug in operator bxor causing erroneuos result when one operand is a big negative integer with the lowest N*W bits as zero and the other operand not larger than N*W bits. N is an integer of 1 or larger and W is 32 or 64 depending on word size.

Own Id: OTP-14514

Fixed bug in binary_to_term and binary_to_atom that could cause VM crash. Typically happens when the last character of an UTF8 string is in the range 128 to 255, but truncated to only one byte. Bug exists in binary_to_term since ERTS version 5.10.2 (OTP_R16B01) and binary_to_atom since ERTS version 9.0 (OTP-20.0).

Own Id: OTP-14590 Aux Id: ERL-474

When calling garbage_collect/[1,2] or check_process_code/[2,3] from a process with a higher priority than the priority of the process operated on, the run queues could end up in an inconsistent state. This bug has now been fixed.

Own Id: OTP-13298 Aux Id: OTP-11388

Fixed a bug that could cause a crash dump to become almost empty.

Own Id: OTP-13150

The 'raw' socket option could not be used multiple times in one call to any e.g gen_tcp function because only one of the occurrences were used. This bug has been fixed, and also a small bug concerning propagating error codes from within inet:setopts/2.

Own Id: OTP-11482 Aux Id: seq12872

The VTS mode in Common Test has been modified to use a private version of the Webtool application (ct_webtool).

Own Id: OTP-12704 Aux Id: OTP-10922

Fix missing quotation in the LM_FIND_EMU_CC autoconf macro which could cause build failures.

Own Id: OTP-12388

Fix erroneous printout of monitors in crashdump file.

Own Id: OTP-12537

The runtime system without SMP support could crash in the BIF port_control/3 if the port that was being accessed died during the call to the BIF.

Own Id: OTP-12544 Aux Id: Seq12777

Avoid corrupt oversized integer to be created from binary matching. Instead throw system_limit exception which is the correct behavior. A peculiar symptom of this bug was that bitwise operations (band, bor, bxor) on such oversized integers could return the empty list []. Credit: Mikael Pettersson, Nico Kruber

Own Id: OTP-12556

A race condition when calling port_info/1 could cause a memory fault has been fixed.

Own Id: OTP-12587

Fix comparison of exact terms. An overflow that could cause faulty comparisons has been fixed. Comparison of exact terms is exclusively used within Maps.

Own Id: OTP-12623

Fix bug in list_to_integer/1 for very long lists that could cause VM crash.

Own Id: OTP-12624

Introduced a runtime system internal 64-bit API for atomic memory operations.

Own Id: OTP-12351

Add command line argument option for the initial size of process dictionaries.

Use '+hpds <size>' to set initial process dictionary size for spawned processes.

Own Id: OTP-12535 Aux Id: seq12809

Fix documentation on $char for Unicode

Own Id: OTP-12545

Fix getifaddrs realloc pointer error

When a buffer was exhausted and subsequently reallocated, we could get an unsafe pointer pointing to faulty memory.

For this to occur we would need to have a large number of interfaces and a reallocation of memory to a lower addresses.

The symptom would be garbage returned from erlang:port_control(Port, 25, []) (prim_inet:getifaddrs(Port) resulting in a badarg) or a segmentation fault.

Own Id: OTP-12445

Don't close all file descriptors twice in child_setup

The commit c2b4eab25c907f453a394d382c04cd04e6c06b49 introduced an error in which child_setup erroneously tried to close all file descriptors twice.

Use closefrom() if available when closing all file descriptors.

The function closefrom() was only used in the vfork() case before but is now also used in the fork() case if available.

Own Id: OTP-12446

During a crashdump all file descriptors are closed to ensure the closing of the epmd port and to reserve a file descriptor for the crashdump file.

If a driver (third party library) cannot handle closing of sockets this could result in a segmentation fault in which case a crashdump would not be produced. This is now fixed by only closing inets sockets via an emergency close callback to the driver and thus closing the epmd socket.

Own Id: OTP-12447

Fix HiPE debug lock checking on OS X 64bit

Position-independent code is mandatory on OS X. We use r11 as an intermediate register to fill BIF_P->hipe.bif_callee. This fixes the following error when doing `make debug FLAVOR=smp`:

clang -cc1as: fatal error: error in backend: 32-bit absolute addressing is not supported in 64-bit mode

Own Id: OTP-12188

Fix race bug that could cause VM crash in erlang:port_get_data/1 if the port was closed by a concurrent process. Also fix fatal bug if port_set_data/2 is called with a non-immediate data term. Both bugs exist since R16B01.

Own Id: OTP-12208

Correct make variable SSL_DED_LD_RUNTIME_LIBRARY_PATH when erl_xcomp_sysroot ends with a slash.

Own Id: OTP-12216 Aux Id: seq12700

Fix two cases of unreachable code caused by false use of assigment operators.

Own Id: OTP-12222

Fix bug when hipe compiled code makes tail call to a BIF that disables GC while trapping (sush as binary_to_list, list_to_binary, binary_to_term, term_to_binary).

Own Id: OTP-12231

Fix bug when a migrated empty memory carrier is reused just before it should be destroyed by the thread that created it.

Own Id: OTP-12249

Prevents compile-time errors in NIFs, when the compiler is instructed to treat missing field initializers as errors, by adding an initializer for the new options field which was added to ErlNifEntry for 17.3.

Own Id: OTP-12266

Fixed CPU topology detection on FreeBSD systems where Erlang/OTP is compiled by new C compilers (including, but possibly not limited to, gcc 4.9 and clang).

Own Id: OTP-12267

Use C99 function isfinite() instead of finite() when available on non GCC compilers.

Own Id: OTP-12268

Fix bug on windows where an incorrect number of links could be returned when doing file:read_file_info on a directory.

Own Id: OTP-12269

Fix rare bug when purging module on VM started with +Meamin.

Own Id: OTP-12273

Repair run_erl terminal window size adjustment sent from to_erl. This was broken in OTP 17.0 which could lead to strange cursor behaviour in the to_erl shell.

Own Id: OTP-12275 Aux Id: seq12739

Fixed bug on windows causing gen_tcp/udp to return an error when given an fd to work with.

Own Id: OTP-12289

Fix various internal erts issues where negating a signed integer in C would trigger undefined behavior. This fixes issues when dividing with bignums and list_to_integer.

Own Id: OTP-12290

When flushing output to stdout on windows, the emulator could sometimes hang indefinitely waiting for the flush to complete. This has been fixed.

Own Id: OTP-12291

Fix so that non-smp emulators with dirty scheduler support shows the correct number of dirty schedulers when calling erlang:system_info(system_version).

Own Id: OTP-12295

Add nif_version to erlang:system_info/1 in order to get the NIF API version of the runtime system in a way similar to driver_version.

Own Id: OTP-12298

Fix bug that could cause the return value from dirty NIF with zero arity to be treated as garbage, leading to VM crash.

Own Id: OTP-12300

Improve allocation carrier migration search logic. This will reduce the risk of failed migrations that could lead to excess memory consumption. It will also improve smp performance due to reduced memory contention on the migration pool.

Own Id: OTP-12323

Introduced support for eager check I/O.

By default eager check I/O will be disabled, but this will most likely be changed in OTP 18. When eager check I/O is enabled, schedulers will more frequently check for I/O work. Outstanding I/O operations will however not be prioritized to the same extent as when eager check I/O is disabled.

Eager check I/O can be enabled using the erl command line argument: +secio true

Characteristics impact when enabled:

Own Id: OTP-12117

Fix erts .app-file

Own Id: OTP-12189

Add configure option --with-ssl-incl=PATH to support OpenSSL installations with headers and libraries at different places.

Own Id: OTP-12215 Aux Id: seq12700

Optimization of atomic memory operations with release barrier semantics on 32-bit PowerPC when using the implementation included in OTP.

Own Id: OTP-12250

Minor adjustment of scheduler activation code making sure that an activation of a scheduler is not prevented by its run-queue being non-empty. (Thanks to Songlu Cai)

Own Id: OTP-12287

Improved support for atomic memory operations provided by the libatomic_ops library. Most importantly support for use of native double word atomics when implemented by libatomic_ops (for example, implemented for ARM).

The $ERL_TOP/HOWTO/INSTALL.md document now also more clearly describes when you want to build together with a libatomic_ops installation.

Own Id: OTP-12302

Add configure option --with-ssl-rpath to control which runtime library path to use for dynamic linkage toward OpenSSL.

Own Id: OTP-12316 Aux Id: seq12753

Added systemd notify support to epmd

Own Id: OTP-12321

Fix bug when an migrated empty memory carrier is reused just before it should be destroyed by the thread that created it.

Own Id: OTP-12249

Repair run_erl terminal window size adjustment sent from to_erl. This was broken in OTP 17.0 which could lead to strange cursor behaviour in the to_erl shell.

Own Id: OTP-12275 Aux Id: seq12739

General documentation updates.

Own Id: OTP-12052

A bug in the VM code implementing sending of signals to ports could cause the receiving port queue to remain in a busy state forever. When this state had been reached, processes sending command signals to the port either got suspended forever, or, if the nosuspend feature was used, always failed to send to the port. This bug was introduced in ERTS version 5.10.

In order for this bug to be triggered on a port, one had to at least once utilize the nosuspend functionality when passing a signal to the port. This by either calling

Thanks Vasily Demidenok for reporting the issue, and Sergey Kudryashov for providing a testcase.

Own Id: OTP-12082 Aux Id: OTP-10336

Fix size overflow bug at memory allocation. A memory allocation call, with an insane size close to the entire address space, could return successfully as if it had allocated just a few bytes. (Thanks to Don A. Bailey for reporting)

Own Id: OTP-12091

Fix various issues where negating a signed integer would trigger undefined behaviour. This fixes issues in the enif_make_int64 interface and some edge cases inside the erlang runtime system.

Own Id: OTP-12097

The documentation erroneously listed the +swct command line argument under +sws.

Own Id: OTP-12102 Aux Id: OTP-10994

Profiling messages could be delivered out of order when profiling on runnable_procs and/or runnable_ports using erlang:system_profile/2. This bug was introduced in ERTS version 5.10.

Own Id: OTP-12105 Aux Id: OTP-10336

Various logging fixes, including: Add run queue index to the process dump in crash dumps.
Add thread index to enomem slogan when crashing.
Remove error logger message for sending messages to old instances of the same node.

Own Id: OTP-12115

Fix compiler warnings reported by LLVM

Own Id: OTP-12138

Correct conversion of MIN_SMALL by list_to_integer/1 and binary_to_integer/1. The bug produced an unnormalized bignum which can cause strange behavior such as comparing different to a correct MIN_SMALL integer. The value MIN_SMALL is -(1 bsl 27) = -134217728 on a 32-bit VM and -(1 bsl 59) = -576460752303423488 on a 64-bit VM. (Thanks to Jesper Louis Andersen, Mikael Pettersson and Anthony Ramine for report, patch and optimization suggestion)

Own Id: OTP-12140

Fix bug in term_to_binary that reallocates binary with inconsistent size information. Bug has never been confirmed to be the cause of any faulty behavior.

Own Id: OTP-12141

Real_path method used while prim loading archive files was not taking into account the fact that windows directory symlinks can be across different drives.

Own Id: OTP-12155

Add log2 histogram to lcnt for lock wait time

Own Id: OTP-12059

Introduced enif_schedule_nif() to the NIF API.

The enif_schedule_nif() function allows a long-running NIF to be broken into separate NIF invocations without the help of a wrapper function written in Erlang. The NIF first executes part of the long-running task, then calls enif_schedule_nif() to schedule a NIF for later execution to continue the task. Any number of NIFs can be scheduled in this manner, one after another. Since the emulator regains control between invocations, this helps avoid problems caused by native code tying up scheduler threads for too long.

The enif_schedule_nif() function also replaces the enif_schedule_dirty_nif() in the experimental dirty NIF API. Note that the only incompatible changes made are in the experimental dirty NIF API.

See the NIF documentation for more information.

Thanks to Steve Vinoski.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12128

OTP-11850 fixed filelib:wildcard/1 to work with broken symlinks. This correction, however, introduced problems since symlinks were no longer followed for functions like filelib:ensure_dir/1, filelib:is_dir/1, filelib:file_size/1, etc. This is now corrected.

Own Id: OTP-12054 Aux Id: seq12660

Fixed ETHR_FORCE_INLINE which caused the build to break on some platforms without adequate thread support (VxWorks).

Own Id: OTP-12010

The documentation for spawn_opt/5 now has a note mentioning that the monitor option is not supported.

Own Id: OTP-11849

Fix broken system monitoring of large_heap for non-smp VM. No message for large_heap was ever sent on non-smp VM. Bug exist since R16B.

Own Id: OTP-11852

The emulator without SMP support crashed when passing a message to a process without enough heap space for the message. This bug was introduced in erts-6.0.

Own Id: OTP-11887 Aux Id: OTP-11388

Fix race between ETS table deletion and unfixation that could cause VM crash. The race could happen between a terminating process that does not own the table but has a fixation on it and another process that deletes the table (maybe the owner terminating) at the same time. Bug existed since R15B02.

Own Id: OTP-11892

The string following the -eval option when invoking erl would not be properly translated from UTF-8 to a list of Unicode characters (as would the arguments for -run).

That bug would cause the build of Erlang/OTP to fail when building in a directory whose pathname contained non-US ASCII characters encoded in UTF-8. (Thanks to Eric Pailleau for reporting this bug.)

Own Id: OTP-11916

Fix erts_debug:size/1 to handle Map sizes

Own Id: OTP-11923

Removed erlang:bitstr_to_list/1 and erlang:list_to_bitstr/1. They were added by mistake, and have always raised an undefined exception when called.

Own Id: OTP-11942

Fixed compilation using mingw-w64 on Windows.

Thanks to Jani Hakala.

Own Id: OTP-11945

The git sha is no longer printed in the shell start header when erlang is built from a tagged git release.

Own Id: OTP-11961

Fixed a bug where send trace events were erroneously dropped when the send was done to a registered process. This bug was introduced in R16B.

Own Id: OTP-11968

The following native functions now bump an appropriate amount of reductions and yield when out of reductions:

Characteristics impact:

Own Id: OTP-11888

The systemd features of epmd have been removed from epmd by default. To enable them you have to build erlang with the configure option --enable-systemd.

Own Id: OTP-11921

Removed Erlang wrapper code used when calling binary_to_term/1, and binary_to_term/2. This improves the performance of these BIFs especially when they are called with small binaries as input.

Own Id: OTP-11931

Add erlang:system_info(tolerant_timeofday), an API to check whether compensation for sudden changes of system time is enabled or not.

Own Id: OTP-11970

Fix broken system monitoring of large_heap for non-smp VM. No message for large_heap was ever sent on non-smp VM. Bug exist since R16B.

Own Id: OTP-11852

Fixed type spec of erlang:system_info/1.

Own Id: OTP-11859 Aux Id: OTP-11615

The option dupnames did not work as intended in re. When looking for names with {capture, [Name, ...]}, re:run returned a random instance of the match for that name, instead of the leftmost matching instance, which was what the documentation stated. This is now corrected to adhere to the documentation. The option {capture,all_names} along with a re:inspect/2 function is also added to further help in using named subpatterns.

Own Id: OTP-11205

Allow loading of NIF library with unicode path name

Own Id: OTP-11408

Allow loading of driver with unicode path name

Own Id: OTP-11549

Fixed a bug where starting Erlang without having an open stdin on fd 0 would sometimes deadlock the emulator when terminating.

Own Id: OTP-11558

The option '-names' in epmd now works on Windows (Thanks to Johannes Weißl)

Own Id: OTP-11565

Correction of the examples in escript documentation. (Thanks to Pierre Fenoll).

Own Id: OTP-11577

Fix bs_get_integer instruction

The instruction bs_get_integer could unnecessarily trigger a garbage collection in failure cases which is unwanted or outright dangerous.

Ex:

<<X:Sz,_/bits>> = <<"some binary">>

Previously, if Sz induced X to a bignum it would reserved memory size this on the heap via a garbage collection before checking if the size could actually match.

It will now check the binary size before triggering a collection.

Own Id: OTP-11581

Remove heap space overestimation in binary_to_term (and remote message reception) for integers in the intervals [-2147483648,-1] and [256,2147483647] on 64-bit emulators.

Own Id: OTP-11585

Add support for detecting the separate tinfo library from ncurses (Thanks to Dirkjan Ochtman)

Own Id: OTP-11590

Deprecation warning for system_flag(cpu_topology) has been extended for removal in OTP 18 (Thanks to Steve Vinoski for the update)

Own Id: OTP-11602

Documentation improvement regarding some awkward wording around the +spp flag. (Thanks to Brian L. Troutwine )

Own Id: OTP-11607

Fixed bug where sendfile would return the wrong error code for a remotely closed socket if the socket was in passive mode. (Thanks to Vincent Siliakus for reporting the bug.)

Own Id: OTP-11614

Increase garbage collection tenure rate

The garbage collector tries to maintain the previous heap block size during a minor gc, i.e. 'need' is not utilized in determining the size of the new heap, instead it relies on tenure and garbage to be sufficiently large.

In instances during intense growing with exclusively live data on the heap coupled with delayed tenure, fullsweeps would be triggered directly after a minor gc to make room for 'need' since the new heap would be full.

To remedy this, the tenure of terms on the minor heap will always happen (if it is below the high watermark) instead of every other minor gc.

Characteristics Impact: Reduced CPU-time spent in garbage collection but may infer delays in collecting garbage from the heap. Tweak 'fullsweep_after' options to increase gc pressure if needed.

Own Id: OTP-11617

Fix bug when comparing integers with floats larger than 2^992. The bug could potentially cause memory corruption on 32-bit emulators.

Own Id: OTP-11618

Cross-compilation fixes for TileraMDE-3.0.1.125620

Own Id: OTP-11635

sendfile no longer uses async threads by default

This has been done because a slow client attack is possible if the async thread pool is used. The scenario is:

Client does a request for a file and then slowly receives the file one byte at a time. This will eventually fill the async thread pool with blocking sendfile operations and thus starving the vm of all file operations.

If you still want to use the async threads pool for sendfile an option to enable it has been introduced.

Thanks to Christopher Faulet for identifying this vulnerability.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11639

Do proper rollback of calls to enif_open_resource_type when load/upgrade callbacks of NIF library return failure.

Own Id: OTP-11722

Changed the default configuration when configuring with $ERL_TOP/configure to be the same as when configuring with $ERL_TOP/otp_build configure.

Previously floating point exceptions got enabled by default on Linux when HiPE was enabled when configuring with $ERL_TOP/configure, but not when configuring with $ERL_TOP/otp_build configure. The default is now in both cases not to use floating point exceptions since there still exist unresolved issues with floating point exceptions on Linux.

For more information see $ERL_TOP/HOWTO/INSTALL.md.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11723

A comment in erl_db_tree.c no longer differ from the code. (Thanks to Cobus Carstens)

Own Id: OTP-11793

Fix epmd debug functionality for VxWorks (Thanks to Jay True)

Own Id: OTP-11808

Use closefrom/2 when available in child_setup (Thanks to Rick Reed and Anthony Ramine)

Own Id: OTP-11809

Fix dtrace/systemtap bug where the probe arguments would be concatenated due to faulty length calculation.

Thanks to Michal Ptaszek and Scott Lystig Fritchie

Own Id: OTP-11816

It is now better documented that the +fn* flags to erl also affect how command line parameters and environment variables are read. (Thanks to Vlad Dumitrescu)

Own Id: OTP-11818

Options to set match_limit and match_limit_recursion are added to re:run. The option report_errors is also added to get more information when re:run fails due to limits or compilation errors.

Own Id: OTP-10285

Dialyzer's unmatched_return warnings have been corrected.

Own Id: OTP-10908

A common case is to wrap an argument to list_to_binary/1 in a list to ensure conversion can happen even though the argument may already be a binary. Take special care of this case and do not copy binary.

Impact: May cause incompatibility since a single binary is no longer copied. Use binary:copy/1,2 instead.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11082

Make erlang:open_port/2 spawn and spawn_executable handle unicode.

Own Id: OTP-11105

Handle unicode (widestring) in erl, erlc, heart, etc on windows.

Own Id: OTP-11135

The version of the PCRE library Used by Erlang's re module is raised to 8.33 from 7.6. This means, among other things, better Unicode and Unicode Character Properties support. New options connected to PCRE 8.33 are also added to the re module (ucd, notempty_atstart, no_start_optimize). PCRE has extended the regular expression syntax between 7.6 and 8.33, why this imposes a potential incompatibility. Only very complicated regular expressions may be affected, but if you know you are using obscure features, please test run your regular expressions and verify that their behavior has not changed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11204

Filenames containing UTF-8 encoded characters can now be handled by erlc.

If you have set the ERLC_EMULATOR environment variable, note that erlc in OTP 17 will only work with erl in OTP 17 since the protocol between the erlc program and the erl_compile module has changed.

Own Id: OTP-11248

By giving --enable-static-{nifs,drivers} to configure it is now possible to statically linking of nifs and drivers to the main Erlang VM binary. At the moment only the asn1 and crypto nifs of the Erlang/OTP nifs and drivers have been prepared to be statically linked. For more details see the Installation Guide in the System documentation.

Own Id: OTP-11258

Erlang/OTP has been ported to the realtime operating system OSE. The port supports both smp and non-smp emulator. For details around the port and how to started see the User's Guide in the ose application.

Note that not all parts of Erlang/OTP has been ported.

Notable things that work are: non-smp and smp emulators, OSE signal interaction, crypto, asn1, run_erl/to_erl, tcp, epmd, distribution and most if not all non-os specific functionality of Erlang.

Notable things that does not work are: udp/sctp, os_mon, erl_interface, binding of schedulers.

Own Id: OTP-11334

Add the {active,N} socket option for TCP, UDP, and SCTP, where N is an integer in the range -32768..32767, to allow a caller to specify the number of data messages to be delivered to the controlling process. Once the socket's delivered message count either reaches 0 or is explicitly set to 0 with inet:setopts/2 or by including {active,0} as an option when the socket is created, the socket transitions to passive ({active, false}) mode and the socket's controlling process receives a message to inform it of the transition. TCP sockets receive {tcp_passive,Socket}, UDP sockets receive {udp_passive,Socket} and SCTP sockets receive {sctp_passive,Socket}.

The socket's delivered message counter defaults to 0, but it can be set using {active,N} via any gen_tcp, gen_udp, or gen_sctp function that takes socket options as arguments, or via inet:setopts/2. New N values are added to the socket's current counter value, and negative numbers can be used to reduce the counter value. Specifying a number that would cause the socket's counter value to go above 32767 causes an einval error. If a negative number is specified such that the counter value would become negative, the socket's counter value is set to 0 and the socket transitions to passive mode. If the counter value is already 0 and inet:setopts(Socket, [{active,0}]) is specified, the counter value remains at 0 but the appropriate passive mode transition message is generated for the socket.

Thanks to Steve Vinoski

Own Id: OTP-11368

A new optional scheduler utilization balancing mechanism has been introduced. For more information see the +sub command line argument.

Characteristics impact: None, when not enabled. When enabled, changed timing in the system, normally a small overhead due to measuring of utilization and calculating balancing information. On some systems, such as old Windows systems, the overhead can be quite substantial. This time measurement overhead highly depend on the underlying primitives provided by the OS.

Own Id: OTP-11385

A call to either the garbage_collect/1 BIF or the check_process_code/2 BIF may trigger garbage collection of another processes than the process calling the BIF. The previous implementations performed these kinds of garbage collections without considering the internal state of the process being garbage collected. In order to be able to more easily and more efficiently implement yielding native code, these types of garbage collections have been rewritten. A garbage collection like this is now triggered by an asynchronous request signal, the actual garbage collection is performed by the process being garbage collected itself, and finalized by a reply signal to the process issuing the request. Using this approach processes can disable garbage collection and yield without having to set up the heap in a state that can be garbage collected.

The garbage_collect/2, and check_process_code/3 BIFs have been introduced. Both taking an option list as last argument. Using these, one can issue asynchronous requests.

code:purge/1 and code:soft_purge/1 have been rewritten to utilize asynchronous check_process_code requests in order to parallelize work.

Characteristics impact: A call to the garbage_collect/1 BIF or the check_process_code/2 BIF will normally take longer time to complete while the system as a whole wont be as much negatively effected by the operation as before. A call to code:purge/1 and code:soft_purge/1 may complete faster or slower depending on the state of the system while the system as a whole wont be as much negatively effected by the operation as before.

Own Id: OTP-11388 Aux Id: OTP-11535, OTP-11648

Cleanup 'Buckets' and 'Time left' fields in crashdump to ease parsing.

Own Id: OTP-11419

Add sync option to file:open/2.

The sync option adds the POSIX O_SYNC flag to the open system call on platforms that support the flag or its equivalent, e.g., FILE_FLAG_WRITE_THROUGH on Windows. For platforms that don't support it, file:open/2 returns {error, enotsup} if the sync option is passed in. Thank to Steve Vinoski and Joseph Blomstedt

Own Id: OTP-11498

erlang:binary_to_term will now cost an appropriate amount of reductions and will interrupt (yield) for reschedule if the term is big. This avoids too long schedules when binary_to_term is used. (Thanks to Svante Karlsson for the original patch)

Impact: Programs running binary_to_term on large binaries will run more smoothly, but rescheduling will impact the single process performance of the BIF. Single threaded benchmarks might show degraded performance of the BIF, while general system behaviour will be improved.

Own Id: OTP-11535 Aux Id: OTP-11388

Added high resolution icon for windows. (Thanks to Daniel Goertz for the inspiration.)

Own Id: OTP-11560

Migration of memory carriers has been enabled by default on all ERTS internal memory allocators based on the alloc_util framework except for temp_alloc. That is, +M<S>acul de is default for these allocators. Note that this also implies changed allocation strategies for all of these allocators. They will all now use the "address order first fit carrier best fit" strategy.

By passing +Muacul 0 on the command line, all configuration changes made by this change will be reverted.

Characteristics impact: Improved memory characteristics with a smaller memory footprint at the expense of a quite small performance cost.

Own Id: OTP-11604 Aux Id: OTP-10279

A clarification has been added to the documentation of -on_load() in the Reference Manual that it is only recommended for loading NIF libraries.

Own Id: OTP-11611

+fnaw is now default when starting the emulator; it used to be +fnl.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11612

EEP43: New data type - Maps

With Maps you may for instance:

For information on how to use Maps please see Map Expressions in the Reference Manual.

The current implementation is without the following features:

Note that Maps is experimental during OTP 17.0.

Own Id: OTP-11616

The previously deprecated driver API function driver_async_cancel() has been removed. Due to this, the driver API version has been bumped to 3.0.

Thanks to Steve Vinoski.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11628

Experimental "dirty scheduler" functionality has been introduced. In order to try the functionality out, you need to pass the command line argument --enable-dirty-schedulers to configure when building the system.

Dirty schedulers can currently only be used by NIFs on a system with SMP support. More information can be found in the erl_nif(3) documentation, the erl(1) documentation, and in the git commit comment of commit 'c1c03ae4ee50e58b7669ea88ec4d29c6b2b67c7b'.

Note that the functionality is experimental, and not supported. This functionality will be subject to backward incompatible changes. You should not enable the dirty scheduler functionality on production systems. It is only provided for testing.

Thanks to Steve Vinoski.

Own Id: OTP-11629

Improve reduction cost and yielding of term_to_binary. The reduction cost is increased and garbage collection is disabled during yield.

Impact: Improves system responsiveness when term_to_binary is called with large terms without significant degradation of single threaded performance.

Own Id: OTP-11648 Aux Id: OTP-11388

By default, the system's version of zlib will be used, provided its version is 1.2.4 or higher; otherwise the built-in zlib will be used. The built-in version of zlib has been bumped to 1.2.8. (Use the --enable-builtin-zlib option to configure to force the use of the built-in zlib.)

Own Id: OTP-11669

The default float encoding in binary_to_term and external_size has been changed to use minor_mode 1 instead of 0.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11738

Introduced the configure option --with-assumed-cache-line-size=SIZE. For more information see $ERL_TOP/HOWTO/INSTALL.md.

Own Id: OTP-11742

Halfword emulator is marked as deprecated. It still works as before but is planned to be removed in a future major release.

Own Id: OTP-11777

The external format for Maps has changed in a way that is not compatible with the format used in OTP 17.0-rc1 and OTP 17.0-rc2.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11782

Fixed faulty make dependency that would make some make versions fail while building gen_git_version.mk.

Own Id: OTP-11784

Introduced functionality for allowing old drivers and NIF libraries to be loaded during a transition period. For more information see the version management section in the erl_driver(3) documentation and the version management section in the erl_nif(3) documentation.

Own Id: OTP-11799

Support file paths longer than 259 characters on Windows. Long absolute paths are automatically converted to UNC format with a \\?\ prefix which is the only way to represent long paths. The 259 character limit still applies for individual file names, relative paths and the current working directory.

Own Id: OTP-11813

Document that escript:create/2 also accepts a 3-elements tuple containing files and zip:create/3 options to build a zip file.

Thanks to Pierre Fenoll

Own Id: OTP-11827

Add systemd socket activation for epmd.

Thanks to Matwey V. Kornilov

Own Id: OTP-11829

When using gen_tcp:connect and the fd option with port and/or ip, the port and ip options were ignored. This has been fixed so that if port and/or ip is specified together with fd a bind is requested for that fd. If port and/or ip is not specified bind will not be called.

Own Id: OTP-12061

When normalizing paths, erl_prim_loader would always convert backslash to forward slash. This is correct on Windows, but not on other operating systems. erl_prim_loader now checks which OS is running before performing this conversion.

Own Id: OTP-11170

Fixed syslog defines and defined LOG_ERR for systems without syslog.h. Thanks to Matt Lewandowsky.

Own Id: OTP-11349

Check all pattern arguments passed to binary:matches/2. Thanks to Mike Sassak.

Own Id: OTP-11350

Fix two small silent rules omissions. Thanks to Anthony Ramine.

Own Id: OTP-11351

Teach configure to detect if posix_memalign cannot align to more than the system page size.

For cross-compiled systems a new environment variable called erl_xcomp_posix_memalign has been introduced to indicate whether posix_memalign should be used.

Own Id: OTP-11371

Fix bsr bug occurring when shifting a huge number a huge number of bits to the right. Thanks to Lars Hesel Christensen.

Own Id: OTP-11381

Fix memory leak for distributed monitors

Own Id: OTP-11410

Fix various typos in erts, kernel and ssh. Thanks to Martin Hässler.

Own Id: OTP-11414

Crashdumps initiated by out-of-memory on process spawn could cause the beam to segfault during crashdump writing due to invalid pointers.

The pointers are invalid since the process creation never finished. This fix removes these processes from the printouts. Reported by Richard Carlsson.

Own Id: OTP-11420

Crash dumps from 64-bit Erlang machines would have all memory addresses truncated to 32 bits, which could cause trouble inspecting processes message queues and stacks in the crashdump viewer.

Own Id: OTP-11450

Threads other than schedulers threads could make thread unsafe accesses when support for migration of memory carriers had been enabled, i.e., when the +M<S>acul command line flag had been passed to erl. This could cause corruption of the VMs internal state.

This bug was introduced in erts-5.10.2 when the support for migration of memory carriers was introduced.

Own Id: OTP-11456 Aux Id: OTP-10279

Fix bug in binary_to_term for invalid bitstrings and very large binaries (>2Gb).

Own Id: OTP-11479

Under rare circumstances a process calling inet:close/1, gen_tcp:close/1, gen_udp:close/1, or gen_sctp:close/1 could hang in the call indefinitely.

Own Id: OTP-11491

Fix bug that could cause a 32-bit emulator to always crash at start (since R16B01) depending on the alignment of static data in the beam executable.

Own Id: OTP-11496

Fix benign bugs regarding bitstring compare. Only a nuisance for debug and valgrind VM.

Own Id: OTP-11501

Silence warnings (Thanks to Anthony Ramine)

Own Id: OTP-11517

The default wordsize of the emulator (beam) is now determined by compiler default on Mac OSX (Darwin). This was previously forced to 32bits by the configure script unless otherwise specified.

Own Id: OTP-11521

A new memory allocation feature called "super carrier" has been introduced. The super carrier feature can be used in different ways. It can for example be used for pre-allocation of all memory that the runtime system should be able to use.

By default the super carrier is disabled. It is enabled by passing the +MMscs <size in MB> command line argument. For more information see the documentation of the +MMsco, +MMscrfsd, +MMscrpm, +MMscs, +MMusac, and, +Mlpm command line arguments in the erts_alloc(3) documentation.

Since it is disabled by default there should be no impact on system characteristics if not used.

This change has been marked as a potential incompatibility since the returned list when calling erlang:system_info({allocator, mseg_alloc}) now also include an {erts_mmap, _} tuple as one element in the list.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11149

Added erlang:system_info(ets_limit) to provide a way to retrieve the runtime's maximum number of ETS tables. Thanks to Steve Vinoski

Own Id: OTP-11362

Add new BIF os:unsetenv/1 which deletes an environment variable. Thanks to Martin Hässler.

Own Id: OTP-11446

Introduced a new guarantee regarding exit signals from ports:

If the process calling one of the synchronous port BIFs listed below is linked to the port identified by the first argument, and the port exits before sending the result of the port operation, the exit signal issued due to this link will be received by the processes before the BIF returns, or fail with an exception due to the port not being open.

The synchronous port BIFs are:

Note that some ports under certain circumstances unlink themselves from the calling process before exiting, i.e. even though the process linked itself to the port there might be no link triggering an exit signal.

Characteristics impact: The return or exception from the synchronous port BIF will be delayed if the port simultaneously exit due to some issue unrelated to the outstanding synchronous port BIF call. In all other cases characteristics are unchanged.

Own Id: OTP-11489

Memory allocators will be able to create sys_alloc carriers as fallback, if mseg_alloc cannot create more carriers, on systems with posix_memalign() support. This is similar to how it worked in pre-R16 releases.

Windows systems will create carriers using _aligned_malloc() and can by this use the new optimized allocator header scheme introduced in R16 on other platforms.

Own Id: OTP-11318

The documentation of predefined types and built-in types has been corrected.

Own Id: OTP-11090

Fix changing terminal parameters in to_erl

Change the behaviour of to_erl to use TCSADRAIN instead of TCSANOW when changing terminal parameters. This makes the serial driver wait for the output queues to be empty before applying the terminal parameter change. Thanks to Stefan Zegenhagen.

Own Id: OTP-11206

The default value of {flush, boolean()} in erlang:halt/2 is documented to be 'true' if the status is an integer. The implementation behaviour was reversed. The Implementation is now corrected to adhere to the documentation. Thanks to Jose Valim for reporting the error.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11218

Fix serious race bug in R16B01 that could cause PID mix-ups when a lot of processes were spawned and terminated in a very rapid pace on an SMP emulator with at least two scheduler threads.

Own Id: OTP-11225

Validating a trace pattern with the option silent no longer incorrectly enables/disables the silent option of the calling process.

Own Id: OTP-11232

Fixed a bug where GCC 4.8 and later use a more aggressive loop optimization algorithm that broke some previously working code in the efile driver. Thanks to Tomas Abrahamsson for reporting this issue.

Own Id: OTP-11246

Fixed bug when printing memory allocator acul option in crash dump.

Own Id: OTP-11264

Opening a new compressed file on Windows could in rare (random) cases result in {error,eisdir} or other error codes although it should have succeeded. This is now corrected.

Own Id: OTP-11265

Fixed a race condition when closing a trace port that would cause the emulator to crash.

Own Id: OTP-11290

There is a new somewhat experimental socket option 'netns' that can set the network namespace for a socket on Linux:es where it is supported. See the documentation.

Own Id: OTP-11157

New allocator strategy aoffcbf (address order first fit carrier best fit). Supports carrier migration but with better CPU performance than aoffcaobf.

Own Id: OTP-11174

Introduced functionality for inspection of system and build configuration.

Own Id: OTP-11196

Fix matching of floating point middle-endian machines. Thanks to Johannes Weissl.

Own Id: OTP-11201

Fix compile error on ARM and GCC versions greater than 4.1.0. Thanks to Johannes Weissl.

Own Id: OTP-11214

run_erl: Redirect standard streams to /dev/null. Thanks to Johannes Weissl.

Own Id: OTP-11215

Misc. corrections in documentation for erl_driver. Thanks to Giacomo Olgeni.

Own Id: OTP-11227

Fix documentation regarding binary_part.

Own Id: OTP-11239

Make edlin understand a few important control keys. Thanks to Stefan Zegenhagen.

Own Id: OTP-11251

Export type zlib:zstream/0. Thanks to Loic Hoguin.

Own Id: OTP-11278

Add erl option to set schedulers by percentages.

For applications where measurements show enhanced performance from the use of a non-default number of emulator scheduler threads, having to accurately set the right number of scheduler threads across multiple hosts each with different numbers of logical processors is difficult because the erl +S option requires absolute numbers of scheduler threads and scheduler threads online to be specified.

To address this issue, add a +SP option to erl, similar to the existing +S option but allowing the number of scheduler threads and scheduler threads online to be set as percentages of logical processors configured and logical processors available, respectively. For example, "+SP 50:25" sets the number of scheduler threads to 50% of the logical processors configured, and the number of scheduler threads online to 25% of the logical processors available. The +SP option also interacts with any settings specified with the +S option, such that the combination of options "+S 4:4 +SP 50:50" (in either order) results in 2 scheduler threads and 2 scheduler threads online.

Thanks to Steve Vinoski

Own Id: OTP-11282

Extend erl_driver interface with lock names

Lock and thread names are already a feature in the driver interface. This extension will let developers read these names which eases debugging.

Own Id: OTP-11303

Fix incorrect values returned by integer_to_binary/2. Thanks to Juan Jose Comellas.

Own Id: OTP-11311

Fix system_flag scheduling_statistics - disable . Thanks to Steve Vinoski.

Own Id: OTP-11317

The documentation of predefined types has been corrected Thanks to Kostis Sagonas.

Own Id: OTP-11321

A bug in prim_inet has been corrected. If the port owner was killed at a bad time while closing the socket port the port could become orphaned hence causing port and socket leaking. Reported by Fred Herbert, Dmitry Belyaev and others.

Own Id: OTP-10497 Aux Id: OTP-10562

Compilation fixes for NetBSD. Thanks to YAMAMOTO Takashi.

Own Id: OTP-10941

Fixed a race condition when using delayed_write when writing to a file which would cause the same data to be written multiple times.

Own Id: OTP-10984

Fix small memory leak from tracing with option meta.

Own Id: OTP-10997

Correct typo in erlsrv usage. Thanks to Bryan Hunter

Own Id: OTP-11002

ct_run: delete unused function. Thanks to Tuncer Ayaz.

Own Id: OTP-11003

Corrections to run_erl/to_erl handshake behaviour.

Own Id: OTP-11012

Fix typo in type: erlang:process_info_item(). Thanks to Andrew Tunnell-Jones.

Own Id: OTP-11024

Fix src/dest overlap issue in ttsl driver. Thanks to Steve Vinoski.

Own Id: OTP-11064

When sending to a port using erlang:send(Port, Msg, [nosuspend]), the send operation was performed synchronously. This bug has now been fixed.

Own Id: OTP-11076 Aux Id: OTP-10336

When converting a faulty binary to a list with unicode:characters_to_list, the error return value could contain a faulty "rest", i.e. the io_list of characters that could not be converted was wrong. This happened only if input was a sub binary and conversion was from utf8. This is now corrected.

Own Id: OTP-11080

Runtime system could crash when reporting stale driver_select().

Own Id: OTP-11084

Fix lock order violation for memory instrumentation (+Mim, +Mis, +Mit).

Own Id: OTP-11085

Fixed some compilation warnings on miscellaneous platforms. Thanks to Anthony Ramine.

Own Id: OTP-11086

Fixed issue when flushing i/o during shutdown on windows where the Erlang VM would sometime hang due to a race condition.

Own Id: OTP-11096

Fixed issue where repeated calls to erlang:nodes() could cause unnecessary contention in the dist_table lock.

Own Id: OTP-11097

Properly guard WIDE_TAG use with HAVE_WCWIDTH in ttsl_drv. Thanks to Anthony Ramine

Own Id: OTP-11106

Fix some Makefile rules that didn't support silent rules. Thanks to Anthony Ramine.

Own Id: OTP-11111

Fix receive support in erl_eval with a BEAM module. Thanks to Anthony Ramine.

Own Id: OTP-11137

erlang:now() could suddenly jump ~24 days into the future on Windows. This is now corrected. Thanks to Garret Smith for reporting and testing fixes.

Own Id: OTP-11146

erlang:term_to_binary will now cost an appropriate amount of reductions and will interrupt (yield) for reschedule if the term is big. This avoids too long schedules when term_to_binary is used.

Impact: Programs running term_to_binary on large terms will run more smothly, but rescheduling will impact the single process performance of the BIF. Single threaded benchmarks will show degraded performance of the BIF when called with very large terms, while general system behaviour will be improved. The overhead for allowing restart and reduction counting also degrades local performance of the BIF with between 5% and 10% even for small terms.

Own Id: OTP-11163

Replaced the lock protecting gathering of garbage collection statistics with a lock-free solution.

Own Id: OTP-10271 Aux Id: kunagi-108 [04c5410f-9cc4-4696-8639-36bf98686c7a-7]

Support for migration of memory carriers between memory allocator instances has been introduced.

By default this feature is not enabled and do not effect the characteristics of the system. When enabled it has the following impact on the characteristics of the system:

For more information, see the documentation of the +M<S>acul command line argument.

Own Id: OTP-10279

Change specs for spawn_opt to use the process_level() type declaration instead of re-defining it in various places. Thanks to Kostis Sagonas.

Own Id: OTP-11008

Postscript files no longer needed for the generation of PDF files have been removed.

Own Id: OTP-11016

Erlang source files with non-ASCII characters are now encoded in UTF-8 (instead of latin1).

Own Id: OTP-11041 Aux Id: OTP-10907

Optimization of simultaneous inet_db operations on the same socket by using a lock free implementation.

Impact on the characteristics of the system: Improved performance.

Own Id: OTP-11074

The high_msgq_watermark and low_msgq_watermark inet socket options introduced in OTP-R16A could only be set on TCP sockets. These options are now generic and can be set on all types of sockets.

Own Id: OTP-11075 Aux Id: OTP-10336

A new better algorithm for management of the process, and port tables has been introduced.

Impact on the characteristics of the system:

Due to the above mensioned potential incompatibility, it will still be possible to enable the old algorithm for some time. The command line argument +P legacy will enable the old algorithm on the process table, and +Q legacy will do the same for the port table. These command line arguments are however deprecated as of their introduction and have been scheduled for removal in OTP-R18.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11077

Support wide characters in the shell through wcwidth(). Thanks to Anthony Ramine. Reported by Loïc Hoguin.

Own Id: OTP-11088

Added total used memory for each process in erlang crash dumps.

Own Id: OTP-11098

Added support for hipe on Raspberry Pi (armv6l). Thanks to Klaus Alfert.

Own Id: OTP-11125

Remove 'query' from the list of reserved words in docs. Thanks to Matthias Endler and Loïc Hoguin.

Own Id: OTP-11158

Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS X. (Thanks to Anthony Ramine)

Own Id: OTP-11159

Miscellaneous native code in OTP misbehave either due to lengthy execution, or due to not bumping reductions properly. Problems typically occur when passing huge sets of data to a misbehaving BIF. Fixing this has high priority and is being worked on, but there will remain issues like this for some time.

In order to alleviate problems with scheduling which might occur when executing misbehaving native code, the command line argument +sfwi has been introduced.

By default this feature is disabled and you are advised not to enable it if you do not encounter problems with misbehaving native code.

When enabled it has the following impact on the characteristics of the system:

Own Id: OTP-11164

A bug in the implementation of offline schedulers has been fixed. The bug was introduced in OTP-R16A/ERTS-5.10, and caused work-stealing between schedulers to fail. This in turn, caused work to accumulate in some run-queues. The bug was only triggered when there were offline schedulers in the system, i.e., when the amount of online schedulers was less than the total amount of schedulers. The effect of the bug got more severe the larger amount of offline schedulers the system had.

Own Id: OTP-11022 Aux Id: OTP-9892

The BIF is_process_alive/1 could prematurely return false while the process being inspected was terminating. This bug was introduced in ERTS-5.10.

Own Id: OTP-10926

Fix a problem in erlang:delete_element/2 where the call could corrupt one word of stack if the heap and stack met during call.

Own Id: OTP-10932

The +sws<value> and +swt<value> system flags failed if no white space were passed between the parameter and value parts of the flags. Upon failure, the runtime system refused to start.

Own Id: OTP-11000

Scheduler threads will now by default be less eager requesting wakeup due to certain cleanup operations. This can also be controlled using the +swct command line argument of erl(1).

Own Id: OTP-10994

Threads created internally in the runtime system by vanilla, fd, and spawn drivers on Windows systems could make thread unsafe calls to driver_select().

Own Id: OTP-10802

Threads created internally in the runtime system by the vanilla, fd, and spawn drivers on Windows systems could make unsafe memory accesses to driver data after port had terminated.

Own Id: OTP-10803

The runtime system could crash when flushing data to standard out or standard error on Windows.

Own Id: OTP-10807

Bugs due to the port optimizations introduced in erts-5.10/OTP-R16A have been fixed:

Own Id: OTP-10809 Aux Id: OTP-10336

Fix {packet,httph} header capitalization for unrecognized header fields longer than 20 charachters such as Sec-Websocket-Version. The limit is simply raised from 20 to 50 characters with the hope that valid headers longer than 50 are not used.

Own Id: OTP-10824

Fix rounding issues in float_to_list/1,2. Thanks to Serge Aleynikov

Own Id: OTP-10837

Fix memory leak in file driver introduced in R16A.

Own Id: OTP-10841

A bug in an ERTS internal queue implementation could cause the loss of a wake up signal to a consumer thread. This has now been fixed.

The effect of this bug, when triggered, was often only a small or even no delay of certain operations. This since, threads often are woken due to other unrelated reasons. However, if the consumer thread was not woken due to other reasons when the bug was triggered, these operations could be left hanging, potentially for ever. Such effects seems to have been very rare, but we have on at least one occasion gotten a report about such an issue.

Operations potentially effected by this bug:

Own Id: OTP-10854

OS X Snow Leopard now only uses write, as writev does not work properly on very large files.

Own Id: OTP-10858

Fixed a bug where line oriented file I/O using read_ahead was very slow for files with very large difference in line length.

Own Id: OTP-10859

In erts-5.10 (R16A) faulty hashvalues were produced for non-ASCII atoms (characters in byte-range 128..255). This has now been fixed and hashvalues conforms to previous OTP releases.

Own Id: OTP-10860

Fixes of memory accesses that might be thread unsafe when the runtime system has been linked against third-party libraries for atomic memory operations during the build. Most builds are uneffected by this bug. If triggered, the runtime system will most likely crash more or less immediately.

Own Id: OTP-10875 Aux Id: OTP-10854

Fixed a bug where it was longer possible to give the +sws proposal flag to non-smp emulators.

Own Id: OTP-10881 Aux Id: seq12258

Faulty type to bytes read for ReadFile on Windows. This could cause windows systems to misbehave. The correct type is now used.

Own Id: OTP-10890

Change default max ports for Windows to 8192. Having a too large value caused Windows to not be able to recover properly. If you want to use another value, pass +Q Value to erl or werl.

Own Id: OTP-10892

Fix rare crash on halfword vm during code loading.

Own Id: OTP-10896

Tuple funs (deprecated in R15B) are no longer supported.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10170

Added four new bifs, erlang:binary_to_integer/1,2, erlang:integer_to_binary/1, erlang:binary_to_float/1 and erlang:float_to_binary/1,2. These bifs work similarly to how their list counterparts work, except they operate on binaries. In most cases converting from and to binaries is faster than converting from and to lists.

These bifs are auto-imported into erlang source files and can therefore be used without the erlang prefix.

Own Id: OTP-10300 Aux Id: kunagi-74 [74]

The experimental support for packages has been removed.

Own Id: OTP-10348 Aux Id: kunagi-316 [227]

The driver API function erl_drv_consume_timeslice(), and the NIF API function enif_consume_timeslice() have been introduced.

These functions are provided in order to better support co-operative scheduling, improve system responsiveness, and to make it easier to prevent misbehaviors of the VM due to a process or port monopolizing a scheduler thread. They can be used when dividing lengthy work into a number of repeated calls without the need to use threads.

Own Id: OTP-10810

The list_to_integer/2 bif has been optimized when used with bases other than 10.

Own Id: OTP-10834 Aux Id: kunagi-74 [74]

The git commit sha of the HEAD commit is now added to the Erlang shell when compiling a non-released Erlang version.

Own Id: OTP-10838

Change caching policy for memory segment allocator. For instance, prefer sbc segments over mbc segments, caching policy is time-arrow aware, evicting older cached segments to store newer segments.

The default number of cachable segment has been increased from five to ten segments. This can be modified, same as before, with the command line option +MMmcs 5

Impact: Increased speed for processing on larger objects, e.g. binaries. Slight increase of mapped and resident memory. Tune your system with memory options to erl for best performance.

Own Id: OTP-10840

Updated config.sub and config.guess to latest version from gnu.org

Own Id: OTP-10848

Add an xcomp file for Blue Gene/Q. Thanks to Kostis Sagonas.

Own Id: OTP-10849

Cleanup of documentation of the type language. Thanks to Kostis Sagonas.

Own Id: OTP-10850

Change the return value of hipe_bifs:remove_refs_from/1. Thanks to Kostis Sagonas.

Own Id: OTP-10851

As of ERTS-5.10/OTP-R16A node names passed in the EPMD protocol are required to be encoded in UTF-8. Since EPMD previously accepted latin1 encoded node names this is an incompatibility. However, since Erlang nodes always have required characters in node names to be 7-bit ASCII characters (and still do require this), this incompatibility should not effect anyone using EPMD as an Erlang Port Mapper Daemon.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10872 Aux Id: OTP-10753

The +pc flag to erl can be used to set the range of characters considered printable. This affects how the shell and io:format("~tp",...) functionality does heuristic string detection. More can be read in STDLIB users guide.

Own Id: OTP-10884

Fix a number of type cast errors related to formatted printing on Win64 that can potentially cause problem when the Erlang VM exceeds 4 GB of ram. (Thanks to Blaine Whittle for the original patch)

Own Id: OTP-10887

The effect of the deprecated environment variable ERL_MAX_PORTS had been removed premeturely. It has now been readded. Note that this is still scheduled to be released in R17B.

Own Id: OTP-10895

Set new peeled off SCTP socket to nonblocking socket (Thanks to Jonas Falkevik)

Own Id: OTP-10491

Fix various typos (thanks to Tuncer Ayaz)

Own Id: OTP-10611

Fix fd leak when using async thread pool

When using the async thread pool, if an erlang process asks to open a file and it gets shutdown/killed while the file:open/2 call hasn't returned, it's possible to leak a file descriptor against the target file. This has now been fixed. (Thanks to Filipe David Manana)

Own Id: OTP-10677

Use sys/types.h instead of string.h to pull ssize_t definition to erl_driver.h. This fixes build issue on NetBSD. (Thanks to Yamamoto Takashi).

Own Id: OTP-10699

Arguments given with the -run or -s flags to erl are now translated according to the file name encoding mode of the runtime system.

Own Id: OTP-10702

The octet counters in the gen_tcp/inet interface could behave in unexpected ways on 64bit platforms. The behaviour is now as expected.

Own Id: OTP-10746

Certain linux kernels, most notably in redhat and CentOS distribution, had a bug in writev which generated an infinite loop in the tcp code of the VM. The bug is now worked around.

Own Id: OTP-10747

A process that got killed (got an exit signal) while operating on a compresseed file, could cause a segmentation fault in the VM. This is now corrected. Thanks to Filipe David Manana for identifying the problem and submitting a solution.

Own Id: OTP-10748

Windows previously used three digit exponent in formatting which caused difference between platforms, as can be seen by float_to_list/1. This has now been fixed.

Own Id: OTP-10751

A boolean socket option 'ipv6_v6only' for IPv6 sockets has been added. The default value of the option is OS dependent, so applications aiming to be portable should consider using {ipv6_v6only,true} when creating an inet6 listening/destination socket, and if necessary also create an inet socket on the same port for IPv4 traffic. See the documentation.

Own Id: OTP-8928 Aux Id: kunagi-193 [104]

It is now allowed to define stubs for BIFs, to allow type specs to be written for BIFs. For example, if there is BIF called lists:member/2, a dummy definition of lists:member/2 is now allowed.

Own Id: OTP-9861

Process optimizations. The most notable:

These changes imply changes of the characteristics the system. Most notable: changed timing in the system.

Own Id: OTP-9892 Aux Id: OTP-10167

Non-blocking code loading. Earlier when an Erlang module was loaded, all other execution in the VM were halted while the load operation was carried out in single threaded mode. Now modules are loaded without blocking the VM. Processes may continue executing undisturbed in parallel during the entire load operation. The load operation is completed by making the loaded code visible to all processes in a consistent way with one single atomic instruction. Non-blocking code loading will improve realtime characteristics when modules are loaded/upgraded on a running SMP system.

Own Id: OTP-9974

In the SMP emulator, turning on and off tracing will no longer take down the system to single-scheduling.

Own Id: OTP-10122

Remove VxWorks support

Own Id: OTP-10146

Added a general framework for executing benchmarks of Erlang/OTP. Benchmarks for the Erlang VM and mnesia have been incorporated in the framework.

For details about how to add more benchmarks see $ERL_TOP/HOWTO/BENCHMARKS.md in the source distribution.

Own Id: OTP-10156

Optimized deletion of ETS-tables which significantly improves performance when large amounts of temporary tables are used.

This change imply changes of the characteristics the system. Most notable: changed timing in the system.

Own Id: OTP-10167 Aux Id: OTP-9892

Tuple funs (deprecated in R15B) are no longer supported.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10170

New internal header scheme for allocators

Impact: Reduces size on object allocated in multiblock carriers by one word

Own Id: OTP-10273 Aux Id: kunagi-20 [20]

Major port improvements. The most notable:

These changes imply changes of the characteristics of the system. The most notable:

Potential incompatibilities:

The driver API version has been bumped to 2.1 from 2.0 due to the above changes in the driver API.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10336 Aux Id: OTP-9892

The experimental support for packages has been removed.

Own Id: OTP-10348 Aux Id: kunagi-316 [227]

Wrong parameters when setting seq_trace-tokens from within a trace-pattern could crash the VM. This is now corrected.

Own Id: OTP-10522

Erlang specification 4.7.3 defines max tuple size to 65535 elements It is now enforced to no more than 16777215 elements (arity 24 bits)

Previous edge cases (28 bits) were not validated and could cause undefined behaviour.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10633

Add insert_element/3 and delete_element/2

Own Id: OTP-10643

The previous default of a maximum of 32768 simultaneous processes has been raised to 262144. This value can be changed using the the +P command line flag of erl(1). Note that the value passed now is considered as a hint, and that actual value chosen in most cases will be a power of two.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10647 Aux Id: OTP-9892, OTP-10336

The previously (in R15) proposed scheduler wakeup strategy is now used by default. This strategy is not as quick to forget about previous overload as the previous strategy.

This change imply changes of the characteristics the system. Most notable: When a small overload comes and then disappears repeatedly, the system will for a bit longer time be willing to wake up schedulers than before. Timing in the system will due to this also change.

The previous strategy can still be enabled by passing the +sws legacy command line flag to erl.

Own Id: OTP-10661 Aux Id: OTP-10033

The +stbt command line argument of erl was added. This argument can be used for trying to set scheduler bind type. Upon failure unbound schedulers will be used.

Own Id: OTP-10668

Support ANSI in console

Unix platforms will no longer filter control sequences to the ttsl driver thus enabling ANSI and colors in console. (Thanks to Pedram Nimreezi)

Own Id: OTP-10678

Add file:allocate/3 operation

This operation allows pre-allocation of space for files. It succeeds only on systems that support such operation. (Thanks to Filipe David Manana)

Own Id: OTP-10680

Treat -Wreturn-type warnings as error when using GCC (Thanks to Tuncer Ayaz)

Own Id: OTP-10683

Implement ./otp_build configure --enable-silent-rules

With silent rules, the output of make is less verbose and compilation warnings are easier to spot. Silent rules are disabled by default and can be disabled or enabled at will by make V=0 and make V=1. (Thanks to Anthony Ramine)

Own Id: OTP-10726

Use share flags for all file operations on Windows. Thanks to Filipe David Borba Manana.

Own Id: OTP-10727

Make/fakefop adjustments. Thanks to Tuncer Ayaz and Sebastian Rasmussen.

Own Id: OTP-10733

The runtime system will now by default use 10 async threads if thread support has been enabled when building the runtime system.

This will prevent long blocking file-operations from blocking scheduler threads for long periods of time, which can be harmful. Apart from file-operations, it also effects other operations scheduled on the async thread pool by user implemented drivers.

The amount of async threads can be controlled by using the +A command line argument of erl(1). When running some offline tools you might want to disable async threads, but you are advised not to in the general case. Instead, you might want to increase the amount of async threads used.

This change imply changes of the characteristics the system compared to the previous default. The responsiveness of the system as a whole will be improved. Operations scheduled on the async thread pool will get an increased latency. The throughput of these operations may increase, or decrease depending on the type of the operations and how they get scheduled. In the case of file operations, the throughput very much depends on how the Erlang application access files. Multiple concurrent accesses to different files have the potential of an increased throughput.

Own Id: OTP-10736

The default reader group limit has been increased to 64 from 8. This limit can be set using the +rg command line argument of erl(1).

This change of default value will reduce lock contention on ETS tables using the read_concurrency option at the expense of memory consumption when the amount of schedulers and logical processors are between 8 and 64. For more information, see documentation of the +rg command line argument of erl(1).

Own Id: OTP-10737

New BIF float_to_list/2 which solves a problem of float_to_list/1 that doesn't allow specifying the number of digits after the decimal point when formatting floats (Thanks to Serge Aleynikov).

Own Id: OTP-10752

Limited support for unicode atoms in the external format and in the internal representation of the vm. This is a preparative feature in order to support communication with future releases of Erlang/OTP that may create unicode atoms.

Own Id: OTP-10753

Increased potential concurrency in ETS for write_concurrency option. The number of internal table locks has increased from 16 to 64. This makes it four times less likely that two concurrent processes writing to the same table would collide and thereby serialized. The cost is an increased constant memory footprint for tables using write_concurrency. The memory consumption per inserted record is not affected. The increased footprint can be particularly large if write_concurrency is combined with read_concurrency.

Own Id: OTP-10787

Create an erl_crash.dump if no heart exists and no ERL_CRASH_DUMP_SECONDS is set (behaviour changed).

Don't create an erl_crash.dump if heart do exists and no ERL_CRASH_DUMP_SECONDS is set (behaviour not changed).

This changes the behaviour back to the R15B02 default considering if a beam was running with no heart.

Own Id: OTP-10602

Fix linking in OpenBSD. (Thanks to Matthew Dempsky)

Own Id: OTP-10395

Fix bug causing fallback atomics to be used even though healthy gcc atomics or libatomic_ops was detected.

Own Id: OTP-10418

Ensure 'erl_crash.dump' when asked for it. This will change erl_crash.dump behaviour.

* Not setting ERL_CRASH_DUMP_SECONDS will now terminate beam immediately on a crash without writing a crash dump file.

* Setting ERL_CRASH_DUMP_SECONDS to 0 will also terminate beam immediately on a crash without writing a crash dump file, i.e. same as not setting ERL_CRASH_DUMP_SECONDS environment variable.

* Setting ERL_CRASH_DUMP_SECONDS to a negative value will let the beam wait indefinitely on the crash dump file being written.

* Setting ERL_CRASH_DUMP_SECONDS to a positive value will let the beam wait that many seconds on the crash dump file being written.

A positive value will set an alarm/timeout for restart both in beam and in heart if heart is running.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10422 Aux Id: kunagi-250 [161]

Fix bug where MSVRT100.dll was not included in the windows installer.

Own Id: OTP-10481

In the expression <<Bin/binary,...>>, if Bin was a bitstring with a size not a multiple of 8, either no exception was generated or an incorrect exception was generated. (Thanks to Adam Rutkowski for reporting this bug.)

Own Id: OTP-10524

The runtime system could crash while scheduling a port task. The port task was scheduled either due to an external I/O event being triggered, a driver timeout being triggered, or data being sent over a distribution channel.

Own Id: OTP-10556

erlang:memory(ets) erroneously included the size of each ETS-table main structure twice.

Own Id: OTP-10558

Fix compile error in generated file hipe_amd64_bifs.S for Solaris.

Own Id: OTP-10577

A faulty spec for process_info/2 could cause false dialyzer warnings. The spec is corrected.

Own Id: OTP-10584

In very rare cases, the VM could crash if a garbage collector was called while executing an appending bit syntax instruction. The symptom was a core when reallocating memory in the function erts_bs_append. The garbage collector bug is now corrected.

Own Id: OTP-10590

Improve support for building and testing in embedded ppc environments.

Own Id: OTP-10265 Aux Id: kunagi-159 [daf97f67-5724-4812-a5b6-7e86990133d2-1]

Due to a race condition on Windows, sometimes when printing to standard output and then immediately terminating erlang all data would not be printed. The emulator now waits for all data to be printed before exiting.

Own Id: OTP-10325 Aux Id: kunagi-166 [dd72d0e2-3e76-4a51-8b56-7564e24eecae]

The frequency with which sleeping schedulers are woken due to outstanding memory deallocation jobs has been reduced.

Own Id: OTP-10476 Aux Id: OTP-10162

Clearer warnings about the dangers of misuse of native functions and drivers have been added to the documentation.

Own Id: OTP-10557

Fix erl_prim_loader errors in handling of primary archive. The following errors have been corrected:

Thanks to Tuncer Ayaz and Shunichi Shinohara for reporting and co-authoring corrections.

Own Id: OTP-10071

Fix: Add port-I/O statistics for active once and true and not only active false.

Own Id: OTP-10073

The 64-bit windows installer did not look in the right directories for 64-bit version of Microsoft Visual C++ 2010 Redistibutable Package and hence took the wrong decision about having to install the redistributable package if the 32-bit version was installed but not the 64-bit and vice versa. This bug has now been fixed Furthermore the sub-installer for the redistributable package is now run in silent mode if the erlang installer is.

Own Id: OTP-10096

epmd would fail to start automatically when starting a distributed erlang node installed in a location with a whitespace in the path.

Own Id: OTP-10106

A more or less harmless bug that sometimes caused memory deallocations to be delayed longer than intended has been fixed.

Own Id: OTP-10116

Fix bug causing emulator crash when running HiPE on ARM. Bug has existed since R15B.

Own Id: OTP-10137

A bug regarding spaces in C function prototypes has been fixed. (Thanks to Richard O'Keefe.)

Own Id: OTP-10138

Corrected dtrace pid length in message related probes. (Thanks to Zheng Siyao)

Own Id: OTP-10142

Correct formating in exit error messages

Ensure displayed sizes are not negative. (Thanks to Michael Santos)

Own Id: OTP-10148

fix escript/primary archive reloading

If the mtime of an escript/primary archive file changes after being added to the code path, correctly reload the archive and update the cache. (Thanks to Tuncer Ayaz)

Own Id: OTP-10151

Doc fix: link from erlang:now/0 to os:timestamp/0

Sometimes os:timestamp/0 is more appropriate than erlang:now/0. The documentation for the former has a link to the latter; this patch adds a link in the other direction to make os:timestamp/0 more visible. Thanks to Magnus Henoch

Own Id: OTP-10180

The caret in the werl window (on Windows) could appear at the wrong place after regaining focus. This is now corrected.

Own Id: OTP-10181

Fix bug that in some cases could cause corrupted binaries in ETS tables with compressed option.

Own Id: OTP-10182

Fix use of "clever" mktime

Commit 1eef765 introduced regression (conditional _always_ evaluates to true) in which erlang:localtime_to_universaltime/2 stopped working on systems configured with timezone without DST (i.e. UTC) on *BSD platforms: 1> erlang:localtime_to_universaltime({{2012,1,1},{0,0,0}}, true). ** exception error: bad argument Thanks to Piotr Sikora

Own Id: OTP-10187

Relocate bodies of DTrace probes to the statically-linked VM.

Due to various operating systems (in both the DTrace and SystemTap worlds) not fully supporting DTrace probes (or SystemTap-compatibility mode probes) in shared libraries, we relocate those probes to the statically-linked virtual machine. This could be seen as pollution of the pristine VM by a (yet) experimental feature. However:

1. This code can be eliminated completely by the C preprocessor. 2. Leaving the probes in the dyntrace NIF shared library simply does not work correctly on too many platforms. *Many* thanks to Macneil Shonle at Basho for assisting when my RSI-injured fingers gave out. (note: Solaris 10 and FreeBSD 9.0-RELEASE can take a long time to compile)

Own Id: OTP-10189

Fix bug in ets:test_ms/2 that could cause emulator crash when using '$_' in match spec.

Own Id: OTP-10190

Supplying a filename longer than the operating system MAX_PATH to file:read_link/1 would cause a crash (Segemntation fault/Critical Error) on all platforms. This is now corrected.

Own Id: OTP-10200

If Perl was configured to interpret files as being encoded in UTF-8, the build would crash in make_preload. (Thanks to Aaron Harnly for noticing this issue.)

Own Id: OTP-10201

Fix the erlc -MP flag

Because of a copy-and-paste error in erlc.c, the -MP flag had the same effect as -MG. As a workaround, you had to pass +makedep_phony to enable the MP option. This patch makes -MP work as intended.

Own Id: OTP-10211

Allow mixed IPv4 and IPv6 addresses to sctp_bindx

Also allow mixed address families to bind, since the first address on a multihomed sctp socket must be bound with bind, while the rest are to be bound using sctp_bindx. At least Linux supports adding address of mixing families. Make inet_set_faddress function available also when HAVE_SCTP is not defined, since we use it to find an address for bind to be able to mix ipv4 and ipv6 addresses. Thanks to Tomas Abrahamsson

Own Id: OTP-10217

Fix support for leap seconds-aware timezones

erlang:universaltime_to_localtime is leap seconds-aware (since 2008), however erlang:localtime_to_universaltime is not, which gives surprising results on systems configured with leap seconds-aware timezones: 1> erlang:universaltime_to_localtime({{2012,1,1},{0,0,0}}). {{2012,1,1},{0,0,0}} 2> erlang:localtime_to_universaltime({{2012,1,1},{0,0,0}}). {{2012,1,1},{0,0,24}} and completely breaks calendar:local_time_to_universal_time_dst: 3> calendar:local_time_to_universal_time_dst({{2011,1,1},{0,0,0}}). [] Thanks to Piotr Sikora

Own Id: OTP-10227

erlsrv: gracefully stop emulator on Windows shutdown

Windows will send the SERVICE_CONTROL_SHUTDOWN event to the service control handler when shutting down the system. Instead of ignoring the event, erlsrv will now invoke the stop action. Likewise, the Erlang emulator (and it's po drivers) must not quit upon reception of the CTRL_SHUTDOWN_EVENT event in th console control handler. Thanks to Jan Kloetzke

Own Id: OTP-10228

Fix dtrace bug in file rename operation.

Own Id: OTP-10234

Fix bug in memory management of driver port data locks (PDL). In some cases PDLs could be deallocated before ready_async or async_free callback was called.

Own Id: OTP-10249

Add port and suspend options to lock-counter profiling. (Thanks to Rick Reed)

Own Id: OTP-10051

Latency when using the active_once option in gen_tcp communication is reduced.

Own Id: OTP-10055 Aux Id: sto139

Remove bit8 option support from inet

Own Id: OTP-10056

The OS Pid of a port program is now available by calling erlang:port_info(Port,os_pid), Thanks to Matthias Lang for the original patch.

Own Id: OTP-10057

Fix openpty usage in run_erl.

Reopening a slave file descriptor which was closed earlier could lead to a misbehaving connection. This has now been remedied.

Own Id: OTP-10076

Remove all code, documentation, options and diagnostic functions which were related to the experimental hybrid heap implementation.

Own Id: OTP-10105

Optimizations of memory deallocations.

Own Id: OTP-10162 Aux Id: OTP-7775

Optimization of process locking.

Own Id: OTP-10163

Added a xcomp example file for powerpc-dso-linux-gnu

Own Id: OTP-10198

Detect when middle endian doubles are used by a platform and account for it when decoding floats. (Thanks to Mike Sperber)

Own Id: OTP-10209

On Linux systems using heart (erl -heart) and a HEAR_BEAT_TIMEOUT less than default, heart could fire even though Erlang was running fine after approx 298 to 497 days (depending on kernel config). This was due to the behaviour of the times(2) system call. Usage of times(2) is now replaced with clock_gettime(2) and the CLOCK_MONOTONIC clock, resulting in a more stable solution. The Erlang VM itself has used clock_gettime(2) on linux since before R12B, so this only affects the heart program.

Own Id: OTP-10111 Aux Id: seq12075

If threads support for the runtime system had been disabled at compile time (--disable-threads had been passed to configure), and the +A command line argument of erl was passed when starting the runtime system, driver_system_info() erroneously claimed that the runtime system had async threads even though it had not.

Due to this bug the file driver did not split tasks into smaller chunks, but instead completed the whole task at once, i.e., the scheduler got occupied with I/O for a longer time than intended.

Own Id: OTP-10059

A proposal for a new scheduler wakeup strategy has been implemented. For more information see the documentation of the +sws command line argument of erl.

Own Id: OTP-10033 Aux Id: Seq12025

A switch for configuration of busy wait length for scheduler threads has been added. For more information see the documentation of the +sbwt command line argument of erl.

Own Id: OTP-10044 Aux Id: Seq11976

The extra memory barriers introduced by bug-fix OTP-9281 were unnecessarily used also on tables without the write_concurrency option enabled. This could unnecessarily degrade performance of ETS tables without write_concurrency on some hardware (e.g. PowerPC) while not effecting performance at all on other hardware (e.g. x86/x86_64).

OTP-9281 (R14B03): ETS tables using the write_concurrency option could potentially get into an internally inconsistent state.

Own Id: OTP-10048 Aux Id: OTP-9281

erlang:system_profile errorneous profiled the profiler process when observing runnable processes. This has been corrected.

Own Id: OTP-9849

Calling trace_info/2 asking for information about a function that had native could could crash the run-time system.

Own Id: OTP-9886

reduce smp locking time range in erts_garbage_collect (thanks to Jovi Zhang)

Own Id: OTP-9912

Fix typo in supervisor behaviour doc (Thanks to Ricardo Catalinas Jiménez)

Own Id: OTP-9924

Correct spelling of registered (Thanks to Richard Carlsson)

Own Id: OTP-9925

erts: Remove unused variable (Thanks to Jovi Zhang)

Own Id: OTP-9926

Fix bug in ETS with compressed option and insertion of term containing large integers (>2G) on 64-bit machines. Seen to cause emulator crash. (Thanks to Diego Llarrull for excellent bug report)

Own Id: OTP-9932

Handle Linux OS where /sys/devices/system/node is only readable by root. Fallback to /sys/devices/system/cpu for topology info.

Own Id: OTP-9978

When an escript ends now all printout to standard output and standard error gets out on the terminal. This bug has been corrected by changing the behaviour of erlang:halt/0,1, which should fix the same problem for other escript-like applications, i.e that data stored in the output port driver buffers got lost when printing on a TTY and exiting through erlang:halt/0,1.

The BIF:s erlang:halt/0,1 has gotten improved semantics and there is a new BIF erlang:halt/2 to accomplish something like the old semantics. See the documentation.

Now erlang:halt/0 and erlang:halt/1 with an integer argument will close all ports and allow all pending async threads operations to finish before exiting the emulator. Previously erlang:halt/0 and erlang:halt(0) would just wait for pending async threads operations but not close ports. And erlang:halt/1 with a non-zero integer argument would not even wait for pending async threads operations.

To roughly the old behaviour, to not wait for ports and async threads operations when you exit the emulator, you use erlang:halt/2 with an integer first argument and an option list containing {flush,false} as the second argument. Note that now is flushing not dependant of the exit code, and you can not only flush async threads operations which we deemed as a strange behaviour anyway.

Also, erlang:halt/1,2 has gotten a new feature: If the first argument is the atom 'abort' the emulator is aborted producing a core dump, if the operating system so allows.

Own Id: OTP-9985

Added check to inet driver to avoid building on operating systems that do not yet have IPv6 compatible socket API. (Thanks to Peer Stritzinger)

Own Id: OTP-9996

Fix bug when the number of CPUs actually found is lower than the configured value. (Thanks to Benjamin Herrenschmidt)

Own Id: OTP-10004

The runtime system without SMP support and without thread support erroneously busy waited when no work was present. This bug first appeared in erts-5.9.

Own Id: OTP-10019

Various typographical errors corrected in documentation for common_test, driver, erl_driver and windows installation instructions. (Thanks to Tuncer Ayaz)

Own Id: OTP-10037

Fix memory leak caused by race on exiting process

Own Id: OTP-10041

Add erlang:statistics(scheduler_wall_time) to ensure correct determination of scheduler utilization. Measuring scheduler utilization is strongly preferred over CPU utilization, since CPU utilization gives very poor indications of actual scheduler/vm usage.

Own Id: OTP-9858

ERTS internal API improvements. In some cases the amount of atomic read operations needed have been reduced due to this.

Own Id: OTP-9922

The DTrace source patch from Scott Lystig Fritchie is integrated in the source tree. Using an emulator with dtrace probe is still not supported for production use, but may be a valuable debugging tool. Configure with --with-dynamic-trace=dtrace (or --with-dynamic-trace=systemtap) to create a build with dtrace probes enabled. See runtime_tools for documentation and examples.

Own Id: OTP-10017

enif_make_copy may invalidate enif_inspect_binary.

Own Id: OTP-9828

A feature test for the lwsync instruction performed on PowerPC hardware at runtime system startup got into an eternal loop if the instruction was not supported. This bug was introduced in erts-5.9/OTP-R15B.

Own Id: OTP-9843

I/O events could potentially be delayed for ever when enabling kernel-poll on a non-SMP runtime system executing on Solaris. When also combined with async-threads the runtime system hung before completing the boot phase. This bug was introduced in erts-5.9/OTP-R15B.

Own Id: OTP-9844

Honor option packet_size for http packet parsing by both TCP socket and erlang:decode_packet. This gives the ability to accept HTTP headers larger than the default setting, but also avoid DoS attacks by accepting lines only up to whatever length you wish to allow. For consistency, packet type line also honor option packet_size. (Thanks to Steve Vinoski)

Own Id: OTP-9389

A few contracts in the lists module have been corrected.

Own Id: OTP-9616

The Unicode noncharacter code points 16#FFFE and 16#FFFE were not allowed to be encoded or decoded using the unicode module or bit syntax. That was inconsistent with the other noncharacters 16#FDD0 to 16#FDEF that could be encoded/decoded. To resolve the inconsistency, 16#FFFE and 16#FFFE can now be encoded and decoded. (Thanks to Alisdair Sullivan.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9624

Make epp search directory of current file first when including another file This completes a partial fix in R11 that only worked for include_lib(). (Thanks to Richard Carlsson)

Own Id: OTP-9645

Fixed memory leak in enif_inspect_io_list_as_binary when applied on a process independent environment.

Own Id: OTP-9668

The number of beam catches allowed in code are no longer statically defined and will grow according to its need.

Own Id: OTP-9692

Add missing parenthesis in heart doc.

Add missing spaces in the Reference Manual distributed section.

In the HTML version of the doc those spaces are necessary to separate those words.

Own Id: OTP-9693

Fixes module erlang doc style: option description (Thanks to Ricardo Catalinas Jiménez)

Own Id: OTP-9697

Specifying a scope to binary:match/3 when using multiple searchstrings resulted in faulty return values. This is now corrected.

Own Id: OTP-9701

The runtime system crashed if more than one thread tried to exit the runtime system at the same time.

Own Id: OTP-9705

Fix documentation for erlang:process_flag/2

For the subsection about process_flag(save_calls, N) there's an unrelated paragraph about process priorities which was copied from the preceding subsection regarding process_flag(priority, Level). (Thanks to Filipe David Manana)

Own Id: OTP-9714

Calls to erlang:system_flag(schedulers_online, N) and/or erlang:system_flag(multi_scheduling, block|unblock) could cause internal data used by this functionality to get into an inconsistent state. When this happened various problems occurred. This bug was quite hard to trigger, so hopefully no-one has been effected by it.

A spinlock used by the run-queue management sometimes got heavily contended. This code has now been rewritten, and the spinlock has been removed.

Own Id: OTP-9727

Use libdlpi to get physical address (Thanks to Trond Norbye)

Own Id: OTP-9818

An option list argument can now be passed to file:read_file_info/2, file:read_link_info/2 and file:write_file_info/3 and set time type information in the call. Valid options are {time, local}, {time, universal} and {time, posix}. In the case of posix time no conversions are made which makes the operation a bit faster.

Own Id: OTP-7687

A number of memory allocation optimizations have been implemented. Most optimizations reduce contention caused by synchronization between threads during allocation and deallocation of memory. Most notably:

Own Id: OTP-7775

The ethread atomic memory operations API used by the runtime system has been extended and improved.

The ethread library now also performs runtime tests for presence of hardware features, such as for example SSE2 instructions, instead of requiring this to be determined at compile time.

All uses of the old deprecated atomic API in the runtime system have been replaced with the use of the new atomic API. In a lot of places this change imply a relaxation of memory barriers used.

Own Id: OTP-9014

gen_sctp:open/0-2 may now return {error,eprotonosupport} if SCTP is not supported

gen_sctp:peeloff/1 has been implemented and creates a one-to-one socket which also are supported now

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9239

Sendfile has been added to the file module's API. sendfile/2 is used to read data from a file and send it to a tcp socket using a zero copying mechanism if available on that OS.

Thanks to Tuncer Ayaz and Steve Vinovski for original implementation

Own Id: OTP-9240

enif_get_reverse_list function added to nif API. This function should be used to reverse small lists which are deep within other structures making it impractical to do the reverse in Erlang.

Own Id: OTP-9392

The deprecated concat_binary/1 BIF has been removed. Use list_to_binary or iolist_to_binary/1 instead.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9421

Erlang/OTP can now be built using parallel make if you limit the number of jobs, for instance using 'make -j6' or 'make -j10'. 'make -j' does not work at the moment because of some missing dependencies.

Own Id: OTP-9451

Line number and filename information are now included in exception backtraces as a fourth element in the MFA tuple. The information will be pretty-printed by the shell and used by common_test to provide better indication of where a test case.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9468

All binary constants used to be handled as heap binaries (i.e. the entire binary would be copied when sent to another process). Binary constants larger than 64 bytes are now refc binaries (i.e. the actual data in the binary will not be copied when sent to another process).

Own Id: OTP-9486

If a float and an integer is compared, the integer is only converted to a float if the float datatype can contain it. Otherwise the float is converted to an integer.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9497

Add NIF function enif_is_number

This function allows for easily determining if a term represents or not a number (integer, float, small or big).(Thanks to Filipe David Manana)

Own Id: OTP-9629

The ERTS internal system block functionality has been replaced by new functionality for blocking the system. The old system block functionality had contention issues and complexity issues. The new functionality piggy-backs on thread progress tracking functionality needed by newly introduced lock-free synchronization in the runtime system. When the functionality for blocking the system isn't used, there is more or less no overhead at all. This since the functionality for tracking thread progress is there and needed anyway.

Own Id: OTP-9631

An ERTS internal, generic, many to one, lock-free queue for communication between threads has been introduced. The many to one scenario is very common in ERTS, so it can be used in a lot of places in the future. Currently it is used by scheduling of certain jobs, and the async thread pool, but more uses are planned for the future.

Drivers using the driver_async functionality are not automatically locked to the system anymore, and can be unloaded as any dynamically linked in driver.

Scheduling of ready async jobs is now also interleaved in between other jobs. Previously all ready async jobs were performed at once.

Own Id: OTP-9632

Tuple funs (a two-element tuple with a module name and a function) are now officially deprecated and will be removed in R16. Use 'fun M:F/A' instead. To make you aware that your system uses tuple funs, the very first time a tuple fun is applied, a warning will be sent to the error logger.

Own Id: OTP-9649

Changed the internal BIF calling convention. Will make simpler faster code and allow BIFs with an arbitrary arity.

Own Id: OTP-9662

Windows native critical sections are now used internally in the runtime system on Windows as mutex implementation. This since they perform better under extreme contention than our own implementation.

Own Id: OTP-9671

Convert some erl_nif macros into inline functions. Allow for better compile time type checking. (Thanks to Tuncer Ayaz)

Own Id: OTP-9675

The +scl command line flag has been added. It can be used for disabling compaction of scheduler load. For more information see the erl(1) documentation.

Own Id: OTP-9695

The build system has been updated so that Erlang/OTP can be built on Mac OS X Lion systems without a GCC compiler. The INSTALL guide has been updated with instructions on how to install a GCC compiler and build Erlang/OTP with it, in order to get a run-time system with better performance.

Own Id: OTP-9712

When loading a module, the system use to run on a single scheduler during the entire loading process. This has been changed to only take down the system just before inserting the loaded code into the system tables, resulting in a much shorter disruption if a module is loaded in a busy system. (Suggested by Bob Ippolito.)

Own Id: OTP-9720

Possible to run HiPE without floating point exceptions (FPE). Useful on platforms that lack reliable FPE. Slower float operations compared to HiPE with FPE.

Own Id: OTP-9724

As of ERTS version 5.9 (OTP-R15B) the runtime system will by default not bind schedulers to logical processors.

If the Erlang runtime system is the only operating system process that binds threads to logical processors, this improves the performance of the runtime system. However, if other operating system processes (as for example another Erlang runtime system) also bind threads to logical processors, there might be a performance penalty instead. In some cases this performance penalty might be severe. Due to this, we change the default so that the user must make an active decision in order to bind schedulers.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9726

The use of erlang:system_flag(scheduler_bind_type, _) and erlang:system_flag(cpu_topology, _) have been deprecated and scheduled for removal in erts-5.10/OTP-R16. For more information see the documentation of erlang:system_flag/2.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9749

An ancient workaround for a Windows bug was removed from the open_port code, open_port({spawn,...}...) is now faster. Thanks to Daniel Goertzen.

Own Id: OTP-9766

The use of deprecated 32bit time_t on 32bit Windows is removed.

Own Id: OTP-9767

The NIF reload mechanism is deprecated. Do not use it as an upgrade method for live production systems. It might be removed in future releases. It can still serve as a development feature but a warning message will be logged each time it is used.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9771

The driver interface has been changed to enable 64-bit aware drivers. Most importantly the return types for ErlDrvEntry callbacks 'call' and 'control' has ben enlarged which require drivers to be changed or they will cause emulator crashes. See Rewrites for 64-bit driver interface in the driver manual.

Due to this driver version management is now mandatory. A driver that is not written with version management or a driver that was compiled with the wrong major version will be not be loaded by the emulator.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9795

Eliminate use of deprecated regexp module

Own Id: OTP-9810

Several bugs concerning constant binary constructions such as <<0:4294967295>> have been corrected. Depending on the actual size of the binary and the type of run-time system (32-bit, halfword, 64-bit), such expression could either crash the run-time system or make the loader refuse loading of the module.

Own Id: OTP-9284

The Erlsrv utility failed to stop the erlang machine if no StopAction was defined when the service was stopped. This is now corrected.

Own Id: OTP-9344

Due to a bug in glibc the runtime system could abort while trying to destroy a mutex. A fix for this was introduced in R14B02. This fix did, however, not solve the problem. The runtime system will now issue a warning instead of aborting.

Own Id: OTP-9373 Aux Id: OTP-9009

Replace atom in DRV macro in prim_file with string

An experimental version of Dialyzer discovered that the atom that replaced the DRV macro in prim_file ends up in calls to erlang:open_port({spawn, Driver}, Portopts) as the Driver argument. The documentation states that this call requires a string there.

This change is also consistent with the one introduced in commit 0f03b1e9d2bef3bc830c31a369261af4c5234727 by Kostis Sagonas.

Own Id: OTP-9377

Fix typos in the epmd documentation (Thanks to Holger Weiß )

Own Id: OTP-9387

Fix faulty integer terms created by NIF API from 64-bit integers on halfword emulator. (Thanks to Paolo Negri and Paul Davis)

Own Id: OTP-9394

Fix epmd crash on vxworks caused by faulty argument to select() system call.

Own Id: OTP-9427 Aux Id: seq11855

The ets:test_ms function could in rare cases truncate the error messages. This is now corrected.

Own Id: OTP-9435

Fix bug related to hibernate and HiPE (clear F_HIBERNATE_SCHED flag)

F_HIBERNATE_SCHED flag that was introduced in b7ecdcd1ae9e11b8f75e must be cleared in hipe_mode_switch as well. Otherwise, processes running HiPE code that hibernate, wake up and then trap into a BIF will not be rescheduled.(Thanks to Paul Guyot)

Own Id: OTP-9452

Fix bug in FreeBSD topology detection code (Thanks to Paul Guyot)

Own Id: OTP-9453

Fix use of logical operator && with constant operand instead of bitwise & (Thanks to Cristian Greco)

Own Id: OTP-9454

inet: error if fd does not match socket domain

If an IPv4 fd is opened as an IPv6 socket, unexpected behaviour can occur. For example, if an IPv4 UDP socket is opened and passed into Erlang as an IPv6 socket, the first 3 bytes (corresponding to 1 byte representing the protocol family, 2 bytes set to the port) are stripped from the payload. The cause of the UDP payload truncation happens in inet_drv.c:packet_inet_input when a call to inet_get_address fails silently because the family is set to PF_INET6 but the buffer len is the size of an IPv4 struct sockaddr_in.

(Thanks to Andrew Tunnell-Jones for finding the bug and the test case!)

Own Id: OTP-9455

erts: use a union to avoid strict aliasing issues

Use a union for pointer type conversion to avoid compiler warnings about strict-aliasing violations with gcc-4.1. gcc >= 4.2 does not emit the warning. erts: adapt matrix_nif to R14 erl_nif API changes (Thanks To Tuncer Ayaz)

Own Id: OTP-9487

fix 64-bit issues in the garbage collection (Thanks to Richard Carlsson)

Own Id: OTP-9488

epmd: fix compiler warnings

Suppress compiler warnings about ignored return values. (Thanks to Michael Santos )

Own Id: OTP-9500

Fix non-existing function (erlang:disconnect/1) in distributed reference manual (Thanks to Fabian Król)

Own Id: OTP-9504

Document fdatasync -lrt requirement (SunOS <= 5.10) (Thanks to Tuncer Ayaz)

Own Id: OTP-9512

Let epmd ignore empty ERL_EPMD_ADDRESS

If the environment variable ERL_EPMD_ADDRESS is set to the empty string, empd now behaves like it does by default when ERL_EPMD_ADDRESS is unset. That is, in this case, epmd now listens on all available interfaces instead of using only the loopback interface, which happened because epmd added the loopback address to the (in this case empty) list of addresses specified via ERL_EPMD_ADDRESS.

Also, epmd now ignores ERL_EPMD_ADDRESS if it contains only separator characters (comma and space).

The same applies to epmd's -address option.(Thanks to Holger Weiß)

Own Id: OTP-9525

Remove dead code in erl_compile (Thanks to Tuncer Ayaz)

Own Id: OTP-9527

Add erlang:external_size/2 BIF

This BIF's second parameter is a list of options. Currently the only allowed option is {minor_version, Version} where version is either 0 (default) or 1. (Thanks to Filipe David Manana )

Own Id: OTP-9528

Fix enif_compare on 64bits machines

In 64bits machines the Sint type has a size of 8 bytes, while on 32bits machines it has a 4 bytes size. enif_compare was ignoring this and therefore returning incorrect values when the result of the CMP function (which returns a Sint value) doesn't fit in 4 bytes. (Thanks to Filipe David Manana)

Own Id: OTP-9533

Implement or fix -Werror option

If -Werror is enabled and there are warnings no output file is written. Also make sure that error/warning reporting is consistent. (Thanks to Tuncer Ayaz)

Own Id: OTP-9536

In some rare cases we did not have a run queue when scheduling misc ops. This is now fixed.

Own Id: OTP-9537

Remove misc. compiler warnings

Own Id: OTP-9542

Two bugs in gen_sctp has been corrected: getopts/setopts hence also send could only be called from socket owner, and options 'linger', 'rcvbuf' and 'sndbuf' was read from wrong protocol layer hence read wrong values by getopts.

Own Id: OTP-9544

Erlang/OTP can now be built on MacOS X Lion.

Own Id: OTP-9547

XML files have been corrected.

Own Id: OTP-9550 Aux Id: OTP-9541

Fix potential errors inspired by running cppcheck(1) (Thanks to Christian von Roques)

Own Id: OTP-9557

When auxiliary work was enqueued on a scheduler, the wakeup of the scheduler in order to handle this work could be lost. Wakeups in order to handle ordinary work were not effected by this bug. The bug only effected runtime systems with SMP support as follows:

Own Id: OTP-9567

Handle rare race in the crypto key server functionality

Own Id: OTP-9586

Types and specifications have been added.

Own Id: OTP-9356

New allocator strategy "address order first fit". May ease the emptying of memory carriers and thereby real release of memory back to the OS.

Own Id: OTP-9424

The new erlang:check_old_code/1 BIF checks whether a module has old code.

Own Id: OTP-9495

Update documentation and specifications of some of the zlib functions.

Own Id: OTP-9506

Detect the available CPUs on IRIX

Add support for querying the number of configured and online processors on SGI systems running IRIX.(Thanks to Holger Weiß)

Own Id: OTP-9531

Fix binary and iolist overflow problems. Typically problems arose in length calculation where the result would exceed (1 bsl 32 - 1).

Own Id: OTP-9118

Using the old erlang shell (i.e. erl instead on werl) on windows and doing several init:restart's would eventually hang the VM. That is no longer the case.

Own Id: OTP-9139

Removed recursive C code when printing Erlang terms to buffers, avoiding stack overflows that could cause VM to crash.

Own Id: OTP-9140

The send_timeout option in gen_tcp did not work properly in active mode or with {active,once} options. This is now corrected.

Own Id: OTP-9145

Fixed various typos across the documentation (Thanks to Tuncer Ayaz)

Own Id: OTP-9154

Remove duplicate stack entries which could occur after calling certain BIFs.

Own Id: OTP-9163

A race when starting two nodes simultaneously using run_erl has been removed.

Own Id: OTP-9164

Add documentation on .erlang processing back again (Thanks to Gabor Liptak)

Own Id: OTP-9189

Remove gratuitous paren in driver_entry(Thanks to Tuncer Ayaz)

Own Id: OTP-9192

Fix some wrong pointer dereferences (Thanks to Cristian Greco)

Own Id: OTP-9194

erts: Remove unused variables (Thanks to Tuncer Ayaz)

Own Id: OTP-9205

The documentation for init:get_args/0 has been removed. init:get_args/0 itself was deprecated in R9C and removed in R12B. (Thanks to Eric Pailleau.)

Own Id: OTP-9209

Allow user to specify the IP address epmd binds to

The IP address(es) epmd binds to can now be specified by the user, either via epmd's new "-address" option or (if that's not used) by setting the environment variable ERL_EPMD_ADDRESS. Multiple addresses may be specified using a comma-separated list. If the loopback address is not in this list, it will be added implicitly, so that the daemon can be queried by an interactive epmd process.(Thanks to Holger Weiß)

Own Id: OTP-9213

epmd: include host address in local access check

In FreeBSD jails, the source and destination address of connections to localhost are changed to be the IP address of the jail. Consider connections from the host's IP address to itself (e.g., the source and destination address match) to be local for the access control checks. (Thanks to Michal Santos and Tom at diogunix.com)

Own Id: OTP-9214

Fix list returned by net_kernel:epmd_module

Function epmd_module of net_kernel returns a list instead of an atom, when the epmd_module-flag is used. (Thanks to Markus Knofe)

Own Id: OTP-9215

Fix epmd's dbg_perror() output

The dbg_perror() function now hands the current errno value over to dbg_gen_printf(). This fixes the problem that errno had been reset to zero by the time it was used (to print the corresponding error message) in the dbg_gen_printf() function. (Thanks to Holger Weiß)

Own Id: OTP-9223

heart: remove garbage appended to heart command

heart:get_cmd/0 is documented to return an empty string if the command is cleared. get_cmd/0 returns 2 extra bytes: 1 byte for the trailing null, 1 byte from the op (the op is an unsigned char and 2 bytes are allocated for it in the returned buffer). (Thanks to Michael Santos)

Own Id: OTP-9224

file: fix hang reading compressed files

The gzio driver goes into an infinite loop when reading past the end of a compressed file. Reported-By: Alex Morarash (Thanks to Michael Santos)

Own Id: OTP-9245

Eliminate alias warnings from gcc 4.5.2

Own Id: OTP-9250

Unsigned integer may overflow in error message (Thanks to Michael Santos)

Own Id: OTP-9251

Driver names should be strings, not atoms

Own Id: OTP-9253

driver_entry: Remove gratuitous paren and fix typo (Thanks to Tuncer Ayaz)

Own Id: OTP-9254

Fix format specifiers in erl_exit messages

Fix an error message by using an unsigned integer specifier as seen in a tweet by @metabrew: #erlang VM crashed with "no next heap size found: -2090496108, offset 0", suddenly allocated all available RAM

Also correct mis-typed string formats in bif.c.(Thanks to Michael Santos)

Own Id: OTP-9262

net_drv: remove unused tcp request id inet_drv: remove gratuitous assignment (Thanks to Tuncer Ayaz)

Own Id: OTP-9263

Teach run_erl RUN_ERL_DISABLE_FLOWCNTRL for disabling flow control

Flow control can cause unwanted behaviour of the beam process, if accidentally hit Ctrl-S (instead of Ctrl-D to detach) the entire beam may be blocked.

Fix this problem by making it possible to turn off flow control by setting the environment variable RUN_ERL_DISABLE_FLOWCNTRL. (Thanks to Jonas Faklkevik)

Own Id: OTP-9270

The following bugs due to missing memory barriers have been fixed:

Own Id: OTP-9281

Fix halfword bug for ETS ordered_set when doing select/match with partly bound key.

Own Id: OTP-9292

Fix bug in code:is_module_native that caused crash for deleted modules.

Own Id: OTP-9298

Calling driver_async_cancel() could cause a scheduler thread to enter an eternal loop doing no useful work. (Thanks to Anders Ramsell)

Own Id: OTP-9302

New enif_is_exception function to allow NIFs to determine whether an ERL_NIF_TERM represents an exception. (Thanks to Steve Vinoski)

Own Id: OTP-9150

A process being garbage collected by another process could be scheduled on another scheduler. This prevented this scheduler from doing any useful work until the garbage collection was done. This either occurred due to a explicit call to the garbage_collect/1 BIF, or due to a garbage collection part of code loading. A process being garbage collected like this will now not be scheduled until the garbage collection has completed.

Own Id: OTP-9211

Remove unnecessary validation copy in prim_file:drv_command/3 (Thanks to Tony Rogvall)

Own Id: OTP-9276

Symbolic link handling on windows have been slightly updated to map error conditions more consequently and correctly read directory links created outside of the Erlang environment.

Own Id: OTP-9279

Due to standard library DLL mismatches between versions of OpenSSL and Erlang/OTP, OpenSSL is now linked statically to the crypto driver on Windows. This fixes problems starting crypto when running Erlang as a service on all Windows versions.

Own Id: OTP-9280

Halfword emulator memory handling improvements:

Own Id: OTP-9291 Aux Id: seq11841

The value set in the undocumented and unsupported ERL_version_FLAGS (e.g. ERL_R14B03_FLAGS) environment variable can now be overridden by the command line (similar to ERL_AFLAGS).

Own Id: OTP-9297

Fix halfword emulator bug in ets:select_delete for ordered_set that caused emulator to crash.

Own Id: OTP-9258 Aux Id: seq11836

Ets table type ordered_set could order large integer keys wrongly on pure 64bit platforms. This is now corrected.

Own Id: OTP-9181

The status of a process was unnecessarily set to waiting before a process was enqueued in a run queue. This bug was harmless up until OTP-R14B01. In OTP-R14B02 erlang:hibernate/3 was fixed (OTP-9125). After the introduction of OTP-9125, the previously harmless process status bug sometimes caused erroneous badarg exceptions from process_info().

OTP-9125 also introduced a thread unsafe access to the status field of a process which now also have been fixed.

*** INCOMPATIBILITY with noxs ***

Own Id: OTP-9197

The scroll wheel now scrolls the werl window on Windows.

Own Id: OTP-8985

Some malformed distribution messages could cause VM to crash, this is now corrected.

Own Id: OTP-8993

The OS function getifaddrs() can return NULL in some address fields for e.g PPP and tunnel devices which caused the emulator to segfault. This bug has now been corrected.

Own Id: OTP-8996

The expression <<A:0>> would always produce an empty binary, even if A was not an integer. Corrected to cause a badarg exception if the type of A is invalid. (Thanks to Zvi.)

Own Id: OTP-8997

A bug that potentially could cause an emulator crash when deleting an ETS-table has been fixed. A resource leak when hitting the maximum amount of ETS-tables allowed has also been fixed.

Own Id: OTP-8999

A bug in the exit/2 BIF could potentially cause an emulator crash.

Own Id: OTP-9005

Due to a bug in glibc the runtime system could abort while trying to destroy a mutex. The runtime system will now issue a warning instead of aborting.

Own Id: OTP-9009

A bug in epmd could create strange behaviour when listen() calls failed. This is now corrected thanks to Steve Vinoski.

Own Id: OTP-9024

When setting file_info the win32_driver will now correctly set access and modified time. Previously these entities were swapped.

Own Id: OTP-9046

Setting scheduler bind type to unbound failed if binding of schedulers wasn't supported, or if CPU topology wasn't present. This even though the documentation stated that it is possible to set the bind type to unbound.

Own Id: OTP-9056 Aux Id: Seq11779

Two problems were fixed in crash dump: The time left for timers are now shown as unsigned integers and the contents of ordered_set ETS tables is no longer included.

Own Id: OTP-9057

The VM could fail to set IP_TOS and SO_PRIORITY in certain situations, either because sockets were supplied as open file descriptors, or because SO_PRIORITY by default was set higher than the user can explicitly set it to. Those situations are now handled.

Own Id: OTP-9069

Wx on MacOS X generated complains on stderr about certain cocoa functions not being called from the "Main thread". This is now corrected.

Own Id: OTP-9081

Fix a couple typos in driver_entry(3) (thanks to Tuncer Ayaz).

Own Id: OTP-9085

Mention that "-detached" implies "-noinput"

Clarify that specifying "-noinput" is unnecessary if the "-detached" flag is given. (thanks to Holger Weiß)

Own Id: OTP-9086

A potential problem (found by code inspection) when calling a fun whose code was not loaded has been fixed.

Own Id: OTP-9095

The emulator could get into a state where it didn't check for I/O.

Own Id: OTP-9105 Aux Id: Seq11798

Attempting to create binaries exceeding 2Gb (using for example term_to_binary/1) would crash the emulator with an attempt to allocate huge amounts of memory. (Thanks to Jon Meredith.)

Own Id: OTP-9117

Fix erlang:hibernate/3 on HiPE enabled emulator (Thanks to Paul Guyot)

Own Id: OTP-9125

From this release, the previously experimental halfword emulator is now official. It can be enabled by giving the --enable-halfword-emulator option to the configure script.

The halfword emulator is a 64-bit application, but uses halfwords (32-bit words) for all data in Erlang processes, therefore using less memory and being faster than the standard 64-bit emulator. The total size of all BEAM code and all process data for all processes is limited to 4Gb, but ETS tables and off-heap binaries are only limited by the amount of available memory.

Own Id: OTP-8941

32-bit atomic memory operations have been introduced internally in the run time system, and are now used where appropriate. There were previously only atomic memory operations of word size available. The 32-bit atomic memory operations slightly reduce memory consumption, and slightly improve performance on 64-bit runtime systems.

Own Id: OTP-8974

Performance enhancements for looking up timer-entries and removing timers from the wheel.

Own Id: OTP-8990

Write accesses to ETS tables have been optimized by reducing the amount of atomic memory operations needed during a write access.

Own Id: OTP-9000

Strange C coding in the VM made the -D_FORTIFY_SOURCE option to gcc-4.5 react badly. The code is now cleaned up so that it's accepted by gcc-4.5.

Own Id: OTP-9025

The memory footprint for loaded code has been somewhat reduced (especially in the 64-bit BEAM machine).

Own Id: OTP-9030

The maximum number of allowed arguments for an Erlang function has been lowered from 256 to 255, so that the number of arguments can now fit in a byte.

Own Id: OTP-9049

Dependency generation for Makefiles has been added to the compiler and erlc. See the manual pages for compile and erlc. (Thanks to Jean-Sebastien Pedron.)

Own Id: OTP-9065

Fix format_man_pages so it handles all man sections and remove warnings/errors in various man pages.

Own Id: OTP-8600

The configure command line argument --enable-ethread-pre-pentium4-compatibility had no effect. This option is now also automatically enabled if required on the build machine.

Own Id: OTP-8847

Windows 2003 and Windows XP pre SP3 would sometimes not start the Erlang R14B VM at all due to a bug in the cpu topology detection. The bug affects Windows only, no other platform is even remotely affected. The bug is now corrected.

Own Id: OTP-8876

The HiPE run-time in the 64-bit emulator could do a 64-bit write to a 32-bit struct field. It happened to be harmless on Intel/AMD processors. Corrected. (Thanks to Mikael Pettersson.)

Own Id: OTP-8877

A bug in erl_drv_tsd_get() and enif_tsd_get() could cause an emulator crash. These functions are currently not used in OTP. That is, the crash only occur on systems with user implemented NIF libraries, or drivers that use one of these functions.

Own Id: OTP-8889

Calling erlang:system_info({cpu_topology, CpuTopologyType}) with another CpuTopologyType element than one of the documented atoms defined, detected, or used caused an emulator crash. (Thanks to Paul Guyot)

Own Id: OTP-8914

The ERTS internal rwlock implementation could get into an inconsistent state. This bug was very seldom triggered, but could be during heavy contention. The bug was introduced in R14B (erts-5.8.1).

The bug was most likely to be triggered when using the read_concurrency option on an ETS table that was frequently accessed from multiple processes doing lots of writes and reads. That is, in a situation where you typically don't want to use the read_concurrency option in the first place.

Own Id: OTP-8925 Aux Id: OTP-8544

Tracing to port could cause an emulator crash when unloading the trace driver.

Own Id: OTP-8932

Removed use of CancelIoEx on Windows that had been shown to cause problems with some drivers.

Own Id: OTP-8937

The fallback implementation used when no native atomic implementation was found did not compile. (Thanks to Patrick Baggett, and Tuncer Ayaz)

Own Id: OTP-8944

Some integer values used during load balancing could under rare circumstances wrap causing a load unbalance between schedulers.

Own Id: OTP-8950

The windows VM now correctly handles appending to large files (> 4GB).

Own Id: OTP-8958

Name resolving of IPv6 addresses has been implemented for Windows versions that support it. The use of ancient resolver flags (AI_V4MAPPED | AI_ADDRCONFIG) to the getaddrinfo() function has been removed since e.g FreeBSD regard mapped IPv4 addresses to be a security problem and the semantics of the address configured flag is uncertain.

Own Id: OTP-8969

The help texts produced by the configure scripts in the top directory and in the erts directory have been aligned and cleaned up.

Own Id: OTP-8859

When the runtime system had fewer schedulers than logical processors, the system could get an unnecessarily large amount reader groups.

Own Id: OTP-8861

run_rel has been updated to support Solaris's /dev/ptmx device and to load the necessary STREAMS modules so that to_erl can provide terminal echo of keyboard input. (Thanks to Ryan Tilder.)

Own Id: OTP-8878

The Erlang VM now supports Unicode filenames. The feature is turned on by default on systems where Unicode filenames are mandatory (Windows and MacOSX), but can be enabled on other systems with the '+fnu' emulator option. Enabling the Unicode filename feature on systems where it is not default is however considered experimental and not to be used for production. Together with the Unicode file name support, the concept of "raw filenames" is introduced, which means filenames provided without implicit unicode encoding translation. Raw filenames are provided as binaries, not lists. For further information, see stdlib users guide and the chapter about using Unicode in Erlang. Also see the file module manual page.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8887

Buffer overflows have been prevented in erlc, dialyzer, typer, run_test, heart, escript, and erlexec.

Own Id: OTP-8892

The runtime system is now less eager to suspend processes sending messages over the distribution. The default value of the distribution buffer busy limit has also been increased from 128 KB to 1 MB. This in order to improve throughput.

Own Id: OTP-8901

The distribution buffer busy limit can now be configured at system startup. For more information see the documentation of the erl +zdbbl command line flag. (Thanks to Scott Lystig Fritchie)

Own Id: OTP-8912

The inet driver internal buffer stack implementation has been rewritten in order to reduce lock contention.

Own Id: OTP-8916

New ETS option compressed, to enable a more compact storage format at the expence of heavier table operations. For test and evaluation, erl +ec can be used to force compression on all ETS tables.

Own Id: OTP-8922 Aux Id: seq11658

There is now a new function inet:getifaddrs/0 modeled after C library function getifaddrs() on BSD and LInux that reports existing interfaces and their addresses on the host. This replaces the undocumented and unsupported inet:getiflist/0 and inet:ifget/2.

Own Id: OTP-8926

Support for detection of CPU topology and binding of schedulers on FreeBSD 8 have been added. (Thanks to Paul Guyot)

Own Id: OTP-8939

Several bugs related to hibernate/3 and HiPE have been corrected. (Thanks to Paul Guyot.)

Own Id: OTP-8952

Support for soft and hard links on Windows versions and filesystems that support them is added.

Own Id: OTP-8955

The win32 virtual machine is now linked large address aware. his allows the Erlang VM to use up to 3 gigs of address space on Windows instead of the default of 2 gigs.

Own Id: OTP-8956

Fix that the documentation top index generator can handle an Ericsson internal application group.

Own Id: OTP-8875

In embedded mode, on_load handlers that called code:priv_dir/1 or other functions in code would hang the system. Since the crypto application now contains an on_loader handler that calls code:priv_dir/1, including the crypto application in the boot file would prevent the system from starting.

Also extended the -init_debug option to print information about on_load handlers being run to facilitate debugging.

Own Id: OTP-8902 Aux Id: seq11703

Windows 2003 and Windows XP pre SP3 would sometimes not start the Erlang R14B VM at all due to a bug in the cpu topology detection. The bug affects Windows only, no other platform is even remotely affected. The bug is now corrected.

Own Id: OTP-8876

Very small floating point numbers generated errors when converting from list to float in some versions of the VM, this is now corrected so that i.e. list_to_float("1.0e-324"). returns 0.0 in all versions of Erlang.

Own Id: OTP-7178

Windows Vista and Windows 7 file system virtualization, which makes "old style" windows programs execute in a file system sandbox, was previously unintentionally turned on for the Erlang VM. This is now corrected so that i.e. writes to C:\Program Files\. without administrator privileges will fail.

Own Id: OTP-7405

Fix faulty 64-bit integer term output from drivers. Large 64-bits integers did not generate correct bignums and could even cause emulator crash. Only affects drivers using ERL_DRV_INT64 or ERL_DRV_UINT64, introduced in R13B03.

Own Id: OTP-8716

Fixed: inet:setopts(S, [{linger,{true,2}}]) returned {error,einval} for SCTP sockets. The inet_drv had a bug when checking the option size.

Own Id: OTP-8726 Aux Id: seq11617

Fix libm linking with --as-needed flag

When building with "--as-needed" linker flags on Linux the build will fail. This has now been fixed.

(Thanks to Christian Faulhammer)

Own Id: OTP-8728

gen_udp:connect/3 was broken for SCTP enabled builds. It did not detect remote end errors as it should.

Own Id: OTP-8729

Reduce the risk of integer wrapping in bin vheap size counting.

The vheap size series will now use the golden ratio instead of doubling and fibonacci sequences.

Own Id: OTP-8730

ETS ordered_set containing [] as key could cause strange thing to happen, like an infinite hanging ets:select.

Own Id: OTP-8732

reference() has been substituted for ref() in the documentation.

Own Id: OTP-8733

When a native compiled module called a not loaded non-native compiled module that had an on_load function, the export entries were trashed after code loading so on the next call from the native compiled module to the non-native compiled the emulator crashed. This bug has now been fixed.

Own Id: OTP-8736

HiPE-enabled Erlang VMs running on BSD systems sometimes generated messages like "Yikes! erts_alloc() returned misaligned address 0x8016a512c". Fixed. (Thanks to Mikael Pettersson.)

Own Id: OTP-8769

A race condition in erts_poll() could cause delay of poll for I/O.

Own Id: OTP-8773

Removed some potential vulnerabilities from the Erlang Port Mapper Daemon (epmd) and straightened up access control. Also removed hazardous interfaces allowing anyone on a machine to forcefully unregister other nodes. This means that the ei_unregister/erl_unregister interfaces in erl_interface is rendered not only error prone and mystifying as before, but totally ineffective. The old behaviour of unchecked node unregistering can be restored if needed, see epmd documentation for details.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8780

Building in a source tree without prebuilt platform independent build results failed on the SSL examples when:

Own Id: OTP-8791

inet:getsockopt for SCTP sctp_default_send_param had a bug to not initialize required feilds causing random answers. It is now corrected.

Own Id: OTP-8795 Aux Id: seq11655

The hipe_bifs:get_hrvtime/0 BIF now always returns a real value even if the "perfctr" Linux kernel extension is not available. It used to return a dummy value. (Thanks to Mikael Pettersson.)

Own Id: OTP-8798

Calling a native-code compiled module with an on_load function could cause a crash. (Thanks to Mikael Pettersson.)

Own Id: OTP-8799

The emulator could crash while writing a crash dump if native-compiled modules had been loaded. (Thanks to Paul Guyot.)

Own Id: OTP-8801

The garbage collector could crash if invoked from native-compiled code after a call to a BIF. (Thanks to Paul Guyot.)

Own Id: OTP-8821

A rare memory leak in binary:matches is removed

Own Id: OTP-8823

For a socket in the HTTP packet mode, the return value from gen_tcp:recv/2,3 if there is an error in the header will be {ok,{http_error,String}} instead of {error,{http_error,String}} to be consistent with ssl:recv/2,3.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8831

ets:select_reverse/{1,2,3} are now documented.

Own Id: OTP-7863

External format of integers changed to make full use of all 32 bits of INTEGER_EXT. This is a compatible change as old code can read full 32-bit integers but only produce 28-bit integers as INTEGER_EXT.

Own Id: OTP-8540 Aux Id: seq11534

Large parts of the ethread library have been rewritten. The ethread library is an Erlang runtime system internal, portable thread library used by the runtime system itself.

Most notable improvement is a reader optimized rwlock implementation which dramatically improve the performance of read-lock/read-unlock operations on multi processor systems by avoiding ping-ponging of the rwlock cache lines. The reader optimized rwlock implementation is used by miscellaneous rwlocks in the runtime system that are known to be read-locked frequently, and can be enabled on ETS tables by passing the {read_concurrency, true} option upon table creation. See the documentation of ets:new/2 for more information. The reader optimized rwlock implementation can be fine tuned when starting the runtime system. For more information, see the documentation of the +rg command line argument of erl.

There is also a new implementation of rwlocks that is not optimized for readers. Both implementations interleaves readers and writers during contention as opposed to, e.g., the NPTL (Linux) pthread rwlock implementation which use either a reader or writer preferred strategy. The reader/writer preferred strategy is problematic since it starves threads doing the non-preferred operation.

The new rwlock implementations in general performs better in ERTS than common pthread implementations. However, in some extremely heavily contended cases this is not the case. Such heavy contention can more or less only appear on ETS tables. This when multiple processes do very large amounts of write locked operations simultaneously on the same table. Such use of ETS is bad regardless of rwlock implementation, will never scale, and is something we strongly advise against.

The new rwlock implementations depend on atomic operations. If no native atomic implementation is found, a fallback solution will be used. Using the fallback implies a performance degradation. That is, it is more important now than before to build OTP with a native atomic implementation.

The ethread library contains native atomic implementations for, x86 (32 and 64 bit), powerpc (32 bit), sparc V9 (32 and 64 bit), and tilera (32 bit). On other hardware gcc's builtin support for atomic memory access will be used if such exists. If no such support is found, configure will warn about no atomic implementation available.

The ethread library can now also use the libatomic_ops library for atomic memory accesses. This makes it possible for the Erlang runtime system to utilize optimized native atomic operations on more platforms than before. If configure warns about no atomic implementation available, try using the libatomic_ops library. Use the --with-libatomic_ops=PATH configure command line argument when specifying where the libatomic_ops installation is located. The libatomic_ops library can be downloaded from: http://www.hpl.hp.com/research/linux/atomic_ops/

The changed API of the ethread library has also caused modifications in the Erlang runtime system. Preparations for the to come "delayed deallocation" feature has also been done since it depends on the ethread library.

Note: When building for x86, the ethread library will now use instructions that first appeared on the pentium 4 processor. If you want the runtime system to be compatible with older processors (back to 486) you need to pass the --enable-ethread-pre-pentium4-compatibility configure command line argument when configuring the system.

Own Id: OTP-8544

erlang:localtime_to_universaltime({{2008, 8, 1}, {0, 0, 0}},true) when TZ=UTC now behaves consistently on all Unix platforms.

The problem fixed was originally reported by Paul Guyot on erlang-bugs mailing list:

http://www.erlang.org/pipermail/erlang-bugs/2008-November/001077.html

Own Id: OTP-8580

Optimization reducing memory consumption by two words per ETS object.

Own Id: OTP-8737

Fixes for unsupported halfword-emulator

Own Id: OTP-8745

NIF 64-bit integer support; enif_get_int64, enif_get_uint64, enif_make_int64, enif_make_uint64.

Own Id: OTP-8746

Alignment of trailing data in messages has been adjusted. This in order to be able to pass data of any type as trailing data in the future.

Own Id: OTP-8754

The obsolete/driver.h header file has been removed. It has been obsolete and deprecated since R8B. Drivers that still include obsolete/driver.h must be updated to include erl_driver.h.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8758

Added erlang:system_info(build_type) which makes it easier to chose drivers, NIF libraries, etc based on build type of the runtime system.

The NIF library for crypto can now be built for valgrind and/or debug as separate NIF libraries that will be automatically loaded if the runtime system has been built with a matching build type.

Own Id: OTP-8760

Further lessened the memory requirements of ETS objects.

Own Id: OTP-8762

The broken elib_malloc alternate memory allocator has been removed. erlang:system_info(elib_malloc) will always return false, and in R15, erlang:system_info(elib_malloc) will fail with a badarg exception.

Own Id: OTP-8764

Calling erlang:system_info/1 with the new argument update_cpu_info will make the runtime system reread and update the internally stored CPU information. For more information see the documentation of erlang:system_info(update_cpu_info).

The CPU topology is now automatically detected on Windows systems with less than 33 logical processors. The runtime system will now, also on Windows, by default bind schedulers to logical processors using the default_bind bind type if the amount of schedulers is at least equal to the amount of logical processors configured, binding of schedulers is supported, and a CPU topology is available at startup.

Own Id: OTP-8765

The SMP ERTS internal child waiter thread used on Linux system with NPTL was unintentionally disabled during cross compilation rewrites (OTP-8323 in R13B03). It has now been re-enabled. Enabling it again gives a slight performance improvement.

Own Id: OTP-8774

epmd used to generate a message to the syslog when it started up, which could be annoying. This has been changed to only generate the message if the debug swith is given. (Thanks to Michael Santos.)

Own Id: OTP-8775

The scheduler wakeup threshold is now possible to adjust at system boot. For more information see the +swt command line argument of erl.

Own Id: OTP-8811

The undocumented function inet:ifget/2 has been improved to return interface hardware address (MAC) on platforms supporting getaddrinfo() (such as BSD unixes). Note it still does not work on all platforms for example not Windows nor Solaris, so the function is still undocumented.

Buffer overflow and field init bugs for inet:ifget/2 and inet:getservbyname/2 has also been fixed.

Thanks to Michael Santos.

Own Id: OTP-8816

Optimizations for MIPS when using gcc atomics. (Thanks to Steve Vinoski)

Own Id: OTP-8834

Lock optimization in timer functionality.

Own Id: OTP-8835

Fix epmd and build environment to build on VxWorks

Own Id: OTP-8838

Driver threads, such as async threads, using port data locks peeked at the port status field without proper locking when looking up the driver queue.

Own Id: OTP-8475

The use of mmap() was unnecessarily disabled when cross compiling.

The configure arguments --with-ssl, and --with-odbc refused to accept libraries outside of $erl_xcomp_sysroot when cross compiling for no good reason.

The configure argument --with-odbc didn't handle the value yes correct.

The configure arguments --with-odbc, and --without-odbc have also been added to the configure help.

(Thanks to Steve Vinoski for reporting these issues)

Own Id: OTP-8484

A call to the BIF unregister(RegName) when a port had the name RegName registered in the runtime system without SMP support caused a runtime system crash. (Thanks to Per Hedeland for the bugfix and test case.)

Own Id: OTP-8487

The runtime system crashed if fewer logical processors were found than reported by sysconf( SC_NPROCESSORS_CONF).

Own Id: OTP-8549

Fix memory management bug causing crash of non-SMP emulator with async threads enabled. The bug did first appear in R13B03.

Own Id: OTP-8591 Aux Id: seq11554

Port locks could be prematurely destroyed.

Own Id: OTP-8612

The empd program could loop and consume 100% CPU time if an unexpected error occurred in listen() or accept(). Now epmd will terminate if a non-recoverable error occurs. (Thanks to Michael Santos.)

Own Id: OTP-8618

When kernel poll has been enabled, a livelock could in rare circumstances occur. Problem reported by Chetan Ahuja, fix by Mikael Pettersson.

Own Id: OTP-8632

Windows: Closing port of program that stalled without reading all data could deadlock scheduler thread.

Own Id: OTP-8641

On some combination of Montavista Linux on Cavium Octeon processors, some socket-related system calls returned other numbers than -1 for errors. This caused a core dump in inet_drv.c. Now the code works around this problem.

Own Id: OTP-8654

Missing memory barriers in erts_poll() could cause the runtime system to hang indefinitely.

Own Id: OTP-8658

ethr_rwmutex_tryrlock() acquired and refused to acquire a lock with inverted logic. The lock was however never acquired in a thread unsafe manner. (Thanks to JR Zhang for noting this issue)

Own Id: OTP-8662

Extreme combinations of register/unregister in a highly parallell SMP application could crash the VM. The error is corrected.

Own Id: OTP-8663

On Windows, files are now opened with FILE_SHARE_DELETE to get closer to Unix semantics.

Own Id: OTP-8667

erlang:system_info(multi_scheduling) sometimes erroneously returned enabled when it should have returned blocked.

Own Id: OTP-8675

Fix bug causing erlang:decode_packet and enif_make_string to generate faulty strings with negative character values for ascii values larger than 127. (Thanks to Paul Guyot)

Own Id: OTP-8685

open_port/2 with the spawn and spawn_executable options can include an {env,Env} option. In some cases unsetting variables would not work on Unix (typically if more variables were unset than were actually present in the environment).

Own Id: OTP-8701

A user defined CPU topology set via a call to erlang:system_flag(cpu_topology, CPUTopology) was not properly verified, and could in worst case cause an emulator crash. The emulator crash could only occur when a user defined CPU topology already existed and was redefined.

Own Id: OTP-8710

The grammar for match specifications in ERTS users guide only described the tracing dialect of match specifications. An informal grammar for the ETS dialect is added.

Own Id: OTP-8086 Aux Id: seq11333

The module binary from EEP31 (and EEP9) is implemented.

Own Id: OTP-8217

New NIF API function enif_make_new_binary

Own Id: OTP-8474

The guard BIF is_boolean/1 (introduced in R10B) has now been included in the lists of BIFs allowed in guards in the Reference Manual.

Own Id: OTP-8477

Added function zip:foldl/3 to iterate over zip archives.

Added functions to create and extract escripts. See escript:create/2 and escript:extract/2.

The undocumented function escript:foldl/3 has been removed. The same functionality can be achieved with the more flexible functions escript:extract/2 and zip:foldl/3.

Record fields has been annotated with type info. Source files as been adapted to fit within 80 chars and trailing whitespace has been removed.

Own Id: OTP-8521

A regular expression with many levels of parenthesis could cause a buffer overflow. That has been corrected. (Thanks to Michael Santos.)

Own Id: OTP-8539

erlang:decode_packet(httph_bin,..) could return corrupt header strings or even crash the VM. This has been fixed. It only happened on 32-bit VM if the header name was unknown and between 16 and 20 characters long. Sockets with simular packet option did not suffer from this bug.

Own Id: OTP-8548

New NIF features:

And some incompatible changes made to the API. For more information see the warning text in erl_nif(3).

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8555

If the 'fop' program (needed for building PDF files) cannot not be found, it is now possible to build the HTML and man pages anyway (there will also be dummy PDF files with no real content created). (Thanks to Tuncer Ayaz.)

Own Id: OTP-8559

When defining macros the closing right parenthesis before the dot is now mandatory.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8562

Local and imported functions now override the auto-imported BIFs when the names clash. The pre R14 behaviour was that auto-imported BIFs would override local functions. To avoid that old programs change behaviour, the following will generate an error:

If the BIF was added or auto-imported in OTP R14A or later, overriding it with an import or a local function will only result in a warning,

To resolve clashes, you can either use the explicit module name erlang to call the BIF, or you can remove the auto-import of that specific BIF by using the new compiler directive -compile({no_auto_import,[F/A]})., which makes all calls to the local or imported function without explicit module name pass without warnings or errors.

The change makes it possible to add auto-imported BIFs without breaking or silently changing old code in the future. However some current code ingeniously utilizing the old behaviour or the nowarn_bif_clash compiler directive, might need changing to be accepted by the compiler.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8579

A bug in re that could cause certain regular expression matches never to terminate is corrected. (Thanks to Michael Santos and Gordon Guthrie.)

Own Id: OTP-8589

The erlang:open_port spawn and spawn_executable directives can include an {env,Env} directive to set up environment variables for the spawned process. A bug prevented applications from using {env,Env} to set an environment variable whose value ended with a '=' (equal sign) character; the trailing equal sign was mistaken as an indication that an environment variable was to be cleared from the environment of the spawned process. (Thanks to Steve Vinoski.)

Own Id: OTP-8614

receive statements that can only read out a newly created reference are now specially optimized so that it will execute in constant time regardless of the number of messages in the receive queue for the process. That optimization will benefit calls to gen_server:call(). (See gen:do_call/4 for an example of a receive statement that will be optimized.)

Own Id: OTP-8623

The functions file:advise/4 and file:datasync/1 have been added. (Thanks to Filipe David Manana.)

Own Id: OTP-8637

New NIF API functions: enif_make_atom_len, enif_make_existing_atom_len, enif_make_string_len, enif_get_atom_length, enif_get_list_length, enif_is_list, enif_is_tuple (by Tuncer Ayaz)

Own Id: OTP-8640

Support for using gcc's built-in functions for atomic memory access has been added. This functionallity will be used if available and no other native atomic implementation in ERTS is available.

Own Id: OTP-8659

The number of spinlocks used when implementing atomic fall-backs when no native atomic implementation is available has been increased from 16 to 1024.

Own Id: OTP-8660

Writer preferred pthread read/write locks has been enabled on Linux.

Own Id: OTP-8661

The runtime system will by default bind schedulers to logical processors using the default_bind bind type if the amount of schedulers are at least equal to the amount of logical processors configured, binding of schedulers is supported, and a CPU topology is available at startup.

NOTE: If the Erlang runtime system is the only operating system process that binds threads to logical processors, this improves the performance of the runtime system. However, if other operating system processes (as for example another Erlang runtime system) also bind threads to logical processors, there might be a performance penalty instead. If this is the case you, are are advised to unbind the schedulers using the +sbtu command line argument, or by invoking erlang:system_flag(scheduler_bind_type, unbound).

Own Id: OTP-8666

The recently added BIFs erlang:min/2, erlang:max/2 and erlang:port_command/3 are now auto-imported (as they were originally intended to be). Due to the recent compiler change (OTP-8579), the only impact on old code defining it's own min/2, max/2 or port_command/3 functions will be a warning, the local functions will still be used. The warning can be removed by using -compile({no_auto_import,[min/2,max/2,port_command/3]}). in the source file.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8669 Aux Id: OTP-8579

There is a new option 'exclusive' to file:open/2 that uses the OS O_EXCL flag where supported to open the file in exclusive mode.

Own Id: OTP-8670

Now, binary_to_term/2 is auto-imported. This will cause a compile warning if and only if a module has got a local function with that name.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8671

Alignment of scheduler data and run queues were adjusted.

Own Id: OTP-8673

Call time breakpoint tracing

Teach call count tracing to use atomics

Own Id: OTP-8677

eprof has been reimplemented with support in the Erlang virtual machine and is now both faster (i.e. slows down the code being measured less) and scales much better. In measurements we saw speed-ups compared to the old eprof ranging from 6 times (for sequential code that only uses one scheduler/core) up to 84 times (for parallel code that uses 8 cores).

Note: The API for the eprof has been cleaned up and extended. See the documentation.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8706

Two bugs in gen_sctp has been corrected: getopts/setopts hence also send could only be called from socket owner, and options 'linger', 'rcvbuf' and 'sndbuf' was read from wrong protocol layer hence read wrong values by getopts.

Own Id: OTP-9544

Driver threads, such as async threads, using port data locks peeked at the port status field without proper locking when looking up the driver queue.

Own Id: OTP-8475

A call to the BIF unregister(RegName) when a port had the name RegName registered in the runtime system without SMP support caused a runtime system crash. (Thanks to Per Hedeland for the bugfix and test case.)

Own Id: OTP-8487

Fix memory management bug causing crash of non-SMP emulator with async threads enabled. The bug did first appear in R13B03.

Own Id: OTP-8591 Aux Id: seq11554

Fix binary_to_term crash on compressed term with corrupt size field.

Own Id: OTP-8336

Processes and/or ports could get stuck on a blocked scheduler when erlang:system_flag(multi_scheduling, block) was used.

Processes and/or ports could get stuck on an offline scheduler when schedulers online were reduced using erlang:system_flag(schedulers_online, SchedulersOnline).

Own Id: OTP-8342

Building on Windows will now work if the paths to mc.exe and rc.exe contain spaces. The README.win32 file has been updated with some information about building using Visual Studio 2008. (Thanks to Andrew Thompson.)

Own Id: OTP-8345

EPMD now correctly handles the extra data field which can be given in the ALIVE2_REQ request and retrieved in the PORT2_RESP response. (Thanks to Klas Johansson.)

Own Id: OTP-8361

The configure test for reliable floating point exceptions has been update to work on modern versions of Mac OS X. (Thanks to Trannie Carter.)

Own Id: OTP-8368

ERTS makefiles used to detect the use of a gcc C compiler by checking if CC equaled gcc. That is, the makefiles failed to detect gcc C compilers with other command line names than gcc. `configure' now substitute GCC into the makefiles. If CC is a gcc C compiler, GCC will have the value yes. (Thanks to Jean-Sébastien Pédron)

Own Id: OTP-8373

ETS bug causing the memory counter from ets:info for ordered_set's to sometimes get out of synch and absurdly high.

Own Id: OTP-8377 Aux Id: seq11442

Immediately repeated multi-scheduling block/unblock cycles using erlang:system_flag(multi_scheduling, block | unblock) could deadlock the runtime system.

Own Id: OTP-8386

A number of bugs concerning re and unicode are corrected:

re:compile no longer loses unicode option, which also fixes bug in re:split.

re:replace now handles unicode charlist replacement argument

re:replace now handles unicode RE charlist argument correctly

re:replace now handles binary unicode output correctly when nothing is replaced.

Most code, testcases and error isolation done by Rory Byrne.

Own Id: OTP-8394

The loading of native code was not properly atomic in the SMP emulator, which could cause crashes. Also a per-MFA information table for the native code has now been protected with a lock since it turns that it could be accessed concurrently in the SMP emulator. (Thanks to Mikael Pettersson.)

Own Id: OTP-8397

Fix processes in exiting status that are about to be scheduled, to not be allowed to garbage collect.

Own Id: OTP-8420

Removed bogus "\011" character sequence in documentation.

Own Id: OTP-8422

The resolver routines failed to look up the own node name as hostname, if the OS native resolver was erroneously configured, bug reported by Yogish Baliga, now fixed.

The resolver routines now tries to parse the hostname as an IP string as most OS resolvers do, unless the native resolver is used.

The DNS resolver inet_res and file resolver inet_hosts now do not read OS configuration files until they are needed. Since the native resolver is default, in most cases they are never needed.

The DNS resolver's automatic updating of OS configuration file data (/etc/resolv.conf) now uses the 'domain' keyword as default search domain if there is no 'search' keyword.

Own Id: OTP-8426 Aux Id: OTP-8381

The re module: A regular expression with an option change at the start of a pattern that had top-level alternatives could cause overwriting and/or a crash. (Thanks to Michael Santos.)

Own Id: OTP-8438

Harmless buffer overflow by one byte in asn1 and ram_file_drv.

Own Id: OTP-8451

Improved GC performance after BIF/NIF call when a lot of heap fragments was created. This will mainly benefit NIFs that return large compound terms.

Own Id: OTP-8240

Incompatible changes in the experimental NIF feature.

Read more in the documentation of erl_nif and erlang:load_nif/2

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8304

Cross compilation improvements and other build system improvements.

Most notable:

(Thanks to Henrik Riomar for suggestions and testing)

(Thanks to Winston Smith for the AVR32-Linux cross configuration and testing)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8323

NIF improvements:

Own Id: OTP-8335

Minor alignment adjustments of scheduler specific data.

Own Id: OTP-8341

The documentation is now possible to build in an open source environment after a number of bugs are fixed and some features are added in the documentation build process.

- The arity calculation is updated.

- The module prefix used in the function names for bif's are removed in the generated links so the links will look like "http://www.erlang.org/doc/man/erlang.html#append_element-2" instead of "http://www.erlang.org/doc/man/erlang.html#erlang:append_element-2".

- Enhanced the menu positioning in the html documentation when a new page is loaded.

- A number of corrections in the generation of man pages (thanks to Sergei Golovan)

- The legal notice is taken from the xml book file so OTP's build process can be used for non OTP applications.

Own Id: OTP-8343

There is new erlang:binary_to_term/2 BIF that takes an option list. The option safe can be used to prevent creation of resources that are not garbage collected (such as atoms). (Thanks to Jayson Vantuyl.)

Own Id: OTP-8367

The default settings for garbage collection of binaries has been adjusted to be less aggressive than in R13B03. It is now also possible configure the settings for binary GC. See the documentation for spawn_opt/2-5, erlang:system_info/1, erlang:system_flag/2, process_flag/2-3, erlang:trace/3, and the documenation for erl for the new command line options +hms and +hmbs.

Own Id: OTP-8370

A bug causing memory corruption in re:run() has been corrected. (Thanks to Yamashina Hio.)

Own Id: OTP-8375

The -Werror option for erlc and the compiler option warnings_as_errors will cause warnings to be treated as errors. (Thanks to Christopher Faulet.)

Own Id: OTP-8382

Explicit top directories in archive files are now optional.

For example, if an archive (app-vsn.ez) just contains an app-vsn/ebin/mod.beam file, the file info for the app-vsn and app-vsn/ebin directories are faked using the file info from the archive file as origin. The virtual direcories can also be listed. For short, the top directories are virtual if they does not exist.

Own Id: OTP-8387

An unnecessary lock operation in os:timestamp/0 has been eliminated, making it slightly more efficient. (Thanks to Jonas Falkevik and Tuncer Ayaz.)

Own Id: OTP-8390

There is a new +t emulator option for changing the maximum number of atoms. (Thanks to Julien Barbot.)

Own Id: OTP-8405

Fixed numerous compiler warnings generated by gcc 4.4.1 and tile-cc 2.0.1.78377 when compiling the runtime system.

Own Id: OTP-8412

configure learned the option --enable-m32-build to force the building of a 32-bit run-time on systems where the default C compiler generates 64-bit executables by default.

Own Id: OTP-8415

HiPE now works in the 64-bit emulator on Mac OS X. (Thanks to Geoff Cant.)

Own Id: OTP-8416

Improved handling of symbolic links to escripts

Own Id: OTP-8417

Add lock profiling tool.

The Lock profiling tool, lcnt, can make use of the internal lock statistics when the runtime system is built with this feature enabled.

This provides a mechanism to examine potential lock bottlenecks within the runtime itself.

- Add erts_debug:lock_counters({copy_save, bool()}). This option enables or disables statistics saving for destroyed processes and ets-tables. Enabling this might consume a lot of memory.

- Add id-numbering for lock classes which is otherwise undefined.

Own Id: OTP-8424

Removed spurious options to the emulator from escript.

Own Id: OTP-8427

Minor documentation fixes. Mainly anchor adjustments.

Own Id: OTP-8457

An insufficient stack allocation was made when reading CPU information on BSD operating systems. (Thanks Michael Turner and Akira Kitada)

Own Id: OTP-8207

A bug when supplying an argument without a dash directly after the program name when starting erlang could prevent distribution to start. This is now corrected.

Own Id: OTP-8209

A ticker process could potentially be blocked indefinitely trying to send a tick to a node not responding. If this happened, the connection would not be brought down as it should.

Own Id: OTP-8218

Using certain firewalls (i.e. MS IAS Client and certain versions of COMODO) could expose an undocumented behaviour in the Win32 socket interface causing the name resolution calls to hang infinitely. This is now worked around by adding possibilities for port programs under Windows to use overlapped I/O on their standard input/output file handles.

Own Id: OTP-8230

Fixed bug on ETS tables with write_concurrency. The emulator could crash when doing a select or match with a bound key without finding any object.

Own Id: OTP-8242

The information-request / information-response, and group-leader-change-request / group-leader-changed-response signal pairs described below did not always adhere to the signal order guarantees of Erlang's signal model in the runtime system with SMP support. These signals could for example sometimes pass exit signals.

The following BIFs behaviors can be modeled as if an asynchronous information-request signal is sent to Pid. If Pid is alive, it responds with an asynchronous information-response signal; otherwise, the runtime system responds with a no-such-process signal. When the response is received, the caller transforms it into the result of the BIF.

When Pid resides on the same node as the caller of group_leader(GroupLeader, Pid), the group_leader/2 BIFs behavior can be modeled as if an asynchronous group-leader-change-request signal is sent to Pid. If Pid is alive, it responds with an asynchronous group-leader-changed-response signal; otherwise, the runtime system responds with a no-such-process signal. When the response is received, the caller transforms it into the result of the BIF. The distributed case which only consists of an asynchronous group-leader-change-request signal and no response is not effected.

Own Id: OTP-8245

Errors in the system_profile documentation has been corrected.

Own Id: OTP-8257

Low watermark socket option modified high watermark instead of low watermark in the inet_driver. (Thanks to Feng Yu and Tuncer Ayaz)

Own Id: OTP-8279

A race condition could cause the runtime system with SMP support to end up in a completely unresponsive state.

Own Id: OTP-8297

The use of pthread_cond_timedwait() have been completely removed from the runtime system. This since its behavior is unpredictable when the system clock is suddenly changed. The previous use of it was harmless.

Own Id: OTP-8193

The documentation is now built with open source tools (xsltproc and fop) that exists on most platforms. One visible change is that the frames are removed.

Own Id: OTP-8201

A new garbage collecting strategy for binaries which is more aggressive than the previous implementation. Binaries now has a virtual binary heap tied to each process. When binaries are created or received to a process it will check if the heap limit has been reached and if a reclaim should be done. This imitates the behavior of ordinary Erlang terms. The virtual heaps are grown and shrunk like ordinary heaps. This will lessen the memory footprint of binaries in a system.

Own Id: OTP-8202

The ErlDrvTermData term types used by driver_output_term() and driver_send_term() have been extended with the term types ERL_DRV_INT64, and ERL_DRV_UINT64 for passing 64-bit integers. Also the 64-bit integer data types ErlDrvSInt64 and ErlDrvUInt64 have been introduced.

For more information see the erl_driver(3) documentation.

Own Id: OTP-8205

[escript] The restriction that the first line in escripts must begin with #! has been removed.

[escript] Some command line options to the escript executable has now been documented. For example you can run an escript in the debugger by just adding a command line option.

[escript] The documentation of the escript header syntax has been clarified. For example the header is optional. This means that it is possible to directly "execute" .erl, .beam and.zip files.

Own Id: OTP-8215

The instruction for building OTP on Windows was outdated and incomplete, the document is updated.

Also the otp_build script required windows drives to show up in Cygwin using the /cygdrive prefix. That requirement is now removed.

Own Id: OTP-8219

A module can have native implemented functions (NIFs) that are dynamically loaded by calling erlang:load_nif/2. This is an experimental feature that is not yet intended for production systems. It is released with intention to get some early feedback on the interfaces before they are carved in stone.

Own Id: OTP-8220

The float/1 BIF would always force a garbage collection. The BIFs size/1, byte_size/1, bit_size/1, abs/1, and round/1 would force a garbage-collection if the result was not a sufficiently small integer.

Own Id: OTP-8221

The erlang:port_command/3 BIF has been added. erlang:port_command/3 is currently not auto imported, but it is planned to be auto imported in OTP R14. For more information see the erlang(3) documentation.

Own Id: OTP-8225

'configure --enable-darwin-64bit' would fail if Snow Leopard had been booted with the 64-bit kernel. (Thanks to Ryan M. Graham.)

Own Id: OTP-8236

On Windows, open_port({spawn,Command},Opts) could not run executables with spaces in the path or filename, regardless of quoting efforts. While open_port({spawn_executable,Exec},Opts) can run any executable, it was still impossible to use 'spawn' to do the same thing. This is now corrected.

Own Id: OTP-8055

The scheduler bind type processor_spread spread schedulers too much on large NUMA systems.

The new scheduler bind type spread spreads schedulers as much as possible, and behaves as processor_spread previously did. For more information see the documentation of the +sbt command line argument in the erl(1) documentation, and the documentation of erlang:system_flag(scheduler_bind_type, SchedulerBindType).

Own Id: OTP-8063

Automatically detected CPU topology on Linux system could erroneously contain logical processors with -1 as identifiers. This happened when sysconf(_SC_NPROCESSORS_CONF) returned a value larger than the amount of logical processors found.

Own Id: OTP-8064

When the minimal term [] (end of list) was sent as the complete message to a process on another node, and received there, it could not be decoded. This bug is now corrected. Fortunately [] is uncommon as the complete message in real applications but it is a serious bug anyway.

Own Id: OTP-8092

A bug when the floating point exception pointer was not initialized has been corrected. It manifested itself on CentOS 5.1 sometimes when a floating point value was sent to a remote node. Bug reported and patch suggested by David Reiss, confirmed by Mikael Pettersson.

Some build problems on IRIX was also corrected. Problem reported by Patrick Baggett, patch by Mikael Pettersson.

Own Id: OTP-8095

A terminating process could erroneously unregister a name for another process. This could occur under the following conditions: The name of the terminating process was unregistered and then registered for another process simultaneously as the process that first had the name was terminating.

Own Id: OTP-8099 Aux Id: seq11344

Running erlc in a very deep directory (with a path length of more 256 or more characters) would cause the emulator to crash in a call to list_to_atom/1. (Thanks to Chris Newcombe.)

Own Id: OTP-8124

A deadlock of the runtime system could occur when unregistering the name of a port.

Own Id: OTP-8145

Makefile.in has been updated to use the LDFLAGS environment variable (if set). (Thanks to Davide Pesavento.)

Own Id: OTP-8157

The pthread rwlock implemention on Linux could cause starvation of writers. We, therefore, now use our own rwlock implementation on Linux.

Own Id: OTP-8158

Open source Erlang builds are updated to work well on Snow Leopard (MacOS X 10.6)

Own Id: OTP-8168

A call to erlang:system_info(schedulers_online) could end up in an infinite loop. This happened if the amount of schedulers was larger than one, the amount of schedulers online was one, and someone was blocking multi-scheduling.

Own Id: OTP-8169

An error in erlang:system_profile/2 could cause timestamped messages to arrive out of order in the SMP case. This has now been fixed.

Own Id: OTP-8171

binary_to_atom/2 and binary_to_existing_atom/2 would leak memory if the binary contained unaligned data.

Own Id: OTP-8192

The async thread pool in the runtime system without SMP support used a memory allocator that was not thread safe for async jobs.

Own Id: OTP-8194

Processor internal NUMA nodes are now supported in the ERTS internal CPU topology representation. For more information see the documentation of the +sct command line argument in the erl(1) documentation, and the documentation of erlang:system_info(cpu_topology).

Own Id: OTP-8041

Documentation for ets improved about concurrency.

Own Id: OTP-8050

Emulator flags in an escript were earlier inherited to emulators started from from the emulator running the escript. For example when an escript invoked os:cmd("erl"), the new emulator were given erroneous emulator flags. This bug has now been fixed

Escript filenames may now contain dots.

Own Id: OTP-8060

Made some BIFs non-recursive (relational operators,hash and phash) to limit internal stack usage.

Own Id: OTP-8065

Fixed Windows specific bug in erl_prim_loader. Now it handles the root directory (e.g. c:/) better. This bug affected the directory listing in the debugger.

Own Id: OTP-8080

A TCP socket with option {packet,4} could crash the emulator if it received a packet header with a very large size value (>2Gb). The same bug caused erlang:decode_packet/3 to return faulty values. (Thanks to Georgos Seganos.)

Own Id: OTP-8102

The maximum size of the export table has been raised from 65536 to 524288 entries.

Own Id: OTP-8104 Aux Id: seq11345

The file module has now a read_line/1 function similar to the io:get_line/2, but with byte oriented semantics. The function file:read_line/1 works for raw files as well, but for good performance it is recommended to use it together with the 'read_ahead' option for raw file access.

Own Id: OTP-8108

Fixed bug causing emulator crash when reading a term in external format containing a corrupt list with a negative length.

Own Id: OTP-8117

New emulator flag +sss, to set stack size of scheduler threads.

Own Id: OTP-8119

The Windows utility Erlsrv, run in interactive mode now accepts options for registering internal service name and description field of Windows registry database.

Own Id: OTP-8132

erlang:demonitor(Mon, [flush]) has been optimized. Previously it always searched the message queue of the caller for a 'DOWN' message. Current implementation only search the message queue when necessary. It is quite common that the search is not necessary.

A new option info has been added to erlang:demonitor/2. For more information see the erlang(3) documentation.

Own Id: OTP-8143

I/O tasks could unnecessarily be rescheduled. This was harmless, but not useful work.

Own Id: OTP-8148

Minor improvements of erlang:memory/[1,2].

Own Id: OTP-8152

New configuration option to enable use of shared zlib.

Own Id: OTP-8155

Fixed smp bug in ETS that could cause emulator crash when table with more than 1000 objects accessed by several processes, including calls to variants of select or match combined with concurrent object deletion.

Own Id: OTP-8166 Aux Id: seq11392

The code path interpretation is now more relaxed. The flag -code_path_choice now defaults to relaxed instead of strict. See the documentation of code and init for more info.

Own Id: OTP-8170

Load balancing of run queues and check for I/O are triggered more often than before in situations where processes are scheduled often but are doing very little work each time they execute.

Own Id: OTP-8172

Call tracing binary comprehensions would cause an emulator crash. (Thanks to Paul Mineiro.)

Own Id: OTP-8179

binary_to_term/1 would crash the emulator instead of generating a badarg exception when given certain invalid terms. (Thanks to Scott Lystig Fritchie.)

Own Id: OTP-8180

Crash dumps should now cause less problems for the crashdump_viewer application. (For processes where arity was non-zero, the arguments are now longer printed - they used to be printed in a format that was not parseable.)

Own Id: OTP-7472 Aux Id: seq11019, 11292

Processes could potentially get stuck on an offline scheduler.

Own Id: OTP-7990

binary_to_atom/2 and binary_to_existing_atom/2 could leak memory if they caused a badarg exception.

Own Id: OTP-7997

A process could under very rare circumstances erroneously be resumed.

Own Id: OTP-8000

Load balancing between schedulers could under rare circumstances cause an emulator crash.

Own Id: OTP-8008

erlang:memory(processes_used) always returned 0 instead of the correct value. (Thanks to Geoff Cant)

Own Id: OTP-8022

Major improvements of the Erlang distribution for Erlang runtime systems with SMP support. Previously distribution port locks were heavily contended, and all encoding and decoding for a specific distribution channel had to be done in sequence. Lock contention due to the distribution is now negligible and both encoding and decoding of Erlang messages can be done in parallel.

The old atom cache protocol used by the Erlang distribution has been dropped since it effectively prevented all parallel encoding and decoding of messages passed over the same distribution channel.

A new atom cache protocol has been introduced which isolates atom cache accesses and makes parallel encoding and decoding of messages passed over the same distribution channel possible. The new atom cache protocol also use an atom cache size 8 times larger than before. The new atom cache protocol is documented in the ERTS users guide.

Erlang messages received via the distribution are now decoded by the receiving Erlang processes without holding any distribution channel specific locks. Erlang messages and signals sent over the distribution are as before encoded by the sending Erlang process, but now without holding any distribution channel specific locks during the encoding. That is, both encoding and decoding can be and are done in parallel regardless of distribution channel used.

The part that cannot be parallelized is the atom cache updates. Atom cache updates are therefore now scheduled on the distribution port. Since it is only one entity per distribution channel doing this work there is no lock contention due to the atom cache updates.

The new runtime system does not understand the old atom cache protocol. New and old runtime systems can however still communicate, but no atom cache will be used.

Own Id: OTP-7774

Fixed a bug that caused error logging from driver_select sometimes with additional symptoms such as failing IP communications or even an emulator crash.

Own Id: OTP-7898 Aux Id: seq11304

Improved SMP concurrency for ETS tables. Several mutating operations can now be performed truly concurrent on different records of the same table. To support this, the table has to be created with option write_concurrency, as it is achieved at the expense of some execution and memory overhead. ets:select and select_count has also been improved for all tables to not acquire exclusive table lock during the iteration.

Own Id: OTP-7922

erl (that is erl.exe and dyn_erl) and erlexec has been made more dynamic so no hard coded paths needs to added at installation time to erl (that is erl.ini and erl). Reltool will make use of this in a future release.

Own Id: OTP-7952

Added functionality to get higher resolution timestamp from system. The erlang:now function returns a timestamp that's not always consistent with the actual operating system time (due to resilience against large time changes in the operating system). The function os:timestamp/0 is added to get a similar timestamp as the one being returned by erlang:now, but untouched by Erlangs time correcting and smoothing algorithms. The timestamp returned by os:timestamp is always consistent with the operating systems view of time, like the calendar functions for getting wall clock time, but with higher resolution. Example of usage can be found in the os manual page.

Own Id: OTP-7971

Two new options are added to open_port - spawn_executable which runs external executables in a controlled way, and spawn_driver which only opens port to loaded Erlang drivers. See the erlang manual page for details.

Own Id: OTP-7995

New functionality in ETS to transfer the ownership of a table. A table can either change owner be declaring an "heir", another process that will inherit the table if the owner terminates. A table can also change owner by calling a new function ets:give_away.

Own Id: OTP-8006

Updates to Tilera build environment.

Own Id: OTP-8009

A stack trace was unnecessarily saved during process termination.

Own Id: OTP-8014

User defined CPU topology and scheduler bind type can now be set from the command line when starting an emulator. For more information see the documentation of the +sct, and the +sbt emulator flags in the erl(1) documentation.

The CPU topologies returned from erlang:system_info/1 and erlang:system_flag/2 now always contain the processor level, also when not strictly necessary.

Own Id: OTP-8030

Various fixes in ETS: ets:first could return a deleted key in a fixated table. ets:lookup could return objects out of order if a deleted object was re-inserted into a fixed bag. ets:delete_object could fail to delete duplicate objects in a duplicate_bag.

Own Id: OTP-8040

Fixed a bug on Windows that could make gen_tcp:send hang trying to send an iolist of more than 16 binaries.

Own Id: OTP-7816

The runtime system could under rare circumstances crash during load balancing.

Own Id: OTP-7908 Aux Id: otp-7500

run_erl uses fallback if Unix98 pseudo-terminal is not present on host.

Own Id: OTP-7916 Aux Id: seq11249

A message buffer memory leak in the runtime system without smp support has been fixed.

Own Id: OTP-7941

Attempting to append a binary of 16Mb or greater to another binary using the bit syntax would cause a system_limit exception. There was also several cases when constructing binaries when a badarg exception was generated when it should have been system_limit.

Own Id: OTP-7942

The runtime system with SMP support failed to terminate the caller of link(RemotePid) properly, if RemotePid was the pid of a process on an unreachable node. The calling process was in this case marked as exiting, but never terminated.

Own Id: OTP-7946

Rudimentary support for cross compiling is added to the source release. The support is still in its infancy and has only been used to cross compile on Linux for a different cpu architecture and a different Linux version, but should be extendible to support other platforms as well. The cross configuration files with examples are placed in $ERL_TOP/xcomp/. View README.xcomp and run $ERL_TOP/otp_build -help for further information.

Own Id: OTP-7854

The escape sequence \{ which was given a new interpretation in R13A has retained its old meaning (the ASCII code for {), which means that codes greater than 255 have to be stated using hexadecimal characters (for example, \x{AAA}). The escape sequence \xH where H is a hexadecimal character followed by something else but a hexadecimal character is no longer valid (incompatibility with R13A). Character codes less than 256 can be stated using two hexadecimal characters (for example, \x0D).

Own Id: OTP-7891 Aux Id: OTP-7855

The term_to_binary/1 BIF used to be implemented with recursive C code, which could cause the Erlang emulator to terminate because of a stack overflow.

Also fixed some minor issues in term_to_binary/1 and binary_to_term/1 pointed out by Matthew Dempsky.

Own Id: OTP-7894

Several glitches and performance issues in the Unicode and I/O-system implementation of R13A have been corrected.

Own Id: OTP-7896 Aux Id: OTP-7648 OTP-7887

Minor documentation improvements of the scheduler_bind_type argument of erlang:system_flag/2, and the scheduler_bind_type, and the scheduler_bindings arguments of erlang:system_info/1.

Own Id: OTP-7901 Aux Id: OTP-7777

There is a new BIF erlang:make_tuple/3.

Own Id: OTP-7913

OpenSource:

FreeBSD leap-seconds are handled according to patch submitted by OpenSource user Kenji Rikitake. No test case covers this functionality (unsupported platform).

Own Id: OTP-7609

A corrected bug in ETS for bag and duplicate_bag. A delete/2 or lookup_element/3 could miss objects in a fixed table if one or more objects with the same key had already been deleted.

Own Id: OTP-7665

A new driver call-back stop_select is introduced to allow drivers to de-select and then close a file descriptor in a safe way in a SMP emulator. The old way was not strictly according to posix standard and could in some rare cases lead to unexpected behavior. A new flag ERL_DRV_USE can be passed to driver_select() to tell it that the descriptor should be closed. stop_select is then called when it is safe to do so. Old drivers will however still work as before.

Own Id: OTP-7670

A bug fixed for TCP sockets with option {packet,http}. An HTTP request with an absolute URI was returned with a corrupt path string. This bug did only exist in R12B-4 and R12B-5.

Own Id: OTP-7682 Aux Id: OTP-7647

run_erl did in some cases fail to extract control sequences from to_erl (like: winsize=X,Y) and did instead send them to be interpreted by the erlang shell.

Own Id: OTP-7688

A bug in the installer on Windows not updating file associations properly is now corrected.

Own Id: OTP-7746

More space than necessary could be allocated in binaries when appending to a binary (also in a binary comprehension) and the data appended did not consist of wholes bytes (e.g. 13 bits).

Own Id: OTP-7747

The gen_sctp option sctp_peer_addr_params, #sctp_paddrparams{address={IP,Port} was erroneously decoded in the inet driver. This bug has now been corrected.

Own Id: OTP-7755

Outstanding async driver jobs leaked memory if the issuing port died before the async jobs completed.

Own Id: OTP-7784

A bug in the dynamic library loading affecting, among others, OpenSolaris is now corrected. (Thanks to Paul Fisher.)

Own Id: OTP-7796

run_erl compile errors fixed for FreeBSD

Own Id: OTP-7817

A bug in the inet driver for SCTP on Solaris showing for e.g gen_sctp:abort/1 and gen_sctp:eof/1 has been corrected. Patch suggestion by Simon Cornish.

Own Id: OTP-7866

The order of objects visited in select for ordered_set is now documented.

Own Id: OTP-7339

The runtime system with SMP support now uses multiple, scheduler specific run queues, instead of one globally shared run queue.

The lock protecting the shared run queue was heavily contended, and the shared run queue also caused Erlang processes to randomly migrate between schedulers with negative cache effects as a result.

With the current scheduler specific run queue solution, lock contention due to run queue protection has been reduced, and Erlang processes are only migrated when needed to balance the load between the schedulers. The reduced amount of migration also reduce lock contention on locks protecting the scheduler specific instances of the erts internal memory allocators.

The scheduler specific run queues are also a necessity for a lot of future planned NUMA (Non-Uniform Memory Access) specific optimizations.

Own Id: OTP-7500

Support for Unicode is implemented as described in EEP10. Formatting and reading of unicode data both from terminals and files is supported by the io and io_lib modules. Files can be opened in modes with automatic translation to and from different unicode formats. The module 'unicode' contains functions for conversion between external and internal unicode formats and the re module has support for unicode data. There is also language syntax for specifying string and character data beyond the ISO-latin-1 range.

The interactive shell will support input and output of unicode characters when the terminal and operating system supports it.

Please see the EEP and the io/io_lib manual pages as well as the stdlib users guide for details.

I/O-protocol incompatibilities:

The io_protocol between io_Server and client is updated to handle protocol data in unicode formats. The updated protocol is now documented. The specification resides in the stdlib users manual, which is a new part of the manual.

io module incompatibilities:

The io:put_chars, io:get_chars and io:get_line all handle and return unicode data. In the case where binaries can be provided (as to io:put_chars), they shall be encoded in UTF-8. When binaries are returned (as by io:get_line/get_chars when the io_server is set in binary mode) the returned data is also always encoded as UTF-8. The file module however still returns byte-oriented data, why file:read can be used instead of io:get_chars to read binary data in ISO-latin-1.

io_lib module incompatibilities:

io_lib:format can, given new format directives (i.e "~ts" and "~tc"), return lists containing integers larger than 255.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7648 Aux Id: OTP-7580 OTP-7514 OTP-7494 OTP-7443 OTP-7181 EEP10 EEP11

The format of the string returned by erlang:system_info(system_version) (as well as the first message when Erlang is started) has changed. The string now contains the both the OTP version number as well as the erts version number.

Own Id: OTP-7649

Message passing has been further optimized for parallel execution. Serial message passing is slightly more expensive than before, but parallel send to a common receiver is much cheaper.

Own Id: OTP-7659

Lock contention on the atom table lock when decoding Erlang terms on the external format has been drastically reduced.

Own Id: OTP-7660

The undocumented, unsupported, and deprecated guard BIF is_constant/1 has been removed.

*** INCOMPATIBILITY with R12B ***

Own Id: OTP-7673

The Erlang process lock implementation has been improved by Mat Hostetter at Tilera Corporation.

Own Id: OTP-7692

A {nodedown, Node} message passed by the net_kernel:monitor_nodes/X functionality is now guaranteed to be sent after Node has been removed from the result returned by erlang:nodes/Y.

Own Id: OTP-7725

The short-circuit operators andalso and orelse no longer guarantees that their second argument is either true or false. As a consequence, andalso/orelse are now tail-recursive.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7748

A new BIF, lists:keyfind/3, has been added. It works like lists:keysearch/3 except that it does not wrap the returned tuple in a value tuple in case of success. (Thanks to James Hague for suggesting this function.)

Own Id: OTP-7752

Optimization for drivers by creating small binaries direct on process heap.

Own Id: OTP-7762

I bsl N could cause the Erlang virtual machine to run of memory instead generating a system_limit if N was absurdly huge. (Thanks to Daniel Hedlund.)

There would always be a garbage collection when evaluating I bsl N or I bsr N if I was a bignum.

If I is an integer and N a bignum, I bsl N will now cause the correct system_limit exception instead of bad_arith as in earlier releases.

If I is an integer and N a bignum, I bsr N will return either 0 or -1 depending on the sign of I instead of causing a bad_arith exception as in earlier releases.

Own Id: OTP-7764

Scheduler threads can now be bound to logical processors on newer Linux and Solaris systems. More systems will be supported in the future.

In some cases performance has increased drastically when binding schedulers. Schedulers are not bound by default, though. This since it might cause a performance degradation if multiple programs have bound to processors, e.g. multiple Erlang runtime systems. For more information see the documentation of erlang:system_flag/2.

In order to bind scheduler threads the CPU topology need to be known. On some newer Linux and Solaris systems the runtime system automatically detects the CPU topology. If the emulator isn't able to automatically detect the CPU topology, the CPU topology can be defined. For more information see the documentation of erlang:system_flag/2.

Own Id: OTP-7777

The compiler will refuse to a compile file where the module name in the file differs from the output file name.

When compiling using erlc, the current working directory will no be included in the code path (unless explicitly added using "-pa .").

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7793

The BIFs atom_to_binary/2, binary_to_atom/2, and binary_to_existing_atom/2 have been added.

Own Id: OTP-7804

The amount of schedulers online can now be changed during operation. The amount of schedulers online defaults to the same amount as available logical processors. For more information see the documentation of erlang:system_flag/2 and erl.

Own Id: OTP-7811

The deprecated functions erlang:fault/1, erlang:fault/2, and file:rawopen/2 have been removed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7812

Erts internal dynamically allocated process and port specific data for rarely used data. This is used to reduce memory usage of processes and ports that do not use specific functionality. More functionality will be moved to process and port specific data in future releases.

Own Id: OTP-7818

New packet type http_bin for gen_tcp sockets and erlang:decode_packet. It works like http except that strings are returned as binaries instead of lists.

Own Id: OTP-7821

The obsolete wd_keeper program for embedded Solaris systems has been removed.

Own Id: OTP-7822

Nodes belonging to different independent clusters can now co-exist on the same host with the help of a new environment variable setting ERL_EPMD_PORT.

Own Id: OTP-7826

There are new functions erlang:min/2 and erlang:max/2 to calculate the minimum and maximum of two terms, respectively. Note that the functions are not auto-imported, so they need to be imported explicitly or the erlang prefix must be used when calling them.

Own Id: OTP-7841

The copyright notices have been updated.

Own Id: OTP-7851

Enhanced build environment for cross compilation to Tilera Tile architecture.

Support for native ethread atomics on Tilera Tile64/TilePro (Thanks to Tilera Corporation).

Own Id: OTP-7852

The escape sequences \x and \{ have been assigned new interpretations (they used to return the ASCII code for x and { respectively). One or more octal characters inside curly brackets after a leading backslash is from now on an alternative to the existing syntax \NNN, but can also be used for codes greater than 255. In a similar fashion, one or more hexadecimal characters can be put inside curly brackets after a leading \x. Furthermore, the escape sequences \xH and \xHH, where N is a hexadecimal character, can be used for codes less than 256.

NOTE: These new escape sequences are still considered experimental and may be changed in the R13B release.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7855

The PCRE library's exported function names are now prefixed with erts_ in the erlang emulator to avoid clashes with dynamically loaded drivers.

Own Id: OTP-7861

A runtime system with SMP support will now be built by default on most platforms if a usable posix thread library or native windows threads are found.

For more information see the top README file.

Own Id: OTP-7872

A corrected bug in ETS for bag and duplicate_bag. A delete/2 or lookup_element/3 could miss objects in a fixed table if one or more objects with the same key had already been deleted.

Own Id: OTP-7665

A bug fixed for TCP sockets with option {packet,http}. An HTTP request with an absolute URI was returned with a corrupt path string. This bug did only exist in R12B-4 and R12B-5.

Own Id: OTP-7682 Aux Id: OTP-7647

Calling gen_tcp:send() from several processes on socket with option send_timeout could lead to much longer timeout than specified. The solution is a new socket option {send_timeout_close,true} that will do automatic close on timeout. Subsequent calls to send will then immediately fail due to the closed connection.

Own Id: OTP-7731 Aux Id: seq11161

A process being garbage collected via the garbage_collect/1 BIF or the check_process_code/2 BIF didn't handle message receive and resume correctly during the garbage collect. When this occurred, the process returned to the state it had before the garbage collect instead of entering the new state.

Own Id: OTP-7738

A bug in inet_drv concerning gen_tcp:connect has been corrected. A connect towards a non-open port through open firewalls could sometimes erroneously be successful. Any subsequent operation would fail, though.

Own Id: OTP-6542

Floating point arithmetics in drivers could cause a runtime system crash and/or unexpected results on runtime systems with floating point exceptions enabled. Floating point exceptions are disabled unless explicitly enabled or if hipe is enabled.

Own Id: OTP-7237

A bug when many sockets got signalled simultaneously causing the emulator to panic with the message "Inconsistent, why isnt io reported?" is now corrected.

Own Id: OTP-7420

Starting erl with option "-detached" now disconnects correctly from terminal session on Unix.

Own Id: OTP-7461

Mended gdb etp-commands for ETS access.

Own Id: OTP-7538

erlang:decode_packet/3 allows white space between HTTP header tag and colon according to RFC2616.

Own Id: OTP-7543

An emulator compiled for SCTP now starts even if the dynamic libraries are not present. The SCTP driver is then of course not loaded.

Own Id: OTP-7551

To build on Mac OS X, 10.3.0 or later is now required because of fixes for two problems:

There would be a resource leak when erl_ddl attempted to unload a driver. This problem has been corrected by using dlopen() (which works on all modern Unix-like platforms) instead of the Mac OS X specific API calls.

Signal handling in the run-time system for HiPE has been updated to work on later versions of Mac OS X than 10.2.x. Therefore, --enable-hipe now works on Mac OS X with Intel CPUs.

Thanks to Geoff Cant for the patches.

Own Id: OTP-7562

Corrected some information about the protocol between EPMD and Erlang nodes. (Thanks to Michael Regen.)

Own Id: OTP-7594