ERTS Release Notes

Erts 5.9.3

Fixed Bugs and Malfunctions

Fix linking in OpenBSD. (Thanks to Matthew Dempsky)

Own Id: OTP-10395

Fix bug causing fallback atomics to be used even though healthy gcc atomics or libatomic_ops was detected.

Own Id: OTP-10418

Ensure 'erl_crash.dump' when asked for it. This will change erl_crash.dump behaviour.

* Not setting ERL_CRASH_DUMP_SECONDS will now terminate beam immediately on a crash without writing a crash dump file.

* Setting ERL_CRASH_DUMP_SECONDS to 0 will also terminate beam immediately on a crash without writing a crash dump file, i.e. same as not setting ERL_CRASH_DUMP_SECONDS environment variable.

* Setting ERL_CRASH_DUMP_SECONDS to a negative value will let the beam wait indefinitely on the crash dump file being written.

* Setting ERL_CRASH_DUMP_SECONDS to a positive value will let the beam wait that many seconds on the crash dump file being written.

A positive value will set an alarm/timeout for restart both in beam and in heart if heart is running.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10422 Aux Id: kunagi-250 [161]

Fix bug where MSVRT100.dll was not included in the windows installer.

Own Id: OTP-10481

In the expression <<Bin/binary,...>>, if Bin was a bitstring with a size not a multiple of 8, either no exception was generated or an incorrect exception was generated. (Thanks to Adam Rutkowski for reporting this bug.)

Own Id: OTP-10524

The runtime system could crash while scheduling a port task. The port task was scheduled either due to an external I/O event being triggered, a driver timeout being triggered, or data being sent over a distribution channel.

Own Id: OTP-10556

erlang:memory(ets) erroneously included the size of each ETS-table main structure twice.

Own Id: OTP-10558

Fix compile error in generated file hipe_amd64_bifs.S for Solaris.

Own Id: OTP-10577

A faulty spec for process_info/2 could cause false dialyzer warnings. The spec is corrected.

Own Id: OTP-10584

In very rare cases, the VM could crash if a garbage collector was called while executing an appending bit syntax instruction. The symptom was a core when reallocating memory in the function erts_bs_append. The garbage collector bug is now corrected.

Own Id: OTP-10590

Improvements and New Features

Improve support for building and testing in embedded ppc environments.

Own Id: OTP-10265 Aux Id: kunagi-159 [daf97f67-5724-4812-a5b6-7e86990133d2-1]

Due to a race condition on Windows, sometimes when printing to standard output and then immediately terminating erlang all data would not be printed. The emulator now waits for all data to be printed before exiting.

Own Id: OTP-10325 Aux Id: kunagi-166 [dd72d0e2-3e76-4a51-8b56-7564e24eecae]

The frequency with which sleeping schedulers are woken due to outstanding memory deallocation jobs has been reduced.

Own Id: OTP-10476 Aux Id: OTP-10162

Clearer warnings about the dangers of misuse of native functions and drivers have been added to the documentation.

Own Id: OTP-10557

Erts 5.9.2

Fixed Bugs and Malfunctions

Fix erl_prim_loader errors in handling of primary archive. The following errors have been corrected:

If primary archive was named "xxx", then a file in the same directory named "xxxyyy" would be interpreted as a file named "yyy" inside the archive. erl_prim_loader did not correctly create and normalize absolute paths for primary archive and files inside it, so unless given with exact same path files inside the archive would not be found. E.g. if escript was started as /full/path/to/xxx then "./xxx/file" would not be found since erl_prim_loader would try to match /full/path/to/xxx with /full/path/to/./xxx. Same problem with ../. Depending on how the primary archive was built, erl_prim_loader:list_dir/1 would sometimes return an empty string inside the file list. This was a virtual element representing the top directory of the archive. This has been removed.

Thanks to Tuncer Ayaz and Shunichi Shinohara for reporting and co-authoring corrections.

Own Id: OTP-10071

Fix: Add port-I/O statistics for active once and true and not only active false.

Own Id: OTP-10073

The 64-bit windows installer did not look in the right directories for 64-bit version of Microsoft Visual C++ 2010 Redistibutable Package and hence took the wrong decision about having to install the redistributable package if the 32-bit version was installed but not the 64-bit and vice versa. This bug has now been fixed Furthermore the sub-installer for the redistributable package is now run in silent mode if the erlang installer is.

Own Id: OTP-10096

epmd would fail to start automatically when starting a distributed erlang node installed in a location with a whitespace in the path.

Own Id: OTP-10106

A more or less harmless bug that sometimes caused memory deallocations to be delayed longer than intended has been fixed.

Own Id: OTP-10116

Fix bug causing emulator crash when running HiPE on ARM. Bug has existed since R15B.

Own Id: OTP-10137

A bug regarding spaces in C function prototypes has been fixed. (Thanks to Richard O'Keefe.)

Own Id: OTP-10138

Corrected dtrace pid length in message related probes. (Thanks to Zheng Siyao)

Own Id: OTP-10142

Correct formating in exit error messages

Ensure displayed sizes are not negative. (Thanks to Michael Santos)

Own Id: OTP-10148

fix escript/primary archive reloading

If the mtime of an escript/primary archive file changes after being added to the code path, correctly reload the archive and update the cache. (Thanks to Tuncer Ayaz)

Own Id: OTP-10151

Doc fix: link from erlang:now/0 to os:timestamp/0

Sometimes os:timestamp/0 is more appropriate than erlang:now/0. The documentation for the former has a link to the latter; this patch adds a link in the other direction to make os:timestamp/0 more visible. Thanks to Magnus Henoch

Own Id: OTP-10180

The caret in the werl window (on Windows) could appear at the wrong place after regaining focus. This is now corrected.

Own Id: OTP-10181

Fix bug that in some cases could cause corrupted binaries in ETS tables with compressed option.

Own Id: OTP-10182

Fix use of "clever" mktime

Commit 1eef765 introduced regression (conditional _always_ evaluates to true) in which erlang:localtime_to_universaltime/2 stopped working on systems configured with timezone without DST (i.e. UTC) on *BSD platforms: 1> erlang:localtime_to_universaltime({{2012,1,1},{0,0,0}}, true). ** exception error: bad argument Thanks to Piotr Sikora

Own Id: OTP-10187

Relocate bodies of DTrace probes to the statically-linked VM.

Due to various operating systems (in both the DTrace and SystemTap worlds) not fully supporting DTrace probes (or SystemTap-compatibility mode probes) in shared libraries, we relocate those probes to the statically-linked virtual machine. This could be seen as pollution of the pristine VM by a (yet) experimental feature. However:

1. This code can be eliminated completely by the C preprocessor. 2. Leaving the probes in the dyntrace NIF shared library simply does not work correctly on too many platforms. *Many* thanks to Macneil Shonle at Basho for assisting when my RSI-injured fingers gave out. (note: Solaris 10 and FreeBSD 9.0-RELEASE can take a long time to compile)

Own Id: OTP-10189

Fix bug in ets:test_ms/2 that could cause emulator crash when using '$_' in match spec.

Own Id: OTP-10190

Supplying a filename longer than the operating system MAX_PATH to file:read_link/1 would cause a crash (Segemntation fault/Critical Error) on all platforms. This is now corrected.

Own Id: OTP-10200

If Perl was configured to interpret files as being encoded in UTF-8, the build would crash in make_preload. (Thanks to Aaron Harnly for noticing this issue.)

Own Id: OTP-10201

Fix the erlc -MP flag

Because of a copy-and-paste error in erlc.c, the -MP flag had the same effect as -MG. As a workaround, you had to pass +makedep_phony to enable the MP option. This patch makes -MP work as intended.

Own Id: OTP-10211

Allow mixed IPv4 and IPv6 addresses to sctp_bindx

Also allow mixed address families to bind, since the first address on a multihomed sctp socket must be bound with bind, while the rest are to be bound using sctp_bindx. At least Linux supports adding address of mixing families. Make inet_set_faddress function available also when HAVE_SCTP is not defined, since we use it to find an address for bind to be able to mix ipv4 and ipv6 addresses. Thanks to Tomas Abrahamsson

Own Id: OTP-10217

Fix support for leap seconds-aware timezones

erlang:universaltime_to_localtime is leap seconds-aware (since 2008), however erlang:localtime_to_universaltime is not, which gives surprising results on systems configured with leap seconds-aware timezones: 1> erlang:universaltime_to_localtime({{2012,1,1},{0,0,0}}). {{2012,1,1},{0,0,0}} 2> erlang:localtime_to_universaltime({{2012,1,1},{0,0,0}}). {{2012,1,1},{0,0,24}} and completely breaks calendar:local_time_to_universal_time_dst: 3> calendar:local_time_to_universal_time_dst({{2011,1,1},{0,0,0}}). [] Thanks to Piotr Sikora

Own Id: OTP-10227

erlsrv: gracefully stop emulator on Windows shutdown

Windows will send the SERVICE_CONTROL_SHUTDOWN event to the service control handler when shutting down the system. Instead of ignoring the event, erlsrv will now invoke the stop action. Likewise, the Erlang emulator (and it's po drivers) must not quit upon reception of the CTRL_SHUTDOWN_EVENT event in th console control handler. Thanks to Jan Kloetzke

Own Id: OTP-10228

Fix dtrace bug in file rename operation.

Own Id: OTP-10234

Fix bug in memory management of driver port data locks (PDL). In some cases PDLs could be deallocated before ready_async or async_free callback was called.

Own Id: OTP-10249

Improvements and New Features

Add port and suspend options to lock-counter profiling. (Thanks to Rick Reed)

Own Id: OTP-10051

Latency when using the active_once option in gen_tcp communication is reduced.

Own Id: OTP-10055 Aux Id: sto139

Remove bit8 option support from inet

Own Id: OTP-10056

The OS Pid of a port program is now available by calling erlang:port_info(Port,os_pid), Thanks to Matthias Lang for the original patch.

Own Id: OTP-10057

Fix openpty usage in run_erl.

Reopening a slave file descriptor which was closed earlier could lead to a misbehaving connection. This has now been remedied.

Own Id: OTP-10076

Remove all code, documentation, options and diagnostic functions which were related to the experimental hybrid heap implementation.

Own Id: OTP-10105

Optimizations of memory deallocations.

Own Id: OTP-10162 Aux Id: OTP-7775

Optimization of process locking.

Own Id: OTP-10163

Added a xcomp example file for powerpc-dso-linux-gnu

Own Id: OTP-10198

Detect when middle endian doubles are used by a platform and account for it when decoding floats. (Thanks to Mike Sperber)

Own Id: OTP-10209

Erts 5.9.1

Fixed Bugs and Malfunctions

erlang:system_profile errorneous profiled the profiler process when observing runnable processes. This has been corrected.

Own Id: OTP-9849

Calling trace_info/2 asking for information about a function that had native could could crash the run-time system.

Own Id: OTP-9886

reduce smp locking time range in erts_garbage_collect (thanks to Jovi Zhang)

Own Id: OTP-9912

Fix typo in supervisor behaviour doc (Thanks to Ricardo Catalinas Jiménez)

Own Id: OTP-9924

Correct spelling of registered (Thanks to Richard Carlsson)

Own Id: OTP-9925

erts: Remove unused variable (Thanks to Jovi Zhang)

Own Id: OTP-9926

Fix bug in ETS with compressed option and insertion of term containing large integers (>2G) on 64-bit machines. Seen to cause emulator crash. (Thanks to Diego Llarrull for excellent bug report)

Own Id: OTP-9932

Handle Linux OS where /sys/devices/system/node is only readable by root. Fallback to /sys/devices/system/cpu for topology info.

Own Id: OTP-9978

When an escript ends now all printout to standard output and standard error gets out on the terminal. This bug has been corrected by changing the behaviour of erlang:halt/0,1, which should fix the same problem for other escript-like applications, i.e that data stored in the output port driver buffers got lost when printing on a TTY and exiting through erlang:halt/0,1.

The BIF:s erlang:halt/0,1 has gotten improved semantics and there is a new BIF erlang:halt/2 to accomplish something like the old semantics. See the documentation.

Now erlang:halt/0 and erlang:halt/1 with an integer argument will close all ports and allow all pending async threads operations to finish before exiting the emulator. Previously erlang:halt/0 and erlang:halt(0) would just wait for pending async threads operations but not close ports. And erlang:halt/1 with a non-zero integer argument would not even wait for pending async threads operations.

To roughly the old behaviour, to not wait for ports and async threads operations when you exit the emulator, you use erlang:halt/2 with an integer first argument and an option list containing {flush,false} as the second argument. Note that now is flushing not dependant of the exit code, and you can not only flush async threads operations which we deemed as a strange behaviour anyway.

Also, erlang:halt/1,2 has gotten a new feature: If the first argument is the atom 'abort' the emulator is aborted producing a core dump, if the operating system so allows.

Own Id: OTP-9985

Added check to inet driver to avoid building on operating systems that do not yet have IPv6 compatible socket API. (Thanks to Peer Stritzinger)

Own Id: OTP-9996

Fix bug when the number of CPUs actually found is lower than the configured value. (Thanks to Benjamin Herrenschmidt)

Own Id: OTP-10004

The runtime system without SMP support and without thread support erroneously busy waited when no work was present. This bug first appeared in erts-5.9.

Own Id: OTP-10019

Various typographical errors corrected in documentation for common_test, driver, erl_driver and windows installation instructions. (Thanks to Tuncer Ayaz)

Own Id: OTP-10037

Fix memory leak caused by race on exiting process

Own Id: OTP-10041

Improvements and New Features

Add erlang:statistics(scheduler_wall_time) to ensure correct determination of scheduler utilization. Measuring scheduler utilization is strongly preferred over CPU utilization, since CPU utilization gives very poor indications of actual scheduler/vm usage.

Own Id: OTP-9858

ERTS internal API improvements. In some cases the amount of atomic read operations needed have been reduced due to this.

Own Id: OTP-9922

The DTrace source patch from Scott Lystig Fritchie is integrated in the source tree. Using an emulator with dtrace probe is still not supported for production use, but may be a valuable debugging tool. Configure with --with-dynamic-trace=dtrace (or --with-dynamic-trace=systemtap) to create a build with dtrace probes enabled. See runtime_tools for documentation and examples.

Own Id: OTP-10017

Known Bugs and Problems

enif_make_copy may invalidate enif_inspect_binary.

Own Id: OTP-9828

Erts 5.9

Fixed Bugs and Malfunctions

Honor option packet_size for http packet parsing by both TCP socket and erlang:decode_packet. This gives the ability to accept HTTP headers larger than the default setting, but also avoid DoS attacks by accepting lines only up to whatever length you wish to allow. For consistency, packet type line also honor option packet_size. (Thanks to Steve Vinoski)

Own Id: OTP-9389

A few contracts in the lists module have been corrected.

Own Id: OTP-9616

The Unicode noncharacter code points 16#FFFE and 16#FFFE were not allowed to be encoded or decoded using the unicode module or bit syntax. That was inconsistent with the other noncharacters 16#FDD0 to 16#FDEF that could be encoded/decoded. To resolve the inconsistency, 16#FFFE and 16#FFFE can now be encoded and decoded. (Thanks to Alisdair Sullivan.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9624

Make epp search directory of current file first when including another file This completes a partial fix in R11 that only worked for include_lib(). (Thanks to Richard Carlsson)

Own Id: OTP-9645

Fixed memory leak in enif_inspect_io_list_as_binary when applied on a process independent environment.

Own Id: OTP-9668

The number of beam catches allowed in code are no longer statically defined and will grow according to its need.

Own Id: OTP-9692

Add missing parenthesis in heart doc.

Add missing spaces in the Reference Manual distributed section.

In the HTML version of the doc those spaces are necessary to separate those words.

Own Id: OTP-9693

Fixes module erlang doc style: option description (Thanks to Ricardo Catalinas Jiménez)

Own Id: OTP-9697

Specifying a scope to binary:match/3 when using multiple searchstrings resulted in faulty return values. This is now corrected.

Own Id: OTP-9701

The runtime system crashed if more than one thread tried to exit the runtime system at the same time.

Own Id: OTP-9705

Fix documentation for erlang:process_flag/2

For the subsection about process_flag(save_calls, N) there's an unrelated paragraph about process priorities which was copied from the preceeding subsection regarding process_flag(priority, Level). (Thanks to Filipe David Manana)

Own Id: OTP-9714

Calls to erlang:system_flag(schedulers_online, N) and/or erlang:system_flag(multi_scheduling, block|unblock) could cause internal data used by this functionality to get into an inconsistent state. When this happened various problems occurred. This bug was quite hard to trigger, so hopefully no-one has been effected by it.

A spinlock used by the run-queue management sometimes got heavily contended. This code has now been rewritten, and the spinlock has been removed.

Own Id: OTP-9727

Use libdlpi to get physical address (Thanks to Trond Norbye)

Own Id: OTP-9818

Improvements and New Features

An option list argument can now be passed to file:read_file_info/2, file:read_link_info/2 and file:write_file_info/3 and set time type information in the call. Valid options are {time, local}, {time, universal} and {time, posix}. In the case of posix time no conversions are made which makes the operation a bit faster.

Own Id: OTP-7687

A number of memory allocation optimizations have been implemented. Most optimizations reduce contention caused by synchronization between threads during allocation and deallocation of memory. Most notably:

Synchronization of memory management in scheduler specific allocator instances has been rewritten to use lock-free synchronization. Synchronization of memory management in scheduler specific pre-allocators has been rewritten to use lock-free synchronization. The 'mseg_alloc' memory segment allocator now use scheduler specific instances instead of one instance. Apart from reducing contention this also ensures that memory allocators always create memory segments on the local NUMA node on a NUMA system.

Own Id: OTP-7775

The ethread atomic memory operations API used by the runtime system has been extended and improved.

The ethread library now also performs runtime tests for presence of hardware features, such as for example SSE2 instructions, instead of requiring this to be determined at compile time.

All uses of the old deprecated atomic API in the runtime system have been replaced with the use of the new atomic API. In a lot of places this change imply a relaxation of memory barriers used.

Own Id: OTP-9014

gen_sctp:open/0-2 may now return {error,eprotonosupport} if SCTP is not supported

gen_sctp:peeloff/1 has been implemented and creates a one-to-one socket which also are supported now

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9239

Sendfile has been added to the file module's API. sendfile/2 is used to read data from a file and send it to a tcp socket using a zero copying mechanism if available on that OS.

Thanks to Tuncer Ayaz and Steve Vinovski for original implementation

Own Id: OTP-9240

enif_get_reverse_list function added to nif API. This function should be used to reverse small lists which are deep within other structures making it impractical to do the reverse in Erlang.

Own Id: OTP-9392

The deprecated concat_binary/1 BIF has been removed. Use list_to_binary or iolist_to_binary/1 instead.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9421

Erlang/OTP can now be built using parallel make if you limit the number of jobs, for instance using 'make -j6' or 'make -j10'. 'make -j' does not work at the moment because of some missing dependencies.

Own Id: OTP-9451

Line number and filename information are now included in exception backtraces as a fourth element in the MFA tuple. The information will be pretty-printed by the shell and used by common_test to provide better indication of where a test case.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9468

All binary constants used to be handled as heap binaries (i.e. the entire binary would be copied when sent to another process). Binary constants larger than 64 bytes are now refc binaries (i.e. the actual data in the binary will not be copied when sent to another process).

Own Id: OTP-9486

If a float and an integer is compared, the integer is only converted to a float if the float datatype can contain it. Otherwise the float is converted to an integer.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9497

Add NIF function enif_is_number

This function allows for easily determining if a term represents or not a number (integer, float, small or big).(Thanks to Filipe David Manana)

Own Id: OTP-9629

The ERTS internal system block functionality has been replaced by new functionality for blocking the system. The old system block functionality had contention issues and complexity issues. The new functionality piggy-backs on thread progress tracking functionality needed by newly introduced lock-free synchronization in the runtime system. When the functionality for blocking the system isn't used, there is more or less no overhead at all. This since the functionality for tracking thread progress is there and needed anyway.

Own Id: OTP-9631

An ERTS internal, generic, many to one, lock-free queue for communication between threads has been introduced. The many to one scenario is very common in ERTS, so it can be used in a lot of places in the future. Currently it is used by scheduling of certain jobs, and the async thread pool, but more uses are planned for the future.

Drivers using the driver_async functionality are not automatically locked to the system anymore, and can be unloaded as any dynamically linked in driver.

Scheduling of ready async jobs is now also interleaved in between other jobs. Previously all ready async jobs were performed at once.

Own Id: OTP-9632

Tuple funs (a two-element tuple with a module name and a function) are now officially deprecated and will be removed in R16. Use 'fun M:F/A' instead. To make you aware that your system uses tuple funs, the very first time a tuple fun is applied, a warning will be sent to the error logger.

Own Id: OTP-9649

Changed the internal BIF calling convention. Will make simpler faster code and allow BIFs with an arbitrary arity.

Own Id: OTP-9662

Windows native critical sections are now used internally in the runtime system on Windows as mutex implementation. This since they perform better under extreme contention than our own implementation.

Own Id: OTP-9671

Convert some erl_nif macros into inline functions. Allow for better compile time type checking. (Thanks to Tuncer Ayaz)

Own Id: OTP-9675

The +scl command line flag has been added. It can be used for disabling compaction of scheduler load. For more information see the erl(1) documentation.

Own Id: OTP-9695

The build system has been updated so that Erlang/OTP can be built on Mac OS X Lion systems without a GCC compiler. The INSTALL guide has been updated with instructions on how to install a GCC compiler and build Erlang/OTP with it, in order to get a run-time system with better performance.

Own Id: OTP-9712

When loading a module, the system use to run on a single scheduler during the entire loading process. This has been changed to only take down the system just before inserting the loaded code into the system tables, resulting in a much shorter disruption if a module is loaded in a busy system. (Suggested by Bob Ippolito.)

Own Id: OTP-9720

Possible to run HiPE without floating point exceptions (FPE). Useful on platforms that lack reliable FPE. Slower float operations compared to HiPE with FPE.

Own Id: OTP-9724

As of ERTS version 5.9 (OTP-R15B) the runtime system will by default not bind schedulers to logical processors.

If the Erlang runtime system is the only operating system process that binds threads to logical processors, this improves the performance of the runtime system. However, if other operating system processes (as for example another Erlang runtime system) also bind threads to logical processors, there might be a performance penalty instead. In some cases this performance penalty might be severe. Due to this, we change the default so that the user must make an active decision in order to bind schedulers.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9726

The use of erlang:system_flag(scheduler_bind_type, _) and erlang:system_flag(cpu_topology, _) have been deprecated and scheduled for removal in erts-5.10/OTP-R16. For more information see the documentation of erlang:system_flag/2.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9749

An ancient workaround for a Windows bug was removed from the open_port code, open_port({spawn,...}...) is now faster. Thanks to Daniel Goertzen.

Own Id: OTP-9766

The use of deprecated 32bit time_t on 32bit Windows is removed.

Own Id: OTP-9767

The NIF reload mechanism is deprecated. Do not use it as an upgrade method for live production systems. It might be removed in future releases. It can still serve as a development feature but a warning message will be logged each time it is used.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9771

The driver interface has been changed to enable 64-bit aware drivers. Most importantly the return types for ErlDrvEntry callbacks 'call' and 'control' has ben enlarged which require drivers to be changed or they will cause emulator crashes. See Rewrites for 64-bit driver interface in the driver manual.

Due to this driver version management is now mandatory. A driver that is not written with version management or a driver that was compiled with the wrong major version will be not be loaded by the emulator.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9795

Eliminate use of deprecated regexp module

Own Id: OTP-9810

Erts 5.8.5

Fixed Bugs and Malfunctions

Several bugs concerning constant binary constructions such as <<0:4294967295>> have been corrected. Depending on the actual size of the binary and the type of run-time system (32-bit, halfword, 64-bit), such expression could either crash the run-time system or make the loader refuse loading of the module.

Own Id: OTP-9284

The Erlsrv utility failed to stop the erlang machine if no StopAction was defined when the service was stopped. This is now corrected.

Own Id: OTP-9344

Due to a bug in glibc the runtime system could abort while trying to destroy a mutex. A fix for this was introduced in R14B02. This fix did, however, not solve the problem. The runtime system will now issue a warning instead of aborting.

Own Id: OTP-9373 Aux Id: OTP-9009

Replace atom in DRV macro in prim_file with string

An experimental version of Dialyzer discovered that the atom that replaced the DRV macro in prim_file ends up in calls to erlang:open_port({spawn, Driver}, Portopts) as the Driver argument. The documentation states that this call requires a string there.

This change is also consistent with the one introduced in commit 0f03b1e9d2bef3bc830c31a369261af4c5234727 by Kostis Sagonas.

Own Id: OTP-9377

Fix typos in the epmd documentation (Thanks to Holger Weiß )

Own Id: OTP-9387

Fix faulty integer terms created by NIF API from 64-bit integers on halfword emulator. (Thanks to Paolo Negri and Paul Davis)

Own Id: OTP-9394

Fix epmd crash on vxworks caused by faulty argument to select() system call.

Own Id: OTP-9427 Aux Id: seq11855

The ets:test_ms function could in rare cases truncate the error messages. This is now corrected.

Own Id: OTP-9435

Fix bug related to hibernate and HiPE (clear F_HIBERNATE_SCHED flag)

F_HIBERNATE_SCHED flag that was introduced in b7ecdcd1ae9e11b8f75e must be cleared in hipe_mode_switch as well. Otherwise, processes running HiPE code that hibernate, wake up and then trap into a BIF will not be rescheduled.(Thanks to Paul Guyot)

Own Id: OTP-9452

Fix bug in FreeBSD topology detection code (Thanks to Paul Guyot)

Own Id: OTP-9453

Fix use of logical operator && with constant operand instead of bitwise & (Thanks to Cristian Greco)

Own Id: OTP-9454

inet: error if fd does not match socket domain

If an IPv4 fd is opened as an IPv6 socket, unexpected behaviour can occur. For example, if an IPv4 UDP socket is opened and passed into Erlang as an IPv6 socket, the first 3 bytes (corresponding to 1 byte representing the protocol family, 2 bytes set to the port) are stripped from the payload. The cause of the UDP payload truncation happens in inet_drv.c:packet_inet_input when a call to inet_get_address fails silently because the family is set to PF_INET6 but the buffer len is the size of an IPv4 struct sockaddr_in.

(Thanks to Andrew Tunnell-Jones for finding the bug and the test case!)

Own Id: OTP-9455

erts: use a union to avoid strict aliasing issues

Use a union for pointer type conversion to avoid compiler warnings about strict-aliasing violations with gcc-4.1. gcc >= 4.2 does not emit the warning. erts: adapt matrix_nif to R14 erl_nif API changes (Thanks To Tuncer Ayaz)

Own Id: OTP-9487

fix 64-bit issues in the garbage collection (Thanks to Richard Carlsson)

Own Id: OTP-9488

epmd: fix compiler warnings

Suppress compiler warnings about ignored return values. (Thanks to Michael Santos )

Own Id: OTP-9500

Fix non-existing function (erlang:disconnect/1) in distributed reference manual (Thanks to Fabian Król)

Own Id: OTP-9504

Document fdatasync -lrt requirement (SunOS <= 5.10) (Thanks to Tuncer Ayaz)

Own Id: OTP-9512

Let epmd ignore empty ERL_EPMD_ADDRESS

If the environment variable ERL_EPMD_ADDRESS is set to the empty string, empd now behaves like it does by default when ERL_EPMD_ADDRESS is unset. That is, in this case, epmd now listens on all available interfaces instead of using only the loopback interface, which happened because epmd added the loopback address to the (in this case empty) list of addresses specified via ERL_EPMD_ADDRESS.

Also, epmd now ignores ERL_EPMD_ADDRESS if it contains only separator characters (comma and space).

The same applies to epmd's -address option.(Thanks to Holger Weiß)

Own Id: OTP-9525

Remove dead code in erl_compile (Thanks to Tuncer Ayaz)

Own Id: OTP-9527

Add erlang:external_size/2 BIF

This BIF's second parameter is a list of options. Currently the only allowed option is {minor_version, Version} where version is either 0 (default) or 1. (Thanks to Filipe David Manana )

Own Id: OTP-9528

Fix enif_compare on 64bits machines

In 64bits machines the Sint type has a size of 8 bytes, while on 32bits machines it has a 4 bytes size. enif_compare was ignoring this and therefore returning incorrect values when the result of the CMP function (which returns a Sint value) doesn't fit in 4 bytes. (Thanks to Filipe David Manana)

Own Id: OTP-9533

Implement or fix -Werror option

If -Werror is enabled and there are warnings no output file is written. Also make sure that error/warning reporting is consistent. (Thanks to Tuncer Ayaz)

Own Id: OTP-9536

In some rare cases we did not have a run queue when scheduling misc ops. This is now fixed.

Own Id: OTP-9537

Remove misc. compiler warnings

Own Id: OTP-9542

Two bugs in gen_sctp has been corrected: getopts/setopts hence also send could only be called from socket owner, and options 'linger', 'rcvbuf' and 'sndbuf' was read from wrong protocol layer hence read wrong values by getopts.

Own Id: OTP-9544

Erlang/OTP can now be built on MacOS X Lion.

Own Id: OTP-9547

XML files have been corrected.

Own Id: OTP-9550 Aux Id: OTP-9541

Fix potential errors inspired by running cppcheck(1) (Thanks to Christian von Roques)

Own Id: OTP-9557

When auxiliary work was enqueued on a scheduler, the wakeup of the scheduler in order to handle this work could be lost. Wakeups in order to handle ordinary work were not effected by this bug. The bug only effected runtime systems with SMP support as follows:

Deallocation of some ETS data structures could be delayed. On Linux systems not using the NPTL thread library (typically ancient systems with kernel versions prior to 2.6) and Windows systems, the {Port, {exit_status, Status}} message from a terminating port program could be delayed. That is, it only effected port programs which had been started by passing exit_status as an option to open_port/2.

Own Id: OTP-9567

Handle rare race in the crypto key server functionality

Own Id: OTP-9586

Improvements and New Features

Types and specifications have been added.

Own Id: OTP-9356

New allocator strategy "address order first fit". May ease the emptying of memory carriers and thereby real release of memory back to the OS.

Own Id: OTP-9424

The new erlang:check_old_code/1 BIF checks whether a module has old code.

Own Id: OTP-9495

Update documentation and specifications of some of the zlib functions.

Own Id: OTP-9506

Detect the available CPUs on IRIX

Add support for querying the number of configured and online processors on SGI systems running IRIX.(Thanks to Holger Weiß)

Own Id: OTP-9531

Erts 5.8.4

Fixed Bugs and Malfunctions

Fix binary and iolist overflow problems. Typically problems arose in length calculation where the result would exceed (1 bsl 32 - 1).

Own Id: OTP-9118

Using the old erlang shell (i.e. erl instead on werl) on windows and doing several init:restart's would eventually hang the VM. That is no longer the case.

Own Id: OTP-9139

Removed recursive C code when printing Erlang terms to buffers, avoiding stack overflows that could cause VM to crash.

Own Id: OTP-9140

The send_timeout option in gen_tcp did not work properly in active mode or with {active,once} options. This is now corrected.

Own Id: OTP-9145

Fixed various typos across the documentation (Thanks to Tuncer Ayaz)

Own Id: OTP-9154

Remove duplicate stack entries which could occur after calling certain BIFs.

Own Id: OTP-9163

A race when starting two nodes simultaneously using run_erl has been removed.

Own Id: OTP-9164

Add documentation on .erlang processing back again (Thanks to Gabor Liptak)

Own Id: OTP-9189

Remove gratuitous paren in driver_entry(Thanks to Tuncer Ayaz)

Own Id: OTP-9192

Fix some wrong pointer dereferences (Thanks to Cristian Greco)

Own Id: OTP-9194

erts: Remove unused variables (Thanks to Tuncer Ayaz)

Own Id: OTP-9205

The documentation for init:get_args/0 has been removed. init:get_args/0 itself was deprecated in R9C and removed in R12B. (Thanks to Eric Pailleau.)

Own Id: OTP-9209

Allow user to specify the IP address epmd binds to

The IP address(es) epmd binds to can now be specified by the user, either via epmd's new "-address" option or (if that's not used) by setting the environment variable ERL_EPMD_ADDRESS. Multiple addresses may be specified using a comma-separated list. If the loopback address is not in this list, it will be added implicitly, so that the daemon can be queried by an interactive epmd process.(Thanks to Holger Weiß)

Own Id: OTP-9213

epmd: include host address in local access check

In FreeBSD jails, the source and destination address of connections to localhost are changed to be the IP address of the jail. Consider connections from the host's IP address to itself (e.g., the source and destination address match) to be local for the access control checks. (Thanks to Michal Santos and Tom at diogunix.com)

Own Id: OTP-9214

Fix list returned by net_kernel:epmd_module

Function epmd_module of net_kernel returns a list instead of an atom, when the epmd_module-flag is used. (Thanks to Markus Knofe)

Own Id: OTP-9215

Fix epmd's dbg_perror() output

The dbg_perror() function now hands the current errno value over to dbg_gen_printf(). This fixes the problem that errno had been reset to zero by the time it was used (to print the corresponding error message) in the dbg_gen_printf() function. (Thanks to Holger Weiß)

Own Id: OTP-9223

heart: remove garbage appended to heart command

heart:get_cmd/0 is documented to return an empty string if the command is cleared. get_cmd/0 returns 2 extra bytes: 1 byte for the trailing null, 1 byte from the op (the op is an unsigned char and 2 bytes are allocated for it in the returned buffer). (Thanks to Michael Santos)

Own Id: OTP-9224

file: fix hang reading compressed files

The gzio driver goes into an infinite loop when reading past the end of a compressed file. Reported-By: Alex Morarash (Thanks to Michael Santos)

Own Id: OTP-9245

Eliminate alias warnings from gcc 4.5.2

Own Id: OTP-9250

Unsigned integer may overflow in error message (Thanks to Michael Santos)

Own Id: OTP-9251

Driver names should be strings, not atoms

Own Id: OTP-9253

driver_entry: Remove gratuitous paren and fix typo (Thanks to Tuncer Ayaz)

Own Id: OTP-9254

Fix format specifiers in erl_exit messages

Fix an error message by using an unsigned integer specifier as seen in a tweet by @metabrew: #erlang VM crashed with "no next heap size found: -2090496108, offset 0", suddenly allocated all available RAM

Also correct mis-typed string formats in bif.c.(Thanks to Michael Santos)

Own Id: OTP-9262

net_drv: remove unused tcp request id inet_drv: remove gratuitous assignment (Thanks to Tuncer Ayaz)

Own Id: OTP-9263

Teach run_erl RUN_ERL_DISABLE_FLOWCNTRL for disabling flow control

Flow control can cause unwanted behaviour of the beam process, if accidentally hit Ctrl-S (instead of Ctrl-D to detach) the entire beam may be blocked.

Fix this problem by making it possible to turn off flow control by setting the environment variable RUN_ERL_DISABLE_FLOWCNTRL. (Thanks to Jonas Faklkevik)

Own Id: OTP-9270

The following bugs due to missing memory barriers have been fixed:

ETS tables using the write_concurrency option could potentially get into an internally inconsistent state.

ETS tables using the ordered_set option could potentially get into an internally inconsistent state.

A number of memory barriers have been added when building with the libatomic_ops API (i.e. when passing --with-libatomic_ops=PATH to configure) and the tilera atomics API (i.e. when building for the tilera chip). Note that these bugs were due to erroneous usage of the APIs, and not in the implementations of the APIs. When using these APIs the following bugs where present:

The BIF erlang:ports/0 could return an erroneous result.

A thread blocking other threads during code loading, or setup of tracing could potentially read invalid data.

Fixation of ETS tables could potentially get into an internally inconsistent state.

Own Id: OTP-9281

Fix halfword bug for ETS ordered_set when doing select/match with partly bound key.

Own Id: OTP-9292

Fix bug in code:is_module_native that caused crash for deleted modules.

Own Id: OTP-9298

Calling driver_async_cancel() could cause a scheduler thread to enter an eternal loop doing no useful work. (Thanks to Anders Ramsell)

Own Id: OTP-9302

Improvements and New Features

New enif_is_exception function to allow NIFs to determine whether an ERL_NIF_TERM represents an exception. (Thanks to Steve Vinoski)

Own Id: OTP-9150

A process being garbage collected by another process could be scheduled on another scheduler. This prevented this scheduler from doing any useful work until the garbage collection was done. This either occurred due to a explicit call to the garbage_collect/1 BIF, or due to a garbage collection part of code loading. A process being garbage collected like this will now not be scheduled until the garbage collection has completed.

Own Id: OTP-9211

Remove unnecessary validation copy in prim_file:drv_command/3 (Thanks to Tony Rogvall)

Own Id: OTP-9276

Symbolic link handling on windows have been slightly updated to map error conditions more consequently and correctly read directory links created outside of the Erlang environment.

Own Id: OTP-9279

Due to standard library DLL mismatches between versions of OpenSSL and Erlang/OTP, OpenSSL is now linked statically to the crypto driver on Windows. This fixes problems starting crypto when running Erlang as a service on all Windows versions.

Own Id: OTP-9280

Halfword emulator memory handling improvements:

Much more of internal memory structures have been made able to use "high" memory and are no longer restricted to the 4Gb limit that still applies for all process heap data.

Fixed faulty values from erlang:memory() caused by 32-bit counter overflow.

New counter low in erlang:memory() that sums up all memory restricted by 4Gb limit.

Own Id: OTP-9291 Aux Id: seq11841

The value set in the undocumented and unsupported ERL_version_FLAGS (e.g. ERL_R14B03_FLAGS) environment variable can now be overridden by the command line (similar to ERL_AFLAGS).

Own Id: OTP-9297

Erts 5.8.3

Fixed Bugs and Malfunctions

The scroll wheel now scrolls the werl window on Windows.

Own Id: OTP-8985

Some malformed distribution messages could cause VM to crash, this is now corrected.

Own Id: OTP-8993

The OS function getifaddrs() can return NULL in some address fields for e.g PPP and tunnel devices which caused the emulator to segfault. This bug has now been corrected.

Own Id: OTP-8996

The expression <<A:0>> would always produce an empty binary, even if A was not an integer. Corrected to cause a badarg exception if the type of A is invalid. (Thanks to Zvi.)

Own Id: OTP-8997

A bug that potentially could cause an emulator crash when deleting an ETS-table has been fixed. A resource leak when hitting the maximum amount of ETS-tables allowed has also been fixed.

Own Id: OTP-8999

A bug in the exit/2 BIF could potentially cause an emulator crash.

Own Id: OTP-9005

Due to a bug in glibc the runtime system could abort while trying to destroy a mutex. The runtime system will now issue a warning instead of aborting.

Own Id: OTP-9009

A bug in epmd could create strange behaviour when listen() calls failed. This is now corrected thanks to Steve Vinoski.

Own Id: OTP-9024

When setting file_info the win32_driver will now correctly set access and modified time. Previously these entities were swapped.

Own Id: OTP-9046

Setting scheduler bind type to unbound failed if binding of schedulers wasn't supported, or if CPU topology wasn't present. This even though the documentation stated that it is possible to set the bind type to unbound.

Own Id: OTP-9056 Aux Id: Seq11779

Two problems were fixed in crash dump: The time left for timers are now shown as unsigned integers and the contents of ordered_set ETS tables is no longer included.

Own Id: OTP-9057

The VM could fail to set IP_TOS and SO_PRIORITY in certain situations, either because sockets were supplied as open file descriptors, or because SO_PRIORITY by default was set higher than the user can explicitly set it to. Those situations are now handled.

Own Id: OTP-9069

Wx on MacOS X generated complains on stderr about certain cocoa functions not beeing called from the "Main thread". This is now corrected.

Own Id: OTP-9081

Fix a couple typos in driver_entry(3) (thanks to Tuncer Ayaz).

Own Id: OTP-9085

Mention that "-detached" implies "-noinput"

Clarify that specifying "-noinput" is unnecessary if the "-detached" flag is given. (thanks to Holger Weiß)

Own Id: OTP-9086

A potential problem (found by code inspection) when calling a fun whose code was not loaded has been fixed.

Own Id: OTP-9095

The emulator could get into a state where it didn't check for I/O.

Own Id: OTP-9105 Aux Id: Seq11798

Attempting to create binaries exceeding 2Gb (using for example term_to_binary/1) would crash the emulator with an attempt to allocate huge amounts of memory. (Thanks to Jon Meredith.)

Own Id: OTP-9117

Fix erlang:hibernate/3 on HiPE enabled emulator (Thanks to Paul Guyot)

Own Id: OTP-9125

Improvements and New Features

From this release, the previously experimental halfword emulator is now official. It can be enabled by giving the --enable-halfword-emulator option to the configure script.

The halfword emulator is a 64-bit application, but uses halfwords (32-bit words) for all data in Erlang processes, therefore using less memory and being faster than the standard 64-bit emulator. The total size of all BEAM code and all process data for all processes is limited to 4Gb, but ETS tables and off-heap binaries are only limited by the amount of available memory.

Own Id: OTP-8941

32-bit atomic memory operations have been introduced internally in the run time system, and are now used where appropriate. There were previously only atomic memory operations of word size available. The 32-bit atomic memory operations slightly reduce memory consumption, and slightly improve performance on 64-bit runtime systems.

Own Id: OTP-8974

Performance enhancements for looking up timer-entries and removing timers from the wheel.

Own Id: OTP-8990

Write accesses to ETS tables have been optimized by reducing the amount of atomic memory operations needed during a write access.

Own Id: OTP-9000

Strange C coding in the VM made the -D_FORTIFY_SOURCE option to gcc-4.5 react badly. The code is now cleaned up so that it's accepted by gcc-4.5.

Own Id: OTP-9025

The memory footprint for loaded code has been somewhat reduced (especially in the 64-bit BEAM machine).

Own Id: OTP-9030

The maximum number of allowed arguments for an Erlang function has been lowered from 256 to 255, so that the number of arguments can now fit in a byte.

Own Id: OTP-9049

Dependency generation for Makefiles has been added to the compiler and erlc. See the manual pages for compile and erlc. (Thanks to Jean-Sebastien Pedron.)

Own Id: OTP-9065

Erts 5.8.2

Fixed Bugs and Malfunctions

Fix format_man_pages so it handles all man sections and remove warnings/errors in various man pages.

Own Id: OTP-8600

The configure command line argument --enable-ethread-pre-pentium4-compatibility had no effect. This option is now also automatically enabled if required on the build machine.

Own Id: OTP-8847

Windows 2003 and Windows XP pre SP3 would sometimes not start the Erlang R14B VM at all due to a bug in the cpu topology detection. The bug affects Windows only, no other platform is even remotely affected. The bug is now corrected.

Own Id: OTP-8876

The HiPE run-time in the 64-bit emulator could do a 64-bit write to a 32-bit struct field. It happened to be harmless on Intel/AMD processors. Corrected. (Thanks to Mikael Pettersson.)

Own Id: OTP-8877

A bug in erl_drv_tsd_get() and enif_tsd_get() could cause an emulator crash. These functions are currently not used in OTP. That is, the crash only occur on systems with user implemented NIF libraries, or drivers that use one of these functions.

Own Id: OTP-8889

Calling erlang:system_info({cpu_topology, CpuTopologyType}) with another CpuTopologyType element than one of the documented atoms defined, detected, or used caused an emulator crash. (Thanks to Paul Guyot)

Own Id: OTP-8914

The ERTS internal rwlock implementation could get into an inconsistent state. This bug was very seldom triggered, but could be during heavy contention. The bug was introduced in R14B (erts-5.8.1).

The bug was most likely to be triggered when using the read_concurrency option on an ETS table that was frequently accessed from multiple processes doing lots of writes and reads. That is, in a situation where you typically don't want to use the read_concurrency option in the first place.

Own Id: OTP-8925 Aux Id: OTP-8544

Tracing to port could cause an emulator crash when unloading the trace driver.

Own Id: OTP-8932

Removed use of CancelIoEx on Windows that had been shown to cause problems with some drivers.

Own Id: OTP-8937

The fallback implementation used when no native atomic implementation was found did not compile. (Thanks to Patrick Baggett, and Tuncer Ayaz)

Own Id: OTP-8944

Some integer values used during load balancing could under rare circumstances wrap causing a load unbalance between schedulers.

Own Id: OTP-8950

The windows VM now correctly handles appending to large files (> 4GB).

Own Id: OTP-8958

Name resolving of IPv6 addresses has been implemented for Windows versions that support it. The use of ancient resolver flags (AI_V4MAPPED | AI_ADDRCONFIG) to the getaddrinfo() function has been removed since e.g FreeBSD regard mapped IPv4 addresses to be a security problem and the semantics of the address configured flag is uncertain.

Own Id: OTP-8969

Improvements and New Features

The help texts produced by the configure scripts in the top directory and in the erts directory have been aligned and cleaned up.

Own Id: OTP-8859

When the runtime system had fewer schedulers than logical processors, the system could get an unnecessarily large amount reader groups.

Own Id: OTP-8861

run_rel has been updated to support Solaris's /dev/ptmx device and to load the necessary STREAMS modules so that to_erl can provide terminal echo of keyboard input. (Thanks to Ryan Tilder.)

Own Id: OTP-8878

The Erlang VM now supports Unicode filenames. The feature is turned on by default on systems where Unicode filenames are mandatory (Windows and MacOSX), but can be enabled on other systems with the '+fnu' emulator option. Enabling the Unicode filename feature on systems where it is not default is however considered experimental and not to be used for production. Together with the Unicode file name support, the concept of "raw filenames" is introduced, which means filenames provided without implicit unicode encoding translation. Raw filenames are provided as binaries, not lists. For further information, see stdlib users guide and the chapter about using Unicode in Erlang. Also see the file module manual page.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8887

Buffer overflows have been prevented in erlc, dialyzer, typer, run_test, heart, escript, and erlexec.

(Thanks to Michael Santos.)

Own Id: OTP-8892

The runtime system is now less eager to suspend processes sending messages over the distribution. The default value of the distribution buffer busy limit has also been increased from 128 KB to 1 MB. This in order to improve throughput.

Own Id: OTP-8901

The distribution buffer busy limit can now be configured at system startup. For more information see the documentation of the erl +zdbbl command line flag. (Thanks to Scott Lystig Fritchie)

Own Id: OTP-8912

The inet driver internal buffer stack implementation has been rewritten in order to reduce lock contention.

Own Id: OTP-8916

New ETS option compressed, to enable a more compact storage format at the expence of heavier table operations. For test and evaluation, erl +ec can be used to force compression on all ETS tables.

Own Id: OTP-8922 Aux Id: seq11658

There is now a new function inet:getifaddrs/0 modeled after C library function getifaddrs() on BSD and LInux that reports existing interfaces and their addresses on the host. This replaces the undocumented and unsupported inet:getiflist/0 and inet:ifget/2.

Own Id: OTP-8926

Support for detection of CPU topology and binding of schedulers on FreeBSD 8 have been added. (Thanks to Paul Guyot)

Own Id: OTP-8939

Several bugs related to hibernate/3 and HiPE have been corrected. (Thanks to Paul Guyot.)

Own Id: OTP-8952

Support for soft and hard links on Windows versions and filesystems that support them is added.

Own Id: OTP-8955

The win32 virtual machine is now linked large address aware. his allows the Erlang VM to use up to 3 gigs of address space on Windows instead of the default of 2 gigs.

Own Id: OTP-8956

Erts 5.8.1

Fixed Bugs and Malfunctions

Very small floating point numbers generated errors when converting from list to float in some versions of the VM, this is now corrected so that i.e. list_to_float("1.0e-324"). returns 0.0 in all versions of Erlang.

Own Id: OTP-7178

Windows Vista and Windows 7 file system virtualization, which makes "old style" windows programs execute in a file system sandbox, was previously unintentionally turned on for the Erlang VM. This is now corrected so that i.e. writes to C:\Program Files\. without administrator privileges will fail.

Own Id: OTP-7405

Fix faulty 64-bit integer term output from drivers. Large 64-bits integers did not generate correct bignums and could even cause emulator crash. Only affects drivers using ERL_DRV_INT64 or ERL_DRV_UINT64, introduced in R13B03.

Own Id: OTP-8716

Fixed: inet:setopts(S, [{linger,{true,2}}]) returned {error,einval} for SCTP sockets. The inet_drv had a bug when checking the option size.

Own Id: OTP-8726 Aux Id: seq11617

Fix libm linking with --as-needed flag

When building with "--as-needed" linker flags on Linux the build will fail. This has now been fixed.

(Thanks to Christian Faulhammer)

Own Id: OTP-8728

gen_udp:connect/3 was broken for SCTP enabled builds. It did not detect remote end errors as it should.

Own Id: OTP-8729

Reduce the risk of integer wrapping in bin vheap size counting.

The vheap size series will now use the golden ratio instead of doubling and fibonacci sequences.

Own Id: OTP-8730

ETS ordered_set containing [] as key could cause strange thing to happen, like an infinite hanging ets:select.

Own Id: OTP-8732

reference() has been substituted for ref() in the documentation.

Own Id: OTP-8733

When a native compiled module called a not loaded non-native compiled module that had an on_load function, the export entries were trashed after code loading so on the next call from the native compiled module to the non-native compiled the emulator crashed. This bug has now been fixed.

Own Id: OTP-8736

HiPE-enabled Erlang VMs running on BSD systems sometimes generated messages like "Yikes! erts_alloc() returned misaligned address 0x8016a512c". Fixed. (Thanks to Mikael Pettersson.)

Own Id: OTP-8769

A race condition in erts_poll() could cause delay of poll for I/O.

Own Id: OTP-8773

Removed some potential vulnerabilities from the Erlang Port Mapper Daemon (epmd) and straightened up access control. Also removed hazardous interfaces allowing anyone on a machine to forcefully unregister other nodes. This means that the ei_unregister/erl_unregister interfaces in erl_interface is rendered not only error prone and mystifying as before, but totally ineffective. The old behaviour of unchecked node unregistering can be restored if needed, see epmd documentation for details.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8780

Building in a source tree without prebuilt platform independent build results failed on the SSL examples when:

cross building. This has been solved by not building the SSL examples during a cross build. building on Windows.

Own Id: OTP-8791

inet:getsockopt for SCTP sctp_default_send_param had a bug to not initialize required feilds causing random answers. It is now corrected.

Own Id: OTP-8795 Aux Id: seq11655

The hipe_bifs:get_hrvtime/0 BIF now always returns a real value even if the "perfctr" Linux kernel extension is not available. It used to return a dummy value. (Thanks to Mikael Pettersson.)

Own Id: OTP-8798

Calling a native-code compiled module with an on_load function could cause a crash. (Thanks to Mikael Pettersson.)

Own Id: OTP-8799

The emulator could crash while writing a crash dump if native-compiled modules had been loaded. (Thanks to Paul Guyot.)

Own Id: OTP-8801

The garbage collector could crash if invoked from native-compiled code after a call to a BIF. (Thanks to Paul Guyot.)

Own Id: OTP-8821

A rare memory leak in binary:matches is removed

Own Id: OTP-8823

For a socket in the HTTP packet mode, the return value from gen_tcp:recv/2,3 if there is an error in the header will be {ok,{http_error,String}} instead of {error,{http_error,String}} to be consistent with ssl:recv/2,3.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8831

Improvements and New Features

ets:select_reverse/{1,2,3} are now documented.

Own Id: OTP-7863

External format of integers changed to make full use of all 32 bits of INTEGER_EXT. This is a compatible change as old code can read full 32-bit integers but only produce 28-bit integers as INTEGER_EXT.

Own Id: OTP-8540 Aux Id: seq11534

Large parts of the ethread library have been rewritten. The ethread library is an Erlang runtime system internal, portable thread library used by the runtime system itself.

Most notable improvement is a reader optimized rwlock implementation which dramatically improve the performance of read-lock/read-unlock operations on multi processor systems by avoiding ping-ponging of the rwlock cache lines. The reader optimized rwlock implementation is used by miscellaneous rwlocks in the runtime system that are known to be read-locked frequently, and can be enabled on ETS tables by passing the {read_concurrency, true} option upon table creation. See the documentation of ets:new/2 for more information. The reader optimized rwlock implementation can be fine tuned when starting the runtime system. For more information, see the documentation of the +rg command line argument of erl.

There is also a new implementation of rwlocks that is not optimized for readers. Both implementations interleaves readers and writers during contention as opposed to, e.g., the NPTL (Linux) pthread rwlock implementation which use either a reader or writer preferred strategy. The reader/writer preferred strategy is problematic since it starves threads doing the non-preferred operation.

The new rwlock implementations in general performs better in ERTS than common pthread implementations. However, in some extremely heavily contended cases this is not the case. Such heavy contention can more or less only appear on ETS tables. This when multiple processes do very large amounts of write locked operations simultaneously on the same table. Such use of ETS is bad regardless of rwlock implementation, will never scale, and is something we strongly advise against.

The new rwlock implementations depend on atomic operations. If no native atomic implementation is found, a fallback solution will be used. Using the fallback implies a performance degradation. That is, it is more important now than before to build OTP with a native atomic implementation.

The ethread library contains native atomic implementations for, x86 (32 and 64 bit), powerpc (32 bit), sparc V9 (32 and 64 bit), and tilera (32 bit). On other hardware gcc's builtin support for atomic memory access will be used if such exists. If no such support is found, configure will warn about no atomic implementation available.

The ethread library can now also use the libatomic_ops library for atomic memory accesses. This makes it possible for the Erlang runtime system to utilize optimized native atomic operations on more platforms than before. If configure warns about no atomic implementation available, try using the libatomic_ops library. Use the --with-libatomic_ops=PATH configure command line argument when specifying where the libatomic_ops installation is located. The libatomic_ops library can be downloaded from: http://www.hpl.hp.com/research/linux/atomic_ops/

The changed API of the ethread library has also caused modifications in the Erlang runtime system. Preparations for the to come "delayed deallocation" feature has also been done since it depends on the ethread library.

Note: When building for x86, the ethread library will now use instructions that first appeared on the pentium 4 processor. If you want the runtime system to be compatible with older processors (back to 486) you need to pass the --enable-ethread-pre-pentium4-compatibility configure command line argument when configuring the system.

Own Id: OTP-8544

erlang:localtime_to_universaltime({{2008, 8, 1}, {0, 0, 0}},true) when TZ=UTC now behaves consistently on all Unix platforms.

The problem fixed was originally reported by Paul Guyot on erlang-bugs mailing list:

http://www.erlang.org/pipermail/erlang-bugs/2008-November/001077.html

Own Id: OTP-8580

Optimization reducing memory consumption by two words per ETS object.

Own Id: OTP-8737

Fixes for unsupported halfword-emulator

Own Id: OTP-8745

NIF 64-bit integer support; enif_get_int64, enif_get_uint64, enif_make_int64, enif_make_uint64.

Own Id: OTP-8746

Alignment of trailing data in messages has been adjusted. This in order to be able to pass data of any type as trailing data in the future.

Own Id: OTP-8754

The obsolete/driver.h header file has been removed. It has been obsolete and deprecated since R8B. Drivers that still include obsolete/driver.h must be updated to include erl_driver.h.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8758

Added erlang:system_info(build_type) which makes it easier to chose drivers, NIF libraries, etc based on build type of the runtime system.

The NIF library for crypto can now be built for valgrind and/or debug as separate NIF libraries that will be automatically loaded if the runtime system has been built with a matching build type.

Own Id: OTP-8760

Further lessened the memory requirements of ETS objects.

Own Id: OTP-8762

The broken elib_malloc alternate memory allocator has been removed. erlang:system_info(elib_malloc) will always return false, and in R15, erlang:system_info(elib_malloc) will fail with a badarg exception.

Own Id: OTP-8764

Calling erlang:system_info/1 with the new argument update_cpu_info will make the runtime system reread and update the internally stored CPU information. For more information see the documentation of erlang:system_info(update_cpu_info).

The CPU topology is now automatically detected on Windows systems with less than 33 logical processors. The runtime system will now, also on Windows, by default bind schedulers to logical processors using the default_bind bind type if the amount of schedulers is at least equal to the amount of logical processors configured, binding of schedulers is supported, and a CPU topology is available at startup.

Own Id: OTP-8765

The SMP ERTS internal child waiter thread used on Linux system with NPTL was unintentionally disabled during cross compilation rewrites (OTP-8323 in R13B03). It has now been re-enabled. Enabling it again gives a slight performance improvement.

Own Id: OTP-8774

epmd used to generate a message to the syslog when it started up, which could be annoying. This has been changed to only generate the message if the debug swith is given. (Thanks to Michael Santos.)

Own Id: OTP-8775

The scheduler wakeup threshold is now possible to adjust at system boot. For more information see the +swt command line argument of erl.

Own Id: OTP-8811

The undocumented function inet:ifget/2 has been improved to return interface hardware address (MAC) on platforms supporting getaddrinfo() (such as BSD unixes). Note it still does not work on all platforms for example not Windows nor Solaris, so the function is still undocumented.

Buffer overflow and field init bugs for inet:ifget/2 and inet:getservbyname/2 has also been fixed.

Thanks to Michael Santos.

Own Id: OTP-8816

Optimizations for MIPS when using gcc atomics. (Thanks to Steve Vinoski)

Own Id: OTP-8834

Lock optimization in timer functionality.

Own Id: OTP-8835

Known Bugs and Problems

Fix epmd and build environment to build on VxWorks

Own Id: OTP-8838

Erts 5.8

Fixed Bugs and Malfunctions

Driver threads, such as async threads, using port data locks peeked at the port status field without proper locking when looking up the driver queue.

Own Id: OTP-8475

The use of mmap() was unnecessarily disabled when cross compiling.

The configure arguments --with-ssl, and --with-odbc refused to accept libraries outside of $erl_xcomp_sysroot when cross compiling for no good reason.

The configure argument --with-odbc didn't handle the value yes correct.

The configure arguments --with-odbc, and --without-odbc have also been added to the configure help.

(Thanks to Steve Vinoski for reporting these issues)

Own Id: OTP-8484

A call to the BIF unregister(RegName) when a port had the name RegName registered in the runtime system without SMP support caused a runtime system crash. (Thanks to Per Hedeland for the bugfix and test case.)

Own Id: OTP-8487

The runtime system crashed if fewer logical processors were found than reported by sysconf( SC_NPROCESSORS_CONF).

Own Id: OTP-8549

Fix memory management bug causing crash of non-SMP emulator with async threads enabled. The bug did first appear in R13B03.

Own Id: OTP-8591 Aux Id: seq11554

Port locks could be prematurely destroyed.

Own Id: OTP-8612

The empd program could loop and consume 100% CPU time if an unexpected error ocurred in listen() or accept(). Now epmd will terminate if a non-recoverable error occurs. (Thanks to Michael Santos.)

Own Id: OTP-8618

When kernel poll has been enabled, a livelock could in rare circumstances occur. Problem reported by Chetan Ahuja, fix by Mikael Pettersson.

Own Id: OTP-8632

Windows: Closing port of program that stalled without reading all data could deadlock scheduler thread.

Own Id: OTP-8641

On some combination of Montavista Linux on Cavium Octeon processors, some socket-related system calls returned other numbers than -1 for errors. This caused a core dump in inet_drv.c. Now the code works around this problem.

Own Id: OTP-8654

Missing memory barriers in erts_poll() could cause the runtime system to hang indefinitely.

Own Id: OTP-8658

ethr_rwmutex_tryrlock() acquired and refused to acquire a lock with inverted logic. The lock was however never acquired in a thread unsafe manner. (Thanks to JR Zhang for noting this issue)

Own Id: OTP-8662

Extreme combinations of register/unregister in a highly parallell SMP application could crash the VM. The error is corrected.

Own Id: OTP-8663

On Windows, files are now opened with FILE_SHARE_DELETE to get closer to Unix semantics.

Own Id: OTP-8667

erlang:system_info(multi_scheduling) sometimes erroneously returned enabled when it should have returned blocked.

Own Id: OTP-8675

Fix bug causing erlang:decode_packet and enif_make_string to generate faulty strings with negative character values for ascii values larger than 127. (Thanks to Paul Guyot)

Own Id: OTP-8685

open_port/2 with the spawn and spawn_executable options can include an {env,Env} option. In some cases unsetting variables would not work on Unix (typically if more variables were unset than were actually present in the environment).

Own Id: OTP-8701

A user defined CPU topology set via a call to erlang:system_flag(cpu_topology, CPUTopology) was not properly verified, and could in worst case cause an emulator crash. The emulator crash could only occur when a user defined CPU topology already existed and was redefined.

Own Id: OTP-8710

Improvements and New Features

The grammar for match specifications in ERTS users guide only described the tracing dialect of match specifications. An informal grammar for the ETS dialect is added.

Own Id: OTP-8086 Aux Id: seq11333

The module binary from EEP31 (and EEP9) is implemented.

Own Id: OTP-8217

New NIF API function enif_make_new_binary

Own Id: OTP-8474

The guard BIF is_boolean/1 (introduced in R10B) has now been included in the lists of BIFs allowed in guards in the Reference Manual.

Own Id: OTP-8477

Added function zip:foldl/3 to iterate over zip archives.

Added functions to create and extract escripts. See escript:create/2 and escript:extract/2.

The undocumented function escript:foldl/3 has been removed. The same functionality can be achieved with the more flexible functions escript:extract/2 and zip:foldl/3.

Record fields has been annotated with type info. Source files as been adapted to fit within 80 chars and trailing whitespace has been removed.

Own Id: OTP-8521

A regular expression with many levels of parenthesis could cause a buffer overflow. That has been corrected. (Thanks to Michael Santos.)

Own Id: OTP-8539

erlang:decode_packet(httph_bin,..) could return corrupt header strings or even crash the VM. This has been fixed. It only happened on 32-bit VM if the header name was unknown and between 16 and 20 characters long. Sockets with simular packet option did not suffer from this bug.

Own Id: OTP-8548

New NIF features:

Send messages from a NIF, or from thread created by NIF, to any local process (enif_send) Store terms between NIF calls (enif_alloc_env, enif_make_copy) Create binary terms with user defined memory management (enif_make_resource_binary)

And some incompatible changes made to the API. For more information see the warning text in erl_nif(3).

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8555

If the 'fop' program (needed for building PDF files) cannot not be found, it is now possible to build the HTML and man pages anyway (there will also be dummy PDF files with no real content created). (Thanks to Tuncer Ayaz.)

Own Id: OTP-8559

When defining macros the closing right parenthesis before the dot is now mandatory.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8562

Local and imported functions now override the auto-imported BIFs when the names clash. The pre R14 behaviour was that auto-imported BIFs would override local functions. To avoid that old programs change behaviour, the following will generate an error:

Doing a call without explicit module name to a local function having a name clashing with the name of an auto-imported BIF that was present (and auto-imported) before OTP R14A

Explicitly importing a function having a name clashing with the name of an autoimported BIF that was present (and autoimported) before OTP R14A

Using any form of the old compiler directive nowarn_bif_clash

If the BIF was added or auto-imported in OTP R14A or later, overriding it with an import or a local function will only result in a warning,

To resolve clashes, you can either use the explicit module name erlang to call the BIF, or you can remove the auto-import of that specific BIF by using the new compiler directive -compile({no_auto_import,[F/A]})., which makes all calls to the local or imported function without explicit module name pass without warnings or errors.

The change makes it possible to add auto-imported BIFs without breaking or silently changing old code in the future. However some current code ingeniously utilizing the old behaviour or the nowarn_bif_clash compiler directive, might need changing to be accepted by the compiler.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8579

A bug in re that could cause certain regular expression matches never to terminate is corrected. (Thanks to Michael Santos and Gordon Guthrie.)

Own Id: OTP-8589

The erlang:open_port spawn and spawn_executable directives can include an {env,Env} directive to set up environment variables for the spawned process. A bug prevented applications from using {env,Env} to set an environment variable whose value ended with a '=' (equal sign) character; the trailing equal sign was mistaken as an indication that an environment variable was to be cleared from the environment of the spawned process. (Thanks to Steve Vinoski.)

Own Id: OTP-8614

receive statements that can only read out a newly created reference are now specially optimized so that it will execute in constant time regardless of the number of messages in the receive queue for the process. That optimization will benefit calls to gen_server:call(). (See gen:do_call/4 for an example of a receive statement that will be optimized.)

Own Id: OTP-8623

The functions file:advise/4 and file:datasync/1 have been added. (Thanks to Filipe David Manana.)

Own Id: OTP-8637

New NIF API functions: enif_make_atom_len, enif_make_existing_atom_len, enif_make_string_len, enif_get_atom_length, enif_get_list_length, enif_is_list, enif_is_tuple (by Tuncer Ayaz)

Own Id: OTP-8640

Support for using gcc's built-in functions for atomic memory access has been added. This functionallity will be used if available and no other native atomic implementation in ERTS is available.

Own Id: OTP-8659

The number of spinlocks used when implementing atomic fall-backs when no native atomic implementation is available has been increased from 16 to 1024.

Own Id: OTP-8660

Writer preferred pthread read/write locks has been enabled on Linux.

Own Id: OTP-8661

The runtime system will by default bind schedulers to logical processors using the default_bind bind type if the amount of schedulers are at least equal to the amount of logical processors configured, binding of schedulers is supported, and a CPU topology is available at startup.

NOTE: If the Erlang runtime system is the only operating system process that binds threads to logical processors, this improves the performance of the runtime system. However, if other operating system processes (as for example another Erlang runtime system) also bind threads to logical processors, there might be a performance penalty instead. If this is the case you, are are advised to unbind the schedulers using the +sbtu command line argument, or by invoking erlang:system_flag(scheduler_bind_type, unbound).

Own Id: OTP-8666

The recently added BIFs erlang:min/2, erlang:max/2 and erlang:port_command/3 are now auto-imported (as they were originally intended to be). Due to the recent compiler change (OTP-8579), the only impact on old code defining it's own min/2, max/2 or port_command/3 functions will be a warning, the local functions will still be used. The warning can be removed by using -compile({no_auto_import,[min/2,max/2,port_command/3]}). in the source file.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8669 Aux Id: OTP-8579

There is a new option 'exclusive' to file:open/2 that uses the OS O_EXCL flag where supported to open the file in exclusive mode.

Own Id: OTP-8670

Now, binary_to_term/2 is auto-imported. This will cause a compile warning if and only if a module has got a local function with that name.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8671

Alignment of scheduler data and run queues were adjusted.

Own Id: OTP-8673

Call time breakpoint tracing

Introduce a call_time option to erlang:trace_pattern/3.This option enables call time breakpoint tracing on code that is executed by processes with call tracing enabled. Call time tracing stores the number of calls and the time spent of each function with this trace pattern enabled. The information can be retrieved with erlang:trace_info/2 Add a scheduler array for BpData. To solve the issue of multiple schedulers constantly updating the head pointer to the bp data wheel, each scheduler now has its own entrypoint to the wheel. This head pointer can be updated without a locking being taken.

Teach call count tracing to use atomics

Call count previously used a global lock for accessing and writing its counter in the breakpoint. This is now changed to atomics instead. The change will let call count tracing and cprof to scale better when increasing the number of schedulers.

Own Id: OTP-8677

eprof has been reimplemented with support in the Erlang virtual machine and is now both faster (i.e. slows down the code being measured less) and scales much better. In measurements we saw speed-ups compared to the old eprof ranging from 6 times (for sequential code that only uses one scheduler/core) up to 84 times (for parallel code that uses 8 cores).

Note: The API for the eprof has been cleaned up and extended. See the documentation.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8706

Erts 5.7.5

Fixed Bugs and Malfunctions

Fix binary_to_term crash on compressed term with corrupt size field.

Own Id: OTP-8336

Processes and/or ports could get stuck on a blocked scheduler when erlang:system_flag(multi_scheduling, block) was used.

Processes and/or ports could get stuck on an offline scheduler when schedulers online were reduced using erlang:system_flag(schedulers_online, SchedulersOnline).

Own Id: OTP-8342

Building on Windows will now work if the paths to mc.exe and rc.exe contain spaces. The README.win32 file has been updated with some information about building using Visual Studio 2008. (Thanks to Andrew Thompson.)

Own Id: OTP-8345

EPMD now correctly handles the extra data field which can be given in the ALIVE2_REQ request and retrieved in the PORT2_RESP response. (Thanks to Klas Johansson.)

Own Id: OTP-8361

The configure test for reliable floating point exceptions has been update to work on modern versions of Mac OS X. (Thanks to Trannie Carter.)

Own Id: OTP-8368

ERTS makefiles used to detect the use of a gcc C compiler by checking if CC equaled gcc. That is, the makefiles failed to detect gcc C compilers with other command line names than gcc. `configure' now substitute GCC into the makefiles. If CC is a gcc C compiler, GCC will have the value yes. (Thanks to Jean-Sébastien Pédron)

Own Id: OTP-8373

ETS bug causing the memory counter from ets:info for ordered_set's to sometimes get out of synch and absurdly high.

Own Id: OTP-8377 Aux Id: seq11442

Immediately repeated multi-scheduling block/unblock cycles using erlang:system_flag(multi_scheduling, block | unblock) could deadlock the runtime system.

Own Id: OTP-8386

A number of bugs concerning re and unicode are corrected:

re:compile no longer loses unicode option, which also fixes bug in re:split.

re:replace now handles unicode charlist replacement argument

re:replace now handles unicode RE charlist argument correctly

re:replace now handles binary unicode output correctly when nothing is replaced.

Most code, testcases and error isolation done by Rory Byrne.

Own Id: OTP-8394

The loading of native code was not properly atomic in the SMP emulator, which could cause crashes. Also a per-MFA information table for the native code has now been protected with a lock since it turns that it could be accessed concurrently in the SMP emulator. (Thanks to Mikael Pettersson.)

Own Id: OTP-8397

Fix processes in exiting status that are about to be scheduled, to not be allowed to garbage collect.

Own Id: OTP-8420

Removed bogus "\011" character sequence in documentation.

Own Id: OTP-8422

The resolver routines failed to look up the own node name as hostname, if the OS native resolver was erroneously configured, bug reported by Yogish Baliga, now fixed.

The resolver routines now tries to parse the hostname as an IP string as most OS resolvers do, unless the native resolver is used.

The DNS resolver inet_res and file resolver inet_hosts now do not read OS configuration files until they are needed. Since the native resolver is default, in most cases they are never needed.

The DNS resolver's automatic updating of OS configuration file data (/etc/resolv.conf) now uses the 'domain' keyword as default search domain if there is no 'search' keyword.

Own Id: OTP-8426 Aux Id: OTP-8381

The re module: A regular expression with an option change at the start of a pattern that had top-level alternatives could cause overwriting and/or a crash. (Thanks to Michael Santos.)

Own Id: OTP-8438

Harmless buffer overflow by one byte in asn1 and ram_file_drv.

Own Id: OTP-8451

Improvements and New Features

Improved GC performance after BIF/NIF call when a lot of heap fragments was created. This will mainly benefit NIFs that return large compound terms.

Own Id: OTP-8240

Incompatible changes in the experimental NIF feature.

Changed the NIF function prototypes in order to allow more than 3 function arguments. enif_get_data renamed as enif_priv_data. enif_make_string got a third argument for character encoding. The return value of erlang:load_nif/2 on error changed.

Read more in the documentation of erl_nif and erlang:load_nif/2

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8304

Cross compilation improvements and other build system improvements.

Most notable:

Lots of cross compilation improvements. The old cross compilation support was more or less non-existing as well as broken. Please, note that the cross compilation support should still be considered as experimental. Also note that old cross compilation configurations cannot be used without modifications. For more information on cross compiling Erlang/OTP see the $ERL_TOP/INSTALL-CROSS.md file. Support for staged install using DESTDIR. The old broken INSTALL_PREFIX has also been fixed. For more information see the $ERL_TOP/INSTALL.md file. Documentation of the release target of the top Makefile. For more information see the $ERL_TOP/INSTALL.md file. make install now by default creates relative symbolic links instead of absolute ones. For more information see the $ERL_TOP/INSTALL.md file. $ERL_TOP/configure --help=recursive now works and prints help for all applications with configure scripts. Doing make install, or make release directly after make all no longer triggers miscellaneous rebuilds. Existing bootstrap system is now used when doing make install, or make release without a preceding make all. The crypto and ssl applications use the same runtime library path when dynamically linking against libssl.so and libcrypto.so. The runtime library search path has also been extended. The configure scripts of erl_interface and odbc now search for thread libraries and thread library quirks the same way as erts do. The configure script of the odbc application now also looks for odbc libraries in lib64 and lib/64 directories when building on a 64-bit system. The config.h.in file in the erl_interface application is now automatically generated in instead of statically updated which reduces the risk of configure tests without any effect.

(Thanks to Henrik Riomar for suggestions and testing)

(Thanks to Winston Smith for the AVR32-Linux cross configuration and testing)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8323

NIF improvements:

Driver API for multi-threading made available for NIFs. Support for mempory managed (garbage collected) resource objects. A way to pass "pointers" to native data structures between C and Erlang in a safe way. Support for references, floats and term comparison. Various new functions, like enif_inspect_iolist_as_binary, enif_make_sub_binary, enif_get_string, enif_get_atom, enif_make_tuple_from_array, enif_make_list_from_array, enif_make_existing_atom.

Own Id: OTP-8335

Minor alignment adjustments of scheduler specific data.

Own Id: OTP-8341

The documentation is now possible to build in an open source environment after a number of bugs are fixed and some features are added in the documentation build process.

- The arity calculation is updated.

- The module prefix used in the function names for bif's are removed in the generated links so the links will look like "http://www.erlang.org/doc/man/erlang.html#append_element-2" instead of "http://www.erlang.org/doc/man/erlang.html#erlang:append_element-2".

- Enhanced the menu positioning in the html documentation when a new page is loaded.

- A number of corrections in the generation of man pages (thanks to Sergei Golovan)

- The legal notice is taken from the xml book file so OTP's build process can be used for non OTP applications.

Own Id: OTP-8343

There is new erlang:binary_to_term/2 BIF that takes an option list. The option safe can be used to prevent creation of resources that are not garbage collected (such as atoms). (Thanks to Jayson Vantuyl.)

Own Id: OTP-8367

The default settings for garbage collection of binaries has been adjusted to be less aggressive than in R13B03. It is now also possible configure the settings for binary GC. See the documentation for spawn_opt/2-5, erlang:system_info/1, erlang:system_flag/2, process_flag/2-3, erlang:trace/3, and the documenation for erl for the new command line options +hms and +hmbs.

Own Id: OTP-8370

A bug causing memory corruption in re:run() has been corrected. (Thanks to Yamashina Hio.)

Own Id: OTP-8375

The -Werror option for erlc and the compiler option warnings_as_errors will cause warnings to be treated as errors. (Thanks to Christopher Faulet.)

Own Id: OTP-8382

Explicit top directories in archive files are now optional.

For example, if an archive (app-vsn.ez) just contains an app-vsn/ebin/mod.beam file, the file info for the app-vsn and app-vsn/ebin directories are faked using the file info from the archive file as origin. The virtual direcories can also be listed. For short, the top directories are virtual if they does not exist.

Own Id: OTP-8387

An unnecessary lock operation in os:timestamp/0 has been eliminated, making it slightly more efficient. (Thanks to Jonas Falkevik and Tuncer Ayaz.)

Own Id: OTP-8390

There is a new +t emulator option for changing the maximum number of atoms. (Thanks to Julien Barbot.)

Own Id: OTP-8405

Fixed numerous compiler warnings generated by gcc 4.4.1 and tile-cc 2.0.1.78377 when compiling the runtime system.

Own Id: OTP-8412

configure learned the option --enable-m32-build to force the building of a 32-bit run-time on systems where the default C compiler generates 64-bit executables by default.

Own Id: OTP-8415

HiPE now works in the 64-bit emulator on Mac OS X. (Thanks to Geoff Cant.)

Own Id: OTP-8416

Improved handling of symbolic links to escripts

Own Id: OTP-8417

Add lock profiling tool.

The Lock profiling tool, lcnt, can make use of the internal lock statistics when the runtime system is built with this feature enabled.

This provides a mechanism to examine potential lock bottlenecks within the runtime itself.

- Add erts_debug:lock_counters({copy_save, bool()}). This option enables or disables statistics saving for destroyed processes and ets-tables. Enabling this might consume a lot of memory.

- Add id-numbering for lock classes which is otherwise undefined.

Own Id: OTP-8424

Removed spurious options to the emulator from escript.

Own Id: OTP-8427

Minor documentation fixes. Mainly anchor adjustments.

Own Id: OTP-8457

Erts 5.7.4

Fixed Bugs and Malfunctions

An insufficient stack allocation was made when reading CPU information on BSD operating systems. (Thanks Michael Turner and Akira Kitada)

Own Id: OTP-8207

A bug when supplying an argument without a dash directly after the program name when starting erlang could prevent distribution to start. This is now corrected.

Own Id: OTP-8209

A ticker process could potentially be blocked indefinitely trying to send a tick to a node not responding. If this happened, the connection would not be brought down as it should.

Own Id: OTP-8218

Using certain firewalls (i.e. MS IAS Client and certain versions of COMODO) could expose an undocumented behaviour in the Win32 socket interface causing the name resolution calls to hang infinitely. This is now worked around by adding possibilities for port programs under Windows to use overlapped I/O on their standard input/output file handles.

Own Id: OTP-8230

Fixed bug on ETS tables with write_concurrency. The emulator could crash when doing a select or match with a bound key without finding any object.

Own Id: OTP-8242

The information-request / information-response, and group-leader-change-request / group-leader-changed-response signal pairs described below did not always adhere to the signal order guarantees of Erlang's signal model in the runtime system with SMP support. These signals could for example sometimes pass exit signals.

The following BIFs behaviors can be modeled as if an asynchronous information-request signal is sent to Pid. If Pid is alive, it responds with an asynchronous information-response signal; otherwise, the runtime system responds with a no-such-process signal. When the response is received, the caller transforms it into the result of the BIF.

is_process_alive(Pid) erlang:process_display(Pid, Type) process_info(Pid) process_info(Pid, ItemSpec)

When Pid resides on the same node as the caller of group_leader(GroupLeader, Pid), the group_leader/2 BIFs behavior can be modeled as if an asynchronous group-leader-change-request signal is sent to Pid. If Pid is alive, it responds with an asynchronous group-leader-changed-response signal; otherwise, the runtime system responds with a no-such-process signal. When the response is received, the caller transforms it into the result of the BIF. The distributed case which only consists of an asynchronous group-leader-change-request signal and no response is not effected.

Own Id: OTP-8245

Errors in the system_profile documentation has been corrected.

Own Id: OTP-8257

Low watermark socket option modified high watermark instead of low watermark in the inet_driver. (Thanks to Feng Yu and Tuncer Ayaz)

Own Id: OTP-8279

A race condition could cause the runtime system with SMP support to end up in a completely unresponsive state.

Own Id: OTP-8297

Improvements and New Features

The use of pthread_cond_timedwait() have been completely removed from the runtime system. This since its behavior is unpredictable when the system clock is suddenly changed. The previous use of it was harmless.

Own Id: OTP-8193

The documentation is now built with open source tools (xsltproc and fop) that exists on most platforms. One visible change is that the frames are removed.

Own Id: OTP-8201

A new garbage collecting strategy for binaries which is more aggressive than the previous implementation. Binaries now has a virtual binary heap tied to each process. When binaries are created or received to a process it will check if the heap limit has been reached and if a reclaim should be done. This imitates the behavior of ordinary Erlang terms. The virtual heaps are grown and shrunk like ordinary heaps. This will lessen the memory footprint of binaries in a system.

Own Id: OTP-8202

The ErlDrvTermData term types used by driver_output_term() and driver_send_term() have been extended with the term types ERL_DRV_INT64, and ERL_DRV_UINT64 for passing 64-bit integers. Also the 64-bit integer data types ErlDrvSInt64 and ErlDrvUInt64 have been introduced.

For more information see the erl_driver(3) documentation.

Own Id: OTP-8205

[escript] The restriction that the first line in escripts must begin with #! has been removed.

[escript] Some command line options to the escript executable has now been documented. For example you can run an escript in the debugger by just adding a command line option.

[escript] The documentation of the escript header syntax has been clarified. For example the header is optional. This means that it is possible to directly "execute" .erl, .beam and.zip files.

Own Id: OTP-8215

The instruction for building OTP on Windows was outdated and incomplete, the document is updated.

Also the otp_build script required windows drives to show up in Cygwin using the /cygdrive prefix. That requirement is now removed.

Own Id: OTP-8219

A module can have native implemented functions (NIFs) that are dynamically loaded by calling erlang:load_nif/2. This is an experimental feature that is not yet intended for production systems. It is released with intention to get some early feedback on the interfaces before they are carved in stone.

Own Id: OTP-8220

The float/1 BIF would always force a garbage collection. The BIFs size/1, byte_size/1, bit_size/1, abs/1, and round/1 would force a garbage-collection if the result was not a sufficiently small integer.

Own Id: OTP-8221

The erlang:port_command/3 BIF has been added. erlang:port_command/3 is currently not auto imported, but it is planned to be auto imported in OTP R14. For more information see the erlang(3) documentation.

Own Id: OTP-8225

'configure --enable-darwin-64bit' would fail if Snow Leopard had been booted with the 64-bit kernel. (Thanks to Ryan M. Graham.)

Own Id: OTP-8236

Erts 5.7.3

Fixed Bugs and Malfunctions

On Windows, open_port({spawn,Command},Opts) could not run executables with spaces in the path or filename, regardless of quoting efforts. While open_port({spawn_executable,Exec},Opts) can run any executable, it was still impossible to use 'spawn' to do the same thing. This is now corrected.

Own Id: OTP-8055

The scheduler bind type processor_spread spread schedulers too much on large NUMA systems.

The new scheduler bind type spread spreads schedulers as much as possible, and behaves as processor_spread previously did. For more information see the documentation of the +sbt command line argument in the erl(1) documentation, and the documentation of erlang:system_flag(scheduler_bind_type, SchedulerBindType).

Own Id: OTP-8063

Automatically detected CPU topology on Linux system could erroneously contain logical processors with -1 as identifiers. This happened when sysconf(_SC_NPROCESSORS_CONF) returned a value larger than the amount of logical processors found.

Own Id: OTP-8064

When the minimal term [] (end of list) was sent as the complete message to a process on another node, and received there, it could not be decoded. This bug is now corrected. Fortunately [] is uncommon as the complete message in real applications but it is a serious bug anyway.

Own Id: OTP-8092

A bug when the floating point exception pointer was not initialized has been corrected. It manifested itself on CentOS 5.1 sometimes when a floating point value was sent to a remote node. Bug reported and patch suggested by David Reiss, confirmed by Mikael Pettersson.

Some build problems on IRIX was also corrected. Problem reported by Patrick Baggett, patch by Mikael Pettersson.

Own Id: OTP-8095

A terminating process could erroneously unregister a name for another process. This could occur under the following conditions: The name of the terminating process was unregistered and then registered for another process simultaneously as the process that first had the name was terminating.

Own Id: OTP-8099 Aux Id: seq11344

Running erlc in a very deep directory (with a path length of more 256 or more characters) would cause the emulator to crash in a call to list_to_atom/1. (Thanks to Chris Newcombe.)

Own Id: OTP-8124

A deadlock of the runtime system could occur when unregistering the name of a port.

Own Id: OTP-8145

Makefile.in has been updated to use the LDFLAGS environment variable (if set). (Thanks to Davide Pesavento.)

Own Id: OTP-8157

The pthread rwlock implemention on Linux could cause starvation of writers. We, therefore, now use our own rwlock implementation on Linux.

Own Id: OTP-8158

Open source Erlang builds are updated to work well on Snow Leopard (MacOS X 10.6)

Own Id: OTP-8168

A call to erlang:system_info(schedulers_online) could end up in an infinite loop. This happened if the amount of schedulers was larger than one, the amount of schedulers online was one, and someone was blocking multi-scheduling.

Own Id: OTP-8169

An error in erlang:system_profile/2 could cause timestamped messages to arrive out of order in the SMP case. This has now been fixed.

Own Id: OTP-8171

binary_to_atom/2 and binary_to_existing_atom/2 would leak memory if the binary contained unaligned data.

Own Id: OTP-8192

The async thread pool in the runtime system without SMP support used a memory allocator that was not thread safe for async jobs.

Own Id: OTP-8194

Improvements and New Features

Processor internal NUMA nodes are now supported in the ERTS internal CPU topology representation. For more information see the documentation of the +sct command line argument in the erl(1) documentation, and the documentation of erlang:system_info(cpu_topology).

Own Id: OTP-8041

Documentation for ets improved about concurrency.

Own Id: OTP-8050

Emulator flags in an escript were earlier inherited to emulators started from from the emulator running the escript. For example when an escript invoked os:cmd("erl"), the new emulator were given erroneous emulator flags. This bug has now been fixed

Escript filenames may now contain dots.

Own Id: OTP-8060

Made some BIFs non-recursive (relational operators,hash and phash) to limit internal stack usage.

Own Id: OTP-8065

Fixed Windows specific bug in erl_prim_loader. Now it handles the root directory (e.g. c:/) better. This bug affected the directory listing in the debugger.

Own Id: OTP-8080

A TCP socket with option {packet,4} could crash the emulator if it received a packet header with a very large size value (>2Gb). The same bug caused erlang:decode_packet/3 to return faulty values. (Thanks to Georgos Seganos.)

Own Id: OTP-8102

The maximum size of the export table has been raised from 65536 to 524288 entries.

Own Id: OTP-8104 Aux Id: seq11345

The file module has now a read_line/1 function similar to the io:get_line/2, but with byte oriented semantics. The function file:read_line/1 works for raw files as well, but for good performance it is recommended to use it together with the 'read_ahead' option for raw file access.

Own Id: OTP-8108

Fixed bug causing emulator crash when reading a term in external format containing a corrupt list with a negative length.

Own Id: OTP-8117

New emulator flag +sss, to set stack size of scheduler threads.

Own Id: OTP-8119

The Windows utility Erlsrv, run in interactive mode now accepts options for registering internal service name and description field of Windows registry database.

Own Id: OTP-8132

erlang:demonitor(Mon, [flush]) has been optimized. Previously it always searched the message queue of the caller for a 'DOWN' message. Current implementation only search the message queue when necessary. It is quite common that the search is not necessary.

A new option info has been added to erlang:demonitor/2. For more information see the erlang(3) documentation.

Own Id: OTP-8143

I/O tasks could unnecessarily be rescheduled. This was harmless, but not useful work.

Own Id: OTP-8148

Minor improvements of erlang:memory/[1,2].

Own Id: OTP-8152

New configuration option to enable use of shared zlib.

Own Id: OTP-8155

Fixed smp bug in ETS that could cause emulator crash when table with more than 1000 objects accessed by several processes, including calls to variants of select or match combined with concurrent object deletion.

Own Id: OTP-8166 Aux Id: seq11392

The code path interpretation is now more relaxed. The flag -code_path_choice now defaults to relaxed instead of strict. See the documentation of code and init for more info.

Own Id: OTP-8170

Load balancing of run queues and check for I/O are triggered more often than before in situations where processes are scheduled often but are doing very little work each time they execute.

Own Id: OTP-8172

Call tracing binary comprehensions would cause an emulator crash. (Thanks to Paul Mineiro.)

Own Id: OTP-8179

binary_to_term/1 would crash the emulator instead of generating a badarg exception when given certain invalid terms. (Thanks to Scott Lystig Fritchie.)

Own Id: OTP-8180

Erts 5.7.2

Fixed Bugs and Malfunctions

Crash dumps should now cause less problems for the crashdump_viewer application. (For processes where arity was non-zero, the arguments are now longer printed - they used to be printed in a format that was not parseable.)

Own Id: OTP-7472 Aux Id: seq11019, 11292

Processes could potentially get stuck on an offline scheduler.

Own Id: OTP-7990

binary_to_atom/2 and binary_to_existing_atom/2 could leak memory if they caused a badarg exception.

Own Id: OTP-7997

A process could under very rare circumstances erroneously be resumed.

Own Id: OTP-8000

Load balancing between schedulers could under rare circumstances cause an emulator crash.

Own Id: OTP-8008

erlang:memory(processes_used) always returned 0 instead of the correct value. (Thanks to Geoff Cant)

Own Id: OTP-8022

Improvements and New Features

Major improvements of the Erlang distribution for Erlang runtime systems with SMP support. Previously distribution port locks were heavily contended, and all encoding and decoding for a specific distribution channel had to be done in sequence. Lock contention due to the distribution is now negligible and both encoding and decoding of Erlang messages can be done in parallel.

The old atom cache protocol used by the Erlang distribution has been dropped since it effectively prevented all parallel encoding and decoding of messages passed over the same distribution channel.

A new atom cache protocol has been introduced which isolates atom cache accesses and makes parallel encoding and decoding of messages passed over the same distribution channel possible. The new atom cache protocol also use an atom cache size 8 times larger than before. The new atom cache protocol is documented in the ERTS users guide.

Erlang messages received via the distribution are now decoded by the receiving Erlang processes without holding any distribution channel specific locks. Erlang messages and signals sent over the distribution are as before encoded by the sending Erlang process, but now without holding any distribution channel specific locks during the encoding. That is, both encoding and decoding can be and are done in parallel regardless of distribution channel used.

The part that cannot be parallelized is the atom cache updates. Atom cache updates are therefore now scheduled on the distribution port. Since it is only one entity per distribution channel doing this work there is no lock contention due to the atom cache updates.

The new runtime system does not understand the old atom cache protocol. New and old runtime systems can however still communicate, but no atom cache will be used.

Own Id: OTP-7774

Fixed a bug that caused error logging from driver_select sometimes with additional symptoms such as failing IP communications or even an emulator crash.

Own Id: OTP-7898 Aux Id: seq11304

Improved SMP concurrency for ETS tables. Several mutating operations can now be performed truly concurrent on different records of the same table. To support this, the table has to be created with option write_concurrency, as it is achieved at the expense of some execution and memory overhead. ets:select and select_count has also been improved for all tables to not acquire exclusive table lock during the iteration.

Own Id: OTP-7922

erl (that is erl.exe and dyn_erl) and erlexec has been made more dynamic so no hard coded paths needs to added at installation time to erl (that is erl.ini and erl). Reltool will make use of this in a future release.

Own Id: OTP-7952

Added functionality to get higher resolution timestamp from system. The erlang:now function returns a timestamp that's not always consistent with the actual operating system time (due to resilience against large time changes in the operating system). The function os:timestamp/0 is added to get a similar timestamp as the one being returned by erlang:now, but untouched by Erlangs time correcting and smoothing algorithms. The timestamp returned by os:timestamp is always consistent with the operating systems view of time, like the calendar functions for getting wall clock time, but with higher resolution. Example of usage can be found in the os manual page.

Own Id: OTP-7971

Two new options are added to open_port - spawn_executable which runs external executables in a controlled way, and spawn_driver which only opens port to loaded Erlang drivers. See the erlang manual page for details.

Own Id: OTP-7995

New functionality in ETS to transfer the ownership of a table. A table can either change owner be declaring an "heir", another process that will inherit the table if the owner terminates. A table can also change owner by calling a new function ets:give_away.

Own Id: OTP-8006

Updates to Tilera build environment.

Own Id: OTP-8009

A stack trace was unnecessarily saved during process termination.

Own Id: OTP-8014

User defined CPU topology and scheduler bind type can now be set from the command line when starting an emulator. For more information see the documentation of the +sct, and the +sbt emulator flags in the erl(1) documentation.

The CPU topologies returned from erlang:system_info/1 and erlang:system_flag/2 now always contain the processor level, also when not strictly necessary.

Own Id: OTP-8030

Various fixes in ETS: ets:first could return a deleted key in a fixated table. ets:lookup could return objects out of order if a deleted object was re-inserted into a fixed bag. ets:delete_object could fail to delete duplicate objects in a duplicate_bag.

Own Id: OTP-8040

Erts 5.7.1

Fixed Bugs and Malfunctions

Fixed a bug on Windows that could make gen_tcp:send hang trying to send an iolist of more than 16 binaries.

Own Id: OTP-7816

The runtime system could under rare circumstances crash during load balancing.

Own Id: OTP-7908 Aux Id: otp-7500

run_erl uses fallback if Unix98 pseudo-terminal is not present on host.

Own Id: OTP-7916 Aux Id: seq11249

A message buffer memory leak in the runtime system without smp support has been fixed.

Own Id: OTP-7941

Attempting to append a binary of 16Mb or greater to another binary using the bit syntax would cause a system_limit exception. There was also several cases when constructing binaries when a badarg exception was generated when it should have been system_limit.

Own Id: OTP-7942

The runtime system with SMP support failed to terminate the caller of link(RemotePid) properly, if RemotePid was the pid of a process on an unreachable node. The calling process was in this case marked as exiting, but never terminated.

Own Id: OTP-7946

Improvements and New Features

Rudimentary support for cross compiling is added to the source release. The support is still in its infancy and has only been used to cross compile on Linux for a different cpu architecture and a different Linux version, but should be extendible to support other platforms as well. The cross configuration files with examples are placed in $ERL_TOP/xcomp/. View README.xcomp and run $ERL_TOP/otp_build -help for further information.

Own Id: OTP-7854

The escape sequence \{ which was given a new interpretation in R13A has retained its old meaning (the ASCII code for {), which means that codes greater than 255 have to be stated using hexadecimal characters (for example, \x{AAA}). The escape sequence \xH where H is a hexadecimal character followed by something else but a hexadecimal character is no longer valid (incompatibility with R13A). Character codes less than 256 can be stated using two hexadecimal characters (for example, \x0D).

Own Id: OTP-7891 Aux Id: OTP-7855

The term_to_binary/1 BIF used to be implemented with recursive C code, which could cause the Erlang emulator to terminate because of a stack overflow.

Also fixed some minor issues in term_to_binary/1 and binary_to_term/1 pointed out by Matthew Dempsky.

Own Id: OTP-7894

Several glitches and performance issues in the Unicode and I/O-system implementation of R13A have been corrected.

Own Id: OTP-7896 Aux Id: OTP-7648 OTP-7887

Minor documentation improvements of the scheduler_bind_type argument of erlang:system_flag/2, and the scheduler_bind_type, and the scheduler_bindings arguments of erlang:system_info/1.

Own Id: OTP-7901 Aux Id: OTP-7777

There is a new BIF erlang:make_tuple/3.

Own Id: OTP-7913

Erts 5.7

Fixed Bugs and Malfunctions

OpenSource:

FreeBSD leap-seconds are handled according to patch submitted by OpenSource user Kenji Rikitake. No test case covers this functionality (unsupported platform).

Own Id: OTP-7609

A corrected bug in ets for bag and duplicate_bag. A delete/2 or lookup_element/3 could miss objects in a fixed table if one or more objects with the same key had already been deleted.

Own Id: OTP-7665

A new driver call-back stop_select is introduced to allow drivers to de-select and then close a file descriptor in a safe way in a SMP emulator. The old way was not strictly according to posix standard and could in some rare cases lead to unexpected behavior. A new flag ERL_DRV_USE can be passed to driver_select() to tell it that the descriptor should be closed. stop_select is then called when it is safe to do so. Old drivers will however still work as before.

Own Id: OTP-7670

A bug fixed for TCP sockets with option {packet,http}. An HTTP request with an absolute URI was returned with a corrupt path string. This bug did only exist in R12B-4 and R12B-5.

Own Id: OTP-7682 Aux Id: OTP-7647

run_erl did in some cases fail to extract control sequences from to_erl (like: winsize=X,Y) and did instead send them to be interpreted by the erlang shell.

Own Id: OTP-7688

A bug in the installer on Windows not updating file associations properly is now corrected.

Own Id: OTP-7746

More space than necessary could be allocated in binaries when appending to a binary (also in a binary comprehension) and the data appended did not consist of wholes bytes (e.g. 13 bits).

Own Id: OTP-7747

The gen_sctp option sctp_peer_addr_params, #sctp_paddrparams{address={IP,Port} was erroneously decoded in the inet driver. This bug has now been corrected.

Own Id: OTP-7755

Outstanding async driver jobs leaked memory if the issuing port died before the async jobs completed.

Own Id: OTP-7784

A bug in the dynamic library loading affecting, among others, OpenSolaris is now corrected. (Thanks to Paul Fisher.)

Own Id: OTP-7796

run_erl compile errors fixed for FreeBSD

Own Id: OTP-7817

A bug in the inet driver for SCTP on Solaris showing for e.g gen_sctp:abort/1 and gen_sctp:eof/1 has been corrected. Patch suggestion by Simon Cornish.

Own Id: OTP-7866

Improvements and New Features

The order of objects visited in select for ordered_set is now documented.

Own Id: OTP-7339

The runtime system with SMP support now uses multiple, scheduler specific run queues, instead of one globally shared run queue.

The lock protecting the shared run queue was heavily contended, and the shared run queue also caused Erlang processes to randomly migrate between schedulers with negative cache effects as a result.

With the current scheduler specific run queue solution, lock contention due to run queue protection has been reduced, and Erlang processes are only migrated when needed to balance the load between the schedulers. The reduced amount of migration also reduce lock contention on locks protecting the scheduler specific instances of the erts internal memory allocators.

The scheduler specific run queues are also a necessity for a lot of future planned NUMA (Non-Uniform Memory Access) specific optimizations.

Own Id: OTP-7500

Support for Unicode is implemented as described in EEP10. Formatting and reading of unicode data both from terminals and files is supported by the io and io_lib modules. Files can be opened in modes with automatic translation to and from different unicode formats. The module 'unicode' contains functions for conversion between external and internal unicode formats and the re module has support for unicode data. There is also language syntax for specifying string and character data beyond the ISO-latin-1 range.

The interactive shell will support input and output of unicode characters when the terminal and operating system supports it.

Please see the EEP and the io/io_lib manual pages as well as the stdlib users guide for details.

I/O-protocol incompatibilities:

The io_protocol between io_Server and client is updated to handle protocol data in unicode formats. The updated protocol is now documented. The specification resides in the stdlib users manual, which is a new part of the manual.

io module incompatibilities:

The io:put_chars, io:get_chars and io:get_line all handle and return unicode data. In the case where binaries can be provided (as to io:put_chars), they shall be encoded in UTF-8. When binaries are returned (as by io:get_line/get_chars when the io_server is set in binary mode) the returned data is also always encoded as UTF-8. The file module however still returns byte-oriented data, why file:read can be used instead of io:get_chars to read binary data in ISO-latin-1.

io_lib module incompatibilities:

io_lib:format can, given new format directives (i.e "~ts" and "~tc"), return lists containing integers larger than 255.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7648 Aux Id: OTP-7580 OTP-7514 OTP-7494 OTP-7443 OTP-7181 EEP10 EEP11

The format of the string returned by erlang:system_info(system_version) (as well as the first message when Erlang is started) has changed. The string now contains the both the OTP version number as well as the erts version number.

Own Id: OTP-7649

Message passing has been further optimized for parallel execution. Serial message passing is slightly more expensive than before, but parallel send to a common receiver is much cheaper.

Own Id: OTP-7659

Lock contention on the atom table lock when decoding Erlang terms on the external format has been drastically reduced.

Own Id: OTP-7660

The undocumented, unsupported, and deprecated guard BIF is_constant/1 has been removed.

*** INCOMPATIBILITY with R12B ***

Own Id: OTP-7673

The Erlang process lock implementation has been improved by Mat Hostetter at Tilera Corporation.

Own Id: OTP-7692

A {nodedown, Node} message passed by the net_kernel:monitor_nodes/X functionality is now guaranteed to be sent after Node has been removed from the result returned by erlang:nodes/Y.

Own Id: OTP-7725

The short-circuit operators andalso and orelse no longer guarantees that their second argument is either true or false. As a consequence, andalso/orelse are now tail-recursive.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7748

A new BIF, lists:keyfind/3, has been added. It works like lists:keysearch/3 except that it does not wrap the returned tuple in a value tuple in case of success. (Thanks to James Hague for suggesting this function.)

Own Id: OTP-7752

Optimization for drivers by creating small binaries direct on process heap.

Own Id: OTP-7762

I bsl N could cause the Erlang virtual machine to run of memory instead generating a system_limit if N was absurdly huge. (Thanks to Daniel Hedlund.)

There would always be a garbage collection when evaluating I bsl N or I bsr N if I was a bignum.

If I is an integer and N a bignum, I bsl N will now cause the correct system_limit exception instead of bad_arith as in earlier releases.

If I is an integer and N a bignum, I bsr N will return either 0 or -1 depending on the sign of I instead of causing a bad_arith exception as in earlier releases.

Own Id: OTP-7764

Scheduler threads can now be bound to logical processors on newer Linux and Solaris systems. More systems will be supported in the future.

In some cases performance has increased drastically when binding schedulers. Schedulers are not bound by default, though. This since it might cause a performance degradation if multiple programs have bound to processors, e.g. multiple Erlang runtime systems. For more information see the documentation of erlang:system_flag/2.

In order to bind scheduler threads the CPU topology need to be known. On some newer Linux and Solaris systems the runtime system automatically detects the CPU topology. If the emulator isn't able to automatically detect the CPU topology, the CPU topology can be defined. For more information see the documentation of erlang:system_flag/2.

Own Id: OTP-7777

The compiler will refuse to a compile file where the module name in the file differs from the output file name.

When compiling using erlc, the current working directory will no be included in the code path (unless explicitly added using "-pa .").

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7793

The BIFs atom_to_binary/2, binary_to_atom/2, and binary_to_existing_atom/2 have been added.

Own Id: OTP-7804

The amount of schedulers online can now be changed during operation. The amount of schedulers online defaults to the same amount as available logical processors. For more information see the documentation of erlang:system_flag/2 and erl.

Own Id: OTP-7811

The deprecated functions erlang:fault/1, erlang:fault/2, and file:rawopen/2 have been removed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7812

Erts internal dynamically allocated process and port specific data for rarely used data. This is used to reduce memory usage of processes and ports that do not use specific functionality. More functionality will be moved to process and port specific data in future releases.

Own Id: OTP-7818

New packet type http_bin for gen_tcp sockets and erlang:decode_packet. It works like http except that strings are returned as binaries instead of lists.

Own Id: OTP-7821

The obsolete wd_keeper program for embedded Solaris systems has been removed.

Own Id: OTP-7822

Nodes belonging to different independent clusters can now co-exist on the same host with the help of a new environment variable setting ERL_EPMD_PORT.

Own Id: OTP-7826

There are new functions erlang:min/2 and erlang:max/2 to calculate the minimum and maximum of two terms, respectively. Note that the functions are not auto-imported, so they need to be imported explicitly or the erlang prefix must be used when calling them.

Own Id: OTP-7841

The copyright notices have been updated.

Own Id: OTP-7851

Enhanced build environment for cross compilation to Tilera Tile architecture.

Support for native ethread atomics on Tilera Tile64/TilePro (Thanks to Tilera Corporation).

Own Id: OTP-7852

The escape sequences \x and \{ have been assigned new interpretations (they used to return the ASCII code for x and { respectively). One or more octal characters inside curly brackets after a leading backslash is from now on an alternative to the existing syntax \NNN, but can also be used for codes greater than 255. In a similar fashion, one or more hexadecimal characters can be put inside curly brackets after a leading \x. Furthermore, the escape sequences \xH and \xHH, where N is a hexadecimal character, can be used for codes less than 256.

NOTE: These new escape sequences are still considered experimental and may be changed in the R13B release.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7855

The PCRE library's exported function names are now prefixed with erts_ in the erlang emulator to avoid clashes with dynamically loaded drivers.

Own Id: OTP-7861

A runtime system with SMP support will now be built by default on most platforms if a usable posix thread library or native windows threads are found.

For more information see the top README file.

Own Id: OTP-7872

Erts 5.6.5

Fixed Bugs and Malfunctions

A bug in inet_drv concerning gen_tcp:connect has been corrected. A connect towards a non-open port through open firewalls could sometimes erroneously be successful. Any subsequent operation would fail, though.

Own Id: OTP-6542

Floating point arithmetics in drivers could cause a runtime system crash and/or unexpected results on runtime systems with floating point exceptions enabled. Floating point exceptions are disabled unless explicitly enabled or if hipe is enabled.

Own Id: OTP-7237

A bug when many sockets got signalled simultaneously causing the emulator to panic with the message "Inconsistent, why isnt io reported?" is now corrected.

Own Id: OTP-7420

Starting erl with option "-detached" now disconnects correctly from terminal session on Unix.

Own Id: OTP-7461

Mended gdb etp-commands for ETS access.

Own Id: OTP-7538

erlang:decode_packet/3 allows white space between HTTP header tag and colon according to RFC2616.

Own Id: OTP-7543

An emulator compiled for SCTP now starts even if the dynamic libraries are not present. The SCTP driver is then of course not loaded.

Own Id: OTP-7551

To build on Mac OS X, 10.3.0 or later is now required because of fixes for two problems:

There would be a resource leak when erl_ddl attempted to unload a driver. This problem has been corrected by using dlopen() (which works on all modern Unix-like platforms) instead of the Mac OS X specific API calls.

Signal handling in the run-time system for HiPE has been updated to work on later versions of Mac OS X than 10.2.x. Therefore, --enable-hipe now works on Mac OS X with Intel CPUs.

Thanks to Geoff Cant for the patches.

Own Id: OTP-7562

Corrected some information about the protocol between EPMD and Erlang nodes. (Thanks to Michael Regen.)

Own Id: OTP-7594

When using erlang:system_monitor(Pid,{long_gc,Time}), and the GC time exceeded 1 second, it sometimes erroneously showed up as about 4300 seconds. (This bug was corrected in R9C, but re-introduced in R12B.) (Thanks to Chris Newcombe.)

Own Id: OTP-7622 Aux Id: OTP-4903, seq8379

Improvements and New Features

The driver entry of a dynamically loaded driver is now copied when loaded which enables some internal optimizations. Note that drivers that modify the driver entry during execution will not work anymore. Such a miss-use of the driver interface is however not supported.

Own Id: OTP-6900

The split function is now added to the re library. Exceptions and errors from both run, replace and split are made more consistent.

Own Id: OTP-7514 Aux Id: OTP-7494

Fixed harmless compiler warnings when building the emulator and minor build changes in order to avoid unnecessary rebuilds.

Own Id: OTP-7530

There is now experimental support for loading of code from archive files. See the documentation of code, init, erl_prim_loader and escript for more info.

The error handling of escripts has been improved.

An escript may now set explicit arguments to the emulator, such as -smp enabled.

An escript may now contain a precompiled beam file.

An escript may now contain an archive file containing one or more applications (experimental).

The internal module code_aux has been removed.

Own Id: OTP-7548 Aux Id: otp-6622

The reallocation functionality part of the ERTS internal memory allocators, now consider current block in combination with surrounding free blocks as an alternative location for a reallocation.

Own Id: OTP-7555

There could remain false references from a process to a module that has been called earlier, so that the process would be killed if the module was reloaded. (Thanks to Richard Carlsson.)

Also, the fix for this bug also made it possible to make stack backtraces (as returned from erlang:get_stacktrace/0 and other functions) more correct in that the immediate caller is always included in the stack backtrace (it could sometimes be missing).

Own Id: OTP-7559

Improved locking in IO-handling for better smp performance.

Own Id: OTP-7560

Improved BIF rescheduling functionality.

Own Id: OTP-7587

Loading a module compiled with Erlang/OTP R9C and calling module_info/0 in the module would crash the emulator. The emulator now refuses to load any module compiled with R9C or earlier. (Note: only trivial modules compiled with R10B or earlier could be loaded anyway.) (Thanks to Martin Kjellin.)

Own Id: OTP-7590

Erts 5.6.4

Fixed Bugs and Malfunctions

Double backslashes in format string passed to the erts internal printf implementation produced erroneous results. No such format strings were passed to the erts internal printf implementation, i.e., the bug was therefore completely harmless. (Thanks to Perry Smith.)

Own Id: OTP-7408

Large files are now handled on Windows, where the filesystem supports it.

Own Id: OTP-7410

Bug fixed for {packet,http} when space follows http headers.

Own Id: OTP-7458

The trace option running could cause an emulator crash if the current function couldn't be determined.

Own Id: OTP-7484

Using 16#ffffFFFF as a timeout value in receive...after would often cause a timeout almost at once due to an 32-bit integer overflow. (Thanks to Serge Aleynikov and Matthias Lang.)

Own Id: OTP-7493

For the process that an escript runs in, the trap_exit process flag is now false instead of true (as in previous releases). Scripts that depend on the previous (counter-intuitive) behaviour might not work. (Thanks to Bengt Kleberg.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7517

A bug in the string:to_integer/1 builtin made the emulator unstable. This is now corrected. (Thanks to Lev Walkin.)

Own Id: OTP-7526

Improvements and New Features

Performance for ETS intensive applications should now be better in the SMP emulator. Also, ETS table identifiers (as returned by ets:new/2) are now spread out in wider range than before (using 28 bits in a 32-bit emulator) to make sure that the table identifier for a deleted table will not be quickly re-used.

NOTE: Table identifiers can now be negative integers. Programs that (incorrectly) assume that table identifiers can only be positive integers might stop to work. (The type of a table identifier is intentionally not documented, and may change in a future release.)

Own Id: OTP-7348

New BIF erlang:decode_packet/3 that extracts a protocol packet from a binary. Similar to the socket option {packet, Type}. Also documented the socket packet type http and made it official. NOTE: The tuple format for http packets sent from an active socket has been changed in an incompatible way.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7404

The source code for the documentation for some applications (erts, kernel, stdlib, and several others) are now included in the source tar ball. There is currently no Makefile support for building HTML files from the source (such support will be included in a future release).

Own Id: OTP-7406

A lot of frequently accessed memory counters (erts internal) have been removed. This since they hurt performance on the runtime system with SMP support. As a result erlang:memory/[0,1] will only deliver a result if all erts_alloc(3) allocators are enabled (default). The result delivered when all erts_alloc(3) allocators are enabled are both more accurate and less accurate than before. More memory than before are included in the result, but the different parts that are summed are not gathered atomically. A call to erlang:memory/[0,1] is much cheaper for the system than before. This since the information isn't gathered atomically anymore which was very expensive.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7468

Pre-allocators used for, for example, timers, and messages have been rewritten to be scheduler specific. That is, different schedulers will now allocate from different pools which reduces lock contention.

Own Id: OTP-7470

On Mac OS X, file:sync/1 now guarantees that all filesystem buffers are written to the disk by using the fcntl() with F_FULLFSYNC option. Previously, file:sync/1 called fsync(), which only guaranteed that the data had been transferred to the disk drive. (Thanks to Jan Lehnardt.)

Own Id: OTP-7471

Termination of a process that takes a long time can now be preempted, i.e., the terminating process will be rescheduled for later continuation of termination so that other processes can execute. Termination of a process that owns many and/or large ets tables typically takes a long time.

Own Id: OTP-7477

A new trace option exiting has been added. The exiting trace is similar to the running trace, but for exiting processes. For more information see the erlang(3) documentation.

The erlang:trace/3 bif now doesn't block other scheduler threads if only one tracee is specified in the call to erlang:trace/3.

Own Id: OTP-7481

The re module is extended with repetitive matches (global option) and replacement function.

Own Id: OTP-7494 Aux Id: OTP-7181

In the section about binary construction, the reference manual now mentions what happens when an integer value does not fit into an integer segment of size N (namely, that the N least significant bits will be put into into the binary and that the most significant bits will be silently discarded). (Thanks to Edwin Fine.)

Own Id: OTP-7506

Setting the {active,once} for a socket (using inets:setopts/2) is now specially optimized (because the {active,once} option is typically used much more frequently than other options).

Own Id: OTP-7520

Known Bugs and Problems

Floating point arithmetics in drivers can cause a runtime system crash and/or unexpected results on runtime systems with floating point exceptions enabled. Floating point exceptions are disabled unless explicitly enabled or if hipe is enabled.

Own Id: OTP-7237

Erts 5.6.3

Fixed Bugs and Malfunctions

Only one to_erl process at a time is allowed to connect to the same run_erl pipe. Prevents buggy behaviour when IO from several to_erl's get interleaved.

Own Id: OTP-5107

IPv6 name resolving has now been fixed to use getaddrinfo() patch (thoroughly reworked) courtesy of Love Hörnquist-Åstrand submitted by Fredrik Thulin. It also can use gethostname2() patch (also reworked) courtesy of Mikael Magnusson for debian submitted by Sergei Golovan.

Own Id: OTP-5382

Improved error handling in run_erl

Own Id: OTP-7252

A permanent fix for the deadlock issue temporarily fixed by OTP-7260.

OTP-7260 The runtime system with SMP support could under rare circumstances deadlock when a distribution channel was taken down while multiple simultaneous operations were performed on it.

Own Id: OTP-7267 Aux Id: OTP-7260

./configure has been improved to find 64-bit OpenSSL libraries.

Own Id: OTP-7270

A terminating process could under very rare circumstances trigger a bug which could crash the runtime system with SMP support.

Own Id: OTP-7272

SCTP_ADDR_CONFIRMED events are now handled by gen_sctp.

Own Id: OTP-7276

binary_to_term/1 would crash the emulator if the binary data contained an external fun with non-atom module and/or function. Corrected to generate a badarg exception.

Own Id: OTP-7281

On Mac OS 10.5 (Leopard), sending to socket which the other end closes could cause the emulator to consume 100% CPU time. (Thanks to Matthias Radestock.)

Own Id: OTP-7289

The vanilla driver used on Windows could crash the emulator and sometimes produced corrupt files. The vanilla driver is the driver that is used when one only pass a filename as first argument to open_port/2. NOTE: This use of open_port/2 is obsolete, and the documentation of this use has previously been removed. The functionality is only present for backward compatibility reasons and will eventually be removed.

Own Id: OTP-7301

Faulty matching in binaries larger than 512Mb on 64bit machines fixed.(On 32bit, the size limit for binaries is still 512Mb). Thanks to Edwin Fine and Per Gustafsson for finding fault and fix.

Own Id: OTP-7309

crypto:start() on Windows caused emulator to hang on error popup window if openssl DLL was not found. Windows error popups now suppressed.

Own Id: OTP-7325

Configuration option without-termcap can be used to disable the use of termcap libraries for terminal cursor control in the shell.

Own Id: OTP-7338

to_erl reports its terminal window size to run_erl in order to get output formatted accordingly

Own Id: OTP-7342

On Solaris, the compressed option for file operations did not work if the file descriptor happened to be greater than 255 (a problem with fopen() and friends in Solaris itself).

Own Id: OTP-7343 Aux Id: seq10949

A race condition in the runtime system with SMP support causing an erroneous removal of a newly created ets table has been fixed.

The race occurred when a process removed a table during termination simultaneously as another process removed the same table via ets:delete/1 and a third process created a table that accidentally got the same internal table index as the table being removed.

Own Id: OTP-7349

zlib:inflate failed when the size of the inflated data was an exact multiple of the internal buffer size (4000 bytes by default).

Own Id: OTP-7359

If the total number of allowed atoms is exceeded, there will now be a controlled termination of the emulator with a crash dump file. The emulator used to simply crash. (Thanks Howard Yeh and Thomas Lindgren.)

Own Id: OTP-7372

The break handler in werl on Windows could cause the emulator to hang or crash, that is now corrected.

Own Id: OTP-7394 Aux Id: seq10969

The configure script now tests for an serious optimization bug in gcc-4.3.0. If the bug is present, the configure script will abort (if this happens, the only way to build Erlang/OTP is to change to another version of gcc). (Thanks to Mikael Pettersson.)

Own Id: OTP-7397

Improvements and New Features

On Unix, the emulator now notices when the width of the terminal has changed. (Thanks to Matthew Dempsky and Patrick Mahoney.)

Own Id: OTP-7290

There is a new function init:stop/1 which can be used to shutdown the system cleanly AND generate a non-zero exit status or crash dump. (Thanks to Magnus Froberg.)

Own Id: OTP-7308

process_info(Pid, garbage_collection) now returns more information

Own Id: OTP-7311

The hide option for open_port/2 is now documented. (Thanks to Richard Carlsson.)

Own Id: OTP-7358

Known Bugs and Problems

Floating point arithmetics in drivers can cause a runtime system crash on runtime systems with floating point exceptions enabled. Floating point exceptions are disabled unless explicitly enabled or if hipe is enabled.

Own Id: OTP-7237

Erts 5.6.2

Fixed Bugs and Malfunctions

The maximum length of an atom of 255 characters is now strictly enforced. binary_to_term/1 will now fail with a badarg if an encoded term contains an atom longer than 255 characters. Atoms created by drivers will now be truncated to 255 characters if necessary. (Thanks to Matthew Dempsky.)

Own Id: OTP-7147

A bug in "bignum handling" on some 64bit architectures could cause rem and div operations on large numbers to hang indefinitely. Rem operations involving the smallest negative number representable in 28 bits or 60 bits could also cause access violation and emulator crash. Both errors are corrected.

Own Id: OTP-7177

When doing the initial garbage collection after waking a hibernated process, a fullsweep garbage collection was unnecessarily triggered.

Own Id: OTP-7212

The beta testing module gen_sctp now supports active mode as stated in the documentation. Active mode is still rather untested, and there are some issues about what should be the right semantics for gen_sctp:connect/5. In particular: should it be blocking or non-blocking or choosable. There is a high probability it will change semantics in a (near) future patch.

Try it, give comments and send in bug reports!

Own Id: OTP-7225

Invalid arguments to ets:update_counter/3 were not handled correctly. A tuple position (Pos) less than 1 caused the element directly following the key to be updated (as if no position at all had been specified). All invalid values for Pos will now fail with badarg.

Own Id: OTP-7226

The runtime system with SMP support could under rare circumstances deadlock when a distribution channel was taken down while multiple simultaneous operations were performed on it.

Own Id: OTP-7260

Improvements and New Features

More checksum/hash algorithms from the zlib library are now available as built in functions (like md5 hashes has been for a long time).

Own Id: OTP-7128

Minor improvements in the garbage collector.

Own Id: OTP-7139 Aux Id: OTP-7132

The switch "-detached" to the windows werl program now can create an erlang virtual machine without any main window and without a temporary console showing.

Own Id: OTP-7142

erlang:system_info/1 now accepts the logical_processors, and debug_compiled arguments. For more info see the, erlang(3) documentation.

The scale factor returned by test_server:timetrap_scale_factor/0 is now also effected if the emulator uses a larger amount of scheduler threads than the amount of logical processors on the system.

Own Id: OTP-7175

A new BIF ets:update_element/3. To update individual elements within an ets-tuple, without having to read, update and write back the entire tuple.

Own Id: OTP-7200

A process executing the processes/0 BIF can now be preempted by other processes during its execution. This in order to disturb the rest of the system as little as possible. The returned result is, of course, still a consistent snapshot of existing processes at a time during the call to processes/0.

The documentation of the processes/0 BIF and the is_process_alive/1 BIF have been updated in order to clarify the difference between an existing process and a process that is alive.

Own Id: OTP-7213

Erts 5.6.1

Fixed Bugs and Malfunctions

The SMP emulator on sparc64 erroneously used the sparc32 atomic and the sparc32 spinlock implementations which caused it to crash.

Own Id: OTP-7006

Call tracing the new guard BIFs byte_size, bit_size, or tuple_size and the loading a module that uses one of those functions, could cause the emulator to terminate.

Own Id: OTP-7008

configuring --enable-darwin-universal or --enable-darwin-64bit on MacOSX could result in a non optimized emulator. Top level configure script now corrected.

Own Id: OTP-7014

configuring --with-gd did not produce correct include flags for percept.

Own Id: OTP-7015

Environment variables weren't handled in thread safe manner in the runtime system with SMP support on Windows.

erl_drv_putenv(), and erl_drv_getenv() has been introduced for use in drivers. Do not use putenv(), or getenv() directly in drivers. For more information see the erl_driver documentation.

Own Id: OTP-7035

HIPE: Corrected the choice of interface to the send/3 and setnode/3 BIFs for native-compiled code. Using the incorrect interface could, in unusual circumstances, lead to random runtime errors.

Own Id: OTP-7067

Garbage collections could become extremely slow when there were many keys in the process dictionary. (Thanks to Fredrik Svahn.)

Own Id: OTP-7068

The duplicate documentation directory in the windows installation is removed.

Own Id: OTP-7070

Documentation bugfixes and clarifications.

(Thanks to Joern (opendev@gmail.com), Matthias Lang, and Richard Carlsson.)

Own Id: OTP-7079

The runtime system with SMP support not using the native atomic integer implementation part of OTP could deadlock when run on a system with more than one logical processor. That is, only the runtime system with SMP support on other hardware platforms than x86, x86_64, sparc32, and powerpc32 were effected by this bug.

Own Id: OTP-7080

The break handling code (run when Ctrl-C is hit) could could potentially deadlock the runtime system with SMP support.

Own Id: OTP-7104

The sctp driver has been updated to work against newer lksctp packages e.g 1.0.7 that uses the API spelling change adaption -> adaptation. Older lksctp (1.0.6) still work. The erlang API in gen_sctp.erl and inet_sctp.hrl now spells 'adaptation' regardless of the underlying C API.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7120

A bug in erlang:phash2/1 on 64-bit platforms has been fixed. (Thanks to Scott Lystig Fritchie.)

Own Id: OTP-7127

The emulator could under rare circumstances crash while garbage collecting.

Own Id: OTP-7132

Improvements and New Features

The documentation has been updated so as to reflect the last updates of the Erlang shell as well as the minor modifications of the control sequence p of the io_lib module.

Superfluous empty lines have been removed from code examples and from Erlang shell examples.

Own Id: OTP-6944 Aux Id: OTP-6554, OTP-6911

Bit syntax construction with a small integer in a non-byte aligned field wider than the CPU's word size could cause garbage bits in the beginning of the field.

Own Id: OTP-7085

All Windows versions older than Windows 2000 are now not supported by the Erlang runtime system. This since there was a need for usage of features introduced in Windows 2000.

Own Id: OTP-7086

Memory management improvements especially for the runtime system with SMP support:

The runtime system with SMP support can now use multiple, thread specific instances of most memory allocators. This improves performance since it reduces lock contention in the memory allocators. It may however increase memory usage for some applications. The runtime system with SMP support will by default enable this feature on most allocators. The amount of instances used can be configured. driver_alloc(), driver_realloc(), and driver_free() now use their own erts specific memory allocator instead of the default malloc() implementation on the system. The default configuration of some allocators have been changed to fit applications that use much memory better. Some new erts_alloc configuration parameters have been added. erts_alloc_config has been modified to be able to create configurations suitable for multiple instances of allocators. The returned value from erlang:system_info({allocator, Alloc}) has been changed. This since an allocator may now run in multiple instances.

If you for some reason want the memory allocators to be configured as before, you can pass the +Mea r11b command-line argument to erl.

For more information see the erts_alloc(3), the erts_alloc_config(3), and the erlang(3) documentation.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7100

On Unix, denormalized floating point numbers could not be created using list_to_float/1 or binary_to_term/1. (Thanks to Matthew Dempsky.)

Own Id: OTP-7122

Native atomic integers and spin-locks are now also available for the runtime system with SMP support on sparc64.

Own Id: OTP-7130

FP exceptions support for sparc64 userspace on Linux has been added. Note that FP exception support is now turned off by default, so to actually enable it you need to do './configure --enable-fp-exceptions'.

Own Id: OTP-7131

Erts 5.6

Fixed Bugs and Malfunctions

A bug for raw files when reading 0 bytes returning 'eof' instead of empty data has been corrected.

Own Id: OTP-6291 Aux Id: OTP-6967

All exported functions in gzio.c have now been renamed to avoid conflict with drivers that are indirectly linked with an external zlib library.

Own Id: OTP-6816 Aux Id: OTP-6591

On the 64-bit Erlang emulator, bit syntax construction with integers containing more than 60 bits ("big numbers") into fields with more than 60 bits could produce incorrect results.

Own Id: OTP-6833

When the runtime system failed to allocate memory for binaries, it could dead lock while writing the erl_crash.dump.

Own Id: OTP-6848

The runtime system with SMP support could deadlock if a process called the erlang:suspend_process(Pid) BIF or the erlang:garbage_collect(Pid) BIF while the process identified by Pid was currently running and the process calling the BIFs was terminated during the call to the BIFs.

Processes suspending each other via the erlang:suspend_process/1 BIF or garbage collecting each other via the erlang:garbage_collect/1 BIF could deadlock each other when the runtime system with SMP support was used.

Own Id: OTP-6920

dbg could leave traced processes in a suspended state if the tracer process was killed with exit reason kill.

erlang:suspend_process/2 has been introduced which accepts an option list as second argument. For more information see the erlang(3) documentation.

Processes suspended via erlang:suspend_process/[1,2] will now be automatically resumed if the process that called erlang:suspend_process/[1,2] terminates.

Processes could previously be suspended by one process and resumed by another unless someone was tracing the suspendee. This is not possible anymore. The process resuming a process has to be the one that suspended it.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6946

file:write_file/3, file:write/2 and file:read/2 could crash (contrary to documentation) for odd enough file system problems, e.g write to full file system. This bug has now been corrected.

In this process the file module has been rewritten to produce better error codes. Posix error codes now originate from the OS file system calls or are generated only for very similar causes (for example 'enomem' is generated if a memory allocation fails, and 'einval' is generated if the file handle in Erlang is a file handle but currently invalid).

More Erlang-ish error codes are now generated. For example {error,badarg} is now returned from file:close/1 if the argument is not of a file handle type. See file(3).

The possibility to write a single byte using file:write/2 instead of a list or binary of one byte, contradictory to the documentation, has been removed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6967 Aux Id: OTP-6597 OTP-6291

Monitor messages produced by the system monitor functionality, and garbage collect trace messages could contain erroneous heap and/or stack sizes when the actual heaps and/or stacks were huge.

As of erts version 5.6 the large_heap option to erlang:system_monitor/[1,2] has been modified. The monitor message is sent if the sum of the sizes of all memory blocks allocated for all heap generations is equal to or larger than the specified size. Previously the monitor message was sent if the memory block allocated for the youngest generation was equal to or larger than the specified size.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6974 Aux Id: seq10796

inet:getopts/2 returned random values on Windows Vista.

Own Id: OTP-7003

Improvements and New Features

The emulator internal process lock implementation has been rewritten and optimized. A slight risk of starvation existed in the previous implementation. This risk has also been eliminated in the new implementation.

Own Id: OTP-6500

Bitstrings (bit-level) binaries and binary comprehensions are now part of the language. See the Reference Manual.

Own Id: OTP-6558

The windows version of erlang now has SMP support. The SMP emulator is run by default on machines which shows more than one virtual or physical processor.

Own Id: OTP-6560 Aux Id: OTP-6925

The details of the compressed term format has been documented in erl_ext_dist.txt. (Thanks to Daniel Goertzen.)

Own Id: OTP-6755

The runtime system with SMP support is now started by default if more than one logical processor are detected. For more information, see the erl(3) documentation.

Own Id: OTP-6756

The external format for Erlang terms and the distribution protocol are now documented in ERTS User's Guide.

Own Id: OTP-6779

New BIF's erlang:system_profile/1 and erlang:system_profile/2. These BIF's controls concurrency profiling options for processes, ports and schedulers.

Own Id: OTP-6783 Aux Id: OTP-6285

The ErlDrvTermData term types used by driver_output_term() and driver_send_term() have been extended with the term types ERL_DRV_BUF2BINARY, ERL_DRV_EXT2TERM, and ERL_DRV_UINT. ERL_DRV_BUF2BINARY is used for passing and creating a binary, ERL_DRV_EXT2TERM is used for passing terms encoded with the external term format, and ERL_DRV_UINT is used for passing unsigned integers.

Also the data types ErlDrvUInt and ErlDrvSInt have been added which makes it more obvious how arguments to term types are interpreted with regards to width and signedness.

The incorrect data types ErlDriverTerm, ErlDriverBinary, and ErlDriverPort in the erl_driver(3) documentation have been replaced with the correct data types ErlDrvTermData, ErlDrvBinary, and ErlDrvPort.

For more information see the erl_driver(3) documentation.

Own Id: OTP-6823

Miscellaneous improvements of the erts internal thread library.

It now support optimized atomic operations and spin-locks on windows.

Fall-backs based on mutexes and/or spin-locks for missing optimized atomic operations, spin-locks, or rwlocks has been implemented. This makes it possible to compile the runtime system with SMP support on a lot more platforms.

Default stack size on OpenBSD has been increased to 256 kilo-words.

Own Id: OTP-6831 Aux Id: OTP-6560

Many bit syntax operations, both construction and matching, are faster. For further information, see the Efficiency Guide.

Own Id: OTP-6838

Literal lists, tuples, and binaries are no longer constructed at run-time as they used to be, but are stored in a per-module constant pool. Literals that are used more than once are stored only once.

This is not a change to the language, only in the details of its implementation. Therefore, the implications of this change is described in the Efficiency Guide.

Example 1: In the expression element(BitNum-1, {1,2,4,8,16,32,64,128}), the tuple used to be constructed every time the expression was executed, which could be detrimental to performance in two ways if the expression was executed in a loop: the time to build the tuple itself and the time spent in garbage collections because the heap filled up with garbage faster.

Example 2: Literal strings, such as "abc", used to be stored in the compiled code compactly as a byte string and expanded to a list at run-time. Now all strings will be stored expanded to lists (such as [$a,$b,$c]) in the constant pool. That means that the string will be faster to use at run-time, but that it will require more space even when not used. If space is an issue, you might want to use binary literals (that is, <<"abc">>) instead of string literals for infrequently used long strings (such as error messages).

Own Id: OTP-6850

The Erlang driver API has been extended with a portable POSIX thread like API for multi-threading. The Erlang driver thread API provides:

Threads Mutexes Condition variables Read/Write locks Thread specific data

For more information see the erl_driver(3) documentation.

Own Id: OTP-6857

Recursive calls now usually consume less stack than in R11B. See the Efficiency Guide.

Own Id: OTP-6862 Aux Id: seq10746

The deprecated BIFs erlang:old_binary_to_term/1 and erlang:info/1 have been removed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6876

Calls to driver call-backs triggered by external events are now scheduled and interleaved with execution of Erlang processes also on the runtime system without SMP support.

Own Id: OTP-6878

Faster arithmetic of integers of more than 27 bits signed (or 60 bits signed on an 64-bit CPU), and also faster integer multiplication. (Thanks to Tony Rogvall.)

Own Id: OTP-6891

Significant improvements of the process_info BIFs:

process_info/2 can now be called with a list of items as second argument in order to atomically retrieve information about multiple items. process_info/[1,2] has been optimized in the runtime system with SMP support. The whole scheduler could previously be blocked for a significant period of time in process_info/[1,2] waiting for a lock on the process being inspected. The Erlang process calling process_info/[1,2] can still be blocked for a significant period of time waiting for the lock, but the scheduler will now be able to run other processes while the process calling process_info/[1,2] waits for the lock. process_info/2 now accept a few more items than before. The documentation of process_info/[1,2] has been improved.

For more information see the erlang(3) documentation.

Own Id: OTP-6899

open_port({}, []) could crash the emulator. (Thanks to Matthew Dempsky.)

Own Id: OTP-6901

Two new guard BIFs have been introduced as a recommended replacement for size/1. (The size/1 BIF will be removed no earlier than in R14B.) The BIFs are tuple_size/1 to calculate the size of a tuple and byte_size/1 to calculate the number of bytes needed for the contents of the binary or bitstring (rounded up to the nearest number of bytes if necessary).

There is also a new bit_size/1 BIF that returns the exact number of bits that a binary or bitstring contains.

Own Id: OTP-6902

The ets:fixtable/2 function, which has been deprecated for several releases, has been removed.

The ets:info/1 function has been reimplemented as a BIF, which guarantees that information returned is consistent.

The ets:info/2 function now fails with reason badarg if the second argument is invalid. (Dialyzer can be used to find buggy code where the second argument is misspelled.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6906

As the linux kernel may generate a minor fault when tracing with CPU timestamps, and there exists no patch to the Linux kernel that fixes the problem, cpu timestamps are disabled on Linux for now.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6922

The functions io:columns/0, io:columns/1, io:rows/0 and io:rows/1 are added to allow the user to get information about the terminal geometry. The shell takes some advantage of this when formatting output. For regular files and other io-devices where height and width are not applicable, the functions return {error,enotsup}.

Potential incompatibility: If one has written a custom io-handler, the handler has to either return an error or take care of io-requests regarding terminal height and width. Usually that is no problem as io-handlers, as a rule of thumb, should give an error reply when receiving unknown io-requests, instead of crashing.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6933

driver_caller() can now also be used from the start callback of a driver.

Own Id: OTP-6951

The emulator can now be compiled for 64bit intel, as well as a 32bit universal binary on darwin/MacOSX 10.4 and 10.5.

Own Id: OTP-6961

If open_port fails because all available ports are already in use, it will now throw a system_limit exception instead of an enfile exception. (enfile might still be thrown if the operating system would return ENFILE.)

Own Id: OTP-6968

The spawn_monitor/1 and spawn_monitor/3 BIFs are now auto-imported (i.e. they no longer need an erlang: prefix).

Own Id: OTP-6975

On Windows, the werl window now handles resize, so that the whole window can be utilized. Text selection is also updated to be line oriented instead of rectangle oriented as in earlier versions.

Own Id: OTP-6994 Aux Id: OTP-6933

Kqueue support (kernel-poll) has been enabled on FreeBSD. The problem with kqueue not detecting writes on a pipe on FreeBSD was actually not a kqueue issue, but a writev on pipes issue. Neither poll(), nor select() detected the write when the bug hit. NetBSD and DragonFlyBSD probably have or have had the same bug. This bug should have been fixed in FreeBSD 6.3 and FreeBSD 7.0 thanks to Jean-Sebastien Pedron.

Own Id: OTP-7001

Erts 5.5.5

Fixed Bugs and Malfunctions

The functions in gzio.c have been renamed to avoid conflict with drivers that indirectly linked with an external zlib library.

Own Id: OTP-6591

The emulator without SMP support dumped core if an async-thread finished a job after the emulator had begun writing an .

Own Id: OTP-6685

In bit syntax matching, integer fields with greater size than 16Mb would fail to match. (Thanks to Bertil Karlsson and Francesco Pierfederici.)

Matching out a 32-bit integer not aligned on a byte boundary from a binary could cause an heap overflow (and subsequent termination of the emulator).

A module that contained bit syntax construction with a literal field size greater than 16Mb would fail to load.

Several other similar bugs having to do with huge field sizes were eliminated.

Attempting to construct a binary longer than 536870911 bytes will now fail with a exception (rather than fail in mysterious ways or construct an binary with incorrect contents). Similarily, attempting to match a binary longer than 536870911 bytes will now fail (instead of producing an incorrect result). This limitation has been documented in the Efficiency Guide. (The limit is in the 32-bit emulator; use the 64-bit emulator if you need to handle larger binaries than 536870911.)

Own Id: OTP-6686

Bugs in rem and div of very large numbers are corrected.

Own Id: OTP-6692

didn't allocate enough heap when a bignum was part of the result which could cause an emulator crash.

Own Id: OTP-6693

It was previously not possible to pass the same configuration via the command-line, as used by default.

A command-line argument that configure a size of some sort can now be passed a value that equals the size of the address space. The value used, in this case, will be .

Own Id: OTP-6699

did not update if was invalid. is now set to if the is invalid.

The driver expects to be updated also when the is invalid. This situation occurs seldom, but when the runtime system has async-threads enabled and ports are killed it can. When it occurred the runtime system crashed.

Own Id: OTP-6729

Improvements and New Features

For scripts written using , there is a new function , which can be used to retrieve the pathame of the script. The documentation has been clarified regarding pre-defined macros such as ?MODULE and the module name.

Own Id: OTP-6593

The section Guards in the chapter The Abstract Format of the ERTS User's Guide has been updated.

Own Id: OTP-6600

Corrected protocol layer flue for socket options SO_LINGER, SO_SNDBUF and SO_RCVBUF, for SCTP.

Own Id: OTP-6625 Aux Id: OTP-6336

The behaviour of the inet option {active,once} on peer close is improved and documented.

Own Id: OTP-6681

The inet option send_timeout for connection oriented sockets is added to allow for timeouts in communicating send requests to the underlying TCP stack.

Own Id: OTP-6684 Aux Id: seq10637 OTP-6681

The command line flag , and the environment variables , and for the command have been added. For more information see the documentation.

Own Id: OTP-6697

The type test has been deprecated. is improperly named and almost entirely undocumented.

Own Id: OTP-6731

Erts 5.5.4

Fixed Bugs and Malfunctions

Corrected misspelling of '' in the help text for . (Thanks to Ulf Wiger.)

Own Id: OTP-6433

The MD5 calculation of a BEAM file done by , , and by the compiler for the default value of the attribute have all been changed so that its result will be the same on all platforms; modules containing funs could get different MD5s on different platforms.

Own Id: OTP-6459

The emulator could dump core while writing an file if there were funs with a large terms in its environment. Since there is no way to inspect a fun's environment in the crashdump_viewer application anyway, a variables in the environment are now set to [] before dumping the fun. (Thanks to Jean-Sebastien Pedron.)

Own Id: OTP-6504

messages from ports opened with the option could under rare circumstances be delayed. This bug was present on Erlang runtime systems without SMP support on all unix operating systems other than SunOS.

Own Id: OTP-6528

A bug in linuxthreads could cause the emulator to dump core when dlerror() was called before the first call to dlopen(). As a workaround the emulator always makes a call to dlopen() on initialization when linuxthreads is used as thread library.

Own Id: OTP-6530

file:sync/1 did not do anything on Windows. Now it calls the system function for flushing buffers (FlushFileBuffers()). (Thanks to Matthew Sackman.)

Own Id: OTP-6531

could on the runtime system with SMP support fail with the wrong exit reason when a port couldn't be created. When this happened the exit reason was typically , or instead of .

Own Id: OTP-6536

The file driver (efile_drv) did not flush data written using the option 'delayed_write' after the set timeout time, rather at the next file operation. This bug has now been corrected.

Own Id: OTP-6538

Improvements and New Features

An interface towards the SCTP Socket API Extensions has been implemented.It is an Open Source patch courtesy of Serge Aleynikov and Leonid Timochouk. The Erlang code parts has been adapted by the OTP team, changing the Erlang API somewhat.

The Erlang interface consists of the module and an include file for option record definitions. The module is documented.

The delivered Open Source patch, before the OTP team rewrites, was written according to http://tools.ietf.org/html/draft-ietf-tsvwg-sctpsocket-13 and was claimed to work fine, tested on Linux Fedora Core 5.0 (kernel 2.6.15-2054 or later) and on Solaris 10 and 11. The OTP team rewrites used the same standard document but might have accidentally broken some functionality. If so it will soon be patched to working state. The tricky parts in C and the general design has essentially not changed. During the rewrites the code was hand tested on SuSE Linux Enterprise Server 10, and briefly on Solaris 10. Feedbach on code and docs is very much appreciated.

The SCTP interface is in beta state. It has only been hand tested and has no automatic test suites in OPT meaning everything is most certainly not tested. Socket active mode is broken. IPv6 is not tested. The documentation has been reworked due to the API changes, but has not been proofread after this.

Thank you from the OTP team to Serge Aleynikov and Leonid Timochouk for a valuable contribution. We hope we have not messed it up too much.

Own Id: OTP-6336

A option is now recognized by . {minor_version,1} will cause floats to be encoded in an exact and more space-efficient way compared to the previous encoding.

Own Id: OTP-6434

There is a new program that can be used for writing scripts in Erlang. Erlang scripts don't need to be compiled and any arguments can be passed to them without risk that they are interpreted by the Erlang system.

Own Id: OTP-6505

Setting and getting socket options in a "raw" fashion is now allowed. Using this feature will inevitably produce non portable code, but will allow setting ang getting arbitrary uncommon options on TCP stacks that do have them.

Own Id: OTP-6519

Miscellaneous signal handling improvements on the Erlang runtime system with SMP support.

The fallback implementation of spin locks and atomic operations are now implemented using pthread spin locks when pthread spin locks are found on the system.

The Erlang runtime system with SMP support can now run on Linux systems that has Linuxthreads instead of NPTL (Native POSIX Thread Library). Note that the SMP support hasn't been as thoroughly tested with Linuxthreads as with NPTL. A runtime system with SMP support will therefore not be built by default on Linux when NPTL isn't found. In order to force a build of the runtime system with SMP support, pass to when building OTP.

Own Id: OTP-6525

Erts 5.5.3.1

Fixed Bugs and Malfunctions

could cause the emulator with SMP support to deadlock.

Own Id: OTP-6431 Aux Id: OTP-6403

The runtime system with SMP support failed to call the driver timeout callback of ports in state closing. This could cause ports to fail flushing their I/O queues.

Own Id: OTP-6432

The message from a closed port could arrive at the port owner before had been removed from the result of in the runtime system with SMP support.

Own Id: OTP-6437

The async id of async jobs created via wasn't created in a thread safe manner in the runtime system with SMP support. This could in worst case cause to cancel another async job than intended.

Own Id: OTP-6438

Under rare circumstances a terminating connection between two nodes could cause an instantaneous reconnect between the two nodes to fail on the runtime system with SMP support.

Own Id: OTP-6447

In the documentation of the driver entry field of erts version 5.5.3 (), the following is stated: "The following fields are ignored if this field is equal to ". This is a documentation bug and has been changed to: "If this field is equal to , all the fields following this field also have to be , or in case it is a pointer field".

The runtime check for detection of old incompatible drivers made when loading drivers has been improved. The emulator can, however, not make sure that a driver that doesn't use the extended driver interface isn't incompatible. Therefore, when loading a driver that doesn't use the extended driver interface, there is a risk that it will be loaded also when the driver is incompatible. When the driver use the extended driver interface, the emulator can verify that it isn't of an incompatible driver version. You are therefore advised to use the extended driver interface.

For more information see the and documentation.

Own Id: OTP-6452 Aux Id: OTP-6330

When terminating ports on the runtime system with SMP support, removal of links to processes was done without locking the link lock on processes. This could cause an emulator crash.

Own Id: OTP-6475

The emulator with SMP support could crash when a port flushed its I/O queue by calling from the timeout driver callback.

Own Id: OTP-6479

Large exit reasons could under rare circumstances cause the runtime system with SMP support to crash.

Own Id: OTP-6521

Improvements and New Features

Faster system calls for keeping the time accurate are used on newer Linux kernels, which can result in a significant speed-up of the emulator on those systems.

Own Id: OTP-6430

Added number of async threads and number of scheduler threads to the system information that can be retrieved via . For more information see the documentation.

Own Id: OTP-6440

When is received by the runtime system with SMP support, the is now written by a special thread, instead of as before directly from the signal handler.

Own Id: OTP-6465

term_to_binary/2 with compression is now faster.

term_to_binary/2 now accepts the option '' for specifying the compression level. must be in the range 0 (no compression) through 9 (highest compression level). Default is 6.

Future compatibility bugfix: binary_to_term/1 did not handle the and fields correctly.

Own Id: OTP-6494

Removed unnecessary reallocation when initializing kernel-poll set.

Own Id: OTP-6516

Erts 5.5.3

Fixed Bugs and Malfunctions

Node and fun table entries could under rare circumstances be deallocated multiple times on the emulator with SMP support.

Own Id: OTP-6369

epoll_wait() can repeat entries for the same file descriptor in the result array. This could cause the ready_input, ready_output, or event callbacks of a driver to unintentionally be called multiple times. We have only noted repeated entries when an error condition has occurred on the file descriptor. In this case, the repeated entries should normally not be a problem for the driver since it should detect the error and de-select the file descriptor. Therefore this bug should be considered harmless. The emulator now coalesce repeated entries into one.

You are only affected by this bug if you are using erts-5.5.2.* and the kernel-poll feature on linux.

Own Id: OTP-6376 Aux Id: OTP-6222

If a process that was waiting in gen_tcp:accept/1 was killed, calling gen_tcp:accept/1 again on the same listen socket would fail with ''.

Own Id: OTP-6381 Aux Id: seq10535

The emulator failed to start on Solaris 10 when kernel-poll support was enabled and the maximum number of filedescriptors configured was less than or equal to 256.

Own Id: OTP-6384 Aux Id: OTP-6222

The R10B compiler could generate unsafe instructions that could cause memory corruption. (The R11B compiler does not have that problem.) The erlang emulator will now refuse to load R10B-compiled modules that contain such unsafe instructions. In addition, the beam_validator module in the compiler will also reject such instructions (in case it is used to validate R10B code). (Thanks to Matthew Reilly.)

Own Id: OTP-6386

Process and port timers could fail to work properly on the runtime system with SMP support. Many thanks to Dmitriy Kargapolov and Serge Aleynikov who tracked down this bug.

Own Id: OTP-6387

Bit syntax code compiled by an R10B compiler that matched out a floating point number would not properly check that the floating point number was valid; if the float was, for instance, NaN the emulator could crash.

Own Id: OTP-6395

statistics(runtime) on Windows used to return the elapsed system time for the process, instead of the user time. Corrected. (Thanks to Robert Virding.)

Own Id: OTP-6407

A loadable driver (loaded by erl_ddll) which used driver_async() would cause the emulator to crash. (Thanks to Dmitriy Kargapolov.)

Own Id: OTP-6410

Under rare circumstances the emulator on unix platforms could fail to terminate the Erlang port corresponding to a port program opened with the option.

Own Id: OTP-6411

A link removed via could under rare circumstances transfer exit signals for a short period of time after the call to had returned when the runtime system with SMP support was used.

Own Id: OTP-6425 Aux Id: OTP-6160

Improvements and New Features

In the runtime system with SMP support, ports are now being scheduled on the scheduler threads interleaved with processes instead of being run in a separate I/O thread.

Own Id: OTP-6262

More interfaces are added in erl_ddll, to support different usage scenarios.

Own Id: OTP-6307 Aux Id: OTP-6234

In the runtime system with SMP support, the global I/O lock has been replaced with a more fine grained port locking scheme. Port locking is either done on driver level, i.e., all ports executing the same driver share a lock, or on port level, i.e., each port has its own lock. Currently the inet driver, the efile driver, and the spawn driver use port level locking and all other statically linked in drivers use driver level locking. By default dynamically linked in drivers will use locking on driver level. For more information on how to enable port level locking see the and the man pages.

As of erts version 5.5.3 the driver interface has been extended. The extended interface introduce version management, the possibility to pass capability flags to the runtime system at driver initialization, and some new driver API functions. For example, the function which can be used to determine if the driver is run in a runtime system with SMP support or not. The extended interface doesn't have to be used, but dynamically linked in driver have to be recompiled. For information see the and the man pages.

NOTE: Dynamically linked in drivers have to be recompiled.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6330 Aux Id: OTP-6262

A test and debug feature which modifies the timing of the runtime system has been added. For more information, see documentation of the ]]> command line argument of .

Own Id: OTP-6382

The version of zlib (http://zlib.net) linked into run-time system has been updated to version 1.2.3.

Own Id: OTP-6394

The program now passes on the and options to the Erlang emulator it starts. This is mainly useful for compiling native code, because native code must be compiled with same type of run-time system as in which it will be run.

If the option is given, now prints a warning that it is deprecated and that it will be removed in R12B.

Own Id: OTP-6398

The option of has been removed, i.e., the number of scheduler threads cannot be changed after emulator boot time any more.

A option has been added to . This option can be used for blocking and unblocking multi-scheduling. For more information see the documentation.

Own Id: OTP-6403

A port program that had been started with the option and closed one of the pipes used for communication with the emulator caused the emulator to continuously poll for termination of the port program. This only became a problem when the emulator didn't have other things to do and the port program closed a pipe much earlier than it terminated. When the emulator had other things to do, such as running Erlang processes, the emulator polled for termination in between scheduling of processes.

Now the emulator doesn't poll for termination of the port program at all; instead, it waits for the child signal from the terminated port program to arrive and then schedules the Erlang port for termination.

The termination of any port programs have also been optimized. Previously the termination of any port program did always cause a scan of a table of the size equal to maximum number of file descriptors. If the maximum number of file descriptors was large, this scan could be quite expensive. Now the search have been reduced to the set of ports started with the option.

Note, all of the above only applies to Erlang emulators on Unix platforms.

Own Id: OTP-6412 Aux Id: seq10194

* BEAM: added support for floating-point exceptions on FreeBSD (32-bit x86)

* SMP: made locking procedures work even when native lock operations aren't implemented

* SMP: improved timing accuracy in the timer thread (if enabled)

Own Id: OTP-6424

Erts 5.5.2

Fixed Bugs and Malfunctions

erlc: A typo in the help text for '-pa path' was corrected.

Own Id: OTP-6218

Failure in port command/control driver callbacks could crash the non-SMP emulator. This bug was introduced in the 5.5 version of erts.

Own Id: OTP-6224

Erroneous "" error reports could sometimes occur when a driver instance terminated in the ready_output() callback of the driver. This bug was only present in emulators that used poll(). Note, that this bug was harmless, the only effect it had was the erroneous error report.

Own Id: OTP-6229 Aux Id: OTP-3993, Seq5266, Seq7247, OTP-4307

The emulator could cause a type assertion failure while writing an erl_crash.dump, causing the erl_crash.dump to be terminated and a core dump generated.

Own Id: OTP-6235 Aux Id: seq10444

The registered name of a process is now the last observable resource removed before links and monitors are triggered when a process terminates.

Previously ets tables were removed after the registered name. This could cause problems on the runtime system with SMP support for code that expected that ets tables owned by a specific process had been removed if the name of the process had been removed.

Own Id: OTP-6237

Failure to fork() a new (os) process could cause the emulator to deadlock. This bug affect all emulators with SMP support, and emulators with async thread support on SunOS.

Own Id: OTP-6241 Aux Id: OTP-3906

Fprof traces could become truncated for the SMP emulator. This bug has now been corrected.

Own Id: OTP-6246

The undocumented functions inet:getiflist/0,1 inet:ifget/2,3 and inet:getif/1 were completely broken on Windows. That has been fixed.

Own Id: OTP-6255

Behavior in case of disappeared nodes when using the dist_auto_connect once got changed in R11B-1. The timeouts regarding normal distributed operations is now reverted to the old (pre R11B-1).

Own Id: OTP-6258 Aux Id: OTP-6200, seq10449

The bsl and bsr operators could cause the emulator to crash if given invalid arguments. (Thanks to datacompboy and Per Gustafsson.)

Own Id: OTP-6259

driver_cancel_timer() could under certain circumstances fail to cancel the timer on the runtime system with SMP support.

Own Id: OTP-6261

A call to erlang:system_info(info) could deadlock the runtime system with SMP support.

Own Id: OTP-6268

Improvements and New Features

Exit signals are now truly asynchronous in the runtime system with SMP support. This simplifies locking in bifs a lot, and makes process termination cheaper.

Own Id: OTP-6160

When tracing on the runtime system with SMP support it can be difficult to know when a trace has been delivered to the tracer. A new built in function has been introduced in order to make it easier to know when the trace has been delivered. See the erlang(3) man page for more information.

Own Id: OTP-6205 Aux Id: OTP-6269

Kernel poll support can now be combined with SMP support. Currently the following kernel poll versions exist: , , and . Linux has been replaced with . Some time in the future there will also be a kernel poll version using Solaris event ports.

The "check io" implementation for unix has been completely rewritten. The current kernel poll implementation reduce the amount of system calls needed compared to the old kernel poll implementation.

When or is used either or is used as fallback. Previously only could be used as fallback. Since now can be used as fallback, kernel poll support is now also available on newer MacOSX. Note however, when is used as fallback, the maximum number of file descriptors is limited to .

Kernel poll support is now enabled by default if , , or is found when building OTP, i.e. you do not have to pass the argument to . As before, kernel poll is disabled by default in the runtime system. In order to enable it, pass the command line argument to .

Note: will refuse to enable kernel poll support on FreeBSD since have problems with (at least) pipes on all version of FreeBSD that we have tested.

Own Id: OTP-6222 Aux Id: seq10380

The module and the code in the emulator have been completely rewritten; several bugs were fixed.

Own Id: OTP-6234

The SMP emulator now avoids locking for the following operations (thus making them as fast as in the UP emulator): , atom comparison, atom hashing, .

Own Id: OTP-6252

There are new BIFs , and the new option for .

The module has been updated to handle the new BIFs.

Own Id: OTP-6281

Erts 5.5.1

Fixed Bugs and Malfunctions

Complex pattern matching of strings would fail in the 64 bits emulator because of a bug in the loader. (Thanks to Igor Goryachev.)

Own Id: OTP-6142

and would be calculated incorrectly. could in unlucky circumstances cause a heap overflow, as could size(Binary) when size of the binary was larger than 128Mb.

Own Id: OTP-6154

erlang:display/1 displayed erroneous values for negative integers.

Big integers (both positive and negative) were previously displayed in hexadecimal form while small integers were displayed in decimal form. All integers are now displayed in decimal form.

NOTE: erlang:display/1 should only be used for debugging.

Own Id: OTP-6156

A call to erlang:trace/3 with erroneous flags caused the SMP emulator to deadlock instead of exiting the calling process with badarg.

Own Id: OTP-6175

A bug causing the emulator to hang when exiting a process that is exception traced has been fixed.

Own Id: OTP-6180

ets:rename/1 could deadlock, or crash the SMP emulator when the table wasn't a named table.

ets:next/2, and ets:prev/2 could return erroneous results on the SMP emulator.

Own Id: OTP-6198 Aux Id: seq10392, seq10415

A memory allocation bug could cause the SMP emulator to crash when a process had executed a with a larger timeout than 10 minutes.

Own Id: OTP-6199

The runtime system with SMP support did not slowly adjust its view of time when the system time suddenly changed.

Timeouts could sometimes timeout too early on the runtime system with SMP support.

Own Id: OTP-6202

Improvements and New Features

The smp runtime system now automatically detects the number of logical processors on MacOSX (darwin) and OpenBSD.

The smp runtime system is now built by default on MacOSX (darwin) on x86.

Own Id: OTP-6119

The command line argument now take the following options: , , or .

Especially the argument is useful since it starts the Erlang runtime system with SMP support if it is available and more than one logical processor are detected; otherwise, it starts the Erlang runtime system without SMP support. For more information see the man page.

Own Id: OTP-6126

Increased the reduction cost for sending messages in the SMP emulator so it behaves more like the non-SMP emulator.

Own Id: OTP-6196

A port running a dynamically linked-in driver that exits due to the driver being unloaded now exits with exit reason . Previously the port exited with exit reason .

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6204

Changed name of the argument to . This since the argument so easily could be mixed up with the argument (both returning integers).

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6208

The changes below were made by Mikael Pettersson, HiPE.

HiPE runtime system:

Reduce overheads in the HiPE runtime system's BIF glue code.

Fix bug when exceptions are thrown from BEAM to HiPE.

Support SPARC on Linux.

Support x86 on FreeBSD.

Floating-point exceptions:

Reduce overheads in checking results of floating-point operations.

Minor bug fix in SSE2 floating-point exception handling.

Support SSE2 floating-point exceptions on 32-bit x86 machines.

Make FP exceptions work in the SMP runtime system on FreeBSD/x86.

Support floating-point exceptions on SPARCs running Linux.

Runtime system:

Minor scheduler optimisation in the non-SMP runtime system.

Substantial reduction of I/O thread overheads in the SMP runtime system if the separate timer thread is used. (In R11B-1, the separate timer thread is not used.)

Own Id: OTP-6211

ERTS 5.5

Fixed Bugs and Malfunctions

Previously and behaved completely asynchronous. This had one undesirable effect, though. You could never know when you were guaranteed not to be affected by a link that you had unlinked or a monitor that you had demonitored.

The new behavior of and can be viewed as two operations performed atomically. Asynchronously send an unlink signal or a demonitor signal, and ignore any future results of the link or monitor.

NOTE: This change can cause some obscure code to fail which previously did not. For example, the following code might hang:

 ok
            %% We were previously guaranteed to get a down message
            %% (since we exited the process ourself), so we could
            %% in this case leave out:
            %% after 0 -> ok
            end,
          ]]>

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-5772

Two bugs fixed: If the environment variable was set, its contents would be appended to the end of the command line even if the command line had an options. Changed to place the options from just before . Also, the and flags no longer have any effect if placed after .

Own Id: OTP-6054

Improvements and New Features

The documentation for writing drivers in the ERTS User's Guide has been expanded and updated.

Own Id: OTP-5192

The and operators are now allowed to be used in guards. That also applies to match specifications.

Own Id: OTP-5894 Aux Id: OTP-5149

There is a new trace match spec function and a corresponding trace message that can be used to trace on any exit from a function, both normal function return and exception return. See the documentation for details.

The trace message is now also generated when the execution returns to a function due to catching an exception.

Own Id: OTP-5956

Erlang runtime system with SMP (symmetric multi processing) support.

The runtime system with SMP support is in this release focused on stability and there are a number of steps with optimizations to follow before it will take full advantage of multi processor systems. The released system is however truly multi threaded and you will notice increased performance for many applications already. We recommend that you evaluate your application on the SMP version of the runtime system and wait for some more optimizations before you use it in a real product. You will then discover if there are any problems in your application that needs to be fixed in order for it to work properly in a multi threaded environment. More optimized versions of the runtime system with SMP support will be included in the R11B maintenance releases.

The SMP enabled runtime system will be started if the command line argument is passed to the command. In order to make use of more than one processor core, multiple scheduler threads are used. By default, the number of scheduler threads will equal the number of processor cores. The number of scheduler threads can be set with the command line argument. For more information see the man page.

A runtime system with SMP support is by default built on the following platforms if posix threads, and a gcc compiler of at least version 2.95 is found:

Linux with at least kernel version 2.6 and the Native POSIX Thread Library on x86, x86_64, and 32-bits PowerPC.

Solaris of at least version 8 on 32-bits SPARC-V9.

MacOSX of at least version 10.4 (Darwin 8.0) on 32-bits PowerPC.

The runtime system with SMP support is known not to build on:

Windows.

Linux with kernel versions less than 2.4, or without the Native POSIX Thread Library.

Other hardware platforms than x86, x86_64, 32-bits SPARC-V9 and 32-bits PowerPC.

Windows will be supported in a future release.

The runtime system with SMP support might build on other operating systems in combination with supported hardware. In order to force a build of a runtime system with SMP support, pass the command line argument to configure. Note, however, that it is not enough that it builds. The underlying thread library and operating system has to provide SMP support as well. If the thread library does not distribute scheduler threads over multiple processor cores then the runtime system will only seemingly provide SMP support. If the runtime system is not built by default on a specific platform, we have not tested it on that platform.

NOTE: The design of SMP support for drivers is ongoing. There will probably be incompatible driver changes (only affecting drivers run on the runtime system with SMP support) released as patches for R11B.

Potential incompatibility: Previously, specific driver call-backs were always called from the same thread. This is not true in the runtime system with SMP support. Calls to call-backs will be made from different threads, e.g., two consecutive calls to exactly the same call-back can be made from two different threads. This will in most cases not be a problem. All calls to call-backs are synchronized, i.e., only one call-back will be called at a time.

In the future the default behavior will probably be the following: Calls to call-backs will, as now, be made from different threads. Calls to call-backs in the same driver instance will be synchronized. It will probably be possible to configure so that all calls to call-backs in all driver instances of a specific driver type will be synchronized. It may be possible to configure so that all calls to call-backs of a driver instance or a of a specific driver type will be made from the same thread.

Parallelism in the Erlang code executed is a necessity for the Erlang runtime system to be able to take advantage of multi-core or multi-processor hardware. There need to be at least as many Erlang processes runnable as processor cores for the Erlang runtime system to be able to take advantage of all processor cores.

An Erlang runtime system with SMP support with only one Erlang process runnable all the time will almost always be slower than the same Erlang runtime system without SMP support. This is due to thread synchronization overhead.

Known major bottleneck in the Erlang runtime system:

Currently the I/O system uses one "big lock", i.e. only one thread can do I/O at a time (with the exception of async threads and threads created by users own linked-in drivers). This is high on the list of things to optimize. Note, code that does not do I/O can be executed at the same time as one thread does I/O.

Some pitfalls which might cause Erlang programs that work on the non-SMP runtime system to fail on the SMP runtime system:

A newly spawned process will often begin executing immediately. Code that expects that the parent process will be able to execute for a while before the child process begins executing is likely to fail.

High priority processes could previously provide mutual exclusion (bad programming style) by preventing normal and low priority processes from being run. High priority processes cannot be used this way to provide mutual exclusion.

could be used to provide some kind of temporary mutual exclusion (also bad programming style). cannot be used to provide any kind of mutual exclusion.

Obscure pitfall, only if a process being traced also sends normal messages to the tracer:

The order between trace messages and normal messages is undefined. I.e. the order between normal messages sent from a tracee to a tracer and the trace messages generated from the same tracee to the same tracer is undefined. The internal order of normal messages and the internal order of trace messages will, of course, be preserved as before.

The kernel poll feature is currently not supported by the runtime system with SMP support. It will probably be supported in a future release.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6006 Aux Id: OTP-6095

Linked-in driver modifications.

Linked-in drivers must be recompiled.

The field in the type has been removed. The reference count can be accessed via API functions. For more information see the man page.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6095 Aux Id: OTP-6006