Fix a bug that could cause a crash when formatting a list of non-characters using the control sequences p or P and limiting the output with the option chars_limit.

Own Id: OTP-15159

gen_statem improvements.

When using an exception that is valid but not allowed in a state enter call, the reason has been changed from {bad_action_from_state_function,Action} to {bad_state_enter_action_from_state_function,Action}.

Timer parsing has been improved. Many erroneous timeout tuples was not handled correctly.

The documentation has been updated, in particular the User's Guide and the pointer to it from the Reference Manual is much more obvious.

Own Id: OTP-14015

The type specifications for file:posix/0 and inet:posix/0 have been updated according to which errors file and socket operations should be able to return.

Own Id: OTP-14019 Aux Id: ERL-550

File operations used to accept filenames containing null characters (integer value zero). This caused the name to be truncated and in some cases arguments to primitive operations to be mixed up. Filenames containing null characters inside the filename are now rejected and will cause primitive file operations to fail.

Also environment variable operations used to accept names and values of environment variables containing null characters (integer value zero). This caused operations to silently produce erroneous results. Environment variable names and values containing null characters inside the name or value are now rejected and will cause environment variable operations to fail.

Primitive environment variable operations also used to accept the $= character in environment variable names causing various problems. $= characters in environment variable names are now also rejected.

Also os:cmd/1 now reject null characters inside its command.

erlang:open_port/2 will also reject null characters inside the port name from now on.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14543 Aux Id: ERL-370

Make io_lib:unscan_format/1 work with pad char and default precision.

Own Id: OTP-14958 Aux Id: PR-1735

The control sequence modifiers t and l can be used together in the same control sequence which makes it possible to have Unicode atoms and no detection of printable character lists at the same time.

Own Id: OTP-14971 Aux Id: PR-1743

Fix a bug in the Erlang code linter: the check of guard expressions no longer returns false if the map syntax is used. The bug affected the Erlang shell, the Debugger, and other modules evaluating abstract code.

Own Id: OTP-15035 Aux Id: ERL-613

A sys debug fun of type {Fun,State} should not be possible to install twice. This was, however, possible if the current State was 'undefined', which was mistaken for non-existing fun. This has been corrected.

Own Id: OTP-15049

Fix io:putchars/2 stacktrace rewriting at errors to point to a valid function.

Own Id: OTP-15101

The gen_server has gotten a new callback handle_continue/2 for check pointing the state. This is useful at least when implementing behaviours on top of gen_server and for some start up scenarios.

Own Id: OTP-13019 Aux Id: PR-1490

The semantics of timeout parameter {clean_timeout,infinity} to gen_statem:call/3 has been changed to use a proxy process for the call. With this change clean_timeout implicates a proxy process with no exceptions. This may be a hard to observe incompatibility: in the presence of network problems a late reply could arrive in the caller's message queue when catching errors. That will not happen after this correction.

The semantics of timeout parameter infinity has not been changed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13073 Aux Id: PR-1595

A new logging API is added to Erlang/OTP, see the logger(3) manual page, and section Logging in the Kernel User's Guide.

Calls to error_logger are automatically redirected to the new API, and legacy error logger event handlers can still be used. It is, however, recommended to use the Logger API directly when writing new code.

Notice the following potential incompatibilities:

Since Logger is new in Erlang/OTP 21.0, we do reserve the right to introduce changes to the Logger API and functionality in patches following this release. These changes might or might not be backwards compatible with the initial version.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13295

Add functions calendar:system_time_to_local_time/2 and calendar:system_time_to_universal_time/2.

Own Id: OTP-13413

Functions rand:uniform_real/0 and rand:uniform_real_s/1 have been added. They produce uniformly distributed numbers in the range 0.0 =< X < 1.0 that are as close to random real numbers as Normalized IEEE 754 Double Precision allows. Because the random real number exactly 0.0 is infinitely improbable they will never return exactly 0.0.

These properties are useful when you need to call for example math:log(X) or 1 / X on a random value X, since that will never fail with a number from these new functions.

Own Id: OTP-13764 Aux Id: PR-1574

Added maps:iterator/0 and maps:next/1 to be used for iterating over the key-value associations in a map.

Own Id: OTP-14012

Changed the default behaviour of .erlang loading: .erlang is no longer loaded from the current directory. c:erlangrc(PathList) can be used to search and load an .erlang file from user specified directories.

escript, erlc, dialyzer and typer no longer load an .erlang at all.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14439

Added new uri_string module to stdlib for handling URIs (RFC 3986).

Own Id: OTP-14496

Update Unicode specification to version 10.0.

Own Id: OTP-14503

filelib:wildcard() now allows characters with a special meaning to be escaped using backslashes.

This is an incompatible change, but note that the use of backslashes in wildcards would already work differently on Windows and Unix. Existing calls to filelib:wildcard() needs to be updated. On Windows, directory separators must always be written as a slash.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14577

The supervisor now stores its child specifications in a map instead of a list. This causes a significant improvement when starting many children under a non-simple_one_for_one supervisor.

Own Id: OTP-14586

The base64 module is optimized.

Note that the functions encode/1, decode/1, and mime_decode/1 fail unless called with an argument of the documented type. They used to accept any iodata().

Own Id: OTP-14624 Aux Id: PR-1565

Add function lists:search/2.

Own Id: OTP-14675 Aux Id: PR-102

uri_string module extended with functions for handling application/x-www-form-urlencoded query strings based on the HTML5 specification.

Own Id: OTP-14747

Add functions calendar:rfc3339_to_system_time/1,2 and calendar:system_time_to_rfc3339/1,2.

Own Id: OTP-14764

The stack traces returned by the functions of the erl_eval module more accurately reflect where the exception occurred.

Own Id: OTP-14826 Aux Id: PR 1540

Add options atime, mtime, ctime, uid, and gid to the erl_tar:add/3,4 functions.

Own Id: OTP-14834 Aux Id: PR 1608

Added ets:whereis/1 for retrieving the table identifier of a named table.

Own Id: OTP-14884

Improved URI normalization functions in the uri_string module.

Own Id: OTP-14910

The new functions io_lib:fwrite/3 and io_lib:format/3 take a third argument, an option list. The only option is chars_limit, which is used for limiting the number of returned characters. The limit is soft, which means that the number of returned characters exceeds the limit with at most a smallish amount. If the limit is set, the functions format/3 and fwrite/3 try to distribute the number of characters evenly over the control sequences pPswW. Furthermore, the control sequences pPwP try to distribute the number of characters evenly over substructures.

A modification of the control sequences pPwW is that even if there is no limit on the number of returned characters, all associations of a map are printed to the same depth. The aim is to give a more consistent output as the order of map keys is not defined. As before, if the depth is less than the number of associations of a map, the selection of associations to print is arbitrary.

Own Id: OTP-14983

Add functions ordsets:is_empty/1 and sets:is_empty/1.

Own Id: OTP-14996 Aux Id: ERL-557, PR-1703

Improve performance of string:uppercase/1, string:lowercase/1 and string:casefold/1 when handling ASCII characters.

Own Id: OTP-14998

External funs with literal values for module, name, and arity (e.g. erlang:abs/1) are now treated as literals. That means more efficient code that produces less garbage on the heap.

Own Id: OTP-15003

sys:statistics(Pid,get) did not report 'out' messages from gen_server. This is now corrected.

Own Id: OTP-15047

A sys debug function can now have the format {Id,Fun,State} in addition to the old {Fun,State}. This allows installing multiple instances of a debug fun.

Own Id: OTP-15048

The lib module is removed:

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-15072 Aux Id: PR 1786, OTP-15114

Function ets:delete_all_objects/1 now yields the scheduler thread for large tables that take significant time to clear. This to improve real time characteristics of other runnable processes.

Own Id: OTP-15078

In control sequences of the functions io:fwrite/2,3 and io_lib:fwrite/2,3 containing p or P, a field width of value 0 means that no line breaks are inserted. This is in contrast to the old behaviour, where 0 used to insert line breaks after every subterm. To insert line breaks after every subterm, a field width of value 1 can be used.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-15103 Aux Id: ERL-607

The Module:init/1 function in gen_statem may return an actions list containing any action, but an erroneous check only allowed state enter actions so e.g {next_event,internal,event} caused a server crash. This bug has been fixed.

Own Id: OTP-13995

Correct filelib:find_source() and filelib:find_file() to by default also search one level below src. This is in accordance with the Design Principles which states that an application can have Erlang source files one level below the src directory.

Own Id: OTP-14832 Aux Id: ERL-527

The contract of erl_tar:table/2 is corrected.

Own Id: OTP-14860 Aux Id: PR 1670

Correct a few contracts.

Own Id: OTP-14889

Fix string:prefix/2 to handle an empty string as second argument.

Own Id: OTP-14942 Aux Id: PR-1702

Make ets:i/1 exit cleaner when ^D is input while browsing a table. Only the old Erlang shell is affected (erl(1) flag -oldshell).

Own Id: OTP-14663

Fixed handling of windows UNC paths in module filename.

Own Id: OTP-14693

Improve performance of the new string functionality when handling ASCII characters.

Own Id: OTP-14670

Added a clarification to the documentation of unicode:characters_to_list/2.

Own Id: OTP-14798

Fix a bug in the Erlang shell where recursively defined records with typed fields could cause a loop.

Own Id: OTP-14488 Aux Id: PR-1489

Make edlin handle grapheme clusters instead of codepoints to improve the handling multi-codepoints characters.

Own Id: OTP-14542

There could be false warnings for erlang:get_stacktrace/0 being used outside of a try block when using multiple catch clauses.

Own Id: OTP-14600 Aux Id: ERL-478

The Erlang code linter no longer checks that the functions mentioned in nowarn_deprecated_function options are declared in the module.

Own Id: OTP-14378

General Unicode improvements.

Own Id: OTP-14462

A bug in proc_lib:format() introduced in Erlang/OTP 20.0 is corrected.

Own Id: OTP-14482 Aux Id: PR-1488

Fix string:len/1 to be compatible with previous versions.

Own Id: OTP-14487 Aux Id: ERIERL-40

In OTP-20.0, the behavior of c, make, and ct_make was changed so that in some cases the beam files by default would be written to the directory where the source files were found. This is now changed back to the old behavior so beam files are by default written to current directory.

Own Id: OTP-14489 Aux Id: ERL-438

For many releases, it has been legal to override a BIF with a local function having the same name. However, calling a local function with the same name as guard BIF as filter in a list comprehension was not allowed.

Own Id: OTP-13690

A new (default) pseudo-random number generator algorithm Xoroshiro116+ has been implemented in the rand module.

The old algorithm implementations had a number of flaws so they are all deprecated, but corrected versions of two of them have been added. See the documentation.

Own Id: OTP-14295 Aux Id: PR-1372

The Erlang shell, qlc:string_to_handle(), and the Debugger (the Evaluator area and Edit variable window of the Bindings area) can parse pids, ports, references, and external funs, as long as they can be created in the running system.

Own Id: OTP-14296

Internal code change: Calls to catch followed by a call to erlang:get_stacktrace/0 has been rewritten to use try instead of catch to make the code future-proof.

Own Id: OTP-14400

The ms_transform module, used by ets:fun2ms/1 and dbg:fun2ms/1, evaluates constant arithmetic expressions. This is necessary since the Erlang compiler, which normally evaluates constant expressions, does not recognize the format generated by ms_transform.

Own Id: OTP-14454 Aux Id: ERIERL-29

The state machine engine gen_statem can now handle generic time-outs (multiple named) as well as absolute time-out time. See the documentation.

The gen_statem callback Module:init/1 has become mandatory to harmonize with other gen_* modules. This may be an incompatibility for gen_statem callback modules that use gen_statem:enter_loop/4-6.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14531

Improved unicode support for strings. Added normalization functions in the unicode module. Extended the string module API with new functions with improved unicode handling and that works on grapheme clusters. The new functions operates on the unicode:chardata() type, thus they also accept UTF-8 binaries as input.

The old string API have been marked as obsolete. The return values have been changed for some error cases.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10289 Aux Id: OTP-10309

There are two new guard BIFs 'floor/1' and 'ceil/1'. They both return integers. In the 'math' module, there are two new BIFs with the same names that return floating point values.

Own Id: OTP-13692

Making code_change, terminate and handle_info callbacks optional in the OTP behaviours.

Own Id: OTP-13801

The support for Dets files created with Erlang/OTP R7 and earlier is removed.

Own Id: OTP-13830

Replaced usage of deprecated symbolic time unit representations.

Own Id: OTP-13831 Aux Id: OTP-13735

The function fmod/2 has been added to the math module.

Own Id: OTP-14000

The EXIT signals received from processes using proc_lib now looks like EXIT signals from processes that were spawned using spawn_link. In particular, that means that the stack trace is now included in the EXIT signal so that it can see where the process crashed.

Own Id: OTP-14001

sets:add_element/2 is faster when adding an element that is already present, and sets:del_element/2 is faster when the element to be deleted is not present. This optimization can make certain operations, such as sets:union/2 with many overlapping elements, up to two orders of magnitude faster.

Own Id: OTP-14035

Add information in doc about supervisor shutdown reason when maximum restart frequency is reached.

Own Id: OTP-14037 Aux Id: PR-1233

Added rand:jump/[0|1] functions.

Own Id: OTP-14038 Aux Id: PR-1235

Functions for detecting changed code has been added. code:modified_modules/0 returns all currently loaded modules that have changed on disk. code:module_status/1 returns the status for a module. In the shell and in c module, mm/0 is short for code:modified_modules/0, and lm/0 reloads all currently loaded modules that have changed on disk.

Own Id: OTP-14059

Each assert macro in assert.hrl now has a corresponding version with an extra argument, for adding comments to assertions. These can for example be printed as part of error reports, to clarify the meaning of the check that failed.

Own Id: OTP-14066

error_logger_tty_h and error_logger_file_h now inserts the node information for nonlocal messages before the message itself instead of after, both for readability and so as not to change the line termination property at the end of the message.

Own Id: OTP-14068

The Erlang code linter checks for badly formed type constraints.

Own Id: OTP-14070 Aux Id: PR-1214

By default, there will now be a warning when export_all is used. The warning can be disabled using nowarn_export_all.

Own Id: OTP-14071

When a gen_server process crashes, the stacktrace for the client will be printed to facilitate debugging.

Own Id: OTP-14089

Optimized ETS operations by changing table identifier type from integer to reference. The reference enables a more direct mapping to the table with less potential lock contention and makes especially creation and deletion of tables scale much better.

The change of the opaque type for the ETS table identifiers may cause failure in code that make faulty assumptions about this opaque type.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14094

take/2 has been added to dict, orddict, and gb_trees. take_any/2 has been added to gb_trees.

Own Id: OTP-14102

Extend gen_event API to handle options as well.

Own Id: OTP-14123

Advice on how to tune the supervisor restart frequency (intensity and period) is added to System Documentation - Design Principles - Supervisor Behaviour.

Own Id: OTP-14168 Aux Id: PR-1289

gen_fsm is deprecated and is replaced by gen_statem, however for backwards compatibility reasons gen_fsm may continue to exist as an undocumented feature for quite some time.

Own Id: OTP-14183

The shell functions c/1 and c/2 have been extended so that if the argument is a module name instead of a file name, it automatically locates the .beam file and the corresponding source file, and then recompiles the module using the same compiler options (plus any options passed to c/2). If compilation fails, the old beam file is preserved. Also adds c(Mod, Opts, Filter), where the Filter argument allows you to remove old compiler options before the new options are added.

New utility functions file_find/2/3 and find_source/1/2/3 have been added to filelib.

Own Id: OTP-14190

erl_tar in previous versions of OTP only supports the USTAR format. That limited path names to at most 255 bytes, and did not support Unicode characters in names in a portable way.

erl_tar now has support for reading tar archives in the formats currently in common use, such as v7, STAR, USTAR, PAX, and GNU tar's extensions to the STAR/USTAR format. When writing tar archives, erl_tar can now write them in the PAX format if necessary (for example, to support very long filenames or filenames with Unicode characters). If possible, erl_tar will still write tar archives in the USTAR for maximum portability.

Own Id: OTP-14226

base64:mime_decode/1 has been optimized so that it is now almost as fast asbase64:decode/1; it used be noticeably slower.

Own Id: OTP-14245

erl_tar will now strip any leading '/' from pathnames when extracting files from a tar archive and write a message to the error logger. There is also new check for directory traversal attacks; if a relative path points above the current working directory the extraction will be aborted.

Own Id: OTP-14278

Miscellaneous updates due to atoms containing arbitrary Unicode characters.

Own Id: OTP-14285

The Crypto application now supports generation of cryptographically strong random numbers (floats < 1.0 and integer arbitrary ranges) as a plugin to the 'rand' module.

Own Id: OTP-14317 Aux Id: PR-1372

Add new function ets:select_replace/2 which performs atomic "compare-and-swap" operations for ETS objects using match specifications.

Own Id: OTP-14319 Aux Id: PR-1076

The Erlang code linter checks for bad dialyzer attributes. It also checks for bad type variables in type declarations.

Own Id: OTP-14323

Two new functions has been implemented in the rand module; normal/2 and normal_s/3, that both produce normal distribution (pseudo) random numbers with mean value and variance according to arguments.

Own Id: OTP-14328 Aux Id: PR-1382

Upgraded the OTP internal PCRE library from version 8.33 to version 8.40. This library is used for implementation of the re regular expressions module.

Besides various bug fixes, the new version allows for better stack protection. In order to utilize this feature, the stack size of normal scheduler threads is now by default set to 128 kilo words on all platforms. The stack size of normal scheduler threads can be set upon system start by passing the +sss command line argument to the erl command.

See http://pcre.org/original/changelog.txt for information about changes made to PCRE between the versions 8.33 and 8.40.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-14331 Aux Id: ERL-208

Added function re:version/0 which returns information about the OTP internal PCRE version used for implementation of the re module.

Own Id: OTP-14347 Aux Id: PR-1412

The format of debug information that is stored in BEAM files (when debug_info is used) has been changed. The purpose of the change is to better support other BEAM-based languages such as Elixir or LFE.

All tools included in OTP (dialyzer, debugger, cover, and so on) will handle both the new format and the previous format. Tools that retrieve the debug information using beam_lib:chunk(Beam, [abstract_code]) will continue to work with both the new and old format. Tools that call beam_lib:chunk(Beam, ["Abst"]) will not work with the new format.

For more information, see the description of debug_info in the documentation for beam_lib and the description of the {debug_info,{Backend,Data}} option in the documentation for compile.

Own Id: OTP-14369 Aux Id: PR-1367

Add option hibernate_after to gen_server, gen_statem and gen_event. Also added to the deprecated gen_fsm behaviour.

Own Id: OTP-14405

The size of crash reports created by gen_server, gen_statem and proc_lib is limited with aid of the Kernel application variable error_logger_format_depth. The purpose is to limit the size of the messages sent to the error_logger process when processes with huge message queues or states crash.

The crash report generated by proc_lib includes the new tag message_queue_len. The neighbour report also includes the new tag current_stacktrace. Finally, the neighbour report no longer includes the tags messages and dictionary.

The new function error_logger:get_format_depth/0 can be used to retrieve the value of the Kernel application variable error_logger_format_depth.

Own Id: OTP-14417

An escript with only two lines would not work.

Own Id: OTP-14098

Characters ($char) can be used in constant pattern expressions. They can also be used in types and contracts.

Own Id: OTP-14103 Aux Id: ERL-313

The signatures of erl_parse:anno_to_term/1 and erl_parse:anno_from_term/1 are corrected. Using these functions no longer results in false Dialyzer warnings.

Own Id: OTP-14131

Pretty-printing of maps is improved.

Own Id: OTP-14175 Aux Id: seq13277

If any of the following functions in the zip module crashed, a file would be left open: extract(), unzip(), create(), or zip(). This has been corrected.

A zip file having a "Unix header" could not be unpacked.

Own Id: OTP-14189 Aux Id: ERL-348, ERL-349

Improve the Erlang shell's tab-completion of long names.

Own Id: OTP-14200 Aux Id: ERL-352

The reference manual for sys had some faulty information about the 'get_modules' message used by processes where modules change dynamically during runtime. The documentation is now corrected.

Own Id: OTP-14248 Aux Id: ERL-367

Bug fixes, new features and improvements to gen_statem:

A new type init_result/1 has replaced the old init_result/0, so if you used that old type (that was never documented) you have to change your code, which may be regarded as a potential incompatibility.

Changing callback modes after code change did not work since the new callback mode was not recorded. This bug has been fixed.

The event types state_timeout and {call,From} could not be generated with a {next_event,EventType,EventContent} action since they did not pass the runtime type check. This bug has now been corrected.

State entry calls can now be repeated using (new) state callback returns {repeat_state,...}, {repeat_state_and_data,_} and repeat_state_and_data.

There have been lots of code cleanup in particular regarding timer handling. For example is async cancel_timer now used. Error handling has also been cleaned up.

To align with probable future changes to the rest of gen_*, terminate/3 has now got a fallback and code_change/4 is not mandatory.

Own Id: OTP-14114

filename:safe_relative_path/1 to sanitize a relative path has been added.

Own Id: OTP-14215

When a simple_one_for_one supervisor is shutting down, and a child exits with an exit reason of the form {shutdown, Term}, an error report was earlier printed. This is now corrected.

Own Id: OTP-13907 Aux Id: PR-1158, ERL-163

Allow empty list as parameter of the fun used with dbg:fun2ms/1.

Own Id: OTP-13974

The new behaviour gen_statem has been improved with 3 new features: the possibility to use old style non-proxy timeouts for gen_statem:call/2,3, state entry code, and state timeouts. These are backwards compatible. Minor code and documentation improvements has been performed including a borderline semantics correction of timeout zero handling.

Own Id: OTP-13929 Aux Id: PR-1170, ERL-284

The zip:unzip/1,2 and zip:extract/1,2 functions have been updated to handle directory traversal exploits. Any element in the zip file that contains a path that points to a directory above the top level working directory, cwd, will instead be extracted in cwd. An error message is printed for any such element in the zip file during the unzip operation. The keep_old_files option determines if a file will overwrite a previous file with the same name within the zip file.

Own Id: OTP-13633

Correct the contracts for ets:match_object/1,3.

Own Id: OTP-13721 Aux Id: PR-1113

Errors in type specification and Emacs template generation for gen_statem:code_change/4 has been fixed from bugs.erlang.org's Jira cases ERL-172 and ERL-187.

Own Id: OTP-13746 Aux Id: ERL-172, ERL-187

gen_statem has been changed to set the callback mode for a server to what Module:callback_mode/0 returns. This facilitates e.g code downgrade since the callback mode now becomes a property of the currently active code, not of the server process.

Exception handling from Module:init/1 has also been improved.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13752

Correct a bug regarding typed records in the Erlang shell. The bug was introduced in OTP-19.0.

Own Id: OTP-13719 Aux Id: ERL-182

Fix a race bug affecting dets:open_file/2.

Own Id: OTP-13260 Aux Id: seq13002

Don't search for non-existing Map keys twice

For maps:get/2,3 and maps:find/2, searching for an immediate key, e.g. an atom, in a small map, the search was performed twice if the key did not exist.

Own Id: OTP-13459

Avoid stray corner-case math errors on Solaris, e.g. an error is thrown on underflows in exp() and pow() when it shouldn't be.

Own Id: OTP-13531

Fix linting of map key variables

Map keys cannot be unbound and then used in parallel matching.

Example: #{ K := V } = #{ k := K } = M. This is illegal if 'K' is not bound.

Own Id: OTP-13534 Aux Id: ERL-135

Fixed a bug in re on openbsd where sometimes re:run would return an incorrect result.

Own Id: OTP-13602

To avoid potential timer bottleneck on supervisor restart, timer server is no longer used when the supervisor is unable to restart a child.

Own Id: OTP-13618 Aux Id: PR-1001

The Erlang code preprocessor (epp) can handle file names spanning over many tokens. Example: -include("a" "file" "name")..

Own Id: OTP-13662 Aux Id: seq13136

The types of The Abstract Format in the erl_parse module have been refined.

Own Id: OTP-10292

Undocumented syntax for function specifications, -spec F/A :: Domain -> Range, has been removed (without deprecation).

Using the is_subtype(V, T) syntax for constraints (in function specifications) is no longer documented, and the newer syntax V :: T should be used instead. The Erlang Parser still recognizes the is_subtype syntax, and will continue to do so for some time.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11879

The 'random' module has been deprecated. Use the 'rand' module instead.

Own Id: OTP-12502 Aux Id: OTP-12501

Background: In record fields with a type declaration but without an initializer, the Erlang parser inserted automatically the singleton type 'undefined' to the list of declared types, if that value was not present there. That is, the record declaration:

-record(rec, {f1 :: float(), f2 = 42 :: integer(), f3 :: some_mod:some_typ()}).

was translated by the parser to:

-record(rec, {f1 :: float() | 'undefined', f2 = 42 :: integer(), f3 :: some_mod:some_typ() | 'undefined'}).

The rationale for this was that creation of a "dummy" #rec{} record should not result in a warning from dialyzer that, for example, the implicit initialization of the #rec.f1 field violates its type declaration.

Problems: This seemingly innocent action has some unforeseen consequences.

For starters, there is no way for programmers to declare that e.g. only floats make sense for the f1 field of #rec{} records when there is no "obvious" default initializer for this field. (This also affects tools like PropEr that use these declarations produced by the Erlang parser to generate random instances of records for testing purposes.)

It also means that dialyzer does not warn if e.g. an is_atom/1 test or something more exotic like an atom_to_list/1 call is performed on the value of the f1 field.

Similarly, there is no way to extend dialyzer to warn if it finds record constructions where f1 is not initialized to some float.

Last but not least, it is semantically problematic when the type of the field is an opaque type: creating a union of an opaque and a structured type is very problematic for analysis because it fundamentally breaks the opacity of the term at that point.

Change: To solve these problems the parser will not automatically insert the 'undefined' value anymore; instead the user has the option to choose the places where this value makes sense (for the field) and where it does not and insert the | 'undefined' there manually.

Consequences of this change: This change means that dialyzer will issue a warning for all places where records with uninitialized fields are created and those fields have a declared type that is incompatible with 'undefined' (e.g. float()). This warning can be suppressed easily by adding | 'undefined' to the type of this field. This also adds documentation that the user really intends to create records where this field is uninitialized.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12719

Remove deprecated functions in the modules erl_scan and erl_parse.

Own Id: OTP-12861

The pre-processor can now expand the ?FUNCTION_NAME and ?FUNCTION_ARITY macros.

Own Id: OTP-13059

A new behaviour gen_statem has been implemented. It has been thoroughly reviewed, is stable enough to be used by at least two heavy OTP applications, and is here to stay. But depending on user feedback, we do not expect but might find it necessary to make minor not backwards compatible changes into OTP-20.0, so its state can be designated as "not quite experimental"...

The gen_statem behaviour is intended to replace gen_fsm for new code. It has the same features and add some really useful:

The callback model(s) for gen_statem differs from the one for gen_fsm, but it is still fairly easy to rewrite from gen_fsm to gen_statem.

Own Id: OTP-13065 Aux Id: PR-960

Optimize binary:split/2 and binary:split/3 with native BIF implementation.

Own Id: OTP-13082

Background: The types of record fields have since R12B been put in a separate form by epp:parse_file(), leaving the record declaration form untyped. The separate form, however, does not follow the syntax of type declarations, and parse transforms inspecting -type() attributes need to know about the special syntax. Since the compiler stores the return value of epp:parse_file() as debug information in the abstract code chunk ("Abst" or abstract_code), tools too need to know about the special syntax, if they inspect -type() attributes in abstract code.

Change: No separate type form is created by epp:parse_file(), but the type information is kept in the record fields. This means that all parse transforms and all tools inspecting -record() declarations need to recognize {typed_record_field, Field, Type}.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13148

Unsized fields of the type bytes in binary generators are now forbidden. (The other ways of writing unsized fields, such as binary, are already forbidden.)

Own Id: OTP-13152

The type map() is built-in, and cannot be redefined.

Own Id: OTP-13153

Let dets:open_file() exit with a badarg message if given a raw file name (a binary).

Own Id: OTP-13229 Aux Id: ERL-55

Add filename:basedir/2,3

basedir returns suitable path(s) for 'user_cache', 'user_config', 'user_data', 'user_log', 'site_config' and 'site_data'. On linux and linux like systems the paths will respect the XDG environment variables.

Own Id: OTP-13392

There are new preprocessor directives -error(Term) and -warning(Term) to cause a compilation error or a compilation warning, respectively.

Own Id: OTP-13476

Optimize '++' operator and lists:append/2 by using a single pass to build a new list while checking for properness.

Own Id: OTP-13487

Add maps:update_with/3,4 and maps:take/2

Own Id: OTP-13522 Aux Id: PR-1025

lists:join/2 has been added. Similar to string:join/2 but works with arbitrary lists.

Own Id: OTP-13523

Obfuscate asserts to make Dialyzer shut up.

Own Id: OTP-13524 Aux Id: PR-1002

Supervisors now explicitly add their callback module in the return from sys:get_status/1,2. This is to simplify custom supervisor implementations. The Misc part of the return value from sys:get_status/1,2 for a supervisor is now:

[{data, [{"State", State}]},{supervisor,[{"Callback",Module}]}]

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-13619 Aux Id: PR-1000

Relax translation of initial calls in proc_lib, i.e. remove the restriction to only do the translation for gen_server and gen_fsm. This enables user defined gen based generic callback modules to be displayed nicely in c:i() and observer.

Own Id: OTP-13623

The function queue:lait/1 (misspelling of liat/1) is now deprecated.

Own Id: OTP-13658

Fix evaluation in matching of bound map key variables in the interpreter.

Prior to this patch, the following code would not evaluate: X = key,(fun(#{X := value}) -> true end)(#{X => value})

Own Id: OTP-13218

Fix erl_eval not using non-local function handler.

Own Id: OTP-13228 Aux Id: ERL-32

The Erlang Code Linter no longer crashes if there is a -deprecated() attribute but no -module() declaration.

Own Id: OTP-13230 Aux Id: ERL-62

The timestamp in the result returned by dets:info(Tab, safe_fixed) was unintentionally broken as a result of the time API rewrites in OTP 18.0. This has now been fixed.

Own Id: OTP-13239 Aux Id: OTP-11997

A rare race condition in beam_lib when using encrypted abstract format has been eliminated.

Own Id: OTP-13278

Improved maps:with/2 and maps:without/2 algorithms

The new implementation speeds up the execution significantly for all sizes of input.

Own Id: OTP-13376

Time warp safety improvements.

Introduced the options monotonic_timestamp, and strict_monotonic_timestamp to the trace, sequential trace, and system profile functionality. This since the already existing timestamp option is not time warp safe.

Introduced the option safe_fixed_monotonic_time to ets:info/2 and dets:info/2. This since the already existing safe_fixed option is not time warp safe.

Own Id: OTP-13222 Aux Id: OTP-11997

In the shell Ctrl+W (delete word) will no longer consider "." as being part of a word.

Own Id: OTP-13281

The Erlang Pretty Printer uses :: for function type constraints.

A bug concerning pretty printing of annotated type union elements in map pair types has been fixed.

Some minor issues regarding the documentation of types and specs have been corrected.

Own Id: OTP-13084

The shell command rp prints strings as lists of integers if pretty printing of lists is set to false.

Own Id: OTP-13145

The shell would crash if a bit syntax expression with conflicting types were given (e.g. if a field type was given as 'integer-binary'). (Thanks to Aleksei Magusev for reporting this bug.)

Own Id: OTP-13157

The rand:export_seed/0 would never return 'undefined' even if no seed has previously been created. Fixed to return 'undefined' if there is no seed in the process dictionary.

Own Id: OTP-13162

Add support for the Delete, Home and End keys in the Erlang shell.

Own Id: OTP-13032

beam_lib:all_chunks/1 and beam_lib:build_module/1 have been documented.

Own Id: OTP-13063

In OTP 18.0, qlc does not handle syntax errors well. This bug has been fixed.

Own Id: OTP-12946

Optimize zip:unzip/2 when uncompressing to memory.

Own Id: OTP-12950

The STDLIB reference manual is updated to show correct information about the return value of gen_fsm:reply/2.

Own Id: OTP-12973

re:split2,3 and re:replace/3,4 now correctly handles pre-compiled patterns that have been compiled using the 'unicode' option.

Own Id: OTP-12977

Export shell:catch_exception/1 as documented.

Own Id: OTP-12990

A mechanism for limiting the amount of text that the built-in error logger events will produce has been introduced. It is useful for limiting both the size of log files and the CPU time used to produce them.

This mechanism is experimental in the sense that it may be changed if it turns out that it does not solve the problem it is supposed to solve. In that case, there may be backward incompatible improvements to this mechanism.

See the documentation for the config parameter error_logger_format_depth in the Kernel application for information about how to turn on this feature.

Own Id: OTP-12864

Fix handling of single dot in filename:join/2

The reference manual says that filename:join(A,B) is equivalent to filename:join([A,B]). In some rare cases this turns out not to be true. For example:

filename:join("/a/.","b") -> "/a/./b" vs filename:join(["/a/.","b"]) -> "/a/b".

This has been corrected. A single dot is now only kept if it occurs at the very beginning or the very end of the resulting path.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12158

The undocumented option generic_debug for gen_server has been removed.

Own Id: OTP-12183

erl_lint:icrt_export/4 has been rewritten to make the code really follow the scoping rules of Erlang, and not just in most situations by accident.

Own Id: OTP-12186

Add 'trim_all' option to binary:split/3

This option can be set to remove _ALL_ empty parts of the result of a call to binary:split/3.

Own Id: OTP-12301

Correct orddict(3) regarding evaluation order of fold() and map().

Own Id: OTP-12651 Aux Id: seq12832

Correct maps module error exceptions

Bad input to maps module function will now yield the following exceptions:

Own Id: OTP-12657

It is now possible to paste text in JCL mode (using Ctrl-Y) that has been copied in the previous shell session. Also a bug that caused the JCL mode to crash when pasting text has been fixed.

Own Id: OTP-12673

Add uptime() shell command.

Own Id: OTP-12752

Cache nowarn_bif_clash functions in erl_lint.

This patch stores nowarn_bif_clash in the lint record. By using erlc +'{eprof,lint_module}' when compiling the erlang parser, we noticed the time spent on nowarn_function/2 reduced from 30% to 0.01%.

Own Id: OTP-12754

Optimize the Erlang Code Linter by using the cached filename information.

Own Id: OTP-12772

If a child of a simple_one_for_one returns ignore from its start function no longer store the child for any restart type. It is not possible to restart or delete the child because the supervisor is a simple_one_for_one.

Own Id: OTP-12793

Make ets:file2tab preserve enabled read_concurrency and write_concurrency options for tables.

Own Id: OTP-12814

There are many cases where user code needs to be able to distinguish between a socket that was closed normally and one that was aborted. Setting the option {show_econnreset, true} enables the user to receive ECONNRESET errors on both active and passive sockets.

Own Id: OTP-12841

Allow maps for supervisor flags and child specs

Earlier, supervisor flags and child specs were given as tuples. While this is kept for backwards compatibility, it is now also allowed to give these parameters as maps, see sup_flags and child_spec.

Own Id: OTP-11043

A new system message, terminate, is added. This can be sent with sys:terminate/2,3. If the receiving process handles system messages properly it will terminate shortly after receiving this message.

The new function proc_lib:stop/1,3 utilizes this new system message and monitors the receiving process in order to facilitate a synchronous stop mechanism for 'special processes'.

proc_lib:stop/1,3 is used by the following functions:

Own Id: OTP-11173 Aux Id: seq12353

Remove the pg module, which has been deprecated through OTP-17, is now removed from the STDLIB application. This module has been marked experimental for more than 15 years, and has largely been superseded by the pg2 module from the Kernel application.

Own Id: OTP-11907

New BIF: erlang:get_keys/0, lists all keys associated with the process dictionary. Note: erlang:get_keys/0 is auto-imported.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12151 Aux Id: seq12521

Add three new functions to io_lib-- scan_format/2, unscan_format/1, and build_text/1-- which expose the parsed form of the format control sequences to make it possible to easily modify or filter the input to io_lib:format/2. This can e.g. be used in order to replace unbounded-size control sequences like ~w or ~p with corresponding depth-limited ~W and ~P before doing the actual formatting.

Own Id: OTP-12167

Introduce the erl_anno module, an abstraction of the second element of tokens and tuples in the abstract format.

Own Id: OTP-12195

Support variables as Map keys in expressions and patterns

Erlang will accept any expression as keys in Map expressions and it will accept literals or bound variables as keys in Map patterns.

Own Id: OTP-12218

The last traces of Mnemosyne Rules have been removed.

Own Id: OTP-12257

Properly support maps in match_specs

Own Id: OTP-12270

New function ets:take/2. Works the same as ets:delete/2 but also returns the deleted object(s).

Own Id: OTP-12309

string:tokens/2 is somewhat faster, especially if the list of separators only contains one separator character.

Own Id: OTP-12422 Aux Id: seq12774

The documentation of the Abstract Format (in the ERTS User's Guide) has been updated with types and specification. (Thanks to Anthony Ramine.)

The explicit representation of parentheses used in types of the abstract format has been removed. Instead the new functions erl_parse:type_inop_prec() and erl_parse:type_preop_prec() can be used for inserting parentheses where needed.

Own Id: OTP-12492

Prevent zip:zip_open/[12] from leaking file descriptors if parent process dies.

Own Id: OTP-12566

Add a new random number generator, see rand module. It have better characteristics and an improved interface.

Own Id: OTP-12586 Aux Id: OTP-12501, OTP-12502

filename:split/1 when given an empty binary will now return an empty list, to make it consistent with return value when given an empty list.

Own Id: OTP-12716

Add sync option to ets:tab2file/3.

Own Id: OTP-12737 Aux Id: seq12805

Add functions gb_sets:iterator_from() and gb_trees:iterator_from(). (Thanks to Kirill Kinduk.)

Own Id: OTP-12742

Add maps:filter/2 to maps module.

Own Id: OTP-12745

Change some internal data structures to Maps in order to speed up compilation time. Measured speed up is around 10%-15%.

Own Id: OTP-12774

Update orddict to use parameterized types and specs. (Thanks to UENISHI Kota.)

Own Id: OTP-12785

The assert macros in eunit has been moved out to stdlib/include/assert.hrl. This files get included by eunit.hrl. Thus, nothing changes for eunit users, but the asserts can now also be included separately.

Own Id: OTP-12808

Behaviour of character types \d, \w and \s has always been to not match characters with value above 255, not 128, i.e. they are limited to ISO-Latin-1 and not ASCII

Own Id: OTP-12521

c:m/1 now displays the module's MD5 sum.

Own Id: OTP-12500

Make ets:i/1 handle binary input from IO server.

Own Id: OTP-12550

The documentation of string:tokens/2 now explicitly specifies that adjacent separator characters do not give any empty strings in the resulting list of tokens.

Own Id: OTP-12036

Fix broken deprecation warnings in ssh application

Own Id: OTP-12187

Maps: Properly align union typed assoc values in documentation

Own Id: OTP-12190

Fix filelib:wildcard/2 when 'Cwd' ends with a dot

Own Id: OTP-12212

Allow Name/Arity syntax in maps values inside attributes.

Own Id: OTP-12213

Fix edlin to correctly save text killed with ctrl-u. Prior to this fix, entering text into the Erlang shell and then killing it with ctrl-u followed by yanking it back with ctrl-y would result in the yanked text being the reverse of the original killed text.

Own Id: OTP-12224

If a callback function was terminated with exit/1, there would be no stack trace in the ERROR REPORT produced by gen_server. This has been corrected.

To keep the backwards compatibility, the actual exit reason for the process is not changed.

Own Id: OTP-12263 Aux Id: seq12733

Warnings produced by ms_transform could point out the wrong line number.

Own Id: OTP-12264

Supports tar file creation on other media than file systems mounted on the local machine.

The erl_tar api is extended with erl_tar:init/3 that enables usage of user provided media storage routines. A ssh-specific set of such routines is hidden in the new function ssh_sftp:open_tar/3 to simplify creating a tar archive on a remote ssh server.

A chunked file reading option is added to erl_tar:add/3,4 to save memory on e.g small embedded systems. The size of the slices read from a file in that case can be specified.

Own Id: OTP-12180 Aux Id: seq12715

I/O requests are optimized for long message queues in the calling process.

Own Id: OTP-12315

The type spec of the FormFunc argument to sys:handle_debug/4 was erroneously pointing to dbg_fun(). This is now corrected and the new type is format_fun().

Own Id: OTP-11800

Behaviors such as gen_fsm and gen_server should always invoke format_status/2 before printing the state to the logs.

Own Id: OTP-11967

The documentation of dets:insert_new/2 has been corrected. (Thanks to Alexei Sholik for reporting the bug.)

Own Id: OTP-12024

Printing a term with io_lib:format and control sequence w, precision P and field width F, where F< P would fail in one of the two following ways:

1) If P < printed length of the term, an infinite loop would be entered, consuming all available memory.

2) If P >= printed length of the term, an exception would be raised.

These two problems are now corrected.

Own Id: OTP-12041

The documentation of maps:values/1 has been corrected.

Own Id: OTP-12055

Expand shell functions in map expressions.

Own Id: OTP-12063

Add maps:with/2

Own Id: OTP-12137

OTP-11850 fixed filelib:wildcard/1 to work with broken symlinks. This correction, however, introduced problems since symlinks were no longer followed for functions like filelib:ensure_dir/1, filelib:is_dir/1, filelib:file_size/1, etc. This is now corrected.

Own Id: OTP-12054 Aux Id: seq12660

filelib:wildcard("broken_symlink") would return an empty list if "broken_symlink" was a symlink that did not point to an existing file.

Own Id: OTP-11850 Aux Id: seq12571

erl_tar can now handle files names that contain Unicode characters. See "UNICODE SUPPORT" in the documentation for erl_tar.

When creating a tar file, erl_tar would sometime write a too short end of tape marker. GNU tar would correctly extract files from such tar file, but would complain about "A lone zero block at...".

Own Id: OTP-11854

When redefining and exporting the type map() the Erlang Code Linter (erl_lint) erroneously emitted an error. This bug has been fixed.

Own Id: OTP-11872

Fix evaluation of map updates in the debugger and erl_eval

Reported-by: José Valim

Own Id: OTP-11922

The following native functions now bump an appropriate amount of reductions and yield when out of reductions:

Characteristics impact:

Own Id: OTP-11888

Add maps:get/3 to maps module. The function will return the supplied default value if the key does not exist in the map.

Own Id: OTP-11951

The option dupnames did not work as intended in re. When looking for names with {capture, [Name, ...]}, re:run returned a random instance of the match for that name, instead of the leftmost matching instance, which was what the documentation stated. This is now corrected to adhere to the documentation. The option {capture,all_names} along with a re:inspect/2 function is also added to further help in using named subpatterns.

Own Id: OTP-11205

If option 'binary' was set for standard_input, then c:i() would hang if the output was more than one page long - i.e. then input after "(c)ontinue (q)uit -->" could not be read. This has been corrected. (Thanks to José Valim)

Own Id: OTP-11589

stdlib/lists: Add function droplast/1 This functions drops the last element of a non-empty list. lists:last/1 and lists:droplast/1 are the dual of hd/1 and tl/1 but for the end of a list. (Thanks to Hans Svensson)

Own Id: OTP-11677

Allow all auto imports to be suppressed at once. Introducing the no_auto_import attribute: -compile(no_auto_import). Useful for code generation tools that always use the qualified function names and want to avoid the auto imported functions clashing with local ones. (Thanks to José Valim.)

Own Id: OTP-11682

supervisor_bridge does no longer report normal termination of children. The reason is that in some cases, for instance when the restart strategy is simple_one_for_one, the log could be completely overloaded with reports about normally terminating processes. (Thanks to Artem Ocheredko)

Own Id: OTP-11685

The type annotations for alternative registries using the {via, Module, Name} syntax for sup_name() and sup_ref() in the supervisor module are now consistent with the documentation. Dialyzer should no longer complain about valid supervisor:start_link() and supervisor:start_child() calls. (Thanks to Caleb Helbling.)

Own Id: OTP-11707

Two Dets bugs have been fixed. When trying to open a short file that is not a Dets file, the file was deleted even with just read access. Calling dets:is_dets_file/1 with a file that is not a Dets file, a file descriptor was left open. (Thanks to Håkan Mattsson for reporting the bugs.)

Own Id: OTP-11709

Fix race bug in ets:all. Concurrent creation of tables could cause other tables to not be included in the result. (Thanks to Florian Schintke for bug report)

Own Id: OTP-11726

erl_eval now properly evaluates '=='/2 when it is used in guards. (Thanks to José Valim)

Own Id: OTP-11747

Calls to proplists:get_value/3 are replaced by the faster lists:keyfind/3 in io_lib_pretty. Elements in the list are always 2-tuples. (Thanks to Andrew Thompson)

Own Id: OTP-11752

A qlc bug where filters were erroneously optimized away has been fixed. Thanks to Sam Bobroff for reporting the bug.

Own Id: OTP-11758

A number of compiler errors where unusual or nonsensical code would crash the compiler have been reported by Ulf Norell and corrected by Anthony Ramine.

Own Id: OTP-11770

Since Erlang/OTP R16B the Erlang Core Linter (erl_lint) has not emitted errors when built-in types were re-defined. This bug has been fixed. (Thanks to Roberto Aloi.)

Own Id: OTP-11772

The functions sys:get_state/1,2 and sys:replace_state/2,3 are fixed so they can now be run while the process is sys suspended. To accomplish this, the new callbacks Mod:system_get_state/1 and Mod:system_replace_state/2 are added, which are also implemented by the generic behaviours gen_server, gen_event and gen_fsm.

The potential incompatibility refers to:

(Thanks to James Fish and Steve Vinoski)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11817

Options to set match_limit and match_limit_recursion are added to re:run. The option report_errors is also added to get more information when re:run fails due to limits or compilation errors.

Own Id: OTP-10285

The pre-defined types array/0, dict/0, digraph/0, gb_set/0, gb_tree/0, queue/0, set/0, and tid/0 have been deprecated. They will be removed in Erlang/OTP 18.0.

Instead the types array:array/0, dict:dict/0, digraph:graph/0, gb_set:set/0, gb_tree:tree/0, queue:queue/0, sets:set/0, and ets:tid/0 can be used. (Note: it has always been necessary to use ets:tid/0.)

It is allowed in Erlang/OTP 17.0 to locally re-define the types array/0, dict/0, and so on.

New types array:array/1, dict:dict/2, gb_sets:set/1, gb_trees:tree/2, queue:queue/1, and sets:set/1 have been added.

A compiler option, nowarn_deprecated_type, has been introduced. By including the attribute

in an Erlang source file, warnings about deprecated types can be avoided in Erlang/OTP 17.0.

The option can also be given as a compiler flag:

Own Id: OTP-10342

Calls to erlang:open_port/2 with 'spawn' are updated to handle space in the command path.

Own Id: OTP-10842

Dialyzer's unmatched_return warnings have been corrected.

Own Id: OTP-10908

Forbid unsized fields in patterns of binary generators and simplified v3_core's translation of bit string generators. (Thanks to Anthony Ramine.)

Own Id: OTP-11186

The version of the PCRE library Used by Erlang's re module is raised to 8.33 from 7.6. This means, among other things, better Unicode and Unicode Character Properties support. New options connected to PCRE 8.33 are also added to the re module (ucd, notempty_atstart, no_start_optimize). PCRE has extended the regular expression syntax between 7.6 and 8.33, why this imposes a potential incompatibility. Only very complicated regular expressions may be affected, but if you know you are using obscure features, please test run your regular expressions and verify that their behavior has not changed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11204

Added dict:is_empty/1 and orddict:is_empty/1. (Thanks to Magnus Henoch.)

Own Id: OTP-11353

A call to either the garbage_collect/1 BIF or the check_process_code/2 BIF may trigger garbage collection of another processes than the process calling the BIF. The previous implementations performed these kinds of garbage collections without considering the internal state of the process being garbage collected. In order to be able to more easily and more efficiently implement yielding native code, these types of garbage collections have been rewritten. A garbage collection like this is now triggered by an asynchronous request signal, the actual garbage collection is performed by the process being garbage collected itself, and finalized by a reply signal to the process issuing the request. Using this approach processes can disable garbage collection and yield without having to set up the heap in a state that can be garbage collected.

The garbage_collect/2, and check_process_code/3 BIFs have been introduced. Both taking an option list as last argument. Using these, one can issue asynchronous requests.

code:purge/1 and code:soft_purge/1 have been rewritten to utilize asynchronous check_process_code requests in order to parallelize work.

Characteristics impact: A call to the garbage_collect/1 BIF or the check_process_code/2 BIF will normally take longer time to complete while the system as a whole wont be as much negatively effected by the operation as before. A call to code:purge/1 and code:soft_purge/1 may complete faster or slower depending on the state of the system while the system as a whole wont be as much negatively effected by the operation as before.

Own Id: OTP-11388 Aux Id: OTP-11535, OTP-11648

Improve the documentation of the supervisor's via reference. (Thanks to MaximMinin.)

Own Id: OTP-11399

orddict:from_list/1 now uses the optimized sort routines in the lists module instead of (essentially) an insertion sort. Depending on the input data, the speed of the new from_list/1 is anything from slightly faster up to several orders of magnitude faster than the old from_list/1.

Own Id: OTP-11552

EEP43: New data type - Maps

With Maps you may for instance:

For information on how to use Maps please see Map Expressions in the Reference Manual.

The current implementation is without the following features:

Note that Maps is experimental during OTP 17.0.

Own Id: OTP-11616

When tab completing the erlang shell now expands zero-arity functions all the way to closing parenthesis, unless there is another function with the same name and a different arity. (Thanks to Pierre Fenoll.)

Own Id: OTP-11684

The Erlang Code Preprocessor (epp) could loop when encountering a circular macro definition in an included file. This bug has been fixed.

Thanks to Maruthavanan Subbarayan for reporting the bug, and to Richard Carlsson for providing a bug fix.

Own Id: OTP-11728

The Erlang Code Linter (erl_lint) has since Erlang/OTP R13B emitted warnings whenever any of the types arity(), bitstring(), iodata(), or boolean() were re-defined. Now errors are emitted instead.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11771

The encoding option of erl_parse:abstract/2 has been extended to include none and a callback function (a predicate).

Own Id: OTP-11807

Export zip option types to allow referal from other modules.

Thanks to Pierre Fenoll and Håkan Mattson

Own Id: OTP-11828

The module pg has been deprecated and will be removed in Erlang/OTP 18.

Own Id: OTP-11840

Fix typo in gen_server.erl. Thanks to Brian L. Troutwine.

Own Id: OTP-11398

Spec for atan2 should be atan2(Y, X), not atan2(X, Y). Thanks to Ary Borenszweig.

Own Id: OTP-11465

Add XML marker for regexp syntax. Thanks to Håkan Mattson.

Own Id: OTP-11442

The functions dets:foldl/3, dets:foldr/3, and dets:traverse/2 did not release the table after having traversed the table to the end. The bug was introduced in R16B. (Thanks to Manuel Duran Aguete.)

Own Id: OTP-11245

If the fun M:F/A construct was used erroneously the linter could crash. (Thanks to Mikhail Sobolev.)

Own Id: OTP-11254

The specifications of io_lib:fread/2,3 have been corrected. (Thanks to Chris King and Kostis Sagonas for pinpointing the bug.)

Own Id: OTP-11261

Fixed type typo in gen_server.

Own Id: OTP-11200

Update type specs in filelib and io_prompt. Thanks to Jose Valim.

Own Id: OTP-11208

Fix typo in abcast() function comment. Thanks to Johannes Weissl.

Own Id: OTP-11219

Make edlin understand a few important control keys. Thanks to Stefan Zegenhagen.

Own Id: OTP-11251

Export the edge/0 type from the digraph module. Thanks to Alex Ronne Petersen.

Own Id: OTP-11266

Fix variable usage tracking in erl_lint and fixed unsafe variable tracking in try expressions. Thanks to Anthony Ramine.

Own Id: OTP-11268

The Erlang scanner no longer accepts floating point numbers in the input string.

Own Id: OTP-10990

When converting a faulty binary to a list with unicode:characters_to_list, the error return value could contain a faulty "rest", i.e. the io_list of characters that could not be converted was wrong. This happened only if input was a sub binary and conversion was from utf8. This is now corrected.

Own Id: OTP-11080

The type hook_function() has been corrected in erl_pp, the Erlang Pretty Printer.

The printing of invalid forms, e.g. record field types, has also been fixed. It has been broken since R16B.

(Thanks to Tomáš Janoušek.)

Own Id: OTP-11100

Fix receive support in erl_eval with a BEAM module. Thanks to Anthony Ramine.

Own Id: OTP-11137

Delete obsolete note about simple-one-for-one supervisor. Thanks to Magnus Henoch.

Own Id: OTP-10938

When selecting encoding of a script written in Erlang (escript) the optional directive on the second line is now recognized.

Own Id: OTP-10951

The function erl_parse:abstract/2 has been documented.

Own Id: OTP-10992

Integrate elliptic curve contribution from Andreas Schultz

In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto.

This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API.

Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments.

Own Id: OTP-11009

Added sys:get_state/1,2 and sys:replace_state/2,3. Thanks to Steve Vinoski.

Own Id: OTP-11013

Optimizations to gen mechanism. Thanks to Loïc Hoguin.

Own Id: OTP-11025

Optimizations to gen.erl. Thanks to Loïc Hoguin.

Own Id: OTP-11035

Use erlang:demonitor(Ref, [flush]) where applicable. Thanks to Loïc Hoguin.

Own Id: OTP-11039

Erlang source files with non-ASCII characters are now encoded in UTF-8 (instead of latin1).

Own Id: OTP-11041 Aux Id: OTP-10907

Fix rest_for_one and one_for_all restarting a child not terminated. Thanks to James Fish.

Own Id: OTP-11042

Fix excessive CPU consumption of timer_server. Thanks to Aliaksey Kandratsenka.

Own Id: OTP-11053

Rename and document lists:zf/2 as lists:filtermap/2. Thanks to Anthony Ramine.

Own Id: OTP-11078

Fixed an inconsistent state in epp. Thanks to Anthony Ramine

Own Id: OTP-11079

c:ls(File) will now print File, similar to ls(1) in Unix. The error messages have also been improved. (Thanks to Bengt Kleberg.)

Own Id: OTP-11108

Support callback attributes in erl_pp. Thanks to Anthony Ramine.

Own Id: OTP-11140

Improve erl_lint performance. Thanks to José Valim.

Own Id: OTP-11143

Bugs related to Unicode have been fixed in the erl_eval module.

Own Id: OTP-10622 Aux Id: kunagi-351 [262]

filelib:wildcard("some/relative/path/*.beam", Path) would fail to match any file. That is, filelib:wildcard/2 would not work if the first component of the pattern did not contain any wildcard characters. (A previous attempt to fix the problem in R15B02 seems to have made matters worse.)

(Thanks to Samuel Rivas and Tuncer Ayaz.)

There is also an incompatible change to the Path argument. It is no longer allowed to be a binary.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10812

The new STDLIB application variable shell_strings can be used for determining how the Erlang shell outputs lists of integers. The new function shell:strings/1 toggles the value of the variable.

The control sequence modifier l can be used for turning off the string recognition of ~p and ~P.

Own Id: OTP-10755

Miscellaneous updates due to Unicode support.

Own Id: OTP-10820

Extend ~ts to handle binaries with characters coded in ISO-latin-1

Own Id: OTP-10836

The +pc flag to erl can be used to set the range of characters considered printable. This affects how the shell and io:format("~tp",...) functionality does heuristic string detection. More can be read in STDLIB users guide.

Own Id: OTP-10884

Wildcards such as "some/path/*" passed to filelib:wildcard/2 would fail to match any file. (Thanks to Samuel Rivas for reporting this bug.)

Own Id: OTP-6874 Aux Id: kunagi-190 [101]

Fixed error handling in proc_lib:start which could hang if the spawned process died in init.

Own Id: OTP-9803 Aux Id: kunagi-209 [120]

Allow ** in filelib:wildcard

Two adjacent * used as a single pattern will match all files and zero or more directories and subdirectories. (Thanks to José Valim)

Own Id: OTP-10431

Add the \gN and \g{N} syntax for back references in re:replace/3,4 to allow use with numeric replacement strings. (Thanks to Vance Shipley)

Own Id: OTP-10455

Export ets:match_pattern/0 type (Thanks to Joseph Wayne Norton)

Own Id: OTP-10472

Fix printing the empty binary at depth 1 with ~W (Thanks to Andrew Thompson)

Own Id: OTP-10504

The type ascii_string() in the base64 module has been corrected. The type file:file_info() has been cleaned up. The type file:fd() has been made opaque in the documentation.

Own Id: OTP-10624 Aux Id: kunagi-352 [263]

Dets tables are no longer fixed while traversing with a bound key (when only the objects with the right key are matched). This optimization affects the functions match/2, match_object/2, select/2, match_delete/2, and select_delete/2.

Own Id: OTP-10097

Support for Unicode has been implemented.

Own Id: OTP-10302

The linter now warns for opaque types that are not exported, as well as for under-specified opaque types.

Own Id: OTP-10436

The type file:name() has been substituted for the type file:filename() in the following functions in the filename module: absname/2, absname_join/2, join/1,2, and split/1.

Own Id: OTP-10474

If a child process fails in its start function, then the error reason was earlier only reported as an error report from the error_handler, and supervisor:start_link would only return {error,shutdown}. This has been changed so the supervisor will now return {error,{shutdown,Reason}}, where Reason identifies the failing child and its error reason. (Thanks to Tomas Pihl)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-10490

Where necessary a comment stating encoding has been added to Erlang files. The comment is meant to be removed in Erlang/OTP R17B when UTF-8 becomes the default encoding.

Own Id: OTP-10630

The contracts and types of the modules erl_scan and sys have been corrected and improved. (Thanks to Kostis Sagonas.)

Own Id: OTP-10658

The Erlang shell now skips the rest of the line when it encounters an Erlang scanner error.

Own Id: OTP-10659

Clean up some specs in the proplists module. (Thanks to Kostis Sagonas.)

Own Id: OTP-10663

Some examples overflowing the width of PDF pages have been corrected.

Own Id: OTP-10665

Enable escript to accept emulator arguments when script file has no shebang. Thanks to Magnus Henoch

Own Id: OTP-10691

Fix bug in queue:out/1, queue:out_r/1 that makes it O(N^2) in worst case. Thanks to Aleksandr Erofeev.

Own Id: OTP-10722

There are new functions in the epp module which read the character encoding from files. See epp(3) for more information.

Own Id: OTP-10742 Aux Id: OTP-10302

The functions in io_lib have been adjusted for Unicode. The existing functions write_string() and so on now take Unicode strings, while the old behavior has been taken over by new functions write_latin1_string() and so on. There are also new functions to write Unicode strings as Latin-1 strings, mainly targetted towards the Erlang pretty printer (erl_pp).

Own Id: OTP-10745 Aux Id: OTP-10302

The new functions proc_lib:format/2 and erl_parse:abstract/2 accept an encoding as second argument.

Own Id: OTP-10749 Aux Id: OTP-10302

Increased potential concurrency in ETS for write_concurrency option. The number of internal table locks has increased from 16 to 64. This makes it four times less likely that two concurrent processes writing to the same table would collide and thereby serialized. The cost is an increased constant memory footprint for tables using write_concurrency. The memory consumption per inserted record is not affected. The increased footprint can be particularly large if write_concurrency is combined with read_concurrency.

Own Id: OTP-10787

Minor test updates

Own Id: OTP-10591

Fixed bug where if given an invalid drive letter on windows ensure dir would go into an infinite loop.

Own Id: OTP-10104

Calls to gen_server:enter_loop/4 where ServerName has a global scope and no timeout is given now works correctly.

Thanks to Sam Bobroff for reporting the issue.

Own Id: OTP-10130

fix escript/primary archive reloading

If the mtime of an escript/primary archive file changes after being added to the code path, correctly reload the archive and update the cache. (Thanks to Tuncer Ayaz)

Own Id: OTP-10151

Fix bug that in some cases could cause corrupted binaries in ETS tables with compressed option.

Own Id: OTP-10182

Fix filename:nativename/1 on Win32

Don't choke on paths given as binary argument on Win32. Thanks to Jan Klötzke

Own Id: OTP-10188

Fix bug in ets:test_ms/2 that could cause emulator crash when using '$_' in match spec.

Own Id: OTP-10190

Fix bug where zip archives wrongly have a first disk number set to 1

Own Id: OTP-10223

The message printed by the Erlang shell as an explanation of the badarith error has been corrected. (Thanks to Matthias Lang.)

Own Id: OTP-10054

References to is_constant/1 (which was removed in the R12 release) has been removed from documentation and code.

Own Id: OTP-6454 Aux Id: seq10407

Leave control back to gen_server during supervisor's restart loop

When an attempt to restart a child failed, supervisor would earlier keep the execution flow and try to restart the child over and over again until it either succeeded or the restart frequency limit was reached. If none of these happened, supervisor would hang forever in this loop.

This commit adds a timer of 0 ms where the control is left back to the gen_server which implements the supervisor. This way any incoming request to the supervisor will be handled - which could help breaking the infinite loop - e.g. shutdown request for the supervisor or for the problematic child.

This introduces some incompatibilities in stdlib due to new return values from supervisor:

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9549

If a temporary child's start function returned 'ignore', then the supervisor would keep it's child specification. This has been corrected. Child specifications for non-existing temporary children shall never be kept.

Own Id: OTP-9782 Aux Id: seq11964

Use universal time as base in error logger

Previous conversion used the deprecated calendar:local_time_to_universal_time/1

Own Id: OTP-9854

Calling a guard test (such as is_list/1) from the top-level in a guard, would cause a compiler crash if there was a local definition with the same name. Corrected to reject the program with an error message.

Own Id: OTP-9866

Fix the type spec from the doc of binary:part/3 (Thanks to Ricardo Catalinas Jiménez)

Own Id: OTP-9920

Correct spelling of registered (Thanks to Richard Carlsson)

Own Id: OTP-9925

Put gb_trees documentation into alphabetical order (Thanks to Aidan Hobson Sayers)

Own Id: OTP-9929

Fix bug in ETS with compressed option and insertion of term containing large integers (>2G) on 64-bit machines. Seen to cause emulator crash. (Thanks to Diego Llarrull for excellent bug report)

Own Id: OTP-9932

Add plugin support for alternative name lookup This patch introduces a new way of locating a behaviour instance: {via, Module, Name}. (Thanks to Ulf Wiger)

Own Id: OTP-9945

The function digraph_utils:condensation/1 used to create a digraph containing loops contradicting the documentation which states that the created digraph is free of cycles. This bug has been fixed. (Thanks to Kostis Sagonas for finding the bug.)

Own Id: OTP-9953

When an escript ends now all printout to standard output and standard error gets out on the terminal. This bug has been corrected by changing the behaviour of erlang:halt/0,1, which should fix the same problem for other escript-like applications, i.e that data stored in the output port driver buffers got lost when printing on a TTY and exiting through erlang:halt/0,1.

The BIF:s erlang:halt/0,1 has gotten improved semantics and there is a new BIF erlang:halt/2 to accomplish something like the old semantics. See the documentation.

Now erlang:halt/0 and erlang:halt/1 with an integer argument will close all ports and allow all pending async threads operations to finish before exiting the emulator. Previously erlang:halt/0 and erlang:halt(0) would just wait for pending async threads operations but not close ports. And erlang:halt/1 with a non-zero integer argument would not even wait for pending async threads operations.

To roughly the old behaviour, to not wait for ports and async threads operations when you exit the emulator, you use erlang:halt/2 with an integer first argument and an option list containing {flush,false} as the second argument. Note that now is flushing not dependant of the exit code, and you can not only flush async threads operations which we deemed as a strange behaviour anyway.

Also, erlang:halt/1,2 has gotten a new feature: If the first argument is the atom 'abort' the emulator is aborted producing a core dump, if the operating system so allows.

Own Id: OTP-9985

Add escript win32 alternative invocation. escript can now be started as both "escript.exe" and "escript" (Thanks to Pierre Rouleau)

Own Id: OTP-9997

Improved algorithm in module random. Avoid seed values that are even divisors of the primes and by that prevent getting sub-seeds that are stuck on zero. Worst case was random:seed(0,0,0) that produced a series of only zeros. This is an incompatible change in the sense that applications that relies on reproducing a specific series for a given seed will fail. The pseudo random output is still deterministic but different compared to earlier versions.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8713

Calls to global:whereis_name/1 have been substituted for calls to global:safe_whereis_name/1 since the latter is not safe at all.

The reason for not doing this earlier is that setting a global lock masked out a bug concerning the restart of supervised children. The bug has now been fixed by a modification of global:whereis_name/1. (Thanks to Ulf Wiger for code contribution.)

A minor race conditions in gen_fsm:start* has been fixed: if one of these functions returned {error, Reason} or ignore, the name could still be registered (either locally or in global. (This is the same modification as was done for gen_server in OTP-7669.)

The undocumented function global:safe_whereis_name/1 has been removed.

Own Id: OTP-9212 Aux Id: seq7117, OTP-4174

If a child of a supervisor terminates with reason {shutdown,Term} it is now handled by the supervisor as if the reason was 'shutdown'.

For children with restart type 'permanent', this implies no change. For children with restart type 'transient', the child will no longer be restarted and no supervisor report will be written. For children with restart type 'temporary', no supervisor report will be written.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9222

Minor improvement of documentation regarding supervisor restart strategy for temporary and transient child processes.

Own Id: OTP-9381

A Dets table with sufficiently large buckets could not always be repaired. This bug has been fixed.

The format of Dets files has been modified. When downgrading tables created with the new system will be repaired. Otherwise the modification should not be noticeable.

Own Id: OTP-9607

A few contracts in the lists module have been corrected.

Own Id: OTP-9616

Add '-callback' attributes in stdlib's behaviours

Replace the behaviour_info(callbacks) export in stdlib's behaviours with -callback' attributes for all the callbacks. Update the documentation with information on the callback attribute Automatically generate 'behaviour_info' function from '-callback' attributes

'behaviour_info(callbacks)' is a special function that is defined in a module which describes a behaviour and returns a list of its callbacks.

This function is now automatically generated using the '-callback' specs. An error is returned by lint if user defines both '-callback' attributes and the behaviour_info/1 function. If no type info is needed for a callback use a generic spec for it. Add '-callback' attribute to language syntax

Behaviours may define specs for their callbacks using the familiar spec syntax, replacing the '-spec' keyword with '-callback'. Simple lint checks are performed to ensure that no callbacks are defined twice and all types referred are declared.

These attributes can be then used by tools to provide documentation to the behaviour or find discrepancies in the callback definitions in the callback module.

Add callback specs into 'application' module in kernel Add callback specs to tftp module following internet documentation Add callback specs to inets_service module following possibly deprecated comments

Own Id: OTP-9621

If a Dets table had been properly closed but the space management data could not been read, it was not possible to repair the file. This bug has been fixed.

Own Id: OTP-9622

The Unicode noncharacter code points 16#FFFE and 16#FFFE were not allowed to be encoded or decoded using the unicode module or bit syntax. That was inconsistent with the other noncharacters 16#FDD0 to 16#FDEF that could be encoded/decoded. To resolve the inconsistency, 16#FFFE and 16#FFFE can now be encoded and decoded. (Thanks to Alisdair Sullivan.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9624

Make epp search directory of current file first when including another file This completes a partial fix in R11 that only worked for include_lib(). (Thanks to Richard Carlsson)

Own Id: OTP-9645

ms_transform: Fix incorrect `variable shadowed' warnings

This patch removes incorrect passing of variable bindings from one function clause to another. (Thanks to Haitao Li)

Own Id: OTP-9646

Explicitly kill dynamic children in supervisors

According to the supervisor's documentation: "Important note on simple-one-for-one supervisors: The dynamically created child processes of a simple-one-for-one supervisor are not explicitly killed, regardless of shutdown strategy, but are expected to terminate when the supervisor does (that is, when an exit signal from the parent process is received)."

All is fine as long as we stop simple_one_for_one supervisor manually. Dynamic children catch the exit signal from the supervisor and leave. But, if this happens when we stop an application, after the top supervisor has stopped, the application master kills all remaining processes associated to this application. So, dynamic children that trap exit signals can be killed during their cleanup (here we mean inside terminate/2). This is unpredictable and highly time-dependent.

In this commit, supervisor module is patched to explicitly terminate dynamic children accordingly to the shutdown strategy.

NOTE: Order in which dynamic children are stopped is not defined. In fact, this is "almost" done at the same time.

Stack errors when dynamic children are stopped

Because a simple_one_for_one supervisor can have many workers, we stack errors during its shutdown to report only one message for each encountered error type. Instead of reporting the child's pid, we use the number of concerned children. (Thanks to Christopher Faulet)

Own Id: OTP-9647

Allow an infinite timeout to shutdown worker processes

Now, in child specification, the shutdown value can also be set to infinity for worker children. This restriction was removed because this is not always possible to predict the shutdown time for a worker. This is highly application-dependent. Add a warning to docs about workers' shutdown strategy (Thanks to Christopher Faulet)

Own Id: OTP-9648

A badarg would sometimes occur in supervisor when printing error reports and the child pid was undefined. This has been corrected.

Own Id: OTP-9669

Fix re:split spec not to accept option 'global'(Thanks to Shunichi Shinohara)

Own Id: OTP-9691

Fix a few tests that used to fail on the HiPE platform.

Own Id: OTP-9637

Variables are now now allowed in 'fun M:F/A' as suggested by Richard O'Keefe in EEP-23.

The representation of 'fun M:F/A' in the abstract format has been changed in an incompatible way. Tools that directly read or manipulate the abstract format (such as parse transforms) may need to be updated. The compiler can handle both the new and the old format (i.e. extracting the abstract format from a pre-R15 BEAM file and compiling it using compile:forms/1,2 will work). The syntax_tools application can also handle both formats.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9643

Tuple funs (a two-element tuple with a module name and a function) are now officially deprecated and will be removed in R16. Use 'fun M:F/A' instead. To make you aware that your system uses tuple funs, the very first time a tuple fun is applied, a warning will be sent to the error logger.

Own Id: OTP-9649

The deprecated 'regexp' module has been removed. Use the 're' module instead.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9737

filename:find_src/1,2 will now work on stripped BEAM files (reported by Per Hedeland). The HiPE compiler will also work on stripped BEAM files. The BEAM compiler will no longer include compilation options given in the source code itself in M:module_info(compile) (because those options will be applied anyway if the module is re-compiled).

Own Id: OTP-9752

erl_tar:extract failed when executed inside a directory with some parent directory to which the user has no read access. This has been corrected.

Own Id: OTP-9368

A bug in erl_scan:set_attribute/3 has been fixed.

Own Id: OTP-9412

The contract of io_lib:fread() has been corrected.

Own Id: OTP-9413 Aux Id: seq11873

A crash in io_lib:fread/2 when end of input data was encountered while trying to match literal characters, which should return {more,_,_,_} but instead crashed, has been corrected. Reported by Klas Johansson.

A similar peculiarity for io:fread when encountering end of file before any field data has also been corrected.

Own Id: OTP-9439

The contract of timer:now_diff() has been corrected. (Thanks to Alex Morarash).

Own Id: OTP-9450

Fix minor typo in gen_fsm documentation (Thanks to Haitao Li)

Own Id: OTP-9456

The contracts of zip:zip_list_dir/1 and zip:zip_get/2 have been corrected.

Own Id: OTP-9471 Aux Id: seq11887, OTP-9472

A bug in zip:zip_open() has been fixed.

Own Id: OTP-9472 Aux Id: seq11887, OTP-9471

Fix trivial documentation errors(Thanks to Matthias Lang)

Own Id: OTP-9498

Add a proplist() type

Recently I was adding specs to an API and found that there is no canonical proplist() type defined. (Thanks to Ryan Zezeski)

Own Id: OTP-9499

fix supervisors restarting temporary children

In the current implementation of supervisors, temporary children should never be restarted. However, when a temporary child is restarted as part of a one_for_all or rest_for_one strategy where the failing process is not the temporary child, the supervisor still tries to restart it.

Because the supervisor doesn't keep some of the MFA information of temporary children, this causes the supervisor to hit its restart limit and crash.

This patch fixes the behaviour by inserting a clause in terminate_children/2-3 (private function) that will omit temporary children when building a list of killed processes, to avoid having the supervisor trying to restart them again.

Only supervisors in need of restarting children used the list, so the change should be of no impact for the functions that called terminate_children/2-3 only to kill all children.

The documentation has been modified to make this behaviour more explicit. (Thanks to Fred Hebert)

Own Id: OTP-9502

fix broken edoc annotations (Thanks to Richard Carlsson)

Own Id: OTP-9516

XML files have been corrected.

Own Id: OTP-9550 Aux Id: OTP-9541

Handle rare race in the crypto key server functionality

Own Id: OTP-9586

Types and specifications have been added.

Own Id: OTP-9356

The contracts of the queue module have been modified.

Own Id: OTP-9418

Contracts in STDLIB and Kernel have been improved and type errors have been corrected.

Own Id: OTP-9485

Types for several BIFs have been extended/corrected. Also the types for types for lists:keyfind/3, lists:keysearch/3, and lists:keyemember/3 have been corrected. The incorrect/incomplete types could cause false dialyzer warnings.

Own Id: OTP-9496

The default value undefined was added to records field types in such a way that the result was not always a well-formed type. This bug has been fixed.

Own Id: OTP-9147

Update index file atomically

Since the log_mf_h index file might be read by other processes than the error handler (e.g. by the rb tool), this file should be updated atomically. This will avoid hitting the time gap between opening the file in write mode (and thus emptying the file) and the actual update with the new contents. To do this, a temporary file is written, and the file:rename/1 used to replace the real index file.

Own Id: OTP-9148

Fixed various typos across the documentation (Thanks to Tuncer Ayaz)

Own Id: OTP-9154

Supervisors should not save child-specs for temporary processes when they terminate as they should not be restarted. Saving the temporary child spec will result in that you can not start a new temporary process with the same child spec as an already terminated temporary process. Since R14B02 you can not restart a temporary temporary process as arguments are no longer saved, it has however always been semantically incorrect to restart a temporary process. Thanks to Filipe David Manana for reporting this and suggesting a solution.

Own Id: OTP-9167 Aux Id: OTP-9064

Various small documentation fixes (Thanks to Bernard Duggan)

Own Id: OTP-9172

Fix format_status bug for unregistered gen_event processes

Port the gen_fsm code for format_status to gen_event in order to prevent a lists:concat([...,pid()]) crash when calling sys:get_status/1 on an unregistered gen_event process.

Refactor format_status header code from gen_* behaviours to module gen.

Extend the format_status tests in gen_event_SUITE to cover format_status bugs with anonymous gen_event processes. (Thanks To Geoff Cant)

Own Id: OTP-9218

List of pids changed to 'set' in supervisor for dynamic temporary children. Accessing the list would not scale well when adding/deleting many children. (Thanks to Evgeniy Khramtsov)

Own Id: OTP-9242

Change pool module to attempt to attach to nodes that are already running

The pool module prints out an error message and takes no further action for nodes that are already running. This patch changes that behavior so that if the return from slave:start/3 is {already_running, Node} then an attempt to attach to the node is still made. This makes sense because the node has been specified by the user in the .hosts.erlang file indicating a wish for the node to be part of the pool and a manual attach can be successfully made after the pool is started.(Thanks to Kelly McLaughlin)

Own Id: OTP-9244

unicode: document 16#FFFE and 16#FFFF (non chars)(Thanks to Tuncer Ayaz)

Own Id: OTP-9256

re: remove gratuitous "it " in manpage (Thanks to Tuncer Ayaz)

Own Id: OTP-9307

A bug in erl_eval(3) has been fixed.

Own Id: OTP-9322

Add timer:tc/1 and remove the catch in tc/2 and tc/3. The time measuring functions will thus no longer trap exits, errors or throws caused by the measured function.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9169

Allow supervisor:terminate_child(SupRef,Pid) for simple_one_for_one supervisors

supervisor:terminate_child/2 was earlier not allowed if the supervisor used restart strategy simple_one_for_one. This is now changed so that children of this type of supervisors can be terminated by specifying the child's Pid.

(Thanks to Vance Shipley.)

Own Id: OTP-9201

Types and specifications have been added.

Own Id: OTP-9267

Erlang types and specifications are used for documentation.

Own Id: OTP-9271

Allow Dets tablenames to be arbitrary terms.

Own Id: OTP-9282

A specification that could cause problems for Dialyzer has been fixed. An opaque type in erl_eval has been turned in to a ordinary type. This is a temporary fix.

Own Id: OTP-9333

Two bugs in io:format for ~F.~Ps has been corrected. When length(S) >= abs(F) > P, the precision P was incorrectly ignored. When F == P > length(S) the result was incorrectly left adjusted. Bug found by Ali Yakout who also provided a fix.

Own Id: OTP-8989 Aux Id: seq11741

Fix exception generation in the io module

Some functions did not generate correct badarg exception on a badarg exception.

Own Id: OTP-9045

Fixes to the dict and orddict module documentation

Fixed grammar and one inconsistency (Key - Value instead of key/value, since everywhere else the former is used). (thanks to Filipe David Manana)

Own Id: OTP-9083

Add ISO week number calculation functions to the calendar module in stdlib

This new feature adds the missing week number function to the calendar module of the stdlib application. The implementation conforms to the ISO 8601 standard. The new feature has been implemented tested and documented (thanks to Imre Horvath).

Own Id: OTP-9087

Implement the 'MAY' clauses from RFC4648 regarding the pad character to make mime_decode() and mime_decode_to_string() functions more tolerant of badly padded base64. The RFC is quoted below for easy reference.

"RFC4648 Section 3.3 with reference to MIME decoding: Furthermore, such specifications MAY ignore the pad character, "=", treating it as non-alphabet data, if it is present before the end of the encoded data. If more than the allowed number of pad characters is found at the end of the string (e.g., a base 64 string terminated with "==="), the excess pad characters MAY also be ignored."

Own Id: OTP-9020

Supervisors will no longer save start parameters for temporary processes as they will not be restarted. In the case of simple_one_for_one workers such as ssl-connection processes this will substantial reduce the memory footprint of the supervisor.

Own Id: OTP-9064

When running escript it is now possible to add the -n flag and the escript will be compiled using +native.

Own Id: OTP-9076

Several type specifications for standard libraries were wrong in the R14B01 release. This is now corrected. The corrections concern types in re,io,filename and the module erlang itself.

Own Id: OTP-9008

When several clients accessed a Dets table simultaneously, one of them calling dets:insert_new/2, the Dets server could crash. Alternatively, under the same conditions, ok was sometimes returned instead of true. (Thanks to John Hughes.)

Own Id: OTP-8856

When several clients accessed a Dets table simultaneously, inserted or updated objects were sometimes lost due to the Dets file being truncated. (Thanks to John Hughes.)

Own Id: OTP-8898

When several clients accessed a Dets table simultaneously, modifications of the Dets server's internal state were sometimes thrown away. The symptoms are diverse: error with reason bad_object; inserted objects not returned by lookup(); et cetera. (Thanks to John Hughes.)

Own Id: OTP-8899

If a Dets table was closed after calling bchunk/2, match/1,3, match_object/1,3, or select/1,3 and then opened again, a subsequent call using the returned continuation would normally return a reply. This bug has fixed; now the call fails with reason badarg.

Own Id: OTP-8903

Cover did not collect coverage data for files such as Yecc parses containing include directives. The bug has been fixed by modifying epp, the Erlang Code Preprocessor.

Own Id: OTP-8911

If a Dets table with fewer slots than keys was opened and then closed after just a lookup, the contents were no longer well-formed. This bug has been fixed. (Thanks to Matthew Evans.)

Own Id: OTP-8923

In a supervisor, when it terminates a child, if that child happens to have exited fractionally early, with normal, the supervisor reports this as an error. This should not be reported as an error.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8938 Aux Id: seq11615

The documentation filelib:wildcard/1,2 now describes the character set syntax for wildcards.

Own Id: OTP-8879 Aux Id: seq11683

Buffer overflows have been prevented in erlc, dialyzer, typer, run_test, heart, escript, and erlexec.

Own Id: OTP-8892

Using a float for the number of copies for string:copies/2 resulted in an infinite loop. Now it will fail with an exception instead. (Thanks to Michael Santos.)

Own Id: OTP-8915

New ETS option compressed, to enable a more compact storage format at the expence of heavier table operations. For test and evaluation, erl +ec can be used to force compression on all ETS tables.

Own Id: OTP-8922 Aux Id: seq11658

The default maximum number of slots of a Dets table has been changed as to be equal to the maximum number of slots. (Thanks to Richard Carlsson.)

Own Id: OTP-8959

reference() has been substituted for ref() in the documentation.

Own Id: OTP-8733

The ms_transform now warns if the fun head shadows surrounding variables (just like the warnings you would get for an ordinary fun in the same context).

Own Id: OTP-6759

ets:select_reverse/{1,2,3} are now documented.

Own Id: OTP-7863

Large parts of the ethread library have been rewritten. The ethread library is an Erlang runtime system internal, portable thread library used by the runtime system itself.

Most notable improvement is a reader optimized rwlock implementation which dramatically improve the performance of read-lock/read-unlock operations on multi processor systems by avoiding ping-ponging of the rwlock cache lines. The reader optimized rwlock implementation is used by miscellaneous rwlocks in the runtime system that are known to be read-locked frequently, and can be enabled on ETS tables by passing the {read_concurrency, true} option upon table creation. See the documentation of ets:new/2 for more information. The reader optimized rwlock implementation can be fine tuned when starting the runtime system. For more information, see the documentation of the +rg command line argument of erl.

There is also a new implementation of rwlocks that is not optimized for readers. Both implementations interleaves readers and writers during contention as opposed to, e.g., the NPTL (Linux) pthread rwlock implementation which use either a reader or writer preferred strategy. The reader/writer preferred strategy is problematic since it starves threads doing the non-preferred operation.

The new rwlock implementations in general performs better in ERTS than common pthread implementations. However, in some extremely heavily contended cases this is not the case. Such heavy contention can more or less only appear on ETS tables. This when multiple processes do very large amounts of write locked operations simultaneously on the same table. Such use of ETS is bad regardless of rwlock implementation, will never scale, and is something we strongly advise against.

The new rwlock implementations depend on atomic operations. If no native atomic implementation is found, a fallback solution will be used. Using the fallback implies a performance degradation. That is, it is more important now than before to build OTP with a native atomic implementation.

The ethread library contains native atomic implementations for, x86 (32 and 64 bit), powerpc (32 bit), sparc V9 (32 and 64 bit), and tilera (32 bit). On other hardware gcc's builtin support for atomic memory access will be used if such exists. If no such support is found, configure will warn about no atomic implementation available.

The ethread library can now also use the libatomic_ops library for atomic memory accesses. This makes it possible for the Erlang runtime system to utilize optimized native atomic operations on more platforms than before. If configure warns about no atomic implementation available, try using the libatomic_ops library. Use the --with-libatomic_ops=PATH configure command line argument when specifying where the libatomic_ops installation is located. The libatomic_ops library can be downloaded from: http://www.hpl.hp.com/research/linux/atomic_ops/

The changed API of the ethread library has also caused modifications in the Erlang runtime system. Preparations for the to come "delayed deallocation" feature has also been done since it depends on the ethread library.

Note: When building for x86, the ethread library will now use instructions that first appeared on the pentium 4 processor. If you want the runtime system to be compatible with older processors (back to 486) you need to pass the --enable-ethread-pre-pentium4-compatibility configure command line argument when configuring the system.

Own Id: OTP-8544

Some Built In Functions (BIFs) from the module erlang was never made autoimported for backward compatibility reasons. As local functions now override autoimports, new autoimports is no longer a problem, why the following BIFs are finally made autoimported: monitor/2, monitor/3, demonitor/2, demonitor/3, error/1, error/2, integer_to_list/2, list_to_integer/2.

Own Id: OTP-8763

The Erlang code preprocessor (epp) sent extra messages on the form {eof,Location} to the client when parsing the file attribute. This bug, introduced in R11B, has been fixed.

Own Id: OTP-8470

The abstract type 'fun' could not be printed by the Erlang pretty printer (erl_pp). This bug has been fixed.

Own Id: OTP-8473

The function erl_scan:reserved_word/1 no longer returns true when given the word spec. This bug was introduced in STDLIB-1.15.3 (R12B-3).

Own Id: OTP-8567

The documentation of lists:keysort/2 states that the sort is stable.

Own Id: OTP-8628 Aux Id: seq11576

The shell's line editing has been improved to more resemble the behaviour of readline and other shells. (Thanks to Dave Peticolas)

Own Id: OTP-8635

The Erlang code preprocessor (epp) did not correctly handle premature end-of-input when defining macros. This bug, introduced in STDLIB 1.16, has been fixed.

Own Id: OTP-8665 Aux Id: OTP-7810

The module binary from EEP31 (and EEP9) is implemented.

Own Id: OTP-8217

The erlang pretty printer (erl_pp) no longer quotes atoms in types.

Own Id: OTP-8501

The Erlang code preprocessor (epp) now considers records with no fields as typed.

Own Id: OTP-8503

Added function zip:foldl/3 to iterate over zip archives.

Added functions to create and extract escripts. See escript:create/2 and escript:extract/2.

The undocumented function escript:foldl/3 has been removed. The same functionality can be achieved with the more flexible functions escript:extract/2 and zip:foldl/3.

Record fields has been annotated with type info. Source files as been adapted to fit within 80 chars and trailing whitespace has been removed.

Own Id: OTP-8521

The Erlang parser no longer duplicates the singleton type undefined in the type of record fields without initial value.

Own Id: OTP-8522

A regular expression with many levels of parenthesis could cause a buffer overflow. That has been corrected. (Thanks to Michael Santos.)

Own Id: OTP-8539

When defining macros the closing right parenthesis before the dot is now mandatory.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8562

Some properties of a compiled re pattern are defined to allow for guard tests.

Own Id: OTP-8577

Local and imported functions now override the auto-imported BIFs when the names clash. The pre R14 behaviour was that auto-imported BIFs would override local functions. To avoid that old programs change behaviour, the following will generate an error:

If the BIF was added or auto-imported in OTP R14A or later, overriding it with an import or a local function will only result in a warning,

To resolve clashes, you can either use the explicit module name erlang to call the BIF, or you can remove the auto-import of that specific BIF by using the new compiler directive -compile({no_auto_import,[F/A]})., which makes all calls to the local or imported function without explicit module name pass without warnings or errors.

The change makes it possible to add auto-imported BIFs without breaking or silently changing old code in the future. However some current code ingeniously utilizing the old behaviour or the nowarn_bif_clash compiler directive, might need changing to be accepted by the compiler.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8579

The undocumented, unsupport, and deprecated function lists:flat_length/1 has been removed.

Own Id: OTP-8584

A bug in re that could cause certain regular expression matches never to terminate is corrected. (Thanks to Michael Santos and Gordon Guthrie.)

Own Id: OTP-8589

Nested records can now be accessed without parenthesis. See the Reference Manual for examples. (Thanks to YAMASHINA Hio and Tuncer Ayaz.)

Own Id: OTP-8597

receive statements that can only read out a newly created reference are now specially optimized so that it will execute in constant time regardless of the number of messages in the receive queue for the process. That optimization will benefit calls to gen_server:call(). (See gen:do_call/4 for an example of a receive statement that will be optimized.)

Own Id: OTP-8623

The beam_lib:cmp/2 function now compares BEAM files in stricter way. The BEAM files will be considered different if there are any changes except in the compilation information ("CInf") chunk. beam_lib:cmp/2 used to ignore differences in the debug information (significant for Dialyzer) and other chunks that did not directly change the run-time behavior.

Own Id: OTP-8625

When a gen_server, gen_fsm process, or gen_event terminates abnormally, sometimes the text representation of the process state can occupy many lines of the error log, depending on the definition of the state term. A mechanism to trim out parts of the state from the log has been added (using a format_status/2 callback). See the documentation.

Own Id: OTP-8630

Calling sys:get_status() for processes that have globally registered names that were not atoms would cause a crash. Corrected. (Thanks to Steve Vinoski.)

Own Id: OTP-8656

The Erlang scanner has been augmented with two new tokens: .. and ....

Own Id: OTP-8657

Expressions evaluating to integers can now be used in types and function specifications where hitherto only integers were allowed ("Erlang_Integer").

Own Id: OTP-8664

The compiler optimizes record operations better.

Own Id: OTP-8668

The recently added BIFs erlang:min/2, erlang:max/2 and erlang:port_command/3 are now auto-imported (as they were originally intended to be). Due to the recent compiler change (OTP-8579), the only impact on old code defining it's own min/2, max/2 or port_command/3 functions will be a warning, the local functions will still be used. The warning can be removed by using -compile({no_auto_import,[min/2,max/2,port_command/3]}). in the source file.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8669 Aux Id: OTP-8579

Now, binary_to_term/2 is auto-imported. This will cause a compile warning if and only if a module has got a local function with that name.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8671

The predefined builtin type tid() has been removed. Instead, ets:tid() should be used.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8687

Because of a race condition, using filelib:ensure_dir/1 from multiple processes to create the same path or parts of the same directory structure, filelib:ensure_dir/1 could return a meaningless {error,eexist}. That race condition has been eliminated, and {error,eexist} will now be returned only if there exists a regular file, device file, or some other non-directory file with the same name. (Thanks to Tuncer Ayaz.)

Own Id: OTP-8389

A number of bugs concerning re and unicode are corrected:

re:compile no longer loses unicode option, which also fixes bug in re:split.

re:replace now handles unicode charlist replacement argument

re:replace now handles unicode RE charlist argument correctly

re:replace now handles binary unicode output correctly when nothing is replaced.

Most code, testcases and error isolation done by Rory Byrne.

Own Id: OTP-8394

The loading of native code was not properly atomic in the SMP emulator, which could cause crashes. Also a per-MFA information table for the native code has now been protected with a lock since it turns that it could be accessed concurrently in the SMP emulator. (Thanks to Mikael Pettersson.)

Own Id: OTP-8397

user.erl (used in oldshell) is updated to handle unicode in prompt strings (io:get_line/{1,2}). io_lib is also updated to format prompts with the 't' modifier (i.e. ~ts instead of ~s).

Own Id: OTP-8418 Aux Id: OTP-8393

The re module: A regular expression with an option change at the start of a pattern that had top-level alternatives could cause overwriting and/or a crash. (Thanks to Michael Santos.)

Own Id: OTP-8438

The ability for the gen_server and gen_fsm callback modules to format their own state for display under the sys:get_status/1,2 calls has been restored and documented. (Thanks to Steve Vinoski.)

Own Id: OTP-8324

c:nc/{1,2} used to assume that the beam file was created in the same directory as the source code and failed to load the code if it was not. Corrected to look for the beam file in the current directory or in the directory specified by the {outdir,Dir} option. (Thanks to Alex Suraci.)

Own Id: OTP-8337

The documentation is now possible to build in an open source environment after a number of bugs are fixed and some features are added in the documentation build process.

- The arity calculation is updated.

- The module prefix used in the function names for bif's are removed in the generated links so the links will look like "http://www.erlang.org/doc/man/erlang.html#append_element-2" instead of "http://www.erlang.org/doc/man/erlang.html#erlang:append_element-2".

- Enhanced the menu positioning in the html documentation when a new page is loaded.

- A number of corrections in the generation of man pages (thanks to Sergei Golovan)

- The legal notice is taken from the xml book file so OTP's build process can be used for non OTP applications.

Own Id: OTP-8343

Shell tab completion now works for quoted module and function names. (Thanks to Ulf Wiger.)

Own Id: OTP-8383

Explicit top directories in archive files are now optional.

For example, if an archive (app-vsn.ez) just contains an app-vsn/ebin/mod.beam file, the file info for the app-vsn and app-vsn/ebin directories are faked using the file info from the archive file as origin. The virtual direcories can also be listed. For short, the top directories are virtual if they does not exist.

Own Id: OTP-8387

Macros overloading has been implemented. (Thanks to Christopher Faulet.)

Own Id: OTP-8388

The new function shell:prompt_func/1 and the new application configuration parameter shell_prompt_func can be used for customizing the Erlang shell prompt.

Own Id: OTP-8393

Improved handling of typed records in escripts

Own Id: OTP-8434

Added supervisor:count_children/1 to count the number of children being managed without the memory impact of which_children/1. (Thanks to Jay Nelson.)

Own Id: OTP-8436

The documentation is now built with open source tools (xsltproc and fop) that exists on most platforms. One visible change is that the frames are removed.

Own Id: OTP-8201

[escript] The restriction that the first line in escripts must begin with #! has been removed.

[escript] Some command line options to the escript executable has now been documented. For example you can run an escript in the debugger by just adding a command line option.

[escript] The documentation of the escript header syntax has been clarified. For example the header is optional. This means that it is possible to directly "execute" .erl, .beam and.zip files.

Own Id: OTP-8215

Optimized array:from_orddict/1, it is now faster and uses less memory if the orddict was sparse.

Changed array:reset/2, it will now never expand the array which it could before for non fixed arrays. See the documentation.

Own Id: OTP-8216

The Erlang Pretty Printer (erl_pp) now puts the leading [ of list comprehensions as well as the leading << of bit string comprehensions on a separate line in order to expose the Cover counter of the template.

Own Id: OTP-8227

The extension ".xrl" used for Leex input files is now recognized by the compiler.

Own Id: OTP-8232

Some clarifications have been made in the documentation regarding gen_server, gen_fsm, and gen_event behavior when handling 'EXIT' messages from the parent process. For more information see the gen_server(3), gen_fsm(3), and gen_event(3) documentation.

Own Id: OTP-8255 Aux Id: seq11419

The -on_load() directive can be used to run a function when a module is loaded. It is documented in the section about code loading in the Reference Manual.

Own Id: OTP-8295

An erroneous type spec for gen:start/6 caused dialyzer to erroneously issue warnings when {spawn_opt, SpawnOptionList} was passed in the option list to the gen_server and gen_fsm start functions.

Own Id: OTP-8068 Aux Id: seq11323, seq11314

The linter used to crash on invalid -opaque declarations.

Own Id: OTP-8051

Bugs in digraph:add_edge/5 and digraph:del_path/3 have been fixed. (Thanks to Crystal Din.)

Own Id: OTP-8066

When trying to insert objects with dets:insert_new() into a Dets table of type duplicate_bag, already existing objects would sometimes be duplicated. This bug has been fixed. (Thanks to Crystal Din.)

Own Id: OTP-8070

Running erlc in a very deep directory (with a path length of more 256 or more characters) would cause the emulator to crash in a call to list_to_atom/1. (Thanks to Chris Newcombe.)

Own Id: OTP-8124

A few minor bugs have been fixed in the Erlang Code Preprocessor (epp).

Own Id: OTP-8130

A bug in The Erlang Meta Interpreter (erl_eval) has been fixed: exceptions generated in the template of bit string comprehensions were not handled properly. (Thanks to Ulf Wiger.)

Own Id: OTP-8133

Option {capture,none} was missing in documentation for re:run/3.

Own Id: OTP-8113

When erl_scan:tokens() returns an error tuple {error, ErrorInfo, EndLocation}, the list LeftOverChars is the remaining characters of the input data, starting from EndLocation. It used to be the empty list.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-8129

The Erlang Meta Interpreter (erl_eval) has been somewhat optimized when it comes to interpreting receive-expressions. (Thanks to Richard Carlsson.)

Own Id: OTP-8139

The Erlang Pretty Printer (erl_pp) has been modified as to handle types.

Own Id: OTP-8150

The text of tokens returned by the Erlang scanner (erl_scan) was sometimes empty when the text option was given and StartLocation was a line. This bug has been fixed.

Own Id: OTP-7965

The documentation for base64:decode/1 has been updated to point out that it strips whitespace.

base64:decode/1 and base64:mime_decode/1 would sometimes fail instead of stripping away non-base64 characters.

Own Id: OTP-7984

Two types in the gen module were corrected.

Own Id: OTP-8029 Aux Id: seq11296

array:from_orddict([]) and array:from_list([]) would construct fixed arrays instead of extendible arrays.

Own Id: OTP-8033

Interpreted escripts are now tail recursive.

The function erl_eval:expr/5 has been introduced.

Own Id: OTP-7933

gen_server:call/2,3 will be somewhat faster if the calling process has a many messages in its message queue.

Own Id: OTP-7979

Random now supports seed with arity one, random:seed/1, which takes a three-tuple.

Own Id: OTP-8019

The regexp module now recognizes the escape sequences \xXY and \x{X...}.

Own Id: OTP-8024

The documentation of dets:open_file/1 now states that the file is repaired if it has not been properly closed. (Thanks to Ulf Wiger.)

Own Id: OTP-7895

The Erlang scanner no longer returns the text of tokens when the start location is a pair of a line and column unless the new option text is supplied (incompatibility with R13A).

There are new functions to access the attributes of tokens: attributes_info/1,2 and set_attribute/3.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7892 Aux Id: OTP-7810

Several glitches and performance issues in the Unicode and I/O-system implementation of R13A have been corrected.

Own Id: OTP-7896 Aux Id: OTP-7648 OTP-7887

The type spec of filelib:wildcard/2 has been corrected.

Own Id: OTP-7915

New functions: gb_sets:is_disjoint/2, ordsets:is_disjoint/2, and gb_sets:is_disjoint/2.

Own Id: OTP-7947

The function gb_trees:map/2 which was added in R13A is now documented.

Own Id: OTP-7948

Fixed a minor race conditions in gen_server:start*: if one of these functions returned {error,Reason} or ignore, the name could still be registered (either locally or in global).

A process started by proc_lib in some cases depended on its process dictionary not to be erased, and would crash when terminating abnormally and not generate a proper crash report. This has been corrected (but the initial call will not be shown in the error report if the process dictionary has been erased). NOTE: There is no longer any need to erase the process dictionary for memory conservation reasons, since the actual call arguments are no longer saved in the process dictionary.

Own Id: OTP-7669

The Erlang preprocessor used wrong line number when stringifying macro arguments. (Thanks to John Hughes.)

Own Id: OTP-7702

A bug in the qlc module has been fixed: merge join sometimes failed to return all answers. (Thanks to Bernard Duggan.)

Own Id: OTP-7714

A new option, key_equality, has been added to qlc:table/2. This option makes it possible for qlc to better handle tables that use ==/2 when comparing keys for equality (examples of such tables are ordered ETS tables and gb_table in qlc(3)).

Own Id: OTP-6674

The functions lists:seq/1,2 return the empty list in a few cases when they used to generate an exception, for example lists:seq(1, 0). See lists(3) for details. (Thanks to Richard O'Keefe.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7230

The order of objects visited in select for ordered_set is now documented.

Own Id: OTP-7339

It is now possible to debug code in escripts and archives.

Own Id: OTP-7626

Support for Unicode is implemented as described in EEP10. Formatting and reading of unicode data both from terminals and files is supported by the io and io_lib modules. Files can be opened in modes with automatic translation to and from different unicode formats. The module 'unicode' contains functions for conversion between external and internal unicode formats and the re module has support for unicode data. There is also language syntax for specifying string and character data beyond the ISO-latin-1 range.

The interactive shell will support input and output of unicode characters when the terminal and operating system supports it.

Please see the EEP and the io/io_lib manual pages as well as the stdlib users guide for details.

I/O-protocol incompatibilities:

The io_protocol between io_Server and client is updated to handle protocol data in unicode formats. The updated protocol is now documented. The specification resides in the stdlib users manual, which is a new part of the manual.

io module incompatibilities:

The io:put_chars, io:get_chars and io:get_line all handle and return unicode data. In the case where binaries can be provided (as to io:put_chars), they shall be encoded in UTF-8. When binaries are returned (as by io:get_line/get_chars when the io_server is set in binary mode) the returned data is also always encoded as UTF-8. The file module however still returns byte-oriented data, why file:read can be used instead of io:get_chars to read binary data in ISO-latin-1.

io_lib module incompatibilities:

io_lib:format can, given new format directives (i.e "~ts" and "~tc"), return lists containing integers larger than 255.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7648 Aux Id: OTP-7580 OTP-7514 OTP-7494 OTP-7443 OTP-7181 EEP10 EEP11

The function pool:attach/1 now returns already_attached if the node is already attached, rather than allready_attached (sic!). (Thanks to Edwin Fine.)

Own Id: OTP-7653 Aux Id: OTP-7603

Preprocessor directives are now allowed in escripts. This means that for example macros may be used in escripts.

Own Id: OTP-7662

When a process started with proc_lib, gen_server, or gen_fsm exits with reason {shutdown,Term}, a crash report will no longer be generated (to allow a clean shutdown, but still provide additional information to process that are linked to the terminating process).

Own Id: OTP-7740 Aux Id: seq10847

A new BIF, lists:keyfind/3, has been added. It works like lists:keysearch/3 except that it does not wrap the returned tuple in a value tuple in case of success. (Thanks to James Hague for suggesting this function.)

Own Id: OTP-7752

lists:suffix(Suffix, List) used to have a a complexity of length(Suffix)*length(List) (which could become quite slow for some inputs). It has now been re-implemented so that its complexity is length(Suffix)+length(List). (Thanks to Richard O'Keefe for the new implementation.)

Own Id: OTP-7797

The Erlang scanner has been augmented as to return white spaces, comments, and exact location of tokens. The functions string/3, tokens/4, and token_info/1,2 are new. See erl_scan(3) for details.

tokens/3,4 have been modified as to return a list of tokens instead of an error when eof is encountered before the dot.

Own Id: OTP-7810

filelib:fold_files/5 now uses the re module instead of the regexp module for regular expression matching. In practice, this change will not be a problem for most regular expressions used for filelib:fold_files/5. (The major difference in regular expression is that parenthesis and curly brackets is treated as literal characters by regexp but as special characters by re; fortunately, those characters are rarely used in filenames.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7819

digraph:new(Type) will now cause a badarg exception if Type is not a valid type. Similarly, digraph_utils:subgraph/2,3 will now cause a badarg if the arguments are invalid. (Those functions used to return error tuples if something was wrong.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7824

The argument passed to random:uniform/1 must now be an integer (as stated in the documentation). In previous releases, a floating point number was also allowed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7827

The copyright notices have been updated.

Own Id: OTP-7851

A few missing match spec functions was added to dbg:fun2ms; exception_trace/0 and trace/2,3.

There is a new function queue:member/2.

A bug in io_lib:fread that made it accidentally concatenate fields separated by newline has been corrected. Reported and analyzed by Matthew Palmer to erlang-patches.

Own Id: OTP-7865

A bug in the qlc module has been fixed: when merge joining two query handles the temporary file used for equivalence classes was not truncated properly which could result in poor performance.

Own Id: OTP-7552

The characters 16#C0 and 16#E0 ("A" and "a" with grave accent), were not properly converted by the string:to_lower/1 and string:to_upper/1 functions. (Thanks to Richard O'Keefe.)

Own Id: OTP-7589

The function pool:attach/1 now returns already_attached if the node is already attached, rather than allready_attached (sic!). (Thanks to Edwin Fine.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7603

The documentation for io:get_line/1,2 now mentions that the return value can also be {error,Reason}.

Own Id: OTP-7604 Aux Id: seq11063

The split function is now added to the re library. Exceptions and errors from both run, replace and split are made more consistent.

Own Id: OTP-7514 Aux Id: OTP-7494

Processes spawned using proc_lib (including gen_server and other library modules that use proc_lib) no longer keep the entire argument list for the initial call, but only the arity.

Also, if proc_lib:spawn/1 is used to spawn a fun, the actual fun is not kept, but only module, function name, and arity of the function that implements the fun.

The reason for the change is that keeping the initial fun (or a fun in an argument list), would prevent upgrading the code for the module. A secondary reason is that keeping the fun and function arguments could waste a significant amount of memory.

The drawback with the change is that the crash reports will provide less precise information about the initial call (only Module:Function/Arity instead of Module:Function(Arguments)). The function proc_lib:initial_call/1 still returns a list, but each argument has been replaced with a dummy atom.

Own Id: OTP-7531 Aux Id: seq11036

There is now experimental support for loading of code from archive files. See the documentation of code, init, erl_prim_loader and escript for more info.

The error handling of escripts has been improved.

An escript may now set explicit arguments to the emulator, such as -smp enabled.

An escript may now contain a precompiled beam file.

An escript may now contain an archive file containing one or more applications (experimental).

The internal module code_aux has been removed.

Own Id: OTP-7548 Aux Id: otp-6622

Enabled explicit control of which types of files that should be compressed in a ZIP archive.

Own Id: OTP-7549 Aux Id: otp-6622

In the job control mode, the "s" and "r" commands now take an optional argument to specify which shell to start. (Thanks to Robert Virding.)

Own Id: OTP-7617

A bug in the calendar module could cause calendar:local_time_to_universal_time_dst/1 to return duplicate identical values for local times in timezones without DST. Multiple values should only be returned when a local time is within the hour occurring twice due to shift from DST to non-DST, and certainly only in timezones with DST. The correct behaviour is now implemented.

Own Id: OTP-7344 Aux Id: seq10960

The documentation of (d)ets:init_table() has been corrected. (Thanks to Paul Mineiro.)

Own Id: OTP-7413

The soft upper limit of 60 on the number of non-white characters on a line, which was introduced in R12B-0 for the control sequences p and P of the functions io:fwrite/2,3 and io_lib:fwrite/2, has been removed. This means that terms whose printed representation fits on a line will have no NEWLINEs. The Erlang shell still uses the 60 character limit, though.

Own Id: OTP-7421 Aux Id: OTP-6708

Some debug code has been removed from Dets.

Own Id: OTP-7424

The documentation of dets:match_delete/2 has been corrected. (Thanks to Paul Mineiro.)

Own Id: OTP-7445

Corrections of digraph(3). (Thanks to Vlad Dumitrescu.)

Own Id: OTP-7492

For the process that an escript runs in, the trap_exit process flag is now false instead of true (as in previous releases). Scripts that depend on the previous (counter-intuitive) behaviour might not work. (Thanks to Bengt Kleberg.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-7517

The documentation of lists:(u)sort/2 now states what is expected of an ordering function.

Own Id: OTP-7489

The re module is extended with repetitive matches (global option) and replacement function.

Own Id: OTP-7494 Aux Id: OTP-7181

The Erlang shell now displays a nicer error message when evaluating an undefined command. (Thanks to Richard Carlsson.)

Own Id: OTP-7495

zip:unzip to/from binary with empty directories did not work. (Thanks to Martin Dvorak.)

Own Id: OTP-7248

The documentation of the control sequence w of the io_lib module now states that floating point numbers are printed accurately.

Own Id: OTP-7324 Aux Id: OTP-7084

zip:unzip was not supporting a flavour of the zip format found in jar-files.

Own Id: OTP-7382 Aux Id: seq10970

An experimental module "re" is added to the emulator which interfaces a publicly available regular expression library for Perl-like regular expressions (PCRE). The interface is purely experimental and *will* be subject to change.

The implementation is for reference and testing in connection to the relevant EEP.

Own Id: OTP-7181

When inserting many small objects, Dets sometimes crashed when reaching the maximum number of slots. (Thanks to Daniel Goertzen.)

Own Id: OTP-7146

Processes linked to the Erlang shell did not get an exit signal when the evaluator process was killed. This bug, introduced in R12B-0, has been fixed.

Own Id: OTP-7184 Aux Id: OTP-6554

Invalid arguments to ets:update_counter/3 were not handled correctly. A tuple position (Pos) less than 1 caused the element directly following the key to be updated (as if no position at all had been specified). All invalid values for Pos will now fail with badarg.

Own Id: OTP-7226

For certain terminals, io:columns/0 could return 0 instead of enotsup. That is now corrected.

Own Id: OTP-7229 Aux Id: seq10886

qlc:info() can now handle port identifiers, pids, references, and funs. (Thanks to Wojciech Kaczmare for reporting this bug.)

When evaluating the parent_fun messages sent to the process calling qlc:cursor() were sometimes erroneously consumed. This bug has been fixed.

Own Id: OTP-7232

erl_parse:abstract() can now handle bit strings.

Own Id: OTP-7234

The queue module has been rewritten to make it easier to use. Suggestions and discussion from and with among others Lev Walkin, Anders Ramsell and Rober Virding in december 2007 on erlang-questions@erlang.org. It was also discussed to change the internal representation to contain length information which would speed up len/1 but that change has been postponed. Anyone interested may write an EEP and try to reach an acceptable compromise for queue overhead and thereby the speed of all other operations than len/1. The queue module is now optimized for fast and minimal garbage in/2 and out/1 and such. See the documentation.

New functions: is_queue/1, get/1, get_r/1, peek/1, peek_r/1, drop/1, drop_r/1 and liat/1. is_queue/1 is a new predicate, liat/1 is a correction of an old misspelling, and the others (get*, peek* and drop*) are new interface functions.

Own Id: OTP-7064

The functions io_lib:write/1,2 and io_lib:print/1,4 have been changed when it comes to writing floating point numbers. This change affects the control sequences p, P, w, and W of the io_lib module. (Thanks to Bob Ippolito for code contribution.)

Own Id: OTP-7084

Updated the documentation for erlang:function_exported/3 and io:format/2 functions to no longer state that those functions are kept mainly for backwards compatibility.

Own Id: OTP-7186

A new BIF ets:update_element/3. To update individual elements within an ets-tuple, without having to read, update and write back the entire tuple.

Own Id: OTP-7200

string:join/2 now accepts an empty list as first argument.

Own Id: OTP-7231 Aux Id: OTP-6671

qlc:info/1,2 accepts a new option, depth. The type SelectedObjects used in the description of qlc:table/2 has been augmented.

Own Id: OTP-7238

tuple_size/1 and byte_size/1 have been substituted for size/1 in the documentation.

Own Id: OTP-7244

Ets:select/3 in combination with ets:repair_continuation/2 and ordered_set data tables could result in function_clause although used as intended. This is now corrected. Thanks to Paul Mineiro for finding and isolating the bug!

Own Id: OTP-7025

The compiler warning for the deprecated function ftp:close/1 now mentions the correct replacement function.

The warning for the removed functions in the httpd_util module have been changed to say they have been removed, not merely deprecated. (Thanks to Fredrik Thulin.)

Own Id: OTP-7034 Aux Id: seq10825

In (Expr)#r{} (no fields are updated), Expr is no longer evaluated more than once. There is also a test that Expr is of the correct record type. (Thanks to Dominic Williams.)

Own Id: OTP-7078 Aux Id: OTP-4962

Documentation bugfixes and clarifications.

Own Id: OTP-7079

Duplicated objects were sometimes not deleted from the list of answers when a QLC table was traversed using a match specification. (Thanks to Dmitri Girenko.)

Own Id: OTP-7114

The documentation has been updated so as to reflect the last updates of the Erlang shell as well as the minor modifications of the control sequence p of the io_lib module.

Superfluous empty lines have been removed from code examples and from Erlang shell examples.

Own Id: OTP-6944 Aux Id: OTP-6554, OTP-6911

tuple_size/1 and byte_size/1 have been substituted for size/1.

Own Id: OTP-7009

It is now possible to hibernate a gen_server/gen_event/gen_fsm. In gen_server and gen_fsm, hibernation is triggered by returning the atom 'hibernate' instead of a timeout value. In the gen_event case hibernation is triggered by a event handler returning a tuple with an extra element containing the atom 'hibernate'.

Own Id: OTP-7026 Aux Id: seq10817

Some undocumented debug functionality has been added to Dets.

Own Id: OTP-7066

The functions digraph_utils:is_tree/1, digraph_utils:is_arborescence/1, and digraph_utils:arborescence_root/1 are new.

Own Id: OTP-7081

The compiler could generate suboptimal code for record updates if the record update code consisted of multiple source code lines.

Own Id: OTP-7101

Bugs have been fixed in qlc:

A few minor optimizations have been implemented as well.

Own Id: OTP-6673

A bug concerning the use of parameterized modules from the shell has been fixed.

Own Id: OTP-6785

A bug regarding the size expression of the bit syntax has been fixed in the erl_eval module.

Own Id: OTP-6787

The log_mf_h event handler didn't close the index file when it was done reading it causing a file descriptor leak.

Own Id: OTP-6800

Definitions for the filename() and dirname() types have been added to the documentation for the filelib module.

Own Id: OTP-6870

file:write_file/3, file:write/2 and file:read/2 could crash (contrary to documentation) for odd enough file system problems, e.g write to full file system. This bug has now been corrected.

In this process the file module has been rewritten to produce better error codes. Posix error codes now originate from the OS file system calls or are generated only for very similar causes (for example 'enomem' is generated if a memory allocation fails, and 'einval' is generated if the file handle in Erlang is a file handle but currently invalid).

More Erlang-ish error codes are now generated. For example {error,badarg} is now returned from file:close/1 if the argument is not of a file handle type. See file(3).

The possibility to write a single byte using file:write/2 instead of a list or binary of one byte, contradictory to the documentation, has been removed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6967 Aux Id: OTP-6597 OTP-6291

A bug concerning the evaluation of the ++/2 operator has been fixed in erl_eval. (Thanks to Matthew Dempsky.)

Own Id: OTP-6977

The behaviour of the internal functions gen:call/3,4 has been changed slightly in the rare case that when the caller was linked to the called server, and the server crashed during the call; its exit signal was consumed by the gen:call/3,4 code and converted to an exit exception. This exit signal is no longer consumed.

To even notice this change, 1) the calling process has to be linked to the called server.

2) the call must not be remote by name that is it must be local or remote by pid, local by name or global by name.

3) the calling process has to have set process_flag(trap_exit, true).

4) the server has to crash during the call.

5) the calling process has to be sensitive to getting previously consumed {'EXIT',Pid,Reason} messages in its message queue.

The old behaviour was once the only way for a client to notice if the server died, but has since erlang:monitor(process, {Name,Node}) was introduced and used in gen:call been regarded as an undesired behaviour if not a bug.

The affected user APIs are: gen_server:call/2,3, gen_fsm:sync_send_event/2,3, gen_fsm:sync_send_all_state_event/2,3, gen_event:_, sys:_ and maybe a few others that hardly will be noticed.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-3954 Aux Id: Seq 4538

When an exception occurs the Erlang shell now displays the class, the reason, and the stacktrace in a clearer way (rather than dumping the raw EXIT tuples as before). proc_lib:format/1 displays the exception of crash reports in the same clearer way.

The new shell command catch_exception and the new application configuration parameter shell_catch_exception can be used for catching exceptions that would normally exit the Erlang shell.

Own Id: OTP-6554 Aux Id: OTP-6289

The function string:join/2 joins strings in a list with a separator. Example: 'string:join(["a", "b", "c"], ", ") gives "a, b, c"'

Own Id: OTP-6671

The control sequence P of the Format argument of the functions io:fwrite/2,3 and io_lib:fwrite/2 now inserts fewer line breaks when printing tuples and lists. A soft upper limit of 60 on the number of non-white characters on a line has been introduced.

Own Id: OTP-6708

The new module array provides a fast functional array implementation.

Own Id: OTP-6733

Functions that have long been deprecated have now been removed from the following modules: dict, erl_eval, erl_pp, io, io_lib, lists, orddict, ordsets, sets, and string.

The undocumented function lists:zf/3 has also been removed (use a list comprehension or lists:zf/2 instead).

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6845

Minor documentation corrections for file:pread/2 and file:pread/3.

Own Id: OTP-6853

Contract directives for modules in Kernel and STDLIB.

Own Id: OTP-6895

The ets:fixtable/2 function, which has been deprecated for several releases, has been removed.

The ets:info/1 function has been reimplemented as a BIF, which guarantees that information returned is consistent.

The ets:info/2 function now fails with reason badarg if the second argument is invalid. (Dialyzer can be used to find buggy code where the second argument is misspelled.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6906

The Erlang pretty printer erl_pp now inserts more newlines in order to facilitate line coverage analysis by Cover. (Thanks to Thomas Arts.)

Own Id: OTP-6911

The documentation for ets:safe_fixtable/2, ets:foldl/3, and ets:foldr/3 is now clearer about what will happen if objects are inserted during table traversals.

Own Id: OTP-6928 Aux Id: seq10779

It is now possible to extract files in tar files directly into binaries. It is also possible to add files to tar files directly from binaries.

Own Id: OTP-6943

The functions keystore/4 and keytake/3 are new in the lists module.

Own Id: OTP-6953

The new qlc option tmpdir_usage can be used for outputting messages onto the error logger when a temporary file is about to be created, or to prohibit the usage of temporary files altogether.

Own Id: OTP-6964

The allowed syntax for -type() and -spec() was updated.

Own Id: OTP-6861 Aux Id: OTP-6834

The compiler will for forward compatibility ignore the -type() and -spec() attributes that will be introduced in the R12B release.

Own Id: OTP-6834

The log_mf_h event handler didn't close the index file when it was done reading it causing a file descriptor leak.

Own Id: OTP-6800

The dict:size/1 and orddict:size/1 functions have been documented.

Own Id: OTP-6818

Bugs have been fixed in Dets concerning comparison (==) and matching (=:=).

The STDLIB manual pages have been updated as to more carefully state when terms are matched and when they are compared.

Own Id: OTP-4738 Aux Id: OTP-4685

The shell has been updated to fix the following flaws: Shell process exit left you with an unresponsive initial shell if not using oldshell. Starting a restricted shell with a nonexisting callback module resulted in a shell where no commands could be used, not even init:stop/0. Fun's could not be used as parameters to local shell functions (in shell_default or user_default) when restricted_shell was active.

Own Id: OTP-6537

A bug in QLC's parse transform has been fixed.

Own Id: OTP-6590

A bug concerning lists:sort/1 and lists:keysort/2 and a mix of floating point numbers and integers has been fixed.

Own Id: OTP-6606

When calling erlang:garbage_collect/0 in the Erlang shell not only the evaluator process (the one returned by calling self() in the Erlang shell) is garbage collected, but also the process holding the history list.

Own Id: OTP-6659

Functions of the beam_lib module that used to catch exceptions and return a tuple {'EXIT',Reason} now exit with the reason Reason.

Own Id: OTP-6711

The erl_eval module now calls the non-local function handler whenever an operator is evaluated (exceptions are andalso, orelse, and catch). The non-local function handler is now also called when the function or operator occurs in a guard test (such calls used to be ignored).

These changes affect the Erlang shell when running in restricted mode: the callback function non_local_allowed/3 is now called for operators such as '!'/2. This means that non_local_allowed/3 may need to be changed as to let operators through. Note that erlang:'!'/2 as well as erlang:send/2,3 have to be restricted in order to stop message passing in the shell.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6714 Aux Id: seq10374

The new compiler option warn_obsolete_guard can be used for turning on warnings for calls to old type testing BIFs.

Own Id: OTP-6585

For scripts written using escript, there is a new function escript:script_name/0, which can be used to retrieve the pathame of the script. The documentation has been clarified regarding pre-defined macros such as ?MODULE and the module name.

Own Id: OTP-6593

Minor Makefile changes.

Own Id: OTP-6689 Aux Id: OTP-6742

The MD5 calculation of a BEAM file done by code:module_md5/1, beam_lib:md5/1, and by the compiler for the default value of the vsn attribute have all been changed so that its result will be the same on all platforms; modules containing funs could get different MD5s on different platforms.

Own Id: OTP-6459

When sorting terms using the file_sorter module (the option Format set to term), file errors were not always properly handled. This bug has been fixed.

The directory supplied with the tmpdir option is no longer checked unless it is actually used. The error reason not_a_directory can no longer be returned; instead a file_error tuple is returned

Own Id: OTP-6526

Bugs regarding try/catch have been fixed in the erl_eval module.

Own Id: OTP-6539

When sorting the operands of a join operation, QLC called file:open/3 with bad arguments. This bug has been fixed.

Own Id: OTP-6562 Aux Id: seq10606

The functions beam_lib:cmp/1 and beam_lib:strip/1 (and similar functions) have been updated to handle optional chunks (such as "FunT") in more general way in order to be future compatible.

The function beam_lib:chunks/3 has been added.

The function beam_lib:md5/1 has been added.

Own Id: OTP-6443

Added base64 as a module to stdlib, encoding and decoding

Own Id: OTP-6470

Added the functions to_upper/1 and to_lower/1 to the string module. These provide case conversion for ISO/IEC 8859-1 characters (Latin1) and strings.

Own Id: OTP-6472

The callback function non_local_allowed/3 used by the restricted shell can now return the value {{restricted,NewFuncSpec,NewArgList},NewState} which can be used for letting the shell call some other function than the one specified.

Own Id: OTP-6497 Aux Id: seq10555

There is a new escript program that can be used for writing scripts in Erlang. Erlang scripts don't need to be compiled and any arguments can be passed to them without risk that they are interpreted by the Erlang system.

Own Id: OTP-6505

The Format argument of the functions io:fwrite/2,3 and io_lib:fwrite/2 is now allowed to be a binary.

Own Id: OTP-6517

The control sequences p and P of the Format argument of the functions io:fwrite/2,3 and io_lib:fwrite/2 could cause a badarg failure when applied to binaries. This bug was introduced in STDLIB 1.14.3. (Thanks to Denis Bilenko.)

Own Id: OTP-6495

Added the option {cwd, Dir} to make zip-archives with relative pathnames without having to do (a global) file:set_cwd.

Own Id: OTP-6491 Aux Id: seq10551

The spawn_opt/2,3,4,5 option monitor -- introduced in Kernel 2.11.2 -- is currently not possible to use when starting a process using proc_lib, that is, also when starting a gen_server, gen_fsm etc.

This limitation has now been properly documented and the behavior of the gen_fsm, gen_server, and proc_lib start and start_link functions when providing this option has been changed from hanging indefinitely to failing with reason badarg.

(Thanks to Fredrik Linder)

Own Id: OTP-6345

The control sequence P of the Format argument of the functions io:fwrite/2,3 and io_lib:fwrite/2 now replaces the tail of binary strings with ... when the maximum depth has been reached. For instance, >, 3]).]]> prints >]]>.

The indentation takes more care not to exceed the right margin, if possible.

If the maximum depth is reached while printing a tuple, ,... is printed instead of |... (this change applies to the control sequence W as well).

Own Id: OTP-6354

The Erlang shell command h/0 that prints the history list now avoids printing (huge) terms referred to by v/1 but instead just prints the call to v/1.

Own Id: OTP-6390

The functions dets:select/1,3, dets:match/1,3, and dets:match_object/1,3 have been changed as to never return {[],Continuation}. This change affects the corresponding functions in Mnesia.

Bugs have been fixed in QLC: qlc:info() could crash if the tmpdir option did not designate a valid directory; the results of looking up keys are kept in RAM, which should improve performance.

Own Id: OTP-6359

A bug in erl_pp:exprs() has been fixed.

Own Id: OTP-6321 Aux Id: seq10497

The control sequences p and P of the Format argument of the functions io:format/2,3 and io_lib:format/2 did not handle binaries very well. This bug, introduced in stdlib-1.14, has been fixed.

Own Id: OTP-6230

filelib:wildcard(Wc, PathWithRedundantSlashes), where PathWithRedundantSlashes is a directory path containing redundant slashes, such as /tmp/ or //tmp, could return incorrect results. (Thanks to Martin Bjorklund.)

Own Id: OTP-6271

The Erlang code preprocessor crashed if the predefined macros ?MODULE or ?MODULE_STRING were used before the module declaration. This bug has been fixed.

Own Id: OTP-6277

Support for faster join of two tables has been added to the qlc module. There are two kinds of fast joins: lookup join that uses existing indices, and merge join that takes two sorted inputs. There is a new join option that can be used to force QLC to use a particular kind of join in some QLC expression.

Several other changes have also been included:

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6038

The Erlang pretty printer (erl_pp) is now much faster when the code is deeply nested. A few minor bugs have been fixed as well.

Own Id: OTP-6227 Aux Id: OTP-5924

The Erlang shell now tries to garbage collect large binaries. Under certain circumstances such binaries could otherwise linger on for an indefinite amount of time.

Own Id: OTP-6239

To help Dialyzer find more bugs, many functions in the Kernel and STDLIB applications now only accept arguments of the type that is documented.

For instance, the functions lists:prefix/2 and lists:suffix/2 are documented to only accept lists as their arguments, but they actually accepted anything and returned false. That has been changed so that the functions cause an exception if one or both arguments are not lists.

Also, the string:strip/3 function is documented to take a character argument that is a character to strip from one or both ends of the string. Given a list instead of a character, it used to do nothing, but will now cause an exception.

Dialyzer will find most cases where those functions are passed arguments of the wrong type.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6295

The functions c:y/1,2 which call yecc:file/1,2 are now listed by c:help/0.

Documentation of c:y/1,2 has been added to c(3).

The fact that the control sequence character s recognizes binaries and deep character lists has been documented in io(3). This feature was added in R11B-0 (OTP-5403).

Own Id: OTP-6140

The shell command rr() sometimes failed to read record definitions from file(s). This problem has been fixed.

Own Id: OTP-6166 Aux Id: OTP-5878

The nonlocal function handler in erl_eval, which is used for implementing the restricted mode of the Erlang shell, did not handle calls to erlang:apply/3 correctly. This bug has been fixed.

Own Id: OTP-6169 Aux Id: seq10374

ets:rename/1 could deadlock, or crash the SMP emulator when the table wasn't a named table.

ets:next/2, and ets:prev/2 could return erroneous results on the SMP emulator.

Own Id: OTP-6198 Aux Id: seq10392, seq10415

When closing a Dets table the space management data was sometimes saved in such a way that opening the table could not be done without repairing the file. This bug has been fixed.

Own Id: OTP-6206

A bugfix in QLC: two of the call-back functions used for implementing QLC tables, TraverseFun and LookupFun, are now allowed to return a term other than a list. Such a term is immediately returned as the results of the current query, and is useful mostly for returning error tuples.

Several other minor bugs have been also been fixed.

Own Id: OTP-5195

The STDLIB modules error_logger_file_h and error_logger_tty_h now read the environment variable utc_log from the SASL application.

Own Id: OTP-5535

ets:info/1 has been corrected to behave according to the documentation and return a list of tuples, not a tuple with tuples.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-5639

Referencing a so far undeclared record from the default value of some record declaration is from now on considered an error by the linter. It is also an error if the default value of a record declaration uses or binds a variable.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-5878

When a file .hrl file is included using -include_lib, the include path is temporarily updated to include the directory the .hrl file was found in, which will allow that .hrl file to itself include files from the same directory as itself using -include. (Thanks to Richard Carlsson.)

Own Id: OTP-5944

Corrected filelib:ensure_dir/1 which sometimes returned true and sometimes ok to always return ok when successful. This goes against the documentation which said true, but ok was judged to be a more logical return value.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-5960 Aux Id: seq10240

The shell now handles records better when used in calls on the form {Module, Function}(ArgList).

Own Id: OTP-5990 Aux Id: OTP-5876

The functions lists:ukeysort/2 and lists:ukeymerge/3 have been changed in such a way that two tuples are considered equal if their keys match.

For the sake of consistency, lists:usort/2 and lists:umerge/3 have been modified too: two elements are considered equal if they compare equal.

The file_sorter module has been modified in a similar way: the unique option now applies to the key (keysort() and keymerge()) and the ordering function (the option {order, Order} ).

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-6019

Correction in documentation for ets:update_counter/3; failure with badarg also if the counter to be updated is the key.

Own Id: OTP-6072

When sorting terms using the file_sorter module and an ordering fun, the sort was not always stable. This bug has been fixed.

Own Id: OTP-6088

Improvements of the linter:

The Erlang code preprocessor epp now recognizes the file attribute. This attribute is meant to be used by tools such as Yecc that generate source code files.

Own Id: OTP-5362

The formatting option ~s of io:fwrite and io_lib:fwrite has been extended to handle arguments that are binaries or I/O lists.

Own Id: OTP-5403

The control sequences p and P of the Format argument of the functions io:format/2,3 and io_lib:format/2 have been changed as to display the contents of binaries containing printable characters as strings.

Own Id: OTP-5485

The linter emits warnings for functions exported more than once in export attributes.

Own Id: OTP-5494

A manual for STDLIB has been added, stdlib(6). It mentions the configuration parameters for the Erlang shell.

Own Id: OTP-5530

Added the zip module with functions for reading and creating zip archives. See zip(3).

Own Id: OTP-5786

Simple-one-for-one supervisors now store the pids of child processes using dict instead of a list. This significantly improves performance when there are many dynamic supervised child processes. (Thanks to Mickaël Rémond et al.)

Own Id: OTP-5898

When given the new option 'strict_record_tests', the compiler will generate code that verifies the record type for 'R#record.field' operations in guards. Code that verifies record types in bodies has already been generated since R10B, but in this release there will be a '{badrecord,RecordTag}' instead of a 'badmatch' if the record verification test fails. See the documentation for the compile module for more information.

The Erlang shell always applies strict record tests.

Own Id: OTP-5915 Aux Id: OTP-5714

The Erlang pretty printer (erl_pp) now tries to insert line breaks at appropriate places.

Own Id: OTP-5924

The public option has been removed from digraph:new/1. The reason is that several functions in the digraph module are implemented using multiple ETS accesses, which is not thread safe. (Thanks to Ulf Wiger.)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-5985

The function lists:keyreplace/4 checks that the fourth argument (NewTuple) is a tuple.

Own Id: OTP-6023

Added an example of how to reconstruct source code from debug info (abstract code) to beam_lib(3). (Thanks to Mats Cronqvist who wrote the example.)

Own Id: OTP-6073

The new compiler option warn_unused_record is used for finding unused locally defined record types.

Own Id: OTP-6105

shell_default:xm/1 has been added. It calls xref:m/1.

Own Id: OTP-5405 Aux Id: OTP-4101

Warnings are output whenever so far undeclared records are referenced from some default value of a record declaration. In STDLIB 1.14 (R11B) such forward references will cause a compilation error.

Own Id: OTP-5878

The linter's check of the deprecated attribute did not take the compile option export_all into account. This bug has been fixed.

Own Id: OTP-5917

The Erlang pretty printer did not handle try/catch correctly. This bug has been fixed.

Own Id: OTP-5926

Corrected documentation for lists:nthtail/3.

Added documentation for lists:keymap/3.

Tried to clarify some other type declarations and function descriptions in lists(3).

Corrected documentation for timer:now_diff/2.

Fixed broken links in gen_fsm(3), gen_server(3), io_lib(3) and lib(3).

Own Id: OTP-5931

Type checks have been added to functions in lists.erl.

Own Id: OTP-5939

The new STDLIB module erl_expand_records expands records in abstract code. It is used by the Erlang shell, which means that Compiler is no longer used by the shell.

Own Id: OTP-5876 Aux Id: OTP-5435

The compiler will now warn that the megaco:format_versions/1 function is deprecated.

Own Id: OTP-5976

When calling gen_server:enter_loop with a registered server name, it was only checked that the registered name existed, not that it actually was the name of the calling process.

Own Id: OTP-5854

More detail on beam_lib:version/1 in documentation.

Own Id: OTP-5789

The new function io:read/3 works like io:read/1,2 but takes a third argument, StartLine.

Own Id: OTP-5813

The new function gen_fsm:enter_loop/4,5,6, similar to gen_server:enter_loop/3,4,5, has been added.

Own Id: OTP-5846 Aux Id: seq10163

The function c:i/1 is now exported.

Own Id: OTP-5848 Aux Id: seq10164

A couple of type errors have been fixed in sofs.

Own Id: OTP-5739

The pre-processor used to complain that the macro definition '-define(S(S), ??S).' was circular, which it isn't. (Thanks to Richard Carlsson.)

Own Id: OTP-5777

The linter, QLC and the module erl_pp did not handle the new 'fun M:F/A' construct in all situations. This problem has been fixed.

Own Id: OTP-5644

The manual pages for most of the Kernel and some of the STDLIB modules have been updated, in particular regarding type definitions.

The documentation of the return value for erts:info/1 has been corrected.

The documentation for erlang:statistics/1 now lists all possible arguments.

Own Id: OTP-5360

Replaced some tuple funs with the new fun M:F/A construct.

The high-order functions in the lists module no longer accept bad funs under any circumstances. 'lists:map(bad_fun, [])' used to return '[]' but now causes an exception.

Unused, broken compatibility code in the ets module was removed. (Thanks to Dialyzer.)

Eliminated 5 discrepancies found by Dialyzer in the Appmon application.

Own Id: OTP-5633

The c:i/0 function will now run in a paged mode if there are more than 100 processes in the system. (Thanks to Ulf Wiger.)

erlang:system_info(process_count) has been optimized and does now return exactly the same value as length(processes()). Previously erlang:system_info(process_count) did not include exiting processes which are included in length(processes()).

The +P flag for erl, which sets the maximum number of processes allowed to exist at the same, no longer accepts values higher than 134217727. (You will still probably run out of memory before you'll be able to reach that limit.)

Own Id: OTP-5645 Aux Id: seq9984

Very minor corrections in beam_lib and its documentation.

Own Id: OTP-5589

The erlang:port_info/1 BIF is now documented. Minor corrections of the documentation for erlang:port_info/2.

Added a note to the documentation of the math module that all functions are not available on all platforms.

Added more information about the '+c' option in the erl man page in the ERTS documentation.

Own Id: OTP-5555

The new fun M:F/A construct creates a fun that refers to the latest version of M:F/A. This syntax is meant to replace tuple funs {M,F} which have many problems.

The new type test is_function(Fun,A) (which may be used in guards) test whether Fun is a fun that can be applied with A arguments. (Currently, Fun can also be a tuple fun.)

Own Id: OTP-5584

filelib:wildcard/2 was broken (it ignored its second argument).

Also, filelib:wildcard("Filename") (where the argument does not contain any meta-characters) would always return ["Filename"]. Corrected so that an empty list will be returned if "Filename" does not actually exist. (Same correction in filelib:wildcard/2.) (This change is a slight incompatibility.)

filelib:wildcard/1,2 will generate a different exception when given bad patterns such as "{a,". The exception used to be caused by 'exit(missing_delimiter)' but is now 'erlang:error({badpattern,missing_delimiter})'.

Own Id: OTP-5523 Aux Id: seq9824

Further improvements of encrypted debug info: New option encrypt_debug_info for compiler.

Own Id: OTP-5541 Aux Id: seq9837

When opening a Dets table read only an attempt was sometimes made to re-hash the table resulting in an error message. This problem has been fixed.

Own Id: OTP-5487 Aux Id: OTP-4989

It is now possible to encrypt the debug information in Beam files, to help keep the source code secret. See the documentation for compile on how to provide the key for encrypting, and the documentation for beam_lib on how to provide the key for decryption so that tools such as the Debugger, Xref, or Cover can be used.

The beam_lib:chunks/2 functions now accepts an additional chunk type compile_info to retrieve the compilation information directly as a term. (Thanks to Tobias Lindahl.)

Own Id: OTP-5460 Aux Id: seq9787

Closing a Dets table kept in RAM would cause a crash if the file could not be written. This problem has been fixed by returning an error tuple.

Own Id: OTP-5402

erl_pp now correctly pretty-prints fun F/A.

Own Id: OTP-5412

The Erlang shell failed if the compiler was not in the code path. This problem has been fixed, but in order to evaluate records the compiler is still needed.

Own Id: OTP-5435

Corrected the example in the documentation for ets:match/2. Also clarified that ets:update_counter/3 updates the counter atomically. (Thanks to Anders Svensson.)

Own Id: OTP-5452 Aux Id: seq9770, seq9789

The possibility to start the Erlang shell in parallel with the rest of the system was reintroduced for backwards compatibility in STDLIB 1.13.1. The flag to be used for this is now called async_shell_start and has been documented. New shells started from the JCL menu are not synchronized with init anymore. This makes it possible to start a new shell (e.g. for debugging purposes) even if the initial shell has not come up.

Own Id: OTP-5406 Aux Id: OTP-5218

The compiler will now produce warnings when using the deprecated functions in the snmp module.

Own Id: OTP-5425

The function c:zi/0 has been removed. Use c:i/0 instead.

Own Id: OTP-5432

Corrected two minor bugs found by the Dialyzer: Calling a parameterized module from a restricted shell (i.e. if shell:start_restricted/1 has been used) would crash the shell evaluator. A debug printout in gen_fsm had a clause that would never match; causing less information to be printed.

And a somewhat more serious one also found by Dialyzer: rpc:yield/1 would crash unless the call started by rpc:async_call/4 had already finished; rpc:nb_yield(Key,infinity) would also crash.

Cleaned up and removed redundant code found by Dialyzer in erlang:dmonitor_p/2.

Own Id: OTP-5462

Bugs in the Erlang shell have been fixed.

Own Id: OTP-5327

Some dead code reported by Dialyzer was eliminated.

A bug in dbg when tracing to wrap trace files has been corrected. It failed to delete any already existing wrap trace files with the same names when starting a new wrap trace.

Own Id: OTP-5329

The linter could output invalid warnings about bit patterns in record initializations. This problem has been fixed.

Own Id: OTP-5338

ordsets:is_set(NoList), where NoList is any term except a list, would crash. For consistency with sets:is_set/1 and gb_sets:is_set/1, it now returns false.

Own Id: OTP-5341

A BIF erlang:raise/3 has been added. See the manual for details. It is intended for internal system programming only, advanced error handling.

Own Id: OTP-5376 Aux Id: OTP-5257

The deprecated attribute is now checked by the linter. See xref(3) for a description of the deprecated attribute.

Own Id: OTP-5276

The restricted shell will now indicate if the return value from a user predicate is on an incorrect form.

Own Id: OTP-5335

Bugs concerning unused and shadowed variables have been fixed in the linter.

Own Id: OTP-5091

A bug in the evaluator that caused the shell to choke on bit syntax expressions has been fixed.

Own Id: OTP-5237

io:format/2 et.al no longer crashes for some combinations of precision and value for format character "g". Previously it crashed if the precision P was 4 or lower and the absolute value of the float to print was lower than 10^4 but 10^(P-1) or higher. Now it will not crash depending on the value of the float.

Own Id: OTP-5263

Bugs in the handling of the bit syntax have been fixed in the Erlang shell.

Own Id: OTP-5269

gb_sets:del_element/2 was changed to do the same as gb_sets:delete_any/2 which was the original intention, not as gb_sets:delete/2. Code that relies on gb_sets:del_element/2 causing an error if the element does not exist must be changed to call gb_sets:delete/2 instead.

The documentation was also updated to explicitly document functions that were only referred to as 'aliases' of a documented function. Also, a list of all functions common to the gb_sets, sets, and ordsets was added.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-5277

Debug messages have been removed from the QLC module.

Own Id: OTP-5283

The size of continuations returned from dets:match/1,3, dets:match_object/1,3, and dets:select/1,3 has been reduced. This affects the amount of data Mnesia sends between nodes while evaluating QLC queries.

Own Id: OTP-5232

The -rsh switch for starting a remote shell (introduced with OTP-5210) clashed with an already existing switch used by slave. Therefore the switch for the remote shell is now instead named -remsh.

Own Id: OTP-5248 Aux Id: OTP-5210

The Pman 'trace shell' functionality was broken as has now been fixed. Furthermore, Pman could not correctly find the pid of the active shell if more than one shell process was running on the node. This has also been corrected.

Own Id: OTP-5191

When the undocumented feature "parameterized modules" was used, the ?MODULE macro did not work correctly.

Own Id: OTP-5224

You can now start Erlang with the -rsh flag which gives you a remote initial shell instead of a local one. Example:

            erl -sname this_node -rsh other_node@other_host
          

Own Id: OTP-5210

The man page for the lists module has been updated with description of the new zip, unzip, and partition/2 functions.

Own Id: OTP-5213

The top level group leader used to be listed as job #1 in the job list in JCL mode. Since there is no shell associated with this process that can be connected to, it will no longer be listed.

Own Id: OTP-5214

The possibility to start the Erlang shell in parallel with the rest of the system has been reintroduced for backwards compatibility. Note that this old behaviour is error prone and should not be used unless for some reason necessary.

Own Id: OTP-5218 Aux Id: seq9534

The shell commands rr/1,2,3 now accepts wildcards when reading record definitions from BEAM files.

Own Id: OTP-5226