aboutsummaryrefslogtreecommitdiffstats
path: root/lib/kernel
AgeCommit message (Collapse)Author
2017-12-13Merge pull request #1654 from bjorng/bjorn/kernel/file_io_serverBjörn Gustavsson
Slightly optimize reading of cooked files in list mode
2017-12-13Merge branch 'maint'Henrik Nord
2017-12-08Update release notesErlang/OTP
2017-12-08Update version numbersErlang/OTP
2017-12-08Slightly optimize reading of cooked files in list modeBjörn Gustavsson
In general, the new NIF-based file routines are faster than the old efile driver. However, on some computers, building the entire OTP system is somewhat slower. It turns out that it is because 'erlc' cheated by turning off the IO thread pool (using '+A0') to avoid context switches between scheduler threads and threads in the IO thread pool. The new file routines perform IO on dirty IO threads, and there is (by intent) no way to force the operations to occur on scheduler threads to avoid the context switches What we can do to is to use a small (4Kb) read-ahead buffer for files opened for reading (only) in list mode (which is how the compiler opens its input files). The buffering reduces the number of context switches between scheduler threads and dirty IO threads. On my computer that seems to slightly speed up building of the entire OTP system. The buffer should do no harm. The only potential for harm I can think of is random access to a file opened in read mode, where the read-ahead buffer could slightly decrease performance. That does not seems to be a likely use case in practice, though.
2017-12-04Merge branch 'raimo/ssl-dist-bench/OTP-14657'Raimo Niskanen
* origin/raimo/ssl-dist-bench/OTP-14657: Write SSL distribution benchmarks Polish SSL distribution Handle whitebox test message Correct distribution doc Use SNI when connecting Use -ssl_dist_optfile options Read in -ssl_dist_optfile to ETS Facilitate test certs with common root Stop checking DNS name for SNI
2017-12-04Correct distribution docRaimo Niskanen
2017-11-30Make file:sendfile/5 follow its documented error behaviorJohn Högberg
2017-11-30Remove doc for sendfile/5 use_threads optionJohn Högberg
The option no longer does anything; systems that lack support for non-blocking sendfile(2) will use the Erlang fallback.
2017-11-30Update file performance adviceJohn Högberg
The parts relating to drivers/ports are now obsolete, and the provided example was far noisier than it had to be; the only relevant metric is the number of calls and it's up to the user to decide how those will be reduced. One could argue for its complete removal, but I'm inclined to leave it be.
2017-11-30Stop testing use_threads in sendfile_SUITEJohn Högberg
The option no longer does anything at all.
2017-11-30Stop testing the artificial size limit on writesJohn Högberg
The old driver didn't fall back to using write(2) if writev(2) failed due to the combined length of the iov overflowing a ssize_t, but the new one doesn't have any problems with it so we failed this test with a case_clause error on 32-bit machines.
2017-11-30get_cwd/1 on non-existent drives should error outJohn Högberg
2017-11-30Reads that draw from both buffer and file must workJohn Högberg
2017-11-30Volume-relative paths must work on WindowsJohn Högberg
2017-11-30Operations on closed raw files should return EINVALJohn Högberg
2017-11-30Use lexemes/2 instead of the deprecated tokens/2John Högberg
2017-11-30Remove disk_log_SUITE:evilJohn Högberg
This test revolves around the possibility that the underlying port can be killed, which is nonsense now that the file suite no longer uses ports for anything.
2017-11-30Ensure that trailing slashes are ignored on list_dirJohn Högberg
2017-11-30Ensure that root paths are translated to our preferred formJohn Högberg
2017-11-30Tighten timings in delayed_writeJohn Högberg
The cumulative wait time was as long as the delay itself in the flush-on-size test, causing the test to pass because the write managed to time out.
2017-11-30pread/2 must always return a list of resultsJohn Högberg
2017-11-30Test opening raw files in the same manner as regular onesJohn Högberg
2017-11-30Add microbenchmarks for file:read/2 and file:write/2John Högberg
2017-11-30Account for new behavior in tests that touch prim_fileJohn Högberg
This also hides the module behind ?PRIM_FILE to make testing new implementations less painful.
2017-11-30Reimplement efile_drv as a dirty NIFJohn Högberg
This improves the latency of file operations as dirty schedulers are a bit more eager to run jobs than async threads, and use a single global queue rather than per-thread queues, eliminating the risk of a job stalling behind a long-running job on the same thread while other async threads sit idle. There's no such thing as a free lunch though; the lowered latency comes at the cost of increased busy-waiting which may have an adverse effect on some applications. This behavior can be tweaked with the +sbwt flag, but unfortunately it affects all types of schedulers and not just dirty ones. We plan to add type-specific flags at a later stage. sendfile has been moved to inet_drv to lessen the effect of a nasty race; the cooperation between inet_drv and efile has never been airtight and the socket dying at the wrong time (Regardless of reason) could result in fd aliasing. Moving it to the inet driver makes it impossible to trigger this by closing the socket in the middle of a sendfile operation, while still allowing it to be aborted -- something that can't be done if it stays in the file driver. The race still occurs if the controlling process dies in the short window between dispatching the sendfile operation and the dup(2) call in the driver, but it's much less likely to happen now. A proper fix is in the works. -- Notable functional differences: * The use_threads option for file:sendfile/5 no longer has any effect. * The file-specific DTrace probes have been removed. The same effect can be achieved with normal tracing together with the nif__entry/nif__return probes to track scheduling. -- OTP-14256
2017-11-27Merge branch 'maint'Dan Gudmundsson
* maint: Update HOWTO with wxWidgets build examples https should be mapped to url and not seealso Update kernel appup for OTP-20.2
2017-11-27Merge branch 'siri/appups-20.2' into maintSiri Hansen
* siri/appups-20.2: Update kernel appup for OTP-20.2
2017-11-23Merge branch 'sverker/async-auto-connect/OTP-14370' AGAINSverker Eriksson
2017-11-23erts: Fix erlang:monitor toward c-nodesSverker Eriksson
by suppressing DOP_MONITOR_P, DOP_MONITOR_P_EXIT and DOP_DEMONITOR_P if not supported by the remote node. In 17e198d6ee60f7dec9abfed272cf4226aea44535 I changed the behavior of erlang:monitor to not raise badarg for c-nodes but instead create a monitor to only supervise the connection. But I forgot to prevent DOP_MONITOR_P and friends from being sent to the node that does not expect them. Note: We test both DFLAG_DIST_MONITOR and DFLAG_DIST_MONITOR_NAME for the node to support process monitoring. This is because erl_interface is buggy as it sets DFLAG_DIST_MONITOR without really supporting it. ToDo: Should erl_interface stop setting DFLAG_DIST_MONITOR or should we change the meaning of these flags?
2017-11-22Fix broken to_dis after merge from maint to masterBjörn Gustavsson
d8d07a7593d811 that added the to_dis option to the compiler no longer works when merged to master. That is because of 79f28cfd8df1b7 that removed some unused code in erts_debug. Fix this by adding a new function erts_debug:dis_to_file/2 and use it from compile module where we have access to the filename (in beam_listing we only have an opened file).
2017-11-20Merge branch 'maint'Lukas Larsson
2017-11-20Merge branch 'lukas/system/improve_profile_docs/OTP-14633' into maintLukas Larsson
* lukas/system/improve_profile_docs/OTP-14633: erts: Add term2bin note about encoding guarantee Fix broken link in INSTALL howto system: Add info to profiling effeciency guide Update message queue and pinfo binary docs kernel: Clarify gen_tcp:listen documentation system: Add External OSS tools section to profiling guilde
2017-11-20Merge branch 'maint'Lukas Larsson
Conflicts: lib/compiler/src/beam_listing.erl
2017-11-20Merge branch 'lukas/compiler/add_to_dis/OTP-14784' into maintLukas Larsson
* lukas/compiler/add_to_dis/OTP-14784: compiler: Add +to_dis option that dumps loaded asm
2017-11-20compiler: Add +to_dis option that dumps loaded asmLukas Larsson
2017-11-20Merge branch 'maint'Lukas Larsson
2017-11-20Merge branch 'lukas/docs/xmllint_fixes/OTP-14721' into maintLukas Larsson
* lukas/docs/xmllint_fixes/OTP-14721: ssl/ssh: Remove/ignore unused XML_FILES doc files Refactor xmllint check and make it fail on failure Add toplevel xmllint make target Conflicts: lib/crypto/doc/src/Makefile
2017-11-15Move new|abort_connection_id to erts_internalSverker Eriksson
and drop _id suffix.
2017-11-15Cleanup net_kernelSverker Eriksson
2017-11-15Improve connection abortingSverker Eriksson
2017-11-15erts: Keep magic ref to DistEntry in net_kernelSverker Eriksson
to make sure it's kept alive.
2017-11-15erts: Introduce asynchronous auto-connectSverker Eriksson
2017-11-15kernel: Move auto connect into net_kernel serverSverker Eriksson
as a preparation for async auto-connect requests.
2017-11-15erts: Make DFLAGS_NEW_FUN_TAGS mandatorySverker Eriksson
and remove ugly encoding fallback as {fun, ...} DFLAGS_NEW_FUN_TAGS has been supported by vm/erl_interface/jinterface since R13 or even older. Renamed test case obsolete_funs to term2bin_tuple_fallbacks and removed test for {fun,...} fallback and added missing test for bitstring fallback {Binary, Bits}.
2017-11-15Merge PR-1605 from sverker/sverker/binary_to_term-used OTP-14780Sverker Eriksson
Add 'used' option for binary_to_term/2
2017-11-10Merge branch 'maint'Ingela Anderton Andin
* maint: Updated OTP version Update release notes Update version numbers inets: Prepare for release inets: Add missing guard Avoid WindowBits=8 as per the manual Fix deflateParams on zlib 1.2.11 Ignore empty binaries in enif_inspect_iovec Emasculate writable binaries on entering an iovec Only apply EOS behaviors if there's pending data Conflicts: OTP_VERSION
2017-11-10Merge branch 'maint-20' into maintIngela Anderton Andin
* maint-20: Updated OTP version Update release notes Update version numbers inets: Prepare for release inets: Add missing guard Avoid WindowBits=8 as per the manual Fix deflateParams on zlib 1.2.11 Ignore empty binaries in enif_inspect_iovec Emasculate writable binaries on entering an iovec Only apply EOS behaviors if there's pending data Stop assuming that all schedulers are managed when updating msacc
2017-11-09Merge branch 'john/erts/fix-gunzip-eos/OTP-14730/ERL-507' into maint-20Erlang/OTP
* john/erts/fix-gunzip-eos/OTP-14730/ERL-507: Only apply EOS behaviors if there's pending data
2017-11-09Avoid WindowBits=8 as per the manualJohn Högberg
The docs for zlib:deflateInit/6 explicitly mention that 8 is broken and should not be used unless we know for certain that our zlib version supports it.