aboutsummaryrefslogtreecommitdiffstats
path: root/lib/kernel
AgeCommit message (Collapse)Author
2018-01-08Merge branch 'john/erts/putenv-thread-safety/OTP-14666'John Högberg
2018-01-03Disallow NULs in filename-encoded stringsJohn Högberg
Previously we accepted trailing NULs, which was backwards compatible as such usage never resulted in misbehavior in the first place. The downside is that it prevented erts_native_filename_need from returning an accurate number of *actual characters*, needlessly complicating encoding-agnostic code like erts_osenv.
2018-01-03Replace the libc environment with a thread-safe emulationJohn Högberg
putenv(3) and friends aren't thread-safe regardless of how you slice it; a global lock around all environment operations (like before) keeps things safe as far as our own operations go, but we have absolutely no control over what libc or a library dragged in by a driver/NIF does -- they're free to call getenv(3) or putenv(3) without honoring our lock. This commit solves this by setting up an "emulated" environment which can't be touched without going through our interfaces. Third-party libraries can still shoot themselves in the foot but benign uses of os:putenv/2 will no longer risk crashing the emulator.
2017-12-21Merge branch 'maint'Sverker Eriksson
2017-12-21Merge branch 'sverker/cuddle-valgrind' into maintSverker Eriksson
2017-12-21Merge pull request #1651 from getong/doc_updateLukas Larsson
false should be never, false not found in the source code
2017-12-19Merge branch 'dgud/kernel/refc_sched_wall_time/OTP-11694'Dan Gudmundsson
* dgud/kernel/refc_sched_wall_time/OTP-11694: test: spawn scheduler_wall_time flag holder Turn on scheduler_wall_time in an alive process Redirect system_flag(scheduler_wall_time,_) to kernel_refc kernel: add a resource reference counter
2017-12-19Redirect system_flag(scheduler_wall_time,_) to kernel_refcRickard Green
2017-12-13Merge pull request #1654 from bjorng/bjorn/kernel/file_io_serverBjörn Gustavsson
Slightly optimize reading of cooked files in list mode
2017-12-13Merge branch 'maint'Henrik Nord
2017-12-12kernel: Try fix/skip prim_file_SUITE:large_write for valgrindSverker Eriksson
by limiting the memory usage. Our valgrind test machine (pharazon) seems to get totally swamped by this test case.
2017-12-08Update release notesErlang/OTP
2017-12-08Update version numbersErlang/OTP
2017-12-08Slightly optimize reading of cooked files in list modeBjörn Gustavsson
In general, the new NIF-based file routines are faster than the old efile driver. However, on some computers, building the entire OTP system is somewhat slower. It turns out that it is because 'erlc' cheated by turning off the IO thread pool (using '+A0') to avoid context switches between scheduler threads and threads in the IO thread pool. The new file routines perform IO on dirty IO threads, and there is (by intent) no way to force the operations to occur on scheduler threads to avoid the context switches What we can do to is to use a small (4Kb) read-ahead buffer for files opened for reading (only) in list mode (which is how the compiler opens its input files). The buffering reduces the number of context switches between scheduler threads and dirty IO threads. On my computer that seems to slightly speed up building of the entire OTP system. The buffer should do no harm. The only potential for harm I can think of is random access to a file opened in read mode, where the read-ahead buffer could slightly decrease performance. That does not seems to be a likely use case in practice, though.
2017-12-04Merge branch 'raimo/ssl-dist-bench/OTP-14657'Raimo Niskanen
* origin/raimo/ssl-dist-bench/OTP-14657: Write SSL distribution benchmarks Polish SSL distribution Handle whitebox test message Correct distribution doc Use SNI when connecting Use -ssl_dist_optfile options Read in -ssl_dist_optfile to ETS Facilitate test certs with common root Stop checking DNS name for SNI
2017-12-04Correct distribution docRaimo Niskanen
2017-12-04false should be never, false not found in the source codegetong
2017-12-01kernel: add a resource reference counterDan Gudmundsson
System resources/functionality may need to be reference counted to be handled correctly when used or enabled/disabled from more than one process or application. It is easier to handle this in erlang code than in erts, so make a process that deals with the housekeeping.
2017-11-30Make file:sendfile/5 follow its documented error behaviorJohn Högberg
2017-11-30Remove doc for sendfile/5 use_threads optionJohn Högberg
The option no longer does anything; systems that lack support for non-blocking sendfile(2) will use the Erlang fallback.
2017-11-30Update file performance adviceJohn Högberg
The parts relating to drivers/ports are now obsolete, and the provided example was far noisier than it had to be; the only relevant metric is the number of calls and it's up to the user to decide how those will be reduced. One could argue for its complete removal, but I'm inclined to leave it be.
2017-11-30Stop testing use_threads in sendfile_SUITEJohn Högberg
The option no longer does anything at all.
2017-11-30Stop testing the artificial size limit on writesJohn Högberg
The old driver didn't fall back to using write(2) if writev(2) failed due to the combined length of the iov overflowing a ssize_t, but the new one doesn't have any problems with it so we failed this test with a case_clause error on 32-bit machines.
2017-11-30get_cwd/1 on non-existent drives should error outJohn Högberg
2017-11-30Reads that draw from both buffer and file must workJohn Högberg
2017-11-30Volume-relative paths must work on WindowsJohn Högberg
2017-11-30Operations on closed raw files should return EINVALJohn Högberg
2017-11-30Use lexemes/2 instead of the deprecated tokens/2John Högberg
2017-11-30Remove disk_log_SUITE:evilJohn Högberg
This test revolves around the possibility that the underlying port can be killed, which is nonsense now that the file suite no longer uses ports for anything.
2017-11-30Ensure that trailing slashes are ignored on list_dirJohn Högberg
2017-11-30Ensure that root paths are translated to our preferred formJohn Högberg
2017-11-30Tighten timings in delayed_writeJohn Högberg
The cumulative wait time was as long as the delay itself in the flush-on-size test, causing the test to pass because the write managed to time out.
2017-11-30pread/2 must always return a list of resultsJohn Högberg
2017-11-30Test opening raw files in the same manner as regular onesJohn Högberg
2017-11-30Add microbenchmarks for file:read/2 and file:write/2John Högberg
2017-11-30Account for new behavior in tests that touch prim_fileJohn Högberg
This also hides the module behind ?PRIM_FILE to make testing new implementations less painful.
2017-11-30Reimplement efile_drv as a dirty NIFJohn Högberg
This improves the latency of file operations as dirty schedulers are a bit more eager to run jobs than async threads, and use a single global queue rather than per-thread queues, eliminating the risk of a job stalling behind a long-running job on the same thread while other async threads sit idle. There's no such thing as a free lunch though; the lowered latency comes at the cost of increased busy-waiting which may have an adverse effect on some applications. This behavior can be tweaked with the +sbwt flag, but unfortunately it affects all types of schedulers and not just dirty ones. We plan to add type-specific flags at a later stage. sendfile has been moved to inet_drv to lessen the effect of a nasty race; the cooperation between inet_drv and efile has never been airtight and the socket dying at the wrong time (Regardless of reason) could result in fd aliasing. Moving it to the inet driver makes it impossible to trigger this by closing the socket in the middle of a sendfile operation, while still allowing it to be aborted -- something that can't be done if it stays in the file driver. The race still occurs if the controlling process dies in the short window between dispatching the sendfile operation and the dup(2) call in the driver, but it's much less likely to happen now. A proper fix is in the works. -- Notable functional differences: * The use_threads option for file:sendfile/5 no longer has any effect. * The file-specific DTrace probes have been removed. The same effect can be achieved with normal tracing together with the nif__entry/nif__return probes to track scheduling. -- OTP-14256
2017-11-27Merge branch 'maint'Dan Gudmundsson
* maint: Update HOWTO with wxWidgets build examples https should be mapped to url and not seealso Update kernel appup for OTP-20.2
2017-11-27Merge branch 'siri/appups-20.2' into maintSiri Hansen
* siri/appups-20.2: Update kernel appup for OTP-20.2
2017-11-23Merge branch 'sverker/async-auto-connect/OTP-14370' AGAINSverker Eriksson
2017-11-23erts: Fix erlang:monitor toward c-nodesSverker Eriksson
by suppressing DOP_MONITOR_P, DOP_MONITOR_P_EXIT and DOP_DEMONITOR_P if not supported by the remote node. In 17e198d6ee60f7dec9abfed272cf4226aea44535 I changed the behavior of erlang:monitor to not raise badarg for c-nodes but instead create a monitor to only supervise the connection. But I forgot to prevent DOP_MONITOR_P and friends from being sent to the node that does not expect them. Note: We test both DFLAG_DIST_MONITOR and DFLAG_DIST_MONITOR_NAME for the node to support process monitoring. This is because erl_interface is buggy as it sets DFLAG_DIST_MONITOR without really supporting it. ToDo: Should erl_interface stop setting DFLAG_DIST_MONITOR or should we change the meaning of these flags?
2017-11-22Fix broken to_dis after merge from maint to masterBjörn Gustavsson
d8d07a7593d811 that added the to_dis option to the compiler no longer works when merged to master. That is because of 79f28cfd8df1b7 that removed some unused code in erts_debug. Fix this by adding a new function erts_debug:dis_to_file/2 and use it from compile module where we have access to the filename (in beam_listing we only have an opened file).
2017-11-20Merge branch 'maint'Lukas Larsson
2017-11-20Merge branch 'lukas/system/improve_profile_docs/OTP-14633' into maintLukas Larsson
* lukas/system/improve_profile_docs/OTP-14633: erts: Add term2bin note about encoding guarantee Fix broken link in INSTALL howto system: Add info to profiling effeciency guide Update message queue and pinfo binary docs kernel: Clarify gen_tcp:listen documentation system: Add External OSS tools section to profiling guilde
2017-11-20Merge branch 'maint'Lukas Larsson
Conflicts: lib/compiler/src/beam_listing.erl
2017-11-20Merge branch 'lukas/compiler/add_to_dis/OTP-14784' into maintLukas Larsson
* lukas/compiler/add_to_dis/OTP-14784: compiler: Add +to_dis option that dumps loaded asm
2017-11-20compiler: Add +to_dis option that dumps loaded asmLukas Larsson
2017-11-20Merge branch 'maint'Lukas Larsson
2017-11-20Merge branch 'lukas/docs/xmllint_fixes/OTP-14721' into maintLukas Larsson
* lukas/docs/xmllint_fixes/OTP-14721: ssl/ssh: Remove/ignore unused XML_FILES doc files Refactor xmllint check and make it fail on failure Add toplevel xmllint make target Conflicts: lib/crypto/doc/src/Makefile
2017-11-15Move new|abort_connection_id to erts_internalSverker Eriksson
and drop _id suffix.