| Age | Commit message (Collapse) | Author |
|---|
|
* sv/file-osync/OTP-11498:
Add sync option to file:open/2
Conflicts:
erts/preloaded/ebin/prim_file.beam
|
|
Conflicts:
erts/preloaded/ebin/prim_inet.beam
lib/kernel/test/gen_sctp_SUITE.erl
|
|
|
|
|
|
The sync option adds the POSIX O_SYNC flag to the open system call on
platforms that support the flag or its equivalent, e.g.,
FILE_FLAG_WRITE_THROUGH on Windows. For platforms that don't support it,
file:open/2 returns {error, enotsup} if the sync option is passed in.
The semantics of O_SYNC are platform-specific. For example, not all
platforms guarantee that all file metadata are written to the disk along
with the file data when the flag is in effect. This issue is noted in the
documentation this commit adds for the sync option.
Add a test for the sync option. Note however that the underlying OS
semantics for O_SYNC can't be tested automatically in any practical way, so
the test assumes the OS does the right thing with the flag when
present. For manual verification, dtruss on OS X and strace on Linux were
both run against beam processes to watch calls to open(), and file:open/2
was called in Erlang shells to open files for writing, both with and
without the sync option. Both the dtruss output and the strace output
showed that the O_SYNC flag was present in the open() calls when sync was
specified and was clear when sync was not specified.
|
|
|
|
|
|
Conflicts:
erts/preloaded/ebin/erlang.beam
|
|
Introduce unconditional ERTS_ASSERT
and use that for both ASSERT and ASSERT_EXPR.
|
|
Add the {active,N} socket option, where N is an integer in the range
-32768..32767, to allow a caller to specify the number of data messages to
be delivered to the controlling process. Once the socket's delivered
message count either reaches 0 or is explicitly set to 0 with
inet:setopts/2 or by including {active,0} as an option when the socket is
created, the socket transitions to passive ({active, false}) mode and the
socket's controlling process receives a message to inform it of the
transition. TCP sockets receive {tcp_passive,Socket}, UDP sockets receive
{udp_passive,Socket} and SCTP sockets receive {sctp_passive,Socket}.
The socket's delivered message counter defaults to 0, but it can be set
using {active,N} via any gen_tcp, gen_udp, or gen_sctp function that takes
socket options as arguments, or via inet:setopts/2. New N values are added
to the socket's current counter value, and negative numbers can be used to
reduce the counter value. Specifying a number that would cause the socket's
counter value to go above 32767 causes an einval error. If a negative
number is specified such that the counter value would become negative, the
socket's counter value is set to 0 and the socket transitions to passive
mode. If the counter value is already 0 and inet:setopts(Socket,
[{active,0}]) is specified, the counter value remains at 0 but the
appropriate passive mode transition message is generated for the socket.
This commit contains a modified preloaded prim_inet.beam due to changes in
prim_inet.erl.
Add tests for {active,N} mode for TCP, UDP, and SCTP sockets.
Add documentation for {active,N} mode for inet, gen_tcp, gen_udp, and
gen_sctp.
|
|
* raimo/fix-signedness-flaws-in-efile_drv:
Fix EV_* macros and functions signedness flaws
|
|
* raimo/linux-network-namespace-sockopt/OTP-11157:
Document socket option 'netns'
Rudimentary test
Make netns option value a string
Implement netns for SCTP + bugfixes
Implement netns option for TCP and UDP
Implement emulator netns support for TCP and UDP
|
|
|
|
The actual port id is used to create a key from the
pointer value which is the ErlDrvPort. To do this
a new driver api function driver_async_port_key is
added and the driver API minor version is updated.
The documentation is updated and the faulty description of
how to spread ports over async threads is updated to
use the new API.
Testcase also added.
|
|
|
|
|
|
This is needed as some gcc versions seems to optimize this undefined
behaviour in a way which breaks this code.
|
|
|
|
|
|
|
|
Bug introduced in a73414d2e8ad03538 and never released.
|
|
* rickard/inet_opts/OTP-11075:
Make high_msgq_watermark and low_msgq_watermark generic inet options
Conflicts:
erts/preloaded/ebin/prim_inet.beam
|
|
|
|
* lukas/erts/efile_delayed_write_fix/OTP-10984:
Do driver_deq in worker threads instead of async_ready
|
|
Doing it in async_ready was needed before the pdl was introduced, but
now with the pdl the deq no longer needs the port lock to protect it.
This was not an issue when async_ready was called in the worker thread,
but now (R15B) that it is signalled back to the scheduler, some very nasty
race conditions could occur when using driver_timer and async jobs.
|
|
|
|
|
|
|
|
* egil/win-efile-bugfix/OTP-10890:
erts: Use correct type for ReadFile bytes read
|
|
|
|
Using a 64bit type for bytes read will not always clear the higher bits.
|
|
* sverk/tcp-exit_on_close-false:
inet_drv: Fix condition to reject INET_REQ_IGNOREFD for UDP and SCTP
A stab at fixing bug with {exit_on_close,false} not working
|
|
|
|
Seems to happen with async threads and when user closes
the file explicitly before the port is closed.
|
|
|
|
when invalid packets are received.
|
|
* pan/fix-compiler-warnings-clang-and-new-gcc:
Fix compiler warnings from GCC 4.7.1 on ARCH Linux
Fix clang compiler warnings on FreeBSD in erts
|
|
|
|
The following are deliberately left, as I have only a list of compiler
warnings and no system to test on:
hipe/hipe_x86_signal.c:264:5: warning: no previous prototype for function '_sigaction' [-Wmissing-prototypes]
int __SIGACTION(int signum, const struct sigaction *act, struct sigaction *oldact)
^
hipe/hipe_x86_signal.c:222:21: note: expanded from macro '__SIGACTION'
^
1 warning generated.
sys/unix/sys_float.c:835:16: warning: declaration of 'struct exception' will not be visible outside of this function [-Wvisibility]
matherr(struct exception *exc)
^
sys/unix/sys_float.c:835:1: warning: no previous prototype for function 'matherr' [-Wmissing-prototypes]
matherr(struct exception *exc)
^
2 warnings generated.
drivers/unix/unix_efile.c:1504:11: warning: implicit declaration of function 'sendfile' [-Wimplicit-function-declaration]
retval = sendfile(in_fd, out_fd, *offset, SENDFILE_CHUNK_SIZE,
^
1 warning generated.
|
|
Also, on 64 bit architectures, use 64 bit int's for the
counters and be specific about the counter variables sizes
utilizing datatypes from sys.h.
|
|
|
|
* pan/R16/redhat_workaround:
Clean up and make the fix work on windows.
Add workaround for CentOS/RedHat writev bug to inet_drv
OTP-10747
|
|
|
|
* fdm/file-allocate/OTP-10680:
Update preloaded prim_file.beam
erts: Fix xcomp configure for fallocate
Add file:allocate/3 operation
|
|
This operation allows pre-allocation of space for files.
It succeeds only on systems that support such operation.
The POSIX standard defines the optional system call
posix_fallocate() to implement this feature. However,
some systems implement more specific functions to
accomplish the same operation.
On Linux, if the more specific function fallocate() is
implemented, it is used instead of posix_fallocate(),
falling back to posix_fallocate() if the fallocate()
call failed (it's only supported for the ext4, ocfs2,
xfs and btrfs file systems at the moment).
On Mac OS X it uses the specific fcntl() operation
F_PREALLOCATE, falling back to posix_fallocate() if
it's available (at the moment Mac OS X doesn't provide
posix_fallocate()).
On any other UNIX system, it uses posix_fallocate() if it's
available. Any other system not providing this system call
or any function to pre-allocate space for files, this operation
always fails with the ENOTSUP POSIX error.
|
|
When using the async thread pool, if an erlang process
asks to open a file and it gets shutdown/killed while
the file:open/2 call hasn't returned, it's possible to
leak a file descriptor against the target file.
This happens because when the file driver is stopped
(file_stop() function is called), an async thread is
executing, about to execute, or executed already the
invoke_open() function. After file_stop() is called,
the file_async_ready() function will not run, and this
function is responsible for setting desc->fd with the
file descriptor that invoke_open() got. The file_stop()
call closes desc->fd if it refers to a valid file
descriptor, which is not the case here, because this
function was called before file_async_ready() could
run.
This leak is easily reproducile in a GNU/Linux system
using the following test code:
-module(t).
-export([t/1]).
t(N) ->
Pid = spawn_link(fun() ->
process_flag(trap_exit, true),
loop(N)
end),
Ref = erlang:monitor(process, Pid),
receive {'DOWN', Ref, _, _, _} ->
ok
end.
loop(0) ->
ok;
loop(N) ->
Name = integer_to_list(N),
Server = self(),
Pid = spawn(fun() ->
Server ! continue,
{ok, FdW} = file:open(Name, [raw, write]),
{ok, FdR} = file:open(Name, [raw, read]),
% Optional close calls, with or without them
% it makes no difference.
%ok = file:close(FdW),
%ok = file:close(FdR),
ok
end),
receive continue -> ok end,
exit(Pid, shutdown),
loop(N - 1).
Running this code with a few iterations is enough to very often
notice, with the lsof command, that the beam.smp process is holding
forever file descriptors open. This issue doesn't happen if the
async thread pool is not used.
Example:
$ erl +A 4
Erlang R15B03 (erts-5.9.3) [source] [64-bit] [smp:4:4] [async-threads:4] [hipe] [kernel-poll:false]
Eshell V5.9.3 (abort with ^G)
1> c(t).
{ok,t}
2> os:getpid().
"31975"
3> t:t(20).
ok
In a separate shell:
$ lsof -p 31975
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
beam.smp 31975 fdmanana cwd DIR 8,18 22736896 32563204 /home/fdmanana/git/hub/otp/tmp
beam.smp 31975 fdmanana rtd DIR 8,1 4096 2 /
beam.smp 31975 fdmanana txt REG 8,1 7600263 1835126 /opt/r15b03/lib/erlang/erts-5.9.3/bin/beam.smp
beam.smp 31975 fdmanana mem REG 8,1 7220736 2497283 /usr/lib/locale/locale-archive
beam.smp 31975 fdmanana mem REG 8,1 10280 2505021 /usr/lib/libsctp.so.1.0.11
beam.smp 31975 fdmanana mem REG 8,1 1811128 917795 /lib/x86_64-linux-gnu/libc-2.15.so
beam.smp 31975 fdmanana mem REG 8,1 31752 917803 /lib/x86_64-linux-gnu/librt-2.15.so
beam.smp 31975 fdmanana mem REG 8,1 135366 917799 /lib/x86_64-linux-gnu/libpthread-2.15.so
beam.smp 31975 fdmanana mem REG 8,1 159200 921249 /lib/x86_64-linux-gnu/libtinfo.so.5.9
beam.smp 31975 fdmanana mem REG 8,1 1030512 917962 /lib/x86_64-linux-gnu/libm-2.15.so
beam.smp 31975 fdmanana mem REG 8,1 14768 917702 /lib/x86_64-linux-gnu/libdl-2.15.so
beam.smp 31975 fdmanana mem REG 8,1 149280 917974 /lib/x86_64-linux-gnu/ld-2.15.so
beam.smp 31975 fdmanana 0u CHR 136,1 4 /dev/pts/1
beam.smp 31975 fdmanana 1u CHR 136,1 4 /dev/pts/1
beam.smp 31975 fdmanana 2u CHR 136,1 4 /dev/pts/1
beam.smp 31975 fdmanana 3r FIFO 0,8 1298297 pipe
beam.smp 31975 fdmanana 4w FIFO 0,8 1298297 pipe
beam.smp 31975 fdmanana 5r FIFO 0,8 1298298 pipe
beam.smp 31975 fdmanana 6w FIFO 0,8 1298298 pipe
beam.smp 31975 fdmanana 7w REG 8,18 0 32564173 /home/fdmanana/git/hub/otp/tmp/20
beam.smp 31975 fdmanana 8w REG 8,18 0 32564176 /home/fdmanana/git/hub/otp/tmp/16
beam.smp 31975 fdmanana 9w REG 8,18 0 32564177 /home/fdmanana/git/hub/otp/tmp/15
beam.smp 31975 fdmanana 10w REG 8,18 0 32564179 /home/fdmanana/git/hub/otp/tmp/12
beam.smp 31975 fdmanana 11w REG 8,18 0 32564180 /home/fdmanana/git/hub/otp/tmp/11
beam.smp 31975 fdmanana 12w REG 8,18 0 32564205 /home/fdmanana/git/hub/otp/tmp/10
beam.smp 31975 fdmanana 13w REG 8,18 0 32564182 /home/fdmanana/git/hub/otp/tmp/8
beam.smp 31975 fdmanana 14w REG 8,18 0 32564183 /home/fdmanana/git/hub/otp/tmp/7
beam.smp 31975 fdmanana 15w REG 8,18 0 32564186 /home/fdmanana/git/hub/otp/tmp/3
|
|
Thanks to Tony Rogvall.
|
|
|
|
rickard/r16/port-optimizations/OTP-10336
* rickard/port-optimizations/OTP-10336:
Change annotate level for emacs-22 in cerl
Update etp-commands
Add documentation on communication in Erlang
Add support for busy port message queue
Add driver callback epilogue
Implement true asynchronous signaling between processes and ports
Add erl_drv_[send|output]_term
Move busy port flag
Use rwlock for driver list
Optimize management of port tasks
Improve configuration of process and port tables
Remove R9 compatibility features
Use ptab functionality also for ports
Prepare for use of ptab functionality also for ports
Atomic port state
Generalize process table implementation
Implement functionality for delaying thread progress from unmanaged threads
Conflicts:
erts/doc/src/erl_driver.xml
erts/doc/src/erlang.xml
erts/emulator/beam/beam_bif_load.c
erts/emulator/beam/beam_bp.c
erts/emulator/beam/beam_emu.c
erts/emulator/beam/bif.c
erts/emulator/beam/copy.c
erts/emulator/beam/erl_alloc.c
erts/emulator/beam/erl_alloc.types
erts/emulator/beam/erl_bif_info.c
erts/emulator/beam/erl_bif_port.c
erts/emulator/beam/erl_bif_trace.c
erts/emulator/beam/erl_init.c
erts/emulator/beam/erl_message.c
erts/emulator/beam/erl_port_task.c
erts/emulator/beam/erl_process.c
erts/emulator/beam/erl_process.h
erts/emulator/beam/erl_process_lock.c
erts/emulator/beam/erl_trace.c
erts/emulator/beam/export.h
erts/emulator/beam/global.h
erts/emulator/beam/io.c
erts/emulator/sys/unix/sys.c
erts/emulator/sys/vxworks/sys.c
erts/emulator/test/port_SUITE.erl
erts/etc/unix/cerl.src
erts/preloaded/ebin/erlang.beam
erts/preloaded/ebin/prim_inet.beam
erts/preloaded/src/prim_inet.erl
lib/hipe/cerl/erl_bif_types.erl
lib/kernel/doc/src/inet.xml
lib/kernel/src/inet.erl
|
|
|
