aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/drivers/common/efile_drv.c
AgeCommit message (Collapse)Author
2014-02-24ose: OSE port related cleanup and fixesJonas Karlsson
Some OSE cross-chains have problems with system includes being used, so for atleast OSE specific parts we use "" instead of <>.
2014-02-24ose: efile driver updates.Jonas Karlsson
2014-02-24Added support for ENEA OSELukas Larsson
This port has support for both non-smp and smp. It contains a new way to do io checking in which erts_poll_wait receives the payload of the polled entity. This has implications for all linked-in drivers.
2014-01-28Merge branch 'lukas/erts/sendfile_no_async_pool/OTP-11639'Lukas Larsson
* lukas/erts/sendfile_no_async_pool/OTP-11639: erts/kernel: sendfile no longer uses async threads
2014-01-27erts/kernel: sendfile no longer uses async threadsLukas Larsson
This has been done because a slow client attack is possible if the async thread pool is used. The scenario is: Client does a request for a file and then slowly receives the file one byte at a time. This will eventually fill the async thread pool with blocking sendfile operations and thus starving the vm of all file operations. If you still want to use the async threads pool for sendfile an option to enable it has been introduced.
2014-01-14Don't make gzio.c dependent on the zutil.h header fileBjörn Gustavsson
gzio.c is our own replacement for zlib's gzopen() etc (based on a version of gzio.c that was included in an old version of zlib). Unfortunately, gzio.c still depends on the *internal* zlib header file zutil.h which is not supposed to be used outside of the zlib source code. The dependencies are the use of the gzFile typedef and the F_OPEN() macro. Instead of gzFile, define and use our own ErtsGzFile. To get rid of the F_OPEN() macro, call open() of _wfopen() directly.
2013-09-02Fix EV_* macros and functions signedness flawsRaimo Niskanen
2013-08-23Create better distribution of files over async threadsPatrik Nyblom
The actual port id is used to create a key from the pointer value which is the ErlDrvPort. To do this a new driver api function driver_async_port_key is added and the driver API minor version is updated. The documentation is updated and the faulty description of how to spread ports over async threads is updated to use the new API. Testcase also added.
2013-08-23Add debug functionality to retrieve async keyPatrik Nyblom
2013-08-20erts: Fix bug in translating ev macros to functionsLukas Larsson
2013-08-13erts: Refactor non-pure macros to functionsLukas Larsson
This is needed as some gcc versions seems to optimize this undefined behaviour in a way which breaks this code.
2013-05-31kernel: Fix bug in file:pwrite for large data setsSverker Eriksson
Bug introduced in a73414d2e8ad03538 and never released.
2013-03-27Merge branch 'lukas/erts/efile_delayed_write_fix/OTP-10984' into maintLukas Larsson
* lukas/erts/efile_delayed_write_fix/OTP-10984: Do driver_deq in worker threads instead of async_ready
2013-03-13Do driver_deq in worker threads instead of async_readyLukas Larsson
Doing it in async_ready was needed before the pdl was introduced, but now with the pdl the deq no longer needs the port lock to protect it. This was not an issue when async_ready was called in the worker thread, but now (R15B) that it is signalled back to the scheduler, some very nasty race conditions could occur when using driver_timer and async jobs.
2013-03-04erts: Fix void * arithmeticBjörn-Egil Dahlberg
2013-03-04erts: Use block comments - ansi styleBjörn-Egil Dahlberg
2013-02-22erts: Use correct type for ReadFile bytes readBjörn-Egil Dahlberg
Using a 64bit type for bytes read will not always clear the higher bits.
2013-02-15invoke_read_line now remembers read_ahead settingsLukas Larsson
2013-02-11erts: Fix memory leak in efile_drv.cSverker Eriksson
Seems to happen with async threads and when user closes the file explicitly before the port is closed.
2013-01-25Update copyright yearsBjörn-Egil Dahlberg
2013-01-23Alternative solution to the efile_drv crash on exitPatrik Nyblom
2013-01-11Merge branch 'fdm/file-allocate/OTP-10680'Björn-Egil Dahlberg
* fdm/file-allocate/OTP-10680: Update preloaded prim_file.beam erts: Fix xcomp configure for fallocate Add file:allocate/3 operation
2013-01-09Add file:allocate/3 operationFilipe David Manana
This operation allows pre-allocation of space for files. It succeeds only on systems that support such operation. The POSIX standard defines the optional system call posix_fallocate() to implement this feature. However, some systems implement more specific functions to accomplish the same operation. On Linux, if the more specific function fallocate() is implemented, it is used instead of posix_fallocate(), falling back to posix_fallocate() if the fallocate() call failed (it's only supported for the ext4, ocfs2, xfs and btrfs file systems at the moment). On Mac OS X it uses the specific fcntl() operation F_PREALLOCATE, falling back to posix_fallocate() if it's available (at the moment Mac OS X doesn't provide posix_fallocate()). On any other UNIX system, it uses posix_fallocate() if it's available. Any other system not providing this system call or any function to pre-allocate space for files, this operation always fails with the ENOTSUP POSIX error.
2013-01-09Fix fd leak when using async thread poolFilipe David Borba Manana
When using the async thread pool, if an erlang process asks to open a file and it gets shutdown/killed while the file:open/2 call hasn't returned, it's possible to leak a file descriptor against the target file. This happens because when the file driver is stopped (file_stop() function is called), an async thread is executing, about to execute, or executed already the invoke_open() function. After file_stop() is called, the file_async_ready() function will not run, and this function is responsible for setting desc->fd with the file descriptor that invoke_open() got. The file_stop() call closes desc->fd if it refers to a valid file descriptor, which is not the case here, because this function was called before file_async_ready() could run. This leak is easily reproducile in a GNU/Linux system using the following test code: -module(t). -export([t/1]). t(N) -> Pid = spawn_link(fun() -> process_flag(trap_exit, true), loop(N) end), Ref = erlang:monitor(process, Pid), receive {'DOWN', Ref, _, _, _} -> ok end. loop(0) -> ok; loop(N) -> Name = integer_to_list(N), Server = self(), Pid = spawn(fun() -> Server ! continue, {ok, FdW} = file:open(Name, [raw, write]), {ok, FdR} = file:open(Name, [raw, read]), % Optional close calls, with or without them % it makes no difference. %ok = file:close(FdW), %ok = file:close(FdR), ok end), receive continue -> ok end, exit(Pid, shutdown), loop(N - 1). Running this code with a few iterations is enough to very often notice, with the lsof command, that the beam.smp process is holding forever file descriptors open. This issue doesn't happen if the async thread pool is not used. Example: $ erl +A 4 Erlang R15B03 (erts-5.9.3) [source] [64-bit] [smp:4:4] [async-threads:4] [hipe] [kernel-poll:false] Eshell V5.9.3 (abort with ^G) 1> c(t). {ok,t} 2> os:getpid(). "31975" 3> t:t(20). ok In a separate shell: $ lsof -p 31975 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME beam.smp 31975 fdmanana cwd DIR 8,18 22736896 32563204 /home/fdmanana/git/hub/otp/tmp beam.smp 31975 fdmanana rtd DIR 8,1 4096 2 / beam.smp 31975 fdmanana txt REG 8,1 7600263 1835126 /opt/r15b03/lib/erlang/erts-5.9.3/bin/beam.smp beam.smp 31975 fdmanana mem REG 8,1 7220736 2497283 /usr/lib/locale/locale-archive beam.smp 31975 fdmanana mem REG 8,1 10280 2505021 /usr/lib/libsctp.so.1.0.11 beam.smp 31975 fdmanana mem REG 8,1 1811128 917795 /lib/x86_64-linux-gnu/libc-2.15.so beam.smp 31975 fdmanana mem REG 8,1 31752 917803 /lib/x86_64-linux-gnu/librt-2.15.so beam.smp 31975 fdmanana mem REG 8,1 135366 917799 /lib/x86_64-linux-gnu/libpthread-2.15.so beam.smp 31975 fdmanana mem REG 8,1 159200 921249 /lib/x86_64-linux-gnu/libtinfo.so.5.9 beam.smp 31975 fdmanana mem REG 8,1 1030512 917962 /lib/x86_64-linux-gnu/libm-2.15.so beam.smp 31975 fdmanana mem REG 8,1 14768 917702 /lib/x86_64-linux-gnu/libdl-2.15.so beam.smp 31975 fdmanana mem REG 8,1 149280 917974 /lib/x86_64-linux-gnu/ld-2.15.so beam.smp 31975 fdmanana 0u CHR 136,1 4 /dev/pts/1 beam.smp 31975 fdmanana 1u CHR 136,1 4 /dev/pts/1 beam.smp 31975 fdmanana 2u CHR 136,1 4 /dev/pts/1 beam.smp 31975 fdmanana 3r FIFO 0,8 1298297 pipe beam.smp 31975 fdmanana 4w FIFO 0,8 1298297 pipe beam.smp 31975 fdmanana 5r FIFO 0,8 1298298 pipe beam.smp 31975 fdmanana 6w FIFO 0,8 1298298 pipe beam.smp 31975 fdmanana 7w REG 8,18 0 32564173 /home/fdmanana/git/hub/otp/tmp/20 beam.smp 31975 fdmanana 8w REG 8,18 0 32564176 /home/fdmanana/git/hub/otp/tmp/16 beam.smp 31975 fdmanana 9w REG 8,18 0 32564177 /home/fdmanana/git/hub/otp/tmp/15 beam.smp 31975 fdmanana 10w REG 8,18 0 32564179 /home/fdmanana/git/hub/otp/tmp/12 beam.smp 31975 fdmanana 11w REG 8,18 0 32564180 /home/fdmanana/git/hub/otp/tmp/11 beam.smp 31975 fdmanana 12w REG 8,18 0 32564205 /home/fdmanana/git/hub/otp/tmp/10 beam.smp 31975 fdmanana 13w REG 8,18 0 32564182 /home/fdmanana/git/hub/otp/tmp/8 beam.smp 31975 fdmanana 14w REG 8,18 0 32564183 /home/fdmanana/git/hub/otp/tmp/7 beam.smp 31975 fdmanana 15w REG 8,18 0 32564186 /home/fdmanana/git/hub/otp/tmp/3
2012-08-30Teach VM not to dump core on long pathnamesPatrik Nyblom
Long input paths (longer than MAX_PATH) would get copied into a buffer of size MAX_PATH for read_link and altname in efile_drv. Also fixed misuse of size_t parameter as wchar_t * string length in win_efile:efile_readlink.
2012-08-27erts: Fix dtrace-bug in file rename opSverker Eriksson
2012-05-09Correct formating in exit error messagesMichael Santos
Ensure displayed sizes are not negative.
2012-03-30Update copyright yearsBjörn-Egil Dahlberg
2012-03-22Correct some errors in the user tag spreadingPatrik Nyblom
2012-03-22Change to more specific configure options for dtracePatrik Nyblom
2012-03-22Add user tag spreading functionality to VM and use in filePatrik Nyblom
User tags in a dynamic trace enabled VM are spread throughout the system in the same way as seq_trace tokens. This is used by the file module and various other modules to get hold of the tag from the user process without changing the protocol.
2012-03-22Update dtrace for changes in R15Björn-Egil Dahlberg
2012-03-22Add DTrace support for OS X, Solaris, and Linux (via SystemTap), 4/4Scott Lystig Fritchie
Add probes to (mostly) the efile_drv.c driver and other file I/O-related source files.
2012-03-20Fix reselecting bug on OS XLukas Larsson
Since stop_select is called at an arbitrary point in the future it would sometime not be alled before the tcp driver started selecting on the fd. So now ERL_DRV_USE_NO_CALLBACK is used so that the stop_select call is never made. This seems to only have happened OS X.
2012-03-20Fix bug when sending long files using selectLukas Larsson
The return value from efile_sendfile was not consistent inbetween platforms. The API should now be working as it was intended. OTP-9994
2012-03-20Fix ifdef to check if we are on OS XLukas Larsson
2012-03-20Fix memory leak when sendfile process crashesLukas Larsson
We use the fact that file_flush is called when there is data in the driver queue when a port is closed to ensure that all data is cleaned up as it should. OTP-9993
2011-12-20erts: rewrite efile_writev to handle partial writes correctlyRaimo Niskanen
2011-12-09Merge branch 'raimo/64-bit-driver-api/OTP-9795'Björn Gustavsson
* raimo/64-bit-driver-api/OTP-9795: (22 commits) driver_SUITE.erl: Fix sys info drivers emulator test drivers: Conform to updated driver API runtime_tools's drivers: Conform to updated driver API ws's xwe_driver.c: Conform to updated driver API megaco's flex scanner: Conform to updated driver API seq_trace_SUITE_data/echo_drv.c: Conform to updated driver API erl_interface tests: Conform port_call_drv.c updated driver API erl_drv_thread_SUITE_data/testcase_driver.c: Conform to updated driver API float_SUITE_data/fp_drv.c: Conform to updated driver API port_SUITE_data/*_drv.c: Conform to updated driver API port_bif_SUITE_data/control_drv.c: Conform to updated driver API send_term_SUITE_data/send_term_drv.c: Conform to updated driver API system_profile_SUITE_data/echo_drv.c: Conform to updated driver API trace_port_SUITE_data/echo_drv.c: Conform to updated driver API Remove support for old drivers without ERL_DRV_EXTENDED_MARKER built-in drivers: Add ERL_DRV_EXTENDED_MARKER and version numbers Bump driver version to 2.0 erl_driver.h: Enlarge type on return value from call erl_driver.h: Enlarge types on driver callbacks output, control and call erl_driver.h: Enlarge types in driver output functions ... Conflicts: erts/emulator/test/driver_SUITE_data/monitor_drv.c erts/emulator/test/driver_SUITE_data/timer_drv.c
2011-12-09erl_driver.h: Enlarge types on driver callbacks output, control and callRaimo Niskanen
2011-12-08Merge branch 'ta/sendfile/OTP-9240'Lukas Larsson
* ta/sendfile/OTP-9240: Do not use async threads on DARWIN Fix cleanup when sendfile process crashes Return {error,closed} from sendfile if closed Do not use SFV_NOWAIT as it does not exist on all solaris Clarify some code comments Make solaris use sendfilev
2011-12-08Do not use async threads on DARWINLukas Larsson
2011-12-08Fix cleanup when sendfile process crashesLukas Larsson
When sendfile crashes during a send there should be no error_logger printouts about stolen fds.
2011-12-07Return {error,closed} from sendfile if closedLukas Larsson
If the socket is closed by the remote end, sendfile now returns {error,closed}.
2011-12-07erl_driver.h: Enlarge types in driver queue and alloc functionsRaimo Niskanen
2011-12-02efile_drv: Fix casting between Sint64 and time_tBjörn-Egil Dahlberg
2011-12-02Use epoch seconds instead of datetime()Björn-Egil Dahlberg
First stage in utc-time for prim_file.
2011-12-02Remove header/trailer supportLukas Larsson
Since the API for headers/trailers seem to be very awkward to work with when using non-blocking io the feature is dropped for now. See unix_efile.c for more details.
2011-12-01Preliminary work on header/trailerLukas Larsson
Have to figure out how to represent progress in header writing when using non-blocking, not sure how to do this.
2011-12-01Use free_sendfile explicitly for non-asyncLukas Larsson
This is needed because aync job will not call free_sendfile if there is an async_ready callback.
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-20 19:53:36 +0000