aboutsummaryrefslogtreecommitdiffstats
path: root/erts/preloaded/src/prim_file.erl
AgeCommit message (Collapse)Author
2019-02-22[logger] Change timestamp from erlang:system_time to os:system_timeSiri Hansen
This is to align the timestamps with external logs.
2019-01-07Spawn prim_file helper as a system processJohn Högberg
2018-11-13Avoid closing files in gc/monitor callbacksJohn Högberg
Closing files in these callbacks could block scheduler progress and cause major system instability. We now defer these operations to a dedicated process instead. This process may in turn block forever and prevent further orphaned files from being closed, but it will keep the emulator itself from misbehaving.
2018-09-13Fix unsafe use of lists:reverse/1John Högberg
We said reverse/2 but used reverse/1 which is unsafe to use in preloaded modules. This didn't have any effect in practice as the affected functions weren't used before the code server was started, but it's still an error.
2018-07-18Fix node crash on passing bad arguments to file:read_file_info/2John Högberg
2018-06-18Update copyright yearHenrik Nord
2018-05-23Use system_time instead of monotonic_time as timestamp in loggerSiri Hansen
2018-04-26Remove error_logger process and add logger processSiri Hansen
2018-03-19Fix file:change_group/change_ownerJohn Högberg
It wasn't possible to change group/owner separately, and our test suite lacked coverage for that. ERL-589
2017-11-30Reimplement efile_drv as a dirty NIFJohn Högberg
This improves the latency of file operations as dirty schedulers are a bit more eager to run jobs than async threads, and use a single global queue rather than per-thread queues, eliminating the risk of a job stalling behind a long-running job on the same thread while other async threads sit idle. There's no such thing as a free lunch though; the lowered latency comes at the cost of increased busy-waiting which may have an adverse effect on some applications. This behavior can be tweaked with the +sbwt flag, but unfortunately it affects all types of schedulers and not just dirty ones. We plan to add type-specific flags at a later stage. sendfile has been moved to inet_drv to lessen the effect of a nasty race; the cooperation between inet_drv and efile has never been airtight and the socket dying at the wrong time (Regardless of reason) could result in fd aliasing. Moving it to the inet driver makes it impossible to trigger this by closing the socket in the middle of a sendfile operation, while still allowing it to be aborted -- something that can't be done if it stays in the file driver. The race still occurs if the controlling process dies in the short window between dispatching the sendfile operation and the dup(2) call in the driver, but it's much less likely to happen now. A proper fix is in the works. -- Notable functional differences: * The use_threads option for file:sendfile/5 no longer has any effect. * The file-specific DTrace probes have been removed. The same effect can be achieved with normal tracing together with the nif__entry/nif__return probes to track scheduling. -- OTP-14256
2016-02-22Fix a few dialyzer warningsHans Bolinder
2015-06-18Change license text to APLv2Bruce Yinhe
2014-01-27erts/kernel: sendfile no longer uses async threadsLukas Larsson
This has been done because a slow client attack is possible if the async thread pool is used. The scenario is: Client does a request for a file and then slowly receives the file one byte at a time. This will eventually fill the async thread pool with blocking sendfile operations and thus starving the vm of all file operations. If you still want to use the async threads pool for sendfile an option to enable it has been introduced.
2013-11-15Add sync option to file:open/2Joseph Blomstedt
The sync option adds the POSIX O_SYNC flag to the open system call on platforms that support the flag or its equivalent, e.g., FILE_FLAG_WRITE_THROUGH on Windows. For platforms that don't support it, file:open/2 returns {error, enotsup} if the sync option is passed in. The semantics of O_SYNC are platform-specific. For example, not all platforms guarantee that all file metadata are written to the disk along with the file data when the flag is in effect. This issue is noted in the documentation this commit adds for the sync option. Add a test for the sync option. Note however that the underlying OS semantics for O_SYNC can't be tested automatically in any practical way, so the test assumes the OS does the right thing with the flag when present. For manual verification, dtruss on OS X and strace on Linux were both run against beam processes to watch calls to open(), and file:open/2 was called in Erlang shells to open files for writing, both with and without the sync option. Both the dtruss output and the strace output showed that the O_SYNC flag was present in the open() calls when sync was specified and was clear when sync was not specified.
2013-02-20prim_file: Fix error return from prim_file:list_dir/1Björn Gustavsson
When the run-time system was started with +fnue, the error tuple indicating a non-translatable filename was added as a non-proper list tail inside an {ok,Files} term.
2013-02-18Fix broken prim_file:list_dir_all/1Björn Gustavsson
6d516de001dde82c02fe050db8e3aab47914fa90 added prim_file:list_dir_all/1. Unfortunately, only the first element in the list would be handled as intended.
2013-02-11prim_file: Add list_dir_all() and read_link_all()Björn Gustavsson
2013-02-11Teach prim_file:set_cwd() to avoid entering non-translatable directoriesBjörn Gustavsson
We have decided that we don't want to deal with the compilations of prim_file:get_cwd() returning a binary when the current directory name cannot be translated losslessly to a list (i.e. when the run-time system was started with +fnu and the current directory name contains bytes that are not part of a valid UTF-8 sequence). Therefore, if prim_file:set_cwd() is given a binary as the pathname, we will need to check the binary to make sure it can be translated to a list. We will introduce a new BIF, called prim_file:is_translatable/1, which will check both filename encoding mode, and if it is one of Unicode modes, the binary as well. We don't need to do anything special if prim_file:set_cwd() is passed a list.
2013-02-11Make prim_file skip invalid filenames in unicode modePatrik Nyblom
The fix affects list_dir and read_link. Raw filenames are now never produced, just consumed even if +fnu or +fna is used on Linux etc. This also adds the options to get error return or error handler warning messages with +fn{u|a}{i|w|e} as an option to erl. This is still not documented and there needs to be other versions of read_dir and read_link to facilitate reading of all types of filenames and links. A check that we will not change to an invalid directory is also needed.
2013-02-11prim_file: Refactor functions that return filenamesBjörn Gustavsson
2013-02-11prim_file: Refactor handling of responsesBjörn Gustavsson
Currently, the format of the return value from drv_command/3 is determined solely by the efile driver's response. In a future commit, we will need to produce different return values that also dependend on which function in prim_file that was called; thus, we will need some way to pass down some sort of state to drv_get_response/2. As a preparation for that, allow the third argument of drv_command/3 to be a fun. That also allows us to remove the convoluted special case handling of the list_dir operation.
2013-02-11prim_file: Always open non-file ports in binary modeBjörn Gustavsson
Ports for operations that did not directly operate on a file (such as listing the files in a directory) was always opened in a binary mode, but there was still code that supported such port opened in non-binary mode. Since we are about to update the code reading directories, and we don't want to bother we supporting non-binary ports, make sure that we force the use of binary mode.
2013-01-25Extend char() to Unicode charactersHans Bolinder
The code related to the introduction of unicode_string() and unicode_char() has been removed. The types char() and string() have been extended to include Unicode characters. In fact char() was changed some time ago; this commit is about cleaning up the documentation and introduce better names for some functions.
2013-01-09Add file:allocate/3 operationFilipe David Manana
This operation allows pre-allocation of space for files. It succeeds only on systems that support such operation. The POSIX standard defines the optional system call posix_fallocate() to implement this feature. However, some systems implement more specific functions to accomplish the same operation. On Linux, if the more specific function fallocate() is implemented, it is used instead of posix_fallocate(), falling back to posix_fallocate() if the fallocate() call failed (it's only supported for the ext4, ocfs2, xfs and btrfs file systems at the moment). On Mac OS X it uses the specific fcntl() operation F_PREALLOCATE, falling back to posix_fallocate() if it's available (at the moment Mac OS X doesn't provide posix_fallocate()). On any other UNIX system, it uses posix_fallocate() if it's available. Any other system not providing this system call or any function to pre-allocate space for files, this operation always fails with the ENOTSUP POSIX error.
2012-08-29Merge branch 'maint'Henrik Nord
2012-07-19erts: Remove VxWorks from preloadedBjörn-Egil Dahlberg
2012-07-05Fix various code typos for R15B02Tuncer Ayaz
2012-03-30Update copyright yearsBjörn-Egil Dahlberg
2012-03-22Merge branch 'maint'Patrik Nyblom
Conflicts: erts/emulator/beam/beam_emu.c erts/emulator/beam/bif.tab erts/preloaded/ebin/prim_file.beam lib/hipe/cerl/erl_bif_types.erl
2012-03-22Remove code causing dialyzer warning from prim_filePatrik Nyblom
2012-03-22Rename dyntrace BIFs to more suiting namesPatrik Nyblom
2012-03-22Add user tag spreading functionality to VM and use in filePatrik Nyblom
User tags in a dynamic trace enabled VM are spread throughout the system in the same way as seq_trace tokens. This is used by the file module and various other modules to get hold of the tag from the user process without changing the protocol.
2012-03-22Update dtrace for changes in R15Björn-Egil Dahlberg
2012-03-22Add DTrace support for OS X, Solaris, and Linux (via SystemTap), 4/4Scott Lystig Fritchie
Add probes to (mostly) the efile_drv.c driver and other file I/O-related source files.
2012-01-27Move types and specs from erl_bif_types.erl to modulesHans Bolinder
2011-12-08Change name of bif universaltime_to_seconds/1Björn-Egil Dahlberg
* erlang:universaltime_to_seconds/1 changed to erlang:universaltime_to_posixtime/1 * erlang:seconds_to_universaltime/1 changed to erlang:posixtime_to_universaltime/1 Let prim_file.erl reflect these changes.
2011-12-08Change options to prim_file:*_file_info/*Björn-Egil Dahlberg
* 'utc' changed to 'universal' * 'epoch' changed to 'posix' This change conforms to other naming already in OTP, e.g. erlang:universaltime_to_localtime/1.
2011-12-08Catch errors from prim_file:*_file_infoBjörn-Egil Dahlberg
We do not want to crash the file server.
2011-12-08Let prim_file validate ctime in file_infoBjörn-Egil Dahlberg
2011-12-02Silence warnings in prim_fileBjörn-Egil Dahlberg
2011-12-02Fix validate structure in file_infoBjörn-Egil Dahlberg
2011-12-02Fix prim_file time conversionsBjörn-Egil Dahlberg
2011-12-02Move datetime into prim_fileBjörn-Egil Dahlberg
2011-12-02Use epoch seconds instead of datetime()Björn-Egil Dahlberg
First stage in utc-time for prim_file.
2011-12-02Remove header/trailer supportLukas Larsson
Since the API for headers/trailers seem to be very awkward to work with when using non-blocking io the feature is dropped for now. See unix_efile.c for more details.
2011-12-02Only allow tcp sockets as target for sendfileLukas Larsson
2011-12-01Preliminary work on header/trailerLukas Larsson
Have to figure out how to represent progress in header writing when using non-blocking, not sure how to do this.
2011-12-01Implement blocking calls for sendfileLukas Larsson
Move sendfile data to invoke data instead of file_descr. Remove usage of ready_output when doing a send. If told to send 0 bytes, file_sendfile now sends the entire file for linux.
2011-12-01Implement sendfile using blocking io in asynch threadsLukas Larsson
Move the command handling to outputv in preparation for header and trailer inclusion in the sendfile api. Use the standard efile communication functions for sendfile.
2011-12-01Create erlang fallback for sendfileLukas Larsson
Created erlang fallback for sendfile in gen_tcp and moved sendfile from file to gen_tcp. Also created testcases for testing all different options to sendfile. For info about how sendfile should work see the BSD man pages as they contain a more complete API than other *nixes.