| Age | Commit message (Collapse) | Author |
|---|
|
* nox/read_file_info-raw/OTP-12325:
kernel: Do not check unsync:ed file size
Introduce new option 'raw' in file_info functions
|
|
This option allows the caller not to go through the file server for information
about files guaranteed to be local.
|
|
file:file_info/1 was removed a long time ago, but its
documentation was not removed.
|
|
* josevalim/set_cwd-typespec:
Correct file:set_cwd/1 typespec
OTP-11787
|
|
file:set_cwd/1 accepts binaries as arguments, however the binaries
must be properly encoded as per file:native_name_encoding/0.
Also update the note under no_translation error to refer that passing
a ISO-latin-1 encoded binary under any unicode file name encoding.
|
|
This has been done because a slow client attack is possible if the
async thread pool is used. The scenario is:
Client does a request for a file and then slowly receives the file one
byte at a time. This will eventually fill the async thread pool with blocking
sendfile operations and thus starving the vm of all file operations.
If you still want to use the async threads pool for sendfile an option to
enable it has been introduced.
|
|
* sv/file-osync/OTP-11498:
Add sync option to file:open/2
Conflicts:
erts/preloaded/ebin/prim_file.beam
|
|
The sync option adds the POSIX O_SYNC flag to the open system call on
platforms that support the flag or its equivalent, e.g.,
FILE_FLAG_WRITE_THROUGH on Windows. For platforms that don't support it,
file:open/2 returns {error, enotsup} if the sync option is passed in.
The semantics of O_SYNC are platform-specific. For example, not all
platforms guarantee that all file metadata are written to the disk along
with the file data when the flag is in effect. This issue is noted in the
documentation this commit adds for the sync option.
Add a test for the sync option. Note however that the underlying OS
semantics for O_SYNC can't be tested automatically in any practical way, so
the test assumes the OS does the right thing with the flag when
present. For manual verification, dtruss on OS X and strace on Linux were
both run against beam processes to watch calls to open(), and file:open/2
was called in Erlang shells to open files for writing, both with and
without the sync option. Both the dtruss output and the strace output
showed that the O_SYNC flag was present in the open() calls when sync was
specified and was clear when sync was not specified.
|
|
|
|
The file module communicates with a file io server with the following
protocol for file operations:
> {file_request,From,ReplyAs,Request}
< {file_reply,ReplyAs,Reply}
The ReplyAs value is sent by the client side to match against when
receiving the reply and is otherwise left untouched and passed as is by
the server.
This commit enables receive optimizations by using the reference of the
server monitor, changing the protocol to:
> {file_request,From,MonitorRef,Request}
< {file_reply,MonitorRef,Reply}
As the shape of the messages is not changed, backwards compatibility is
not a concern.
|
|
|
|
The recommended type for filenames is a list of characters (which
may be Unicode characters greater than 255). Change the
file:filename() to reflect that.
For the filename module we still need a type that can be either
a string or a binary, so we need to introduce the type
file:filename_all().
|
|
* pan/unicode_filename_warnings:
Add file:list_dir_all/1 and file:read_link_all/1
prim_file: Add list_dir_all() and read_link_all()
Teach prim_file:set_cwd() to avoid entering non-translatable directories
Make prim_file skip invalid filenames in unicode mode
prim_file: Refactor functions that return filenames
prim_file: Refactor handling of responses
prim_file: Always open non-file ports in binary mode
Test that list_dir("non-existing-dir") fails with the correct error
|
|
|
|
|
|
|
|
This operation allows pre-allocation of space for files.
It succeeds only on systems that support such operation.
The POSIX standard defines the optional system call
posix_fallocate() to implement this feature. However,
some systems implement more specific functions to
accomplish the same operation.
On Linux, if the more specific function fallocate() is
implemented, it is used instead of posix_fallocate(),
falling back to posix_fallocate() if the fallocate()
call failed (it's only supported for the ext4, ocfs2,
xfs and btrfs file systems at the moment).
On Mac OS X it uses the specific fcntl() operation
F_PREALLOCATE, falling back to posix_fallocate() if
it's available (at the moment Mac OS X doesn't provide
posix_fallocate()).
On any other UNIX system, it uses posix_fallocate() if it's
available. Any other system not providing this system call
or any function to pre-allocate space for files, this operation
always fails with the ENOTSUP POSIX error.
|
|
Expect modifications, additions and corrections.
There is a kludge in file_io_server and
erl_scan:continuation_location() that's not so pleasing.
|
|
The mode() type is used in path_open and copy, two functions that
do _not_ accept the ram option (or is at least not supposed to),
why I moved the option 'ram' from the type down to the spec for
open itself. That also makes the option visible directly under open/2
in the documentation.
|
|
|
|
* maint:
Fix chunk usage check
|
|
|
|
|
|
Conflicts:
erts/emulator/beam/beam_emu.c
erts/emulator/beam/bif.tab
erts/preloaded/ebin/prim_file.beam
lib/hipe/cerl/erl_bif_types.erl
|
|
|
|
|
|
User tags in a dynamic trace enabled VM are spread throughout the system
in the same way as seq_trace tokens. This is used by the file module
and various other modules to get hold of the tag from the user process
without changing the protocol.
|
|
|
|
Add probes to (mostly) the efile_drv.c driver and other
file I/O-related source files.
|
|
|
|
* ta/sendfile/OTP-9240:
Do not use async threads on DARWIN
Fix cleanup when sendfile process crashes
Return {error,closed} from sendfile if closed
Do not use SFV_NOWAIT as it does not exist on all solaris
Clarify some code comments
Make solaris use sendfilev
|
|
* egil/file-info-opt-utc/OTP-7687: (39 commits)
Remove time_t specific test in prim_file_SUITE
Update prim_file.beam and prim_zip.beam
Add types for posixtime_to_universaltime and the reverse
Set BASEYEAR to 1902
Set lower limit of years handled to 1601
Emulate localtime, gmtime and mktime to enable negative time_t
Document file:*_file_info/2
Fix compiler warning in unix_efile.c
Change name of bif universaltime_to_seconds/1
Change options to prim_file:*_file_info/*
Remove dead code
Catch errors from prim_file:*_file_info
Testcase for utc <-> seconds conversion
Fix negative time in seconds_to_universaltime/1
Remove OS taint from datetime conversion
Add utc <-> seconds conversions bifs
Let prim_file validate ctime in file_info
Teach #file_info spec unix epochs for file times
Add file_info_opt tests in prim_file_SUITE
unix_efile: Zero is a valid number in utime
...
Conflicts:
erts/emulator/beam/erl_time_sup.c
erts/emulator/sys/win32/erl_win_sys.h
erts/emulator/sys/win32/sys_time.c
|
|
* file:read_file_info/2
* file:write_file_info/3
Document options and time behavior.
|
|
|
|
If the socket is closed by the remote end, sendfile
now returns {error,closed}.
|
|
* ta/docs:
Fix misspelling of chosen
Fix typos in erl(1)
Fix misspelling of 'turn off'
Fix typo in reltool(3)
Fix typo in gen_sctp(3)
Fix typo in mod_responsecontrol.erl
Fix typo in erts_alloc(3)
Fix typos in ssl_cipher.erl
Fix typos in expressions.xml
file: correct make_link/2 and make_symlink/2 docs
Fix typo in test_server(3)
Fix typos in io_protocol.xml
Fix typo in erlang(3)
Fix typos in tar_SUITE
Fix typos in erts/preloaded/src
OTP-9787
|
|
* ao/fix-change-time-inval-dates:
Fix a crash when file:change_time/2,3 are called with invalid dates
OTP-9785
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Since the API for headers/trailers seem to be very awkward to
work with when using non-blocking io the feature is dropped
for now. See unix_efile.c for more details.
|
|
|
|
Since sendfile could in theory be used to send to any type
of file descriptor in *nix, it is a better fit to have it
in file.
|
|
Created erlang fallback for sendfile in gen_tcp and
moved sendfile from file to gen_tcp. Also created testcases
for testing all different options to sendfile.
For info about how sendfile should work see the BSD man pages
as they contain a more complete API than other *nixes.
|
|
Allow Erlang code to use sendfile() where available by wrapping it as
file:sendfile/4 and file:sendfile/2.
sendfile(2) - Linux man page:
"sendfile() copies data between one file descriptor and another.
Because this copying is done within the kernel, sendfile() is more
efficient than the combination of read(2) and write(2), which would
require transferring data to and from user space."
|
|
Calling file:change_time/2,3 with an invalid date tuple
(e.g file:change_time("file.txt", {undefined, undefined}))
will cause file_server_2 to crash.
error_logger will shutdown and the whole VM will stop.
Change behavior to validate given dates on system boundaries.
(i.e before issuing a server call).
|
|
In the location information tables in the run-time system, source
filenames that are the same as the module name plus ".erl" extension
are not stored explicitly, thus saving memory.
To take advantage of that optimization, avoid complicating the names of
files in the current working directory; specifically, make sure that
"./" is not prepended to the name.
|
