| Age | Commit message (Collapse) | Author |
|---|
|
Kostis Sagonas pointed out that there is a dialyzer warning
for constructing an improper list in the following clause:
translate_response(?FILE_RESP_N2DATA = X,
[<<_:64, _:64, _:64>> | <<>>] = Data) ->
{error, {bad_response_from_port, [X | Data]}};
I don't want to change the code to somehow eliminate the warning. An
improper list has already been constructed in the efile driver itself,
and that would be difficult to fix.
Therefore, tell dialyzer to ignore warnings for improper lists
in translate_response/2.
|
|
|
|
This has been done because a slow client attack is possible if the
async thread pool is used. The scenario is:
Client does a request for a file and then slowly receives the file one
byte at a time. This will eventually fill the async thread pool with blocking
sendfile operations and thus starving the vm of all file operations.
If you still want to use the async threads pool for sendfile an option to
enable it has been introduced.
|
|
The sync option adds the POSIX O_SYNC flag to the open system call on
platforms that support the flag or its equivalent, e.g.,
FILE_FLAG_WRITE_THROUGH on Windows. For platforms that don't support it,
file:open/2 returns {error, enotsup} if the sync option is passed in.
The semantics of O_SYNC are platform-specific. For example, not all
platforms guarantee that all file metadata are written to the disk along
with the file data when the flag is in effect. This issue is noted in the
documentation this commit adds for the sync option.
Add a test for the sync option. Note however that the underlying OS
semantics for O_SYNC can't be tested automatically in any practical way, so
the test assumes the OS does the right thing with the flag when
present. For manual verification, dtruss on OS X and strace on Linux were
both run against beam processes to watch calls to open(), and file:open/2
was called in Erlang shells to open files for writing, both with and
without the sync option. Both the dtruss output and the strace output
showed that the O_SYNC flag was present in the open() calls when sync was
specified and was clear when sync was not specified.
|
|
When the run-time system was started with +fnue, the error tuple
indicating a non-translatable filename was added as a non-proper
list tail inside an {ok,Files} term.
|
|
6d516de001dde82c02fe050db8e3aab47914fa90 added prim_file:list_dir_all/1.
Unfortunately, only the first element in the list would be handled
as intended.
|
|
|
|
We have decided that we don't want to deal with the compilations
of prim_file:get_cwd() returning a binary when the current
directory name cannot be translated losslessly to a list (i.e.
when the run-time system was started with +fnu and the current directory
name contains bytes that are not part of a valid UTF-8 sequence).
Therefore, if prim_file:set_cwd() is given a binary as the pathname,
we will need to check the binary to make sure it can be translated
to a list. We will introduce a new BIF, called prim_file:is_translatable/1,
which will check both filename encoding mode, and if it is one of
Unicode modes, the binary as well.
We don't need to do anything special if prim_file:set_cwd() is passed
a list.
|
|
The fix affects list_dir and read_link. Raw filenames are now
never produced, just consumed even if +fnu or +fna is used on
Linux etc.
This also adds the options to get error return or error handler
warning messages with +fn{u|a}{i|w|e} as an option to erl.
This is still not documented and there needs to be other versions
of read_dir and read_link to facilitate reading of all types
of filenames and links.
A check that we will not change to an invalid directory is also needed.
|
|
|
|
Currently, the format of the return value from drv_command/3 is
determined solely by the efile driver's response. In a future
commit, we will need to produce different return values that
also dependend on which function in prim_file that was called;
thus, we will need some way to pass down some sort of state
to drv_get_response/2.
As a preparation for that, allow the third argument of drv_command/3
to be a fun. That also allows us to remove the convoluted special
case handling of the list_dir operation.
|
|
Ports for operations that did not directly operate on a file (such
as listing the files in a directory) was always opened in a binary
mode, but there was still code that supported such port opened in
non-binary mode.
Since we are about to update the code reading directories, and we
don't want to bother we supporting non-binary ports, make sure that
we force the use of binary mode.
|
|
The code related to the introduction of unicode_string() and
unicode_char() has been removed. The types char() and string() have
been extended to include Unicode characters.
In fact char() was changed some time ago; this commit is about
cleaning up the documentation and introduce better names for some
functions.
|
|
This operation allows pre-allocation of space for files.
It succeeds only on systems that support such operation.
The POSIX standard defines the optional system call
posix_fallocate() to implement this feature. However,
some systems implement more specific functions to
accomplish the same operation.
On Linux, if the more specific function fallocate() is
implemented, it is used instead of posix_fallocate(),
falling back to posix_fallocate() if the fallocate()
call failed (it's only supported for the ext4, ocfs2,
xfs and btrfs file systems at the moment).
On Mac OS X it uses the specific fcntl() operation
F_PREALLOCATE, falling back to posix_fallocate() if
it's available (at the moment Mac OS X doesn't provide
posix_fallocate()).
On any other UNIX system, it uses posix_fallocate() if it's
available. Any other system not providing this system call
or any function to pre-allocate space for files, this operation
always fails with the ENOTSUP POSIX error.
|
|
|
|
|
|
|
|
|
|
Conflicts:
erts/emulator/beam/beam_emu.c
erts/emulator/beam/bif.tab
erts/preloaded/ebin/prim_file.beam
lib/hipe/cerl/erl_bif_types.erl
|
|
|
|
|
|
User tags in a dynamic trace enabled VM are spread throughout the system
in the same way as seq_trace tokens. This is used by the file module
and various other modules to get hold of the tag from the user process
without changing the protocol.
|
|
|
|
Add probes to (mostly) the efile_drv.c driver and other
file I/O-related source files.
|
|
|
|
* erlang:universaltime_to_seconds/1 changed to
erlang:universaltime_to_posixtime/1
* erlang:seconds_to_universaltime/1 changed to
erlang:posixtime_to_universaltime/1
Let prim_file.erl reflect these changes.
|
|
* 'utc' changed to 'universal'
* 'epoch' changed to 'posix'
This change conforms to other naming already in OTP, e.g.
erlang:universaltime_to_localtime/1.
|
|
We do not want to crash the file server.
|
|
|
|
|
|
|
|
|
|
|
|
First stage in utc-time for prim_file.
|
|
Since the API for headers/trailers seem to be very awkward to
work with when using non-blocking io the feature is dropped
for now. See unix_efile.c for more details.
|
|
|
|
Have to figure out how to represent progress in header writing when
using non-blocking, not sure how to do this.
|
|
Move sendfile data to invoke data instead of file_descr.
Remove usage of ready_output when doing a send.
If told to send 0 bytes, file_sendfile now sends the entire file
for linux.
|
|
Move the command handling to outputv in preparation for
header and trailer inclusion in the sendfile api.
Use the standard efile communication functions for sendfile.
|
|
Created erlang fallback for sendfile in gen_tcp and
moved sendfile from file to gen_tcp. Also created testcases
for testing all different options to sendfile.
For info about how sendfile should work see the BSD man pages
as they contain a more complete API than other *nixes.
|
|
Allow Erlang code to use sendfile() where available by wrapping it as
file:sendfile/4 and file:sendfile/2.
sendfile(2) - Linux man page:
"sendfile() copies data between one file descriptor and another.
Because this copying is done within the kernel, sendfile() is more
efficient than the combination of read(2) and write(2), which would
require transferring data to and from user space."
|
|
Conflicts:
erts/aclocal.m4
erts/include/internal/ethread_header_config.h.in
|
|
|
|
|
|
The efile driver will now use chunked data on list_dir. This will
lessen the number of sends to prim_file and hence improve
performance. This method is utilized in both direct and async cases.
|
|
An experimental version of Dialyzer discovered that the atom that
replaced the DRV macro in prim_file ends up in calls to
erlang:open_port({spawn, Driver}, Portopts) as the Driver argument. The
documentation states that this call requires a string there.
This change is also consistent with the one introduced in commit
0f03b1e9d2bef3bc830c31a369261af4c5234727 by Kostis Sagonas.
|
|
|
|
|
|
|
|
|
