| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
The sync option adds the POSIX O_SYNC flag to the open system call on
platforms that support the flag or its equivalent, e.g.,
FILE_FLAG_WRITE_THROUGH on Windows. For platforms that don't support it,
file:open/2 returns {error, enotsup} if the sync option is passed in.
The semantics of O_SYNC are platform-specific. For example, not all
platforms guarantee that all file metadata are written to the disk along
with the file data when the flag is in effect. This issue is noted in the
documentation this commit adds for the sync option.
Add a test for the sync option. Note however that the underlying OS
semantics for O_SYNC can't be tested automatically in any practical way, so
the test assumes the OS does the right thing with the flag when
present. For manual verification, dtruss on OS X and strace on Linux were
both run against beam processes to watch calls to open(), and file:open/2
was called in Erlang shells to open files for writing, both with and
without the sync option. Both the dtruss output and the strace output
showed that the O_SYNC flag was present in the open() calls when sync was
specified and was clear when sync was not specified.
|
|
files as delimiters.
While working on a tool that processes Erlang code and testing it against this repo,
I found out about those little sneaky 0xff. I thought it may be of help to other
people build such tools to remove non-conforming-to-standard characters.
|
|
|
|
|
|
|
|
Using a 64bit type for bytes read will not always clear the higher bits.
|
|
|
|
Some file operations provided by the Erlang file module
didn't open the target file with all the file share flags.
This made some concurrent file operations against the
same file fail on Windows, while on other platforms such
as GNU/Linux or Mac OS X they succeed. The operations will
fail only if they're performed concurrently by different
threads (async IO threads or scheduler threads).
For example, one Erlang process does a file:delete/1 call
while another Erlang process is doing a filelib:file_size/1
call. This made the former process get an eacces error from
the file:delete/1 call. On GNU/Linux or Mac OS X the call would
succeed. Another example is if one Erlang process attempts to
open a file for reading while another one is in the middle of
a file:read_file_info/1 call (after it opened the file and
before it closed the file).
It's easy to verify that if a file is not open with all the
share flags, it's impossible for other threads (even if they
belong to the same OS process) to open the file while the
file is not closed by the first thread.
The following test program shows this:
#include <windows.h>
#include <iostream>
// Must be an existing file
//#define SHARE_FLAGS (FILE_SHARE_READ)
static DWORD WINAPI MyThreadFunction(LPVOID lpParam);
static char *lastError();
int main(int argc, char *argv[])
{
DWORD threadId;
HANDLE threadHandle, hFile;
hFile = CreateFile(FILENAME, GENERIC_READ, SHARE_FLAGS, NULL,
OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
if (hFile == INVALID_HANDLE_VALUE) {
std::cerr << "File open error from main: " <<
lastError() << std::endl;
return 1;
}
std::cout << "Main thread opened file successfully" << std::endl;
threadHandle = CreateThread(NULL, 0, MyThreadFunction, NULL, 0, &threadId);
if (threadHandle == INVALID_HANDLE_VALUE) {
std::cerr << "Thread create error from main: " <<
lastError() << std::endl;
return 1;
}
WaitForSingleObject(threadHandle, INFINITE);
CloseHandle(threadHandle);
CloseHandle(hFile);
return 0;
}
static DWORD WINAPI MyThreadFunction( LPVOID lpParam )
{
HANDLE hFile;
hFile = CreateFile(FILENAME, GENERIC_READ, SHARE_FLAGS, NULL,
OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
if (hFile == INVALID_HANDLE_VALUE) {
std::cerr << "File open error from second thread: " <<
lastError() << std::endl;
return 1;
}
std::cout << "Second thread opened file successfully" << std::endl;
CloseHandle(hFile);
return 0;
}
static char *lastError()
{
static char *buf = NULL;
DWORD dw = GetLastError();
if (buf != NULL) {
LocalFree((LPTSTR) &buf);
}
FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM |
FORMAT_MESSAGE_IGNORE_INSERTS,
NULL, dw, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
(LPTSTR) &buf, 0, NULL);
return buf;
}
Rnning this program with SHARE_FLAGS set to 0 (as efile_fileinfo()
does for e.g.), shows that the second thread is unable to open the
file:
C:\cygwin\home\fdmanana\tmp>touch foo.bar
C:\cygwin\home\fdmanana\tmp>threads_fopen_test.exe
Main thread opened file successfully
File open error from second thread: The process cannot access the
file because it is being used by another process.
Changing the program's SHARE_FLAGS to FILE_SHARE_READ, shows that
both threads are able to open the file:
C:\cygwin\home\fdmanana\tmp>touch foo.bar
C:\cygwin\home\fdmanana\tmp>threads_test.exe
Main thread opened file successfully
Second thread opened file successfully
Same logic applies to opening files for writing or deleting and
renaming files while they're open by some other thread that didn't
specify the flags FILE_SHARE_WRITE and FILE_SHARE_DELETE.
|
|
This operation allows pre-allocation of space for files.
It succeeds only on systems that support such operation.
The POSIX standard defines the optional system call
posix_fallocate() to implement this feature. However,
some systems implement more specific functions to
accomplish the same operation.
On Linux, if the more specific function fallocate() is
implemented, it is used instead of posix_fallocate(),
falling back to posix_fallocate() if the fallocate()
call failed (it's only supported for the ext4, ocfs2,
xfs and btrfs file systems at the moment).
On Mac OS X it uses the specific fcntl() operation
F_PREALLOCATE, falling back to posix_fallocate() if
it's available (at the moment Mac OS X doesn't provide
posix_fallocate()).
On any other UNIX system, it uses posix_fallocate() if it's
available. Any other system not providing this system call
or any function to pre-allocate space for files, this operation
always fails with the ENOTSUP POSIX error.
|
|
|
|
|
|
Long input paths (longer than MAX_PATH) would get copied
into a buffer of size MAX_PATH for read_link and altname
in efile_drv.
Also fixed misuse of size_t parameter as wchar_t *
string length in win_efile:efile_readlink.
|
|
Incorrect window was used to calculate x position.
|
|
|
|
* raimo/64-bit-driver-api/OTP-9795: (22 commits)
driver_SUITE.erl: Fix sys info drivers
emulator test drivers: Conform to updated driver API
runtime_tools's drivers: Conform to updated driver API
ws's xwe_driver.c: Conform to updated driver API
megaco's flex scanner: Conform to updated driver API
seq_trace_SUITE_data/echo_drv.c: Conform to updated driver API
erl_interface tests: Conform port_call_drv.c updated driver API
erl_drv_thread_SUITE_data/testcase_driver.c: Conform to updated driver API
float_SUITE_data/fp_drv.c: Conform to updated driver API
port_SUITE_data/*_drv.c: Conform to updated driver API
port_bif_SUITE_data/control_drv.c: Conform to updated driver API
send_term_SUITE_data/send_term_drv.c: Conform to updated driver API
system_profile_SUITE_data/echo_drv.c: Conform to updated driver API
trace_port_SUITE_data/echo_drv.c: Conform to updated driver API
Remove support for old drivers without ERL_DRV_EXTENDED_MARKER
built-in drivers: Add ERL_DRV_EXTENDED_MARKER and version numbers
Bump driver version to 2.0
erl_driver.h: Enlarge type on return value from call
erl_driver.h: Enlarge types on driver callbacks output, control and call
erl_driver.h: Enlarge types in driver output functions
...
Conflicts:
erts/emulator/test/driver_SUITE_data/monitor_drv.c
erts/emulator/test/driver_SUITE_data/timer_drv.c
|
|
|
|
|
|
|
|
* ctime were never defined for invalid file handles
* fix epoch <-> fileinfo conversions
|
|
|
|
Conflicts:
erts/emulator/drivers/win32/win_efile.c
|
|
Almost all uses of the 'long' datatype is removed from VM and tests
Emulator test now runs w/o drivers crashing
Nasty abs bug fixed in VM as well as type errors in allocator debug functions
Still one allocator test that fails, domain knowledge is needed to fix that.
Fix type inconsistency in beam_load causing crashes
|
|
Still does not run, just compiles.
|
|
|
|
|
|
|
|
When setting file_info it will now correctly set access and
modified time. Previously these entities were swapped.
|
|
|
|
|
|
Also close find-handles in altname and other minor corrections to patch
|
|
|
|
* jr/windows-file-append:
Fix appending to large files (>4GB) on Windows
OTP-8958
|
|
Hard linking requires XP or later and a NTFS formatted drive.
Symbolic linking requires Windows Vista, Windows Server 2008,
Windows 7 or later. Symbolic linking on earlier versions of
Windows will return {error, enotsup}. file:make_link/2,
file:make_sym_link/2 and file:read_link/1 are all implemented.
file:read_file_info/1 does NOT return the number of hard
links on the file as this results in performance problems
for read_file_info/1, however this can be enabled by uncommenting
a few lines of code.
The original path was submitted on 9/2/2009 by Alex Tearse-Doyle.
|
|
Append mode doesn't work for files larger that 4GB on Windows.
Caused by incorrect usage of SetFilePointer in win_efile.c.
Fix uses OVERLAPPED structure to specify write position (EOF).
http://msdn.microsoft.com/en-us/library/aa365747.aspx
Opening file in append mode was also considered, but rejected, because
it might cause backwards incompatibility by limiting applicable
operations on the descriptor.
SetFilePointerEx was not used because it would caused more system calls
than using OVERLAPPED structure.
|
|
The mem_drv driver was only useful when elib_malloc was
active.
|
|
Add an option that atomically tests for the existence of a file and
creates it if the file does not exist, by passing the O_EXCL flag
to open() on Unix and CREATE_NEW flag on Windows. Support for O_EXCL
varies across platforms and filesystems.
{ok, Fd} = file:open("/tmp/foo", [write,exclusive]),
{error, eexist} = file:open("/tmp/foo", [write,exclusive]).
|
|
See dwShareMode on
http://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx
|
|
* fm/file-operations:
Update preloaded modules
Add file:advise/4 - a wrapper to the POSIX syscall posix_fadvise
Add file:datasync/1 for syncing file contents only
sys.h: Correct the get_int64() macro
OTP-8637 fm/file-operations
The functions file:advise/4 and file:datasync/1 have been added. (Thanks to
Filipe David Manana.)
|
|
Useful for informing the Operating System about the access pattern
for a file's data, so that it can adapt the caching strategy to
maximize disk IO performance.
|
|
file:datasync/1 invokes the POSIX system call "int fdatasync(int fd)".
This system call is similar to "fsync" but, unlike fsync, it does not
update the metadata associated with the file (like the access time for
example). It's used by many DBMSs (MySQL and SQLite of example) to
increase disk IO performance, as it avoids disk seeks and disk write
operations compared to fsync.
More details on it at:
http://linux.die.net/man/2/fdatasync
An example, from the MySQL source:
http://bazaar.launchpad.net/~mysql/mysql-server/mysql-5.1-telco-6.1/annotate/head%3A/mysys/my_sync.c#L61
This new function just calls fsync on systems not implementing fdatasync.
|
|
|
