diff options
Diffstat (limited to 'lib/kernel/doc/src/file.xml')
| -rw-r--r-- | lib/kernel/doc/src/file.xml | 399 |
1 files changed, 179 insertions, 220 deletions
diff --git a/lib/kernel/doc/src/file.xml b/lib/kernel/doc/src/file.xml index b674b3ca93..fc25e83d40 100644 --- a/lib/kernel/doc/src/file.xml +++ b/lib/kernel/doc/src/file.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -28,20 +28,23 @@ <date></date> <rev></rev> </header> - <module>file</module> + <module since="">file</module> <modulesummary>File interface module.</modulesummary> <description> <p>This module provides an interface to the file system.</p> - <p>On operating systems with thread support, - file operations can be performed in threads of their own, allowing - other Erlang processes to continue executing in parallel with - the file operations. See command-line flag - <c>+A</c> in <seealso marker="erts:erl"><c>erl(1)</c></seealso>.</p> + <warning> + <p>File operations are only guaranteed to appear atomic when going + through the same file server. A NIF or other OS process may observe + intermediate steps on certain operations on some operating systems, + eg. renaming an existing file on Windows, or + <seealso marker="#write_file_info/2"><c>write_file_info/2</c> + </seealso> on any OS at the time of writing.</p> + </warning> <p>Regarding filename encoding, the Erlang VM can operate in two modes. The current mode can be queried using function - <seealso marker="#native_name_encoding"><c>native_name_encoding/0</c></seealso>. + <seealso marker="#native_name_encoding/0"><c>native_name_encoding/0</c></seealso>. It returns <c>latin1</c> or <c>utf8</c>.</p> <p>In <c>latin1</c> mode, the Erlang VM does not change the @@ -59,7 +62,7 @@ terminal supports UTF-8, otherwise <c>latin1</c>. The default can be overridden using <c>+fnl</c> (to force <c>latin1</c> mode) or <c>+fnu</c> (to force <c>utf8</c> mode) when starting - <seealso marker="erts:erl"><c>erts:erl</c></seealso>.</p> + <seealso marker="erts:erl"><c>erl</c></seealso>.</p> <p>On operating systems with transparent naming, files can be inconsistently named, for example, some files are encoded in UTF-8 while @@ -81,6 +84,16 @@ <p>See also section <seealso marker="stdlib:unicode_usage#notes-about-raw-filenames">Notes About Raw Filenames</seealso> in the STDLIB User's Guide.</p> + <note><p> + File operations used to accept filenames containing + null characters (integer value zero). This caused + the name to be truncated and in some cases arguments + to primitive operations to be mixed up. Filenames + containing null characters inside the filename + are now <em>rejected</em> and will cause primitive + file operations fail. + </p></note> + </description> <datatypes> @@ -96,9 +109,21 @@ </datatype> <datatype> <name name="filename"/> + <desc> + <p> + See also the documentation of the + <seealso marker="#type-name_all"><c>name_all()</c></seealso> type. + </p> + </desc> </datatype> <datatype> <name name="filename_all"/> + <desc> + <p> + See also the documentation of the + <seealso marker="#type-name_all"><c>name_all()</c></seealso> type. + </p> + </desc> </datatype> <datatype> <name name="io_device"/> @@ -112,21 +137,23 @@ <name name="name"/> <desc> <p>If VM is in Unicode filename mode, <c>string()</c> and <c>char()</c> - are allowed to be > 255. + are allowed to be > 255. See also the documentation of the + <seealso marker="#type-name_all"><c>name_all()</c></seealso> type. </p> </desc> </datatype> <datatype> <name name="name_all"/> <desc> - <p>If VM is in Unicode filename mode, <c>string()</c> and <c>char()</c> + <p>If VM is in Unicode filename mode, characters are allowed to be > 255. <c><anno>RawFilename</anno></c> is a filename not subject to Unicode translation, meaning that it can contain characters not conforming to the Unicode encoding expected from the file system (that is, non-UTF-8 characters although the VM is started - in Unicode filename mode). + in Unicode filename mode). Null characters (integer value zero) + are <em>not</em> allowed in filenames (not even at the end). </p> </desc> </datatype> @@ -159,7 +186,7 @@ <funcs> <func> - <name name="advise" arity="4"/> + <name name="advise" arity="4" since="OTP R14B"/> <fsummary>Predeclare an access pattern for file data.</fsummary> <type name="posix_file_advise"/> <desc> @@ -170,7 +197,7 @@ </desc> </func> <func> - <name name="allocate" arity="3"/> + <name name="allocate" arity="3" since="OTP R16B"/> <fsummary>Allocate file space.</fsummary> <desc> <p><c>allocate/3</c> can be used to preallocate space for a file.</p> @@ -182,7 +209,7 @@ </desc> </func> <func> - <name name="change_group" arity="2"/> + <name name="change_group" arity="2" since=""/> <fsummary>Change group of a file.</fsummary> <desc> <p>Changes group of a file. See @@ -190,7 +217,7 @@ </desc> </func> <func> - <name name="change_mode" arity="2"/> + <name name="change_mode" arity="2" since="OTP R14B"/> <fsummary>Change permissions of a file.</fsummary> <desc> <p>Changes permissions of a file. See @@ -198,7 +225,7 @@ </desc> </func> <func> - <name name="change_owner" arity="2"/> + <name name="change_owner" arity="2" since=""/> <fsummary>Change owner of a file.</fsummary> <desc> <p>Changes owner of a file. See @@ -206,7 +233,7 @@ </desc> </func> <func> - <name name="change_owner" arity="3"/> + <name name="change_owner" arity="3" since=""/> <fsummary>Change owner and group of a file.</fsummary> <desc> <p>Changes owner and group of a file. See @@ -214,7 +241,7 @@ </desc> </func> <func> - <name name="change_time" arity="2"/> + <name name="change_time" arity="2" since=""/> <fsummary>Change the modification time of a file.</fsummary> <desc> <p>Changes the modification and access times of a file. See @@ -222,7 +249,7 @@ </desc> </func> <func> - <name name="change_time" arity="3"/> + <name name="change_time" arity="3" since=""/> <fsummary>Change the modification and last access time of a file.</fsummary> <desc> <p>Changes the modification and last access times of a file. See @@ -230,7 +257,7 @@ </desc> </func> <func> - <name name="close" arity="1"/> + <name name="close" arity="1" since=""/> <fsummary>Close a file.</fsummary> <desc> <p>Closes the file referenced by <c><anno>IoDevice</anno></c>. It mostly @@ -243,7 +270,7 @@ </desc> </func> <func> - <name name="consult" arity="1"/> + <name name="consult" arity="1" since=""/> <fsummary>Read Erlang terms from a file.</fsummary> <desc> <p>Reads Erlang terms, separated by '.', from @@ -281,8 +308,8 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="copy" arity="2"/> - <name name="copy" arity="3"/> + <name name="copy" arity="2" since=""/> + <name name="copy" arity="3" since=""/> <fsummary>Copy file contents.</fsummary> <desc> <p>Copies <c><anno>ByteCount</anno></c> bytes from @@ -319,7 +346,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="datasync" arity="1"/> + <name name="datasync" arity="1" since="OTP R14B"/> <fsummary>Synchronize the in-memory data of a file, ignoring most of its metadata, with that on the physical medium.</fsummary> <desc> <p>Ensures that any buffers kept by the operating system @@ -342,7 +369,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="del_dir" arity="1"/> + <name name="del_dir" arity="1" since=""/> <fsummary>Delete a directory.</fsummary> <desc> <p>Tries to delete directory <c><anno>Dir</anno></c>. @@ -378,7 +405,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="delete" arity="1"/> + <name name="delete" arity="1" since=""/> <fsummary>Delete a file.</fsummary> <desc> <p>Tries to delete file <c><anno>Filename</anno></c>. @@ -415,7 +442,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="eval" arity="1"/> + <name name="eval" arity="1" since=""/> <fsummary>Evaluate Erlang expressions in a file.</fsummary> <desc> <p>Reads and evaluates Erlang expressions, separated by '.' (or @@ -449,7 +476,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="eval" arity="2"/> + <name name="eval" arity="2" since=""/> <fsummary>Evaluate Erlang expressions in a file.</fsummary> <desc> <p>The same as <c>eval/1</c>, but the variable bindings @@ -459,7 +486,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="format_error" arity="1"/> + <name name="format_error" arity="1" since=""/> <fsummary>Return a descriptive string for an error reason.</fsummary> <desc> <p>Given the error reason returned by any function in this @@ -467,7 +494,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="get_cwd" arity="0"/> + <name name="get_cwd" arity="0" since=""/> <fsummary>Get the current working directory.</fsummary> <desc> <p>Returns <c>{ok, <anno>Dir</anno>}</c>, where <c><anno>Dir</anno></c> @@ -489,7 +516,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="get_cwd" arity="1"/> + <name name="get_cwd" arity="1" since=""/> <fsummary>Get the current working directory for the specified drive.</fsummary> <desc> <p>Returns <c>{ok, <anno>Dir</anno>}</c> or @@ -520,7 +547,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="list_dir" arity="1"/> + <name name="list_dir" arity="1" since=""/> <fsummary>List files in a directory.</fsummary> <desc> <p>Lists all files in a directory, <em>except</em> files @@ -551,7 +578,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="list_dir_all" arity="1"/> + <name name="list_dir_all" arity="1" since="OTP R16B"/> <fsummary>List all files in a directory.</fsummary> <desc> <p><marker id="list_dir_all"/>Lists all the files in a directory, @@ -576,7 +603,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="make_dir" arity="1"/> + <name name="make_dir" arity="1" since=""/> <fsummary>Make a directory.</fsummary> <desc> <p>Tries to create directory <c><anno>Dir</anno></c>. Missing parent @@ -610,7 +637,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="make_link" arity="2"/> + <name name="make_link" arity="2" since=""/> <fsummary>Make a hard link to a file.</fsummary> <desc> <p>Makes a hard link from <c><anno>Existing</anno></c> to @@ -639,7 +666,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="make_symlink" arity="2"/> + <name name="make_symlink" arity="2" since=""/> <fsummary>Make a symbolic link to a file or directory.</fsummary> <desc> <p>Creates a symbolic link <c><anno>New</anno></c> to @@ -675,7 +702,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="native_name_encoding" arity="0"/> + <name name="native_name_encoding" arity="0" since="OTP R14B01"/> <fsummary>Return the configured filename encoding of the VM.</fsummary> <desc> <p><marker id="native_name_encoding"/>Returns @@ -687,7 +714,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="open" arity="2"/> + <name name="open" arity="2" since=""/> <fsummary>Open a file.</fsummary> <desc> <p>Opens file <c><anno>File</anno></c> in the mode determined @@ -954,8 +981,7 @@ f.txt: {person, "kalle", 25}. </item> <tag><c>eisdir</c></tag> <item> - <p>The named file is not a regular file. It can be a - directory, a FIFO, or a device.</p> + <p>The named file is a directory.</p> </item> <tag><c>enotdir</c></tag> <item> @@ -971,7 +997,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="path_consult" arity="2"/> + <name name="path_consult" arity="2" since=""/> <fsummary>Read Erlang terms from a file.</fsummary> <desc> <p>Searches the path <c><anno>Path</anno></c> (a list of directory @@ -1013,7 +1039,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="path_eval" arity="2"/> + <name name="path_eval" arity="2" since=""/> <fsummary>Evaluate Erlang expressions in a file.</fsummary> <desc> <p>Searches the path <c><anno>Path</anno></c> (a list of directory @@ -1059,7 +1085,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="path_open" arity="3"/> + <name name="path_open" arity="3" since=""/> <fsummary>Open a file.</fsummary> <desc> <p>Searches the path <c><anno>Path</anno></c> (a list of directory @@ -1088,7 +1114,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="path_script" arity="2"/> + <name name="path_script" arity="2" since=""/> <fsummary>Evaluate and return the value of Erlang expressions in a file.</fsummary> <desc> <p>Searches the path <c><anno>Path</anno></c> (a list of directory @@ -1132,7 +1158,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="path_script" arity="3"/> + <name name="path_script" arity="3" since=""/> <fsummary>Evaluate and return the value of Erlang expressions in a file.</fsummary> <desc> <p>The same as <c>path_script/2</c> but the variable bindings @@ -1142,7 +1168,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="pid2name" arity="1"/> + <name name="pid2name" arity="1" since=""/> <fsummary>Return the name of the file handled by a pid.</fsummary> <desc> <p>If <c><anno>Pid</anno></c> is an I/O device, that is, a pid returned from @@ -1167,7 +1193,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="position" arity="2"/> + <name name="position" arity="2" since=""/> <fsummary>Set position in a file.</fsummary> <desc> <p>Sets the position of the file referenced by <c><anno>IoDevice</anno></c> @@ -1219,7 +1245,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="pread" arity="2"/> + <name name="pread" arity="2" since=""/> <fsummary>Read from a file at certain positions.</fsummary> <desc> <p>Performs a sequence of <c>pread/3</c> in one operation, @@ -1237,7 +1263,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="pread" arity="3"/> + <name name="pread" arity="3" since=""/> <fsummary>Read from a file at a certain position.</fsummary> <desc> <p>Combines <c>position/2</c> and <c>read/2</c> in one @@ -1257,7 +1283,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="pwrite" arity="2"/> + <name name="pwrite" arity="2" since=""/> <fsummary>Write to a file at certain positions.</fsummary> <desc> <p>Performs a sequence of <c>pwrite/3</c> in one operation, @@ -1272,7 +1298,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="pwrite" arity="3"/> + <name name="pwrite" arity="3" since=""/> <fsummary>Write to a file at a certain position.</fsummary> <desc> <p>Combines <c>position/2</c> and <c>write/2</c> in one @@ -1291,7 +1317,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="read" arity="2"/> + <name name="read" arity="2" since=""/> <fsummary>Read from a file.</fsummary> <desc> <p>Reads <c><anno>Number</anno></c> bytes/characters from the file @@ -1345,7 +1371,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="read_file" arity="1"/> + <name name="read_file" arity="1" since=""/> <fsummary>Read a file.</fsummary> <desc> <p>Returns <c>{ok, <anno>Binary</anno>}</c>, where @@ -1381,8 +1407,8 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="read_file_info" arity="1"/> - <name name="read_file_info" arity="2"/> + <name name="read_file_info" arity="1" since=""/> + <name name="read_file_info" arity="2" since="OTP R15B"/> <fsummary>Retrieve information about a file.</fsummary> <desc> <p>Retrieves information about a file. Returns @@ -1407,8 +1433,12 @@ f.txt: {person, "kalle", 25}. which is 1970-01-01 00:00 UTC.</p></item> </taglist> <p>Default is <c>{time, local}</c>.</p> - <p>If the option <c>raw</c> is set, the file server is not called - and only information about local files is returned.</p> + <p>If the option <c>raw</c> is set, the file server is not called and + only information about local files is returned. Note that this will + break this module's atomicity guarantees as it can race with a + concurrent call to + <seealso marker="#write_file_info/2"><c>write_file_info/1,2</c> + </seealso></p> <note> <p>As file times are stored in POSIX time on most OS, it is faster to query file information with option <c>posix</c>.</p> @@ -1532,7 +1562,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="read_line" arity="1"/> + <name name="read_line" arity="1" since=""/> <fsummary>Read a line from a file.</fsummary> <desc> <p>Reads a line of bytes/characters from the file referenced by @@ -1589,7 +1619,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="read_link" arity="1"/> + <name name="read_link" arity="1" since=""/> <fsummary>See what a link is pointing to.</fsummary> <desc> <p><marker id="read_link_all"/>Returns @@ -1619,7 +1649,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="read_link_all" arity="1"/> + <name name="read_link_all" arity="1" since="OTP R16B"/> <fsummary>See what a link is pointing to.</fsummary> <desc> <p>Returns <c>{ok, <anno>Filename</anno>}</c> if @@ -1647,8 +1677,8 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="read_link_info" arity="1"/> - <name name="read_link_info" arity="2"/> + <name name="read_link_info" arity="1" since=""/> + <name name="read_link_info" arity="2" since="OTP R15B"/> <fsummary>Retrieve information about a link or file.</fsummary> <desc> <p>Works like @@ -1656,8 +1686,12 @@ f.txt: {person, "kalle", 25}. except that if <c><anno>Name</anno></c> is a symbolic link, information about the link is returned in the <c>file_info</c> record and the <c>type</c> field of the record is set to <c>symlink</c>.</p> - <p>If the option <c>raw</c> is set, the file server is not called - and only information about local files is returned.</p> + <p>If the option <c>raw</c> is set, the file server is not called and + only information about local files is returned. Note that this will + break this module's atomicity guarantees as it can race with a + concurrent call to + <seealso marker="#write_file_info/2"><c>write_file_info/1,2</c> + </seealso></p> <p>If <c><anno>Name</anno></c> is not a symbolic link, this function returns the same result as <c>read_file_info/1</c>. On platforms that do not support symbolic links, this function @@ -1665,7 +1699,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="rename" arity="2"/> + <name name="rename" arity="2" since=""/> <fsummary>Rename a file.</fsummary> <desc> <p>Tries to rename the file <c><anno>Source</anno></c> to @@ -1728,7 +1762,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="script" arity="1"/> + <name name="script" arity="1" since=""/> <fsummary>Evaluate and return the value of Erlang expressions in a file.</fsummary> <desc> <p>Reads and evaluates Erlang expressions, separated by '.' (or @@ -1763,7 +1797,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="script" arity="2"/> + <name name="script" arity="2" since=""/> <fsummary>Evaluate and return the value of Erlang expressions in a file.</fsummary> <desc> <p>The same as <c>script/1</c> but the variable bindings @@ -1773,7 +1807,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="sendfile" arity="2"/> + <name name="sendfile" arity="2" since="OTP R15B"/> <fsummary>Send a file to a socket.</fsummary> <desc> <p>Sends the file <c>Filename</c> to <c>Socket</c>. @@ -1782,7 +1816,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="sendfile" arity="5"/> + <name name="sendfile" arity="5" since="OTP R15B"/> <fsummary>Send a file to a socket.</fsummary> <type name="sendfile_option"/> <desc> @@ -1795,29 +1829,21 @@ f.txt: {person, "kalle", 25}. <p>The file used must be opened using the <c>raw</c> flag, and the process calling <c>sendfile</c> must be the controlling process of the socket. See <seealso marker="gen_tcp#controlling_process-2"><c>gen_tcp:controlling_process/2</c></seealso>.</p> - <p>If the OS used does not support <c>sendfile</c>, an Erlang fallback - using - <seealso marker="#read/2"><c>read/2</c></seealso> and - <seealso marker="gen_tcp#send/2"><c>gen_tcp:send/2</c></seealso> is used.</p> + <p>If the OS used does not support non-blocking <c>sendfile</c>, an + Erlang fallback using <seealso marker="#read/2"><c>read/2</c></seealso> + and <seealso marker="gen_tcp#send/2"><c>gen_tcp:send/2</c></seealso> is + used.</p> <p>The option list can contain the following options:</p> <taglist> <tag><c>chunk_size</c></tag> <item><p>The chunk size used by the Erlang fallback to send data. If using the fallback, set this to a value that comfortably fits in the systems memory. Default is 20 MB.</p></item> - <tag><c>use_threads</c></tag> - <item><p>Instructs the emulator to use the <c>async</c> thread pool for the - <c>sendfile</c> system call. This can be useful if the OS you are running - on does not properly support non-blocking <c>sendfile</c> calls. Notice that - using <c>async</c> threads potentially makes your system vulnerable to slow - client attacks. If set to <c>true</c> and no <c>async</c> threads are available, - the <c>sendfile</c> call returns <c>{error,einval}</c>. - Introduced in Erlang/OTP 17.0. Default is <c>false</c>.</p></item> </taglist> </desc> </func> <func> - <name name="set_cwd" arity="1"/> + <name name="set_cwd" arity="1" since=""/> <fsummary>Set the current working directory.</fsummary> <desc> <p>Sets the current working directory of the file server to @@ -1825,7 +1851,7 @@ f.txt: {person, "kalle", 25}. <p>The functions in the module <c>file</c> usually treat binaries as raw filenames, that is, they are passed "as is" even when the encoding of the binary does not agree with - <seealso marker="#native_name_encoding"><c>native_name_encoding()</c></seealso>. + <seealso marker="#native_name_encoding/0"><c>native_name_encoding()</c></seealso>. However, this function expects binaries to be encoded according to the value returned by <c>native_name_encoding()</c>.</p> <p>Typical error reasons are:</p> @@ -1864,7 +1890,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="sync" arity="1"/> + <name name="sync" arity="1" since=""/> <fsummary>Synchronize the in-memory state of a file with that on the physical medium.</fsummary> <desc> <p>Ensures that any buffers kept by the operating system @@ -1880,7 +1906,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="truncate" arity="1"/> + <name name="truncate" arity="1" since=""/> <fsummary>Truncate a file.</fsummary> <desc> <p>Truncates the file referenced by <c><anno>IoDevice</anno></c> at @@ -1889,7 +1915,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="write" arity="2"/> + <name name="write" arity="2" since=""/> <fsummary>Write to a file.</fsummary> <desc> <p>Writes <c><anno>Bytes</anno></c> to the file referenced by @@ -1915,7 +1941,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="write_file" arity="2"/> + <name name="write_file" arity="2" since=""/> <fsummary>Write a file.</fsummary> <desc> <p>Writes the contents of the <c>iodata</c> term <c><anno>Bytes</anno></c> @@ -1952,7 +1978,7 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="write_file" arity="3"/> + <name name="write_file" arity="3" since=""/> <fsummary>Write a file.</fsummary> <desc> <p>Same as <c>write_file/2</c>, but takes a third argument @@ -1963,8 +1989,8 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> - <name name="write_file_info" arity="2"/> - <name name="write_file_info" arity="3"/> + <name name="write_file_info" arity="2" since=""/> + <name name="write_file_info" arity="3" since="OTP R15B"/> <fsummary>Change file information.</fsummary> <desc> <p>Changes file information. Returns <c>ok</c> if successful, @@ -2117,144 +2143,77 @@ f.txt: {person, "kalle", 25}. <section> <title>Performance</title> - <p>Some operating system file operations, for example, a - <c>sync/1</c> or <c>close/1</c> on a huge file, can block their - calling thread for seconds. If this affects the emulator main - thread, the response time is no longer in the order of - milliseconds, depending on the definition of "soft" in soft - real-time system.</p> - <p>If the device driver thread pool is active, file operations are - done through those threads instead, so the emulator can go on - executing Erlang processes. Unfortunately, the time for serving a - file operation increases because of the extra scheduling required - from the operating system.</p> - <p>If the device driver thread pool is disabled or of size 0, large - file reads and writes are segmented into many smaller, which - enable the emulator to serve other processes during the file - operation. This has the same effect as when using the thread - pool, but with larger overhead. Other file operations, for - example, <c>sync/1</c> or <c>close/1</c> on a huge file, still are - a problem.</p> - <p>For increased performance, raw files are recommended. Raw files - use the file system of the host machine of the node.</p> + <p>For increased performance, raw files are recommended.</p> + <p>A normal file is really a process so it can be used as an I/O + device (see <seealso marker="stdlib:io"><c>io</c></seealso>). + Therefore, when data is written to a normal file, the sending of the + data to the file process, copies all data that are not binaries. Opening + the file in binary mode and writing binaries is therefore recommended. + If the file is opened on another node, or if the file server runs as + slave to the file server of another node, also binaries are copied.</p> <note> - <p> - For normal files (non-raw), the file server is used to find the files, - and if the node is running its file server as slave to the file server - of another node, and the other node runs on some other host machine, - they can have different file systems. - However, this is seldom a problem.</p> + <p>Raw files use the file system of the host machine of the node. + For normal files (non-raw), the file server is used to find the files, + and if the node is running its file server as slave to the file server + of another node, and the other node runs on some other host machine, + they can have different file systems. + However, this is seldom a problem.</p> </note> - <p>A normal file is really a process so it can be used as an I/O - device (see - <seealso marker="stdlib:io"><c>io</c></seealso>). - Therefore, when data is written to a - normal file, the sending of the data to the file process, copies - all data that are not binaries. Opening the file in binary mode - and writing binaries is therefore recommended. If the file is - opened on another node, or if the file server runs as slave to - the file server of another node, also binaries are copied.</p> - <p>Caching data to reduce the number of file operations, or rather - the number of calls to the file driver, generally increases - performance. The following function writes 4 MBytes in 23 - seconds when tested:</p> + <p><seealso marker="#open/2"><c>open/2</c></seealso> can be given the + options <c>delayed_write</c> and <c>read_ahead</c> to turn on caching, + which will reduce the number of operating system calls and greatly + improve performance for small reads and writes. However, the overhead + won't disappear completely and it's best to keep the number of file + operations to a minimum. As a contrived example, the following function + writes 4MB in 2.5 seconds when tested:</p> + <code type="none"><![CDATA[ -create_file_slow(Name, N) when integer(N), N >= 0 -> - {ok, FD} = file:open(Name, [raw, write, delayed_write, binary]), - ok = create_file_slow(FD, 0, N), - ok = ?FILE_MODULE:close(FD), - ok. - -create_file_slow(FD, M, M) -> +create_file_slow(Name) -> + {ok, Fd} = file:open(Name, [raw, write, delayed_write, binary]), + create_file_slow_1(Fd, 4 bsl 20), + file:close(Fd). + +create_file_slow_1(_Fd, 0) -> ok; -create_file_slow(FD, M, N) -> - ok = file:write(FD, <<M:32/unsigned>>), - create_file_slow(FD, M+1, N).]]></code> +create_file_slow_1(Fd, M) -> + ok = file:write(Fd, <<0>>), + create_file_slow_1(Fd, M - 1).]]></code> + + <p>The following functionally equivalent code writes 128 bytes per call + to <seealso marker="#write/2"><c>write/2</c></seealso> and so does the + same work in 0.08 seconds, which is roughly 30 times faster:</p> - <p>The following, functionally equivalent, function collects 1024 - entries into a list of 128 32-byte binaries before each call to - <seealso marker="#write/2"><c>write/2</c></seealso> and so - does the same work in 0.52 seconds, - which is 44 times faster:</p> <code type="none"><![CDATA[ -create_file(Name, N) when integer(N), N >= 0 -> - {ok, FD} = file:open(Name, [raw, write, delayed_write, binary]), - ok = create_file(FD, 0, N), - ok = ?FILE_MODULE:close(FD), +create_file(Name) -> + {ok, Fd} = file:open(Name, [raw, write, delayed_write, binary]), + create_file_1(Fd, 4 bsl 20), + file:close(Fd), ok. - -create_file(FD, M, M) -> + +create_file_1(_Fd, 0) -> ok; -create_file(FD, M, N) when M + 1024 =< N -> - create_file(FD, M, M + 1024, []), - create_file(FD, M + 1024, N); -create_file(FD, M, N) -> - create_file(FD, M, N, []). - -create_file(FD, M, M, R) -> - ok = file:write(FD, R); -create_file(FD, M, N0, R) when M + 8 =< N0 -> - N1 = N0-1, N2 = N0-2, N3 = N0-3, N4 = N0-4, - N5 = N0-5, N6 = N0-6, N7 = N0-7, N8 = N0-8, - create_file(FD, M, N8, - [<<N8:32/unsigned, N7:32/unsigned, - N6:32/unsigned, N5:32/unsigned, - N4:32/unsigned, N3:32/unsigned, - N2:32/unsigned, N1:32/unsigned>> | R]); -create_file(FD, M, N0, R) -> - N1 = N0-1, - create_file(FD, M, N1, [<<N1:32/unsigned>> | R]).]]></code> +create_file_1(Fd, M) when M >= 128 -> + ok = file:write(Fd, <<0:(128)/unit:8>>), + create_file_1(Fd, M - 128); +create_file_1(Fd, M) -> + ok = file:write(Fd, <<0:(M)/unit:8>>), + create_file_1(Fd, M - 1).]]></code> - <note> - <p>Trust only your own benchmarks. If the list length in - <c>create_file/2</c> above is increased, it runs slightly - faster, but consumes more memory and causes more memory - fragmentation. How much this affects your application is - something that this simple benchmark cannot predict.</p> - <p>If the size of each binary is increased to 64 bytes, it - also runs slightly faster, but the code is then twice as clumsy. - In the current implementation, binaries larger than 64 bytes are - stored in memory common to all processes and not copied when - sent between processes, while these smaller binaries are stored - on the process heap and copied when sent like any other term.</p> - <p>So, with a binary size of 68 bytes, <c>create_file/2</c> runs - 30 percent slower than with 64 bytes, and causes much more - memory fragmentation. Notice that if the binaries were to be sent - between processes (for example, a non-raw file), the results - would probably be completely different.</p> - </note> - <p>A raw file is really a port. When writing data to a port, it is - efficient to write a list of binaries. It is not needed to - flatten a deep list before writing. On Unix hosts, scatter output, - which writes a set of buffers in one operation, is used when - possible. In this way <c>write(FD, [Bin1, Bin2 | Bin3])</c> - writes the contents of the binaries without copying the data - at all, except for perhaps deep down in the operating system - kernel.</p> - <p>For raw files, <c>pwrite/2</c> and <c>pread/2</c> are - efficiently implemented. The file driver is called only once for - the whole operation, and the list iteration is done in the file - driver.</p> - <p>The options <c>delayed_write</c> and <c>read_ahead</c> to - <seealso marker="#open/2"><c>open/2</c></seealso> - make the file driver cache data to reduce - the number of operating system calls. The function - <c>create_file/2</c> in the recent example takes 60 seconds - without option <c>delayed_write</c>, which is 2.6 - times slower.</p> - <p>As a bad example, <c>create_file_slow/2</c> - without options <c>raw</c>, <c>binary</c>, and <c>delayed_write</c>, - meaning it calls <c>open(Name, [write])</c>, needs - 1 min 20 seconds for the job, which is 3.5 times slower than - the first example, and 150 times slower than the optimized - <c>create_file/2</c>.</p> - <warning> - <p>If an error occurs when accessing an open file with module - <seealso marker="stdlib:io"><c>io</c></seealso>, - the process handling the file exits. The dead - file process can hang if a process tries to access it later. - This will be fixed in a future release.</p> - </warning> + <p>When writing data it's generally more efficient to write a list of + binaries rather than a list of integers. It is not needed to + flatten a deep list before writing. On Unix hosts, scatter output, + which writes a set of buffers in one operation, is used when + possible. In this way <c>write(FD, [Bin1, Bin2 | Bin3])</c> + writes the contents of the binaries without copying the data + at all, except for perhaps deep down in the operating system + kernel.</p> + <warning> + <p>If an error occurs when accessing an open file with module + <seealso marker="stdlib:io"><c>io</c></seealso>, the process + handling the file exits. The dead file process can hang if a process + tries to access it later. This will be fixed in a future release. + </p> + </warning> </section> <section> |