aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorRickard Green <[email protected]>2012-11-16 09:31:56 +0100
committerRickard Green <[email protected]>2012-11-16 09:31:56 +0100
commitcb11b4d6ec812813d0df07facc02ac5bd47e6851 (patch)
tree355c2048bf1fcee6392a32be9ed921b3caa3c188
parent9b7d4359fd771fc3e7589dc91e091c7c35a65807 (diff)
parent15f3cb7cb7074d507f70e778862e285e844b8546 (diff)
downloadotp-cb11b4d6ec812813d0df07facc02ac5bd47e6851.tar.gz
otp-cb11b4d6ec812813d0df07facc02ac5bd47e6851.tar.bz2
otp-cb11b4d6ec812813d0df07facc02ac5bd47e6851.zip
Merge branch 'maint'
* maint: Add clearer warnings about misuse of NIF and driver functionality Fix erlang:memory(ets) Conflicts: erts/emulator/beam/erl_db.c
-rw-r--r--erts/doc/src/driver_entry.xml23
-rw-r--r--erts/doc/src/erl_driver.xml57
-rw-r--r--erts/doc/src/erl_nif.xml97
-rw-r--r--erts/emulator/beam/erl_db.c4
4 files changed, 138 insertions, 43 deletions
diff --git a/erts/doc/src/driver_entry.xml b/erts/doc/src/driver_entry.xml
index a2efdf3ebc..929c485c36 100644
--- a/erts/doc/src/driver_entry.xml
+++ b/erts/doc/src/driver_entry.xml
@@ -34,6 +34,29 @@
<lib>driver_entry</lib>
<libsummary>The driver-entry structure used by erlang drivers.</libsummary>
<description>
+ <marker id="WARNING"/>
+ <warning><p><em>Use this functionality with extreme care!</em></p>
+ <p>A driver callback is executed as a direct extension of the
+ native code of the VM. Execution is not made in a safe environment.
+ The VM can <em>not</em> provide the same services as provided when
+ executing Erlang code, such as preemptive scheduling or memory
+ protection. If the driver callback function doesn't behave well,
+ the whole VM will misbehave.</p>
+ <list>
+ <item><p>A driver callback that crash will crash the whole VM.</p></item>
+ <item><p>An erroneously implemented driver callback might cause
+ a VM internal state inconsistency which may cause a crash of the VM,
+ or miscellaneous misbehaviors of the VM at any point after the call
+ to the driver callback.</p></item>
+ <item><p>A driver callback that do
+ <seealso marker="erl_driver#lengthy_work">lengthy work</seealso>
+ before returning will degrade responsiveness of the VM,
+ and may cause miscellaneous strange behaviors. Such strange behaviors
+ include, but are not limited to, extreme memory usage, and bad load
+ balancing between schedulers. Strange behaviors that might occur due
+ to lengthy work may also vary between OTP releases.</p></item>
+ </list>
+ </warning>
<p>
As of erts version 5.9 (OTP release R15B) the driver interface
has been changed with larger types for the callbacks
diff --git a/erts/doc/src/erl_driver.xml b/erts/doc/src/erl_driver.xml
index 187c263b60..e16fd744c0 100644
--- a/erts/doc/src/erl_driver.xml
+++ b/erts/doc/src/erl_driver.xml
@@ -34,6 +34,32 @@
<lib>erl_driver</lib>
<libsummary>API functions for an Erlang driver</libsummary>
<description>
+ <p>An Erlang driver is a library containing a set of native driver
+ callback functions that the Erlang VM calls when certain
+ events occur. There may be multiple instances of a driver, each
+ instance is associated with an Erlang port.</p>
+ <marker id="WARNING"/>
+ <warning><p><em>Use this functionality with extreme care!</em></p>
+ <p>A driver callback is executed as a direct extension of the
+ native code of the VM. Execution is not made in a safe environment.
+ The VM can <em>not</em> provide the same services as provided when
+ executing Erlang code, such as preemptive scheduling or memory
+ protection. If the driver callback function doesn't behave well,
+ the whole VM will misbehave.</p>
+ <list>
+ <item><p>A driver callback that crash will crash the whole VM.</p></item>
+ <item><p>An erroneously implemented driver callback might cause
+ a VM internal state inconsistency which may cause a crash of the VM,
+ or miscellaneous misbehaviors of the VM at any point after the call
+ to the driver callback.</p></item>
+ <item><p>A driver callback that do <seealso marker="#lengthy_work">lengthy
+ work</seealso> before returning will degrade responsiveness of the VM,
+ and may cause miscellaneous strange behaviors. Such strange behaviors
+ include, but are not limited to, extreme memory usage, and bad load
+ balancing between schedulers. Strange behaviors that might occur due
+ to lengthy work may also vary between OTP releases.</p></item>
+ </list>
+ </warning>
<p>As of erts version 5.5.3 the driver interface has been extended
(see <seealso marker="driver_entry#extended_marker">extended marker</seealso>).
The extended interface introduce
@@ -53,16 +79,12 @@
<p>The driver calls back to the emulator, using the API
functions declared in <c>erl_driver.h</c>. They are used for
outputting data from the driver, using timers, etc.</p>
- <p>A driver is a library with a set of function that the emulator
- calls, in response to Erlang functions and message
- sending. There may be multiple instances of a driver, each
- instance is connected to an Erlang port. Every port has a port
- owner process. Communication with the port is normally done
- through the port owner process.</p>
- <p>Most of the functions take the <c>port</c> handle as an
- argument. This identifies the driver instance. Note that this
- port handle must be stored by the driver, it is not given when
- the driver is called from the emulator (see
+ <p>Each driver instance is associated with a port. Every port
+ has a port owner process. Communication with the port is normally
+ done through the port owner process. Most of the functions take
+ the <c>port</c> handle as an argument. This identifies the driver
+ instance. Note that this port handle must be stored by the driver,
+ it is not given when the driver is called from the emulator (see
<seealso marker="driver_entry#emulator">driver_entry</seealso>).</p>
<p>Some of the functions take a parameter of type
<c>ErlDrvBinary</c>, a driver binary. It should be both
@@ -129,6 +151,21 @@
are <em>only</em> thread safe when used in a runtime
system with SMP support.</p>
</note>
+ <p><marker id="lengthy_work"/>
+ As mentioned in the <seealso marker="#WARNING">warning</seealso> text at
+ the beginning of this document it is of vital importance that a driver callback
+ does return relatively fast. It is hard to give an exact maximum amount
+ of time that a driver callback is allowed to work, but as a rule of thumb
+ a well behaving driver callback should return before a millisecond has
+ passed. This can be achieved using different approaches.
+ If you have full control over the code that are to execute in the driver
+ callback, the best approach is to divide the work into multiple chunks of
+ work and trigger multiple calls to the
+ <seealso marker="driver_entry#timeout">timeout callback</seealso> using
+ zero timeouts. This might, however, not always be possible, e.g. when
+ calling third party libraries. In this case you typically want to dispatch
+ the work to another thread. Information about thread primitives can be
+ found below.</p>
</description>
<section>
diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml
index f484e9eaf7..f00f7b9f46 100644
--- a/erts/doc/src/erl_nif.xml
+++ b/erts/doc/src/erl_nif.xml
@@ -34,30 +34,6 @@
<lib>erl_nif</lib>
<libsummary>API functions for an Erlang NIF library</libsummary>
<description>
- <note><p>The NIF concept is officially supported from R14B. NIF source code
- written for earlier experimental versions might need adaption to run on R14B.</p>
- <p>No incompatible changes between <em>R14B</em> and R14A.</p>
- <p>Incompatible changes between <em>R14A</em> and R13B04:</p>
- <list>
- <item>Environment argument removed for <c>enif_alloc</c>,
- <c>enif_realloc</c>, <c>enif_free</c>, <c>enif_alloc_binary</c>,
- <c>enif_realloc_binary</c>, <c>enif_release_binary</c>,
- <c>enif_alloc_resource</c>, <c>enif_release_resource</c>,
- <c>enif_is_identical</c> and <c>enif_compare</c>.</item>
- <item>Character encoding argument added to <c>enif_get_atom</c>
- and <c>enif_make_existing_atom</c>.</item>
- <item>Module argument added to <c>enif_open_resource_type</c>
- while changing name spaces of resource types from global to module local.</item>
- </list>
- <p>Incompatible changes between <em>R13B04</em> and R13B03:</p>
- <list>
- <item>The function prototypes of the NIFs have changed to expect <c>argc</c> and <c>argv</c>
- arguments. The arity of a NIF is by that no longer limited to 3.</item>
- <item><c>enif_get_data</c> renamed as <c>enif_priv_data</c>.</item>
- <item><c>enif_make_string</c> got a third argument for character encoding.</item>
- </list>
- </note>
-
<p>A NIF library contains native implementation of some functions
of an Erlang module. The native implemented functions (NIFs) are
called like any other functions without any difference to the
@@ -67,6 +43,57 @@
is to throw an exception. But it can also be used as a fallback
implementation if the NIF library is not implemented for some
architecture.</p>
+ <marker id="WARNING"/>
+ <warning><p><em>Use this functionality with extreme care!</em></p>
+ <p>A native function is executed as a direct extension of the
+ native code of the VM. Execution is not made in a safe environment.
+ The VM can <em>not</em> provide the same services as provided when
+ executing Erlang code, such as preemptive scheduling or memory
+ protection. If the native function doesn't behave well, the whole
+ VM will misbehave.</p>
+ <list>
+ <item><p>A native function that crash will crash the whole VM.</p></item>
+ <item><p>An erroneously implemented native function might cause
+ a VM internal state inconsistency which may cause a crash of the VM,
+ or miscellaneous misbehaviors of the VM at any point after the call
+ to the native function.</p></item>
+ <item><p>A native function that do <seealso marker="#lengthy_work">lengthy
+ work</seealso> before returning will degrade responsiveness of the VM,
+ and may cause miscellaneous strange behaviors. Such strange behaviors
+ include, but are not limited to, extreme memory usage, and bad load
+ balancing between schedulers. Strange behaviors that might occur due
+ to lengthy work may also vary between OTP releases.</p></item>
+ </list>
+ </warning>
+
+ <p>The NIF concept is officially supported from R14B. NIF source code
+ written for earlier experimental versions might need adaption to run on R14B
+ or later versions:</p>
+ <list>
+ <item>No incompatible changes between <em>R14B</em> and R14A.</item>
+ <item>Incompatible changes between <em>R14A</em> and R13B04:
+ <list>
+ <item>Environment argument removed for <c>enif_alloc</c>,
+ <c>enif_realloc</c>, <c>enif_free</c>, <c>enif_alloc_binary</c>,
+ <c>enif_realloc_binary</c>, <c>enif_release_binary</c>,
+ <c>enif_alloc_resource</c>, <c>enif_release_resource</c>,
+ <c>enif_is_identical</c> and <c>enif_compare</c>.</item>
+ <item>Character encoding argument added to <c>enif_get_atom</c>
+ and <c>enif_make_existing_atom</c>.</item>
+ <item>Module argument added to <c>enif_open_resource_type</c>
+ while changing name spaces of resource types from global to module local.</item>
+ </list>
+ </item>
+ <item>Incompatible changes between <em>R13B04</em> and R13B03:
+ <list>
+ <item>The function prototypes of the NIFs have changed to expect <c>argc</c> and <c>argv</c>
+ arguments. The arity of a NIF is by that no longer limited to 3.</item>
+ <item><c>enif_get_data</c> renamed as <c>enif_priv_data</c>.</item>
+ <item><c>enif_make_string</c> got a third argument for character encoding.</item>
+ </list>
+ </item>
+ </list>
+
<p>A minimal example of a NIF library can look like this:</p>
<p/>
<code type="none">
@@ -136,7 +163,23 @@ ok
then retrieved by calling <seealso marker="#enif_priv_data">enif_priv_data</seealso>.</p>
<p>There is no way to explicitly unload a NIF library. A library will be
automatically unloaded when the module code that it belongs to is purged
- by the code server.</p>
+ by the code server.</p>
+
+ <p><marker id="lengthy_work"/>
+ As mentioned in the <seealso marker="#WARNING">warning</seealso> text at
+ the beginning of this document it is of vital importance that a native function
+ does return relatively fast. It is hard to give an exact maximum amount
+ of time that a native function is allowed to work, but as a rule of thumb
+ a well behaving native function should return to its caller before a
+ millisecond has passed. This can be achieved using different approaches.
+ If you have full control over the code that are to execute in the native
+ function, the best approach is to divide the work into multiple chunks of
+ work and call the native function multiple times. This might, however,
+ not always be possible, e.g. when calling third party libraries. In this
+ case you typically want to dispatch the work to another thread, return
+ from the native function, and wait for the result. The thread can send
+ the result back to the calling thread using message passing. Information
+ about thread primitives can be found below.</p>
</description>
<section>
<title>FUNCTIONALITY</title>
@@ -266,10 +309,6 @@ ok
mutable.</p>
<p>The library initialization callbacks <c>load</c>, <c>reload</c> and
<c>upgrade</c> are all thread-safe even for shared state data.</p>
- <p>Avoid doing lengthy work in NIF calls as that may degrade the
- responsiveness of the VM. NIFs are called directly by the same scheduler
- thread that executed the calling Erlang code. The calling scheduler will thus
- be blocked from doing any other work until the NIF returns.</p>
</item>
</taglist>
</section>
diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c
index 4c30905495..1ba1048afa 100644
--- a/erts/emulator/beam/erl_db.c
+++ b/erts/emulator/beam/erl_db.c
@@ -251,7 +251,6 @@ free_dbtable(void *vtb)
#endif
ASSERT(is_immed(tb->common.heir_data));
erts_db_free(ERTS_ALC_T_DB_TABLE, tb, (void *) tb, sizeof(DbTable));
- ERTS_ETS_MISC_MEM_ADD(-sizeof(DbTable));
}
static void schedule_free_dbtable(DbTable* tb)
@@ -1423,7 +1422,6 @@ BIF_RETTYPE ets_new_2(BIF_ALIST_2)
erts_smp_atomic_init_nob(&init_tb.common.memory_size, 0);
tb = (DbTable*) erts_db_alloc(ERTS_ALC_T_DB_TABLE,
&init_tb, sizeof(DbTable));
- ERTS_ETS_MISC_MEM_ADD(sizeof(DbTable));
erts_smp_atomic_init_nob(&tb->common.memory_size,
erts_smp_atomic_read_nob(&init_tb.common.memory_size));
}
@@ -2867,7 +2865,6 @@ void init_db(void)
meta_pid_to_tab = (DbTable*) erts_db_alloc(ERTS_ALC_T_DB_TABLE,
&init_tb,
sizeof(DbTable));
- ERTS_ETS_MISC_MEM_ADD(sizeof(DbTable));
erts_smp_atomic_init_nob(&meta_pid_to_tab->common.memory_size,
erts_smp_atomic_read_nob(&init_tb.common.memory_size));
@@ -2899,7 +2896,6 @@ void init_db(void)
meta_pid_to_fixed_tab = (DbTable*) erts_db_alloc(ERTS_ALC_T_DB_TABLE,
&init_tb,
sizeof(DbTable));
- ERTS_ETS_MISC_MEM_ADD(sizeof(DbTable));
erts_smp_atomic_init_nob(&meta_pid_to_fixed_tab->common.memory_size,
erts_smp_atomic_read_nob(&init_tb.common.memory_size));