diff options
371 files changed, 22464 insertions, 24522 deletions
diff --git a/HOWTO/BENCHMARKS.md b/HOWTO/BENCHMARKS.md index b3df4638ea..28a590dfd2 100644 --- a/HOWTO/BENCHMARKS.md +++ b/HOWTO/BENCHMARKS.md @@ -18,13 +18,13 @@ benchmarks you have to [install the tests][]. To get a listing of all benchmarks you have available call `ts:benchmarks()`. To run all benchmarks call `ts:bench()`. This will run all benchmarks using -the emulator which is in you `$PATH` (Note that this does not have to be the +the emulator which is in your `$PATH` (Note that this does not have to be the same as from which the benchmarks were built from). All the results of the benchmarks are put in a folder in `$TESTROOT/test_server/` called `YYYY_MO_DDTHH_MI_SS`. Each benchmark is run multiple times and the data for all runs is collected in -the files within the benchmark folder. All benchmarks are written so that a +the files within the benchmark folder. All benchmarks are written so that higher values are better. Writing benchmarks diff --git a/erts/doc/src/Makefile b/erts/doc/src/Makefile index 91e823fdb1..89d7c85a86 100644 --- a/erts/doc/src/Makefile +++ b/erts/doc/src/Makefile @@ -77,6 +77,7 @@ XML_CHAPTER_FILES = \ inet_cfg.xml \ erl_ext_dist.xml \ erl_dist_protocol.xml \ + communication.xml \ notes.xml \ notes_history.xml diff --git a/erts/doc/src/communication.xml b/erts/doc/src/communication.xml new file mode 100644 index 0000000000..6049123f6a --- /dev/null +++ b/erts/doc/src/communication.xml @@ -0,0 +1,89 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2012</year><year>2012</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Communication in Erlang</title> + <prepared>Rickard Green</prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date>2012-12-03</date> + <rev>PA1</rev> + <file>communication.xml</file> + </header> + <p>Communication in Erlang is conceptually performed using + asynchronous signaling. All different executing entities + such as processes, and ports communicate via asynchronous + signals. The most commonly used signal is a message. Other + common signals are exit, link, unlink, monitor, demonitor + signals.</p> + <section> + <title>Passing of Signals</title> + <p>The amount of time that passes between a signal being sent + and the arrival of the signal at the destination is unspecified + but positive. If the receiver has terminated, the signal will + not arrive, but it is possible that it triggers another signal. + For example, a link signal sent to a non-existing process will + trigger an exit signal which will be sent back to where the link + signal originated from. When communicating over the distribution, + signals may be lost if the distribution channel goes down.</p> + <p>The only signal ordering guarantee given is the following. If + an entity sends multiple signals to the same destination entity, + the order will be preserved. That is, if <c>A</c> send + a signal <c>S1</c> to <c>B</c>, and later sends + the signal <c>S2</c> to <c>B</c>, <c>S1</c> is guaranteed not to + arrive after <c>S2</c>.</p> + </section> + <section> + <title>Synchronous Communication</title> + <p>Some communication is synchronous. If broken down into pieces, + a synchronous communication operation, consists of two asynchronous + signals. One request signal and one reply signal. An example of + such a synchronous communication is a call to <c>process_info/2</c> + when the first argument is not <c>self()</c>. The caller will send + an asynchronous signal requesting information, and will then + wait for the reply signal containing the requested information. When + the request signal reaches its destination the destination process + replies with the requested information.</p> + </section> + <section> + <title>Implementation</title> + <p>The implementation of different asynchronous signals in the + VM may vary over time, but the behavior will always respect this + concept of asynchronous signals being passed between entities + as described above.</p> + <p>By inspecting the implementation you might notice that some + specific signal actually gives a stricter guarantee than described + above. It is of vital importance that such knowledge about the + implementation is <em>not</em> used by Erlang code, since the + implementation might change at any time without prior notice.</p> + <p>Some example of major implementation changes:</p> + <list type="bulleted"> + <item>As of ERTS version 5.5.2 exit signals to processes are truly + asynchronously delivered.</item> + <item>As of ERTS version 5.10 all signals from processes to ports + are truly asynchronously delivered.</item> + </list> + </section> +</chapter> + diff --git a/erts/doc/src/driver_entry.xml b/erts/doc/src/driver_entry.xml index 929c485c36..c37138e7b1 100644 --- a/erts/doc/src/driver_entry.xml +++ b/erts/doc/src/driver_entry.xml @@ -4,7 +4,7 @@ <cref> <header> <copyright> - <year>2001</year><year>2011</year> + <year>2001</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -400,11 +400,11 @@ typedef struct erl_drv_entry { <tag><marker id="driver_flags"/>int driver_flags</tag> <item> - <p>This field is used to pass driver capability information to the - runtime system. If the <c>extended_marker</c> field equals - <c>ERL_DRV_EXTENDED_MARKER</c>, it should contain <c>0</c> or - driver flags (<c>ERL_DRV_FLAG_*</c>) ored bitwise. Currently - the following driver flags exist: + <p>This field is used to pass driver capability and other + information to the runtime system. If the + <c>extended_marker</c> field equals <c>ERL_DRV_EXTENDED_MARKER</c>, + it should contain <c>0</c> or driver flags (<c>ERL_DRV_FLAG_*</c>) + ored bitwise. Currently the following driver flags exist: </p> <taglist> <tag><c>ERL_DRV_FLAG_USE_PORT_LOCKING</c></tag> @@ -427,6 +427,12 @@ typedef struct erl_drv_entry { by the Erlang distribution (the behaviour has always been required by drivers used by the distribution). </item> + <tag><c>ERL_DRV_FLAG_NO_BUSY_MSGQ</c></tag> + <item>Disable busy port message queue functionality. For + more information, see the documentation of the + <seealso marker="erl_driver#erl_drv_busy_msgq_limits">erl_drv_busy_msgq_limits()</seealso> + function. + </item> </taglist> </item> <tag>void *handle2</tag> diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index 7cdb3a4dfe..99f2466d79 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -582,13 +582,71 @@ <seealso marker="erts_alloc">erts_alloc(3)</seealso> for further information.</p> </item> + <tag><c><![CDATA[+n Behavior]]></c></tag> + <item> + <p>Control behavior of signals to ports.</p> + <p>As of OTP-R16 signals to ports are truly asynchronously + delivered. Note that signals always have been documented as + asynchronous. The underlying implementation has, however, + previously delivered these signals synchronously. Correctly + written Erlang programs should be able to handle this without + any issues. Bugs in existing Erlang programs that make false + assumptions about signals to ports may, however, be tricky to + find. This switch has been introduced in order to at least + make it easier to compare behaviors during a transition period. + Note that <em>this flag is deprecated</em> as of its + introduction, and is scheduled for removal in OTP-R17. + <c>Behavior</c> should be one of the following characters:</p> + <taglist> + <tag><c>d</c></tag> + <item>The default. Asynchronous signals. A process that sends + a signal to a port may continue execution before the signal + has been delivered to the port.</item> + <tag><c>s</c></tag> + <item>Synchronous signals. A processes that sends a signal + to a port will not continue execution until the signal has + been delivered. Should <em>only</em> be used for testing and + debugging.</item> + <tag><c>a</c></tag> + <item>Asynchronous signals. As the default, but a processes + that sends a signal will even more frequently continue + execution before the signal has been delivered to the + port. Should <em>only</em> be used for testing and + debugging.</item> + </taglist> + </item> <tag><marker id="max_processes"><c><![CDATA[+P Number]]></c></marker></tag> <item> - <p>Sets the maximum number of concurrent processes for this - system. <c><![CDATA[Number]]></c> must be in the range 16..134217727. - Default is 32768.</p> - <p><em>NOTE</em>: It is possible to choose any value in the range - but powers of 2 perform best.</p> + <p>Sets the maximum number of simultaneously existing processes for this + system. Valid range for <c>Number</c> is <c>[1024-134217727]</c></p> + <p><em>NOTE</em>: The actual maximum chosen may be much larger than + the <c>Number</c> passed. Currently the runtime system often, + but not always, chooses a value that is a power of 2. This might, + however, be changed in the future. The actual value chosen can be + checked by calling + <seealso marker="erlang#system_info_process_limit">erlang:system_info(process_limit)</seealso>.</p> + <p>The default value is <c>262144</c></p> + </item> + <tag><marker id="max_ports"><c><![CDATA[+Q Number]]></c></marker></tag> + <item> + <p>Sets the maximum number of simultaneously existing ports for this + system. Valid range for <c>Number</c> is <c>[1024-134217727]</c></p> + <p><em>NOTE</em>: The actual maximum chosen may be much larger than + the actual <c>Number</c> passed. Currently the runtime system often, + but not always, chooses a value that is a power of 2. This might, + however, be changed in the future. The actual value chosen can be + checked by calling + <seealso marker="erlang#system_info_port_limit">erlang:system_info(port_limit)</seealso>.</p> + <p>The default value used is normally <c>65536</c>. However, if + the runtime system is able to determine maximum amount of file + descriptors that it is allowed to open and this value is larger + than <c>65536</c>, the chosen value will increased to a value + larger or equal to the maximum amount of file descriptors that + can be opened.</p> + <p>Previously the environment variable <c>ERL_MAX_PORTS</c> was used + for setting the maximum number of simultaneously existing ports. This + environment variable is deprecated, and scheduled for removal in + OTP-R17, but can still be used.</p> </item> <tag><marker id="compat_rel"><c><![CDATA[+R ReleaseNumber]]></c></marker></tag> <item> @@ -597,21 +655,14 @@ default. This flags sets the emulator in compatibility mode with an earlier Erlang/OTP release <c><![CDATA[ReleaseNumber]]></c>. The release number must be in the range - <c><![CDATA[7..<current release>]]></c>. This limits the emulator, - making it possible for it to communicate with Erlang nodes - (as well as C- and Java nodes) running that earlier release.</p> - <p>For example, an R10 node is not automatically compatible - with an R9 node, but R10 nodes started with the <c><![CDATA[+R 9]]></c> - flag can co-exist with R9 nodes in the same distributed - Erlang system, they are R9-compatible.</p> + <c><![CDATA[<current release>-2..<current release>]]></c>. This + limits the emulator, making it possible for it to communicate + with Erlang nodes (as well as C- and Java nodes) running that + earlier release.</p> <p>Note: Make sure all nodes (Erlang-, C-, and Java nodes) of a distributed Erlang system is of the same Erlang/OTP release, or from two different Erlang/OTP releases X and Y, where <em>all</em> Y nodes have compatibility mode X.</p> - <p>For example: A distributed Erlang system can consist of - R10 nodes, or of R9 nodes and R9-compatible R10 nodes, but - not of R9 nodes, R9-compatible R10 nodes and "regular" R10 - nodes, as R9 and "regular" R10 nodes are not compatible.</p> </item> <tag><c><![CDATA[+r]]></c></tag> <item> @@ -934,6 +985,22 @@ without prior notice. </p> </item> + <tag><marker id="+spp"><c>+spp Bool</c></marker></tag> + <item> + <p>Set default scheduler hint for port parallelism. If set to + <c>true</c>, the VM will schedule port tasks when it by this can + improve the parallelism in the system. If set to <c>false</c>, + the VM will try to perform port tasks immediately and by this + improve latency at the expense of parallelism. If this + flag has not been passed, the default scheduler hint for port + parallelism is currently <c>false</c>. The default used can be + inspected in runtime by calling + <seealso marker="erlang#system_info_port_parallelism">erlang:system_info(port_parallelism)</seealso>. + The default can be overriden on port creation by passing the + <seealso marker="erlang#open_port_parallelism">parallelism</seealso> + option to + <seealso marker="erlang#open_port/2">open_port/2</seealso></p>. + </item> <tag><marker id="sched_thread_stack_size"><c><![CDATA[+sss size]]></c></marker></tag> <item> <p>Suggested stack size, in kilowords, for scheduler threads. @@ -1074,7 +1141,7 @@ </item> </taglist> </item> - <tag><c><![CDATA[ERL_AFLAGS]]></c></tag> + <tag><marker id="ERL_AFLAGS"><c><![CDATA[ERL_AFLAGS]]></c></marker></tag> <item> <p>The content of this environment variable will be added to the beginning of the command line for <c><![CDATA[erl]]></c>.</p> @@ -1084,7 +1151,7 @@ the <c><![CDATA[-extra]]></c> section, i.e. the end of the command line following after an <c><![CDATA[-extra]]></c> flag.</p> </item> - <tag><c><![CDATA[ERL_ZFLAGS]]></c> and <c><![CDATA[ERL_FLAGS]]></c></tag> + <tag><marker id="ERL_ZFLAGS"><c><![CDATA[ERL_ZFLAGS]]></c></marker> and <marker id="ERL_FLAGS"><c><![CDATA[ERL_FLAGS]]></c></marker></tag> <item> <p>The content of these environment variables will be added to the end of the command line for <c><![CDATA[erl]]></c>.</p> diff --git a/erts/doc/src/erl_driver.xml b/erts/doc/src/erl_driver.xml index e16fd744c0..13f42a74a7 100644 --- a/erts/doc/src/erl_driver.xml +++ b/erts/doc/src/erl_driver.xml @@ -145,12 +145,20 @@ different threads. This, however, is not a problem for any function in this API, since the emulator has control over these threads.</p> - <note> - <p>Functions not explicitly documented as thread-safe are - <em>not</em> thread-safe. Also note that some functions + <warning> + <p>Functions not explicitly documented as thread safe are + <em>not</em> thread safe. Also note that some functions are <em>only</em> thread safe when used in a runtime system with SMP support.</p> - </note> + <p>A function not explicitly documented as thread safe may at + some point in time have a thread safe implementation in the + runtime system. Such an implementation may however change to + a thread <em>unsafe</em> implementation at any time <em>without + any notice</em> at all. + </p> + <p><em>Only use functions explicitly documented as thread safe + from arbitrary threads.</em></p> + </warning> <p><marker id="lengthy_work"/> As mentioned in the <seealso marker="#WARNING">warning</seealso> text at the beginning of this document it is of vital importance that a driver callback @@ -1529,16 +1537,81 @@ typedef struct ErlIOVec { </desc> </func> <func> + <name><ret>void</ret><nametext>erl_drv_busy_msgq_limits(ErlDrvPort port, ErlDrvSizeT *low, ErlDrvSizeT *high)</nametext></name> + <fsummary>Set and get limits for busy port message queue</fsummary> + <desc> + <marker id="erl_drv_busy_msgq_limits"></marker> + <p>Sets and gets limits that will be used for controling the + busy state of the port message queue.</p> + <p>The port message queue will be set into a busy + state when the amount of command data queued on the + message queue reaches the <c>high</c> limit. The port + message queue will be set into a not busy state when the + amount of command data queued on the message queue falls + below the <c>low</c> limit. Command data is in this + context data passed to the port using either + <c>Port ! {Owner, {command, Data}}</c>, or + <c>port_command/[2,3]</c>. Note that these limits + only concerns command data that have not yet reached the + port. The <seealso marker="#set_busy_port">busy port</seealso> + feature can be used for data that has reached the port.</p> + + <p>Valid limits are values in the range + <c>[ERL_DRV_BUSY_MSGQ_LIM_MIN, ERL_DRV_BUSY_MSGQ_LIM_MAX]</c>. + Limits will be automatically adjusted to be sane. That is, + the system will adjust values so that the low limit used is + lower or equal to the high limit used. By default the high + limit will be 8 kB and the low limit will be 4 kB.</p> + + <p>By passing a pointer to an integer variable containing + the value <c>ERL_DRV_BUSY_MSGQ_READ_ONLY</c>, currently used + limit will be read and written back to the integer variable. + A new limit can be set by passing a pointer to an integer + variable containing a valid limit. The passed value will be + written to the internal limit. The internal limit will then + be adjusted. After this the adjusted limit will be written + back to the integer variable from which the new value was + read. Values are in bytes.</p> + + <p>The busy message queue feature can be disabled either + by setting the <c>ERL_DRV_FLAG_NO_BUSY_MSGQ</c> + <seealso marker="driver_entry#driver_flags">driver flag</seealso> + in the <seealso marker="driver_entry">driver_entry</seealso> + used by the driver, or by calling this function with + <c>ERL_DRV_BUSY_MSGQ_DISABLED</c> as a limit (either low or + high). When this feature has been disabled it cannot be + enabled again. When reading the limits both of them + will be <c>ERL_DRV_BUSY_MSGQ_DISABLED</c>, if this + feature has been disabled.</p> + + <p>Processes sending command data to the port will be suspended + if either the port is busy or if the port message queue is + busy. Suspended processes will be resumed when neither the + port is busy, nor the port message queue is busy.</p> + + <p>For information about busy port functionality + see the documentation of the + <seealso marker="#set_busy_port">set_busy_port()</seealso> + function.</p> + </desc> + </func> + <func> <name><ret>void</ret><nametext>set_busy_port(ErlDrvPort port, int on)</nametext></name> <fsummary>Signal or unsignal port as busy</fsummary> <desc> <marker id="set_busy_port"></marker> - <p>This function set and resets the busy status of the port. If - <c>on</c> is 1, the port is set to busy, if it's 0 the port - is set to not busy.</p> - <p>When the port is busy, sending to it with <c>Port ! Data</c> - or <c>port_command/2</c>, will block the port owner process, - until the port is signaled as not busy.</p> + <p>This function set and unset the busy state of the port. If + <c>on</c> is non-zero, the port is set to busy, if it's zero the port + is set to not busy. You typically want to combine + this feature with the <seealso marker="#erl_drv_busy_msgq_limits">busy + port message queue</seealso> functionality.</p> + <p>Processes sending command data to the port will be suspended + if either the port is busy or if the port message queue + is busy. Suspended processes will be resumed when neither the + port is busy, nor the port message queue is busy. Command data + is in this context data passed to the port using either + <c>Port ! {Owner, {command, Data}}</c>, or + <c>port_command/[2,3]</c>.</p> <p>If the <seealso marker="driver_entry#driver_flags"><![CDATA[ERL_DRV_FLAG_SOFT_BUSY]]></seealso> has been set in the @@ -1547,6 +1620,10 @@ typedef struct ErlIOVec { <seealso marker="erlang#port_command/3">port_command(Port, Data, [force])</seealso> even though the driver has signaled that it is busy. </p> + <p>For information about busy port message queue functionality + see the documentation of the + <seealso marker="#erl_drv_busy_msgq_limits">erl_drv_busy_msgq_limits()</seealso> + function.</p> </desc> </func> <func> @@ -1607,6 +1684,8 @@ typedef struct ErlIOVec { <desc> <marker id="driver_connected"></marker> <p>This function returns the port owner process.</p> + <p>Note that this function is <em>not</em> thread-safe, not + even when the emulator with SMP support is used.</p> </desc> </func> <func> @@ -1634,22 +1713,32 @@ typedef struct ErlIOVec { <tag><seealso marker="driver_entry#call">call</seealso></tag> <item>Called from <c>erlang:port_call/3</c></item> </taglist> + <p>Note that this function is <em>not</em> thread-safe, not + even when the emulator with SMP support is used.</p> </desc> </func> <func> - <name><ret>int</ret><nametext>driver_output_term(ErlDrvPort port, ErlDrvTermData* term, int n)</nametext></name> + <name><ret>int</ret><nametext>erl_drv_output_term(ErlDrvTermData port, ErlDrvTermData* term, int n)</nametext></name> <fsummary>Send term data from driver to port owner</fsummary> <desc> - <marker id="driver_output_term"></marker> + <marker id="erl_drv_output_term"></marker> <p>This functions sends data in the special driver term - format. This is a fast way to deliver term data from a - driver. It also needs no binary conversion, so the port - owner process receives data as normal Erlang terms.</p> + format to the port owner process. This is a fast way to + deliver term data from a driver. It also needs no binary + conversion, so the port owner process receives data as + normal Erlang terms. The + <seealso marker="#erl_drv_send_term">erl_drv_send_term()</seealso> + functions can be used for sending to any arbitrary process + on the local node.</p> + <note><p>Note that the <c>port</c> parameter is <em>not</em> + an ordinary port handle, but a port handle converted using + <c>driver_mk_port()</c>.</p></note> <p>The <c>term</c> parameter points to an array of <c>ErlDrvTermData</c>, with <c>n</c> elements. This array contains terms described in the driver term format. Every term consists of one to four elements in the array. The - term first has a term type, and then arguments.</p> + term first has a term type, and then arguments. The + <c>port</c> parameter specifies the sending port.</p> <p>Tuple and lists (with the exception of strings, see below), are built in reverse polish notation, so that to build a tuple, the elements are given first, and then the tuple @@ -1701,17 +1790,17 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len ErlDrvPort port = ... ErlDrvTermData spec[] = { ERL_DRV_ATOM, driver_mk_atom("tcp"), - ERL_DRV_PORT, driver_mk_port(port), + ERL_DRV_PORT, driver_mk_port(drvport), ERL_DRV_INT, 100, ERL_DRV_BINARY, bin, 50, 0, ERL_DRV_LIST, 2, ERL_DRV_TUPLE, 3, }; - driver_output_term(port, spec, sizeof(spec) / sizeof(spec[0])); + erl_drv_output_term(driver_mk_port(drvport), spec, sizeof(spec) / sizeof(spec[0])); ]]> </code> <p>Where <c>bin</c> is a driver binary of length at least 50 - and <c>port</c> is a port handle. Note that the <c>ERL_DRV_LIST</c> + and <c>drvport</c> is a port handle. Note that the <c>ERL_DRV_LIST</c> comes after the elements of the list, likewise the <c>ERL_DRV_TUPLE</c>.</p> <p>The term <c>ERL_DRV_STRING_CONS</c> is a way to construct @@ -1732,7 +1821,7 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len ERL_DRV_NIL, ERL_DRV_LIST, 4 }; - driver_output_term(port, spec, sizeof(spec) / sizeof(spec[0])); + erl_drv_output_term(driver_mk_port(drvport), spec, sizeof(spec) / sizeof(spec[0])); ]]></code> <p></p> <code type="none"><![CDATA[ @@ -1742,7 +1831,7 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len ERL_DRV_STRING_CONS, (ErlDrvTermData)"123", 3, ERL_DRV_STRING_CONS, (ErlDrvTermData)"abc", 3, }; - driver_output_term(port, spec, sizeof(spec) / sizeof(spec[0])); + erl_drv_output_term(driver_mk_port(drvport), spec, sizeof(spec) / sizeof(spec[0])); ]]></code> <p>The <c>ERL_DRV_EXT2TERM</c> term type is used for passing a term encoded with the @@ -1762,7 +1851,7 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len ERL_DRV_EXT2TERM, (ErlDrvTermData) binp->orig_bytes, binp->orig_size ERL_DRV_TUPLE, 2, }; - driver_output_term(port, spec, sizeof(spec) / sizeof(spec[0])); + erl_drv_output_term(driver_mk_port(drvport), spec, sizeof(spec) / sizeof(spec[0])); ]]></code> <p>If you want to pass a binary and don't already have the content of the binary in an <c>ErlDrvBinary</c>, you can benefit from using @@ -1778,6 +1867,22 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len <c>ERL_DRV_EXT2TERM</c> term types were introduced in the 5.6 version of erts. </p> + <p>This function is only thread-safe when the emulator with SMP + support is used.</p> + </desc> + </func> + <func> + <name><ret>int</ret><nametext>driver_output_term(ErlDrvPort port, ErlDrvTermData* term, int n)</nametext></name> + <fsummary>Send term data from driver to port owner</fsummary> + <desc> + <marker id="driver_output_term"></marker> + <warning><p><c>driver_output_term()</c> is deferred and will + be removed in the OTP-R17 release. Use + <seealso marker="#erl_drv_send_term">erl_drv_output_term()</seealso> + instead.</p> + </warning> + <p>The parameters <c>term</c> and <c>n</c> do the same thing + as in <seealso marker="#erl_drv_output_term">erl_drv_output_term()</seealso>.</p> <p>Note that this function is <em>not</em> thread-safe, not even when the emulator with SMP support is used.</p> </desc> @@ -1791,6 +1896,8 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len <c>string</c>. The atom is created and won't change, so the return value may be saved and reused, which is faster than looking up the atom several times.</p> + <p>Note that this function is <em>not</em> thread-safe, not + even when the emulator with SMP support is used.</p> </desc> </func> <func> @@ -1799,20 +1906,46 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len <desc> <marker id="driver_mk_port"></marker> <p>This function converts a port handle to the erlang term - format, usable in the <c>driver_output_send</c> function.</p> + format, usable in the <seealso marker="#erl_drv_output_term">erl_drv_output_term()</seealso>, and <seealso marker="#erl_drv_send_term">erl_drv_send_term()</seealso> functions.</p> + <p>Note that this function is <em>not</em> thread-safe, not + even when the emulator with SMP support is used.</p> </desc> </func> <func> - <name><ret>int</ret><nametext>driver_send_term(ErlDrvPort port, ErlDrvTermData receiver, ErlDrvTermData* term, int n)</nametext></name> + <name><ret>int</ret><nametext>erl_drv_send_term(ErlDrvTermData port, ErlDrvTermData receiver, ErlDrvTermData* term, int n)</nametext></name> <fsummary>Send term data to other process than port owner process</fsummary> <desc> - <marker id="driver_send_term"></marker> + <marker id="erl_drv_send_term"></marker> <p>This function is the only way for a driver to send data to <em>other</em> processes than the port owner process. The <c>receiver</c> parameter specifies the process to receive the data.</p> + <note><p>Note that the <c>port</c> parameter is <em>not</em> + an ordinary port handle, but a port handle converted using + <c>driver_mk_port()</c>.</p></note> + <p>The parameters <c>port</c>, <c>term</c> and <c>n</c> do the same thing + as in <seealso marker="#erl_drv_output_term">erl_drv_output_term()</seealso>.</p> + <p>This function is only thread-safe when the emulator with SMP + support is used.</p> + </desc> + </func> + <func> + <name><ret>int</ret><nametext>driver_send_term(ErlDrvPort port, ErlDrvTermData receiver, ErlDrvTermData* term, int n)</nametext></name> + <fsummary>Send term data to other process than port owner process</fsummary> + <desc> + <marker id="driver_send_term"></marker> + <warning><p><c>driver_send_term()</c> is deferred and will + be removed in the OTP-R17 release. Use + <seealso marker="#erl_drv_send_term">erl_drv_send_term()</seealso> + instead.</p> + <p>Also note that parameters of <c>driver_send_term()</c> + cannot be properly checked by the runtime system when + executed by arbitrary threads. This may cause the + <c>driver_send_term()</c> function not to fail when + it should.</p> + </warning> <p>The parameters <c>term</c> and <c>n</c> do the same thing - as in <seealso marker="#driver_output_term">driver_output_term</seealso>.</p> + as in <seealso marker="#erl_drv_output_term">erl_drv_output_term()</seealso>.</p> <p>This function is only thread-safe when the emulator with SMP support is used.</p> </desc> diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index ef0e6fea84..1a97a84b42 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -854,10 +854,10 @@ b</pre> </func> <func> <name name="exit" arity="2"/> - <fsummary>Send an exit signal to a process</fsummary> + <fsummary>Send an exit signal to a process or a port</fsummary> <desc> <p>Sends an exit signal with exit reason <c><anno>Reason</anno></c> to - the process <c><anno>Pid</anno></c>.</p> + the process or port identified by <c><anno>Pid</anno></c>.</p> <p>The following behavior apply if <c><anno>Reason</anno></c> is any term except <c>normal</c> or <c>kill</c>:</p> <p>If <c><anno>Pid</anno></c> is not trapping exits, <c><anno>Pid</anno></c> itself will @@ -1497,13 +1497,6 @@ os_prompt% </pre> applied with <c><anno>Arity</anno></c> number of arguments; otherwise returns <c>false</c>.</p> <p>Allowed in guard tests.</p> - <warning> - <p>Currently, <c>is_function/2</c> will also return - <c>true</c> if the first argument is a tuple fun (a tuple - containing two atoms). In a future release, tuple funs will - no longer be supported and <c>is_function/2</c> will return - <c>false</c> if given a tuple fun.</p> - </warning> </desc> </func> <func> @@ -2767,6 +2760,18 @@ os_prompt% </pre> console window when spawning the port program. (This option has no effect on other platforms.)</p> </item> + <tag><marker id="open_port_parallelism"><c>{parallelism, Boolean}</c></marker></tag> + <item> + <p>Set scheduler hint for port parallelism. If set to <c>true</c>, + the VM will schedule port tasks when it by this can improve the + parallelism in the system. If set to <c>false</c>, the VM will + try to perform port tasks immediately and by this improving the + latency at the expense of parallelism. The default can be set on + system startup by passing the + <seealso marker="erl#+spp">+spp</seealso> command line argument + to <seealso marker="erl">erl(1)</seealso>. + </p> + </item> </taglist> <p>The default is <c>stream</c> for all types of port and <c>use_stdio</c> for spawned ports.</p> @@ -2819,10 +2824,11 @@ os_prompt% </pre> the owning process using signals of the form <c>{'EXIT', Port, PosixCode}</c>. See <c>file(3)</c> for possible values of <c>PosixCode</c>.</p> - <p><marker id="ERL_MAX_PORTS"></marker> - The maximum number of ports that can be open at the same - time is 1024 by default, but can be configured by - the environment variable <c>ERL_MAX_PORTS</c>.</p> + <p>The maximum number of ports that can be open at the same + time can be configured by passing the + <seealso marker="erl#max_ports"><c>+Q</c></seealso> + command line flag to + <seealso marker="erl"><c>erl(1)</c></seealso>.</p> </desc> </func> <func> @@ -2880,10 +2886,10 @@ os_prompt% </pre> <desc> <p>Closes an open port. Roughly the same as <c><anno>Port</anno> ! {self(), close}</c> except for the error behaviour - (see below), and that the port does <em>not</em> reply with - <c>{Port, closed}</c>. Any process may close a port with - <c>port_close/1</c>, not only the port owner (the connected - process).</p> + (see below), being synchronous, and that the port does + <em>not</em> reply with <c>{Port, closed}</c>. Any process may + close a port with <c>port_close/1</c>, not only the port owner + (the connected process).</p> <p>For comparison: <c><anno>Port</anno> ! {self(), close}</c> fails with <c>badarg</c> if <c><anno>Port</anno></c> cannot be sent to (i.e., <c><anno>Port</anno></c> refers neither to a port nor to a process). If @@ -2895,8 +2901,12 @@ os_prompt% </pre> <p>Note that any process can close a port using <c><anno>Port</anno> ! {PortOwner, close}</c> just as if it itself was the port owner, but the reply always goes to the port owner.</p> - <p>In short: <c>port_close(Port)</c> has a cleaner and more - logical behaviour than <c><anno>Port</anno> ! {self(), close}</c>.</p> + <p>As of OTP-R16 <c><anno>Port</anno> ! {PortOwner, close}</c> is truly + asynchronous. Note that this operation has always been + documented as an asynchronous operation, while the underlying + implementation has been synchronous. <c>port_close/1</c> is + however still fully synchronous. This due to its error + behavior.</p> <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not an open port or the registered name of an open port.</p> </desc> @@ -2906,11 +2916,11 @@ os_prompt% </pre> <fsummary>Send data to a port</fsummary> <desc> <p>Sends data to a port. Same as - <c><anno>Port</anno> ! {self(), {command, Data}}</c> except for the error - behaviour (see below). Any process may send data to a port - with <c>port_command/2</c>, not only the port owner - (the connected process).</p> - <p>For comparison: <c><anno>Port</anno> ! {self(), {command, Data}}</c> + <c><anno>Port</anno> ! {PortOwner, {command, Data}}</c> except for the error + behaviour and being synchronous (see below). Any process may + send data to a port with <c>port_command/2</c>, not only the + port owner (the connected process).</p> + <p>For comparison: <c><anno>Port</anno> ! {PortOwner, {command, Data}}</c> fails with <c>badarg</c> if <c><anno>Port</anno></c> cannot be sent to (i.e., <c><anno>Port</anno></c> refers neither to a port nor to a process). If <c><anno>Port</anno></c> is a closed port the data message disappears @@ -2921,11 +2931,14 @@ os_prompt% </pre> <p>Note that any process can send to a port using <c><anno>Port</anno> ! {PortOwner, {command, <anno>Data</anno>}}</c> just as if it itself was the port owner.</p> - <p>In short: <c>port_command(<anno>Port</anno>, <anno>Data</anno>)</c> has a cleaner and - more logical behaviour than - <c><anno>Port</anno> ! {self(), {command, Data}}</c>.</p> <p>If the port is busy, the calling process will be suspended until the port is not busy anymore.</p> + <p>As of OTP-R16 <c><anno>Port</anno> ! {PortOwner, {command, Data}}</c> is + truly asynchronous. Note that this operation has always been + documented as an asynchronous operation, while the underlying + implementation has been synchronous. <c>port_command/2</c> is + however still fully synchronous. This due to its error + behavior.</p> <p>Failures:</p> <taglist> <tag><c>badarg</c></tag> @@ -2999,7 +3012,7 @@ os_prompt% </pre> <fsummary>Set the owner of a port</fsummary> <desc> <p>Sets the port owner (the connected port) to <c><anno>Pid</anno></c>. - Roughly the same as <c><anno>Port</anno> ! {self(), {connect, <anno>Pid</anno>}}</c> + Roughly the same as <c><anno>Port</anno> ! {Owner, {connect, <anno>Pid</anno>}}</c> except for the following:</p> <list type="bulleted"> <item> @@ -3010,6 +3023,9 @@ os_prompt% </pre> <c>{Port,connected}</c>.</p> </item> <item> + <p><c>port_connect/1</c> is synchronous, see below.</p> + </item> + <item> <p>The new port owner gets linked to the port.</p> </item> </list> @@ -3033,11 +3049,14 @@ os_prompt% </pre> <c><anno>Port</anno> ! {PortOwner, {connect, <anno>Pid</anno>}}</c> just as if it itself was the port owner, but the reply always goes to the port owner.</p> - <p>In short: <c>port_connect(<anno>Port</anno>, <anno>Pid</anno>)</c> has a cleaner and - more logical behaviour than - <c><anno>Port</anno> ! {self(),{connect,<anno>Pid</anno>}}</c>.</p> + <p>As of OTP-R16 <c><anno>Port</anno> ! {PortOwner, {connect, <anno>Pid</anno>}}</c> is + truly asynchronous. Note that this operation has always been + documented as an asynchronous operation, while the underlying + implementation has been synchronous. <c>port_connect/2</c> is + however still fully synchronous. This due to its error + behavior.</p> <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not an open port - or the registered name of an open port, or if <c><anno>Pid</anno></c> is + or the registered name of an open port, or if <c>Pid</c> is not an existing local pid.</p> </desc> </func> @@ -3084,71 +3103,187 @@ os_prompt% </pre> </func> <func> <name name="port_info" arity="1"/> - <type name="port_info_result_item"/> <fsummary>Information about a port</fsummary> <desc> <p>Returns a list containing tuples with information about the <c><anno>Port</anno></c>, or <c>undefined</c> if the port is not open. The order of the tuples is not defined, nor are all the tuples mandatory.</p> - <taglist> - <tag><c>{registered_name, <anno>RegName</anno>}</c></tag> - <item> - <p><c><anno>RegName</anno></c> (an atom) is the registered name of - the port. If the port has no registered name, this tuple - is not present in the list.</p> - </item> - <tag><c>{id, <anno>Index</anno>}</c></tag> - <item> - <p><c><anno>Index</anno></c> (an integer) is the internal index of the - port. This index may be used to separate ports.</p> - </item> - <tag><c>{connected, <anno>Pid</anno>}</c></tag> - <item> - <p><c><anno>Pid</anno></c> is the process connected to the port.</p> - </item> - <tag><c>{links, <anno>Pids</anno>}</c></tag> - <item> - <p><c><anno>Pids</anno></c> is a list of pids to which processes the - port is linked.</p> - </item> - <tag><c>{name, <anno>String</anno>}</c></tag> - <item> - <p><c><anno>String</anno></c> is the command name set by - <c>open_port</c>.</p> - </item> - <tag><c>{input, <anno>Bytes</anno>}</c></tag> - <item> - <p><c><anno>Bytes</anno></c> is the total number of bytes read from - the port.</p> - </item> - <tag><c>{output, <anno>Bytes</anno>}</c></tag> - <item> - <p><c><anno>Bytes</anno></c> is the total number of bytes written to - the port.</p> - </item> - <tag><c>{os_pid, <anno>OsPid</anno> | undefined}</c></tag> - <item> - <p><c> <anno>OsPid</anno></c> is the process identifier (or equivalent) of an OS process created with <c>open_port({spawn | spawn_executable, Command}, Options)</c>. If the port is not the result of spawning an OS process, the value is <c>undefined</c>.</p> - </item> - </taglist> - <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local port.</p> + <p>Currently the result will containt information about the + following <c>Item</c>s: <c>registered_name</c> (if the port has + a registered name), <c>id</c>, <c>connected</c>, <c>links</c>, + <c>name</c>, <c>input</c>, and <c>output</c>. For more information + about the different <c>Item</c>s, see + <seealso marker="#port_info/2">port_info/2</seealso>.</p> + <p>Failure: <c>badarg</c> if <c>Port</c> is not a local port + identifier, or an atom.</p> </desc> </func> <func> - <name name="port_info" arity="2"/> - <type name="port_info_item"/> - <type name="port_info_result_item"/> - <fsummary>Information about a port</fsummary> + <name name="port_info" arity="2" clause_i="1"/> + <fsummary>Information about the connected process of a port</fsummary> <desc> - <p>Returns information about <c><anno>Port</anno></c> as specified - by <c><anno>Item</anno></c>, or <c>undefined</c> if the port is not open. - Also, if <c>Item =:= registered_name</c> and the port has no - registered name, <c>[]</c> is returned.</p> - <p>For valid values of <c><anno>Item</anno></c>, and corresponding - values of <c><anno>Result</anno></c>, see - <seealso marker="#port_info/1">erlang:port_info/1</seealso>.</p> - <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local port.</p> + <p><c><anno>Pid</anno></c> is the process identifier of the process + connected to the port.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="2"/> + <fsummary>Information about the internal index of a port</fsummary> + <desc> + <p><c><anno>Index</anno></c> is the internal index of the port. This + index may be used to separate ports.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="3"/> + <fsummary>Information about the input of a port</fsummary> + <desc> + <p><c><anno>Bytes</anno></c> is the total number of bytes + read from the port.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="4"/> + <fsummary>Information about the links of a port</fsummary> + <desc> + <p><c><anno>Pids</anno></c> is a list of the process identifiers + of the processes that the port is linked to.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="5"/> + <fsummary>Information about the locking of a port</fsummary> + <desc> + <p><c><anno>Locking</anno></c> is currently either <c>false</c> + (emulator without SMP support), <c>port_level</c> (port specific + locking), or <c>driver_level</c> (driver specific locking). Note + that these results are highly implementation specific and might + change in the future.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="6"/> + <fsummary>Information about the memory size of a port</fsummary> + <desc> + <p><c><anno>Bytes</anno></c> is the total amount of memory, + in bytes, allocated for this port by the runtime system. Note + that the port itself might have allocated memory which is not + included in <c><anno>Bytes</anno></c>.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="7"/> + <fsummary>Information about the monitors of a port</fsummary> + <desc> + <p><c><anno>Monitors</anno></c> represent processes that this port + is monitoring.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="8"/> + <fsummary>Information about the name of a port</fsummary> + <desc> + <p><c><anno>Name</anno></c> is the command name set by + <seealso marker="#open_port/2">open_port/2</seealso>.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="9"/> + <fsummary>Information about the OS pid of a port</fsummary> + <desc> + <p><c><anno>OsPid</anno></c> is the process identifier (or equivalent) + of an OS process created with + <seealso marker="#open_port/2">open_port({spawn | spawn_executable, + Command}, Options)</seealso>. If the port is not the result of spawning + an OS process, the value is <c>undefined</c>.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="10"/> + <fsummary>Information about the output of a port</fsummary> + <desc> + <p><c><anno>Bytes</anno></c> is the total number of bytes written + to the port from Erlang processes using either + <seealso marker="#port_command/2">port_command/2</seealso>, + <seealso marker="#port_command/3">port_command/3</seealso>, + or <c><anno>Port</anno> ! {Owner, {command, Data}</c>. + </p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="11"/> + <fsummary>Information about the parallelism hint of a port</fsummary> + <desc> + <p><c><anno>Boolean</anno></c> corresponds to the port parallelism + hint being used by this port. For more information see + the <seealso marker="#open_port_parallelism">parallelism</seealso> + option of <seealso marker="#open_port/2">open_port/2</seealso>.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="12"/> + <fsummary>Information about the queue size of a port</fsummary> + <desc> + <p><c><anno>Bytes</anno></c> is the total amount of data, + in bytes, queued by the port using the ERTS driver queue + implementation.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="13"/> + <fsummary>Information about the registered name of a port</fsummary> + <desc> + <p><c><anno>RegisteredName</anno></c> is the registered name of + the port. If the port has no registered name, <c>[]</c> is returned.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> </desc> </func> <func> @@ -3168,7 +3303,10 @@ os_prompt% </pre> <name name="ports" arity="0"/> <fsummary>All open ports</fsummary> <desc> - <p>Returns a list of all ports on the local node.</p> + <p>Returns a list of port identifiers corresponding to all the + ports currently existing on the local node.</p> + + <p>Note that a port that is exiting, exists but is not open.</p> </desc> </func> <func> @@ -5501,19 +5639,40 @@ ok <item> <p>Returns a string containing the OTP release number.</p> </item> + <tag><marker id="system_info_port_parallelism"><c>port_parallelism</c></marker></tag> + <item><p>Returns the default port parallelism scheduling hint used. + For more information see the + <seealso marker="erl#+spp">+spp</seealso> command line argument + of <seealso marker="erl">erl(1)</seealso>.</p></item> + <tag><c>process_count</c></tag> + <item> + <p>Returns the number of ports currently existing at + the local node as an integer. The same value as + <c>length(erlang:ports())</c> returns.</p> + </item> + <tag><marker id="system_info_port_limit"><c>port_limit</c></marker></tag> + <item> + <p>Returns the maximum number of simultaneously existing + ports at the local node as an integer. This limit + can be configured at startup by using the + <seealso marker="erl#max_ports"><c>+Q</c></seealso> + command line flag of + <seealso marker="erl"><c>erl(1)</c></seealso>.</p> + </item> <tag><c>process_count</c></tag> <item> <p>Returns the number of processes currently existing at the local node as an integer. The same value as <c>length(processes())</c> returns.</p> </item> - <tag><c>process_limit</c></tag> + <tag><marker id="system_info_process_limit"><c>process_limit</c></marker></tag> <item> - <p>Returns the maximum number of concurrently existing + <p>Returns the maximum number of simultaneously existing processes at the local node as an integer. This limit - can be configured at startup by using the command line - flag <c>+P</c>, see - <seealso marker="erts:erl#max_processes">erl(1)</seealso>.</p> + can be configured at startup by using the + <seealso marker="erl#max_processes"><c>+P</c></seealso> + command line flag of + <seealso marker="erl"><c>erl(1)</c></seealso>.</p> </item> <tag><c>procs</c></tag> <item> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index cdb72b2b98..e996d3e8e3 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -30,6 +30,179 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 5.9.3.1</title> + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + Create an erl_crash.dump if no heart exists and no + ERL_CRASH_DUMP_SECONDS is set (behaviour changed).</p> + <p> + Don't create an erl_crash.dump if heart do exists and no + ERL_CRASH_DUMP_SECONDS is set (behaviour not changed).</p> + <p> + This changes the behaviour back to the R15B02 default + considering if a beam was running with no heart.</p> + <p> + Own Id: OTP-10602</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 5.9.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix linking in OpenBSD. (Thanks to Matthew Dempsky)</p> + <p> + Own Id: OTP-10395</p> + </item> + <item> + <p> + Fix bug causing fallback atomics to be used even though + healthy gcc atomics or libatomic_ops was detected.</p> + <p> + Own Id: OTP-10418</p> + </item> + <item> + <p> + Ensure 'erl_crash.dump' when asked for it. This will + change erl_crash.dump behaviour.</p> + <p> + * Not setting ERL_CRASH_DUMP_SECONDS will now terminate + beam immediately on a crash without writing a crash dump + file.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to 0 will also terminate + beam immediately on a crash without writing a crash dump + file, i.e. same as not setting ERL_CRASH_DUMP_SECONDS + environment variable.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to a negative value will + let the beam wait indefinitely on the crash dump file + being written.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to a positive value will + let the beam wait that many seconds on the crash dump + file being written.</p> + <p> + A positive value will set an alarm/timeout for restart + both in beam and in heart if heart is running.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-10422 Aux Id: kunagi-250 [161] </p> + </item> + <item> + <p> + Fix bug where MSVRT100.dll was not included in the + windows installer.</p> + <p> + Own Id: OTP-10481</p> + </item> + <item> + <p>In the expression + <c><<Bin/binary,...>></c>, if <c>Bin</c> was + a bitstring with a size not a multiple of 8, either no + exception was generated or an incorrect exception was + generated. (Thanks to Adam Rutkowski for reporting this + bug.)</p> + <p> + Own Id: OTP-10524</p> + </item> + <item> + <p> + The runtime system could crash while scheduling a port + task. The port task was scheduled either due to an + external I/O event being triggered, a driver timeout + being triggered, or data being sent over a distribution + channel.</p> + <p> + Own Id: OTP-10556</p> + </item> + <item> + <p> + <c>erlang:memory(ets)</c> erroneously included the size + of each ETS-table main structure twice.</p> + <p> + Own Id: OTP-10558</p> + </item> + <item> + <p> + Fix compile error in generated file hipe_amd64_bifs.S for + Solaris.</p> + <p> + Own Id: OTP-10577</p> + </item> + <item> + <p> + A faulty spec for process_info/2 could cause false + dialyzer warnings. The spec is corrected.</p> + <p> + Own Id: OTP-10584</p> + </item> + <item> + <p> + In very rare cases, the VM could crash if a garbage + collector was called while executing an appending bit + syntax instruction. The symptom was a core when + reallocating memory in the function erts_bs_append. The + garbage collector bug is now corrected.</p> + <p> + Own Id: OTP-10590</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Improve support for building and testing in embedded ppc + environments.</p> + <p> + Own Id: OTP-10265 Aux Id: kunagi-159 + [daf97f67-5724-4812-a5b6-7e86990133d2-1] </p> + </item> + <item> + <p> + Due to a race condition on Windows, sometimes when + printing to standard output and then immediately + terminating erlang all data would not be printed. The + emulator now waits for all data to be printed before + exiting.</p> + <p> + Own Id: OTP-10325 Aux Id: kunagi-166 + [dd72d0e2-3e76-4a51-8b56-7564e24eecae] </p> + </item> + <item> + <p> + The frequency with which sleeping schedulers are woken + due to outstanding memory deallocation jobs has been + reduced.</p> + <p> + Own Id: OTP-10476 Aux Id: OTP-10162 </p> + </item> + <item> + <p> + Clearer warnings about the dangers of misuse of <seealso + marker="erl_nif#WARNING">native functions</seealso> and + <seealso marker="erl_driver#WARNING">drivers</seealso> + have been added to the documentation.</p> + <p> + Own Id: OTP-10557</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 5.9.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/doc/src/part.xml b/erts/doc/src/part.xml index e27b722721..2c178556d4 100644 --- a/erts/doc/src/part.xml +++ b/erts/doc/src/part.xml @@ -31,6 +31,7 @@ <description> <p>The Erlang Runtime System Application <em>ERTS</em>.</p> </description> + <xi:include href="communication.xml"/> <xi:include href="match_spec.xml"/> <xi:include href="crash_dump.xml"/> <xi:include href="alt_dist.xml"/> diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in index 7e966c81bb..67320697ef 100644 --- a/erts/emulator/Makefile.in +++ b/erts/emulator/Makefile.in @@ -560,7 +560,8 @@ $(PRELOAD_SRC): $(ERL_TOP)/erts/preloaded/ebin/otp_ring0.beam \ $(ERL_TOP)/erts/preloaded/ebin/zlib.beam \ $(ERL_TOP)/erts/preloaded/ebin/prim_zip.beam \ $(ERL_TOP)/erts/preloaded/ebin/erl_prim_loader.beam \ - $(ERL_TOP)/erts/preloaded/ebin/erlang.beam + $(ERL_TOP)/erts/preloaded/ebin/erlang.beam \ + $(ERL_TOP)/erts/preloaded/ebin/erts_internal.beam LANG=C $(PERL) utils/make_preload $(MAKE_PRELOAD_EXTRA) -rc $^ > $@ else PRELOAD_OBJ = $(OBJDIR)/preload.o @@ -572,7 +573,8 @@ $(PRELOAD_SRC): $(ERL_TOP)/erts/preloaded/ebin/otp_ring0.beam \ $(ERL_TOP)/erts/preloaded/ebin/zlib.beam \ $(ERL_TOP)/erts/preloaded/ebin/prim_zip.beam \ $(ERL_TOP)/erts/preloaded/ebin/erl_prim_loader.beam \ - $(ERL_TOP)/erts/preloaded/ebin/erlang.beam + $(ERL_TOP)/erts/preloaded/ebin/erlang.beam \ + $(ERL_TOP)/erts/preloaded/ebin/erts_internal.beam LANG=C $(PERL) utils/make_preload -old $^ > $@ endif @@ -735,7 +737,8 @@ RUN_OBJS = \ $(OBJDIR)/packet_parser.o $(OBJDIR)/safe_hash.o \ $(OBJDIR)/erl_zlib.o $(OBJDIR)/erl_nif.o \ $(OBJDIR)/erl_bif_binary.o $(OBJDIR)/erl_ao_firstfit_alloc.o \ - $(OBJDIR)/erl_thr_queue.o $(OBJDIR)/erl_sched_spec_pre_alloc.o + $(OBJDIR)/erl_thr_queue.o $(OBJDIR)/erl_sched_spec_pre_alloc.o \ + $(OBJDIR)/erl_ptab.o ifeq ($(TARGET),win32) DRV_OBJS = \ diff --git a/erts/emulator/beam/atom.names b/erts/emulator/beam/atom.names index afcbd732df..c47a608215 100644 --- a/erts/emulator/beam/atom.names +++ b/erts/emulator/beam/atom.names @@ -94,6 +94,7 @@ atom asynchronous atom atom atom atom_used atom attributes +atom await_port_send_result atom await_proc_exit atom await_sched_wall_time_modifications atom awaiting_load @@ -152,6 +153,7 @@ atom connection_closed atom cons atom const atom context_switches +atom control atom copy atom cpu atom cpu_timestamp @@ -204,6 +206,7 @@ atom erlang atom ERROR='ERROR' atom error_handler atom error_logger +atom erts_internal atom ets atom ETS_TRANSFER='ETS-TRANSFER' atom event @@ -237,6 +240,7 @@ atom gc_end atom gc_start atom Ge='>=' atom generational +atom get_data atom get_seq_token atom get_tcw atom getenv @@ -408,6 +412,7 @@ atom overlapped_io atom owner atom packet atom packet_size +atom parallelism atom Plus='+' atom pause atom pending @@ -419,12 +424,12 @@ atom pid atom port atom ports atom port_count +atom port_limit atom print atom priority atom private atom process atom processes -atom processes_trap atom processes_used atom process_count atom process_display @@ -434,6 +439,7 @@ atom procs atom profile atom protected atom protection +atom ptab_list_continue atom public atom purify atom quantify @@ -481,6 +487,7 @@ atom sequential_trace_token atom serial atom set atom set_cpu_topology +atom set_data atom set_on_first_link atom set_on_first_spawn atom set_on_link diff --git a/erts/emulator/beam/beam_bif_load.c b/erts/emulator/beam/beam_bif_load.c index 9e4add823d..e0a4f86d2d 100644 --- a/erts/emulator/beam/beam_bif_load.c +++ b/erts/emulator/beam/beam_bif_load.c @@ -438,8 +438,7 @@ check_process_code_2(BIF_ALIST_2) if (is_internal_pid(BIF_ARG_1)) { Eterm res; ErtsCodeIndex code_ix; - if (internal_pid_index(BIF_ARG_1) >= erts_max_processes) - goto error; + code_ix = erts_active_code_ix(); modp = erts_get_module(BIF_ARG_2, code_ix); if (modp == NULL) { /* Doesn't exist. */ diff --git a/erts/emulator/beam/beam_bp.c b/erts/emulator/beam/beam_bp.c index 58e0090a76..2168d410b5 100644 --- a/erts/emulator/beam/beam_bp.c +++ b/erts/emulator/beam/beam_bp.c @@ -714,7 +714,7 @@ erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr* I) IS_TRACED_FL(p, F_TRACE_CALLS)) { int local = !!(bp_flags & ERTS_BPF_LOCAL_TRACE); flags = erts_call_trace(p, ep->code, bp->local_ms, args, - local, &p->tracer_proc); + local, &ERTS_TRACER_PROC(p)); } if (bp_flags & ERTS_BPF_META_TRACE) { Eterm tpid1, tpid2; @@ -800,7 +800,7 @@ erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr* I) } if (flags & MATCH_SET_EXCEPTION_TRACE) { erts_trace_exception(p, ep->code, class, value, - &p->tracer_proc); + &ERTS_TRACER_PROC(p)); } if ((flags & MATCH_SET_RETURN_TO_TRACE) && p->catches > 0) { /* can only happen if(local)*/ @@ -825,7 +825,7 @@ erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr* I) UnUseTmpHeapNoproc(3); if ((flags_meta|flags) & MATCH_SET_EXCEPTION_TRACE) { erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); - p->trace_flags |= F_EXCEPTION_TRACE; + ERTS_TRACE_FLAGS(p) |= F_EXCEPTION_TRACE; erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL_MINOR); } } @@ -835,7 +835,7 @@ erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr* I) } /* MATCH_SET_RETURN_TO_TRACE cannot occur if(meta) */ if (flags & MATCH_SET_RX_TRACE) { - erts_trace_return(p, ep->code, result, &p->tracer_proc); + erts_trace_return(p, ep->code, result, &ERTS_TRACER_PROC(p)); } if (flags & MATCH_SET_RETURN_TO_TRACE) { /* can only happen if(local)*/ @@ -935,7 +935,7 @@ do_call_trace(Process* c_p, BeamInstr* I, Eterm* reg, c_p->cp = (flags & MATCH_SET_EXCEPTION_TRACE) ? beam_exception_trace : beam_return_trace; erts_smp_proc_lock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); - c_p->trace_flags |= F_EXCEPTION_TRACE; + ERTS_TRACE_FLAGS(c_p) |= F_EXCEPTION_TRACE; erts_smp_proc_unlock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); } c_p->stop = E; @@ -974,7 +974,7 @@ erts_trace_time_call(Process* c_p, BeamInstr* I, BpDataTime* bdt) ASSERT(pbt->pc); /* add time to previous code */ bp_time_diff(&sitem, pbt, ms, s, us); - sitem.pid = c_p->id; + sitem.pid = c_p->common.id; sitem.count = 0; /* previous breakpoint */ @@ -997,7 +997,7 @@ erts_trace_time_call(Process* c_p, BeamInstr* I, BpDataTime* bdt) } /* Add count to this code */ - sitem.pid = c_p->id; + sitem.pid = c_p->common.id; sitem.count = 1; sitem.s_time = 0; sitem.us_time = 0; @@ -1055,7 +1055,7 @@ erts_trace_time_return(Process *p, BeamInstr *pc) ASSERT(pbt->pc); bp_time_diff(&sitem, pbt, ms, s, us); - sitem.pid = p->id; + sitem.pid = p->common.id; sitem.count = 0; /* previous breakpoint */ @@ -1386,7 +1386,7 @@ void erts_schedule_time_break(Process *p, Uint schedule) { if (pbdt) { get_sys_now(&ms,&s,&us); bp_time_diff(&sitem, pbt, ms, s, us); - sitem.pid = p->id; + sitem.pid = p->common.id; sitem.count = 0; h = &(pbdt->hash[bp_sched2ix_proc(p)]); diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index 7b1ae624ce..0e9d140908 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -516,7 +516,7 @@ extern int count_instructions; # define Dispatchfun() DispatchMacroFun() #endif -#define Self(R) R = c_p->id +#define Self(R) R = c_p->common.id #define Node(R) R = erts_this_node->sysname #define Arg(N) I[(N)+1] @@ -1074,11 +1074,11 @@ init_emulator(void) void dtrace_drvport_str(ErlDrvPort drvport, char *port_buf) { - Port *port = erts_drvport2port(drvport); + Port *port = erts_drvport2port(drvport, NULL); erts_snprintf(port_buf, DTRACE_TERM_BUF_SIZE, "#Port<%lu.%lu>", - port_channel_no(port->id), - port_number(port->id)); + port_channel_no(port->common.id), + port_number(port->common.id)); } #endif /* @@ -1195,7 +1195,7 @@ void process_main(void) c_p = schedule(c_p, reds_used); ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); #ifdef DEBUG - pid = c_p->id; /* Save for debugging purpouses */ + pid = c_p->common.id; /* Save for debugging purpouses */ #endif ERTS_SMP_REQ_PROC_MAIN_LOCK(c_p); PROCESS_MAIN_CHK_LOCKS(c_p); @@ -1227,7 +1227,7 @@ void process_main(void) reds = c_p->fcalls; if (ERTS_PROC_GET_SAVED_CALLS_BUF(c_p) - && (c_p->trace_flags & F_SENSITIVE) == 0) { + && (ERTS_TRACE_FLAGS(c_p) & F_SENSITIVE) == 0) { neg_o_reds = -reds; FCALLS = REDS_IN(c_p) = 0; } else { @@ -1591,6 +1591,7 @@ void process_main(void) reg[0] = r(0); result = erl_send(c_p, r(0), x(1)); PreFetch(0, next); + ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); ERTS_SMP_REQ_PROC_MAIN_LOCK(c_p); PROCESS_MAIN_CHK_LOCKS(c_p); if (c_p->mbuf || MSO(c_p).overhead >= BIN_VHEAP_SZ(c_p)) { @@ -1866,14 +1867,14 @@ void process_main(void) erts_fprintf(stderr, "Dtrace -> (%T) stop spreading " "tag %T with message %T\r\n", - c_p->id,DT_UTAG(c_p),ERL_MESSAGE_TERM(msgp)); + c_p->common.id,DT_UTAG(c_p),ERL_MESSAGE_TERM(msgp)); #endif } else { #ifdef DTRACE_TAG_HARDDEBUG erts_fprintf(stderr, "Dtrace -> (%T) kill tag %T with " "message %T\r\n", - c_p->id,DT_UTAG(c_p),ERL_MESSAGE_TERM(msgp)); + c_p->common.id,DT_UTAG(c_p),ERL_MESSAGE_TERM(msgp)); #endif DT_UTAG(c_p) = NIL; SEQ_TRACE_TOKEN(c_p) = NIL; @@ -1898,7 +1899,7 @@ void process_main(void) erts_fprintf(stderr, "Dtrace -> (%T) receive tag (%T) " "with message %T\r\n", - c_p->id, DT_UTAG(c_p), ERL_MESSAGE_TERM(msgp)); + c_p->common.id, DT_UTAG(c_p), ERL_MESSAGE_TERM(msgp)); #endif } else { #endif @@ -1914,7 +1915,7 @@ void process_main(void) } msg = ERL_MESSAGE_TERM(msgp); seq_trace_output(SEQ_TRACE_TOKEN(c_p), msg, SEQ_TRACE_RECEIVE, - c_p->id, c_p); + c_p->common.id, c_p); #ifdef USE_VM_PROBES } #endif @@ -2567,6 +2568,7 @@ void process_main(void) reg[0] = r(0); result = (*bf)(c_p, reg, I); ASSERT(!ERTS_PROC_IS_EXITING(c_p) || is_non_value(result)); + ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); ERTS_HOLE_CHECK(c_p); ERTS_SMP_REQ_PROC_MAIN_LOCK(c_p); PROCESS_MAIN_CHK_LOCKS(c_p); @@ -3301,7 +3303,6 @@ void process_main(void) PROCESS_MAIN_CHK_LOCKS(c_p); bif_nif_arity = I[-1]; ERTS_SMP_UNREQ_PROC_MAIN_LOCK(c_p); - ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); ASSERT(!ERTS_PROC_IS_EXITING(c_p)); { @@ -3346,7 +3347,6 @@ void process_main(void) bif_nif_arity = I[-1]; ASSERT(bif_nif_arity <= 3); ERTS_SMP_UNREQ_PROC_MAIN_LOCK(c_p); - ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); reg[0] = r(0); { Eterm (*bf)(Process*, Eterm*, BeamInstr*) = vbf; @@ -5257,7 +5257,7 @@ terminate_proc(Process* c_p, Eterm Value) /* EXF_LOG is a primary exception flag */ if (c_p->freason & EXF_LOG) { erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); - erts_dsprintf(dsbufp, "Error in process %T ", c_p->id); + erts_dsprintf(dsbufp, "Error in process %T ", c_p->common.id); if (erts_is_alive) erts_dsprintf(dsbufp, "on node %T ", erts_this_node->sysname); erts_dsprintf(dsbufp,"with exit value: %0.*T\n", display_items, Value); @@ -6186,7 +6186,7 @@ new_fun(Process* p, Eterm* reg, ErlFunEntry* fe, int num_free) MSO(p).first = (struct erl_off_heap_header*) funp; funp->fe = fe; funp->num_free = num_free; - funp->creator = p->id; + funp->creator = p->common.id; #ifdef HIPE funp->native_address = fe->native_address; #endif diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index 1cdce49eef..c4ff4fe982 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -37,10 +37,13 @@ #include "erl_db_util.h" #include "register.h" #include "erl_thr_progress.h" +#define ERTS_PTAB_WANT_BIF_IMPL__ +#include "erl_ptab.h" static Export* flush_monitor_message_trap = NULL; static Export* set_cpu_topology_trap = NULL; static Export* await_proc_exit_trap = NULL; +static Export* await_port_send_result_trap = NULL; Export* erts_format_cpu_topology_trap = NULL; static Export *await_sched_wall_time_mod_trap; @@ -83,8 +86,10 @@ static int insert_internal_link(Process* p, Eterm rpid) ASSERT(is_internal_pid(rpid)); #ifdef ERTS_SMP - if (IS_TRACED(p) && (p->trace_flags & (F_TRACE_SOL|F_TRACE_SOL1))) + if (IS_TRACED(p) + && (ERTS_TRACE_FLAGS(p) & (F_TRACE_SOL|F_TRACE_SOL1))) { rp_locks = ERTS_PROC_LOCKS_ALL; + } erts_smp_proc_lock(p, ERTS_PROC_LOCK_LINK); #endif @@ -100,27 +105,27 @@ static int insert_internal_link(Process* p, Eterm rpid) } if (p != rp) { - erts_add_link(&(p->nlinks), LINK_PID, rp->id); - erts_add_link(&(rp->nlinks), LINK_PID, p->id); + erts_add_link(&ERTS_P_LINKS(p), LINK_PID, rp->common.id); + erts_add_link(&ERTS_P_LINKS(rp), LINK_PID, p->common.id); - ASSERT(is_nil(p->tracer_proc) - || is_internal_pid(p->tracer_proc) - || is_internal_port(p->tracer_proc)); + ASSERT(is_nil(ERTS_TRACER_PROC(p)) + || is_internal_pid(ERTS_TRACER_PROC(p)) + || is_internal_port(ERTS_TRACER_PROC(p))); if (IS_TRACED(p)) { - if (p->trace_flags & (F_TRACE_SOL|F_TRACE_SOL1)) { - rp->trace_flags |= (p->trace_flags & TRACEE_FLAGS); - rp->tracer_proc = p->tracer_proc; /* maybe steal */ + if (ERTS_TRACE_FLAGS(p) & (F_TRACE_SOL|F_TRACE_SOL1)) { + ERTS_TRACE_FLAGS(rp) |= (ERTS_TRACE_FLAGS(p) & TRACEE_FLAGS); + ERTS_TRACER_PROC(rp) = ERTS_TRACER_PROC(p); /* maybe steal */ - if (p->trace_flags & F_TRACE_SOL1) { /* maybe override */ - rp->trace_flags &= ~(F_TRACE_SOL1 | F_TRACE_SOL); - p->trace_flags &= ~(F_TRACE_SOL1 | F_TRACE_SOL); + if (ERTS_TRACE_FLAGS(p) & F_TRACE_SOL1) { /* maybe override */ + ERTS_TRACE_FLAGS(rp) &= ~(F_TRACE_SOL1 | F_TRACE_SOL); + ERTS_TRACE_FLAGS(p) &= ~(F_TRACE_SOL1 | F_TRACE_SOL); } } } } if (IS_TRACED_FL(rp, F_TRACE_PROCS)) - trace_proc(p, rp, am_getting_linked, p->id); + trace_proc(p, rp, am_getting_linked, p->common.id); if (p == rp) erts_smp_proc_unlock(p, rp_locks & ~ERTS_PROC_LOCK_MAIN); @@ -144,10 +149,6 @@ BIF_RETTYPE link_1(BIF_ALIST_1) /* check that the pid or port which is our argument is OK */ if (is_internal_pid(BIF_ARG_1)) { - if (internal_pid_index(BIF_ARG_1) >= erts_max_processes) { - BIF_ERROR(BIF_P, BADARG); - } - if (insert_internal_link(BIF_P, BIF_ARG_1)) { BIF_RET(am_true); } @@ -157,19 +158,37 @@ BIF_RETTYPE link_1(BIF_ALIST_1) } if (is_internal_port(BIF_ARG_1)) { - Port *pt = erts_id2port(BIF_ARG_1, BIF_P, ERTS_PROC_LOCK_MAIN); - if (!pt) { + int send_link_signal = 0; + Port *prt = erts_port_lookup(BIF_ARG_1, ERTS_PORT_SFLGS_INVALID_LOOKUP); + if (!prt) { goto res_no_proc; } erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK); - if (erts_add_link(&(BIF_P->nlinks), LINK_PID, BIF_ARG_1) >= 0) - erts_add_link(&(pt->nlinks), LINK_PID, BIF_P->id); + if (erts_add_link(&ERTS_P_LINKS(BIF_P), LINK_PID, BIF_ARG_1) >= 0) + send_link_signal = 1; /* else: already linked */ erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK); - erts_smp_port_unlock(pt); + + if (send_link_signal) { + Eterm ref; + Eterm *refp = erts_port_synchronous_ops ? &ref : NULL; + + switch (erts_port_link(BIF_P, prt, BIF_P->common.id, refp)) { + case ERTS_PORT_OP_DROPPED: + case ERTS_PORT_OP_BADARG: + goto res_no_proc; + case ERTS_PORT_OP_SCHEDULED: + if (refp) { + ASSERT(is_internal_ref(ref)); + BIF_TRAP3(await_port_send_result_trap, BIF_P, ref, am_true, am_true); + } + default: + break; + } + } BIF_RET(am_true); } else if (is_external_port(BIF_ARG_1) @@ -182,7 +201,7 @@ BIF_RETTYPE link_1(BIF_ALIST_1) erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK); /* We may earn time by checking first that we're not linked already */ - if (erts_lookup_link(BIF_P->nlinks, BIF_ARG_1) != NULL) { + if (erts_lookup_link(ERTS_P_LINKS(BIF_P), BIF_ARG_1) != NULL) { erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK); BIF_RET(am_true); } @@ -209,10 +228,10 @@ BIF_RETTYPE link_1(BIF_ALIST_1) erts_smp_de_links_lock(dep); - erts_add_link(&(BIF_P->nlinks), LINK_PID, BIF_ARG_1); + erts_add_link(&ERTS_P_LINKS(BIF_P), LINK_PID, BIF_ARG_1); lnk = erts_add_or_lookup_link(&(dep->nlinks), LINK_PID, - BIF_P->id); + BIF_P->common.id); ASSERT(lnk != NULL); erts_add_link(&ERTS_LINK_ROOT(lnk), LINK_PID, BIF_ARG_1); @@ -220,7 +239,7 @@ BIF_RETTYPE link_1(BIF_ALIST_1) erts_smp_de_runlock(dep); erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK); - code = erts_dsig_send_link(&dsd, BIF_P->id, BIF_ARG_1); + code = erts_dsig_send_link(&dsd, BIF_P->common.id, BIF_ARG_1); if (code == ERTS_DSIG_SEND_YIELD) ERTS_BIF_YIELD_RETURN(BIF_P, am_true); BIF_RET(am_true); @@ -289,7 +308,7 @@ remote_demonitor(Process *c_p, DistEntry *dep, Eterm ref, Eterm to) if (dmon) erts_destroy_monitor(dmon); } - mon = erts_remove_monitor(&c_p->monitors, ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(c_p), ref); erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_LINK); res = ERTS_DEMONITOR_TRUE; @@ -298,7 +317,7 @@ remote_demonitor(Process *c_p, DistEntry *dep, Eterm ref, Eterm to) case ERTS_DSIG_PREP_CONNECTED: erts_smp_de_links_lock(dep); - mon = erts_remove_monitor(&c_p->monitors, ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(c_p), ref); dmon = erts_remove_monitor(&dep->monitors, ref); erts_smp_de_links_unlock(dep); erts_smp_de_runlock(dep); @@ -325,7 +344,7 @@ remote_demonitor(Process *c_p, DistEntry *dep, Eterm ref, Eterm to) * the atom is stored there. Yield if necessary. */ code = erts_dsig_send_demonitor(&dsd, - c_p->id, + c_p->common.id, (mon->name != NIL ? mon->name : mon->pid), @@ -387,7 +406,7 @@ static int demonitor(Process *c_p, Eterm ref) goto done; /* Cannot be this monitor's ref */ } - mon = erts_lookup_monitor(c_p->monitors, ref); + mon = erts_lookup_monitor(ERTS_P_MONITORS(c_p), ref); if (!mon) { res = ERTS_DEMONITOR_FALSE; goto done; @@ -426,7 +445,7 @@ static int demonitor(Process *c_p, Eterm ref) to, ERTS_PROC_LOCK_LINK, ERTS_P2P_FLG_ALLOW_OTHER_X); - mon = erts_remove_monitor(&c_p->monitors, ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(c_p), ref); #ifndef ERTS_SMP ASSERT(mon); #else @@ -440,7 +459,7 @@ static int demonitor(Process *c_p, Eterm ref) } if (rp) { ErtsMonitor *rmon; - rmon = erts_remove_monitor(&(rp->monitors), ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), ref); if (rp != c_p) erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); if (rmon != NULL) @@ -582,7 +601,7 @@ local_pid_monitor(Process *p, Eterm target) mon_ref = erts_make_ref(p); ERTS_BIF_PREP_RET(ret, mon_ref); - if (target == p->id) { + if (target == p->common.id) { return ret; } @@ -599,8 +618,8 @@ local_pid_monitor(Process *p, Eterm target) else { ASSERT(rp != p); - erts_add_monitor(&(p->monitors), MON_ORIGIN, mon_ref, target, NIL); - erts_add_monitor(&(rp->monitors), MON_TARGET, mon_ref, p->id, NIL); + erts_add_monitor(&ERTS_P_MONITORS(p), MON_ORIGIN, mon_ref, target, NIL); + erts_add_monitor(&ERTS_P_MONITORS(rp), MON_TARGET, mon_ref, p->common.id, NIL); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); } @@ -635,9 +654,9 @@ local_name_monitor(Process *p, Eterm target_name) UnUseTmpHeap(3,p); } else if (rp != p) { - erts_add_monitor(&(p->monitors), MON_ORIGIN, mon_ref, rp->id, + erts_add_monitor(&ERTS_P_MONITORS(p), MON_ORIGIN, mon_ref, rp->common.id, target_name); - erts_add_monitor(&(rp->monitors), MON_TARGET, mon_ref, p->id, + erts_add_monitor(&ERTS_P_MONITORS(rp), MON_TARGET, mon_ref, p->common.id, target_name); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); } @@ -689,16 +708,16 @@ remote_monitor(Process *p, Eterm bifarg1, Eterm bifarg2, erts_smp_de_links_lock(dep); - erts_add_monitor(&(p->monitors), MON_ORIGIN, mon_ref, p_trgt, + erts_add_monitor(&ERTS_P_MONITORS(p), MON_ORIGIN, mon_ref, p_trgt, p_name); - erts_add_monitor(&(dep->monitors), MON_TARGET, mon_ref, p->id, + erts_add_monitor(&(dep->monitors), MON_TARGET, mon_ref, p->common.id, d_name); erts_smp_de_links_unlock(dep); erts_smp_de_runlock(dep); erts_smp_proc_unlock(p, ERTS_PROC_LOCK_LINK); - code = erts_dsig_send_monitor(&dsd, p->id, target, mon_ref); + code = erts_dsig_send_monitor(&dsd, p->common.id, target, mon_ref); if (code == ERTS_DSIG_SEND_YIELD) ERTS_BIF_PREP_YIELD_RETURN(ret, p, mon_ref); else @@ -941,36 +960,39 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) } if (is_internal_port(BIF_ARG_1)) { - Port *pt = erts_id2port_sflgs(BIF_ARG_1, - BIF_P, - ERTS_PROC_LOCK_MAIN, - ERTS_PORT_SFLGS_DEAD); - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCK_STATUS); #ifdef ERTS_SMP - if (ERTS_PROC_PENDING_EXIT(BIF_P)) { - if (pt) - erts_smp_port_unlock(pt); + if (ERTS_PROC_PENDING_EXIT(BIF_P)) goto handle_pending_exit; - } #endif - l = erts_remove_link(&BIF_P->nlinks, BIF_ARG_1); - - ASSERT(pt || !l); - - if (pt) { - rl = erts_remove_link(&pt->nlinks, BIF_P->id); - erts_smp_port_unlock(pt); - if (rl) - erts_destroy_link(rl); - } + l = erts_remove_link(&ERTS_P_LINKS(BIF_P), BIF_ARG_1); erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCK_STATUS); - if (l) + if (l) { + Port *prt; + erts_destroy_link(l); + /* Send unlink signal */ + prt = erts_port_lookup(BIF_ARG_1, ERTS_PORT_SFLGS_DEAD); + if (prt) { + ErtsPortOpResult res; + Eterm ref; + Eterm *refp = erts_port_synchronous_ops ? &ref : NULL; +#ifdef DEBUG + ref = NIL; +#endif + res = erts_port_unlink(BIF_P, prt, BIF_P->common.id, refp); + + if (refp && res == ERTS_PORT_OP_SCHEDULED) { + ASSERT(is_internal_ref(ref)); + BIF_TRAP3(await_port_send_result_trap, BIF_P, ref, am_true, am_true); + } + } + } + BIF_RET(am_true); } else if (is_external_port(BIF_ARG_1) @@ -993,7 +1015,7 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) if (ERTS_PROC_PENDING_EXIT(BIF_P)) goto handle_pending_exit; #endif - l = erts_remove_link(&BIF_P->nlinks,BIF_ARG_1); + l = erts_remove_link(&ERTS_P_LINKS(BIF_P), BIF_ARG_1); erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCK_STATUS); @@ -1022,8 +1044,8 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) #endif case ERTS_DSIG_PREP_CONNECTED: - erts_remove_dist_link(&dld, BIF_P->id, BIF_ARG_1, dep); - code = erts_dsig_send_unlink(&dsd, BIF_P->id, BIF_ARG_1); + erts_remove_dist_link(&dld, BIF_P->common.id, BIF_ARG_1, dep); + code = erts_dsig_send_unlink(&dsd, BIF_P->common.id, BIF_ARG_1); erts_destroy_dist_link(&dld); if (code == ERTS_DSIG_SEND_YIELD) ERTS_BIF_YIELD_RETURN(BIF_P, am_true); @@ -1037,10 +1059,6 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) /* Internal pid... */ - /* process ok ? */ - if (internal_pid_index(BIF_ARG_1) >= erts_max_processes) - BIF_ERROR(BIF_P, BADARG); - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCK_STATUS); /* get process struct */ @@ -1059,7 +1077,7 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) #endif /* unlink and ignore errors */ - l = erts_remove_link(&BIF_P->nlinks,BIF_ARG_1); + l = erts_remove_link(&ERTS_P_LINKS(BIF_P), BIF_ARG_1); if (l != NULL) erts_destroy_link(l); @@ -1067,12 +1085,12 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) ERTS_SMP_ASSERT_IS_NOT_EXITING(BIF_P); } else { - rl = erts_remove_link(&(rp->nlinks),BIF_P->id); + rl = erts_remove_link(&ERTS_P_LINKS(rp), BIF_P->common.id); if (rl != NULL) erts_destroy_link(rl); if (IS_TRACED_FL(rp, F_TRACE_PROCS) && rl != NULL) { - trace_proc(BIF_P, rp, am_getting_unlinked, BIF_P->id); + trace_proc(BIF_P, rp, am_getting_unlinked, BIF_P->common.id); } if (rp != BIF_P) @@ -1345,15 +1363,28 @@ BIF_RETTYPE exit_2(BIF_ALIST_2) */ if (is_internal_port(BIF_ARG_1)) { - Port *prt; - erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); - prt = erts_id2port(BIF_ARG_1, NULL, 0); + Port *prt = erts_port_lookup(BIF_ARG_1, ERTS_PORT_SFLGS_INVALID_LOOKUP); + if (prt) { - erts_do_exit_port(prt, BIF_P->id, BIF_ARG_2); - erts_port_release(prt); + Eterm ref; + Eterm *refp = erts_port_synchronous_ops ? &ref : NULL; + ErtsPortOpResult res; + +#ifdef DEBUG + ref = NIL; +#endif + + res = erts_port_exit(BIF_P, 0, prt, BIF_P->common.id, BIF_ARG_2, refp); + + ERTS_BIF_CHK_EXITED(BIF_P); + + if (refp && res == ERTS_PORT_OP_SCHEDULED) { + ASSERT(is_internal_ref(ref)); + BIF_TRAP3(await_port_send_result_trap, BIF_P, ref, am_true, am_true); + } + } - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_MAIN); - ERTS_BIF_CHK_EXITED(BIF_P); + BIF_RET(am_true); } else if(is_external_port(BIF_ARG_1) @@ -1379,7 +1410,7 @@ BIF_RETTYPE exit_2(BIF_ALIST_2) case ERTS_DSIG_PREP_NOT_CONNECTED: BIF_TRAP2(dexit_trap, BIF_P, BIF_ARG_1, BIF_ARG_2); case ERTS_DSIG_PREP_CONNECTED: - code = erts_dsig_send_exit2(&dsd, BIF_P->id, BIF_ARG_1, BIF_ARG_2); + code = erts_dsig_send_exit2(&dsd, BIF_P->common.id, BIF_ARG_1, BIF_ARG_2); if (code == ERTS_DSIG_SEND_YIELD) ERTS_BIF_YIELD_RETURN(BIF_P, am_true); BIF_RET(am_true); @@ -1397,9 +1428,7 @@ BIF_RETTYPE exit_2(BIF_ALIST_2) */ ErtsProcLocks rp_locks; - if (internal_pid_index(BIF_ARG_1) >= erts_max_processes) - BIF_ERROR(BIF_P, BADARG); - if (BIF_ARG_1 == BIF_P->id) { + if (BIF_ARG_1 == BIF_P->common.id) { rp_locks = ERTS_PROC_LOCKS_ALL; rp = BIF_P; erts_smp_proc_lock(rp, ERTS_PROC_LOCKS_ALL_MINOR); @@ -1417,7 +1446,7 @@ BIF_RETTYPE exit_2(BIF_ALIST_2) * Send an exit signal. */ erts_send_exit_signal(BIF_P, - BIF_P->id, + BIF_P->common.id, rp, &rp_locks, BIF_ARG_2, @@ -1519,22 +1548,24 @@ BIF_RETTYPE process_flag_2(BIF_ALIST_2) } /* * NOTE: It is important that we check for pending exit signals - * and handle them before flag trap_exit is set to true. - * For more info, see implementation of erts_send_exit_signal(). + * and handle them before returning if trap_exit is set to + * true. For more info, see implementation of + * erts_send_exit_signal(). */ - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_STATUS); - ERTS_SMP_LC_ASSERT((ERTS_PROC_LOCK_MAIN|ERTS_PROC_LOCK_STATUS) - & erts_proc_lc_my_proc_locks(BIF_P)); - ERTS_SMP_BIF_CHK_PENDING_EXIT(BIF_P, - ERTS_PROC_LOCK_MAIN|ERTS_PROC_LOCK_STATUS); if (trap_exit) - state = erts_smp_atomic32_read_bor_nob(&BIF_P->state, - ERTS_PSFLG_TRAP_EXIT); + state = erts_smp_atomic32_read_bor_mb(&BIF_P->state, + ERTS_PSFLG_TRAP_EXIT); else - state = erts_smp_atomic32_read_band_nob(&BIF_P->state, - ~ERTS_PSFLG_TRAP_EXIT); + state = erts_smp_atomic32_read_band_mb(&BIF_P->state, + ~ERTS_PSFLG_TRAP_EXIT); +#ifdef ERTS_SMP + if (ERTS_PROC_PENDING_EXIT(BIF_P)) { + erts_handle_pending_exit(BIF_P, ERTS_PROC_LOCK_MAIN); + ERTS_BIF_EXITED(BIF_P); + } +#endif + old_value = (state & ERTS_PSFLG_TRAP_EXIT) ? am_true : am_false; - erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_STATUS); BIF_RET(old_value); } else if (BIF_ARG_1 == am_scheduler) { @@ -1617,11 +1648,13 @@ BIF_RETTYPE process_flag_2(BIF_ALIST_2) goto error; } erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCKS_ALL_MINOR); - old_value = BIF_P->trace_flags & F_SENSITIVE ? am_true : am_false; + old_value = (ERTS_TRACE_FLAGS(BIF_P) & F_SENSITIVE + ? am_true + : am_false); if (is_sensitive) { - BIF_P->trace_flags |= F_SENSITIVE; + ERTS_TRACE_FLAGS(BIF_P) |= F_SENSITIVE; } else { - BIF_P->trace_flags &= ~F_SENSITIVE; + ERTS_TRACE_FLAGS(BIF_P) &= ~F_SENSITIVE; } erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCKS_ALL_MINOR); BIF_RET(old_value); @@ -1747,8 +1780,9 @@ ebif_bang_2(BIF_ALIST_2) #define SEND_BADARG (-4) #define SEND_USER_ERROR (-5) #define SEND_INTERNAL_ERROR (-6) +#define SEND_AWAIT_RESULT (-7) -Sint do_send(Process *p, Eterm to, Eterm msg, int suspend); +Sint do_send(Process *p, Eterm to, Eterm msg, int suspend, Eterm *refp); static Sint remote_send(Process *p, DistEntry *dep, Eterm to, Eterm full_to, Eterm msg, int suspend) @@ -1802,7 +1836,7 @@ static Sint remote_send(Process *p, DistEntry *dep, } Sint -do_send(Process *p, Eterm to, Eterm msg, int suspend) { +do_send(Process *p, Eterm to, Eterm msg, int suspend, Eterm *refp) { Eterm portid; Port *pt; Process* rp; @@ -1814,16 +1848,10 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { trace_send(p, to, msg); if (ERTS_PROC_GET_SAVED_CALLS_BUF(p)) save_calls(p, &exp_send); - - if (internal_pid_index(to) >= erts_max_processes) - return SEND_BADARG; - rp = erts_proc_lookup_raw(to); - - if (!rp) { - ERTS_SMP_ASSERT_IS_NOT_EXITING(p); + rp = erts_proc_lookup_raw(to); + if (!rp) return 0; - } } else if (is_external_pid(to)) { dep = external_pid_dist_entry(to); if(dep == erts_this_dist_entry) { @@ -1832,7 +1860,7 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { "Discarding message %T from %T to %T in an old " "incarnation (%d) of this node (%d)\n", msg, - p->id, + p->common.id, to, external_pid_creation(to), erts_this_node->creation); @@ -1841,45 +1869,24 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { } return remote_send(p, dep, to, to, msg, suspend); } else if (is_atom(to)) { - - /* Need to virtual schedule out sending process - * because of lock wait. This is only necessary - * for internal port calling but the lock is bundled - * with name lookup. - */ - - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_out); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_inactive); - } - erts_whereis_name(p, ERTS_PROC_LOCK_MAIN, - to, - &rp, 0, 0, - &pt); + Eterm id = erts_whereis_name_to_id(p, to); + + rp = erts_proc_lookup(id); + if (rp) + goto send_message; + pt = erts_port_lookup(id, ERTS_PORT_SFLGS_INVALID_LOOKUP); if (pt) { - portid = pt->id; + portid = id; goto port_common; } - - /* Not a port virtually schedule the process back in */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_in); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_active); - } if (IS_TRACED(p)) trace_send(p, to, msg); if (ERTS_PROC_GET_SAVED_CALLS_BUF(p)) save_calls(p, &exp_send); - if (!rp) { - return SEND_BADARG; - } + return SEND_BADARG; } else if (is_external_port(to) && (external_port_dist_entry(to) == erts_this_dist_entry)) { @@ -1888,50 +1895,56 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { "Discarding message %T from %T to %T in an old " "incarnation (%d) of this node (%d)\n", msg, - p->id, + p->common.id, to, external_port_creation(to), erts_this_node->creation); erts_send_error_to_logger(p->group_leader, dsbufp); return 0; } else if (is_internal_port(to)) { + int ret_val; portid = to; - /* schedule out calling process, waiting for lock*/ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_out); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_inactive); - } - pt = erts_id2port(to, p, ERTS_PROC_LOCK_MAIN); + + pt = erts_port_lookup(portid, ERTS_PORT_SFLGS_INVALID_LOOKUP); + port_common: - ERTS_SMP_LC_ASSERT(!pt || erts_lc_is_port_locked(pt)); + ret_val = 0; - /* We have waited for locks, trace schedule ports */ - if (pt && IS_TRACED_FL(pt, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(pt, am_in, am_command); - } - if (pt && erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(pt)) { - profile_runnable_port(pt, am_active); - } - - /* XXX let port_command handle the busy stuff !!! */ - if (pt && (pt->status & ERTS_PORT_SFLG_PORT_BUSY)) { - if (suspend) { - erts_suspend(p, ERTS_PROC_LOCK_MAIN, pt); - if (erts_system_monitor_flags.busy_port) { - monitor_generic(p, am_busy_port, portid); + if (pt) { + int ps_flags = suspend ? 0 : ERTS_PORT_SIG_FLG_NOSUSPEND; + *refp = NIL; + + switch (erts_port_command(p, ps_flags, pt, msg, refp)) { + case ERTS_PORT_OP_CALLER_EXIT: + /* We are exiting... */ + return SEND_USER_ERROR; + case ERTS_PORT_OP_BUSY: + /* Nothing has been sent */ + if (suspend) + erts_suspend(p, ERTS_PROC_LOCK_MAIN, pt); + return SEND_YIELD; + case ERTS_PORT_OP_BUSY_SCHEDULED: + /* Message was sent */ + if (suspend) { + erts_suspend(p, ERTS_PROC_LOCK_MAIN, pt); + ret_val = SEND_YIELD_RETURN; + break; } + /* Fall through */ + case ERTS_PORT_OP_SCHEDULED: + if (is_not_nil(*refp)) { + ASSERT(is_internal_ref(*refp)); + ret_val = SEND_AWAIT_RESULT; + } + break; + case ERTS_PORT_OP_DROPPED: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DONE: + break; + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_command() result"); + break; } - /* Virtually schedule out the port before releasing */ - if (IS_TRACED_FL(pt, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(pt, am_out, am_command); - } - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(pt)) { - profile_runnable_port(pt, am_inactive); - } - erts_port_release(pt); - return SEND_YIELD; } if (IS_TRACED(p)) /* trace once only !! */ @@ -1949,30 +1962,11 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { SEQ_TRACE_SEND, portid, p); } - /* XXX NO GC in port command */ - erts_port_command(p, p->id, pt, msg); - if (pt) { - /* Virtually schedule out the port before releasing */ - if (IS_TRACED_FL(pt, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(pt, am_out, am_command); - } - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(pt)) { - profile_runnable_port(pt, am_inactive); - } - erts_port_release(pt); - } - /* Virtually schedule in process */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_in); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_active); - } if (ERTS_PROC_IS_EXITING(p)) { KILL_CATCHES(p); /* Must exit */ return SEND_USER_ERROR; } - return 0; + return ret_val; } else if (is_tuple(to)) { /* Remote send */ int ret; tp = tuple_val(to); @@ -1988,47 +1982,24 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { dep = erts_sysname_to_connected_dist_entry(tp[2]); if (dep == erts_this_dist_entry) { + Eterm id; erts_deref_dist_entry(dep); if (IS_TRACED(p)) trace_send(p, to, msg); if (ERTS_PROC_GET_SAVED_CALLS_BUF(p)) save_calls(p, &exp_send); - - /* Need to virtual schedule out sending process - * because of lock wait. This is only necessary - * for internal port calling but the lock is bundled. - */ - - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_out); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_inactive); - } - erts_whereis_name(p, ERTS_PROC_LOCK_MAIN, - tp[1], - &rp, 0, 0, - &pt); + id = erts_whereis_name_to_id(p, tp[1]); + + rp = erts_proc_lookup_raw(id); + if (rp) + goto send_message; + pt = erts_port_lookup(id, ERTS_PORT_SFLGS_INVALID_LOOKUP); if (pt) { - portid = pt->id; + portid = id; goto port_common; } - /* Port lookup failed, virtually schedule the process - * back in. - */ - - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_in); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_active); - } - - if (!rp) { - return 0; - } - goto send_message; + return 0; } ret = remote_send(p, dep, tp[1], to, msg, suspend); @@ -2067,6 +2038,7 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { BIF_RETTYPE send_3(BIF_ALIST_3) { + Eterm ref; Process *p = BIF_P; Eterm to = BIF_ARG_1; Eterm msg = BIF_ARG_2; @@ -2090,12 +2062,18 @@ BIF_RETTYPE send_3(BIF_ALIST_3) if(!is_nil(l)) { BIF_ERROR(p, BADARG); } - - result = do_send(p, to, msg, suspend); + +#ifdef DEBUG + ref = NIL; +#endif + + result = do_send(p, to, msg, suspend, &ref); if (result > 0) { ERTS_VBUMP_REDS(p, result); BIF_RET(am_ok); - } else switch (result) { + } + + switch (result) { case 0: BIF_RET(am_ok); break; @@ -2118,6 +2096,9 @@ BIF_RETTYPE send_3(BIF_ALIST_3) ERTS_BIF_YIELD_RETURN(p, am_ok); else BIF_RET(am_nosuspend); + case SEND_AWAIT_RESULT: + ASSERT(is_internal_ref(ref)); + BIF_TRAP3(await_port_send_result_trap, p, ref, am_nosuspend, am_ok); case SEND_BADARG: BIF_ERROR(p, BADARG); break; @@ -2142,12 +2123,21 @@ BIF_RETTYPE send_2(BIF_ALIST_2) Eterm erl_send(Process *p, Eterm to, Eterm msg) { - Sint result = do_send(p, to, msg, !0); + Eterm ref; + Sint result; + +#ifdef DEBUG + ref = NIL; +#endif + + result = do_send(p, to, msg, !0, &ref); if (result > 0) { ERTS_VBUMP_REDS(p, result); BIF_RET(msg); - } else switch (result) { + } + + switch (result) { case 0: BIF_RET(msg); break; @@ -2159,6 +2149,9 @@ Eterm erl_send(Process *p, Eterm to, Eterm msg) break; case SEND_YIELD_RETURN: ERTS_BIF_YIELD_RETURN(p, msg); + case SEND_AWAIT_RESULT: + ASSERT(is_internal_ref(ref)); + BIF_TRAP3(await_port_send_result_trap, p, ref, msg, msg); case SEND_BADARG: BIF_ERROR(p, BADARG); break; @@ -2428,9 +2421,7 @@ BIF_RETTYPE setelement_3(BIF_ALIST_3) /* copy the tuple */ resp = hp; - while (size--) { /* XXX use memcpy? */ - *hp++ = *ptr++; - } + sys_memcpy(hp, ptr, sizeof(Eterm)*size); resp[ix] = BIF_ARG_3; BIF_RET(make_tuple(resp)); } @@ -2443,7 +2434,7 @@ BIF_RETTYPE make_tuple_2(BIF_ALIST_2) Eterm* hp; Eterm res; - if (is_not_small(BIF_ARG_1) || (n = signed_val(BIF_ARG_1)) < 0) { + if (is_not_small(BIF_ARG_1) || (n = signed_val(BIF_ARG_1)) < 0 || n > ERTS_MAX_TUPLE_SIZE) { BIF_ERROR(BIF_P, BADARG); } hp = HAlloc(BIF_P, n+1); @@ -2464,7 +2455,7 @@ BIF_RETTYPE make_tuple_3(BIF_ALIST_3) Eterm list = BIF_ARG_3; Eterm* tup; - if (is_not_small(BIF_ARG_1) || (n = signed_val(BIF_ARG_1)) < 0) { + if (is_not_small(BIF_ARG_1) || (n = signed_val(BIF_ARG_1)) < 0 || n > ERTS_MAX_TUPLE_SIZE) { error: BIF_ERROR(BIF_P, BADARG); } @@ -2516,11 +2507,16 @@ BIF_RETTYPE append_element_2(BIF_ALIST_2) Eterm res; if (is_not_tuple(BIF_ARG_1)) { + error: BIF_ERROR(BIF_P, BADARG); } - ptr = tuple_val(BIF_ARG_1); + ptr = tuple_val(BIF_ARG_1); arity = arityval(*ptr); - hp = HAlloc(BIF_P, arity + 2); + + if (arity + 1 > ERTS_MAX_TUPLE_SIZE) + goto error; + + hp = HAlloc(BIF_P, arity + 2); res = make_tuple(hp); *hp = make_arityval(arity+1); while (arity--) { @@ -3104,7 +3100,7 @@ BIF_RETTYPE list_to_tuple_1(BIF_ALIST_1) Eterm* hp; int len; - if ((len = list_length(list)) < 0) { + if ((len = list_length(list)) < 0 || len > ERTS_MAX_TUPLE_SIZE) { BIF_ERROR(BIF_P, BADARG); } @@ -3126,7 +3122,7 @@ BIF_RETTYPE list_to_tuple_1(BIF_ALIST_1) BIF_RETTYPE self_0(BIF_ALIST_0) { - BIF_RET(BIF_P->id); + BIF_RET(BIF_P->common.id); } /**********************************************************************/ @@ -3163,11 +3159,9 @@ static erts_smp_spinlock_t make_ref_lock; static erts_smp_mtx_t ports_snapshot_mtx; erts_smp_atomic_t erts_dead_ports_ptr; /* To store dying ports during snapshot */ -Eterm erts_make_ref_in_buffer(Eterm buffer[REF_THING_SIZE]) +void +erts_make_ref_in_array(Uint32 ref[ERTS_MAX_REF_NUMBERS]) { - Eterm* hp = buffer; - Uint32 ref0, ref1, ref2; - erts_smp_spin_lock(&make_ref_lock); reference0++; @@ -3179,24 +3173,36 @@ Eterm erts_make_ref_in_buffer(Eterm buffer[REF_THING_SIZE]) } } - ref0 = reference0; - ref1 = reference1; - ref2 = reference2; + ref[0] = reference0; + ref[1] = reference1; + ref[2] = reference2; erts_smp_spin_unlock(&make_ref_lock); +} - write_ref_thing(hp, ref0, ref1, ref2); +Eterm erts_make_ref_in_buffer(Eterm buffer[REF_THING_SIZE]) +{ + Eterm* hp = buffer; + Uint32 ref[ERTS_MAX_REF_NUMBERS]; + + erts_make_ref_in_array(ref); + write_ref_thing(hp, ref[0], ref[1], ref[2]); return make_internal_ref(hp); } Eterm erts_make_ref(Process *p) { Eterm* hp; + Uint32 ref[ERTS_MAX_REF_NUMBERS]; ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_MAIN & erts_proc_lc_my_proc_locks(p)); hp = HAlloc(p, REF_THING_SIZE); - return erts_make_ref_in_buffer(hp); + + erts_make_ref_in_array(ref); + write_ref_thing(hp, ref[0], ref[1], ref[2]); + + return make_internal_ref(hp); } BIF_RETTYPE make_ref_0(BIF_ALIST_0) @@ -3460,7 +3466,7 @@ BIF_RETTYPE garbage_collect_1(BIF_ALIST_1) BIF_ERROR(BIF_P, BADARG); } - if (BIF_P->id == BIF_ARG_1) + if (BIF_P->common.id == BIF_ARG_1) rp = BIF_P; else { #ifdef ERTS_SMP @@ -3500,71 +3506,23 @@ BIF_RETTYPE garbage_collect_0(BIF_ALIST_0) } /**********************************************************************/ -/* Return a list of active ports */ +/* + * The erlang:processes/0 BIF. + */ -BIF_RETTYPE ports_0(BIF_ALIST_0) +BIF_RETTYPE processes_0(BIF_ALIST_0) { - Eterm res = NIL; - Eterm* port_buf = erts_alloc(ERTS_ALC_T_TMP, - sizeof(Eterm)*erts_max_ports); - Eterm* pp = port_buf; - Eterm* dead_ports; - int alive, dead; - Uint32 next_ss; - int i; - - /* To get a consistent snapshot... - * We add alive ports from start of the buffer - * while dying ports are added from the other end by the killing threads. - */ - - erts_smp_mtx_lock(&ports_snapshot_mtx); /* One snapshot at a time */ - - erts_smp_atomic_set_nob(&erts_dead_ports_ptr, - (erts_aint_t) (port_buf + erts_max_ports)); - - next_ss = erts_smp_atomic32_inc_read_relb(&erts_ports_snapshot); - - for (i = erts_max_ports-1; i >= 0; i--) { - Port* prt = &erts_port[i]; - erts_smp_port_state_lock(prt); - if (!(prt->status & ERTS_PORT_SFLGS_DEAD) - && prt->snapshot != next_ss) { - ASSERT(prt->snapshot == next_ss - 1); - *pp++ = prt->id; - prt->snapshot = next_ss; /* Consumed by this snapshot */ - } - erts_smp_port_state_unlock(prt); - } - - dead_ports = (Eterm*)erts_smp_atomic_xchg_nob(&erts_dead_ports_ptr, - (erts_aint_t) NULL); - erts_smp_mtx_unlock(&ports_snapshot_mtx); - - ASSERT(pp <= dead_ports); - - alive = pp - port_buf; - dead = port_buf + erts_max_ports - dead_ports; - - ASSERT((alive+dead) <= erts_max_ports); - - if (alive+dead > 0) { - erts_aint_t i; - Eterm *hp = HAlloc(BIF_P, (alive+dead)*2); - - for (i = 0; i < alive; i++) { - res = CONS(hp, port_buf[i], res); - hp += 2; - } - for (i = 0; i < dead; i++) { - res = CONS(hp, dead_ports[i], res); - hp += 2; - } - } + return erts_ptab_list(BIF_P, &erts_proc); +} - erts_free(ERTS_ALC_T_TMP, port_buf); +/**********************************************************************/ +/* + * The erlang:ports/0 BIF. + */ - BIF_RET(res); +BIF_RETTYPE ports_0(BIF_ALIST_0) +{ + return erts_ptab_list(BIF_P, &erts_port); } /**********************************************************************/ @@ -4194,12 +4152,13 @@ BIF_RETTYPE system_flag_2(BIF_ALIST_2) BIF_RET(old_value); } } else if (BIF_ARG_1 == make_small(1)) { - Uint i; + int i, max; ErlMessage* mp; erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); erts_smp_thr_progress_block(); - for (i = 0; i < erts_max_processes; i++) { + max = erts_ptab_max(&erts_proc); + for (i = 0; i < max; i++) { Process *p = erts_pix2proc(i); if (p) { #ifdef USE_VM_PROBES @@ -4546,6 +4505,8 @@ void erts_init_bif(void) am_format_cpu_topology, 1); await_proc_exit_trap = erts_export_put(am_erlang,am_await_proc_exit,3); + await_port_send_result_trap + = erts_export_put(am_erts_internal, am_await_port_send_result, 3); await_sched_wall_time_mod_trap = erts_export_put(am_erlang, am_await_sched_wall_time_modifications, 2); erts_smp_atomic32_init_nob(&sched_wall_time, 0); @@ -4561,19 +4522,18 @@ bif erlang:send_to_logger/2 BIF_RETTYPE send_to_logger_2(BIF_ALIST_2) { byte *buf; - int len; + ErlDrvSizeT len; if (!is_atom(BIF_ARG_1) || !(is_list(BIF_ARG_2) || is_nil(BIF_ARG_1))) { BIF_ERROR(BIF_P,BADARG); } - len = io_list_len(BIF_ARG_2); - if (len < 0) + if (erts_iolist_size(BIF_ARG_2, &len) != 0) BIF_ERROR(BIF_P,BADARG); else if (len == 0) buf = ""; else { #ifdef DEBUG - int len2; + ErlDrvSizeT len2; #endif buf = (byte *) erts_alloc(ERTS_ALC_T_TMP, len+1); #ifdef DEBUG @@ -4581,7 +4541,7 @@ BIF_RETTYPE send_to_logger_2(BIF_ALIST_2) #else (void) #endif - io_list_to_buf(BIF_ARG_2, buf, len); + erts_iolist_to_buf(BIF_ARG_2, buf, len); ASSERT(len2 == len); buf[len] = '\0'; switch (BIF_ARG_1) { @@ -4675,7 +4635,6 @@ BIF_RETTYPE dt_prepend_vm_tag_data_1(BIF_ALIST_1) #ifdef USE_VM_PROBES Eterm b; Eterm *hp; - hp = HAlloc(BIF_P,2); if (is_binary((DT_UTAG(BIF_P)))) { Uint sz = binary_size(DT_UTAG(BIF_P)); int i; @@ -4692,6 +4651,7 @@ BIF_RETTYPE dt_prepend_vm_tag_data_1(BIF_ALIST_1) } else { b = new_binary(BIF_P,(byte *)"\0",1); } + hp = HAlloc(BIF_P,2); BIF_RET(CONS(hp,b,BIF_ARG_1)); #else BIF_RET(BIF_ARG_1); @@ -4702,7 +4662,6 @@ BIF_RETTYPE dt_append_vm_tag_data_1(BIF_ALIST_1) #ifdef USE_VM_PROBES Eterm b; Eterm *hp; - hp = HAlloc(BIF_P,2); if (is_binary((DT_UTAG(BIF_P)))) { Uint sz = binary_size(DT_UTAG(BIF_P)); int i; @@ -4719,6 +4678,7 @@ BIF_RETTYPE dt_append_vm_tag_data_1(BIF_ALIST_1) } else { b = new_binary(BIF_P,(byte *)"\0",1); } + hp = HAlloc(BIF_P,2); BIF_RET(CONS(hp,BIF_ARG_1,b)); #else BIF_RET(BIF_ARG_1); @@ -4742,14 +4702,14 @@ BIF_RETTYPE dt_spread_tag_1(BIF_ALIST_1) #ifdef DTRACE_TAG_HARDDEBUG erts_fprintf(stderr, "Dtrace -> (%T) start spreading tag %T\r\n", - BIF_P->id,DT_UTAG(BIF_P)); + BIF_P->common.id,DT_UTAG(BIF_P)); #endif } else { DT_UTAG_FLAGS(BIF_P) &= ~DT_UTAG_SPREADING; #ifdef DTRACE_TAG_HARDDEBUG erts_fprintf(stderr, "Dtrace -> (%T) stop spreading tag %T\r\n", - BIF_P->id,DT_UTAG(BIF_P)); + BIF_P->common.id,DT_UTAG(BIF_P)); #endif } } @@ -4775,7 +4735,7 @@ BIF_RETTYPE dt_restore_tag_1(BIF_ALIST_1) #ifdef DTRACE_TAG_HARDDEBUG erts_fprintf(stderr, "Dtrace -> (%T) restore Killing tag!\r\n", - BIF_P->id); + BIF_P->common.id); #endif } DT_UTAG(BIF_P) = NIL; @@ -4792,12 +4752,12 @@ BIF_RETTYPE dt_restore_tag_1(BIF_ALIST_1) erts_fprintf(stderr, "Dtrace -> (%T) restore stop spreading " "tag %T\r\n", - BIF_P->id, tpl[2]); + BIF_P->common.id, tpl[2]); } else if ((x & DT_UTAG_SPREADING) && !(DT_UTAG_FLAGS(BIF_P) & DT_UTAG_SPREADING)) { erts_fprintf(stderr, "Dtrace -> (%T) restore start spreading " - "tag %T\r\n",BIF_P->id,tpl[2]); + "tag %T\r\n",BIF_P->common.id,tpl[2]); } #endif DT_UTAG_FLAGS(BIF_P) = x; diff --git a/erts/emulator/beam/bif.h b/erts/emulator/beam/bif.h index 7cb2c78815..71f232035d 100644 --- a/erts/emulator/beam/bif.h +++ b/erts/emulator/beam/bif.h @@ -322,27 +322,6 @@ do { \ ERTS_BIF_EXITED((PROC)); \ } while (0) -#ifdef ERTS_SMP -#define ERTS_SMP_BIF_CHK_PENDING_EXIT(P, L) \ -do { \ - ERTS_SMP_LC_ASSERT((L) == erts_proc_lc_my_proc_locks((P))); \ - ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_MAIN & (L)); \ - if (!((L) & ERTS_PROC_LOCK_STATUS)) \ - erts_smp_proc_lock((P), ERTS_PROC_LOCK_STATUS); \ - if (ERTS_PROC_PENDING_EXIT((P))) { \ - erts_handle_pending_exit((P), (L)|ERTS_PROC_LOCK_STATUS); \ - erts_smp_proc_unlock((P), \ - (((L)|ERTS_PROC_LOCK_STATUS) \ - & ~ERTS_PROC_LOCK_MAIN)); \ - ERTS_BIF_EXITED((P)); \ - } \ - if (!((L) & ERTS_PROC_LOCK_STATUS)) \ - erts_smp_proc_unlock((P), ERTS_PROC_LOCK_STATUS); \ -} while (0) -#else -#define ERTS_SMP_BIF_CHK_PENDING_EXIT(P, L) -#endif - /* * The ERTS_BIF_*_AWAIT_X_*_TRAP makros either exits the caller, or * sets up a trap to erlang:await_proc_exit/3. diff --git a/erts/emulator/beam/bif.tab b/erts/emulator/beam/bif.tab index f7dad2767f..9a1a25031c 100644 --- a/erts/emulator/beam/bif.tab +++ b/erts/emulator/beam/bif.tab @@ -180,10 +180,6 @@ bif 'erl.lang.port':open/2 ebif_open_port_2 open_port_2 bif erlang:pid_to_list/1 bif 'erl.lang.proc':pid_to_string/1 ebif_pid_to_string_1 pid_to_list_1 -bif erlang:port_info/1 -bif 'erl.lang.port':info/1 ebif_port_info_1 -bif erlang:port_info/2 -bif 'erl.lang.port':info/2 ebif_port_info_2 bif erlang:ports/0 bif 'erl.lang.node':ports/0 ebif_ports_0 bif erlang:pre_loaded/0 @@ -252,24 +248,16 @@ bif erlang:setnode/2 bif erlang:setnode/3 bif erlang:dist_exit/3 -bif erlang:port_call/2 -bif 'erl.lang.port':call/2 ebif_port_call_2 -bif erlang:port_call/3 -bif 'erl.lang.port':call/3 ebif_port_call_3 -bif erlang:port_command/2 -bif 'erl.lang.port':command/2 ebif_port_command_2 -bif erlang:port_command/3 -bif 'erl.lang.port':command/3 ebif_port_command_3 -bif erlang:port_control/3 -bif 'erl.lang.port':control/3 ebif_port_control_3 -bif erlang:port_close/1 -bif 'erl.lang.port':close/1 ebif_port_close_1 -bif erlang:port_connect/2 -bif 'erl.lang.port':connect/2 ebif_port_connect_2 -bif erlang:port_set_data/2 -bif 'erl.lang.port':set_data/2 ebif_port_set_data_2 -bif erlang:port_get_data/1 -bif 'erl.lang.port':get_data/1 ebif_port_get_data_1 +# Static native functions in erts_internal +bif erts_internal:port_info/1 +bif erts_internal:port_info/2 +bif erts_internal:port_call/3 +bif erts_internal:port_command/3 +bif erts_internal:port_control/3 +bif erts_internal:port_close/1 +bif erts_internal:port_connect/2 +bif erts_internal:port_set_data/2 +bif erts_internal:port_get_data/1 # Tracing & debugging. bif erlang:trace_pattern/2 diff --git a/erts/emulator/beam/binary.c b/erts/emulator/beam/binary.c index 3d2725e239..4441dab181 100644 --- a/erts/emulator/beam/binary.c +++ b/erts/emulator/beam/binary.c @@ -355,10 +355,10 @@ BIF_RETTYPE bitstring_to_list_1(BIF_ALIST_1) BIF_RETTYPE erts_list_to_binary_bif(Process *p, Eterm arg) { Eterm bin; - Uint size; + ErlDrvSizeT size; byte* bytes; #ifdef DEBUG - int offset; + ErlDrvSizeT offset; #endif if (is_nil(arg)) { @@ -377,7 +377,7 @@ BIF_RETTYPE erts_list_to_binary_bif(Process *p, Eterm arg) #ifdef DEBUG offset = #endif - io_list_to_buf(arg, (char*) bytes, size); + erts_iolist_to_buf(arg, (char*) bytes, size); ASSERT(offset == 0); BIF_RET(bin); diff --git a/erts/emulator/beam/break.c b/erts/emulator/beam/break.c index 63136d86c9..9aa1e5f30d 100644 --- a/erts/emulator/beam/break.c +++ b/erts/emulator/beam/break.c @@ -61,16 +61,19 @@ extern char* erts_system_version[]; static void port_info(int to, void *to_arg) { - int i; - for (i = 0; i < erts_max_ports; i++) - print_port_info(to, to_arg, i); + int i, max = erts_ptab_max(&erts_port); + for (i = 0; i < max; i++) { + Port *p = erts_pix2port(i); + if (p) + print_port_info(p, to, to_arg); + } } void process_info(int to, void *to_arg) { - int i; - for (i = 0; i < erts_max_processes; i++) { + int i, max = erts_ptab_max(&erts_proc); + for (i = 0; i < max; i++) { Process *p = erts_pix2proc(i); if (p && p->i != ENULL) { if (!ERTS_PROC_IS_EXITING(p)) @@ -84,12 +87,12 @@ process_info(int to, void *to_arg) static void process_killer(void) { - int i, j; + int i, j, max = erts_ptab_max(&erts_proc); Process* rp; erts_printf("\n\nProcess Information\n\n"); erts_printf("--------------------------------------------------\n"); - for (i = erts_max_processes-1; i >= 0; i--) { + for (i = max-1; i >= 0; i--) { rp = erts_pix2proc(i); if (rp && rp->i != ENULL) { int br; @@ -196,7 +199,7 @@ print_process_info(int to, void *to_arg, Process *p) erts_aint32_t state; /* display the PID */ - erts_print(to, to_arg, "=proc:%T\n", p->id); + erts_print(to, to_arg, "=proc:%T\n", p->common.id); /* Display the state */ erts_print(to, to_arg, "State: "); @@ -226,8 +229,8 @@ print_process_info(int to, void *to_arg, Process *p) * If the process is registered as a global process, display the * registered name */ - if (p->reg != NULL) - erts_print(to, to_arg, "Name: %T\n", p->reg->name); + if (p->common.u.alive.reg) + erts_print(to, to_arg, "Name: %T\n", p->common.u.alive.reg->name); /* * Display the initial function name @@ -301,11 +304,11 @@ print_process_info(int to, void *to_arg, Process *p) } /* display the links only if there are any*/ - if (p->nlinks != NULL || p->monitors != NULL) { + if (ERTS_P_LINKS(p) || ERTS_P_MONITORS(p)) { PrintMonitorContext context = {1,to}; erts_print(to, to_arg,"Link list: ["); - erts_doforall_links(p->nlinks, &doit_print_link, &context); - erts_doforall_monitors(p->monitors, &doit_print_monitor, &context); + erts_doforall_links(ERTS_P_LINKS(p), &doit_print_link, &context); + erts_doforall_monitors(ERTS_P_MONITORS(p), &doit_print_monitor, &context); erts_print(to, to_arg,"]\n"); } @@ -625,9 +628,9 @@ bin_check(void) { Process *rp; struct erl_off_heap_header* hdr; - int i, printed = 0; + int i, printed = 0, max = erts_ptab_max(&erts_proc); - for (i=0; i < erts_max_processes; i++) { + for (i=0; i < max; i++) { rp = erts_pix2proc(i); if (!rp) continue; @@ -635,7 +638,7 @@ bin_check(void) if (hdr->thing_word == HEADER_PROC_BIN) { ProcBin *bp = (ProcBin*) hdr; if (!printed) { - erts_printf("Process %T holding binary data \n", rp->id); + erts_printf("Process %T holding binary data \n", rp->common.id); printed = 1; } erts_printf("%p orig_size: %bpd, norefs = %bpd\n", @@ -670,6 +673,7 @@ erl_crash_dump_v(char *file, int line, char* fmt, va_list args) char dumpnamebuf[MAXPATHLEN]; char* dumpname; int secs; + int env_erl_crash_dump_seconds_set = 1; if (ERTS_SOMEONE_IS_CRASH_DUMPING) return; @@ -694,6 +698,8 @@ erl_crash_dump_v(char *file, int line, char* fmt, va_list args) envsz = sizeof(env); /* ERL_CRASH_DUMP_SECONDS not set + * if we have a heart port, break immediately + * otherwise dump crash indefinitely (until crash is complete) * same as ERL_CRASH_DUMP_SECONDS = 0 * - do not write dump * - do not set an alarm @@ -715,8 +721,10 @@ erl_crash_dump_v(char *file, int line, char* fmt, va_list args) */ if (erts_sys_getenv__("ERL_CRASH_DUMP_SECONDS", env, &envsz) != 0) { - return; /* break immediately */ + env_erl_crash_dump_seconds_set = 0; + secs = -1; } else { + env_erl_crash_dump_seconds_set = 1; secs = atoi(env); } @@ -724,7 +732,16 @@ erl_crash_dump_v(char *file, int line, char* fmt, va_list args) return; } - erts_sys_prepare_crash_dump(secs); + /* erts_sys_prepare_crash_dump returns 1 if heart port is found, otherwise 0 + * If we don't find heart (0) and we don't have ERL_CRASH_DUMP_SECONDS set + * we should continue writing a dump + * + * beware: secs -1 means no alarm + */ + + if (erts_sys_prepare_crash_dump(secs) && !env_erl_crash_dump_seconds_set ) { + return; + } if (erts_sys_getenv__("ERL_CRASH_DUMP",&dumpnamebuf[0],&dumpnamebufsize) != 0) dumpname = "erl_crash.dump"; @@ -752,7 +769,7 @@ erl_crash_dump_v(char *file, int line, char* fmt, va_list args) erts_print_nif_taints(fd, NULL); erts_fdprintf(fd, "Atoms: %d\n", atom_table_size()); info(fd, NULL); /* General system info */ - if (erts_proc.tab) + if (erts_ptab_initialized(&erts_proc)) process_info(fd, NULL); /* Info about each process and port */ db_info(fd, NULL, 0); erts_print_bif_timer_info(fd, NULL); diff --git a/erts/emulator/beam/code_ix.h b/erts/emulator/beam/code_ix.h index bc10e956f5..e2bd0da112 100644 --- a/erts/emulator/beam/code_ix.h +++ b/erts/emulator/beam/code_ix.h @@ -18,7 +18,7 @@ */ /* Description: - * This is the interface that facilitate changing the beam code + * This is the interface that facilitates changing the beam code * (load,upgrade,delete) while allowing executing Erlang processes to * access the code without any locks or other expensive memory barriers. * @@ -35,7 +35,7 @@ * The third code index is not explicitly used. It can be thought of as * the "previous active" or the "next staging" index. It is needed to make * sure that we do not reuse a code index for staging until we are sure - * that no executing BIFs are still referring it. + * that no executing BIFs are still referencing it. * We could get by with only two (0 and 1), but that would require that we * must wait for all schedulers to re-schedule before each code change * operation can start staging. diff --git a/erts/emulator/beam/copy.c b/erts/emulator/beam/copy.c index 36eda04de2..23c0fca6aa 100644 --- a/erts/emulator/beam/copy.c +++ b/erts/emulator/beam/copy.c @@ -47,7 +47,7 @@ copy_object(Eterm obj, Process* to) if (DTRACE_ENABLED(copy_object)) { DTRACE_CHARBUF(proc_name, 64); - erts_snprintf(proc_name, sizeof(proc_name), "%T", to->id); + erts_snprintf(proc_name, sizeof(proc_name), "%T", to->common.id); DTRACE2(copy_object, proc_name, size); } #endif diff --git a/erts/emulator/beam/dist.c b/erts/emulator/beam/dist.c index 28c4621ff2..f169a1a34a 100644 --- a/erts/emulator/beam/dist.c +++ b/erts/emulator/beam/dist.c @@ -124,6 +124,13 @@ static void send_nodes_mon_msgs(Process *, Eterm, Eterm, Eterm, Eterm); static void init_nodes_monitors(void); static erts_smp_atomic_t no_caches; +static erts_smp_atomic_t no_nodes; + +struct { + Eterm reason; + ErlHeapFragment *bp; +} nodedown; + static void delete_cache(ErtsAtomCache *cache) @@ -144,7 +151,7 @@ create_cache(DistEntry *dep) ERTS_SMP_LC_ASSERT( is_internal_port(dep->cid) - && erts_lc_is_port_locked(&erts_port[internal_port_index(dep->cid)])); + && erts_lc_is_port_locked(erts_port_lookup_raw(dep->cid))); ASSERT(!dep->cache); dep->cache = cp = (ErtsAtomCache*) erts_alloc(ERTS_ALC_T_DCACHE, @@ -171,11 +178,10 @@ get_suspended_on_de(DistEntry *dep, Uint32 unset_qflgs) return NULL; } else { - ErtsProcList *plp; - plp = dep->suspended.first; - dep->suspended.first = NULL; - dep->suspended.last = NULL; - return plp; + ErtsProcList *suspended = dep->suspended; + dep->suspended = NULL; + erts_proclist_fetch(&suspended, NULL); + return suspended; } } @@ -252,7 +258,7 @@ static void doit_monitor_net_exits(ErtsMonitor *mon, void *vnecp) if (mon->type == MON_ORIGIN) { /* local pid is beeing monitored */ - rmon = erts_remove_monitor(&(rp->monitors),mon->ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), mon->ref); /* ASSERT(rmon != NULL); nope, can happen during process exit */ if (rmon != NULL) { erts_destroy_monitor(rmon); @@ -262,7 +268,7 @@ static void doit_monitor_net_exits(ErtsMonitor *mon, void *vnecp) Eterm watched; UseTmpHeapNoproc(3); ASSERT(mon->type == MON_TARGET); - rmon = erts_remove_monitor(&(rp->monitors),mon->ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), mon->ref); /* ASSERT(rmon != NULL); can happen during process exit */ if (rmon != NULL) { ASSERT(is_atom(rmon->name) || is_nil(rmon->name)); @@ -311,7 +317,7 @@ static void doit_link_net_exits_sub(ErtsLink *sublnk, void *vlnecp) goto done; } - rlnk = erts_remove_link(&(rp->nlinks), sublnk->pid); + rlnk = erts_remove_link(&ERTS_P_LINKS(rp), sublnk->pid); xres = erts_send_exit_signal(NULL, sublnk->pid, rp, @@ -370,7 +376,7 @@ static void doit_node_link_net_exits(ErtsLink *lnk, void *vnecp) if (!rp) { goto done; } - rlnk = erts_remove_link(&(rp->nlinks), name); + rlnk = erts_remove_link(&ERTS_P_LINKS(rp), name); if (rlnk != NULL) { ASSERT(is_atom(rlnk->pid) && (rlnk->type == LINK_NODE)); erts_destroy_link(rlnk); @@ -394,6 +400,47 @@ static void doit_node_link_net_exits(ErtsLink *lnk, void *vnecp) erts_destroy_link(lnk); } +static void +set_node_not_alive(void *unused) +{ + ErlHeapFragment *bp; + Eterm nodename = erts_this_dist_entry->sysname; + + ASSERT(erts_smp_atomic_read_nob(&no_nodes) == 0); + + erts_smp_thr_progress_block(); + erts_set_this_node(am_Noname, 0); + erts_is_alive = 0; + send_nodes_mon_msgs(NULL, am_nodedown, nodename, am_visible, nodedown.reason); + nodedown.reason = NIL; + bp = nodedown.bp; + nodedown.bp = NULL; + erts_smp_thr_progress_unblock(); + if (bp) + free_message_buffer(bp); +} + +static ERTS_INLINE void +dec_no_nodes(void) +{ + erts_aint_t no = erts_smp_atomic_dec_read_mb(&no_nodes); + ASSERT(no >= 0); + ASSERT(erts_get_scheduler_id()); /* Need to be a scheduler */ + if (no == 0) + erts_schedule_misc_aux_work(erts_get_scheduler_id(), + set_node_not_alive, + NULL); +} + +static ERTS_INLINE void +inc_no_nodes(void) +{ +#ifdef DEBUG + erts_aint_t no = erts_smp_atomic_read_nob(&no_nodes); + ASSERT(erts_is_alive ? no > 0 : no == 0); +#endif + erts_smp_atomic_inc_mb(&no_nodes); +} /* * proc is currently running or exiting process. @@ -403,47 +450,76 @@ int erts_do_net_exits(DistEntry *dep, Eterm reason) Eterm nodename; if (dep == erts_this_dist_entry) { /* Net kernel has died (clean up!!) */ + DistEntry *tdep; + int no_dist_port = 0; Eterm nd_reason = (reason == am_no_network ? am_no_network : am_net_kernel_terminated); erts_smp_rwmtx_rwlock(&erts_dist_table_rwmtx); + for (tdep = erts_hidden_dist_entries; tdep; tdep = tdep->next) + no_dist_port++; + for (tdep = erts_visible_dist_entries; tdep; tdep = tdep->next) + no_dist_port++; + /* KILL all port controllers */ - while(erts_visible_dist_entries || erts_hidden_dist_entries) { - DistEntry *tdep; - Eterm prt_id; - Port *prt; - if(erts_hidden_dist_entries) - tdep = erts_hidden_dist_entries; - else - tdep = erts_visible_dist_entries; - prt_id = tdep->cid; - ASSERT(is_internal_port(prt_id)); + if (no_dist_port == 0) erts_smp_rwmtx_rwunlock(&erts_dist_table_rwmtx); + else { + Eterm def_buf[128]; + int i = 0; + Eterm *dist_port; - prt = erts_id2port(prt_id, NULL, 0); - if (prt) { - ASSERT(prt->status & ERTS_PORT_SFLG_DISTRIBUTION); - ASSERT(prt->dist_entry); - /* will call do_net_exists !!! */ - erts_do_exit_port(prt, prt_id, nd_reason); - erts_port_release(prt); + if (no_dist_port <= sizeof(def_buf)/sizeof(def_buf[0])) + dist_port = &def_buf[0]; + else + dist_port = erts_alloc(ERTS_ALC_T_TMP, + sizeof(Eterm)*no_dist_port); + for (tdep = erts_hidden_dist_entries; tdep; tdep = tdep->next) { + ASSERT(is_internal_port(tdep->cid)); + dist_port[i++] = tdep->cid; + } + for (tdep = erts_visible_dist_entries; tdep; tdep = tdep->next) { + ASSERT(is_internal_port(tdep->cid)); + dist_port[i++] = tdep->cid; } + erts_smp_rwmtx_rwunlock(&erts_dist_table_rwmtx); - erts_smp_rwmtx_rwlock(&erts_dist_table_rwmtx); - } + for (i = 0; i < no_dist_port; i++) { + Port *prt = erts_port_lookup(dist_port[i], + ERTS_PORT_SFLGS_INVALID_LOOKUP); + if (!prt) + continue; + ASSERT(erts_atomic32_read_nob(&prt->state) + & ERTS_PORT_SFLG_DISTRIBUTION); - erts_smp_rwmtx_rwunlock(&erts_dist_table_rwmtx); + erts_port_exit(NULL, ERTS_PORT_SIG_FLG_FORCE_SCHED, + prt, dist_port[i], nd_reason, NULL); + } - nodename = erts_this_dist_entry->sysname; - erts_smp_thr_progress_block(); - erts_set_this_node(am_Noname, 0); - erts_is_alive = 0; - send_nodes_mon_msgs(NULL, am_nodedown, nodename, am_visible, nd_reason); - erts_smp_thr_progress_unblock(); + if (dist_port != &def_buf[0]) + erts_free(ERTS_ALC_T_TMP, dist_port); + } + /* + * When last dist port exits, node will be taken + * from alive to not alive. + */ + ASSERT(is_nil(nodedown.reason) && !nodedown.bp); + if (is_immed(nd_reason)) + nodedown.reason = nd_reason; + else { + Eterm *hp; + Uint sz = size_object(nd_reason); + nodedown.bp = new_message_buffer(sz); + hp = nodedown.bp->mem; + nodedown.reason = copy_struct(nd_reason, + sz, + &hp, + &nodedown.bp->off_heap); + } } - else { /* recursive call via erts_do_exit_port() will end up here */ + else { /* Call from distribution port */ NetExitsContext nec = {dep}; ErtsLink *nlinks; ErtsLink *node_links; @@ -454,10 +530,10 @@ int erts_do_net_exits(DistEntry *dep, Eterm reason) erts_smp_de_rwlock(dep); ERTS_SMP_LC_ASSERT(is_internal_port(dep->cid) - && erts_lc_is_port_locked(&erts_port[internal_port_index(dep->cid)])); + && erts_lc_is_port_locked(erts_port_lookup_raw(dep->cid))); if (erts_port_task_is_scheduled(&dep->dist_cmd)) - erts_port_task_abort(dep->cid, &dep->dist_cmd); + erts_port_task_abort(&dep->dist_cmd); if (dep->status & ERTS_DE_SFLG_EXITING) { #ifdef DEBUG @@ -503,6 +579,9 @@ int erts_do_net_exits(DistEntry *dep, Eterm reason) clear_dist_entry(dep); } + + dec_no_nodes(); + return 1; } @@ -516,6 +595,10 @@ void init_dist(void) { init_nodes_monitors(); + nodedown.reason = NIL; + nodedown.bp = NULL; + + erts_smp_atomic_init_nob(&no_nodes, 0); erts_smp_atomic_init_nob(&no_caches, 0); /* Lookup/Install all references to trap functions */ @@ -769,7 +852,7 @@ erts_dsig_send_msg(ErtsDSigData *dsdp, Eterm remote, Eterm message) *node_name = *sender_name = *receiver_name = '\0'; if (DTRACE_ENABLED(message_send) || DTRACE_ENABLED(message_send_remote)) { erts_snprintf(node_name, sizeof(node_name), "%T", dsdp->dep->sysname); - erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->id); + erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->common.id); erts_snprintf(receiver_name, sizeof(receiver_name), "%T", remote); msize = size_object(message); if (token != NIL && token != am_have_dt_utag) { @@ -826,7 +909,7 @@ erts_dsig_send_reg_msg(ErtsDSigData *dsdp, Eterm remote_name, Eterm message) *node_name = *sender_name = *receiver_name = '\0'; if (DTRACE_ENABLED(message_send) || DTRACE_ENABLED(message_send_remote)) { erts_snprintf(node_name, sizeof(node_name), "%T", dsdp->dep->sysname); - erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->id); + erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->common.id); erts_snprintf(receiver_name, sizeof(receiver_name), "{%T,%s}", remote_name, node_name); msize = size_object(message); @@ -840,10 +923,10 @@ erts_dsig_send_reg_msg(ErtsDSigData *dsdp, Eterm remote_name, Eterm message) if (token != NIL) ctl = TUPLE5(&ctl_heap[0], make_small(DOP_REG_SEND_TT), - sender->id, am_Cookie, remote_name, token); + sender->common.id, am_Cookie, remote_name, token); else ctl = TUPLE4(&ctl_heap[0], make_small(DOP_REG_SEND), - sender->id, am_Cookie, remote_name); + sender->common.id, am_Cookie, remote_name); DTRACE6(message_send, sender_name, receiver_name, msize, tok_label, tok_lastcnt, tok_serial); DTRACE7(message_send_remote, sender_name, node_name, receiver_name, @@ -889,7 +972,7 @@ erts_dsig_send_exit_tt(ErtsDSigData *dsdp, Eterm local, Eterm remote, *node_name = *sender_name = *remote_name = '\0'; if (DTRACE_ENABLED(process_exit_signal_remote)) { erts_snprintf(node_name, sizeof(node_name), "%T", dsdp->dep->sysname); - erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->id); + erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->common.id); erts_snprintf(remote_name, sizeof(remote_name), "{%T,%s}", remote, node_name); erts_snprintf(reason_str, sizeof(reason), "%T", reason); @@ -1141,7 +1224,7 @@ int erts_net_message(Port *prt, } erts_smp_de_links_lock(dep); - res = erts_add_link(&(rp->nlinks), LINK_PID, from); + res = erts_add_link(&ERTS_P_LINKS(rp), LINK_PID, from); if (res < 0) { /* It was already there! Lets skip the rest... */ @@ -1149,7 +1232,7 @@ int erts_net_message(Port *prt, erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); break; } - lnk = erts_add_or_lookup_link(&(dep->nlinks), LINK_PID, rp->id); + lnk = erts_add_or_lookup_link(&(dep->nlinks), LINK_PID, rp->common.id); erts_add_link(&(ERTS_LINK_ROOT(lnk)), LINK_PID, from); erts_smp_de_links_unlock(dep); @@ -1176,7 +1259,7 @@ int erts_net_message(Port *prt, if (!rp) break; - lnk = erts_remove_link(&(rp->nlinks), from); + lnk = erts_remove_link(&ERTS_P_LINKS(rp), from); if (IS_TRACED_FL(rp, F_TRACE_PROCS) && lnk != NULL) { trace_proc(NULL, rp, am_getting_unlinked, from); @@ -1233,10 +1316,10 @@ int erts_net_message(Port *prt, } else { if (is_atom(watched)) - watched = rp->id; + watched = rp->common.id; erts_smp_de_links_lock(dep); erts_add_monitor(&(dep->monitors), MON_ORIGIN, ref, watched, name); - erts_add_monitor(&(rp->monitors), MON_TARGET, ref, watcher, name); + erts_add_monitor(&ERTS_P_MONITORS(rp), MON_TARGET, ref, watcher, name); erts_smp_de_links_unlock(dep); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); } @@ -1275,7 +1358,7 @@ int erts_net_message(Port *prt, if (!rp) { break; } - mon = erts_remove_monitor(&(rp->monitors),ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(rp), ref); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); ASSERT(mon != NULL); if (mon == NULL) { @@ -1432,7 +1515,7 @@ int erts_net_message(Port *prt, erts_destroy_monitor(mon); - mon = erts_remove_monitor(&(rp->monitors),ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(rp), ref); if (mon == NULL) { erts_smp_proc_unlock(rp, rp_locks); @@ -1483,7 +1566,7 @@ int erts_net_message(Port *prt, if (!rp) lnk = NULL; else { - lnk = erts_remove_link(&(rp->nlinks), from); + lnk = erts_remove_link(&ERTS_P_LINKS(rp), from); /* If lnk == NULL, we have unlinked on this side, i.e. * ignore exit. @@ -1597,7 +1680,7 @@ int erts_net_message(Port *prt, erts_free(ERTS_ALC_T_DCTRL_BUF, (void *) ctl); } UnUseTmpHeapNoproc(DIST_CTL_DEFAULT_SIZE); - erts_do_exit_port(prt, dep->cid, am_killed); + erts_deliver_port_exit(prt, dep->cid, am_killed, 0); ERTS_SMP_CHK_NO_PROC_LOCKS; return -1; } @@ -1693,7 +1776,6 @@ dsig_send(ErtsDSigData *dsdp, Eterm ctl, Eterm msg, int force_busy) erts_smp_mtx_unlock(&dep->qlock); plp = erts_proclist_create(c_p); - plp->next = NULL; erts_suspend(c_p, ERTS_PROC_LOCK_MAIN, NULL); suspended = 1; erts_smp_mtx_lock(&dep->qlock); @@ -1726,11 +1808,7 @@ dsig_send(ErtsDSigData *dsdp, Eterm ctl, Eterm msg, int force_busy) else { /* Enqueue suspended process on dist entry */ ASSERT(plp); - if (dep->suspended.last) - dep->suspended.last->next = plp; - else - dep->suspended.first = plp; - dep->suspended.last = plp; + erts_proclist_store_last(&dep->suspended, plp); } } @@ -1779,7 +1857,7 @@ dsig_send(ErtsDSigData *dsdp, Eterm ctl, Eterm msg, int force_busy) erts_snprintf(port_str, sizeof(port_str), "%T", cid); erts_snprintf(remote_str, sizeof(remote_str), "%T", dep->sysname); - erts_snprintf(pid_str, sizeof(pid_str), "%T", c_p->id); + erts_snprintf(pid_str, sizeof(pid_str), "%T", c_p->common.id); DTRACE4(dist_port_busy, erts_this_node_sysname, port_str, remote_str, pid_str); } @@ -1812,7 +1890,7 @@ dist_port_command(Port *prt, ErtsDistOutputBuf *obuf) DTRACE_CHARBUF(port_str, 64); DTRACE_CHARBUF(remote_str, 64); - erts_snprintf(port_str, sizeof(port_str), "%T", prt->id); + erts_snprintf(port_str, sizeof(port_str), "%T", prt->common.id); erts_snprintf(remote_str, sizeof(remote_str), "%T", prt->dist_entry->sysname); DTRACE4(dist_output, erts_this_node_sysname, port_str, @@ -1866,7 +1944,7 @@ dist_port_commandv(Port *prt, ErtsDistOutputBuf *obuf) DTRACE_CHARBUF(port_str, 64); DTRACE_CHARBUF(remote_str, 64); - erts_snprintf(port_str, sizeof(port_str), "%T", prt->id); + erts_snprintf(port_str, sizeof(port_str), "%T", prt->common.id); erts_snprintf(remote_str, sizeof(remote_str), "%T", prt->dist_entry->sysname); DTRACE4(dist_outputv, erts_this_node_sysname, port_str, @@ -1903,13 +1981,13 @@ int erts_dist_command(Port *prt, int reds_limit) { Sint reds = ERTS_PORT_REDS_DIST_CMD_START; - int prt_busy; Uint32 status; Uint32 flags; Sint obufsize = 0; ErtsDistOutputQueue oq, foq; DistEntry *dep = prt->dist_entry; Uint (*send)(Port *prt, ErtsDistOutputBuf *obuf); + erts_aint32_t sched_flags; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); @@ -1925,7 +2003,7 @@ erts_dist_command(Port *prt, int reds_limit) erts_smp_de_runlock(dep); if (status & ERTS_DE_SFLG_EXITING) { - erts_do_exit_port(prt, prt->id, am_killed); + erts_deliver_port_exit(prt, prt->common.id, am_killed, 0); erts_deref_dist_entry(dep); return reds + ERTS_PORT_REDS_DIST_CMD_EXIT; } @@ -1952,12 +2030,12 @@ erts_dist_command(Port *prt, int reds_limit) dep->finalized_out_queue.first = NULL; dep->finalized_out_queue.last = NULL; + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (reds > reds_limit) goto preempted; - prt_busy = (int) (prt->status & ERTS_PORT_SFLG_PORT_BUSY); - - if (!prt_busy && foq.first) { + if (!(sched_flags & ERTS_PTS_FLG_BUSY_PORT) && foq.first) { int preempt = 0; do { Uint size; @@ -1974,11 +2052,10 @@ erts_dist_command(Port *prt, int reds_limit) obufsize += size_obuf(fob); foq.first = foq.first->next; free_dist_obuf(fob); - preempt = reds > reds_limit || (prt->status & ERTS_PORT_SFLGS_DEAD); - if (prt->status & ERTS_PORT_SFLG_PORT_BUSY) { - prt_busy = 1; + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + preempt = reds > reds_limit || (sched_flags & ERTS_PTS_FLG_EXIT); + if (sched_flags & ERTS_PTS_FLG_BUSY_PORT) break; - } } while (foq.first && !preempt); if (!foq.first) foq.last = NULL; @@ -1986,7 +2063,7 @@ erts_dist_command(Port *prt, int reds_limit) goto preempted; } - if (prt_busy) { + if (sched_flags & ERTS_PTS_FLG_BUSY_PORT) { if (oq.first) { ErtsDistOutputBuf *ob; int preempt; @@ -2058,12 +2135,10 @@ erts_dist_command(Port *prt, int reds_limit) obufsize += size_obuf(fob); oq.first = oq.first->next; free_dist_obuf(fob); - preempt = reds > reds_limit || (prt->status & ERTS_PORT_SFLGS_DEAD); - if (prt->status & ERTS_PORT_SFLG_PORT_BUSY) { - prt_busy = 1; - if (oq.first && !preempt) - goto finalize_only; - } + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + preempt = reds > reds_limit || (sched_flags & ERTS_PTS_FLG_EXIT); + if ((sched_flags & ERTS_PTS_FLG_BUSY_PORT) && oq.first && !preempt) + goto finalize_only; } ASSERT(!oq.first || preempt); @@ -2091,7 +2166,7 @@ erts_dist_command(Port *prt, int reds_limit) ASSERT(dep->qsize >= obufsize); dep->qsize -= obufsize; obufsize = 0; - if (!prt_busy + if (!(sched_flags & ERTS_PTS_FLG_BUSY_PORT) && (dep->qflgs & ERTS_DE_QFLG_BUSY) && dep->qsize < erts_dist_buf_busy_limit) { ErtsProcList *suspendees; @@ -2137,11 +2212,15 @@ erts_dist_command(Port *prt, int reds_limit) return reds; preempted: + /* + * Here we assume that state has been read + * since last call to driver. + */ ASSERT(oq.first || !oq.last); ASSERT(!oq.first || oq.last); - if (prt->status & ERTS_PORT_SFLGS_DEAD) { + if (sched_flags & ERTS_PTS_FLG_EXIT) { /* * Port died during port command; clean up 'oq' * and 'foq'. Things buffered in dist entry after @@ -2199,7 +2278,7 @@ erts_dist_port_not_busy(Port *prt) DTRACE_CHARBUF(port_str, 64); DTRACE_CHARBUF(remote_str, 64); - erts_snprintf(port_str, sizeof(port_str), "%T", prt->id); + erts_snprintf(port_str, sizeof(port_str), "%T", prt->common.id); erts_snprintf(remote_str, sizeof(remote_str), "%T", prt->dist_entry->sysname); DTRACE3(dist_port_not_busy, erts_this_node_sysname, @@ -2241,7 +2320,7 @@ static void doit_print_monitor_info(ErtsMonitor *mon, void *vptdp) Process *rp; ErtsMonitor *rmon; rp = erts_proc_lookup(mon->pid); - if (!rp || (rmon = erts_lookup_monitor(rp->monitors, mon->ref)) == NULL) { + if (!rp || (rmon = erts_lookup_monitor(ERTS_P_MONITORS(rp), mon->ref)) == NULL) { erts_print(to, arg, "Warning, stray monitor for: %T\n", mon->pid); } else if (mon->type == MON_ORIGIN) { /* Local pid is being monitored */ @@ -2486,6 +2565,7 @@ BIF_RETTYPE setnode_2(BIF_ALIST_2) erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); erts_smp_thr_progress_block(); + inc_no_nodes(); erts_set_this_node(BIF_ARG_1, (Uint32) creation); erts_is_alive = 1; send_nodes_mon_msgs(NULL, am_nodeup, BIF_ARG_1, am_visible, NIL); @@ -2554,9 +2634,9 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) /* DFLAG_EXTENDED_REFERENCES is compulsory from R9 and forward */ if (!(DFLAG_EXTENDED_REFERENCES & flags)) { erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); - erts_dsprintf(dsbufp, "%T", BIF_P->id); - if (BIF_P->reg) - erts_dsprintf(dsbufp, " (%T)", BIF_P->reg->name); + erts_dsprintf(dsbufp, "%T", BIF_P->common.id); + if (BIF_P->common.u.alive.reg) + erts_dsprintf(dsbufp, " (%T)", BIF_P->common.u.alive.reg->name); erts_dsprintf(dsbufp, " attempted to enable connection to node %T " "which is not able to handle extended references.\n", @@ -2576,10 +2656,14 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) else if (!dep) goto system_limit; /* Should never happen!!! */ - pp = erts_id2port(BIF_ARG_2, BIF_P, ERTS_PROC_LOCK_MAIN); + pp = erts_id2port_sflgs(BIF_ARG_2, + BIF_P, + ERTS_PROC_LOCK_MAIN, + ERTS_PORT_SFLGS_INVALID_LOOKUP); erts_smp_de_rwlock(dep); - if (!pp || (pp->status & ERTS_PORT_SFLG_EXITING)) + if (!pp || (erts_atomic32_read_nob(&pp->state) + & ERTS_PORT_SFLG_EXITING)) goto badarg; if ((pp->drv_ptr->flags & ERL_DRV_FLAG_SOFT_BUSY) == 0) @@ -2594,11 +2678,7 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) plp->next = NULL; erts_suspend(BIF_P, ERTS_PROC_LOCK_MAIN, NULL); erts_smp_mtx_lock(&dep->qlock); - if (dep->suspended.last) - dep->suspended.last->next = plp; - else - dep->suspended.first = plp; - dep->suspended.last = plp; + erts_proclist_store_last(&dep->suspended, plp); erts_smp_mtx_unlock(&dep->qlock); goto yield; } @@ -2608,7 +2688,16 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) if (pp->dist_entry || is_not_nil(dep->cid)) goto badarg; - erts_port_status_bor_set(pp, ERTS_PORT_SFLG_DISTRIBUTION); + erts_atomic32_read_bor_nob(&pp->state, ERTS_PORT_SFLG_DISTRIBUTION); + + /* + * Dist-ports do not use the "busy port message queue" functionality, but + * instead use "busy dist entry" functionality. + */ + { + ErlDrvSizeT disable = ERL_DRV_BUSY_MSGQ_DISABLED; + erl_drv_busy_msgq_limits((ErlDrvPort) pp, &disable, NULL); + } pp->dist_entry = dep; @@ -2640,6 +2729,8 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) erts_smp_de_rwunlock(dep); dep = NULL; /* inc of refc transferred to port (dist_entry field) */ + inc_no_nodes(); + send_nodes_mon_msgs(BIF_P, am_nodeup, BIF_ARG_1, @@ -2653,7 +2744,7 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) } if (pp) - erts_smp_port_unlock(pp); + erts_port_release(pp); return ret; @@ -2697,7 +2788,7 @@ BIF_RETTYPE dist_exit_3(BIF_ALIST_3) if (is_internal_pid(local)) { Process *lp; ErtsProcLocks lp_locks; - if (BIF_P->id == local) { + if (BIF_P->common.id == local) { lp_locks = ERTS_PROC_LOCKS_ALL; lp = BIF_P; erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCKS_ALL_MINOR); @@ -2725,11 +2816,17 @@ BIF_RETTYPE dist_exit_3(BIF_ALIST_3) #endif erts_smp_proc_unlock(lp, lp_locks); if (lp == BIF_P) { + erts_aint32_t state = erts_smp_atomic32_read_acqb(&BIF_P->state); /* * We may have exited current process and may have to take action. */ - ERTS_BIF_CHK_EXITED(BIF_P); - ERTS_SMP_BIF_CHK_PENDING_EXIT(BIF_P, ERTS_PROC_LOCK_MAIN); + if (state & (ERTS_PSFLG_EXITING|ERTS_PSFLG_PENDING_EXIT)) { +#ifdef ERTS_SMP + if (state & ERTS_PSFLG_PENDING_EXIT) + erts_handle_pending_exit(BIF_P, ERTS_PROC_LOCK_MAIN); +#endif + ERTS_BIF_EXITED(BIF_P); + } } } else if (is_external_pid(local) @@ -2927,23 +3024,23 @@ monitor_node(Process* p, Eterm Node, Eterm Bool, Eterm Options) if (Bool == am_true) { ASSERT(dep->cid != NIL); lnk = erts_add_or_lookup_link(&(dep->node_links), LINK_NODE, - p->id); + p->common.id); ++ERTS_LINK_REFC(lnk); - lnk = erts_add_or_lookup_link(&(p->nlinks), LINK_NODE, Node); + lnk = erts_add_or_lookup_link(&ERTS_P_LINKS(p), LINK_NODE, Node); ++ERTS_LINK_REFC(lnk); } else { - lnk = erts_lookup_link(dep->node_links, p->id); + lnk = erts_lookup_link(dep->node_links, p->common.id); if (lnk != NULL) { if ((--ERTS_LINK_REFC(lnk)) == 0) { erts_destroy_link(erts_remove_link(&(dep->node_links), - p->id)); + p->common.id)); } } - lnk = erts_lookup_link(p->nlinks, Node); + lnk = erts_lookup_link(ERTS_P_LINKS(p), Node); if (lnk != NULL) { if ((--ERTS_LINK_REFC(lnk)) == 0) { - erts_destroy_link(erts_remove_link(&(p->nlinks), + erts_destroy_link(erts_remove_link(&ERTS_P_LINKS(p), Node)); } } @@ -3505,7 +3602,7 @@ erts_processes_monitoring_nodes(Process *c_p) olist = erts_bld_cons(hpp, szp, am_nodedown_reason, olist); res = erts_bld_cons(hpp, szp, erts_bld_tuple(hpp, szp, 2, - nmp->proc->id, + nmp->proc->common.id, olist), res); } diff --git a/erts/emulator/beam/dist.h b/erts/emulator/beam/dist.h index 845151c895..2bc3d9c881 100644 --- a/erts/emulator/beam/dist.h +++ b/erts/emulator/beam/dist.h @@ -187,11 +187,12 @@ void erts_schedule_dist_command(Port *prt, DistEntry *dist_entry) if (prt) { ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - ASSERT((erts_port_status_get(prt) & ERTS_PORT_SFLGS_DEAD) == 0); + ASSERT((erts_atomic32_read_nob(&prt->state) + & ERTS_PORT_SFLGS_DEAD) == 0); ASSERT(prt->dist_entry); dep = prt->dist_entry; - id = prt->id; + id = prt->common.id; } else { ASSERT(dist_entry); @@ -203,13 +204,8 @@ void erts_schedule_dist_command(Port *prt, DistEntry *dist_entry) id = dep->cid; } - if (!erts_smp_atomic_xchg_mb(&dep->dist_cmd_scheduled, 1)) { - (void) erts_port_task_schedule(id, - &dep->dist_cmd, - ERTS_PORT_TASK_DIST_CMD, - (ErlDrvEvent) -1, - NULL); - } + if (!erts_smp_atomic_xchg_mb(&dep->dist_cmd_scheduled, 1)) + erts_port_task_schedule(id, &dep->dist_cmd, ERTS_PORT_TASK_DIST_CMD); } #endif diff --git a/erts/emulator/beam/erl_alloc.c b/erts/emulator/beam/erl_alloc.c index 3eee53eba3..04d807b780 100644 --- a/erts/emulator/beam/erl_alloc.c +++ b/erts/emulator/beam/erl_alloc.c @@ -2128,6 +2128,7 @@ erts_memory(int *print_to_p, void *print_to_arg, void *proc, Eterm earg) if (want_tot_or_sys || want.processes || want.processes_used) { + int max_processes = erts_ptab_max(&erts_proc); UWord tmp; if (ERTS_MEM_NEED_ALL_ALCU) @@ -2137,7 +2138,7 @@ erts_memory(int *print_to_p, void *print_to_arg, void *proc, Eterm earg) fi, ERTS_ALC_NO_FIXED_SIZES); tmp = alcu_size(ERTS_ALC_A_EHEAP, NULL, 0); } - tmp += erts_max_processes*sizeof(Process*); + tmp += max_processes*sizeof(erts_smp_atomic_t); tmp += erts_bif_timer_memory_size(); tmp += erts_tot_link_lh_size(); @@ -2268,6 +2269,8 @@ erts_allocated_areas(int *print_to_p, void *print_to_arg, void *proc) Eterm res = THE_NON_VALUE; int i, length; Uint reserved_atom_space, atom_space; + int max_processes = erts_ptab_max(&erts_proc); + int max_ports = erts_ptab_max(&erts_port); if (proc) { ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_MAIN @@ -2299,7 +2302,7 @@ erts_allocated_areas(int *print_to_p, void *print_to_arg, void *proc) values[i].arity = 2; values[i].name = "static"; values[i].ui[0] = - erts_max_ports*sizeof(Port) /* Port table */ + max_ports*sizeof(erts_smp_atomic_t) /* Port table */ + erts_timer_wheel_memory_size(); /* Timer wheel */ i++; @@ -2378,7 +2381,7 @@ erts_allocated_areas(int *print_to_p, void *print_to_arg, void *proc) values[i].arity = 2; values[i].name = "process_table"; - values[i].ui[0] = erts_max_processes*sizeof(Process*); + values[i].ui[0] = max_processes*sizeof(Process*); i++; values[i].arity = 2; diff --git a/erts/emulator/beam/erl_alloc.h b/erts/emulator/beam/erl_alloc.h index e475f9d8a2..ba5ec9c367 100644 --- a/erts/emulator/beam/erl_alloc.h +++ b/erts/emulator/beam/erl_alloc.h @@ -267,6 +267,8 @@ typedef void (*erts_alloc_verify_func_t)(Allctr_t *); erts_alloc_verify_func_t erts_alloc_get_verify_unused_temp_alloc(Allctr_t **allctr); +#define ERTS_ALC_DATA_ALIGN_SIZE(SZ) \ + (((((SZ) - 1) / 8) + 1) * 8) #define ERTS_ALC_CACHE_LINE_ALIGN_SIZE(SZ) \ (((((SZ) - 1) / ERTS_CACHE_LINE_SIZE) + 1) * ERTS_CACHE_LINE_SIZE) diff --git a/erts/emulator/beam/erl_alloc.types b/erts/emulator/beam/erl_alloc.types index 0a4407f009..d4de0d076a 100644 --- a/erts/emulator/beam/erl_alloc.types +++ b/erts/emulator/beam/erl_alloc.types @@ -146,6 +146,7 @@ class SYSTEM system_data type SBMBC SBMBC SYSTEM small_block_mbc type PROC FIXED_SIZE PROCESSES proc +type PORT DRIVER SYSTEM port type ATOM LONG_LIVED ATOM atom_entry type MODULE LONG_LIVED CODE module_entry type REG_PROC STANDARD PROCESSES reg_proc @@ -189,7 +190,10 @@ type PORT_TABLE LONG_LIVED SYSTEM port_tab type TIMER_WHEEL LONG_LIVED SYSTEM timer_wheel type DRV DRIVER SYSTEM drv_internal type DRV_BINARY BINARY BINARIES drv_binary -type DRIVER STANDARD SYSTEM driver +type DRIVER DRIVER SYSTEM driver +type DRV_CMD_DATA DRIVER SYSTEM driver_command_data +type DRV_CTRL_DATA DRIVER SYSTEM driver_control_data +type DRV_CALL_DATA DRIVER SYSTEM driver_call_data type NIF DRIVER SYSTEM nif_internal type BINARY BINARY BINARIES binary type NBIF_TABLE SYSTEM SYSTEM nbif_tab @@ -197,14 +201,12 @@ type ARG_REG STANDARD PROCESSES arg_reg type PROC_DICT STANDARD PROCESSES proc_dict type CALLS_BUF STANDARD PROCESSES calls_buf type BPD STANDARD SYSTEM bpd -type PORT_NAME STANDARD SYSTEM port_name type LINEBUF STANDARD SYSTEM line_buf type IOQ STANDARD SYSTEM io_queue type BITS_BUF STANDARD SYSTEM bits_buf type TMP_DIST_BUF TEMPORARY SYSTEM tmp_dist_buf type ASYNC_DATA LONG_LIVED SYSTEM internal_async_data type ESTACK TEMPORARY SYSTEM estack -type PORT_CALL_BUF TEMPORARY SYSTEM port_call_buf type DB_TABLE ETS ETS db_tab type DB_FIXATION SHORT_LIVED ETS db_fixation type DB_FIX_DEL SHORT_LIVED ETS fixed_del @@ -234,14 +236,14 @@ type DDLL_HANDLE STANDARD SYSTEM ddll_handle type DDLL_ERRCODES LONG_LIVED SYSTEM ddll_errcodes type DDLL_TMP_BUF TEMPORARY SYSTEM ddll_tmp_buf type PORT_TASK SHORT_LIVED SYSTEM port_task -type PORT_TASKQ SHORT_LIVED SYSTEM port_task_queue +type PT_HNDL_LIST SHORT_LIVED SYSTEM port_task_handle_list type MISC_OP_LIST SHORT_LIVED SYSTEM misc_op_list type PORT_NAMES SHORT_LIVED SYSTEM port_names -type PORT_DATA_LOCK STANDARD SYSTEM port_data_lock +type PORT_DATA_LOCK DRIVER SYSTEM port_data_lock type NODES_MON STANDARD PROCESSES nodes_monitor -type PROCS_TPROC_EL SHORT_LIVED PROCESSES processes_term_proc_el -type PROCS_CNKINF SHORT_LIVED PROCESSES processes_chunk_info -type PROCS_PIDS SHORT_LIVED PROCESSES processes_pids +type PTAB_LIST_DEL SHORT_LIVED PROCESSES ptab_list_deleted_el +type PTAB_LIST_CNKI SHORT_LIVED PROCESSES ptab_list_chunk_info +type PTAB_LIST_PIDS SHORT_LIVED PROCESSES ptab_list_pids type RE_TMP_BUF TEMPORARY SYSTEM re_tmp_buf type RE_SUBJECT SHORT_LIVED SYSTEM re_subject type RE_HEAP STANDARD SYSTEM re_heap @@ -266,6 +268,8 @@ type AUX_WORK_TMO LONG_LIVED SYSTEM aux_work_timeouts type MISC_AUX_WORK_Q LONG_LIVED SYSTEM misc_aux_work_q type CODE_IX_LOCK_Q SHORT_LIVED SYSTEM code_ix_lock_q type PROC_INTERVAL LONG_LIVED SYSTEM process_interval +type BUSY_CALLER_TAB SHORT_LIVED SYSTEM busy_caller_table +type BUSY_CALLER SHORT_LIVED SYSTEM busy_caller +if threads_no_smp # Need thread safe allocs, but std_alloc and fix_alloc are not; diff --git a/erts/emulator/beam/erl_async.c b/erts/emulator/beam/erl_async.c index c5f432bea1..f2ca193ace 100644 --- a/erts/emulator/beam/erl_async.c +++ b/erts/emulator/beam/erl_async.c @@ -382,10 +382,15 @@ static ERTS_INLINE ErtsAsync *async_get(ErtsThrQ_t *q, static ERTS_INLINE void call_async_ready(ErtsAsync *a) { +#if ERTS_USE_ASYNC_READY_Q Port *p = erts_id2port_sflgs(a->port, NULL, 0, ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP); +#else + Port *p = erts_thr_id2port_sflgs(a->port, + ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP); +#endif if (!p) { if (a->async_free) a->async_free(a->async_data); @@ -395,7 +400,11 @@ static ERTS_INLINE void call_async_ready(ErtsAsync *a) if (a->async_free) a->async_free(a->async_data); } +#if ERTS_USE_ASYNC_READY_Q erts_port_release(p); +#else + erts_thr_port_release(p); +#endif } if (a->pdl) driver_pdl_dec_refc(a->pdl); @@ -603,7 +612,7 @@ long driver_async(ErlDrvPort ix, unsigned int* key, sched_id = 1; #endif - prt = erts_drvport2port(ix); + prt = erts_drvport2port(ix, NULL); if (!prt) return -1; @@ -615,7 +624,7 @@ long driver_async(ErlDrvPort ix, unsigned int* key, a->sched_id = sched_id; #endif a->hndl = (DE_Handle*)prt->drv_ptr->handle; - a->port = prt->id; + a->port = prt->common.id; a->pdl = NULL; a->async_data = async_data; a->async_invoke = async_invoke; diff --git a/erts/emulator/beam/erl_bif_ddll.c b/erts/emulator/beam/erl_bif_ddll.c index 7f7c975e78..7cbea55eac 100644 --- a/erts/emulator/beam/erl_bif_ddll.c +++ b/erts/emulator/beam/erl_bif_ddll.c @@ -104,16 +104,49 @@ static void dereference_all_processes(DE_Handle *dh); static void restore_process_references(DE_Handle *dh); static void ddll_no_more_references(void *vdh); -#define lock_drv_list() erts_smp_mtx_lock(&erts_driver_list_lock) -#define unlock_drv_list() erts_smp_mtx_unlock(&erts_driver_list_lock) +#define lock_drv_list() erts_smp_rwmtx_rwlock(&erts_driver_list_lock) +#define unlock_drv_list() erts_smp_rwmtx_rwunlock(&erts_driver_list_lock) #define assert_drv_list_locked() \ - ERTS_SMP_LC_ASSERT(erts_smp_lc_mtx_is_locked(&erts_driver_list_lock)) + ERTS_SMP_LC_ASSERT(erts_smp_lc_rwmtx_is_rwlocked(&erts_driver_list_lock) \ + || erts_smp_lc_rwmtx_is_rlocked(&erts_driver_list_lock)) +#define assert_drv_list_rwlocked() \ + ERTS_SMP_LC_ASSERT(erts_smp_lc_rwmtx_is_rwlocked(&erts_driver_list_lock)) +#define assert_drv_list_rlocked() \ + ERTS_SMP_LC_ASSERT(erts_smp_lc_rwmtx_is_rlocked(&erts_driver_list_lock)) #define assert_drv_list_not_locked() \ - ERTS_SMP_LC_ASSERT(!erts_smp_lc_mtx_is_locked(&erts_driver_list_lock)) + ERTS_SMP_LC_ASSERT(!erts_smp_lc_rwmtx_is_rwlocked(&erts_driver_list_lock) \ + && !erts_smp_lc_rwmtx_is_rlocked(&erts_driver_list_lock)) #define FREE_PORT_FLAGS (ERTS_PORT_SFLGS_DEAD & (~ERTS_PORT_SFLG_INITIALIZING)) +static void +kill_ports_driver_unloaded(DE_Handle *dh) +{ + int ix, max = erts_ptab_max(&erts_port); + + for (ix = 0; ix < max; ix++) { + erts_aint32_t state; + Port* prt = erts_pix2port(ix); + if (!prt) + continue; + + ERTS_SMP_DATA_DEPENDENCY_READ_MEMORY_BARRIER; + + state = erts_atomic32_read_nob(&prt->state); + if (state & FREE_PORT_FLAGS) + continue; + + erts_smp_port_lock(prt); + + state = erts_atomic32_read_nob(&prt->state); + if (!(state & ERTS_PORT_SFLGS_DEAD) && prt->drv_ptr->handle == dh) + driver_failure_atom((ErlDrvPort) prt, "driver_unloaded"); + + erts_port_release(prt); + } +} + /* * try_load(Path, Name, OptionList) -> {ok,Status} | * {ok, PendingStatus, Ref} | @@ -149,7 +182,7 @@ BIF_RETTYPE erl_ddll_try_load_3(BIF_ALIST_3) Eterm name_term = BIF_ARG_2; Eterm options = BIF_ARG_3; char *path = NULL; - Uint path_len; + ErlDrvSizeT path_len; char *name = NULL; DE_Handle *dh; erts_driver_t *drv; @@ -228,7 +261,7 @@ BIF_RETTYPE erl_ddll_try_load_3(BIF_ALIST_3) goto error; } path = erts_alloc(ERTS_ALC_T_DDLL_TMP_BUF, path_len + 1 /* might need path separator */ + sys_strlen(name) + 1); - if (io_list_to_buf(path_term, path, path_len) != 0) { + if (erts_iolist_to_buf(path_term, path, path_len) != 0) { goto error; } while (path_len > 0 && (path[path_len-1] == '\\' || path[path_len-1] == '/')) { @@ -356,40 +389,16 @@ BIF_RETTYPE erl_ddll_try_load_3(BIF_ALIST_3) ok_term = mkatom("loaded"); } } - assert_drv_list_locked(); + assert_drv_list_rwlocked(); if (kill_ports) { - int j; - /* Avoid closing the driver by referencing it */ + /* Avoid closing the driver by referencing it */ erts_ddll_reference_driver(dh); ASSERT(dh->status == ERL_DE_RELOAD); dh->status = ERL_DE_FORCE_RELOAD; #if DDLL_SMP unlock_drv_list(); #endif - for (j = 0; j < erts_max_ports; j++) { - Port* prt = &erts_port[j]; - erts_smp_port_state_lock(prt); - if (!(prt->status & FREE_PORT_FLAGS) && - prt->drv_ptr->handle == dh) { - erts_smp_atomic_inc_nob(&prt->refc); -#if DDLL_SMP - /* Extremely rare spinlock */ - while(prt->status & ERTS_PORT_SFLG_INITIALIZING) { - erts_smp_port_state_unlock(prt); - erts_smp_port_state_lock(prt); - } - erts_smp_port_state_unlock(prt); - erts_smp_mtx_lock(prt->lock); - if (!(prt->status & ERTS_PORT_SFLGS_DEAD)) { - driver_failure_atom(j, "driver_unloaded"); - } -#else - driver_failure_atom(j, "driver_unloaded"); -#endif - erts_port_release(prt); - } - else erts_smp_port_state_unlock(prt); - } + kill_ports_driver_unloaded(dh); /* Dereference, eventually causing driver destruction */ #if DDLL_SMP lock_drv_list(); @@ -579,45 +588,21 @@ Eterm erl_ddll_try_unload_2(BIF_ALIST_2) dh->reload_full_path = dh->reload_driver_name = NULL; dh->reload_flags = 0; } - if (dh->port_count > 0) { + if (erts_smp_atomic32_read_nob(&dh->port_count) > 0) { ++kill_ports; } dh->status = ERL_DE_UNLOAD; ok_term = am_pending_driver; done: - assert_drv_list_locked(); + assert_drv_list_rwlocked(); if (kill_ports > 1) { - int j; /* Avoid closing the driver by referencing it */ erts_ddll_reference_driver(dh); dh->status = ERL_DE_FORCE_UNLOAD; #if DDLL_SMP unlock_drv_list(); #endif - for (j = 0; j < erts_max_ports; j++) { - Port* prt = &erts_port[j]; - erts_smp_port_state_lock(prt); - if (!(prt->status & FREE_PORT_FLAGS) - && prt->drv_ptr->handle == dh) { - erts_smp_atomic_inc_nob(&prt->refc); -#if DDLL_SMP - /* Extremely rare spinlock */ - while(prt->status & ERTS_PORT_SFLG_INITIALIZING) { - erts_smp_port_state_unlock(prt); - erts_smp_port_state_lock(prt); - } - erts_smp_port_state_unlock(prt); - erts_smp_mtx_lock(prt->lock); - if (!(prt->status & ERTS_PORT_SFLGS_DEAD)) { - driver_failure_atom(j, "driver_unloaded"); - } -#else - driver_failure_atom(j, "driver_unloaded"); -#endif - erts_port_release(prt); - } - else erts_smp_port_state_unlock(prt); - } + kill_ports_driver_unloaded(dh); #if DDLL_SMP lock_drv_list(); #endif @@ -787,7 +772,7 @@ BIF_RETTYPE erl_ddll_info_2(BIF_ALIST_2) } else if (drv->handle->status == ERL_DE_PERMANENT) { res = am_permanent; } else { - res = make_small(drv->handle->port_count); + res = make_small(erts_smp_atomic32_read_nob(&drv->handle->port_count)); } goto done; case am_linked_in_driver: @@ -1045,38 +1030,16 @@ void erts_ddll_proc_dead(Process *p, ErtsProcLocks plocks) } dh->status = ERL_DE_UNLOAD; } - if (!left && drv->handle->port_count > 0) { + if (!left + && erts_smp_atomic32_read_nob(&drv->handle->port_count) > 0) { if (kill_ports) { - int j; DE_Handle *dh = drv->handle; erts_ddll_reference_driver(dh); dh->status = ERL_DE_FORCE_UNLOAD; #if DDLL_SMP unlock_drv_list(); #endif - for (j = 0; j < erts_max_ports; j++) { - Port* prt = &erts_port[j]; - erts_smp_port_state_lock(prt); - if (!(prt->status & FREE_PORT_FLAGS) && - prt->drv_ptr->handle == dh) { - erts_smp_atomic_inc_nob(&prt->refc); -#if DDLL_SMP - while(prt->status & ERTS_PORT_SFLG_INITIALIZING) { - erts_smp_port_state_unlock(prt); - erts_smp_port_state_lock(prt); - } - erts_smp_port_state_unlock(prt); - erts_smp_mtx_lock(prt->lock); - if (!(prt->status & ERTS_PORT_SFLGS_DEAD)) { - driver_failure_atom(j, "driver_unloaded"); - } -#else - driver_failure_atom(j, "driver_unloaded"); -#endif - erts_port_release(prt); - } - else erts_smp_port_state_unlock(prt); - } + kill_ports_driver_unloaded(dh); #if DDLL_SMP lock_drv_list(); /* Needed for future list operations */ #endif @@ -1098,7 +1061,7 @@ void erts_ddll_proc_dead(Process *p, ErtsProcLocks plocks) void erts_ddll_lock_driver(DE_Handle *dh, char *name) { DE_ProcEntry *p,*q; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); notify_all(dh, name, ERL_DE_PROC_AWAIT_LOAD, am_UP, am_permanent); notify_all(dh, name, @@ -1121,19 +1084,22 @@ void erts_ddll_lock_driver(DE_Handle *dh, char *name) void erts_ddll_increment_port_count(DE_Handle *dh) { assert_drv_list_locked(); - dh->port_count++; + erts_smp_atomic32_inc_nob(&dh->port_count); } void erts_ddll_decrement_port_count(DE_Handle *dh) { assert_drv_list_locked(); - ASSERT(dh->port_count > 0); - dh->port_count--; +#if DEBUG + ASSERT(erts_smp_atomic32_dec_read_nob(&dh->port_count) >= 0); +#else + erts_smp_atomic32_dec_nob(&dh->port_count); +#endif } static void first_ddll_reference(DE_Handle *dh) { - assert_drv_list_locked(); + assert_drv_list_rwlocked(); erts_refc_init(&(dh->refc),1); } @@ -1161,7 +1127,7 @@ void erts_ddll_dereference_driver(DE_Handle *dh) static void dereference_all_processes(DE_Handle *dh) { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); for(p = dh->procs;p != NULL; p = p->next) { if (p->awaiting_status == ERL_DE_PROC_LOADED) { ASSERT(!(p->flags & ERL_DE_FL_DEREFERENCED)); @@ -1174,7 +1140,7 @@ static void dereference_all_processes(DE_Handle *dh) static void restore_process_references(DE_Handle *dh) { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); ASSERT(erts_refc_read(&(dh->refc),0) == 0); for(p = dh->procs;p != NULL; p = p->next) { if (p->awaiting_status == ERL_DE_PROC_LOADED) { @@ -1402,7 +1368,7 @@ static int is_last_user(DE_Handle *dh, Process *proc) { DE_ProcEntry *p = dh->procs; int found = 0; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); while (p != NULL) { if (p->proc == proc && p->awaiting_status == ERL_DE_PROC_LOADED) { @@ -1423,7 +1389,7 @@ static DE_ProcEntry *find_proc_entry(DE_Handle *dh, Process *proc, Uint status) { DE_ProcEntry *p = dh->procs; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); while (p != NULL) { if (p->proc == proc && p->awaiting_status == status) { @@ -1450,7 +1416,7 @@ static int num_procs(DE_Handle *dh, Uint status) { DE_ProcEntry *p = dh->procs; int i = 0; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); while (p != NULL) { if (p->awaiting_status == status) { @@ -1465,7 +1431,7 @@ static int num_entries(DE_Handle *dh, Process *proc, Uint status) { DE_ProcEntry *p = dh->procs; int i = 0; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); while (p != NULL) { if (p->awaiting_status == status && p->proc == proc) { ++i; @@ -1478,7 +1444,7 @@ static int num_entries(DE_Handle *dh, Process *proc, Uint status) { static void add_proc_loaded(DE_Handle *dh, Process *proc) { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); p = erts_alloc(ERTS_ALC_T_DDLL_PROCESS, sizeof(DE_ProcEntry)); p->proc = proc; p->flags = 0; @@ -1490,7 +1456,7 @@ static void add_proc_loaded(DE_Handle *dh, Process *proc) static void add_proc_loaded_deref(DE_Handle *dh, Process *proc) { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); p = erts_alloc(ERTS_ALC_T_DDLL_PROCESS, sizeof(DE_ProcEntry)); p->proc = proc; p->awaiting_status = ERL_DE_PROC_LOADED; @@ -1510,7 +1476,7 @@ static void add_proc_waiting(DE_Handle *dh, Process *proc, Uint status, Eterm ref) { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); p = erts_alloc(ERTS_ALC_T_DDLL_PROCESS, sizeof(DE_ProcEntry)); p->proc = proc; p->flags = 0; @@ -1524,7 +1490,7 @@ static Eterm add_monitor(Process *p, DE_Handle *dh, Uint status) { Eterm r; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); r = erts_make_ref(p); add_proc_waiting(dh, p, status, r); return r; @@ -1535,7 +1501,7 @@ static void set_driver_reloading(DE_Handle *dh, Process *proc, char *path, char { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); p = erts_alloc(ERTS_ALC_T_DDLL_PROCESS, sizeof(DE_ProcEntry)); p->proc = proc; p->awaiting_status = ERL_DE_OK; @@ -1556,7 +1522,7 @@ static int do_load_driver_entry(DE_Handle *dh, char *path, char *name) int res; ErlDrvEntry *dp; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); if ((res = erts_sys_ddll_open(path, &(dh->handle))) != ERL_DE_NO_ERROR) { return res; @@ -1594,7 +1560,7 @@ static int do_load_driver_entry(DE_Handle *dh, char *path, char *name) goto error; } erts_smp_atomic_init_nob(&(dh->refc), (erts_aint_t) 0); - dh->port_count = 0; + erts_smp_atomic32_init_nob(&dh->port_count, 0); dh->full_path = erts_alloc(ERTS_ALC_T_DDLL_HANDLE, sys_strlen(path) + 1); sys_strcpy(dh->full_path, path); dh->flags = 0; @@ -1620,7 +1586,7 @@ static int do_unload_driver_entry(DE_Handle *dh, Eterm *save_name) { erts_driver_t *q, *p = driver_list; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); while (p != NULL) { if (p->handle == dh) { @@ -1660,11 +1626,11 @@ static int load_driver_entry(DE_Handle **dhp, char *path, char *name) int res; DE_Handle *dh = erts_alloc(ERTS_ALC_T_DDLL_HANDLE, sizeof(DE_Handle)); - assert_drv_list_locked(); + assert_drv_list_rwlocked(); dh->handle = NULL; dh->procs = NULL; - dh->port_count = 0; + erts_smp_atomic32_init_nob(&dh->port_count, 0); erts_refc_init(&(dh->refc), (erts_aint_t) 0); dh->status = -1; dh->reload_full_path = NULL; @@ -1698,7 +1664,7 @@ static int reload_driver_entry(DE_Handle *dh) int loadres; Uint flags = dh->reload_flags; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); dh->reload_full_path = NULL; dh->reload_driver_name = NULL; @@ -1736,7 +1702,7 @@ static void notify_proc(Process *proc, Eterm ref, Eterm driver_name, Eterm type, ErtsProcLocks rp_locks = 0; ERTS_SMP_CHK_NO_PROC_LOCKS; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); if (errcode != 0) { int need = load_error_need(errcode); Eterm e; @@ -1769,7 +1735,7 @@ static void notify_all(DE_Handle *dh, char *name, Uint awaiting, Eterm type, Ete { DE_ProcEntry **p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); p = &(dh->procs); while (*p != NULL) { @@ -1875,7 +1841,7 @@ static Eterm mkatom(char *str) static char *pick_list_or_atom(Eterm name_term) { char *name = NULL; - Uint name_len; + ErlDrvSizeT name_len; if (is_atom(name_term)) { Atom *ap = atom_tab(atom_val(name_term)); if (ap->len == 0) { @@ -1891,7 +1857,7 @@ static char *pick_list_or_atom(Eterm name_term) goto error; } name = erts_alloc(ERTS_ALC_T_DDLL_TMP_BUF, name_len + 1); - if (io_list_to_buf(name_term, name, name_len) != 0) { + if (erts_iolist_to_buf(name_term, name, name_len) != 0) { goto error; } name[name_len] = '\0'; @@ -1912,10 +1878,10 @@ static int build_proc_info(DE_Handle *dh, ProcEntryInfo **out_pei, Uint filter) int i; DE_ProcEntry *pe; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); for (pe = dh->procs; pe != NULL; pe = pe->next) { - Eterm id = pe->proc->id; + Eterm id = pe->proc->common.id; Uint stat = pe->awaiting_status; if (stat == ERL_DE_PROC_AWAIT_UNLOAD_ONLY) { stat = ERL_DE_PROC_AWAIT_UNLOAD; diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index a3811ccdb0..fabddffc68 100755 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -40,6 +40,8 @@ #include "erl_cpu_topology.h" #include "erl_async.h" #include "erl_thr_progress.h" +#define ERTS_PTAB_WANT_DEBUG_FUNCS__ +#include "erl_ptab.h" #ifdef HIPE #include "hipe_arch.h" #endif @@ -128,8 +130,6 @@ static char erts_system_version[] = ("Erlang " ERLANG_OTP_RELEASE static Eterm os_type_tuple; static Eterm os_version_tuple; -static BIF_RETTYPE port_info(Process* p, Eterm portid, Eterm item); - static Eterm current_function(Process* p, Process* rp, Eterm** hpp, int full_info); static Eterm current_stacktrace(Process* p, Process* rp, Eterm** hpp); @@ -873,8 +873,7 @@ BIF_RETTYPE process_info_1(BIF_ALIST_1) && external_pid_dist_entry(BIF_ARG_1) == erts_this_dist_entry) BIF_RET(am_undefined); - if (is_not_internal_pid(BIF_ARG_1) - || internal_pid_index(BIF_ARG_1) >= erts_max_processes) { + if (is_not_internal_pid(BIF_ARG_1)) { BIF_ERROR(BIF_P, BADARG); } @@ -909,8 +908,7 @@ BIF_RETTYPE process_info_2(BIF_ALIST_2) && external_pid_dist_entry(pid) == erts_this_dist_entry) BIF_RET(am_undefined); - if (is_not_internal_pid(pid) - || internal_pid_index(BIF_ARG_1) >= erts_max_processes) { + if (is_not_internal_pid(pid)) { BIF_ERROR(BIF_P, BADARG); } @@ -1002,9 +1000,9 @@ process_info_aux(Process *BIF_P, switch (item) { case am_registered_name: - if (rp->reg != NULL) { + if (rp->common.u.alive.reg) { hp = HAlloc(BIF_P, 3); - res = rp->reg->name; + res = rp->common.u.alive.reg->name; } else { if (always_wrap) { hp = HAlloc(BIF_P, 3); @@ -1050,7 +1048,7 @@ process_info_aux(Process *BIF_P, ERTS_SMP_MSGQ_MV_INQ2PRIVQ(rp); n = rp->msg.len; - if (n == 0 || rp->trace_flags & F_SENSITIVE) { + if (n == 0 || ERTS_TRACE_FLAGS(rp) & F_SENSITIVE) { hp = HAlloc(BIF_P, 3); } else { int remove_bad_messages = 0; @@ -1209,7 +1207,7 @@ process_info_aux(Process *BIF_P, INIT_MONITOR_INFOS(mic); - erts_doforall_links(rp->nlinks,&collect_one_link,&mic); + erts_doforall_links(ERTS_P_LINKS(rp),&collect_one_link,&mic); hp = HAlloc(BIF_P, 3 + mic.sz); res = NIL; @@ -1227,7 +1225,7 @@ process_info_aux(Process *BIF_P, int i; INIT_MONITOR_INFOS(mic); - erts_doforall_monitors(rp->monitors,&collect_one_origin_monitor,&mic); + erts_doforall_monitors(ERTS_P_MONITORS(rp),&collect_one_origin_monitor,&mic); hp = HAlloc(BIF_P, 3 + mic.sz); res = NIL; for (i = 0; i < mic.mi_i; i++) { @@ -1264,7 +1262,7 @@ process_info_aux(Process *BIF_P, Eterm item; INIT_MONITOR_INFOS(mic); - erts_doforall_monitors(rp->monitors,&collect_one_target_monitor,&mic); + erts_doforall_monitors(ERTS_P_MONITORS(rp),&collect_one_target_monitor,&mic); hp = HAlloc(BIF_P, 3 + mic.sz); res = NIL; @@ -1330,7 +1328,7 @@ process_info_aux(Process *BIF_P, } case am_dictionary: - if (rp->trace_flags & F_SENSITIVE) { + if (ERTS_TRACE_FLAGS(rp) & F_SENSITIVE) { res = NIL; } else { res = erts_dictionary_copy(BIF_P, rp->dictionary); @@ -1426,8 +1424,8 @@ process_info_aux(Process *BIF_P, ERTS_SMP_MSGQ_MV_INQ2PRIVQ(rp); - erts_doforall_links(rp->nlinks, &one_link_size, &size); - erts_doforall_monitors(rp->monitors, &one_mon_size, &size); + erts_doforall_links(ERTS_P_LINKS(rp), &one_link_size, &size); + erts_doforall_monitors(ERTS_P_MONITORS(rp), &one_mon_size, &size); size += (rp->heap_sz + rp->mbuf_sz) * sizeof(Eterm); if (rp->old_hend && rp->old_heap) size += (rp->old_hend - rp->old_heap) * sizeof(Eterm); @@ -1500,7 +1498,7 @@ process_info_aux(Process *BIF_P, case am_trace: hp = HAlloc(BIF_P, 3); - res = make_small(rp->trace_flags & TRACEE_FLAGS); + res = make_small(ERTS_TRACE_FLAGS(rp) & TRACEE_FLAGS); break; case am_binary: { @@ -1605,7 +1603,7 @@ current_function(Process* BIF_P, Process* rp, Eterm** hpp, int full_info) } } - if (BIF_P->id == rp->id) { + if (BIF_P == rp) { FunctionInfo fi2; /* @@ -1837,17 +1835,17 @@ info_1_tuple(Process* BIF_P, /* Pointer to current process. */ # define ERTS_ERROR_CHECKER_PRINTF_XML VALGRIND_PRINTF_XML # endif #endif - Uint buf_size = 8*1024; /* Try with 8KB first */ + ErlDrvSizeT buf_size = 8*1024; /* Try with 8KB first */ char *buf = erts_alloc(ERTS_ALC_T_TMP, buf_size); - int r = io_list_to_buf(*tp, (char*) buf, buf_size - 1); - if (r < 0) { + ErlDrvSizeT r = erts_iolist_to_buf(*tp, (char*) buf, buf_size - 1); + if (ERTS_IOLIST_TO_BUF_FAILED(r)) { erts_free(ERTS_ALC_T_TMP, (void *) buf); if (erts_iolist_size(*tp, &buf_size)) { goto badarg; } buf_size++; buf = erts_alloc(ERTS_ALC_T_TMP, buf_size); - r = io_list_to_buf(*tp, (char*) buf, buf_size - 1); + r = erts_iolist_to_buf(*tp, (char*) buf, buf_size - 1); ASSERT(r == buf_size - 1); } buf[buf_size - 1 - r] = '\0'; @@ -2159,9 +2157,13 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) res = TUPLE2(hp, am_min_bin_vheap_size,make_small(BIN_VH_MIN_SIZE)); BIF_RET(res); } else if (BIF_ARG_1 == am_process_count) { - BIF_RET(make_small(erts_process_count())); + BIF_RET(make_small(erts_ptab_count(&erts_proc))); } else if (BIF_ARG_1 == am_process_limit) { - BIF_RET(make_small(erts_max_processes)); + BIF_RET(make_small(erts_ptab_max(&erts_proc))); + } else if (BIF_ARG_1 == am_port_count) { + BIF_RET(make_small(erts_ptab_count(&erts_port))); + } else if (BIF_ARG_1 == am_port_limit) { + BIF_RET(make_small(erts_ptab_max(&erts_port))); } else if (BIF_ARG_1 == am_info || BIF_ARG_1 == am_procs || BIF_ARG_1 == am_loaded @@ -2534,6 +2536,9 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) } else if (ERTS_IS_ATOM_STR("run_queues", BIF_ARG_1)) { res = make_small(erts_no_run_queues); BIF_RET(res); + } else if (ERTS_IS_ATOM_STR("port_parallelism", BIF_ARG_1)) { + res = erts_port_parallelism ? am_true : am_false; + BIF_RET(res); } else if (ERTS_IS_ATOM_STR("c_compiler_used", BIF_ARG_1)) { Eterm *hp = NULL; Uint sz = 0; @@ -2701,66 +2706,6 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) BIF_ERROR(BIF_P, BADARG); } -BIF_RETTYPE -port_info_1(BIF_ALIST_1) -{ - Process* p = BIF_P; - Eterm pid = BIF_ARG_1; - static Eterm keys[] = { - am_name, - am_links, - am_id, - am_connected, - am_input, - am_output, - am_os_pid - }; - Eterm items[ASIZE(keys)]; - Eterm result = NIL; - Eterm reg_name; - Eterm* hp; - Uint need; - int i; - - /* - * Collect all information about the port. - */ - - for (i = 0; i < ASIZE(keys); i++) { - Eterm item; - - item = port_info(p, pid, keys[i]); - if (is_non_value(item)) { - return THE_NON_VALUE; - } - if (item == am_undefined) { - return am_undefined; - } - items[i] = item; - } - reg_name = port_info(p, pid, am_registered_name); - - /* - * Build the resulting list. - */ - - need = 2*ASIZE(keys); - if (is_tuple(reg_name)) { - need += 2; - } - hp = HAlloc(p, need); - for (i = ASIZE(keys) - 1; i >= 0; i--) { - result = CONS(hp, items[i], result); - hp += 2; - } - if (is_tuple(reg_name)) { - result = CONS(hp, reg_name, result); - } - - return result; -} - - /**********************************************************************/ /* Return information on ports */ /* Info: @@ -2773,38 +2718,20 @@ port_info_1(BIF_ALIST_1) ** os_pid The child's process ID */ -BIF_RETTYPE port_info_2(BIF_ALIST_2) -{ - return port_info(BIF_P, BIF_ARG_1, BIF_ARG_2); -} - -static BIF_RETTYPE port_info(Process* p, Eterm portid, Eterm item) +Eterm +erts_bld_port_info(Eterm **hpp, ErlOffHeap *ohp, Uint *szp, Port *prt, Eterm item) { - BIF_RETTYPE ret; - Port *prt; - Eterm res; - Eterm* hp; - int count; - - if (is_internal_port(portid)) - prt = erts_id2port(portid, p, ERTS_PROC_LOCK_MAIN); - else if (is_atom(portid)) - erts_whereis_name(p, ERTS_PROC_LOCK_MAIN, - portid, NULL, 0, 0, &prt); - else if (is_external_port(portid) - && external_port_dist_entry(portid) == erts_this_dist_entry) - BIF_RET(am_undefined); - else { - BIF_ERROR(p, BADARG); - } + Eterm res = THE_NON_VALUE; - if (!prt) { - BIF_RET(am_undefined); - } + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); if (item == am_id) { - hp = HAlloc(p, 3); - res = make_small(internal_port_number(portid)); + if (hpp) + res = make_small(internal_port_index(prt->common.id)); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_links) { MonitorInfoCollection mic; @@ -2813,17 +2740,26 @@ static BIF_RETTYPE port_info(Process* p, Eterm portid, Eterm item) INIT_MONITOR_INFOS(mic); - erts_doforall_links(prt->nlinks, &collect_one_link, &mic); + erts_doforall_links(ERTS_P_LINKS(prt), &collect_one_link, &mic); - hp = HAlloc(p, 3 + mic.sz); - res = NIL; - for (i = 0; i < mic.mi_i; i++) { - item = STORE_NC(&hp, &MSO(p), mic.mi[i].entity); - res = CONS(hp, item, res); - hp += 2; + if (szp) + *szp += mic.sz; + + if (hpp) { + res = NIL; + for (i = 0; i < mic.mi_i; i++) { + item = STORE_NC(hpp, ohp, mic.mi[i].entity); + res = CONS(*hpp, item, res); + *hpp += 2; + } } + DESTROY_MONITOR_INFOS(mic); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_monitors) { MonitorInfoCollection mic; @@ -2832,79 +2768,96 @@ static BIF_RETTYPE port_info(Process* p, Eterm portid, Eterm item) INIT_MONITOR_INFOS(mic); - erts_doforall_monitors(prt->monitors, &collect_one_origin_monitor, &mic); + erts_doforall_monitors(ERTS_P_MONITORS(prt), &collect_one_origin_monitor, &mic); - hp = HAlloc(p, 3 + mic.sz); - res = NIL; - for (i = 0; i < mic.mi_i; i++) { - Eterm t; - item = STORE_NC(&hp, &MSO(p), mic.mi[i].entity); - t = TUPLE2(hp, am_process, item); - hp += 3; - res = CONS(hp, t, res); - hp += 2; + if (szp) + *szp += mic.sz; + + if (hpp) { + res = NIL; + for (i = 0; i < mic.mi_i; i++) { + Eterm t; + item = STORE_NC(hpp, ohp, mic.mi[i].entity); + t = TUPLE2(*hpp, am_process, item); + *hpp += 3; + res = CONS(*hpp, t, res); + *hpp += 2; + } } + DESTROY_MONITOR_INFOS(mic); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_name) { - count = sys_strlen(prt->name); + int count = sys_strlen(prt->name); + + if (hpp) + res = buf_to_intlist(hpp, prt->name, count, NIL); - hp = HAlloc(p, 3 + 2*count); - res = buf_to_intlist(&hp, prt->name, count, NIL); + if (szp) { + *szp += 2*count; + res = am_true; + goto done; + } } else if (item == am_connected) { - hp = HAlloc(p, 3); - res = prt->connected; /* internal pid */ + if (hpp) + res = ERTS_PORT_GET_CONNECTED(prt); /* internal pid */ + if (szp) { + res = am_true; + goto done; + } } else if (item == am_input) { - Uint hsz = 3; - Uint n = prt->bytes_in; - (void) erts_bld_uint(NULL, &hsz, n); - hp = HAlloc(p, hsz); - res = erts_bld_uint(&hp, NULL, n); + res = erts_bld_uint(hpp, szp, prt->bytes_in); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_output) { - Uint hsz = 3; - Uint n = prt->bytes_out; - (void) erts_bld_uint(NULL, &hsz, n); - hp = HAlloc(p, hsz); - res = erts_bld_uint(&hp, NULL, n); + res = erts_bld_uint(hpp, szp, prt->bytes_out); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_os_pid) { - if (prt->os_pid >= 0) { - Uint hsz = 3; - UWord n = prt->os_pid; - (void) erts_bld_uword(NULL, &hsz, n); - hp = HAlloc(p, hsz); - res = erts_bld_uword(&hp, NULL, n); - } else { - hp = HAlloc(p, 3); - res = am_undefined; - } + res = (prt->os_pid < 0 + ? am_undefined + : erts_bld_uword(hpp, szp, (UWord) prt->os_pid)); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_registered_name) { - RegProc *reg; - reg = prt->reg; - if (reg == NULL) { - ERTS_BIF_PREP_RET(ret, NIL); - goto done; - } else { - hp = HAlloc(p, 3); + RegProc *reg = prt->common.u.alive.reg; + if (reg) { res = reg->name; + if (szp) { + res = am_true; + goto done; + } + } + else { + if (szp) + return am_undefined; + return NIL; } } else if (item == am_memory) { /* All memory consumed in bytes (the Port struct should not be included though). */ - Uint hsz = 3; Uint size = 0; ErlHeapFragment* bp; - hp = HAlloc(p, 3); - - erts_doforall_links(prt->nlinks, &one_link_size, &size); + erts_doforall_links(ERTS_P_LINKS(prt), &one_link_size, &size); for (bp = prt->bp; bp; bp = bp->next) size += sizeof(ErlHeapFragment) + (bp->alloc_size - 1)*sizeof(Eterm); @@ -2918,51 +2871,72 @@ static BIF_RETTYPE port_info(Process* p, Eterm portid, Eterm item) /* All memory allocated by the driver should be included, but it is hard to retrieve... */ - (void) erts_bld_uint(NULL, &hsz, size); - hp = HAlloc(p, hsz); - res = erts_bld_uint(&hp, NULL, size); + res = erts_bld_uint(hpp, szp, size); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_queue_size) { Uint ioq_size = erts_port_ioq_size(prt); - Uint hsz = 3; - (void) erts_bld_uint(NULL, &hsz, ioq_size); - hp = HAlloc(p, hsz); - res = erts_bld_uint(&hp, NULL, ioq_size); + res = erts_bld_uint(hpp, szp, ioq_size); + if (szp) { + res = am_true; + goto done; + } } else if (ERTS_IS_ATOM_STR("locking", item)) { - hp = HAlloc(p, 3); + if (hpp) { #ifndef ERTS_SMP - res = am_false; + res = am_false; #else - if (prt->status & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK) { - DECL_AM(port_level); - ASSERT(prt->drv_ptr->flags - & ERL_DRV_FLAG_USE_PORT_LOCKING); - res = AM_port_level; + if (erts_atomic32_read_nob(&prt->state) + & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK) { + DECL_AM(port_level); + ASSERT(prt->drv_ptr->flags + & ERL_DRV_FLAG_USE_PORT_LOCKING); + res = AM_port_level; + } + else { + DECL_AM(driver_level); + ASSERT(!(prt->drv_ptr->flags + & ERL_DRV_FLAG_USE_PORT_LOCKING)); + res = AM_driver_level; + } +#endif } - else { - DECL_AM(driver_level); - ASSERT(!(prt->drv_ptr->flags - & ERL_DRV_FLAG_USE_PORT_LOCKING)); - res = AM_driver_level; + if (szp) { + res = am_true; + goto done; } -#endif + } + else if (item == am_parallelism) { + if (szp) { + res = am_true; + goto done; + } + res = ((ERTS_PTS_FLG_PARALLELISM & + erts_smp_atomic32_read_nob(&prt->sched.flags)) + ? am_true + : am_false); } else { - ERTS_BIF_PREP_ERROR(ret, p, BADARG); - goto done; + if (szp) + return am_false; + return THE_NON_VALUE; } - ERTS_BIF_PREP_RET(ret, TUPLE2(hp, item, res)); - - done: - - erts_smp_port_unlock(prt); +done: + if (szp) + *szp += 3; + if (hpp) { + res = TUPLE2(*hpp, item, res); + *hpp += 3; + } - return ret; + return res; } - BIF_RETTYPE fun_info_2(BIF_ALIST_2) { @@ -3094,12 +3068,9 @@ BIF_RETTYPE is_process_alive_1(BIF_ALIST_1) if(is_internal_pid(BIF_ARG_1)) { Process *rp; - if (BIF_ARG_1 == BIF_P->id) + if (BIF_ARG_1 == BIF_P->common.id) BIF_RET(am_true); - if(internal_pid_index(BIF_ARG_1) >= erts_max_processes) - BIF_ERROR(BIF_P, BADARG); - rp = erts_proc_lookup(BIF_ARG_1); if (!rp) { BIF_RET(am_false); @@ -3317,10 +3288,9 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1) /* Used by node_container_SUITE (emulator) */ Eterm res; if (ERTS_IS_ATOM_STR("next_pid", BIF_ARG_1)) - res = erts_test_next_pid(0, 0); - else { - res = erts_test_next_port(0, 0); - } + res = erts_ptab_test_next_id(&erts_proc, 0, 0); + else + res = erts_ptab_test_next_id(&erts_port, 0, 0); if (res < 0) BIF_RET(am_false); BIF_RET(erts_make_integer(res, BIF_P)); @@ -3356,11 +3326,11 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1) } else if (ERTS_IS_ATOM_STR("processes", BIF_ARG_1)) { /* Used by process_SUITE (emulator) */ - BIF_RET(erts_debug_processes(BIF_P)); + BIF_RET(erts_debug_ptab_list(BIF_P, &erts_proc)); } else if (ERTS_IS_ATOM_STR("processes_bif_info", BIF_ARG_1)) { /* Used by process_SUITE (emulator) */ - BIF_RET(erts_debug_processes_bif_info(BIF_P)); + BIF_RET(erts_debug_ptab_list_bif_info(BIF_P, &erts_proc)); } else if (ERTS_IS_ATOM_STR("max_atom_out_cache_index", BIF_ARG_1)) { /* Used by distribution_SUITE (emulator) */ @@ -3421,17 +3391,20 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1) ERTS_SMP_ASSERT_IS_NOT_EXITING(BIF_P); BIF_RET(am_undefined); } - res = make_link_list(BIF_P, p->nlinks, NIL); + res = make_link_list(BIF_P, ERTS_P_LINKS(p), NIL); erts_smp_proc_unlock(p, ERTS_PROC_LOCK_LINK); BIF_RET(res); } else if(is_internal_port(tp[2])) { Eterm res; - Port *p = erts_id2port(tp[2], BIF_P, ERTS_PROC_LOCK_MAIN); + Port *p = erts_id2port_sflgs(tp[2], + BIF_P, + ERTS_PROC_LOCK_MAIN, + ERTS_PORT_SFLGS_INVALID_LOOKUP); if(!p) BIF_RET(am_undefined); - res = make_link_list(BIF_P, p->nlinks, NIL); - erts_smp_port_unlock(p); + res = make_link_list(BIF_P, ERTS_P_LINKS(p), NIL); + erts_port_release(p); BIF_RET(res); } else if(is_node_name_atom(tp[2])) { @@ -3463,7 +3436,7 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1) ERTS_SMP_ASSERT_IS_NOT_EXITING(BIF_P); BIF_RET(am_undefined); } - res = make_monitor_list(BIF_P, p->monitors); + res = make_monitor_list(BIF_P, ERTS_P_MONITORS(p)); erts_smp_proc_unlock(p, ERTS_PROC_LOCK_LINK); BIF_RET(res); } else if(is_node_name_atom(tp[2])) { @@ -3606,7 +3579,7 @@ BIF_RETTYPE erts_debug_set_internal_state_2(BIF_ALIST_2) erts_aint_t prev_on = erts_smp_atomic_xchg_nob(&available_internal_state, on); if (on) { erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); - erts_dsprintf(dsbufp, "Process %T ", BIF_P->id); + erts_dsprintf(dsbufp, "Process %T ", BIF_P->common.id); if (erts_is_alive) erts_dsprintf(dsbufp, "on node %T ", erts_this_node->sysname); erts_dsprintf(dsbufp, @@ -3674,10 +3647,9 @@ BIF_RETTYPE erts_debug_set_internal_state_2(BIF_ALIST_2) Eterm res; if (ERTS_IS_ATOM_STR("next_pid", BIF_ARG_1)) - res = erts_test_next_pid(1, next); - else { - res = erts_test_next_port(1, next); - } + res = erts_ptab_test_next_id(&erts_proc, 1, next); + else + res = erts_ptab_test_next_id(&erts_port, 1, next); if (res < 0) BIF_RET(am_false); BIF_RET(erts_make_integer(res, BIF_P)); @@ -3948,8 +3920,8 @@ static Eterm lcnt_build_lock_term(Eterm **hpp, Uint *szp, erts_lcnt_lock_t *lock } else if (lock->flag & ERTS_LCNT_LT_PROCLOCK) { /* use registered names as id's for process locks if available */ proc = erts_proc_lookup(lock->id); - if (proc && proc->reg) { - id = proc->reg->name; + if (proc && proc->common.u.alive.reg) { + id = proc->common.u.alive.reg->name; } else { /* otherwise use process id */ id = lock->id; diff --git a/erts/emulator/beam/erl_bif_op.c b/erts/emulator/beam/erl_bif_op.c index 13f8b1f63c..d4cd9c89b3 100644 --- a/erts/emulator/beam/erl_bif_op.c +++ b/erts/emulator/beam/erl_bif_op.c @@ -261,11 +261,6 @@ Eterm erl_is_function(Process* p, Eterm arg1, Eterm arg2) if (exp->code[2] == (Uint) arity) { BIF_RET(am_true); } - } else if (is_tuple(arg1)) { - Eterm* tp = tuple_val(arg1); - if (tp[0] == make_arityval(2) && is_atom(tp[1]) && is_atom(tp[2])) { - BIF_RET(am_true); - } } BIF_RET(am_false); } diff --git a/erts/emulator/beam/erl_bif_port.c b/erts/emulator/beam/erl_bif_port.c index f9009166c0..81146e38d7 100644 --- a/erts/emulator/beam/erl_bif_port.c +++ b/erts/emulator/beam/erl_bif_port.c @@ -42,28 +42,26 @@ #include "erl_bits.h" #include "dtrace-wrapper.h" -static int open_port(Process* p, Eterm name, Eterm settings, int *err_nump); +static Port *open_port(Process* p, Eterm name, Eterm settings, int *err_typep, int *err_nump); static byte* convert_environment(Process* p, Eterm env); static char **convert_args(Eterm); static void free_args(char **); char *erts_default_arg0 = "default"; -static BIF_RETTYPE -port_call(Process* p, Eterm arg1, Eterm arg2, Eterm arg3); - BIF_RETTYPE open_port_2(BIF_ALIST_2) { - int port_num; - Eterm port_val; + Port *port; + Eterm port_id; char *str; - int err_num; + int err_type, err_num; - if ((port_num = open_port(BIF_P, BIF_ARG_1, BIF_ARG_2, &err_num)) < 0) { - if (port_num == -3) { + port = open_port(BIF_P, BIF_ARG_1, BIF_ARG_2, &err_type, &err_num); + if (!port) { + if (err_type == -3) { ASSERT(err_num == BADARG || err_num == SYSTEM_LIMIT); BIF_ERROR(BIF_P, err_num); - } else if (port_num == -2) { + } else if (err_type == -2) { str = erl_errno_id(err_num); } else { str = "einval"; @@ -74,546 +72,408 @@ BIF_RETTYPE open_port_2(BIF_ALIST_2) erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK); - port_val = erts_port[port_num].id; - erts_add_link(&(erts_port[port_num].nlinks), LINK_PID, BIF_P->id); - erts_add_link(&(BIF_P->nlinks), LINK_PID, port_val); + port_id = port->common.id; + erts_add_link(&ERTS_P_LINKS(port), LINK_PID, BIF_P->common.id); + erts_add_link(&ERTS_P_LINKS(BIF_P), LINK_PID, port_id); erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK); - erts_port_release(&erts_port[port_num]); + erts_port_release(port); - BIF_RET(port_val); + BIF_RET(port_id); } -/**************************************************************************** - - PORT BIFS: - - port_command/2 -- replace Port ! {..., {command, Data}} - port_command(Port, Data) -> true - when port(Port), io-list(Data) - - port_control/3 -- new port_control(Port, Ctl, Data) -> Reply - port_control(Port, Ctl, Data) -> Reply - where integer(Ctl), io-list(Data), io-list(Reply) - - port_close/1 -- replace Port ! {..., close} - port_close(Port) -> true - when port(Port) - - port_connect/2 -- replace Port ! {..., {connect, Pid}} - port_connect(Port, Pid) - when port(Port), pid(Pid) - - ***************************************************************************/ - -static Port* -id_or_name2port(Process *c_p, Eterm id) +static ERTS_INLINE Port * +lookup_port(Process *c_p, Eterm id_or_name) { - Port *port; - if (is_not_atom(id)) - port = erts_id2port(id, c_p, ERTS_PROC_LOCK_MAIN); + /* TODO: Implement nicer lookup in register... */ + Eterm id; + if (is_atom(id_or_name)) + id = erts_whereis_name_to_id(c_p, id_or_name); else - erts_whereis_name(c_p, ERTS_PROC_LOCK_MAIN, id, NULL, 0, 0, &port); - return port; + id = id_or_name; + return erts_port_lookup(id, ERTS_PORT_SFLGS_INVALID_LOOKUP); } -#define ERTS_PORT_COMMAND_FLAG_FORCE (((Uint32) 1) << 0) -#define ERTS_PORT_COMMAND_FLAG_NOSUSPEND (((Uint32) 1) << 1) +/* + * erts_internal:port_command/3 is used by the + * erlang:port_command/2 and erlang:port_command/3 + * BIFs. + */ -static BIF_RETTYPE -do_port_command(Process *BIF_P, Eterm arg1, Eterm arg2, Eterm arg3, - Uint32 flags) +BIF_RETTYPE erts_internal_port_command_3(BIF_ALIST_3) { BIF_RETTYPE res; - Port *p; - - /* Trace sched out before lock check wait */ - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_out); - } - - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_inactive); - } - - p = id_or_name2port(BIF_P, arg1); - if (!p) { - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_in); + Port *prt; + int flags = 0; + Eterm ref; + + if (is_not_nil(BIF_ARG_3)) { + Eterm l = BIF_ARG_3; + while (is_list(l)) { + Eterm* cons = list_val(l); + Eterm car = CAR(cons); + if (car == am_force) + flags |= ERTS_PORT_SIG_FLG_FORCE; + else if (car == am_nosuspend) + flags |= ERTS_PORT_SIG_FLG_NOSUSPEND; + else + BIF_RET(am_badarg); + l = CDR(cons); } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_active); - } - BIF_ERROR(BIF_P, BADARG); + if (!is_nil(l)) + BIF_RET(am_badarg); } - - /* Trace port in, id_or_name2port causes wait */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_in, am_command); - } - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_active); + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + if (flags & ERTS_PORT_SIG_FLG_FORCE) { + if (!(prt->drv_ptr->flags & ERL_DRV_FLAG_SOFT_BUSY)) + BIF_RET(am_notsup); } - ERTS_BIF_PREP_RET(res, am_true); +#ifdef DEBUG + ref = NIL; +#endif - if ((flags & ERTS_PORT_COMMAND_FLAG_FORCE) - && !(p->drv_ptr->flags & ERL_DRV_FLAG_SOFT_BUSY)) { - ERTS_BIF_PREP_ERROR(res, BIF_P, EXC_NOTSUP); - } - else if (!(flags & ERTS_PORT_COMMAND_FLAG_FORCE) - && p->status & ERTS_PORT_SFLG_PORT_BUSY) { - if (flags & ERTS_PORT_COMMAND_FLAG_NOSUSPEND) { + switch (erts_port_output(BIF_P, flags, prt, prt->common.id, BIF_ARG_2, &ref)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + ERTS_BIF_PREP_RET(res, am_badarg); + break; + case ERTS_PORT_OP_BUSY: + ASSERT(!(flags & ERTS_PORT_SIG_FLG_FORCE)); + if (flags & ERTS_PORT_SIG_FLG_NOSUSPEND) ERTS_BIF_PREP_RET(res, am_false); - } else { - erts_suspend(BIF_P, ERTS_PROC_LOCK_MAIN, p); - if (erts_system_monitor_flags.busy_port) { - monitor_generic(BIF_P, am_busy_port, p->id); - } - ERTS_BIF_PREP_YIELD3(res, bif_export[BIF_port_command_3], BIF_P, - arg1, arg2, arg3); - } - } else { - int wres; - erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); - ERTS_SMP_CHK_NO_PROC_LOCKS; - wres = erts_write_to_port(BIF_P->id, p, arg2); - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_MAIN); - if (wres != 0) { - ERTS_BIF_PREP_ERROR(res, BIF_P, BADARG); + erts_suspend(BIF_P, ERTS_PROC_LOCK_MAIN, prt); + ERTS_BIF_PREP_YIELD3(res, bif_export[BIF_erts_internal_port_command_3], + BIF_P, BIF_ARG_1, BIF_ARG_2, BIF_ARG_3); } - } - - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_out, am_command); - } - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_inactive); + break; + case ERTS_PORT_OP_BUSY_SCHEDULED: + ASSERT(!(flags & ERTS_PORT_SIG_FLG_FORCE)); + /* Fall through... */ + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(ref)); + ERTS_BIF_PREP_RET(res, ref); + break; + case ERTS_PORT_OP_DONE: + ERTS_BIF_PREP_RET(res, am_true); + break; + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_output() result"); + break; } - erts_port_release(p); - /* Trace sched in after port release */ - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_in); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_active); - } - if (ERTS_PROC_IS_EXITING(BIF_P)) { KILL_CATCHES(BIF_P); /* Must exit */ ERTS_BIF_PREP_ERROR(res, BIF_P, EXC_ERROR); } - return res; -} -BIF_RETTYPE port_command_2(BIF_ALIST_2) -{ - return do_port_command(BIF_P, BIF_ARG_1, BIF_ARG_2, NIL, 0); + return res; } -BIF_RETTYPE port_command_3(BIF_ALIST_3) +BIF_RETTYPE erts_internal_port_call_3(BIF_ALIST_3) { - Eterm l = BIF_ARG_3; - Uint32 flags = 0; - while (is_list(l)) { - Eterm* cons = list_val(l); - Eterm car = CAR(cons); - if (car == am_force) { - flags |= ERTS_PORT_COMMAND_FLAG_FORCE; - } else if (car == am_nosuspend) { - flags |= ERTS_PORT_COMMAND_FLAG_NOSUSPEND; - } else { - BIF_ERROR(BIF_P, BADARG); - } - l = CDR(cons); - } - if(!is_nil(l)) { - BIF_ERROR(BIF_P, BADARG); + Port* prt; + Eterm retval; + Uint uint_op; + unsigned int op; + erts_aint32_t state; + + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + if (!term_to_Uint(BIF_ARG_2, &uint_op)) + BIF_RET(am_badarg); + + if (uint_op > (Uint) UINT_MAX) + BIF_RET(am_badarg); + + op = (unsigned int) uint_op; + + switch (erts_port_call(BIF_P, prt, op, BIF_ARG_3, &retval)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_DROPPED: + case ERTS_PORT_OP_BADARG: + retval = am_badarg; + break; + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(retval)); + break; + case ERTS_PORT_OP_DONE: + ASSERT(is_not_internal_ref(retval)); + break; + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_call() result"); + retval = am_internal_error; + break; } - return do_port_command(BIF_P, BIF_ARG_1, BIF_ARG_2, BIF_ARG_3, flags); -} -BIF_RETTYPE port_call_2(BIF_ALIST_2) -{ - return port_call(BIF_P,BIF_ARG_1, make_small(0), BIF_ARG_2); -} + state = erts_smp_atomic32_read_acqb(&BIF_P->state); + if (state & (ERTS_PSFLG_EXITING|ERTS_PSFLG_PENDING_EXIT)) { +#ifdef ERTS_SMP + if (state & ERTS_PSFLG_PENDING_EXIT) + erts_handle_pending_exit(BIF_P, ERTS_PROC_LOCK_MAIN); +#endif + ERTS_BIF_EXITED(BIF_P); + } -BIF_RETTYPE port_call_3(BIF_ALIST_3) -{ - return port_call(BIF_P, BIF_ARG_1, BIF_ARG_2, BIF_ARG_3); + BIF_RET(retval); } -static BIF_RETTYPE -port_call(Process* c_p, Eterm arg1, Eterm arg2, Eterm arg3) +BIF_RETTYPE erts_internal_port_control_3(BIF_ALIST_3) { - Uint op; - Port *p; - Uint size; - byte *bytes; - byte *endp; - ErlDrvSizeT real_size; - erts_driver_t *drv; - byte port_input[256]; /* Default input buffer to encode in */ - byte port_result[256]; /* Buffer for result from port. */ - byte* port_resp; /* Pointer to result buffer. */ - char *prc; - ErlDrvSSizeT ret; - Eterm res; - Sint result_size; - Eterm *hp; - Eterm *hp_end; - unsigned ret_flags = 0U; - int fpe_was_unmasked; - - bytes = &port_input[0]; - port_resp = port_result; - /* trace of port scheduling with virtual process descheduling - * lock wait - */ - if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(c_p, am_out); - } - - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(c_p, am_inactive); + Port* prt; + Eterm retval; + Uint uint_op; + unsigned int op; + erts_aint32_t state; + + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + if (!term_to_Uint(BIF_ARG_2, &uint_op)) + BIF_RET(am_badarg); + + if (uint_op > (Uint) UINT_MAX) + BIF_RET(am_badarg); + + op = (unsigned int) uint_op; + + switch (erts_port_control(BIF_P, prt, op, BIF_ARG_3, &retval)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + retval = am_badarg; + break; + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(retval)); + break; + case ERTS_PORT_OP_DONE: + ASSERT(is_not_internal_ref(retval)); + break; + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_control() result"); + retval = am_internal_error; + break; } - p = id_or_name2port(c_p, arg1); - if (!p) { - error: - if (port_resp != port_result && - !(ret_flags & DRIVER_CALL_KEEP_BUFFER)) { - driver_free(port_resp); - } - if (bytes != &port_input[0]) - erts_free(ERTS_ALC_T_PORT_CALL_BUF, bytes); - /* Need to virtual schedule in the process if there - * was an error. - */ - if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(c_p, am_in); - } - - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(c_p, am_active); - } - - if (p) - erts_port_release(p); + state = erts_smp_atomic32_read_acqb(&BIF_P->state); + if (state & (ERTS_PSFLG_EXITING|ERTS_PSFLG_PENDING_EXIT)) { #ifdef ERTS_SMP - ERTS_SMP_BIF_CHK_PENDING_EXIT(c_p, ERTS_PROC_LOCK_MAIN); -#else - ERTS_BIF_CHK_EXITED(c_p); + if (state & ERTS_PSFLG_PENDING_EXIT) + erts_handle_pending_exit(BIF_P, ERTS_PROC_LOCK_MAIN); #endif - BIF_ERROR(c_p, BADARG); - } - - if ((drv = p->drv_ptr) == NULL) { - goto error; - } - if (drv->call == NULL) { - goto error; - } - if (!term_to_Uint(arg2, &op)) { - goto error; - } - p->caller = c_p->id; - - /* Lock taken, virtual schedule of port */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_in, am_call); - } - - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_active); + ERTS_BIF_EXITED(BIF_P); } - size = erts_encode_ext_size(arg3); - if (size > sizeof(port_input)) - bytes = erts_alloc(ERTS_ALC_T_PORT_CALL_BUF, size); - endp = bytes; - erts_encode_ext(arg3, &endp); + BIF_RET(retval); +} - real_size = endp - bytes; - if (real_size > size) { - erl_exit(1, "%s, line %d: buffer overflow: %d word(s)\n", - __FILE__, __LINE__, endp - (bytes + size)); - } - erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_MAIN); -#ifdef USE_VM_PROBES - if (DTRACE_ENABLED(driver_call)) { - DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); - DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); +/* + * erts_internal:port_close/1 is used by the + * erlang:port_close/1 BIF. + */ +BIF_RETTYPE erts_internal_port_close_1(BIF_ALIST_1) +{ + Eterm ref; + Port *prt; - dtrace_pid_str(p->connected, process_str); - dtrace_port_str(p, port_str); - DTRACE5(driver_call, process_str, port_str, p->name, op, real_size); - } -#endif - prc = (char *) port_resp; - fpe_was_unmasked = erts_block_fpe(); - ret = drv->call((ErlDrvData)p->drv_data, - (unsigned) op, - (char *) bytes, - (int) real_size, - &prc, - (int) sizeof(port_result), - &ret_flags); - erts_unblock_fpe(fpe_was_unmasked); - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_out, am_call); - } - - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_inactive); - } - - port_resp = (byte *) prc; - p->caller = NIL; - erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_MAIN); -#ifdef HARDDEBUG - { - ErlDrvSizeT z; - printf("real_size = %ld,%d, ret = %ld,%d\r\n", (unsigned long) real_size, - (int) real_size, (unsigned long)ret, (int) ret); - printf("["); - for(z = 0; z < real_size; ++z) { - printf("%d, ",(int) bytes[z]); - } - printf("]\r\n"); - printf("["); - for(z = 0; z < ret; ++z) { - printf("%d, ",(int) port_resp[z]); - } - printf("]\r\n"); - } -#endif - if (ret <= 0 || port_resp[0] != VERSION_MAGIC) { - /* Error or a binary without magic/ with wrong magic */ - goto error; - } - result_size = erts_decode_ext_size(port_resp, ret); - if (result_size < 0) { - goto error; - } - hp = HAlloc(c_p, result_size); - hp_end = hp + result_size; - endp = port_resp; - res = erts_decode_ext(&hp, &MSO(c_p), &endp); - if (res == THE_NON_VALUE) { - goto error; - } - HRelease(c_p, hp_end, hp); - if (port_resp != port_result && !(ret_flags & DRIVER_CALL_KEEP_BUFFER)) { - driver_free(port_resp); - } - if (bytes != &port_input[0]) - erts_free(ERTS_ALC_T_PORT_CALL_BUF, bytes); - if (p) - erts_port_release(p); -#ifdef ERTS_SMP - ERTS_SMP_BIF_CHK_PENDING_EXIT(c_p, ERTS_PROC_LOCK_MAIN); -#else - ERTS_BIF_CHK_EXITED(c_p); +#ifdef DEBUG + ref = NIL; #endif - if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(c_p, am_in); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(c_p, am_active); + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + + switch (erts_port_exit(BIF_P, 0, prt, prt->common.id, am_normal, &ref)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_badarg); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(ref)); + BIF_RET(ref); + case ERTS_PORT_OP_DONE: + BIF_RET(am_true); + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_exit() result"); + BIF_RET(am_internal_error); } - - return res; } - -BIF_RETTYPE port_control_3(BIF_ALIST_3) -{ - Port* p; - Uint op; - Eterm res = THE_NON_VALUE; - - /* Virtual schedule out calling process before lock wait */ - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_out); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_inactive); - } +/* + * erts_internal:port_connect/2 is used by the + * erlang:port_connect/2 BIF. + */ +BIF_RETTYPE erts_internal_port_connect_2(BIF_ALIST_2) +{ + Eterm ref; + Port* prt; - p = id_or_name2port(BIF_P, BIF_ARG_1); - if (!p) { - /* Schedule the process before exiting */ - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_in); - } - - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_active); - } - - BIF_ERROR(BIF_P, BADARG); - } + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); - /* Trace the port for scheduling in */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_in, am_control); - } - - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_active); - } +#ifdef DEBUG + ref = NIL; +#endif - if (term_to_Uint(BIF_ARG_2, &op)) - res = erts_port_control(BIF_P, p, op, BIF_ARG_3); - - /* Trace the port for scheduling out */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_out, am_control); + switch (erts_port_connect(BIF_P, 0, prt, prt->common.id, BIF_ARG_2, &ref)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_badarg); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(ref)); + BIF_RET(ref); + break; + case ERTS_PORT_OP_DONE: + BIF_RET(am_true); + break; + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_connect() result"); + BIF_RET(am_internal_error); } +} - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_inactive); - } +BIF_RETTYPE erts_internal_port_info_1(BIF_ALIST_1) +{ + Eterm retval; + Port* prt; - erts_port_release(p); -#ifdef ERTS_SMP - ERTS_SMP_BIF_CHK_PENDING_EXIT(BIF_P, ERTS_PROC_LOCK_MAIN); -#else - ERTS_BIF_CHK_EXITED(BIF_P); -#endif - - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_in); + if (is_internal_port(BIF_ARG_1) || is_atom(BIF_ARG_1)) { + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_undefined); } - - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_active); + else if (is_external_port(BIF_ARG_1)) { + if (external_port_dist_entry(BIF_ARG_1) == erts_this_dist_entry) + BIF_RET(am_undefined); + else + BIF_RET(am_badarg); } - - if (is_non_value(res)) { - BIF_ERROR(BIF_P, BADARG); + else { + BIF_RET(am_badarg); + } + + switch (erts_port_info(BIF_P, prt, THE_NON_VALUE, &retval)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + BIF_RET(am_badarg); + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_undefined); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(retval)); + BIF_RET(retval); + case ERTS_PORT_OP_DONE: + ASSERT(is_not_internal_ref(retval)); + BIF_RET(retval); + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_info() result"); + BIF_RET(am_internal_error); } - BIF_RET(res); } -BIF_RETTYPE port_close_1(BIF_ALIST_1) -{ - Port* p; - erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); - p = id_or_name2port(NULL, BIF_ARG_1); - if (!p) { - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_MAIN); - BIF_ERROR(BIF_P, BADARG); - } - erts_do_exit_port(p, p->connected, am_normal); - /* if !ERTS_SMP: since we terminate port with reason normal - we SHOULD never get an exit signal ourselves - */ - erts_port_release(p); - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_MAIN); - BIF_RET(am_true); -} -BIF_RETTYPE port_connect_2(BIF_ALIST_2) +BIF_RETTYPE erts_internal_port_info_2(BIF_ALIST_2) { + Eterm retval; Port* prt; - Process* rp; - Eterm pid = BIF_ARG_2; - if (is_not_internal_pid(pid)) { - error: - BIF_ERROR(BIF_P, BADARG); + if (is_internal_port(BIF_ARG_1) || is_atom(BIF_ARG_1)) { + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_undefined); } - prt = id_or_name2port(BIF_P, BIF_ARG_1); - if (!prt) { - goto error; + else if (is_external_port(BIF_ARG_1)) { + if (external_port_dist_entry(BIF_ARG_1) == erts_this_dist_entry) + BIF_RET(am_undefined); + else + BIF_RET(am_badarg); } - - rp = erts_pid2proc(BIF_P, ERTS_PROC_LOCK_MAIN, - pid, ERTS_PROC_LOCK_LINK); - if (!rp) { - erts_smp_port_unlock(prt); - ERTS_SMP_ASSERT_IS_NOT_EXITING(BIF_P); - goto error; - } - - erts_add_link(&(rp->nlinks), LINK_PID, prt->id); - erts_add_link(&(prt->nlinks), LINK_PID, pid); - - erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); - - prt->connected = pid; /* internal pid */ - erts_smp_port_unlock(prt); -#ifdef USE_VM_PROBES - if (DTRACE_ENABLED(port_connect)) { - DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); - DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); - DTRACE_CHARBUF(newprocess_str, DTRACE_TERM_BUF_SIZE); - - dtrace_pid_str(prt->connected, process_str); - erts_snprintf(port_str, sizeof(port_str), "%T", prt->id); - dtrace_proc_str(rp, newprocess_str); - DTRACE4(port_connect, process_str, port_str, prt->name, newprocess_str); + else { + BIF_RET(am_badarg); + } + + switch (erts_port_info(BIF_P, prt, BIF_ARG_2, &retval)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + BIF_RET(am_badarg); + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_undefined); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(retval)); + BIF_RET(retval); + case ERTS_PORT_OP_DONE: + ASSERT(is_not_internal_ref(retval)); + BIF_RET(retval); + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_info() result"); + BIF_RET(am_internal_error); } -#endif - BIF_RET(am_true); } -BIF_RETTYPE port_set_data_2(BIF_ALIST_2) + +BIF_RETTYPE erts_internal_port_set_data_2(BIF_ALIST_2) { + Eterm ref; Port* prt; - Eterm portid = BIF_ARG_1; - Eterm data = BIF_ARG_2; - prt = id_or_name2port(BIF_P, portid); - if (!prt) { - BIF_ERROR(BIF_P, BADARG); - } - if (prt->bp != NULL) { - free_message_buffer(prt->bp); - prt->bp = NULL; - } - if (IS_CONST(data)) { - prt->data = data; - } else { - Uint size; - ErlHeapFragment* bp; - Eterm* hp; - - size = size_object(data); - prt->bp = bp = new_message_buffer(size); - hp = bp->mem; - prt->data = copy_struct(data, size, &hp, &bp->off_heap); + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + switch (erts_port_set_data(BIF_P, prt, BIF_ARG_2, &ref)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_badarg); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(ref)); + BIF_RET(ref); + case ERTS_PORT_OP_DONE: + BIF_RET(am_true); + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_set_data() result"); + BIF_RET(am_internal_error); } - erts_smp_port_unlock(prt); - BIF_RET(am_true); } -BIF_RETTYPE port_get_data_1(BIF_ALIST_1) +BIF_RETTYPE erts_internal_port_get_data_1(BIF_ALIST_1) { - BIF_RETTYPE res; + Eterm retval; Port* prt; - Eterm portid = BIF_ARG_1; - prt = id_or_name2port(BIF_P, portid); - if (!prt) { - BIF_ERROR(BIF_P, BADARG); - } - if (prt->bp == NULL) { /* MUST be CONST! */ - res = prt->data; - } else { - Eterm* hp = HAlloc(BIF_P, prt->bp->used_size); - res = copy_struct(prt->data, prt->bp->used_size, &hp, &MSO(BIF_P)); + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + switch (erts_port_get_data(BIF_P, prt, &retval)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_badarg); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(retval)); + BIF_RET(retval); + case ERTS_PORT_OP_DONE: + ASSERT(is_not_internal_ref(retval)); + BIF_RET(retval); + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_get_data() result"); + BIF_RET(am_internal_error); } - erts_smp_port_unlock(prt); - BIF_RET(res); } /* @@ -625,11 +485,10 @@ BIF_RETTYPE port_get_data_1(BIF_ALIST_1) * either BADARG or SYSTEM_LIMIT). */ -static int -open_port(Process* p, Eterm name, Eterm settings, int *err_nump) +static Port * +open_port(Process* p, Eterm name, Eterm settings, int *err_typep, int *err_nump) { -#define OPEN_PORT_ERROR(VAL) do { port_num = (VAL); goto do_return; } while (0) - int i, port_num; + int i; Eterm option; Uint arity; Eterm* tp; @@ -637,11 +496,11 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) erts_driver_t* driver; char* name_buf = NULL; SysDriverOpts opts; - int binary_io; - int soft_eof; Sint linebuf; Eterm edir = NIL; byte dir[MAXPATHLEN]; + erts_aint32_t sflgs = 0; + Port *port; /* These are the defaults */ opts.packet_bytes = 0; @@ -655,8 +514,7 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) opts.overlapped_io = 0; opts.spawn_type = ERTS_SPAWN_ANY; opts.argv = NULL; - binary_io = 0; - soft_eof = 0; + opts.parallelism = erts_port_parallelism; linebuf = 0; *err_nump = 0; @@ -734,6 +592,13 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) } } else if (option == am_cd) { edir = *tp; + } else if (option == am_parallelism) { + if (*tp == am_true) + opts.parallelism = 1; + else if (*tp == am_false) + opts.parallelism = 0; + else + goto badarg; } else { goto badarg; } @@ -748,13 +613,13 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) } else if (*nargs == am_nouse_stdio) { opts.use_stdio = 0; } else if (*nargs == am_binary) { - binary_io = 1; + sflgs |= ERTS_PORT_SFLG_BINARY_IO; } else if (*nargs == am_in) { opts.read_write |= DO_READ; } else if (*nargs == am_out) { opts.read_write |= DO_WRITE; } else if (*nargs == am_eof) { - soft_eof = 1; + sflgs |= ERTS_PORT_SFLG_SOFT_EOF; } else if (*nargs == am_hide) { opts.hide_window = 1; } else if (*nargs == am_exit_status) { @@ -902,9 +767,9 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) heap[2] = make_small(0); heap[3] = NIL; iolist = make_list(heap); - r = io_list_to_buf(iolist, (char*) dir, MAXPATHLEN); + r = erts_iolist_to_buf(iolist, (char*) dir, MAXPATHLEN); UnUseTmpHeap(4,p); - if (r < 0) { + if (ERTS_IOLIST_TO_BUF_FAILED(r)) { goto badarg; } opts.wd = (char *) dir; @@ -926,44 +791,40 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) erts_smp_proc_unlock(p, ERTS_PROC_LOCK_MAIN); - port_num = erts_open_driver(driver, p->id, name_buf, &opts, err_nump); + port = erts_open_driver(driver, p->common.id, name_buf, &opts, err_typep, err_nump); #ifdef USE_VM_PROBES - if (port_num >= 0 && DTRACE_ENABLED(port_open)) { + if (port && DTRACE_ENABLED(port_open)) { DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); dtrace_proc_str(p, process_str); - erts_snprintf(port_str, sizeof(port_str), "%T", erts_port[port_num].id); + erts_snprintf(port_str, sizeof(port_str), "%T", port->common.id); DTRACE3(port_open, process_str, name_buf, port_str); } #endif erts_smp_proc_lock(p, ERTS_PROC_LOCK_MAIN); - if (port_num < 0) { - DEBUGF(("open_driver returned %d(%d)\n", port_num, *err_nump)); + if (!port) { + DEBUGF(("open_driver returned (%d:%d)\n", + err_typep ? *err_typep : 4711, + err_nump ? *err_nump : 4711)); if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { trace_virtual_sched(p, am_in); } - OPEN_PORT_ERROR(port_num); + goto do_return; } if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { trace_virtual_sched(p, am_in); } - if (binary_io) { - erts_port_status_bor_set(&erts_port[port_num], - ERTS_PORT_SFLG_BINARY_IO); - } - if (soft_eof) { - erts_port_status_bor_set(&erts_port[port_num], - ERTS_PORT_SFLG_SOFT_EOF); - } - if (linebuf && erts_port[port_num].linebuf == NULL){ - erts_port[port_num].linebuf = allocate_linebuf(linebuf); - erts_port_status_bor_set(&erts_port[port_num], - ERTS_PORT_SFLG_LINEBUF_IO); + if (linebuf && port->linebuf == NULL){ + port->linebuf = allocate_linebuf(linebuf); + sflgs |= ERTS_PORT_SFLG_LINEBUF_IO; } + + if (sflgs) + erts_atomic32_read_bor_relb(&port->state, sflgs); do_return: if (name_buf) @@ -974,13 +835,15 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) if (opts.wd && opts.wd != ((char *)dir)) { erts_free(ERTS_ALC_T_TMP, (void *) opts.wd); } - return port_num; + return port; badarg: - *err_nump = BADARG; - OPEN_PORT_ERROR(-3); + if (err_typep) + *err_typep = -3; + if (err_nump) + *err_nump = BADARG; + port = NULL; goto do_return; -#undef OPEN_PORT_ERROR } /* Arguments can be given i unicode and as raw binaries, convert filename is used to convert */ diff --git a/erts/emulator/beam/erl_bif_re.c b/erts/emulator/beam/erl_bif_re.c index b036c5ef5c..88f980d19f 100644 --- a/erts/emulator/beam/erl_bif_re.c +++ b/erts/emulator/beam/erl_bif_re.c @@ -413,7 +413,7 @@ build_compile_result(Process *p, Eterm error_tag, pcre *result, int errcode, con static BIF_RETTYPE re_compile(Process* p, Eterm arg1, Eterm arg2) { - Uint slen; + ErlDrvSizeT slen; char *expr; pcre *result; int errcode = 0; @@ -444,7 +444,7 @@ re_compile(Process* p, Eterm arg1, Eterm arg2) BIF_ERROR(p,BADARG); } expr = erts_alloc(ERTS_ALC_T_RE_TMP_BUF, slen + 1); - if (io_list_to_buf(arg1, expr, slen) != 0) { + if (erts_iolist_to_buf(arg1, expr, slen) != 0) { erts_free(ERTS_ALC_T_RE_TMP_BUF, expr); BIF_ERROR(p,BADARG); } @@ -797,7 +797,7 @@ build_capture(Eterm capture_spec[CAPSPEC_SIZE], const pcre *code) memcpy(tmpb,ap->name,ap->len); tmpb[ap->len] = '\0'; } else { - Uint slen; + ErlDrvSizeT slen; if (erts_iolist_size(val, &slen)) { goto error; } @@ -809,7 +809,7 @@ build_capture(Eterm capture_spec[CAPSPEC_SIZE], const pcre *code) (tmpbsiz = slen + 1)); } } - if (io_list_to_buf(val, tmpb, slen) != 0) { + if (erts_iolist_to_buf(val, tmpb, slen) != 0) { goto error; } tmpb[slen] = '\0'; @@ -853,7 +853,7 @@ re_run(Process *p, Eterm arg1, Eterm arg2, Eterm arg3) const pcre *code_tmp; RestartContext restart; byte *temp_alloc = NULL; - Uint slength; + ErlDrvSizeT slength; int startoffset = 0; int options = 0, comp_options = 0; int ovsize; @@ -877,7 +877,7 @@ re_run(Process *p, Eterm arg1, Eterm arg2, Eterm arg3) if (is_not_tuple(arg2) || (arityval(*tuple_val(arg2)) != 4)) { if (is_binary(arg2) || is_list(arg2) || is_nil(arg2)) { /* Compile from textual RE */ - Uint slen; + ErlDrvSizeT slen; char *expr; pcre *result; int errcode = 0; @@ -896,7 +896,7 @@ re_run(Process *p, Eterm arg1, Eterm arg2, Eterm arg3) } expr = erts_alloc(ERTS_ALC_T_RE_TMP_BUF, slen + 1); - if (io_list_to_buf(arg2, expr, slen) != 0) { + if (erts_iolist_to_buf(arg2, expr, slen) != 0) { erts_free(ERTS_ALC_T_RE_TMP_BUF, expr); BIF_ERROR(p,BADARG); } @@ -1039,7 +1039,7 @@ handle_iolist: } restart.subject = erts_alloc(ERTS_ALC_T_RE_SUBJECT, slength); - if (io_list_to_buf(arg1, restart.subject, slength) != 0) { + if (erts_iolist_to_buf(arg1, restart.subject, slength) != 0) { erts_free(ERTS_ALC_T_RE_SUBJECT, restart.ovector); erts_free(ERTS_ALC_T_RE_SUBJECT, restart.code); erts_free(ERTS_ALC_T_RE_SUBJECT, restart.subject); diff --git a/erts/emulator/beam/erl_bif_timer.c b/erts/emulator/beam/erl_bif_timer.c index 525b11f61c..d67695e533 100644 --- a/erts/emulator/beam/erl_bif_timer.c +++ b/erts/emulator/beam/erl_bif_timer.c @@ -265,10 +265,10 @@ link_proc(Process *p, ErtsBifTimer* btm) { btm->receiver.proc.ess = p; btm->receiver.proc.prev = NULL; - btm->receiver.proc.next = p->bif_timers; - if (p->bif_timers) - p->bif_timers->receiver.proc.prev = btm; - p->bif_timers = btm; + btm->receiver.proc.next = p->u.bif_timers; + if (p->u.bif_timers) + p->u.bif_timers->receiver.proc.prev = btm; + p->u.bif_timers = btm; } static ERTS_INLINE void @@ -277,7 +277,7 @@ unlink_proc(ErtsBifTimer* btm) if (btm->receiver.proc.prev) btm->receiver.proc.prev->receiver.proc.next = btm->receiver.proc.next; else - btm->receiver.proc.ess->bif_timers = btm->receiver.proc.next; + btm->receiver.proc.ess->u.bif_timers = btm->receiver.proc.next; if (btm->receiver.proc.next) btm->receiver.proc.next->receiver.proc.prev = btm->receiver.proc.prev; } @@ -613,7 +613,7 @@ erts_print_bif_timer_info(int to, void *to_arg) for (btm = bif_timer_tab[i]; btm; btm = btm->tab.next) { Eterm receiver = (btm->flags & BTM_FLG_BYNAME ? btm->receiver.name - : btm->receiver.proc.ess->id); + : btm->receiver.proc.ess->common.id); erts_print(to, to_arg, "=timer:%T\n", receiver); erts_print(to, to_arg, "Message: %T\n", btm->message); erts_print(to, to_arg, "Time left: %u ms\n", @@ -637,7 +637,7 @@ erts_cancel_bif_timers(Process *p, ErtsProcLocks plocks) erts_smp_proc_lock(p, plocks); } - btm = p->bif_timers; + btm = p->u.bif_timers; while (btm) { ErtsBifTimer *tmp_btm; ASSERT(!(btm->flags & BTM_FLG_CANCELED)); @@ -647,7 +647,7 @@ erts_cancel_bif_timers(Process *p, ErtsProcLocks plocks) erts_cancel_timer(&tmp_btm->tm); } - p->bif_timers = NULL; + p->u.bif_timers = NULL; erts_smp_btm_rwunlock(); } @@ -696,7 +696,7 @@ erts_bif_timer_foreach(void (*func)(Eterm, Eterm, ErlHeapFragment *, void *), for (btm = bif_timer_tab[i]; btm; btm = btm->tab.next) { (*func)((btm->flags & BTM_FLG_BYNAME ? btm->receiver.name - : btm->receiver.proc.ess->id), + : btm->receiver.proc.ess->common.id), btm->message, btm->bp, arg); diff --git a/erts/emulator/beam/erl_bif_trace.c b/erts/emulator/beam/erl_bif_trace.c index 805d788177..99a4394666 100644 --- a/erts/emulator/beam/erl_bif_trace.c +++ b/erts/emulator/beam/erl_bif_trace.c @@ -124,7 +124,7 @@ trace_pattern(Process* p, Eterm MFA, Eterm Pattern, Eterm flaglist) struct trace_pattern_flags flags = erts_trace_pattern_flags_off; int is_global; Process *meta_tracer_proc = p; - Eterm meta_tracer_pid = p->id; + Eterm meta_tracer_pid = p->common.id; if (!erts_try_seize_code_write_permission(p)) { ERTS_BIF_YIELD3(bif_export[BIF_trace_pattern_3], p, MFA, Pattern, flaglist); @@ -171,14 +171,12 @@ trace_pattern(Process* p, Eterm MFA, Eterm Pattern, Eterm flaglist) } } else if (is_internal_port(meta_tracer_pid)) { Port *meta_tracer_port; - meta_tracer_proc = NULL; - if (internal_port_index(meta_tracer_pid) >= erts_max_ports) + meta_tracer_proc = NULL; + meta_tracer_port = (erts_port_lookup( + meta_tracer_pid, + ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP)); + if (!meta_tracer_port) goto error; - meta_tracer_port = - &erts_port[internal_port_index(meta_tracer_pid)]; - if (INVALID_TRACER_PORT(meta_tracer_port, meta_tracer_pid)) { - goto error; - } } else { goto error; } @@ -254,7 +252,7 @@ trace_pattern(Process* p, Eterm MFA, Eterm Pattern, Eterm flaglist) MatchSetRef(erts_default_meta_match_spec); erts_default_meta_tracer_pid = meta_tracer_pid; if (meta_tracer_proc) { - meta_tracer_proc->trace_flags |= F_TRACER; + ERTS_TRACE_FLAGS(meta_tracer_proc) |= F_TRACER; } } else if (! flags.breakpoint) { MatchSetUnref(erts_default_meta_match_spec); @@ -342,7 +340,7 @@ trace_pattern(Process* p, Eterm MFA, Eterm Pattern, Eterm flaglist) } if (meta_tracer_proc) { - meta_tracer_proc->trace_flags |= F_TRACER; + ERTS_TRACE_FLAGS(meta_tracer_proc) |= F_TRACER; } matches = erts_set_trace_pattern(p, mfa, specified, @@ -526,24 +524,24 @@ Eterm trace_3(BIF_ALIST_3) if (is_nil(tracer) || is_internal_pid(tracer)) { Process *tracer_proc = erts_pid2proc(p, ERTS_PROC_LOCK_MAIN, - is_nil(tracer) ? p->id : tracer, + is_nil(tracer) ? p->common.id : tracer, ERTS_PROC_LOCKS_ALL); if (!tracer_proc) goto error; - tracer_proc->trace_flags |= F_TRACER; + ERTS_TRACE_FLAGS(tracer_proc) |= F_TRACER; erts_smp_proc_unlock(tracer_proc, (tracer_proc == p ? ERTS_PROC_LOCKS_ALL_MINOR : ERTS_PROC_LOCKS_ALL)); } else if (is_internal_port(tracer)) { - Port *tracer_port = erts_id2port(tracer, p, ERTS_PROC_LOCK_MAIN); - if (!erts_is_valid_tracer_port(tracer)) { - if (tracer_port) - erts_smp_port_unlock(tracer_port); + Port *tracer_port = erts_id2port_sflgs(tracer, + p, + ERTS_PROC_LOCK_MAIN, + ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); + if (!tracer_port) goto error; - } - tracer_port->trace_flags |= F_TRACER; - erts_smp_port_unlock(tracer_port); + ERTS_TRACE_FLAGS(tracer_port) |= F_TRACER; + erts_port_release(tracer_port); } else goto error; @@ -554,7 +552,7 @@ Eterm trace_3(BIF_ALIST_3) case am_true: on = 1; if (is_nil(tracer)) - tracer = p->id; + tracer = p->common.id; break; default: goto error; @@ -576,26 +574,29 @@ Eterm trace_3(BIF_ALIST_3) if (pid_spec == tracer) goto error; - tracee_port = erts_id2port(pid_spec, p, ERTS_PROC_LOCK_MAIN); + tracee_port = erts_id2port_sflgs(pid_spec, + p, + ERTS_PROC_LOCK_MAIN, + ERTS_PORT_SFLGS_INVALID_LOOKUP); if (!tracee_port) goto error; if (tracer != NIL && port_already_traced(p, tracee_port, tracer)) { - erts_smp_port_unlock(tracee_port); + erts_port_release(tracee_port); goto already_traced; } if (on) - tracee_port->trace_flags |= mask; + ERTS_TRACE_FLAGS(tracee_port) |= mask; else - tracee_port->trace_flags &= ~mask; + ERTS_TRACE_FLAGS(tracee_port) &= ~mask; - if (!tracee_port->trace_flags) - tracee_port->tracer_proc = NIL; + if (!ERTS_TRACE_FLAGS(tracee_port)) + ERTS_TRACER_PROC(tracee_port) = NIL; else if (tracer != NIL) - tracee_port->tracer_proc = tracer; + ERTS_TRACER_PROC(tracee_port) = tracer; - erts_smp_port_unlock(tracee_port); + erts_port_release(tracee_port); matches = 1; } else if (is_pid(pid_spec)) { @@ -627,14 +628,14 @@ Eterm trace_3(BIF_ALIST_3) } if (on) - tracee_p->trace_flags |= mask; + ERTS_TRACE_FLAGS(tracee_p) |= mask; else - tracee_p->trace_flags &= ~mask; + ERTS_TRACE_FLAGS(tracee_p) &= ~mask; - if ((tracee_p->trace_flags & TRACEE_FLAGS) == 0) - tracee_p->tracer_proc = NIL; + if ((ERTS_TRACE_FLAGS(tracee_p) & TRACEE_FLAGS) == 0) + ERTS_TRACER_PROC(tracee_p) = NIL; else if (tracer != NIL) - tracee_p->tracer_proc = tracer; + ERTS_TRACER_PROC(tracee_p) = tracer; erts_smp_proc_unlock(tracee_p, (tracee_p == p @@ -708,47 +709,56 @@ Eterm trace_3(BIF_ALIST_3) ok = 1; if (procs || mods) { + int max = erts_ptab_max(&erts_proc); /* tracing of processes */ - for (i = 0; i < erts_max_processes; i++) { + for (i = 0; i < max; i++) { Process* tracee_p = erts_pix2proc(i); if (! tracee_p) continue; if (tracer != NIL) { - if (tracee_p->id == tracer) + if (tracee_p->common.id == tracer) continue; if (already_traced(NULL, tracee_p, tracer)) continue; } if (on) { - tracee_p->trace_flags |= mask; + ERTS_TRACE_FLAGS(tracee_p) |= mask; } else { - tracee_p->trace_flags &= ~mask; + ERTS_TRACE_FLAGS(tracee_p) &= ~mask; } - if(!(tracee_p->trace_flags & TRACEE_FLAGS)) { - tracee_p->tracer_proc = NIL; + if(!(ERTS_TRACE_FLAGS(tracee_p) & TRACEE_FLAGS)) { + ERTS_TRACER_PROC(tracee_p) = NIL; } else if (tracer != NIL) { - tracee_p->tracer_proc = tracer; + ERTS_TRACER_PROC(tracee_p) = tracer; } matches++; } } if (ports || mods) { + int max = erts_ptab_max(&erts_port); /* tracing of ports */ - for (i = 0; i < erts_max_ports; i++) { - Port *tracee_port = &erts_port[i]; - if (tracee_port->status & ERTS_PORT_SFLGS_DEAD) continue; + for (i = 0; i < max; i++) { + erts_aint32_t state; + Port *tracee_port = erts_pix2port(i); + if (!tracee_port) + continue; + state = erts_atomic32_read_nob(&tracee_port->state); + if (state & ERTS_PORT_SFLGS_DEAD) + continue; if (tracer != NIL) { - if (tracee_port->id == tracer) continue; - if (port_already_traced(NULL, tracee_port, tracer)) continue; + if (tracee_port->common.id == tracer) + continue; + if (port_already_traced(NULL, tracee_port, tracer)) + continue; } - if (on) tracee_port->trace_flags |= mask; - else tracee_port->trace_flags &= ~mask; + if (on) ERTS_TRACE_FLAGS(tracee_port) |= mask; + else ERTS_TRACE_FLAGS(tracee_port) &= ~mask; - if (!(tracee_port->trace_flags & TRACEE_FLAGS)) { - tracee_port->tracer_proc = NIL; + if (!(ERTS_TRACE_FLAGS(tracee_port) & TRACEE_FLAGS)) { + ERTS_TRACER_PROC(tracee_port) = NIL; } else if (tracer != NIL) { - tracee_port->tracer_proc = tracer; + ERTS_TRACER_PROC(tracee_port) = tracer; } /* matches are not counted for ports since it would violate compatibility */ /* This could be a reason to modify this function or make a new one. */ @@ -817,20 +827,20 @@ static int port_already_traced(Process *c_p, Port *tracee_port, Eterm tracer) * * main lock is held on c_p * * all locks are held on port tracee_p */ - if ((tracee_port->trace_flags & TRACEE_FLAGS) - && tracee_port->tracer_proc != tracer) { + if ((ERTS_TRACE_FLAGS(tracee_port) & TRACEE_FLAGS) + && ERTS_TRACER_PROC(tracee_port) != tracer) { /* This tracee is already being traced, and not by the * tracer to be */ - if (is_internal_port(tracee_port->tracer_proc)) { - if (!erts_is_valid_tracer_port(tracee_port->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(tracee_port))) { + if (!erts_is_valid_tracer_port(ERTS_TRACER_PROC(tracee_port))) { /* Current trace port now invalid * - discard it and approve the new. */ goto remove_tracer; } else return 1; } - else if(is_internal_pid(tracee_port->tracer_proc)) { - Process *tracer_p = erts_proc_lookup(tracee_port->tracer_proc); + else if(is_internal_pid(ERTS_TRACER_PROC(tracee_port))) { + Process *tracer_p = erts_proc_lookup(ERTS_TRACER_PROC(tracee_port)); if (!tracer_p) { /* Current trace process now invalid * - discard it and approve the new. */ @@ -840,8 +850,8 @@ static int port_already_traced(Process *c_p, Port *tracee_port, Eterm tracer) } else { remove_tracer: - tracee_port->trace_flags &= ~TRACEE_FLAGS; - tracee_port->tracer_proc = NIL; + ERTS_TRACE_FLAGS(tracee_port) &= ~TRACEE_FLAGS; + ERTS_TRACER_PROC(tracee_port) = NIL; } } return 0; @@ -857,20 +867,22 @@ static int already_traced(Process *c_p, Process *tracee_p, Eterm tracer) * * main lock is held on c_p * * all locks multiple are held on tracee_p */ - if ((tracee_p->trace_flags & TRACEE_FLAGS) - && tracee_p->tracer_proc != tracer) { + if ((ERTS_TRACE_FLAGS(tracee_p) & TRACEE_FLAGS) + && ERTS_TRACER_PROC(tracee_p) != tracer) { /* This tracee is already being traced, and not by the * tracer to be */ - if (is_internal_port(tracee_p->tracer_proc)) { - if (!erts_is_valid_tracer_port(tracee_p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(tracee_p))) { + if (!erts_is_valid_tracer_port(ERTS_TRACER_PROC(tracee_p))) { /* Current trace port now invalid * - discard it and approve the new. */ goto remove_tracer; } else return 1; } - else if(is_internal_pid(tracee_p->tracer_proc)) { - Process *tracer_p = erts_proc_lookup(tracee_p->tracer_proc); + else if(is_internal_pid(ERTS_TRACER_PROC(tracee_p))) { + Process *tracer_p; + + tracer_p = erts_proc_lookup(ERTS_TRACER_PROC(tracee_p)); if (!tracer_p) { /* Current trace process now invalid * - discard it and approve the new. */ @@ -880,8 +892,8 @@ static int already_traced(Process *c_p, Process *tracee_p, Eterm tracer) } else { remove_tracer: - tracee_p->trace_flags &= ~TRACEE_FLAGS; - tracee_p->tracer_proc = NIL; + ERTS_TRACE_FLAGS(tracee_p) &= ~TRACEE_FLAGS; + ERTS_TRACER_PROC(tracee_p) = NIL; } } return 0; @@ -925,8 +937,7 @@ trace_info_pid(Process* p, Eterm pid_spec, Eterm key) if (pid_spec == am_new) { erts_get_default_tracing(&trace_flags, &tracer); - } else if (is_internal_pid(pid_spec) - && internal_pid_index(pid_spec) < erts_max_processes) { + } else if (is_internal_pid(pid_spec)) { Process *tracee; tracee = erts_pid2proc(p, ERTS_PROC_LOCK_MAIN, pid_spec, ERTS_PROC_LOCKS_ALL); @@ -934,16 +945,16 @@ trace_info_pid(Process* p, Eterm pid_spec, Eterm key) if (!tracee) { return am_undefined; } else { - tracer = tracee->tracer_proc; - trace_flags = tracee->trace_flags; + tracer = ERTS_TRACER_PROC(tracee); + trace_flags = ERTS_TRACE_FLAGS(tracee); } if (is_internal_pid(tracer)) { if (!erts_proc_lookup(tracer)) { reset_tracer: - tracee->trace_flags &= ~TRACEE_FLAGS; - trace_flags = tracee->trace_flags; - tracer = tracee->tracer_proc = NIL; + ERTS_TRACE_FLAGS(tracee) &= ~TRACEE_FLAGS; + trace_flags = ERTS_TRACE_FLAGS(tracee); + tracer = ERTS_TRACER_PROC(tracee) = NIL; } } else if (is_internal_port(tracer)) { @@ -1877,7 +1888,7 @@ new_seq_trace_token(Process* p) SEQ_TRACE_TOKEN(p) = TUPLE5(hp, make_small(0), /* Flags */ make_small(0), /* Label */ make_small(0), /* Serial */ - p->id, /* Internal pid */ /* From */ + p->common.id, /* Internal pid */ /* From */ make_small(p->seq_trace_lastcnt)); } } @@ -2247,9 +2258,11 @@ BIF_RETTYPE system_profile_2(BIF_ALIST_2) if (!profiler_p) goto error; } else if (is_internal_port(profiler)) { - if (internal_port_index(profiler) >= erts_max_ports) goto error; - profiler_port = &erts_port[internal_port_index(profiler)]; - if (INVALID_TRACER_PORT(profiler_port, profiler)) goto error; + profiler_port = (erts_port_lookup( + profiler, + ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP)); + if (!profiler_port) + goto error; } else { goto error; } @@ -2313,8 +2326,7 @@ trace_delivered_1(BIF_ALIST_1) p = NULL; } else if (! (p = erts_pid2proc(BIF_P, ERTS_PROC_LOCK_MAIN, BIF_ARG_1, ERTS_PROC_LOCKS_ALL))) { - if (is_not_internal_pid(BIF_ARG_1) - || internal_pid_index(BIF_ARG_1) >= erts_max_processes) { + if (is_not_internal_pid(BIF_ARG_1)) { BIF_ERROR(BIF_P, BADARG); } } @@ -2333,7 +2345,7 @@ trace_delivered_1(BIF_ALIST_1) msg = TUPLE3(hp, AM_trace_delivered, BIF_ARG_1, msg_ref); #ifdef ERTS_SMP - erts_send_sys_msg_proc(BIF_P->id, BIF_P->id, msg, bp); + erts_send_sys_msg_proc(BIF_P->common.id, BIF_P->common.id, msg, bp); if (p) erts_smp_proc_unlock(p, (BIF_P == p diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c index 1ba1048afa..48a95cdf32 100644 --- a/erts/emulator/beam/erl_db.c +++ b/erts/emulator/beam/erl_db.c @@ -425,7 +425,8 @@ DbTable* db_get_table_aux(Process *p, if (tb) { db_lock(tb, kind); if (tb->common.id != id - || ((tb->common.status & what) == 0 && p->id != tb->common.owner)) { + || ((tb->common.status & what) == 0 + && p->common.id != tb->common.owner)) { db_unlock(tb, kind); tb = NULL; } @@ -615,7 +616,7 @@ BIF_RETTYPE ets_safe_fixtable_2(BIF_ALIST_2) #ifdef HARDDEBUG erts_fprintf(stderr, "ets:safe_fixtable(%T,%T); Process: %T, initial: %T:%T/%bpu\n", - BIF_ARG_1, BIF_ARG_2, BIF_P->id, + BIF_ARG_1, BIF_ARG_2, BIF_P->common.id, BIF_P->initial[0], BIF_P->initial[1], BIF_P->initial[2]); #endif kind = (BIF_ARG_2 == am_true) ? LCK_READ : LCK_WRITE_REC; @@ -1194,7 +1195,7 @@ BIF_RETTYPE ets_rename_2(BIF_ALIST_2) #ifdef HARDDEBUG erts_fprintf(stderr, "ets:rename(%T,%T); Process: %T, initial: %T:%T/%bpu\n", - BIF_ARG_1, BIF_ARG_2, BIF_P->id, + BIF_ARG_1, BIF_ARG_2, BIF_P->common.id, BIF_P->initial[0], BIF_P->initial[1], BIF_P->initial[2]); #endif @@ -1437,7 +1438,7 @@ BIF_RETTYPE ets_new_2(BIF_ALIST_2) db_init_lock(tb, status & (DB_FINE_LOCKED|DB_FREQ_READ), "db_tab", "db_tab_fix"); tb->common.keypos = keypos; - tb->common.owner = BIF_P->id; + tb->common.owner = BIF_P->common.id; set_heir(BIF_P, tb, heir, heir_data); erts_smp_atomic_init_nob(&tb->common.nitems, 0); @@ -1506,7 +1507,7 @@ BIF_RETTYPE ets_new_2(BIF_ALIST_2) #ifdef HARDDEBUG erts_fprintf(stderr, "ets:new(%T,%T)=%T; Process: %T, initial: %T:%T/%bpu\n", - BIF_ARG_1, BIF_ARG_2, ret, BIF_P->id, + BIF_ARG_1, BIF_ARG_2, ret, BIF_P->common.id, BIF_P->initial[0], BIF_P->initial[1], BIF_P->initial[2]); erts_fprintf(stderr, "ets: new: meta_pid_to_tab common.memory_size = %ld\n", erts_smp_atomic_read_nob(&meta_pid_to_tab->common.memory_size)); @@ -1518,7 +1519,9 @@ BIF_RETTYPE ets_new_2(BIF_ALIST_2) db_meta_lock(meta_pid_to_tab, LCK_WRITE_REC); if (db_put_hash(meta_pid_to_tab, - TUPLE2(meta_tuple, BIF_P->id, make_small(slot)), + TUPLE2(meta_tuple, + BIF_P->common.id, + make_small(slot)), 0) != DB_ERROR_NONE) { erl_exit(1,"Could not update ets metadata."); } @@ -1637,7 +1640,7 @@ BIF_RETTYPE ets_delete_1(BIF_ALIST_1) #ifdef HARDDEBUG erts_fprintf(stderr, "ets:delete(%T); Process: %T, initial: %T:%T/%bpu\n", - BIF_ARG_1, BIF_P->id, + BIF_ARG_1, BIF_P->common.id, BIF_P->initial[0], BIF_P->initial[1], BIF_P->initial[2]); #endif @@ -1654,7 +1657,7 @@ BIF_RETTYPE ets_delete_1(BIF_ALIST_1) tb->common.status &= ~(DB_PROTECTED|DB_PUBLIC|DB_PRIVATE); tb->common.status |= DB_DELETE; - if (tb->common.owner != BIF_P->id) { + if (tb->common.owner != BIF_P->common.id) { DeclareTmpHeap(meta_tuple,3,BIF_P); /* @@ -1669,10 +1672,12 @@ BIF_RETTYPE ets_delete_1(BIF_ALIST_1) make_small(tb->common.slot)); BIF_P->flags |= F_USING_DB; - tb->common.owner = BIF_P->id; + tb->common.owner = BIF_P->common.id; db_put_hash(meta_pid_to_tab, - TUPLE2(meta_tuple,BIF_P->id,make_small(tb->common.slot)), + TUPLE2(meta_tuple, + BIF_P->common.id, + make_small(tb->common.slot)), 0); db_meta_unlock(meta_pid_to_tab, LCK_WRITE_REC); UnUseTmpHeap(3,BIF_P); @@ -1748,7 +1753,7 @@ BIF_RETTYPE ets_give_away_3(BIF_ALIST_3) } if ((tb = db_get_table(BIF_P, BIF_ARG_1, DB_WRITE, LCK_WRITE)) == NULL - || tb->common.owner != BIF_P->id) { + || tb->common.owner != BIF_P->common.id) { goto badarg; } from_pid = tb->common.owner; @@ -1771,7 +1776,10 @@ BIF_RETTYPE ets_give_away_3(BIF_ALIST_3) db_unlock(tb,LCK_WRITE); erts_send_message(BIF_P, to_proc, &to_locks, - TUPLE4(buf, am_ETS_TRANSFER, tb->common.id, from_pid, BIF_ARG_3), + TUPLE4(buf, am_ETS_TRANSFER, + tb->common.id, + from_pid, + BIF_ARG_3), 0); erts_smp_proc_unlock(to_proc, to_locks); UnUseTmpHeap(5,BIF_P); @@ -1833,7 +1841,7 @@ BIF_RETTYPE ets_setopts_2(BIF_ALIST_2) if (tail != NIL || (tb = db_get_table(BIF_P, BIF_ARG_1, DB_WRITE, LCK_WRITE)) == NULL - || tb->common.owner != BIF_P->id) { + || tb->common.owner != BIF_P->common.id) { goto badarg; } @@ -2647,7 +2655,7 @@ BIF_RETTYPE ets_info_1(BIF_ALIST_1) */ /* If/when we implement lockless private tables: - if ((tb->common.status & DB_PRIVATE) && owner != BIF_P->id) { + if ((tb->common.status & DB_PRIVATE) && owner != BIF_P->common.id) { db_unlock(tb, LCK_READ); rp = erts_pid2proc_not_running(BIF_P, ERTS_PROC_LOCK_MAIN, owner, ERTS_PROC_LOCK_MAIN); @@ -3039,9 +3047,9 @@ static int give_away_to_heir(Process* p, DbTable* tb) Eterm to_pid; UWord heir_data; - ASSERT(tb->common.owner == p->id); + ASSERT(tb->common.owner == p->common.id); ASSERT(is_internal_pid(tb->common.heir)); - ASSERT(tb->common.heir != p->id); + ASSERT(tb->common.heir != p->common.id); retry: to_pid = tb->common.heir; to_proc = erts_pid2proc_opt(p, ERTS_PROC_LOCK_MAIN, @@ -3054,7 +3062,7 @@ retry: db_lock(tb,LCK_WRITE); ASSERT(tb != NULL); - if (tb->common.owner != p->id) { + if (tb->common.owner != p->common.id) { if (to_proc != NULL ) { erts_smp_proc_unlock(to_proc, to_locks); } @@ -3065,7 +3073,7 @@ retry: if (to_proc != NULL ) { erts_smp_proc_unlock(to_proc, to_locks); } - if (to_pid == p->id || to_pid == am_none) { + if (to_pid == p->common.id || to_pid == am_none) { return 0; /* no real heir, table still mine */ } goto retry; @@ -3074,7 +3082,8 @@ retry: if (to_proc == NULL) { return 0; /* heir not alive, table still mine */ } - if (to_proc->started_interval != tb->common.heir_started_interval) { + if (to_proc->common.u.alive.started_interval + != tb->common.heir_started_interval) { erts_smp_proc_unlock(to_proc, to_locks); return 0; /* heir dead and pid reused, table still mine */ } @@ -3099,7 +3108,11 @@ retry: heir_data = tpv[1]; } erts_send_message(p, to_proc, &to_locks, - TUPLE4(buf, am_ETS_TRANSFER, tb->common.id, p->id, heir_data), + TUPLE4(buf, + am_ETS_TRANSFER, + tb->common.id, + p->common.id, + heir_data), 0); erts_smp_proc_unlock(to_proc, to_locks); return !0; @@ -3108,7 +3121,7 @@ retry: /* * erts_db_process_exiting() is called when a process terminates. * It returns 0 when completely done, and !0 when it wants to - * yield. c_p->u.exit_data can hold a pointer to a state while + * yield. c_p->u.terminate can hold a pointer to a state while * yielding. */ #define ERTS_DB_INTERNAL_ERROR(LSTR) \ @@ -3118,8 +3131,8 @@ retry: int erts_db_process_exiting(Process *c_p, ErtsProcLocks c_p_locks) { - ErtsDbProcCleanupState *state = (ErtsDbProcCleanupState *) c_p->u.exit_data; - Eterm pid = c_p->id; + ErtsDbProcCleanupState *state = (ErtsDbProcCleanupState *) c_p->u.terminate; + Eterm pid = c_p->common.id; ErtsDbProcCleanupState default_state; int ret; @@ -3300,7 +3313,7 @@ erts_db_process_exiting(Process *c_p, ErtsProcLocks c_p_locks) if (state != &default_state) erts_free(ERTS_ALC_T_DB_PROC_CLEANUP, state); - c_p->u.exit_data = NULL; + c_p->u.terminate = NULL; return 0; default: @@ -3321,13 +3334,13 @@ erts_db_process_exiting(Process *c_p, ErtsProcLocks c_p_locks) break; } - ASSERT(c_p->u.exit_data == (void *) state + ASSERT(c_p->u.terminate == (void *) state || state == &default_state); if (state == &default_state) { - c_p->u.exit_data = erts_alloc(ERTS_ALC_T_DB_PROC_CLEANUP, + c_p->u.terminate = erts_alloc(ERTS_ALC_T_DB_PROC_CLEANUP, sizeof(ErtsDbProcCleanupState)); - sys_memcpy(c_p->u.exit_data, + sys_memcpy(c_p->u.terminate, (void*) state, sizeof(ErtsDbProcCleanupState)); } @@ -3353,7 +3366,7 @@ static void fix_table_locked(Process* p, DbTable* tb) } else { for (; fix != NULL; fix = fix->next) { - if (fix->pid == p->id) { + if (fix->pid == p->common.id) { ++(fix->counter); #ifdef ERTS_SMP erts_smp_mtx_unlock(&tb->common.fixlock); @@ -3365,7 +3378,7 @@ static void fix_table_locked(Process* p, DbTable* tb) fix = (DbFixation *) erts_db_alloc(ERTS_ALC_T_DB_FIXATION, tb, sizeof(DbFixation)); ERTS_ETS_MISC_MEM_ADD(sizeof(DbFixation)); - fix->pid = p->id; + fix->pid = p->common.id; fix->counter = 1; fix->next = tb->common.fixations; tb->common.fixations = fix; @@ -3376,7 +3389,9 @@ static void fix_table_locked(Process* p, DbTable* tb) UseTmpHeap(3,p); db_meta_lock(meta_pid_to_fixed_tab, LCK_WRITE_REC); if (db_put_hash(meta_pid_to_fixed_tab, - TUPLE2(meta_tuple, p->id, make_small(tb->common.slot)), + TUPLE2(meta_tuple, + p->common.id, + make_small(tb->common.slot)), 0) != DB_ERROR_NONE) { UnUseTmpHeap(3,p); erl_exit(1,"Could not insert ets metadata in safe_fixtable."); @@ -3396,7 +3411,7 @@ static void unfix_table_locked(Process* p, DbTable* tb, erts_smp_mtx_lock(&tb->common.fixlock); #endif for (pp = &tb->common.fixations; *pp != NULL; pp = &(*pp)->next) { - if ((*pp)->pid == p->id) { + if ((*pp)->pid == p->common.id) { DbFixation* fix = *pp; erts_refc_dec(&tb->common.ref,0); --(fix->counter); @@ -3410,7 +3425,7 @@ static void unfix_table_locked(Process* p, DbTable* tb, #endif db_meta_lock(meta_pid_to_fixed_tab, LCK_WRITE_REC); db_erase_bag_exact2(meta_pid_to_fixed_tab, - p->id, make_small(tb->common.slot)); + p->common.id, make_small(tb->common.slot)); db_meta_unlock(meta_pid_to_fixed_tab, LCK_WRITE_REC); erts_db_free(ERTS_ALC_T_DB_FIXATION, tb, (void *) fix, sizeof(DbFixation)); @@ -3469,15 +3484,15 @@ static void set_heir(Process* me, DbTable* tb, Eterm heir, UWord heir_data) if (heir == am_none) { return; } - if (heir == me->id) { - erts_ensure_later_proc_interval(me->started_interval); - tb->common.heir_started_interval = me->started_interval; + if (heir == me->common.id) { + erts_ensure_later_proc_interval(me->common.u.alive.started_interval); + tb->common.heir_started_interval = me->common.u.alive.started_interval; } else { Process* heir_proc= erts_proc_lookup(heir); if (heir_proc != NULL) { - erts_ensure_later_proc_interval(heir_proc->started_interval); - tb->common.heir_started_interval = heir_proc->started_interval; + erts_ensure_later_proc_interval(heir_proc->common.u.alive.started_interval); + tb->common.heir_started_interval = heir_proc->common.u.alive.started_interval; } else { tb->common.heir = am_none; } diff --git a/erts/emulator/beam/erl_db_util.c b/erts/emulator/beam/erl_db_util.c index 0c9ca83ce4..bcdb630140 100644 --- a/erts/emulator/beam/erl_db_util.c +++ b/erts/emulator/beam/erl_db_util.c @@ -138,21 +138,23 @@ set_tracee_flags(Process *tracee_p, Eterm tracer, Uint d_flags, Uint e_flags) { Uint flags; if (tracer == NIL) { - flags = tracee_p->trace_flags & ~TRACEE_FLAGS; + flags = ERTS_TRACE_FLAGS(tracee_p) & ~TRACEE_FLAGS; } else { - flags = ((tracee_p->trace_flags & ~d_flags) | e_flags); + flags = ((ERTS_TRACE_FLAGS(tracee_p) & ~d_flags) | e_flags); if (! flags) tracer = NIL; } - ret = tracee_p->tracer_proc != tracer || tracee_p->trace_flags != flags - ? am_true : am_false; - tracee_p->tracer_proc = tracer; - tracee_p->trace_flags = flags; + ret = ((ERTS_TRACER_PROC(tracee_p) != tracer + || ERTS_TRACE_FLAGS(tracee_p) != flags) + ? am_true + : am_false); + ERTS_TRACER_PROC(tracee_p) = tracer; + ERTS_TRACE_FLAGS(tracee_p) = flags; return ret; } /* ** Assuming all locks on tracee_p on entry ** -** Changes tracee_p->trace_flags and tracee_p->tracer_proc +** Changes ERTS_TRACE_FLAGS(tracee_p) and ERTS_TRACER_PROC(tracee_p) ** according to input disable/enable flags and tracer. ** ** Returns am_true|am_false on success, am_true if value changed, @@ -173,17 +175,20 @@ set_match_trace(Process *tracee_p, Eterm fail_term, Eterm tracer, tracer, ERTS_PROC_LOCKS_ALL))) { if (tracee_p != tracer_p) { ret = set_tracee_flags(tracee_p, tracer, d_flags, e_flags); - tracer_p->trace_flags |= tracee_p->trace_flags ? F_TRACER : 0; + ERTS_TRACE_FLAGS(tracer_p) |= (ERTS_TRACE_FLAGS(tracee_p) + ? F_TRACER + : 0); erts_smp_proc_unlock(tracer_p, ERTS_PROC_LOCKS_ALL); } } else if (is_internal_port(tracer)) { Port *tracer_port = - erts_id2port(tracer, tracee_p, ERTS_PROC_LOCKS_ALL); + erts_id2port_sflgs(tracer, + tracee_p, + ERTS_PROC_LOCKS_ALL, + ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); if (tracer_port) { - if (! INVALID_TRACER_PORT(tracer_port, tracer)) { - ret = set_tracee_flags(tracee_p, tracer, d_flags, e_flags); - } - erts_smp_port_unlock(tracer_port); + ret = set_tracee_flags(tracee_p, tracer, d_flags, e_flags); + erts_port_release(tracer_port); } } else { ASSERT(is_nil(tracer)); @@ -2174,7 +2179,7 @@ restart: pc += n; break; case matchSelf: - *esp++ = c_p->id; + *esp++ = c_p->common.id; break; case matchWaste: --esp; @@ -2261,7 +2266,7 @@ restart: case matchEnableTrace: if ( (n = erts_trace_flag2bit(esp[-1]))) { BEGIN_ATOMIC_TRACE(c_p); - set_tracee_flags(c_p, c_p->tracer_proc, 0, n); + set_tracee_flags(c_p, ERTS_TRACER_PROC(c_p), 0, n); esp[-1] = am_true; } else { esp[-1] = FAIL_TERM; @@ -2274,7 +2279,7 @@ restart: BEGIN_ATOMIC_TRACE(c_p); if ( (tmpp = get_proc(c_p, 0, esp[0], 0))) { /* Always take over the tracer of the current process */ - set_tracee_flags(tmpp, c_p->tracer_proc, 0, n); + set_tracee_flags(tmpp, ERTS_TRACER_PROC(c_p), 0, n); esp[-1] = am_true; } } @@ -2282,7 +2287,7 @@ restart: case matchDisableTrace: if ( (n = erts_trace_flag2bit(esp[-1]))) { BEGIN_ATOMIC_TRACE(c_p); - set_tracee_flags(c_p, c_p->tracer_proc, n, 0); + set_tracee_flags(c_p, ERTS_TRACER_PROC(c_p), n, 0); esp[-1] = am_true; } else { esp[-1] = FAIL_TERM; @@ -2295,7 +2300,7 @@ restart: BEGIN_ATOMIC_TRACE(c_p); if ( (tmpp = get_proc(c_p, 0, esp[0], 0))) { /* Always take over the tracer of the current process */ - set_tracee_flags(tmpp, c_p->tracer_proc, n, 0); + set_tracee_flags(tmpp, ERTS_TRACER_PROC(c_p), n, 0); esp[-1] = am_true; } } @@ -2316,12 +2321,12 @@ restart: --esp; if (*esp == am_true) { erts_smp_proc_lock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); - c_p->trace_flags |= F_TRACE_SILENT; + ERTS_TRACE_FLAGS(c_p) |= F_TRACE_SILENT; erts_smp_proc_unlock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); } else if (*esp == am_false) { erts_smp_proc_lock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); - c_p->trace_flags &= ~F_TRACE_SILENT; + ERTS_TRACE_FLAGS(c_p) &= ~F_TRACE_SILENT; erts_smp_proc_unlock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); } break; @@ -2329,11 +2334,11 @@ restart: { /* disable enable */ Uint d_flags = 0, e_flags = 0; /* process trace flags */ - Eterm tracer = c_p->tracer_proc; + Eterm tracer = ERTS_TRACER_PROC(c_p); /* XXX Atomicity note: Not fully atomic. Default tracer * is sampled from current process but applied to * tracee and tracer later after releasing main - * locks on current process, so c_p->tracer_proc + * locks on current process, so ERTS_TRACER_PROC(c_p) * may actually have changed when tracee and tracer * gets updated. I do not think nobody will notice. * It is just the default value that is not fully atomic. @@ -2358,7 +2363,7 @@ restart: { /* disable enable */ Uint d_flags = 0, e_flags = 0; /* process trace flags */ - Eterm tracer = c_p->tracer_proc; + Eterm tracer = ERTS_TRACER_PROC(c_p); /* XXX Atomicity note. Not fully atomic. See above. * Above it could possibly be solved, but not here. */ diff --git a/erts/emulator/beam/erl_db_util.h b/erts/emulator/beam/erl_db_util.h index dcecc4251a..d8f6e40d2e 100644 --- a/erts/emulator/beam/erl_db_util.h +++ b/erts/emulator/beam/erl_db_util.h @@ -320,10 +320,10 @@ ERTS_GLB_INLINE int db_eq(DbTableCommon* tb, Eterm a, DbTerm* b) #define DB_INFO (DB_PROTECTED|DB_PUBLIC|DB_PRIVATE) #define ONLY_WRITER(P,T) (((T)->common.status & (DB_PRIVATE|DB_PROTECTED)) \ - && (T)->common.owner == (P)->id) + && (T)->common.owner == (P)->common.id) #define ONLY_READER(P,T) (((T)->common.status & DB_PRIVATE) && \ -(T)->common.owner == (P)->id) +(T)->common.owner == (P)->common.id) /* Function prototypes */ BIF_RETTYPE db_get_trace_control_word(Process* p); diff --git a/erts/emulator/beam/erl_debug.c b/erts/emulator/beam/erl_debug.c index 22e873afc6..b90d00f236 100644 --- a/erts/emulator/beam/erl_debug.c +++ b/erts/emulator/beam/erl_debug.c @@ -252,16 +252,16 @@ void erts_check_stack(Process *p) if (p->stop > stack_start) erl_exit(1, "<%lu.%lu.%lu>: Stack underflow\n", - internal_pid_channel_no(p->id), - internal_pid_number(p->id), - internal_pid_serial(p->id)); + internal_pid_channel_no(p->common.id), + internal_pid_number(p->common.id), + internal_pid_serial(p->common.id)); if (p->stop < stack_end) erl_exit(1, "<%lu.%lu.%lu>: Stack overflow\n", - internal_pid_channel_no(p->id), - internal_pid_number(p->id), - internal_pid_serial(p->id)); + internal_pid_channel_no(p->common.id), + internal_pid_number(p->common.id), + internal_pid_serial(p->common.id)); for (elemp = p->stop; elemp < stack_start; elemp++) { int in_mbuf = 0; @@ -284,9 +284,9 @@ void erts_check_stack(Process *p) erl_exit(1, "<%lu.%lu.%lu>: Wild stack pointer\n", - internal_pid_channel_no(p->id), - internal_pid_number(p->id), - internal_pid_serial(p->id)); + internal_pid_channel_no(p->common.id), + internal_pid_number(p->common.id), + internal_pid_serial(p->common.id)); } } @@ -387,16 +387,16 @@ void verify_process(Process *p) #define VERIFY_AREA(name,ptr,sz) { \ int n = (sz); \ while (n--) if(!verify_eterm(p,*(ptr+n))) \ - erl_exit(1,"Wild pointer found in " name " of %T!\n",p->id); } + erl_exit(1,"Wild pointer found in " name " of %T!\n",p->common.id); } #define VERIFY_ETERM(name,eterm) { \ if(!verify_eterm(p,eterm)) \ - erl_exit(1,"Wild pointer found in " name " of %T!\n",p->id); } + erl_exit(1,"Wild pointer found in " name " of %T!\n",p->common.id); } ErlMessage* mp = p->msg.first; - VERBOSE(DEBUG_MEMORY,("Verify process: %T...\n",p->id)); + VERBOSE(DEBUG_MEMORY,("Verify process: %T...\n",p->common.id)); while (mp != NULL) { VERIFY_ETERM("message term",ERL_MESSAGE_TERM(mp)); @@ -516,7 +516,7 @@ static void print_process_memory(Process *p) ErlHeapFragment* bp = MBUF(p); erts_printf("==============================\n"); - erts_printf("|| Memory info for %T ||\n",p->id); + erts_printf("|| Memory info for %T ||\n",p->common.id); erts_printf("==============================\n"); erts_printf("-- %-*s ---%s-%s-%s-%s--\n", @@ -601,7 +601,7 @@ void print_memory_info(Process *p) { if (p != NULL) { erts_printf("======================================\n"); - erts_printf("|| Memory info for %-12T ||\n",p->id); + erts_printf("|| Memory info for %-12T ||\n",p->common.id); erts_printf("======================================\n"); erts_printf("+- local heap ----%s-%s-%s-%s-+\n", dashes,dashes,dashes,dashes); diff --git a/erts/emulator/beam/erl_driver.h b/erts/emulator/beam/erl_driver.h index 771ee46d2b..046b46513f 100644 --- a/erts/emulator/beam/erl_driver.h +++ b/erts/emulator/beam/erl_driver.h @@ -133,7 +133,7 @@ typedef struct { #define ERL_DRV_EXTENDED_MARKER (0xfeeeeeed) #define ERL_DRV_EXTENDED_MAJOR_VERSION 2 -#define ERL_DRV_EXTENDED_MINOR_VERSION 0 +#define ERL_DRV_EXTENDED_MINOR_VERSION 1 /* * The emulator will refuse to load a driver with different major @@ -154,6 +154,7 @@ typedef struct { #define ERL_DRV_FLAG_USE_PORT_LOCKING (1 << 0) #define ERL_DRV_FLAG_SOFT_BUSY (1 << 1) +#define ERL_DRV_FLAG_NO_BUSY_MSGQ (1 << 2) /* * Integer types @@ -207,8 +208,8 @@ typedef struct erl_drv_binary { typedef struct _erl_drv_data* ErlDrvData; /* Data to be used by the driver itself. */ #ifndef ERL_SYS_DRV typedef struct _erl_drv_event* ErlDrvEvent; /* An event to be selected on. */ -typedef struct _erl_drv_port* ErlDrvPort; /* A port descriptor. */ #endif +typedef struct _erl_drv_port* ErlDrvPort; /* A port descriptor. */ typedef struct _erl_drv_port* ErlDrvThreadData; /* Thread data. */ #if !defined(__WIN32__) && !defined(_WIN32) && !defined(_WIN32_) && !defined(USE_SELECT) @@ -377,9 +378,18 @@ typedef struct erl_drv_entry { ErlDrvEntry* driver_init(void) #endif +#define ERL_DRV_BUSY_MSGQ_DISABLED (~((ErlDrvSizeT) 0)) +#define ERL_DRV_BUSY_MSGQ_READ_ONLY ((ErlDrvSizeT) 0) +#define ERL_DRV_BUSY_MSGQ_LIM_MAX (ERL_DRV_BUSY_MSGQ_DISABLED - 1) +#define ERL_DRV_BUSY_MSGQ_LIM_MIN ((ErlDrvSizeT) 1) + /* * These are the functions available for driver writers. */ +EXTERN void erl_drv_busy_msgq_limits(ErlDrvPort port, + ErlDrvSizeT *low, + ErlDrvSizeT *high); + EXTERN int driver_select(ErlDrvPort port, ErlDrvEvent event, int mode, int on); EXTERN int driver_event(ErlDrvPort port, ErlDrvEvent event, ErlDrvEventData event_data); @@ -594,11 +604,33 @@ EXTERN ErlDrvPort driver_create_port(ErlDrvPort creator_port, ErlDrvData drv_data); +/* + * driver_output_term() is deprecated, and scheduled for removal in + * OTP-R17. Use erl_drv_output_term() instead. For more information + * see the erl_driver(3) documentation. + */ +EXTERN int driver_output_term(ErlDrvPort ix, + ErlDrvTermData* data, + int len) ERL_DRV_DEPRECATED_FUNC; +/* + * driver_send_term() is deprecated, and scheduled for removal in + * OTP-R17. Use erl_drv_send_term() instead. For more information + * see the erl_driver(3) documentation. + */ +EXTERN int driver_send_term(ErlDrvPort ix, + ErlDrvTermData to, + ErlDrvTermData* data, + int len) ERL_DRV_DEPRECATED_FUNC; + /* output term data to the port owner */ -EXTERN int driver_output_term(ErlDrvPort ix, ErlDrvTermData* data, int len); +EXTERN int erl_drv_output_term(ErlDrvTermData port, + ErlDrvTermData* data, + int len); /* output term data to a specific process */ -EXTERN int driver_send_term(ErlDrvPort ix, ErlDrvTermData to, - ErlDrvTermData* data, int len); +EXTERN int erl_drv_send_term(ErlDrvTermData port, + ErlDrvTermData to, + ErlDrvTermData* data, + int len); /* Async IO functions */ EXTERN long driver_async(ErlDrvPort ix, diff --git a/erts/emulator/beam/erl_gc.c b/erts/emulator/beam/erl_gc.c index 6075a527c3..d377ba8f31 100644 --- a/erts/emulator/beam/erl_gc.c +++ b/erts/emulator/beam/erl_gc.c @@ -59,7 +59,7 @@ static Uint reclaimed; /* no of words reclaimed in GCs */ erts_fprintf(stderr, "htop=%p\n", (p)->htop); \ erts_fprintf(stderr, "heap=%p\n", (p)->heap); \ erl_exit(ERTS_ABORT_EXIT, "%s, line %d: %T: Overrun stack and heap\n", \ - __FILE__,__LINE__,(P)->id); \ + __FILE__,__LINE__,(P)->common.id); \ } #ifdef DEBUG @@ -336,6 +336,19 @@ erts_gc_after_bif_call(Process* p, Eterm result, Eterm* regs, Uint arity) return result; } +static ERTS_INLINE void reset_active_writer(Process *p) +{ + struct erl_off_heap_header* ptr; + ptr = MSO(p).first; + while (ptr) { + if (ptr->thing_word == HEADER_PROC_BIN) { + ProcBin *pbp = (ProcBin*) ptr; + pbp->flags &= ~PB_ACTIVE_WRITER; + } + ptr = ptr->next; + } +} + /* * Garbage collect a process. * @@ -391,6 +404,7 @@ erts_garbage_collect(Process* p, int need, Eterm* objv, int nobj) DTRACE2(gc_minor_end, pidbuf, reclaimed_now); } } + reset_active_writer(p); /* * Finish. @@ -1932,9 +1946,9 @@ setup_rootset(Process *p, Eterm *objv, int nobj, Rootset *rootset) n++; } #endif - ASSERT(is_nil(p->tracer_proc) || - is_internal_pid(p->tracer_proc) || - is_internal_port(p->tracer_proc)); + ASSERT(is_nil(ERTS_TRACER_PROC(p)) || + is_internal_pid(ERTS_TRACER_PROC(p)) || + is_internal_port(ERTS_TRACER_PROC(p))); ASSERT(is_pid(follow_moved(p->group_leader))); if (is_not_immed(p->group_leader)) { @@ -2166,7 +2180,6 @@ link_live_proc_bin(struct shrink_cand_data *shrink, if (pbp->flags & PB_ACTIVE_WRITER) { - pbp->flags &= ~PB_ACTIVE_WRITER; shrink->no_of_active++; } else { /* inactive */ diff --git a/erts/emulator/beam/erl_init.c b/erts/emulator/beam/erl_init.c index 4b90e5394a..175f04b15b 100644 --- a/erts/emulator/beam/erl_init.c +++ b/erts/emulator/beam/erl_init.c @@ -44,6 +44,7 @@ #include "erl_thr_progress.h" #include "erl_thr_queue.h" #include "erl_async.h" +#include "erl_ptab.h" #ifdef HIPE #include "hipe_mode_switch.h" /* for hipe_mode_switch_init() */ @@ -109,6 +110,11 @@ const int etp_lock_check = 1; #else const int etp_lock_check = 0; #endif +#ifdef WORDS_BIGENDIAN +const int etp_big_endian = 1; +#else +const int etp_big_endian = 0; +#endif /* * Note about VxWorks: All variables must be initialized by executable code, * not by an initializer. Otherwise a new instance of the emulator will @@ -121,9 +127,10 @@ extern void ConNormalExit(void); extern void ConWaitForExit(void); #endif -static void erl_init(int ncpu); - -#define ERTS_MIN_COMPAT_REL 7 +static void erl_init(int ncpu, + int proc_tab_sz, + int port_tab_sz, + int port_tab_sz_ignore_files); static erts_atomic_t exiting; @@ -206,8 +213,6 @@ ErtsModifiedTimings erts_modified_timings[] = { Export *erts_delay_trap = NULL; -int erts_use_r9_pids_ports; - int ignore_break; int replace_intr; @@ -271,12 +276,18 @@ void erts_short_init(void) { int ncpu = early_init(NULL, NULL); - erl_init(ncpu); + erl_init(ncpu, + ERTS_DEFAULT_MAX_PROCESSES, + ERTS_DEFAULT_MAX_PORTS, + 0); erts_initialized = 1; } static void -erl_init(int ncpu) +erl_init(int ncpu, + int proc_tab_sz, + int port_tab_sz, + int port_tab_sz_ignore_files) { init_benchmarking(); @@ -284,7 +295,7 @@ erl_init(int ncpu) erts_init_gc(); erts_init_time(); erts_init_sys_common_misc(); - erts_init_process(ncpu); + erts_init_process(ncpu, proc_tab_sz); erts_init_scheduling(no_schedulers, no_schedulers_online); erts_init_cpu_topology(); /* Must be after init_scheduling */ @@ -306,6 +317,7 @@ erl_init(int ncpu) erts_bif_info_init(); erts_ddll_init(); init_emulator(); + erts_ptab_init(); /* Must be after init_emulator() */ erts_bp_init(); init_db(); /* Must be after init_emulator */ erts_bif_timer_init(); @@ -313,7 +325,7 @@ erl_init(int ncpu) init_dist(); erl_drv_thr_init(); erts_init_async(); - init_io(); + erts_init_io(port_tab_sz, port_tab_sz_ignore_files); init_load(); erts_init_bif(); erts_init_bif_chksum(); @@ -457,6 +469,7 @@ load_preloaded(void) /* be helpful (or maybe downright rude:-) */ void erts_usage(void) { + int this_rel = this_rel_num(); erts_fprintf(stderr, "Usage: %s [flags] [ -- [init_args] ]\n", progname(program)); erts_fprintf(stderr, "The flags are:\n\n"); @@ -490,16 +503,20 @@ void erts_usage(void) /* erts_fprintf(stderr, "-i module set the boot module (default init)\n"); */ erts_fprintf(stderr, "-K boolean enable or disable kernel poll\n"); - + erts_fprintf(stderr, "-n[s|a|d] Control behavior of signals to ports\n"); + erts_fprintf(stderr, " Note that this flag is deprecated!\n"); erts_fprintf(stderr, "-M<X> <Y> memory allocator switches,\n"); erts_fprintf(stderr, " see the erts_alloc(3) documentation for more info.\n"); erts_fprintf(stderr, "-P number set maximum number of processes on this node,\n"); erts_fprintf(stderr, " valid range is [%d-%d]\n", - ERTS_MIN_PROCESSES, ERTS_MAX_PROCESSES); + ERTS_MIN_PROCESSES, ERTS_MAX_PROCESSES); + erts_fprintf(stderr, "-Q number set maximum number of ports on this node,\n"); + erts_fprintf(stderr, " valid range is [%d-%d]\n", + ERTS_MIN_PORTS, ERTS_MAX_PORTS); erts_fprintf(stderr, "-R number set compatibility release number,\n"); erts_fprintf(stderr, " valid range [%d-%d]\n", - ERTS_MIN_COMPAT_REL, this_rel_num()); + this_rel-2, this_rel); erts_fprintf(stderr, "-r force ets memory block to be moved on realloc\n"); erts_fprintf(stderr, "-rg amount set reader groups limit\n"); @@ -519,6 +536,7 @@ void erts_usage(void) erts_fprintf(stderr, " valid range is [%d-%d]\n", ERTS_SCHED_THREAD_MIN_STACK_SIZE, ERTS_SCHED_THREAD_MAX_STACK_SIZE); + erts_fprintf(stderr, "-spp Bool set port parallelism scheduling hint\n"); erts_fprintf(stderr, "-S n1:n2 set number of schedulers (n1), and number of\n"); erts_fprintf(stderr, " schedulers online (n2), valid range for both\n"); erts_fprintf(stderr, " numbers are [1-%d]\n", @@ -612,7 +630,6 @@ early_init(int *argc, char **argv) /* erts_printf_eterm_func = erts_printf_term; erts_disable_tolerant_timeofday = 0; display_items = 200; - erts_proc.max = ERTS_DEFAULT_MAX_PROCESSES; erts_backtrace_depth = DEFAULT_BACKTRACE_SIZE; erts_async_max_threads = 0; erts_async_thread_suggested_stack_size = ERTS_ASYNC_THREAD_MIN_STACK_SIZE; @@ -641,8 +658,6 @@ early_init(int *argc, char **argv) /* erts_compat_rel = this_rel_num(); - erts_use_r9_pids_ports = 0; - erts_sys_pre_init(); erts_atomic_init_nob(&exiting, 0); #ifdef ERTS_SMP @@ -897,11 +912,13 @@ erl_start(int argc, char **argv) { int i = 1; char* arg=NULL; - char* Parg = NULL; int have_break_handler = 1; char envbuf[21]; /* enough for any 64-bit integer */ size_t envbufsz; int ncpu = early_init(&argc, argv); + int proc_tab_sz = ERTS_DEFAULT_MAX_PROCESSES; + int port_tab_sz = ERTS_DEFAULT_MAX_PORTS; + int port_tab_sz_ignore_files = 0; envbufsz = sizeof(envbuf); if (erts_sys_getenv_raw(ERL_MAX_ETS_TABLES_ENV, envbuf, &envbufsz) == 0) @@ -1152,12 +1169,53 @@ erl_start(int argc, char **argv) arg); break; - case 'P': - /* set maximum number of processes */ - Parg = get_arg(argv[i]+2, argv[i+1], &i); - erts_proc.max = atoi(Parg); - /* Check of result is delayed until later. This is because +R - may be given after +P. */ + case 'n': + arg = get_arg(argv[i]+2, argv[i+1], &i); + switch (arg[0]) { + case 's': /* synchronous */ + erts_port_synchronous_ops = 1; + erts_port_schedule_all_ops = 0; + break; + case 'a': /* asynchronous */ + erts_port_synchronous_ops = 0; + erts_port_schedule_all_ops = 1; + break; + case 'd': /* Default - schedule on conflict (asynchronous) */ + erts_port_synchronous_ops = 0; + erts_port_schedule_all_ops = 0; + break; + default: + bad_n_option: + erts_fprintf(stderr, "bad -n option %s\n", arg); + erts_usage(); + } + if (arg[1] != '\0') + goto bad_n_option; + break; + + case 'P': /* set maximum number of processes */ + arg = get_arg(argv[i]+2, argv[i+1], &i); + errno = 0; + proc_tab_sz = strtol(arg, NULL, 10); + if (errno != 0 + || proc_tab_sz < ERTS_MIN_PROCESSES + || ERTS_MAX_PROCESSES < proc_tab_sz) { + erts_fprintf(stderr, "bad number of processes %s\n", arg); + erts_usage(); + } + break; + + case 'Q': /* set maximum number of ports */ + arg = get_arg(argv[i]+2, argv[i+1], &i); + errno = 0; + port_tab_sz = strtol(arg, NULL, 10); + if (errno != 0 + || port_tab_sz < ERTS_MIN_PROCESSES + || ERTS_MAX_PROCESSES < port_tab_sz) { + erts_fprintf(stderr, "bad number of ports %s\n", arg); + erts_usage(); + } + port_tab_sz_ignore_files = 1; break; case 'S' : /* Was handled in early_init() just read past it */ @@ -1259,6 +1317,19 @@ erl_start(int argc, char **argv) erts_usage(); } } + else if (has_prefix("pp", sub_param)) { + arg = get_arg(sub_param+2, argv[i+1], &i); + if (sys_strcmp(arg, "true") == 0) + erts_port_parallelism = 1; + else if (sys_strcmp(arg, "false") == 0) + erts_port_parallelism = 0; + else { + erts_fprintf(stderr, + "bad port parallelism scheduling hint %s\n", + arg); + erts_usage(); + } + } else if (sys_strcmp("nsp", sub_param) == 0) erts_use_sender_punish = 0; else if (sys_strcmp("wt", sub_param) == 0) { @@ -1340,22 +1411,19 @@ erl_start(int argc, char **argv) case 'R': { /* set compatibility release */ + int this_rel; arg = get_arg(argv[i]+2, argv[i+1], &i); erts_compat_rel = atoi(arg); - if (erts_compat_rel < ERTS_MIN_COMPAT_REL - || erts_compat_rel > this_rel_num()) { + this_rel = this_rel_num(); + if (erts_compat_rel < this_rel - 2 || this_rel < erts_compat_rel) { erts_fprintf(stderr, "bad compatibility release number %s\n", arg); erts_usage(); } - ASSERT(ERTS_MIN_COMPAT_REL >= 7); switch (erts_compat_rel) { - case 7: - case 8: - case 9: - erts_use_r9_pids_ports = 1; + /* Currently no compat features... */ default: break; } @@ -1397,8 +1465,6 @@ erl_start(int argc, char **argv) } break; } - case 'n': /* XXX obsolete */ - break; case 'c': if (argv[i][2] == 0) { /* -c: documented option */ erts_disable_tolerant_timeofday = 1; @@ -1453,15 +1519,6 @@ erl_start(int argc, char **argv) i++; } - /* Delayed check of +P flag */ - if (erts_proc.max < ERTS_MIN_PROCESSES - || erts_proc.max > ERTS_MAX_PROCESSES - || (erts_use_r9_pids_ports - && erts_proc.max > ERTS_MAX_R9_PROCESSES)) { - erts_fprintf(stderr, "bad number of processes %s\n", Parg); - erts_usage(); - } - /* Restart will not reinstall the break handler */ #ifdef __WIN32__ if (ignore_break) @@ -1482,7 +1539,10 @@ erl_start(int argc, char **argv) boot_argc = argc - i; /* Number of arguments to init */ boot_argv = &argv[i]; - erl_init(ncpu); + erl_init(ncpu, + proc_tab_sz, + port_tab_sz, + port_tab_sz_ignore_files); load_preloaded(); erts_end_staging_code_ix(); diff --git a/erts/emulator/beam/erl_lock_check.c b/erts/emulator/beam/erl_lock_check.c index 314d2f6a9c..69bb4be717 100644 --- a/erts/emulator/beam/erl_lock_check.c +++ b/erts/emulator/beam/erl_lock_check.c @@ -95,7 +95,6 @@ static erts_lc_lock_order_t erts_lock_order[] = { { "dist_entry_links", "address" }, { "code_write_permission", NULL }, { "proc_status", "pid" }, - { "proc_tab", NULL }, { "ports_snapshot", NULL }, { "meta_name_tab", "address" }, { "meta_main_tab_slot", "address" }, @@ -115,9 +114,6 @@ static erts_lc_lock_order_t erts_lock_order[] = { #if defined(ENABLE_CHILD_WAITER_THREAD) || defined(ERTS_SMP) { "child_status", NULL }, #endif -#ifdef __WIN32__ - { "sys_driver_data_lock", NULL }, -#endif { "drv_ev_state_grow", NULL, }, { "drv_ev_state", "address" }, { "safe_hash", "address" }, @@ -127,6 +123,7 @@ static erts_lc_lock_order_t erts_lock_order[] = { { "schdlr_sspnd", NULL }, { "migration_info_update", NULL }, { "run_queue", "address" }, + { "process_table", NULL }, { "cpu_info", NULL }, { "pollset", "address" }, #ifdef __WIN32__ @@ -157,12 +154,10 @@ static erts_lc_lock_order_t erts_lock_order[] = { { "pmmap", NULL }, #endif #ifdef ERTS_SMP + { "port_sched_lock", "port_id" }, { "port_task_pre_alloc_lock", "address" }, - { "port_taskq_pre_alloc_lock", "address" }, { "proclist_pre_alloc_lock", "address" }, - { "port_tasks_lock", NULL }, - { "get_free_port", NULL }, - { "port_state", "address" }, + { "port_table", NULL }, { "xports_list_pre_alloc_lock", "address" }, { "inet_buffer_stack_lock", NULL }, { "gc_info", NULL }, @@ -247,6 +242,7 @@ typedef struct { typedef struct erts_lc_locked_locks_t_ erts_lc_locked_locks_t; struct erts_lc_locked_locks_t_ { char *thread_name; + int emu_thread; erts_tid_t tid; erts_lc_locked_locks_t *next; erts_lc_locked_locks_t *prev; @@ -364,6 +360,7 @@ create_locked_locks(char *thread_name) if (!l_lcks->thread_name) lc_abort(); + l_lcks->emu_thread = 0; l_lcks->tid = erts_thr_self(); l_lcks->required.first = NULL; l_lcks->required.last = NULL; @@ -671,7 +668,7 @@ erts_lc_set_thread_name(char *thread_name) { erts_lc_locked_locks_t *l_lcks = get_my_locked_locks(); if (!l_lcks) - (void) create_locked_locks(thread_name); + l_lcks = create_locked_locks(thread_name); else { ASSERT(l_lcks->thread_name); free((void *) l_lcks->thread_name); @@ -679,6 +676,14 @@ erts_lc_set_thread_name(char *thread_name) if (!l_lcks->thread_name) lc_abort(); } + l_lcks->emu_thread = 1; +} + +int +erts_lc_is_emu_thr(void) +{ + erts_lc_locked_locks_t *l_lcks = get_my_locked_locks(); + return l_lcks->emu_thread; } int diff --git a/erts/emulator/beam/erl_lock_check.h b/erts/emulator/beam/erl_lock_check.h index b67f36fa06..068340abe7 100644 --- a/erts/emulator/beam/erl_lock_check.h +++ b/erts/emulator/beam/erl_lock_check.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2011. All Rights Reserved. + * Copyright Ericsson AB 2005-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -102,9 +102,10 @@ void erts_lc_unrequire_lock_flg(erts_lc_lock_t *lck, Uint16 op_flags); void erts_lc_require_lock(erts_lc_lock_t *lck); void erts_lc_unrequire_lock(erts_lc_lock_t *lck); +int erts_lc_is_emu_thr(void); #define ERTS_LC_ASSERT(A) \ - ((void) ((A) ? 1 : erts_lc_assert_failed(__FILE__, __LINE__, #A))) + ((void) (((A) || ERTS_SOMEONE_IS_CRASH_DUMPING) ? 1 : erts_lc_assert_failed(__FILE__, __LINE__, #A))) #ifdef ERTS_SMP #define ERTS_SMP_LC_ASSERT(A) ERTS_LC_ASSERT(A) #else diff --git a/erts/emulator/beam/erl_message.c b/erts/emulator/beam/erl_message.c index d5b7d01048..325d77e911 100644 --- a/erts/emulator/beam/erl_message.c +++ b/erts/emulator/beam/erl_message.c @@ -495,7 +495,7 @@ queue_message(Process *c_p, #ifndef ERTS_SMP res = receiver->msg.len; #else - res = receiver->u.alive.msg_inq.len; + res = receiver->msg_inq.len; if (*receiver_locks & ERTS_PROC_LOCK_MAIN) { /* * We move 'in queue' to 'private queue' and place @@ -894,8 +894,8 @@ erts_send_message(Process* sender, #ifdef USE_VM_PROBES *sender_name = *receiver_name = '\0'; if (DTRACE_ENABLED(message_send)) { - erts_snprintf(sender_name, sizeof(DTRACE_CHARBUF_NAME(sender_name)), "%T", sender->id); - erts_snprintf(receiver_name, sizeof(DTRACE_CHARBUF_NAME(receiver_name)), "%T", receiver->id); + erts_snprintf(sender_name, sizeof(DTRACE_CHARBUF_NAME(sender_name)), "%T", sender->common.id); + erts_snprintf(receiver_name, sizeof(DTRACE_CHARBUF_NAME(receiver_name)), "%T", receiver->common.id); } #endif if (SEQ_TRACE_TOKEN(sender) != NIL && !(flags & ERTS_SND_FLG_NO_SEQ_TRACE)) { @@ -917,7 +917,7 @@ erts_send_message(Process* sender, seq_trace_update_send(sender); seq_trace_output(stoken, message, SEQ_TRACE_SEND, - receiver->id, sender); + receiver->common.id, sender); seq_trace_size = 6; /* TUPLE5 */ #ifdef USE_VM_PROBES } @@ -948,7 +948,7 @@ erts_send_message(Process* sender, #ifdef DTRACE_TAG_HARDDEBUG erts_fprintf(stderr, "Dtrace -> (%T) Spreading tag (%T) with " - "message %T!\r\n",sender->id, utag, message); + "message %T!\r\n",sender->common.id, utag, message); #endif } #endif @@ -1136,7 +1136,7 @@ erts_deliver_exit_message(Eterm from, Process *to, ErtsProcLocks *to_locksp, save = TUPLE3(hp, am_EXIT, from_copy, mess); hp += 4; /* the trace token must in this case be updated by the caller */ - seq_trace_output(token, save, SEQ_TRACE_SEND, to->id, NULL); + seq_trace_output(token, save, SEQ_TRACE_SEND, to->common.id, NULL); temptoken = copy_struct(token, sz_token, &hp, &bp->off_heap); erts_queue_message(to, to_locksp, bp, save, temptoken #ifdef USE_VM_PROBES diff --git a/erts/emulator/beam/erl_message.h b/erts/emulator/beam/erl_message.h index 2ae94965b1..771eba431f 100644 --- a/erts/emulator/beam/erl_message.h +++ b/erts/emulator/beam/erl_message.h @@ -127,21 +127,21 @@ typedef struct { /* Move in message queue to end of private message queue */ #define ERTS_SMP_MSGQ_MV_INQ2PRIVQ(P) \ do { \ - if ((P)->u.alive.msg_inq.first) { \ - *(P)->msg.last = (P)->u.alive.msg_inq.first; \ - (P)->msg.last = (P)->u.alive.msg_inq.last; \ - (P)->msg.len += (P)->u.alive.msg_inq.len; \ - (P)->u.alive.msg_inq.first = NULL; \ - (P)->u.alive.msg_inq.last = &(P)->u.alive.msg_inq.first; \ - (P)->u.alive.msg_inq.len = 0; \ + if ((P)->msg_inq.first) { \ + *(P)->msg.last = (P)->msg_inq.first; \ + (P)->msg.last = (P)->msg_inq.last; \ + (P)->msg.len += (P)->msg_inq.len; \ + (P)->msg_inq.first = NULL; \ + (P)->msg_inq.last = &(P)->msg_inq.first; \ + (P)->msg_inq.len = 0; \ } \ } while (0) /* Add message last in message queue */ #define LINK_MESSAGE(p, mp) do { \ - *(p)->u.alive.msg_inq.last = (mp); \ - (p)->u.alive.msg_inq.last = &(mp)->next; \ - (p)->u.alive.msg_inq.len++; \ + *(p)->msg_inq.last = (mp); \ + (p)->msg_inq.last = &(mp)->next; \ + (p)->msg_inq.len++; \ } while(0) #else @@ -245,6 +245,9 @@ void erts_move_msg_attached_data_to_heap(Eterm **, ErlOffHeap *, ErlMessage *); Eterm erts_msg_distext2heap(Process *, ErtsProcLocks *, ErlHeapFragment **, Eterm *, ErtsDistExternal *); +void erts_cleanup_offheap(ErlOffHeap *offheap); + + ERTS_GLB_INLINE Uint erts_msg_used_frag_sz(const ErlMessage *msg); ERTS_GLB_INLINE Uint erts_msg_attached_data_size(ErlMessage *msg); diff --git a/erts/emulator/beam/erl_monitors.c b/erts/emulator/beam/erl_monitors.c index 1a84950120..63175c44d6 100644 --- a/erts/emulator/beam/erl_monitors.c +++ b/erts/emulator/beam/erl_monitors.c @@ -971,7 +971,7 @@ Eterm erts_debug_dump_monitors_1(BIF_ALIST_1) } } else { erts_printf("Dumping pid monitors--------------------\n"); - erts_dump_monitors(rp->monitors,0); + erts_dump_monitors(ERTS_P_MONITORS(rp),0); erts_printf("Monitors dumped-------------------------\n"); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); BIF_RET(am_true); @@ -985,12 +985,15 @@ Eterm erts_debug_dump_links_1(BIF_ALIST_1) Process *rp; DistEntry *dep; if (is_internal_port(pid)) { - Port *rport = erts_id2port(pid, p, ERTS_PROC_LOCK_MAIN); + Port *rport = erts_id2port_sflgs(pid, + p, + ERTS_PROC_LOCK_MAIN, + ERTS_PORT_SFLGS_INVALID_LOOKUP); if (rport) { erts_printf("Dumping port links----------------------\n"); - erts_dump_links(rport->nlinks,0); + erts_dump_links(ERTS_P_LINKS(rport), 0); erts_printf("Links dumped----------------------------\n"); - erts_smp_port_unlock(rport); + erts_port_release(rport); BIF_RET(am_true); } else { BIF_ERROR(p,BADARG); @@ -1014,7 +1017,7 @@ Eterm erts_debug_dump_links_1(BIF_ALIST_1) } else { erts_printf("Dumping pid links-----------------------\n"); - erts_dump_links(rp->nlinks,0); + erts_dump_links(ERTS_P_LINKS(rp), 0); erts_printf("Links dumped----------------------------\n"); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); BIF_RET(am_true); diff --git a/erts/emulator/beam/erl_monitors.h b/erts/emulator/beam/erl_monitors.h index d3f6d410dd..a7fa4e0145 100644 --- a/erts/emulator/beam/erl_monitors.h +++ b/erts/emulator/beam/erl_monitors.h @@ -137,8 +137,6 @@ typedef struct erts_suspend_monitor { #define ERTS_LINK_ROOT(Linkp) ((Linkp)->shared.root) #define ERTS_LINK_REFC(Linkp) ((Linkp)->shared.refc) -#define ERTS_LINK_ROOT_AS_UINT(Linkp) (*((Uint *) &((Linkp)->root))) - Uint erts_tot_link_lh_size(void); diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index 0dca588847..1bd2d933b2 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -263,7 +263,7 @@ ErlNifEnv* enif_alloc_env(void) HEAP_LIMIT(&msg_env->phony_proc) = phony_heap; HEAP_END(&msg_env->phony_proc) = phony_heap; MBUF(&msg_env->phony_proc) = NULL; - msg_env->phony_proc.id = ERTS_INVALID_PID; + msg_env->phony_proc.common.id = ERTS_INVALID_PID; #ifdef FORCE_HEAP_FRAGS msg_env->phony_proc.space_verified = 0; msg_env->phony_proc.space_verified_from = NULL; @@ -287,7 +287,7 @@ void enif_clear_env(ErlNifEnv* env) struct enif_msg_environment_t* menv = (struct enif_msg_environment_t*)env; Process* p = &menv->phony_proc; ASSERT(p == menv->env.proc); - ASSERT(p->id == ERTS_INVALID_PID); + ASSERT(p->common.id == ERTS_INVALID_PID); ASSERT(MBUF(p) == menv->env.heap_frag); if (MBUF(p) != NULL) { erts_cleanup_offheap(&MSO(p)); @@ -319,7 +319,7 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, if (env != NULL) { c_p = env->proc; - if (receiver == c_p->id) { + if (receiver == c_p->common.id) { rp_locks = ERTS_PROC_LOCK_MAIN; flush_me = 1; } @@ -341,7 +341,7 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, : erts_pid2proc_opt(c_p, ERTS_PROC_LOCK_MAIN, receiver, rp_locks, ERTS_P2P_FLG_SMP_INC_REFC)); if (rp == NULL) { - ASSERT(env == NULL || receiver != c_p->id); + ASSERT(env == NULL || receiver != c_p->common.id); return 0; } flush_env(msg_env); @@ -397,7 +397,7 @@ static int is_offheap(const ErlOffHeap* oh) ErlNifPid* enif_self(ErlNifEnv* caller_env, ErlNifPid* pid) { - pid->pid = caller_env->proc->id; + pid->pid = caller_env->proc->common.id; return pid; } int enif_get_local_pid(ErlNifEnv* env, ERL_NIF_TERM term, ErlNifPid* pid) @@ -505,7 +505,7 @@ int enif_inspect_iolist_as_binary(ErlNifEnv* env, Eterm term, ErlNifBinary* bin) { struct enif_tmp_obj_t* tobj; ErtsAlcType_t allocator; - Uint sz; + ErlDrvSizeT sz; if (is_binary(term)) { return enif_inspect_binary(env,term,bin); } @@ -531,7 +531,7 @@ int enif_inspect_iolist_as_binary(ErlNifEnv* env, Eterm term, ErlNifBinary* bin) bin->size = sz; bin->bin_term = THE_NON_VALUE; bin->ref_bin = NULL; - io_list_to_buf(term, (char*) bin->data, sz); + erts_iolist_to_buf(term, (char*) bin->data, sz); ADD_READONLY_CHECK(env, bin->data, bin->size); return 1; } @@ -1396,6 +1396,44 @@ size_t enif_sizeof_resource(void* obj) return ERTS_MAGIC_BIN_DATA_SIZE(bin) - offsetof(ErlNifResource,data); } + +void* enif_dlopen(const char* lib, + void (*err_handler)(void*,const char*), void* err_arg) +{ + ErtsSysDdllError errdesc = ERTS_SYS_DDLL_ERROR_INIT; + void* handle; + void* init_func; + if (erts_sys_ddll_open2(lib, &handle, &errdesc) == ERL_DE_NO_ERROR) { + if (erts_sys_ddll_load_nif_init(handle, &init_func, &errdesc) == ERL_DE_NO_ERROR) { + erts_sys_ddll_call_nif_init(init_func); + } + } + else { + if (err_handler != NULL) { + (*err_handler)(err_arg, errdesc.str); + } + handle = NULL; + } + erts_sys_ddll_free_error(&errdesc); + return handle; +} + +void* enif_dlsym(void* handle, const char* symbol, + void (*err_handler)(void*,const char*), void* err_arg) +{ + ErtsSysDdllError errdesc = ERTS_SYS_DDLL_ERROR_INIT; + void* ret; + if (erts_sys_ddll_sym2(handle, symbol, &ret, &errdesc) != ERL_DE_NO_ERROR) { + if (err_handler != NULL) { + (*err_handler)(err_arg, errdesc.str); + } + erts_sys_ddll_free_error(&errdesc); + return NULL; + } + return ret; +} + + /*************************************************************************** ** load_nif/2 ** ***************************************************************************/ @@ -1806,7 +1844,7 @@ void erl_nif_init() #ifdef USE_VM_PROBES void dtrace_nifenv_str(ErlNifEnv *env, char *process_buf) { - dtrace_pid_str(env->proc->id, process_buf); + dtrace_pid_str(env->proc->common.id, process_buf); } #endif diff --git a/erts/emulator/beam/erl_nif_api_funcs.h b/erts/emulator/beam/erl_nif_api_funcs.h index 6396af09d0..51ff1eaa48 100644 --- a/erts/emulator/beam/erl_nif_api_funcs.h +++ b/erts/emulator/beam/erl_nif_api_funcs.h @@ -138,6 +138,8 @@ ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM,enif_make_uint64,(ErlNifEnv*, ErlNifUInt64)); ERL_NIF_API_FUNC_DECL(int,enif_is_exception,(ErlNifEnv*, ERL_NIF_TERM term)); ERL_NIF_API_FUNC_DECL(int,enif_make_reverse_list,(ErlNifEnv*, ERL_NIF_TERM term, ERL_NIF_TERM *list)); ERL_NIF_API_FUNC_DECL(int,enif_is_number,(ErlNifEnv*, ERL_NIF_TERM term)); +ERL_NIF_API_FUNC_DECL(void*,enif_dlopen,(const char* lib, void (*err_handler)(void*,const char*), void* err_arg)); +ERL_NIF_API_FUNC_DECL(void*,enif_dlsym,(void* handle, const char* symbol, void (*err_handler)(void*,const char*), void* err_arg)); /* ** Add new entries here to keep compatibility on Windows!!! @@ -260,6 +262,8 @@ ERL_NIF_API_FUNC_DECL(int,enif_is_number,(ErlNifEnv*, ERL_NIF_TERM term)); # define enif_is_exception ERL_NIF_API_FUNC_MACRO(enif_is_exception) # define enif_make_reverse_list ERL_NIF_API_FUNC_MACRO(enif_make_reverse_list) # define enif_is_number ERL_NIF_API_FUNC_MACRO(enif_is_number) +# define enif_dlopen ERL_NIF_API_FUNC_MACRO(enif_dlopen) +# define enif_dlsym ERL_NIF_API_FUNC_MACRO(enif_dlsym) /* ** Add new entries here diff --git a/erts/emulator/beam/erl_node_container_utils.h b/erts/emulator/beam/erl_node_container_utils.h index 7b4cb7b042..667bda255b 100644 --- a/erts/emulator/beam/erl_node_container_utils.h +++ b/erts/emulator/beam/erl_node_container_utils.h @@ -20,7 +20,7 @@ #ifndef ERL_NODE_CONTAINER_UTILS_H__ #define ERL_NODE_CONTAINER_UTILS_H__ -#include "erl_term.h" +#include "erl_ptab.h" /* * Note regarding node containers: @@ -29,9 +29,6 @@ * the emulator) for the Erlang data types that contain a reference * to a node, i.e. pids, ports, and references. * - * Observe! The layouts of the node container data types have been - * changed in R9. - * * Node containers are divided into internal and external node containers. * An internal node container refer to the current incarnation of the * node which it reside on. An external node container refer to @@ -52,13 +49,6 @@ * reference is a boxed data type. An internal node container have an * implicit reference to the 'erts_this_node' element in the node table. * - * Due to the R9 changes in layouts of node containers there are room to - * store more data than previously. Today (R9) this extra space is unused, - * but it is planned to be used in the future. For example only 18 bits - * are used for data in a pid but there is room for 28 bits of data (on a - * 32-bit machine). Some preparations have been made in the emulator for - * usage of this extra space. - * * OBSERVE! Pids doesn't use fixed size 'serial' and 'number' fields any * more. Previously the 15 bit 'number' field of a pid was used as index * into the process table, and the 3 bit 'serial' field was used as a @@ -104,8 +94,6 @@ #define internal_dist_entry(x) (erts_this_node->dist_entry) #define external_dist_entry(x) (external_node((x))->dist_entry) -extern int erts_use_r9_pids_ports; - /* * For this node (and previous incarnations of this node), 0 is used as * channel no. For other nodes, the atom index of the atom corresponding @@ -128,47 +116,20 @@ extern int erts_use_r9_pids_ports; * Pids * \* */ -#define erts_max_processes erts_proc.max - -typedef struct { - erts_smp_atomic_t *tab; - int max; - int tab_cache_lines; - int pix_per_cache_line; - int pix_cl_mask; - int pix_cl_shift; - int pix_cli_mask; - int pix_cli_shift; -} ErtsProcTab; - -extern ErtsProcTab erts_proc; - -ERTS_GLB_INLINE int erts_pid_data2ix(Eterm pid_data); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE int erts_pid_data2ix(Eterm pid_data) -{ - int n, pix; - - n = (int) pid_data; - if (erts_proc.pix_cl_mask) { - pix = ((n & erts_proc.pix_cl_mask) << erts_proc.pix_cl_shift); - pix += ((n >> erts_proc.pix_cli_shift) & erts_proc.pix_cli_mask); - } - else { - n %= erts_proc.max; - pix = n % erts_proc.tab_cache_lines; - pix *= erts_proc.pix_per_cache_line; - pix += n / erts_proc.tab_cache_lines; - } - ASSERT(0 <= pix && pix < erts_proc.max); - return pix; -} +extern ErtsPTab erts_proc; -#endif +#define make_internal_pid(D) erts_ptab_make_id(&erts_proc, \ + (D), \ + _TAG_IMMED1_PID) -#define internal_pid_index(x) erts_pid_data2ix(internal_pid_data((x))) +#define internal_pid_index(PID) (ASSERT_EXPR(is_internal_pid((PID))), \ + erts_ptab_id2pix(&erts_proc, (PID))) + +#define internal_pid_data(PID) (ASSERT_EXPR(is_internal_pid((PID))), \ + erts_ptab_id2data(&erts_proc, (PID))) + +#define internal_pid_number(x) _GET_PID_NUM(internal_pid_data((x))) +#define internal_pid_serial(x) _GET_PID_SER(internal_pid_data((x))) #define internal_pid_node_name(x) (internal_pid_node((x))->sysname) #define external_pid_node_name(x) (external_pid_node((x))->sysname) @@ -208,34 +169,37 @@ ERTS_GLB_INLINE int erts_pid_data2ix(Eterm pid_data) || is_external_pid((x))) #define is_not_pid(x) (!is_pid(x)) -#define ERTS_MAX_R9_PROCESSES (1 << ERTS_R9_PROC_BITS) - /* * Maximum number of processes. We want the number to fit in a SMALL on * 32-bit CPU. */ -#define ERTS_MAX_PROCESSES ((SWORD_CONSTANT(1) << 27)-1) -#if (ERTS_MAX_PROCESSES > MAX_SMALL) -# error "The maximum number of processes must fit in a SMALL." -#endif - +#define ERTS_MAX_PROCESSES (ERTS_PTAB_MAX_SIZE-1) #define ERTS_MAX_PID_DATA ((1 << _PID_DATA_SIZE) - 1) #define ERTS_MAX_PID_NUMBER ((1 << _PID_NUM_SIZE) - 1) #define ERTS_MAX_PID_SERIAL ((1 << _PID_SER_SIZE) - 1) -#define ERTS_MAX_PID_R9_SERIAL ((1 << _PID_R9_SER_SIZE) - 1) -#define ERTS_R9_PROC_BITS (_PID_R9_SER_SIZE + _PID_NUM_SIZE) #define ERTS_PROC_BITS (_PID_SER_SIZE + _PID_NUM_SIZE) -#define ERTS_INVALID_PID make_internal_pid(ERTS_MAX_PID_DATA) +#define ERTS_INVALID_PID ERTS_PTAB_INVALID_ID(_TAG_IMMED1_PID) /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ * Ports * \* */ -#define internal_port_index(x) (internal_port_data((x)) \ - & erts_port_tab_index_mask) +extern ErtsPTab erts_port; + +#define make_internal_port(D) erts_ptab_make_id(&erts_port, \ + (D), \ + _TAG_IMMED1_PORT) + +#define internal_port_index(PRT) (ASSERT_EXPR(is_internal_port((PRT))), \ + erts_ptab_id2pix(&erts_port, (PRT))) + +#define internal_port_data(PRT) (ASSERT_EXPR(is_internal_port((PRT))), \ + erts_ptab_id2data(&erts_port, (PRT))) + +#define internal_port_number(x) _GET_PORT_NUM(internal_port_data((x))) #define internal_port_node_name(x) (internal_port_node((x))->sysname) #define external_port_node_name(x) (external_port_node((x))->sysname) @@ -274,18 +238,18 @@ ERTS_GLB_INLINE int erts_pid_data2ix(Eterm pid_data) #define is_not_port(x) (!is_port(x)) /* Highest port-ID part in a term of type Port - Not necessarily the same as the variable erts_max_ports + Not necessarily the same as current maximum port table size which defines the maximum number of simultaneous Ports in the Erlang node. ERTS_MAX_PORTS is a hard upper limit. */ -#define ERTS_MAX_R9_PORTS (1 << ERTS_R9_PORTS_BITS) -#define ERTS_MAX_PORTS (1 << ERTS_PORTS_BITS) - +#define ERTS_MAX_PORTS (ERTS_PTAB_MAX_SIZE-1) #define ERTS_MAX_PORT_DATA ((1 << _PORT_DATA_SIZE) - 1) #define ERTS_MAX_PORT_NUMBER ((1 << _PORT_NUM_SIZE) - 1) -#define ERTS_R9_PORTS_BITS (_PORT_R9_NUM_SIZE) #define ERTS_PORTS_BITS (_PORT_NUM_SIZE) + +#define ERTS_INVALID_PORT ERTS_PTAB_INVALID_ID(_TAG_IMMED1_PORT) + /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ * Refs * \* */ diff --git a/erts/emulator/beam/erl_node_tables.c b/erts/emulator/beam/erl_node_tables.c index 40837d3817..ebfba065d1 100644 --- a/erts/emulator/beam/erl_node_tables.c +++ b/erts/emulator/beam/erl_node_tables.c @@ -116,8 +116,7 @@ dist_table_alloc(void *dep_tmpl) dep->qsize = 0; dep->out_queue.first = NULL; dep->out_queue.last = NULL; - dep->suspended.first = NULL; - dep->suspended.last = NULL; + dep->suspended = NULL; dep->finalized_out_queue.first = NULL; dep->finalized_out_queue.last = NULL; @@ -769,8 +768,7 @@ void erts_init_node_tables(void) erts_this_dist_entry->qsize = 0; erts_this_dist_entry->out_queue.first = NULL; erts_this_dist_entry->out_queue.last = NULL; - erts_this_dist_entry->suspended.first = NULL; - erts_this_dist_entry->suspended.last = NULL; + erts_this_dist_entry->suspended = NULL; erts_this_dist_entry->finalized_out_queue.first = NULL; erts_this_dist_entry->finalized_out_queue.last = NULL; @@ -1268,7 +1266,7 @@ setup_reference_table(void) ErlHeapFragment *hfp; DistEntry *dep; HashInfo hi; - int i; + int i, max; DeclareTmpHeapNoproc(heap,3); inserted_bins = NULL; @@ -1297,8 +1295,9 @@ setup_reference_table(void) UnUseTmpHeapNoproc(3); + max = erts_ptab_max(&erts_proc); /* Insert all processes */ - for (i = 0; i < erts_max_processes; i++) { + for (i = 0; i < max; i++) { Process *proc = erts_pix2proc(i); if (proc) { ErlMessage *msg; @@ -1306,12 +1305,12 @@ setup_reference_table(void) /* Insert Heap */ insert_offheap(&(proc->off_heap), HEAP_REF, - proc->id); + proc->common.id); /* Insert message buffers */ for(hfp = proc->mbuf; hfp; hfp = hfp->next) insert_offheap(&(hfp->off_heap), HEAP_REF, - proc->id); + proc->common.id); /* Insert msg msg buffers */ for (msg = proc->msg.first; msg; msg = msg->next) { ErlHeapFragment *heap_frag = NULL; @@ -1321,7 +1320,7 @@ setup_reference_table(void) else { if (msg->data.dist_ext->dep) insert_dist_entry(msg->data.dist_ext->dep, - HEAP_REF, proc->id, 0); + HEAP_REF, proc->common.id, 0); if (is_not_nil(ERL_MESSAGE_TOKEN(msg))) heap_frag = erts_dist_ext_trailer(msg->data.dist_ext); } @@ -1329,10 +1328,10 @@ setup_reference_table(void) if (heap_frag) insert_offheap(&(heap_frag->off_heap), HEAP_REF, - proc->id); + proc->common.id); } #ifdef ERTS_SMP - for (msg = proc->u.alive.msg_inq.first; msg; msg = msg->next) { + for (msg = proc->msg_inq.first; msg; msg = msg->next) { ErlHeapFragment *heap_frag = NULL; if (msg->data.attached) { if (is_value(ERL_MESSAGE_TERM(msg))) @@ -1340,7 +1339,7 @@ setup_reference_table(void) else { if (msg->data.dist_ext->dep) insert_dist_entry(msg->data.dist_ext->dep, - HEAP_REF, proc->id, 0); + HEAP_REF, proc->common.id, 0); if (is_not_nil(ERL_MESSAGE_TOKEN(msg))) heap_frag = erts_dist_ext_trailer(msg->data.dist_ext); } @@ -1348,19 +1347,19 @@ setup_reference_table(void) if (heap_frag) insert_offheap(&(heap_frag->off_heap), HEAP_REF, - proc->id); + proc->common.id); } #endif /* Insert links */ - if(proc->nlinks) - insert_links(proc->nlinks, proc->id); - if(proc->monitors) - insert_monitors(proc->monitors, proc->id); + if (ERTS_P_LINKS(proc)) + insert_links(ERTS_P_LINKS(proc), proc->common.id); + if (ERTS_P_MONITORS(proc)) + insert_monitors(ERTS_P_MONITORS(proc), proc->common.id); /* Insert controller */ { DistEntry *dep = ERTS_PROC_GET_DIST_ENTRY(proc); if (dep) - insert_dist_entry(dep, CTRL_REF, proc->id, 0); + insert_dist_entry(dep, CTRL_REF, proc->common.id, 0); } } } @@ -1370,21 +1369,33 @@ setup_reference_table(void) #endif /* Insert all ports */ - for (i = 0; i < erts_max_ports; i++) { - if (erts_port[i].status & ERTS_PORT_SFLGS_DEAD) + max = erts_ptab_max(&erts_port); + for (i = 0; i < max; i++) { + erts_aint32_t state; + Port *prt; + + prt = erts_pix2port(i); + if (!prt) + continue; + + state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_DEAD) continue; /* Insert links */ - if(erts_port[i].nlinks) - insert_links(erts_port[i].nlinks, erts_port[i].id); + if (ERTS_P_LINKS(prt)) + insert_links(ERTS_P_LINKS(prt), prt->common.id); + /* Insert monitors */ + if (ERTS_P_MONITORS(prt)) + insert_monitors(ERTS_P_MONITORS(prt), prt->common.id); /* Insert port data */ - for(hfp = erts_port[i].bp; hfp; hfp = hfp->next) - insert_offheap(&(hfp->off_heap), HEAP_REF, erts_port[i].id); + for(hfp = prt->bp; hfp; hfp = hfp->next) + insert_offheap(&(hfp->off_heap), HEAP_REF, prt->common.id); /* Insert controller */ - if (erts_port[i].dist_entry) - insert_dist_entry(erts_port[i].dist_entry, + if (prt->dist_entry) + insert_dist_entry(prt->dist_entry, CTRL_REF, - erts_port[i].id, + prt->common.id, 0); } diff --git a/erts/emulator/beam/erl_node_tables.h b/erts/emulator/beam/erl_node_tables.h index 4a015bdef9..af60071ea5 100644 --- a/erts/emulator/beam/erl_node_tables.h +++ b/erts/emulator/beam/erl_node_tables.h @@ -84,10 +84,6 @@ typedef struct { } ErtsDistOutputQueue; struct ErtsProcList_; -typedef struct { - struct ErtsProcList_ *first; - struct ErtsProcList_ *last; -} ErtsDistSuspended; /* * Lock order: @@ -100,7 +96,6 @@ typedef struct { */ struct erl_link; -struct port; typedef struct dist_entry_ { HashBucket hash_bucket; /* Hash bucket */ @@ -135,13 +130,13 @@ typedef struct dist_entry_ { Uint32 qflgs; Sint qsize; ErtsDistOutputQueue out_queue; - ErtsDistSuspended suspended; + struct ErtsProcList_ *suspended; ErtsDistOutputQueue finalized_out_queue; erts_smp_atomic_t dist_cmd_scheduled; ErtsPortTaskHandle dist_cmd; - Uint (*send)(struct port *prt, ErtsDistOutputBuf *obuf); + Uint (*send)(Port *prt, ErtsDistOutputBuf *obuf); struct cache* cache; /* The atom cache */ } DistEntry; diff --git a/erts/emulator/beam/erl_port.h b/erts/emulator/beam/erl_port.h new file mode 100644 index 0000000000..f4d73e716a --- /dev/null +++ b/erts/emulator/beam/erl_port.h @@ -0,0 +1,942 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +#ifndef ERL_PORT_TYPE__ +#define ERL_PORT_TYPE__ +typedef struct _erl_drv_port Port; +typedef struct ErtsProc2PortSigData_ ErtsProc2PortSigData; +#endif + +#if !defined(ERL_PORT_H__) && !defined(ERL_PORT_GET_PORT_TYPE_ONLY__) +#define ERL_PORT_H__ + +#include "erl_port_task.h" +#include "erl_ptab.h" +#include "erl_thr_progress.h" +#include "erl_trace.h" + +#define ERTS_DEFAULT_MAX_PORTS (1 << 16) +#define ERTS_MIN_PORTS 1024 + +extern int erts_port_synchronous_ops; +extern int erts_port_schedule_all_ops; +extern int erts_port_parallelism; + +typedef struct erts_driver_t_ erts_driver_t; + +#define ERTS_INVALID_ERL_DRV_PORT ((ErlDrvPort) (SWord) -1) +#define SMALL_IO_QUEUE 5 /* Number of fixed elements */ + +typedef struct { + ErlDrvSizeT size; /* total size in bytes */ + + SysIOVec* v_start; + SysIOVec* v_end; + SysIOVec* v_head; + SysIOVec* v_tail; + SysIOVec v_small[SMALL_IO_QUEUE]; + + ErlDrvBinary** b_start; + ErlDrvBinary** b_end; + ErlDrvBinary** b_head; + ErlDrvBinary** b_tail; + ErlDrvBinary* b_small[SMALL_IO_QUEUE]; +} ErlIOQueue; + +typedef struct line_buf { /* Buffer used in line oriented I/O */ + ErlDrvSizeT bufsiz; /* Size of character buffer */ + ErlDrvSizeT ovlen; /* Length of overflow data */ + ErlDrvSizeT ovsiz; /* Actual size of overflow buffer */ + char data[1]; /* Starting point of buffer data, + data[0] is a flag indicating an unprocess CR, + The rest is the overflow buffer. */ +} LineBuf; + +/* + * Items part of erlang:port_info/1 result. Note am_registered_name + * *need* to be first. + */ + +#define ERTS_PORT_INFO_1_ITEMS \ + { am_registered_name, /* Needs to be first */ \ + am_name, \ + am_links, \ + am_id, \ + am_connected, \ + am_input, \ + am_output, \ + am_os_pid } + +/* + * Port Specific Data. + * + * Only use PrtSD for very rarely used data. + */ + +#define ERTS_PRTSD_SCHED_ID 0 + +#define ERTS_PRTSD_SIZE 1 + +typedef struct { + void *data[ERTS_PRTSD_SIZE]; +} ErtsPrtSD; + +#ifdef ERTS_SMP +typedef struct ErtsXPortsList_ ErtsXPortsList; +#endif + +/* + * Port locking: + * + * Locking is done either driver specific or port specific. When + * driver specific locking is used, all instances of the driver, + * i.e. ports running the driver, share the same lock. When port + * specific locking is used each instance have its own lock. + * + * Most fields in the Port structure are protected by the lock + * referred to by the 'lock' field. This lock is shared between + * all ports running the same driver when driver specific locking + * is used. + * + * The 'sched' field is protected by the run queue lock that the + * port currently is assigned to. + * + */ + +struct _erl_drv_port { + ErtsPTabElementCommon common; /* *Need* to be first in struct */ + + ErtsPortTaskSched sched; + ErtsPortTaskHandle timeout_task; +#ifdef ERTS_SMP + erts_mtx_t *lock; + ErtsXPortsList *xports; + erts_smp_atomic_t run_queue; +#else + erts_atomic32_t refc; + int cleanup; +#endif + erts_atomic_t connected; /* A connected process */ + Eterm caller; /* Current caller. */ + Eterm data; /* Data associated with port. */ + ErlHeapFragment* bp; /* Heap fragment holding data (NULL if imm data). */ + Uint bytes_in; /* Number of bytes read */ + Uint bytes_out; /* Number of bytes written */ + + ErlIOQueue ioq; /* driver accessible i/o queue */ + DistEntry *dist_entry; /* Dist entry used in DISTRIBUTION */ + char *name; /* String used in the open */ + erts_driver_t* drv_ptr; + UWord drv_data; + SWord os_pid; /* Child process ID */ + ErtsProcList *suspended; /* List of suspended processes. */ + LineBuf *linebuf; /* Buffer to hold data not ready for + process to get (line oriented I/O)*/ + erts_atomic32_t state; /* Status and type flags */ + int control_flags; /* Flags for port_control() */ + ErlDrvPDL port_data_lock; + + ErtsPrtSD *psd; /* Port specific data */ +}; + +#define ERTS_PORT_GET_CONNECTED(PRT) \ + ((Eterm) erts_atomic_read_nob(&(PRT)->connected)) +#define ERTS_PORT_SET_CONNECTED(PRT, PID) \ + erts_atomic_set_relb(&(PRT)->connected, (erts_aint_t) (PID)) +#define ERTS_PORT_INIT_CONNECTED(PRT, PID) \ + erts_atomic_init_nob(&(PRT)->connected, (erts_aint_t) (PID)) + + +struct erl_drv_port_data_lock { + erts_mtx_t mtx; + erts_atomic_t refc; + Port *prt; +}; + +ERTS_GLB_INLINE ErtsRunQueue *erts_port_runq(Port *prt); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE ErtsRunQueue * +erts_port_runq(Port *prt) +{ +#ifdef ERTS_SMP + ErtsRunQueue *rq1, *rq2; + rq1 = (ErtsRunQueue *) erts_smp_atomic_read_nob(&prt->run_queue); + if (!rq1) + return NULL; + while (1) { + erts_smp_runq_lock(rq1); + rq2 = (ErtsRunQueue *) erts_smp_atomic_read_nob(&prt->run_queue); + if (rq1 == rq2) + return rq1; + erts_smp_runq_unlock(rq1); + rq1 = rq2; + if (!rq1) + return NULL; + } +#else + return ERTS_RUNQ_IX(0); +#endif +} + +#endif + + +ERTS_GLB_INLINE void *erts_prtsd_get(Port *p, int ix); +ERTS_GLB_INLINE void *erts_prtsd_set(Port *p, int ix, void *new); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE void * +erts_prtsd_get(Port *prt, int ix) +{ + return prt->psd ? prt->psd->data[ix] : NULL; +} + +ERTS_GLB_INLINE void * +erts_prtsd_set(Port *prt, int ix, void *data) +{ + if (prt->psd) { + void *old = prt->psd->data[ix]; + prt->psd->data[ix] = data; + return old; + } + else { + prt->psd = erts_alloc(ERTS_ALC_T_PRTSD, sizeof(ErtsPrtSD)); + prt->psd->data[ix] = data; + return NULL; + } +} + +#endif + +extern erts_smp_atomic_t erts_bytes_out; /* no bytes written out */ +extern erts_smp_atomic_t erts_bytes_in; /* no bytes sent into the system */ + + +/* port status flags */ + +#define ERTS_PORT_SFLG_CONNECTED ((Uint32) (1 << 0)) +/* Port have begun exiting */ +#define ERTS_PORT_SFLG_EXITING ((Uint32) (1 << 1)) +/* Distribution port */ +#define ERTS_PORT_SFLG_DISTRIBUTION ((Uint32) (1 << 2)) +#define ERTS_PORT_SFLG_BINARY_IO ((Uint32) (1 << 3)) +#define ERTS_PORT_SFLG_SOFT_EOF ((Uint32) (1 << 4)) +/* Flow control */ +/* Port is closing (no i/o accepted) */ +#define ERTS_PORT_SFLG_CLOSING ((Uint32) (1 << 5)) +/* Send a closed message when terminating */ +#define ERTS_PORT_SFLG_SEND_CLOSED ((Uint32) (1 << 6)) +/* Line orinted io on port */ +#define ERTS_PORT_SFLG_LINEBUF_IO ((Uint32) (1 << 7)) +/* Immortal port (only certain system ports) */ +#define ERTS_PORT_SFLG_FREE ((Uint32) (1 << 8)) +#define ERTS_PORT_SFLG_INITIALIZING ((Uint32) (1 << 9)) +/* Port uses port specific locking (opposed to driver specific locking) */ +#define ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK ((Uint32) (1 << 10)) +#define ERTS_PORT_SFLG_INVALID ((Uint32) (1 << 11)) +/* Last port to terminate halts the emulator */ +#define ERTS_PORT_SFLG_HALT ((Uint32) (1 << 12)) +#ifdef DEBUG +/* Only debug: make sure all flags aren't cleared unintentionally */ +#define ERTS_PORT_SFLG_PORT_DEBUG ((Uint32) (1 << 31)) +#endif + +/* Combinations of port status flags */ +#define ERTS_PORT_SFLGS_DEAD \ + (ERTS_PORT_SFLG_FREE | ERTS_PORT_SFLG_INITIALIZING) +#define ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP \ + (ERTS_PORT_SFLGS_DEAD | ERTS_PORT_SFLG_INVALID) +#define ERTS_PORT_SFLGS_INVALID_LOOKUP \ + (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP \ + | ERTS_PORT_SFLG_EXITING \ + | ERTS_PORT_SFLG_CLOSING) +#define ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP \ + (ERTS_PORT_SFLGS_INVALID_LOOKUP \ + | ERTS_PORT_SFLG_DISTRIBUTION) + + +/* + * Costs in reductions for some port operations. + */ +#define ERTS_PORT_REDS_EXECUTE 10 +#define ERTS_PORT_REDS_FREE 100 +#define ERTS_PORT_REDS_TIMEOUT 400 +#define ERTS_PORT_REDS_INPUT 400 +#define ERTS_PORT_REDS_OUTPUT 400 +#define ERTS_PORT_REDS_EVENT 400 +#define ERTS_PORT_REDS_CMD_OUTPUTV 400 +#define ERTS_PORT_REDS_CMD_OUTPUT 400 +#define ERTS_PORT_REDS_EXIT 300 +#define ERTS_PORT_REDS_CONNECT 40 +#define ERTS_PORT_REDS_UNLINK 40 +#define ERTS_PORT_REDS_LINK 40 +#define ERTS_PORT_REDS_BADSIG 40 +#define ERTS_PORT_REDS_CONTROL 400 +#define ERTS_PORT_REDS_CALL 400 +#define ERTS_PORT_REDS_INFO 100 +#define ERTS_PORT_REDS_SET_DATA 40 +#define ERTS_PORT_REDS_GET_DATA 40 +#define ERTS_PORT_REDS_TERMINATE 200 + +void print_port_info(Port *, int, void *); +void erts_port_free(Port *); +#ifndef ERTS_SMP +void erts_port_cleanup(Port *); +#endif +void erts_fire_port_monitor(Port *prt, Eterm ref); +#ifdef ERTS_SMP +int erts_port_handle_xports(Port *); +#endif + +#if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) +int erts_lc_is_port_locked(Port *); +#endif + +ERTS_GLB_INLINE void erts_port_inc_refc(Port *prt); +ERTS_GLB_INLINE void erts_port_dec_refc(Port *prt); +ERTS_GLB_INLINE void erts_port_add_refc(Port *prt, Sint32 add_refc); + +ERTS_GLB_INLINE int erts_smp_port_trylock(Port *prt); +ERTS_GLB_INLINE void erts_smp_port_lock(Port *prt); +ERTS_GLB_INLINE void erts_smp_port_unlock(Port *prt); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE void erts_port_inc_refc(Port *prt) +{ +#ifdef ERTS_SMP + erts_ptab_inc_refc(&prt->common); +#else + erts_atomic32_inc_nob(&prt->refc); +#endif +} + +ERTS_GLB_INLINE void erts_port_dec_refc(Port *prt) +{ +#ifdef ERTS_SMP + int referred = erts_ptab_dec_test_refc(&prt->common); + if (!referred) + erts_port_free(prt); +#else + int refc = erts_atomic32_dec_read_nob(&prt->refc); + if (refc == 0) + erts_port_free(prt); +#endif +} + +ERTS_GLB_INLINE void erts_port_add_refc(Port *prt, Sint32 add_refc) +{ +#ifdef ERTS_SMP + int referred = erts_ptab_add_test_refc(&prt->common, add_refc); + if (!referred) + erts_port_free(prt); +#else + int refc = erts_atomic32_add_read_nob(&prt->refc, add_refc); + if (refc == 0) + erts_port_free(prt); +#endif +} + +ERTS_GLB_INLINE int +erts_smp_port_trylock(Port *prt) +{ +#ifdef ERTS_SMP + /* *Need* to be a managed thread */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); + return erts_mtx_trylock(prt->lock); +#else + return 0; +#endif +} + +ERTS_GLB_INLINE void +erts_smp_port_lock(Port *prt) +{ +#ifdef ERTS_SMP + /* *Need* to be a managed thread */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); + erts_mtx_lock(prt->lock); +#endif +} + +ERTS_GLB_INLINE void +erts_smp_port_unlock(Port *prt) +{ +#ifdef ERTS_SMP + /* *Need* to be a managed thread */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); + erts_mtx_unlock(prt->lock); +#endif +} + +#endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ + + +#define ERTS_INVALID_PORT_OPT(PP, ID, FLGS) \ + (!(PP) \ + || (erts_atomic32_read_nob(&(PP)->state) & (FLGS)) \ + || (PP)->common.id != (ID)) + +/* port lookup */ + +#define INVALID_PORT(PP, ID) \ + ERTS_INVALID_PORT_OPT((PP), (ID), ERTS_PORT_SFLGS_INVALID_LOOKUP) + +/* Invalidate trace port if anything suspicious, for instance + * that the port is a distribution port or it is busy. + */ +#define INVALID_TRACER_PORT(PP, ID) \ + ERTS_INVALID_PORT_OPT((PP), (ID), ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP) + +#define ERTS_PORT_SCHED_ID(P, ID) \ + ((Uint) (UWord) erts_prtsd_set((P), ERTS_PSD_SCHED_ID, (void *) (UWord) (ID))) + +extern const Port erts_invalid_port; +#define ERTS_PORT_LOCK_BUSY ((Port *) &erts_invalid_port) + +int erts_is_port_ioq_empty(Port *); +void erts_terminate_port(Port *); + +#ifdef ERTS_SMP +Port *erts_de2port(DistEntry *, Process *, ErtsProcLocks); +#endif + +ERTS_GLB_INLINE Port *erts_pix2port(int); +ERTS_GLB_INLINE Port *erts_port_lookup_raw(Eterm); +ERTS_GLB_INLINE Port *erts_port_lookup(Eterm, Uint32); +ERTS_GLB_INLINE Port*erts_id2port(Eterm id); +ERTS_GLB_INLINE Port *erts_id2port_sflgs(Eterm, Process *, ErtsProcLocks, Uint32); +ERTS_GLB_INLINE void erts_port_release(Port *); +#ifdef ERTS_SMP +ERTS_GLB_INLINE Port *erts_thr_id2port_sflgs(Eterm id, Uint32 invalid_sflgs); +ERTS_GLB_INLINE void erts_thr_port_release(Port *prt); +#endif +ERTS_GLB_INLINE Port *erts_thr_drvport2port_raw(ErlDrvPort); +ERTS_GLB_INLINE Port *erts_drvport2port_raw(ErlDrvPort drvport); +ERTS_GLB_INLINE Port *erts_drvport2port(ErlDrvPort, erts_aint32_t *); +ERTS_GLB_INLINE Port *erts_drvportid2port(Eterm); +ERTS_GLB_INLINE Eterm erts_drvport2id(ErlDrvPort); +ERTS_GLB_INLINE Uint32 erts_portid2status(Eterm); +ERTS_GLB_INLINE int erts_is_port_alive(Eterm); +ERTS_GLB_INLINE int erts_is_valid_tracer_port(Eterm); +ERTS_GLB_INLINE int erts_port_driver_callback_epilogue(Port *, erts_aint32_t *); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE Port *erts_pix2port(int ix) +{ + Port *prt; + ASSERT(0 <= ix && ix < erts_ptab_max(&erts_port)); + prt = (Port *) erts_ptab_pix2intptr_nob(&erts_port, ix); + return prt == ERTS_PORT_LOCK_BUSY ? NULL : prt; +} + +ERTS_GLB_INLINE Port * +erts_port_lookup_raw(Eterm id) +{ + Port *prt; + + ERTS_SMP_LC_ASSERT(erts_thr_progress_lc_is_delaying()); + + if (is_not_internal_port(id)) + return NULL; + + prt = (Port *) erts_ptab_pix2intptr_ddrb(&erts_port, + internal_port_index(id)); + return prt && prt->common.id == id ? prt : NULL; +} + +ERTS_GLB_INLINE Port * +erts_port_lookup(Eterm id, Uint32 invalid_sflgs) +{ + Port *prt = erts_port_lookup_raw(id); + return (!prt + ? NULL + : ((invalid_sflgs & erts_atomic32_read_nob(&prt->state)) + ? NULL + : prt)); +} + + +ERTS_GLB_INLINE Port* +erts_id2port(Eterm id) +{ + erts_aint32_t state; + Port *prt; + + /* Only allowed to be called from managed threads */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); + + if (is_not_internal_port(id)) + return NULL; + + prt = (Port *) erts_ptab_pix2intptr_ddrb(&erts_port, + internal_port_index(id)); + + if (!prt || prt->common.id != id) + return NULL; + + erts_smp_port_lock(prt); + state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_INVALID_LOOKUP) { + erts_smp_port_unlock(prt); + return NULL; + } + + return prt; +} + + +ERTS_GLB_INLINE Port* +erts_id2port_sflgs(Eterm id, + Process *c_p, ErtsProcLocks c_p_locks, + Uint32 invalid_sflgs) +{ +#ifdef ERTS_SMP + int no_proc_locks = !c_p || !c_p_locks; +#endif + erts_aint32_t state; + Port *prt; + + /* Only allowed to be called from managed threads */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); + + if (is_not_internal_port(id)) + return NULL; + + prt = (Port *) erts_ptab_pix2intptr_ddrb(&erts_port, + internal_port_index(id)); + + if (!prt || prt->common.id != id) + return NULL; + +#ifdef ERTS_SMP + if (no_proc_locks) + erts_smp_port_lock(prt); + else if (erts_smp_port_trylock(prt) == EBUSY) { + /* Unlock process locks, and acquire locks in lock order... */ + erts_smp_proc_unlock(c_p, c_p_locks); + erts_smp_port_lock(prt); + erts_smp_proc_lock(c_p, c_p_locks); + } +#endif + state = erts_atomic32_read_nob(&prt->state); + if (state & invalid_sflgs) { +#ifdef ERTS_SMP + erts_smp_port_unlock(prt); +#endif + return NULL; + } + + return prt; +} + +ERTS_GLB_INLINE void +erts_port_release(Port *prt) +{ + /* Only allowed to be called from managed threads */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); +#ifdef ERTS_SMP + erts_smp_port_unlock(prt); +#else + if (prt->cleanup) { + prt->cleanup = 0; + erts_port_cleanup(prt); + } +#endif +} + +#ifdef ERTS_SMP + +/* + * erts_thr_id2port_sflgs() and erts_thr_port_release() can + * be used by unmanaged threads in the SMP case. + */ +ERTS_GLB_INLINE Port * +erts_thr_id2port_sflgs(Eterm id, Uint32 invalid_sflgs) +{ + Port *prt; + ErtsThrPrgrDelayHandle dhndl; + + if (is_not_internal_port(id)) + return NULL; + + dhndl = erts_thr_progress_unmanaged_delay(); + + prt = (Port *) erts_ptab_pix2intptr_ddrb(&erts_port, + internal_port_index(id)); + + if (!prt || prt->common.id != id) { + erts_thr_progress_unmanaged_continue(dhndl); + prt = NULL; + } + else { + erts_aint32_t state; + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) { + erts_port_inc_refc(prt); + erts_thr_progress_unmanaged_continue(dhndl); + } + + erts_mtx_lock(prt->lock); + state = erts_atomic32_read_nob(&prt->state); + if (state & invalid_sflgs) { + erts_mtx_unlock(prt->lock); + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_port_dec_refc(prt); + prt = NULL; + } + } + + return prt; +} + +ERTS_GLB_INLINE void +erts_thr_port_release(Port *prt) +{ + erts_mtx_unlock(prt->lock); +#ifdef ERTS_SMP + if (!erts_thr_progress_is_managed_thread()) + erts_port_dec_refc(prt); +#endif +} + +#endif + +ERTS_GLB_INLINE Port* +erts_thr_drvport2port_raw(ErlDrvPort drvport) +{ +#if ERTS_ENABLE_LOCK_CHECK + int emu_thread = erts_lc_is_emu_thr(); +#endif + if (drvport == ERTS_INVALID_ERL_DRV_PORT) + return NULL; + else { + Port *prt = (Port *) drvport; +#if ERTS_ENABLE_LOCK_CHECK + if (!ERTS_IS_CRASH_DUMPING) { + if (emu_thread) { + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + ERTS_LC_ASSERT(!prt->port_data_lock + || erts_lc_mtx_is_locked(&prt->port_data_lock->mtx)); + } + else { + ERTS_LC_ASSERT(prt->port_data_lock); + ERTS_LC_ASSERT(erts_lc_mtx_is_locked(&prt->port_data_lock->mtx)); + } + } +#endif + return prt; + } +} + +ERTS_GLB_INLINE Port* +erts_drvport2port_raw(ErlDrvPort drvport) +{ + ERTS_LC_ASSERT(erts_lc_is_emu_thr()); + if (drvport == ERTS_INVALID_ERL_DRV_PORT) + return NULL; + else { + Port *prt = (Port *) drvport; + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt) + || ERTS_IS_CRASH_DUMPING); + return prt; + } +} + +ERTS_GLB_INLINE Port* +erts_drvport2port(ErlDrvPort drvport, erts_aint32_t *statep) +{ + Port *prt = erts_drvport2port_raw(drvport); + erts_aint32_t state; + if (!prt) + return NULL; + state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) + return NULL; + if (statep) + *statep = state; + return prt; +} + +ERTS_GLB_INLINE Port* +erts_drvportid2port(Eterm id) +{ + Port *prt; + erts_aint32_t state; + if (is_not_internal_port(id)) + return NULL; + prt = (Port *) erts_ptab_pix2intptr_nob(&erts_port, + internal_port_index(id)); + if (!prt) + return NULL; + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt) + || ERTS_IS_CRASH_DUMPING); + if (prt->common.id != id) + return NULL; + state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) + return NULL; + return prt; +} + +ERTS_GLB_INLINE Eterm +erts_drvport2id(ErlDrvPort drvport) +{ + Port *prt = erts_drvport2port_raw(drvport); + if (!prt) + return am_undefined; + else + return prt->common.id; +} + +ERTS_GLB_INLINE Uint32 +erts_portid2status(Eterm id) +{ + Port *prt = erts_port_lookup_raw(id); + if (prt) + return (Uint32) erts_atomic32_read_acqb(&prt->state); + else + return ERTS_PORT_SFLG_INVALID; +} + +ERTS_GLB_INLINE int +erts_is_port_alive(Eterm id) +{ + return !(erts_portid2status(id) & (ERTS_PORT_SFLG_INVALID + | ERTS_PORT_SFLGS_DEAD)); +} + +ERTS_GLB_INLINE int +erts_is_valid_tracer_port(Eterm id) +{ + return !(erts_portid2status(id) & ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); +} + +ERTS_GLB_INLINE int +erts_port_driver_callback_epilogue(Port *prt, erts_aint32_t *statep) +{ + int reds = 0; + erts_aint32_t state; + + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + + state = erts_atomic32_read_nob(&prt->state); + if ((state & ERTS_PORT_SFLG_CLOSING) && erts_is_port_ioq_empty(prt)) { + reds += ERTS_PORT_REDS_TERMINATE; + erts_terminate_port(prt); + state = erts_atomic32_read_nob(&prt->state); + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + } + +#ifdef ERTS_SMP + if (prt->xports) { + reds += erts_port_handle_xports(prt); + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + ASSERT(!prt->xports); + } +#endif + + if (statep) + *statep = state; + + return reds; +} + +#endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ + +void erts_port_resume_procs(Port *); + +struct binary; + +#define ERTS_P2P_SIG_TYPE_BAD 0 +#define ERTS_P2P_SIG_TYPE_OUTPUT 1 +#define ERTS_P2P_SIG_TYPE_OUTPUTV 2 +#define ERTS_P2P_SIG_TYPE_CONNECT 3 +#define ERTS_P2P_SIG_TYPE_EXIT 4 +#define ERTS_P2P_SIG_TYPE_CONTROL 5 +#define ERTS_P2P_SIG_TYPE_CALL 6 +#define ERTS_P2P_SIG_TYPE_INFO 7 +#define ERTS_P2P_SIG_TYPE_LINK 8 +#define ERTS_P2P_SIG_TYPE_UNLINK 9 +#define ERTS_P2P_SIG_TYPE_SET_DATA 10 +#define ERTS_P2P_SIG_TYPE_GET_DATA 11 + +#define ERTS_P2P_SIG_TYPE_BITS 4 +#define ERTS_P2P_SIG_TYPE_MASK \ + ((1 << ERTS_P2P_SIG_TYPE_BITS) - 1) + +#define ERTS_P2P_SIG_DATA_FLG(N) \ + (1 << (ERTS_P2P_SIG_TYPE_BITS + (N))) +#define ERTS_P2P_SIG_DATA_FLG_BANG_OP ERTS_P2P_SIG_DATA_FLG(0) +#define ERTS_P2P_SIG_DATA_FLG_REPLY ERTS_P2P_SIG_DATA_FLG(1) +#define ERTS_P2P_SIG_DATA_FLG_NOSUSPEND ERTS_P2P_SIG_DATA_FLG(2) +#define ERTS_P2P_SIG_DATA_FLG_FORCE ERTS_P2P_SIG_DATA_FLG(3) +#define ERTS_P2P_SIG_DATA_FLG_BAD_OUTPUT ERTS_P2P_SIG_DATA_FLG(4) +#define ERTS_P2P_SIG_DATA_FLG_BROKEN_LINK ERTS_P2P_SIG_DATA_FLG(5) +#define ERTS_P2P_SIG_DATA_FLG_SCHED ERTS_P2P_SIG_DATA_FLG(6) + +struct ErtsProc2PortSigData_ { + int flags; + Eterm caller; + Uint32 ref[ERTS_MAX_REF_NUMBERS]; + union { + struct { + Eterm from; + ErlIOVec *evp; + ErlDrvBinary *cbinp; + } outputv; + struct { + Eterm from; + char *bufp; + ErlDrvSizeT size; + } output; + struct { + Eterm from; + Eterm connected; + } connect; + struct { + Eterm from; + Eterm reason; + ErlHeapFragment *bp; + } exit; + struct { + struct binary *binp; + unsigned int command; + char *bufp; + ErlDrvSizeT size; + } control; + struct { + unsigned int command; + char *bufp; + ErlDrvSizeT size; + } call; + struct { + Eterm item; + } info; + struct { + Eterm port; + Eterm to; + } link; + struct { + Eterm from; + } unlink; + struct { + ErlHeapFragment *bp; + Eterm data; + } set_data; + } u; +} ; + +ERTS_GLB_INLINE int +erts_proc2port_sig_is_command_op(ErtsProc2PortSigData *sigdp); +ERTS_GLB_INLINE ErlDrvSizeT +erts_proc2port_sig_command_data_size(ErtsProc2PortSigData *sigdp); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE int +erts_proc2port_sig_is_command_op(ErtsProc2PortSigData *sigdp) +{ + switch (sigdp->flags & ERTS_P2P_SIG_TYPE_MASK) { + case ERTS_P2P_SIG_TYPE_OUTPUT: return !0; + case ERTS_P2P_SIG_TYPE_OUTPUTV: return !0; + default: return 0; + } +} + +ERTS_GLB_INLINE ErlDrvSizeT +erts_proc2port_sig_command_data_size(ErtsProc2PortSigData *sigdp) +{ + switch (sigdp->flags & ERTS_P2P_SIG_TYPE_MASK) { + case ERTS_P2P_SIG_TYPE_OUTPUT: return sigdp->u.output.size; + case ERTS_P2P_SIG_TYPE_OUTPUTV: return sigdp->u.outputv.evp->size; + default: return (ErlDrvSizeT) 0; + } +} + +#endif + +#define ERTS_PROC2PORT_SIG_EXEC 0 +#define ERTS_PROC2PORT_SIG_ABORT 1 +#define ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND 2 +#define ERTS_PROC2PORT_SIG_ABORT_CLOSED 3 + +typedef int (*ErtsProc2PortSigCallback)(Port *, + erts_aint32_t, + int, + ErtsProc2PortSigData *); + +typedef enum { + ERTS_PORT_OP_BADARG, + ERTS_PORT_OP_CALLER_EXIT, + ERTS_PORT_OP_BUSY, + ERTS_PORT_OP_BUSY_SCHEDULED, + ERTS_PORT_OP_SCHEDULED, + ERTS_PORT_OP_DROPPED, + ERTS_PORT_OP_DONE +} ErtsPortOpResult; + +ErtsPortOpResult +erts_schedule_proc2port_signal(Process *, + Port *, + Eterm, + Eterm *, + ErtsProc2PortSigData *, + int, + ErtsProc2PortSigCallback); + +int erts_deliver_port_exit(Port *, Eterm, Eterm, int); + +/* + * Port signal flags + */ +#define ERTS_PORT_SIG_FLG_BANG_OP ERTS_P2P_SIG_DATA_FLG_BANG_OP +#define ERTS_PORT_SIG_FLG_NOSUSPEND ERTS_P2P_SIG_DATA_FLG_NOSUSPEND +#define ERTS_PORT_SIG_FLG_FORCE ERTS_P2P_SIG_DATA_FLG_FORCE +#define ERTS_PORT_SIG_FLG_BROKEN_LINK ERTS_P2P_SIG_DATA_FLG_BROKEN_LINK +#define ERTS_PORT_SIG_FLG_BAD_OUTPUT ERTS_P2P_SIG_DATA_FLG_BAD_OUTPUT +#define ERTS_PORT_SIG_FLG_FORCE_SCHED ERTS_P2P_SIG_DATA_FLG_SCHED +/* ERTS_PORT_SIG_FLG_FORCE_IMM_CALL only when crash dumping... */ +#define ERTS_PORT_SIG_FLG_FORCE_IMM_CALL ERTS_P2P_SIG_DATA_FLG_BAD_OUTPUT + +/* + * Port ! {Owner, {command, Data}} + * Port ! {Owner, {connect, NewOwner}} + * Port ! {Owner, close} + */ +ErtsPortOpResult erts_port_command(Process *, int, Port *, Eterm, Eterm *); + +/* + * Signals from processes to ports. + */ +ErtsPortOpResult erts_port_output(Process *, int, Port *, Eterm, Eterm, Eterm *); +ErtsPortOpResult erts_port_exit(Process *, int, Port *, Eterm, Eterm, Eterm *); +ErtsPortOpResult erts_port_connect(Process *, int, Port *, Eterm, Eterm, Eterm *); +ErtsPortOpResult erts_port_link(Process *, Port *, Eterm, Eterm *); +ErtsPortOpResult erts_port_unlink(Process *, Port *, Eterm, Eterm *); +ErtsPortOpResult erts_port_control(Process *, Port *, unsigned int, Eterm, Eterm *); +ErtsPortOpResult erts_port_call(Process *, Port *, unsigned int, Eterm, Eterm *); +ErtsPortOpResult erts_port_info(Process *, Port *, Eterm, Eterm *); +ErtsPortOpResult erts_port_set_data(Process *, Port *, Eterm, Eterm *); +ErtsPortOpResult erts_port_get_data(Process *, Port *, Eterm *); + +#endif diff --git a/erts/emulator/beam/erl_port_task.c b/erts/emulator/beam/erl_port_task.c index b6bc59a1c3..b661c26036 100644 --- a/erts/emulator/beam/erl_port_task.c +++ b/erts/emulator/beam/erl_port_task.c @@ -33,36 +33,29 @@ #include "erl_port_task.h" #include "dist.h" #include "dtrace-wrapper.h" +#include <stdarg.h> #if defined(DEBUG) && 0 -#define HARD_DEBUG +#define ERTS_HARD_DEBUG_TASK_QUEUES +#else +#undef ERTS_HARD_DEBUG_TASK_QUEUES #endif -/* - * Costs in reductions for some port operations. - */ -#define ERTS_PORT_REDS_EXECUTE 0 -#define ERTS_PORT_REDS_FREE 50 -#define ERTS_PORT_REDS_TIMEOUT 200 -#define ERTS_PORT_REDS_INPUT 200 -#define ERTS_PORT_REDS_OUTPUT 200 -#define ERTS_PORT_REDS_EVENT 200 -#define ERTS_PORT_REDS_TERMINATE 100 - - -#define ERTS_PORT_TASK_INVALID_PORT(P, ID) \ - ((erts_port_status_get((P)) & ERTS_PORT_SFLGS_DEAD) || (P)->id != (ID)) - -#define ERTS_PORT_IS_IN_RUNQ(RQ, P) \ - ((P)->sched.next || (P)->sched.prev || (RQ)->ports.start == (P)) +#ifdef ERTS_HARD_DEBUG_TASK_QUEUES +static void chk_task_queues(Port *pp, ErtsPortTask *execq, int processing_busy_queue); +#define ERTS_PT_DBG_CHK_TASK_QS(PP, EQ, PBQ) \ + chk_task_queues((PP), (EQ), (PBQ)) +#else +#define ERTS_PT_DBG_CHK_TASK_QS(PP, EQ, PBQ) +#endif #ifdef USE_VM_PROBES #define DTRACE_DRIVER(PROBE_NAME, PP) \ - if (DTRACE_ENABLED(driver_ready_input)) { \ + if (DTRACE_ENABLED(PROBE_NAME)) { \ DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); \ DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); \ \ - dtrace_pid_str(PP->connected, process_str); \ + dtrace_pid_str(ERTS_PORT_GET_CONNECTED(PP), process_str); \ dtrace_port_str(PP, port_str); \ DTRACE3(PROBE_NAME, process_str, port_str, PP->name); \ } @@ -72,83 +65,766 @@ erts_smp_atomic_t erts_port_task_outstanding_io_tasks; -struct ErtsPortTaskQueue_ { - ErtsPortTask *first; - ErtsPortTask *last; - Port *port; -}; +#define ERTS_PT_STATE_SCHEDULED 0 +#define ERTS_PT_STATE_ABORTED 1 +#define ERTS_PT_STATE_EXECUTING 2 + +typedef union { + struct { /* I/O tasks */ + ErlDrvEvent event; + ErlDrvEventData event_data; + } io; + struct { + ErtsProc2PortSigCallback callback; + ErtsProc2PortSigData data; + } psig; +} ErtsPortTaskTypeData; struct ErtsPortTask_ { - ErtsPortTask *prev; - ErtsPortTask *next; - ErtsPortTaskQueue *queue; - ErtsPortTaskHandle *handle; + erts_smp_atomic32_t state; ErtsPortTaskType type; - ErlDrvEvent event; - ErlDrvEventData event_data; + union { + struct { + ErtsPortTask *next; + ErtsPortTaskHandle *handle; + int flags; + Uint32 ref[ERTS_MAX_REF_NUMBERS]; + ErtsPortTaskTypeData td; + } alive; + ErtsThrPrgrLaterOp release; + } u; }; -#ifdef HARD_DEBUG -#define ERTS_PT_CHK_PORTQ(RQ) check_port_queue((RQ), NULL, 0) -#define ERTS_PT_CHK_PRES_PORTQ(RQ, PP) check_port_queue((RQ), (PP), -1) -#define ERTS_PT_CHK_IN_PORTQ(RQ, PP) check_port_queue((RQ), (PP), 1) -#define ERTS_PT_CHK_NOT_IN_PORTQ(RQ, PP) check_port_queue((RQ), (PP), 0) -#define ERTS_PT_CHK_TASKQ(Q) check_task_queue((Q), NULL, 0) -#define ERTS_PT_CHK_IN_TASKQ(Q, T) check_task_queue((Q), (T), 1) -#define ERTS_PT_CHK_NOT_IN_TASKQ(Q, T) check_task_queue((Q), (T), 0) -static void -check_port_queue(Port *chk_pp, int inq); -static void -check_task_queue(ErtsPortTaskQueue *ptqp, - ErtsPortTask *chk_ptp, - int inq); -#else -#define ERTS_PT_CHK_PORTQ(RQ) -#define ERTS_PT_CHK_PRES_PORTQ(RQ, PP) -#define ERTS_PT_CHK_IN_PORTQ(RQ, PP) -#define ERTS_PT_CHK_NOT_IN_PORTQ(RQ, PP) -#define ERTS_PT_CHK_TASKQ(Q) -#define ERTS_PT_CHK_IN_TASKQ(Q, T) -#define ERTS_PT_CHK_NOT_IN_TASKQ(Q, T) +struct ErtsPortTaskHandleList_ { + ErtsPortTaskHandle handle; + union { + ErtsPortTaskHandleList *next; +#ifdef ERTS_SMP + ErtsThrPrgrLaterOp release; #endif + } u; +}; + +typedef struct ErtsPortTaskBusyCaller_ ErtsPortTaskBusyCaller; +struct ErtsPortTaskBusyCaller_ { + ErtsPortTaskBusyCaller *next; + Eterm caller; + SWord count; + ErtsPortTask *last; +}; + +#define ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS 17 +struct ErtsPortTaskBusyCallerTable_ { + ErtsPortTaskBusyCaller *bucket[ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS]; + ErtsPortTaskBusyCaller pre_alloc_busy_caller; +}; + -static void handle_remaining_tasks(ErtsRunQueue *runq, Port *pp); +static void begin_port_cleanup(Port *pp, ErtsPortTask **execq); ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(port_task, ErtsPortTask, - 200, + 1000, ERTS_ALC_T_PORT_TASK) -ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(port_taskq, - ErtsPortTaskQueue, + +ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(busy_caller_table, + ErtsPortTaskBusyCallerTable, 50, - ERTS_ALC_T_PORT_TASKQ) + ERTS_ALC_T_BUSY_CALLER_TAB) + +#ifdef ERTS_SMP +static void +call_port_task_free(void *vptp) +{ + port_task_free((ErtsPortTask *) vptp); +} +#endif + +static ERTS_INLINE void +schedule_port_task_free(ErtsPortTask *ptp) +{ +#ifdef ERTS_SMP + erts_schedule_thr_prgr_later_op(call_port_task_free, + (void *) ptp, + &ptp->u.release); +#else + port_task_free(ptp); +#endif +} + +static ERTS_INLINE ErtsPortTask * +p2p_sig_data_to_task(ErtsProc2PortSigData *sigdp) +{ + ErtsPortTask *ptp; + char *ptr = (char *) sigdp; + ptr -= offsetof(ErtsPortTask, u.alive.td.psig.data); + ptp = (ErtsPortTask *) ptr; + ASSERT(ptp->type == ERTS_PORT_TASK_PROC_SIG); + return ptp; +} + +ErtsProc2PortSigData * +erts_port_task_alloc_p2p_sig_data(void) +{ + ErtsPortTask *ptp = port_task_alloc(); + + ptp->type = ERTS_PORT_TASK_PROC_SIG; + ptp->u.alive.flags = ERTS_PT_FLG_SIG_DEP; + erts_smp_atomic32_init_nob(&ptp->state, ERTS_PT_STATE_SCHEDULED); + + ASSERT(ptp == p2p_sig_data_to_task(&ptp->u.alive.td.psig.data)); + + return &ptp->u.alive.td.psig.data; +} + +static ERTS_INLINE Eterm +task_caller(ErtsPortTask *ptp) +{ + Eterm caller; + + ASSERT(ptp->type == ERTS_PORT_TASK_PROC_SIG); + + caller = ptp->u.alive.td.psig.data.caller; + + ASSERT(is_internal_pid(caller) || is_internal_port(caller)); + + return caller; +} + +/* + * Busy queue management + */ + +static ERTS_INLINE int +caller2bix(Eterm caller) +{ + ASSERT(is_internal_pid(caller) || is_internal_port(caller)); + return (int) (_GET_PID_DATA(caller) % ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS); +} + + +static void +popped_from_busy_queue(Port *pp, ErtsPortTask *ptp, int last) +{ + ErtsPortTaskBusyCaller **prev_bcpp = NULL, *bcp; + ErtsPortTaskBusyCallerTable *tabp = pp->sched.taskq.local.busy.table; + Eterm caller = task_caller(ptp); + int bix = caller2bix(caller); + + ASSERT(is_internal_pid(caller)); + + ASSERT(tabp); + bcp = tabp->bucket[bix]; + prev_bcpp = &tabp->bucket[bix]; + ASSERT(bcp); + while (bcp->caller != caller) { + prev_bcpp = &bcp->next; + bcp = bcp->next; + ASSERT(bcp); + } + ASSERT(bcp->count > 0); + if (--bcp->count != 0) { + ASSERT(!last); + } + else { + *prev_bcpp = bcp->next; + if (bcp == &tabp->pre_alloc_busy_caller) + bcp->caller = am_undefined; + else + erts_free(ERTS_ALC_T_BUSY_CALLER, bcp); + if (last) { +#ifdef DEBUG + erts_aint32_t flags = +#endif + erts_smp_atomic32_read_band_nob( + &pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_BUSY_TASKS); + ASSERT(flags & ERTS_PTS_FLG_HAVE_BUSY_TASKS); +#ifdef DEBUG + for (bix = 0; bix < ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS; bix++) { + ASSERT(!tabp->bucket[bix]); + } +#endif + busy_caller_table_free(tabp); + pp->sched.taskq.local.busy.first = NULL; + pp->sched.taskq.local.busy.last = NULL; + pp->sched.taskq.local.busy.table = NULL; + } + } +} + +static void +busy_wait_move_to_busy_queue(Port *pp, ErtsPortTask *ptp) +{ + ErtsPortTaskBusyCallerTable *tabp = pp->sched.taskq.local.busy.table; + Eterm caller = task_caller(ptp); + ErtsPortTaskBusyCaller *bcp; + int bix; + + ASSERT(is_internal_pid(caller)); + /* + * Port is busy and this task type needs to wait until not busy. + */ + + ASSERT(ptp->u.alive.flags & ERTS_PT_FLG_WAIT_BUSY); + + ptp->u.alive.next = NULL; + if (pp->sched.taskq.local.busy.last) { + ASSERT(pp->sched.taskq.local.busy.first); + pp->sched.taskq.local.busy.last->u.alive.next = ptp; + } + else { + int i; + erts_aint32_t flags; + + pp->sched.taskq.local.busy.first = ptp; + flags = erts_smp_atomic32_read_bor_nob(&pp->sched.flags, + ERTS_PTS_FLG_HAVE_BUSY_TASKS); + ASSERT(!(flags & ERTS_PTS_FLG_HAVE_BUSY_TASKS)); + + ASSERT(!tabp); + + tabp = busy_caller_table_alloc(); + pp->sched.taskq.local.busy.table = tabp; + for (i = 0; i < ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS; i++) + tabp->bucket[i] = NULL; + tabp->pre_alloc_busy_caller.caller = am_undefined; + } + pp->sched.taskq.local.busy.last = ptp; + + bix = caller2bix(caller); + ASSERT(tabp); + bcp = tabp->bucket[bix]; + + while (bcp && bcp->caller != caller) + bcp = bcp->next; + + if (bcp) + bcp->count++; + else { + if (tabp->pre_alloc_busy_caller.caller == am_undefined) + bcp = &tabp->pre_alloc_busy_caller; + else + bcp = erts_alloc(ERTS_ALC_T_BUSY_CALLER, + sizeof(ErtsPortTaskBusyCaller)); + bcp->caller = caller; + bcp->count = 1; + bcp->next = tabp->bucket[bix]; + tabp->bucket[bix] = bcp; + } + + bcp->last = ptp; +} + +static ERTS_INLINE int +check_sig_dep_move_to_busy_queue(Port *pp, ErtsPortTask *ptp) +{ + ErtsPortTaskBusyCallerTable *tabp = pp->sched.taskq.local.busy.table; + ErtsPortTask *last_ptp; + ErtsPortTaskBusyCaller *bcp; + int bix; + Eterm caller; + + ASSERT(ptp->u.alive.flags & ERTS_PT_FLG_SIG_DEP); + ASSERT(pp->sched.taskq.local.busy.last); + ASSERT(tabp); + + + /* + * We are either not busy, or the task does not imply wait on busy port. + * However, due to the signaling order requirements the task might depend + * on other tasks in the busy queue. + */ + + caller = task_caller(ptp); + bix = caller2bix(caller); + bcp = tabp->bucket[bix]; + while (bcp && bcp->caller != caller) + bcp = bcp->next; + + if (!bcp) + return 0; + + /* + * There are other tasks that we depend on in the busy queue; + * move into busy queue. + */ + + bcp->count++; + last_ptp = bcp->last; + ptp->u.alive.next = last_ptp->u.alive.next; + if (!ptp->u.alive.next) { + ASSERT(pp->sched.taskq.local.busy.last == last_ptp); + pp->sched.taskq.local.busy.last = ptp; + } + last_ptp->u.alive.next = ptp; + bcp->last = ptp; + + return 1; +} + +static void +no_sig_dep_move_from_busyq(Port *pp) +{ + ErtsPortTaskBusyCallerTable *tabp = pp->sched.taskq.local.busy.table; + ErtsPortTask *first_ptp, *last_ptp, *ptp; + ErtsPortTaskBusyCaller **prev_bcpp = NULL, *bcp = NULL; + + /* + * Move tasks at the head of the busy queue that no longer + * have any dependencies to busy wait tasks into the ordinary + * queue. + */ + + first_ptp = ptp = pp->sched.taskq.local.busy.first; + + ASSERT(ptp && !(ptp->u.alive.flags & ERTS_PT_FLG_WAIT_BUSY)); + ASSERT(tabp); + + do { + Eterm caller = task_caller(ptp); + + if (!bcp || bcp->caller != caller) { + int bix = caller2bix(caller); + + prev_bcpp = &tabp->bucket[bix]; + bcp = tabp->bucket[bix]; + ASSERT(bcp); + while (bcp->caller != caller) { + ASSERT(bcp); + prev_bcpp = &bcp->next; + bcp = bcp->next; + } + } + + ASSERT(bcp->caller == caller); + ASSERT(bcp->count > 0); + + if (--bcp->count == 0) { + *prev_bcpp = bcp->next; + if (bcp == &tabp->pre_alloc_busy_caller) + bcp->caller = am_undefined; + else + erts_free(ERTS_ALC_T_BUSY_CALLER, bcp); + } + + last_ptp = ptp; + ptp = ptp->u.alive.next; + } while (ptp && !(ptp->u.alive.flags & ERTS_PT_FLG_WAIT_BUSY)); + + pp->sched.taskq.local.busy.first = last_ptp->u.alive.next; + if (!pp->sched.taskq.local.busy.first) { +#ifdef DEBUG + int bix; + erts_aint32_t flags = +#endif + erts_smp_atomic32_read_band_nob( + &pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_BUSY_TASKS); + ASSERT(flags & ERTS_PTS_FLG_HAVE_BUSY_TASKS); +#ifdef DEBUG + for (bix = 0; bix < ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS; bix++) { + ASSERT(!tabp->bucket[bix]); + } +#endif + busy_caller_table_free(tabp); + pp->sched.taskq.local.busy.last = NULL; + pp->sched.taskq.local.busy.table = NULL; + } + last_ptp->u.alive.next = pp->sched.taskq.local.first; + pp->sched.taskq.local.first = first_ptp; +} + +#ifdef ERTS_HARD_DEBUG_TASK_QUEUES + +static void +chk_task_queues(Port *pp, ErtsPortTask *execq, int processing_busy_queue) +{ + Sint tot_count, tot_table_count; + int bix; + ErtsPortTask *ptp, *last; + ErtsPortTask *first = processing_busy_queue ? execq : pp->sched.taskq.local.busy.first; + ErtsPortTask *nb_task_queue = processing_busy_queue ? pp->sched.taskq.local.first : execq; + ErtsPortTaskBusyCallerTable *tabp = pp->sched.taskq.local.busy.table; + ErtsPortTaskBusyCaller *bcp; + + if (!first) { + ASSERT(!tabp); + ASSERT(!pp->sched.taskq.local.busy.last); + ASSERT(!(erts_smp_atomic32_read_nob(&pp->sched.flags) & ERTS_PTS_FLG_HAVE_BUSY_TASKS)); + return; + } + + ASSERT(erts_smp_atomic32_read_nob(&pp->sched.flags) & ERTS_PTS_FLG_HAVE_BUSY_TASKS); + ASSERT(tabp); + + tot_count = 0; + ptp = first; + while (ptp) { + Sint count = 0; + Eterm caller = task_caller(ptp); + int bix = caller2bix(caller); + for (bcp = tabp->bucket[bix]; bcp; bcp = bcp->next) + if (bcp->caller == caller) + break; + ASSERT(bcp && bcp->caller == caller); + + ASSERT(bcp->last); + while (1) { + ErtsPortTask *ptp2; + + ASSERT(caller == task_caller(ptp)); + count++; + tot_count++; + last = ptp; + + for (ptp2 = nb_task_queue; ptp2; ptp2 = ptp2->u.alive.next) { + ASSERT(ptp != ptp2); + } + + if (ptp == bcp->last) + break; + ptp = ptp->u.alive.next; + } + + ASSERT(count == bcp->count); + ptp = ptp->u.alive.next; + } + + tot_table_count = 0; + for (bix = 0; bix < ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS; bix++) { + for (bcp = tabp->bucket[bix]; bcp; bcp = bcp->next) + tot_table_count += bcp->count; + } + + ASSERT(tot_count == tot_table_count); + + ASSERT(last == pp->sched.taskq.local.busy.last); +} + +#endif /* ERTS_HARD_DEBUG_TASK_QUEUES */ /* * Task handle manipulation. */ +static ERTS_INLINE void +reset_port_task_handle(ErtsPortTaskHandle *pthp) +{ + erts_smp_atomic_set_relb(pthp, (erts_aint_t) NULL); +} + static ERTS_INLINE ErtsPortTask * handle2task(ErtsPortTaskHandle *pthp) { - return (ErtsPortTask *) erts_smp_atomic_read_nob(pthp); + return (ErtsPortTask *) erts_smp_atomic_read_acqb(pthp); } static ERTS_INLINE void reset_handle(ErtsPortTask *ptp) { - if (ptp->handle) { - ASSERT(ptp == handle2task(ptp->handle)); - erts_smp_atomic_set_nob(ptp->handle, (erts_aint_t) NULL); + if (ptp->u.alive.handle) { + ASSERT(ptp == handle2task(ptp->u.alive.handle)); + reset_port_task_handle(ptp->u.alive.handle); } } static ERTS_INLINE void set_handle(ErtsPortTask *ptp, ErtsPortTaskHandle *pthp) { - ptp->handle = pthp; + ptp->u.alive.handle = pthp; if (pthp) { - erts_smp_atomic_set_nob(pthp, (erts_aint_t) ptp); - ASSERT(ptp == handle2task(ptp->handle)); + erts_smp_atomic_set_relb(pthp, (erts_aint_t) ptp); + ASSERT(ptp == handle2task(ptp->u.alive.handle)); + } +} + + +/* + * Busy port queue management + */ + +static erts_aint32_t +check_unset_busy_port_q(Port *pp, + erts_aint32_t flags, + ErtsPortTaskBusyPortQ *bpq) +{ + ErlDrvSizeT qsize, low; + int resume_procs = 0; + + ASSERT(bpq); + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); + + erts_port_task_sched_lock(&pp->sched); + qsize = (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->size); + low = (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->low); + if (qsize < low) { + erts_aint32_t mask = ~(ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q + | ERTS_PTS_FLG_BUSY_PORT_Q); + flags = erts_smp_atomic32_read_band_relb(&pp->sched.flags, mask); + if ((flags & ERTS_PTS_FLGS_BUSY) == ERTS_PTS_FLG_BUSY_PORT_Q) + resume_procs = 1; + } + else if (flags & ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q) { + flags = erts_smp_atomic32_read_band_relb(&pp->sched.flags, + ~ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q); + flags &= ~ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q; + } + erts_port_task_sched_unlock(&pp->sched); + if (resume_procs) + erts_port_resume_procs(pp); + + return flags; +} + +static ERTS_INLINE void +aborted_proc2port_data(Port *pp, ErlDrvSizeT size) +{ + ErtsPortTaskBusyPortQ *bpq; + erts_aint32_t flags; + ErlDrvSizeT qsz; + + ASSERT(pp->sched.taskq.bpq); + + if (size == 0) + return; + + bpq = pp->sched.taskq.bpq; + + qsz = (ErlDrvSizeT) erts_smp_atomic_add_read_acqb(&bpq->size, + (erts_aint_t) -size); + ASSERT(qsz + size > qsz); + flags = erts_smp_atomic32_read_nob(&pp->sched.flags); + ASSERT(pp->sched.taskq.bpq); + if ((flags & (ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q + | ERTS_PTS_FLG_BUSY_PORT_Q)) != ERTS_PTS_FLG_BUSY_PORT_Q) + return; + if (qsz < (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->low)) + erts_smp_atomic32_read_bor_nob(&pp->sched.flags, + ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q); +} + +static ERTS_INLINE void +dequeued_proc2port_data(Port *pp, ErlDrvSizeT size) +{ + ErtsPortTaskBusyPortQ *bpq; + erts_aint32_t flags; + ErlDrvSizeT qsz; + + ASSERT(pp->sched.taskq.bpq); + + if (size == 0) + return; + + bpq = pp->sched.taskq.bpq; + + qsz = (ErlDrvSizeT) erts_smp_atomic_add_read_acqb(&bpq->size, + (erts_aint_t) -size); + ASSERT(qsz + size > qsz); + flags = erts_smp_atomic32_read_nob(&pp->sched.flags); + if (!(flags & ERTS_PTS_FLG_BUSY_PORT_Q)) + return; + if (qsz < (ErlDrvSizeT) erts_smp_atomic_read_acqb(&bpq->low)) + check_unset_busy_port_q(pp, flags, bpq); +} + +static ERTS_INLINE erts_aint32_t +enqueue_proc2port_data(Port *pp, + ErtsProc2PortSigData *sigdp, + erts_aint32_t flags) +{ + ErtsPortTaskBusyPortQ *bpq = pp->sched.taskq.bpq; + if (sigdp && bpq) { + ErlDrvSizeT size = erts_proc2port_sig_command_data_size(sigdp); + if (size) { + erts_aint_t asize = erts_smp_atomic_add_read_acqb(&bpq->size, + (erts_aint_t) size); + ErlDrvSizeT qsz = (ErlDrvSizeT) asize; + + ASSERT(qsz - size < qsz); + + if (!(flags & ERTS_PTS_FLG_BUSY_PORT_Q) && qsz > bpq->high) { + flags = erts_smp_atomic32_read_bor_acqb(&pp->sched.flags, + ERTS_PTS_FLG_BUSY_PORT_Q); + flags |= ERTS_PTS_FLG_BUSY_PORT_Q; + qsz = (ErlDrvSizeT) erts_smp_atomic_read_acqb(&bpq->size); + if (qsz < (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->low)) { + flags = (erts_smp_atomic32_read_bor_relb( + &pp->sched.flags, + ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q)); + flags |= ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q; + } + } + ASSERT(!(flags & ERTS_PTS_FLG_EXIT)); + } + } + return flags; +} + +/* + * erl_drv_busy_msgq_limits() is called by drivers either reading or + * writing the limits. + * + * A limit of zero is interpreted as a read only request (using a + * limit of zero would not be useful). Other values are interpreted + * as a write-read request. + */ + +void +erl_drv_busy_msgq_limits(ErlDrvPort dport, ErlDrvSizeT *lowp, ErlDrvSizeT *highp) +{ + Port *pp = erts_drvport2port(dport, NULL); + ErtsPortTaskBusyPortQ *bpq = pp->sched.taskq.bpq; + int written = 0, resume_procs = 0; + ErlDrvSizeT low, high; + + if (!pp || !bpq) { + if (lowp) + *lowp = ERL_DRV_BUSY_MSGQ_DISABLED; + if (highp) + *highp = ERL_DRV_BUSY_MSGQ_DISABLED; + return; + } + + low = lowp ? *lowp : 0; + high = highp ? *highp : 0; + + erts_port_task_sched_lock(&pp->sched); + + if (low == ERL_DRV_BUSY_MSGQ_DISABLED + || high == ERL_DRV_BUSY_MSGQ_DISABLED) { + /* Disable busy msgq feature */ + erts_aint32_t flags; + pp->sched.taskq.bpq = NULL; + flags = ~(ERTS_PTS_FLG_BUSY_PORT_Q|ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q); + flags = erts_smp_atomic32_read_band_acqb(&pp->sched.flags, flags); + if ((flags & ERTS_PTS_FLGS_BUSY) == ERTS_PTS_FLG_BUSY_PORT_Q) + resume_procs = 1; + } + else { + + if (!low) + low = (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->low); + else { + if (bpq->high < low) + bpq->high = low; + erts_smp_atomic_set_relb(&bpq->low, (erts_aint_t) low); + written = 1; + } + + if (!high) + high = bpq->high; + else { + if (low > high) { + low = high; + erts_smp_atomic_set_relb(&bpq->low, (erts_aint_t) low); + } + bpq->high = high; + written = 1; + } + + if (written) { + ErlDrvSizeT size = (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->size); + if (size > high) + erts_smp_atomic32_read_bor_relb(&pp->sched.flags, + ERTS_PTS_FLG_BUSY_PORT_Q); + else if (size < low) + erts_smp_atomic32_read_bor_relb(&pp->sched.flags, + ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q); + } + } + + erts_port_task_sched_unlock(&pp->sched); + + if (resume_procs) + erts_port_resume_procs(pp); + if (lowp) + *lowp = low; + if (highp) + *highp = high; +} + +/* + * No-suspend handles. + */ + +#ifdef ERTS_SMP +static void +free_port_task_handle_list(void *vpthlp) +{ + erts_free(ERTS_ALC_T_PT_HNDL_LIST, vpthlp); +} +#endif + +static void +schedule_port_task_handle_list_free(ErtsPortTaskHandleList *pthlp) +{ +#ifdef ERTS_SMP + erts_schedule_thr_prgr_later_op(free_port_task_handle_list, + (void *) pthlp, + &pthlp->u.release); +#else + erts_free(ERTS_ALC_T_PT_HNDL_LIST, pthlp); +#endif +} + +static ERTS_INLINE void +abort_nosuspend_task(Port *pp, + ErtsPortTaskType type, + ErtsPortTaskTypeData *tdp) +{ + + ASSERT(type == ERTS_PORT_TASK_PROC_SIG); + + if (!pp->sched.taskq.bpq) + tdp->psig.callback(NULL, + ERTS_PORT_SFLG_INVALID, + ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND, + &tdp->psig.data); + else { + ErlDrvSizeT size = erts_proc2port_sig_command_data_size(&tdp->psig.data); + tdp->psig.callback(NULL, + ERTS_PORT_SFLG_INVALID, + ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND, + &tdp->psig.data); + aborted_proc2port_data(pp, size); + } +} + +static ErtsPortTaskHandleList * +get_free_nosuspend_handles(Port *pp) +{ + ErtsPortTaskHandleList *nshp, *last_nshp = NULL; + + ERTS_SMP_LC_ASSERT(erts_port_task_sched_lock_is_locked(&pp->sched)); + + nshp = pp->sched.taskq.local.busy.nosuspend; + + while (nshp && !erts_port_task_is_scheduled(&nshp->handle)) { + last_nshp = nshp; + nshp = nshp->u.next; + } + + if (!last_nshp) + nshp = NULL; + else { + nshp = pp->sched.taskq.local.busy.nosuspend; + pp->sched.taskq.local.busy.nosuspend = last_nshp->u.next; + last_nshp->u.next = NULL; + if (!pp->sched.taskq.local.busy.nosuspend) + erts_smp_atomic32_read_band_nob(&pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_NS_TASKS); + } + return nshp; +} + +static void +free_nosuspend_handles(ErtsPortTaskHandleList *free_nshp) +{ + while (free_nshp) { + ErtsPortTaskHandleList *nshp = free_nshp; + free_nshp = free_nshp->u.next; + schedule_port_task_handle_list_free(nshp); } } @@ -161,7 +837,6 @@ enqueue_port(ErtsRunQueue *runq, Port *pp) { ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(runq)); pp->sched.next = NULL; - pp->sched.in_runq = 1; if (runq->ports.end) { ASSERT(runq->ports.start); runq->ports.end->sched.next = pp; @@ -199,285 +874,423 @@ pop_port(ErtsRunQueue *runq) return pp; } +/* + * Task queue operations + */ -#ifdef HARD_DEBUG +static ERTS_INLINE int +enqueue_task(Port *pp, + ErtsPortTask *ptp, + ErtsProc2PortSigData *sigdp, + ErtsPortTaskHandleList *ns_pthlp, + erts_aint32_t *flagsp) -static void -check_port_queue(ErtsRunQueue *runq, Port *chk_pp, int inq) { - Port *pp; - Port *last_pp; - Port *first_pp = runq->ports.start; - int no_forward = 0, no_backward = 0; - int found_forward = 0, found_backward = 0; - if (!first_pp) { - ASSERT(!runq->ports.end); - } + int res; + erts_aint32_t fail_flags = ERTS_PTS_FLG_EXIT; + erts_aint32_t flags; + ptp->u.alive.next = NULL; + if (ns_pthlp) + fail_flags |= ERTS_PTS_FLG_BUSY_PORT; + erts_port_task_sched_lock(&pp->sched); + flags = erts_smp_atomic32_read_nob(&pp->sched.flags); + if (flags & fail_flags) + res = 0; else { - ASSERT(!first_pp->sched.prev); - for (pp = first_pp; pp; pp = pp->sched.next) { - ASSERT(pp->sched.taskq); - if (pp->sched.taskq->first) - no_forward++; - if (chk_pp == pp) - found_forward = 1; - if (!pp->sched.prev) { - ASSERT(first_pp == pp); - } - if (!pp->sched.next) { - ASSERT(runq->ports.end == pp); - last_pp = pp; - } - } - for (pp = last_pp; pp; pp = pp->sched.prev) { - ASSERT(pp->sched.taskq); - if (pp->sched.taskq->last) - no_backward++; - if (chk_pp == pp) - found_backward = 1; - if (!pp->sched.prev) { - ASSERT(first_pp == pp); - } - if (!pp->sched.next) { - ASSERT(runq->ports.end == pp); - } - check_task_queue(pp->sched.taskq, NULL, 0); + if (ns_pthlp) { + ns_pthlp->u.next = pp->sched.taskq.local.busy.nosuspend; + pp->sched.taskq.local.busy.nosuspend = ns_pthlp; } - ASSERT(no_forward == no_backward); - } - ASSERT(no_forward == RUNQ_READ_LEN(&runq->ports.info.len)); - if (chk_pp) { - if (chk_pp->sched.taskq || chk_pp->sched.exe_taskq) { - ASSERT(chk_pp->sched.taskq != chk_pp->sched.exe_taskq); - } - ASSERT(!chk_pp->sched.taskq || chk_pp->sched.taskq->first); - if (inq < 0) - inq = chk_pp->sched.taskq && !chk_pp->sched.exe_taskq; - if (inq) { - ASSERT(found_forward && found_backward); + if (pp->sched.taskq.in.last) { + ASSERT(pp->sched.taskq.in.first); + ASSERT(!pp->sched.taskq.in.last->u.alive.next); + + pp->sched.taskq.in.last->u.alive.next = ptp; } else { - ASSERT(!found_forward && !found_backward); - } - } -} - -#endif + ASSERT(!pp->sched.taskq.in.first); -/* - * Task queue operations - */ - -static ERTS_INLINE ErtsPortTaskQueue * -port_taskq_init(ErtsPortTaskQueue *ptqp, Port *pp) -{ - if (ptqp) { - ptqp->first = NULL; - ptqp->last = NULL; - ptqp->port = pp; + pp->sched.taskq.in.first = ptp; + } + pp->sched.taskq.in.last = ptp; + flags = enqueue_proc2port_data(pp, sigdp, flags); + res = 1; } - return ptqp; + erts_port_task_sched_unlock(&pp->sched); + *flagsp = flags; + return res; } static ERTS_INLINE void -enqueue_task(ErtsPortTaskQueue *ptqp, ErtsPortTask *ptp) +prepare_exec(Port *pp, ErtsPortTask **execqp, int *processing_busy_q_p) { - ERTS_PT_CHK_NOT_IN_TASKQ(ptqp, ptp); - ptp->next = NULL; - ptp->prev = ptqp->last; - ptp->queue = ptqp; - if (ptqp->last) { - ASSERT(ptqp->first); - ptqp->last->next = ptp; + erts_aint32_t act = erts_smp_atomic32_read_nob(&pp->sched.flags); + + if (!pp->sched.taskq.local.busy.first || (act & ERTS_PTS_FLG_BUSY_PORT)) { + *execqp = pp->sched.taskq.local.first; + *processing_busy_q_p = 0; } else { - ASSERT(!ptqp->first); - ptqp->first = ptp; + *execqp = pp->sched.taskq.local.busy.first; + *processing_busy_q_p = 1; + } + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + + while (1) { + erts_aint32_t new, exp; + + new = exp = act; + + new &= ~ERTS_PTS_FLG_IN_RUNQ; + new |= ERTS_PTS_FLG_EXEC; + + act = erts_smp_atomic32_cmpxchg_nob(&pp->sched.flags, new, exp); + + ASSERT(act & ERTS_PTS_FLG_IN_RUNQ); + + if (exp == act) + break; } - ptqp->last = ptp; - ERTS_PT_CHK_IN_TASKQ(ptqp, ptp); } -static ERTS_INLINE void -push_task(ErtsPortTaskQueue *ptqp, ErtsPortTask *ptp) +/* finalize_exec() return value != 0 if port should remain active */ +static ERTS_INLINE int +finalize_exec(Port *pp, ErtsPortTask **execq, int processing_busy_q) { - ERTS_PT_CHK_NOT_IN_TASKQ(ptqp, ptp); - ptp->next = ptqp->first; - ptp->prev = NULL; - ptp->queue = ptqp; - if (ptqp->first) { - ASSERT(ptqp->last); - ptqp->first->prev = ptp; - } + erts_aint32_t act; + + if (!processing_busy_q) + pp->sched.taskq.local.first = *execq; else { - ASSERT(!ptqp->last); - ptqp->last = ptp; + pp->sched.taskq.local.busy.first = *execq; + ASSERT(*execq); + } + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execq, processing_busy_q); + + *execq = NULL; + + act = erts_smp_atomic32_read_nob(&pp->sched.flags); + if (act & ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q) + act = check_unset_busy_port_q(pp, act, pp->sched.taskq.bpq); + + while (1) { + erts_aint32_t new, exp; + + new = exp = act; + + new &= ~ERTS_PTS_FLG_EXEC; + if (act & ERTS_PTS_FLG_HAVE_TASKS) + new |= ERTS_PTS_FLG_IN_RUNQ; + + act = erts_smp_atomic32_cmpxchg_relb(&pp->sched.flags, new, exp); + + ASSERT(!(act & ERTS_PTS_FLG_IN_RUNQ)); + + if (exp == act) + break; } - ptqp->first = ptp; - ERTS_PT_CHK_IN_TASKQ(ptqp, ptp); + + return (act & ERTS_PTS_FLG_HAVE_TASKS) != 0; } -static ERTS_INLINE void -dequeue_task(ErtsPortTask *ptp) +static ERTS_INLINE erts_aint32_t +select_queue_for_exec(Port *pp, ErtsPortTask **execqp, int *processing_busy_q_p) { - ASSERT(ptp); - ASSERT(ptp->queue); - ERTS_PT_CHK_IN_TASKQ(ptp->queue, ptp); - if (ptp->next) - ptp->next->prev = ptp->prev; - else { - ASSERT(ptp->queue->last == ptp); - ptp->queue->last = ptp->prev; + erts_aint32_t flags = erts_smp_atomic32_read_nob(&pp->sched.flags); + + if (flags & ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q) + flags = check_unset_busy_port_q(pp, flags, pp->sched.taskq.bpq); + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + + if (flags & ERTS_PTS_FLG_BUSY_PORT) { + if (*processing_busy_q_p) { + ErtsPortTask *ptp; + + ptp = pp->sched.taskq.local.busy.first = *execqp; + if (!ptp) + pp->sched.taskq.local.busy.last = NULL; + else if (!(ptp->u.alive.flags & ERTS_PT_FLG_WAIT_BUSY)) + no_sig_dep_move_from_busyq(pp); + + *execqp = pp->sched.taskq.local.first; + *processing_busy_q_p = 0; + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + } + + return flags; } - if (ptp->prev) - ptp->prev->next = ptp->next; - else { - ASSERT(ptp->queue->first == ptp); - ptp->queue->first = ptp->next; + + /* Not busy */ + + if (!*processing_busy_q_p && pp->sched.taskq.local.busy.first) { + pp->sched.taskq.local.first = *execqp; + *execqp = pp->sched.taskq.local.busy.first; + *processing_busy_q_p = 1; + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); } - ASSERT(ptp->queue->first || !ptp->queue->last); - ASSERT(ptp->queue->last || !ptp->queue->first); - ERTS_PT_CHK_NOT_IN_TASKQ(ptp->queue, ptp); + return flags; } -static ERTS_INLINE ErtsPortTask * -pop_task(ErtsPortTaskQueue *ptqp) +/* + * check_task_for_exec() returns a value !0 if the task + * is ok to execute; otherwise 0. + */ +static ERTS_INLINE int +check_task_for_exec(Port *pp, + erts_aint32_t flags, + ErtsPortTask **execqp, + int *processing_busy_q_p, + ErtsPortTask *ptp) { - ErtsPortTask *ptp = ptqp->first; - if (!ptp) { - ASSERT(!ptqp->last); + + if (!*processing_busy_q_p) { + /* Processing normal queue */ + + ERTS_PT_DBG_CHK_TASK_QS(pp, ptp, *processing_busy_q_p); + + if ((flags & ERTS_PTS_FLG_BUSY_PORT) + && (ptp->u.alive.flags & ERTS_PT_FLG_WAIT_BUSY)) { + + busy_wait_move_to_busy_queue(pp, ptp); + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + + return 0; + } + + if (pp->sched.taskq.local.busy.last + && (ptp->u.alive.flags & ERTS_PT_FLG_SIG_DEP)) { + + int res = !check_sig_dep_move_to_busy_queue(pp, ptp); + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + + return res; + } + } else { - ERTS_PT_CHK_IN_TASKQ(ptqp, ptp); - ASSERT(!ptp->prev); - ptqp->first = ptp->next; - if (ptqp->first) - ptqp->first->prev = NULL; - else { - ASSERT(ptqp->last == ptp); - ptqp->last = NULL; + /* Processing busy queue */ + + ASSERT(!(flags & ERTS_PTS_FLG_BUSY_PORT)); + + ERTS_PT_DBG_CHK_TASK_QS(pp, ptp, *processing_busy_q_p); + + popped_from_busy_queue(pp, ptp, !*execqp); + + if (!*execqp) { + *execqp = pp->sched.taskq.local.first; + *processing_busy_q_p = 0; } - ASSERT(ptp->queue->first || !ptp->queue->last); - ASSERT(ptp->queue->last || !ptp->queue->first); + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + } - ERTS_PT_CHK_NOT_IN_TASKQ(ptqp, ptp); - return ptp; + + return 1; } -#ifdef HARD_DEBUG +static ErtsPortTask * +fetch_in_queue(Port *pp, ErtsPortTask **execqp) +{ + ErtsPortTask *ptp; + ErtsPortTaskHandleList *free_nshp = NULL; -static void -check_task_queue(ErtsPortTaskQueue *ptqp, - ErtsPortTask *chk_ptp, - int inq) + erts_port_task_sched_lock(&pp->sched); + + ptp = pp->sched.taskq.in.first; + pp->sched.taskq.in.first = NULL; + pp->sched.taskq.in.last = NULL; + if (ptp) + *execqp = ptp->u.alive.next; + else + erts_smp_atomic32_read_band_nob(&pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_TASKS); + + + if (pp->sched.taskq.local.busy.nosuspend) + free_nshp = get_free_nosuspend_handles(pp); + + erts_port_task_sched_unlock(&pp->sched); + + if (free_nshp) + free_nosuspend_handles(free_nshp); + + return ptp; +} + +static ERTS_INLINE ErtsPortTask * +select_task_for_exec(Port *pp, + ErtsPortTask **execqp, + int *processing_busy_q_p) { ErtsPortTask *ptp; - ErtsPortTask *last_ptp; - ErtsPortTask *first_ptp = ptqp->first; - int found_forward = 0, found_backward = 0; - if (!first_ptp) { - ASSERT(!ptqp->last); - } - else { - ASSERT(!first_ptp->prev); - for (ptp = first_ptp; ptp; ptp = ptp->next) { - ASSERT(ptp->queue == ptqp); - if (chk_ptp == ptp) - found_forward = 1; - if (!ptp->prev) { - ASSERT(first_ptp == ptp); - } - if (!ptp->next) { - ASSERT(ptqp->last == ptp); - last_ptp = ptp; - } - } - for (ptp = last_ptp; ptp; ptp = ptp->prev) { - ASSERT(ptp->queue == ptqp); - if (chk_ptp == ptp) - found_backward = 1; - if (!ptp->prev) { - ASSERT(first_ptp == ptp); - } - if (!ptp->next) { - ASSERT(ptqp->last == ptp); - } - } - } - if (chk_ptp) { - if (inq) { - ASSERT(found_forward && found_backward); - } + erts_aint32_t flags; + + flags = select_queue_for_exec(pp, execqp, processing_busy_q_p); + + while (1) { + ptp = *execqp; + if (ptp) + *execqp = ptp->u.alive.next; else { - ASSERT(!found_forward && !found_backward); + ptp = fetch_in_queue(pp, execqp); + if (!ptp) + return NULL; } + if (check_task_for_exec(pp, flags, execqp, processing_busy_q_p, ptp)) + return ptp; } } -#endif /* * Abort a scheduled task. */ int -erts_port_task_abort(Eterm id, ErtsPortTaskHandle *pthp) +erts_port_task_abort(ErtsPortTaskHandle *pthp) { - ErtsRunQueue *runq; - ErtsPortTaskQueue *ptqp; + int res; ErtsPortTask *ptp; - Port *pp; - - pp = &erts_port[internal_port_index(id)]; - runq = erts_port_runq(pp); - if (!runq) - return 1; +#ifdef ERTS_SMP + ErtsThrPrgrDelayHandle dhndl = erts_thr_progress_unmanaged_delay(); +#endif ptp = handle2task(pthp); + if (!ptp) + res = -1; + else { + erts_aint32_t old_state; + +#ifdef DEBUG + ErtsPortTaskHandle *saved_pthp = ptp->u.alive.handle; + ERTS_SMP_READ_MEMORY_BARRIER; + old_state = erts_smp_atomic32_read_nob(&ptp->state); + if (old_state == ERTS_PT_STATE_SCHEDULED) { + ASSERT(saved_pthp == pthp); + } +#endif - if (!ptp) { - erts_smp_runq_unlock(runq); - return 1; + old_state = erts_smp_atomic32_cmpxchg_nob(&ptp->state, + ERTS_PT_STATE_ABORTED, + ERTS_PT_STATE_SCHEDULED); + if (old_state != ERTS_PT_STATE_SCHEDULED) + res = - 1; /* Task already aborted, executing, or executed */ + else { + + reset_port_task_handle(pthp); + + switch (ptp->type) { + case ERTS_PORT_TASK_INPUT: + case ERTS_PORT_TASK_OUTPUT: + case ERTS_PORT_TASK_EVENT: + ASSERT(erts_smp_atomic_read_nob( + &erts_port_task_outstanding_io_tasks) > 0); + erts_smp_atomic_dec_relb(&erts_port_task_outstanding_io_tasks); + break; + case ERTS_PORT_TASK_PROC_SIG: + ERTS_INTERNAL_ERROR("Aborted process to port signal"); + break; + default: + break; + } + + res = 0; + } } - ASSERT(ptp->handle == pthp); - ptqp = ptp->queue; - ASSERT(pp == ptqp->port); +#ifdef ERTS_SMP + erts_thr_progress_unmanaged_continue(dhndl); +#endif - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - ASSERT(ptqp); - ASSERT(ptqp->first); + return res; +} - dequeue_task(ptp); - reset_handle(ptp); +void +erts_port_task_abort_nosuspend_tasks(Port *pp) +{ + erts_aint32_t flags; + ErtsPortTaskHandleList *abort_list; +#ifdef ERTS_SMP + ErtsThrPrgrDelayHandle dhndl = ERTS_THR_PRGR_DHANDLE_INVALID; +#endif - switch (ptp->type) { - case ERTS_PORT_TASK_INPUT: - case ERTS_PORT_TASK_OUTPUT: - case ERTS_PORT_TASK_EVENT: - ASSERT(erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) > 0); - erts_smp_atomic_dec_relb(&erts_port_task_outstanding_io_tasks); - break; - default: - break; - } + erts_port_task_sched_lock(&pp->sched); + flags = erts_smp_atomic32_read_band_nob(&pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_NS_TASKS); + abort_list = pp->sched.taskq.local.busy.nosuspend; + pp->sched.taskq.local.busy.nosuspend = NULL; + erts_port_task_sched_unlock(&pp->sched); - ASSERT(ptqp == pp->sched.taskq || ptqp == pp->sched.exe_taskq); + while (abort_list) { +#ifdef DEBUG + ErtsPortTaskHandle *saved_pthp; +#endif + ErtsPortTaskType type; + ErtsPortTaskTypeData td; + ErtsPortTaskHandle *pthp; + ErtsPortTask *ptp; + ErtsPortTaskHandleList *pthlp; + erts_aint32_t old_state; - if (ptqp->first || pp->sched.taskq != ptqp) - ptqp = NULL; - else - pp->sched.taskq = NULL; + pthlp = abort_list; + abort_list = pthlp->u.next; - ERTS_PT_CHK_PRES_PORTQ(runq, pp); +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + dhndl = erts_thr_progress_unmanaged_delay(); +#endif - erts_smp_runq_unlock(runq); + pthp = &pthlp->handle; + ptp = handle2task(pthp); + if (!ptp) { +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue(dhndl); +#endif + schedule_port_task_handle_list_free(pthlp); + continue; + } - port_task_free(ptp); - if (ptqp) - port_taskq_free(ptqp); +#ifdef DEBUG + saved_pthp = ptp->u.alive.handle; + ERTS_SMP_READ_MEMORY_BARRIER; + old_state = erts_smp_atomic32_read_nob(&ptp->state); + if (old_state == ERTS_PT_STATE_SCHEDULED) { + ASSERT(saved_pthp == pthp); + } +#endif - return 0; + old_state = erts_smp_atomic32_cmpxchg_nob(&ptp->state, + ERTS_PT_STATE_ABORTED, + ERTS_PT_STATE_SCHEDULED); + if (old_state != ERTS_PT_STATE_SCHEDULED) { + /* Task already aborted, executing, or executed */ +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue(dhndl); +#endif + schedule_port_task_handle_list_free(pthlp); + continue; + } + + reset_port_task_handle(pthp); + + type = ptp->type; + td = ptp->u.alive.td; + +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue(dhndl); +#endif + schedule_port_task_handle_list_free(pthlp); + + abort_nosuspend_task(pp, type, &td); + } } /* @@ -488,243 +1301,264 @@ int erts_port_task_schedule(Eterm id, ErtsPortTaskHandle *pthp, ErtsPortTaskType type, - ErlDrvEvent event, - ErlDrvEventData event_data) + ...) { + ErtsProc2PortSigData *sigdp = NULL; + ErtsPortTaskHandleList *ns_pthlp = NULL; +#ifdef ERTS_SMP + ErtsRunQueue *xrunq; + ErtsThrPrgrDelayHandle dhndl; +#endif ErtsRunQueue *runq; Port *pp; - ErtsPortTask *ptp; - int enq_port = 0; - - /* - * NOTE: We might not have the port lock here. We are only - * allowed to access the 'sched', 'tab_status', - * and 'id' fields of the port struct while - * tasks_lock is held. - */ + ErtsPortTask *ptp = NULL; + erts_aint32_t act, add_flags; if (pthp && erts_port_task_is_scheduled(pthp)) { ASSERT(0); - erts_port_task_abort(id, pthp); + erts_port_task_abort(pthp); } - ptp = port_task_alloc(); - ASSERT(is_internal_port(id)); - pp = &erts_port[internal_port_index(id)]; - runq = erts_port_runq(pp); - if (!runq || ERTS_PORT_TASK_INVALID_PORT(pp, id)) { - if (runq) - erts_smp_runq_unlock(runq); - return -1; - } - - ASSERT(!erts_port_task_is_scheduled(pthp)); +#ifdef ERTS_SMP + dhndl = erts_thr_progress_unmanaged_delay(); +#endif - ERTS_PT_CHK_PRES_PORTQ(runq, pp); + pp = erts_port_lookup_raw(id); - if (!pp->sched.taskq && !pp->sched.in_runq && !pp->sched.exe_taskq) { #ifdef ERTS_SMP - ErtsRunQueue *xrunq = erts_check_emigration_need(runq, ERTS_PORT_PRIO_LEVEL); - if (xrunq) { - /* Port emigrated ... */ - erts_smp_atomic_set_nob(&pp->run_queue, (erts_aint_t) xrunq); - erts_smp_runq_unlock(runq); - runq = erts_port_runq(pp); - if (!runq) - return -1; - } - enq_port = !pp->sched.taskq && !pp->sched.in_runq && !pp->sched.exe_taskq; -#else - enq_port = 1; -#endif + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) { + if (pp) + erts_port_inc_refc(pp); + erts_thr_progress_unmanaged_continue(dhndl); } +#endif - ASSERT(!enq_port - || !(ERTS_RUNQ_FLGS_GET_NOB(runq) & ERTS_RUNQ_FLG_SUSPENDED)); + if (!pp) + goto fail; - if (!pp->sched.taskq) - pp->sched.taskq = port_taskq_init(port_taskq_alloc(), pp); + if (type != ERTS_PORT_TASK_PROC_SIG) { + ptp = port_task_alloc(); - ASSERT(ptp); + ptp->type = type; + ptp->u.alive.flags = 0; - ptp->type = type; - ptp->event = event; - ptp->event_data = event_data; + erts_smp_atomic32_init_nob(&ptp->state, ERTS_PT_STATE_SCHEDULED); - set_handle(ptp, pthp); + set_handle(ptp, pthp); + } switch (type) { - case ERTS_PORT_TASK_FREE: - erl_exit(ERTS_ABORT_EXIT, - "erts_port_task_schedule(): Cannot schedule free task\n"); - break; case ERTS_PORT_TASK_INPUT: - case ERTS_PORT_TASK_OUTPUT: - case ERTS_PORT_TASK_EVENT: + case ERTS_PORT_TASK_OUTPUT: { + va_list argp; + va_start(argp, type); + ptp->u.alive.td.io.event = va_arg(argp, ErlDrvEvent); + va_end(argp); erts_smp_atomic_inc_relb(&erts_port_task_outstanding_io_tasks); - /* Fall through... */ + break; + } + case ERTS_PORT_TASK_EVENT: { + va_list argp; + va_start(argp, type); + ptp->u.alive.td.io.event = va_arg(argp, ErlDrvEvent); + ptp->u.alive.td.io.event_data = va_arg(argp, ErlDrvEventData); + va_end(argp); + erts_smp_atomic_inc_relb(&erts_port_task_outstanding_io_tasks); + break; + } + case ERTS_PORT_TASK_PROC_SIG: { + va_list argp; + ASSERT(!pthp); + va_start(argp, type); + sigdp = va_arg(argp, ErtsProc2PortSigData *); + ptp = p2p_sig_data_to_task(sigdp); + ptp->u.alive.td.psig.callback = va_arg(argp, ErtsProc2PortSigCallback); + ptp->u.alive.flags |= va_arg(argp, int); + va_end(argp); + if (!(ptp->u.alive.flags & ERTS_PT_FLG_NOSUSPEND)) + set_handle(ptp, pthp); + else { + ns_pthlp = erts_alloc(ERTS_ALC_T_PT_HNDL_LIST, + sizeof(ErtsPortTaskHandleList)); + set_handle(ptp, &ns_pthlp->handle); + } + break; + } default: - enqueue_task(pp->sched.taskq, ptp); break; } -#ifndef ERTS_SMP - /* - * When (!enq_port && !pp->sched.exe_taskq) is true in the smp case, - * the port might not be in the run queue. If this is the case, another - * thread is in the process of enqueueing the port. This very seldom - * occur, but do occur and is a valid scenario. Debug info showing this - * enqueue in progress must be introduced before we can enable (modified - * versions of these) assertions in the smp case again. - */ -#if defined(HARD_DEBUG) - if (pp->sched.exe_taskq || enq_port) - ERTS_PT_CHK_NOT_IN_PORTQ(runq, pp); - else - ERTS_PT_CHK_IN_PORTQ(runq, pp); -#elif defined(DEBUG) - if (!enq_port && !pp->sched.exe_taskq) { - /* We should be in port run q */ - ASSERT(pp->sched.in_runq); + if (!enqueue_task(pp, ptp, sigdp, ns_pthlp, &act)) { + reset_handle(ptp); + if (ns_pthlp && !(act & ERTS_PTS_FLG_EXIT)) + goto abort_nosuspend; + else + goto fail; } -#endif -#endif - if (!enq_port) { - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - erts_smp_runq_unlock(runq); - } - else { - enqueue_port(runq, pp); - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - - if (erts_system_profile_flags.runnable_ports) { - profile_runnable_port(pp, am_active); + add_flags = ERTS_PTS_FLG_HAVE_TASKS; + if (ns_pthlp) + add_flags |= ERTS_PTS_FLG_HAVE_NS_TASKS; + + while (1) { + erts_aint32_t new, exp; + + if ((act & add_flags) == add_flags + && (act & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC))) + goto done; /* Done */ + + new = exp = act; + new |= add_flags; + if (!(act & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC))) + new |= ERTS_PTS_FLG_IN_RUNQ; + + act = erts_smp_atomic32_cmpxchg_relb(&pp->sched.flags, new, exp); + + if (exp == act) { + if (!(act & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC))) + break; /* Need to enqueue port */ + goto done; /* Done */ } + if (act & ERTS_PTS_FLG_EXIT) + goto done; /* Died after our task insert... */ + } + + /* Enqueue port on run-queue */ + + runq = erts_port_runq(pp); + if (!runq) + ERTS_INTERNAL_ERROR("Missing run-queue"); + +#ifdef ERTS_SMP + xrunq = erts_check_emigration_need(runq, ERTS_PORT_PRIO_LEVEL); + if (xrunq) { + /* Port emigrated ... */ + erts_smp_atomic_set_nob(&pp->run_queue, (erts_aint_t) xrunq); erts_smp_runq_unlock(runq); + runq = erts_port_runq(pp); + if (!runq) + ERTS_INTERNAL_ERROR("Missing run-queue"); + } +#endif - erts_smp_notify_inc_runq(runq); + enqueue_port(runq, pp); + + if (erts_system_profile_flags.runnable_ports) { + profile_runnable_port(pp, am_active); } + + erts_smp_runq_unlock(runq); + + erts_smp_notify_inc_runq(runq); + +done: + +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_port_dec_refc(pp); +#endif + + return 0; + +abort_nosuspend: + +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_port_dec_refc(pp); +#endif + + abort_nosuspend_task(pp, ptp->type, &ptp->u.alive.td); + + ASSERT(ns_pthlp); + erts_free(ERTS_ALC_T_PT_HNDL_LIST, ns_pthlp); + if (ptp) + port_task_free(ptp); + return 0; + +fail: + +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_port_dec_refc(pp); +#endif + + if (ns_pthlp) + erts_free(ERTS_ALC_T_PT_HNDL_LIST, ns_pthlp); + + if (ptp) + port_task_free(ptp); + + return -1; } void erts_port_task_free_port(Port *pp) { + ErtsProcList *suspended; + erts_aint32_t flags; ErtsRunQueue *runq; - ErtsPortTaskQueue *ptqp; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); - ASSERT(!(pp->status & ERTS_PORT_SFLGS_DEAD)); + ASSERT(!(erts_atomic32_read_nob(&pp->state) & ERTS_PORT_SFLGS_DEAD)); + runq = erts_port_runq(pp); - ASSERT(runq); - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - ptqp = pp->sched.exe_taskq; - if (ptqp) { - /* I (this thread) am currently executing this port, free it - when scheduled out... */ - ErtsPortTask *ptp; - enqueue_free: - ptp = port_task_alloc(); - erts_smp_port_state_lock(pp); - pp->status &= ~ERTS_PORT_SFLG_CLOSING; - pp->status |= ERTS_PORT_SFLG_FREE_SCHEDULED; - erts_may_save_closed_port(pp); - erts_smp_port_state_unlock(pp); - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&pp->refc) > 1); - ptp->type = ERTS_PORT_TASK_FREE; - ptp->event = (ErlDrvEvent) -1; - ptp->event_data = NULL; - set_handle(ptp, NULL); - push_task(ptqp, ptp); - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - erts_smp_runq_unlock(runq); - } - else { - if (pp->sched.in_runq) { - ptqp = pp->sched.taskq; - if (!ptqp) - pp->sched.taskq = ptqp = port_taskq_init(port_taskq_alloc(), pp); - goto enqueue_free; - } - ASSERT(!pp->sched.taskq); - erts_smp_port_state_lock(pp); - pp->status &= ~ERTS_PORT_SFLG_CLOSING; - pp->status |= ERTS_PORT_SFLG_FREE_SCHEDULED; - erts_may_save_closed_port(pp); - erts_smp_port_state_unlock(pp); - erts_smp_atomic_dec_nob(&pp->refc); /* Not alive */ - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&pp->refc) > 0); /* Lock */ - handle_remaining_tasks(runq, pp); /* May release runq lock */ - ASSERT(!pp->sched.exe_taskq && (!ptqp || !ptqp->first)); - pp->sched.taskq = NULL; - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - erts_smp_runq_unlock(runq); - } -} + if (!runq) + ERTS_INTERNAL_ERROR("Missing run-queue"); + erts_port_task_sched_lock(&pp->sched); + flags = erts_smp_atomic32_read_bor_relb(&pp->sched.flags, + ERTS_PTS_FLG_EXIT); + suspended = pp->suspended; + pp->suspended = NULL; + erts_port_task_sched_unlock(&pp->sched); + erts_atomic32_read_bset_relb(&pp->state, + (ERTS_PORT_SFLG_CLOSING + | ERTS_PORT_SFLG_FREE), + ERTS_PORT_SFLG_FREE); -typedef struct { - ErtsRunQueue *runq; - int *resp; -} ErtsPortTaskExeBlockData; + erts_smp_runq_unlock(runq); + + if (erts_proclist_fetch(&suspended, NULL)) + erts_resume_processes(suspended); + + if (!(flags & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC))) + begin_port_cleanup(pp, NULL); +} /* - * Run all scheduled tasks for the first port in run queue. If - * new tasks appear while running reschedule port (free task is - * an exception; it is always handled instantly). + * Execute scheduled tasks of a port. * * erts_port_task_execute() is called by scheduler threads between - * scheduleing of processes. Sched lock should be held by caller. + * scheduling of processes. Run-queue lock should be held by caller. */ int erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) { Port *pp; - ErtsPortTaskQueue *ptqp; - ErtsPortTask *ptp; + ErtsPortTask *execq; + int processing_busy_q; int res = 0; int reds = ERTS_PORT_REDS_EXECUTE; erts_aint_t io_tasks_executed = 0; int fpe_was_unmasked; + erts_aint32_t state; + int active; ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(runq)); - ERTS_PT_CHK_PORTQ(runq); - pp = pop_port(runq); if (!pp) { res = 0; goto done; } - ASSERT(pp->sched.in_runq); - pp->sched.in_runq = 0; - if (!pp->sched.taskq) { - if (erts_system_profile_flags.runnable_ports) - profile_runnable_port(pp, am_inactive); - res = (erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) - != (erts_aint_t) 0); - goto done; - } + erts_smp_runq_unlock(runq); *curr_port_pp = pp; - - ASSERT(pp->sched.taskq->first); - ptqp = pp->sched.taskq; - pp->sched.taskq = NULL; - - ASSERT(!pp->sched.exe_taskq); - pp->sched.exe_taskq = ptqp; - - if (erts_smp_port_trylock(pp) == EBUSY) { - erts_smp_runq_unlock(runq); - erts_smp_port_lock(pp); - erts_smp_runq_lock(runq); - } if (erts_sched_stat.enabled) { ErtsSchedulerData *esdp = erts_get_scheduler_data(); @@ -741,77 +1575,94 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) erts_smp_spin_unlock(&erts_sched_stat.lock); } + prepare_exec(pp, &execq, &processing_busy_q); + + erts_smp_port_lock(pp); + /* trace port scheduling, in */ if (IS_TRACED_FL(pp, F_TRACE_SCHED_PORTS)) { trace_sched_ports(pp, am_in); } - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); + fpe_was_unmasked = erts_block_fpe(); - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - ptp = pop_task(ptqp); + state = erts_atomic32_read_nob(&pp->state); + goto begin_handle_tasks; - fpe_was_unmasked = erts_block_fpe(); + while (1) { + erts_aint32_t task_state; + ErtsPortTask *ptp; - while (ptp) { - ASSERT(pp->sched.taskq != pp->sched.exe_taskq); + ptp = select_task_for_exec(pp, &execq, &processing_busy_q); + if (!ptp) + break; + + task_state = erts_smp_atomic32_cmpxchg_nob(&ptp->state, + ERTS_PT_STATE_EXECUTING, + ERTS_PT_STATE_SCHEDULED); + if (task_state != ERTS_PT_STATE_SCHEDULED) { + ASSERT(task_state == ERTS_PT_STATE_ABORTED); + goto aborted_port_task; + } reset_handle(ptp); - erts_smp_runq_unlock(runq); ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); ERTS_SMP_CHK_NO_PROC_LOCKS; ASSERT(pp->drv_ptr); switch (ptp->type) { - case ERTS_PORT_TASK_FREE: /* May be pushed in q at any time */ - reds += ERTS_PORT_REDS_FREE; - erts_smp_runq_lock(runq); - - erts_unblock_fpe(fpe_was_unmasked); - ASSERT(pp->status & ERTS_PORT_SFLG_FREE_SCHEDULED); - if (ptqp->first || (pp->sched.taskq && pp->sched.taskq->first)) - handle_remaining_tasks(runq, pp); - ASSERT(!ptqp->first - && (!pp->sched.taskq || !pp->sched.taskq->first)); - erts_smp_atomic_dec_nob(&pp->refc); /* Not alive */ - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&pp->refc) > 0); /* Lock */ - - port_task_free(ptp); - if (pp->sched.taskq) - port_taskq_free(pp->sched.taskq); - pp->sched.taskq = NULL; - - goto tasks_done; case ERTS_PORT_TASK_TIMEOUT: reds += ERTS_PORT_REDS_TIMEOUT; - if (!(pp->status & ERTS_PORT_SFLGS_DEAD)) { + if (!(state & ERTS_PORT_SFLGS_DEAD)) { DTRACE_DRIVER(driver_timeout, pp); (*pp->drv_ptr->timeout)((ErlDrvData) pp->drv_data); } break; case ERTS_PORT_TASK_INPUT: reds += ERTS_PORT_REDS_INPUT; - ASSERT((pp->status & ERTS_PORT_SFLGS_DEAD) == 0); + ASSERT((state & ERTS_PORT_SFLGS_DEAD) == 0); DTRACE_DRIVER(driver_ready_input, pp); /* NOTE some windows drivers use ->ready_input for input and output */ - (*pp->drv_ptr->ready_input)((ErlDrvData) pp->drv_data, ptp->event); + (*pp->drv_ptr->ready_input)((ErlDrvData) pp->drv_data, + ptp->u.alive.td.io.event); io_tasks_executed++; break; case ERTS_PORT_TASK_OUTPUT: reds += ERTS_PORT_REDS_OUTPUT; - ASSERT((pp->status & ERTS_PORT_SFLGS_DEAD) == 0); + ASSERT((state & ERTS_PORT_SFLGS_DEAD) == 0); DTRACE_DRIVER(driver_ready_output, pp); - (*pp->drv_ptr->ready_output)((ErlDrvData) pp->drv_data, ptp->event); + (*pp->drv_ptr->ready_output)((ErlDrvData) pp->drv_data, + ptp->u.alive.td.io.event); io_tasks_executed++; break; case ERTS_PORT_TASK_EVENT: reds += ERTS_PORT_REDS_EVENT; - ASSERT((pp->status & ERTS_PORT_SFLGS_DEAD) == 0); + ASSERT((state & ERTS_PORT_SFLGS_DEAD) == 0); DTRACE_DRIVER(driver_event, pp); - (*pp->drv_ptr->event)((ErlDrvData) pp->drv_data, ptp->event, ptp->event_data); + (*pp->drv_ptr->event)((ErlDrvData) pp->drv_data, + ptp->u.alive.td.io.event, + ptp->u.alive.td.io.event_data); io_tasks_executed++; break; + case ERTS_PORT_TASK_PROC_SIG: { + ErtsProc2PortSigData *sigdp = &ptp->u.alive.td.psig.data; + ASSERT((state & ERTS_PORT_SFLGS_DEAD) == 0); + if (!pp->sched.taskq.bpq) + reds += ptp->u.alive.td.psig.callback(pp, + state, + ERTS_PROC2PORT_SIG_EXEC, + sigdp); + else { + ErlDrvSizeT size = erts_proc2port_sig_command_data_size(sigdp); + reds += ptp->u.alive.td.psig.callback(pp, + state, + ERTS_PROC2PORT_SIG_EXEC, + sigdp); + dequeued_proc2port_data(pp, size); + } + break; + } case ERTS_PORT_TASK_DIST_CMD: reds += erts_dist_command(pp, CONTEXT_REDS-reds); break; @@ -822,33 +1673,31 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) break; } - if ((pp->status & ERTS_PORT_SFLG_CLOSING) - && erts_is_port_ioq_empty(pp)) { - reds += ERTS_PORT_REDS_TERMINATE; - erts_terminate_port(pp); - } + reds += erts_port_driver_callback_epilogue(pp, &state); - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); + aborted_port_task: + schedule_port_task_free(ptp); -#ifdef ERTS_SMP - if (pp->xports) - erts_smp_xports_unlock(pp); - ASSERT(!pp->xports); -#endif - - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); + begin_handle_tasks: + if (state & ERTS_PORT_SFLG_FREE) { + reds += ERTS_PORT_REDS_FREE; - port_task_free(ptp); + begin_port_cleanup(pp, &execq); - erts_smp_runq_lock(runq); + break; + } - ptp = pop_task(ptqp); + if (reds >= CONTEXT_REDS) + break; } - tasks_done: - erts_unblock_fpe(fpe_was_unmasked); + /* trace port scheduling, out */ + if (IS_TRACED_FL(pp, F_TRACE_SCHED_PORTS)) { + trace_sched_ports(pp, am_out); + } + if (io_tasks_executed) { ASSERT(erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) >= io_tasks_executed); @@ -856,15 +1705,19 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) -1*io_tasks_executed); } - *curr_port_pp = NULL; - #ifdef ERTS_SMP ASSERT(runq == (ErtsRunQueue *) erts_smp_atomic_read_nob(&pp->run_queue)); #endif - if (!pp->sched.taskq) { - ASSERT(pp->sched.exe_taskq); - pp->sched.exe_taskq = NULL; + active = finalize_exec(pp, &execq, processing_busy_q); + + erts_port_release(pp); + + *curr_port_pp = NULL; + + erts_smp_runq_lock(runq); + + if (!active) { if (erts_system_profile_flags.runnable_ports) profile_runnable_port(pp, am_inactive); } @@ -873,16 +1726,13 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) ErtsRunQueue *xrunq; #endif - ASSERT(!(pp->status & ERTS_PORT_SFLGS_DEAD)); - ASSERT(pp->sched.taskq->first); + ASSERT(!(erts_atomic32_read_nob(&pp->state) & ERTS_PORT_SFLGS_DEAD)); #ifdef ERTS_SMP xrunq = erts_check_emigration_need(runq, ERTS_PORT_PRIO_LEVEL); if (!xrunq) { #endif enqueue_port(runq, pp); - ASSERT(pp->sched.exe_taskq); - pp->sched.exe_taskq = NULL; /* No need to notify ourselves about inc in runq. */ #ifdef ERTS_SMP } @@ -892,49 +1742,20 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) erts_smp_runq_unlock(runq); xrunq = erts_port_runq(pp); - if (xrunq) { - enqueue_port(xrunq, pp); - ASSERT(pp->sched.exe_taskq); - pp->sched.exe_taskq = NULL; - erts_smp_runq_unlock(xrunq); - erts_smp_notify_inc_runq(xrunq); - } + ASSERT(xrunq); + enqueue_port(xrunq, pp); + erts_smp_runq_unlock(xrunq); + erts_smp_notify_inc_runq(xrunq); erts_smp_runq_lock(runq); } #endif } + done: res = (erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) != (erts_aint_t) 0); - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - - port_taskq_free(ptqp); - - /* trace port scheduling, out */ - if (IS_TRACED_FL(pp, F_TRACE_SCHED_PORTS)) { - trace_sched_ports(pp, am_out); - } -#ifndef ERTS_SMP - erts_port_release(pp); -#else - { - erts_aint_t refc; - erts_smp_mtx_unlock(pp->lock); - refc = erts_smp_atomic_dec_read_nob(&pp->refc); - ASSERT(refc >= 0); - if (refc == 0) { - erts_smp_runq_unlock(runq); - erts_port_cleanup(pp); /* Might aquire runq lock */ - erts_smp_runq_lock(runq); - res = (erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) - != (erts_aint_t) 0); - } - } -#endif - - done: runq->scheduler->reductions += reds; ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(runq)); @@ -943,78 +1764,146 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) return res; } -/* - * Handle remaining tasks after a free task. - */ +#ifdef ERTS_SMP +static void +release_port(void *vport) +{ + erts_port_dec_refc((Port *) vport); +} +#endif static void -handle_remaining_tasks(ErtsRunQueue *runq, Port *pp) +begin_port_cleanup(Port *pp, ErtsPortTask **execqp) { - int i; - ErtsPortTask *ptp; - ErtsPortTaskQueue *ptqps[] = {pp->sched.exe_taskq, pp->sched.taskq}; + int i, max; + ErtsPortTask *qs[2]; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); - for (i = 0; i < sizeof(ptqps)/sizeof(ErtsPortTaskQueue *); i++) { - if (!ptqps[i]) - continue; - ptp = pop_task(ptqps[i]); - while (ptp) { + /* + * Handle remaining tasks... + */ + + max = 0; + if (execqp && *execqp) { + qs[max++] = *execqp; + *execqp = NULL; + } + + erts_port_task_sched_lock(&pp->sched); + qs[max] = pp->sched.taskq.in.first; + pp->sched.taskq.in.first = NULL; + pp->sched.taskq.in.last = NULL; + erts_port_task_sched_unlock(&pp->sched); + if (qs[max]) + max++; + + for (i = 0; i < max; i++) { + while (1) { + erts_aint32_t state; + ErtsPortTask *ptp = qs[i]; + if (!ptp) + break; + + qs[i] = ptp->u.alive.next; + + /* Normal case here is aborted tasks... */ + state = erts_smp_atomic32_read_nob(&ptp->state); + if (state == ERTS_PT_STATE_ABORTED) + goto aborted_port_task; + + state = erts_smp_atomic32_cmpxchg_nob(&ptp->state, + ERTS_PT_STATE_EXECUTING, + ERTS_PT_STATE_SCHEDULED); + if (state != ERTS_PT_STATE_SCHEDULED) { + ASSERT(state == ERTS_PT_STATE_ABORTED); + goto aborted_port_task; + } + reset_handle(ptp); - erts_smp_runq_unlock(runq); switch (ptp->type) { - case ERTS_PORT_TASK_FREE: case ERTS_PORT_TASK_TIMEOUT: break; case ERTS_PORT_TASK_INPUT: - erts_stale_drv_select(pp->id, ptp->event, DO_READ, 1); + erts_stale_drv_select(pp->common.id, + ptp->u.alive.td.io.event, + DO_READ, + 1); break; case ERTS_PORT_TASK_OUTPUT: - erts_stale_drv_select(pp->id, ptp->event, DO_WRITE, 1); + erts_stale_drv_select(pp->common.id, + ptp->u.alive.td.io.event, + DO_WRITE, + 1); break; case ERTS_PORT_TASK_EVENT: - erts_stale_drv_select(pp->id, ptp->event, 0, 1); + erts_stale_drv_select(pp->common.id, + ptp->u.alive.td.io.event, + 0, + 1); break; case ERTS_PORT_TASK_DIST_CMD: break; + case ERTS_PORT_TASK_PROC_SIG: { + ErtsProc2PortSigData *sigdp = &ptp->u.alive.td.psig.data; + if (!pp->sched.taskq.bpq) + ptp->u.alive.td.psig.callback(NULL, + ERTS_PORT_SFLG_INVALID, + ERTS_PROC2PORT_SIG_ABORT_CLOSED, + sigdp); + else { + ErlDrvSizeT size = erts_proc2port_sig_command_data_size(sigdp); + ptp->u.alive.td.psig.callback(NULL, + ERTS_PORT_SFLG_INVALID, + ERTS_PROC2PORT_SIG_ABORT_CLOSED, + sigdp); + aborted_proc2port_data(pp, size); + } + break; + } default: erl_exit(ERTS_ABORT_EXIT, "Invalid port task type: %d\n", (int) ptp->type); } - port_task_free(ptp); - - erts_smp_runq_lock(runq); - ptp = pop_task(ptqps[i]); + aborted_port_task: + schedule_port_task_free(ptp); } } - ASSERT(!pp->sched.taskq || !pp->sched.taskq->first); + erts_smp_atomic32_read_band_nob(&pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_TASKS); + + /* + * Schedule cleanup of port structure... + */ +#ifdef ERTS_SMP + erts_schedule_thr_prgr_later_op(release_port, + (void *) pp, + &pp->common.u.release); +#else + pp->cleanup = 1; +#endif } int erts_port_is_scheduled(Port *pp) { - int res; - ErtsRunQueue *runq = erts_port_runq(pp); - if (!runq) - return 0; - res = pp->sched.taskq || pp->sched.exe_taskq; - erts_smp_runq_unlock(runq); - return res; + erts_aint32_t flags = erts_smp_atomic32_read_acqb(&pp->sched.flags); + return (flags & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC)) != 0; } #ifdef ERTS_SMP + void erts_enqueue_port(ErtsRunQueue *rq, Port *pp) { ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(rq)); ASSERT(rq == (ErtsRunQueue *) erts_smp_atomic_read_nob(&pp->run_queue)); - ASSERT(pp->sched.in_runq); + ASSERT(erts_smp_atomic32_read_nob(&pp->sched.flags) & ERTS_PTS_FLG_IN_RUNQ); enqueue_port(rq, pp); } @@ -1026,7 +1915,8 @@ erts_dequeue_port(ErtsRunQueue *rq) pp = pop_port(rq); ASSERT(!pp || rq == (ErtsRunQueue *) erts_smp_atomic_read_nob(&pp->run_queue)); - ASSERT(!pp || pp->sched.in_runq); + ASSERT(!pp || (erts_smp_atomic32_read_nob(&pp->sched.flags) + & ERTS_PTS_FLG_IN_RUNQ)); return pp; } @@ -1041,5 +1931,5 @@ erts_port_task_init(void) erts_smp_atomic_init_nob(&erts_port_task_outstanding_io_tasks, (erts_aint_t) 0); init_port_task_alloc(); - init_port_taskq_alloc(); + init_busy_caller_table_alloc(); } diff --git a/erts/emulator/beam/erl_port_task.h b/erts/emulator/beam/erl_port_task.h index fd88b1c1ff..ae6cd69ae2 100644 --- a/erts/emulator/beam/erl_port_task.h +++ b/erts/emulator/beam/erl_port_task.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2006-2011. All Rights Reserved. + * Copyright Ericsson AB 2006-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -27,6 +27,9 @@ #define ERTS_PORT_TASK_H_BASIC_TYPES__ #include "erl_sys_driver.h" #include "erl_smp.h" +#define ERL_PORT_GET_PORT_TYPE_ONLY__ +#include "erl_port.h" +#undef ERL_PORT_GET_PORT_TYPE_ONLY__ typedef erts_smp_atomic_t ErtsPortTaskHandle; #endif @@ -43,13 +46,19 @@ typedef erts_smp_atomic_t ErtsPortTaskHandle; #define ERTS_INCLUDE_SCHEDULER_INTERNALS #endif +#define ERTS_PT_FLG_WAIT_BUSY (1 << 0) +#define ERTS_PT_FLG_SIG_DEP (1 << 1) +#define ERTS_PT_FLG_NOSUSPEND (1 << 2) +#define ERTS_PT_FLG_REF (1 << 3) +#define ERTS_PT_FLG_BAD_OUTPUT (1 << 4) + typedef enum { - ERTS_PORT_TASK_FREE, ERTS_PORT_TASK_INPUT, ERTS_PORT_TASK_OUTPUT, ERTS_PORT_TASK_EVENT, ERTS_PORT_TASK_TIMEOUT, - ERTS_PORT_TASK_DIST_CMD + ERTS_PORT_TASK_DIST_CMD, + ERTS_PORT_TASK_PROC_SIG } ErtsPortTaskType; #ifdef ERTS_INCLUDE_SCHEDULER_INTERNALS @@ -57,19 +66,76 @@ typedef enum { extern erts_smp_atomic_t erts_port_task_outstanding_io_tasks; #endif +#define ERTS_PTS_FLG_IN_RUNQ (((erts_aint32_t) 1) << 0) +#define ERTS_PTS_FLG_EXEC (((erts_aint32_t) 1) << 1) +#define ERTS_PTS_FLG_HAVE_TASKS (((erts_aint32_t) 1) << 2) +#define ERTS_PTS_FLG_EXIT (((erts_aint32_t) 1) << 3) +#define ERTS_PTS_FLG_BUSY_PORT (((erts_aint32_t) 1) << 4) +#define ERTS_PTS_FLG_BUSY_PORT_Q (((erts_aint32_t) 1) << 5) +#define ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q (((erts_aint32_t) 1) << 6) +#define ERTS_PTS_FLG_HAVE_BUSY_TASKS (((erts_aint32_t) 1) << 7) +#define ERTS_PTS_FLG_HAVE_NS_TASKS (((erts_aint32_t) 1) << 8) +#define ERTS_PTS_FLG_PARALLELISM (((erts_aint32_t) 1) << 9) +#define ERTS_PTS_FLG_FORCE_SCHED (((erts_aint32_t) 1) << 10) + +#define ERTS_PTS_FLGS_BUSY \ + (ERTS_PTS_FLG_BUSY_PORT | ERTS_PTS_FLG_BUSY_PORT_Q) + +#define ERTS_PTS_FLGS_FORCE_SCHEDULE_OP \ + (ERTS_PTS_FLG_EXIT \ + | ERTS_PTS_FLG_HAVE_BUSY_TASKS \ + | ERTS_PTS_FLG_HAVE_TASKS \ + | ERTS_PTS_FLG_EXEC \ + | ERTS_PTS_FLG_FORCE_SCHED) + +#define ERTS_PORT_TASK_DEFAULT_BUSY_PORT_Q_HIGH 8192 +#define ERTS_PORT_TASK_DEFAULT_BUSY_PORT_Q_LOW 4096 + +typedef struct { + ErlDrvSizeT high; + erts_smp_atomic_t low; + erts_smp_atomic_t size; +} ErtsPortTaskBusyPortQ; + typedef struct ErtsPortTask_ ErtsPortTask; -typedef struct ErtsPortTaskQueue_ ErtsPortTaskQueue; +typedef struct ErtsPortTaskBusyCallerTable_ ErtsPortTaskBusyCallerTable; +typedef struct ErtsPortTaskHandleList_ ErtsPortTaskHandleList; typedef struct { Port *next; - int in_runq; - ErtsPortTaskQueue *taskq; - ErtsPortTaskQueue *exe_taskq; + struct { + struct { + struct { + ErtsPortTask *first; + ErtsPortTask *last; + ErtsPortTaskBusyCallerTable *table; + ErtsPortTaskHandleList *nosuspend; + } busy; + ErtsPortTask *first; + } local; + struct { + ErtsPortTask *first; + ErtsPortTask *last; + } in; + ErtsPortTaskBusyPortQ *bpq; + } taskq; + erts_smp_atomic32_t flags; +#ifdef ERTS_SMP + erts_mtx_t mtx; +#endif } ErtsPortTaskSched; ERTS_GLB_INLINE void erts_port_task_handle_init(ErtsPortTaskHandle *pthp); ERTS_GLB_INLINE int erts_port_task_is_scheduled(ErtsPortTaskHandle *pthp); -ERTS_GLB_INLINE void erts_port_task_init_sched(ErtsPortTaskSched *ptsp); +ERTS_GLB_INLINE void erts_port_task_pre_init_sched(ErtsPortTaskSched *ptsp, + ErtsPortTaskBusyPortQ *bpq); +ERTS_GLB_INLINE void erts_port_task_init_sched(ErtsPortTaskSched *ptsp, + Eterm id); +ERTS_GLB_INLINE void erts_port_task_fini_sched(ErtsPortTaskSched *ptsp); +ERTS_GLB_INLINE void erts_port_task_sched_lock(ErtsPortTaskSched *ptsp); +ERTS_GLB_INLINE void erts_port_task_sched_unlock(ErtsPortTaskSched *ptsp); +ERTS_GLB_INLINE int erts_port_task_sched_lock_is_locked(ErtsPortTaskSched *ptsp); + #ifdef ERTS_INCLUDE_SCHEDULER_INTERNALS ERTS_GLB_INLINE int erts_port_task_have_outstanding_io_tasks(void); #endif @@ -88,13 +154,75 @@ erts_port_task_is_scheduled(ErtsPortTaskHandle *pthp) return ((void *) erts_smp_atomic_read_nob(pthp)) != NULL; } +ERTS_GLB_INLINE void erts_port_task_pre_init_sched(ErtsPortTaskSched *ptsp, + ErtsPortTaskBusyPortQ *bpq) +{ + if (bpq) { + erts_aint_t low = (erts_aint_t) ERTS_PORT_TASK_DEFAULT_BUSY_PORT_Q_LOW; + erts_smp_atomic_init_nob(&bpq->low, low); + bpq->high = (ErlDrvSizeT) ERTS_PORT_TASK_DEFAULT_BUSY_PORT_Q_HIGH; + erts_smp_atomic_init_nob(&bpq->size, (erts_aint_t) 0); + } + ptsp->taskq.bpq = bpq; +} + ERTS_GLB_INLINE void -erts_port_task_init_sched(ErtsPortTaskSched *ptsp) +erts_port_task_init_sched(ErtsPortTaskSched *ptsp, Eterm instr_id) { +#ifdef ERTS_SMP + char *lock_str = "port_sched_lock"; +#endif ptsp->next = NULL; - ptsp->in_runq = 0; - ptsp->taskq = NULL; - ptsp->exe_taskq = NULL; + ptsp->taskq.local.busy.first = NULL; + ptsp->taskq.local.busy.last = NULL; + ptsp->taskq.local.busy.table = NULL; + ptsp->taskq.local.busy.nosuspend = NULL; + ptsp->taskq.local.first = NULL; + ptsp->taskq.in.first = NULL; + ptsp->taskq.in.last = NULL; + erts_smp_atomic32_init_nob(&ptsp->flags, 0); +#ifdef ERTS_SMP +#ifdef ERTS_ENABLE_LOCK_COUNT + if (!(erts_lcnt_rt_options & ERTS_LCNT_OPT_PORTLOCK)) + lock_str = NULL; +#endif + erts_mtx_init_x(&ptsp->mtx, lock_str, instr_id); +#endif +} + +ERTS_GLB_INLINE void +erts_port_task_sched_lock(ErtsPortTaskSched *ptsp) +{ +#ifdef ERTS_SMP + erts_mtx_lock(&ptsp->mtx); +#endif +} + +ERTS_GLB_INLINE void +erts_port_task_sched_unlock(ErtsPortTaskSched *ptsp) +{ +#ifdef ERTS_SMP + erts_mtx_unlock(&ptsp->mtx); +#endif +} + +ERTS_GLB_INLINE int +erts_port_task_sched_lock_is_locked(ErtsPortTaskSched *ptsp) +{ +#if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) + return erts_lc_mtx_is_locked(&ptsp->mtx); +#else + return 0; +#endif +} + + +ERTS_GLB_INLINE void +erts_port_task_fini_sched(ErtsPortTaskSched *ptsp) +{ +#ifdef ERTS_SMP + erts_mtx_destroy(&ptsp->mtx); +#endif } #ifdef ERTS_INCLUDE_SCHEDULER_INTERNALS @@ -115,14 +243,16 @@ int erts_port_task_execute(ErtsRunQueue *, Port **); void erts_port_task_init(void); #endif -int erts_port_task_abort(Eterm id, ErtsPortTaskHandle *); +int erts_port_task_abort(ErtsPortTaskHandle *); +void erts_port_task_abort_nosuspend_tasks(Port *); + int erts_port_task_schedule(Eterm, ErtsPortTaskHandle *, ErtsPortTaskType, - ErlDrvEvent, - ErlDrvEventData); + ...); void erts_port_task_free_port(Port *); int erts_port_is_scheduled(Port *); +ErtsProc2PortSigData *erts_port_task_alloc_p2p_sig_data(void); #ifdef ERTS_SMP void erts_enqueue_port(ErtsRunQueue *rq, Port *pp); diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index 61554780c4..aaca4b5f59 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -42,6 +42,7 @@ #include "erl_thr_queue.h" #include "erl_async.h" #include "dtrace-wrapper.h" +#include "erl_ptab.h" #define ERTS_DELAYED_WAKEUP_INFINITY (~(Uint64) 0) #define ERTS_DELAYED_WAKEUP_REDUCTIONS ((Uint64) CONTEXT_REDS/2) @@ -136,155 +137,16 @@ do { \ #define ERTS_EMPTY_RUNQ_PORTS(RQ) \ (RUNQ_READ_LEN(&(RQ)->ports.info.len) == 0 && (RQ)->misc.start == NULL) +const Process erts_invalid_process = {{ERTS_INVALID_PID}}; + extern BeamInstr beam_apply[]; extern BeamInstr beam_exit[]; extern BeamInstr beam_continue_exit[]; -#ifdef ARCH_32 - -union { - erts_smp_dw_atomic_t pid_data; - char align[ERTS_CACHE_LINE_SIZE]; -} last erts_align_attribute(ERTS_CACHE_LINE_SIZE); - - -static ERTS_INLINE Uint64 -dw_aint_to_uint64(erts_dw_aint_t *dw) -{ -#ifdef ETHR_SU_DW_NAINT_T__ - return (Uint64) dw->dw_sint; -#else - Uint64 res; - res = (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_HIGH_WORD]); - res <<= 32; - res |= (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_LOW_WORD]); - return res; -#endif -} - -static void -unint64_to_dw_aint(erts_dw_aint_t *dw, Uint64 val) -{ -#ifdef ETHR_SU_DW_NAINT_T__ - dw->dw_sint = (ETHR_SU_DW_NAINT_T__) val; -#else - dw->sint[ERTS_DW_AINT_LOW_WORD] = (erts_aint_t) (val & 0xffffffff); - dw->sint[ERTS_DW_AINT_HIGH_WORD] = (erts_aint_t) ((val >> 32) & 0xffffffff); -#endif -} - -static ERTS_INLINE void -last_pid_data_init_nob(Uint64 val) -{ - erts_dw_aint_t dw; - unint64_to_dw_aint(&dw, val); - erts_smp_dw_atomic_init_nob(&last.pid_data, &dw); -} - -static ERTS_INLINE void -last_pid_data_set_relb(Uint64 val) -{ - erts_dw_aint_t dw; - unint64_to_dw_aint(&dw, val); - erts_smp_dw_atomic_set_relb(&last.pid_data, &dw); -} - -static ERTS_INLINE Uint64 -last_pid_data_read_nob(void) -{ - erts_dw_aint_t dw; - erts_smp_dw_atomic_read_nob(&last.pid_data, &dw); - return dw_aint_to_uint64(&dw); -} - -static ERTS_INLINE Uint64 -last_pid_data_read_acqb(void) -{ - erts_dw_aint_t dw; - erts_smp_dw_atomic_read_acqb(&last.pid_data, &dw); - return dw_aint_to_uint64(&dw); -} - -static ERTS_INLINE Uint64 -last_pid_data_cmpxchg_relb(Uint64 new, Uint64 exp) -{ - erts_dw_aint_t dw_new, dw_xchg; - - unint64_to_dw_aint(&dw_new, new); - unint64_to_dw_aint(&dw_xchg, exp); - - if (erts_smp_dw_atomic_cmpxchg_relb(&last.pid_data, &dw_new, &dw_xchg)) - return exp; - else - return dw_aint_to_uint64(&dw_xchg); -} - -#elif defined(ARCH_64) - -union { - erts_smp_atomic_t pid_data; - char align[ERTS_CACHE_LINE_SIZE]; -} last erts_align_attribute(ERTS_CACHE_LINE_SIZE); - -static ERTS_INLINE void -last_pid_data_init_nob(Uint64 val) -{ - erts_smp_atomic_init_nob(&last.pid_data, (erts_aint_t) val); -} - -static ERTS_INLINE void -last_pid_data_set_relb(Uint64 val) -{ - erts_smp_atomic_set_relb(&last.pid_data, (erts_aint_t) val); -} - -static ERTS_INLINE Uint64 -last_pid_data_read_nob(void) -{ - return (Uint64) erts_smp_atomic_read_nob(&last.pid_data); -} - -static ERTS_INLINE Uint64 -last_pid_data_read_acqb(void) -{ - return (Uint64) erts_smp_atomic_read_acqb(&last.pid_data); -} - -static ERTS_INLINE Uint64 -last_pid_data_cmpxchg_relb(Uint64 new, Uint64 exp) -{ - return (Uint64) erts_smp_atomic_cmpxchg_relb(&last.pid_data, - (erts_aint_t) new, - (erts_aint_t) exp); -} - -#else -# error "Not 64-bit, nor 32-bit architecture..." -#endif - -static ERTS_INLINE int -last_pid_data_cmp(Uint64 lpd1, Uint64 lpd2) -{ - Uint64 lpd1_wrap; - - if (lpd1 == lpd2) - return 0; - - lpd1_wrap = lpd1 + (((Uint64) 1) << 63); - - if (lpd1 < lpd1_wrap) - return (lpd1 < lpd2 && lpd2 < lpd1_wrap) ? -1 : 1; - else - return (lpd1_wrap <= lpd2 && lpd2 < lpd1) ? 1 : -1; -} - - -#define ERTS_PID_DATA_MASK__ ((1 << _PID_DATA_SIZE) - 1) - int erts_sched_compact_load; Uint erts_no_schedulers; -ErtsProcTab erts_proc erts_align_attribute(ERTS_CACHE_LINE_SIZE); +ErtsPTab erts_proc erts_align_attribute(ERTS_CACHE_LINE_SIZE); int erts_sched_thread_suggested_stack_size = -1; @@ -371,8 +233,6 @@ erts_sched_stat_t erts_sched_stat; static erts_tsd_key_t sched_data_key; #endif -erts_smp_rwmtx_t erts_proc_tab_rwmtx; - static erts_smp_atomic32_t function_calls; #ifdef ERTS_SMP @@ -409,29 +269,6 @@ struct erts_system_profile_flags_t erts_system_profile_flags; #if ERTS_MAX_PROCESSES > 0x7fffffff #error "Need to store process_count in another type" #endif -static erts_smp_atomic32_t process_count; - -typedef struct ErtsTermProcElement_ ErtsTermProcElement; -struct ErtsTermProcElement_ { - ErtsTermProcElement *next; - ErtsTermProcElement *prev; - int ix; - union { - struct { - Eterm pid; - Uint64 spawned; - Uint64 exited; - } process; - struct { - Uint64 interval; - } bif_invocation; - } u; -}; - -static struct { - ErtsTermProcElement *start; - ErtsTermProcElement *end; -} saved_term_procs; ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(misc_op_list, ErtsMiscOpList, @@ -493,8 +330,6 @@ do { \ * Local functions. */ -static void init_processes_bif(void); -static void save_terminating_process(Process *p); static void exec_misc_ops(ErtsRunQueue *); static void print_function_from_pc(int to, void *to_arg, BeamInstr* x); static int stack_element_dump(int to, void *to_arg, Process* p, Eterm* sp, @@ -562,39 +397,17 @@ erts_smp_lc_runq_is_locked(ErtsRunQueue *runq) } #endif -static erts_interval_t *proc_interval; - -static void -proc_interval_init(void) -{ - proc_interval = erts_alloc_permanent_cache_aligned( - ERTS_ALC_T_PROC_INTERVAL, - sizeof(erts_interval_t)); - erts_smp_interval_init(proc_interval); -} - -static ERTS_INLINE Uint64 -get_proc_interval(void) -{ - return erts_smp_current_interval_nob(proc_interval); -} static ERTS_INLINE Uint64 ensure_later_proc_interval(Uint64 interval) { - return erts_smp_ensure_later_interval_nob(proc_interval, interval); -} - -static ERTS_INLINE Uint64 -step_proc_interval(void) -{ - return erts_smp_step_interval_nob(proc_interval); + return erts_smp_ensure_later_interval_nob(erts_ptab_interval(&erts_proc), interval); } Uint64 erts_get_proc_interval(void) { - return get_proc_interval(); + return erts_smp_current_interval_nob(erts_ptab_interval(&erts_proc)); } Uint64 @@ -606,7 +419,7 @@ erts_ensure_later_proc_interval(Uint64 interval) Uint64 erts_step_proc_interval(void) { - return step_proc_interval(); + return erts_smp_step_interval_nob(erts_ptab_interval(&erts_proc)); } void @@ -654,20 +467,18 @@ erts_pre_init_process(void) #endif } +#ifdef ERTS_SMP +static void +release_process(void *vproc) +{ + erts_smp_proc_dec_refc((Process *) vproc); +} +#endif + /* initialize the scheduler */ void -erts_init_process(int ncpu) +erts_init_process(int ncpu, int proc_tab_size) { - int proc_tab_sz; - int max_proc_bits; - int proc_bits = ERTS_PROC_BITS; - erts_smp_atomic_t *proc_entry; - char *proc_tab_end; - erts_smp_rwmtx_opt_t proc_tab_rwmtx_opts = ERTS_SMP_RWMTX_OPT_DEFAULT_INITER; - proc_tab_rwmtx_opts.type = ERTS_SMP_RWMTX_TYPE_EXTREMELY_FREQUENT_READ; - proc_tab_rwmtx_opts.lived = ERTS_SMP_RWMTX_LONG_LIVED; - - proc_interval_init(); #ifdef ERTS_SMP erts_disable_proc_not_running_opt = 0; @@ -676,52 +487,16 @@ erts_init_process(int ncpu) init_proclist_alloc(); - erts_smp_atomic32_init_nob(&process_count, 0); - - if (erts_use_r9_pids_ports) - proc_bits = ERTS_R9_PROC_BITS; - - if (erts_proc.max > (1 << proc_bits)) - erts_proc.max = 1 << proc_bits; - - proc_tab_sz = ERTS_ALC_CACHE_LINE_ALIGN_SIZE(erts_proc.max - * sizeof(erts_smp_atomic_t)); - erts_proc.tab = erts_alloc(ERTS_ALC_T_PROC_TABLE, proc_tab_sz); - proc_tab_end = ((char *) erts_proc.tab) + proc_tab_sz; - proc_entry = erts_proc.tab; - while (proc_tab_end > ((char *) proc_entry)) { - erts_smp_atomic_init_nob(proc_entry, ERTS_AINT_NULL); - proc_entry++; - } - - erts_smp_rwmtx_init_opt(&erts_proc_tab_rwmtx, - &proc_tab_rwmtx_opts, - "proc_tab"); - last_pid_data_init_nob(~((Uint64) 0)); - - max_proc_bits = erts_fit_in_bits_int32((Sint32) erts_proc.max - 1); - - erts_proc.tab_cache_lines = proc_tab_sz/ERTS_CACHE_LINE_SIZE; - erts_proc.pix_per_cache_line = ERTS_CACHE_LINE_SIZE/sizeof(erts_smp_atomic_t); - if ((erts_proc.max & (erts_proc.max - 1)) - | (erts_proc.pix_per_cache_line & (erts_proc.pix_per_cache_line - 1))) { - /* - * erts_proc.max or erts_proc.pix_per_cache_line - * not a power of 2 :( - */ - erts_proc.pix_cl_mask = 0; - erts_proc.pix_cl_shift = 0; - erts_proc.pix_cli_mask = 0; - erts_proc.pix_cli_shift = 0; - } - else { - ASSERT((erts_proc.tab_cache_lines - & (erts_proc.tab_cache_lines - 1)) == 0); - erts_proc.pix_cl_mask = erts_proc.tab_cache_lines-1; - erts_proc.pix_cl_shift = erts_fit_in_bits_int32(erts_proc.pix_per_cache_line-1); - erts_proc.pix_cli_shift = erts_fit_in_bits_int32(erts_proc.pix_cl_mask); - erts_proc.pix_cli_mask = (1 << (max_proc_bits - erts_proc.pix_cli_shift)) - 1; - } + erts_ptab_init_table(&erts_proc, + ERTS_ALC_T_PROC_TABLE, +#ifdef ERTS_SMP + release_process, +#else + NULL, +#endif + (ErtsPTabElementCommon *) &erts_invalid_process.common, + proc_tab_size, + "process_table"); last_reductions = 0; last_exact_reductions = 0; @@ -732,7 +507,6 @@ void erts_late_init_process(void) { int ix; - init_processes_bif(); erts_smp_spinlock_init(&erts_sched_stat.lock, "sched_stat"); for (ix = 0; ix < ERTS_NO_PRIO_LEVELS; ix++) { @@ -980,9 +754,9 @@ static ERTS_INLINE ErtsProcList * proclist_create(Process *p) { ErtsProcList *plp = proclist_alloc(); - ensure_later_proc_interval(p->started_interval); - plp->pid = p->id; - plp->started_interval = p->started_interval; + ensure_later_proc_interval(p->common.u.alive.started_interval); + plp->pid = p->common.id; + plp->started_interval = p->common.u.alive.started_interval; return plp; } @@ -992,12 +766,6 @@ proclist_destroy(ErtsProcList *plp) proclist_free(plp); } -static ERTS_INLINE int -proclist_same(ErtsProcList *plp, Process *p) -{ - return plp->pid == p->id && plp->started_interval == p->started_interval; -} - ErtsProcList * erts_proclist_create(Process *p) { @@ -1010,12 +778,6 @@ erts_proclist_destroy(ErtsProcList *plp) proclist_destroy(plp); } -int -erts_proclist_same(ErtsProcList *plp, Process *p) -{ - return proclist_same(plp, p); -} - void * erts_psd_set_init(Process *p, ErtsProcLocks plocks, int ix, void *data) { @@ -1776,37 +1538,32 @@ handle_reap_ports(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting) unset_aux_work_flags(awdp->ssi, ERTS_SSI_AUX_WORK_REAP_PORTS); awdp->esdp->run_queue->halt_in_progress = 1; if (erts_smp_atomic32_dec_read_acqb(&erts_halt_progress) == 0) { - int i; + int i, max = erts_ptab_max(&erts_port); erts_smp_atomic32_set_nob(&erts_halt_progress, 1); - for (i = 0; i < erts_max_ports; i++) { - Port *prt = &erts_port[i]; - erts_smp_port_state_lock(prt); - if ((prt->status & (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP - | ERTS_PORT_SFLG_HALT))) { - erts_smp_port_state_unlock(prt); - continue; - } - /* We need to set the halt flag - get the port lock */ -#ifdef ERTS_SMP - erts_smp_atomic_inc_nob(&prt->refc); -#endif - erts_smp_port_state_unlock(prt); -#ifdef ERTS_SMP - erts_smp_mtx_lock(prt->lock); -#endif - if ((prt->status & (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP - | ERTS_PORT_SFLG_HALT))) { - erts_port_release(prt); + for (i = 0; i < max; i++) { + erts_aint32_t state; + Port *prt = erts_pix2port(i); + if (!prt) continue; - } - erts_port_status_bor_set(prt, ERTS_PORT_SFLG_HALT); - erts_smp_atomic32_inc_nob(&erts_halt_progress); - if (prt->status & (ERTS_PORT_SFLG_EXITING - | ERTS_PORT_SFLG_CLOSING)) { - erts_port_release(prt); + state = erts_atomic32_read_acqb(&prt->state); + if (state & (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP + | ERTS_PORT_SFLG_HALT)) continue; + + /* We need to set the halt flag - get the port lock */ + + erts_smp_port_lock(prt); + + state = erts_atomic32_read_nob(&prt->state); + if (!(state & (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP + | ERTS_PORT_SFLG_HALT))) { + state = erts_atomic32_read_bor_relb(&prt->state, + ERTS_PORT_SFLG_HALT); + erts_smp_atomic32_inc_nob(&erts_halt_progress); + if (!(state & (ERTS_PORT_SFLG_EXITING|ERTS_PORT_SFLG_CLOSING))) + erts_deliver_port_exit(prt, prt->common.id, am_killed, 0); } - erts_do_exit_port(prt, prt->id, am_killed); + erts_port_release(prt); } if (erts_smp_atomic32_dec_read_nob(&erts_halt_progress) == 0) { @@ -3183,12 +2940,12 @@ resume_run_queue(ErtsRunQueue *rq) erts_smp_runq_lock(rq); - (void) ERTS_RUNQ_FLGS_MASK_SET(rq, - (ERTS_RUNQ_FLG_OUT_OF_WORK - | ERTS_RUNQ_FLG_HALFTIME_OUT_OF_WORK - | ERTS_RUNQ_FLG_SUSPENDED), - (ERTS_RUNQ_FLG_OUT_OF_WORK - | ERTS_RUNQ_FLG_HALFTIME_OUT_OF_WORK)); + (void) ERTS_RUNQ_FLGS_READ_BSET(rq, + (ERTS_RUNQ_FLG_OUT_OF_WORK + | ERTS_RUNQ_FLG_HALFTIME_OUT_OF_WORK + | ERTS_RUNQ_FLG_SUSPENDED), + (ERTS_RUNQ_FLG_OUT_OF_WORK + | ERTS_RUNQ_FLG_HALFTIME_OUT_OF_WORK)); rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; for (pix = 0; pix < ERTS_NO_PROC_PRIO_LEVELS; pix++) { @@ -4210,7 +3967,7 @@ erts_fprintf(stderr, "--------------------------------\n"); ERTS_DBG_CHK_FULL_REDS_HISTORY(rq); rq->out_of_work_count = 0; - (void) ERTS_RUNQ_FLGS_MASK_SET(rq, ERTS_RUNQ_FLGS_MIGRATION_INFO, flags); + (void) ERTS_RUNQ_FLGS_READ_BSET(rq, ERTS_RUNQ_FLGS_MIGRATION_INFO, flags); rq->max_len = rq->len; for (pix = 0; pix < ERTS_NO_PRIO_LEVELS; pix++) { @@ -4896,6 +4653,11 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online) erts_smp_atomic32_init_relb(&erts_halt_progress, -1); erts_halt_code = 0; + +#if !defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) + erts_lc_set_thread_name("scheduler 1"); +#endif + } ErtsRunQueue * @@ -5573,8 +5335,7 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) else if (on) { /* ------ BLOCK ------ */ if (schdlr_sspnd.msb.procs) { plp = proclist_create(p); - plp->next = schdlr_sspnd.msb.procs; - schdlr_sspnd.msb.procs = plp; + erts_proclist_store_last(&schdlr_sspnd.msb.procs, plp); p->flags |= F_HAVE_BLCKD_MSCHED; ASSERT(erts_smp_atomic32_read_nob(&schdlr_sspnd.active) == 1); ASSERT(p->scheduler_data->no == 1); @@ -5659,8 +5420,7 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) ~ERTS_SCHDLR_SSPND_CHNG_WAITER); } plp = proclist_create(p); - plp->next = schdlr_sspnd.msb.procs; - schdlr_sspnd.msb.procs = plp; + erts_proclist_store_last(&schdlr_sspnd.msb.procs, plp); ASSERT(p->scheduler_data); } } @@ -5671,20 +5431,16 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) } else { /* ------ UNBLOCK ------ */ if (p->flags & F_HAVE_BLCKD_MSCHED) { - ErtsProcList **plpp = &schdlr_sspnd.msb.procs; - plp = schdlr_sspnd.msb.procs; + ErtsProcList *plp = erts_proclist_peek_first(schdlr_sspnd.msb.procs); while (plp) { - if (!proclist_same(plp, p)){ - plpp = &plp->next; - plp = plp->next; - } - else { - *plpp = plp->next; - proclist_destroy(plp); + ErtsProcList *tmp_plp = plp; + plp = erts_proclist_peek_next(schdlr_sspnd.msb.procs, plp); + if (erts_proclist_same(tmp_plp, p)) { + erts_proclist_remove(&schdlr_sspnd.msb.procs, tmp_plp); + proclist_destroy(tmp_plp); if (!all) break; - plp = *plpp; } } } @@ -5753,23 +5509,25 @@ erts_multi_scheduling_blockers(Process *p) Eterm res = NIL; erts_smp_mtx_lock(&schdlr_sspnd.mtx); - if (schdlr_sspnd.msb.procs) { + if (!erts_proclist_is_empty(schdlr_sspnd.msb.procs)) { Eterm *hp, *hp_end; ErtsProcList *plp1, *plp2; - Uint max_size; - ASSERT(schdlr_sspnd.msb.procs); - for (max_size = 0, plp1 = schdlr_sspnd.msb.procs; + Uint max_size = 0; + + for (plp1 = erts_proclist_peek_first(schdlr_sspnd.msb.procs); plp1; - plp1 = plp1->next) { + plp1 = erts_proclist_peek_next(schdlr_sspnd.msb.procs, plp1)) { max_size += 2; } ASSERT(max_size); hp = HAlloc(p, max_size); hp_end = hp + max_size; - for (plp1 = schdlr_sspnd.msb.procs; plp1; plp1 = plp1->next) { - for (plp2 = schdlr_sspnd.msb.procs; + for (plp1 = erts_proclist_peek_first(schdlr_sspnd.msb.procs); + plp1; + plp1 = erts_proclist_peek_next(schdlr_sspnd.msb.procs, plp1)) { + for (plp2 = erts_proclist_peek_first(schdlr_sspnd.msb.procs); plp2->pid != plp1->pid; - plp2 = plp2->next); + plp2 = erts_proclist_peek_next(schdlr_sspnd.msb.procs, plp2)); if (plp2 == plp1) { res = CONS(hp, plp1->pid, res); hp += 2; @@ -6021,7 +5779,7 @@ handle_pend_sync_suspend(Process *suspendee, ASSERT(is_nil(suspender->suspendee)); if (suspendee_alive) { erts_suspend(suspendee, suspendee_locks, NULL); - suspender->suspendee = suspendee->id; + suspender->suspendee = suspendee->common.id; } /* suspender is suspended waiting for suspendee to suspend; resume suspender */ @@ -6042,7 +5800,7 @@ pid2proc_not_running(Process *c_p, ErtsProcLocks c_p_locks, ERTS_SMP_LC_ASSERT(c_p_locks & ERTS_PROC_LOCK_MAIN); ERTS_SMP_LC_ASSERT(pid_locks & (ERTS_PROC_LOCK_MAIN|ERTS_PROC_LOCK_STATUS)); - if (c_p->id == pid) + if (c_p->common.id == pid) return erts_pid2proc(c_p, c_p_locks, pid, pid_locks); if (c_p_locks & ERTS_PROC_LOCK_STATUS) @@ -6094,7 +5852,7 @@ pid2proc_not_running(Process *c_p, ErtsProcLocks c_p_locks, */ if (!c_p->pending_suspenders) { /* Mark rp pending for suspend by c_p */ - add_pend_suspend(rp, c_p->id, handle_pend_sync_suspend); + add_pend_suspend(rp, c_p->common.id, handle_pend_sync_suspend); ASSERT(is_nil(c_p->suspendee)); /* Suspend c_p; when rp is suspended c_p will be resumed. */ @@ -6200,20 +5958,20 @@ handle_pend_bif_sync_suspend(Process *suspendee, ASSERT(is_nil(suspender->suspendee)); if (!suspendee_alive) erts_delete_suspend_monitor(&suspender->suspend_monitors, - suspendee->id); + suspendee->common.id); else { #ifdef DEBUG int res; #endif ErtsSuspendMonitor *smon; smon = erts_lookup_suspend_monitor(suspender->suspend_monitors, - suspendee->id); + suspendee->common.id); #ifdef DEBUG res = #endif do_bif_suspend_process(suspendee, smon, suspendee); ASSERT(!smon || res != 0); - suspender->suspendee = suspendee->id; + suspender->suspendee = suspendee->common.id; } /* suspender is suspended waiting for suspendee to suspend; resume suspender */ @@ -6242,14 +6000,14 @@ handle_pend_bif_async_suspend(Process *suspendee, ASSERT(is_nil(suspender->suspendee)); if (!suspendee_alive) erts_delete_suspend_monitor(&suspender->suspend_monitors, - suspendee->id); + suspendee->common.id); else { #ifdef DEBUG int res; #endif ErtsSuspendMonitor *smon; smon = erts_lookup_suspend_monitor(suspender->suspend_monitors, - suspendee->id); + suspendee->common.id); #ifdef DEBUG res = #endif @@ -6294,7 +6052,7 @@ suspend_process_2(BIF_ALIST_2) int unless_suspending = 0; - if (BIF_P->id == BIF_ARG_1) + if (BIF_P->common.id == BIF_ARG_1) goto badarg; /* We are not allowed to suspend ourselves */ if (is_not_nil(BIF_ARG_2)) { @@ -6390,7 +6148,7 @@ suspend_process_2(BIF_ALIST_2) if (!do_bif_suspend_process(BIF_P, smon, suspendee)) add_pend_suspend(suspendee, - BIF_P->id, + BIF_P->common.id, handle_pend_bif_async_suspend); res = am_true; @@ -6453,7 +6211,7 @@ suspend_process_2(BIF_ALIST_2) else { /* Mark suspendee pending for suspend by BIF_P */ add_pend_suspend(suspendee, - BIF_P->id, + BIF_P->common.id, handle_pend_bif_sync_suspend); ASSERT(is_nil(BIF_P->suspendee)); @@ -6527,7 +6285,7 @@ resume_process_1(BIF_ALIST_1) Process *suspendee; int is_active; - if (BIF_P->id == BIF_ARG_1) + if (BIF_P->common.id == BIF_ARG_1) BIF_ERROR(BIF_P, BADARG); erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK); @@ -6652,7 +6410,8 @@ erts_process_status(Process *c_p, ErtsProcLocks c_p_locks, for (i = 0; i < erts_no_schedulers; i++) { esdp = ERTS_SCHEDULER_IX(i); erts_smp_runq_lock(esdp->run_queue); - if (esdp->free_process && esdp->free_process->id == rpid) { + if (esdp->free_process + && esdp->free_process->common.id == rpid) { res = am_free; erts_smp_runq_unlock(esdp->run_queue); break; @@ -6673,27 +6432,31 @@ erts_process_status(Process *c_p, ErtsProcLocks c_p_locks, void erts_suspend(Process* c_p, ErtsProcLocks c_p_locks, Port *busy_port) { -#ifdef DEBUG - int res; -#endif + int suspend; + ASSERT(c_p == erts_get_current_process()); ERTS_SMP_LC_ASSERT(c_p_locks == erts_proc_lc_my_proc_locks(c_p)); if (!(c_p_locks & ERTS_PROC_LOCK_STATUS)) erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_STATUS); + if (busy_port) + suspend = erts_save_suspend_process_on_port(busy_port, c_p); + else + suspend = 1; + + if (suspend) { #ifdef DEBUG - res = + int res = #endif - suspend_process(c_p, c_p); - - ASSERT(res); - - if (busy_port) - erts_wake_process_later(busy_port, c_p); + suspend_process(c_p, c_p); + ASSERT(res); + } if (!(c_p_locks & ERTS_PROC_LOCK_STATUS)) erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_STATUS); + if (suspend && busy_port && erts_system_monitor_flags.busy_port) + monitor_generic(c_p, am_busy_port, busy_port->common.id); } void @@ -6708,16 +6471,19 @@ erts_resume(Process* process, ErtsProcLocks process_locks) } int -erts_resume_processes(ErtsProcList *plp) +erts_resume_processes(ErtsProcList *list) { + /* 'list' is expected to have been fetched (i.e. not a ring anymore) */ int nresumed = 0; + ErtsProcList *plp = list; + while (plp) { Process *proc; ErtsProcList *fplp; ASSERT(is_internal_pid(plp->pid)); proc = erts_pid2proc(NULL, 0, plp->pid, ERTS_PROC_LOCK_STATUS); if (proc) { - if (proclist_same(plp, proc)) { + if (erts_proclist_same(plp, proc)) { resume_process(proc); nresumed++; } @@ -6870,9 +6636,8 @@ Process *schedule(Process *p, int calls) state = erts_smp_atomic32_read_acqb(&p->state); if (IS_TRACED(p)) { - if (IS_TRACED_FL(p, F_TRACE_CALLS) && !(state & ERTS_PSFLG_FREE)) { + if (IS_TRACED_FL(p, F_TRACE_CALLS) && !(state & ERTS_PSFLG_FREE)) erts_schedule_time_break(p, ERTS_BP_CALL_TIME_SCHEDULE_OUT); - } if (state & (ERTS_PSFLG_FREE|ERTS_PSFLG_EXITING)) { if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_EXIT)) trace_sched(p, ((state & ERTS_PSFLG_FREE) @@ -6885,7 +6650,7 @@ Process *schedule(Process *p, int calls) else if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_PROCS)) trace_virtual_sched(p, am_out); } - } + } #ifdef ERTS_SMP if (state & ERTS_PSFLG_PENDING_EXIT) @@ -6949,9 +6714,8 @@ Process *schedule(Process *p, int calls) #ifdef ERTS_SMP { ErtsProcList *pnd_xtrs = rq->procs.pending_exiters; - rq->procs.pending_exiters = NULL; - - if (pnd_xtrs) { + if (erts_proclist_fetch(&pnd_xtrs, NULL)) { + rq->procs.pending_exiters = NULL; erts_smp_runq_unlock(rq); handle_pending_exiters(pnd_xtrs); erts_smp_runq_lock(rq); @@ -7247,7 +7011,7 @@ Process *schedule(Process *p, int calls) erts_smp_proc_unlock(p, ERTS_PROC_LOCK_STATUS); #ifdef ERTS_SMP - if (is_not_nil(p->tracer_proc)) + if (is_not_nil(ERTS_TRACER_PROC(p))) erts_check_my_tracer_proc(p); #endif @@ -7450,70 +7214,6 @@ erts_get_exact_total_reductions(Process *c_p, Uint *redsp, Uint *diffp) erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_MAIN); } -/* - * erts_test_next_pid() is only used for testing. - */ -Sint -erts_test_next_pid(int set, Uint next) -{ - Uint64 lpd; - Sint res; - Eterm pid_data; - int first_pix = -1; - - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - - if (!set) - lpd = last_pid_data_read_nob(); - else { - - lpd = (Uint64) next; - pid_data = (Eterm) (lpd & ERTS_PID_DATA_MASK__); - if (ERTS_INVALID_PID == make_internal_pid(pid_data)) { - lpd += erts_proc.max; - ASSERT(erts_pid_data2ix(pid_data) - == erts_pid_data2ix(lpd & ERTS_PID_DATA_MASK__)); - } - last_pid_data_set_relb(lpd); - } - - while (1) { - int pix; - lpd++; - pix = (int) (lpd % erts_proc.max); - if (first_pix < 0) - first_pix = pix; - else if (pix == first_pix) { - res = -1; - break; - } - if (ERTS_AINT_NULL == erts_smp_atomic_read_nob(&erts_proc.tab[pix])) { - pid_data = (Eterm) (lpd & ERTS_PID_DATA_MASK__); - if (ERTS_INVALID_PID == make_internal_pid(pid_data)) { - lpd += erts_proc.max; - ASSERT(erts_pid_data2ix(pid_data) - == erts_pid_data2ix(lpd & ERTS_PID_DATA_MASK__)); - } - res = lpd & ERTS_PID_DATA_MASK__; - break; - } - } - - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - - return res; - -} - -Uint erts_process_count(void) -{ - erts_aint32_t res = erts_smp_atomic32_read_nob(&process_count); - if (res > erts_proc.max) - return erts_proc.max; - ASSERT(res >= 0); - return (Uint) res; -} - void erts_free_proc(Process *p) { @@ -7523,132 +7223,66 @@ erts_free_proc(Process *p) erts_free(ERTS_ALC_T_PROC, (void *) p); } +typedef struct { + Process *proc; + erts_aint32_t state; + ErtsRunQueue *run_queue; +} ErtsEarlyProcInit; + +static void early_init_process_struct(void *varg, Eterm data) +{ + ErtsEarlyProcInit *arg = (ErtsEarlyProcInit *) varg; + Process *proc = arg->proc; + + proc->common.id = make_internal_pid(data); + erts_smp_atomic32_init_relb(&proc->state, arg->state); + +#ifdef ERTS_SMP + RUNQ_SET_RQ(&proc->run_queue, arg->run_queue); + + erts_proc_lock_init(proc); /* All locks locked */ +#endif + +} + /* ** Allocate process and find out where to place next process. */ static Process* alloc_process(ErtsRunQueue *rq, erts_aint32_t state) { - int pix; - Process* p; - Uint64 lpd, exp_lpd; - Eterm pid_data; - erts_aint32_t proc_count; -#ifdef DEBUG - Eterm pid; -#endif - - erts_smp_rwmtx_rlock(&erts_proc_tab_rwmtx); - - proc_count = erts_smp_atomic32_inc_read_acqb(&process_count); - if (proc_count > erts_proc.max) { - while (1) { - erts_aint32_t act_proc_count; - - act_proc_count = erts_smp_atomic32_cmpxchg_relb(&process_count, - proc_count-1, - proc_count); - if (act_proc_count == proc_count) - goto system_limit; - proc_count = act_proc_count; - if (proc_count <= erts_proc.max) - break; - } - } + ErtsEarlyProcInit init_arg; + Process *p; - p = (Process*) erts_alloc_fnf(ERTS_ALC_T_PROC, sizeof(Process)); + p = erts_alloc_fnf(ERTS_ALC_T_PROC, sizeof(Process)); if (!p) - goto enomem; - - p->approx_started = erts_get_approx_time(); - p->started_interval = get_proc_interval(); - - lpd = last_pid_data_read_acqb(); - - /* Reserve slot */ - while (1) { - lpd++; - pix = erts_pid_data2ix((Eterm) (lpd & ERTS_PID_DATA_MASK__)); - if (erts_smp_atomic_read_nob(&erts_proc.tab[pix]) == ERTS_AINT_NULL) { - erts_aint_t val; - val = erts_smp_atomic_cmpxchg_relb(&erts_proc.tab[pix], - ((erts_aint_t) - ERTS_PROC_LOCK_BUSY), - ERTS_AINT_NULL); - - if (ERTS_AINT_NULL == val) - break; - } - } - - pid_data = (Eterm) lpd & ERTS_PID_DATA_MASK__; + return NULL; - p->id = make_internal_pid(pid_data); - if (p->id == ERTS_INVALID_PID) { - /* Do not use the invalid pid; change serial */ - lpd += erts_proc.max; - ASSERT(pix == erts_pid_data2ix((Eterm) (lpd & ERTS_PID_DATA_MASK__))); - pid_data = (Eterm) lpd & ERTS_PID_DATA_MASK__; - p->id = make_internal_pid(pid_data); - ASSERT(p->id != ERTS_INVALID_PID); - } + init_arg.proc = (Process *) p; + init_arg.run_queue = rq; + init_arg.state = state; - exp_lpd = last_pid_data_read_nob(); + ASSERT(((char *) p) == ((char *) &p->common)); - /* Move last pid data forward */ - while (1) { - Uint64 act_lpd; - if (last_pid_data_cmp(lpd, exp_lpd) < 0) - break; - act_lpd = last_pid_data_cmpxchg_relb(lpd, exp_lpd); - if (act_lpd == exp_lpd) - break; - exp_lpd = act_lpd; + if (!erts_ptab_new_element(&erts_proc, + &p->common, + (void *) &init_arg, + early_init_process_struct)) { + erts_free(ERTS_ALC_T_PROC, p); + return NULL; } -#ifdef ERTS_SMP - RUNQ_SET_RQ(&p->run_queue, rq); -#endif - - erts_smp_atomic32_init_relb(&p->state, state); - -#ifdef DEBUG - pid = p->id; -#endif - -#ifdef ERTS_SMP - erts_proc_lock_init(p); /* All locks locked */ -#endif - - /* Move into slot reserved */ -#ifdef DEBUG - ASSERT(ERTS_PROC_LOCK_BUSY - == (Process *) erts_smp_atomic_xchg_relb(&erts_proc.tab[pix], - (erts_aint_t) p)); -#else - erts_smp_atomic_set_relb(&erts_proc.tab[pix], (erts_aint_t) p); -#endif - - ASSERT(internal_pid_serial(p->id) <= (erts_use_r9_pids_ports - ? ERTS_MAX_PID_R9_SERIAL - : ERTS_MAX_PID_SERIAL)); - - erts_smp_rwmtx_runlock(&erts_proc_tab_rwmtx); - + ASSERT(internal_pid_serial(p->common.id) <= ERTS_MAX_PID_SERIAL); + + p->approx_started = erts_get_approx_time(); p->rcount = 0; - ASSERT(p == (Process *) - erts_smp_atomic_read_nob( - &erts_proc.tab[internal_pid_index(pid)])); - - return p; -enomem: -system_limit: - - erts_smp_rwmtx_runlock(&erts_proc_tab_rwmtx); - return NULL; + ASSERT(p == (Process *) (erts_ptab_pix2intptr_nob( + &erts_proc, + internal_pid_index(p->common.id)))); + return p; } Eterm @@ -7795,21 +7429,21 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). p->reds = 0; #ifdef ERTS_SMP - p->u.alive.ptimer = NULL; + p->common.u.alive.ptimer = NULL; #else - sys_memset(&p->u.alive.tm, 0, sizeof(ErlTimer)); + sys_memset(&p->common.u.alive.tm, 0, sizeof(ErlTimer)); #endif - p->reg = NULL; - p->nlinks = NULL; - p->monitors = NULL; + p->common.u.alive.reg = NULL; + ERTS_P_LINKS(p) = NULL; + ERTS_P_MONITORS(p) = NULL; p->nodes_monitors = NULL; p->suspend_monitors = NULL; ASSERT(is_pid(parent->group_leader)); if (parent->group_leader == ERTS_INVALID_PID) - p->group_leader = p->id; + p->group_leader = p->common.id; else { /* Needs to be done after the heap has been set up */ p->group_leader = @@ -7818,18 +7452,18 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). : STORE_NC(&p->htop, &p->off_heap, parent->group_leader); } - erts_get_default_tracing(&p->trace_flags, &p->tracer_proc); + erts_get_default_tracing(&ERTS_TRACE_FLAGS(p), &ERTS_TRACER_PROC(p)); p->msg.first = NULL; p->msg.last = &p->msg.first; p->msg.save = &p->msg.first; p->msg.len = 0; #ifdef ERTS_SMP - p->u.alive.msg_inq.first = NULL; - p->u.alive.msg_inq.last = &p->u.alive.msg_inq.first; - p->u.alive.msg_inq.len = 0; + p->msg_inq.first = NULL; + p->msg_inq.last = &p->msg_inq.first; + p->msg_inq.len = 0; #endif - p->bif_timers = NULL; + p->u.bif_timers = NULL; p->mbuf = NULL; p->mbuf_sz = 0; p->psd = NULL; @@ -7841,7 +7475,9 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). DT_UTAG(p) = NIL; DT_UTAG_FLAGS(p) = 0; #endif - p->parent = parent->id == ERTS_INVALID_PID ? NIL : parent->id; + p->parent = (parent->common.id == ERTS_INVALID_PID + ? NIL + : parent->common.id); INIT_HOLE_CHECK(p); #ifdef DEBUG @@ -7849,18 +7485,19 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). #endif if (IS_TRACED(parent)) { - if (parent->trace_flags & F_TRACE_SOS) { - p->trace_flags |= (parent->trace_flags & TRACEE_FLAGS); - p->tracer_proc = parent->tracer_proc; + if (ERTS_TRACE_FLAGS(parent) & F_TRACE_SOS) { + ERTS_TRACE_FLAGS(p) |= (ERTS_TRACE_FLAGS(parent) & TRACEE_FLAGS); + ERTS_TRACER_PROC(p) = ERTS_TRACER_PROC(parent); } if (ARE_TRACE_FLAGS_ON(parent, F_TRACE_PROCS)) { - trace_proc_spawn(parent, p->id, mod, func, args); + trace_proc_spawn(parent, p->common.id, mod, func, args); } - if (parent->trace_flags & F_TRACE_SOS1) { /* Overrides TRACE_CHILDREN */ - p->trace_flags |= (parent->trace_flags & TRACEE_FLAGS); - p->tracer_proc = parent->tracer_proc; - p->trace_flags &= ~(F_TRACE_SOS1 | F_TRACE_SOS); - parent->trace_flags &= ~(F_TRACE_SOS1 | F_TRACE_SOS); + if (ERTS_TRACE_FLAGS(parent) & F_TRACE_SOS1) { + /* Overrides TRACE_CHILDREN */ + ERTS_TRACE_FLAGS(p) |= (ERTS_TRACE_FLAGS(parent) & TRACEE_FLAGS); + ERTS_TRACER_PROC(p) = ERTS_TRACER_PROC(parent); + ERTS_TRACE_FLAGS(p) &= ~(F_TRACE_SOS1 | F_TRACE_SOS); + ERTS_TRACE_FLAGS(parent) &= ~(F_TRACE_SOS1 | F_TRACE_SOS); } } @@ -7873,27 +7510,27 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). int ret; #endif if (IS_TRACED_FL(parent, F_TRACE_PROCS)) { - trace_proc(parent, parent, am_link, p->id); + trace_proc(parent, parent, am_link, p->common.id); } #ifdef DEBUG - ret = erts_add_link(&(parent->nlinks), LINK_PID, p->id); + ret = erts_add_link(&ERTS_P_LINKS(parent), LINK_PID, p->common.id); ASSERT(ret == 0); - ret = erts_add_link(&(p->nlinks), LINK_PID, parent->id); + ret = erts_add_link(&ERTS_P_LINKS(p), LINK_PID, parent->common.id); ASSERT(ret == 0); #else - erts_add_link(&(parent->nlinks), LINK_PID, p->id); - erts_add_link(&(p->nlinks), LINK_PID, parent->id); + erts_add_link(&ERTS_P_LINKS(parent), LINK_PID, p->common.id); + erts_add_link(&ERTS_P_LINKS(p), LINK_PID, parent->common.id); #endif if (IS_TRACED(parent)) { - if (parent->trace_flags & (F_TRACE_SOL|F_TRACE_SOL1)) { - p->trace_flags |= (parent->trace_flags & TRACEE_FLAGS); - p->tracer_proc = parent->tracer_proc; /* maybe steal */ + if (ERTS_TRACE_FLAGS(parent) & (F_TRACE_SOL|F_TRACE_SOL1)) { + ERTS_TRACE_FLAGS(p) |= (ERTS_TRACE_FLAGS(parent)&TRACEE_FLAGS); + ERTS_TRACER_PROC(p) = ERTS_TRACER_PROC(parent); /*maybe steal*/ - if (parent->trace_flags & F_TRACE_SOL1) { /* maybe override */ - p ->trace_flags &= ~(F_TRACE_SOL1 | F_TRACE_SOL); - parent->trace_flags &= ~(F_TRACE_SOL1 | F_TRACE_SOL); + if (ERTS_TRACE_FLAGS(parent) & F_TRACE_SOL1) {/*maybe override*/ + ERTS_TRACE_FLAGS(p) &= ~(F_TRACE_SOL1 | F_TRACE_SOL); + ERTS_TRACE_FLAGS(parent) &= ~(F_TRACE_SOL1 | F_TRACE_SOL); } } } @@ -7906,8 +7543,8 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). Eterm mref; mref = erts_make_ref(parent); - erts_add_monitor(&(parent->monitors), MON_ORIGIN, mref, p->id, NIL); - erts_add_monitor(&(p->monitors), MON_TARGET, mref, parent->id, NIL); + erts_add_monitor(&ERTS_P_MONITORS(parent), MON_ORIGIN, mref, p->common.id, NIL); + erts_add_monitor(&ERTS_P_MONITORS(p), MON_TARGET, mref, parent->common.id, NIL); so->mref = mref; } @@ -7915,8 +7552,8 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). p->scheduler_data = NULL; p->suspendee = NIL; p->pending_suspenders = NULL; - p->u.alive.pending_exit.reason = THE_NON_VALUE; - p->u.alive.pending_exit.bp = NULL; + p->pending_exit.reason = THE_NON_VALUE; + p->pending_exit.bp = NULL; #endif #if !defined(NO_FPE_SIGNALS) || defined(HIPE) @@ -7925,7 +7562,7 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL); - res = p->id; + res = p->common.id; /* * Schedule process for execution. @@ -7933,7 +7570,7 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). schedule_process(p, state, 0); - VERBOSE(DEBUG_PROCESSES, ("Created a new process: %T\n",p->id)); + VERBOSE(DEBUG_PROCESSES, ("Created a new process: %T\n",p->common.id)); #ifdef USE_VM_PROBES if (DTRACE_ENABLED(process_spawn)) { @@ -7968,10 +7605,10 @@ void erts_init_empty_process(Process *p) p->min_heap_size = 0; p->min_vheap_size = 0; p->rcount = 0; - p->id = ERTS_INVALID_PID; + p->common.id = ERTS_INVALID_PID; p->reds = 0; - p->tracer_proc = NIL; - p->trace_flags = F_INITIAL_TRACE_FLAGS; + ERTS_TRACER_PROC(p) = NIL; + ERTS_TRACE_FLAGS(p) = F_INITIAL_TRACE_FLAGS; p->group_leader = ERTS_INVALID_PID; p->flags = 0; p->fvalue = NIL; @@ -7984,14 +7621,14 @@ void erts_init_empty_process(Process *p) p->bin_old_vheap = 0; p->bin_vheap_mature = 0; #ifdef ERTS_SMP - p->u.alive.ptimer = NULL; + p->common.u.alive.ptimer = NULL; #else - memset(&(p->u.alive.tm), 0, sizeof(ErlTimer)); + memset(&(p->common.u.alive.tm), 0, sizeof(ErlTimer)); #endif p->next = NULL; p->off_heap.first = NULL; p->off_heap.overhead = 0; - p->reg = NULL; + p->common.u.alive.reg = NULL; p->heap_sz = 0; p->high_water = NULL; p->old_hend = NULL; @@ -8000,15 +7637,15 @@ void erts_init_empty_process(Process *p) p->mbuf = NULL; p->mbuf_sz = 0; p->psd = NULL; - p->monitors = NULL; - p->nlinks = NULL; /* List of links */ + ERTS_P_MONITORS(p) = NULL; + ERTS_P_LINKS(p) = NULL; /* List of links */ p->nodes_monitors = NULL; p->suspend_monitors = NULL; p->msg.first = NULL; p->msg.last = &p->msg.first; p->msg.save = &p->msg.first; p->msg.len = 0; - p->bif_timers = NULL; + p->u.bif_timers = NULL; p->dictionary = NULL; p->seq_trace_clock = 0; p->seq_trace_lastcnt = 0; @@ -8036,7 +7673,7 @@ void erts_init_empty_process(Process *p) p->parent = NIL; p->approx_started = 0; - p->started_interval = 0; + p->common.u.alive.started_interval = 0; #ifdef HIPE hipe_init_process(&p->hipe); @@ -8054,13 +7691,13 @@ void erts_init_empty_process(Process *p) #ifdef ERTS_SMP p->scheduler_data = NULL; - p->u.alive.msg_inq.first = NULL; - p->u.alive.msg_inq.last = &p->u.alive.msg_inq.first; - p->u.alive.msg_inq.len = 0; + p->msg_inq.first = NULL; + p->msg_inq.last = &p->msg_inq.first; + p->msg_inq.len = 0; p->suspendee = NIL; p->pending_suspenders = NULL; - p->u.alive.pending_exit.reason = THE_NON_VALUE; - p->u.alive.pending_exit.bp = NULL; + p->pending_exit.reason = THE_NON_VALUE; + p->pending_exit.bp = NULL; erts_proc_lock_init(p); erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL); RUNQ_SET_RQ(&p->run_queue, ERTS_RUNQ_IX(0)); @@ -8082,25 +7719,25 @@ erts_debug_verify_clean_empty_process(Process* p) ASSERT(p->stop == NULL); ASSERT(p->hend == NULL); ASSERT(p->heap == NULL); - ASSERT(p->id == ERTS_INVALID_PID); - ASSERT(p->tracer_proc == NIL); - ASSERT(p->trace_flags == F_INITIAL_TRACE_FLAGS); + ASSERT(p->common.id == ERTS_INVALID_PID); + ASSERT(ERTS_TRACER_PROC(p) == NIL); + ASSERT(ERTS_TRACE_FLAGS(p) == F_INITIAL_TRACE_FLAGS); ASSERT(p->group_leader == ERTS_INVALID_PID); ASSERT(p->next == NULL); - ASSERT(p->reg == NULL); + ASSERT(p->common.u.alive.reg == NULL); ASSERT(p->heap_sz == 0); ASSERT(p->high_water == NULL); ASSERT(p->old_hend == NULL); ASSERT(p->old_htop == NULL); ASSERT(p->old_heap == NULL); - ASSERT(p->monitors == NULL); - ASSERT(p->nlinks == NULL); + ASSERT(ERTS_P_MONITORS(p) == NULL); + ASSERT(ERTS_P_LINKS(p) == NULL); ASSERT(p->nodes_monitors == NULL); ASSERT(p->suspend_monitors == NULL); ASSERT(p->msg.first == NULL); ASSERT(p->msg.len == 0); - ASSERT(p->bif_timers == NULL); + ASSERT(p->u.bif_timers == NULL); ASSERT(p->dictionary == NULL); ASSERT(p->catches == 0); ASSERT(p->cp == NULL); @@ -8110,12 +7747,12 @@ erts_debug_verify_clean_empty_process(Process* p) ASSERT(p->parent == NIL); #ifdef ERTS_SMP - ASSERT(p->u.alive.msg_inq.first == NULL); - ASSERT(p->u.alive.msg_inq.len == 0); + ASSERT(p->msg_inq.first == NULL); + ASSERT(p->msg_inq.len == 0); ASSERT(p->suspendee == NIL); ASSERT(p->pending_suspenders == NULL); - ASSERT(p->u.alive.pending_exit.reason == THE_NON_VALUE); - ASSERT(p->u.alive.pending_exit.bp == NULL); + ASSERT(p->pending_exit.reason == THE_NON_VALUE); + ASSERT(p->pending_exit.bp == NULL); #endif /* Thing that erts_cleanup_empty_process() cleans up */ @@ -8157,7 +7794,7 @@ delete_process(Process* p) { ErlMessage* mp; - VERBOSE(DEBUG_PROCESSES, ("Removing process: %T\n",p->id)); + VERBOSE(DEBUG_PROCESSES, ("Removing process: %T\n",p->common.id)); /* Cleanup psd */ @@ -8231,8 +7868,6 @@ delete_process(Process* p) mp = next_mp; } - ASSERT(!p->monitors); - ASSERT(!p->nlinks); ASSERT(!p->nodes_monitors); ASSERT(!p->suspend_monitors); @@ -8296,7 +7931,7 @@ void erts_handle_pending_exit(Process *c_p, ErtsProcLocks locks) { ErtsProcLocks xlocks; - ASSERT(is_value(c_p->u.alive.pending_exit.reason)); + ASSERT(is_value(c_p->pending_exit.reason)); ERTS_SMP_LC_ASSERT(erts_proc_lc_my_proc_locks(c_p) == locks); ERTS_SMP_LC_ASSERT(locks & ERTS_PROC_LOCK_MAIN); ERTS_SMP_LC_ASSERT(!((ERTS_PSFLG_EXITING|ERTS_PSFLG_FREE) @@ -8315,10 +7950,10 @@ erts_handle_pending_exit(Process *c_p, ErtsProcLocks locks) set_proc_exiting(c_p, erts_smp_atomic32_read_acqb(&c_p->state), - c_p->u.alive.pending_exit.reason, - c_p->u.alive.pending_exit.bp); - c_p->u.alive.pending_exit.reason = THE_NON_VALUE; - c_p->u.alive.pending_exit.bp = NULL; + c_p->pending_exit.reason, + c_p->pending_exit.bp); + c_p->pending_exit.reason = THE_NON_VALUE; + c_p->pending_exit.bp = NULL; if (xlocks) erts_smp_proc_unlock(c_p, xlocks); @@ -8327,12 +7962,14 @@ erts_handle_pending_exit(Process *c_p, ErtsProcLocks locks) static void handle_pending_exiters(ErtsProcList *pnd_xtrs) { + /* 'list' is expected to have been fetched (i.e. not a ring anymore) */ ErtsProcList *plp = pnd_xtrs; - ErtsProcList *free_plp; + while (plp) { + ErtsProcList *free_plp; Process *p = erts_pid2proc(NULL, 0, plp->pid, ERTS_PROC_LOCKS_ALL); if (p) { - if (proclist_same(plp, p)) { + if (erts_proclist_same(plp, p)) { erts_aint32_t state = erts_smp_atomic32_read_acqb(&p->state); if (!(state & ERTS_PSFLG_RUNNING)) { ASSERT(state & ERTS_PSFLG_PENDING_EXIT); @@ -8361,8 +7998,7 @@ save_pending_exiter(Process *p) erts_smp_runq_lock(rq); - plp->next = rq->procs.pending_exiters; - rq->procs.pending_exiters = plp; + erts_proclist_store_last(&rq->procs.pending_exiters, plp); erts_smp_runq_unlock(rq); wake_scheduler(rq, 1); @@ -8409,7 +8045,7 @@ send_exit_message(Process *to, ErtsProcLocks *to_locksp, hp = bp->mem; mess = copy_struct(exit_term, term_size, &hp, &bp->off_heap); /* the trace token must in this case be updated by the caller */ - seq_trace_output(token, mess, SEQ_TRACE_SEND, to->id, NULL); + seq_trace_output(token, mess, SEQ_TRACE_SEND, to->common.id, NULL); temp_token = copy_struct(token, sz_token, &hp, &bp->off_heap); erts_queue_message(to, to_locksp, bp, mess, temp_token #ifdef USE_VM_PROBES @@ -8539,7 +8175,7 @@ send_exit_signal(Process *c_p, /* current process if and only else if (reason != am_normal || (flags & ERTS_XSIG_FLG_NO_IGN_NORMAL)) { #ifdef ERTS_SMP if (!(state & (ERTS_PSFLG_EXITING|ERTS_PSFLG_PENDING_EXIT))) { - ASSERT(!rp->u.alive.pending_exit.bp); + ASSERT(!rp->pending_exit.bp); if (rp == c_p && (*rp_locks & ERTS_PROC_LOCK_MAIN)) { /* Ensure that all locks on c_p are locked before @@ -8589,7 +8225,7 @@ send_exit_signal(Process *c_p, /* current process if and only set_pending_exit: if (is_immed(rsn)) { - rp->u.alive.pending_exit.reason = rsn; + rp->pending_exit.reason = rsn; } else { Eterm *hp; @@ -8597,11 +8233,11 @@ send_exit_signal(Process *c_p, /* current process if and only ErlHeapFragment *bp = new_message_buffer(sz); hp = &bp->mem[0]; - rp->u.alive.pending_exit.reason = copy_struct(rsn, - sz, - &hp, - &bp->off_heap); - rp->u.alive.pending_exit.bp = bp; + rp->pending_exit.reason = copy_struct(rsn, + sz, + &hp, + &bp->off_heap); + rp->pending_exit.bp = bp; } erts_smp_atomic32_read_bor_relb(&rp->state, ERTS_PSFLG_PENDING_EXIT); @@ -8700,7 +8336,7 @@ static void doit_exit_monitor(ErtsMonitor *mon, void *vpcontext) if (!rp) { goto done; } - rmon = erts_remove_monitor(&(rp->monitors),mon->ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), mon->ref); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); if (rmon == NULL) { goto done; @@ -8735,7 +8371,7 @@ static void doit_exit_monitor(ErtsMonitor *mon, void *vpcontext) ASSERT(mon->type == MON_TARGET); ASSERT(is_pid(mon->pid) || is_internal_port(mon->pid)); if (is_internal_port(mon->pid)) { - Port *prt = erts_id2port(mon->pid, NULL, 0); + Port *prt = erts_id2port(mon->pid); if (prt == NULL) { goto done; } @@ -8751,13 +8387,13 @@ static void doit_exit_monitor(ErtsMonitor *mon, void *vpcontext) goto done; } UseTmpHeapNoproc(3); - rmon = erts_remove_monitor(&(rp->monitors),mon->ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), mon->ref); if (rmon) { erts_destroy_monitor(rmon); watched = (is_atom(mon->name) ? TUPLE2(lhp, mon->name, erts_this_dist_entry->sysname) - : pcontext->p->id); + : pcontext->p->common.id); erts_queue_monitor_message(rp, &rp_locks, mon->ref, am_process, watched, pcontext->reason); } @@ -8822,21 +8458,22 @@ static void doit_exit_link(ErtsLink *lnk, void *vpcontext) switch(lnk->type) { case LINK_PID: if(is_internal_port(item)) { - Port *prt = erts_id2port(item, NULL, 0); - if (prt) { - rlnk = erts_remove_link(&prt->nlinks, p->id); - if (rlnk) - erts_destroy_link(rlnk); - erts_do_exit_port(prt, p->id, reason); - erts_port_release(prt); - } + Port *prt = erts_port_lookup(item, ERTS_PORT_SFLGS_INVALID_LOOKUP); + if (prt) + erts_port_exit(NULL, + (ERTS_PORT_SIG_FLG_FORCE_SCHED + | ERTS_PORT_SIG_FLG_BROKEN_LINK), + prt, + p->common.id, + reason, + NULL); } else if(is_external_port(item)) { erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); erts_dsprintf(dsbufp, "Erroneous link between %T and external port %T " "found\n", - p->id, + p->common.id, item); erts_send_error_to_logger_nogl(dsbufp); ASSERT(0); /* It isn't possible to setup such a link... */ @@ -8846,14 +8483,14 @@ static void doit_exit_link(ErtsLink *lnk, void *vpcontext) | ERTS_PROC_LOCKS_XSIG_SEND); rp = erts_pid2proc(NULL, 0, item, rp_locks); if (rp) { - rlnk = erts_remove_link(&(rp->nlinks), p->id); + rlnk = erts_remove_link(&ERTS_P_LINKS(rp), p->common.id); /* If rlnk == NULL, we got unlinked while exiting, i.e., do nothing... */ if (rlnk) { int xres; erts_destroy_link(rlnk); xres = send_exit_signal(NULL, - p->id, + p->common.id, rp, &rp_locks, reason, @@ -8865,7 +8502,7 @@ static void doit_exit_link(ErtsLink *lnk, void *vpcontext) if (xres >= 0 && IS_TRACED_FL(rp, F_TRACE_PROCS)) { /* We didn't exit the process and it is traced */ if (IS_TRACED_FL(rp, F_TRACE_PROCS)) { - trace_proc(p, rp, am_getting_unlinked, p->id); + trace_proc(p, rp, am_getting_unlinked, p->common.id); } } } @@ -8879,12 +8516,12 @@ static void doit_exit_link(ErtsLink *lnk, void *vpcontext) ErtsDSigData dsd; int code; ErtsDistLinkData dld; - erts_remove_dist_link(&dld, p->id, item, dep); + erts_remove_dist_link(&dld, p->common.id, item, dep); erts_smp_proc_lock(p, ERTS_PROC_LOCK_MAIN); code = erts_dsig_prepare(&dsd, dep, p, ERTS_DSP_NO_LOCK, 0); if (code == ERTS_DSIG_PREP_CONNECTED) { - code = erts_dsig_send_exit_tt(&dsd, p->id, item, reason, - SEQ_TRACE_TOKEN(p)); + code = erts_dsig_send_exit_tt(&dsd, p->common.id, item, + reason, SEQ_TRACE_TOKEN(p)); ASSERT(code == ERTS_DSIG_SEND_OK); } erts_smp_proc_unlock(p, ERTS_PROC_LOCK_MAIN); @@ -8899,7 +8536,7 @@ static void doit_exit_link(ErtsLink *lnk, void *vpcontext) /* dist entries have node links in a separate structure to avoid confusion */ erts_smp_de_links_lock(dep); - rlnk = erts_remove_link(&(dep->node_links), p->id); + rlnk = erts_remove_link(&(dep->node_links), p->common.id); erts_smp_de_links_unlock(dep); if (rlnk) erts_destroy_link(rlnk); @@ -8927,15 +8564,6 @@ resume_suspend_monitor(ErtsSuspendMonitor *smon, void *vc_p) erts_destroy_suspend_monitor(smon); } -#ifdef ERTS_SMP -static void -proc_dec_refc(void *vproc) -{ - erts_smp_proc_dec_refc((Process *) vproc); -} -#endif - - /* this function fishishes a process and propagates exit messages - called by process_main when a process dies */ void @@ -8973,10 +8601,10 @@ erts_do_exit_process(Process* p, Eterm reason) state = set_proc_exiting_state(p, erts_smp_atomic32_read_nob(&p->state)); if (state & ERTS_PSFLG_PENDING_EXIT) { /* Process exited before pending exit was received... */ - p->u.alive.pending_exit.reason = THE_NON_VALUE; - if (p->u.alive.pending_exit.bp) { - free_message_buffer(p->u.alive.pending_exit.bp); - p->u.alive.pending_exit.bp = NULL; + p->pending_exit.reason = THE_NON_VALUE; + if (p->pending_exit.bp) { + free_message_buffer(p->pending_exit.bp); + p->pending_exit.bp = NULL; } } @@ -8993,23 +8621,25 @@ erts_do_exit_process(Process* p, Eterm reason) trace_proc(p, p, am_exit, reason); } - erts_trace_check_exiting(p->id); + erts_trace_check_exiting(p->common.id); - ASSERT((p->trace_flags & F_INITIAL_TRACE_FLAGS) == F_INITIAL_TRACE_FLAGS); + ASSERT((ERTS_TRACE_FLAGS(p) & F_INITIAL_TRACE_FLAGS) + == F_INITIAL_TRACE_FLAGS); cancel_timer(p); /* Always cancel timer just in case */ - /* - * The timer of this process can *not* be used anymore. The field used - * for the timer is now used for misc exiting data. - */ - p->u.exit_data = NULL; - - if (p->bif_timers) + if (p->u.bif_timers) erts_cancel_bif_timers(p, ERTS_PROC_LOCKS_ALL); erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL_MINOR); + /* + * The p->u.bif_timers of this process can *not* be used anymore; + * will be overwritten by misc termination data. + */ + p->u.terminate = NULL; + + erts_continue_exit_process(p); } @@ -9081,9 +8711,9 @@ erts_continue_exit_process(Process *p) * The registered name *should* be the last "erlang resource" to * cleanup. */ - if (p->reg) { + if (p->common.u.alive.reg) { (void) erts_unregister_name(p, ERTS_PROC_LOCK_MAIN, NULL, THE_NON_VALUE); - ASSERT(!p->reg); + ASSERT(!p->common.u.alive.reg); } erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); @@ -9097,22 +8727,18 @@ erts_continue_exit_process(Process *p) yield_allowed = 0; #endif + /* + * Note! The monitor and link fields will be overwritten + * by erts_ptab_delete_element() below. + */ + mon = ERTS_P_MONITORS(p); + lnk = ERTS_P_LINKS(p); + { - int maybe_save; - int pix; /* Do *not* use erts_get_runq_proc() */ ErtsRunQueue *rq; rq = erts_get_runq_current(ERTS_GET_SCHEDULER_DATA_FROM_PROC(p)); - pix = internal_pid_index(p->id); - - erts_smp_rwmtx_rlock(&erts_proc_tab_rwmtx); - maybe_save = saved_term_procs.end != NULL; - if (maybe_save) { - erts_smp_rwmtx_runlock(&erts_proc_tab_rwmtx); - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - } - erts_smp_runq_lock(rq); #ifdef ERTS_SMP @@ -9123,22 +8749,11 @@ erts_continue_exit_process(Process *p) p->scheduler_data->current_process = NULL; p->scheduler_data->free_process = p; #endif - /* Time of death! */ - erts_smp_atomic_set_relb(&erts_proc.tab[pix], ERTS_AINT_NULL); - ASSERT(erts_smp_atomic32_read_nob(&process_count) > 0); - erts_smp_atomic32_dec_relb(&process_count); + /* Time of death! */ + erts_ptab_delete_element(&erts_proc, &p->common); erts_smp_runq_unlock(rq); - - if (!maybe_save) - erts_smp_rwmtx_runlock(&erts_proc_tab_rwmtx); - else { - if (saved_term_procs.end) - save_terminating_process(p); - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - } - } /* @@ -9148,12 +8763,6 @@ erts_continue_exit_process(Process *p) * when the monitors and/or links hit. */ - mon = p->monitors; - p->monitors = NULL; /* to avoid recursive deletion during traversal */ - - lnk = p->nlinks; - p->nlinks = NULL; - { /* Inactivate and notify free */ erts_aint32_t n, e, a = erts_smp_atomic32_read_nob(&p->state); @@ -9197,7 +8806,7 @@ erts_continue_exit_process(Process *p) UseTmpHeap(4,p); hp = &tmp_heap[0]; - exit_tuple = TUPLE3(hp, am_EXIT, p->id, reason); + exit_tuple = TUPLE3(hp, am_EXIT, p->common.id, reason); exit_tuple_sz = size_object(exit_tuple); @@ -9223,9 +8832,6 @@ erts_continue_exit_process(Process *p) delete_process(p); #ifdef ERTS_SMP - erts_schedule_thr_prgr_later_op(proc_dec_refc, - (void *) p, - &p->u.release_data); erts_smp_proc_lock(p, ERTS_PROC_LOCK_MAIN); ERTS_SMP_CHK_HAVE_ONLY_MAIN_PROC_LOCK(p); #endif @@ -9277,9 +8883,9 @@ cancel_timer(Process* p) ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_MAIN & erts_proc_lc_my_proc_locks(p)); p->flags &= ~(F_INSLPQUEUE|F_TIMO); #ifdef ERTS_SMP - erts_cancel_smp_ptimer(p->u.alive.ptimer); + erts_cancel_smp_ptimer(p->common.u.alive.ptimer); #else - erts_cancel_timer(&p->u.alive.tm); + erts_cancel_timer(&p->common.u.alive.tm); #endif } @@ -9300,12 +8906,12 @@ set_timer(Process* p, Uint timeout) p->flags &= ~F_TIMO; #ifdef ERTS_SMP - erts_create_smp_ptimer(&p->u.alive.ptimer, - p->id, + erts_create_smp_ptimer(&p->common.u.alive.ptimer, + p->common.id, (ErlTimeoutProc) timeout_proc, timeout); #else - erts_set_timer(&p->u.alive.tm, + erts_set_timer(&p->common.u.alive.tm, (ErlTimeoutProc) timeout_proc, NULL, (void*) p, @@ -9323,7 +8929,7 @@ erts_stack_dump(int to, void *to_arg, Process *p) Eterm* sp; int yreg = -1; - if (p->trace_flags & F_SENSITIVE) { + if (ERTS_TRACE_FLAGS(p) & F_SENSITIVE) { return; } erts_program_counter_info(to, to_arg, p); @@ -9411,1068 +9017,6 @@ stack_element_dump(int to, void *to_arg, Process* p, Eterm* sp, int yreg) return yreg; } -/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ - * The processes/0 BIF implementation. * -\* */ - - -#define ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED 25 -#define ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE 1000 -#define ERTS_PROCESSES_BIF_MIN_START_REDS \ - (ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE \ - / ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED) - -#define ERTS_PROCESSES_BIF_TAB_FREE_TERM_PROC_REDS 1 - -#define ERTS_PROCESSES_BIF_INSPECT_TERM_PROC_PER_RED 10 - -#define ERTS_PROCESSES_INSPECT_TERM_PROC_MAX_REDS \ - (ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE \ - / ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED) - - -#define ERTS_PROCESSES_BIF_BUILD_RESULT_CONSES_PER_RED 75 - -#define ERTS_PROCS_DBG_DO_TRACE 0 - -#ifdef DEBUG -# define ERTS_PROCESSES_BIF_DEBUGLEVEL 100 -#else -# define ERTS_PROCESSES_BIF_DEBUGLEVEL 0 -#endif - -#define ERTS_PROCS_DBGLVL_CHK_HALLOC 1 -#define ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS 5 -#define ERTS_PROCS_DBGLVL_CHK_PIDS 10 -#define ERTS_PROCS_DBGLVL_CHK_TERM_PROC_LIST 20 -#define ERTS_PROCS_DBGLVL_CHK_RESLIST 20 - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL == 0 -# define ERTS_PROCS_ASSERT(EXP) -#else -# define ERTS_PROCS_ASSERT(EXP) \ - ((void) ((EXP) \ - ? 1 \ - : (debug_processes_assert_error(#EXP, __FILE__, __LINE__), 0))) -#endif - - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_HALLOC -# define ERTS_PROCS_DBG_SAVE_HEAP_ALLOC(PBDP, HP, SZ) \ -do { \ - ERTS_PROCS_ASSERT(!(PBDP)->debug.heap); \ - ERTS_PROCS_ASSERT(!(PBDP)->debug.heap_size); \ - (PBDP)->debug.heap = (HP); \ - (PBDP)->debug.heap_size = (SZ); \ -} while (0) -# define ERTS_PROCS_DBG_VERIFY_HEAP_ALLOC_USED(PBDP, HP) \ -do { \ - ERTS_PROCS_ASSERT((PBDP)->debug.heap); \ - ERTS_PROCS_ASSERT((PBDP)->debug.heap_size); \ - ERTS_PROCS_ASSERT((PBDP)->debug.heap + (PBDP)->debug.heap_size == (HP));\ - (PBDP)->debug.heap = NULL; \ - (PBDP)->debug.heap_size = 0; \ -} while (0) -# define ERTS_PROCS_DBG_HEAP_ALLOC_INIT(PBDP) \ -do { \ - (PBDP)->debug.heap = NULL; \ - (PBDP)->debug.heap_size = 0; \ -} while (0) -#else -# define ERTS_PROCS_DBG_SAVE_HEAP_ALLOC(PBDP, HP, SZ) -# define ERTS_PROCS_DBG_VERIFY_HEAP_ALLOC_USED(PBDP, HP) -# define ERTS_PROCS_DBG_HEAP_ALLOC_INIT(PBDP) -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_RESLIST -# define ERTS_PROCS_DBG_CHK_RESLIST(R) debug_processes_check_res_list((R)) -#else -# define ERTS_PROCS_DBG_CHK_RESLIST(R) -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_PIDS -# define ERTS_PROCS_DBG_SAVE_PIDS(PBDP) debug_processes_save_all_pids((PBDP)) -# define ERTS_PROCS_DBG_VERIFY_PIDS(PBDP) \ -do { \ - if (!(PBDP)->debug.correct_pids_verified) \ - debug_processes_verify_all_pids((PBDP)); \ -} while (0) -# define ERTS_PROCS_DBG_CLEANUP_CHK_PIDS(PBDP) \ -do { \ - if ((PBDP)->debug.correct_pids) { \ - erts_free(ERTS_ALC_T_PROCS_PIDS, \ - (PBDP)->debug.correct_pids); \ - (PBDP)->debug.correct_pids = NULL; \ - } \ -} while(0) -# define ERTS_PROCS_DBG_CHK_PIDS_INIT(PBDP) \ -do { \ - (PBDP)->debug.correct_pids_verified = 0; \ - (PBDP)->debug.correct_pids = NULL; \ -} while (0) -#else -# define ERTS_PROCS_DBG_SAVE_PIDS(PBDP) -# define ERTS_PROCS_DBG_VERIFY_PIDS(PBDP) -# define ERTS_PROCS_DBG_CLEANUP_CHK_PIDS(PBDP) -# define ERTS_PROCS_DBG_CHK_PIDS_INIT(PBDP) -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS -# define ERTS_PROCS_DBG_CHK_PID_FOUND(PBDP, PID, IC) \ - debug_processes_check_found_pid((PBDP), (PID), (IC), 1) -# define ERTS_PROCS_DBG_CHK_PID_NOT_FOUND(PBDP, PID, IC) \ - debug_processes_check_found_pid((PBDP), (PID), (IC), 0) -#else -# define ERTS_PROCS_DBG_CHK_PID_FOUND(PBDP, PID, IC) -# define ERTS_PROCS_DBG_CHK_PID_NOT_FOUND(PBDP, PID, IC) -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_TERM_PROC_LIST -# define ERTS_PROCS_DBG_CHK_TPLIST() \ - debug_processes_check_term_proc_list() -# define ERTS_PROCS_DBG_CHK_FREELIST(FL) \ - debug_processes_check_term_proc_free_list(FL) -#else -# define ERTS_PROCS_DBG_CHK_TPLIST() -# define ERTS_PROCS_DBG_CHK_FREELIST(FL) -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL == 0 -#if ERTS_PROCS_DBG_DO_TRACE -# define ERTS_PROCS_DBG_INIT(P, PBDP) (PBDP)->debug.caller = (P)->id -# else -# define ERTS_PROCS_DBG_INIT(P, PBDP) -# endif -# define ERTS_PROCS_DBG_CLEANUP(PBDP) -#else -# define ERTS_PROCS_DBG_INIT(P, PBDP) \ -do { \ - (PBDP)->debug.caller = (P)->id; \ - ERTS_PROCS_DBG_HEAP_ALLOC_INIT((PBDP)); \ - ERTS_PROCS_DBG_CHK_PIDS_INIT((PBDP)); \ -} while (0) -# define ERTS_PROCS_DBG_CLEANUP(PBDP) \ -do { \ - ERTS_PROCS_DBG_CLEANUP_CHK_PIDS((PBDP)); \ -} while (0) -#endif - -#if ERTS_PROCS_DBG_DO_TRACE -# define ERTS_PROCS_DBG_TRACE(PID, FUNC, WHAT) \ - erts_fprintf(stderr, "%T %s:%d:%s(): %s\n", \ - (PID), __FILE__, __LINE__, #FUNC, #WHAT) -#else -# define ERTS_PROCS_DBG_TRACE(PID, FUNC, WHAT) -#endif - -static Uint processes_bif_tab_chunks; -static Export processes_trap_export; - -typedef struct { - Uint64 interval; -} ErtsProcessesBifChunkInfo; - -typedef enum { - INITIALIZING, - INSPECTING_TABLE, - INSPECTING_TERMINATED_PROCESSES, - BUILDING_RESULT, - RETURN_RESULT -} ErtsProcessesBifState; - -typedef struct { - ErtsProcessesBifState state; - Eterm caller; - ErtsProcessesBifChunkInfo *chunk; - int tix; - int pid_ix; - int pid_sz; - Eterm *pid; - ErtsTermProcElement *bif_invocation; /* Only used when > 1 chunk */ - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL != 0 || ERTS_PROCS_DBG_DO_TRACE - struct { - Eterm caller; -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - Uint64 *pid_started; -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_HALLOC - Eterm *heap; - Uint heap_size; -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_PIDS - int correct_pids_verified; - Eterm *correct_pids; -#endif - } debug; -#endif - -} ErtsProcessesBifData; - - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL != 0 -static void debug_processes_assert_error(char* expr, char* file, int line); -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_RESLIST -static void debug_processes_check_res_list(Eterm list); -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_PIDS -static void debug_processes_save_all_pids(ErtsProcessesBifData *pbdp); -static void debug_processes_verify_all_pids(ErtsProcessesBifData *pbdp); -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS -static void debug_processes_check_found_pid(ErtsProcessesBifData *pbdp, - Eterm pid, - Uint64 ic, - int pid_should_be_found); -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_TERM_PROC_LIST -static void debug_processes_check_term_proc_list(void); -static void debug_processes_check_term_proc_free_list(ErtsTermProcElement *tpep); -#endif - -static void -save_terminating_process(Process *p) -{ - ErtsTermProcElement *tpep = erts_alloc(ERTS_ALC_T_PROCS_TPROC_EL, - sizeof(ErtsTermProcElement)); - ERTS_PROCS_ASSERT(saved_term_procs.start && saved_term_procs.end); - ERTS_SMP_LC_ASSERT(erts_lc_rwmtx_is_rwlocked(&erts_proc_tab_rwmtx)); - - ERTS_PROCS_DBG_CHK_TPLIST(); - - tpep->prev = saved_term_procs.end; - tpep->next = NULL; - tpep->ix = internal_pid_index(p->id); - tpep->u.process.pid = p->id; - tpep->u.process.spawned = p->started_interval; - tpep->u.process.exited = get_proc_interval(); - - saved_term_procs.end->next = tpep; - saved_term_procs.end = tpep; - - ERTS_PROCS_DBG_CHK_TPLIST(); - - ERTS_PROCS_ASSERT(tpep->prev->ix >= 0 - ? (tpep->u.process.exited - >= tpep->prev->u.process.exited) - : (tpep->u.process.exited - >= tpep->prev->u.bif_invocation.interval)); -} - -static void -cleanup_processes_bif_data(Binary *bp) -{ - ErtsProcessesBifData *pbdp = ERTS_MAGIC_BIN_DATA(bp); - - ERTS_PROCS_DBG_TRACE(pbdp->debug.caller, cleanup_processes_bif_data, call); - - if (pbdp->state != INITIALIZING) { - - if (pbdp->chunk) { - erts_free(ERTS_ALC_T_PROCS_CNKINF, pbdp->chunk); - pbdp->chunk = NULL; - } - if (pbdp->pid) { - erts_free(ERTS_ALC_T_PROCS_PIDS, pbdp->pid); - pbdp->pid = NULL; - } - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - if (pbdp->debug.pid_started) { - erts_free(ERTS_ALC_T_PROCS_PIDS, pbdp->debug.pid_started); - pbdp->debug.pid_started = NULL; - } -#endif - - if (pbdp->bif_invocation) { - ErtsTermProcElement *tpep; - - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - - ERTS_PROCS_DBG_TRACE(pbdp->debug.caller, - cleanup_processes_bif_data, - term_proc_cleanup); - - tpep = pbdp->bif_invocation; - pbdp->bif_invocation = NULL; - - ERTS_PROCS_DBG_CHK_TPLIST(); - - if (tpep->prev) { - /* - * Only remove this bif invokation when we - * have preceding invokations. - */ - tpep->prev->next = tpep->next; - if (tpep->next) - tpep->next->prev = tpep->prev; - else { - /* - * At the time of writing this branch cannot be - * reached. I don't want to remove this code though - * since it may be possible to reach this line - * in the future if the cleanup order in - * erts_do_exit_process() is changed. The ASSERT(0) - * is only here to make us aware that the reorder - * has happened. /rickard - */ - ASSERT(0); - saved_term_procs.end = tpep->prev; - } - erts_free(ERTS_ALC_T_PROCS_TPROC_EL, tpep); - } - else { - /* - * Free all elements until next bif invokation - * is found. - */ - ERTS_PROCS_ASSERT(saved_term_procs.start == tpep); - do { - ErtsTermProcElement *ftpep = tpep; - tpep = tpep->next; - erts_free(ERTS_ALC_T_PROCS_TPROC_EL, ftpep); - } while (tpep && tpep->ix >= 0); - saved_term_procs.start = tpep; - if (tpep) - tpep->prev = NULL; - else - saved_term_procs.end = NULL; - } - - ERTS_PROCS_DBG_CHK_TPLIST(); - - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - - } - } - - ERTS_PROCS_DBG_TRACE(pbdp->debug.caller, - cleanup_processes_bif_data, - return); - ERTS_PROCS_DBG_CLEANUP(pbdp); -} - -static int -processes_bif_engine(Process *p, Eterm *res_accp, Binary *mbp) -{ - ErtsProcessesBifData *pbdp = ERTS_MAGIC_BIN_DATA(mbp); - int have_reds; - int reds; - int locked = 0; - - do { - switch (pbdp->state) { - case INITIALIZING: - pbdp->chunk = erts_alloc(ERTS_ALC_T_PROCS_CNKINF, - (sizeof(ErtsProcessesBifChunkInfo) - * processes_bif_tab_chunks)); - pbdp->tix = 0; - pbdp->pid_ix = 0; - - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - locked = 1; - - ERTS_PROCS_DBG_TRACE(p->id, processes_bif_engine, init); - - pbdp->pid_sz = erts_process_count(); - pbdp->pid = erts_alloc(ERTS_ALC_T_PROCS_PIDS, - sizeof(Eterm)*pbdp->pid_sz); - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - pbdp->debug.pid_started = erts_alloc(ERTS_ALC_T_PROCS_PIDS, - sizeof(Uint64)*pbdp->pid_sz); -#endif - - ERTS_PROCS_DBG_SAVE_PIDS(pbdp); - - if (processes_bif_tab_chunks == 1) - pbdp->bif_invocation = NULL; - else { - /* - * We will have to access the table multiple times - * releasing the table lock in between chunks. - */ - pbdp->bif_invocation = erts_alloc(ERTS_ALC_T_PROCS_TPROC_EL, - sizeof(ErtsTermProcElement)); - pbdp->bif_invocation->ix = -1; - pbdp->bif_invocation->u.bif_invocation.interval - = step_proc_interval(); - ERTS_PROCS_DBG_CHK_TPLIST(); - - pbdp->bif_invocation->next = NULL; - if (saved_term_procs.end) { - pbdp->bif_invocation->prev = saved_term_procs.end; - saved_term_procs.end->next = pbdp->bif_invocation; - ERTS_PROCS_ASSERT(saved_term_procs.start); - } - else { - pbdp->bif_invocation->prev = NULL; - saved_term_procs.start = pbdp->bif_invocation; - } - saved_term_procs.end = pbdp->bif_invocation; - - ERTS_PROCS_DBG_CHK_TPLIST(); - - } - - pbdp->state = INSPECTING_TABLE; - /* Fall through */ - - case INSPECTING_TABLE: { - int ix = pbdp->tix; - int indices = ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE; - int cix = ix / ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE; - int end_ix = ix + indices; - Uint64 *invocation_interval_p; - - invocation_interval_p - = (pbdp->bif_invocation - ? &pbdp->bif_invocation->u.bif_invocation.interval - : NULL); - - ERTS_PROCS_ASSERT(is_nil(*res_accp)); - if (!locked) { - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - locked = 1; - } - - ERTS_SMP_LC_ASSERT(erts_lc_rwmtx_is_rwlocked(&erts_proc_tab_rwmtx)); - ERTS_PROCS_DBG_TRACE(p->id, processes_bif_engine, insp_table); - - if (cix != 0) - pbdp->chunk[cix].interval = step_proc_interval(); - else if (pbdp->bif_invocation) - pbdp->chunk[0].interval = *invocation_interval_p; - /* else: interval is irrelevant */ - - if (end_ix >= erts_proc.max) { - ERTS_PROCS_ASSERT(cix+1 == processes_bif_tab_chunks); - end_ix = erts_proc.max; - indices = end_ix - ix; - /* What to do when done with this chunk */ - pbdp->state = (processes_bif_tab_chunks == 1 - ? BUILDING_RESULT - : INSPECTING_TERMINATED_PROCESSES); - } - - for (; ix < end_ix; ix++) { - Process *rp = erts_pix2proc(ix); - if (rp - && (!invocation_interval_p - || rp->started_interval < *invocation_interval_p)) { - ERTS_PROCS_ASSERT(is_internal_pid(rp->id)); - pbdp->pid[pbdp->pid_ix] = rp->id; - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - pbdp->debug.pid_started[pbdp->pid_ix] = rp->started_interval; -#endif - - pbdp->pid_ix++; - ERTS_PROCS_ASSERT(pbdp->pid_ix <= pbdp->pid_sz); - } - } - - pbdp->tix = end_ix; - - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - locked = 0; - - reds = indices/ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED; - BUMP_REDS(p, reds); - - have_reds = ERTS_BIF_REDS_LEFT(p); - - if (have_reds && pbdp->state == INSPECTING_TABLE) { - ix = pbdp->tix; - indices = ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE; - end_ix = ix + indices; - if (end_ix > erts_proc.max) { - end_ix = erts_proc.max; - indices = end_ix - ix; - } - - reds = indices/ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED; - - /* Pretend we have no reds left if we haven't got enough - reductions to complete next chunk */ - if (reds > have_reds) - have_reds = 0; - } - - break; - } - - case INSPECTING_TERMINATED_PROCESSES: { - int i; - int max_reds; - int free_term_procs = 0; - Uint64 invocation_interval; - ErtsTermProcElement *tpep; - ErtsTermProcElement *free_list = NULL; - - tpep = pbdp->bif_invocation; - ERTS_PROCS_ASSERT(tpep); - invocation_interval = tpep->u.bif_invocation.interval; - - max_reds = have_reds = ERTS_BIF_REDS_LEFT(p); - if (max_reds > ERTS_PROCESSES_INSPECT_TERM_PROC_MAX_REDS) - max_reds = ERTS_PROCESSES_INSPECT_TERM_PROC_MAX_REDS; - - reds = 0; - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - ERTS_PROCS_DBG_TRACE(p->id, processes_bif_engine, insp_term_procs); - - ERTS_PROCS_DBG_CHK_TPLIST(); - - if (tpep->prev) - tpep->prev->next = tpep->next; - else { - ERTS_PROCS_ASSERT(saved_term_procs.start == tpep); - saved_term_procs.start = tpep->next; - - if (saved_term_procs.start && saved_term_procs.start->ix >= 0) { - free_list = saved_term_procs.start; - free_term_procs = 1; - } - } - - if (tpep->next) - tpep->next->prev = tpep->prev; - else - saved_term_procs.end = tpep->prev; - - tpep = tpep->next; - - i = 0; - while (reds < max_reds && tpep) { - if (tpep->ix < 0) { - if (free_term_procs) { - ERTS_PROCS_ASSERT(free_list); - ERTS_PROCS_ASSERT(tpep->prev); - - tpep->prev->next = NULL; /* end of free_list */ - saved_term_procs.start = tpep; - tpep->prev = NULL; - free_term_procs = 0; - } - } - else { - int cix = tpep->ix/ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE; - Uint64 chunk_interval = pbdp->chunk[cix].interval; - Eterm pid = tpep->u.process.pid; - ERTS_PROCS_ASSERT(is_internal_pid(pid)); - - if (tpep->u.process.spawned < invocation_interval) { - if (tpep->u.process.exited < chunk_interval) { - ERTS_PROCS_DBG_CHK_PID_NOT_FOUND(pbdp, - pid, - tpep->u.process.spawned); - pbdp->pid[pbdp->pid_ix] = pid; -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - pbdp->debug.pid_started[pbdp->pid_ix] - = tpep->u.process.spawned; -#endif - pbdp->pid_ix++; - ERTS_PROCS_ASSERT(pbdp->pid_ix <= pbdp->pid_sz); - } - else { - ERTS_PROCS_DBG_CHK_PID_FOUND(pbdp, - pid, - tpep->u.process.spawned); - } - } - else { - ERTS_PROCS_DBG_CHK_PID_NOT_FOUND(pbdp, - pid, - tpep->u.process.spawned); - } - - i++; - if (i == ERTS_PROCESSES_BIF_INSPECT_TERM_PROC_PER_RED) { - reds++; - i = 0; - } - if (free_term_procs) - reds += ERTS_PROCESSES_BIF_TAB_FREE_TERM_PROC_REDS; - } - tpep = tpep->next; - } - - if (free_term_procs) { - ERTS_PROCS_ASSERT(free_list); - saved_term_procs.start = tpep; - if (!tpep) - saved_term_procs.end = NULL; - else { - ERTS_PROCS_ASSERT(tpep->prev); - tpep->prev->next = NULL; /* end of free_list */ - tpep->prev = NULL; - } - } - - if (!tpep) { - /* Done */ - ERTS_PROCS_ASSERT(pbdp->pid_ix == pbdp->pid_sz); - pbdp->state = BUILDING_RESULT; - pbdp->bif_invocation->next = free_list; - free_list = pbdp->bif_invocation; - pbdp->bif_invocation = NULL; - } - else { - /* Link in bif_invocation again where we left off */ - pbdp->bif_invocation->prev = tpep->prev; - pbdp->bif_invocation->next = tpep; - tpep->prev = pbdp->bif_invocation; - if (pbdp->bif_invocation->prev) - pbdp->bif_invocation->prev->next = pbdp->bif_invocation; - else { - ERTS_PROCS_ASSERT(saved_term_procs.start == tpep); - saved_term_procs.start = pbdp->bif_invocation; - } - } - - ERTS_PROCS_DBG_CHK_TPLIST(); - ERTS_PROCS_DBG_CHK_FREELIST(free_list); - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - - /* - * We do the actual free of term proc structures now when we - * have released the table lock instead of when we encountered - * them. This since free() isn't for free and we don't want to - * unnecessarily block other schedulers. - */ - while (free_list) { - tpep = free_list; - free_list = tpep->next; - erts_free(ERTS_ALC_T_PROCS_TPROC_EL, tpep); - } - - have_reds -= reds; - if (have_reds < 0) - have_reds = 0; - BUMP_REDS(p, reds); - break; - } - - case BUILDING_RESULT: { - int conses, ix, min_ix; - Eterm *hp; - Eterm res = *res_accp; - - ERTS_PROCS_DBG_VERIFY_PIDS(pbdp); - ERTS_PROCS_DBG_CHK_RESLIST(res); - - ERTS_PROCS_DBG_TRACE(p->id, processes_bif_engine, begin_build_res); - - have_reds = ERTS_BIF_REDS_LEFT(p); - conses = ERTS_PROCESSES_BIF_BUILD_RESULT_CONSES_PER_RED*have_reds; - min_ix = pbdp->pid_ix - conses; - if (min_ix < 0) { - min_ix = 0; - conses = pbdp->pid_ix; - } - - hp = HAlloc(p, conses*2); - ERTS_PROCS_DBG_SAVE_HEAP_ALLOC(pbdp, hp, conses*2); - - for (ix = pbdp->pid_ix - 1; ix >= min_ix; ix--) { - ERTS_PROCS_ASSERT(is_internal_pid(pbdp->pid[ix])); - res = CONS(hp, pbdp->pid[ix], res); - hp += 2; - } - - ERTS_PROCS_DBG_VERIFY_HEAP_ALLOC_USED(pbdp, hp); - - pbdp->pid_ix = min_ix; - if (min_ix == 0) - pbdp->state = RETURN_RESULT; - else { - pbdp->pid_sz = min_ix; - pbdp->pid = erts_realloc(ERTS_ALC_T_PROCS_PIDS, - pbdp->pid, - sizeof(Eterm)*pbdp->pid_sz); -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - pbdp->debug.pid_started = erts_realloc(ERTS_ALC_T_PROCS_PIDS, - pbdp->debug.pid_started, - (sizeof(Uint64) - * pbdp->pid_sz)); -#endif - } - reds = conses/ERTS_PROCESSES_BIF_BUILD_RESULT_CONSES_PER_RED; - BUMP_REDS(p, reds); - have_reds -= reds; - - ERTS_PROCS_DBG_CHK_RESLIST(res); - ERTS_PROCS_DBG_TRACE(p->id, processes_bif_engine, end_build_res); - *res_accp = res; - break; - } - case RETURN_RESULT: - cleanup_processes_bif_data(mbp); - return 1; - - default: - erl_exit(ERTS_ABORT_EXIT, - "erlang:processes/0: Invalid state: %d\n", - (int) pbdp->state); - } - - - } while (have_reds || pbdp->state == RETURN_RESULT); - - return 0; -} - -/* - * processes_trap/2 is a hidden BIF that processes/0 traps to. - */ - -static BIF_RETTYPE processes_trap(BIF_ALIST_2) -{ - Eterm res_acc; - Binary *mbp; - - /* - * This bif cannot be called from erlang code. It can only be - * trapped to from processes/0; therefore, a bad argument - * is a processes/0 internal error. - */ - - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_trap, call); - ERTS_PROCS_ASSERT(is_nil(BIF_ARG_1) || is_list(BIF_ARG_1)); - - res_acc = BIF_ARG_1; - - ERTS_PROCS_ASSERT(ERTS_TERM_IS_MAGIC_BINARY(BIF_ARG_2)); - - mbp = ((ProcBin *) binary_val(BIF_ARG_2))->val; - - ERTS_PROCS_ASSERT(ERTS_MAGIC_BIN_DESTRUCTOR(mbp) - == cleanup_processes_bif_data); - ERTS_PROCS_ASSERT( - ((ErtsProcessesBifData *) ERTS_MAGIC_BIN_DATA(mbp))->debug.caller - == BIF_P->id); - - if (processes_bif_engine(BIF_P, &res_acc, mbp)) { - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_trap, return); - BIF_RET(res_acc); - } - else { - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_trap, trap); - ERTS_BIF_YIELD2(&processes_trap_export, BIF_P, res_acc, BIF_ARG_2); - } -} - - - -/* - * The actual processes/0 BIF. - */ - -BIF_RETTYPE processes_0(BIF_ALIST_0) -{ - /* - * A requirement: The list of pids returned should be a consistent - * snapshot of all processes existing at some point - * in time during the execution of processes/0. Since - * processes might terminate while processes/0 is - * executing, we have to keep track of terminated - * processes and add them to the result. We also - * ignore processes created after processes/0 has - * begun executing. - */ - Eterm res_acc = NIL; - Binary *mbp = erts_create_magic_binary(sizeof(ErtsProcessesBifData), - cleanup_processes_bif_data); - ErtsProcessesBifData *pbdp = ERTS_MAGIC_BIN_DATA(mbp); - - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_0, call); - pbdp->state = INITIALIZING; - ERTS_PROCS_DBG_INIT(BIF_P, pbdp); - - if (ERTS_BIF_REDS_LEFT(BIF_P) >= ERTS_PROCESSES_BIF_MIN_START_REDS - && processes_bif_engine(BIF_P, &res_acc, mbp)) { - erts_bin_free(mbp); - ERTS_PROCS_DBG_CHK_RESLIST(res_acc); - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_0, return); - BIF_RET(res_acc); - } - else { - Eterm *hp; - Eterm magic_bin; - ERTS_PROCS_DBG_CHK_RESLIST(res_acc); - hp = HAlloc(BIF_P, PROC_BIN_SIZE); - ERTS_PROCS_DBG_SAVE_HEAP_ALLOC(pbdp, hp, PROC_BIN_SIZE); - magic_bin = erts_mk_magic_binary_term(&hp, &MSO(BIF_P), mbp); - ERTS_PROCS_DBG_VERIFY_HEAP_ALLOC_USED(pbdp, hp); - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_0, trap); - ERTS_BIF_YIELD2(&processes_trap_export, BIF_P, res_acc, magic_bin); - } -} - -static void -init_processes_bif(void) -{ - saved_term_procs.start = NULL; - saved_term_procs.end = NULL; - processes_bif_tab_chunks = (((erts_proc.max - 1) - / ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE) - + 1); - - /* processes_trap/2 is a hidden BIF that the processes/0 BIF traps to. */ - erts_init_trap_export(&processes_trap_export, am_erlang, am_processes_trap, 2, - &processes_trap); - -} - -/* - * Debug stuff - */ - -#if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) -int -erts_dbg_check_halloc_lock(Process *p) -{ - if (ERTS_PROC_LOCK_MAIN & erts_proc_lc_my_proc_locks(p)) - return 1; - if (p->id == ERTS_INVALID_PID) - return 1; - if (p->scheduler_data && p == p->scheduler_data->match_pseudo_process) - return 1; - if (erts_thr_progress_is_blocking()) - return 1; - return 0; -} -#endif - -Eterm -erts_debug_processes(Process *c_p) -{ - /* This is the old processes/0 BIF. */ - int i; - Uint need; - Eterm res; - Eterm* hp; - Process *p; - Eterm *hp_end; - - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - - res = NIL; - need = erts_process_count() * 2; - hp = HAlloc(c_p, need); /* we need two heap words for each pid */ - hp_end = hp + need; - - /* make the list by scanning bakward */ - - - for (i = erts_proc.max-1; i >= 0; i--) { - p = erts_pix2proc(i); - if (p) { - res = CONS(hp, p->id, res); - hp += 2; - } - } - - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - - HRelease(c_p, hp_end, hp); - - return res; -} - -Eterm -erts_debug_processes_bif_info(Process *c_p) -{ - ERTS_DECL_AM(processes_bif_info); - Eterm elements[] = { - AM_processes_bif_info, - make_small((Uint) ERTS_PROCESSES_BIF_MIN_START_REDS), - make_small((Uint) processes_bif_tab_chunks), - make_small((Uint) ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE), - make_small((Uint) ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED), - make_small((Uint) ERTS_PROCESSES_BIF_TAB_FREE_TERM_PROC_REDS), - make_small((Uint) ERTS_PROCESSES_BIF_INSPECT_TERM_PROC_PER_RED), - make_small((Uint) ERTS_PROCESSES_INSPECT_TERM_PROC_MAX_REDS), - make_small((Uint) ERTS_PROCESSES_BIF_BUILD_RESULT_CONSES_PER_RED), - make_small((Uint) ERTS_PROCESSES_BIF_DEBUGLEVEL) - }; - Uint sz = 0; - Eterm *hp; - (void) erts_bld_tuplev(NULL, &sz, sizeof(elements)/sizeof(Eterm), elements); - hp = HAlloc(c_p, sz); - return erts_bld_tuplev(&hp, NULL, sizeof(elements)/sizeof(Eterm), elements); -} - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS -static void -debug_processes_check_found_pid(ErtsProcessesBifData *pbdp, - Eterm pid, - Uint64 ic, - int pid_should_be_found) -{ - int i; - for (i = 0; i < pbdp->pid_ix; i++) { - if (pbdp->pid[i] == pid && pbdp->debug.pid_started[i] == ic) { - ERTS_PROCS_ASSERT(pid_should_be_found); - return; - } - } - ERTS_PROCS_ASSERT(!pid_should_be_found); -} -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_RESLIST -static void -debug_processes_check_res_list(Eterm list) -{ - while (is_list(list)) { - Eterm* consp = list_val(list); - Eterm hd = CAR(consp); - ERTS_PROCS_ASSERT(is_internal_pid(hd)); - list = CDR(consp); - } - - ERTS_PROCS_ASSERT(is_nil(list)); -} -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_PIDS - -static void -debug_processes_save_all_pids(ErtsProcessesBifData *pbdp) -{ - int ix, tix, cpix; - pbdp->debug.correct_pids_verified = 0; - pbdp->debug.correct_pids = erts_alloc(ERTS_ALC_T_PROCS_PIDS, - sizeof(Eterm)*pbdp->pid_sz); - - for (tix = 0, cpix = 0; tix < erts_proc.max; tix++) { - Process *rp = erts_pix2proc(tix); - if (rp) { - ERTS_PROCS_ASSERT(is_internal_pid(rp->id)); - pbdp->debug.correct_pids[cpix++] = rp->id; - ERTS_PROCS_ASSERT(cpix <= pbdp->pid_sz); - } - } - ERTS_PROCS_ASSERT(cpix == pbdp->pid_sz); - - for (ix = 0; ix < pbdp->pid_sz; ix++) - pbdp->pid[ix] = make_small(ix); -} - -static void -debug_processes_verify_all_pids(ErtsProcessesBifData *pbdp) -{ - int ix, cpix; - - ERTS_PROCS_ASSERT(pbdp->pid_ix == pbdp->pid_sz); - - for (ix = 0; ix < pbdp->pid_sz; ix++) { - int found = 0; - Eterm pid = pbdp->pid[ix]; - ERTS_PROCS_ASSERT(is_internal_pid(pid)); - for (cpix = ix; cpix < pbdp->pid_sz; cpix++) { - if (pbdp->debug.correct_pids[cpix] == pid) { - pbdp->debug.correct_pids[cpix] = NIL; - found = 1; - break; - } - } - if (!found) { - for (cpix = 0; cpix < ix; cpix++) { - if (pbdp->debug.correct_pids[cpix] == pid) { - pbdp->debug.correct_pids[cpix] = NIL; - found = 1; - break; - } - } - } - ERTS_PROCS_ASSERT(found); - } - pbdp->debug.correct_pids_verified = 1; - - erts_free(ERTS_ALC_T_PROCS_PIDS, pbdp->debug.correct_pids); - pbdp->debug.correct_pids = NULL; -} -#endif /* ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_PIDS */ - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_TERM_PROC_LIST -static void -debug_processes_check_term_proc_list(void) -{ - ERTS_SMP_LC_ASSERT(erts_lc_rwmtx_is_rwlocked(&erts_proc_tab_rwmtx)); - if (!saved_term_procs.start) - ERTS_PROCS_ASSERT(!saved_term_procs.end); - else { - Uint64 curr_interval = get_proc_interval(); - Uint64 *prev_x_interval_p = NULL; - ErtsTermProcElement *tpep; - - for (tpep = saved_term_procs.start; tpep; tpep = tpep->next) { - if (!tpep->prev) - ERTS_PROCS_ASSERT(saved_term_procs.start == tpep); - else - ERTS_PROCS_ASSERT(tpep->prev->next == tpep); - if (!tpep->next) - ERTS_PROCS_ASSERT(saved_term_procs.end == tpep); - else - ERTS_PROCS_ASSERT(tpep->next->prev == tpep); - if (tpep->ix < 0) { - Uint64 interval = tpep->u.bif_invocation.interval; - ERTS_PROCS_ASSERT(interval <= curr_interval); - } - else { - Uint64 s_interval = tpep->u.process.spawned; - Uint64 x_interval = tpep->u.process.exited; - - ERTS_PROCS_ASSERT(s_interval <= x_interval); - if (prev_x_interval_p) - ERTS_PROCS_ASSERT(*prev_x_interval_p <= x_interval); - prev_x_interval_p = &tpep->u.process.exited; - ERTS_PROCS_ASSERT(is_internal_pid(tpep->u.process.pid)); - ERTS_PROCS_ASSERT(tpep->ix - == internal_pid_index(tpep->u.process.pid)); - } - } - - } -} - -static void -debug_processes_check_term_proc_free_list(ErtsTermProcElement *free_list) -{ - if (saved_term_procs.start) { - ErtsTermProcElement *ftpep; - ErtsTermProcElement *tpep; - - for (ftpep = free_list; ftpep; ftpep = ftpep->next) { - for (tpep = saved_term_procs.start; tpep; tpep = tpep->next) - ERTS_PROCS_ASSERT(ftpep != tpep); - } - } -} - -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL != 0 - -static void -debug_processes_assert_error(char* expr, char* file, int line) -{ - fflush(stdout); - erts_fprintf(stderr, "%s:%d: Assertion failed: %s\n", file, line, expr); - fflush(stderr); - abort(); -} - -#endif - -/* *\ - * End of the processes/0 BIF implementation. * -\* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */ - /* * A nice system halt closing all open port goes as follows: * 1) This function schedules the aux work ERTS_SSI_AUX_WORK_REAP_PORTS @@ -10499,3 +9043,19 @@ void erl_halt(int code) notify_reap_ports_relb(); } } + +#if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) +int +erts_dbg_check_halloc_lock(Process *p) +{ + if (ERTS_PROC_LOCK_MAIN & erts_proc_lc_my_proc_locks(p)) + return 1; + if (p->common.id == ERTS_INVALID_PID) + return 1; + if (p->scheduler_data && p == p->scheduler_data->match_pseudo_process) + return 1; + if (erts_thr_progress_is_blocking()) + return 1; + return 0; +} +#endif diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index e789c873fb..6d1032c292 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -42,6 +42,9 @@ typedef struct process Process; #include "erl_process_lock.h" /* Only pull out important types... */ #undef ERTS_PROCESS_LOCK_ONLY_PROC_LOCK_TYPE__ +#define ERL_PORT_GET_PORT_TYPE_ONLY__ +#include "erl_port.h" +#undef ERL_PORT_GET_PORT_TYPE_ONLY__ #include "erl_vm.h" #include "erl_smp.h" #include "erl_message.h" @@ -66,11 +69,10 @@ typedef struct process Process; #undef ERL_THR_PROGRESS_TSD_TYPE_ONLY struct ErtsNodesMonitor_; -struct port; #define ERTS_MAX_NO_OF_SCHEDULERS 1024 -#define ERTS_DEFAULT_MAX_PROCESSES (1 << 15) +#define ERTS_DEFAULT_MAX_PROCESSES (1 << 18) #define ERTS_HEAP_ALLOC(Type, Size) \ erts_alloc((Type), (Size)) @@ -205,32 +207,10 @@ extern int erts_sched_thread_suggested_stack_size; ((Uint32) erts_smp_atomic32_read_nob(&(RQ)->flags)) #define ERTS_RUNQ_FLGS_GET_MB(RQ) \ ((Uint32) erts_smp_atomic32_read_mb(&(RQ)->flags)) -#define ERTS_RUNQ_FLGS_MASK_SET(RQ, MSK, FLGS) \ - ((Uint32) erts_smp_atomic32_mask_set_relb(&(RQ)->flags, \ - (erts_aint32_t) (MSK), \ - (erts_aint32_t) (FLGS))) - -ERTS_GLB_INLINE erts_aint32_t -erts_smp_atomic32_mask_set_relb(erts_smp_atomic32_t *a32p, - erts_aint32_t mask, - erts_aint32_t set); -#if ERTS_GLB_INLINE_INCL_FUNC_DEF -ERTS_GLB_INLINE erts_aint32_t -erts_smp_atomic32_mask_set_relb(erts_smp_atomic32_t *a32p, - erts_aint32_t mask, - erts_aint32_t set) -{ - erts_aint32_t act = erts_smp_atomic32_read_nob(a32p); - while (1) { - erts_aint32_t exp = act; - erts_aint32_t new = exp & ~mask; - new |= (mask & set); - act = erts_smp_atomic32_cmpxchg_relb(a32p, new, exp); - if (act == exp) - return act; - } -} -#endif +#define ERTS_RUNQ_FLGS_READ_BSET(RQ, MSK, FLGS) \ + ((Uint32) erts_smp_atomic32_read_bset_relb(&(RQ)->flags, \ + (erts_aint32_t) (MSK), \ + (erts_aint32_t) (FLGS))) typedef enum { ERTS_SCHDLR_SSPND_DONE_MSCHED_BLOCKED, @@ -311,6 +291,7 @@ struct ErtsProcList_ { Eterm pid; Uint64 started_interval; ErtsProcList* next; + ErtsProcList* prev; }; typedef struct ErtsMiscOpList_ ErtsMiscOpList; @@ -401,8 +382,8 @@ struct ErtsRunQueue_ { struct { ErtsRunQueueInfo info; - struct port *start; - struct port *end; + Port *start; + Port *end; } ports; }; @@ -512,7 +493,7 @@ struct ErtsSchedulerData_ { ErtsSchedulerSleepInfo *ssi; Process *current_process; Uint no; /* Scheduler number */ - struct port *current_port; + Port *current_port; ErtsRunQueue *run_queue; int virtual_reds; int cpu_id; /* >= 0 when bound */ @@ -737,8 +718,8 @@ struct ErtsPendingSuspend_ { # define BIN_OLD_VHEAP(p) (p)->bin_old_vheap struct process { - Eterm id; /* The pid of this process - (need to be first in struct) */ + ErtsPTabElementCommon common; /* *Need* to be first in struct */ + /* All fields in the PCB that differs between different heap * architectures, have been moved to the end of this struct to * make sure that as few offsets as possible differ. Different @@ -782,12 +763,8 @@ struct process { * Only valid for the current process. */ Uint32 rcount; /* suspend count */ - int prio; /* Priority of process */ int schedule_count; /* Times left to reschedule a low prio process */ Uint reds; /* No of reductions for this process */ - Eterm tracer_proc; /* If proc is traced, this is the tracer - (can NOT be boxed) */ - Uint trace_flags; /* Trace flags (used to be in flags) */ Eterm group_leader; /* Pid in charge (can be boxed) */ Uint flags; /* Trap exit, etc (no trace flags anymore) */ @@ -797,10 +774,6 @@ struct process { Process *next; /* Pointer to next process in run queue */ - struct reg_proc *reg; /* NULL iff not registered */ - ErtsLink *nlinks; - ErtsMonitor *monitors; /* The process monitors, both ends */ - struct ErtsNodesMonitor_ *nodes_monitors; ErtsSuspendMonitor *suspend_monitors; /* Processes suspended by @@ -809,7 +782,10 @@ struct process { ErlMessageQueue msg; /* Message queue */ - ErtsBifTimer *bif_timers; /* Bif timers aiming at this process */ + union { + ErtsBifTimer *bif_timers; /* Bif timers aiming at this process */ + void *terminate; + } u; ProcDict *dictionary; /* Process dictionary, may be NULL */ @@ -834,7 +810,6 @@ struct process { */ Eterm parent; /* Pid of process that created this process. */ erts_approx_time_t approx_started; /* Time when started. */ - Uint64 started_interval; /* This is the place, where all fields that differs between memory * architectures, have gone to. @@ -856,25 +831,11 @@ struct process { Uint64 bin_old_vheap_sz; /* Virtual old heap block size for binaries */ Uint64 bin_old_vheap; /* Virtual old heap size for binaries */ - union { - struct { -#ifdef ERTS_SMP - ErtsSmpPTimer *ptimer; - ErlMessageInQueue msg_inq; - ErtsPendExit pending_exit; -#else - ErlTimer tm; /* Timer entry */ -#endif - } alive; /* when process is alive */ -#ifdef ERTS_SMP - ErtsThrPrgrLaterOp release_data; /* when releasing process struct */ -#endif - void *exit_data; /* Misc data referred during termination */ - } u; - erts_smp_atomic32_t state; /* Process state flags (see ERTS_PSFLG_*) */ #ifdef ERTS_SMP + ErlMessageInQueue msg_inq; + ErtsPendExit pending_exit; erts_proc_lock_t lock; ErtsSchedulerData *scheduler_data; Eterm suspendee; @@ -904,6 +865,8 @@ struct process { #endif }; +extern const Process erts_invalid_process; + #ifdef CHECK_FOR_HOLES # define INIT_HOLE_CHECK(p) \ do { \ @@ -1028,8 +991,6 @@ Eterm* erts_heap_alloc(Process* p, Uint need, Uint xtra); Eterm* erts_set_hole_marker(Eterm* ptr, Uint sz); #endif -extern erts_smp_rwmtx_t erts_proc_tab_rwmtx; -extern erts_smp_atomic_t *erts_proc_tab; extern Uint erts_default_process_flags; extern erts_smp_rwmtx_t erts_cpu_bind_rwmtx; /* If any of the erts_system_monitor_* variables are set (enabled), @@ -1057,10 +1018,6 @@ struct erts_system_profile_flags_t { unsigned int exclusive : 1; }; extern struct erts_system_profile_flags_t erts_system_profile_flags; - -#define IS_TRACED(p) ( (p)->tracer_proc != NIL ) -#define ARE_TRACE_FLAGS_ON(p,tf) ( ((p)->trace_flags & (tf|F_SENSITIVE)) == (tf) ) -#define IS_TRACED_FL(p,tf) ( IS_TRACED(p) && ARE_TRACE_FLAGS_ON(p,tf) ) /* process flags */ #define F_HIBERNATE_SCHED (1 << 0) /* Schedule out after hibernate op */ @@ -1171,7 +1128,172 @@ Uint64 erts_step_proc_interval(void); ErtsProcList *erts_proclist_create(Process *); void erts_proclist_destroy(ErtsProcList *); -int erts_proclist_same(ErtsProcList *, Process *); + +ERTS_GLB_INLINE int erts_proclist_same(ErtsProcList *, Process *); +ERTS_GLB_INLINE void erts_proclist_store_first(ErtsProcList **, ErtsProcList *); +ERTS_GLB_INLINE void erts_proclist_store_last(ErtsProcList **, ErtsProcList *); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_first(ErtsProcList *); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_last(ErtsProcList *); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_next(ErtsProcList *, ErtsProcList *); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_prev(ErtsProcList *, ErtsProcList *); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_fetch_first(ErtsProcList **); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_fetch_last(ErtsProcList **); +ERTS_GLB_INLINE int erts_proclist_fetch(ErtsProcList **, ErtsProcList **); +ERTS_GLB_INLINE void erts_proclist_remove(ErtsProcList **, ErtsProcList *); +ERTS_GLB_INLINE int erts_proclist_is_empty(ErtsProcList *); +ERTS_GLB_INLINE int erts_proclist_is_first(ErtsProcList *, ErtsProcList *); +ERTS_GLB_INLINE int erts_proclist_is_last(ErtsProcList *, ErtsProcList *); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE int +erts_proclist_same(ErtsProcList *plp, Process *p) +{ + return (plp->pid == p->common.id + && (plp->started_interval + == p->common.u.alive.started_interval)); +} + +ERTS_GLB_INLINE void erts_proclist_store_first(ErtsProcList **list, + ErtsProcList *element) +{ + if (!*list) + element->next = element->prev = element; + else { + element->prev = (*list)->prev; + element->next = *list; + element->prev->next = element; + element->next->prev = element; + } + *list = element; +} + +ERTS_GLB_INLINE void erts_proclist_store_last(ErtsProcList **list, + ErtsProcList *element) +{ + if (!*list) { + element->next = element->prev = element; + *list = element; + } + else { + element->prev = (*list)->prev; + element->next = *list; + element->prev->next = element; + element->next->prev = element; + } +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_first(ErtsProcList *list) +{ + return list; +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_last(ErtsProcList *list) +{ + if (!list) + return NULL; + else + return list->prev; +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_next(ErtsProcList *list, + ErtsProcList *element) +{ + ErtsProcList *next; + ASSERT(list && element); + next = element->next; + return list == next ? NULL : next; +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_prev(ErtsProcList *list, + ErtsProcList *element) +{ + ErtsProcList *prev; + ASSERT(list && element); + prev = element->prev; + return list == element ? NULL : prev; +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_fetch_first(ErtsProcList **list) +{ + if (!*list) + return NULL; + else { + ErtsProcList *res = *list; + if (res == *list) + *list = NULL; + else + *list = res->next; + res->next->prev = res->prev; + res->prev->next = res->next; + return res; + } +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_fetch_last(ErtsProcList **list) +{ + if (!*list) + return NULL; + else { + ErtsProcList *res = (*list)->prev; + if (res == *list) + *list = NULL; + res->next->prev = res->prev; + res->prev->next = res->next; + return res; + } +} + +ERTS_GLB_INLINE int erts_proclist_fetch(ErtsProcList **list_first, + ErtsProcList **list_last) +{ + if (!*list_first) { + if (list_last) + *list_last = NULL; + return 0; + } + else { + if (list_last) + *list_last = (*list_first)->prev; + (*list_first)->prev->next = NULL; + (*list_first)->prev = NULL; + return !0; + } +} + +ERTS_GLB_INLINE void erts_proclist_remove(ErtsProcList **list, + ErtsProcList *element) +{ + ASSERT(list && *list); + if (*list == element) { + *list = element->next; + if (*list == element) + *list = NULL; + } + element->next->prev = element->prev; + element->prev->next = element->next; +} + +ERTS_GLB_INLINE int erts_proclist_is_empty(ErtsProcList *list) +{ + return list == NULL; +} + +ERTS_GLB_INLINE int erts_proclist_is_first(ErtsProcList *list, + ErtsProcList *element) +{ + ASSERT(list && element); + return list == element; +} + +ERTS_GLB_INLINE int erts_proclist_is_last(ErtsProcList *list, + ErtsProcList *element) +{ + ASSERT(list && element); + return list->prev == element; +} + +#endif int erts_sched_set_wakeup_other_thresold(char *str); int erts_sched_set_wakeup_other_type(char *str); @@ -1221,7 +1343,7 @@ void erts_schedule_multi_misc_aux_work(int ignore_self, erts_aint32_t erts_set_aux_work_timeout(int, erts_aint32_t, int); void erts_sched_notify_check_cpu_bind(void); Uint erts_active_schedulers(void); -void erts_init_process(int); +void erts_init_process(int, int); Eterm erts_process_status(Process *, ErtsProcLocks, Process *, Eterm); Uint erts_run_queues_len(Uint *); void erts_add_to_runq(Process *); @@ -1240,7 +1362,6 @@ void set_timer(Process*, Uint); void cancel_timer(Process*); /* Begin System profile */ Uint erts_runnable_process_count(void); -Uint erts_process_count(void); /* End System profile */ void erts_init_empty_process(Process *p); void erts_cleanup_empty_process(Process* p); @@ -1264,7 +1385,7 @@ Eterm erts_sched_stat_term(Process *p, int total); void erts_free_proc(Process *); -void erts_suspend(Process*, ErtsProcLocks, struct port*); +void erts_suspend(Process*, ErtsProcLocks, Port*); void erts_resume(Process*, ErtsProcLocks); int erts_resume_processes(ErtsProcList *); @@ -1289,9 +1410,6 @@ void erts_deep_process_dump(int, void *); Eterm erts_get_reader_groups_map(Process *c_p); Eterm erts_debug_reader_groups_map(Process *c_p, int groups); -Sint erts_test_next_pid(int, Uint); -Eterm erts_debug_processes(Process *c_p); -Eterm erts_debug_processes_bif_info(Process *c_p); Uint erts_debug_nbalance(void); int erts_debug_wait_deallocations(Process *c_p); @@ -1603,7 +1721,7 @@ ERTS_GLB_INLINE Eterm erts_get_current_pid(void) { Process *proc = erts_get_current_process(); - return proc ? proc->id : THE_NON_VALUE; + return proc ? proc->common.id : THE_NON_VALUE; } ERTS_GLB_INLINE @@ -1812,10 +1930,10 @@ extern int erts_disable_proc_not_running_opt; /* Minimum NUMBER of processes for a small system to start */ -#ifdef ERTS_SMP +#define ERTS_MIN_PROCESSES 1024 +#if defined(ERTS_SMP) && ERTS_MIN_PROCESSES < ERTS_NO_OF_PIX_LOCKS +#undef ERTS_MIN_PROCESSES #define ERTS_MIN_PROCESSES ERTS_NO_OF_PIX_LOCKS -#else -#define ERTS_MIN_PROCESSES 16 #endif void erts_smp_notify_inc_runq(ErtsRunQueue *runq); diff --git a/erts/emulator/beam/erl_process_dict.c b/erts/emulator/beam/erl_process_dict.c index 93466da3aa..bf384c66e1 100644 --- a/erts/emulator/beam/erl_process_dict.c +++ b/erts/emulator/beam/erl_process_dict.c @@ -360,7 +360,7 @@ static void pd_hash_erase(Process *p, Eterm id, Eterm *ret) erts_fprintf(stderr, "Process dictionary for process %T is broken, trying to " "display term found in line %d:\n" - "%T\n", p->id, __LINE__, old); + "%T\n", p->common.id, __LINE__, old); #endif erl_exit(1, "Damaged process dictionary found during erase/1."); } @@ -405,7 +405,7 @@ Eterm erts_pd_hash_get(Process *p, Eterm id) erts_fprintf(stderr, "Process dictionary for process %T is broken, trying to " "display term found in line %d:\n" - "%T\n", p->id, __LINE__, tmp); + "%T\n", p->common.id, __LINE__, tmp); #endif erl_exit(1, "Damaged process dictionary found during get/1."); } @@ -614,7 +614,7 @@ static Eterm pd_hash_put(Process *p, Eterm id, Eterm value) erts_fprintf(stderr, "Process dictionary for process %T is broken, trying to " "display term found in line %d:\n" - "%T\n", p->id, __LINE__, old); + "%T\n", p->common.id, __LINE__, old); #endif erl_exit(1, "Damaged process dictionary found during put/2."); diff --git a/erts/emulator/beam/erl_process_dump.c b/erts/emulator/beam/erl_process_dump.c index 542c5ed0d9..ba74dfd6a1 100644 --- a/erts/emulator/beam/erl_process_dump.c +++ b/erts/emulator/beam/erl_process_dump.c @@ -60,11 +60,11 @@ extern BeamInstr beam_continue_exit[]; void erts_deep_process_dump(int to, void *to_arg) { - int i; + int i, max = erts_ptab_max(&erts_proc); all_binaries = NULL; - for (i = 0; i < erts_max_processes; i++) { + for (i = 0; i < max; i++) { Process *p = erts_pix2proc(i); if (p && p->i != ENULL) { erts_aint32_t state = erts_smp_atomic32_read_acqb(&p->state); @@ -85,8 +85,8 @@ dump_process_info(int to, void *to_arg, Process *p) ERTS_SMP_MSGQ_MV_INQ2PRIVQ(p); - if ((p->trace_flags & F_SENSITIVE) == 0 && p->msg.first) { - erts_print(to, to_arg, "=proc_messages:%T\n", p->id); + if ((ERTS_TRACE_FLAGS(p) & F_SENSITIVE) == 0 && p->msg.first) { + erts_print(to, to_arg, "=proc_messages:%T\n", p->common.id); for (mp = p->msg.first; mp != NULL; mp = mp->next) { Eterm mesg = ERL_MESSAGE_TERM(mp); if (is_value(mesg)) @@ -100,21 +100,21 @@ dump_process_info(int to, void *to_arg, Process *p) } } - if ((p->trace_flags & F_SENSITIVE) == 0) { + if ((ERTS_TRACE_FLAGS(p) & F_SENSITIVE) == 0) { if (p->dictionary) { - erts_print(to, to_arg, "=proc_dictionary:%T\n", p->id); + erts_print(to, to_arg, "=proc_dictionary:%T\n", p->common.id); erts_deep_dictionary_dump(to, to_arg, p->dictionary, dump_element_nl); } } - if ((p->trace_flags & F_SENSITIVE) == 0) { - erts_print(to, to_arg, "=proc_stack:%T\n", p->id); + if ((ERTS_TRACE_FLAGS(p) & F_SENSITIVE) == 0) { + erts_print(to, to_arg, "=proc_stack:%T\n", p->common.id); for (sp = p->stop; sp < STACK_START(p); sp++) { yreg = stack_element_dump(to, to_arg, p, sp, yreg); } - erts_print(to, to_arg, "=proc_heap:%T\n", p->id); + erts_print(to, to_arg, "=proc_heap:%T\n", p->common.id); for (sp = p->stop; sp < STACK_START(p); sp++) { Eterm term = *sp; diff --git a/erts/emulator/beam/erl_process_lock.c b/erts/emulator/beam/erl_process_lock.c index 84a8270d06..2db5df06b4 100644 --- a/erts/emulator/beam/erl_process_lock.c +++ b/erts/emulator/beam/erl_process_lock.c @@ -66,8 +66,7 @@ #endif #include "erl_process.h" - -const Process erts_proc_lock_busy = {ERTS_INVALID_PID}; +#include "erl_thr_progress.h" #ifdef ERTS_SMP @@ -399,7 +398,7 @@ wait_for_locks(Process *p, ErtsProcLocks need_locks, ErtsProcLocks olflgs) { - erts_pix_lock_t *pix_lock = pixlck ? pixlck : ERTS_PID2PIXLOCK(p->id); + erts_pix_lock_t *pix_lock = pixlck ? pixlck : ERTS_PID2PIXLOCK(p->common.id); erts_tse_t *wtr; /* Acquire a waiter object on which this thread can wait. */ @@ -553,7 +552,7 @@ erts_proc_unlock_failed(Process *p, erts_pix_lock_t *pixlck, ErtsProcLocks wait_locks) { - erts_pix_lock_t *pix_lock = pixlck ? pixlck : ERTS_PID2PIXLOCK(p->id); + erts_pix_lock_t *pix_lock = pixlck ? pixlck : ERTS_PID2PIXLOCK(p->common.id); #if ERTS_PROC_LOCK_ATOMIC_IMPL erts_pix_lock(pix_lock); @@ -580,7 +579,7 @@ erts_proc_lock_prepare_proc_lock_waiter(void) */ static void -proc_safelock(int is_sched, +proc_safelock(int is_managed, Process *a_proc, ErtsProcLocks a_have_locks, ErtsProcLocks a_need_locks, @@ -603,40 +602,40 @@ proc_safelock(int is_sched, * Locks with the same lock order should be locked on p1 before p2. */ if (a_proc) { - if (a_proc->id < b_proc->id) { + if (a_proc->common.id < b_proc->common.id) { p1 = a_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid1 = a_proc->id; + pid1 = a_proc->common.id; #endif need_locks1 = a_need_locks; have_locks1 = a_have_locks; p2 = b_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid2 = b_proc->id; + pid2 = b_proc->common.id; #endif need_locks2 = b_need_locks; have_locks2 = b_have_locks; } - else if (a_proc->id > b_proc->id) { + else if (a_proc->common.id > b_proc->common.id) { p1 = b_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid1 = b_proc->id; + pid1 = b_proc->common.id; #endif need_locks1 = b_need_locks; have_locks1 = b_have_locks; p2 = a_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid2 = a_proc->id; + pid2 = a_proc->common.id; #endif need_locks2 = a_need_locks; have_locks2 = a_have_locks; } else { ERTS_LC_ASSERT(a_proc == b_proc); - ERTS_LC_ASSERT(a_proc->id == b_proc->id); + ERTS_LC_ASSERT(a_proc->common.id == b_proc->common.id); p1 = a_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid1 = a_proc->id; + pid1 = a_proc->common.id; #endif need_locks1 = a_need_locks | b_need_locks; have_locks1 = a_have_locks | b_have_locks; @@ -651,7 +650,7 @@ proc_safelock(int is_sched, else { p1 = b_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid1 = b_proc->id; + pid1 = b_proc->common.id; #endif need_locks1 = b_need_locks; have_locks1 = b_have_locks; @@ -706,7 +705,7 @@ proc_safelock(int is_sched, if (unlock_locks) { have_locks1 &= ~unlock_locks; need_locks1 |= unlock_locks; - if (!is_sched && !have_locks1) { + if (!is_managed && !have_locks1) { refc1 = 1; erts_smp_proc_inc_refc(p1); } @@ -716,7 +715,7 @@ proc_safelock(int is_sched, if (unlock_locks) { have_locks2 &= ~unlock_locks; need_locks2 |= unlock_locks; - if (!is_sched && !have_locks2) { + if (!is_managed && !have_locks2) { refc2 = 1; erts_smp_proc_inc_refc(p2); } @@ -797,7 +796,7 @@ proc_safelock(int is_sched, } #endif - if (!is_sched) { + if (!is_managed) { if (refc1) erts_smp_proc_dec_refc(p1); if (refc2) @@ -830,7 +829,7 @@ erts_pid2proc_opt(Process *c_p, int flags) { Process *dec_refc_proc = NULL; - int need_ptl; + ErtsThrPrgrDelayHandle dhndl; ErtsProcLocks need_locks; Uint pix; Process *proc; @@ -853,8 +852,8 @@ erts_pid2proc_opt(Process *c_p, ERTS_LC_ASSERT((pid_need_locks & ERTS_PROC_LOCKS_ALL) == pid_need_locks); need_locks = pid_need_locks; - if (c_p && c_p->id == pid) { - ASSERT(c_p->id != ERTS_INVALID_PID); + if (c_p && c_p->common.id == pid) { + ASSERT(c_p->common.id != ERTS_INVALID_PID); ASSERT(c_p == erts_pix2proc(pix)); if (!(flags & ERTS_P2P_FLG_ALLOW_OTHER_X) @@ -868,15 +867,12 @@ erts_pid2proc_opt(Process *c_p, } } - need_ptl = !erts_get_scheduler_id(); - - if (need_ptl) - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); + dhndl = erts_thr_progress_unmanaged_delay(); - proc = (Process *) erts_smp_atomic_read_ddrb(&erts_proc.tab[pix]); + proc = (Process *) erts_ptab_pix2intptr_ddrb(&erts_proc, pix); if (proc) { - if (proc->id != pid) + if (proc->common.id != pid) proc = NULL; else if (!need_locks) { if (flags & ERTS_P2P_FLG_SMP_INC_REFC) @@ -935,6 +931,7 @@ erts_pid2proc_opt(Process *c_p, if (flags & ERTS_P2P_FLG_TRY_LOCK) proc = ERTS_PROC_LOCK_BUSY; else { + int managed; if (flags & ERTS_P2P_FLG_SMP_INC_REFC) erts_smp_proc_inc_refc(proc); @@ -942,14 +939,21 @@ erts_pid2proc_opt(Process *c_p, erts_lcnt_proc_lock_unaquire(&proc->lock, lcnt_locks); #endif - if (need_ptl) { + managed = dhndl == ERTS_THR_PRGR_DHANDLE_MANAGED; + if (!managed) { erts_smp_proc_inc_refc(proc); + erts_thr_progress_unmanaged_continue(dhndl); dec_refc_proc = proc; - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - need_ptl = 0; + + /* + * We don't want to call + * erts_thr_progress_unmanaged_continue() + * again. + */ + dhndl = ERTS_THR_PRGR_DHANDLE_MANAGED; } - proc_safelock(!need_ptl, + proc_safelock(managed, c_p, c_p_have_locks, c_p_have_locks, @@ -961,8 +965,8 @@ erts_pid2proc_opt(Process *c_p, } } - if (need_ptl) - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue(dhndl); if (need_locks && proc @@ -970,7 +974,7 @@ erts_pid2proc_opt(Process *c_p, && (!(flags & ERTS_P2P_FLG_ALLOW_OTHER_X) ? ERTS_PROC_IS_EXITING(proc) : (proc - != (Process *) erts_smp_atomic_read_nob(&erts_proc.tab[pix])))) { + != (Process *) erts_ptab_pix2intptr_nob(&erts_proc, pix)))) { erts_smp_proc_unlock(proc, need_locks); @@ -1012,22 +1016,22 @@ erts_proc_lock_init(Process *p) erts_proc_lc_trylock(p, ERTS_PROC_LOCKS_ALL, 1); #endif #elif ERTS_PROC_LOCK_RAW_MUTEX_IMPL - erts_mtx_init_x(&p->lock.main, "proc_main", p->id); + erts_mtx_init_x(&p->lock.main, "proc_main", p->common.id); ethr_mutex_lock(&p->lock.main.mtx); #ifdef ERTS_ENABLE_LOCK_CHECK erts_lc_trylock(1, &p->lock.main.lc); #endif - erts_mtx_init_x(&p->lock.link, "proc_link", p->id); + erts_mtx_init_x(&p->lock.link, "proc_link", p->common.id); ethr_mutex_lock(&p->lock.link.mtx); #ifdef ERTS_ENABLE_LOCK_CHECK erts_lc_trylock(1, &p->lock.link.lc); #endif - erts_mtx_init_x(&p->lock.msgq, "proc_msgq", p->id); + erts_mtx_init_x(&p->lock.msgq, "proc_msgq", p->common.id); ethr_mutex_lock(&p->lock.msgq.mtx); #ifdef ERTS_ENABLE_LOCK_CHECK erts_lc_trylock(1, &p->lock.msgq.lc); #endif - erts_mtx_init_x(&p->lock.status, "proc_status", p->id); + erts_mtx_init_x(&p->lock.status, "proc_status", p->common.id); ethr_mutex_lock(&p->lock.status.mtx); #ifdef ERTS_ENABLE_LOCK_CHECK erts_lc_trylock(1, &p->lock.status.lc); @@ -1064,11 +1068,11 @@ erts_proc_lock_fin(Process *p) #if ERTS_PROC_LOCK_OWN_IMPL && defined(ERTS_ENABLE_LOCK_COUNT) void erts_lcnt_proc_lock_init(Process *p) { if (erts_lcnt_rt_options & ERTS_LCNT_OPT_PROCLOCK) { - if (p->id != ERTS_INVALID_PID) { - erts_lcnt_init_lock_x(&(p->lock.lcnt_main), "proc_main", ERTS_LCNT_LT_PROCLOCK, p->id); - erts_lcnt_init_lock_x(&(p->lock.lcnt_msgq), "proc_msgq", ERTS_LCNT_LT_PROCLOCK, p->id); - erts_lcnt_init_lock_x(&(p->lock.lcnt_link), "proc_link", ERTS_LCNT_LT_PROCLOCK, p->id); - erts_lcnt_init_lock_x(&(p->lock.lcnt_status), "proc_status", ERTS_LCNT_LT_PROCLOCK, p->id); + if (p->common.id != ERTS_INVALID_PID) { + erts_lcnt_init_lock_x(&(p->lock.lcnt_main), "proc_main", ERTS_LCNT_LT_PROCLOCK, p->common.id); + erts_lcnt_init_lock_x(&(p->lock.lcnt_msgq), "proc_msgq", ERTS_LCNT_LT_PROCLOCK, p->common.id); + erts_lcnt_init_lock_x(&(p->lock.lcnt_link), "proc_link", ERTS_LCNT_LT_PROCLOCK, p->common.id); + erts_lcnt_init_lock_x(&(p->lock.lcnt_status), "proc_status", ERTS_LCNT_LT_PROCLOCK, p->common.id); } else { erts_lcnt_init_lock(&(p->lock.lcnt_main), "proc_main", ERTS_LCNT_LT_PROCLOCK); erts_lcnt_init_lock(&(p->lock.lcnt_msgq), "proc_msgq", ERTS_LCNT_LT_PROCLOCK); @@ -1176,10 +1180,11 @@ void erts_lcnt_proc_trylock(erts_proc_lock_t *lock, ErtsProcLocks locks, int res } -void erts_lcnt_enable_proc_lock_count(int enable) { - int i; +void erts_lcnt_enable_proc_lock_count(int enable) +{ + int i, max = erts_ptab_max(&erts_proc); - for (i = 0; i < erts_max_processes; ++i) { + for (i = 0; i < max; ++i) { Process* p = erts_pix2proc(i); if (p) { if (enable) { @@ -1208,7 +1213,7 @@ void erts_proc_lc_lock(Process *p, ErtsProcLocks locks) { erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_MAIN) { lck.id = lc_id.proc_lock_main; @@ -1232,7 +1237,7 @@ void erts_proc_lc_trylock(Process *p, ErtsProcLocks locks, int locked) { erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_MAIN) { lck.id = lc_id.proc_lock_main; @@ -1256,7 +1261,7 @@ void erts_proc_lc_unlock(Process *p, ErtsProcLocks locks) { erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_STATUS) { lck.id = lc_id.proc_lock_status; @@ -1283,7 +1288,7 @@ erts_proc_lc_might_unlock(Process *p, ErtsProcLocks locks) { #if ERTS_PROC_LOCK_OWN_IMPL erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_STATUS) { lck.id = lc_id.proc_lock_status; @@ -1318,7 +1323,7 @@ erts_proc_lc_require_lock(Process *p, ErtsProcLocks locks) { #if ERTS_PROC_LOCK_OWN_IMPL erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_MAIN) { lck.id = lc_id.proc_lock_main; @@ -1353,7 +1358,7 @@ erts_proc_lc_unrequire_lock(Process *p, ErtsProcLocks locks) { #if ERTS_PROC_LOCK_OWN_IMPL erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_STATUS) { lck.id = lc_id.proc_lock_status; @@ -1390,7 +1395,7 @@ erts_proc_lc_trylock_force_busy(Process *p, ErtsProcLocks locks) { if (locks & ERTS_PROC_LOCKS_ALL) { erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_MAIN) @@ -1415,7 +1420,7 @@ void erts_proc_lc_chk_only_proc_main(Process *p) { #if ERTS_PROC_LOCK_OWN_IMPL erts_lc_lock_t proc_main = ERTS_LC_LOCK_INIT(lc_id.proc_lock_main, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); erts_lc_check_exact(&proc_main, 1); #elif ERTS_PROC_LOCK_RAW_MUTEX_IMPL @@ -1439,19 +1444,19 @@ erts_proc_lc_chk_have_proc_locks(Process *p, ErtsProcLocks locks) ERTS_PROC_LC_EMPTY_LOCK_INIT}; if (locks & ERTS_PROC_LOCK_MAIN) { have_locks[have_locks_len].id = lc_id.proc_lock_main; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_LINK) { have_locks[have_locks_len].id = lc_id.proc_lock_link; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_MSGQ) { have_locks[have_locks_len].id = lc_id.proc_lock_msgq; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_STATUS) { have_locks[have_locks_len].id = lc_id.proc_lock_status; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } #elif ERTS_PROC_LOCK_RAW_MUTEX_IMPL erts_lc_lock_t have_locks[4]; @@ -1484,35 +1489,35 @@ erts_proc_lc_chk_proc_locks(Process *p, ErtsProcLocks locks) if (locks & ERTS_PROC_LOCK_MAIN) { have_locks[have_locks_len].id = lc_id.proc_lock_main; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } else { have_not_locks[have_not_locks_len].id = lc_id.proc_lock_main; - have_not_locks[have_not_locks_len++].extra = p->id; + have_not_locks[have_not_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_LINK) { have_locks[have_locks_len].id = lc_id.proc_lock_link; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } else { have_not_locks[have_not_locks_len].id = lc_id.proc_lock_link; - have_not_locks[have_not_locks_len++].extra = p->id; + have_not_locks[have_not_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_MSGQ) { have_locks[have_locks_len].id = lc_id.proc_lock_msgq; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } else { have_not_locks[have_not_locks_len].id = lc_id.proc_lock_msgq; - have_not_locks[have_not_locks_len++].extra = p->id; + have_not_locks[have_not_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_STATUS) { have_locks[have_locks_len].id = lc_id.proc_lock_status; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } else { have_not_locks[have_not_locks_len].id = lc_id.proc_lock_status; - have_not_locks[have_not_locks_len++].extra = p->id; + have_not_locks[have_not_locks_len++].extra = p->common.id; } #elif ERTS_PROC_LOCK_RAW_MUTEX_IMPL erts_lc_lock_t have_locks[4]; @@ -1547,16 +1552,16 @@ erts_proc_lc_my_proc_locks(Process *p) ErtsProcLocks res = 0; #if ERTS_PROC_LOCK_OWN_IMPL erts_lc_lock_t locks[4] = {ERTS_LC_LOCK_INIT(lc_id.proc_lock_main, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK), ERTS_LC_LOCK_INIT(lc_id.proc_lock_link, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK), ERTS_LC_LOCK_INIT(lc_id.proc_lock_msgq, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK), ERTS_LC_LOCK_INIT(lc_id.proc_lock_status, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK)}; #elif ERTS_PROC_LOCK_RAW_MUTEX_IMPL erts_lc_lock_t locks[4] = {p->lock.main.lc, diff --git a/erts/emulator/beam/erl_process_lock.h b/erts/emulator/beam/erl_process_lock.h index 4aec19c8c3..9dd503f3cb 100644 --- a/erts/emulator/beam/erl_process_lock.h +++ b/erts/emulator/beam/erl_process_lock.h @@ -176,8 +176,8 @@ typedef struct erts_proc_lock_t_ { * on multiple processes, locks on processes with low process ids * have to be locked before locks on processes with high process * ids. E.g., if the main and the message queue locks are to be - * locked on processes p1 and p2 and p1->id < p2->id, then locks - * should be locked in the following order: + * locked on processes p1 and p2 and p1->common.id < p2->common.id, + * then locks should be locked in the following order: * 1. main lock on p1 * 2. main lock on p2 * 3. message queue lock on p1 @@ -203,7 +203,7 @@ typedef struct erts_proc_lock_t_ { & ~ERTS_PROC_LOCK_MAIN) -#define ERTS_PIX_LOCKS_BITS 8 +#define ERTS_PIX_LOCKS_BITS 10 #define ERTS_NO_OF_PIX_LOCKS (1 << ERTS_PIX_LOCKS_BITS) @@ -767,7 +767,7 @@ erts_smp_proc_lock(Process *p, ErtsProcLocks locks) #if ERTS_PROC_LOCK_ATOMIC_IMPL NULL, #else - ERTS_PID2PIXLOCK(p->id), + ERTS_PID2PIXLOCK(p->common.id), #endif /*ERTS_PROC_LOCK_ATOMIC_IMPL*/ locks, file, line); #elif defined(ERTS_SMP) @@ -775,7 +775,7 @@ erts_smp_proc_lock(Process *p, ErtsProcLocks locks) #if ERTS_PROC_LOCK_ATOMIC_IMPL NULL, #else - ERTS_PID2PIXLOCK(p->id), + ERTS_PID2PIXLOCK(p->common.id), #endif /*ERTS_PROC_LOCK_ATOMIC_IMPL*/ locks); #endif /*ERTS_SMP*/ @@ -789,7 +789,7 @@ erts_smp_proc_unlock(Process *p, ErtsProcLocks locks) #if ERTS_PROC_LOCK_ATOMIC_IMPL NULL, #else - ERTS_PID2PIXLOCK(p->id), + ERTS_PID2PIXLOCK(p->common.id), #endif locks); #endif @@ -805,7 +805,7 @@ erts_smp_proc_trylock(Process *p, ErtsProcLocks locks) #if ERTS_PROC_LOCK_ATOMIC_IMPL NULL, #else - ERTS_PID2PIXLOCK(p->id), + ERTS_PID2PIXLOCK(p->common.id), #endif locks); #endif @@ -814,21 +814,15 @@ erts_smp_proc_trylock(Process *p, ErtsProcLocks locks) ERTS_GLB_INLINE void erts_smp_proc_inc_refc(Process *p) { #ifdef ERTS_SMP -#ifdef ERTS_ENABLE_LOCK_CHECK - erts_aint32_t refc = erts_atomic32_inc_read_nob(&p->lock.refc); - ERTS_SMP_LC_ASSERT(refc > 1); -#else - erts_atomic32_inc_nob(&p->lock.refc); -#endif + erts_ptab_inc_refc(&p->common); #endif } ERTS_GLB_INLINE void erts_smp_proc_dec_refc(Process *p) { #ifdef ERTS_SMP - erts_aint32_t refc = erts_atomic32_dec_read_nob(&p->lock.refc); - ERTS_SMP_LC_ASSERT(refc >= 0); - if (refc == 0) + int referred = erts_ptab_dec_test_refc(&p->common); + if (!referred) erts_free_proc(p); #endif } @@ -836,10 +830,8 @@ ERTS_GLB_INLINE void erts_smp_proc_dec_refc(Process *p) ERTS_GLB_INLINE void erts_smp_proc_add_refc(Process *p, Sint32 add_refc) { #ifdef ERTS_SMP - erts_aint32_t refc = erts_atomic32_add_read_nob(&p->lock.refc, - (erts_aint32_t) add_refc); - ERTS_SMP_LC_ASSERT(refc >= 0); - if (refc == 0) + int referred = erts_ptab_add_test_refc(&p->common, add_refc); + if (!referred) erts_free_proc(p); #endif } @@ -875,8 +867,7 @@ void erts_proc_safelock(Process *a_proc, #define ERTS_P2P_FLG_TRY_LOCK (1 << 1) #define ERTS_P2P_FLG_SMP_INC_REFC (1 << 2) -#define ERTS_PROC_LOCK_BUSY ((Process *) &erts_proc_lock_busy) -extern const Process erts_proc_lock_busy; +#define ERTS_PROC_LOCK_BUSY ((Process *) &erts_invalid_process) #define erts_pid2proc(PROC, HL, PID, NL) \ erts_pid2proc_opt((PROC), (HL), (PID), (NL), 0) @@ -896,33 +887,24 @@ Process *erts_pid2proc_opt(Process *, ErtsProcLocks, Eterm, ErtsProcLocks, int); ERTS_GLB_INLINE Process *erts_pix2proc(int ix) { Process *proc; - ASSERT(0 <= ix && ix < erts_proc.max); - proc = (Process *) erts_smp_atomic_read_nob(&erts_proc.tab[ix]); + ASSERT(0 <= ix && ix < erts_ptab_max(&erts_proc)); + proc = (Process *) erts_ptab_pix2intptr_nob(&erts_proc, ix); return proc == ERTS_PROC_LOCK_BUSY ? NULL : proc; } ERTS_GLB_INLINE Process *erts_proc_lookup_raw(Eterm pid) { Process *proc; - int pix; - /* - * In SMP case: Only scheduler threads are allowed - * to use this function. Other threads need to - * atomicaly increment refc at lookup, i.e., use - * erts_pid2proc_opt() with ERTS_P2P_FLG_SMP_INC_REFC. - */ - ERTS_SMP_LC_ASSERT(erts_get_scheduler_id()); + ERTS_SMP_LC_ASSERT(erts_thr_progress_lc_is_delaying()); if (is_not_internal_pid(pid)) return NULL; - pix = internal_pid_index(pid); - proc = (Process *) erts_smp_atomic_read_ddrb(&erts_proc.tab[pix]); - - if (proc && proc->id != pid) + proc = (Process *) erts_ptab_pix2intptr_ddrb(&erts_proc, + internal_pid_index(pid)); + if (proc && proc->common.id != pid) return NULL; - return proc; } diff --git a/erts/emulator/beam/erl_ptab.c b/erts/emulator/beam/erl_ptab.c new file mode 100644 index 0000000000..87beeafa1a --- /dev/null +++ b/erts/emulator/beam/erl_ptab.c @@ -0,0 +1,1566 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +/* + * Description: Process/Port table implementation. + * + * Author: Rickard Green + */ + +#ifdef HAVE_CONFIG_H +# include "config.h" +#endif +#define ERTS_PTAB_WANT_BIF_IMPL__ +#define ERTS_PTAB_WANT_DEBUG_FUNCS__ +#include "erl_ptab.h" +#include "global.h" +#include "erl_binary.h" + +typedef struct ErtsPTabListBifData_ ErtsPTabListBifData; + +#define ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED 25 +#define ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE 1000 +#define ERTS_PTAB_LIST_BIF_MIN_START_REDS \ + (ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE \ + / ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED) + +#define ERTS_PTAB_LIST_BIF_TAB_FREE_DELETED_REDS 1 + +#define ERTS_PTAB_LIST_BIF_INSPECT_DELETED_PER_RED 10 + +#define ERTS_PTAB_LIST_INSPECT_DELETED_MAX_REDS \ + (ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE \ + / ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED) + + +#define ERTS_PTAB_LIST_BIF_BUILD_RESULT_CONSES_PER_RED 75 + +#define ERTS_PTAB_LIST_DBG_DO_TRACE 0 + +#ifdef DEBUG +# define ERTS_PTAB_LIST_BIF_DEBUGLEVEL 100 +#else +# define ERTS_PTAB_LIST_BIF_DEBUGLEVEL 0 +#endif + +#define ERTS_PTAB_LIST_DBGLVL_CHK_HALLOC 1 +#define ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS 5 +#define ERTS_PTAB_LIST_DBGLVL_CHK_PIDS 10 +#define ERTS_PTAB_LIST_DBGLVL_CHK_DEL_LIST 20 +#define ERTS_PTAB_LIST_DBGLVL_CHK_RESLIST 20 + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL == 0 +# define ERTS_PTAB_LIST_ASSERT(EXP) +#else +# define ERTS_PTAB_LIST_ASSERT(EXP) \ + ((void) ((EXP) \ + ? 1 \ + : (debug_ptab_list_assert_error(#EXP, \ + __FILE__, \ + __LINE__, \ + __func__), \ + 0))) +#endif + + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_HALLOC +# define ERTS_PTAB_LIST_DBG_SAVE_HEAP_ALLOC(PTLBDP, HP, SZ) \ +do { \ + ERTS_PTAB_LIST_ASSERT(!(PTLBDP)->debug.heap); \ + ERTS_PTAB_LIST_ASSERT(!(PTLBDP)->debug.heap_size); \ + (PTLBDP)->debug.heap = (HP); \ + (PTLBDP)->debug.heap_size = (SZ); \ +} while (0) +# define ERTS_PTAB_LIST_DBG_VERIFY_HEAP_ALLOC_USED(PTLBDP, HP) \ +do { \ + ERTS_PTAB_LIST_ASSERT((PTLBDP)->debug.heap); \ + ERTS_PTAB_LIST_ASSERT((PTLBDP)->debug.heap_size); \ + ERTS_PTAB_LIST_ASSERT(((PTLBDP)->debug.heap \ + + (PTLBDP)->debug.heap_size) \ + == (HP)); \ + (PTLBDP)->debug.heap = NULL; \ + (PTLBDP)->debug.heap_size = 0; \ +} while (0) +# define ERTS_PTAB_LIST_DBG_HEAP_ALLOC_INIT(PTLBDP) \ +do { \ + (PTLBDP)->debug.heap = NULL; \ + (PTLBDP)->debug.heap_size = 0; \ +} while (0) +#else +# define ERTS_PTAB_LIST_DBG_SAVE_HEAP_ALLOC(PTLBDP, HP, SZ) +# define ERTS_PTAB_LIST_DBG_VERIFY_HEAP_ALLOC_USED(PTLBDP, HP) +# define ERTS_PTAB_LIST_DBG_HEAP_ALLOC_INIT(PTLBDP) +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_RESLIST +# define ERTS_PTAB_LIST_DBG_CHK_RESLIST(R) \ + debug_ptab_list_check_res_list((R)) +#else +# define ERTS_PTAB_LIST_DBG_CHK_RESLIST(R) +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_PIDS +# define ERTS_PTAB_LIST_DBG_SAVE_PIDS(PTLBDP) \ + debug_ptab_list_save_all_pids((PTLBDP)) +# define ERTS_PTAB_LIST_DBG_VERIFY_PIDS(PTLBDP) \ +do { \ + if (!(PTLBDP)->debug.correct_pids_verified) \ + debug_ptab_list_verify_all_pids((PTLBDP)); \ +} while (0) +# define ERTS_PTAB_LIST_DBG_CLEANUP_CHK_PIDS(PTLBDP) \ +do { \ + if ((PTLBDP)->debug.correct_pids) { \ + erts_free(ERTS_ALC_T_PTAB_LIST_PIDS, \ + (PTLBDP)->debug.correct_pids); \ + (PTLBDP)->debug.correct_pids = NULL; \ + } \ +} while(0) +# define ERTS_PTAB_LIST_DBG_CHK_PIDS_INIT(PTLBDP) \ +do { \ + (PTLBDP)->debug.correct_pids_verified = 0; \ + (PTLBDP)->debug.correct_pids = NULL; \ +} while (0) +#else +# define ERTS_PTAB_LIST_DBG_SAVE_PIDS(PTLBDP) +# define ERTS_PTAB_LIST_DBG_VERIFY_PIDS(PTLBDP) +# define ERTS_PTAB_LIST_DBG_CLEANUP_CHK_PIDS(PTLBDP) +# define ERTS_PTAB_LIST_DBG_CHK_PIDS_INIT(PTLBDP) +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS +# define ERTS_PTAB_LIST_DBG_CHK_PID_FOUND(PTLBDP, PID, IC) \ + debug_ptab_list_check_found_pid((PTLBDP), (PID), (IC), 1) +# define ERTS_PTAB_LIST_DBG_CHK_PID_NOT_FOUND(PTLBDP, PID, IC) \ + debug_ptab_list_check_found_pid((PTLBDP), (PID), (IC), 0) +#else +# define ERTS_PTAB_LIST_DBG_CHK_PID_FOUND(PTLBDP, PID, IC) +# define ERTS_PTAB_LIST_DBG_CHK_PID_NOT_FOUND(PTLBDP, PID, IC) +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_DEL_LIST +# define ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(PTab) \ + debug_ptab_list_check_del_list((PTab)) +# define ERTS_PTAB_LIST_DBG_CHK_FREELIST(PTab, FL) \ + debug_ptab_list_check_del_free_list((PTab), (FL)) +#else +# define ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(PTab) +# define ERTS_PTAB_LIST_DBG_CHK_FREELIST(PTab, FL) +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL == 0 +#if ERTS_PTAB_LIST_DBG_DO_TRACE +# define ERTS_PTAB_LIST_DBG_INIT(P, PTLBDP) \ + (PTLBDP)->debug.caller = (P)->common.id +# else +# define ERTS_PTAB_LIST_DBG_INIT(P, PTLBDP) +# endif +# define ERTS_PTAB_LIST_DBG_CLEANUP(PTLBDP) +#else +# define ERTS_PTAB_LIST_DBG_INIT(P, PTLBDP) \ +do { \ + (PTLBDP)->debug.caller = (P)->common.id; \ + ERTS_PTAB_LIST_DBG_HEAP_ALLOC_INIT((PTLBDP)); \ + ERTS_PTAB_LIST_DBG_CHK_PIDS_INIT((PTLBDP)); \ +} while (0) +# define ERTS_PTAB_LIST_DBG_CLEANUP(PTLBDP) \ +do { \ + ERTS_PTAB_LIST_DBG_CLEANUP_CHK_PIDS((PTLBDP)); \ +} while (0) +#endif + +#if ERTS_PTAB_LIST_DBG_DO_TRACE +# define ERTS_PTAB_LIST_DBG_TRACE(PID, WHAT) \ + erts_fprintf(stderr, "%T %s:%d:%s(): %s\n", \ + (PID), __FILE__, __LINE__, __func__, #WHAT) +#else +# define ERTS_PTAB_LIST_DBG_TRACE(PID, WHAT) +#endif + + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL != 0 +static void debug_ptab_list_assert_error(char* expr, + const char* file, + int line, + const char *func); +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_RESLIST +static void debug_ptab_list_check_res_list(Eterm list); +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_PIDS +static void debug_ptab_list_save_all_pids(ErtsPTabListBifData *ptlbdp); +static void debug_ptab_list_verify_all_pids(ErtsPTabListBifData *ptlbdp); +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS +static void debug_ptab_list_check_found_pid(ErtsPTabListBifData *ptlbdp, + Eterm pid, + Uint64 ic, + int pid_should_be_found); +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_DEL_LIST +static void debug_ptab_list_check_del_list(ErtsPTab *ptab); +static void debug_ptab_list_check_del_free_list(ErtsPTab *ptab, + ErtsPTabDeletedElement *ptdep); +#endif + +struct ErtsPTabDeletedElement_ { + ErtsPTabDeletedElement *next; + ErtsPTabDeletedElement *prev; + int ix; + union { + struct { + Eterm id; + Uint64 inserted; + Uint64 deleted; + } element; + struct { + Uint64 interval; + } bif_invocation; + } u; +}; + +static Export ptab_list_continue_export; + +typedef struct { + Uint64 interval; +} ErtsPTabListBifChunkInfo; + +typedef enum { + INITIALIZING, + INSPECTING_TABLE, + INSPECTING_DELETED, + BUILDING_RESULT, + RETURN_RESULT +} ErtsPTabListBifState; + +struct ErtsPTabListBifData_ { + ErtsPTab *ptab; + ErtsPTabListBifState state; + Eterm caller; + ErtsPTabListBifChunkInfo *chunk; + int tix; + int pid_ix; + int pid_sz; + Eterm *pid; + ErtsPTabDeletedElement *bif_invocation; /* Only used when > 1 chunk */ + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL != 0 || ERTS_PTAB_LIST_DBG_DO_TRACE + struct { + Eterm caller; +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + Uint64 *pid_started; +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_HALLOC + Eterm *heap; + Uint heap_size; +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_PIDS + int correct_pids_verified; + Eterm *correct_pids; +#endif + } debug; +#endif + +}; + +#ifdef ARCH_32 + +static ERTS_INLINE Uint64 +dw_aint_to_uint64(erts_dw_aint_t *dw) +{ +#ifdef ETHR_SU_DW_NAINT_T__ + return (Uint64) dw->dw_sint; +#else + Uint64 res; + res = (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_HIGH_WORD]); + res <<= 32; + res |= (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_LOW_WORD]); + return res; +#endif +} + +static void +unint64_to_dw_aint(erts_dw_aint_t *dw, Uint64 val) +{ +#ifdef ETHR_SU_DW_NAINT_T__ + dw->dw_sint = (ETHR_SU_DW_NAINT_T__) val; +#else + dw->sint[ERTS_DW_AINT_LOW_WORD] = (erts_aint_t) (val & 0xffffffff); + dw->sint[ERTS_DW_AINT_HIGH_WORD] = (erts_aint_t) ((val >> 32) & 0xffffffff); +#endif +} + +static ERTS_INLINE void +last_data_init_nob(ErtsPTab *ptab, Uint64 val) +{ + erts_dw_aint_t dw; + unint64_to_dw_aint(&dw, val); + erts_smp_dw_atomic_init_nob(&ptab->vola.tile.last_data, &dw); +} + +static ERTS_INLINE void +last_data_set_relb(ErtsPTab *ptab, Uint64 val) +{ + erts_dw_aint_t dw; + unint64_to_dw_aint(&dw, val); + erts_smp_dw_atomic_set_relb(&ptab->vola.tile.last_data, &dw); +} + +static ERTS_INLINE Uint64 +last_data_read_nob(ErtsPTab *ptab) +{ + erts_dw_aint_t dw; + erts_smp_dw_atomic_read_nob(&ptab->vola.tile.last_data, &dw); + return dw_aint_to_uint64(&dw); +} + +static ERTS_INLINE Uint64 +last_data_read_acqb(ErtsPTab *ptab) +{ + erts_dw_aint_t dw; + erts_smp_dw_atomic_read_acqb(&ptab->vola.tile.last_data, &dw); + return dw_aint_to_uint64(&dw); +} + +static ERTS_INLINE Uint64 +last_data_cmpxchg_relb(ErtsPTab *ptab, Uint64 new, Uint64 exp) +{ + erts_dw_aint_t dw_new, dw_xchg; + + unint64_to_dw_aint(&dw_new, new); + unint64_to_dw_aint(&dw_xchg, exp); + + if (erts_smp_dw_atomic_cmpxchg_relb(&ptab->vola.tile.last_data, + &dw_new, + &dw_xchg)) + return exp; + else + return dw_aint_to_uint64(&dw_xchg); +} + +#elif defined(ARCH_64) + +union { + erts_smp_atomic_t pid_data; + char align[ERTS_CACHE_LINE_SIZE]; +} last erts_align_attribute(ERTS_CACHE_LINE_SIZE); + +static ERTS_INLINE void +last_data_init_nob(ErtsPTab *ptab, Uint64 val) +{ + erts_smp_atomic_init_nob(&ptab->vola.tile.last_data, (erts_aint_t) val); +} + +static ERTS_INLINE void +last_data_set_relb(ErtsPTab *ptab, Uint64 val) +{ + erts_smp_atomic_set_relb(&ptab->vola.tile.last_data, (erts_aint_t) val); +} + +static ERTS_INLINE Uint64 +last_data_read_nob(ErtsPTab *ptab) +{ + return (Uint64) erts_smp_atomic_read_nob(&ptab->vola.tile.last_data); +} + +static ERTS_INLINE Uint64 +last_data_read_acqb(ErtsPTab *ptab) +{ + return (Uint64) erts_smp_atomic_read_acqb(&ptab->vola.tile.last_data); +} + +static ERTS_INLINE Uint64 +last_data_cmpxchg_relb(ErtsPTab *ptab, Uint64 new, Uint64 exp) +{ + return (Uint64) erts_smp_atomic_cmpxchg_relb(&ptab->vola.tile.last_data, + (erts_aint_t) new, + (erts_aint_t) exp); +} + +#else +# error "Not 64-bit, nor 32-bit architecture..." +#endif + +static ERTS_INLINE int +last_data_cmp(Uint64 ld1, Uint64 ld2) +{ + Uint64 ld1_wrap; + + if (ld1 == ld2) + return 0; + + ld1_wrap = ld1 + (((Uint64) 1) << 63); + + if (ld1 < ld1_wrap) + return (ld1 < ld2 && ld2 < ld1_wrap) ? -1 : 1; + else + return (ld1_wrap <= ld2 && ld2 < ld1) ? 1 : -1; +} + +#define ERTS_PTAB_LastData2EtermData(LD) \ + ((Eterm) ((LD) & ~(~((Uint64) 0) << ERTS_PTAB_ID_DATA_SIZE))) + +void +erts_ptab_init_table(ErtsPTab *ptab, + ErtsAlcType_t atype, + void (*release_element)(void *), + ErtsPTabElementCommon *invalid_element, + int size, + char *name) +{ + size_t tab_sz; + int bits; + char *tab_end; + erts_smp_atomic_t *tab_entry; + erts_smp_rwmtx_opt_t rwmtx_opts = ERTS_SMP_RWMTX_OPT_DEFAULT_INITER; + rwmtx_opts.type = ERTS_SMP_RWMTX_TYPE_EXTREMELY_FREQUENT_READ; + rwmtx_opts.lived = ERTS_SMP_RWMTX_LONG_LIVED; + + erts_smp_rwmtx_init_opt(&ptab->list.data.rwmtx, &rwmtx_opts, name); + erts_smp_atomic32_init_nob(&ptab->vola.tile.count, 0); + last_data_init_nob(ptab, ~((Uint64) 0)); + + /* A size that is a power of 2 is to prefer performance wise */ + bits = erts_fit_in_bits_int32(size-1); + size = 1 << bits; + if (size > ERTS_PTAB_MAX_SIZE) { + size = ERTS_PTAB_MAX_SIZE; + bits = erts_fit_in_bits_int32((Sint32) size - 1); + } + + ptab->r.o.max = size; + + tab_sz = ERTS_ALC_CACHE_LINE_ALIGN_SIZE(size*sizeof(erts_smp_atomic_t)); + ptab->r.o.tab = erts_alloc_permanent_cache_aligned(atype, tab_sz); + tab_end = ((char *) ptab->r.o.tab) + tab_sz; + tab_entry = ptab->r.o.tab; + while (tab_end > ((char *) tab_entry)) { + erts_smp_atomic_init_nob(tab_entry, ERTS_AINT_NULL); + tab_entry++; + } + + ptab->r.o.tab_cache_lines = tab_sz/ERTS_CACHE_LINE_SIZE; + ptab->r.o.pix_per_cache_line = (ERTS_CACHE_LINE_SIZE + / sizeof(erts_smp_atomic_t)); + ASSERT((ptab->r.o.max & (ptab->r.o.max - 1)) == 0); /* power of 2 */ + ASSERT((ptab->r.o.pix_per_cache_line + & (ptab->r.o.pix_per_cache_line - 1)) == 0); /* power of 2 */ + ASSERT((ptab->r.o.tab_cache_lines + & (ptab->r.o.tab_cache_lines - 1)) == 0); /* power of 2 */ + + ptab->r.o.pix_mask + = (1 << bits) - 1; + ptab->r.o.pix_cl_mask + = ptab->r.o.tab_cache_lines-1; + ptab->r.o.pix_cl_shift + = erts_fit_in_bits_int32(ptab->r.o.pix_per_cache_line-1); + ptab->r.o.pix_cli_shift + = erts_fit_in_bits_int32(ptab->r.o.pix_cl_mask); + ptab->r.o.pix_cli_mask + = (1 << (bits - ptab->r.o.pix_cli_shift)) - 1; + + ASSERT(ptab->r.o.pix_cl_shift + ptab->r.o.pix_cli_shift == bits); + + ptab->r.o.invalid_element = invalid_element; + ptab->r.o.invalid_data = erts_ptab_id2data(ptab, invalid_element->id); + ptab->r.o.release_element = release_element; + + erts_smp_interval_init(&ptab->list.data.interval); + ptab->list.data.deleted.start = NULL; + ptab->list.data.deleted.end = NULL; + ptab->list.data.chunks = (((ptab->r.o.max - 1) + / ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE) + + 1); + + if (size == ERTS_PTAB_MAX_SIZE) { + int pix; + /* + * We want a table size of a power of 2 which ERTS_PTAB_MAX_SIZE + * is. We only have ERTS_PTAB_MAX_SIZE-1 unique identifiers and + * we don't want to shrink the size to ERTS_PTAB_MAX_SIZE/2. + * + * In order to fix this, we insert a pointer from the table + * to the invalid_element, wich will be interpreted as a + * slot currently being modified. This way we will be able to + * have ERTS_PTAB_MAX_SIZE-1 valid elements in the table while + * still having a table size of the power of 2. + */ + erts_smp_atomic32_inc_nob(&ptab->vola.tile.count); + pix = erts_ptab_data2pix(ptab, ptab->r.o.invalid_data); + erts_smp_atomic_set_relb(&ptab->r.o.tab[pix], + (erts_aint_t) ptab->r.o.invalid_element); + } + +} + +int +erts_ptab_initialized(ErtsPTab *ptab) +{ + return ptab->r.o.tab != NULL; +} + +int +erts_ptab_new_element(ErtsPTab *ptab, + ErtsPTabElementCommon *ptab_el, + void *init_arg, + void (*init_ptab_el)(void *, Eterm)) +{ + int pix; + Uint64 ld, exp_ld; + Eterm data; + erts_aint32_t count; + erts_aint_t invalid = (erts_aint_t) ptab->r.o.invalid_element; + + erts_ptab_rlock(ptab); + + count = erts_smp_atomic32_inc_read_acqb(&ptab->vola.tile.count); + if (count > ptab->r.o.max) { + while (1) { + erts_aint32_t act_count; + + act_count = erts_smp_atomic32_cmpxchg_relb(&ptab->vola.tile.count, + count-1, + count); + if (act_count == count) { + erts_ptab_runlock(ptab); + return 0; + } + count = act_count; + if (count <= ptab->r.o.max) + break; + } + } + + ptab_el->u.alive.started_interval + = erts_smp_current_interval_nob(erts_ptab_interval(ptab)); + + ld = last_data_read_acqb(ptab); + + /* Reserve slot */ + while (1) { + ld++; + pix = erts_ptab_data2pix(ptab, ERTS_PTAB_LastData2EtermData(ld)); + if (erts_smp_atomic_read_nob(&ptab->r.o.tab[pix]) == ERTS_AINT_NULL) { + erts_aint_t val; + val = erts_smp_atomic_cmpxchg_relb(&ptab->r.o.tab[pix], + invalid, + ERTS_AINT_NULL); + + if (ERTS_AINT_NULL == val) + break; + } + } + + data = ERTS_PTAB_LastData2EtermData(ld); + + if (data == ptab->r.o.invalid_data) { + /* Do not use invalid data; fix it... */ + ld += ptab->r.o.max; + ASSERT(pix == erts_ptab_data2pix(ptab, + ERTS_PTAB_LastData2EtermData(ld))); + data = ERTS_PTAB_LastData2EtermData(ld); + ASSERT(data != ptab->r.o.invalid_data); + } + + exp_ld = last_data_read_nob(ptab); + + /* Move last data forward */ + while (1) { + Uint64 act_ld; + if (last_data_cmp(ld, exp_ld) < 0) + break; + act_ld = last_data_cmpxchg_relb(ptab, ld, exp_ld); + if (act_ld == exp_ld) + break; + exp_ld = act_ld; + } + + init_ptab_el(init_arg, data); + +#ifdef ERTS_SMP + erts_smp_atomic32_init_nob(&ptab_el->refc, 1); +#endif + + /* Move into slot reserved */ +#ifdef DEBUG + ASSERT(invalid == erts_smp_atomic_xchg_relb(&ptab->r.o.tab[pix], + (erts_aint_t) ptab_el)); +#else + erts_smp_atomic_set_relb(&ptab->r.o.tab[pix], (erts_aint_t) ptab_el); +#endif + + erts_ptab_runlock(ptab); + + return 1; +} + +static void +save_deleted_element(ErtsPTab *ptab, ErtsPTabElementCommon *ptab_el) +{ + ErtsPTabDeletedElement *ptdep = erts_alloc(ERTS_ALC_T_PTAB_LIST_DEL, + sizeof(ErtsPTabDeletedElement)); + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start + && ptab->list.data.deleted.end); + ERTS_SMP_LC_ASSERT(erts_smp_lc_ptab_is_rwlocked(ptab)); + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + ptdep->prev = ptab->list.data.deleted.end; + ptdep->next = NULL; + ptdep->ix = erts_ptab_id2pix(ptab, ptab_el->id); + ptdep->u.element.id = ptab_el->id; + ptdep->u.element.inserted = ptab_el->u.alive.started_interval; + ptdep->u.element.deleted = + erts_smp_current_interval_nob(erts_ptab_interval(ptab)); + + ptab->list.data.deleted.end->next = ptdep; + ptab->list.data.deleted.end = ptdep; + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + ERTS_PTAB_LIST_ASSERT(ptdep->prev->ix >= 0 + ? (ptdep->u.element.deleted + >= ptdep->prev->u.element.deleted) + : (ptdep->u.element.deleted + >= ptdep->prev->u.bif_invocation.interval)); +} + +void +erts_ptab_delete_element(ErtsPTab *ptab, + ErtsPTabElementCommon *ptab_el) +{ + int maybe_save; + int pix = erts_ptab_id2pix(ptab, ptab_el->id); + + ASSERT(erts_get_scheduler_id()); /* *Need* to be a scheduler */ + + erts_ptab_rlock(ptab); + maybe_save = ptab->list.data.deleted.end != NULL; + if (maybe_save) { + erts_ptab_runlock(ptab); + erts_ptab_rwlock(ptab); + } + + erts_smp_atomic_set_relb(&ptab->r.o.tab[pix], ERTS_AINT_NULL); + + ASSERT(erts_smp_atomic32_read_nob(&ptab->vola.tile.count) > 0); + erts_smp_atomic32_dec_relb(&ptab->vola.tile.count); + + if (!maybe_save) + erts_ptab_runlock(ptab); + else { + if (ptab->list.data.deleted.end) + save_deleted_element(ptab, ptab_el); + erts_ptab_rwunlock(ptab); + } + + if (ptab->r.o.release_element) + erts_schedule_thr_prgr_later_op(ptab->r.o.release_element, + (void *) ptab_el, + &ptab_el->u.release); +} + +/* + * erts_ptab_list() implements BIFs listing the content of the table, + * e.g. erlang:processes/0. + */ +static void cleanup_ptab_list_bif_data(Binary *bp); +static int ptab_list_bif_engine(Process *c_p, Eterm *res_accp, Binary *mbp); + + +BIF_RETTYPE +erts_ptab_list(Process *c_p, ErtsPTab *ptab) +{ + /* + * A requirement: The list of identifiers returned should be a + * consistent snapshot of all elements existing + * in the table at some point in time during the + * execution of the BIF calling this function. + * Since elements might be deleted while the BIF + * is executing, we have to keep track of all + * deleted elements and add them to the result. + * We also ignore elements created after the BIF + * has begun executing. + */ + BIF_RETTYPE ret_val; + Eterm res_acc = NIL; + Binary *mbp = erts_create_magic_binary(sizeof(ErtsPTabListBifData), + cleanup_ptab_list_bif_data); + ErtsPTabListBifData *ptlbdp = ERTS_MAGIC_BIN_DATA(mbp); + + ERTS_PTAB_LIST_DBG_TRACE(c_p->common.id, call); + ptlbdp->ptab = ptab; + ptlbdp->state = INITIALIZING; + ERTS_PTAB_LIST_DBG_INIT(c_p, ptlbdp); + + if (ERTS_BIF_REDS_LEFT(c_p) >= ERTS_PTAB_LIST_BIF_MIN_START_REDS + && ptab_list_bif_engine(c_p, &res_acc, mbp)) { + erts_bin_free(mbp); + ERTS_PTAB_LIST_DBG_CHK_RESLIST(res_acc); + ERTS_PTAB_LIST_DBG_TRACE(c_p->common.id, return); + ERTS_BIF_PREP_RET(ret_val, res_acc); + } + else { + Eterm *hp; + Eterm magic_bin; + ERTS_PTAB_LIST_DBG_CHK_RESLIST(res_acc); + hp = HAlloc(c_p, PROC_BIN_SIZE); + ERTS_PTAB_LIST_DBG_SAVE_HEAP_ALLOC(ptlbdp, hp, PROC_BIN_SIZE); + magic_bin = erts_mk_magic_binary_term(&hp, &MSO(c_p), mbp); + ERTS_PTAB_LIST_DBG_VERIFY_HEAP_ALLOC_USED(ptlbdp, hp); + ERTS_PTAB_LIST_DBG_TRACE(c_p->common.id, trap); + ERTS_BIF_PREP_YIELD2(ret_val, + &ptab_list_continue_export, + c_p, + res_acc, + magic_bin); + } + return ret_val; +} + +static void +cleanup_ptab_list_bif_data(Binary *bp) +{ + ErtsPTabListBifData *ptlbdp = ERTS_MAGIC_BIN_DATA(bp); + ErtsPTab *ptab = ptlbdp->ptab; + + ERTS_PTAB_LIST_DBG_TRACE(ptlbdp->debug.caller, call); + + if (ptlbdp->state != INITIALIZING) { + + if (ptlbdp->chunk) { + erts_free(ERTS_ALC_T_PTAB_LIST_CNKI, ptlbdp->chunk); + ptlbdp->chunk = NULL; + } + if (ptlbdp->pid) { + erts_free(ERTS_ALC_T_PTAB_LIST_PIDS, ptlbdp->pid); + ptlbdp->pid = NULL; + } + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + if (ptlbdp->debug.pid_started) { + erts_free(ERTS_ALC_T_PTAB_LIST_PIDS, ptlbdp->debug.pid_started); + ptlbdp->debug.pid_started = NULL; + } +#endif + + if (ptlbdp->bif_invocation) { + ErtsPTabDeletedElement *ptdep; + + erts_ptab_rwlock(ptab); + + ERTS_PTAB_LIST_DBG_TRACE(ptlbdp->debug.caller, deleted_cleanup); + + ptdep = ptlbdp->bif_invocation; + ptlbdp->bif_invocation = NULL; + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + if (ptdep->prev) { + /* + * Only remove this bif invokation when we + * have preceding invokations. + */ + ptdep->prev->next = ptdep->next; + if (ptdep->next) + ptdep->next->prev = ptdep->prev; + else { + /* + * At the time of writing this branch cannot be + * reached. I don't want to remove this code though + * since it may be possible to reach this line + * in the future if the cleanup order in + * erts_do_exit_process() is changed. The ASSERT(0) + * is only here to make us aware that the reorder + * has happened. /rickard + */ + ASSERT(0); + ptab->list.data.deleted.end = ptdep->prev; + } + erts_free(ERTS_ALC_T_PTAB_LIST_DEL, ptdep); + } + else { + /* + * Free all elements until next bif invokation + * is found. + */ + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start == ptdep); + do { + ErtsPTabDeletedElement *fptdep = ptdep; + ptdep = ptdep->next; + erts_free(ERTS_ALC_T_PTAB_LIST_DEL, fptdep); + } while (ptdep && ptdep->ix >= 0); + ptab->list.data.deleted.start = ptdep; + if (ptdep) + ptdep->prev = NULL; + else + ptab->list.data.deleted.end = NULL; + } + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + erts_ptab_rwunlock(ptab); + + } + } + + ERTS_PTAB_LIST_DBG_TRACE(ptlbdp->debug.caller, return); + ERTS_PTAB_LIST_DBG_CLEANUP(ptlbdp); +} + +static int +ptab_list_bif_engine(Process *c_p, Eterm *res_accp, Binary *mbp) +{ + ErtsPTabListBifData *ptlbdp = ERTS_MAGIC_BIN_DATA(mbp); + ErtsPTab *ptab = ptlbdp->ptab; + int have_reds; + int reds; + int locked = 0; + + do { + switch (ptlbdp->state) { + case INITIALIZING: + ptlbdp->chunk = erts_alloc(ERTS_ALC_T_PTAB_LIST_CNKI, + (sizeof(ErtsPTabListBifChunkInfo) + * ptab->list.data.chunks)); + ptlbdp->tix = 0; + ptlbdp->pid_ix = 0; + + erts_ptab_rwlock(ptab); + locked = 1; + + ERTS_PTAB_LIST_DBG_TRACE(c_p->common.id, init); + + ptlbdp->pid_sz = erts_ptab_count(ptab); + ptlbdp->pid = erts_alloc(ERTS_ALC_T_PTAB_LIST_PIDS, + sizeof(Eterm)*ptlbdp->pid_sz); + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + ptlbdp->debug.pid_started + = erts_alloc(ERTS_ALC_T_PTAB_LIST_PIDS, + sizeof(Uint64)*ptlbdp->pid_sz); +#endif + + ERTS_PTAB_LIST_DBG_SAVE_PIDS(ptlbdp); + + if (ptab->list.data.chunks == 1) + ptlbdp->bif_invocation = NULL; + else { + /* + * We will have to access the table multiple times + * releasing the table lock in between chunks. + */ + ptlbdp->bif_invocation + = erts_alloc(ERTS_ALC_T_PTAB_LIST_DEL, + sizeof(ErtsPTabDeletedElement)); + ptlbdp->bif_invocation->ix = -1; + ptlbdp->bif_invocation->u.bif_invocation.interval + = erts_smp_step_interval_nob(erts_ptab_interval(ptab)); + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + ptlbdp->bif_invocation->next = NULL; + if (ptab->list.data.deleted.end) { + ptlbdp->bif_invocation->prev = ptab->list.data.deleted.end; + ptab->list.data.deleted.end->next = ptlbdp->bif_invocation; + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start); + } + else { + ptlbdp->bif_invocation->prev = NULL; + ptab->list.data.deleted.start = ptlbdp->bif_invocation; + } + ptab->list.data.deleted.end = ptlbdp->bif_invocation; + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + } + + ptlbdp->state = INSPECTING_TABLE; + /* Fall through */ + + case INSPECTING_TABLE: { + int ix = ptlbdp->tix; + int indices = ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE; + int cix = ix / ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE; + int end_ix = ix + indices; + Uint64 *invocation_interval_p; + ErtsPTabElementCommon *invalid_element; + + invocation_interval_p + = (ptlbdp->bif_invocation + ? &ptlbdp->bif_invocation->u.bif_invocation.interval + : NULL); + + ERTS_PTAB_LIST_ASSERT(is_nil(*res_accp)); + if (!locked) { + erts_ptab_rwlock(ptab); + locked = 1; + } + + ERTS_SMP_LC_ASSERT(erts_smp_lc_ptab_is_rwlocked(ptab)); + ERTS_PTAB_LIST_DBG_TRACE(p->common.id, insp_table); + + if (cix != 0) + ptlbdp->chunk[cix].interval + = erts_smp_step_interval_nob(erts_ptab_interval(ptab)); + else if (ptlbdp->bif_invocation) + ptlbdp->chunk[0].interval = *invocation_interval_p; + /* else: interval is irrelevant */ + + if (end_ix >= ptab->r.o.max) { + ERTS_PTAB_LIST_ASSERT(cix+1 == ptab->list.data.chunks); + end_ix = ptab->r.o.max; + indices = end_ix - ix; + /* What to do when done with this chunk */ + ptlbdp->state = (ptab->list.data.chunks == 1 + ? BUILDING_RESULT + : INSPECTING_DELETED); + } + + invalid_element = ptab->r.o.invalid_element; + for (; ix < end_ix; ix++) { + ErtsPTabElementCommon *el; + el = (ErtsPTabElementCommon *) erts_ptab_pix2intptr_nob(ptab, + ix); + if (el + && el != invalid_element + && (!invocation_interval_p + || el->u.alive.started_interval < *invocation_interval_p)) { + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(el->id)); + ptlbdp->pid[ptlbdp->pid_ix] = el->id; + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + ptlbdp->debug.pid_started[ptlbdp->pid_ix] + = el->u.alive.started_interval; +#endif + + ptlbdp->pid_ix++; + ERTS_PTAB_LIST_ASSERT(ptlbdp->pid_ix <= ptlbdp->pid_sz); + } + } + + ptlbdp->tix = end_ix; + + erts_ptab_rwunlock(ptab); + locked = 0; + + reds = indices/ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED; + BUMP_REDS(c_p, reds); + + have_reds = ERTS_BIF_REDS_LEFT(c_p); + + if (have_reds && ptlbdp->state == INSPECTING_TABLE) { + ix = ptlbdp->tix; + indices = ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE; + end_ix = ix + indices; + if (end_ix > ptab->r.o.max) { + end_ix = ptab->r.o.max; + indices = end_ix - ix; + } + + reds = indices/ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED; + + /* Pretend we have no reds left if we haven't got enough + reductions to complete next chunk */ + if (reds > have_reds) + have_reds = 0; + } + + break; + } + + case INSPECTING_DELETED: { + int i; + int max_reds; + int free_deleted = 0; + Uint64 invocation_interval; + ErtsPTabDeletedElement *ptdep; + ErtsPTabDeletedElement *free_list = NULL; + + ptdep = ptlbdp->bif_invocation; + ERTS_PTAB_LIST_ASSERT(ptdep); + invocation_interval = ptdep->u.bif_invocation.interval; + + max_reds = have_reds = ERTS_BIF_REDS_LEFT(c_p); + if (max_reds > ERTS_PTAB_LIST_INSPECT_DELETED_MAX_REDS) + max_reds = ERTS_PTAB_LIST_INSPECT_DELETED_MAX_REDS; + + reds = 0; + erts_ptab_rwlock(ptab); + ERTS_PTAB_LIST_DBG_TRACE(p->common.id, insp_term_procs); + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + if (ptdep->prev) + ptdep->prev->next = ptdep->next; + else { + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start == ptdep); + ptab->list.data.deleted.start = ptdep->next; + + if (ptab->list.data.deleted.start + && ptab->list.data.deleted.start->ix >= 0) { + free_list = ptab->list.data.deleted.start; + free_deleted = 1; + } + } + + if (ptdep->next) + ptdep->next->prev = ptdep->prev; + else + ptab->list.data.deleted.end = ptdep->prev; + + ptdep = ptdep->next; + + i = 0; + while (reds < max_reds && ptdep) { + if (ptdep->ix < 0) { + if (free_deleted) { + ERTS_PTAB_LIST_ASSERT(free_list); + ERTS_PTAB_LIST_ASSERT(ptdep->prev); + + ptdep->prev->next = NULL; /* end of free_list */ + ptab->list.data.deleted.start = ptdep; + ptdep->prev = NULL; + free_deleted = 0; + } + } + else { + int cix = ptdep->ix/ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE; + Uint64 chunk_interval = ptlbdp->chunk[cix].interval; + Eterm pid = ptdep->u.element.id; + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(pid)); + + if (ptdep->u.element.inserted < invocation_interval) { + if (ptdep->u.element.deleted < chunk_interval) { + ERTS_PTAB_LIST_DBG_CHK_PID_NOT_FOUND( + ptlbdp, + pid, + ptdep->u.element.inserted); + ptlbdp->pid[ptlbdp->pid_ix] = pid; +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + ptlbdp->debug.pid_started[ptlbdp->pid_ix] + = ptdep->u.element.inserted; +#endif + ptlbdp->pid_ix++; + ERTS_PTAB_LIST_ASSERT(ptlbdp->pid_ix + <= ptlbdp->pid_sz); + } + else { + ERTS_PTAB_LIST_DBG_CHK_PID_FOUND( + ptlbdp, + pid, + ptdep->u.element.inserted); + } + } + else { + ERTS_PTAB_LIST_DBG_CHK_PID_NOT_FOUND( + ptlbdp, + pid, + ptdep->u.element.inserted); + } + + i++; + if (i == ERTS_PTAB_LIST_BIF_INSPECT_DELETED_PER_RED) { + reds++; + i = 0; + } + if (free_deleted) + reds += ERTS_PTAB_LIST_BIF_TAB_FREE_DELETED_REDS; + } + ptdep = ptdep->next; + } + + if (free_deleted) { + ERTS_PTAB_LIST_ASSERT(free_list); + ptab->list.data.deleted.start = ptdep; + if (!ptdep) + ptab->list.data.deleted.end = NULL; + else { + ERTS_PTAB_LIST_ASSERT(ptdep->prev); + ptdep->prev->next = NULL; /* end of free_list */ + ptdep->prev = NULL; + } + } + + if (!ptdep) { + /* Done */ + ERTS_PTAB_LIST_ASSERT(ptlbdp->pid_ix == ptlbdp->pid_sz); + ptlbdp->state = BUILDING_RESULT; + ptlbdp->bif_invocation->next = free_list; + free_list = ptlbdp->bif_invocation; + ptlbdp->bif_invocation = NULL; + } + else { + /* Link in bif_invocation again where we left off */ + ptlbdp->bif_invocation->prev = ptdep->prev; + ptlbdp->bif_invocation->next = ptdep; + ptdep->prev = ptlbdp->bif_invocation; + if (ptlbdp->bif_invocation->prev) + ptlbdp->bif_invocation->prev->next = ptlbdp->bif_invocation; + else { + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start + == ptdep); + ptab->list.data.deleted.start = ptlbdp->bif_invocation; + } + } + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + ERTS_PTAB_LIST_DBG_CHK_FREELIST(ptab, free_list); + erts_ptab_rwunlock(ptab); + + /* + * We do the actual free of deleted structures now when we + * have released the table lock instead of when we encountered + * them. This since free() isn't for free and we don't want to + * unnecessarily block other schedulers. + */ + while (free_list) { + ptdep = free_list; + free_list = ptdep->next; + erts_free(ERTS_ALC_T_PTAB_LIST_DEL, ptdep); + } + + have_reds -= reds; + if (have_reds < 0) + have_reds = 0; + BUMP_REDS(c_p, reds); + break; + } + + case BUILDING_RESULT: { + int conses, ix, min_ix; + Eterm *hp; + Eterm res = *res_accp; + + ERTS_PTAB_LIST_DBG_VERIFY_PIDS(ptlbdp); + ERTS_PTAB_LIST_DBG_CHK_RESLIST(res); + + ERTS_PTAB_LIST_DBG_TRACE(p->common.id, begin_build_res); + + have_reds = ERTS_BIF_REDS_LEFT(c_p); + conses = ERTS_PTAB_LIST_BIF_BUILD_RESULT_CONSES_PER_RED*have_reds; + min_ix = ptlbdp->pid_ix - conses; + if (min_ix < 0) { + min_ix = 0; + conses = ptlbdp->pid_ix; + } + + if (conses) { + hp = HAlloc(c_p, conses*2); + ERTS_PTAB_LIST_DBG_SAVE_HEAP_ALLOC(ptlbdp, hp, conses*2); + + for (ix = ptlbdp->pid_ix - 1; ix >= min_ix; ix--) { + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(ptlbdp->pid[ix])); + res = CONS(hp, ptlbdp->pid[ix], res); + hp += 2; + } + + ERTS_PTAB_LIST_DBG_VERIFY_HEAP_ALLOC_USED(ptlbdp, hp); + } + + ptlbdp->pid_ix = min_ix; + if (min_ix == 0) + ptlbdp->state = RETURN_RESULT; + else { + ptlbdp->pid_sz = min_ix; + ptlbdp->pid = erts_realloc(ERTS_ALC_T_PTAB_LIST_PIDS, + ptlbdp->pid, + sizeof(Eterm)*ptlbdp->pid_sz); +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + ptlbdp->debug.pid_started + = erts_realloc(ERTS_ALC_T_PTAB_LIST_PIDS, + ptlbdp->debug.pid_started, + sizeof(Uint64) * ptlbdp->pid_sz); +#endif + } + reds = conses/ERTS_PTAB_LIST_BIF_BUILD_RESULT_CONSES_PER_RED; + BUMP_REDS(c_p, reds); + have_reds -= reds; + + ERTS_PTAB_LIST_DBG_CHK_RESLIST(res); + ERTS_PTAB_LIST_DBG_TRACE(c_p->common.id, end_build_res); + *res_accp = res; + break; + } + case RETURN_RESULT: + cleanup_ptab_list_bif_data(mbp); + return 1; + + default: + erl_exit(ERTS_ABORT_EXIT, + "%s:%d:ptab_list_bif_engine(): Invalid state: %d\n", + __FILE__, __LINE__, (int) ptlbdp->state); + } + + + } while (have_reds || ptlbdp->state == RETURN_RESULT); + + return 0; +} + +/* + * ptab_list_continue/2 is a hidden BIF that the original BIF traps to + * if there are too much work to do in one go. + */ + +static BIF_RETTYPE ptab_list_continue(BIF_ALIST_2) +{ + Eterm res_acc; + Binary *mbp; + + /* + * This bif cannot be called from erlang code. It can only be + * trapped to from other BIFs; therefore, a bad argument + * is an internal error and should never occur... + */ + + ERTS_PTAB_LIST_DBG_TRACE(BIF_P->common.id, call); + ERTS_PTAB_LIST_ASSERT(is_nil(BIF_ARG_1) || is_list(BIF_ARG_1)); + + res_acc = BIF_ARG_1; + + ERTS_PTAB_LIST_ASSERT(ERTS_TERM_IS_MAGIC_BINARY(BIF_ARG_2)); + + mbp = ((ProcBin *) binary_val(BIF_ARG_2))->val; + + ERTS_PTAB_LIST_ASSERT(ERTS_MAGIC_BIN_DESTRUCTOR(mbp) + == cleanup_ptab_list_bif_data); + ERTS_PTAB_LIST_ASSERT( + ((ErtsPTabListBifData *) ERTS_MAGIC_BIN_DATA(mbp))->debug.caller + == BIF_P->common.id); + + if (ptab_list_bif_engine(BIF_P, &res_acc, mbp)) { + ERTS_PTAB_LIST_DBG_TRACE(BIF_P->common.id, return); + BIF_RET(res_acc); + } + else { + ERTS_PTAB_LIST_DBG_TRACE(BIF_P->common.id, trap); + ERTS_BIF_YIELD2(&ptab_list_continue_export, BIF_P, res_acc, BIF_ARG_2); + } +} + +void +erts_ptab_init(void) +{ + /* ptab_list_continue/2 is a hidden BIF that the original BIF traps to. */ + erts_init_trap_export(&ptab_list_continue_export, + am_erlang, am_ptab_list_continue, 2, + &ptab_list_continue); + +} + +/* + * Debug stuff + */ + +Sint +erts_ptab_test_next_id(ErtsPTab *ptab, int set, Uint next) +{ + Uint64 ld; + Sint res; + Eterm data; + int first_pix = -1; + + erts_ptab_rwlock(ptab); + + if (!set) + ld = last_data_read_nob(ptab); + else { + + ld = (Uint64) next; + data = ERTS_PTAB_LastData2EtermData(ld); + if (ptab->r.o.invalid_data == data) { + ld += ptab->r.o.max; + ASSERT(erts_ptab_data2pix(ptab, data) + == erts_ptab_data2pix(ptab, + ERTS_PTAB_LastData2EtermData(ld))); + } + last_data_set_relb(ptab, ld); + } + + while (1) { + int pix; + ld++; + pix = (int) (ld % ptab->r.o.max); + if (first_pix < 0) + first_pix = pix; + else if (pix == first_pix) { + res = -1; + break; + } + if (ERTS_AINT_NULL == erts_ptab_pix2intptr_nob(ptab, pix)) { + data = ERTS_PTAB_LastData2EtermData(ld); + if (ptab->r.o.invalid_data == data) { + ld += ptab->r.o.max; + ASSERT(erts_ptab_data2pix(ptab, data) + == erts_ptab_data2pix(ptab, + ERTS_PTAB_LastData2EtermData(ld))); + data = ERTS_PTAB_LastData2EtermData(ld); + } + res = data; + break; + } + } + + erts_ptab_rwunlock(ptab); + + return res; +} + +static ERTS_INLINE ErtsPTabElementCommon * +ptab_pix2el(ErtsPTab *ptab, int ix) +{ + ErtsPTabElementCommon *ptab_el; + ASSERT(0 <= ix && ix < ptab->r.o.max); + ptab_el = (ErtsPTabElementCommon *) erts_ptab_pix2intptr_nob(ptab, ix); + if (ptab_el == ptab->r.o.invalid_element) + return NULL; + else + return ptab_el; +} + +Eterm +erts_debug_ptab_list(Process *c_p, ErtsPTab *ptab) +{ + int i; + Uint need; + Eterm res; + Eterm* hp; + Eterm *hp_end; + + erts_ptab_rwlock(ptab); + + res = NIL; + need = erts_ptab_count(ptab) * 2; + hp = HAlloc(c_p, need); /* we need two heap words for each id */ + hp_end = hp + need; + + /* make the list by scanning bakward */ + + + for (i = ptab->r.o.max-1; i >= 0; i--) { + ErtsPTabElementCommon *el = ptab_pix2el(ptab, i); + if (el) { + res = CONS(hp, el->id, res); + hp += 2; + } + } + + erts_ptab_rwunlock(ptab); + + HRelease(c_p, hp_end, hp); + + return res; +} + +Eterm +erts_debug_ptab_list_bif_info(Process *c_p, ErtsPTab *ptab) +{ + ERTS_DECL_AM(ptab_list_bif_info); + Eterm elements[] = { + AM_ptab_list_bif_info, + make_small((Uint) ERTS_PTAB_LIST_BIF_MIN_START_REDS), + make_small((Uint) ptab->list.data.chunks), + make_small((Uint) ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE), + make_small((Uint) ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED), + make_small((Uint) ERTS_PTAB_LIST_BIF_TAB_FREE_DELETED_REDS), + make_small((Uint) ERTS_PTAB_LIST_BIF_INSPECT_DELETED_PER_RED), + make_small((Uint) ERTS_PTAB_LIST_INSPECT_DELETED_MAX_REDS), + make_small((Uint) ERTS_PTAB_LIST_BIF_BUILD_RESULT_CONSES_PER_RED), + make_small((Uint) ERTS_PTAB_LIST_BIF_DEBUGLEVEL) + }; + Uint sz = 0; + Eterm *hp; + (void) erts_bld_tuplev(NULL, &sz, sizeof(elements)/sizeof(Eterm), elements); + hp = HAlloc(c_p, sz); + return erts_bld_tuplev(&hp, NULL, sizeof(elements)/sizeof(Eterm), elements); +} + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS +static void +debug_ptab_list_check_found_pid(ErtsPTabListBifData *ptlbdp, + Eterm pid, + Uint64 ic, + int pid_should_be_found) +{ + int i; + for (i = 0; i < ptlbdp->pid_ix; i++) { + if (ptlbdp->pid[i] == pid && ptlbdp->debug.pid_started[i] == ic) { + ERTS_PTAB_LIST_ASSERT(pid_should_be_found); + return; + } + } + ERTS_PTAB_LIST_ASSERT(!pid_should_be_found); +} +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_RESLIST +static void +debug_ptab_list_check_res_list(Eterm list) +{ + while (is_list(list)) { + Eterm* consp = list_val(list); + Eterm hd = CAR(consp); + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(hd)); + list = CDR(consp); + } + + ERTS_PTAB_LIST_ASSERT(is_nil(list)); +} +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_PIDS + +static void +debug_ptab_list_save_all_pids(ErtsPTabListBifData *ptlbdp) +{ + int ix, tix, cpix; + ErtsPTab *ptab = ptlbdp->ptab; + ptlbdp->debug.correct_pids_verified = 0; + ptlbdp->debug.correct_pids = erts_alloc(ERTS_ALC_T_PTAB_LIST_PIDS, + sizeof(Eterm)*ptlbdp->pid_sz); + + for (tix = 0, cpix = 0; tix < ptab->r.o.max; tix++) { + ErtsPTabElementCommon *el = ptab_pix2el(ptab, tix); + if (el) { + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(el->id)); + ptlbdp->debug.correct_pids[cpix++] = el->id; + ERTS_PTAB_LIST_ASSERT(cpix <= ptlbdp->pid_sz); + } + } + ERTS_PTAB_LIST_ASSERT(cpix == ptlbdp->pid_sz); + + for (ix = 0; ix < ptlbdp->pid_sz; ix++) + ptlbdp->pid[ix] = make_small(ix); +} + +static void +debug_ptab_list_verify_all_pids(ErtsPTabListBifData *ptlbdp) +{ + int ix, cpix; + + ERTS_PTAB_LIST_ASSERT(ptlbdp->pid_ix == ptlbdp->pid_sz); + + for (ix = 0; ix < ptlbdp->pid_sz; ix++) { + int found = 0; + Eterm pid = ptlbdp->pid[ix]; + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(pid)); + for (cpix = ix; cpix < ptlbdp->pid_sz; cpix++) { + if (ptlbdp->debug.correct_pids[cpix] == pid) { + ptlbdp->debug.correct_pids[cpix] = NIL; + found = 1; + break; + } + } + if (!found) { + for (cpix = 0; cpix < ix; cpix++) { + if (ptlbdp->debug.correct_pids[cpix] == pid) { + ptlbdp->debug.correct_pids[cpix] = NIL; + found = 1; + break; + } + } + } + ERTS_PTAB_LIST_ASSERT(found); + } + ptlbdp->debug.correct_pids_verified = 1; + + erts_free(ERTS_ALC_T_PTAB_LIST_PIDS, ptlbdp->debug.correct_pids); + ptlbdp->debug.correct_pids = NULL; +} +#endif /* ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_PIDS */ + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_DEL_LIST +static void +debug_ptab_list_check_del_list(ErtsPTab *ptab) +{ + ERTS_SMP_LC_ASSERT(erts_smp_lc_ptab_is_rwlocked(ptab)); + if (!ptab->list.data.deleted.start) + ERTS_PTAB_LIST_ASSERT(!ptab->list.data.deleted.end); + else { + Uint64 curr_interval = erts_smp_current_interval_nob(erts_ptab_interval(ptab)); + Uint64 *prev_x_interval_p = NULL; + ErtsPTabDeletedElement *ptdep; + + for (ptdep = ptab->list.data.deleted.start; + ptdep; + ptdep = ptdep->next) { + if (!ptdep->prev) + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start == ptdep); + else + ERTS_PTAB_LIST_ASSERT(ptdep->prev->next == ptdep); + if (!ptdep->next) + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.end == ptdep); + else + ERTS_PTAB_LIST_ASSERT(ptdep->next->prev == ptdep); + if (ptdep->ix < 0) { + Uint64 interval = ptdep->u.bif_invocation.interval; + ERTS_PTAB_LIST_ASSERT(interval <= curr_interval); + } + else { + Uint64 s_interval = ptdep->u.element.inserted; + Uint64 x_interval = ptdep->u.element.deleted; + + ERTS_PTAB_LIST_ASSERT(s_interval <= x_interval); + if (prev_x_interval_p) + ERTS_PTAB_LIST_ASSERT(*prev_x_interval_p <= x_interval); + prev_x_interval_p = &ptdep->u.element.deleted; + ERTS_PTAB_LIST_ASSERT( + erts_ptab_is_valid_id(ptdep->u.element.id)); + ERTS_PTAB_LIST_ASSERT(erts_ptab_id2pix(ptab, + ptdep->u.element.id) + == ptdep->ix); + + } + } + + } +} + +static void +debug_ptab_list_check_del_free_list(ErtsPTab *ptab, + ErtsPTabDeletedElement *free_list) +{ + if (ptab->list.data.deleted.start) { + ErtsPTabDeletedElement *fptdep; + ErtsPTabDeletedElement *ptdep; + + for (fptdep = free_list; fptdep; fptdep = fptdep->next) { + for (ptdep = ptab->list.data.deleted.start; + ptdep; + ptdep = ptdep->next) { + ERTS_PTAB_LIST_ASSERT(fptdep != ptdep); + } + } + } +} + +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL != 0 + +static void +debug_ptab_list_assert_error(char* expr, const char* file, int line, const char *func) +{ + fflush(stdout); + erts_fprintf(stderr, "%s:%d:%s(): Assertion failed: %s\n", + (char *) file, line, (char *) func, expr); + fflush(stderr); + abort(); +} + +#endif diff --git a/erts/emulator/beam/erl_ptab.h b/erts/emulator/beam/erl_ptab.h new file mode 100644 index 0000000000..8a130f42a3 --- /dev/null +++ b/erts/emulator/beam/erl_ptab.h @@ -0,0 +1,472 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +/* + * Description: Process/Port table implementation. + * + * Author: Rickard Green + */ + +#ifndef ERL_PTAB_H__ +#define ERL_PTAB_H__ + +#include "sys.h" +#include "erl_term.h" +#include "erl_time.h" +#include "erl_utils.h" +#define ERL_THR_PROGRESS_TSD_TYPE_ONLY +#include "erl_thr_progress.h" +#undef ERL_THR_PROGRESS_TSD_TYPE_ONLY +#include "erl_alloc.h" +#include "erl_monitors.h" + +#define ERTS_TRACER_PROC(P) ((P)->common.tracer_proc) +#define ERTS_TRACE_FLAGS(P) ((P)->common.trace_flags) + +#define ERTS_P_LINKS(P) ((P)->common.u.alive.links) +#define ERTS_P_MONITORS(P) ((P)->common.u.alive.monitors) + +#define IS_TRACED(p) \ + (ERTS_TRACER_PROC((p)) != NIL) +#define ARE_TRACE_FLAGS_ON(p,tf) \ + ((ERTS_TRACE_FLAGS((p)) & (tf|F_SENSITIVE)) == (tf)) +#define IS_TRACED_FL(p,tf) \ + ( IS_TRACED(p) && ARE_TRACE_FLAGS_ON(p,tf) ) + +typedef struct { + Eterm id; +#ifdef ERTS_SMP + erts_atomic32_t refc; +#endif + Eterm tracer_proc; + Uint trace_flags; + union { + /* --- While being alive --- */ + struct { + Uint64 started_interval; + struct reg_proc *reg; + ErtsLink *links; + ErtsMonitor *monitors; +#ifdef ERTS_SMP + ErtsSmpPTimer *ptimer; +#else + ErlTimer tm; +#endif + } alive; + + /* --- While being released --- */ + ErtsThrPrgrLaterOp release; + } u; +} ErtsPTabElementCommon; + +typedef struct ErtsPTabDeletedElement_ ErtsPTabDeletedElement; + +typedef struct { + erts_smp_rwmtx_t rwmtx; + erts_interval_t interval; + struct { + ErtsPTabDeletedElement *start; + ErtsPTabDeletedElement *end; + } deleted; + int chunks; +} ErtsPTabListData; + +typedef struct { +#ifdef ARCH_32 + erts_smp_dw_atomic_t last_data; +#else + erts_smp_atomic_t last_data; +#endif + erts_smp_atomic32_t count; +} ErtsPTabVolatileData; + +typedef struct { + erts_smp_atomic_t *tab; + Uint32 max; + Uint32 tab_cache_lines; + Uint32 pix_per_cache_line; + Uint32 pix_mask; + Uint32 pix_cl_mask; + Uint32 pix_cl_shift; + Uint32 pix_cli_mask; + Uint32 pix_cli_shift; + ErtsPTabElementCommon *invalid_element; + Eterm invalid_data; + void (*release_element)(void *); +} ErtsPTabReadOnlyData; + +typedef struct { + /* + * Data mainly modified when someone is listing + * the content of the table. + */ + union { + ErtsPTabListData data; + char algn[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(ErtsPTabListData))]; + } list; + + /* + * Frequently modified data. + */ + union { + ErtsPTabVolatileData tile; + char algn[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(ErtsPTabVolatileData))]; + } vola; + + /* + * Read only data. + */ + union { + ErtsPTabReadOnlyData o; + char algn[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(ErtsPTabReadOnlyData))]; + } r; +} ErtsPTab; + +#define ERTS_PTAB_ID_DATA_SIZE 28 +#define ERTS_PTAB_ID_DATA_SHIFT (_TAG_IMMED1_SIZE) +/* ERTS_PTAB_MAX_SIZE must be a power of 2 */ +#define ERTS_PTAB_MAX_SIZE (SWORD_CONSTANT(1) << 27) +#if (ERTS_PTAB_MAX_SIZE-1) > MAX_SMALL +# error "The maximum number of processes/ports must fit in a SMALL." +#endif + + +/* + * Currently pids and ports are allowed. + */ +#if _PID_DATA_SIZE != ERTS_PTAB_ID_DATA_SIZE +# error "Unexpected pid data size" +#endif +#if _PID_DATA_SHIFT != ERTS_PTAB_ID_DATA_SHIFT +# error "Unexpected pid tag size" +#endif +#if _PORT_DATA_SIZE != ERTS_PTAB_ID_DATA_SIZE +# error "Unexpected port data size" +#endif +#if _PORT_DATA_SHIFT != ERTS_PTAB_ID_DATA_SHIFT +# error "Unexpected port tag size" +#endif + +#define ERTS_PTAB_INVALID_ID(TAG) \ + ((Eterm) \ + ((((1 << ERTS_PTAB_ID_DATA_SIZE) - 1) << ERTS_PTAB_ID_DATA_SHIFT) \ + | (TAG))) + +#define erts_ptab_is_valid_id(ID) \ + (is_internal_pid((ID)) || is_internal_port((ID))) + +void erts_ptab_init(void); +void erts_ptab_init_table(ErtsPTab *ptab, + ErtsAlcType_t atype, + void (*release_element)(void *), + ErtsPTabElementCommon *invalid_element, + int size, + char *name); +int erts_ptab_new_element(ErtsPTab *ptab, + ErtsPTabElementCommon *ptab_el, + void *init_arg, + void (*init_ptab_el)(void *, Eterm)); +void erts_ptab_delete_element(ErtsPTab *ptab, + ErtsPTabElementCommon *ptab_el); +int erts_ptab_initialized(ErtsPTab *ptab); + +ERTS_GLB_INLINE erts_interval_t *erts_ptab_interval(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_ptab_max(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_ptab_count(ErtsPTab *ptab); +ERTS_GLB_INLINE Uint erts_ptab_pixdata2data(ErtsPTab *ptab, Eterm pixdata); +ERTS_GLB_INLINE Uint32 erts_ptab_pixdata2pix(ErtsPTab *ptab, Eterm pixdata); +ERTS_GLB_INLINE Uint32 erts_ptab_data2pix(ErtsPTab *ptab, Eterm data); +ERTS_GLB_INLINE Uint erts_ptab_data2pixdata(ErtsPTab *ptab, Eterm data); +ERTS_GLB_INLINE Eterm erts_ptab_make_id(ErtsPTab *ptab, Eterm data, Eterm tag); +ERTS_GLB_INLINE int erts_ptab_id2pix(ErtsPTab *ptab, Eterm id); +ERTS_GLB_INLINE Uint erts_ptab_id2data(ErtsPTab *ptab, Eterm id); +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_nob(ErtsPTab *ptab, int ix); +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_ddrb(ErtsPTab *ptab, int ix); +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_rb(ErtsPTab *ptab, int ix); +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_acqb(ErtsPTab *ptab, int ix); +ERTS_GLB_INLINE void erts_ptab_inc_refc(ErtsPTabElementCommon *ptab_el); +ERTS_GLB_INLINE int erts_ptab_dec_test_refc(ErtsPTabElementCommon *ptab_el); +ERTS_GLB_INLINE int erts_ptab_add_test_refc(ErtsPTabElementCommon *ptab_el, + Sint32 add_refc); +ERTS_GLB_INLINE void erts_ptab_rlock(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_ptab_tryrlock(ErtsPTab *ptab); +ERTS_GLB_INLINE void erts_ptab_runlock(ErtsPTab *ptab); +ERTS_GLB_INLINE void erts_ptab_rwlock(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_ptab_tryrwlock(ErtsPTab *ptab); +ERTS_GLB_INLINE void erts_ptab_rwunlock(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_smp_lc_ptab_is_rlocked(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_smp_lc_ptab_is_rwlocked(ErtsPTab *ptab); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE erts_interval_t * +erts_ptab_interval(ErtsPTab *ptab) +{ + return &ptab->list.data.interval; +} + +ERTS_GLB_INLINE int +erts_ptab_max(ErtsPTab *ptab) +{ + int max = ptab->r.o.max; + return max == ERTS_PTAB_MAX_SIZE ? max - 1 : max; +} + +ERTS_GLB_INLINE int +erts_ptab_count(ErtsPTab *ptab) +{ + int max = ptab->r.o.max; + erts_aint32_t res = erts_smp_atomic32_read_nob(&ptab->vola.tile.count); + if (max == ERTS_PTAB_MAX_SIZE) { + max--; + res--; + } + if (res > max) + return max; + ASSERT(res >= 0); + return (int) res; + +} + +ERTS_GLB_INLINE Uint erts_ptab_pixdata2data(ErtsPTab *ptab, Eterm pixdata) +{ + Uint32 data = ((Uint32) pixdata) & ~ptab->r.o.pix_mask; + data |= (pixdata >> ptab->r.o.pix_cl_shift) & ptab->r.o.pix_cl_mask; + data |= (pixdata & ptab->r.o.pix_cli_mask) << ptab->r.o.pix_cli_shift; + return data; +} + +ERTS_GLB_INLINE Uint32 erts_ptab_pixdata2pix(ErtsPTab *ptab, Eterm pixdata) +{ + return ((Uint32) pixdata) & ptab->r.o.pix_mask; +} + +ERTS_GLB_INLINE Uint32 erts_ptab_data2pix(ErtsPTab *ptab, Eterm data) +{ + Uint32 n, pix; + n = (Uint32) data; + pix = ((n & ptab->r.o.pix_cl_mask) << ptab->r.o.pix_cl_shift); + pix += ((n >> ptab->r.o.pix_cli_shift) & ptab->r.o.pix_cli_mask); + ASSERT(0 <= pix && pix < ptab->r.o.max); + return pix; +} + +ERTS_GLB_INLINE Uint erts_ptab_data2pixdata(ErtsPTab *ptab, Eterm data) +{ + Uint pixdata = data & ~((Uint) ptab->r.o.pix_mask); + pixdata |= (Uint) erts_ptab_data2pix(ptab, data); + ASSERT(data == erts_ptab_pixdata2data(ptab, pixdata)); + return pixdata; +} + +#if ERTS_SIZEOF_TERM == 8 + +ERTS_GLB_INLINE Eterm +erts_ptab_make_id(ErtsPTab *ptab, Eterm data, Eterm tag) +{ + HUint huint; + Uint32 low_data = (Uint32) data; + low_data &= (1 << ERTS_PTAB_ID_DATA_SIZE) - 1; + low_data <<= ERTS_PTAB_ID_DATA_SHIFT; + huint.hval[ERTS_HUINT_HVAL_HIGH] = erts_ptab_data2pix(ptab, data); + huint.hval[ERTS_HUINT_HVAL_LOW] = low_data | ((Uint32) tag); + return (Eterm) huint.val; +} + +ERTS_GLB_INLINE int +erts_ptab_id2pix(ErtsPTab *ptab, Eterm id) +{ + HUint huint; + huint.val = id; + return (int) huint.hval[ERTS_HUINT_HVAL_HIGH]; +} + +ERTS_GLB_INLINE Uint +erts_ptab_id2data(ErtsPTab *ptab, Eterm id) +{ + HUint huint; + huint.val = id; + return (Uint) (huint.hval[ERTS_HUINT_HVAL_LOW] >> ERTS_PTAB_ID_DATA_SHIFT); +} + +#elif ERTS_SIZEOF_TERM == 4 + +ERTS_GLB_INLINE Eterm +erts_ptab_make_id(ErtsPTab *ptab, Eterm data, Eterm tag) +{ + Eterm id; + data &= ((1 << ERTS_PTAB_ID_DATA_SIZE) - 1); + id = (Eterm) erts_ptab_data2pixdata(ptab, data); + return (id << ERTS_PTAB_ID_DATA_SHIFT) | tag; +} + +ERTS_GLB_INLINE int +erts_ptab_id2pix(ErtsPTab *ptab, Eterm id) +{ + Uint pixdata = (Uint) id; + pixdata >>= ERTS_PTAB_ID_DATA_SHIFT; + return (int) erts_ptab_pixdata2pix(ptab, pixdata); +} + +ERTS_GLB_INLINE Uint +erts_ptab_id2data(ErtsPTab *ptab, Eterm id) +{ + Uint pixdata = (Uint) id; + pixdata >>= ERTS_PTAB_ID_DATA_SHIFT; + return erts_ptab_pixdata2data(ptab, pixdata); +} + +#else +#error "Unsupported size of term" +#endif + +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_nob(ErtsPTab *ptab, int ix) +{ + ASSERT(0 <= ix && ix < ptab->r.o.max); + return erts_smp_atomic_read_nob(&ptab->r.o.tab[ix]); +} + +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_ddrb(ErtsPTab *ptab, int ix) +{ + ASSERT(0 <= ix && ix < ptab->r.o.max); + return erts_smp_atomic_read_ddrb(&ptab->r.o.tab[ix]); +} + +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_rb(ErtsPTab *ptab, int ix) +{ + ASSERT(0 <= ix && ix < ptab->r.o.max); + return erts_smp_atomic_read_rb(&ptab->r.o.tab[ix]); +} + +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_acqb(ErtsPTab *ptab, int ix) +{ + ASSERT(0 <= ix && ix < ptab->r.o.max); + return erts_smp_atomic_read_acqb(&ptab->r.o.tab[ix]); +} + +ERTS_GLB_INLINE void erts_ptab_inc_refc(ErtsPTabElementCommon *ptab_el) +{ +#ifdef ERTS_SMP +#ifdef ERTS_ENABLE_LOCK_CHECK + erts_aint32_t refc = erts_atomic32_inc_read_nob(&ptab_el->refc); + ERTS_SMP_LC_ASSERT(refc > 1); +#else + erts_atomic32_inc_nob(&ptab_el->refc); +#endif +#endif +} + +ERTS_GLB_INLINE int erts_ptab_dec_test_refc(ErtsPTabElementCommon *ptab_el) +{ +#ifdef ERTS_SMP + erts_aint32_t refc = erts_atomic32_dec_read_nob(&ptab_el->refc); + ERTS_SMP_LC_ASSERT(refc >= 0); + return (int) refc; +#else + return 0; +#endif +} + +ERTS_GLB_INLINE int erts_ptab_add_test_refc(ErtsPTabElementCommon *ptab_el, + Sint32 add_refc) +{ +#ifdef ERTS_SMP + erts_aint32_t refc; + +#ifndef ERTS_ENABLE_LOCK_CHECK + if (add_refc >= 0) { + erts_atomic32_add_nob(&ptab_el->refc, + (erts_aint32_t) add_refc); + return 1; + } +#endif + + refc = erts_atomic32_add_read_nob(&ptab_el->refc, + (erts_aint32_t) add_refc); + ERTS_SMP_LC_ASSERT(refc >= 0); + return (int) refc; +#else + return 0; +#endif +} + +ERTS_GLB_INLINE void erts_ptab_rlock(ErtsPTab *ptab) +{ + erts_smp_rwmtx_rlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE int erts_ptab_tryrlock(ErtsPTab *ptab) +{ + return erts_smp_rwmtx_tryrlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE void erts_ptab_runlock(ErtsPTab *ptab) +{ + erts_smp_rwmtx_runlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE void erts_ptab_rwlock(ErtsPTab *ptab) +{ + erts_smp_rwmtx_rwlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE int erts_ptab_tryrwlock(ErtsPTab *ptab) +{ + return erts_smp_rwmtx_tryrwlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE void erts_ptab_rwunlock(ErtsPTab *ptab) +{ + erts_smp_rwmtx_rwunlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE int erts_smp_lc_ptab_is_rlocked(ErtsPTab *ptab) +{ + return erts_smp_lc_rwmtx_is_rlocked(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE int erts_smp_lc_ptab_is_rwlocked(ErtsPTab *ptab) +{ + return erts_smp_lc_rwmtx_is_rwlocked(&ptab->list.data.rwmtx); +} + +#endif + +#endif + +#if defined(ERTS_PTAB_WANT_BIF_IMPL__) && !defined(ERTS_PTAB_LIST__) +#define ERTS_PTAB_LIST__ + +#include "erl_process.h" +#include "bif.h" + +BIF_RETTYPE erts_ptab_list(struct process *c_p, ErtsPTab *ptab); + +#endif + +#if defined(ERTS_PTAB_WANT_DEBUG_FUNCS__) && !defined(ERTS_PTAB_DEBUG_FUNCS__) +#define ERTS_PTAB_DEBUG_FUNCS__ +#include "erl_process.h" + +/* Debug functions */ +Sint erts_ptab_test_next_id(ErtsPTab *ptab, int set, Uint next); +Eterm erts_debug_ptab_list(Process *c_p, ErtsPTab *ptab); +Eterm erts_debug_ptab_list_bif_info(Process *c_p, ErtsPTab *ptab); + +#endif diff --git a/erts/emulator/beam/erl_smp.h b/erts/emulator/beam/erl_smp.h index a32e9d9d7c..34c90c0bda 100644 --- a/erts/emulator/beam/erl_smp.h +++ b/erts/emulator/beam/erl_smp.h @@ -274,6 +274,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_nob erts_atomic_read_band_nob #define erts_smp_atomic_xchg_nob erts_atomic_xchg_nob #define erts_smp_atomic_cmpxchg_nob erts_atomic_cmpxchg_nob +#define erts_smp_atomic_read_bset_nob erts_atomic_read_bset_nob #define erts_smp_atomic_init_mb erts_atomic_init_mb #define erts_smp_atomic_set_mb erts_atomic_set_mb @@ -288,6 +289,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_mb erts_atomic_read_band_mb #define erts_smp_atomic_xchg_mb erts_atomic_xchg_mb #define erts_smp_atomic_cmpxchg_mb erts_atomic_cmpxchg_mb +#define erts_smp_atomic_read_bset_mb erts_atomic_read_bset_mb #define erts_smp_atomic_init_acqb erts_atomic_init_acqb #define erts_smp_atomic_set_acqb erts_atomic_set_acqb @@ -302,6 +304,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_acqb erts_atomic_read_band_acqb #define erts_smp_atomic_xchg_acqb erts_atomic_xchg_acqb #define erts_smp_atomic_cmpxchg_acqb erts_atomic_cmpxchg_acqb +#define erts_smp_atomic_read_bset_acqb erts_atomic_read_bset_acqb #define erts_smp_atomic_init_relb erts_atomic_init_relb #define erts_smp_atomic_set_relb erts_atomic_set_relb @@ -316,6 +319,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_relb erts_atomic_read_band_relb #define erts_smp_atomic_xchg_relb erts_atomic_xchg_relb #define erts_smp_atomic_cmpxchg_relb erts_atomic_cmpxchg_relb +#define erts_smp_atomic_read_bset_relb erts_atomic_read_bset_relb #define erts_smp_atomic_init_ddrb erts_atomic_init_ddrb #define erts_smp_atomic_set_ddrb erts_atomic_set_ddrb @@ -330,6 +334,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_ddrb erts_atomic_read_band_ddrb #define erts_smp_atomic_xchg_ddrb erts_atomic_xchg_ddrb #define erts_smp_atomic_cmpxchg_ddrb erts_atomic_cmpxchg_ddrb +#define erts_smp_atomic_read_bset_ddrb erts_atomic_read_bset_ddrb #define erts_smp_atomic_init_rb erts_atomic_init_rb #define erts_smp_atomic_set_rb erts_atomic_set_rb @@ -344,6 +349,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_rb erts_atomic_read_band_rb #define erts_smp_atomic_xchg_rb erts_atomic_xchg_rb #define erts_smp_atomic_cmpxchg_rb erts_atomic_cmpxchg_rb +#define erts_smp_atomic_read_bset_rb erts_atomic_read_bset_rb #define erts_smp_atomic_init_wb erts_atomic_init_wb #define erts_smp_atomic_set_wb erts_atomic_set_wb @@ -358,6 +364,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_wb erts_atomic_read_band_wb #define erts_smp_atomic_xchg_wb erts_atomic_xchg_wb #define erts_smp_atomic_cmpxchg_wb erts_atomic_cmpxchg_wb +#define erts_smp_atomic_read_bset_wb erts_atomic_read_bset_wb /* 32-bit atomics */ @@ -374,6 +381,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_nob erts_atomic32_read_band_nob #define erts_smp_atomic32_xchg_nob erts_atomic32_xchg_nob #define erts_smp_atomic32_cmpxchg_nob erts_atomic32_cmpxchg_nob +#define erts_smp_atomic32_read_bset_nob erts_atomic32_read_bset_nob #define erts_smp_atomic32_init_mb erts_atomic32_init_mb #define erts_smp_atomic32_set_mb erts_atomic32_set_mb @@ -388,6 +396,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_mb erts_atomic32_read_band_mb #define erts_smp_atomic32_xchg_mb erts_atomic32_xchg_mb #define erts_smp_atomic32_cmpxchg_mb erts_atomic32_cmpxchg_mb +#define erts_smp_atomic32_read_bset_mb erts_atomic32_read_bset_mb #define erts_smp_atomic32_init_acqb erts_atomic32_init_acqb #define erts_smp_atomic32_set_acqb erts_atomic32_set_acqb @@ -402,6 +411,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_acqb erts_atomic32_read_band_acqb #define erts_smp_atomic32_xchg_acqb erts_atomic32_xchg_acqb #define erts_smp_atomic32_cmpxchg_acqb erts_atomic32_cmpxchg_acqb +#define erts_smp_atomic32_read_bset_acqb erts_atomic32_read_bset_acqb #define erts_smp_atomic32_init_relb erts_atomic32_init_relb #define erts_smp_atomic32_set_relb erts_atomic32_set_relb @@ -416,6 +426,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_relb erts_atomic32_read_band_relb #define erts_smp_atomic32_xchg_relb erts_atomic32_xchg_relb #define erts_smp_atomic32_cmpxchg_relb erts_atomic32_cmpxchg_relb +#define erts_smp_atomic32_read_bset_relb erts_atomic32_read_bset_relb #define erts_smp_atomic32_init_ddrb erts_atomic32_init_ddrb #define erts_smp_atomic32_set_ddrb erts_atomic32_set_ddrb @@ -430,6 +441,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_ddrb erts_atomic32_read_band_ddrb #define erts_smp_atomic32_xchg_ddrb erts_atomic32_xchg_ddrb #define erts_smp_atomic32_cmpxchg_ddrb erts_atomic32_cmpxchg_ddrb +#define erts_smp_atomic32_read_bset_ddrb erts_atomic32_read_bset_ddrb #define erts_smp_atomic32_init_rb erts_atomic32_init_rb #define erts_smp_atomic32_set_rb erts_atomic32_set_rb @@ -444,6 +456,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_rb erts_atomic32_read_band_rb #define erts_smp_atomic32_xchg_rb erts_atomic32_xchg_rb #define erts_smp_atomic32_cmpxchg_rb erts_atomic32_cmpxchg_rb +#define erts_smp_atomic32_read_bset_rb erts_atomic32_read_bset_rb #define erts_smp_atomic32_init_wb erts_atomic32_init_wb #define erts_smp_atomic32_set_wb erts_atomic32_set_wb @@ -458,6 +471,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_wb erts_atomic32_read_band_wb #define erts_smp_atomic32_xchg_wb erts_atomic32_xchg_wb #define erts_smp_atomic32_cmpxchg_wb erts_atomic32_cmpxchg_wb +#define erts_smp_atomic32_read_bset_wb erts_atomic32_read_bset_wb #else /* !ERTS_SMP */ @@ -513,6 +527,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_nob erts_no_atomic_read_band #define erts_smp_atomic_xchg_nob erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_nob erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_nob erts_no_atomic_read_bset #define erts_smp_atomic_init_mb erts_no_atomic_set #define erts_smp_atomic_set_mb erts_no_atomic_set @@ -527,6 +542,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_mb erts_no_atomic_read_band #define erts_smp_atomic_xchg_mb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_mb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_mb erts_no_atomic_read_bset #define erts_smp_atomic_init_acqb erts_no_atomic_set #define erts_smp_atomic_set_acqb erts_no_atomic_set @@ -541,6 +557,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_acqb erts_no_atomic_read_band #define erts_smp_atomic_xchg_acqb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_acqb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_acqb erts_no_atomic_read_bset #define erts_smp_atomic_init_relb erts_no_atomic_set #define erts_smp_atomic_set_relb erts_no_atomic_set @@ -555,6 +572,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_relb erts_no_atomic_read_band #define erts_smp_atomic_xchg_relb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_relb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_relb erts_no_atomic_read_bset #define erts_smp_atomic_init_ddrb erts_no_atomic_set #define erts_smp_atomic_set_ddrb erts_no_atomic_set @@ -569,6 +587,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_ddrb erts_no_atomic_read_band #define erts_smp_atomic_xchg_ddrb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_ddrb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_ddrb erts_no_atomic_read_bset #define erts_smp_atomic_init_rb erts_no_atomic_set #define erts_smp_atomic_set_rb erts_no_atomic_set @@ -583,6 +602,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_rb erts_no_atomic_read_band #define erts_smp_atomic_xchg_rb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_rb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_rb erts_no_atomic_read_bset #define erts_smp_atomic_init_wb erts_no_atomic_set #define erts_smp_atomic_set_wb erts_no_atomic_set @@ -597,6 +617,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_wb erts_no_atomic_read_band #define erts_smp_atomic_xchg_wb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_wb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_wb erts_no_atomic_read_bset /* 32-bit atomics */ @@ -613,6 +634,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_nob erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_nob erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_nob erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_nob erts_no_atomic32_read_bset #define erts_smp_atomic32_init_mb erts_no_atomic32_set #define erts_smp_atomic32_set_mb erts_no_atomic32_set @@ -627,6 +649,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_mb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_mb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_mb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_mb erts_no_atomic32_read_bset #define erts_smp_atomic32_init_acqb erts_no_atomic32_set #define erts_smp_atomic32_set_acqb erts_no_atomic32_set @@ -641,6 +664,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_acqb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_acqb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_acqb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_acqb erts_no_atomic32_read_bset #define erts_smp_atomic32_init_relb erts_no_atomic32_set #define erts_smp_atomic32_set_relb erts_no_atomic32_set @@ -655,6 +679,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_relb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_relb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_relb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_relb erts_no_atomic32_read_bset #define erts_smp_atomic32_init_ddrb erts_no_atomic32_set #define erts_smp_atomic32_set_ddrb erts_no_atomic32_set @@ -669,6 +694,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_ddrb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_ddrb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_ddrb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_ddrb erts_no_atomic32_read_bset #define erts_smp_atomic32_init_rb erts_no_atomic32_set #define erts_smp_atomic32_set_rb erts_no_atomic32_set @@ -683,6 +709,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_rb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_rb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_rb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_rb erts_no_atomic32_read_bset #define erts_smp_atomic32_init_wb erts_no_atomic32_set #define erts_smp_atomic32_set_wb erts_no_atomic32_set @@ -697,6 +724,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_wb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_wb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_wb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_wb erts_no_atomic32_read_bset #endif /* !ERTS_SMP */ diff --git a/erts/emulator/beam/erl_sys_driver.h b/erts/emulator/beam/erl_sys_driver.h index d429d0ce96..b991a2840c 100644 --- a/erts/emulator/beam/erl_sys_driver.h +++ b/erts/emulator/beam/erl_sys_driver.h @@ -31,7 +31,6 @@ #define ERL_SYS_DRV typedef long ErlDrvEvent; /* An event to be selected on. */ -typedef long ErlDrvPort; /* A port descriptor. */ /* typedef struct _SysDriverOpts SysDriverOpts; defined in sys.h */ diff --git a/erts/emulator/beam/erl_term.c b/erts/emulator/beam/erl_term.c index bf7774f882..4587cd84d1 100644 --- a/erts/emulator/beam/erl_term.c +++ b/erts/emulator/beam/erl_term.c @@ -133,7 +133,7 @@ ET_DEFINE_CHECKED(Uint,unsigned_val,Eterm,is_small); ET_DEFINE_CHECKED(Sint,signed_val,Eterm,is_small); ET_DEFINE_CHECKED(Uint,atom_val,Eterm,is_atom); ET_DEFINE_CHECKED(Uint,header_arity,Eterm,is_header); -ET_DEFINE_CHECKED(Uint,arityval,Eterm,is_arity_value); +ET_DEFINE_CHECKED(Uint,arityval,Eterm,is_sane_arity_value); ET_DEFINE_CHECKED(Uint,thing_arityval,Eterm,is_thing); ET_DEFINE_CHECKED(Uint,thing_subtag,Eterm,is_thing); ET_DEFINE_CHECKED(Eterm*,binary_val,Wterm,is_binary); @@ -144,9 +144,7 @@ ET_DEFINE_CHECKED(Uint,bignum_header_arity,Eterm,_is_bignum_header); ET_DEFINE_CHECKED(Eterm*,big_val,Wterm,is_big); ET_DEFINE_CHECKED(Eterm*,float_val,Wterm,is_float); ET_DEFINE_CHECKED(Eterm*,tuple_val,Wterm,is_tuple); -ET_DEFINE_CHECKED(Uint,internal_pid_data,Eterm,is_internal_pid); ET_DEFINE_CHECKED(struct erl_node_*,internal_pid_node,Eterm,is_internal_pid); -ET_DEFINE_CHECKED(Uint,internal_port_data,Eterm,is_internal_port); ET_DEFINE_CHECKED(struct erl_node_*,internal_port_node,Eterm,is_internal_port); ET_DEFINE_CHECKED(Eterm*,internal_ref_val,Wterm,is_internal_ref); ET_DEFINE_CHECKED(Uint,internal_ref_data_words,Wterm,is_internal_ref); diff --git a/erts/emulator/beam/erl_term.h b/erts/emulator/beam/erl_term.h index c270d13365..fb3ef9cd6c 100644 --- a/erts/emulator/beam/erl_term.h +++ b/erts/emulator/beam/erl_term.h @@ -300,8 +300,17 @@ _ET_DECLARE_CHECKED(Uint,header_arity,Eterm) #define header_arity(x) _ET_APPLY(header_arity,(x)) /* arityval access methods */ +/* Erlang Spec. 4.7.3 defines max arity to 65535 + * we will however enforce max arity of 16777215 (24 bits) + * (checked in bifs and asserted in debug) + */ +#define MAX_ARITYVAL ((((Uint)1) << 24) - 1) +#define ERTS_MAX_TUPLE_SIZE MAX_ARITYVAL + #define make_arityval(sz) _make_header((sz),_TAG_HEADER_ARITYVAL) #define is_arity_value(x) (((x) & _TAG_HEADER_MASK) == _TAG_HEADER_ARITYVAL) +#define is_sane_arity_value(x) ((((x) & _TAG_HEADER_MASK) == _TAG_HEADER_ARITYVAL) && \ + (((x) >> _HEADER_ARITY_OFFS) <= MAX_ARITYVAL)) #define is_not_arity_value(x) (!is_arity_value((x))) #define _unchecked_arityval(x) _unchecked_header_arity((x)) _ET_DECLARE_CHECKED(Uint,arityval,Eterm) @@ -542,12 +551,6 @@ _ET_DECLARE_CHECKED(Eterm*,tuple_val,Wterm) #define _GETBITS(X,Pos,Size) (((X) >> (Pos)) & ~(~((Uint) 0) << (Size))) /* - * Observe! New layout for pids, ports and references in R9 (see also note - * in erl_node_container_utils.h). - */ - - -/* * Creation in node specific data (pids, ports, refs) */ @@ -584,7 +587,6 @@ _ET_DECLARE_CHECKED(Eterm*,tuple_val,Wterm) * */ -#define _PID_R9_SER_SIZE 3 #define _PID_SER_SIZE (_PID_DATA_SIZE - _PID_NUM_SIZE) #define _PID_NUM_SIZE 15 @@ -598,23 +600,13 @@ _ET_DECLARE_CHECKED(Eterm*,tuple_val,Wterm) #define make_pid_data(Ser, Num) \ ((Uint) ((Ser) << _PID_NUM_SIZE | (Num))) -#define make_internal_pid(X) \ - ((Eterm) (((X) << _PID_DATA_SHIFT) | _TAG_IMMED1_PID)) - #define is_internal_pid(x) (((x) & _TAG_IMMED1_MASK) == _TAG_IMMED1_PID) #define is_not_internal_pid(x) (!is_internal_pid((x))) -#define _unchecked_internal_pid_data(x) _GET_PID_DATA((x)) -_ET_DECLARE_CHECKED(Uint,internal_pid_data,Eterm) -#define internal_pid_data(x) _ET_APPLY(internal_pid_data,(x)) - #define _unchecked_internal_pid_node(x) erts_this_node _ET_DECLARE_CHECKED(struct erl_node_*,internal_pid_node,Eterm) #define internal_pid_node(x) _ET_APPLY(internal_pid_node,(x)) -#define internal_pid_number(x) _GET_PID_NUM(internal_pid_data((x))) -#define internal_pid_serial(x) _GET_PID_SER(internal_pid_data((x))) - #define internal_pid_data_words(x) (1) /* @@ -644,7 +636,6 @@ _ET_DECLARE_CHECKED(struct erl_node_*,internal_pid_node,Eterm) * N : node number * */ -#define _PORT_R9_NUM_SIZE 18 #define _PORT_NUM_SIZE _PORT_DATA_SIZE #define _PORT_DATA_SIZE 28 @@ -654,18 +645,9 @@ _ET_DECLARE_CHECKED(struct erl_node_*,internal_pid_node,Eterm) #define _GET_PORT_NUM(X) _GETBITS((X), 0, _PORT_NUM_SIZE) -#define make_internal_port(X) \ - ((Eterm) (((X) << _PORT_DATA_SHIFT) | _TAG_IMMED1_PORT)) - #define is_internal_port(x) (((x) & _TAG_IMMED1_MASK) == _TAG_IMMED1_PORT) #define is_not_internal_port(x) (!is_internal_port(x)) -#define _unchecked_internal_port_data(x) _GET_PORT_DATA((x)) -_ET_DECLARE_CHECKED(Uint,internal_port_data,Eterm) -#define internal_port_data(x) _ET_APPLY(internal_port_data,(x)) - -#define internal_port_number(x) _GET_PORT_NUM(internal_port_data((x))) - #define _unchecked_internal_port_node(x) erts_this_node _ET_DECLARE_CHECKED(struct erl_node_*,internal_port_node,Eterm) #define internal_port_node(x) _ET_APPLY(internal_port_node,(x)) diff --git a/erts/emulator/beam/erl_thr_progress.c b/erts/emulator/beam/erl_thr_progress.c index 88524bdd4c..9678d7e08b 100644 --- a/erts/emulator/beam/erl_thr_progress.c +++ b/erts/emulator/beam/erl_thr_progress.c @@ -96,17 +96,14 @@ #define ERTS_THR_PRGR_LFLG_BLOCK (((erts_aint32_t) 1) << 31) #define ERTS_THR_PRGR_LFLG_NO_LEADER (((erts_aint32_t) 1) << 30) -#define ERTS_THR_PRGR_LFLG_ACTIVE_MASK (~(ERTS_THR_PRGR_LFLG_NO_LEADER \ - | ERTS_THR_PRGR_LFLG_BLOCK)) +#define ERTS_THR_PRGR_LFLG_WAITING_UM (((erts_aint32_t) 1) << 29) +#define ERTS_THR_PRGR_LFLG_ACTIVE_MASK (~(ERTS_THR_PRGR_LFLG_NO_LEADER \ + | ERTS_THR_PRGR_LFLG_BLOCK \ + | ERTS_THR_PRGR_LFLG_WAITING_UM)) -#define ERTS_THR_PRGR_LFLGS_ACTIVE(LFLGS) \ +#define ERTS_THR_PRGR_LFLGS_ACTIVE(LFLGS) \ ((LFLGS) & ERTS_THR_PRGR_LFLG_ACTIVE_MASK) -#define ERTS_THR_PRGR_LFLGS_ALL_WAITING(LFLGS) \ - (((LFLGS) & (ERTS_THR_PRGR_LFLG_NO_LEADER \ - |ERTS_THR_PRGR_LFLG_ACTIVE_MASK)) \ - == ERTS_THR_PRGR_LFLG_NO_LEADER) - /* * We use a 64-bit value for thread progress. By this wrapping of * the thread progress will more or less never occur. @@ -262,6 +259,11 @@ typedef struct { erts_atomic32_t managed_count; erts_atomic32_t managed_id; erts_atomic32_t unmanaged_id; + int chk_next_ix; + struct { + int waiting; + erts_atomic32_t current; + } umrefc_ix; } ErtsThrPrgrMiscData; typedef struct { @@ -276,12 +278,18 @@ typedef union { char align__[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(ErtsThrPrgrElement))]; } ErtsThrPrgrArray; +typedef union { + erts_atomic_t refc; + char align__[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(erts_atomic_t))]; +} ErtsThrPrgrUnmanagedRefc; + typedef struct { union { ErtsThrPrgrMiscData data; char align__[ERTS_ALC_CACHE_LINE_ALIGN_SIZE( sizeof(ErtsThrPrgrMiscData))]; } misc; + ErtsThrPrgrUnmanagedRefc umrefc[2]; ErtsThrPrgrArray *thr; struct { int no; @@ -346,7 +354,9 @@ init_tmp_thr_prgr_data(ErtsThrPrgrData *tpd) tpd->is_managed = 0; tpd->is_blocking = 0; tpd->is_temporary = 1; - +#ifdef ERTS_ENABLE_LOCK_CHECK + tpd->is_delaying = 0; +#endif erts_tsd_set(erts_thr_prgr_data_key__, (void *) tpd); } @@ -461,6 +471,12 @@ erts_thr_progress_init(int no_schedulers, int managed, int unmanaged) erts_atomic32_init_nob(&intrnl->misc.data.managed_count, 0); erts_atomic32_init_nob(&intrnl->misc.data.managed_id, no_schedulers); erts_atomic32_init_nob(&intrnl->misc.data.unmanaged_id, -1); + intrnl->misc.data.chk_next_ix = 0; + intrnl->misc.data.umrefc_ix.waiting = -1; + erts_atomic32_init_nob(&intrnl->misc.data.umrefc_ix.current, 0); + + erts_atomic_init_nob(&intrnl->umrefc[0].refc, (erts_aint_t) 0); + erts_atomic_init_nob(&intrnl->umrefc[1].refc, (erts_aint_t) 0); intrnl->thr = (ErtsThrPrgrArray *) ptr; ptr += thr_arr_sz; @@ -547,6 +563,9 @@ erts_thr_progress_register_unmanaged_thread(ErtsThrPrgrCallbacks *callbacks) tpd->is_managed = 0; tpd->is_blocking = is_blocking; tpd->is_temporary = 0; +#ifdef ERTS_ENABLE_LOCK_CHECK + tpd->is_delaying = 0; +#endif ASSERT(tpd->id >= 0); if (tpd->id >= intrnl->unmanaged.no) erl_exit(ERTS_ABORT_EXIT, @@ -600,6 +619,9 @@ erts_thr_progress_register_managed_thread(ErtsSchedulerData *esdp, tpd->is_managed = 1; tpd->is_blocking = is_blocking; tpd->is_temporary = 0; +#ifdef ERTS_ENABLE_LOCK_CHECK + tpd->is_delaying = 1; +#endif init_wakeup_request_array(&tpd->wakeup_request[0]); @@ -607,8 +629,8 @@ erts_thr_progress_register_managed_thread(ErtsSchedulerData *esdp, tpd->leader = 0; tpd->active = 1; - tpd->previous.local = 0; - tpd->previous.current = ERTS_THR_PRGR_VAL_WAITING; + tpd->confirmed = 0; + tpd->leader_state.current = ERTS_THR_PRGR_VAL_WAITING; erts_tsd_set(erts_thr_prgr_data_key__, (void *) tpd); erts_atomic32_inc_nob(&intrnl->misc.data.lflgs); @@ -651,60 +673,113 @@ leader_update(ErtsThrPrgrData *tpd) block_thread(tpd); } else { + ErtsThrPrgrVal current; + int ix, chk_next_ix, umrefc_ix, my_ix, no_managed, waiting_unmanaged; erts_aint32_t lflgs; ErtsThrPrgrVal next; - int ix, sz, make_progress; + erts_aint_t refc; - if (tpd->previous.current == ERTS_THR_PRGR_VAL_WAITING) { - /* Took over as leader from another thread */ - tpd->previous.current = read_acqb(&erts_thr_prgr__.current); - tpd->previous.next = tpd->previous.current; - tpd->previous.next++; - if (tpd->previous.next == ERTS_THR_PRGR_VAL_WAITING) - tpd->previous.next = 0; - } + my_ix = tpd->id; - if (tpd->previous.local == tpd->previous.current) { - ErtsThrPrgrVal val = tpd->previous.current + 1; - if (val == ERTS_THR_PRGR_VAL_WAITING) - val = 0; - tpd->previous.local = val; - set_mb(&intrnl->thr[tpd->id].data.current, val); + if (tpd->leader_state.current == ERTS_THR_PRGR_VAL_WAITING) { + /* Took over as leader from another thread */ + tpd->leader_state.current = read_nob(&erts_thr_prgr__.current); + tpd->leader_state.next = tpd->leader_state.current; + tpd->leader_state.next++; + if (tpd->leader_state.next == ERTS_THR_PRGR_VAL_WAITING) + tpd->leader_state.next = 0; + tpd->leader_state.chk_next_ix = intrnl->misc.data.chk_next_ix; + tpd->leader_state.umrefc_ix.waiting = intrnl->misc.data.umrefc_ix.waiting; + tpd->leader_state.umrefc_ix.current = + (int) erts_atomic32_read_nob(&intrnl->misc.data.umrefc_ix.current); + + if (tpd->confirmed == tpd->leader_state.current) { + ErtsThrPrgrVal val = tpd->leader_state.current + 1; + if (val == ERTS_THR_PRGR_VAL_WAITING) + val = 0; + tpd->confirmed = val; + set_mb(&intrnl->thr[my_ix].data.current, val); + } } - next = tpd->previous.next; - make_progress = 1; - sz = intrnl->managed.no; - for (ix = 0; ix < sz; ix++) { - ErtsThrPrgrVal tmp; - tmp = read_nob(&intrnl->thr[ix].data.current); - if (tmp != next && tmp != ERTS_THR_PRGR_VAL_WAITING) { - make_progress = 0; - ASSERT(erts_thr_progress_has_passed__(next, tmp)); - break; + next = tpd->leader_state.next; + + waiting_unmanaged = 0; + umrefc_ix = -1; /* Shut up annoying warning */ + + chk_next_ix = tpd->leader_state.chk_next_ix; + no_managed = intrnl->managed.no; + ASSERT(0 <= chk_next_ix && chk_next_ix <= no_managed); + /* Check manged threads */ + if (chk_next_ix < no_managed) { + for (ix = chk_next_ix; ix < no_managed; ix++) { + ErtsThrPrgrVal tmp; + if (ix == my_ix) + continue; + tmp = read_nob(&intrnl->thr[ix].data.current); + if (tmp != next && tmp != ERTS_THR_PRGR_VAL_WAITING) { + tpd->leader_state.chk_next_ix = ix; + ASSERT(erts_thr_progress_has_passed__(next, tmp)); + goto done; + } } } - if (make_progress) { - ErtsThrPrgrVal current = next; + /* Check unmanged threads */ + waiting_unmanaged = tpd->leader_state.umrefc_ix.waiting != -1; + umrefc_ix = (waiting_unmanaged + ? tpd->leader_state.umrefc_ix.waiting + : tpd->leader_state.umrefc_ix.current); + refc = erts_atomic_read_nob(&intrnl->umrefc[umrefc_ix].refc); + ASSERT(refc >= 0); + if (refc != 0) { + int new_umrefc_ix; + + if (waiting_unmanaged) + goto done; + + new_umrefc_ix = (umrefc_ix + 1) & 0x1; + tpd->leader_state.umrefc_ix.waiting = umrefc_ix; + tpd->leader_state.chk_next_ix = no_managed; + erts_atomic32_set_nob(&intrnl->misc.data.umrefc_ix.current, + (erts_aint32_t) new_umrefc_ix); + ETHR_MEMBAR(ETHR_StoreLoad); + refc = erts_atomic_read_nob(&intrnl->umrefc[umrefc_ix].refc); + ASSERT(refc >= 0); + waiting_unmanaged = 1; + if (refc != 0) + goto done; + } - next++; - if (next == ERTS_THR_PRGR_VAL_WAITING) - next = 0; + /* Make progress */ + current = next; - set_nob(&intrnl->thr[tpd->id].data.current, next); - set_mb(&erts_thr_prgr__.current, current); - tpd->previous.local = next; - tpd->previous.next = next; - tpd->previous.current = current; + next++; + if (next == ERTS_THR_PRGR_VAL_WAITING) + next = 0; + + set_nob(&intrnl->thr[my_ix].data.current, next); + set_mb(&erts_thr_prgr__.current, current); + tpd->confirmed = next; + tpd->leader_state.next = next; + tpd->leader_state.current = current; #if ERTS_THR_PRGR_PRINT_VAL - if (current % 1000 == 0) - erts_fprintf(stderr, "%b64u\n", current); + if (current % 1000 == 0) + erts_fprintf(stderr, "%b64u\n", current); #endif - handle_wakeup_requests(current); + handle_wakeup_requests(current); + + if (waiting_unmanaged) { + waiting_unmanaged = 0; + tpd->leader_state.umrefc_ix.waiting = -1; + erts_atomic32_read_band_nob(&intrnl->misc.data.lflgs, + ~ERTS_THR_PRGR_LFLG_WAITING_UM); } + tpd->leader_state.chk_next_ix = 0; + + done: if (tpd->active) { lflgs = erts_atomic32_read_nob(&intrnl->misc.data.lflgs); @@ -712,20 +787,44 @@ leader_update(ErtsThrPrgrData *tpd) (void) block_thread(tpd); } else { + int force_wakeup_check = 0; + erts_aint32_t set_flags = ERTS_THR_PRGR_LFLG_NO_LEADER; tpd->leader = 0; - tpd->previous.current = ERTS_THR_PRGR_VAL_WAITING; + tpd->leader_state.current = ERTS_THR_PRGR_VAL_WAITING; #if ERTS_THR_PRGR_PRINT_LEADER erts_fprintf(stderr, "L <- %d\n", tpd->id); #endif ERTS_THR_PROGRESS_STATE_DEBUG_SET_LEADER(tpd->id, 0); + if (waiting_unmanaged) + set_flags |= ERTS_THR_PRGR_LFLG_WAITING_UM; + lflgs = erts_atomic32_read_bor_relb(&intrnl->misc.data.lflgs, - ERTS_THR_PRGR_LFLG_NO_LEADER); + set_flags); + lflgs |= set_flags; if (lflgs & ERTS_THR_PRGR_LFLG_BLOCK) lflgs = block_thread(tpd); - if (ERTS_THR_PRGR_LFLGS_ACTIVE(lflgs) == 0 && got_sched_wakeups()) + + if (waiting_unmanaged) { + /* Need to check umrefc again */ + ETHR_MEMBAR(ETHR_StoreLoad); + refc = erts_atomic_read_nob(&intrnl->umrefc[umrefc_ix].refc); + if (refc == 0) { + /* Need to force wakeup check */ + force_wakeup_check = 1; + } + } + + if ((force_wakeup_check + || ((lflgs & (ERTS_THR_PRGR_LFLG_NO_LEADER + | ERTS_THR_PRGR_LFLG_WAITING_UM + | ERTS_THR_PRGR_LFLG_ACTIVE_MASK)) + == ERTS_THR_PRGR_LFLG_NO_LEADER)) + && got_sched_wakeups()) { + /* Someone need to make progress */ wakeup_managed(0); + } } } @@ -744,11 +843,11 @@ update(ErtsThrPrgrData *tpd) erts_aint32_t lflgs; res = 0; val = read_acqb(&erts_thr_prgr__.current); - if (tpd->previous.local == val) { + if (tpd->confirmed == val) { val++; if (val == ERTS_THR_PRGR_VAL_WAITING) val = 0; - tpd->previous.local = val; + tpd->confirmed = val; set_mb(&intrnl->thr[tpd->id].data.current, val); } @@ -801,12 +900,19 @@ erts_thr_progress_prepare_wait(ErtsSchedulerData *esdp) block_count_dec(); - tpd->previous.local = ERTS_THR_PRGR_VAL_WAITING; + tpd->confirmed = ERTS_THR_PRGR_VAL_WAITING; set_mb(&intrnl->thr[tpd->id].data.current, ERTS_THR_PRGR_VAL_WAITING); lflgs = erts_atomic32_read_nob(&intrnl->misc.data.lflgs); - if (ERTS_THR_PRGR_LFLGS_ALL_WAITING(lflgs) && got_sched_wakeups()) - wakeup_managed(0); /* Someone need to make progress */ + + if ((lflgs & (ERTS_THR_PRGR_LFLG_NO_LEADER + | ERTS_THR_PRGR_LFLG_WAITING_UM + | ERTS_THR_PRGR_LFLG_ACTIVE_MASK)) + == ERTS_THR_PRGR_LFLG_NO_LEADER + && got_sched_wakeups()) { + /* Someone need to make progress */ + wakeup_managed(0); + } } void @@ -828,7 +934,7 @@ erts_thr_progress_finalize_wait(ErtsSchedulerData *esdp) val++; if (val == ERTS_THR_PRGR_VAL_WAITING) val = 0; - tpd->previous.local = val; + tpd->confirmed = val; set_mb(&intrnl->thr[tpd->id].data.current, val); val = read_acqb(&erts_thr_prgr__.current); if (current == val) @@ -875,6 +981,68 @@ erts_thr_progress_active(ErtsSchedulerData *esdp, int on) } +static ERTS_INLINE void +unmanaged_continue(ErtsThrPrgrDelayHandle handle) +{ + int umrefc_ix = (int) handle; + erts_aint_t refc; + + ASSERT(umrefc_ix == 0 || umrefc_ix == 1); + refc = erts_atomic_dec_read_relb(&intrnl->umrefc[umrefc_ix].refc); + ASSERT(refc >= 0); + if (refc == 0) { + erts_aint_t lflgs; + ERTS_THR_READ_MEMORY_BARRIER; + lflgs = erts_atomic32_read_nob(&intrnl->misc.data.lflgs); + if ((lflgs & (ERTS_THR_PRGR_LFLG_NO_LEADER + | ERTS_THR_PRGR_LFLG_WAITING_UM + | ERTS_THR_PRGR_LFLG_ACTIVE_MASK)) + == (ERTS_THR_PRGR_LFLG_NO_LEADER|ERTS_THR_PRGR_LFLG_WAITING_UM) + && got_sched_wakeups()) { + /* Others waiting for us... */ + wakeup_managed(0); + } + } +} + +void +erts_thr_progress_unmanaged_continue__(ErtsThrPrgrDelayHandle handle) +{ +#ifdef ERTS_ENABLE_LOCK_CHECK + ErtsThrPrgrData *tpd = perhaps_thr_prgr_data(NULL); + ERTS_LC_ASSERT(tpd && tpd->is_delaying); + tpd->is_delaying = 0; + return_tmp_thr_prgr_data(tpd); +#endif + ASSERT(!erts_thr_progress_is_managed_thread()); + + unmanaged_continue(handle); +} + +ErtsThrPrgrDelayHandle +erts_thr_progress_unmanaged_delay__(void) +{ + int umrefc_ix; + ASSERT(!erts_thr_progress_is_managed_thread()); + umrefc_ix = (int) erts_atomic32_read_acqb(&intrnl->misc.data.umrefc_ix.current); + while (1) { + int tmp_ix; + erts_atomic_inc_acqb(&intrnl->umrefc[umrefc_ix].refc); + tmp_ix = (int) erts_atomic32_read_acqb(&intrnl->misc.data.umrefc_ix.current); + if (tmp_ix == umrefc_ix) + break; + unmanaged_continue(umrefc_ix); + umrefc_ix = tmp_ix; + } +#ifdef ERTS_ENABLE_LOCK_CHECK + { + ErtsThrPrgrData *tpd = tmp_thr_prgr_data(NULL); + tpd->is_delaying = 1; + } +#endif + return (ErtsThrPrgrDelayHandle) umrefc_ix; +} + static ERTS_INLINE int has_reached_wakeup(ErtsThrPrgrVal wakeup) { @@ -931,7 +1099,7 @@ request_wakeup_managed(ErtsThrPrgrData *tpd, ErtsThrPrgrVal value) */ ASSERT(tpd->is_managed); - ASSERT(tpd->previous.local != ERTS_THR_PRGR_VAL_WAITING); + ASSERT(tpd->confirmed != ERTS_THR_PRGR_VAL_WAITING); if (has_reached_wakeup(value)) { wakeup_managed(tpd->id); @@ -946,7 +1114,7 @@ request_wakeup_managed(ErtsThrPrgrData *tpd, ErtsThrPrgrVal value) tpd->wakeup_request[wix])); - if (tpd->previous.local == value) { + if (tpd->confirmed == value) { /* * We have already confirmed this value. We need to request * wakeup for a value later than our latest confirmed value in diff --git a/erts/emulator/beam/erl_thr_progress.h b/erts/emulator/beam/erl_thr_progress.h index e72321cf48..1416aa6166 100644 --- a/erts/emulator/beam/erl_thr_progress.h +++ b/erts/emulator/beam/erl_thr_progress.h @@ -53,9 +53,22 @@ typedef Uint64 ErtsThrPrgrVal; #define ERTS_THR_PRGR_WAKEUP_DATA_SIZE 4 /* Need to be an even power of 2. */ typedef struct { + ErtsThrPrgrVal next; + ErtsThrPrgrVal current; + int chk_next_ix; + struct { + int current; + int waiting; + } umrefc_ix; +} ErtsThrPrgrLeaderState; + +typedef struct { int id; int is_managed; int is_blocking; +#ifdef ERTS_ENABLE_LOCK_CHECK + int is_delaying; /* managed is always delaying */ +#endif int is_temporary; /* --- Part below only for registered threads --- */ @@ -66,11 +79,8 @@ typedef struct { int leader; /* Needs to be first in the managed threads part */ int active; - struct { - ErtsThrPrgrVal local; - ErtsThrPrgrVal next; - ErtsThrPrgrVal current; - } previous; + ErtsThrPrgrVal confirmed; + ErtsThrPrgrLeaderState leader_state; } ErtsThrPrgrData; void erts_thr_progress_fatal_error_block(SWord timeout, @@ -121,6 +131,11 @@ typedef struct { ERTS_THR_PRGR_ATOMIC current; } ErtsThrPrgr; +typedef int ErtsThrPrgrDelayHandle; +#define ERTS_THR_PRGR_DHANDLE_MANAGED ((ErtsThrPrgrDelayHandle) -1) +/* ERTS_THR_PRGR_DHANDLE_MANAGED implies managed thread */ +#define ERTS_THR_PRGR_DHANDLE_INVALID ((ErtsThrPrgrDelayHandle) -2) + extern ErtsThrPrgr erts_thr_prgr__; void erts_thr_progress_pre_init(void); @@ -136,6 +151,8 @@ int erts_thr_progress_update(ErtsSchedulerData *esdp); int erts_thr_progress_leader_update(ErtsSchedulerData *esdp); void erts_thr_progress_prepare_wait(ErtsSchedulerData *esdp); void erts_thr_progress_finalize_wait(ErtsSchedulerData *esdp); +ErtsThrPrgrDelayHandle erts_thr_progress_unmanaged_delay__(void); +void erts_thr_progress_unmanaged_continue__(int umrefc_ix); void erts_thr_progress_dbg_print_state(void); @@ -148,6 +165,11 @@ ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_prgr_read_acqb__(ERTS_THR_PRGR_ATOMIC *a ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_prgr_read_mb__(ERTS_THR_PRGR_ATOMIC *atmc); ERTS_GLB_INLINE int erts_thr_progress_is_managed_thread(void); +ERTS_GLB_INLINE ErtsThrPrgrDelayHandle erts_thr_progress_unmanaged_delay(void); +ERTS_GLB_INLINE void erts_thr_progress_unmanaged_continue(ErtsThrPrgrDelayHandle handle); +#ifdef ERTS_ENABLE_LOCK_CHECK +ERTS_GLB_INLINE int erts_thr_progress_lc_is_delaying(void); +#endif ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_progress_current_to_later__(ErtsThrPrgrVal val); ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_progress_later(ErtsSchedulerData *); ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_progress_current(void); @@ -229,6 +251,35 @@ erts_thr_progress_is_managed_thread(void) return tpd && tpd->is_managed; } +ERTS_GLB_INLINE ErtsThrPrgrDelayHandle +erts_thr_progress_unmanaged_delay(void) +{ + if (erts_thr_progress_is_managed_thread()) + return ERTS_THR_PRGR_DHANDLE_MANAGED; /* Nothing to do */ + else + return erts_thr_progress_unmanaged_delay__(); +} + +ERTS_GLB_INLINE void +erts_thr_progress_unmanaged_continue(ErtsThrPrgrDelayHandle handle) +{ + ASSERT(handle != ERTS_THR_PRGR_DHANDLE_MANAGED + || erts_thr_progress_is_managed_thread()); + if (handle != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue__(handle); +} + +#ifdef ERTS_ENABLE_LOCK_CHECK + +ERTS_GLB_INLINE int +erts_thr_progress_lc_is_delaying(void) +{ + ErtsThrPrgrData *tpd = erts_tsd_get(erts_thr_prgr_data_key__); + return tpd && tpd->is_delaying; +} + +#endif + ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_progress_current_to_later__(ErtsThrPrgrVal val) { @@ -248,7 +299,7 @@ erts_thr_progress_later(ErtsSchedulerData *esdp) if (esdp) { tpd = &esdp->thr_progress_data; managed_thread: - val = tpd->previous.local; + val = tpd->confirmed; ERTS_THR_MEMORY_BARRIER; } else { diff --git a/erts/emulator/beam/erl_threads.h b/erts/emulator/beam/erl_threads.h index 17628286bc..1dc3ffeb3c 100644 --- a/erts/emulator/beam/erl_threads.h +++ b/erts/emulator/beam/erl_threads.h @@ -533,6 +533,9 @@ ERTS_GLB_INLINE erts_aint_t erts_no_atomic_xchg(erts_no_atomic_t *xchgp, ERTS_GLB_INLINE erts_aint_t erts_no_atomic_cmpxchg(erts_no_atomic_t *xchgp, erts_aint_t new, erts_aint_t expected); +ERTS_GLB_INLINE erts_aint_t erts_no_atomic_read_bset(erts_no_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); ERTS_GLB_INLINE void erts_no_atomic32_set(erts_no_atomic32_t *var, erts_aint32_t i); ERTS_GLB_INLINE erts_aint32_t erts_no_atomic32_read(erts_no_atomic32_t *var); @@ -553,6 +556,9 @@ ERTS_GLB_INLINE erts_aint32_t erts_no_atomic32_xchg(erts_no_atomic32_t *xchgp, ERTS_GLB_INLINE erts_aint32_t erts_no_atomic32_cmpxchg(erts_no_atomic32_t *xchgp, erts_aint32_t new, erts_aint32_t expected); +ERTS_GLB_INLINE erts_aint32_t erts_no_atomic32_read_bset(erts_no_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); ERTS_GLB_INLINE void erts_spinlock_init_x_opt(erts_spinlock_t *lock, char *name, @@ -612,6 +618,78 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #ifdef USE_THREADS +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_nob(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_ddrb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_rb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_wb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_acqb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_relb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_mb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_nob(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_ddrb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_rb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_wb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_acqb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_relb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_mb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +#define ERTS_ATOMIC_BSET_IMPL__(Type, ReadOp, CmpxchgOp, VarP, Mask, Set) \ +do { \ + Type act = ReadOp((VarP)); \ + while (1) { \ + Type exp = act; \ + Type new = exp & ~(Mask); \ + new |= ((Mask) & (Set)); \ + act = CmpxchgOp((VarP), new, exp); \ + if (act == exp) \ + return act; \ + } \ +} while (0) +#endif + /* * See "Documentation of atomics and memory barriers" at the top * of this file for info on atomics. @@ -670,6 +748,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_nob ethr_atomic_xchg #define erts_atomic_cmpxchg_nob ethr_atomic_cmpxchg +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_nob(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg, + var, mask, set); +} +#endif + #define erts_atomic_init_mb ethr_atomic_init_mb #define erts_atomic_set_mb ethr_atomic_set_mb #define erts_atomic_read_mb ethr_atomic_read_mb @@ -684,6 +775,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_mb ethr_atomic_xchg_mb #define erts_atomic_cmpxchg_mb ethr_atomic_cmpxchg_mb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_mb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_mb, + var, mask, set); +} +#endif + #define erts_atomic_init_acqb ethr_atomic_init_acqb #define erts_atomic_set_acqb ethr_atomic_set_acqb #define erts_atomic_read_acqb ethr_atomic_read_acqb @@ -698,6 +802,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_acqb ethr_atomic_xchg_acqb #define erts_atomic_cmpxchg_acqb ethr_atomic_cmpxchg_acqb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_acqb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_acqb, + var, mask, set); +} +#endif + #define erts_atomic_init_relb ethr_atomic_init_relb #define erts_atomic_set_relb ethr_atomic_set_relb #define erts_atomic_read_relb ethr_atomic_read_relb @@ -712,6 +829,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_relb ethr_atomic_xchg_relb #define erts_atomic_cmpxchg_relb ethr_atomic_cmpxchg_relb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_relb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_relb, + var, mask, set); +} +#endif + #define erts_atomic_init_ddrb ethr_atomic_init_ddrb #define erts_atomic_set_ddrb ethr_atomic_set_ddrb #define erts_atomic_read_ddrb ethr_atomic_read_ddrb @@ -726,6 +856,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_ddrb ethr_atomic_xchg_ddrb #define erts_atomic_cmpxchg_ddrb ethr_atomic_cmpxchg_ddrb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_ddrb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_ddrb, + var, mask, set); +} +#endif + #define erts_atomic_init_rb ethr_atomic_init_rb #define erts_atomic_set_rb ethr_atomic_set_rb #define erts_atomic_read_rb ethr_atomic_read_rb @@ -740,6 +883,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_rb ethr_atomic_xchg_rb #define erts_atomic_cmpxchg_rb ethr_atomic_cmpxchg_rb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_rb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_rb, + var, mask, set); +} +#endif + #define erts_atomic_init_wb ethr_atomic_init_wb #define erts_atomic_set_wb ethr_atomic_set_wb #define erts_atomic_read_wb ethr_atomic_read_wb @@ -754,6 +910,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_wb ethr_atomic_xchg_wb #define erts_atomic_cmpxchg_wb ethr_atomic_cmpxchg_wb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_wb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_wb, + var, mask, set); +} +#endif + /* 32-bit atomics */ #define erts_atomic32_init_nob ethr_atomic32_init @@ -770,6 +939,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_nob ethr_atomic32_xchg #define erts_atomic32_cmpxchg_nob ethr_atomic32_cmpxchg +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_nob(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg, + var, mask, set); +} +#endif + #define erts_atomic32_init_mb ethr_atomic32_init_mb #define erts_atomic32_set_mb ethr_atomic32_set_mb #define erts_atomic32_read_mb ethr_atomic32_read_mb @@ -784,6 +966,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_mb ethr_atomic32_xchg_mb #define erts_atomic32_cmpxchg_mb ethr_atomic32_cmpxchg_mb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_mb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_mb, + var, mask, set); +} +#endif + #define erts_atomic32_init_acqb ethr_atomic32_init_acqb #define erts_atomic32_set_acqb ethr_atomic32_set_acqb #define erts_atomic32_read_acqb ethr_atomic32_read_acqb @@ -798,6 +993,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_acqb ethr_atomic32_xchg_acqb #define erts_atomic32_cmpxchg_acqb ethr_atomic32_cmpxchg_acqb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_acqb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_acqb, + var, mask, set); +} +#endif + #define erts_atomic32_init_relb ethr_atomic32_init_relb #define erts_atomic32_set_relb ethr_atomic32_set_relb #define erts_atomic32_read_relb ethr_atomic32_read_relb @@ -812,6 +1020,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_relb ethr_atomic32_xchg_relb #define erts_atomic32_cmpxchg_relb ethr_atomic32_cmpxchg_relb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_relb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_relb, + var, mask, set); +} +#endif + #define erts_atomic32_init_ddrb ethr_atomic32_init_ddrb #define erts_atomic32_set_ddrb ethr_atomic32_set_ddrb #define erts_atomic32_read_ddrb ethr_atomic32_read_ddrb @@ -826,6 +1047,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_ddrb ethr_atomic32_xchg_ddrb #define erts_atomic32_cmpxchg_ddrb ethr_atomic32_cmpxchg_ddrb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_ddrb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_ddrb, + var, mask, set); +} +#endif + #define erts_atomic32_init_rb ethr_atomic32_init_rb #define erts_atomic32_set_rb ethr_atomic32_set_rb #define erts_atomic32_read_rb ethr_atomic32_read_rb @@ -840,6 +1074,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_rb ethr_atomic32_xchg_rb #define erts_atomic32_cmpxchg_rb ethr_atomic32_cmpxchg_rb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_rb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_rb, + var, mask, set); +} +#endif + #define erts_atomic32_init_wb ethr_atomic32_init_wb #define erts_atomic32_set_wb ethr_atomic32_set_wb #define erts_atomic32_read_wb ethr_atomic32_read_wb @@ -854,6 +1101,21 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_wb ethr_atomic32_xchg_wb #define erts_atomic32_cmpxchg_wb ethr_atomic32_cmpxchg_wb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_wb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_wb, + var, mask, set); +} +#endif + +#undef ERTS_ATOMIC_BSET_IMPL__ + #else /* !USE_THREADS */ /* Double word size atomics */ @@ -908,6 +1170,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_nob erts_no_atomic_read_band #define erts_atomic_xchg_nob erts_no_atomic_xchg #define erts_atomic_cmpxchg_nob erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_nob erts_no_atomic_read_bset #define erts_atomic_init_mb erts_no_atomic_set #define erts_atomic_set_mb erts_no_atomic_set @@ -922,6 +1185,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_mb erts_no_atomic_read_band #define erts_atomic_xchg_mb erts_no_atomic_xchg #define erts_atomic_cmpxchg_mb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_mb erts_no_atomic_read_bset #define erts_atomic_init_acqb erts_no_atomic_set #define erts_atomic_set_acqb erts_no_atomic_set @@ -936,6 +1200,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_acqb erts_no_atomic_read_band #define erts_atomic_xchg_acqb erts_no_atomic_xchg #define erts_atomic_cmpxchg_acqb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_acqb erts_no_atomic_read_bset #define erts_atomic_init_relb erts_no_atomic_set #define erts_atomic_set_relb erts_no_atomic_set @@ -950,6 +1215,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_relb erts_no_atomic_read_band #define erts_atomic_xchg_relb erts_no_atomic_xchg #define erts_atomic_cmpxchg_relb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_relb erts_no_atomic_read_bset #define erts_atomic_init_ddrb erts_no_atomic_set #define erts_atomic_set_ddrb erts_no_atomic_set @@ -964,6 +1230,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_ddrb erts_no_atomic_read_band #define erts_atomic_xchg_ddrb erts_no_atomic_xchg #define erts_atomic_cmpxchg_ddrb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_ddrb erts_no_atomic_read_bset #define erts_atomic_init_rb erts_no_atomic_set #define erts_atomic_set_rb erts_no_atomic_set @@ -978,6 +1245,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_rb erts_no_atomic_read_band #define erts_atomic_xchg_rb erts_no_atomic_xchg #define erts_atomic_cmpxchg_rb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_rb erts_no_atomic_read_bset #define erts_atomic_init_wb erts_no_atomic_set #define erts_atomic_set_wb erts_no_atomic_set @@ -992,6 +1260,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_wb erts_no_atomic_read_band #define erts_atomic_xchg_wb erts_no_atomic_xchg #define erts_atomic_cmpxchg_wb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_wb erts_no_atomic_read_bset /* 32-bit atomics */ @@ -1008,6 +1277,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_nob erts_no_atomic32_read_band #define erts_atomic32_xchg_nob erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_nob erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_nob erts_no_atomic32_read_bset #define erts_atomic32_init_mb erts_no_atomic32_set #define erts_atomic32_set_mb erts_no_atomic32_set @@ -1022,6 +1292,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_mb erts_no_atomic32_read_band #define erts_atomic32_xchg_mb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_mb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_mb erts_no_atomic32_read_bset #define erts_atomic32_init_acqb erts_no_atomic32_set #define erts_atomic32_set_acqb erts_no_atomic32_set @@ -1036,6 +1307,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_acqb erts_no_atomic32_read_band #define erts_atomic32_xchg_acqb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_acqb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_acqb erts_no_atomic32_read_bset #define erts_atomic32_init_relb erts_no_atomic32_set #define erts_atomic32_set_relb erts_no_atomic32_set @@ -1050,6 +1322,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_relb erts_no_atomic32_read_band #define erts_atomic32_xchg_relb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_relb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_relb erts_no_atomic32_read_bset #define erts_atomic32_init_ddrb erts_no_atomic32_set #define erts_atomic32_set_ddrb erts_no_atomic32_set @@ -1064,6 +1337,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_ddrb erts_no_atomic32_read_band #define erts_atomic32_xchg_ddrb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_ddrb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_ddrb erts_no_atomic32_read_bset #define erts_atomic32_init_rb erts_no_atomic32_set #define erts_atomic32_set_rb erts_no_atomic32_set @@ -1078,6 +1352,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_rb erts_no_atomic32_read_band #define erts_atomic32_xchg_rb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_rb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_rb erts_no_atomic32_read_bset #define erts_atomic32_init_wb erts_no_atomic32_set #define erts_atomic32_set_wb erts_no_atomic32_set @@ -1092,6 +1367,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_wb erts_no_atomic32_read_band #define erts_atomic32_xchg_wb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_wb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_wb erts_no_atomic32_read_bset #endif /* !USE_THREADS */ @@ -1856,6 +2132,17 @@ erts_no_atomic_cmpxchg(erts_no_atomic_t *xchgp, return old; } +ERTS_GLB_INLINE erts_aint_t +erts_no_atomic_read_bset(erts_no_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + erts_aint_t old = *var; + *var &= ~mask; + *var |= (mask & set); + return old; +} + /* atomic32 */ ERTS_GLB_INLINE void @@ -1943,6 +2230,17 @@ erts_no_atomic32_cmpxchg(erts_no_atomic32_t *xchgp, return old; } +ERTS_GLB_INLINE erts_aint32_t +erts_no_atomic32_read_bset(erts_no_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + erts_aint32_t old = *var; + *var &= ~mask; + *var |= (mask & set); + return old; +} + /* spinlock */ ERTS_GLB_INLINE void diff --git a/erts/emulator/beam/erl_trace.c b/erts/emulator/beam/erl_trace.c index d04a91f18c..41f33efa79 100644 --- a/erts/emulator/beam/erl_trace.c +++ b/erts/emulator/beam/erl_trace.c @@ -156,8 +156,8 @@ do { (RES) = (TPID); } while(0) #define ERTS_TRACER_REF_TYPE Process * #define ERTS_GET_TRACER_REF(RES, TPID, TRACEE_FLGS) \ do { \ - (RES) = erts_proc_lookup((TPID)); \ - if (!(RES) || !((RES)->trace_flags & F_TRACER)) { \ + (RES) = erts_proc_lookup((TPID)); \ + if (!(RES) || !(ERTS_TRACE_FLAGS((RES)) & F_TRACER)) { \ (TPID) = NIL; \ (TRACEE_FLGS) &= ~TRACEE_FLAGS; \ return; \ @@ -409,7 +409,7 @@ WRITE_SYS_MSG_TO_PORT(Eterm unused_to, } #ifndef ERTS_SMP - if (!INVALID_TRACER_PORT(trace_port, trace_port->id)) + if (!INVALID_TRACER_PORT(trace_port, trace_port->common.id)) #endif erts_raw_port_command(trace_port, buffer, ptr-buffer); @@ -441,7 +441,7 @@ do_send_schedfix_to_port(Port *trace_port, Eterm pid, Eterm timestamp) { message = TUPLE5(hp, am_trace_ts, pid, am_out, mfarity, timestamp); /* Note, hp is deliberately NOT incremented since it will be reused */ - do_send_to_port(trace_port->id, + do_send_to_port(trace_port->common.id, trace_port, pid, SYS_MSG_TYPE_UNDEFINED, @@ -451,7 +451,7 @@ do_send_schedfix_to_port(Port *trace_port, Eterm pid, Eterm timestamp) { hp += 5; hp = patch_ts(message, hp); - do_send_to_port(trace_port->id, + do_send_to_port(trace_port->common.id, trace_port, pid, SYS_MSG_TYPE_UNDEFINED, @@ -508,7 +508,7 @@ send_to_port(Process *c_p, Eterm message, #endif do_send_to_port(*tracer_pid, trace_port, - c_p ? c_p->id : NIL, + c_p ? c_p->common.id : NIL, SYS_MSG_TYPE_TRACE, message); #ifndef ERTS_SMP @@ -543,7 +543,7 @@ send_to_port(Process *c_p, Eterm message, trace_port->control_flags &= ~PORT_CONTROL_FLAG_HEAVY; do_send_to_port(*tracer_pid, trace_port, - c_p ? c_p->id : NIL, + c_p ? c_p->common.id : NIL, SYS_MSG_TYPE_TRACE, message); @@ -557,7 +557,7 @@ send_to_port(Process *c_p, Eterm message, * just after writning the real trace message, and now gets scheduled * in again. */ - do_send_schedfix_to_port(trace_port, c_p->id, ts); + do_send_schedfix_to_port(trace_port, c_p->common.id, ts); } erts_port_release(trace_port); @@ -599,15 +599,14 @@ profile_send(Eterm from, Eterm message) { if (profiler_port) { do_send_to_port(profiler, profiler_port, - NIL, /* or current process->id */ + NIL, /* or current process->common.id */ SYS_MSG_TYPE_SYSPROF, message); erts_port_release(profiler_port); } } else { - ASSERT(is_internal_pid(profiler) - && internal_pid_index(profiler) < erts_max_processes); + ASSERT(is_internal_pid(profiler)); profile_p = erts_proc_lookup(profiler); @@ -673,7 +672,7 @@ seq_trace_send_to_port(Process *c_p, #endif do_send_to_port(seq_tracer, trace_port, - c_p ? c_p->id : NIL, + c_p ? c_p->common.id : NIL, SYS_MSG_TYPE_SEQTRACE, message); @@ -704,7 +703,7 @@ seq_trace_send_to_port(Process *c_p, trace_port->control_flags &= ~PORT_CONTROL_FLAG_HEAVY; do_send_to_port(seq_tracer, trace_port, - c_p ? c_p->id : NIL, + c_p ? c_p->common.id : NIL, SYS_MSG_TYPE_SEQTRACE, message); @@ -718,7 +717,7 @@ seq_trace_send_to_port(Process *c_p, * just after writing the real trace message, and now gets scheduled * in again. */ - do_send_schedfix_to_port(trace_port, c_p->id, ts); + do_send_schedfix_to_port(trace_port, c_p->common.id, ts); } erts_port_release(trace_port); @@ -729,7 +728,9 @@ seq_trace_send_to_port(Process *c_p, } #define TS_HEAP_WORDS 5 -#define TS_SIZE(p) (((p)->trace_flags & F_TIMESTAMP) ? TS_HEAP_WORDS : 0) +#define TS_SIZE(p) ((ERTS_TRACE_FLAGS((p)) & F_TIMESTAMP) \ + ? TS_HEAP_WORDS \ + : 0) /* * Patch a timestamp into a tuple. The tuple must be the last thing @@ -764,17 +765,17 @@ send_to_tracer(Process *tracee, erts_smp_mtx_lock(&smq_mtx); - if (tracee->trace_flags & F_TIMESTAMP) + if (ERTS_TRACE_FLAGS(tracee) & F_TIMESTAMP) *hpp = patch_ts(msg, *hpp); - if (is_internal_pid(tracee->tracer_proc)) - ERTS_ENQ_TRACE_MSG(tracee->id, tracer_ref, msg, bp); + if (is_internal_pid(ERTS_TRACER_PROC(tracee))) + ERTS_ENQ_TRACE_MSG(tracee->common.id, tracer_ref, msg, bp); else { - ASSERT(is_internal_port(tracee->tracer_proc)); + ASSERT(is_internal_port(ERTS_TRACER_PROC(tracee))); send_to_port(no_fake_sched ? NULL : tracee, msg, - &tracee->tracer_proc, - &tracee->trace_flags); + &ERTS_TRACER_PROC(tracee), + &ERTS_TRACE_FLAGS(tracee)); } erts_smp_mtx_unlock(&smq_mtx); @@ -792,7 +793,7 @@ trace_sched_aux(Process *p, Eterm what, int never_fake_sched) ERTS_TRACER_REF_TYPE tracer_ref = ERTS_NULL_TRACER_REF; int sched_no, curr_func, to_port, no_fake_sched; - if (is_nil(p->tracer_proc)) + if (is_nil(ERTS_TRACER_PROC(p))) return; no_fake_sched = never_fake_sched; @@ -812,13 +813,14 @@ trace_sched_aux(Process *p, Eterm what, int never_fake_sched) } sched_no = IS_TRACED_FL(p, F_TRACE_SCHED_NO); - to_port = is_internal_port(p->tracer_proc); + to_port = is_internal_port(ERTS_TRACER_PROC(p)); if (!to_port) { - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); } if (ERTS_PROC_IS_EXITING(p)) @@ -851,7 +853,7 @@ trace_sched_aux(Process *p, Eterm what, int never_fake_sched) } if (!sched_no) { - mess = TUPLE4(hp, am_trace, p->id, what, tmp); + mess = TUPLE4(hp, am_trace, p->common.id, what, tmp); hp += 5; } else { @@ -860,7 +862,7 @@ trace_sched_aux(Process *p, Eterm what, int never_fake_sched) #else Eterm sched_id = make_small(1); #endif - mess = TUPLE5(hp, am_trace, p->id, what, sched_id, tmp); + mess = TUPLE5(hp, am_trace, p->common.id, what, sched_id, tmp); hp += 6; } @@ -912,19 +914,19 @@ trace_send(Process *p, Eterm to, Eterm msg) operation = am_atom_put(s, sys_strlen(s)); } - if (is_internal_port(p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(p))) { #define LOCAL_HEAP_SIZE (11) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); hp = local_heap; - mess = TUPLE5(hp, am_trace, p->id, operation, msg, to); + mess = TUPLE5(hp, am_trace, p->common.id, operation, msg, to); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - send_to_port(p, mess, &p->tracer_proc, &p->trace_flags); + send_to_port(p, mess, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -934,10 +936,11 @@ trace_send(Process *p, Eterm to, Eterm msg) ErlOffHeap *off_heap; ERTS_TRACER_REF_TYPE tracer_ref; - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); sz_msg = size_object(msg); sz_to = size_object(to); @@ -953,16 +956,16 @@ trace_send(Process *p, Eterm to, Eterm msg) sz_msg, &hp, off_heap); - mess = TUPLE5(hp, am_trace, p->id/* Local pid */, operation, msg, to); + mess = TUPLE5(hp, am_trace, p->common.id, operation, msg, to); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -977,19 +980,19 @@ trace_receive(Process *rp, Eterm msg) size_t sz_msg; Eterm* hp; - if (is_internal_port(rp->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(rp))) { #define LOCAL_HEAP_SIZE (10) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); hp = local_heap; - mess = TUPLE4(hp, am_trace, rp->id, am_receive, msg); + mess = TUPLE4(hp, am_trace, rp->common.id, am_receive, msg); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (rp->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(rp) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - send_to_port(rp, mess, &rp->tracer_proc, &rp->trace_flags); + send_to_port(rp, mess, &ERTS_TRACER_PROC(rp), &ERTS_TRACE_FLAGS(rp)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -999,10 +1002,11 @@ trace_receive(Process *rp, Eterm msg) ErlOffHeap *off_heap; ERTS_TRACER_REF_TYPE tracer_ref; - ASSERT(is_internal_pid(rp->tracer_proc) - && internal_pid_index(rp->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(rp))); - ERTS_GET_TRACER_REF(tracer_ref, rp->tracer_proc, rp->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(rp), + ERTS_TRACE_FLAGS(rp)); sz_msg = size_object(msg); @@ -1011,16 +1015,16 @@ trace_receive(Process *rp, Eterm msg) hp = ERTS_ALLOC_SYSMSG_HEAP(hsz, &bp, &off_heap, tracer_ref); msg = copy_struct(msg, sz_msg, &hp, off_heap); - mess = TUPLE4(hp, am_trace, rp->id/* Local pid */, am_receive, msg); + mess = TUPLE4(hp, am_trace, rp->common.id, am_receive, msg); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (rp->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(rp) & F_TIMESTAMP) { patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(rp->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(rp->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -1030,14 +1034,14 @@ seq_trace_update_send(Process *p) { Eterm seq_tracer = erts_get_system_seq_tracer(); ASSERT((is_tuple(SEQ_TRACE_TOKEN(p)) || is_nil(SEQ_TRACE_TOKEN(p)))); - if ( (p->id == seq_tracer) || (SEQ_TRACE_TOKEN(p) == NIL) + if ( (p->common.id == seq_tracer) || (SEQ_TRACE_TOKEN(p) == NIL) #ifdef USE_VM_PROBES || (SEQ_TRACE_TOKEN(p) == am_have_dt_utag) #endif ) { return 0; } - SEQ_TRACE_TOKEN_SENDER(p) = p->id; /* Internal pid */ + SEQ_TRACE_TOKEN_SENDER(p) = p->common.id; SEQ_TRACE_TOKEN_SERIAL(p) = make_small(++(p -> seq_trace_clock)); SEQ_TRACE_TOKEN_LASTCNT(p) = @@ -1074,7 +1078,7 @@ seq_trace_output_generic(Eterm token, Eterm msg, Uint type, ASSERT(is_tuple(token) || is_nil(token)); if (SEQ_TRACE_T_SENDER(token) == seq_tracer || token == NIL || - (process && process->trace_flags & F_SENSITIVE)) { + (process && ERTS_TRACE_FLAGS(process) & F_SENSITIVE)) { return; } @@ -1138,8 +1142,7 @@ seq_trace_output_generic(Eterm token, Eterm msg, Uint type, Uint sz_label, sz_lastcnt_serial, sz_msg, sz_ts, sz_sender, sz_exitfrom, sz_receiver; - ASSERT(is_internal_pid(seq_tracer) - && internal_pid_index(seq_tracer) < erts_max_processes); + ASSERT(is_internal_pid(seq_tracer)); #ifndef ERTS_SMP @@ -1253,17 +1256,17 @@ erts_trace_return_to(Process *p, BeamInstr *pc) hp += 4; } - mess = TUPLE4(hp, am_trace, p->id, am_return_to, mfa); + mess = TUPLE4(hp, am_trace, p->common.id, am_return_to, mfa); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - if (is_internal_port(p->tracer_proc)) { - send_to_port(p, mess, &p->tracer_proc, &p->trace_flags); + if (is_internal_port(ERTS_TRACER_PROC(p))) { + send_to_port(p, mess, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); } else { ErlHeapFragment *bp; ErlOffHeap *off_heap; @@ -1273,10 +1276,11 @@ erts_trace_return_to(Process *p, BeamInstr *pc) /* * Find the tracer. */ - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); size = size_object(mess); @@ -1286,7 +1290,7 @@ erts_trace_return_to(Process *p, BeamInstr *pc) * Copy the trace message into the buffer and enqueue it. */ mess = copy_struct(mess, size, &hp, off_heap); - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, mess, bp); } UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE @@ -1315,25 +1319,25 @@ erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid) /* Breakpoint trace enabled without specifying tracer => * use process tracer and flags */ - tracer_pid = &p->tracer_proc; + tracer_pid = &ERTS_TRACER_PROC(p); } if (is_nil(*tracer_pid)) { /* Trace disabled */ return; } ASSERT(is_internal_pid(*tracer_pid) || is_internal_port(*tracer_pid)); - if (*tracer_pid == p->id) { + if (*tracer_pid == p->common.id) { /* Do not generate trace messages to oneself */ return; } - if (tracer_pid == &p->tracer_proc) { + if (tracer_pid == &ERTS_TRACER_PROC(p)) { /* Tracer specified in process structure => * non-breakpoint trace => * use process flags */ - tracee_flags = &p->trace_flags; + tracee_flags = &ERTS_TRACE_FLAGS(p); #ifdef ERTS_SMP - tracee = p->id; + tracee = p->common.id; #endif } else { /* Tracer not specified in process structure => @@ -1362,7 +1366,7 @@ erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid) hp = local_heap; mfa = TUPLE3(hp, mod, name, make_small(arity)); hp += 4; - mess = TUPLE5(hp, am_trace, p->id, am_return_from, mfa, retval); + mess = TUPLE5(hp, am_trace, p->common.id, am_return_from, mfa, retval); hp += 6; erts_smp_mtx_lock(&smq_mtx); if (*tracee_flags & F_TIMESTAMP) { @@ -1382,8 +1386,7 @@ erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid) Eterm* limit; #endif - ASSERT(is_internal_pid(*tracer_pid) - && internal_pid_index(*tracer_pid) < erts_max_processes); + ASSERT(is_internal_pid(*tracer_pid)); ERTS_GET_TRACER_REF(tracer_ref, *tracer_pid, *tracee_flags); @@ -1405,7 +1408,7 @@ erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid) mfa = TUPLE3(hp, mod, name, make_small(arity)); hp += 4; retval = copy_struct(retval, retval_size, &hp, off_heap); - mess = TUPLE5(hp, am_trace, p->id/* Local pid */, am_return_from, mfa, retval); + mess = TUPLE5(hp, am_trace, p->common.id, am_return_from, mfa, retval); hp += 6; erts_smp_mtx_lock(&smq_mtx); @@ -1446,25 +1449,25 @@ erts_trace_exception(Process* p, BeamInstr mfa[3], Eterm class, Eterm value, /* Breakpoint trace enabled without specifying tracer => * use process tracer and flags */ - tracer_pid = &p->tracer_proc; + tracer_pid = &ERTS_TRACER_PROC(p); } if (is_nil(*tracer_pid)) { /* Trace disabled */ return; } ASSERT(is_internal_pid(*tracer_pid) || is_internal_port(*tracer_pid)); - if (*tracer_pid == p->id) { + if (*tracer_pid == p->common.id) { /* Do not generate trace messages to oneself */ return; } - if (tracer_pid == &p->tracer_proc) { + if (tracer_pid == &ERTS_TRACER_PROC(p)) { /* Tracer specified in process structure => * non-breakpoint trace => * use process flags */ - tracee_flags = &p->trace_flags; + tracee_flags = &ERTS_TRACE_FLAGS(p); #ifdef ERTS_SMP - tracee = p->id; + tracee = p->common.id; #endif if (! (*tracee_flags & F_TRACE_CALLS)) { return; @@ -1492,7 +1495,7 @@ erts_trace_exception(Process* p, BeamInstr mfa[3], Eterm class, Eterm value, hp += 4; cv = TUPLE2(hp, class, value); hp += 3; - mess = TUPLE5(hp, am_trace, p->id, am_exception_from, mfa_tuple, cv); + mess = TUPLE5(hp, am_trace, p->common.id, am_exception_from, mfa_tuple, cv); hp += 6; ASSERT((hp - local_heap) <= LOCAL_HEAP_SIZE); erts_smp_mtx_lock(&smq_mtx); @@ -1514,8 +1517,7 @@ erts_trace_exception(Process* p, BeamInstr mfa[3], Eterm class, Eterm value, Eterm* limit; #endif - ASSERT(is_internal_pid(*tracer_pid) - && internal_pid_index(*tracer_pid) < erts_max_processes); + ASSERT(is_internal_pid(*tracer_pid)); ERTS_GET_TRACER_REF(tracer_ref, *tracer_pid, *tracee_flags); @@ -1539,7 +1541,7 @@ erts_trace_exception(Process* p, BeamInstr mfa[3], Eterm class, Eterm value, value = copy_struct(value, value_size, &hp, off_heap); cv = TUPLE2(hp, class, value); hp += 3; - mess = TUPLE5(hp, am_trace, p->id/* Local pid */, + mess = TUPLE5(hp, am_trace, p->common.id, am_exception_from, mfa_tuple, cv); hp += 6; @@ -1593,25 +1595,25 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, /* Breakpoint trace enabled without specifying tracer => * use process tracer and flags */ - tracer_pid = &p->tracer_proc; + tracer_pid = &ERTS_TRACER_PROC(p); } if (is_nil(*tracer_pid)) { /* Trace disabled */ return 0; } ASSERT(is_internal_pid(*tracer_pid) || is_internal_port(*tracer_pid)); - if (*tracer_pid == p->id) { + if (*tracer_pid == p->common.id) { /* Do not generate trace messages to oneself */ return 0; } - if (tracer_pid == &p->tracer_proc) { + if (tracer_pid == &ERTS_TRACER_PROC(p)) { /* Tracer specified in process structure => * non-breakpoint trace => * use process flags */ - tracee_flags = &p->trace_flags; + tracee_flags = &ERTS_TRACE_FLAGS(p); #ifdef ERTS_SMP - tracee = p->id; + tracee = p->common.id; #endif } else { /* Tracer not specified in process structure => @@ -1619,7 +1621,7 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, * meta trace => * use fixed flag set instead of process flags */ - if (p->trace_flags & F_SENSITIVE) { + if (ERTS_TRACE_FLAGS(p) & F_SENSITIVE) { /* No trace messages for sensitive processes. */ return 0; } @@ -1677,7 +1679,7 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, if (!erts_is_valid_tracer_port(*tracer_pid)) { #ifdef ERTS_SMP - ASSERT(is_nil(tracee) || tracer_pid == &p->tracer_proc); + ASSERT(is_nil(tracee) || tracer_pid == &ERTS_TRACER_PROC(p)); if (is_not_nil(tracee)) erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); #endif @@ -1779,7 +1781,7 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, * Build the trace tuple and send it to the port. */ - mess = TUPLE4(hp, am_trace, p->id, am_call, mfa_tuple); + mess = TUPLE4(hp, am_trace, p->common.id, am_call, mfa_tuple); hp += 5; if (pam_result != am_true) { hp[-5] = make_arityval(5); @@ -1814,21 +1816,21 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, Eterm* limit; #endif - ASSERT(is_internal_pid(*tracer_pid) - && internal_pid_index(*tracer_pid) < erts_max_processes); + ASSERT(is_internal_pid(*tracer_pid)); tracer = erts_pid2proc(p, ERTS_PROC_LOCK_MAIN, *tracer_pid, ERTS_PROC_LOCK_STATUS); if (!tracer) invalid_tracer = 1; else { - invalid_tracer = (tracer->trace_flags & F_TRACER) == 0; + invalid_tracer = !(ERTS_TRACE_FLAGS(tracer) & F_TRACER); erts_smp_proc_unlock(tracer, ERTS_PROC_LOCK_STATUS); } if (invalid_tracer) { #ifdef ERTS_SMP - ASSERT(is_nil(tracee) || tracer_pid == &p->tracer_proc); + ASSERT(is_nil(tracee) + || tracer_pid == &ERTS_TRACER_PROC(p)); if (is_not_nil(tracee)) erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); #endif @@ -1952,7 +1954,7 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, * Build the trace tuple and enqueue it. */ - mess = TUPLE4(hp, am_trace, p->id/* Local pid */, am_call, mfa_tuple); + mess = TUPLE4(hp, am_trace, p->common.id, am_call, mfa_tuple); hp += 5; if (pam_result != am_true) { hp[-5] = make_arityval(5); @@ -1990,17 +1992,17 @@ trace_proc(Process *c_p, Process *t_p, Eterm what, Eterm data) ERTS_SMP_LC_ASSERT((erts_proc_lc_my_proc_locks(t_p) != 0) || erts_thr_progress_is_blocking()); - if (is_internal_port(t_p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(t_p))) { #define LOCAL_HEAP_SIZE (5+5) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); hp = local_heap; - mess = TUPLE4(hp, am_trace, t_p->id, what, data); + mess = TUPLE4(hp, am_trace, t_p->common.id, what, data); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (t_p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(t_p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } send_to_port( @@ -2011,7 +2013,9 @@ trace_proc(Process *c_p, Process *t_p, Eterm what, Eterm data) /* Fake schedule out and in are never sent when smp enabled */ c_p, #endif - mess, &t_p->tracer_proc, &t_p->trace_flags); + mess, + &ERTS_TRACER_PROC(t_p), + &ERTS_TRACE_FLAGS(t_p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -2022,10 +2026,11 @@ trace_proc(Process *c_p, Process *t_p, Eterm what, Eterm data) ERTS_TRACER_REF_TYPE tracer_ref; size_t sz_data; - ASSERT(is_internal_pid(t_p->tracer_proc) - && internal_pid_index(t_p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(t_p))); - ERTS_GET_TRACER_REF(tracer_ref, t_p->tracer_proc, t_p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(t_p), + ERTS_TRACE_FLAGS(t_p)); sz_data = size_object(data); @@ -2034,16 +2039,16 @@ trace_proc(Process *c_p, Process *t_p, Eterm what, Eterm data) hp = ERTS_ALLOC_SYSMSG_HEAP(need, &bp, &off_heap, tracer_ref); tmp = copy_struct(data, sz_data, &hp, off_heap); - mess = TUPLE4(hp, am_trace, t_p->id/* Local pid */, what, tmp); + mess = TUPLE4(hp, am_trace, t_p->common.id, what, tmp); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (t_p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(t_p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(t_p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(t_p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -2064,7 +2069,7 @@ trace_proc_spawn(Process *p, Eterm pid, Eterm mess; Eterm* hp; - if (is_internal_port(p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(p))) { #define LOCAL_HEAP_SIZE (4+6+5) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); @@ -2072,13 +2077,13 @@ trace_proc_spawn(Process *p, Eterm pid, hp = local_heap; mfa = TUPLE3(hp, mod, func, args); hp += 4; - mess = TUPLE5(hp, am_trace, p->id, am_spawn, pid, mfa); + mess = TUPLE5(hp, am_trace, p->common.id, am_spawn, pid, mfa); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - send_to_port(p, mess, &p->tracer_proc, &p->trace_flags); + send_to_port(p, mess, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -2090,10 +2095,11 @@ trace_proc_spawn(Process *p, Eterm pid, size_t sz_args, sz_pid; Uint need; - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); sz_args = size_object(args); sz_pid = size_object(pid); @@ -2105,16 +2111,16 @@ trace_proc_spawn(Process *p, Eterm pid, mfa = TUPLE3(hp, mod, func, tmp); hp += 4; tmp = copy_struct(pid, sz_pid, &hp, off_heap); - mess = TUPLE5(hp, am_trace, p->id, am_spawn, tmp, mfa); + mess = TUPLE5(hp, am_trace, p->common.id, am_spawn, tmp, mfa); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -2204,7 +2210,7 @@ trace_gc(Process *p, Eterm what) UseTmpHeap(LOCAL_HEAP_SIZE,p); - if (is_internal_port(p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(p))) { hp = local_heap; #ifdef DEBUG size = 0; @@ -2216,10 +2222,11 @@ trace_gc(Process *p, Eterm what) size += 5/*4-tuple*/ + TS_SIZE(p); #endif } else { - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); size = 0; (void) erts_bld_atom_uint_2tup_list(NULL, @@ -2243,19 +2250,19 @@ trace_gc(Process *p, Eterm what) tags, values); - msg = TUPLE4(hp, am_trace, p->id/* Local pid */, what, msg); + msg = TUPLE4(hp, am_trace, p->common.id, what, msg); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(msg, hp); } ASSERT(hp == limit); - if (is_internal_port(p->tracer_proc)) - send_to_port(p, msg, &p->tracer_proc, &p->trace_flags); + if (is_internal_port(ERTS_TRACER_PROC(p))) + send_to_port(p, msg, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); else - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, msg, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, msg, bp); erts_smp_mtx_unlock(&smq_mtx); UnUseTmpHeap(LOCAL_HEAP_SIZE,p); #undef LOCAL_HEAP_SIZE @@ -2295,8 +2302,7 @@ monitor_long_gc(Process *p, Uint time) { #endif #ifndef ERTS_SMP - ASSERT(is_internal_pid(system_monitor) - && internal_pid_index(system_monitor) < erts_max_processes); + ASSERT(is_internal_pid(system_monitor)); monitor_p = erts_proc_lookup(system_monitor); if (!monitor_p || p == monitor_p) return; @@ -2321,7 +2327,7 @@ monitor_long_gc(Process *p, Uint time) { sizeof(values)/sizeof(Uint), tags, values); - msg = TUPLE4(hp, am_monitor, p->id/* Local pid */, am_long_gc, list); + msg = TUPLE4(hp, am_monitor, p->common.id, am_long_gc, list); #ifdef DEBUG hp += 5 /* 4-tuple */; @@ -2329,7 +2335,7 @@ monitor_long_gc(Process *p, Uint time) { #endif #ifdef ERTS_SMP - enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->id, NIL, msg, bp); + enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->common.id, NIL, msg, bp); #else erts_queue_message(monitor_p, NULL, bp, msg, NIL #ifdef USE_VM_PROBES @@ -2370,8 +2376,7 @@ monitor_large_heap(Process *p) { #ifndef ERTS_SMP - ASSERT(is_internal_pid(system_monitor) - && internal_pid_index(system_monitor) < erts_max_processes); + ASSERT(is_internal_pid(system_monitor)); monitor_p = erts_proc_lookup(system_monitor); if (monitor_p || p == monitor_p) { return; @@ -2397,7 +2402,7 @@ monitor_large_heap(Process *p) { sizeof(values)/sizeof(Uint), tags, values); - msg = TUPLE4(hp, am_monitor, p->id/* Local pid */, am_large_heap, list); + msg = TUPLE4(hp, am_monitor, p->common.id, am_large_heap, list); #ifdef DEBUG hp += 5 /* 4-tuple */; @@ -2405,7 +2410,7 @@ monitor_large_heap(Process *p) { #endif #ifdef ERTS_SMP - enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->id, NIL, msg, bp); + enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->common.id, NIL, msg, bp); #else erts_queue_message(monitor_p, NULL, bp, msg, NIL #ifdef USE_VM_PROBES @@ -2425,8 +2430,7 @@ monitor_generic(Process *p, Eterm type, Eterm spec) { Eterm *hp, msg; #ifndef ERTS_SMP - ASSERT(is_internal_pid(system_monitor) - && internal_pid_index(system_monitor) < erts_max_processes); + ASSERT(is_internal_pid(system_monitor)); monitor_p = erts_proc_lookup(system_monitor); if (!monitor_p || p == monitor_p) return; @@ -2434,11 +2438,11 @@ monitor_generic(Process *p, Eterm type, Eterm spec) { hp = ERTS_ALLOC_SYSMSG_HEAP(5, &bp, &off_heap, monitor_p); - msg = TUPLE4(hp, am_monitor, p->id/* Local pid */, type, spec); + msg = TUPLE4(hp, am_monitor, p->common.id, type, spec); hp += 5; #ifdef ERTS_SMP - enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->id, NIL, msg, bp); + enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->common.id, NIL, msg, bp); #else erts_queue_message(monitor_p, NULL, bp, msg, NIL #ifdef USE_VM_PROBES @@ -2560,21 +2564,21 @@ trace_port_open(Port *p, Eterm calling_pid, Eterm drv_name) { Eterm mess; Eterm* hp; - if (is_internal_port(p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(p))) { #define LOCAL_HEAP_SIZE (5+6) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); hp = local_heap; - mess = TUPLE5(hp, am_trace, calling_pid, am_open, p->id, drv_name); + mess = TUPLE5(hp, am_trace, calling_pid, am_open, p->common.id, drv_name); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } /* No fake schedule */ - send_to_port(NULL, mess, &p->tracer_proc, &p->trace_flags); + send_to_port(NULL, mess, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -2584,25 +2588,26 @@ trace_port_open(Port *p, Eterm calling_pid, Eterm drv_name) { size_t sz_data; ERTS_TRACER_REF_TYPE tracer_ref; - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); sz_data = 6 + TS_SIZE(p); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); hp = ERTS_ALLOC_SYSMSG_HEAP(sz_data, &bp, &off_heap, tracer_ref); - mess = TUPLE5(hp, am_trace, calling_pid, am_open, p->id, drv_name); + mess = TUPLE5(hp, am_trace, calling_pid, am_open, p->common.id, drv_name); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } @@ -2623,20 +2628,20 @@ trace_port(Port *t_p, Eterm what, Eterm data) { ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(t_p) || erts_thr_progress_is_blocking()); - if (is_internal_port(t_p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(t_p))) { #define LOCAL_HEAP_SIZE (5+5) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); hp = local_heap; - mess = TUPLE4(hp, am_trace, t_p->id, what, data); + mess = TUPLE4(hp, am_trace, t_p->common.id, what, data); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (t_p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(t_p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } /* No fake schedule */ - send_to_port(NULL, mess, &t_p->tracer_proc, &t_p->trace_flags); + send_to_port(NULL,mess,&ERTS_TRACER_PROC(t_p),&ERTS_TRACE_FLAGS(t_p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -2646,25 +2651,26 @@ trace_port(Port *t_p, Eterm what, Eterm data) { size_t sz_data; ERTS_TRACER_REF_TYPE tracer_ref; - ASSERT(is_internal_pid(t_p->tracer_proc) - && internal_pid_index(t_p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(t_p))); sz_data = 5 + TS_SIZE(t_p); - ERTS_GET_TRACER_REF(tracer_ref, t_p->tracer_proc, t_p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(t_p), + ERTS_TRACE_FLAGS(t_p)); hp = ERTS_ALLOC_SYSMSG_HEAP(sz_data, &bp, &off_heap, tracer_ref); - mess = TUPLE4(hp, am_trace, t_p->id, what, data); + mess = TUPLE4(hp, am_trace, t_p->common.id, what, data); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (t_p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(t_p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(t_p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(t_p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -2689,7 +2695,7 @@ trace_sched_ports_where(Port *p, Eterm what, Eterm where) { int ws = 5; Eterm sched_id = am_undefined; - if (is_internal_port(p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(p))) { #define LOCAL_HEAP_SIZE (5+6) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); @@ -2704,21 +2710,21 @@ trace_sched_ports_where(Port *p, Eterm what, Eterm where) { #else sched_id = make_small(1); #endif - mess = TUPLE5(hp, am_trace, p->id, what, sched_id, where); + mess = TUPLE5(hp, am_trace, p->common.id, what, sched_id, where); ws = 6; } else { - mess = TUPLE4(hp, am_trace, p->id, what, where); + mess = TUPLE4(hp, am_trace, p->common.id, what, where); ws = 5; } hp += ws; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } /* No fake scheduling */ - send_to_port(NULL, mess, &p->tracer_proc, &p->trace_flags); + send_to_port(NULL, mess, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -2727,12 +2733,13 @@ trace_sched_ports_where(Port *p, Eterm what, Eterm where) { ErlOffHeap *off_heap; ERTS_TRACER_REF_TYPE tracer_ref; - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); if (IS_TRACED_FL(p, F_TRACE_SCHED_NO)) ws = 6; /* Make place for scheduler id */ - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); hp = ERTS_ALLOC_SYSMSG_HEAP(ws+TS_SIZE(p), &bp, &off_heap, tracer_ref); @@ -2744,19 +2751,19 @@ trace_sched_ports_where(Port *p, Eterm what, Eterm where) { #else sched_id = make_small(1); #endif - mess = TUPLE5(hp, am_trace, p->id, what, sched_id, where); + mess = TUPLE5(hp, am_trace, p->common.id, what, sched_id, where); } else { - mess = TUPLE4(hp, am_trace, p->id, what, where); + mess = TUPLE4(hp, am_trace, p->common.id, what, where); } hp += ws; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -2792,14 +2799,14 @@ profile_runnable_port(Port *p, Eterm status) { GET_NOW(&Ms, &s, &us); timestamp = TUPLE3(hp, make_small(Ms), make_small(s), make_small(us)); hp += 4; - msg = TUPLE5(hp, am_profile, p->id, status, count, timestamp); hp += 6; + msg = TUPLE5(hp, am_profile, p->common.id, status, count, timestamp); hp += 6; #ifndef ERTS_SMP - profile_send(p->id, msg); + profile_send(p->common.id, msg); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE #else - enqueue_sys_msg_unlocked(SYS_MSG_TYPE_SYSPROF, p->id, NIL, msg, bp); + enqueue_sys_msg_unlocked(SYS_MSG_TYPE_SYSPROF, p->common.id, NIL, msg, bp); #endif erts_smp_mtx_unlock(&smq_mtx); } @@ -2846,13 +2853,13 @@ profile_runnable_proc(Process *p, Eterm status){ GET_NOW(&Ms, &s, &us); timestamp = TUPLE3(hp, make_small(Ms), make_small(s), make_small(us)); hp += 4; - msg = TUPLE5(hp, am_profile, p->id, status, where, timestamp); hp += 6; + msg = TUPLE5(hp, am_profile, p->common.id, status, where, timestamp); hp += 6; #ifndef ERTS_SMP - profile_send(p->id, msg); + profile_send(p->common.id, msg); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE #else - enqueue_sys_msg_unlocked(SYS_MSG_TYPE_SYSPROF, p->id, NIL, msg, bp); + enqueue_sys_msg_unlocked(SYS_MSG_TYPE_SYSPROF, p->common.id, NIL, msg, bp); #endif erts_smp_mtx_unlock(&smq_mtx); } @@ -2865,16 +2872,19 @@ profile_runnable_proc(Process *p, Eterm status){ void erts_check_my_tracer_proc(Process *p) { - if (is_internal_pid(p->tracer_proc)) { - Process *tracer = erts_pid2proc(p, ERTS_PROC_LOCK_MAIN, - p->tracer_proc, ERTS_PROC_LOCK_STATUS); - int invalid_tracer = !tracer || !(tracer->trace_flags & F_TRACER); + if (is_internal_pid(ERTS_TRACER_PROC(p))) { + Process *tracer = erts_pid2proc(p, + ERTS_PROC_LOCK_MAIN, + ERTS_TRACER_PROC(p), + ERTS_PROC_LOCK_STATUS); + int invalid_tracer = (!tracer + || !(ERTS_TRACE_FLAGS(tracer) & F_TRACER)); if (tracer) erts_smp_proc_unlock(tracer, ERTS_PROC_LOCK_STATUS); if (invalid_tracer) { erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); - p->trace_flags &= ~TRACEE_FLAGS; - p->tracer_proc = NIL; + ERTS_TRACE_FLAGS(p) &= ~TRACEE_FLAGS; + ERTS_TRACER_PROC(p) = NIL; erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL_MINOR); } } @@ -3218,7 +3228,7 @@ sys_msg_dispatcher_func(void *unused) proc = erts_pid2proc(NULL, 0, receiver, proc_locks); if (!proc || (smqp->type == SYS_MSG_TYPE_TRACE - && !(proc->trace_flags & F_TRACER))) { + && !(ERTS_TRACE_FLAGS(proc) & F_TRACER))) { /* Bad tracer */ #ifdef DEBUG_PRINTOUTS if (smqp->type == SYS_MSG_TYPE_TRACE && proc) @@ -3245,16 +3255,14 @@ sys_msg_dispatcher_func(void *unused) proc = erts_whereis_process(NULL,0,receiver,proc_locks,0); if (!proc) goto failure; - else if (smqp->from == proc->id) + else if (smqp->from == proc->common.id) goto drop_sys_msg; else goto queue_proc_msg; } else if (is_internal_port(receiver)) { - port = erts_id2port_sflgs(receiver, - NULL, - 0, - ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); + port = erts_thr_id2port_sflgs(receiver, + ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); if (!port) goto failure; else { @@ -3268,7 +3276,7 @@ sys_msg_dispatcher_func(void *unused) #ifdef DEBUG_PRINTOUTS erts_fprintf(stderr, "delivered\n"); #endif - erts_port_release(port); + erts_thr_port_release(port); if (smqp->bp) free_message_buffer(smqp->bp); } diff --git a/erts/emulator/beam/erl_trace.h b/erts/emulator/beam/erl_trace.h new file mode 100644 index 0000000000..50fb27aab0 --- /dev/null +++ b/erts/emulator/beam/erl_trace.h @@ -0,0 +1,141 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + + +#ifndef ERL_TRACE_H__ +#define ERL_TRACE_H__ + +struct binary; + +/* erl_bif_trace.c */ +Eterm erl_seq_trace_info(Process *p, Eterm arg1); +void erts_system_monitor_clear(Process *c_p); +void erts_system_profile_clear(Process *c_p); + +/* erl_trace.c */ +void erts_init_trace(void); +void erts_trace_check_exiting(Eterm exiting); +Eterm erts_set_system_seq_tracer(Process *c_p, + ErtsProcLocks c_p_locks, + Eterm new); +Eterm erts_get_system_seq_tracer(void); +void erts_change_default_tracing(int setflags, Uint *flagsp, Eterm *tracerp); +void erts_get_default_tracing(Uint *flagsp, Eterm *tracerp); +void erts_set_system_monitor(Eterm monitor); +Eterm erts_get_system_monitor(void); + +#ifdef ERTS_SMP +void erts_check_my_tracer_proc(Process *); +void erts_block_sys_msg_dispatcher(void); +void erts_release_sys_msg_dispatcher(void); +void erts_foreach_sys_msg_in_q(void (*func)(Eterm, + Eterm, + Eterm, + ErlHeapFragment *)); +void erts_queue_error_logger_message(Eterm, Eterm, ErlHeapFragment *); +#endif + +void erts_send_sys_msg_proc(Eterm, Eterm, Eterm, ErlHeapFragment *); +void trace_send(Process*, Eterm, Eterm); +void trace_receive(Process*, Eterm); +Uint32 erts_call_trace(Process *p, BeamInstr mfa[], struct binary *match_spec, Eterm* args, + int local, Eterm *tracer_pid); +void erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid); +void erts_trace_exception(Process* p, BeamInstr mfa[], Eterm class, Eterm value, + Eterm *tracer); +void erts_trace_return_to(Process *p, BeamInstr *pc); +void trace_sched(Process*, Eterm); +void trace_proc(Process*, Process*, Eterm, Eterm); +void trace_proc_spawn(Process*, Eterm pid, Eterm mod, Eterm func, Eterm args); +void save_calls(Process *p, Export *); +void trace_gc(Process *p, Eterm what); +/* port tracing */ +void trace_virtual_sched(Process*, Eterm); +void trace_sched_ports(Port *pp, Eterm); +void trace_sched_ports_where(Port *pp, Eterm, Eterm); +void trace_port(Port *, Eterm what, Eterm data); +void trace_port_open(Port *, Eterm calling_pid, Eterm drv_name); + +/* system_profile */ +void erts_set_system_profile(Eterm profile); +Eterm erts_get_system_profile(void); +void profile_scheduler(Eterm scheduler_id, Eterm); +void profile_scheduler_q(Eterm scheduler_id, Eterm state, Eterm no_schedulers, Uint Ms, Uint s, Uint us); +void profile_runnable_proc(Process* p, Eterm status); +void profile_runnable_port(Port* p, Eterm status); +void erts_system_profile_setup_active_schedulers(void); + +/* system_monitor */ +void monitor_long_gc(Process *p, Uint time); +void monitor_large_heap(Process *p); +void monitor_generic(Process *p, Eterm type, Eterm spec); +Uint erts_trace_flag2bit(Eterm flag); +int erts_trace_flags(Eterm List, + Uint *pMask, Eterm *pTracer, int *pCpuTimestamp); +Eterm erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr *I); + +#ifdef ERTS_SMP +void erts_send_pending_trace_msgs(ErtsSchedulerData *esdp); +#define ERTS_SMP_CHK_PEND_TRACE_MSGS(ESDP) \ +do { \ + if ((ESDP)->pending_trace_msgs) \ + erts_send_pending_trace_msgs((ESDP)); \ +} while (0) +#else +#define ERTS_SMP_CHK_PEND_TRACE_MSGS(ESDP) +#endif + +#define seq_trace_output(token, msg, type, receiver, process) \ +seq_trace_output_generic((token), (msg), (type), (receiver), (process), NIL) +#define seq_trace_output_exit(token, msg, type, receiver, exitfrom) \ +seq_trace_output_generic((token), (msg), (type), (receiver), NULL, (exitfrom)) +void seq_trace_output_generic(Eterm token, Eterm msg, Uint type, + Eterm receiver, Process *process, Eterm exitfrom); + +int seq_trace_update_send(Process *process); + +Eterm erts_seq_trace(Process *process, + Eterm atom_type, Eterm atom_true_or_false, + int build_result); + +struct trace_pattern_flags { + unsigned int breakpoint : 1; /* Set if any other is set */ + unsigned int local : 1; /* Local call trace breakpoint */ + unsigned int meta : 1; /* Metadata trace breakpoint */ + unsigned int call_count : 1; /* Fast call count breakpoint */ + unsigned int call_time : 1; /* Fast call time breakpoint */ +}; +extern const struct trace_pattern_flags erts_trace_pattern_flags_off; +extern int erts_call_time_breakpoint_tracing; +int erts_set_trace_pattern(Process*p, Eterm* mfa, int specified, + struct binary* match_prog_set, + struct binary *meta_match_prog_set, + int on, struct trace_pattern_flags, + Eterm meta_tracer_pid, int is_blocking); +void +erts_get_default_trace_pattern(int *trace_pattern_is_on, + struct binary **match_spec, + struct binary **meta_match_spec, + struct trace_pattern_flags *trace_pattern_flags, + Eterm *meta_tracer_pid); +int erts_is_default_trace_enabled(void); +void erts_bif_trace_init(void); +int erts_finish_breakpointing(void); + +#endif /* ERL_TRACE_H__ */ diff --git a/erts/emulator/beam/erl_utils.h b/erts/emulator/beam/erl_utils.h new file mode 100644 index 0000000000..a2064bd8a3 --- /dev/null +++ b/erts/emulator/beam/erl_utils.h @@ -0,0 +1,215 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +#ifndef ERL_UTILS_H__ +#define ERL_UTILS_H__ + +#include "sys.h" +#include "erl_smp.h" +#include "erl_printf.h" + +typedef struct { +#ifdef DEBUG + int smp_api; +#endif + union { + Uint64 not_atomic; +#ifdef ARCH_64 + erts_atomic_t atomic; +#else + erts_dw_atomic_t atomic; +#endif + } counter; +} erts_interval_t; + +void erts_interval_init(erts_interval_t *); +void erts_smp_interval_init(erts_interval_t *); +Uint64 erts_step_interval_nob(erts_interval_t *); +Uint64 erts_step_interval_relb(erts_interval_t *); +Uint64 erts_smp_step_interval_nob(erts_interval_t *); +Uint64 erts_smp_step_interval_relb(erts_interval_t *); +Uint64 erts_ensure_later_interval_nob(erts_interval_t *, Uint64); +Uint64 erts_ensure_later_interval_acqb(erts_interval_t *, Uint64); +Uint64 erts_smp_ensure_later_interval_nob(erts_interval_t *, Uint64); +Uint64 erts_smp_ensure_later_interval_acqb(erts_interval_t *, Uint64); +#ifdef ARCH_32 +ERTS_GLB_INLINE Uint64 erts_interval_dw_aint_to_val__(erts_dw_aint_t *); +#endif +ERTS_GLB_INLINE Uint64 erts_current_interval_nob__(erts_interval_t *); +ERTS_GLB_INLINE Uint64 erts_current_interval_acqb__(erts_interval_t *); +ERTS_GLB_INLINE Uint64 erts_current_interval_nob(erts_interval_t *); +ERTS_GLB_INLINE Uint64 erts_current_interval_acqb(erts_interval_t *); +ERTS_GLB_INLINE Uint64 erts_smp_current_interval_nob(erts_interval_t *); +ERTS_GLB_INLINE Uint64 erts_smp_current_interval_acqb(erts_interval_t *); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +#ifdef ARCH_32 + +ERTS_GLB_INLINE Uint64 +erts_interval_dw_aint_to_val__(erts_dw_aint_t *dw) +{ +#ifdef ETHR_SU_DW_NAINT_T__ + return (Uint64) dw->dw_sint; +#else + Uint64 res; + res = (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_HIGH_WORD]); + res <<= 32; + res |= (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_LOW_WORD]); + return res; +#endif +} + +#endif + +ERTS_GLB_INLINE Uint64 +erts_current_interval_nob__(erts_interval_t *icp) +{ +#ifdef ARCH_64 + return (Uint64) erts_atomic_read_nob(&icp->counter.atomic); +#else + erts_dw_aint_t dw; + erts_dw_atomic_read_nob(&icp->counter.atomic, &dw); + return erts_interval_dw_aint_to_val__(&dw); +#endif +} + +ERTS_GLB_INLINE Uint64 +erts_current_interval_acqb__(erts_interval_t *icp) +{ +#ifdef ARCH_64 + return (Uint64) erts_atomic_read_acqb(&icp->counter.atomic); +#else + erts_dw_aint_t dw; + erts_dw_atomic_read_acqb(&icp->counter.atomic, &dw); + return erts_interval_dw_aint_to_val__(&dw); +#endif +} + +ERTS_GLB_INLINE Uint64 +erts_current_interval_nob(erts_interval_t *icp) +{ + ASSERT(!icp->smp_api); + return erts_current_interval_nob__(icp); +} + +ERTS_GLB_INLINE Uint64 +erts_current_interval_acqb(erts_interval_t *icp) +{ + ASSERT(!icp->smp_api); + return erts_current_interval_acqb__(icp); +} + +ERTS_GLB_INLINE Uint64 +erts_smp_current_interval_nob(erts_interval_t *icp) +{ + ASSERT(icp->smp_api); +#ifdef ERTS_SMP + return erts_current_interval_nob__(icp); +#else + return icp->counter.not_atomic; +#endif +} + +ERTS_GLB_INLINE Uint64 +erts_smp_current_interval_acqb(erts_interval_t *icp) +{ + ASSERT(icp->smp_api); +#ifdef ERTS_SMP + return erts_current_interval_acqb__(icp); +#else + return icp->counter.not_atomic; +#endif +} + +#endif /* ERTS_GLB_INLINE_INCL_FUNC_DEF */ + +/* + * To be used to silence unused result warnings, but do not abuse it. + */ +void erts_silence_warn_unused_result(long unused); + + +int erts_fit_in_bits_int64(Sint64); +int erts_fit_in_bits_int32(Sint32); +int list_length(Eterm); +int erts_is_builtin(Eterm, Eterm, int); +Uint32 make_broken_hash(Eterm); +Uint32 block_hash(byte *, unsigned, Uint32); +Uint32 make_hash2(Eterm); +Uint32 make_hash(Eterm); + + +Eterm erts_bld_atom(Uint **hpp, Uint *szp, char *str); +Eterm erts_bld_uint(Uint **hpp, Uint *szp, Uint ui); +Eterm erts_bld_uword(Uint **hpp, Uint *szp, UWord uw); +Eterm erts_bld_uint64(Uint **hpp, Uint *szp, Uint64 ui64); +Eterm erts_bld_sint64(Uint **hpp, Uint *szp, Sint64 si64); +Eterm erts_bld_cons(Uint **hpp, Uint *szp, Eterm car, Eterm cdr); +Eterm erts_bld_tuple(Uint **hpp, Uint *szp, Uint arity, ...); +Eterm erts_bld_tuplev(Uint **hpp, Uint *szp, Uint arity, Eterm terms[]); +Eterm erts_bld_string_n(Uint **hpp, Uint *szp, const char *str, Sint len); +#define erts_bld_string(hpp,szp,str) erts_bld_string_n(hpp,szp,str,strlen(str)) +Eterm erts_bld_list(Uint **hpp, Uint *szp, Sint length, Eterm terms[]); +Eterm erts_bld_2tup_list(Uint **hpp, Uint *szp, + Sint length, Eterm terms1[], Uint terms2[]); +Eterm +erts_bld_atom_uint_2tup_list(Uint **hpp, Uint *szp, + Sint length, Eterm atoms[], Uint uints[]); +Eterm +erts_bld_atom_2uint_3tup_list(Uint **hpp, Uint *szp, Sint length, + Eterm atoms[], Uint uints1[], Uint uints2[]); + +void erts_init_utils(void); +void erts_init_utils_mem(void); + +erts_dsprintf_buf_t *erts_create_tmp_dsbuf(Uint); +void erts_destroy_tmp_dsbuf(erts_dsprintf_buf_t *); + +#if HALFWORD_HEAP +int eq_rel(Eterm a, Eterm* a_base, Eterm b, Eterm* b_base); +# define eq(A,B) eq_rel(A,NULL,B,NULL) +#else +int eq(Eterm, Eterm); +# define eq_rel(A,A_BASE,B,B_BASE) eq(A,B) +#endif + +#define EQ(x,y) (((x) == (y)) || (is_not_both_immed((x),(y)) && eq((x),(y)))) + +#if HALFWORD_HEAP +Sint cmp_rel(Eterm, Eterm*, Eterm, Eterm*); +#define CMP(A,B) cmp_rel(A,NULL,B,NULL) +#else +Sint cmp(Eterm, Eterm); +#define cmp_rel(A,A_BASE,B,B_BASE) cmp(A,B) +#define CMP(A,B) cmp(A,B) +#endif +#define cmp_lt(a,b) (CMP((a),(b)) < 0) +#define cmp_le(a,b) (CMP((a),(b)) <= 0) +#define cmp_eq(a,b) (CMP((a),(b)) == 0) +#define cmp_ne(a,b) (CMP((a),(b)) != 0) +#define cmp_ge(a,b) (CMP((a),(b)) >= 0) +#define cmp_gt(a,b) (CMP((a),(b)) > 0) + +#define CMP_LT(a,b) ((a) != (b) && cmp_lt((a),(b))) +#define CMP_GE(a,b) ((a) == (b) || cmp_ge((a),(b))) +#define CMP_EQ(a,b) ((a) == (b) || cmp_eq((a),(b))) +#define CMP_NE(a,b) ((a) != (b) && cmp_ne((a),(b))) + +#endif diff --git a/erts/emulator/beam/global.h b/erts/emulator/beam/global.h index 4c0d3421c8..298241618f 100755 --- a/erts/emulator/beam/global.h +++ b/erts/emulator/beam/global.h @@ -39,38 +39,8 @@ #include "erl_sys_driver.h" #include "erl_debug.h" #include "error.h" - -typedef struct port Port; -#include "erl_port_task.h" - -typedef struct erts_driver_t_ erts_driver_t; - -#define SMALL_IO_QUEUE 5 /* Number of fixed elements */ - -typedef struct { - ErlDrvSizeT size; /* total size in bytes */ - - SysIOVec* v_start; - SysIOVec* v_end; - SysIOVec* v_head; - SysIOVec* v_tail; - SysIOVec v_small[SMALL_IO_QUEUE]; - - ErlDrvBinary** b_start; - ErlDrvBinary** b_end; - ErlDrvBinary** b_head; - ErlDrvBinary** b_tail; - ErlDrvBinary* b_small[SMALL_IO_QUEUE]; -} ErlIOQueue; - -typedef struct line_buf { /* Buffer used in line oriented I/O */ - ErlDrvSizeT bufsiz; /* Size of character buffer */ - ErlDrvSizeT ovlen; /* Length of overflow data */ - ErlDrvSizeT ovsiz; /* Actual size of overflow buffer */ - char data[1]; /* Starting point of buffer data, - data[0] is a flag indicating an unprocess CR, - The rest is the overflow buffer. */ -} LineBuf; +#include "erl_utils.h" +#include "erl_port.h" struct enif_environment_t /* ErlNifEnv */ { @@ -90,162 +60,6 @@ extern void erts_print_nif_taints(int to, void* to_arg); void erts_unload_nif(struct erl_module_nif* nif); extern void erl_nif_init(void); -/* - * Port Specific Data. - * - * Only use PrtSD for very rarely used data. - */ - -#define ERTS_PRTSD_SCHED_ID 0 - -#define ERTS_PRTSD_SIZE 1 - -typedef struct { - void *data[ERTS_PRTSD_SIZE]; -} ErtsPrtSD; - -#ifdef ERTS_SMP -typedef struct ErtsXPortsList_ ErtsXPortsList; -#endif - -/* - * Port locking: - * - * Locking is done either driver specific or port specific. When - * driver specific locking is used, all instances of the driver, - * i.e. ports running the driver, share the same lock. When port - * specific locking is used each instance have its own lock. - * - * Most fields in the Port structure are protected by the lock - * referred to by the lock field. I'v called it the port lock. - * This lock is shared between all ports running the same driver - * when driver specific locking is used. - * - * The 'sched' field is protected by the port tasks lock - * (see erl_port_tasks.c) - * - * The 'status' field is protected by a combination of the port lock, - * the port tasks lock, and the state_lck. It may be read if - * the state_lck, or the port lock is held. It may only be - * modified if both the port lock and the state_lck is held - * (with one exception; see below). When changeing status from alive - * to dead or vice versa, also the port task lock has to be held. - * This in order to guarantee that tasks are scheduled only for - * ports that are alive. - * - * The status field may be modified with only the state_lck - * held when status is changed from dead to alive. This since no - * threads can have any references to the port other than via the - * port table. - * - * /rickard - */ - -struct port { - ErtsPortTaskSched sched; - ErtsPortTaskHandle timeout_task; - erts_smp_atomic_t refc; -#ifdef ERTS_SMP - erts_smp_mtx_t *lock; - ErtsXPortsList *xports; - erts_smp_atomic_t run_queue; - erts_smp_spinlock_t state_lck; /* protects: id, status, snapshot */ -#endif - Eterm id; /* The Port id of this port */ - Eterm connected; /* A connected process */ - Eterm caller; /* Current caller. */ - Eterm data; /* Data associated with port. */ - ErlHeapFragment* bp; /* Heap fragment holding data (NULL if imm data). */ - ErtsLink *nlinks; - ErtsMonitor *monitors; /* Only MON_ORIGIN monitors of pid's */ - Uint bytes_in; /* Number of bytes read */ - Uint bytes_out; /* Number of bytes written */ -#ifdef ERTS_SMP - ErtsSmpPTimer *ptimer; -#else - ErlTimer tm; /* Timer entry */ -#endif - - Eterm tracer_proc; /* If the port is traced, this is the tracer */ - Uint trace_flags; /* Trace flags */ - - ErlIOQueue ioq; /* driver accessible i/o queue */ - DistEntry *dist_entry; /* Dist entry used in DISTRIBUTION */ - char *name; /* String used in the open */ - erts_driver_t* drv_ptr; - UWord drv_data; - SWord os_pid; /* Child process ID */ - ErtsProcList *suspended; /* List of suspended processes. */ - LineBuf *linebuf; /* Buffer to hold data not ready for - process to get (line oriented I/O)*/ - Uint32 status; /* Status and type flags */ - int control_flags; /* Flags for port_control() */ - erts_aint32_t snapshot; /* Next snapshot that port should be part of */ - struct reg_proc *reg; - ErlDrvPDL port_data_lock; - - ErtsPrtSD *psd; /* Port specific data */ -}; - - -ERTS_GLB_INLINE ErtsRunQueue *erts_port_runq(Port *prt); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE ErtsRunQueue * -erts_port_runq(Port *prt) -{ -#ifdef ERTS_SMP - ErtsRunQueue *rq1, *rq2; - rq1 = (ErtsRunQueue *) erts_smp_atomic_read_nob(&prt->run_queue); - if (!rq1) - return NULL; - while (1) { - erts_smp_runq_lock(rq1); - rq2 = (ErtsRunQueue *) erts_smp_atomic_read_nob(&prt->run_queue); - if (rq1 == rq2) - return rq1; - erts_smp_runq_unlock(rq1); - rq1 = rq2; - if (!rq1) - return NULL; - } -#else - return ERTS_RUNQ_IX(0); -#endif -} - -#endif - - -ERTS_GLB_INLINE void *erts_prtsd_get(Port *p, int ix); -ERTS_GLB_INLINE void *erts_prtsd_set(Port *p, int ix, void *new); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE void * -erts_prtsd_get(Port *prt, int ix) -{ - return prt->psd ? prt->psd->data[ix] : NULL; -} - -ERTS_GLB_INLINE void * -erts_prtsd_set(Port *prt, int ix, void *data) -{ - if (prt->psd) { - void *old = prt->psd->data[ix]; - prt->psd->data[ix] = data; - return old; - } - else { - prt->psd = erts_alloc(ERTS_ALC_T_PRTSD, sizeof(ErtsPrtSD)); - prt->psd->data[ix] = data; - return NULL; - } -} - -#endif - /* Driver handle (wrapper for old plain handle) */ #define ERL_DE_OK 0 #define ERL_DE_UNLOAD 1 @@ -297,7 +111,7 @@ typedef struct { or that wait for it to change state */ erts_refc_t refc; /* Number of ports/processes having references to the driver */ - Uint port_count; /* Number of ports using the driver */ + erts_smp_atomic32_t port_count; /* Number of ports using the driver */ Uint flags; /* ERL_DE_FL_KILL_PORTS */ int status; /* ERL_DE_xxx */ char *full_path; /* Full path of the driver */ @@ -349,7 +163,7 @@ struct erts_driver_t_ { }; extern erts_driver_t *driver_list; -extern erts_smp_mtx_t erts_driver_list_lock; +extern erts_smp_rwmtx_t erts_driver_list_lock; extern void erts_ddll_init(void); extern void erts_ddll_lock_driver(DE_Handle *dh, char *name); @@ -529,40 +343,9 @@ union erl_off_heap_ptr { void* voidp; }; -/* arrays that get malloced at startup */ -extern Port* erts_port; - -extern Uint erts_max_ports; -extern Uint erts_port_tab_index_mask; -extern erts_smp_atomic32_t erts_ports_snapshot; -extern erts_smp_atomic_t erts_dead_ports_ptr; - -ERTS_GLB_INLINE void erts_may_save_closed_port(Port *prt); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE void erts_may_save_closed_port(Port *prt) -{ - ERTS_SMP_LC_ASSERT(erts_smp_lc_spinlock_is_locked(&prt->state_lck)); - if (prt->snapshot != erts_smp_atomic32_read_acqb(&erts_ports_snapshot)) { - /* Dead ports are added from the end of the snapshot buffer */ - Eterm* tombstone; - tombstone = (Eterm*) erts_smp_atomic_add_read_nob(&erts_dead_ports_ptr, - -(erts_aint_t)sizeof(Eterm)); - ASSERT(tombstone+1 != NULL); - ASSERT(prt->snapshot == erts_smp_atomic32_read_nob(&erts_ports_snapshot) - 1); - *tombstone = prt->id; - } - /*else no ongoing snapshot or port was already included or created after snapshot */ -} - -#endif - /* controls warning mapping in error_logger */ extern Eterm node_cookie; -extern erts_smp_atomic_t erts_bytes_out; /* no bytes written out */ -extern erts_smp_atomic_t erts_bytes_in; /* no bytes sent into the system */ extern Uint display_items; /* no of items to display in traces etc */ extern int erts_backtrace_depth; @@ -700,54 +483,6 @@ do { \ #define WSTACK_ISEMPTY(s) (WSTK_CONCAT(s,_sp) == WSTK_CONCAT(s,_start)) #define WSTACK_POP(s) (*(--WSTK_CONCAT(s,_sp))) - -/* port status flags */ - -#define ERTS_PORT_SFLG_CONNECTED ((Uint32) (1 << 0)) -/* Port have begun exiting */ -#define ERTS_PORT_SFLG_EXITING ((Uint32) (1 << 1)) -/* Distribution port */ -#define ERTS_PORT_SFLG_DISTRIBUTION ((Uint32) (1 << 2)) -#define ERTS_PORT_SFLG_BINARY_IO ((Uint32) (1 << 3)) -#define ERTS_PORT_SFLG_SOFT_EOF ((Uint32) (1 << 4)) -/* Flow control */ -#define ERTS_PORT_SFLG_PORT_BUSY ((Uint32) (1 << 5)) -/* Port is closing (no i/o accepted) */ -#define ERTS_PORT_SFLG_CLOSING ((Uint32) (1 << 6)) -/* Send a closed message when terminating */ -#define ERTS_PORT_SFLG_SEND_CLOSED ((Uint32) (1 << 7)) -/* Line orinted io on port */ -#define ERTS_PORT_SFLG_LINEBUF_IO ((Uint32) (1 << 8)) -/* Immortal port (only certain system ports) */ -#define ERTS_PORT_SFLG_IMMORTAL ((Uint32) (1 << 9)) -#define ERTS_PORT_SFLG_FREE ((Uint32) (1 << 10)) -#define ERTS_PORT_SFLG_FREE_SCHEDULED ((Uint32) (1 << 11)) -#define ERTS_PORT_SFLG_INITIALIZING ((Uint32) (1 << 12)) -/* Port uses port specific locking (opposed to driver specific locking) */ -#define ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK ((Uint32) (1 << 13)) -#define ERTS_PORT_SFLG_INVALID ((Uint32) (1 << 14)) -/* Last port to terminate halts the emulator */ -#define ERTS_PORT_SFLG_HALT ((Uint32) (1 << 15)) -#ifdef DEBUG -/* Only debug: make sure all flags aren't cleared unintentionally */ -#define ERTS_PORT_SFLG_PORT_DEBUG ((Uint32) (1 << 31)) -#endif - -/* Combinations of port status flags */ -#define ERTS_PORT_SFLGS_DEAD \ - (ERTS_PORT_SFLG_FREE \ - | ERTS_PORT_SFLG_FREE_SCHEDULED \ - | ERTS_PORT_SFLG_INITIALIZING) -#define ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP \ - (ERTS_PORT_SFLGS_DEAD | ERTS_PORT_SFLG_INVALID) -#define ERTS_PORT_SFLGS_INVALID_LOOKUP \ - (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP \ - | ERTS_PORT_SFLG_CLOSING) -#define ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP \ - (ERTS_PORT_SFLGS_INVALID_LOOKUP \ - | ERTS_PORT_SFLG_PORT_BUSY \ - | ERTS_PORT_SFLG_DISTRIBUTION) - /* binary.c */ void erts_emasculate_writable_binary(ProcBin* pb); @@ -758,11 +493,35 @@ Eterm erts_realloc_binary(Eterm bin, size_t size); /* erl_bif_info.c */ +Eterm +erts_bld_port_info(Eterm **hpp, + ErlOffHeap *ohp, + Uint *szp, + Port *prt, + Eterm item); + void erts_bif_info_init(void); /* bif.c */ Eterm erts_make_ref(Process *); Eterm erts_make_ref_in_buffer(Eterm buffer[REF_THING_SIZE]); +void erts_make_ref_in_array(Uint32 ref[ERTS_MAX_REF_NUMBERS]); + +ERTS_GLB_INLINE Eterm +erts_proc_store_ref(Process *c_p, Uint32 ref[ERTS_MAX_REF_NUMBERS]); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE Eterm +erts_proc_store_ref(Process *c_p, Uint32 ref[ERTS_MAX_REF_NUMBERS]) +{ + Eterm *hp = HAlloc(c_p, REF_THING_SIZE); + write_ref_thing(hp, ref[0], ref[1], ref[2]); + return make_internal_ref(hp); +} + +#endif + void erts_queue_monitor_message(Process *, ErtsProcLocks*, Eterm, @@ -778,13 +537,6 @@ Eterm erl_send(Process *p, Eterm to, Eterm msg); Eterm erl_is_function(Process* p, Eterm arg1, Eterm arg2); -/* erl_bif_port.c */ - -/* erl_bif_trace.c */ -Eterm erl_seq_trace_info(Process *p, Eterm arg1); -void erts_system_monitor_clear(Process *c_p); -void erts_system_profile_clear(Process *c_p); - /* beam_load.c */ typedef struct { BeamInstr* current; /* Pointer to: Mod, Name, Arity */ @@ -960,11 +712,6 @@ void erts_free_heap_frags(Process* p); /* io.c */ -struct erl_drv_port_data_lock { - erts_mtx_t mtx; - erts_atomic_t refc; -}; - typedef struct { char *name; char *driver_name; @@ -973,477 +720,33 @@ typedef struct { #define ERTS_SPAWN_DRIVER 1 #define ERTS_SPAWN_EXECUTABLE 2 #define ERTS_SPAWN_ANY (ERTS_SPAWN_DRIVER | ERTS_SPAWN_EXECUTABLE) - int erts_add_driver_entry(ErlDrvEntry *drv, DE_Handle *handle, int driver_list_locked); void erts_destroy_driver(erts_driver_t *drv); -void erts_wake_process_later(Port*, Process*); -int erts_open_driver(erts_driver_t*, Eterm, char*, SysDriverOpts*, int *); -int erts_is_port_ioq_empty(Port *); -void erts_terminate_port(Port *); -void close_port(Eterm); -void init_io(void); -void cleanup_io(void); -void erts_do_exit_port(Port *, Eterm, Eterm); -void erts_port_command(Process *, Eterm, Port *, Eterm); -Eterm erts_port_control(Process*, Port*, Uint, Eterm); -int erts_write_to_port(Eterm caller_id, Port *p, Eterm list); -void print_port_info(int, void *, int); +int erts_save_suspend_process_on_port(Port*, Process*); +Port *erts_open_driver(erts_driver_t*, Eterm, char*, SysDriverOpts*, int *, int *); +void erts_init_io(int, int); void erts_raw_port_command(Port*, byte*, Uint); -void driver_report_exit(int, int); +void driver_report_exit(ErlDrvPort, int); LineBuf* allocate_linebuf(int); int async_ready(Port *, void*); -Sint erts_test_next_port(int, Uint); ErtsPortNames *erts_get_port_names(Eterm); void erts_free_port_names(ErtsPortNames *); Uint erts_port_ioq_size(Port *pp); void erts_stale_drv_select(Eterm, ErlDrvEvent, int, int); -void erts_port_cleanup(Port *); -void erts_fire_port_monitor(Port *prt, Eterm ref); Port *erts_get_heart_port(void); -#ifdef ERTS_SMP -void erts_smp_xports_unlock(Port *); -#endif - #if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_COUNT) void erts_lcnt_enable_io_lock_count(int enable); #endif -#if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) -int erts_lc_is_port_locked(Port *); -#endif - -ERTS_GLB_INLINE void erts_smp_port_state_lock(Port*); -ERTS_GLB_INLINE void erts_smp_port_state_unlock(Port*); - -ERTS_GLB_INLINE int erts_smp_port_trylock(Port *prt); -ERTS_GLB_INLINE void erts_smp_port_lock(Port *prt); -ERTS_GLB_INLINE void erts_smp_port_unlock(Port *prt); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE void -erts_smp_port_state_lock(Port* prt) -{ -#ifdef ERTS_SMP - erts_smp_spin_lock(&prt->state_lck); -#endif -} - -ERTS_GLB_INLINE void -erts_smp_port_state_unlock(Port *prt) -{ -#ifdef ERTS_SMP - erts_smp_spin_unlock(&prt->state_lck); -#endif -} - - -ERTS_GLB_INLINE int -erts_smp_port_trylock(Port *prt) -{ - int res; - - ASSERT(erts_smp_atomic_read_nob(&prt->refc) > 0); - erts_smp_atomic_inc_nob(&prt->refc); - -#ifdef ERTS_SMP - res = erts_smp_mtx_trylock(prt->lock); - if (res == EBUSY) { - erts_smp_atomic_dec_nob(&prt->refc); - } -#else - res = 0; -#endif - - return res; -} - -ERTS_GLB_INLINE void -erts_smp_port_lock(Port *prt) -{ - ASSERT(erts_smp_atomic_read_nob(&prt->refc) > 0); - erts_smp_atomic_inc_nob(&prt->refc); -#ifdef ERTS_SMP - erts_smp_mtx_lock(prt->lock); -#endif -} - -ERTS_GLB_INLINE void -erts_smp_port_unlock(Port *prt) -{ - erts_aint_t refc; -#ifdef ERTS_SMP - erts_smp_mtx_unlock(prt->lock); -#endif - refc = erts_smp_atomic_dec_read_nob(&prt->refc); - ASSERT(refc >= 0); - if (refc == 0) - erts_port_cleanup(prt); -} - -#endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ - - -#define ERTS_INVALID_PORT_OPT(PP, ID, FLGS) \ - (!(PP) || ((PP)->status & (FLGS)) || (PP)->id != (ID)) - -/* port lookup */ - -#define INVALID_PORT(PP, ID) \ - ERTS_INVALID_PORT_OPT((PP), (ID), ERTS_PORT_SFLGS_INVALID_LOOKUP) - -/* Invalidate trace port if anything suspicious, for instance - * that the port is a distribution port or it is busy. - */ -#define INVALID_TRACER_PORT(PP, ID) \ - ERTS_INVALID_PORT_OPT((PP), (ID), ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP) - -#define ERTS_PORT_SCHED_ID(P, ID) \ - ((Uint) (UWord) erts_prtsd_set((P), ERTS_PSD_SCHED_ID, (void *) (UWord) (ID))) - -#ifdef ERTS_SMP -Port *erts_de2port(DistEntry *, Process *, ErtsProcLocks); -#endif - -#define erts_id2port(ID, P, PL) \ - erts_id2port_sflgs((ID), (P), (PL), ERTS_PORT_SFLGS_INVALID_LOOKUP) - -ERTS_GLB_INLINE Port*erts_id2port_sflgs(Eterm, Process *, ErtsProcLocks, Uint32); -ERTS_GLB_INLINE void erts_port_release(Port *); -ERTS_GLB_INLINE Port*erts_drvport2port(ErlDrvPort); -ERTS_GLB_INLINE Port*erts_drvportid2port(Eterm); -ERTS_GLB_INLINE Uint32 erts_portid2status(Eterm id); -ERTS_GLB_INLINE int erts_is_port_alive(Eterm id); -ERTS_GLB_INLINE int erts_is_valid_tracer_port(Eterm id); -ERTS_GLB_INLINE void erts_port_status_bandor_set(Port *, Uint32, Uint32); -ERTS_GLB_INLINE void erts_port_status_band_set(Port *, Uint32); -ERTS_GLB_INLINE void erts_port_status_bor_set(Port *, Uint32); -ERTS_GLB_INLINE void erts_port_status_set(Port *, Uint32); -ERTS_GLB_INLINE Uint32 erts_port_status_get(Port *); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE Port* -erts_id2port_sflgs(Eterm id, Process *c_p, ErtsProcLocks c_p_locks, Uint32 sflgs) -{ -#ifdef ERTS_SMP - int no_proc_locks = !c_p || !c_p_locks; -#endif - Port *prt; - - if (is_not_internal_port(id)) - return NULL; - - prt = &erts_port[internal_port_index(id)]; - - erts_smp_port_state_lock(prt); - if (ERTS_INVALID_PORT_OPT(prt, id, sflgs)) { - erts_smp_port_state_unlock(prt); - prt = NULL; - } - else { - erts_smp_atomic_inc_nob(&prt->refc); - erts_smp_port_state_unlock(prt); - -#ifdef ERTS_SMP - if (no_proc_locks) - erts_smp_mtx_lock(prt->lock); - else if (erts_smp_mtx_trylock(prt->lock) == EBUSY) { - /* Unlock process locks, and acquire locks in lock order... */ - erts_smp_proc_unlock(c_p, c_p_locks); - erts_smp_mtx_lock(prt->lock); - erts_smp_proc_lock(c_p, c_p_locks); - } - - /* The id may not have changed... */ - ERTS_SMP_LC_ASSERT(prt->id == id); - /* ... but status may have... */ - if (prt->status & sflgs) { - erts_smp_port_unlock(prt); /* Also decrements refc... */ - prt = NULL; - } -#endif - - } - - return prt; -} - -ERTS_GLB_INLINE void -erts_port_release(Port *prt) -{ - erts_smp_port_unlock(prt); -} - -ERTS_GLB_INLINE Port* -erts_drvport2port(ErlDrvPort drvport) -{ - int ix = (int) drvport; - if (ix < 0 || erts_max_ports <= ix) - return NULL; - if (erts_port[ix].status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) - return NULL; - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(&erts_port[ix])); - return &erts_port[ix]; -} - -ERTS_GLB_INLINE Port* -erts_drvportid2port(Eterm id) -{ - int ix; - if (is_not_internal_port(id)) - return NULL; - ix = (int) internal_port_index(id); - if (erts_max_ports <= ix) - return NULL; - if (erts_port[ix].status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) - return NULL; - if (erts_port[ix].id != id) - return NULL; - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(&erts_port[ix])); - return &erts_port[ix]; -} - -ERTS_GLB_INLINE Uint32 -erts_portid2status(Eterm id) -{ - if (is_not_internal_port(id)) - return ERTS_PORT_SFLG_INVALID; - else { - Uint32 status; - int ix = internal_port_index(id); - if (erts_max_ports <= ix) - return ERTS_PORT_SFLG_INVALID; - erts_smp_port_state_lock(&erts_port[ix]); - if (erts_port[ix].id == id) - status = erts_port[ix].status; - else - status = ERTS_PORT_SFLG_INVALID; - erts_smp_port_state_unlock(&erts_port[ix]); - return status; - } -} - -ERTS_GLB_INLINE int -erts_is_port_alive(Eterm id) -{ - return !(erts_portid2status(id) & (ERTS_PORT_SFLG_INVALID - | ERTS_PORT_SFLGS_DEAD)); -} - -ERTS_GLB_INLINE int -erts_is_valid_tracer_port(Eterm id) -{ - return !(erts_portid2status(id) & ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); -} - -ERTS_GLB_INLINE void erts_port_status_bandor_set(Port *prt, - Uint32 band_status, - Uint32 bor_status) -{ - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - erts_smp_port_state_lock(prt); - prt->status &= band_status; - prt->status |= bor_status; - erts_smp_port_state_unlock(prt); -} - -ERTS_GLB_INLINE void erts_port_status_band_set(Port *prt, Uint32 status) -{ - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - erts_smp_port_state_lock(prt); - prt->status &= status; - erts_smp_port_state_unlock(prt); -} - -ERTS_GLB_INLINE void erts_port_status_bor_set(Port *prt, Uint32 status) -{ - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - erts_smp_port_state_lock(prt); - prt->status |= status; - erts_smp_port_state_unlock(prt); -} - -ERTS_GLB_INLINE void erts_port_status_set(Port *prt, Uint32 status) -{ - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - erts_smp_port_state_lock(prt); - prt->status = status; - erts_smp_port_state_unlock(prt); -} - -ERTS_GLB_INLINE Uint32 erts_port_status_get(Port *prt) -{ - Uint32 res; - erts_smp_port_state_lock(prt); - res = prt->status; - erts_smp_port_state_unlock(prt); - return res; -} -#endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ - /* erl_drv_thread.c */ void erl_drv_thr_init(void); -/* time.c */ - /* utils.c */ - -typedef struct { -#ifdef DEBUG - int smp_api; -#endif - union { - Uint64 not_atomic; -#ifdef ARCH_64 - erts_atomic_t atomic; -#else - erts_dw_atomic_t atomic; -#endif - } counter; -} erts_interval_t; - -void erts_interval_init(erts_interval_t *); -void erts_smp_interval_init(erts_interval_t *); -Uint64 erts_step_interval_nob(erts_interval_t *); -Uint64 erts_step_interval_relb(erts_interval_t *); -Uint64 erts_smp_step_interval_nob(erts_interval_t *); -Uint64 erts_smp_step_interval_relb(erts_interval_t *); -Uint64 erts_ensure_later_interval_nob(erts_interval_t *, Uint64); -Uint64 erts_ensure_later_interval_acqb(erts_interval_t *, Uint64); -Uint64 erts_smp_ensure_later_interval_nob(erts_interval_t *, Uint64); -Uint64 erts_smp_ensure_later_interval_acqb(erts_interval_t *, Uint64); -#ifdef ARCH_32 -ERTS_GLB_INLINE Uint64 erts_interval_dw_aint_to_val__(erts_dw_aint_t *); -#endif -ERTS_GLB_INLINE Uint64 erts_current_interval_nob__(erts_interval_t *); -ERTS_GLB_INLINE Uint64 erts_current_interval_acqb__(erts_interval_t *); -ERTS_GLB_INLINE Uint64 erts_current_interval_nob(erts_interval_t *); -ERTS_GLB_INLINE Uint64 erts_current_interval_acqb(erts_interval_t *); -ERTS_GLB_INLINE Uint64 erts_smp_current_interval_nob(erts_interval_t *); -ERTS_GLB_INLINE Uint64 erts_smp_current_interval_acqb(erts_interval_t *); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -#ifdef ARCH_32 - -ERTS_GLB_INLINE Uint64 -erts_interval_dw_aint_to_val__(erts_dw_aint_t *dw) -{ -#ifdef ETHR_SU_DW_NAINT_T__ - return (Uint64) dw->dw_sint; -#else - Uint64 res; - res = (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_HIGH_WORD]); - res <<= 32; - res |= (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_LOW_WORD]); - return res; -#endif -} - -#endif - -ERTS_GLB_INLINE Uint64 -erts_current_interval_nob__(erts_interval_t *icp) -{ -#ifdef ARCH_64 - return (Uint64) erts_atomic_read_nob(&icp->counter.atomic); -#else - erts_dw_aint_t dw; - erts_dw_atomic_read_nob(&icp->counter.atomic, &dw); - return erts_interval_dw_aint_to_val__(&dw); -#endif -} - -ERTS_GLB_INLINE Uint64 -erts_current_interval_acqb__(erts_interval_t *icp) -{ -#ifdef ARCH_64 - return (Uint64) erts_atomic_read_acqb(&icp->counter.atomic); -#else - erts_dw_aint_t dw; - erts_dw_atomic_read_acqb(&icp->counter.atomic, &dw); - return erts_interval_dw_aint_to_val__(&dw); -#endif -} - -ERTS_GLB_INLINE Uint64 -erts_current_interval_nob(erts_interval_t *icp) -{ - ASSERT(!icp->smp_api); - return erts_current_interval_nob__(icp); -} - -ERTS_GLB_INLINE Uint64 -erts_current_interval_acqb(erts_interval_t *icp) -{ - ASSERT(!icp->smp_api); - return erts_current_interval_acqb__(icp); -} - -ERTS_GLB_INLINE Uint64 -erts_smp_current_interval_nob(erts_interval_t *icp) -{ - ASSERT(icp->smp_api); -#ifdef ERTS_SMP - return erts_current_interval_nob__(icp); -#else - return icp->counter.not_atomic; -#endif -} - -ERTS_GLB_INLINE Uint64 -erts_smp_current_interval_acqb(erts_interval_t *icp) -{ - ASSERT(icp->smp_api); -#ifdef ERTS_SMP - return erts_current_interval_acqb__(icp); -#else - return icp->counter.not_atomic; -#endif -} - -#endif /* ERTS_GLB_INLINE_INCL_FUNC_DEF */ - -/* - * To be used to silence unused result warnings, but do not abuse it. - */ -void erts_silence_warn_unused_result(long unused); - void erts_cleanup_offheap(ErlOffHeap *offheap); -int erts_fit_in_bits_int64(Sint64); -int erts_fit_in_bits_int32(Sint32); -int list_length(Eterm); Export* erts_find_function(Eterm, Eterm, unsigned int, ErtsCodeIndex); -int erts_is_builtin(Eterm, Eterm, int); -Uint32 make_broken_hash(Eterm); -Uint32 block_hash(byte *, unsigned, Uint32); -Uint32 make_hash2(Eterm); -Uint32 make_hash(Eterm); - - -Eterm erts_bld_atom(Uint **hpp, Uint *szp, char *str); -Eterm erts_bld_uint(Uint **hpp, Uint *szp, Uint ui); -Eterm erts_bld_uword(Uint **hpp, Uint *szp, UWord uw); -Eterm erts_bld_uint64(Uint **hpp, Uint *szp, Uint64 ui64); -Eterm erts_bld_sint64(Uint **hpp, Uint *szp, Sint64 si64); -Eterm erts_bld_cons(Uint **hpp, Uint *szp, Eterm car, Eterm cdr); -Eterm erts_bld_tuple(Uint **hpp, Uint *szp, Uint arity, ...); -Eterm erts_bld_tuplev(Uint **hpp, Uint *szp, Uint arity, Eterm terms[]); -Eterm erts_bld_string_n(Uint **hpp, Uint *szp, const char *str, Sint len); -#define erts_bld_string(hpp,szp,str) erts_bld_string_n(hpp,szp,str,strlen(str)) -Eterm erts_bld_list(Uint **hpp, Uint *szp, Sint length, Eterm terms[]); -Eterm erts_bld_2tup_list(Uint **hpp, Uint *szp, - Sint length, Eterm terms1[], Uint terms2[]); -Eterm -erts_bld_atom_uint_2tup_list(Uint **hpp, Uint *szp, - Sint length, Eterm atoms[], Uint uints[]); -Eterm -erts_bld_atom_2uint_3tup_list(Uint **hpp, Uint *szp, Sint length, - Eterm atoms[], Uint uints1[], Uint uints2[]); Eterm store_external_or_ref_in_proc_(Process *, Eterm); Eterm store_external_or_ref_(Uint **, ErlOffHeap*, Eterm); @@ -1458,42 +761,6 @@ Eterm store_external_or_ref_(Uint **, ErlOffHeap*, Eterm); (ASSERT_EXPR(is_node_container((NC))), \ IS_CONST((NC)) ? (NC) : store_external_or_ref_in_proc_((Pp), (NC))) -void erts_init_utils(void); -void erts_init_utils_mem(void); - -erts_dsprintf_buf_t *erts_create_tmp_dsbuf(Uint); -void erts_destroy_tmp_dsbuf(erts_dsprintf_buf_t *); - -#if HALFWORD_HEAP -int eq_rel(Eterm a, Eterm* a_base, Eterm b, Eterm* b_base); -# define eq(A,B) eq_rel(A,NULL,B,NULL) -#else -int eq(Eterm, Eterm); -# define eq_rel(A,A_BASE,B,B_BASE) eq(A,B) -#endif - -#define EQ(x,y) (((x) == (y)) || (is_not_both_immed((x),(y)) && eq((x),(y)))) - -#if HALFWORD_HEAP -Sint cmp_rel(Eterm, Eterm*, Eterm, Eterm*); -#define CMP(A,B) cmp_rel(A,NULL,B,NULL) -#else -Sint cmp(Eterm, Eterm); -#define cmp_rel(A,A_BASE,B,B_BASE) cmp(A,B) -#define CMP(A,B) cmp(A,B) -#endif -#define cmp_lt(a,b) (CMP((a),(b)) < 0) -#define cmp_le(a,b) (CMP((a),(b)) <= 0) -#define cmp_eq(a,b) (CMP((a),(b)) == 0) -#define cmp_ne(a,b) (CMP((a),(b)) != 0) -#define cmp_ge(a,b) (CMP((a),(b)) >= 0) -#define cmp_gt(a,b) (CMP((a),(b)) > 0) - -#define CMP_LT(a,b) ((a) != (b) && cmp_lt((a),(b))) -#define CMP_GE(a,b) ((a) == (b) || cmp_ge((a),(b))) -#define CMP_EQ(a,b) ((a) == (b) || cmp_eq((a),(b))) -#define CMP_NE(a,b) ((a) != (b) && cmp_ne((a),(b))) - /* duplicates from big.h */ int term_to_Uint(Eterm term, Uint *up); int term_to_UWord(Eterm, UWord*); @@ -1530,79 +797,6 @@ Eterm erts_convert_native_to_filename(Process *p, byte *bytes); #define ERTS_UTF8_ERROR 2 #define ERTS_UTF8_ANALYZE_MORE 3 -/* erl_trace.c */ -void erts_init_trace(void); -void erts_trace_check_exiting(Eterm exiting); -Eterm erts_set_system_seq_tracer(Process *c_p, - ErtsProcLocks c_p_locks, - Eterm new); -Eterm erts_get_system_seq_tracer(void); -void erts_change_default_tracing(int setflags, Uint *flagsp, Eterm *tracerp); -void erts_get_default_tracing(Uint *flagsp, Eterm *tracerp); -void erts_set_system_monitor(Eterm monitor); -Eterm erts_get_system_monitor(void); - -#ifdef ERTS_SMP -void erts_check_my_tracer_proc(Process *); -void erts_block_sys_msg_dispatcher(void); -void erts_release_sys_msg_dispatcher(void); -void erts_foreach_sys_msg_in_q(void (*func)(Eterm, - Eterm, - Eterm, - ErlHeapFragment *)); -void erts_queue_error_logger_message(Eterm, Eterm, ErlHeapFragment *); -#endif - -void erts_send_sys_msg_proc(Eterm, Eterm, Eterm, ErlHeapFragment *); -void trace_send(Process*, Eterm, Eterm); -void trace_receive(Process*, Eterm); -Uint32 erts_call_trace(Process *p, BeamInstr mfa[], Binary *match_spec, Eterm* args, - int local, Eterm *tracer_pid); -void erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid); -void erts_trace_exception(Process* p, BeamInstr mfa[], Eterm class, Eterm value, - Eterm *tracer); -void erts_trace_return_to(Process *p, BeamInstr *pc); -void trace_sched(Process*, Eterm); -void trace_proc(Process*, Process*, Eterm, Eterm); -void trace_proc_spawn(Process*, Eterm pid, Eterm mod, Eterm func, Eterm args); -void save_calls(Process *p, Export *); -void trace_gc(Process *p, Eterm what); -/* port tracing */ -void trace_virtual_sched(Process*, Eterm); -void trace_sched_ports(Port *pp, Eterm); -void trace_sched_ports_where(Port *pp, Eterm, Eterm); -void trace_port(Port *, Eterm what, Eterm data); -void trace_port_open(Port *, Eterm calling_pid, Eterm drv_name); - -/* system_profile */ -void erts_set_system_profile(Eterm profile); -Eterm erts_get_system_profile(void); -void profile_scheduler(Eterm scheduler_id, Eterm); -void profile_scheduler_q(Eterm scheduler_id, Eterm state, Eterm no_schedulers, Uint Ms, Uint s, Uint us); -void profile_runnable_proc(Process* p, Eterm status); -void profile_runnable_port(Port* p, Eterm status); -void erts_system_profile_setup_active_schedulers(void); - -/* system_monitor */ -void monitor_long_gc(Process *p, Uint time); -void monitor_large_heap(Process *p); -void monitor_generic(Process *p, Eterm type, Eterm spec); -Uint erts_trace_flag2bit(Eterm flag); -int erts_trace_flags(Eterm List, - Uint *pMask, Eterm *pTracer, int *pCpuTimestamp); -Eterm erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr *I); - -#ifdef ERTS_SMP -void erts_send_pending_trace_msgs(ErtsSchedulerData *esdp); -#define ERTS_SMP_CHK_PEND_TRACE_MSGS(ESDP) \ -do { \ - if ((ESDP)->pending_trace_msgs) \ - erts_send_pending_trace_msgs((ESDP)); \ -} while (0) -#else -#define ERTS_SMP_CHK_PEND_TRACE_MSGS(ESDP) -#endif - void bin_write(int, void*, byte*, size_t); int intlist_to_buf(Eterm, char*, int); /* most callers pass plain char*'s */ @@ -1620,9 +814,16 @@ char* Sint_to_buf(Sint, struct Sint_buf*); #define ERTS_IOLIST_TYPE 2 Eterm buf_to_intlist(Eterm**, char*, size_t, Eterm); /* most callers pass plain char*'s */ -int io_list_to_buf(Eterm, char*, int); -int io_list_to_buf2(Eterm, char*, int); -int erts_iolist_size(Eterm, Uint *); + +#define ERTS_IOLIST_TO_BUF_OVERFLOW (~((ErlDrvSizeT) 0)) +#define ERTS_IOLIST_TO_BUF_TYPE_ERROR (~((ErlDrvSizeT) 1)) +#define ERTS_IOLIST_TO_BUF_FAILED(R) \ + (((R) & (~((ErlDrvSizeT) 1))) == (~((ErlDrvSizeT) 1))) +#define ERTS_IOLIST_TO_BUF_SUCCEEDED(R) \ + (!ERTS_IOLIST_TO_BUF_FAILED((R))) + +ErlDrvSizeT erts_iolist_to_buf(Eterm, char*, ErlDrvSizeT); +int erts_iolist_size(Eterm, ErlDrvSizeT *); int is_string(Eterm); void erl_at_exit(void (*) (void*), void*); Eterm collect_memory(Process *); @@ -1667,41 +868,6 @@ Uint erts_current_reductions(Process* current, Process *p); int erts_print_system_version(int to, void *arg, Process *c_p); int erts_hibernate(Process* c_p, Eterm module, Eterm function, Eterm args, Eterm* reg); -#define seq_trace_output(token, msg, type, receiver, process) \ -seq_trace_output_generic((token), (msg), (type), (receiver), (process), NIL) -#define seq_trace_output_exit(token, msg, type, receiver, exitfrom) \ -seq_trace_output_generic((token), (msg), (type), (receiver), NULL, (exitfrom)) -void seq_trace_output_generic(Eterm token, Eterm msg, Uint type, - Eterm receiver, Process *process, Eterm exitfrom); - -int seq_trace_update_send(Process *process); - -Eterm erts_seq_trace(Process *process, - Eterm atom_type, Eterm atom_true_or_false, - int build_result); - -struct trace_pattern_flags { - unsigned int breakpoint : 1; /* Set if any other is set */ - unsigned int local : 1; /* Local call trace breakpoint */ - unsigned int meta : 1; /* Metadata trace breakpoint */ - unsigned int call_count : 1; /* Fast call count breakpoint */ - unsigned int call_time : 1; /* Fast call time breakpoint */ -}; -extern const struct trace_pattern_flags erts_trace_pattern_flags_off; -extern int erts_call_time_breakpoint_tracing; -int erts_set_trace_pattern(Process*p, Eterm* mfa, int specified, - Binary* match_prog_set, Binary *meta_match_prog_set, - int on, struct trace_pattern_flags, - Eterm meta_tracer_pid, int is_blocking); -void -erts_get_default_trace_pattern(int *trace_pattern_is_on, - Binary **match_spec, - Binary **meta_match_spec, - struct trace_pattern_flags *trace_pattern_flags, - Eterm *meta_tracer_pid); -int erts_is_default_trace_enabled(void); -void erts_bif_trace_init(void); -int erts_finish_breakpointing(void); /* ** Call_trace uses this API for the parameter matching functions @@ -1943,15 +1109,15 @@ dtrace_pid_str(Eterm pid, char *process_buf) ERTS_GLB_INLINE void dtrace_proc_str(Process *process, char *process_buf) { - dtrace_pid_str(process->id, process_buf); + dtrace_pid_str(process->common.id, process_buf); } ERTS_GLB_INLINE void dtrace_port_str(Port *port, char *port_buf) { erts_snprintf(port_buf, DTRACE_TERM_BUF_SIZE, "#Port<%lu.%lu>", - port_channel_no(port->id), - port_number(port->id)); + port_channel_no(port->common.id), + port_number(port->common.id)); } ERTS_GLB_INLINE void diff --git a/erts/emulator/beam/index.c b/erts/emulator/beam/index.c index c981a0a55e..79c3ecf1b3 100644 --- a/erts/emulator/beam/index.c +++ b/erts/emulator/beam/index.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2009. All Rights Reserved. + * Copyright Ericsson AB 1996-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/beam/io.c b/erts/emulator/beam/io.c index 60b9238d38..be094862d4 100644 --- a/erts/emulator/beam/io.c +++ b/erts/emulator/beam/io.c @@ -43,6 +43,8 @@ #include "erl_version.h" #include "error.h" #include "erl_async.h" +#define ERTS_WANT_EXTERNAL_TAGS +#include "external.h" #include "dtrace-wrapper.h" extern ErlDrvEntry fd_driver_entry; @@ -51,25 +53,29 @@ extern ErlDrvEntry spawn_driver_entry; extern ErlDrvEntry *driver_tab[]; /* table of static drivers, only used during initialization */ erts_driver_t *driver_list; /* List of all drivers, static and dynamic. */ -erts_smp_mtx_t erts_driver_list_lock; /* Mutex for driver list */ +erts_smp_rwmtx_t erts_driver_list_lock; /* Mutex for driver list */ static erts_smp_tsd_key_t driver_list_lock_status_key; /*stop recursive locks when calling driver init */ static erts_smp_tsd_key_t driver_list_last_error_key; /* Save last DDLL error on a per thread basis (for BC interfaces) */ -Port* erts_port; /* The port table */ +ErtsPTab erts_port erts_align_attribute(ERTS_CACHE_LINE_SIZE); /* The port table */ erts_smp_atomic_t erts_bytes_out; /* No bytes sent out of the system */ erts_smp_atomic_t erts_bytes_in; /* No bytes gotten into the system */ -Uint erts_max_ports; -Uint erts_port_tab_index_mask; - const ErlDrvTermData driver_term_nil = (ErlDrvTermData)NIL; +const Port erts_invalid_port = {{ERTS_INVALID_PORT}}; + erts_driver_t vanilla_driver; erts_driver_t spawn_driver; erts_driver_t fd_driver; +int erts_port_synchronous_ops = 0; +int erts_port_schedule_all_ops = 0; +int erts_port_parallelism = 0; + +static void deliver_result(Eterm sender, Eterm pid, Eterm res); static int init_driver(erts_driver_t *, ErlDrvEntry *, DE_Handle *); static void terminate_port(Port *p); static void pdl_init(void); @@ -89,36 +95,12 @@ static void driver_monitor_unlock_pdl(Port *p); static ERTS_INLINE ErlIOQueue* drvport2ioq(ErlDrvPort drvport) { - int ix = (int) drvport; - Uint32 status; - - if (ix < 0 || erts_max_ports <= ix) + Port *prt = erts_thr_drvport2port_raw(drvport); + erts_aint32_t state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) return NULL; - - if (erts_get_scheduler_data()) { - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(&erts_port[ix])); - ERTS_LC_ASSERT(!erts_port[ix].port_data_lock - || erts_lc_mtx_is_locked( - &erts_port[ix].port_data_lock->mtx)); - - status = erts_port[ix].status; - } - else { - erts_smp_port_state_lock(&erts_port[ix]); - status = erts_port[ix].status; - erts_smp_port_state_unlock(&erts_port[ix]); - - ERTS_LC_ASSERT((status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) - || erts_port[ix].port_data_lock); - ERTS_LC_ASSERT(!erts_port[ix].port_data_lock - || erts_lc_mtx_is_locked( - &erts_port[ix].port_data_lock->mtx)); - - } - - return ((status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) - ? NULL - : &erts_port[ix].ioq); + else + return &prt->ioq; } static ERTS_INLINE int @@ -196,27 +178,13 @@ typedef struct line_buf_context { dtrace_port_str((PORT), port_str); #endif -/* The 'number' field in a port now has two parts: the lowest bits - contain the index in the port table, and the higher bits are a counter - which is incremented each time we look for a free port and start from - the beginning of the table. erts_max_ports is the number of file descriptors, - rounded up to a power of 2. - To get the index from a port, use the macro 'internal_port_index'; - 'port_number' returns the whole number field. -*/ - -static erts_smp_spinlock_t get_free_port_lck; -static Uint last_port_num; -static Uint port_num_mask; -erts_smp_atomic32_t erts_ports_snapshot; /* Identifies the _next_ snapshot (not the ongoing) */ - - static ERTS_INLINE void kill_port(Port *pp) { ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); + erts_ptab_delete_element(&erts_port, &pp->common); /* Time of death */ erts_port_task_free_port(pp); - ASSERT(pp->status & ERTS_PORT_SFLGS_DEAD); + /* In non-smp case the port structure may have been deallocated now */ } #ifdef ERTS_SMP @@ -227,146 +195,280 @@ erts_lc_is_port_locked(Port *prt) { if (!prt) return 0; + ERTS_SMP_LC_ASSERT(prt->lock); return erts_smp_lc_mtx_is_locked(prt->lock); } #endif #endif /* #ifdef ERTS_SMP */ -static int -get_free_port(void) -{ - Uint num; - Uint tries = erts_max_ports; - Port* port; +static void initq(Port* prt); - erts_smp_spin_lock(&get_free_port_lck); - num = last_port_num + 1; - for (;; ++num) { - port = &erts_port[num & erts_port_tab_index_mask]; +#if defined(ERTS_ENABLE_LOCK_CHECK) || defined(ERTS_ENABLE_LOCK_COUNT) +#define ERTS_PORT_INIT_INSTR_NEED_ID 1 +#else +#define ERTS_PORT_INIT_INSTR_NEED_ID 0 +#endif - erts_smp_port_state_lock(port); - if (port->status & ERTS_PORT_SFLG_FREE) { - last_port_num = num; - erts_smp_spin_unlock(&get_free_port_lck); - break; - } - erts_smp_port_state_unlock(port); +static ERTS_INLINE void port_init_instr(Port *prt +#if ERTS_PORT_INIT_INSTR_NEED_ID + , Eterm id +#endif + ) +{ +#if !ERTS_PORT_INIT_INSTR_NEED_ID + Eterm id = NIL; /* Not used */ +#endif - if (--tries == 0) { - erts_smp_spin_unlock(&get_free_port_lck); - return -1; - } + /* + * Stuff that need to be initialized with the port id + * in the instrumented case, but not in the normal case. + */ +#ifdef ERTS_SMP + ASSERT(prt->drv_ptr && prt->lock); + if (!prt->drv_ptr->lock) { + char *lock_str = "port_lock"; +#ifdef ERTS_ENABLE_LOCK_COUNT + if (!(erts_lcnt_rt_options & ERTS_LCNT_OPT_PORTLOCK)) + lock_str = NULL; +#endif + erts_mtx_init_locked_x(prt->lock, lock_str, id); } - port->status = ERTS_PORT_SFLG_INITIALIZING; - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&port->refc) == 0); - erts_smp_atomic_set_nob(&port->refc, 2); /* Port alive + lock */ - erts_smp_port_state_unlock(port); - return num & port_num_mask; +#endif + erts_port_task_init_sched(&prt->sched, id); } -/* - * erts_test_next_port() is only used for testing. - */ -Sint -erts_test_next_port(int set, Uint next) +#if !ERTS_PORT_INIT_INSTR_NEED_ID +static ERTS_INLINE void port_init_instr_abort(Port *prt) { - Uint i, num; - Sint res = -1; - - erts_smp_spin_lock(&get_free_port_lck); - if (set) { - last_port_num = (next - 1) & port_num_mask; +#ifdef ERTS_SMP + ASSERT(prt->drv_ptr && prt->lock); + if (!prt->drv_ptr->lock) { + erts_mtx_unlock(prt->lock); + erts_mtx_destroy(prt->lock); } - num = last_port_num + 1; +#endif + erts_port_task_fini_sched(&prt->sched); +} +#endif - for (i=0; i < erts_max_ports && res<0; ++i, ++num) { - - Port* port = &erts_port[num & erts_port_tab_index_mask]; +static void insert_port_struct(void *vprt, Eterm data) +{ + Port *prt = (Port *) vprt; + Eterm id = make_internal_port(data); +#if ERTS_PORT_INIT_INSTR_NEED_ID + /* + * This cannot be done earlier in the instrumented + * case since we don't now 'id' until now. + */ + port_init_instr(prt, id); +#endif + prt->common.id = id; + erts_atomic32_init_relb(&prt->state, ERTS_PORT_SFLG_INITIALIZING); +} - erts_smp_port_state_lock(port); +#define ERTS_CREATE_PORT_FLAG_PARALLELISM (1 << 0) - if (port->status & ERTS_PORT_SFLG_FREE) { - last_port_num = num - 1; - res = num & port_num_mask; - } - erts_smp_port_state_unlock(port); +static Port *create_port(char *name, + erts_driver_t *driver, + erts_mtx_t *driver_lock, + int create_flags, + Eterm pid, + int *enop) +{ + ErtsPortTaskBusyPortQ *busy_port_queue; + Port *prt; + char *p; + size_t port_size, busy_port_queue_size, size; + erts_aint32_t state = ERTS_PORT_SFLG_CONNECTED; + erts_aint32_t x_pts_flgs = 0; +#ifdef DEBUG + /* Make sure the debug flags survives until port is freed */ + state |= ERTS_PORT_SFLG_PORT_DEBUG; +#endif + +#ifdef ERTS_SMP + if (!driver_lock) { + /* Align size for mutex following port struct */ + port_size = size = ERTS_ALC_DATA_ALIGN_SIZE(sizeof(Port)); + size += sizeof(erts_mtx_t); } - erts_smp_spin_unlock(&get_free_port_lck); - return res; -} + else +#endif + port_size = size = ERTS_ALC_DATA_ALIGN_SIZE(sizeof(Port)); + busy_port_queue_size + = ((driver->flags & ERL_DRV_FLAG_NO_BUSY_MSGQ) + ? 0 + : ERTS_ALC_DATA_ALIGN_SIZE(sizeof(ErtsPortTaskBusyPortQ))); + size += busy_port_queue_size; -static void port_cleanup(Port *prt); + size += sys_strlen(name) + 1; -#ifdef ERTS_SMP + p = erts_alloc_fnf(ERTS_ALC_T_PORT, size); + if (!p) { + if (enop) + *enop = ENOMEM; + return NULL; + } -static void -sched_port_cleanup(void *vprt) -{ - Port *prt = (Port *) vprt; - erts_smp_mtx_lock(prt->lock); - port_cleanup(prt); -} + prt = (Port *) p; + p += port_size; -#endif + if (!busy_port_queue_size) + busy_port_queue = NULL; + else { + busy_port_queue = (ErtsPortTaskBusyPortQ *) p; + p += busy_port_queue_size; + } -void -erts_port_cleanup(Port *prt) -{ #ifdef ERTS_SMP - if (erts_smp_mtx_trylock(prt->lock) == EBUSY) - erts_schedule_misc_op(sched_port_cleanup, (void *) prt); - else + if (driver_lock) { + prt->lock = driver_lock; + erts_mtx_lock(driver_lock); + } + else { + prt->lock = (erts_mtx_t *) p; + p += sizeof(erts_mtx_t); + state |= ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK; + } + erts_smp_atomic_set_nob(&prt->run_queue, + (erts_aint_t) erts_get_runq_current(NULL)); + prt->xports = NULL; +#else + erts_atomic32_init_nob(&prt->refc, 1); + prt->cleanup = 0; #endif - port_cleanup(prt); -} + + erts_port_task_pre_init_sched(&prt->sched, busy_port_queue); -void -port_cleanup(Port *prt) -{ + prt->name = p; + sys_strcpy(p, name); + prt->drv_ptr = driver; + ERTS_P_LINKS(prt) = NULL; + ERTS_P_MONITORS(prt) = NULL; + prt->linebuf = NULL; + prt->bp = NULL; + prt->suspended = NULL; + prt->data = am_undefined; + prt->port_data_lock = NULL; + prt->control_flags = 0; + prt->bytes_in = 0; + prt->bytes_out = 0; + prt->dist_entry = NULL; + ERTS_PORT_INIT_CONNECTED(prt, pid); + prt->common.u.alive.reg = NULL; #ifdef ERTS_SMP - Uint32 port_specific; - erts_smp_mtx_t *mtx; + prt->common.u.alive.ptimer = NULL; +#else + sys_memset(&prt->common.u.alive.tm, 0, sizeof(ErlTimer)); #endif - erts_driver_t *driver; + erts_port_task_handle_init(&prt->timeout_task); + prt->psd = NULL; + prt->drv_data = (SWord) 0; + prt->os_pid = -1; - erts_smp_port_state_lock(prt); + /* Set default tracing */ + erts_get_default_tracing(&ERTS_TRACE_FLAGS(prt), &ERTS_TRACER_PROC(prt)); - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - driver = prt->drv_ptr; - prt->drv_ptr = NULL; - ASSERT(driver); + ASSERT(((char *) prt) == ((char *) &prt->common)); - ASSERT(prt->status & ERTS_PORT_SFLG_FREE_SCHEDULED); - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&prt->refc) == 0); +#if !ERTS_PORT_INIT_INSTR_NEED_ID + /* + * When 'id' isn't needed (the normal case), it is better to + * do the initialization here avoiding unnecessary contention + * on table... + */ + port_init_instr(prt); +#endif - ASSERT(prt->status & ERTS_PORT_SFLG_PORT_DEBUG); - ASSERT(!(prt->status & ERTS_PORT_SFLG_FREE)); - prt->status = ERTS_PORT_SFLG_FREE; + if (!erts_ptab_new_element(&erts_port, + &prt->common, + (void *) prt, + insert_port_struct)) { +#if !ERTS_PORT_INIT_INSTR_NEED_ID + port_init_instr_abort(prt); +#endif #ifdef ERTS_SMP + if (driver_lock) + erts_mtx_unlock(driver_lock); +#endif + if (enop) + *enop = 0; + return NULL; + } - port_specific = (prt->status & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK); + ASSERT(prt == (Port *) (erts_ptab_pix2intptr_nob( + &erts_port, + internal_port_index(prt->common.id)))); - mtx = prt->lock; - ASSERT(mtx); + initq(prt); - prt->lock = NULL; + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - erts_smp_port_state_unlock(prt); - erts_smp_mtx_unlock(mtx); + if (erts_port_schedule_all_ops) + x_pts_flgs |= ERTS_PTS_FLG_FORCE_SCHED; - if (port_specific) { - erts_smp_mtx_destroy(mtx); - erts_free(ERTS_ALC_T_PORT_LOCK, mtx); - } -#endif + if (create_flags & ERTS_CREATE_PORT_FLAG_PARALLELISM) + x_pts_flgs |= ERTS_PTS_FLG_PARALLELISM; - if (driver->handle) - erts_ddll_dereference_driver(driver->handle); + if (x_pts_flgs) + erts_smp_atomic32_read_bor_nob(&prt->sched.flags, x_pts_flgs); + + erts_atomic32_set_relb(&prt->state, state); + return prt; +} + +#ifndef ERTS_SMP +void +erts_port_cleanup(Port *prt) +{ + if (prt->drv_ptr && prt->drv_ptr->handle) + erts_ddll_dereference_driver(prt->drv_ptr->handle); + prt->drv_ptr = NULL; + erts_port_dec_refc(prt); } +#endif +void +erts_port_free(Port *prt) +{ +#if defined(ERTS_SMP) || defined(DEBUG) || defined(ERTS_ENABLE_LOCK_CHECK) + erts_aint32_t state = erts_atomic32_read_nob(&prt->state); +#endif + ERTS_LC_ASSERT(state & (ERTS_PORT_SFLG_INITIALIZING + | ERTS_PORT_SFLG_FREE)); + ASSERT(state & ERTS_PORT_SFLG_PORT_DEBUG); + +#ifdef ERTS_SMP + ERTS_LC_ASSERT(erts_atomic32_read_nob(&prt->common.refc) == 0); +#else + ERTS_LC_ASSERT(erts_atomic32_read_nob(&prt->refc) == 0); +#endif + + erts_port_task_fini_sched(&prt->sched); + +#ifdef ERTS_SMP + ASSERT(prt->lock); + if (state & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK) + erts_mtx_destroy(prt->lock); + + /* + * We cannot dereference a driver using driver + * locking until here in smp case. Otherwise, + * the driver lock may still be in use by others. + * + * In the non-smp case we cannot do it here since + * this function may be called by non-scheduler + * threads. This is done in erts_port_cleanup() + * in the non-smp case. + */ + if (prt->drv_ptr->handle) + erts_ddll_dereference_driver(prt->drv_ptr->handle); +#endif + erts_free(ERTS_ALC_T_PORT, prt); +} /* ** Initialize v_start to point to the small fixed vector. @@ -414,94 +516,21 @@ static void stopq(Port* prt) if (prt->port_data_lock) { driver_pdl_unlock(prt->port_data_lock); driver_pdl_dec_refc(prt->port_data_lock); - prt->port_data_lock = NULL; - } -} - - - -static void -setup_port(Port* prt, Eterm pid, erts_driver_t *driver, - ErlDrvData drv_data, char *name, Uint32 xstatus) -{ - ErtsRunQueue *runq = erts_get_runq_current(NULL); - char *new_name, *old_name; -#ifdef DEBUG - /* Make sure the debug flags survives until port is freed */ - xstatus |= ERTS_PORT_SFLG_PORT_DEBUG; -#endif - ASSERT(runq); - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - - - new_name = (char*) erts_alloc(ERTS_ALC_T_PORT_NAME, sys_strlen(name)+1); - sys_strcpy(new_name, name); - erts_smp_runq_lock(runq); - erts_smp_port_state_lock(prt); - prt->os_pid = -1; - prt->status = ERTS_PORT_SFLG_CONNECTED | xstatus; - prt->snapshot = erts_smp_atomic32_read_nob(&erts_ports_snapshot); - old_name = prt->name; - prt->name = new_name; -#ifdef ERTS_SMP - erts_smp_atomic_set_nob(&prt->run_queue, (erts_aint_t) runq); -#endif - ASSERT(!prt->drv_ptr); - prt->drv_ptr = driver; - erts_smp_port_state_unlock(prt); - erts_smp_runq_unlock(runq); -#ifdef ERTS_SMP - ASSERT(!prt->xports); -#endif - if (old_name) { - erts_free(ERTS_ALC_T_PORT_NAME, (void *) old_name); } - - prt->control_flags = 0; - prt->connected = pid; - prt->drv_data = (SWord) drv_data; - prt->bytes_in = 0; - prt->bytes_out = 0; - prt->dist_entry = NULL; - prt->reg = NULL; -#ifdef ERTS_SMP - prt->ptimer = NULL; -#else - sys_memset(&prt->tm, 0, sizeof(ErlTimer)); -#endif - erts_port_task_handle_init(&prt->timeout_task); - prt->suspended = NULL; - sys_strcpy(prt->name, name); - prt->nlinks = NULL; - prt->monitors = NULL; - prt->linebuf = NULL; - prt->bp = NULL; - prt->data = am_undefined; - /* Set default tracing */ - erts_get_default_tracing(&(prt->trace_flags), &(prt->tracer_proc)); - - prt->psd = NULL; - - initq(prt); } -void -erts_wake_process_later(Port *prt, Process *process) +int +erts_save_suspend_process_on_port(Port *prt, Process *process) { - ErtsProcList** p; - ErtsProcList* new_p; - - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - - if (prt->status & ERTS_PORT_SFLGS_DEAD) - return; - - for (p = &(prt->suspended); *p != NULL; p = &((*p)->next)) - /* Empty loop body */; - - new_p = erts_proclist_create(process); - new_p->next = NULL; - *p = new_p; + int saved; + erts_aint32_t flags; + erts_port_task_sched_lock(&prt->sched); + flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + saved = (flags & ERTS_PTS_FLGS_BUSY) && !(flags & ERTS_PTS_FLG_EXIT); + if (saved) + erts_proclist_store_last(&prt->suspended, erts_proclist_create(process)); + erts_port_task_sched_unlock(&prt->sched); + return saved; } /* @@ -513,47 +542,44 @@ erts_wake_process_later(Port *prt, Process *process) (*error_number_ptr must contain either BADARG or SYSTEM_LIMIT). The driver start function must obey the same conventions. */ -int +Port * erts_open_driver(erts_driver_t* driver, /* Pointer to driver. */ Eterm pid, /* Current process. */ char* name, /* Driver name. */ SysDriverOpts* opts, /* Options. */ - int *error_number_ptr) /* errno in case -2 is returned */ + int *error_type_ptr, /* error type */ + int *error_number_ptr) /* errno in case of error type -2 */ { - int port_num; - int port_ix; + +#undef ERTS_OPEN_DRIVER_RET +#define ERTS_OPEN_DRIVER_RET(Prt, EType, ENo) \ + do { \ + if (error_type_ptr) \ + *error_type_ptr = (EType); \ + if (error_number_ptr) \ + *error_number_ptr = (ENo); \ + return (Prt); \ + } while (0) + ErlDrvData drv_data = 0; - Uint32 xstatus = 0; Port *port; int fpe_was_unmasked; - - if (error_number_ptr) - *error_number_ptr = 0; + int error_type, error_number; + int port_errno = 0; + erts_mtx_t *driver_lock = NULL; + int cprt_flgs = 0; ERTS_SMP_CHK_NO_PROC_LOCKS; - if ((port_num = get_free_port()) < 0) { - if (error_number_ptr) { - *error_number_ptr = SYSTEM_LIMIT; - } - return -3; - } - - port_ix = port_num & erts_port_tab_index_mask; - port = &erts_port[port_ix]; - port->id = make_internal_port(port_num); - - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rlock(&erts_driver_list_lock); if (!driver) { for (driver = driver_list; driver; driver = driver->next) { if (sys_strcmp(driver->name, name) == 0) break; } if (!driver) { - erts_smp_mtx_unlock(&erts_driver_list_lock); - if (error_number_ptr) - *error_number_ptr = BADARG; - return -3; + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + ERTS_OPEN_DRIVER_RET(NULL, -3, BADARG); } } if (driver == &spawn_driver) { @@ -597,59 +623,49 @@ erts_open_driver(erts_driver_t* driver, /* Pointer to driver. */ } if (driver == NULL || (driver != &spawn_driver && opts->exit_status)) { - erts_smp_mtx_unlock(&erts_driver_list_lock); - if (error_number_ptr) { - *error_number_ptr = BADARG; - } - /* Need to mark the port as free again */ - erts_smp_port_state_lock(port); - port->status = ERTS_PORT_SFLG_FREE; - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&port->refc) == 2); - erts_smp_atomic_set_nob(&port->refc, 0); - erts_smp_port_state_unlock(port); - return -3; + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + ERTS_OPEN_DRIVER_RET(NULL, -3, BADARG); } - /* - * We'll set up the port before calling the start function, - * to allow message sending and setting timers in the start function. - */ - #ifdef ERTS_SMP - ASSERT(!port->lock); - port->lock = driver->lock; - if (!port->lock) { - port->lock = erts_alloc(ERTS_ALC_T_PORT_LOCK, - sizeof(erts_smp_mtx_t)); - erts_smp_mtx_init_x(port->lock, -#ifdef ERTS_ENABLE_LOCK_COUNT - (erts_lcnt_rt_options & ERTS_LCNT_OPT_PORTLOCK) ? "port_lock" : NULL, -#else - "port_lock", -#endif - port->id); - xstatus |= ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK; - } + driver_lock = driver->lock; #endif if (driver->handle != NULL) { erts_ddll_increment_port_count(driver->handle); erts_ddll_reference_driver(driver->handle); } - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); -#ifdef ERTS_SMP - erts_smp_mtx_lock(port->lock); -#endif + /* + * We'll set up the port before calling the start function, + * to allow message sending and setting timers in the start function. + */ + + if (opts->parallelism) + cprt_flgs |= ERTS_CREATE_PORT_FLAG_PARALLELISM; - setup_port(port, pid, driver, drv_data, name, xstatus); + port = create_port(name, driver, driver_lock, cprt_flgs, pid, &port_errno); + if (!port) { + if (driver->handle) { + erts_smp_rwmtx_rlock(&erts_driver_list_lock); + erts_ddll_decrement_port_count(driver->handle); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + erts_ddll_dereference_driver(driver->handle); + } + if (port_errno) + ERTS_OPEN_DRIVER_RET(NULL, -2, port_errno); + else + ERTS_OPEN_DRIVER_RET(NULL, -3, SYSTEM_LIMIT); + } if (IS_TRACED_FL(port, F_TRACE_PORTS)) { trace_port_open(port, pid, am_atom_put(port->name, strlen(port->name))); } - + + error_number = error_type = 0; if (driver->start) { if (IS_TRACED_FL(port, F_TRACE_SCHED_PORTS)) { trace_sched_ports_where(port, am_in, am_start); @@ -662,56 +678,63 @@ erts_open_driver(erts_driver_t* driver, /* Pointer to driver. */ } #endif fpe_was_unmasked = erts_block_fpe(); - drv_data = (*driver->start)((ErlDrvPort)(port_ix), - name, opts); + drv_data = (*driver->start)((ErlDrvPort) port, name, opts); + if (((SWord) drv_data) == -1) + error_type = -1; + else if (((SWord) drv_data) == -2) { + /* + * We need to save errno quickly after the + * call to the 'start' callback before + * something else modify it. + */ + error_type = -2; + error_number = errno; + } + else if (((SWord) drv_data) == -3) { + error_type = -3; + error_number = BADARG; + } + erts_unblock_fpe(fpe_was_unmasked); port->caller = NIL; if (IS_TRACED_FL(port, F_TRACE_SCHED_PORTS)) { trace_sched_ports_where(port, am_out, am_start); } - if (error_number_ptr && ((SWord) drv_data) == (SWord) -2) - *error_number_ptr = errno; #ifdef ERTS_SMP if (port->xports) - erts_smp_xports_unlock(port); + erts_port_handle_xports(port); ASSERT(!port->xports); #endif } - if (((SWord)drv_data) == -1 || - ((SWord)drv_data) == -2 || - ((SWord)drv_data) == -3) { - int res = (int) ((SWord) drv_data); - - if (res == -3 && error_number_ptr) { - *error_number_ptr = BADARG; - } - + if (error_type) { /* * Must clean up the port. */ #ifdef ERTS_SMP - erts_cancel_smp_ptimer(port->ptimer); + erts_cancel_smp_ptimer(port->common.u.alive.ptimer); #else - erts_cancel_timer(&(port->tm)); + erts_cancel_timer(&(port->common.u.alive.tm)); #endif stopq(port); - kill_port(port); if (port->linebuf != NULL) { erts_free(ERTS_ALC_T_LINEBUF, (void *) port->linebuf); port->linebuf = NULL; } if (driver->handle != NULL) { - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rlock(&erts_driver_list_lock); erts_ddll_decrement_port_count(driver->handle); - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); } + kill_port(port); erts_port_release(port); - return res; + ERTS_OPEN_DRIVER_RET(NULL, error_type, error_number); } - port->drv_data = (SWord) drv_data; - return port_ix; + port->drv_data = (UWord) drv_data; + ERTS_OPEN_DRIVER_RET(port, 0, 0); + +#undef ERTS_OPEN_DRIVER_RET } #ifdef ERTS_SMP @@ -736,102 +759,122 @@ driver_create_port(ErlDrvPort creator_port_ix, /* Creating port */ char* name, /* Driver name */ ErlDrvData drv_data) /* Driver data */ { + int cprt_flgs = 0; Port *creator_port; Port* port; erts_driver_t *driver; Process *rp; - int port_num; - Eterm port_id; - Uint32 xstatus = 0; + erts_mtx_t *driver_lock = NULL; ERTS_SMP_CHK_NO_PROC_LOCKS; - creator_port = erts_drvport2port(creator_port_ix); + /* Need to be called from a scheduler thread */ + if (!erts_get_scheduler_id()) + return ERTS_INVALID_ERL_DRV_PORT; + + creator_port = erts_drvport2port(creator_port_ix, NULL); if (!creator_port) - return (ErlDrvTermData) -1; + return ERTS_INVALID_ERL_DRV_PORT; + + rp = erts_proc_lookup(pid); + if (!rp) + return ERTS_INVALID_ERL_DRV_PORT; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(creator_port)); driver = creator_port->drv_ptr; - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rlock(&erts_driver_list_lock); if (!erts_ddll_driver_ok(driver->handle)) { - erts_smp_mtx_unlock(&erts_driver_list_lock); - return (ErlDrvTermData) -1; + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + return ERTS_INVALID_ERL_DRV_PORT; } - rp = erts_pid2proc(NULL, 0, pid, ERTS_PROC_LOCK_LINK); - if (!rp) { - erts_smp_mtx_unlock(&erts_driver_list_lock); - return (ErlDrvTermData) -1; /* pid does not exist */ + if (driver->handle != NULL) { + erts_ddll_increment_port_count(driver->handle); + erts_ddll_reference_referenced_driver(driver->handle); + } + +#ifdef ERTS_SMP + driver_lock = driver->lock; +#endif + + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + + /* Inherit parallelism flag from parent */ + if (ERTS_PTS_FLG_PARALLELISM & + erts_smp_atomic32_read_nob(&creator_port->sched.flags)) + cprt_flgs |= ERTS_CREATE_PORT_FLAG_PARALLELISM; + port = create_port(name, driver, driver_lock, cprt_flgs, pid, NULL); + if (!port) { + if (driver->handle) { + erts_smp_rwmtx_rlock(&erts_driver_list_lock); + erts_ddll_decrement_port_count(driver->handle); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + erts_ddll_dereference_driver(driver->handle); + } + return ERTS_INVALID_ERL_DRV_PORT; } - if ((port_num = get_free_port()) < 0) { - errno = SYSTEM_LIMIT; + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(port)); + + erts_smp_proc_lock(rp, ERTS_PROC_LOCK_LINK); + if (ERTS_PROC_IS_EXITING(rp)) { erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); - erts_smp_mtx_unlock(&erts_driver_list_lock); - return (ErlDrvTermData) -1; + if (driver->handle) { + erts_smp_rwmtx_rlock(&erts_driver_list_lock); + erts_ddll_decrement_port_count(driver->handle); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + } + kill_port(port); + erts_port_release(port); + return ERTS_INVALID_ERL_DRV_PORT; } - port_id = make_internal_port(port_num); - port = &erts_port[port_num & erts_port_tab_index_mask]; + erts_add_link(&ERTS_P_LINKS(port), LINK_PID, pid); + erts_add_link(&ERTS_P_LINKS(rp), LINK_PID, port->common.id); + erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); #ifdef ERTS_SMP - ASSERT(!port->lock); - port->lock = driver->lock; - if (!port->lock) { + if (!driver_lock) { ErtsXPortsList *xplp = xports_list_alloc(); xplp->port = port; xplp->next = creator_port->xports; creator_port->xports = xplp; - port->lock = erts_alloc(ERTS_ALC_T_PORT_LOCK, - sizeof(erts_smp_mtx_t)); - erts_smp_mtx_init_locked_x(port->lock, -#ifdef ERTS_ENABLE_LOCK_COUNT - (erts_lcnt_rt_options & ERTS_LCNT_OPT_PORTLOCK) ? "port_lock" : NULL, -#else - "port_lock", -#endif - port_id); - xstatus |= ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK; } - #endif - if (driver->handle != NULL) { - erts_ddll_increment_port_count(driver->handle); - erts_ddll_reference_referenced_driver(driver->handle); - } - erts_smp_mtx_unlock(&erts_driver_list_lock); - - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(port)); + port->drv_data = (UWord) drv_data; - setup_port(port, pid, driver, drv_data, name, xstatus); - port->id = port_id; - - erts_add_link(&(port->nlinks), LINK_PID, pid); - erts_add_link(&(rp->nlinks), LINK_PID, port_id); - erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); - return port_num & erts_port_tab_index_mask; + return (ErlDrvPort) port; } #ifdef ERTS_SMP -void -erts_smp_xports_unlock(Port *prt) +int erts_port_handle_xports(Port *prt) { + int reds = 0; ErtsXPortsList *xplp; ASSERT(prt); xplp = prt->xports; ASSERT(xplp); while (xplp) { + Port *rprt = xplp->port; ErtsXPortsList *free_xplp; - if (xplp->port->xports) - erts_smp_xports_unlock(xplp->port); - erts_port_release(xplp->port); + erts_aint32_t state; + if (rprt->xports) + reds += erts_port_handle_xports(rprt); + state = erts_atomic32_read_nob(&rprt->state); + if ((state & ERTS_PORT_SFLG_CLOSING) && erts_is_port_ioq_empty(rprt)) { + terminate_port(rprt); + reds += ERTS_PORT_REDS_TERMINATE; + } + erts_port_release(rprt); free_xplp = xplp; xplp = xplp->next; xports_list_free(free_xplp); + reds++; } prt->xports = NULL; + return reds; } #endif @@ -866,8 +909,8 @@ io_list_to_vec(Eterm obj, /* io-list */ DECLARE_ESTACK(s); Eterm* objp; char *buf = cbin->orig_bytes; - ErlDrvSizeT len = cbin->orig_size; - ErlDrvSizeT csize = 0; + Uint len = cbin->orig_size; + Uint csize = 0; int vlen = 0; char* cptr = buf; @@ -982,7 +1025,7 @@ io_list_to_vec(Eterm obj, /* io-list */ #define IO_LIST_VEC_COUNT(obj) \ do { \ - ErlDrvSizeT _size = binary_size(obj); \ + Uint _size = binary_size(obj); \ Eterm _real; \ ERTS_DECLARE_DUMMY(Uint _offset); \ int _bitoffs; \ @@ -1033,8 +1076,9 @@ do { \ */ static int -io_list_vec_len(Eterm obj, Uint* vsize, Uint* csize, - Uint* pvsize, Uint* pcsize, Uint* total_size) +io_list_vec_len(Eterm obj, int* vsize, Uint* csize, + Uint* pvsize, Uint* pcsize, + ErlDrvSizeT* total_size) { DECLARE_ESTACK(s); Eterm* objp; @@ -1045,7 +1089,7 @@ io_list_vec_len(Eterm obj, Uint* vsize, Uint* csize, Uint p_v_size = 0; Uint p_c_size = 0; Uint p_in_clist = 0; - Uint total; + Uint total; /* Uint due to halfword emulator */ goto L_jump_start; /* avoid a push */ @@ -1105,7 +1149,7 @@ io_list_vec_len(Eterm obj, Uint* vsize, Uint* csize, if (total < c_size) { goto L_overflow_error; } - *total_size = total; + *total_size = (ErlDrvSizeT) total; DESTROY_ESTACK(s); *vsize = v_size; @@ -1120,56 +1164,724 @@ io_list_vec_len(Eterm obj, Uint* vsize, Uint* csize, return 1; } -/* write data to a port */ -int erts_write_to_port(Eterm caller_id, Port *p, Eterm list) -{ - char *buf; - erts_driver_t *drv = p->drv_ptr; - Uint size; +typedef enum { + ERTS_TRY_IMM_DRV_CALL_OK, + ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK, + ERTS_TRY_IMM_DRV_CALL_INVALID_PORT, + ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS +} ErtsTryImmDrvCallResult; + +typedef struct { + Process *c_p; /* Currently executing process (unlocked) */ + Port *port; /* Port to operate on */ + Eterm port_op; /* port operation as an atom */ + erts_aint32_t state; /* in: invalid state; out: read state (if read) */ + erts_aint32_t sched_flags; /* in: invalid flags; out: read flags (if read) */ + int async; /* Asynchronous operation */ + int pre_chk_sched_flags; /* Check sched flags before lock? */ int fpe_was_unmasked; - - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(p) || ERTS_IS_CRASH_DUMPING); +} ErtsTryImmDrvCallState; + +#define ERTS_INIT_TRY_IMM_DRV_CALL_STATE(C_P, PRT, SFLGS, PTS_FLGS, A, PRT_OP) \ + {(C_P), (PRT), (PRT_OP), (SFLGS), (PTS_FLGS), (A), 1, 0} + +/* + * Try doing an immediate driver callback call from a process. If + * this fail, the operation should be scheduled in the normal case... + * + */ +static ERTS_INLINE ErtsTryImmDrvCallResult +try_imm_drv_call(ErtsTryImmDrvCallState *sp) +{ + ErtsTryImmDrvCallResult res; + erts_aint32_t invalid_state, invalid_sched_flags; + Port *prt = sp->port; + Process *c_p = sp->c_p; + + ASSERT(is_atom(sp->port_op)); + + invalid_sched_flags = ERTS_PTS_FLGS_FORCE_SCHEDULE_OP; + invalid_sched_flags |= sp->sched_flags; + if (sp->async) + invalid_sched_flags |= ERTS_PTS_FLG_PARALLELISM; + + if (sp->pre_chk_sched_flags) { + sp->sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (sp->sched_flags & invalid_sched_flags) + return ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS; + } + + if (erts_smp_port_trylock(prt) == EBUSY) + return ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK; + + invalid_state = sp->state; + sp->state = erts_atomic32_read_nob(&prt->state); + if (sp->state & invalid_state) { + res = ERTS_TRY_IMM_DRV_CALL_INVALID_PORT; + goto locked_fail; + } + + sp->sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (sp->sched_flags & invalid_sched_flags) { + res = ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS; + goto locked_fail; + } + + if (c_p) { + if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) + trace_virtual_sched(c_p, am_out); + if (erts_system_profile_flags.runnable_procs + && erts_system_profile_flags.exclusive) + profile_runnable_proc(c_p, am_inactive); + + erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_MAIN); + } + ERTS_SMP_CHK_NO_PROC_LOCKS; - p->caller = caller_id; - if (drv->outputv != NULL) { - Uint vsize; - Uint csize; - Uint pvsize; - Uint pcsize; - ErlDrvSizeT blimit; + if (IS_TRACED_FL(prt, F_TRACE_SCHED_PORTS)) + trace_sched_ports_where(prt, am_in, sp->port_op); + if (erts_system_profile_flags.runnable_ports + && !erts_port_is_scheduled(prt)) + profile_runnable_port(prt, am_active); + + sp->fpe_was_unmasked = erts_block_fpe(); + + return ERTS_TRY_IMM_DRV_CALL_OK; + +locked_fail: + erts_port_release(prt); + return res; +} + +static ERTS_INLINE void +finalize_imm_drv_call(ErtsTryImmDrvCallState *sp) +{ + Port *prt = sp->port; + Process *c_p = sp->c_p; + + erts_port_driver_callback_epilogue(prt, NULL); + + erts_unblock_fpe(sp->fpe_was_unmasked); + + if (IS_TRACED_FL(prt, F_TRACE_SCHED_PORTS)) + trace_sched_ports_where(prt, am_out, sp->port_op); + if (erts_system_profile_flags.runnable_ports + && !erts_port_is_scheduled(prt)) + profile_runnable_port(prt, am_inactive); + + erts_port_release(prt); + + if (c_p) { + erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_MAIN); + + if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) + trace_virtual_sched(c_p, am_in); + if (erts_system_profile_flags.runnable_procs + && erts_system_profile_flags.exclusive) + profile_runnable_proc(c_p, am_active); + } +} + +/* + * force_imm_drv_call()/finalize_force_imm_drv_call() should *only* + * be used while crash dumping... + */ +static ErtsTryImmDrvCallResult +force_imm_drv_call(ErtsTryImmDrvCallState *sp) +{ + erts_aint32_t invalid_state; + Port *prt = sp->port; + + ASSERT(ERTS_IS_CRASH_DUMPING) + ASSERT(is_atom(sp->port_op)); + + invalid_state = sp->state; + sp->state = erts_atomic32_read_nob(&prt->state); + if (sp->state & invalid_state) + return ERTS_TRY_IMM_DRV_CALL_INVALID_PORT; + + sp->fpe_was_unmasked = erts_block_fpe(); + + return ERTS_TRY_IMM_DRV_CALL_OK; +} + +static void +finalize_force_imm_drv_call(ErtsTryImmDrvCallState *sp) +{ + erts_unblock_fpe(sp->fpe_was_unmasked); +} + +#define ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE (REF_THING_SIZE + 3) + +static ERTS_INLINE void +queue_port_sched_op_reply(Process *rp, + ErtsProcLocks *rp_locksp, + Eterm *hp_start, + Eterm *hp, + Uint h_size, + ErlHeapFragment* bp, + Uint32 *ref_num, + Eterm msg) +{ + Eterm ref = make_internal_ref(hp); + write_ref_thing(hp, ref_num[0], ref_num[1], ref_num[2]); + hp += REF_THING_SIZE; + + msg = TUPLE2(hp, ref, msg); + hp += 3; + + if (!bp) { + HRelease(rp, hp_start + h_size, hp); + } + else { + Uint used_h_size = hp - hp_start; + ASSERT(h_size >= used_h_size); + if (h_size > used_h_size) + bp = erts_resize_message_buffer(bp, used_h_size, &msg, 1); + } + + erts_queue_message(rp, + rp_locksp, + bp, + msg, + NIL +#ifdef USE_VM_PROBES + , NIL +#endif + ); +} + +static void +port_sched_op_reply(Eterm to, Uint32 *ref_num, Eterm msg) +{ + Process *rp = erts_proc_lookup_raw(to); + if (rp) { + ErlOffHeap *ohp; + ErlHeapFragment* bp; + Eterm msg_copy; + Uint hsz, msg_sz; + Eterm *hp, *hp_start; + ErtsProcLocks rp_locks = 0; + + hsz = ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE; + if (is_immed(msg)) + msg_sz = 0; + else { + msg_sz = size_object(msg); + hsz += msg_sz; + } + + hp_start = hp = erts_alloc_message_heap(hsz, + &bp, + &ohp, + rp, + &rp_locks); + if (is_immed(msg)) + msg_copy = msg; + else + msg_copy = copy_struct(msg, msg_sz, &hp, ohp); + + queue_port_sched_op_reply(rp, + &rp_locks, + hp_start, + hp, + hsz, + bp, + ref_num, + msg_copy); + + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } +} + + +ErtsPortOpResult +erts_schedule_proc2port_signal(Process *c_p, + Port *prt, + Eterm caller, + Eterm *refp, + ErtsProc2PortSigData *sigdp, + int task_flags, + ErtsProc2PortSigCallback callback) +{ + int sched_res; + if (!refp) { + if (c_p) + erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_MAIN); + } + else { + ASSERT(c_p); + sigdp->flags |= ERTS_P2P_SIG_DATA_FLG_REPLY; + erts_make_ref_in_array(sigdp->ref); + *refp = erts_proc_store_ref(c_p, sigdp->ref); + + /* + * Caller needs to wait for a message containing + * the ref that we just created. No such message + * can exist in callers message queue at this time. + * We therefore move the save pointer of the + * callers message queue to the end of the queue. + * + * NOTE: It is of vital importance that the caller + * immediately do a receive unconditionaly + * waiting for the message with the reference; + * otherwise, next receive will *not* work + * as expected! + */ + erts_smp_proc_lock(c_p, ERTS_PROC_LOCKS_MSG_RECEIVE); + + if (ERTS_PROC_PENDING_EXIT(c_p)) { + /* need to exit caller instead */ + erts_smp_proc_unlock(c_p, ERTS_PROC_LOCKS_MSG_RECEIVE); + KILL_CATCHES(c_p); + c_p->freason = EXC_EXIT; + return ERTS_PORT_OP_CALLER_EXIT; + } + + ERTS_SMP_MSGQ_MV_INQ2PRIVQ(c_p); + c_p->msg.save = c_p->msg.last; + + erts_smp_proc_unlock(c_p, + (ERTS_PROC_LOCK_MAIN + | ERTS_PROC_LOCKS_MSG_RECEIVE)); + } + + + sigdp->caller = caller; + + /* Schedule port close call for later execution... */ + sched_res = erts_port_task_schedule(prt->common.id, + NULL, + ERTS_PORT_TASK_PROC_SIG, + sigdp, + callback, + task_flags); + + if (c_p) + erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_MAIN); + + if (sched_res != 0) { + if (refp) + *refp = NIL; + return ERTS_PORT_OP_DROPPED; + } + return ERTS_PORT_OP_SCHEDULED; +} + +static ERTS_INLINE void +send_badsig(Port *prt) +{ + ErtsProcLocks rp_locks = ERTS_PROC_LOCKS_XSIG_SEND; + Process* rp; + Eterm connected = ERTS_PORT_GET_CONNECTED(prt); + + ERTS_SMP_CHK_NO_PROC_LOCKS; + ERTS_LC_ASSERT(erts_get_scheduler_id()); + + ASSERT(is_internal_pid(connected)); + + rp = erts_proc_lookup_raw(connected); + if (rp) { + erts_smp_proc_lock(rp, rp_locks); + if (!ERTS_PROC_IS_EXITING(rp)) + (void) erts_send_exit_signal(NULL, + prt->common.id, + rp, + &rp_locks, + am_badsig, + NIL, + NULL, + 0); + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } +} + +static void +badsig_received(int bang_op, + Port *prt, + erts_aint32_t state, + int bad_output_value) +{ + /* + * if (bang_op) + * we are part of a "Prt ! Something" operation + * else + * we are part of a call to a port BIF + * behave accordingly... + */ + if (!(state & ERTS_PORT_SFLGS_INVALID_LOOKUP)) { + if (bad_output_value) { + erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); + erts_dsprintf(dsbufp, "Bad value on output port '%s'\n", prt->name); + erts_send_error_to_logger_nogl(dsbufp); + } + if (bang_op) + send_badsig(prt); + } +} + +static int +port_badsig(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + if (op == ERTS_PROC2PORT_SIG_EXEC) + badsig_received(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BANG_OP, + prt, + state, + sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BAD_OUTPUT); + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, am_badarg); + return ERTS_PORT_REDS_BADSIG; +} + + +/* + * bad_port_signal() will + * - preserve signal order of signals. + * - send a 'badsig' exit signal to connected process if 'from' is an + * internal pid and the port is alive when the bad signal reaches + * it. + */ +static ErtsPortOpResult +bad_port_signal(Process *c_p, + int flags, + Port *prt, + Eterm from, + Eterm *refp, + Eterm port_op) +{ + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + !refp, + port_op); + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: + badsig_received(flags & ERTS_PORT_SIG_FLG_BANG_OP, + prt, + try_call_state.state, + flags & ERTS_PORT_SIG_FLG_BAD_OUTPUT); + finalize_imm_drv_call(&try_call_state); + return ERTS_PORT_OP_BADARG; + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule badsig() call instead... */ + break; + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = (flags & ~ERTS_P2P_SIG_TYPE_MASK) | ERTS_P2P_SIG_TYPE_BAD; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + refp, + sigdp, + 0, + port_badsig); +} + + +/* + * Driver outputv() callback + */ + +static ERTS_INLINE void +call_driver_outputv(int bang_op, + Eterm caller, + Eterm from, + Port *prt, + erts_driver_t *drv, + ErlIOVec *evp) +{ + /* + * if (bang_op) + * we are part of a "Prt ! {From, {command, Data}}" operation + * else + * we are part of a call to port_command/[2,3] + * behave accordingly... + */ + if (bang_op && from != ERTS_PORT_GET_CONNECTED(prt)) + send_badsig(prt); + else { + ErlDrvSizeT size = evp->size; + + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt) + || ERTS_IS_CRASH_DUMPING); + +#ifdef USE_VM_PROBES + if (DTRACE_ENABLED(driver_outputv)) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(caller, prt); + DTRACE4(driver_outputv, process_str, port_str, prt->name, size); + } +#endif + + prt->caller = caller; + (*drv->outputv)((ErlDrvData) prt->drv_data, evp); + prt->caller = NIL; + + prt->bytes_out += size; + erts_smp_atomic_add_nob(&erts_bytes_out, size); + } +} + +static ERTS_INLINE void +cleanup_scheduled_outputv(ErlIOVec *ev, ErlDrvBinary *cbinp) +{ + int i; + /* Need to free all binaries */ + for (i = 1; i < ev->vsize; i++) + if (ev->binv[i]) + driver_free_binary(ev->binv[i]); + if (cbinp) + driver_free_binary(cbinp); + erts_free(ERTS_ALC_T_DRV_CMD_DATA, ev); +} + +static int +port_sig_outputv(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + Eterm reply; + + switch (op) { + case ERTS_PROC2PORT_SIG_EXEC: + /* Execution of a scheduled outputv() call */ + + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + + if (state & ERTS_PORT_SFLGS_INVALID_LOOKUP) + reply = am_badarg; + else { + call_driver_outputv(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BANG_OP, + sigdp->caller, + sigdp->u.outputv.from, + prt, + prt->drv_ptr, + sigdp->u.outputv.evp); + reply = am_true; + } + break; + case ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND: + reply = am_false; + break; + default: + reply = am_badarg; + break; + } + + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, reply); + + cleanup_scheduled_outputv(sigdp->u.outputv.evp, + sigdp->u.outputv.cbinp); + + return ERTS_PORT_REDS_CMD_OUTPUTV; +} + +/* + * Driver output() callback + */ + +static ERTS_INLINE void +call_driver_output(int bang_op, + Eterm caller, + Eterm from, + Port *prt, + erts_driver_t *drv, + char *bufp, + ErlDrvSizeT size) +{ + /* + * if (bang_op) + * we are part of a "Prt ! {From, {command, Data}}" operation + * else + * we are part of a call to port_command/[2,3] + * behave accordingly... + */ + if (bang_op && from != ERTS_PORT_GET_CONNECTED(prt)) + send_badsig(prt); + else { + + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt) + || ERTS_IS_CRASH_DUMPING); + +#ifdef USE_VM_PROBES + if (DTRACE_ENABLED(driver_output)) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(caller, prt); + DTRACE4(driver_output, process_str, port_str, prt->name, size); + } +#endif + + prt->caller = caller; + (*drv->output)((ErlDrvData) prt->drv_data, bufp, size); + prt->caller = NIL; + + prt->bytes_out += size; + erts_smp_atomic_add_nob(&erts_bytes_out, size); + } +} + +static ERTS_INLINE void +cleanup_scheduled_output(char *bufp) +{ + erts_free(ERTS_ALC_T_DRV_CMD_DATA, bufp); +} + +static int +port_sig_output(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + Eterm reply; + + switch (op) { + case ERTS_PROC2PORT_SIG_EXEC: + /* Execution of a scheduled output() call */ + + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + + if (state & ERTS_PORT_SFLGS_INVALID_LOOKUP) + reply = am_badarg; + else { + call_driver_output(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BANG_OP, + sigdp->caller, + sigdp->u.output.from, + prt, + prt->drv_ptr, + sigdp->u.output.bufp, + sigdp->u.output.size); + reply = am_true; + } + break; + case ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND: + reply = am_false; + break; + default: + reply = am_badarg; + break; + } + + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, reply); + + cleanup_scheduled_output(sigdp->u.output.bufp); + + return ERTS_PORT_REDS_CMD_OUTPUT; +} + +ErtsPortOpResult +erts_port_output(Process *c_p, + int flags, + Port *prt, + Eterm from, + Eterm list, + Eterm *refp) +{ + ErtsPortOpResult res; + ErtsProc2PortSigData *sigdp; + erts_driver_t *drv = prt->drv_ptr; + size_t size; + int try_call; + erts_aint32_t sched_flags, busy_flgs, invalid_flags; + int task_flags; + ErtsProc2PortSigCallback port_sig_callback; + ErlDrvBinary *cbin = NULL; + ErlIOVec *evp = NULL; + char *buf = NULL; + int force_immediate_call = (flags & ERTS_PORT_SIG_FLG_FORCE_IMM_CALL); + + ASSERT((flags & ~(ERTS_PORT_SIG_FLG_BANG_OP + | ERTS_PORT_SIG_FLG_NOSUSPEND + | ERTS_PORT_SIG_FLG_FORCE + | ERTS_PORT_SIG_FLG_FORCE_IMM_CALL)) == 0); + + busy_flgs = ((flags & ERTS_PORT_SIG_FLG_FORCE) + ? ((erts_aint32_t) 0) + : ERTS_PTS_FLGS_BUSY); + invalid_flags = busy_flgs; + if (!refp) + invalid_flags |= ERTS_PTS_FLG_PARALLELISM; + + /* + * Assumes caller have checked that port is valid... + */ + + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (sched_flags & (busy_flgs|ERTS_PTS_FLG_EXIT)) + return ((sched_flags & ERTS_PTS_FLG_EXIT) + ? ERTS_PORT_OP_DROPPED + : ERTS_PORT_OP_BUSY); + + try_call = (force_immediate_call /* crash dumping */ + || !(sched_flags & (invalid_flags + | ERTS_PTS_FLGS_FORCE_SCHEDULE_OP))); + +#ifdef USE_VM_PROBES + if(DTRACE_ENABLED(port_command)) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(c_p ? c_p->common.id : ERTS_INVALID_PID, prt); + DTRACE4(port_command, process_str, port_str, prt->name, "command"); + } +#endif + + if (drv->outputv) { + ErlIOVec ev; SysIOVec iv[SMALL_WRITE_VEC]; ErlDrvBinary* bv[SMALL_WRITE_VEC]; SysIOVec* ivp; ErlDrvBinary** bvp; - ErlDrvBinary* cbin; - ErlIOVec ev; + int vsize; + Uint csize; + Uint pvsize; + Uint pcsize; + Uint blimit; + size_t iov_offset, binv_offset, alloc_size; - if (io_list_vec_len(list, &vsize, &csize, - &pvsize, &pcsize, &size)) { + if (io_list_vec_len(list, &vsize, &csize, &pvsize, &pcsize, &size)) goto bad_value; + + iov_offset = ERTS_ALC_DATA_ALIGN_SIZE(sizeof(ErlIOVec)); + binv_offset = iov_offset; + binv_offset += ERTS_ALC_DATA_ALIGN_SIZE((vsize+1)*sizeof(SysIOVec)); + alloc_size = binv_offset; + alloc_size += (vsize+1)*sizeof(ErlDrvBinary *); + + if (try_call && vsize < SMALL_WRITE_VEC) { + ivp = ev.iov = iv; + bvp = ev.binv = bv; + evp = &ev; + } + else { + char *ptr = erts_alloc((try_call + ? ERTS_ALC_T_TMP + : ERTS_ALC_T_DRV_CMD_DATA), alloc_size); + + evp = (ErlIOVec *) ptr; + ivp = evp->iov = (SysIOVec *) (ptr + iov_offset); + bvp = evp->binv = (ErlDrvBinary **) (ptr + binv_offset); } + /* To pack or not to pack (small binaries) ...? */ - vsize++; - if (vsize <= SMALL_WRITE_VEC) { + if (vsize < SMALL_WRITE_VEC) { /* Do NOT pack */ blimit = 0; - } else { + } + else { /* Do pack */ vsize = pvsize + 1; csize = pcsize; blimit = ERL_SMALL_IO_BIN_LIMIT; } /* Use vsize and csize from now on */ - if (vsize <= SMALL_WRITE_VEC) { - ivp = iv; - bvp = bv; - } else { - ivp = (SysIOVec *) erts_alloc(ERTS_ALC_T_TMP, - vsize * sizeof(SysIOVec)); - bvp = (ErlDrvBinary**) erts_alloc(ERTS_ALC_T_TMP, - vsize * sizeof(ErlDrvBinary*)); - } + cbin = driver_alloc_binary(csize); if (!cbin) erts_alloc_enomem(ERTS_ALC_T_DRV_BINARY, ERTS_SIZEOF_Binary(csize)); @@ -1178,210 +1890,759 @@ int erts_write_to_port(Eterm caller_id, Port *p, Eterm list) ivp[0].iov_base = NULL; ivp[0].iov_len = 0; bvp[0] = NULL; - ev.vsize = io_list_to_vec(list, ivp+1, bvp+1, cbin, blimit); - if (ev.vsize < 0) { - if (ivp != iv) { - erts_free(ERTS_ALC_T_TMP, (void *) ivp); - } - if (bvp != bv) { - erts_free(ERTS_ALC_T_TMP, (void *) bvp); - } + evp->vsize = io_list_to_vec(list, ivp+1, bvp+1, cbin, blimit); + if (evp->vsize < 0) { + if (evp != &ev) + erts_free(try_call ? ERTS_ALC_T_TMP : ERTS_ALC_T_DRV_CMD_DATA, + evp); driver_free_binary(cbin); goto bad_value; } - ev.vsize++; #if 0 /* This assertion may say something useful, but it can be falsified during the emulator test suites. */ - ASSERT(ev.vsize == vsize); + ASSERT(evp->vsize == vsize); #endif - ev.size = size; /* total size */ - ev.iov = ivp; - ev.binv = bvp; -#ifdef USE_VM_PROBES - if (DTRACE_ENABLED(driver_outputv)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(caller_id, p) - DTRACE4(driver_outputv, process_str, port_str, p->name, size); - } -#endif - fpe_was_unmasked = erts_block_fpe(); - (*drv->outputv)((ErlDrvData)p->drv_data, &ev); - erts_unblock_fpe(fpe_was_unmasked); - if (ivp != iv) { - erts_free(ERTS_ALC_T_TMP, (void *) ivp); - } - if (bvp != bv) { - erts_free(ERTS_ALC_T_TMP, (void *) bvp); + evp->vsize++; + evp->size = size; /* total size */ + + if (!try_call) { + int i; + /* Need to increase refc on all binaries */ + for (i = 1; i < evp->vsize; i++) + if (bvp[i]) + driver_binary_inc_refc(bvp[i]); } - driver_free_binary(cbin); - } else { - int r; - - /* Try with an 8KB buffer first (will often be enough I guess). */ - size = 8*1024; - /* See below why the extra byte is added. */ - buf = erts_alloc(ERTS_ALC_T_TMP, size+1); - r = io_list_to_buf(list, buf, size); + else { + int i; + ErlIOVec *new_evp; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + invalid_flags, + !refp, + am_command); + + try_call_state.pre_chk_sched_flags = 0; /* already checked */ + if (force_immediate_call) + try_call_res = force_imm_drv_call(&try_call_state); + else + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: + call_driver_outputv(flags & ERTS_PORT_SIG_FLG_BANG_OP, + c_p ? c_p->common.id : ERTS_INVALID_PID, + from, + prt, + drv, + evp); + if (force_immediate_call) + finalize_force_imm_drv_call(&try_call_state); + else + finalize_imm_drv_call(&try_call_state); + /* Fall through... */ + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + driver_free_binary(cbin); + if (evp != &ev) + erts_free(ERTS_ALC_T_TMP, evp); + if (try_call_res == ERTS_TRY_IMM_DRV_CALL_OK) + return ERTS_PORT_OP_DONE; + else + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + sched_flags = try_call_state.sched_flags; + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule outputv() call instead... */ + break; + } -#ifdef USE_VM_PROBES - if(DTRACE_ENABLED(port_command)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(caller_id, p) - DTRACE4(port_command, process_str, port_str, p->name, "command"); - } + /* Need to increase refc on all binaries */ + for (i = 1; i < evp->vsize; i++) + if (bvp[i]) + driver_binary_inc_refc(bvp[i]); + + new_evp = erts_alloc(ERTS_ALC_T_DRV_CMD_DATA, alloc_size); + + if (evp != &ev) { + sys_memcpy((void *) new_evp, (void *) evp, alloc_size); + new_evp->iov = (SysIOVec *) (((char *) new_evp) + + iov_offset); + bvp = new_evp->binv = (ErlDrvBinary **) (((char *) new_evp) + + binv_offset); + +#ifdef DEBUG + ASSERT(new_evp->vsize == evp->vsize); + ASSERT(new_evp->size == evp->size); + for (i = 0; i < evp->vsize; i++) { + ASSERT(new_evp->iov[i].iov_len == evp->iov[i].iov_len); + ASSERT(new_evp->iov[i].iov_base == evp->iov[i].iov_base); + ASSERT(new_evp->binv[i] == evp->binv[i]); + } #endif - if (r >= 0) { - size -= r; -#ifdef USE_VM_PROBES - if (DTRACE_ENABLED(driver_output)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(caller_id, p) - DTRACE4(driver_output, process_str, port_str, p->name, size); - } + erts_free(ERTS_ALC_T_TMP, evp); + } + else { /* from stack allocated structure; offsets may differ */ + + sys_memcpy((void *) new_evp, (void *) evp, sizeof(ErlIOVec)); + new_evp->iov = (SysIOVec *) (((char *) new_evp) + + iov_offset); + sys_memcpy((void *) new_evp->iov, + (void *) evp->iov, + evp->vsize * sizeof(SysIOVec)); + new_evp->binv = (ErlDrvBinary **) (((char *) new_evp) + + binv_offset); + sys_memcpy((void *) new_evp->binv, + (void *) evp->binv, + evp->vsize * sizeof(ErlDrvBinary *)); + +#ifdef DEBUG + ASSERT(new_evp->vsize == evp->vsize); + ASSERT(new_evp->size == evp->size); + for (i = 0; i < evp->vsize; i++) { + ASSERT(new_evp->iov[i].iov_len == evp->iov[i].iov_len); + ASSERT(new_evp->iov[i].iov_base == evp->iov[i].iov_base); + ASSERT(new_evp->binv[i] == evp->binv[i]); + } #endif - fpe_was_unmasked = erts_block_fpe(); - (*drv->output)((ErlDrvData)p->drv_data, buf, size); - erts_unblock_fpe(fpe_was_unmasked); - erts_free(ERTS_ALC_T_TMP, buf); + + } + + evp = new_evp; } - else if (r == -2) { - erts_free(ERTS_ALC_T_TMP, buf); - goto bad_value; + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_OUTPUTV; + sigdp->u.outputv.from = from; + sigdp->u.outputv.evp = evp; + sigdp->u.outputv.cbinp = cbin; + port_sig_callback = port_sig_outputv; + } + else { + ErlDrvSizeT r; + + /* + * Apperently there exist code that write 1 byte to + * much in buffer. Where it resides I don't know, but + * we can live with one byte extra allocated... + */ + + if (!try_call) { + if (erts_iolist_size(list, &size)) + goto bad_value; + + buf = erts_alloc(ERTS_ALC_T_DRV_CMD_DATA, size + 1); + + r = erts_iolist_to_buf(list, buf, size); + ASSERT(ERTS_IOLIST_TO_BUF_SUCCEEDED(r)); } else { - ASSERT(r == -1); /* Overflow */ - erts_free(ERTS_ALC_T_TMP, buf); - if (erts_iolist_size(list, &size)) { - goto bad_value; + char *new_buf; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + invalid_flags, + !refp, + am_command); + + /* Try with an 8KB buffer first (will often be enough I guess). */ + size = 8*1024; + + buf = erts_alloc(ERTS_ALC_T_TMP, size + 1); + r = erts_iolist_to_buf(list, buf, size); + + if (ERTS_IOLIST_TO_BUF_SUCCEEDED(r)) { + ASSERT(r <= size); + size -= r; + } + else { + erts_free(ERTS_ALC_T_TMP, buf); + if (r == ERTS_IOLIST_TO_BUF_TYPE_ERROR) + goto bad_value; + ASSERT(r == ERTS_IOLIST_TO_BUF_OVERFLOW); + if (erts_iolist_size(list, &size)) + goto bad_value; + buf = erts_alloc(ERTS_ALC_T_TMP, size + 1); + r = erts_iolist_to_buf(list, buf, size); + ASSERT(ERTS_IOLIST_TO_BUF_SUCCEEDED(r)); } - /* - * I know drivers that pad space with '\0' this is clearly - * incorrect but I don't feel like fixing them now, insted - * add ONE extra byte. - */ - buf = erts_alloc(ERTS_ALC_T_TMP, size+1); - r = io_list_to_buf(list, buf, size); -#ifdef USE_VM_PROBES - if (DTRACE_ENABLED(driver_output)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(caller_id, p) - DTRACE4(driver_output, process_str, port_str, p->name, size); - } -#endif - fpe_was_unmasked = erts_block_fpe(); - (*drv->output)((ErlDrvData)p->drv_data, buf, size); - erts_unblock_fpe(fpe_was_unmasked); + try_call_state.pre_chk_sched_flags = 0; /* already checked */ + if (force_immediate_call) + try_call_res = force_imm_drv_call(&try_call_state); + else + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: + call_driver_output(flags & ERTS_PORT_SIG_FLG_BANG_OP, + c_p ? c_p->common.id : ERTS_INVALID_PID, + from, + prt, + drv, + buf, + size); + if (force_immediate_call) + finalize_force_imm_drv_call(&try_call_state); + else + finalize_imm_drv_call(&try_call_state); + /* Fall through... */ + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + erts_free(ERTS_ALC_T_TMP, buf); + if (try_call_res == ERTS_TRY_IMM_DRV_CALL_OK) + return ERTS_PORT_OP_DONE; + else + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + sched_flags = try_call_state.sched_flags; + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule outputv() call instead... */ + break; + } + + new_buf = erts_alloc(ERTS_ALC_T_DRV_CMD_DATA, size + 1); + sys_memcpy(new_buf, buf, size); erts_free(ERTS_ALC_T_TMP, buf); + buf = new_buf; } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_OUTPUT; + sigdp->u.output.from = from; + sigdp->u.output.bufp = buf; + sigdp->u.output.size = size; + port_sig_callback = port_sig_output; + } + + task_flags = ERTS_PT_FLG_WAIT_BUSY; + sigdp->flags |= flags; + if (flags & (ERTS_P2P_SIG_DATA_FLG_FORCE|ERTS_P2P_SIG_DATA_FLG_NOSUSPEND)) { + task_flags = 0; + if (flags & ERTS_P2P_SIG_DATA_FLG_FORCE) + sigdp->flags &= ~ERTS_P2P_SIG_DATA_FLG_NOSUSPEND; + else if (flags & ERTS_P2P_SIG_DATA_FLG_NOSUSPEND) + task_flags = ERTS_PT_FLG_NOSUSPEND; + } + + res = erts_schedule_proc2port_signal(c_p, + prt, + c_p ? c_p->common.id : ERTS_INVALID_PID, + refp, + sigdp, + task_flags, + port_sig_callback); + + if (res != ERTS_PORT_OP_SCHEDULED) { + if (drv->outputv) + cleanup_scheduled_outputv(evp, cbin); + else + cleanup_scheduled_output(buf); + return res; } - p->bytes_out += size; - erts_smp_atomic_add_nob(&erts_bytes_out, size); -#ifdef ERTS_SMP - if (p->xports) - erts_smp_xports_unlock(p); - ASSERT(!p->xports); + if (!(sched_flags & ERTS_PTS_FLG_EXIT) && (sched_flags & busy_flgs)) + return ERTS_PORT_OP_BUSY_SCHEDULED; + + return res; + +bad_value: + + flags |= ERTS_PORT_SIG_FLG_BAD_OUTPUT; + return bad_port_signal(c_p, flags, prt, from, refp, am_command); +} + +static ERTS_INLINE ErtsPortOpResult +call_deliver_port_exit(int bang_op, + Eterm from, + Port *prt, + erts_aint32_t state, + Eterm reason, + int broken_link) +{ + /* + * if (bang_op) + * we are part of a "Prt ! {From, close}" operation + * else + * we are part of a call to port_close(Port) + * behave accordingly... + */ + + if (state & ERTS_PORT_SFLGS_INVALID_LOOKUP) + return ERTS_PORT_OP_DROPPED; + + if (bang_op && from != ERTS_PORT_GET_CONNECTED(prt)) { + send_badsig(prt); + return ERTS_PORT_OP_DROPPED; + } + + if (broken_link) { + ErtsLink *lnk = erts_remove_link(&ERTS_P_LINKS(prt), from); + if (lnk) + erts_destroy_link(lnk); + else + return ERTS_PORT_OP_DROPPED; + } + + if (!erts_deliver_port_exit(prt, from, reason, bang_op)) + return ERTS_PORT_OP_DROPPED; + +#ifdef USE_VM_PROBES + if(DTRACE_ENABLED(port_command) && bang_op) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(from, prt); + DTRACE4(port_command, process_str, port_str, prt->name, "close"); + } #endif - p->caller = NIL; - return 0; - bad_value: - p->caller = NIL; - { - erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); - erts_dsprintf(dsbufp, "Bad value on output port '%s'\n", p->name); - erts_send_error_to_logger_nogl(dsbufp); - return 1; + return ERTS_PORT_OP_DONE; +} + +static int +port_sig_exit(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + Eterm msg = am_badarg; + if (op == ERTS_PROC2PORT_SIG_EXEC) { + ErtsPortOpResult res; + int bang_op = sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BANG_OP; + int broken_link = sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BROKEN_LINK; + res = call_deliver_port_exit(bang_op, + sigdp->u.exit.from, + prt, + state, + sigdp->u.exit.reason, + broken_link); + + if (res == ERTS_PORT_OP_DONE) + msg = am_true; + } + if (sigdp->u.exit.bp) + free_message_buffer(sigdp->u.exit.bp); + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, msg); + + return ERTS_PORT_REDS_EXIT; +} + +ErtsPortOpResult +erts_port_exit(Process *c_p, + int flags, + Port *prt, + Eterm from, + Eterm reason, + Eterm *refp) +{ + ErtsPortOpResult res; + ErtsProc2PortSigData *sigdp; + ErlHeapFragment *bp = NULL; + + ASSERT((flags & ~(ERTS_PORT_SIG_FLG_BANG_OP + | ERTS_PORT_SIG_FLG_BROKEN_LINK + | ERTS_PORT_SIG_FLG_FORCE_SCHED)) == 0); + + if (!(flags & ERTS_PORT_SIG_FLG_FORCE_SCHED)) { + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE(c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + !refp, + am_exit); + + + switch (try_imm_drv_call(&try_call_state)) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + res = call_deliver_port_exit(flags & ERTS_PORT_SIG_FLG_BANG_OP, + from, + prt, + try_call_state.state, + reason, + flags & ERTS_PORT_SIG_FLG_BROKEN_LINK); + finalize_imm_drv_call(&try_call_state); + return res; + } + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + default: + /* Schedule call instead... */ + break; + } } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_EXIT | flags; + sigdp->u.exit.from = from; + + if (is_immed(reason)) { + sigdp->u.exit.reason = reason; + sigdp->u.exit.bp = NULL; + } + else { + Eterm *hp; + Uint hsz = size_object(reason); + bp = new_message_buffer(hsz); + sigdp->u.exit.bp = bp; + hp = bp->mem; + sigdp->u.exit.reason = copy_struct(reason, + hsz, + &hp, + &bp->off_heap); + } + + res = erts_schedule_proc2port_signal(c_p, + prt, + c_p ? c_p->common.id : from, + refp, + sigdp, + 0, + port_sig_exit); + + if (res == ERTS_PORT_OP_DROPPED) { + if (bp) + free_message_buffer(bp); + } + + return res; } -/* initialize the port array */ -void init_io(void) +static ErtsPortOpResult +set_port_connected(int bang_op, + Eterm from, + Port *prt, + erts_aint32_t state, + Eterm connect) { - int i; - ErlDrvEntry** dp; - char maxports[21]; /* enough for any 64-bit integer */ - size_t maxportssize = sizeof(maxports); - Uint ports_bits = ERTS_PORTS_BITS; - Sint port_extra_shift; + /* + * if (bang_op) + * we are part of a "Prt ! {From, {connect, Connect}}" operation + * else + * we are part of a call to port_connect(Port, Connect) + * behave accordingly... + */ -#ifdef ERTS_SMP - init_xports_list_alloc(); + if (state & ERTS_PORT_SFLGS_INVALID_LOOKUP) + return ERTS_PORT_OP_DROPPED; + + if (bang_op) { /* Bang operation */ + if (is_not_internal_pid(connect) || ERTS_PORT_GET_CONNECTED(prt) != from) { + send_badsig(prt); + return ERTS_PORT_OP_DROPPED; + } + + ERTS_PORT_SET_CONNECTED(prt, connect); + deliver_result(prt->common.id, from, am_connected); + +#ifdef USE_VM_PROBES + if(DTRACE_ENABLED(port_command)) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(from, prt); + DTRACE4(port_command, process_str, port_str, prt->name, "connect"); + } #endif + } + else { /* Port BIF operation */ + Process *rp = erts_proc_lookup_raw(connect); + if (!rp) + return ERTS_PORT_OP_DROPPED; + erts_smp_proc_lock(rp, ERTS_PROC_LOCK_LINK); + if (ERTS_PROC_IS_EXITING(rp)) { + erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); + return ERTS_PORT_OP_DROPPED; + } - pdl_init(); + erts_add_link(&ERTS_P_LINKS(rp), LINK_PID, prt->common.id); + erts_add_link(&ERTS_P_LINKS(prt), LINK_PID, connect); + + ERTS_PORT_SET_CONNECTED(prt, connect); + + erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); + +#ifdef USE_VM_PROBES + if (DTRACE_ENABLED(port_connect)) { + DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); + DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); + DTRACE_CHARBUF(newprocess_str, DTRACE_TERM_BUF_SIZE); + + dtrace_pid_str(connect, process_str); + erts_snprintf(port_str, sizeof(port_str), "%T", prt->common.id); + dtrace_proc_str(rp, newprocess_str); + DTRACE4(port_connect, process_str, port_str, prt->name, newprocess_str); + } +#endif + } + + return ERTS_PORT_OP_DONE; +} - if (erts_sys_getenv_raw("ERL_MAX_PORTS", maxports, &maxportssize) == 0) - erts_max_ports = atoi(maxports); +static int +port_sig_connect(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + Eterm msg = am_badarg; + if (op == ERTS_PROC2PORT_SIG_EXEC) { + ErtsPortOpResult res; + res = set_port_connected(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BANG_OP, + sigdp->u.connect.from, + prt, + state, + sigdp->u.connect.connected); + if (res == ERTS_PORT_OP_DONE) + msg = am_true; + } + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, msg); + return ERTS_PORT_REDS_CONNECT; +} + +ErtsPortOpResult +erts_port_connect(Process *c_p, + int flags, + Port *prt, + Eterm from, + Eterm connect, + Eterm *refp) +{ + ErtsProc2PortSigData *sigdp; + Eterm connect_id; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE(c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + !refp, + am_connect); + + ASSERT((flags & ~ERTS_PORT_SIG_FLG_BANG_OP) == 0); + + if (is_not_internal_pid(connect)) + connect_id = NIL; /* Fail in op (for signal order) */ else - erts_max_ports = sys_max_files(); + connect_id = connect; + + switch (try_imm_drv_call(&try_call_state)) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + ErtsPortOpResult res; + res = set_port_connected(flags & ERTS_PORT_SIG_FLG_BANG_OP, + from, + prt, + try_call_state.state, + connect_id); + finalize_imm_drv_call(&try_call_state); + return res; + } + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + default: + /* Schedule call instead... */ + break; + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_CONNECT | flags; + + sigdp->u.connect.from = from; + sigdp->u.connect.connected = connect_id; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + refp, + sigdp, + 0, + port_sig_connect); +} - if (erts_max_ports > ERTS_MAX_PORTS) - erts_max_ports = ERTS_MAX_PORTS; - if (erts_max_ports < 1024) - erts_max_ports = 1024; +static void +port_unlink(Port *prt, Eterm from) +{ + ErtsLink *lnk = erts_remove_link(&ERTS_P_LINKS(prt), from); + if (lnk) + erts_destroy_link(lnk); +} - if (erts_use_r9_pids_ports) { - ports_bits = ERTS_R9_PORTS_BITS; - if (erts_max_ports > ERTS_MAX_R9_PORTS) - erts_max_ports = ERTS_MAX_R9_PORTS; +static int +port_sig_unlink(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + if (op == ERTS_PROC2PORT_SIG_EXEC) + port_unlink(prt, sigdp->u.unlink.from); + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, am_true); + return ERTS_PORT_REDS_UNLINK; +} + +ErtsPortOpResult +erts_port_unlink(Process *c_p, Port *prt, Eterm from, Eterm *refp) +{ + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE(c_p, + prt, + ERTS_PORT_SFLGS_DEAD, + 0, + !refp, + am_unlink); + + switch (try_imm_drv_call(&try_call_state)) { + case ERTS_TRY_IMM_DRV_CALL_OK: + port_unlink(prt, from); + finalize_imm_drv_call(&try_call_state); + return ERTS_PORT_OP_DONE; + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + default: + /* Schedule call instead... */ + break; } - port_extra_shift = erts_fit_in_bits_int32(erts_max_ports - 1); - port_num_mask = (1 << ports_bits) - 1; + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_UNLINK; + sigdp->u.unlink.from = from; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p ? c_p->common.id : from, + refp, + sigdp, + 0, + port_sig_unlink); +} - erts_port_tab_index_mask = ~(~((Uint) 0) << port_extra_shift); - erts_max_ports = 1 << port_extra_shift; +static void +port_link_failure(Eterm port_id, Eterm linker) +{ + Process *rp; + ErtsProcLocks rp_locks = ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCKS_XSIG_SEND; + ASSERT(is_internal_pid(linker)); + rp = erts_pid2proc(NULL, 0, linker, rp_locks); + if (rp) { + ErtsLink *rlnk = erts_remove_link(&ERTS_P_LINKS(rp), port_id); + if (rlnk) { + int xres = erts_send_exit_signal(NULL, + port_id, + rp, + &rp_locks, + am_noproc, + NIL, + NULL, + 0); + if (xres >= 0 && IS_TRACED_FL(rp, F_TRACE_PROCS)) { + /* We didn't exit the process and it is traced */ + if (IS_TRACED_FL(rp, F_TRACE_PROCS)) + trace_proc(NULL, rp, am_getting_unlinked, port_id); + } + } + } +} - erts_smp_mtx_init(&erts_driver_list_lock,"driver_list"); - driver_list = NULL; - erts_smp_tsd_key_create(&driver_list_lock_status_key); - erts_smp_tsd_key_create(&driver_list_last_error_key); +static void +port_link(Port *prt, erts_aint32_t state, Eterm to) +{ + if (!(state & ERTS_PORT_SFLGS_INVALID_LOOKUP)) + erts_add_link(&ERTS_P_LINKS(prt), LINK_PID, to); + else + port_link_failure(prt->common.id, to); +} - if (erts_max_ports * sizeof(Port) <= erts_max_ports) { - /* More memory needed than the whole address space. */ - erts_alloc_enomem(ERTS_ALC_T_PORT_TABLE, ~((Uint) 0)); +static int +port_sig_link(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + if (op == ERTS_PROC2PORT_SIG_EXEC) + port_link(prt, state, sigdp->u.link.to); + else + port_link_failure(sigdp->u.link.port, sigdp->u.link.to); + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, am_true); + return ERTS_PORT_REDS_LINK; +} + +ErtsPortOpResult +erts_port_link(Process *c_p, Port *prt, Eterm to, Eterm *refp) +{ + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE(c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + !refp, + am_link); + + switch (try_imm_drv_call(&try_call_state)) { + case ERTS_TRY_IMM_DRV_CALL_OK: + port_link(prt, try_call_state.state, to); + finalize_imm_drv_call(&try_call_state); + return ERTS_PORT_OP_DONE; + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_BADARG; + default: + /* Schedule call instead... */ + break; } - erts_port = (Port *) erts_alloc(ERTS_ALC_T_PORT_TABLE, - erts_max_ports * sizeof(Port)); + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_LINK; + sigdp->u.link.port = prt->common.id; + sigdp->u.link.to = to; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p ? c_p->common.id : to, + refp, + sigdp, + 0, + port_sig_link); +} - erts_smp_atomic_init_nob(&erts_bytes_out, 0); - erts_smp_atomic_init_nob(&erts_bytes_in, 0); +void erts_init_io(int port_tab_size, + int port_tab_size_ignore_files) +{ + ErlDrvEntry** dp; + erts_smp_rwmtx_opt_t drv_list_rwmtx_opts = ERTS_SMP_RWMTX_OPT_DEFAULT_INITER; + drv_list_rwmtx_opts.type = ERTS_SMP_RWMTX_TYPE_EXTREMELY_FREQUENT_READ; + drv_list_rwmtx_opts.lived = ERTS_SMP_RWMTX_LONG_LIVED; - for (i = 0; i < erts_max_ports; i++) { - erts_port_task_init_sched(&erts_port[i].sched); - erts_smp_atomic_init_nob(&erts_port[i].refc, 0); #ifdef ERTS_SMP - erts_port[i].lock = NULL; - erts_port[i].xports = NULL; - erts_smp_spinlock_init_x(&erts_port[i].state_lck, -#ifdef ERTS_ENABLE_LOCK_COUNT - (erts_lcnt_rt_options & ERTS_LCNT_OPT_PORTLOCK) ? "port_state" : NULL, -#else - "port_state", -#endif - make_small(0)); + init_xports_list_alloc(); #endif - erts_port[i].tracer_proc = NIL; - erts_port[i].trace_flags = 0; - erts_port[i].drv_ptr = NULL; - erts_port[i].status = ERTS_PORT_SFLG_FREE; - erts_port[i].name = NULL; - erts_port[i].nlinks = NULL; - erts_port[i].monitors = NULL; - erts_port[i].linebuf = NULL; - erts_port[i].port_data_lock = NULL; + pdl_init(); + + if (!port_tab_size_ignore_files) { + int max_files = sys_max_files(); + if (port_tab_size < max_files) + port_tab_size = max_files; } - erts_smp_atomic32_init_nob(&erts_ports_snapshot, (erts_aint32_t) 0); - last_port_num = 0; - erts_smp_spinlock_init(&get_free_port_lck, "get_free_port"); + if (port_tab_size > ERTS_MAX_PORTS) + port_tab_size = ERTS_MAX_PORTS; + else if (port_tab_size < ERTS_MIN_PORTS) + port_tab_size = ERTS_MIN_PORTS; + + erts_smp_rwmtx_init_opt(&erts_driver_list_lock, + &drv_list_rwmtx_opts, + "driver_list"); + driver_list = NULL; + erts_smp_tsd_key_create(&driver_list_lock_status_key); + erts_smp_tsd_key_create(&driver_list_last_error_key); + + erts_ptab_init_table(&erts_port, + ERTS_ALC_T_PORT_TABLE, + NULL, + (ErtsPTabElementCommon *) &erts_invalid_port.common, + port_tab_size, + "port_table"); + + erts_smp_atomic_init_nob(&erts_bytes_out, 0); + erts_smp_atomic_init_nob(&erts_bytes_in, 0); sys_init_io(); erts_smp_tsd_set(driver_list_lock_status_key, (void *) 1); - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rwlock(&erts_driver_list_lock); init_driver(&fd_driver, &fd_driver_entry, NULL); init_driver(&vanilla_driver, &vanilla_driver_entry, NULL); @@ -1390,27 +2651,63 @@ void init_io(void) erts_add_driver_entry(*dp, NULL, 1); erts_smp_tsd_set(driver_list_lock_status_key, NULL); - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } #if defined(ERTS_ENABLE_LOCK_COUNT) && defined(ERTS_SMP) -void erts_lcnt_enable_io_lock_count(int enable) { - int i; - for (i = 0; i < erts_max_ports; i++) { - Port* p = &erts_port[i]; - if (enable) { - erts_lcnt_init_lock_x(&p->state_lck.lcnt, "port_state", ERTS_LCNT_LT_SPINLOCK, make_small(i)); - if (p->lock) { - erts_lcnt_init_lock_x(&p->lock->lcnt, "port_lock", ERTS_LCNT_LT_MUTEX, make_small(i)); - } - } else { - erts_lcnt_destroy_lock(&p->state_lck.lcnt); - if (p->lock) { - erts_lcnt_destroy_lock(&p->lock->lcnt); - } - } +static ERTS_INLINE void lcnt_enable_drv_lock_count(erts_driver_t *dp, int enable) +{ + if (dp->lock) { + if (enable) + erts_lcnt_init_lock_x(&dp->lock->lcnt, + "driver_lock", + ERTS_LCNT_LT_MUTEX, + am_atom_put(dp->name, + sys_strlen(dp->name))); + else + erts_lcnt_destroy_lock(&dp->lock->lcnt); + + } +} + +static ERTS_INLINE void lcnt_enable_port_lock_count(Port *prt, int enable) +{ + erts_aint32_t state = erts_atomic32_read_nob(&prt->state); + if (!enable) { + erts_lcnt_destroy_lock(&prt->sched.mtx.lcnt); + if (state & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK) + erts_lcnt_destroy_lock(&prt->lock->lcnt); } + else { + erts_lcnt_init_lock_x(&prt->sched.mtx.lcnt, + "port_sched_lock", + ERTS_LCNT_LT_MUTEX, + prt->common.id); + if (state & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK) + erts_lcnt_init_lock_x(&prt->lock->lcnt, + "port_lock", + ERTS_LCNT_LT_MUTEX, + prt->common.id); + } +} + +void erts_lcnt_enable_io_lock_count(int enable) +{ + erts_driver_t *dp; + int i, max = erts_ptab_max(&erts_port); + + for (i = 0; i < max; i++) { + Port *prt = erts_pix2port(i); + if (prt) + lcnt_enable_port_lock_count(prt, enable); + } + + lcnt_enable_drv_lock_count(&vanilla_driver, enable); + lcnt_enable_drv_lock_count(&spawn_driver, enable); + lcnt_enable_drv_lock_count(&fd_driver, enable); + for (dp = driver_list; dp; dp = dp->next) + lcnt_enable_drv_lock_count(dp, enable); } #endif @@ -1594,9 +2891,7 @@ deliver_result(Eterm sender, Eterm pid, Eterm res) ERTS_SMP_CHK_NO_PROC_LOCKS; - ASSERT(is_internal_port(sender) - && is_internal_pid(pid) - && internal_pid_index(pid) < erts_max_processes); + ASSERT(is_internal_port(sender) && is_internal_pid(pid)); rp = (scheduler ? erts_proc_lookup(pid) @@ -1608,16 +2903,19 @@ deliver_result(Eterm sender, Eterm pid, Eterm res) ErlOffHeap *ohp; Eterm* hp; Uint sz_res; - sz_res = size_object(res); - hp = erts_alloc_message_heap(sz_res + 3, &bp, &ohp, rp, &rp_locks); - res = copy_struct(res, sz_res, &hp, ohp); - tuple = TUPLE2(hp, sender, res); + + sz_res = size_object(res); + hp = erts_alloc_message_heap(sz_res + 3, &bp, &ohp, rp, &rp_locks); + res = copy_struct(res, sz_res, &hp, ohp); + tuple = TUPLE2(hp, sender, res); erts_queue_message(rp, &rp_locks, bp, tuple, NIL #ifdef USE_VM_PROBES , NIL #endif ); - erts_smp_proc_unlock(rp, rp_locks); + + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); if (!scheduler) erts_smp_proc_dec_refc(rp); @@ -1633,7 +2931,7 @@ deliver_result(Eterm sender, Eterm pid, Eterm res) * len -- length of data */ -static void deliver_read_message(Port* prt, Eterm to, +static void deliver_read_message(Port* prt, erts_aint32_t state, Eterm to, char *hbuf, ErlDrvSizeT hlen, char *buf, ErlDrvSizeT len, int eol) { @@ -1651,10 +2949,11 @@ static void deliver_read_message(Port* prt, Eterm to, ERTS_SMP_CHK_NO_PROC_LOCKS; need = 3 + 3 + 2*hlen; - if (prt->status & ERTS_PORT_SFLG_LINEBUF_IO) { + + if (state & ERTS_PORT_SFLG_LINEBUF_IO) { need += 3; } - if (prt->status & ERTS_PORT_SFLG_BINARY_IO && buf != NULL) { + if ((state & ERTS_PORT_SFLG_BINARY_IO) && buf != NULL) { need += PROC_BIN_SIZE; } else { need += 2*len; @@ -1670,7 +2969,7 @@ static void deliver_read_message(Port* prt, Eterm to, hp = erts_alloc_message_heap(need, &bp, &ohp, rp, &rp_locks); listp = NIL; - if ((prt->status & ERTS_PORT_SFLG_BINARY_IO) == 0) { + if ((state & ERTS_PORT_SFLG_BINARY_IO) == 0) { listp = buf_to_intlist(&hp, buf, len, listp); } else if (buf != NULL) { ProcBin* pb; @@ -1701,14 +3000,14 @@ static void deliver_read_message(Port* prt, Eterm to, listp = buf_to_intlist(&hp, hbuf, hlen, listp); } - if (prt->status & ERTS_PORT_SFLG_LINEBUF_IO){ + if (state & ERTS_PORT_SFLG_LINEBUF_IO){ listp = TUPLE2(hp, (eol) ? am_eol : am_noeol, listp); hp += 3; } tuple = TUPLE2(hp, am_data, listp); hp += 3; - tuple = TUPLE2(hp, prt->id, tuple); + tuple = TUPLE2(hp, prt->common.id, tuple); hp += 3; erts_queue_message(rp, &rp_locks, bp, tuple, am_undefined @@ -1726,7 +3025,8 @@ static void deliver_read_message(Port* prt, Eterm to, * Deliver all lines in a line buffer, repeats calls to * deliver_read_message, and takes the same parameters. */ -static void deliver_linebuf_message(Port* prt, Eterm to, +static void deliver_linebuf_message(Port* prt, erts_aint_t state, + Eterm to, char* hbuf, ErlDrvSizeT hlen, char *buf, ErlDrvSizeT len) { @@ -1735,7 +3035,7 @@ static void deliver_linebuf_message(Port* prt, Eterm to, if(init_linebuf_context(&lc,&(prt->linebuf), buf, len) < 0) return; while((ret = read_linebuf(&lc)) > LINEBUF_EMPTY) - deliver_read_message(prt, to, hbuf, hlen, LINEBUF_DATA(lc), + deliver_read_message(prt, state, to, hbuf, hlen, LINEBUF_DATA(lc), LINEBUF_DATALEN(lc), (ret == LINEBUF_EOL)); } @@ -1746,20 +3046,25 @@ static void deliver_linebuf_message(Port* prt, Eterm to, * Parameters: * prt - Pointer to a Port structure for this port. */ -static void flush_linebuf_messages(Port *prt) +static void flush_linebuf_messages(Port *prt, erts_aint32_t state) { LineBufContext lc; int ret; ERTS_SMP_LC_ASSERT(!prt || erts_lc_is_port_locked(prt)); - if(prt == NULL || !(prt->status & ERTS_PORT_SFLG_LINEBUF_IO)) + + if (!prt) + return; + + if (!(state & ERTS_PORT_SFLG_LINEBUF_IO)) return; if(init_linebuf_context(&lc,&(prt->linebuf), NULL, 0) < 0) return; while((ret = flush_linebuf(&lc)) > LINEBUF_EMPTY) deliver_read_message(prt, - prt->connected, + state, + ERTS_PORT_GET_CONNECTED(prt), NULL, 0, LINEBUF_DATA(lc), @@ -1787,6 +3092,7 @@ deliver_vec_message(Port* prt, /* Port */ ErlOffHeap *ohp; ErtsProcLocks rp_locks = 0; int scheduler = erts_get_scheduler_id() != 0; + erts_aint32_t state; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -1802,12 +3108,13 @@ deliver_vec_message(Port* prt, /* Port */ if (!rp) return; + state = erts_atomic32_read_nob(&prt->state); /* * Calculate the exact number of heap words needed. */ need = 3 + 3; /* Heap space for two tuples */ - if (prt->status & ERTS_PORT_SFLG_BINARY_IO) { + if (state & ERTS_PORT_SFLG_BINARY_IO) { need += (2+PROC_BIN_SIZE)*vsize - 2 + hlen*2; } else { need += (hlen+csize)*2; @@ -1818,7 +3125,7 @@ deliver_vec_message(Port* prt, /* Port */ listp = NIL; iov += vsize; - if ((prt->status & ERTS_PORT_SFLG_BINARY_IO) == 0) { + if ((state & ERTS_PORT_SFLG_BINARY_IO) == 0) { Eterm* thp = hp; while (vsize--) { iov--; @@ -1871,7 +3178,7 @@ deliver_vec_message(Port* prt, /* Port */ tuple = TUPLE2(hp, am_data, listp); hp += 3; - tuple = TUPLE2(hp, prt->id, tuple); + tuple = TUPLE2(hp, prt->common.id, tuple); hp += 3; erts_queue_message(rp, &rp_locks, bp, tuple, am_undefined @@ -1904,7 +3211,7 @@ static void deliver_bin_message(Port* prt, /* port */ /* * Note. * - * The test for (p->status & ERTS_PORT_SFLGS_DEAD) == 0 is important since the + * The test for ERTS_PORT_SFLGS_DEAD is important since the * driver's flush function might call driver_async, which when using no * threads and being short circuited will notice that the io queue is empty * (after calling the driver's async_ready) and recursively call @@ -1920,7 +3227,7 @@ static void flush_port(Port *p) if (p->drv_ptr->flush != NULL) { #ifdef USE_VM_PROBES if (DTRACE_ENABLED(driver_flush)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(p->connected, p) + DTRACE_FORMAT_COMMON_PID_AND_PORT(ERTS_PORT_GET_CONNECTED(p), p) DTRACE3(driver_flush, process_str, port_str, p->name); } #endif @@ -1935,11 +3242,12 @@ static void flush_port(Port *p) } #ifdef ERTS_SMP if (p->xports) - erts_smp_xports_unlock(p); + erts_port_handle_xports(p); ASSERT(!p->xports); #endif } - if ((p->status & ERTS_PORT_SFLGS_DEAD) == 0 && is_port_ioq_empty(p)) { + if ((erts_atomic32_read_nob(&p->state) & ERTS_PORT_SFLGS_DEAD) == 0 + && is_port_ioq_empty(p)) { terminate_port(p); } } @@ -1951,29 +3259,29 @@ terminate_port(Port *prt) Eterm send_closed_port_id; Eterm connected_id = NIL /* Initialize to silence compiler */; erts_driver_t *drv; - int halt; + erts_aint32_t state; ERTS_SMP_CHK_NO_PROC_LOCKS; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - ASSERT(!prt->nlinks); - ASSERT(!prt->monitors); + ASSERT(!ERTS_P_LINKS(prt)); + ASSERT(!ERTS_P_MONITORS(prt)); - /* prt->status may be altered by kill_port()below */ - halt = (prt->status & ERTS_PORT_SFLG_HALT) != 0; - if (prt->status & ERTS_PORT_SFLG_SEND_CLOSED) { - erts_port_status_band_set(prt, ~ERTS_PORT_SFLG_SEND_CLOSED); - send_closed_port_id = prt->id; - connected_id = prt->connected; + /* state may be altered by kill_port() below */ + state = erts_atomic32_read_band_nob(&prt->state, + ~ERTS_PORT_SFLG_SEND_CLOSED); + if (state & ERTS_PORT_SFLG_SEND_CLOSED) { + send_closed_port_id = prt->common.id; + connected_id = ERTS_PORT_GET_CONNECTED(prt); } else { send_closed_port_id = NIL; } #ifdef ERTS_SMP - erts_cancel_smp_ptimer(prt->ptimer); + erts_cancel_smp_ptimer(prt->common.u.alive.ptimer); #else - erts_cancel_timer(&prt->tm); + erts_cancel_timer(&prt->common.u.alive.tm); #endif drv = prt->drv_ptr; @@ -1981,7 +3289,7 @@ terminate_port(Port *prt) int fpe_was_unmasked = erts_block_fpe(); #ifdef USE_VM_PROBES if (DTRACE_ENABLED(driver_stop)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(prt->connected, prt) + DTRACE_FORMAT_COMMON_PID_AND_PORT(connected_id, prt) DTRACE3(driver_stop, process_str, drv->name, port_str); } #endif @@ -1989,14 +3297,14 @@ terminate_port(Port *prt) erts_unblock_fpe(fpe_was_unmasked); #ifdef ERTS_SMP if (prt->xports) - erts_smp_xports_unlock(prt); + erts_port_handle_xports(prt); ASSERT(!prt->xports); #endif } if(drv->handle != NULL) { - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rlock(&erts_driver_list_lock); erts_ddll_decrement_port_count(drv->handle); - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); } stopq(prt); /* clear queue memory */ if(prt->linebuf != NULL){ @@ -2012,20 +3320,21 @@ terminate_port(Port *prt) if (prt->psd) erts_free(ERTS_ALC_T_PRTSD, prt->psd); + ASSERT(prt->dist_entry == NULL); + kill_port(prt); /* * We don't want to send the closed message until after the * port has been removed from the port table (in kill_port()). */ - if (halt && (erts_smp_atomic32_dec_read_nob(&erts_halt_progress) == 0)) { - erts_smp_port_unlock(prt); /* We will exit and never return */ + if ((state & ERTS_PORT_SFLG_HALT) + && (erts_smp_atomic32_dec_read_nob(&erts_halt_progress) == 0)) { + erts_port_release(prt); /* We will exit and never return */ erl_exit_flush_async(erts_halt_code, ""); } if (is_internal_port(send_closed_port_id)) deliver_result(send_closed_port_id, connected_id, am_closed); - - ASSERT(prt->dist_entry == NULL); } void @@ -2045,7 +3354,7 @@ static void sweep_one_monitor(ErtsMonitor *mon, void *vpsc) if (!rp) { goto done; } - rmon = erts_remove_monitor(&(rp->monitors),mon->ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), mon->ref); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); if (rmon == NULL) { goto done; @@ -2099,7 +3408,7 @@ static void sweep_one_link(ErtsLink *lnk, void *vpsc) ASSERT(is_internal_pid(lnk->pid)); rp = erts_pid2proc(NULL, 0, lnk->pid, rp_locks); if (rp) { - ErtsLink *rlnk = erts_remove_link(&(rp->nlinks), psc->port); + ErtsLink *rlnk = erts_remove_link(&ERTS_P_LINKS(rp), psc->port); if (rlnk) { int xres = erts_send_exit_signal(NULL, @@ -2135,11 +3444,13 @@ static void sweep_one_link(ErtsLink *lnk, void *vpsc) * that is to kill a port till reason kill. Then the port is stopped. * */ -void -erts_do_exit_port(Port *p, Eterm from, Eterm reason) + +int +erts_deliver_port_exit(Port *p, Eterm from, Eterm reason, int send_closed) { ErtsLink *lnk; Eterm rreason; + erts_aint32_t state; ERTS_SMP_CHK_NO_PROC_LOCKS; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(p)); @@ -2159,66 +3470,76 @@ erts_do_exit_port(Port *p, Eterm from, Eterm reason) } #endif - if ((p->status & (ERTS_PORT_SFLGS_DEAD - | ERTS_PORT_SFLG_EXITING - | ERTS_PORT_SFLG_IMMORTAL)) - || ((reason == am_normal) && - ((from != p->connected) && (from != p->id)))) { - return; - } + state = erts_atomic32_read_nob(&p->state); + if (state & (ERTS_PORT_SFLGS_DEAD + | ERTS_PORT_SFLG_EXITING + | ERTS_PORT_SFLG_CLOSING)) + return 0; + + if (reason == am_normal && from != ERTS_PORT_GET_CONNECTED(p) && from != p->common.id) + return 0; + + if (send_closed) + erts_atomic32_read_bor_relb(&p->state, + ERTS_PORT_SFLG_SEND_CLOSED); if (IS_TRACED_FL(p, F_TRACE_PORTS)) { trace_port(p, am_closed, reason); } - erts_trace_check_exiting(p->id); + erts_trace_check_exiting(p->common.id); /* * Setting the port to not busy here, frees the list of pending * processes and makes them runnable. */ - set_busy_port((ErlDrvPort)internal_port_index(p->id), 0); + set_busy_port((ErlDrvPort) p, 0); - if (p->reg != NULL) - (void) erts_unregister_name(NULL, 0, p, p->reg->name); + if (p->common.u.alive.reg != NULL) + (void) erts_unregister_name(NULL, 0, p, p->common.u.alive.reg->name); - erts_port_status_bor_set(p, ERTS_PORT_SFLG_EXITING); + state = erts_atomic32_read_bor_relb(&p->state, ERTS_PORT_SFLG_EXITING); { - SweepContext sc = {p->id, rreason}; - lnk = p->nlinks; - p->nlinks = NULL; + SweepContext sc = {p->common.id, rreason}; + lnk = ERTS_P_LINKS(p); + ERTS_P_LINKS(p) = NULL; erts_sweep_links(lnk, &sweep_one_link, &sc); } DRV_MONITOR_LOCK_PDL(p); { - ErtsMonitor *moni = p->monitors; - p->monitors = NULL; + ErtsMonitor *moni = ERTS_P_MONITORS(p); + ERTS_P_MONITORS(p) = NULL; erts_sweep_monitors(moni, &sweep_one_monitor, NULL); } DRV_MONITOR_UNLOCK_PDL(p); - if ((p->status & ERTS_PORT_SFLG_DISTRIBUTION) && p->dist_entry) { + if ((state & ERTS_PORT_SFLG_DISTRIBUTION) && p->dist_entry) { erts_do_net_exits(p->dist_entry, rreason); erts_deref_dist_entry(p->dist_entry); - p->dist_entry = NULL; - erts_port_status_band_set(p, ~ERTS_PORT_SFLG_DISTRIBUTION); + p->dist_entry = NULL; + erts_atomic32_read_band_relb(&p->state, + ~ERTS_PORT_SFLG_DISTRIBUTION); } if ((reason != am_kill) && !is_port_ioq_empty(p)) { - erts_port_status_bandor_set(p, - ~ERTS_PORT_SFLG_EXITING, /* must turn it off */ - ERTS_PORT_SFLG_CLOSING); + /* must turn exiting flag off */ + erts_atomic32_read_bset_relb(&p->state, + (ERTS_PORT_SFLG_EXITING + | ERTS_PORT_SFLG_CLOSING), + ERTS_PORT_SFLG_CLOSING); flush_port(p); } else { terminate_port(p); } + + return 1; } /* About the states ERTS_PORT_SFLG_EXITING and ERTS_PORT_SFLG_CLOSING used above. ** -** ERTS_PORT_SFLG_EXITING is a recursion protection for erts_do_exit_port(). +** ERTS_PORT_SFLG_EXITING is a recursion protection for erts_deliver_port_exit(). ** It is unclear whether this state is necessary or not, it might be possible ** to merge it with ERTS_PORT_SFLG_CLOSING. ERTS_PORT_SFLG_EXITING only persists ** over a section of sequential (but highly recursive) code. @@ -2234,232 +3555,1108 @@ erts_do_exit_port(Port *p, Eterm from, Eterm reason) ** {PID, close} ** {PID, {command, io-list}} ** {PID, {connect, New_PID}} -** -** */ -void erts_port_command(Process *proc, - Eterm caller_id, - Port *port, - Eterm command) +ErtsPortOpResult +erts_port_command(Process *c_p, + int flags, + Port *port, + Eterm command, + Eterm *refp) { Eterm *tp; - Eterm pid; - if (!port) - return; + ASSERT(port); - erts_smp_proc_unlock(proc, ERTS_PROC_LOCK_MAIN); - ERTS_SMP_CHK_NO_PROC_LOCKS; - ASSERT(!INVALID_PORT(port, port->id)); + flags |= ERTS_PORT_SIG_FLG_BANG_OP; if (is_tuple_arity(command, 2)) { + Eterm cntd; tp = tuple_val(command); - if ((pid = port->connected) == tp[1]) { - /* PID must be connected */ + cntd = tp[1]; + if (is_internal_pid(cntd)) { if (tp[2] == am_close) { - erts_port_status_bor_set(port, ERTS_PORT_SFLG_SEND_CLOSED); - erts_do_exit_port(port, pid, am_normal); - -#ifdef USE_VM_PROBES - if(DTRACE_ENABLED(port_command)) { - DTRACE_FORMAT_COMMON_PROC_AND_PORT(proc, port) - DTRACE4(port_command, process_str, port_str, port->name, "close"); - } -#endif - goto done; + if (!erts_port_synchronous_ops) + refp = NULL; + flags &= ~ERTS_PORT_SIG_FLG_NOSUSPEND; + return erts_port_exit(c_p, flags, port, cntd, am_normal, refp); } else if (is_tuple_arity(tp[2], 2)) { tp = tuple_val(tp[2]); if (tp[1] == am_command) { - if (erts_write_to_port(caller_id, port, tp[2]) == 0) - goto done; - } else if ((tp[1] == am_connect) && is_internal_pid(tp[2])) { -#ifdef USE_VM_PROBES - if(DTRACE_ENABLED(port_command)) { - DTRACE_FORMAT_COMMON_PROC_AND_PORT(proc, port) - DTRACE4(port_command, process_str, port_str, port->name, "connect"); - } -#endif - port->connected = tp[2]; - deliver_result(port->id, pid, am_connected); - goto done; + if (!(flags & ERTS_PORT_SIG_FLG_NOSUSPEND) + && !erts_port_synchronous_ops) + refp = NULL; + return erts_port_output(c_p, flags, port, cntd, tp[2], refp); + } + else if (tp[1] == am_connect) { + if (!erts_port_synchronous_ops) + refp = NULL; + flags &= ~ERTS_PORT_SIG_FLG_NOSUSPEND; + return erts_port_connect(c_p, flags, port, cntd, tp[2], refp); } } } } - { - ErtsProcLocks rp_locks = ERTS_PROC_LOCKS_XSIG_SEND; - Process* rp = erts_pid2proc(NULL, 0, - port->connected, rp_locks); - if (rp) { - (void) erts_send_exit_signal(NULL, - port->id, - rp, - &rp_locks, - am_badsig, - NIL, - NULL, - 0); - erts_smp_proc_unlock(rp, rp_locks); - } + /* badsig */ + if (!erts_port_synchronous_ops) + refp = NULL; + flags &= ~ERTS_PORT_SIG_FLG_NOSUSPEND; + return bad_port_signal(c_p, flags, port, c_p->common.id, refp, am_command); +} + +static ERTS_INLINE ErtsPortOpResult +call_driver_control(Eterm caller, + Port *prt, + unsigned int command, + char *bufp, + ErlDrvSizeT size, + char **resp_bufp, + ErlDrvSizeT *from_size) +{ + ErlDrvSSizeT cres; + + if (!prt->drv_ptr->control) + return ERTS_PORT_OP_BADARG; + +#ifdef USE_VM_PROBES + if (DTRACE_ENABLED(port_control) || DTRACE_ENABLED(driver_control)) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(caller, prt); + DTRACE4(port_control, process_str, port_str, prt->name, command); + DTRACE5(driver_control, process_str, port_str, prt->name, + command, size); } - done: - erts_smp_proc_lock(proc, ERTS_PROC_LOCK_MAIN); +#endif + + prt->caller = caller; + cres = prt->drv_ptr->control((ErlDrvData) prt->drv_data, + command, + bufp, + size, + resp_bufp, + *from_size); + prt->caller = NIL; + + if (cres < 0) + return ERTS_PORT_OP_BADARG; + + *from_size = (ErlDrvSizeT) cres; + + return ERTS_PORT_OP_DONE; } -/* - * Control a port synchronously. - * Returns either a list or a binary. - */ -Eterm -erts_port_control(Process* p, Port* prt, Uint command, Eterm iolist) -{ - byte* to_port = NULL; /* Buffer to write to port. */ - /* Initialization is for shutting up - warning about use before set. */ - Uint to_len = 0; /* Length of buffer. */ - int must_free = 0; /* True if the buffer should be freed. */ - char port_result[ERL_ONHEAP_BIN_LIMIT]; /* Default buffer for result from port. */ - char* port_resp; /* Pointer to result buffer. */ - ErlDrvSSizeT n; - ErlDrvSSizeT (*control) - (ErlDrvData, unsigned, char*, ErlDrvSizeT, char**, ErlDrvSizeT); - int fpe_was_unmasked; +static void +cleanup_scheduled_control(Binary *binp, char *bufp) +{ + if (binp) { + if (erts_refc_dectest(&binp->refc, 0) == 0) + erts_bin_free(binp); + } + else { + if (bufp) + erts_free(ERTS_ALC_T_DRV_CTRL_DATA, bufp); + } +} - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - if ((control = prt->drv_ptr->control) == NULL) { - return THE_NON_VALUE; +static ERTS_INLINE Uint +port_control_result_size(int control_flags, + char *resp_bufp, + ErlDrvSizeT *resp_size, + char *pre_alloc_buf) +{ + if (!resp_bufp) + return (Uint) 0; + + if (control_flags & PORT_CONTROL_FLAG_BINARY) { + if (resp_bufp != pre_alloc_buf) { + ErlDrvBinary *dbin = (ErlDrvBinary *) resp_bufp; + *resp_size = dbin->orig_size; + if (*resp_size > ERL_ONHEAP_BIN_LIMIT) + return PROC_BIN_SIZE; + } + ASSERT(*resp_size <= ERL_ONHEAP_BIN_LIMIT); + return (Uint) heap_bin_size((*resp_size)); } - /* - * Convert the iolist to a buffer, pointed to by to_port, - * and with its length in to_len. - */ - if (is_binary(iolist) && binary_bitoffset(iolist) == 0) { + return (Uint) 2*(*resp_size); +} + +static ERTS_INLINE Eterm +write_port_control_result(int control_flags, + char *resp_bufp, + ErlDrvSizeT resp_size, + char *pre_alloc_buf, + Eterm **hpp, + ErlHeapFragment *bp, + ErlOffHeap *ohp) +{ + Eterm res; + if (!resp_bufp) + return NIL; + if (control_flags & PORT_CONTROL_FLAG_BINARY) { + /* Binary result */ + ErlDrvBinary *dbin; + ErlHeapBin *hbin; + + if (resp_bufp == pre_alloc_buf) + dbin = NULL; + else { + dbin = (ErlDrvBinary *) resp_bufp; + if (dbin->orig_size > ERL_ONHEAP_BIN_LIMIT) { + ProcBin* pb = (ProcBin *) *hpp; + *hpp += PROC_BIN_SIZE; + pb->thing_word = HEADER_PROC_BIN; + pb->size = dbin->orig_size; + pb->next = ohp->first; + ohp->first = (struct erl_off_heap_header *) pb; + pb->val = ErlDrvBinary2Binary(dbin); + pb->bytes = (byte*) dbin->orig_bytes; + pb->flags = 0; + OH_OVERHEAD(ohp, dbin->orig_size / sizeof(Eterm)); + return make_binary(pb); + } + resp_bufp = dbin->orig_bytes; + resp_size = dbin->orig_size; + } + + hbin = (ErlHeapBin *) *hpp; + *hpp += heap_bin_size(resp_size); + ASSERT(resp_size <= ERL_ONHEAP_BIN_LIMIT); + hbin->thing_word = header_heap_bin(resp_size); + hbin->size = resp_size; + sys_memcpy(hbin->data, resp_bufp, resp_size); + if (dbin) + driver_free_binary(dbin); + return make_binary(hbin); + } + + /* List result */ + res = buf_to_intlist(hpp, resp_bufp, resp_size, NIL); + if (resp_bufp != pre_alloc_buf) + driver_free(resp_bufp); + return res; +} + +static int +port_sig_control(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + ASSERT(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY); + + if (op == ERTS_PROC2PORT_SIG_EXEC) { + char resp_buf[ERL_ONHEAP_BIN_LIMIT]; + ErlDrvSizeT resp_size = sizeof(resp_buf); + char *resp_bufp = &resp_buf[0]; + ErtsPortOpResult res; + + res = call_driver_control(sigdp->caller, + prt, + sigdp->u.control.command, + sigdp->u.control.bufp, + sigdp->u.control.size, + &resp_bufp, + &resp_size); + + if (res == ERTS_PORT_OP_DONE) { + Eterm msg; + Eterm *hp, *hp_start; + ErlHeapFragment *bp; + ErlOffHeap *ohp; + Process *rp; + ErtsProcLocks rp_locks = 0; + Uint hsz; + int control_flags; + + rp = erts_proc_lookup_raw(sigdp->caller); + if (!rp) + goto done; + + control_flags = prt->control_flags; + + hsz = ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE; + hsz += port_control_result_size(control_flags, + resp_bufp, + &resp_size, + &resp_buf[0]); + + hp_start = hp = erts_alloc_message_heap(hsz, + &bp, + &ohp, + rp, + &rp_locks); + + msg = write_port_control_result(control_flags, + resp_bufp, + resp_size, + &resp_buf[0], + &hp, + bp, + ohp); + + queue_port_sched_op_reply(rp, + &rp_locks, + hp_start, + hp, + hsz, + bp, + sigdp->ref, + msg); + + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + goto done; + } + } + + /* failure */ + + port_sched_op_reply(sigdp->caller, sigdp->ref, am_badarg); + +done: + + cleanup_scheduled_control(sigdp->u.control.binp, + sigdp->u.control.bufp); + + return ERTS_PORT_REDS_CONTROL; +} + + +ErtsPortOpResult +erts_port_control(Process* c_p, + Port *prt, + unsigned int command, + Eterm data, + Eterm *retvalp) +{ + ErtsPortOpResult res; + char *bufp = NULL; + ErlDrvSizeT size = 0; + int try_call; + int tmp_alloced = 0; + erts_aint32_t sched_flags; + Binary *binp; + int copy; + ErtsProc2PortSigData *sigdp; + + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (sched_flags & ERTS_PTS_FLG_EXIT) + return ERTS_PORT_OP_BADARG; + + try_call = !(sched_flags & ERTS_PTS_FLGS_FORCE_SCHEDULE_OP); + + if (is_binary(data) && binary_bitoffset(data) == 0) { + byte *bytep; ERTS_DECLARE_DUMMY(Uint bitoffs); ERTS_DECLARE_DUMMY(Uint bitsize); - ERTS_GET_BINARY_BYTES(iolist, to_port, bitoffs, bitsize); - to_len = binary_size(iolist); + ERTS_GET_BINARY_BYTES(data, bytep, bitoffs, bitsize); + bufp = (char *) bytep; + size = binary_size(data); } else { int r; - /* Try with an 8KB buffer first (will often be enough I guess). */ - to_len = 8*1024; - to_port = erts_alloc(ERTS_ALC_T_TMP, to_len); - must_free = 1; + if (!try_call) { + if (erts_iolist_size(data, &size)) + return ERTS_PORT_OP_BADARG; + bufp = erts_alloc(ERTS_ALC_T_DRV_CTRL_DATA, size); + r = erts_iolist_to_buf(data, bufp, size); + ASSERT(r == 0); + } + else { + /* Try with an 8KB buffer first (will often be enough I guess). */ + size = 8*1024; + bufp = erts_alloc(ERTS_ALC_T_TMP, size); + tmp_alloced = 1; + + r = erts_iolist_to_buf(data, bufp, size); + if (ERTS_IOLIST_TO_BUF_SUCCEEDED(r)) { + size -= r; + } else { + if (r == ERTS_IOLIST_TO_BUF_TYPE_ERROR) { /* Type error */ + erts_free(ERTS_ALC_T_TMP, bufp); + return ERTS_PORT_OP_BADARG; + } + else { + ASSERT(r == ERTS_IOLIST_TO_BUF_OVERFLOW); /* Overflow */ + erts_free(ERTS_ALC_T_TMP, bufp); + if (erts_iolist_size(data, &size)) + return ERTS_PORT_OP_BADARG; /* Type error */ + } + bufp = erts_alloc(ERTS_ALC_T_TMP, size); + r = erts_iolist_to_buf(data, bufp, size); + ASSERT(r == 0); + } + } + } - /* - * In versions before R10B, we used to reserve random - * amounts of extra memory. From R10B, we allocate the - * exact amount. - */ - r = io_list_to_buf(iolist, (char*) to_port, to_len); - if (r >= 0) { - to_len -= r; - } else if (r == -2) { /* Type error */ - erts_free(ERTS_ALC_T_TMP, (void *) to_port); - return THE_NON_VALUE; - } else { - ASSERT(r == -1); /* Overflow */ - erts_free(ERTS_ALC_T_TMP, (void *) to_port); - if (erts_iolist_size(iolist, &to_len)) { /* Type error */ - return THE_NON_VALUE; + if (try_call) { + char resp_buf[ERL_ONHEAP_BIN_LIMIT]; + char* resp_bufp = &resp_buf[0]; + ErlDrvSizeT resp_size = sizeof(resp_buf); + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + 0, + am_control); + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + Eterm *hp; + Uint hsz; + int control_flags; + + res = call_driver_control(c_p->common.id, + prt, + command, + bufp, + size, + &resp_bufp, + &resp_size); + finalize_imm_drv_call(&try_call_state); + if (tmp_alloced) + erts_free(ERTS_ALC_T_TMP, bufp); + if (res == ERTS_PORT_OP_BADARG) { + return ERTS_PORT_OP_BADARG; } - must_free = 1; - to_port = erts_alloc(ERTS_ALC_T_TMP, to_len); - r = io_list_to_buf(iolist, (char*) to_port, to_len); - ASSERT(r == 0); + + control_flags = prt->control_flags; + + hsz = port_control_result_size(control_flags, + resp_bufp, + &resp_size, + &resp_buf[0]); + hp = HAlloc(c_p, hsz); + *retvalp = write_port_control_result(control_flags, + resp_bufp, + resp_size, + &resp_buf[0], + &hp, + NULL, + &c_p->off_heap); + return ERTS_PORT_OP_DONE; + } + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + if (tmp_alloced) + erts_free(ERTS_ALC_T_TMP, bufp); + return ERTS_PORT_OP_BADARG; + default: + /* Schedule control() call instead... */ + break; } } - prt->caller = p->id; /* Internal pid */ + /* Convert data into something that can be scheduled */ - erts_smp_proc_unlock(p, ERTS_PROC_LOCK_MAIN); - ERTS_SMP_CHK_NO_PROC_LOCKS; + copy = tmp_alloced; + + binp = NULL; + + if (is_binary(data) && binary_bitoffset(data) == 0) { + Eterm *ebinp = binary_val_rel(data, NULL); + ASSERT(!tmp_alloced); + if (*ebinp == HEADER_SUB_BIN) + ebinp = binary_val_rel(((ErlSubBin *) ebinp)->orig, NULL); + if (*ebinp != HEADER_PROC_BIN) + copy = 1; + else { + binp = ((ProcBin *) ebinp)->val; + ASSERT(bufp < bufp + size); + ASSERT(binp->orig_bytes <= bufp + && bufp + size <= binp->orig_bytes + binp->orig_size); + erts_refc_inc(&binp->refc, 1); + } + } + + if (copy) { + char *old_bufp = bufp; + bufp = erts_alloc(ERTS_ALC_T_DRV_CTRL_DATA, size); + sys_memcpy(bufp, old_bufp, size); + if (tmp_alloced) + erts_free(ERTS_ALC_T_TMP, old_bufp); + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_CONTROL; + sigdp->u.control.binp = binp; + sigdp->u.control.command = command; + sigdp->u.control.bufp = bufp; + sigdp->u.control.size = size; + + res = erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + retvalp, + sigdp, + 0, + port_sig_control); + if (res != ERTS_PORT_OP_SCHEDULED) { + cleanup_scheduled_control(binp, bufp); + return ERTS_PORT_OP_BADARG; + } + return res; +} + +static ERTS_INLINE ErtsPortOpResult +call_driver_call(Eterm caller, + Port *prt, + unsigned int command, + char *bufp, + ErlDrvSizeT size, + char **resp_bufp, + ErlDrvSizeT *from_size, + unsigned *ret_flagsp) +{ + ErlDrvSSizeT cres; + + if (!prt->drv_ptr->call) + return ERTS_PORT_OP_BADARG; #ifdef USE_VM_PROBES - if (DTRACE_ENABLED(port_control) || DTRACE_ENABLED(driver_control)) { - DTRACE_FORMAT_COMMON_PROC_AND_PORT(p, prt); - DTRACE4(port_control, process_str, port_str, prt->name, command); - DTRACE5(driver_control, process_str, port_str, prt->name, - command, to_len); + if (DTRACE_ENABLED(driver_call)) { + DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); + DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); + + dtrace_pid_str(caller, process_str); + dtrace_port_str(prt, port_str); + DTRACE5(driver_call, process_str, port_str, prt->name, command, size); } #endif - /* - * Call the port's control routine. - */ + prt->caller = caller; + cres = prt->drv_ptr->call((ErlDrvData) prt->drv_data, + command, + bufp, + size, + resp_bufp, + *from_size, + ret_flagsp); + prt->caller = NIL; - port_resp = port_result; - fpe_was_unmasked = erts_block_fpe(); - n = control((ErlDrvData)prt->drv_data, command, (char*)to_port, to_len, - &port_resp, sizeof(port_result)); - erts_unblock_fpe(fpe_was_unmasked); - if (must_free) { - erts_free(ERTS_ALC_T_TMP, (void *) to_port); + if (cres <= 0 + || ((byte) (*resp_bufp)[0]) != VERSION_MAGIC) + return ERTS_PORT_OP_BADARG; + + *from_size = (ErlDrvSizeT) cres; + + return ERTS_PORT_OP_DONE; +} + + +static +void cleanup_scheduled_call(char *bufp) +{ + if (bufp) + erts_free(ERTS_ALC_T_DRV_CALL_DATA, bufp); +} + +static int +port_sig_call(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + char resp_buf[256]; + ErlDrvSizeT resp_size = sizeof(resp_buf); + char *resp_bufp = &resp_buf[0]; + unsigned ret_flags = 0U; + + + ASSERT(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY); + + if (op == ERTS_PROC2PORT_SIG_EXEC) { + ErtsPortOpResult res; + + res = call_driver_call(sigdp->caller, + prt, + sigdp->u.call.command, + sigdp->u.call.bufp, + sigdp->u.call.size, + &resp_bufp, + &resp_size, + &ret_flags); + + if (res == ERTS_PORT_OP_DONE) { + Eterm msg; + Eterm *hp; + ErlHeapFragment *bp; + ErlOffHeap *ohp; + Process *rp; + ErtsProcLocks rp_locks = 0; + Uint hsz; + + rp = erts_proc_lookup_raw(sigdp->caller); + if (!rp) + goto done; + + hsz = erts_decode_ext_size((byte *) resp_bufp, resp_size); + if (hsz >= 0) { + Eterm *hp_start; + byte *endp; + + hsz += 3; /* ok tuple */ + hsz += ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE; + + hp_start = hp = erts_alloc_message_heap(hsz, + &bp, + &ohp, + rp, + &rp_locks); + endp = (byte *) resp_bufp; + msg = erts_decode_ext(&hp, ohp, &endp); + if (is_value(msg)) { + msg = TUPLE2(hp, am_ok, msg); + hp += 3; + + queue_port_sched_op_reply(rp, + &rp_locks, + hp_start, + hp, + hsz, + bp, + sigdp->ref, + msg); + + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + goto done; + } + if (bp) + free_message_buffer(bp); + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } + } } - prt->caller = NIL; -#ifdef ERTS_SMP - if (prt->xports) - erts_smp_xports_unlock(prt); - ASSERT(!prt->xports); -#endif - erts_smp_proc_lock(p, ERTS_PROC_LOCK_MAIN); - /* - * Handle the result. - */ + port_sched_op_reply(sigdp->caller, sigdp->ref, am_badarg); + +done: - if (n < 0) { - return THE_NON_VALUE; + if (resp_bufp != &resp_buf[0] && !(ret_flags & DRIVER_CALL_KEEP_BUFFER)) + driver_free(resp_bufp); + + cleanup_scheduled_call(sigdp->u.call.bufp); + + return ERTS_PORT_REDS_CALL; +} + + +ErtsPortOpResult +erts_port_call(Process* c_p, + Port *prt, + unsigned int command, + Eterm data, + Eterm *retvalp) +{ + ErtsPortOpResult res; + char input_buf[256]; + char *bufp; + byte *endp; + ErlDrvSizeT size; + int try_call; + erts_aint32_t sched_flags; + ErtsProc2PortSigData *sigdp; + + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (sched_flags & ERTS_PTS_FLG_EXIT) { + return ERTS_PORT_OP_BADARG; } - if ((prt->control_flags & PORT_CONTROL_FLAG_BINARY) == 0) { /* List result */ - Eterm ret; - Eterm* hp = HAlloc(p, 2*n); - ret = buf_to_intlist(&hp, port_resp, n, NIL); - if (port_resp != port_result) { - driver_free(port_resp); + try_call = !(sched_flags & ERTS_PTS_FLGS_FORCE_SCHEDULE_OP); + + size = erts_encode_ext_size(data); + + if (!try_call) + bufp = erts_alloc(ERTS_ALC_T_DRV_CALL_DATA, size); + else if (size <= sizeof(input_buf)) + bufp = &input_buf[0]; + else + bufp = erts_alloc(ERTS_ALC_T_TMP, size); + + endp = (byte *) bufp; + erts_encode_ext(data, &endp); + + if (endp - (byte *) bufp > size) + ERTS_INTERNAL_ERROR("erts_internal:port_call() - Buffer overflow"); + + size = endp - (byte *) bufp; + + if (try_call) { + char resp_buf[255]; + char* resp_bufp = &resp_buf[0]; + ErlDrvSizeT resp_size = sizeof(resp_buf); + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + 0, + am_call); + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + Eterm *hp, *hp_end; + Uint hsz; + unsigned ret_flags = 0U; + Eterm term; + + res = call_driver_call(c_p->common.id, + prt, + command, + bufp, + size, + &resp_bufp, + &resp_size, + &ret_flags); + + finalize_imm_drv_call(&try_call_state); + if (bufp != &input_buf[0]) + erts_free(ERTS_ALC_T_TMP, bufp); + if (res == ERTS_PORT_OP_BADARG) + return ERTS_PORT_OP_BADARG; + hsz = erts_decode_ext_size((byte *) resp_bufp, resp_size); + if (hsz < 0) + return ERTS_PORT_OP_BADARG; + hsz += 3; + hp = HAlloc(c_p, hsz); + hp_end = hp + hsz; + endp = (byte *) resp_bufp; + term = erts_decode_ext(&hp, &MSO(c_p), &endp); + if (term == THE_NON_VALUE) + return ERTS_PORT_OP_BADARG; + *retvalp = TUPLE2(hp, am_ok, term); + hp += 3; + HRelease(c_p, hp_end, hp); + if (resp_buf != &resp_buf[0] + && !(ret_flags & DRIVER_CALL_KEEP_BUFFER)) + driver_free(resp_buf); + return ERTS_PORT_OP_DONE; } - return ret; - } - else if (port_resp == NULL) { - return NIL; - } - else { /* Binary result */ - ErlDrvBinary *dbin; - ErlHeapBin *hbin; - if (port_resp != port_result) { - dbin = (ErlDrvBinary *) port_resp; - if (dbin->orig_size > ERL_ONHEAP_BIN_LIMIT) { - ProcBin* pb = (ProcBin *) HAlloc(p, PROC_BIN_SIZE); - pb->thing_word = HEADER_PROC_BIN; - pb->size = dbin->orig_size; - pb->next = MSO(p).first; - MSO(p).first = (struct erl_off_heap_header*)pb; - pb->val = ErlDrvBinary2Binary(dbin); - pb->bytes = (byte*) dbin->orig_bytes; - pb->flags = 0; - OH_OVERHEAD(&(MSO(p)), dbin->orig_size / sizeof(Eterm)); - return make_binary(pb); - } - port_resp = dbin->orig_bytes; - n = dbin->orig_size; - } else { - dbin = NULL; + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + if (bufp != &input_buf[0]) + erts_free(ERTS_ALC_T_TMP, bufp); + return ERTS_PORT_OP_BADARG; + default: + /* Schedule call() call instead... */ + break; } - hbin = (ErlHeapBin*) HAlloc(p, heap_bin_size(n)); - ASSERT(n <= ERL_ONHEAP_BIN_LIMIT); - hbin->thing_word = header_heap_bin(n); - hbin->size = n; - sys_memcpy(hbin->data, port_resp, n); - if (dbin != NULL) { - driver_free_binary(dbin); + } + + /* Convert data into something that can be scheduled */ + + if (bufp == &input_buf[0] || try_call) { + char *new_bufp = erts_alloc(ERTS_ALC_T_DRV_CALL_DATA, size); + sys_memcpy(new_bufp, bufp, size); + if (bufp != &input_buf[0]) + erts_free(ERTS_ALC_T_TMP, bufp); + bufp = new_bufp; + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_CALL; + sigdp->u.call.command = command; + sigdp->u.call.bufp = bufp; + sigdp->u.call.size = size; + + res = erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + retvalp, + sigdp, + 0, + port_sig_call); + if (res != ERTS_PORT_OP_SCHEDULED) { + cleanup_scheduled_call(bufp); + return ERTS_PORT_OP_BADARG; + } + return res; +} + +static Eterm +make_port_info_term(Eterm **hpp_start, + Eterm **hpp, + Uint *hszp, + ErlHeapFragment **bpp, + Port *prt, + Eterm item) +{ + ErlOffHeap *ohp; + + if (is_value(item)) { + if (erts_bld_port_info(NULL, NULL, hszp, prt, item) == am_false) + return THE_NON_VALUE; + if (*hszp) { + *bpp = new_message_buffer(*hszp); + *hpp_start = *hpp = (*bpp)->mem; + ohp = &(*bpp)->off_heap; } - return make_binary(hbin); + else { + *bpp = NULL; + *hpp_start = *hpp = NULL; + ohp = NULL; + } + return erts_bld_port_info(hpp, ohp, NULL, prt, item); + } + else { + int i; + int len; + int start; + static Eterm item[] = ERTS_PORT_INFO_1_ITEMS; + static Eterm value[sizeof(item)/sizeof(item[0])]; + + start = 0; + len = sizeof(item)/sizeof(item[0]); + + for (i = start; i < sizeof(item)/sizeof(item[0]); i++) { + ASSERT(is_atom(item[i])); + value[i] = erts_bld_port_info(NULL, NULL, hszp, prt, item[i]); + } + + if (value[0] == am_undefined) { + start++; + len--; + } + + erts_bld_list(NULL, hszp, len, &value[start]); + + *bpp = new_message_buffer(*hszp); + *hpp_start = *hpp = (*bpp)->mem; + ohp = &(*bpp)->off_heap; + + for (i = start; i < sizeof(item)/sizeof(item[0]); i++) + value[i] = erts_bld_port_info(hpp, ohp, NULL, prt, item[i]); + + return erts_bld_list(hpp, NULL, len, &value[start]); + } +} + +static int +port_sig_info(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + ASSERT(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY); + if (op != ERTS_PROC2PORT_SIG_EXEC) + port_sched_op_reply(sigdp->caller, sigdp->ref, am_undefined); + else { + Eterm *hp, *hp_start; + Uint hsz; + ErlHeapFragment *bp; + Eterm value; + Process *rp; + ErtsProcLocks rp_locks = 0; + + rp = erts_proc_lookup_raw(sigdp->caller); + if (!rp) + return ERTS_PORT_REDS_INFO; + + hsz = ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE; + value = make_port_info_term(&hp_start, + &hp, + &hsz, + &bp, + prt, + sigdp->u.info.item); + if (is_value(value)) { + queue_port_sched_op_reply(rp, + &rp_locks, + hp_start, + hp, + hsz, + bp, + sigdp->ref, + value); + } + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } + return ERTS_PORT_REDS_INFO; +} + +ErtsPortOpResult +erts_port_info(Process* c_p, + Port *prt, + Eterm item, + Eterm *retvalp) +{ + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + 0, + am_info); + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + Eterm *hp, *hp_start; + ErlHeapFragment *bp; + Uint hsz = 0; + Eterm value = make_port_info_term(&hp_start, &hp, &hsz, &bp, prt, item); + finalize_imm_drv_call(&try_call_state); + if (is_non_value(value)) + return ERTS_PORT_OP_BADARG; + else if (is_immed(value)) + *retvalp = value; + else { + Uint used_h_size = hp - hp_start; + hp = HAlloc(c_p, used_h_size); + *retvalp = copy_struct(value, used_h_size, &hp, &MSO(c_p)); + free_message_buffer(bp); + } + return ERTS_PORT_OP_DONE; + } + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule call instead... */ + break; + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_INFO; + sigdp->u.info.item = item; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + retvalp, + sigdp, + 0, + port_sig_info); +} + +static int +port_sig_set_data(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + ASSERT(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY); + + if (op == ERTS_PROC2PORT_SIG_EXEC) { + if (prt->bp) + free_message_buffer(prt->bp); + prt->bp = sigdp->u.set_data.bp; + prt->data = sigdp->u.set_data.data; + port_sched_op_reply(sigdp->caller, sigdp->ref, am_true); + } + else { + if (sigdp->u.set_data.bp) + free_message_buffer(sigdp->u.set_data.bp); + port_sched_op_reply(sigdp->caller, sigdp->ref, am_badarg); + } + return ERTS_PORT_REDS_SET_DATA; +} + +ErtsPortOpResult +erts_port_set_data(Process* c_p, + Port *prt, + Eterm data, + Eterm *refp) +{ + ErtsPortOpResult res; + Eterm set_data; + ErlHeapFragment *bp; + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + !refp, + am_set_data); + + if (is_immed(data)) { + set_data = data; + bp = NULL; + } + else { + Eterm *hp; + Uint sz = size_object(data); + bp = new_message_buffer(sz); + hp = bp->mem; + set_data = copy_struct(data, sz, &hp, &bp->off_heap); + } + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: + if (prt->bp) + free_message_buffer(prt->bp); + prt->bp = bp; + prt->data = set_data; + finalize_imm_drv_call(&try_call_state); + return ERTS_PORT_OP_DONE; + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule call instead... */ + break; + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_SET_DATA; + sigdp->u.set_data.data = set_data; + sigdp->u.set_data.bp = bp; + + res = erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + refp, + sigdp, + 0, + port_sig_set_data); + if (res != ERTS_PORT_OP_SCHEDULED && bp) + free_message_buffer(bp); + return res; +} + +static int +port_sig_get_data(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + ASSERT(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY); + if (op != ERTS_PROC2PORT_SIG_EXEC) + port_sched_op_reply(sigdp->caller, sigdp->ref, am_badarg); + else { + Process *rp; + ErtsProcLocks rp_locks = 0; + + rp = erts_proc_lookup_raw(sigdp->caller); + if (rp) { + Uint hsz; + Eterm *hp, *hp_start; + Eterm data, msg; + ErlHeapFragment *bp; + ErlOffHeap *ohp; + + hsz = ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE; + hsz += 3; + if (prt->bp) + hsz += prt->bp->used_size; + + hp_start = hp = erts_alloc_message_heap(hsz, + &bp, + &ohp, + rp, + &rp_locks); + + if (is_immed(prt->data)) + data = prt->data; + else + data = copy_struct(prt->data, + prt->bp->used_size, + &hp, + &bp->off_heap); + + + + msg = TUPLE2(hp, am_ok, data); + hp += 3; + + queue_port_sched_op_reply(rp, + &rp_locks, + hp_start, + hp, + hsz, + bp, + sigdp->ref, + msg); + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } + } + return ERTS_PORT_REDS_GET_DATA; +} + +ErtsPortOpResult +erts_port_get_data(Process* c_p, + Port *prt, + Eterm *retvalp) +{ + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + 0, + am_get_data); + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + Eterm *hp; + Eterm data; + ErlHeapFragment *bp; + Uint sz; + if (is_immed(prt->data)) { + bp = NULL; + data = prt->data; + } + else { + bp = new_message_buffer(prt->bp->used_size); + data = copy_struct(prt->data, + prt->bp->used_size, + &hp, + &bp->off_heap); + } + finalize_imm_drv_call(&try_call_state); + if (is_immed(data)) + sz = 0; + else + sz = bp->used_size; + + hp = HAlloc(c_p, sz + 3); + if (is_not_immed(data)) { + data = copy_struct(data, bp->used_size, &hp, &MSO(c_p)); + free_message_buffer(bp); + } + *retvalp = TUPLE2(hp, am_ok, data); + return ERTS_PORT_OP_DONE; + } + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule call instead... */ + break; } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_GET_DATA; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + retvalp, + sigdp, + 0, + port_sig_get_data); } typedef struct { @@ -2480,39 +4677,39 @@ static void prt_one_lnk(ErtsLink *lnk, void *vprtd) } void -print_port_info(int to, void *arg, int i) +print_port_info(Port *p, int to, void *arg) { - Port* p = &erts_port[i]; + erts_aint32_t state = erts_atomic32_read_nob(&p->state); - if (p->status & ERTS_PORT_SFLGS_DEAD) + if (state & ERTS_PORT_SFLGS_DEAD) return; - erts_print(to, arg, "=port:%T\n", p->id); - erts_print(to, arg, "Slot: %d\n", i); - if (p->status & ERTS_PORT_SFLG_CONNECTED) { - erts_print(to, arg, "Connected: %T", p->connected); + erts_print(to, arg, "=port:%T\n", p->common.id); + erts_print(to, arg, "Slot: %d\n", internal_port_index(p->common.id)); + if (state & ERTS_PORT_SFLG_CONNECTED) { + erts_print(to, arg, "Connected: %T", ERTS_PORT_GET_CONNECTED(p)); erts_print(to, arg, "\n"); } - if (p->nlinks != NULL) { + if (ERTS_P_LINKS(p)) { prt_one_lnk_data prtd; prtd.to = to; prtd.arg = arg; erts_print(to, arg, "Links: "); - erts_doforall_links(p->nlinks, &prt_one_lnk, &prtd); + erts_doforall_links(ERTS_P_LINKS(p), &prt_one_lnk, &prtd); erts_print(to, arg, "\n"); } - if (p->monitors != NULL) { + if (ERTS_P_MONITORS(p)) { prt_one_lnk_data prtd; prtd.to = to; prtd.arg = arg; erts_print(to, arg, "Monitors: "); - erts_doforall_monitors(p->monitors, &prt_one_monitor, &prtd); + erts_doforall_monitors(ERTS_P_MONITORS(p), &prt_one_monitor, &prtd); erts_print(to, arg, "\n"); } - if (p->reg != NULL) - erts_print(to, arg, "Registered as: %T\n", p->reg->name); + if (p->common.u.alive.reg != NULL) + erts_print(to, arg, "Registered as: %T\n", p->common.u.alive.reg->name); if (p->drv_ptr == &fd_driver) { erts_print(to, arg, "Port is UNIX fd not opened by emulator: %s\n", p->name); @@ -2526,109 +4723,143 @@ print_port_info(int to, void *arg, int i) } void -set_busy_port(ErlDrvPort port_num, int on) +set_busy_port(ErlDrvPort dprt, int on) { + Port *prt; + erts_aint32_t flags; + #ifdef USE_VM_PROBES DTRACE_CHARBUF(port_str, 16); #endif ERTS_SMP_CHK_NO_PROC_LOCKS; - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(&erts_port[port_num])); + prt = erts_drvport2port_raw(dprt); + if (!prt) + return; if (on) { - erts_port_status_bor_set(&erts_port[port_num], - ERTS_PORT_SFLG_PORT_BUSY); + flags = erts_smp_atomic32_read_bor_acqb(&prt->sched.flags, + ERTS_PTS_FLG_BUSY_PORT); + if (flags & ERTS_PTS_FLG_BUSY_PORT) + return; /* Already busy */ + + if (flags & ERTS_PTS_FLG_HAVE_NS_TASKS) + erts_port_task_abort_nosuspend_tasks(prt); + #ifdef USE_VM_PROBES if (DTRACE_ENABLED(port_busy)) { erts_snprintf(port_str, sizeof(port_str), - "%T", erts_port[port_num].id); + "%T", prt->common.id); DTRACE1(port_busy, port_str); } #endif } else { - ErtsProcList* plp = erts_port[port_num].suspended; - erts_port_status_band_set(&erts_port[port_num], - ~ERTS_PORT_SFLG_PORT_BUSY); - erts_port[port_num].suspended = NULL; + flags = erts_smp_atomic32_read_band_acqb(&prt->sched.flags, + ~ERTS_PTS_FLG_BUSY_PORT); + if (!(flags & ERTS_PTS_FLG_BUSY_PORT)) + return; /* Already non-busy */ #ifdef USE_VM_PROBES if (DTRACE_ENABLED(port_not_busy)) { erts_snprintf(port_str, sizeof(port_str), - "%T", erts_port[port_num].id); + "%T", prt->common.id); DTRACE1(port_not_busy, port_str); } #endif - if (erts_port[port_num].dist_entry) { + if (prt->dist_entry) { /* * Processes suspended on distribution ports are * normally queued on the dist entry. */ - erts_dist_port_not_busy(&erts_port[port_num]); + erts_dist_port_not_busy(prt); } - /* - * Resume, in a round-robin fashion, all processes waiting on the port. - * - * This version submitted by Tony Rogvall. The earlier version used - * to resume the processes in order, which caused starvation of all but - * the first process. - */ + if (!(flags & ERTS_PTS_FLG_BUSY_PORT_Q)) + erts_port_resume_procs(prt); + } +} + +void +erts_port_resume_procs(Port *prt) +{ + /* + * Resume, in a round-robin fashion, all processes waiting on the port. + * + * This version submitted by Tony Rogvall. The earlier version used + * to resume the processes in order, which caused starvation of all but + * the first process. + */ + ErtsProcList *plp; + + erts_port_task_sched_lock(&prt->sched); + plp = prt->suspended; + prt->suspended = NULL; + erts_port_task_sched_unlock(&prt->sched); + + if (erts_proclist_fetch(&plp, NULL)) { - if (plp) { #ifdef USE_VM_PROBES - /* - * Hrm, for blocked dist ports, plp always seems to be NULL. - * That's not so fun. - * Well, another way to get the same info is using a D - * script to correlate an earlier process-port_blocked+pid - * event with a later process-scheduled event. That's - * subject to the multi-CPU races with how events are - * handled, but hey, that way works most of the time. - */ - if (DTRACE_ENABLED(process_port_unblocked)) { - DTRACE_CHARBUF(pid_str, 16); - ErtsProcList* plp2 = plp; - - erts_snprintf(port_str, sizeof(port_str), - "%T", erts_port[port_num]); - while (plp2 != NULL) { - erts_snprintf(pid_str, sizeof(pid_str), "%T", plp2->pid); - DTRACE2(process_port_unblocked, pid_str, port_str); - } - } -#endif - /* First proc should be resumed last */ - if (plp->next) { - erts_resume_processes(plp->next); - plp->next = NULL; + /* + * Hrm, for blocked dist ports, plp always seems to be NULL. + * That's not so fun. + * Well, another way to get the same info is using a D + * script to correlate an earlier process-port_blocked+pid + * event with a later process-scheduled event. That's + * subject to the multi-CPU races with how events are + * handled, but hey, that way works most of the time. + */ + if (DTRACE_ENABLED(process_port_unblocked)) { + DTRACE_CHARBUF(port_str, 16); + DTRACE_CHARBUF(pid_str, 16); + ErtsProcList* plp2 = plp; + + erts_snprintf(port_str, sizeof(port_str), "%T", prt->common.id); + while (plp2 != NULL) { + erts_snprintf(pid_str, sizeof(pid_str), "%T", plp2->pid); + DTRACE2(process_port_unblocked, pid_str, port_str); } - erts_resume_processes(plp); - } + } +#endif + + /* First proc should be resumed last */ + if (plp->next) { + plp->next->prev = NULL; + erts_resume_processes(plp->next); + plp->next = NULL; + } + erts_resume_processes(plp); } } void set_port_control_flags(ErlDrvPort port_num, int flags) { - - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(&erts_port[port_num])); - - erts_port[port_num].control_flags = flags; + Port *prt = erts_drvport2port_raw(port_num); + if (prt) + prt->control_flags = flags; } -int get_port_flags(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); +int get_port_flags(ErlDrvPort ix) +{ + int flags; + Port *prt; + erts_aint32_t state; + + prt = erts_drvport2port(ix, &state); + if (!prt) + return 0; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - if (prt == NULL) - return 0; + flags = 0; + if (state & ERTS_PORT_SFLG_BINARY_IO) + flags |= PORT_FLAG_BINARY; + if (state & ERTS_PORT_SFLG_LINEBUF_IO) + flags |= PORT_FLAG_LINE; - return (prt->status & ERTS_PORT_SFLG_BINARY_IO ? PORT_FLAG_BINARY : 0) - | (prt->status & ERTS_PORT_SFLG_LINEBUF_IO ? PORT_FLAG_LINE : 0); + return flags; } - void erts_raw_port_command(Port* p, byte* buf, Uint len) { int fpe_was_unmasked; @@ -2665,25 +4896,18 @@ int async_ready(Port *p, void* data) if (p) { ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(p)); - ASSERT(!(p->status & ERTS_PORT_SFLGS_DEAD)); if (p->drv_ptr->ready_async != NULL) { #ifdef USE_VM_PROBES if (DTRACE_ENABLED(driver_ready_async)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(p->connected, p) + DTRACE_FORMAT_COMMON_PID_AND_PORT(ERTS_PORT_GET_CONNECTED(p), p) DTRACE3(driver_ready_async, process_str, port_str, p->name); } #endif (*p->drv_ptr->ready_async)((ErlDrvData)p->drv_data, data); need_free = 0; -#ifdef ERTS_SMP - if (p->xports) - erts_smp_xports_unlock(p); - ASSERT(!p->xports); -#endif - } - if ((p->status & ERTS_PORT_SFLG_CLOSING) && is_port_ioq_empty(p)) { - terminate_port(p); + } + erts_port_driver_callback_epilogue(p, NULL); } return need_free; } @@ -2691,12 +4915,12 @@ int async_ready(Port *p, void* data) static void report_missing_drv_callback(Port *p, char *drv_type, char *callback) { - ErtsPortNames *pnp = erts_get_port_names(p->id); + ErtsPortNames *pnp = erts_get_port_names(p->common.id); char *unknown = "<unknown>"; char *drv_name = pnp->driver_name ? pnp->driver_name : unknown; char *prt_name = pnp->name ? pnp->name : unknown; erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); - erts_dsprintf(dsbufp, "%T: %s driver '%s' ", p->id, drv_type, drv_name); + erts_dsprintf(dsbufp, "%T: %s driver '%s' ", p->common.id, drv_type, drv_name); if (sys_strcmp(drv_name, prt_name) != 0) erts_dsprintf(dsbufp, "(%s) ", prt_name); erts_dsprintf(dsbufp, "does not implement the %s callback!\n", callback); @@ -2711,7 +4935,7 @@ erts_stale_drv_select(Eterm port, int deselect) { char *type; - ErlDrvPort drv_port = internal_port_index(port); + ErlDrvPort drv_port = (ErlDrvPort) erts_port_lookup_raw(port); ErtsPortNames *pnp = erts_get_port_names(port); erts_dsprintf_buf_t *dsbufp; @@ -2751,16 +4975,16 @@ erts_stale_drv_select(Eterm port, ErtsPortNames * erts_get_port_names(Eterm id) { + Port *prt = erts_port_lookup_raw(id); ErtsPortNames *pnp; ASSERT(is_nil(id) || is_internal_port(id)); - - if (is_not_internal_port(id)) { + + if (!prt) { pnp = erts_alloc(ERTS_ALC_T_PORT_NAMES, sizeof(ErtsPortNames)); pnp->name = NULL; pnp->driver_name = NULL; } else { - Port* prt = &erts_port[internal_port_index(id)]; int do_realloc = 1; int len = -1; size_t pnp_len = sizeof(ErtsPortNames); @@ -2776,17 +5000,10 @@ erts_get_port_names(Eterm id) pnp_len = sizeof(ErtsPortNames) + len; pnp = erts_alloc(ERTS_ALC_T_PORT_NAMES, pnp_len); } - erts_smp_port_state_lock(prt); - if (id != prt->id) { - len = nlen = 0; - name = driver_name = NULL; - } - else { - name = prt->name; - len = nlen = name ? sys_strlen(name) + 1 : 0; - driver_name = (prt->drv_ptr ? prt->drv_ptr->name : NULL); - len += driver_name ? sys_strlen(driver_name) + 1 : 0; - } + name = prt->name; + len = nlen = name ? sys_strlen(name) + 1 : 0; + driver_name = (prt->drv_ptr ? prt->drv_ptr->name : NULL); + len += driver_name ? sys_strlen(driver_name) + 1 : 0; if (len <= pnp_len - sizeof(ErtsPortNames)) { if (!name) pnp->name = NULL; @@ -2804,7 +5021,6 @@ erts_get_port_names(Eterm id) } do_realloc = 0; } - erts_smp_port_state_unlock(prt); } while (do_realloc); } return pnp; @@ -2829,11 +5045,9 @@ static void schedule_port_timeout(Port *p) * /Rickard */ ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(p)); - (void) erts_port_task_schedule(p->id, - &p->timeout_task, - ERTS_PORT_TASK_TIMEOUT, - (ErlDrvEvent) -1, - NULL); + erts_port_task_schedule(p->common.id, + &p->timeout_task, + ERTS_PORT_TASK_TIMEOUT); } ErlDrvTermData driver_mk_term_nil(void) @@ -2841,9 +5055,9 @@ ErlDrvTermData driver_mk_term_nil(void) return driver_term_nil; } -void driver_report_exit(int ix, int status) +void driver_report_exit(ErlDrvPort ix, int status) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); Eterm* hp; Eterm tuple; Process *rp; @@ -2856,7 +5070,7 @@ void driver_report_exit(int ix, int status) ERTS_SMP_CHK_NO_PROC_LOCKS; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - pid = prt->connected; + pid = ERTS_PORT_GET_CONNECTED(prt); ASSERT(is_internal_pid(pid)); rp = (scheduler @@ -2869,7 +5083,7 @@ void driver_report_exit(int ix, int status) tuple = TUPLE2(hp, am_exit_status, make_small(status)); hp += 3; - tuple = TUPLE2(hp, prt->id, tuple); + tuple = TUPLE2(hp, prt->common.id, tuple); erts_queue_message(rp, &rp_locks, bp, tuple, am_undefined #ifdef USE_VM_PROBES @@ -2882,28 +5096,6 @@ void driver_report_exit(int ix, int status) erts_smp_proc_dec_refc(rp); } - -static ERTS_INLINE int -deliver_term_check_port(ErlDrvPort drvport) -{ - int res; - int ix = (int) drvport; - if (ix < 0 || erts_max_ports <= ix) - res = -1; /* invalid */ - else { - Port* prt = &erts_port[ix]; - erts_smp_port_state_lock(prt); - if (!(prt->status & ERTS_PORT_SFLGS_INVALID_LOOKUP)) - res = 1; /* ok */ - else if (prt->status & ERTS_PORT_SFLG_CLOSING) - res = 0; /* closing */ - else - res = -1; /* invalid (dead) */ - erts_smp_port_state_unlock(prt); - } - return res; -} - #define ERTS_B2T_STATES_DEF_STATES_SZ 5 #define ERTS_B2T_STATES_DEF_STATES_INC 100 @@ -2991,10 +5183,7 @@ cleanup_b2t_states(struct b2t_states__ *b2tsp) */ static int -driver_deliver_term(ErlDrvPort port, - Eterm to, - ErlDrvTermData* data, - int len) +driver_deliver_term(Eterm to, ErlDrvTermData* data, int len) { #define ERTS_DDT_FAIL do { res = -1; goto done; } while (0) Uint need = 0; @@ -3196,11 +5385,8 @@ driver_deliver_term(ErlDrvPort port, b2t.ix = 0; /* - * The term is OK. Go ahead and validate the port and process. + * The term is OK. Go ahead and validate the process. */ - res = deliver_term_check_port(port); - if (res <= 0) - goto done; /* * Increase refc on proc if done from a non-scheduler thread. @@ -3470,25 +5656,115 @@ driver_deliver_term(ErlDrvPort port, #undef ERTS_DDT_FAIL } +static ERTS_INLINE int +deliver_term_check_port(ErlDrvTermData port_id, Eterm *connected_p) +{ +#ifdef ERTS_SMP + ErtsThrPrgrDelayHandle dhndl = erts_thr_progress_unmanaged_delay(); +#endif + Port *prt = erts_port_lookup_raw((Eterm) port_id); + erts_aint32_t state = erts_atomic32_read_nob(&prt->state); + if (connected_p) { +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + ETHR_MEMBAR(ETHR_LoadLoad); +#endif + *connected_p = ERTS_PORT_GET_CONNECTED(prt); + } +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) { + erts_thr_progress_unmanaged_continue(dhndl); + ETHR_MEMBAR(ETHR_LoadLoad|ETHR_LoadStore); + } +#endif + ERTS_SMP_LC_ASSERT(dhndl == ERTS_THR_PRGR_DHANDLE_MANAGED + ? erts_lc_is_port_locked(prt) + : !erts_lc_is_port_locked(prt)); + return ((state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) + ? -1 + : ((state & ERTS_PORT_SFLG_CLOSING) ? 0 : 1)); +} + +int erl_drv_output_term(ErlDrvTermData port_id, ErlDrvTermData* data, int len) +{ + /* May be called from arbitrary thread */ + Eterm connected; + int res = deliver_term_check_port(port_id, &connected); + if (res <= 0) + return res; + return driver_deliver_term(connected, data, len); +} +/* + * driver_output_term() is deprecated, and has been scheduled for + * removal in OTP-R17. It is replaced by erl_drv_output_term() + * above. + */ int -driver_output_term(ErlDrvPort ix, ErlDrvTermData* data, int len) +driver_output_term(ErlDrvPort drvport, ErlDrvTermData* data, int len) { - Port* prt = erts_drvport2port(ix); + erts_aint32_t state; + Port* prt; ERTS_SMP_CHK_NO_PROC_LOCKS; + /* NOTE! It *not* safe to access 'drvport' from unmanaged threads. */ + prt = erts_drvport2port(drvport, &state); + if (!prt) + return -1; /* invalid (dead) */ + ERTS_SMP_CHK_NO_PROC_LOCKS; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - - if (prt == NULL) + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) return -1; - return driver_deliver_term(ix, prt->connected, data, len); + else if (state & ERTS_PORT_SFLG_CLOSING) + return 0; + + return driver_deliver_term(ERTS_PORT_GET_CONNECTED(prt), data, len); } +int erl_drv_send_term(ErlDrvTermData port_id, + ErlDrvTermData to, + ErlDrvTermData* data, + int len) +{ + /* May be called from arbitrary thread */ + int res = deliver_term_check_port(port_id, NULL); + if (res <= 0) + return res; + return driver_deliver_term(to, data, len); +} +/* + * driver_send_term() is deprecated, and has been scheduled for + * removal in OTP-R17. It is replaced by erl_drv_send_term() above. + */ int -driver_send_term(ErlDrvPort ix, ErlDrvTermData to, ErlDrvTermData* data, int len) +driver_send_term(ErlDrvPort drvport, + ErlDrvTermData to, + ErlDrvTermData* data, + int len) { - return driver_deliver_term(ix, to, data, len); + /* + * NOTE! It is *not* safe to access the 'drvport' parameter + * from unmanaged threads. Also note that it is impossible + * to make this access safe without using a less efficient + * internal data representation for ErlDrvPort. + */ + ERTS_SMP_CHK_NO_PROC_LOCKS; +#ifdef ERTS_SMP + if (erts_thr_progress_is_managed_thread()) +#endif + { + erts_aint32_t state; + Port* prt = erts_drvport2port(drvport, &state); + if (!prt) + return -1; /* invalid (dead) */ + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) + return -1; + else if (state & ERTS_PORT_SFLG_CLOSING) + return 0; + } + return driver_deliver_term(to, data, len); } @@ -3500,26 +5776,27 @@ driver_send_term(ErlDrvPort ix, ErlDrvTermData to, ErlDrvTermData* data, int len int driver_output_binary(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, ErlDrvBinary* bin, ErlDrvSizeT offs, ErlDrvSizeT len) { - Port* prt = erts_drvport2port(ix); + erts_aint32_t state; + Port* prt = erts_drvport2port(ix, &state); ERTS_SMP_CHK_NO_PROC_LOCKS; if (prt == NULL) return -1; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - if (prt->status & ERTS_PORT_SFLG_CLOSING) + if (state & ERTS_PORT_SFLG_CLOSING) return 0; prt->bytes_in += (hlen + len); erts_smp_atomic_add_nob(&erts_bytes_in, (erts_aint_t) (hlen + len)); - if (prt->status & ERTS_PORT_SFLG_DISTRIBUTION) { + if (state & ERTS_PORT_SFLG_DISTRIBUTION) { return erts_net_message(prt, prt->dist_entry, (byte*) hbuf, hlen, (byte*) (bin->orig_bytes+offs), len); } else - deliver_bin_message(prt, prt->connected, + deliver_bin_message(prt, ERTS_PORT_GET_CONNECTED(prt), hbuf, hlen, bin, offs, len); return 0; } @@ -3534,7 +5811,8 @@ int driver_output_binary(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, int driver_output2(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, char* buf, ErlDrvSizeT len) { - Port* prt = erts_drvport2port(ix); + erts_aint32_t state; + Port* prt = erts_drvport2port(ix, &state); ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -3543,12 +5821,12 @@ int driver_output2(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - if (prt->status & ERTS_PORT_SFLG_CLOSING) + if (state & ERTS_PORT_SFLG_CLOSING) return 0; prt->bytes_in += (hlen + len); erts_smp_atomic_add_nob(&erts_bytes_in, (erts_aint_t) (hlen + len)); - if (prt->status & ERTS_PORT_SFLG_DISTRIBUTION) { + if (state & ERTS_PORT_SFLG_DISTRIBUTION) { if (len == 0) return erts_net_message(prt, prt->dist_entry, @@ -3560,10 +5838,12 @@ int driver_output2(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, (byte*) hbuf, hlen, (byte*) buf, len); } - else if(prt->status & ERTS_PORT_SFLG_LINEBUF_IO) - deliver_linebuf_message(prt, prt->connected, hbuf, hlen, buf, len); + else if (state & ERTS_PORT_SFLG_LINEBUF_IO) + deliver_linebuf_message(prt, state, ERTS_PORT_GET_CONNECTED(prt), + hbuf, hlen, buf, len); else - deliver_read_message(prt, prt->connected, hbuf, hlen, buf, len, 0); + deliver_read_message(prt, state, ERTS_PORT_GET_CONNECTED(prt), + hbuf, hlen, buf, len, 0); return 0; } @@ -3584,6 +5864,7 @@ int driver_outputv(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, SysIOVec* iov; ErlDrvBinary** binv; Port* prt; + erts_aint32_t state; ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -3596,13 +5877,13 @@ int driver_outputv(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, if (hlen < 0) hlen = 0; - prt = erts_drvport2port(ix); + prt = erts_drvport2port(ix, &state); if (prt == NULL) return -1; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - if (prt->status & ERTS_PORT_SFLG_CLOSING) + if (state & ERTS_PORT_SFLG_CLOSING) return 0; /* size > 0 ! */ @@ -3627,7 +5908,8 @@ int driver_outputv(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, /* XXX handle distribution !!! */ prt->bytes_in += (hlen + size); erts_smp_atomic_add_nob(&erts_bytes_in, (erts_aint_t) (hlen + size)); - deliver_vec_message(prt, prt->connected, hbuf, hlen, binv, iov, n, size); + deliver_vec_message(prt, ERTS_PORT_GET_CONNECTED(prt), hbuf, hlen, + binv, iov, n, size); return 0; } @@ -3738,8 +6020,7 @@ ErlDrvBinary* driver_realloc_binary(ErlDrvBinary* bin, ErlDrvSizeT size) } -void driver_free_binary(dbin) -ErlDrvBinary* dbin; +void driver_free_binary(ErlDrvBinary* dbin) { Binary *bin; if (!dbin) { @@ -3835,6 +6116,7 @@ static ERTS_INLINE void pdl_destroy(ErlDrvPDL pdl) { ERTS_LC_ASSERT(driver_pdl_get_refc(pdl) == 0); erts_mtx_destroy(&pdl->mtx); + erts_port_dec_refc(pdl->prt); erts_free(ERTS_ALC_T_PORT_DATA_LOCK, pdl); } @@ -3872,16 +6154,18 @@ ErlDrvPDL driver_pdl_create(ErlDrvPort dp) { ErlDrvPDL pdl; - Port *pp = erts_drvport2port(dp); + Port *pp = erts_drvport2port(dp, NULL); if (!pp || pp->port_data_lock) return NULL; pdl = erts_alloc(ERTS_ALC_T_PORT_DATA_LOCK, sizeof(struct erl_drv_port_data_lock)); erts_mtx_init(&pdl->mtx, "port_data_lock"); pdl_init_refc(pdl); + erts_port_inc_refc(pp); + pdl->prt = pp; pp->port_data_lock = pdl; #ifdef HARDDEBUG - erts_fprintf(stderr, "driver_pdl_create(%T) -> 0x%08X\r\n",pp->id,(unsigned) pdl); + erts_fprintf(stderr, "driver_pdl_create(%T) -> 0x%08X\r\n",pp->common.id,(unsigned) pdl); #endif return pdl; } @@ -4319,33 +6603,33 @@ static ERTS_INLINE void drv_cancel_timer(Port *prt) { #ifdef ERTS_SMP - erts_cancel_smp_ptimer(prt->ptimer); + erts_cancel_smp_ptimer(prt->common.u.alive.ptimer); #else - erts_cancel_timer(&prt->tm); + erts_cancel_timer(&prt->common.u.alive.tm); #endif if (erts_port_task_is_scheduled(&prt->timeout_task)) - erts_port_task_abort(prt->id, &prt->timeout_task); + erts_port_task_abort(&prt->timeout_task); } int driver_set_timer(ErlDrvPort ix, unsigned long t) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); ERTS_SMP_CHK_NO_PROC_LOCKS; if (prt == NULL) return -1; - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + if (prt->drv_ptr->timeout == NULL) return -1; drv_cancel_timer(prt); #ifdef ERTS_SMP - erts_create_smp_ptimer(&prt->ptimer, - prt->id, + erts_create_smp_ptimer(&prt->common.u.alive.ptimer, + prt->common.id, (ErlTimeoutProc) schedule_port_timeout, t); #else - erts_set_timer(&prt->tm, + erts_set_timer(&prt->common.u.alive.tm, (ErlTimeoutProc) schedule_port_timeout, NULL, prt, @@ -4356,7 +6640,7 @@ int driver_set_timer(ErlDrvPort ix, unsigned long t) int driver_cancel_timer(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); if (prt == NULL) return -1; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); @@ -4368,7 +6652,7 @@ int driver_cancel_timer(ErlDrvPort ix) int driver_read_timer(ErlDrvPort ix, unsigned long* t) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -4376,9 +6660,11 @@ driver_read_timer(ErlDrvPort ix, unsigned long* t) return -1; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); #ifdef ERTS_SMP - *t = prt->ptimer ? erts_time_left(&prt->ptimer->timer.tm) : 0; + *t = (prt->common.u.alive.ptimer + ? erts_time_left(&prt->common.u.alive.ptimer->timer.tm) + : 0); #else - *t = erts_time_left(&prt->tm); + *t = erts_time_left(&prt->common.u.alive.tm); #endif return 0; } @@ -4429,8 +6715,8 @@ static int do_driver_monitor_process(Port *prt, } ref = erts_make_ref_in_buffer(buf); - erts_add_monitor(&(prt->monitors), MON_ORIGIN, ref, rp->id, NIL); - erts_add_monitor(&(rp->monitors), MON_TARGET, ref, prt->id, NIL); + erts_add_monitor(&ERTS_P_MONITORS(prt), MON_ORIGIN, ref, rp->common.id, NIL); + erts_add_monitor(&ERTS_P_MONITORS(rp), MON_TARGET, ref, prt->common.id, NIL); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); ref_to_driver_monitor(ref,monitor); @@ -4440,31 +6726,24 @@ static int do_driver_monitor_process(Port *prt, /* * This can be called from a non scheduler thread iff a port_data_lock exists */ -int driver_monitor_process(ErlDrvPort port, +int driver_monitor_process(ErlDrvPort drvport, ErlDrvTermData process, ErlDrvMonitor *monitor) { Port *prt; int ret; - Uint32 status; + erts_aint32_t state; +#if !HEAP_ON_C_STACK || (defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK)) ErtsSchedulerData *sched = erts_get_scheduler_data(); - int ix = (int) port; - if (ix < 0 || erts_max_ports <= ix) { - return -1; - } - prt = &erts_port[ix]; +#endif + + prt = erts_thr_drvport2port_raw(drvport); DRV_MONITOR_LOCK_PDL(prt); - if (sched) { - status = erts_port[ix].status; - } else { - erts_smp_port_state_lock(prt); - status = erts_port[ix].status; - erts_smp_port_state_unlock(prt); - } + state = erts_atomic32_read_nob(&prt->state); - if (status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { DRV_MONITOR_UNLOCK_PDL(prt); return -1; } @@ -4502,7 +6781,7 @@ static int do_driver_demonitor_process(Port *prt, Eterm *buf, memcpy(buf,monitor,sizeof(Eterm)*REF_THING_SIZE); ref = make_internal_ref(buf); - mon = erts_lookup_monitor(prt->monitors, ref); + mon = erts_lookup_monitor(ERTS_P_MONITORS(prt), ref); if (mon == NULL) { return 1; } @@ -4514,13 +6793,13 @@ static int do_driver_demonitor_process(Port *prt, Eterm *buf, to, ERTS_PROC_LOCK_LINK, ERTS_P2P_FLG_ALLOW_OTHER_X); - mon = erts_remove_monitor(&(prt->monitors), ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(prt), ref); if (mon) { erts_destroy_monitor(mon); } if (rp) { ErtsMonitor *rmon; - rmon = erts_remove_monitor(&(rp->monitors), ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), ref); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); if (rmon != NULL) { erts_destroy_monitor(rmon); @@ -4529,30 +6808,23 @@ static int do_driver_demonitor_process(Port *prt, Eterm *buf, return 0; } -int driver_demonitor_process(ErlDrvPort port, +int driver_demonitor_process(ErlDrvPort drvport, const ErlDrvMonitor *monitor) { Port *prt; int ret; - Uint32 status; + erts_aint32_t state; +#if !HEAP_ON_C_STACK || (defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK)) ErtsSchedulerData *sched = erts_get_scheduler_data(); - int ix = (int) port; - if (ix < 0 || erts_max_ports <= ix) { - return -1; - } - prt = &erts_port[ix]; +#endif + + prt = erts_thr_drvport2port_raw(drvport); DRV_MONITOR_LOCK_PDL(prt); - if (sched) { - status = erts_port[ix].status; - } else { - erts_smp_port_state_lock(prt); - status = erts_port[ix].status; - erts_smp_port_state_unlock(prt); - } + state = erts_atomic32_read_nob(&prt->state); - if (status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { DRV_MONITOR_UNLOCK_PDL(prt); return -1; } @@ -4588,7 +6860,7 @@ static ErlDrvTermData do_driver_get_monitored_process(Port *prt, Eterm *buf, memcpy(buf,monitor,sizeof(Eterm)*REF_THING_SIZE); ref = make_internal_ref(buf); - mon = erts_lookup_monitor(prt->monitors, ref); + mon = erts_lookup_monitor(ERTS_P_MONITORS(prt), ref); if (mon == NULL) { return driver_term_nil; } @@ -4599,30 +6871,22 @@ static ErlDrvTermData do_driver_get_monitored_process(Port *prt, Eterm *buf, } -ErlDrvTermData driver_get_monitored_process(ErlDrvPort port, +ErlDrvTermData driver_get_monitored_process(ErlDrvPort drvport, const ErlDrvMonitor *monitor) { Port *prt; ErlDrvTermData ret; - Uint32 status; + erts_aint32_t state; +#if !HEAP_ON_C_STACK || (defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK)) ErtsSchedulerData *sched = erts_get_scheduler_data(); - int ix = (int) port; - if (ix < 0 || erts_max_ports <= ix) { - return driver_term_nil; - } - prt = &erts_port[ix]; +#endif - DRV_MONITOR_LOCK_PDL(prt); + prt = erts_thr_drvport2port_raw(drvport); - if (sched) { - status = erts_port[ix].status; - } else { - erts_smp_port_state_lock(prt); - status = erts_port[ix].status; - erts_smp_port_state_unlock(prt); - } + DRV_MONITOR_LOCK_PDL(prt); - if (status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { + state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { DRV_MONITOR_UNLOCK_PDL(prt); return driver_term_nil; } @@ -4667,7 +6931,7 @@ void erts_fire_port_monitor(Port *prt, Eterm ref) ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); ASSERT(prt->drv_ptr != NULL); DRV_MONITOR_LOCK_PDL(prt); - if (erts_lookup_monitor(prt->monitors,ref) == NULL) { + if (erts_lookup_monitor(ERTS_P_MONITORS(prt), ref) == NULL) { DRV_MONITOR_UNLOCK_PDL(prt); return; } @@ -4677,7 +6941,7 @@ void erts_fire_port_monitor(Port *prt, Eterm ref) DRV_MONITOR_UNLOCK_PDL(prt); #ifdef USE_VM_PROBES if (DTRACE_ENABLED(driver_process_exit)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(prt->connected, prt) + DTRACE_FORMAT_COMMON_PID_AND_PORT(ERTS_PORT_GET_CONNECTED(prt), prt) DTRACE3(driver_process_exit, process_str, port_str, prt->name); } #endif @@ -4686,7 +6950,7 @@ void erts_fire_port_monitor(Port *prt, Eterm ref) erts_unblock_fpe(fpe_was_unmasked); DRV_MONITOR_LOCK_PDL(prt); /* remove monitor *after* callback */ - rmon = erts_remove_monitor(&(prt->monitors),ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(prt), ref); DRV_MONITOR_UNLOCK_PDL(prt); if (rmon) { erts_destroy_monitor(rmon); @@ -4697,7 +6961,8 @@ void erts_fire_port_monitor(Port *prt, Eterm ref) static int driver_failure_term(ErlDrvPort ix, Eterm term, int eof) { - Port* prt = erts_drvport2port(ix); + erts_aint32_t state; + Port* prt = erts_drvport2port(ix, &state); ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -4705,19 +6970,19 @@ driver_failure_term(ErlDrvPort ix, Eterm term, int eof) return -1; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); if (eof) - flush_linebuf_messages(prt); - if (prt->status & ERTS_PORT_SFLG_CLOSING) { + flush_linebuf_messages(prt, state); + if (state & ERTS_PORT_SFLG_CLOSING) { terminate_port(prt); - } else if (eof && (prt->status & ERTS_PORT_SFLG_SOFT_EOF)) { - deliver_result(prt->id, prt->connected, am_eof); + } else if (eof && (state & ERTS_PORT_SFLG_SOFT_EOF)) { + deliver_result(prt->common.id, ERTS_PORT_GET_CONNECTED(prt), am_eof); } else { - /* XXX UGLY WORK AROUND, Let do_exit_port terminate the port */ + /* XXX UGLY WORK AROUND, Let erts_deliver_port_exit() terminate the port */ if (prt->port_data_lock) driver_pdl_lock(prt->port_data_lock); prt->ioq.size = 0; if (prt->port_data_lock) driver_pdl_unlock(prt->port_data_lock); - erts_do_exit_port(prt, prt->id, eof ? am_normal : term); + erts_deliver_port_exit(prt, prt->common.id, eof ? am_normal : term, 0); } return 0; } @@ -4730,23 +6995,23 @@ driver_failure_term(ErlDrvPort ix, Eterm term, int eof) */ int driver_exit(ErlDrvPort ix, int err) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); Process* rp; ErtsLink *lnk, *rlnk = NULL; + Eterm connected; ERTS_SMP_CHK_NO_PROC_LOCKS; if (prt == NULL) return -1; - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - - rp = erts_pid2proc(NULL, 0, prt->connected, ERTS_PROC_LOCK_LINK); + connected = ERTS_PORT_GET_CONNECTED(prt); + rp = erts_pid2proc(NULL, 0, connected, ERTS_PROC_LOCK_LINK); if (rp) { - rlnk = erts_remove_link(&(rp->nlinks),prt->id); + rlnk = erts_remove_link(&ERTS_P_LINKS(rp),prt->common.id); } - lnk = erts_remove_link(&(prt->nlinks),prt->connected); + lnk = erts_remove_link(&ERTS_P_LINKS(prt), connected); #ifdef ERTS_SMP if (rp) @@ -4803,24 +7068,24 @@ ErlDrvTermData driver_mk_atom(char* string) ErlDrvTermData driver_mk_port(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - return (ErlDrvTermData) prt->id; + return (ErlDrvTermData) prt->common.id; } ErlDrvTermData driver_connected(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); ERTS_SMP_CHK_NO_PROC_LOCKS; if (prt == NULL) return NIL; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - return prt->connected; + return ERTS_PORT_GET_CONNECTED(prt); } ErlDrvTermData driver_caller(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); ERTS_SMP_CHK_NO_PROC_LOCKS; if (prt == NULL) return NIL; @@ -4830,25 +7095,25 @@ ErlDrvTermData driver_caller(ErlDrvPort ix) int driver_lock_driver(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); DE_Handle* dh; ERTS_SMP_CHK_NO_PROC_LOCKS; - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rwlock(&erts_driver_list_lock); if (prt == NULL) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); return -1; } ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); if ((dh = (DE_Handle*)prt->drv_ptr->handle ) == NULL) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); return -1; } erts_ddll_lock_driver(dh, prt->drv_ptr->name); - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); return 0; } @@ -4858,7 +7123,7 @@ static int maybe_lock_driver_list(void) void *rec_lock; rec_lock = erts_smp_tsd_get(driver_list_lock_status_key); if (rec_lock == 0) { - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rwlock(&erts_driver_list_lock); return 1; } return 0; @@ -4866,7 +7131,7 @@ static int maybe_lock_driver_list(void) static void maybe_unlock_driver_list(int doit) { if (doit) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } } /* @@ -5035,7 +7300,7 @@ no_event_callback(ErlDrvData drv_data, ErlDrvEvent event, ErlDrvEventData event_ { Port *prt = get_current_port(); report_missing_drv_callback(prt, "Event", "event()"); - driver_event((ErlDrvPort) internal_port_index(prt->id), event, NULL); + driver_event((ErlDrvPort) prt, event, NULL); } static void @@ -5043,7 +7308,7 @@ no_ready_input_callback(ErlDrvData drv_data, ErlDrvEvent event) { Port *prt = get_current_port(); report_missing_drv_callback(prt, "Input", "ready_input()"); - driver_select((ErlDrvPort) internal_port_index(prt->id), event, + driver_select((ErlDrvPort) prt, event, (ERL_DRV_READ | ERL_DRV_USE_NO_CALLBACK), 0); } @@ -5052,7 +7317,7 @@ no_ready_output_callback(ErlDrvData drv_data, ErlDrvEvent event) { Port *prt = get_current_port(); report_missing_drv_callback(prt, "Output", "ready_output()"); - driver_select((ErlDrvPort) internal_port_index(prt->id), event, + driver_select((ErlDrvPort) prt, event, (ERL_DRV_WRITE | ERL_DRV_USE_NO_CALLBACK), 0); } @@ -5087,13 +7352,13 @@ init_driver(erts_driver_t *drv, ErlDrvEntry *de, DE_Handle *handle) drv->lock = NULL; else { drv->lock = erts_alloc(ERTS_ALC_T_DRIVER_LOCK, - sizeof(erts_smp_mtx_t)); - erts_smp_mtx_init_x(drv->lock, - "driver_lock", + sizeof(erts_mtx_t)); + erts_mtx_init_x(drv->lock, + "driver_lock", #if defined(ERTS_ENABLE_LOCK_CHECK) || defined(ERTS_ENABLE_LOCK_COUNT) - am_atom_put(drv->name, sys_strlen(drv->name)) + am_atom_put(drv->name, sys_strlen(drv->name)) #else - NIL + NIL #endif ); } @@ -5165,7 +7430,7 @@ int erts_add_driver_entry(ErlDrvEntry *de, DE_Handle *handle, int driver_list_lo int res; if (!driver_list_locked) { - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rwlock(&erts_driver_list_lock); } dp->next = driver_list; @@ -5194,7 +7459,7 @@ int erts_add_driver_entry(ErlDrvEntry *de, DE_Handle *handle, int driver_list_lo if (!driver_list_locked) { erts_smp_tsd_set(driver_list_lock_status_key, NULL); - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } return res; } @@ -5207,7 +7472,7 @@ int remove_driver_entry(ErlDrvEntry *drv) rec_lock = erts_smp_tsd_get(driver_list_lock_status_key); if (rec_lock == NULL) { - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rwlock(&erts_driver_list_lock); } dp = driver_list; while (dp && dp->entry != drv) @@ -5215,7 +7480,7 @@ int remove_driver_entry(ErlDrvEntry *drv) if (dp) { if (dp->handle) { if (rec_lock == NULL) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } return -1; } @@ -5229,12 +7494,12 @@ int remove_driver_entry(ErlDrvEntry *drv) } erts_destroy_driver(dp); if (rec_lock == NULL) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } return 1; } if (rec_lock == NULL) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } return 0; } @@ -5264,18 +7529,22 @@ erl_drv_getenv(char *key, char *value, size_t *value_size) * - uses the fact that heart_port is registered when starting heart */ -Port *erts_get_heart_port() { +Port *erts_get_heart_port(void) +{ + int ix, max = erts_ptab_max(&erts_port); - Port* port; - Uint ix; + for (ix = 0; ix < max; ix++) { + struct reg_proc *reg; + Port *port = erts_pix2port(ix); - for(ix = 0; ix < erts_max_ports; ix++) { - port = &erts_port[ix]; + if (!port) + continue; /* only examine undead or alive ports */ - if (port->status & ERTS_PORT_SFLGS_DEAD) + if (erts_atomic32_read_nob(&port->state) & ERTS_PORT_SFLGS_DEAD) continue; /* immediate atom compare */ - if (port->reg && port->reg->name == am_heart_port) { + reg = port->common.u.alive.reg; + if (reg && reg->name == am_heart_port) { return port; } } diff --git a/erts/emulator/beam/register.c b/erts/emulator/beam/register.c index c02872ef80..757e2800e6 100644 --- a/erts/emulator/beam/register.c +++ b/erts/emulator/beam/register.c @@ -175,14 +175,14 @@ int erts_register_name(Process *c_p, Eterm name, Eterm id) if (is_not_atom(name) || name == am_undefined) return res; - if (c_p->id == id) /* A very common case I think... */ + if (c_p->common.id == id) /* A very common case I think... */ proc = c_p; else { if (is_not_internal_pid(id) && is_not_internal_port(id)) return res; erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_MAIN); if (is_internal_port(id)) { - port = erts_id2port(id, NULL, 0); + port = erts_id2port(id); if (!port) goto done; } @@ -204,7 +204,7 @@ int erts_register_name(Process *c_p, Eterm name, Eterm id) r.p = proc; if (!proc) goto done; - if (proc->reg) + if (proc->common.u.alive.reg) goto done; r.pt = NULL; } @@ -212,7 +212,7 @@ int erts_register_name(Process *c_p, Eterm name, Eterm id) ASSERT(!INVALID_PORT(port, id)); ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(port)); r.pt = port; - if (r.pt->reg) + if (r.pt->common.u.alive.reg) goto done; r.p = NULL; } @@ -224,23 +224,24 @@ int erts_register_name(Process *c_p, Eterm name, Eterm id) if (IS_TRACED_FL(proc, F_TRACE_PROCS)) { trace_proc(c_p, proc, am_register, name); } - proc->reg = rp; + proc->common.u.alive.reg = rp; } else if (port && rp->pt == port) { if (IS_TRACED_FL(port, F_TRACE_PORTS)) { trace_port(port, am_register, name); } - port->reg = rp; + port->common.u.alive.reg = rp; } - if ((rp->p && rp->p->id == id) || (rp->pt && rp->pt->id == id)) { + if ((rp->p && rp->p->common.id == id) + || (rp->pt && rp->pt->common.id == id)) { res = 1; } done: reg_write_unlock(); if (port) - erts_smp_port_unlock(port); + erts_port_release(port); if (c_p != proc) { if (proc) erts_smp_proc_unlock(proc, ERTS_PROC_LOCK_MAIN); @@ -291,9 +292,9 @@ erts_whereis_name_to_id(Process *c_p, Eterm name) * is read only. */ if (rp->p) - res = rp->p->id; + res = rp->p->common.id; else if (rp->pt) - res = rp->pt->id; + res = rp->pt->common.id; break; } b = b->next; @@ -396,28 +397,26 @@ erts_whereis_name(Process *c_p, if (!rp || !rp->pt) *port = NULL; else { -#ifndef ERTS_SMP - erts_smp_atomic_inc_nob(&rp->pt->refc); -#else +#ifdef ERTS_SMP if (pending_port == rp->pt) pending_port = NULL; else { if (pending_port) { /* Ahh! Registered port changed while reg lock was unlocked... */ - erts_smp_port_unlock(pending_port); + erts_port_release(pending_port); pending_port = NULL; } if (erts_smp_port_trylock(rp->pt) == EBUSY) { - Eterm id = rp->pt->id; /* id read only... */ + Eterm id = rp->pt->common.id; /* id read only... */ /* Unlock all locks, acquire port lock, and restart... */ if (current_c_p_locks) { erts_smp_proc_unlock(c_p, current_c_p_locks); current_c_p_locks = 0; } reg_read_unlock(); - pending_port = erts_id2port(id, NULL, 0); + pending_port = erts_id2port(id); goto restart; } } @@ -431,7 +430,7 @@ erts_whereis_name(Process *c_p, if (c_p && !current_c_p_locks) erts_smp_proc_lock(c_p, c_p_locks); if (pending_port) - erts_smp_port_unlock(pending_port); + erts_port_release(pending_port); #endif reg_read_unlock(); @@ -493,8 +492,8 @@ int erts_unregister_name(Process *c_p, current_c_p_locks = c_p_locks; } #endif - if (c_p->reg) { - r.name = c_p->reg->name; + if (c_p->common.u.alive.reg) { + r.name = c_p->common.u.alive.reg->name; } else { /* Name got unregistered while main lock was released */ res = 0; @@ -505,24 +504,22 @@ int erts_unregister_name(Process *c_p, if ((rp = (RegProc*) hash_get(&process_reg, (void*) &r)) != NULL) { if (rp->pt) { if (port != rp->pt) { -#ifndef ERTS_SMP - erts_smp_atomic_inc_nob(&rp->pt->refc); -#else +#ifdef ERTS_SMP if (port) { ASSERT(port != c_prt); - erts_smp_port_unlock(port); + erts_port_release(port); port = NULL; } if (erts_smp_port_trylock(rp->pt) == EBUSY) { - Eterm id = rp->pt->id; /* id read only... */ + Eterm id = rp->pt->common.id; /* id read only... */ /* Unlock all locks, acquire port lock, and restart... */ if (current_c_p_locks) { erts_smp_proc_unlock(c_p, current_c_p_locks); current_c_p_locks = 0; } reg_write_unlock(); - port = erts_id2port(id, NULL, 0); + port = erts_id2port(id); goto restart; } #endif @@ -532,7 +529,7 @@ int erts_unregister_name(Process *c_p, ASSERT(rp->pt == port); ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(port)); - rp->pt->reg = NULL; + rp->pt->common.u.alive.reg = NULL; if (IS_TRACED_FL(port, F_TRACE_PORTS)) { trace_port(port, am_unregister, r.name); @@ -549,7 +546,7 @@ int erts_unregister_name(Process *c_p, ERTS_PROC_LOCK_MAIN); current_c_p_locks = c_p_locks; #endif - rp->p->reg = NULL; + rp->p->common.u.alive.reg = NULL; if (IS_TRACED_FL(rp->p, F_TRACE_PROCS)) { trace_proc(c_p, rp->p, am_unregister, r.name); } @@ -568,7 +565,7 @@ int erts_unregister_name(Process *c_p, reg_write_unlock(); if (c_prt != port) { if (port) { - erts_smp_port_unlock(port); + erts_port_release(port); } if (c_prt) { erts_smp_port_lock(c_prt); diff --git a/erts/emulator/beam/register.h b/erts/emulator/beam/register.h index 38e8cfbf28..7170463375 100644 --- a/erts/emulator/beam/register.h +++ b/erts/emulator/beam/register.h @@ -24,26 +24,19 @@ #ifndef __REGPROC_H__ #define __REGPROC_H__ -#ifndef __SYS_H__ #include "sys.h" -#endif - -#ifndef __HASH_H__ #include "hash.h" -#endif - -#ifndef __PROCESS_H__ #include "erl_process.h" -#endif - -struct port; +#define ERL_PORT_GET_PORT_TYPE_ONLY__ +#include "erl_port.h" +#undef ERL_PORT_GET_PORT_TYPE_ONLY__ typedef struct reg_proc { HashBucket bucket; /* MUST BE LOCATED AT TOP OF STRUCT!!! */ Process *p; /* The process registered (only one of this and 'pt' is non-NULL */ - struct port *pt; /* The port registered */ + Port *pt; /* The port registered */ Eterm name; /* Atom name */ } RegProc; @@ -55,12 +48,12 @@ int erts_register_name(Process *, Eterm, Eterm); Eterm erts_whereis_name_to_id(Process *, Eterm); void erts_whereis_name(Process *, ErtsProcLocks, Eterm, Process**, ErtsProcLocks, int, - struct port**); + Port**); Process *erts_whereis_process(Process *, ErtsProcLocks, Eterm, ErtsProcLocks, int); -int erts_unregister_name(Process *, ErtsProcLocks, struct port *, Eterm); +int erts_unregister_name(Process *, ErtsProcLocks, Port *, Eterm); #endif diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h index 8957bb5bde..898a30b010 100644 --- a/erts/emulator/beam/sys.h +++ b/erts/emulator/beam/sys.h @@ -116,6 +116,16 @@ typedef ERTS_SYS_FD_TYPE ErtsSysFdType; # define ERTS_DECLARE_DUMMY(X) X #endif +#if !defined(__func__) +# if !defined(__STDC_VERSION__) || __STDC_VERSION__ < 199901L +# if !defined(__GNUC__) || __GNUC__ < 2 +# define __func__ "[unknown_function]" +# else +# define __func__ __FUNCTION__ +# endif +# endif +#endif + #if defined(DEBUG) || defined(ERTS_ENABLE_LOCK_CHECK) # undef ERTS_CAN_INLINE # define ERTS_CAN_INLINE 0 @@ -231,9 +241,11 @@ void erl_assert_error(char* expr, char* file, int line); #if SIZEOF_VOID_P == 8 #undef ARCH_32 #define ARCH_64 +#define ERTS_SIZEOF_TERM 8 #elif SIZEOF_VOID_P == 4 #define ARCH_32 #undef ARCH_64 +#define ERTS_SIZEOF_TERM 4 #else #error Neither 32 nor 64 bit architecture #endif @@ -241,6 +253,8 @@ void erl_assert_error(char* expr, char* file, int line); # define HALFWORD_HEAP 1 # define HALFWORD_ASSERT 0 # define ASSERT_HALFWORD(COND) ASSERT(COND) +# undef ERTS_SIZEOF_TERM +# define ERTS_SIZEOF_TERM 4 #else # define HALFWORD_HEAP 0 # define HALFWORD_ASSERT 0 @@ -367,6 +381,27 @@ typedef unsigned char byte; #error 64-bit architecture, but no appropriate type to use for Uint64 and Sint64 found #endif +#ifdef WORDS_BIGENDIAN +# define ERTS_HUINT_HVAL_HIGH 0 +# define ERTS_HUINT_HVAL_LOW 1 +#else +# define ERTS_HUINT_HVAL_HIGH 1 +# define ERTS_HUINT_HVAL_LOW 0 +#endif +#if ERTS_SIZEOF_TERM == 8 +typedef union { + Uint val; + Uint32 hval[2]; +} HUint; +#elif ERTS_SIZEOF_TERM == 4 +typedef union { + Uint val; + Uint16 hval[2]; +} HUint; +#else +#error "Unsupported size of term" +#endif + # define ERTS_EXTRA_DATA_ALIGN_SZ(X) \ (((size_t) 8) - (((size_t) (X)) & ((size_t) 7))) @@ -505,6 +540,10 @@ __decl_noreturn void __noreturn erl_exit(int n, char*, ...); #define ERTS_ABORT_EXIT (INT_MIN + 1) /* no crash dump; only abort() */ #define ERTS_DUMP_EXIT (INT_MIN + 2) /* crash dump; then exit() */ +#define ERTS_INTERNAL_ERROR(What) \ + erl_exit(ERTS_ABORT_EXIT, "%s:%d:%s(): Internal error: %s\n", \ + __FILE__, __LINE__, __func__, What) + Eterm erts_check_io_info(void *p); /* Size of misc memory allocated from system dependent code */ @@ -579,6 +618,7 @@ typedef struct _SysDriverOpts { char *wd; /* Working directory. */ unsigned spawn_type; /* Bitfield of ERTS_SPAWN_DRIVER | ERTS_SPAWN_EXTERNAL | both*/ + int parallelism; /* Optimize for parallelism */ } SysDriverOpts; extern char *erts_default_arg0; @@ -618,7 +658,7 @@ typedef struct { #define ERTS_SYS_DDLL_ERROR_INIT {NULL} extern void erts_sys_ddll_free_error(ErtsSysDdllError*); extern void erl_sys_ddll_init(void); /* to initialize mutexes etc */ -extern int erts_sys_ddll_open2(char *path, void **handle, ErtsSysDdllError*); +extern int erts_sys_ddll_open2(const char *path, void **handle, ErtsSysDdllError*); #define erts_sys_ddll_open(P,H) erts_sys_ddll_open2(P,H,NULL) extern int erts_sys_ddll_open_noext(char *path, void **handle, ErtsSysDdllError*); extern int erts_sys_ddll_load_driver_init(void *handle, void **function); @@ -627,7 +667,7 @@ extern int erts_sys_ddll_close2(void *handle, ErtsSysDdllError*); #define erts_sys_ddll_close(H) erts_sys_ddll_close2(H,NULL) extern void *erts_sys_ddll_call_init(void *function); extern void *erts_sys_ddll_call_nif_init(void *function); -extern int erts_sys_ddll_sym2(void *handle, char *name, void **function, ErtsSysDdllError*); +extern int erts_sys_ddll_sym2(void *handle, const char *name, void **function, ErtsSysDdllError*); #define erts_sys_ddll_sym(H,N,F) erts_sys_ddll_sym2(H,N,F,NULL) extern char *erts_sys_ddll_error(int code); @@ -644,7 +684,7 @@ void erts_sys_schedule_interrupt_timed(int set, erts_short_time_t msec); void erts_sys_main_thread(void); #endif -extern void erts_sys_prepare_crash_dump(int secs); +extern int erts_sys_prepare_crash_dump(int secs); extern void erts_sys_pre_init(void); extern void erl_sys_init(void); extern void erl_sys_args(int *argc, char **argv); diff --git a/erts/emulator/beam/utils.c b/erts/emulator/beam/utils.c index 1969fc762c..5261effef9 100644 --- a/erts/emulator/beam/utils.c +++ b/erts/emulator/beam/utils.c @@ -46,6 +46,7 @@ #include "erl_thr_queue.h" #include "erl_sched_spec_pre_alloc.h" #include "beam_bp.h" +#include "erl_ptab.h" #undef M_TRIM_THRESHOLD #undef M_TOP_PAD @@ -3016,12 +3017,13 @@ buf_to_intlist(Eterm** hpp, char *buf, size_t len, Eterm tail) ** ; ** ** Return remaining bytes in buffer on success -** -1 on overflow -** -2 on type error (including that result would not be a whole number of bytes) +** ERTS_IOLIST_TO_BUF_OVERFLOW on overflow +** ERTS_IOLIST_TO_BUF_TYPE_ERROR on type error (including that result would not be a whole number of bytes) */ -int io_list_to_buf(Eterm obj, char* buf, int len) +ErlDrvSizeT erts_iolist_to_buf(Eterm obj, char* buf, ErlDrvSizeT alloced_len) { + ErlDrvSizeT len = (ErlDrvSizeT) alloced_len; Eterm* objp; DECLARE_ESTACK(s); goto L_again; @@ -3114,20 +3116,20 @@ int io_list_to_buf(Eterm obj, char* buf, int len) L_type_error: DESTROY_ESTACK(s); - return -2; + return ERTS_IOLIST_TO_BUF_TYPE_ERROR; L_overflow: DESTROY_ESTACK(s); - return -1; + return ERTS_IOLIST_TO_BUF_OVERFLOW; } /* * Return 0 if successful, and non-zero if unsuccessful. */ -int erts_iolist_size(Eterm obj, Uint* sizep) +int erts_iolist_size(Eterm obj, ErlDrvSizeT* sizep) { Eterm* objp; - Uint size = 0; + Uint size = 0; /* Intentionally Uint due to halfword heap */ DECLARE_ESTACK(s); goto L_again; @@ -3179,7 +3181,7 @@ int erts_iolist_size(Eterm obj, Uint* sizep) #undef SAFE_ADD DESTROY_ESTACK(s); - *sizep = size; + *sizep = (ErlDrvSizeT) size; return ERTS_IOLIST_OK; L_overflow_error: diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 3210ffa92a..236b8710fb 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -678,6 +678,8 @@ static int my_strncasecmp(const char *s1, const char *s2, size_t n) #define INET_LOPT_UDP_READ_PACKETS 33 /* Number of packets to read */ #define INET_OPT_RAW 34 /* Raw socket options */ #define INET_LOPT_TCP_SEND_TIMEOUT_CLOSE 35 /* auto-close on send timeout or not */ +#define INET_LOPT_TCP_MSGQ_HIWTRMRK 36 /* set local high watermark */ +#define INET_LOPT_TCP_MSGQ_LOWTRMRK 37 /* set local low watermark */ /* SCTP options: a separate range, from 100: */ #define SCTP_OPT_RTOINFO 100 #define SCTP_OPT_ASSOCINFO 101 @@ -788,6 +790,8 @@ static int my_strncasecmp(const char *s1, const char *s2, size_t n) #define INET_HIGH_WATERMARK (1024*8) /* 8k pending high => busy */ #define INET_LOW_WATERMARK (1024*4) /* 4k pending => allow more */ +#define INET_HIGH_MSGQ_WATERMARK (1024*8) /* 8k pending high => busy */ +#define INET_LOW_MSGQ_WATERMARK (1024*4) /* 4k pending => allow more */ #define INET_INFINITY 0xffffffff /* infinity value */ @@ -879,7 +883,7 @@ typedef struct subs_list_ { #define NO_PROCESS 0 #define NO_SUBSCRIBERS(SLP) ((SLP)->subscriber == NO_PROCESS) -static void send_to_subscribers(ErlDrvPort, subs_list *, int, +static void send_to_subscribers(ErlDrvTermData, subs_list *, int, ErlDrvTermData [], int); static void free_subscribers(subs_list*); static int save_subscriber(subs_list *, ErlDrvTermData); @@ -1873,8 +1877,7 @@ static int deq_async(inet_descriptor* desc, int* ap, ErlDrvTermData* cp, int* rp ** {inet_async, Port, Ref, ok} */ static int -send_async_ok(ErlDrvPort port, ErlDrvTermData Port, int Ref, - ErlDrvTermData recipient) +send_async_ok(ErlDrvTermData Port, int Ref,ErlDrvTermData recipient) { ErlDrvTermData spec[2*LOAD_ATOM_CNT + LOAD_PORT_CNT + LOAD_INT_CNT + LOAD_TUPLE_CNT]; @@ -1888,14 +1891,14 @@ send_async_ok(ErlDrvPort port, ErlDrvTermData Port, int Ref, ASSERT(i == sizeof(spec)/sizeof(*spec)); - return driver_send_term(port, recipient, spec, i); + return erl_drv_send_term(Port, recipient, spec, i); } /* send message: ** {inet_async, Port, Ref, {ok,Port2}} */ static int -send_async_ok_port(ErlDrvPort port, ErlDrvTermData Port, int Ref, +send_async_ok_port(ErlDrvTermData Port, int Ref, ErlDrvTermData recipient, ErlDrvTermData Port2) { ErlDrvTermData spec[2*LOAD_ATOM_CNT + 2*LOAD_PORT_CNT + @@ -1914,14 +1917,14 @@ send_async_ok_port(ErlDrvPort port, ErlDrvTermData Port, int Ref, ASSERT(i == sizeof(spec)/sizeof(*spec)); - return driver_send_term(port, recipient, spec, i); + return erl_drv_send_term(Port, recipient, spec, i); } /* send message: ** {inet_async, Port, Ref, {error,Reason}} */ static int -send_async_error(ErlDrvPort port, ErlDrvTermData Port, int Ref, +send_async_error(ErlDrvTermData Port, int Ref, ErlDrvTermData recipient, ErlDrvTermData Reason) { ErlDrvTermData spec[3*LOAD_ATOM_CNT + LOAD_PORT_CNT + @@ -1939,7 +1942,7 @@ send_async_error(ErlDrvPort port, ErlDrvTermData Port, int Ref, i = LOAD_TUPLE(spec, i, 4); ASSERT(i == sizeof(spec)/sizeof(*spec)); DEBUGF(("send_async_error %ld %ld\r\n", recipient, Reason)); - return driver_send_term(port, recipient, spec, i); + return erl_drv_send_term(Port, recipient, spec, i); } @@ -1951,7 +1954,7 @@ static int async_ok(inet_descriptor* desc) if (deq_async(desc, &aid, &caller, &req) < 0) return -1; - return send_async_ok(desc->port, desc->dport, aid, caller); + return send_async_ok(desc->dport, aid, caller); } static int async_ok_port(inet_descriptor* desc, ErlDrvTermData Port2) @@ -1962,7 +1965,7 @@ static int async_ok_port(inet_descriptor* desc, ErlDrvTermData Port2) if (deq_async(desc, &aid, &caller, &req) < 0) return -1; - return send_async_ok_port(desc->port, desc->dport, aid, caller, Port2); + return send_async_ok_port(desc->dport, aid, caller, Port2); } static int async_error_am(inet_descriptor* desc, ErlDrvTermData reason) @@ -1973,8 +1976,7 @@ static int async_error_am(inet_descriptor* desc, ErlDrvTermData reason) if (deq_async(desc, &aid, &caller, &req) < 0) return -1; - return send_async_error(desc->port, desc->dport, aid, caller, - reason); + return send_async_error(desc->dport, aid, caller, reason); } /* dequeue all operations */ @@ -1985,8 +1987,7 @@ static int async_error_am_all(inet_descriptor* desc, ErlDrvTermData reason) ErlDrvTermData caller; while (deq_async(desc, &aid, &caller, &req) == 0) { - send_async_error(desc->port, desc->dport, aid, caller, - reason); + send_async_error(desc->dport, aid, caller, reason); } return 0; } @@ -2014,7 +2015,7 @@ static int inet_reply_ok(inet_descriptor* desc) ASSERT(i == sizeof(spec)/sizeof(*spec)); desc->caller = 0; - return driver_send_term(desc->port, caller, spec, i); + return erl_drv_send_term(desc->dport, caller, spec, i); } #ifdef HAVE_SCTP @@ -2033,7 +2034,7 @@ static int inet_reply_ok_port(inet_descriptor* desc, ErlDrvTermData dport) ASSERT(i == sizeof(spec)/sizeof(*spec)); desc->caller = 0; - return driver_send_term(desc->port, caller, spec, i); + return erl_drv_send_term(desc->dport, caller, spec, i); } #endif @@ -2056,7 +2057,7 @@ static int inet_reply_error_am(inet_descriptor* desc, ErlDrvTermData reason) desc->caller = 0; DEBUGF(("inet_reply_error_am %ld %ld\r\n", caller, reason)); - return driver_send_term(desc->port, caller, spec, i); + return erl_drv_send_term(desc->dport, caller, spec, i); } /* send: @@ -2165,12 +2166,12 @@ static int http_response_inetdrv(void *arg, int major, int minor, i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i<=27); - return driver_send_term(desc->inet.port, caller, spec, i); + return erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { i = LOAD_TUPLE(spec, i, 3); ASSERT(i<=27); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } } @@ -2262,12 +2263,12 @@ http_request_inetdrv(void* arg, const http_atom_t* meth, const char* meth_ptr, i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 43); - return driver_send_term(desc->inet.port, caller, spec, i); + return erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 43); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } } @@ -2316,12 +2317,12 @@ http_header_inetdrv(void* arg, const http_atom_t* name, const char* name_ptr, i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 26); - return driver_send_term(desc->inet.port, caller, spec, i); + return erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 26); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } } @@ -2347,7 +2348,7 @@ static int http_eoh_inetdrv(void* arg) i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 14); - return driver_send_term(desc->inet.port, caller, spec, i); + return erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { /* {http, S, http_eoh} */ @@ -2356,7 +2357,7 @@ static int http_eoh_inetdrv(void* arg) i = LOAD_ATOM(spec, i, am_http_eoh); i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 14); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } } @@ -2384,7 +2385,7 @@ static int http_error_inetdrv(void* arg, const char* buf, int len) i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 19); - return driver_send_term(desc->inet.port, caller, spec, i); + return erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { /* {http, S, {http_error,Line} */ @@ -2395,7 +2396,7 @@ static int http_error_inetdrv(void* arg, const char* buf, int len) i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 19); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } } @@ -2448,11 +2449,11 @@ int ssl_tls_inetdrv(void* arg, unsigned type, unsigned major, unsigned minor, i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 28); - ret = driver_send_term(desc->inet.port, caller, spec, i); + ret = erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { ASSERT(i <= 28); - ret = driver_output_term(desc->inet.port, spec, i); + ret = erl_drv_output_term(desc->inet.dport, spec, i); } done: driver_free_binary(bin); @@ -2502,7 +2503,7 @@ static int inet_async_data(inet_descriptor* desc, const char* buf, int len) i = LOAD_TUPLE(spec, i, 4); ASSERT(i == 15); desc->caller = 0; - return driver_send_term(desc->port, caller, spec, i); + return erl_drv_send_term(desc->dport, caller, spec, i); } else { /* INET_MODE_BINARY => [H1,H2,...HSz | Binary] */ @@ -2516,7 +2517,7 @@ static int inet_async_data(inet_descriptor* desc, const char* buf, int len) i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 20); desc->caller = 0; - code = driver_send_term(desc->port, caller, spec, i); + code = erl_drv_send_term(desc->dport, caller, spec, i); return code; } } @@ -3109,7 +3110,7 @@ inet_async_binary_data ASSERT(i <= PACKET_ERL_DRV_TERM_DATA_LEN); desc->caller = 0; - return driver_send_term(desc->port, caller, spec, i); + return erl_drv_send_term(desc->dport, caller, spec, i); } /* @@ -3132,7 +3133,7 @@ static int tcp_message(inet_descriptor* desc, const char* buf, int len) i = LOAD_STRING(spec, i, buf, len); /* => [H1,H2,...Hn] */ i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 20); - return driver_output_term(desc->port, spec, i); + return erl_drv_output_term(desc->dport, spec, i); } else { /* INET_MODE_BINARY => [H1,H2,...HSz | Binary] */ @@ -3144,7 +3145,7 @@ static int tcp_message(inet_descriptor* desc, const char* buf, int len) i = LOAD_STRING_CONS(spec, i, buf, hsz); i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 20); - code = driver_output_term(desc->port, spec, i); + code = erl_drv_output_term(desc->dport, spec, i); return code; } } @@ -3179,7 +3180,7 @@ tcp_binary_message(inet_descriptor* desc, ErlDrvBinary* bin, int offs, int len) } i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 20); - return driver_output_term(desc->port, spec, i); + return erl_drv_output_term(desc->dport, spec, i); } /* @@ -3198,7 +3199,7 @@ static int tcp_closed_message(tcp_descriptor* desc) i = LOAD_PORT(spec, i, desc->inet.dport); i = LOAD_TUPLE(spec, i, 2); ASSERT(i <= 6); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } return 0; } @@ -3219,7 +3220,7 @@ static int tcp_error_message(tcp_descriptor* desc, int err) i = LOAD_ATOM(spec, i, am_err); i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 8); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } /* @@ -3310,7 +3311,7 @@ static int packet_binary_message /* Close up the outer 5-tuple: */ i = LOAD_TUPLE(spec, i, 5); ASSERT(i <= PACKET_ERL_DRV_TERM_DATA_LEN); - return driver_output_term(desc->port, spec, i); + return erl_drv_output_term(desc->dport, spec, i); } /* @@ -3337,7 +3338,7 @@ static int packet_error_message(udp_descriptor* udesc, int err) i = LOAD_ATOM(spec, i, am_err); i = LOAD_TUPLE(spec, i, 3); ASSERT(i == sizeof(spec)/sizeof(*spec)); - return driver_output_term(desc->port, spec, i); + return erl_drv_output_term(desc->dport, spec, i); } @@ -5465,6 +5466,28 @@ static int inet_set_opts(inet_descriptor* desc, char* ptr, int len) } continue; + case INET_LOPT_TCP_MSGQ_HIWTRMRK: + if (desc->stype == SOCK_STREAM) { + ErlDrvSizeT high; + if (ival < ERL_DRV_BUSY_MSGQ_LIM_MIN + || ERL_DRV_BUSY_MSGQ_LIM_MAX < ival) + return -1; + high = (ErlDrvSizeT) ival; + erl_drv_busy_msgq_limits(desc->port, NULL, &high); + } + continue; + + case INET_LOPT_TCP_MSGQ_LOWTRMRK: + if (desc->stype == SOCK_STREAM) { + ErlDrvSizeT low; + if (ival < ERL_DRV_BUSY_MSGQ_LIM_MIN + || ERL_DRV_BUSY_MSGQ_LIM_MAX < ival) + return -1; + low = (ErlDrvSizeT) ival; + erl_drv_busy_msgq_limits(desc->port, &low, NULL); + } + continue; + case INET_LOPT_TCP_SEND_TIMEOUT: if (desc->stype == SOCK_STREAM) { tcp_descriptor* tdesc = (tcp_descriptor*) desc; @@ -6365,6 +6388,32 @@ static ErlDrvSSizeT inet_fill_opts(inet_descriptor* desc, } continue; + case INET_LOPT_TCP_MSGQ_HIWTRMRK: + if (desc->stype == SOCK_STREAM) { + ErlDrvSizeT high = ERL_DRV_BUSY_MSGQ_READ_ONLY; + *ptr++ = opt; + erl_drv_busy_msgq_limits(desc->port, NULL, &high); + ival = high > INT_MAX ? INT_MAX : (int) high; + put_int32(ival, ptr); + } + else { + TRUNCATE_TO(0,ptr); + } + continue; + + case INET_LOPT_TCP_MSGQ_LOWTRMRK: + if (desc->stype == SOCK_STREAM) { + ErlDrvSizeT low = ERL_DRV_BUSY_MSGQ_READ_ONLY; + *ptr++ = opt; + erl_drv_busy_msgq_limits(desc->port, &low, NULL); + ival = low > INT_MAX ? INT_MAX : (int) low; + put_int32(ival, ptr); + } + else { + TRUNCATE_TO(0,ptr); + } + continue; + case INET_LOPT_TCP_SEND_TIMEOUT: if (desc->stype == SOCK_STREAM) { *ptr++ = opt; @@ -7278,7 +7327,7 @@ static ErlDrvSSizeT sctp_fill_opts(inet_descriptor* desc, i = LOAD_TUPLE(spec, i, 3); /* Now, convert "spec" into the returnable term: */ - driver_send_term(desc->port, driver_caller(desc->port), spec, i); + erl_drv_send_term(desc->dport, driver_caller(desc->port), spec, i); FREE(spec); (*dest)[0] = INET_REP; @@ -7360,7 +7409,7 @@ send_empty_out_q_msgs(inet_descriptor* desc) ASSERT(msg_len == sizeof(msg)/sizeof(*msg)); - send_to_subscribers(desc->port, + send_to_subscribers(desc->dport, &desc->empty_out_q_subs, 1, msg, @@ -8007,6 +8056,7 @@ static int tcp_inet_init(void) static ErlDrvData tcp_inet_start(ErlDrvPort port, char* args) { + ErlDrvSizeT q_low, q_high; tcp_descriptor* desc; DEBUGF(("tcp_inet_start(%ld) {\r\n", (long)port)); @@ -8016,6 +8066,17 @@ static ErlDrvData tcp_inet_start(ErlDrvPort port, char* args) return ERL_DRV_ERROR_ERRNO; desc->high = INET_HIGH_WATERMARK; desc->low = INET_LOW_WATERMARK; + q_high = INET_HIGH_MSGQ_WATERMARK; + q_low = INET_LOW_MSGQ_WATERMARK; + if (q_low < ERL_DRV_BUSY_MSGQ_LIM_MIN) + q_low = ERL_DRV_BUSY_MSGQ_LIM_MIN; + else if (q_low > ERL_DRV_BUSY_MSGQ_LIM_MAX) + q_low = ERL_DRV_BUSY_MSGQ_LIM_MAX; + if (q_high < ERL_DRV_BUSY_MSGQ_LIM_MIN) + q_high = ERL_DRV_BUSY_MSGQ_LIM_MIN; + else if (q_high > ERL_DRV_BUSY_MSGQ_LIM_MAX) + q_high = ERL_DRV_BUSY_MSGQ_LIM_MAX; + erl_drv_busy_msgq_limits(port, &q_low, &q_high); desc->send_timeout = INET_INFINITY; desc->send_timeout_close = 0; desc->busy_on_send = 0; @@ -8039,6 +8100,7 @@ static ErlDrvData tcp_inet_start(ErlDrvPort port, char* args) static tcp_descriptor* tcp_inet_copy(tcp_descriptor* desc,SOCKET s, ErlDrvTermData owner, int* err) { + ErlDrvSizeT q_low, q_high; ErlDrvPort port = desc->inet.port; tcp_descriptor* copy_desc; @@ -8076,6 +8138,13 @@ static tcp_descriptor* tcp_inet_copy(tcp_descriptor* desc,SOCKET s, FREE(copy_desc); return NULL; } + + /* Read busy msgq limits of parent */ + q_low = q_high = ERL_DRV_BUSY_MSGQ_READ_ONLY; + erl_drv_busy_msgq_limits(desc->inet.port, &q_low, &q_high); + /* Write same busy msgq limits to child */ + erl_drv_busy_msgq_limits(port, &q_low, &q_high); + copy_desc->inet.port = port; copy_desc->inet.dport = driver_mk_port(port); *err = 0; @@ -8108,7 +8177,7 @@ static void tcp_close_check(tcp_descriptor* desc) desc->inet.state = INET_STATE_LISTENING; while (deq_multi_op(desc,&id,&req,&caller,NULL,&monitor) == 0) { driver_demonitor_process(desc->inet.port, &monitor); - send_async_error(desc->inet.port, desc->inet.dport, id, caller, am_closed); + send_async_error(desc->inet.dport, id, caller, am_closed); } clean_multi_timers(&(desc->mtd), desc->inet.port); } @@ -8532,7 +8601,7 @@ static void tcp_inet_multi_timeout(ErlDrvData e, ErlDrvTermData caller) sock_select(INETP(desc),FD_ACCEPT,0); desc->inet.state = INET_STATE_LISTENING; /* restore state */ } - send_async_error(desc->inet.port, desc->inet.dport, id, caller, am_timeout); + send_async_error(desc->inet.dport, id, caller, am_timeout); } @@ -9273,7 +9342,7 @@ static int tcp_inet_input(tcp_descriptor* desc, HANDLE event) if (s == INVALID_SOCKET) { /* Not ERRNO_BLOCK, that's handled right away */ - ret = send_async_error(desc->inet.port, desc->inet.dport, + ret = send_async_error(desc->inet.dport, id, caller, error_atom(sock_errno())); goto done; } @@ -9283,7 +9352,7 @@ static int tcp_inet_input(tcp_descriptor* desc, HANDLE event) if ((accept_desc = tcp_inet_copy(desc,s,caller,&err)) == NULL) { sock_close(s); - ret = send_async_error(desc->inet.port, desc->inet.dport, + ret = send_async_error(desc->inet.dport, id, caller, error_atom(err)); goto done; } @@ -9294,7 +9363,7 @@ static int tcp_inet_input(tcp_descriptor* desc, HANDLE event) ERL_DRV_READ, 1); #endif accept_desc->inet.state = INET_STATE_CONNECTED; - ret = send_async_ok_port(desc->inet.port, desc->inet.dport, + ret = send_async_ok_port(desc->inet.dport, id, caller, accept_desc->inet.dport); } } @@ -10936,7 +11005,7 @@ subs_list *subs; static void send_to_subscribers ( - ErlDrvPort port, + ErlDrvTermData port, subs_list *subs, int free_subs, ErlDrvTermData msg[], @@ -10953,7 +11022,7 @@ static void send_to_subscribers this = subs; while(this) { - (void) driver_send_term(port, this->subscriber, msg, msg_len); + (void) erl_drv_send_term(port, this->subscriber, msg, msg_len); if(free_subs && !first) { next = this->next; diff --git a/erts/emulator/hipe/hipe_amd64_bifs.m4 b/erts/emulator/hipe/hipe_amd64_bifs.m4 index 42e8b2a9b0..0de69a617f 100644 --- a/erts/emulator/hipe/hipe_amd64_bifs.m4 +++ b/erts/emulator/hipe/hipe_amd64_bifs.m4 @@ -2,7 +2,7 @@ changecom(`/*', `*/')dnl /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2011. All Rights Reserved. + * Copyright Ericsson AB 2004-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/hipe/hipe_bif_list.m4 b/erts/emulator/hipe/hipe_bif_list.m4 index ea4f68ef24..764b8d180c 100644 --- a/erts/emulator/hipe/hipe_bif_list.m4 +++ b/erts/emulator/hipe/hipe_bif_list.m4 @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2011. All Rights Reserved. + * Copyright Ericsson AB 2004-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -145,6 +145,7 @@ * Zero-arity BIFs that can fail. */ standard_bif_interface_0(nbif_processes_0, processes_0) +standard_bif_interface_0(nbif_ports_0, ports_0) /* * BIFs and primops that may do a GC (change heap limit and walk the native stack). diff --git a/erts/emulator/hipe/hipe_debug.c b/erts/emulator/hipe/hipe_debug.c index 37615bf718..f2e9d03607 100644 --- a/erts/emulator/hipe/hipe_debug.c +++ b/erts/emulator/hipe/hipe_debug.c @@ -189,11 +189,10 @@ void hipe_print_pcb(Process *p) U("old_head ", old_heap); U("min_heap_..", min_heap_size); U("rcount ", rcount); - U("id ", id); - U("prio ", prio); + U("id ", common.id); U("reds ", reds); - U("tracer_pr..", tracer_proc); - U("trace_fla..", trace_flags); + U("tracer_pr..", common.tracer_proc); + U("trace_fla..", common.trace_flags); U("group_lea..", group_leader); U("flags ", flags); U("fvalue ", fvalue); @@ -202,8 +201,8 @@ void hipe_print_pcb(Process *p) /*XXX: ErlTimer tm; */ U("next ", next); /*XXX: ErlOffHeap off_heap; */ - U("reg ", reg); - U("nlinks ", nlinks); + U("reg ", common.u.alive.reg); + U("nlinks ", common.u.alive.links); /*XXX: ErlMessageQueue msg; */ U("mbuf ", mbuf); U("mbuf_sz ", mbuf_sz); diff --git a/erts/emulator/hipe/hipe_mkliterals.c b/erts/emulator/hipe/hipe_mkliterals.c index cbbf1db2e5..5911d76ad9 100644 --- a/erts/emulator/hipe/hipe_mkliterals.c +++ b/erts/emulator/hipe/hipe_mkliterals.c @@ -267,7 +267,7 @@ static const struct literal { { "P_HP_LIMIT", offsetof(struct process, stop) }, { "P_OFF_HEAP_FIRST", offsetof(struct process, off_heap.first) }, { "P_MBUF", offsetof(struct process, mbuf) }, - { "P_ID", offsetof(struct process, id) }, + { "P_ID", offsetof(struct process, common.id) }, { "P_FLAGS", offsetof(struct process, flags) }, { "P_FVALUE", offsetof(struct process, fvalue) }, { "P_FREASON", offsetof(struct process, freason) }, diff --git a/erts/emulator/hipe/hipe_x86.c b/erts/emulator/hipe/hipe_x86.c index 24d232c968..4281730ae2 100644 --- a/erts/emulator/hipe/hipe_x86.c +++ b/erts/emulator/hipe/hipe_x86.c @@ -265,7 +265,7 @@ void *hipe_make_native_stub(void *beamAddress, unsigned int beamArity) void hipe_arch_print_pcb(struct hipe_process_state *p) { #define U(n,x) \ - printf(" % 4d | %s | 0x%08x | |\r\n", offsetof(struct hipe_process_state,x), n, (unsigned)p->x) + printf(" % 4d | %s | 0x%08x | |\r\n", (int)offsetof(struct hipe_process_state,x), n, (unsigned)p->x) U("ncsp ", ncsp); U("narity ", narity); #undef U diff --git a/erts/emulator/hipe/hipe_x86_bifs.m4 b/erts/emulator/hipe/hipe_x86_bifs.m4 index 8cc6340933..dd6980f555 100644 --- a/erts/emulator/hipe/hipe_x86_bifs.m4 +++ b/erts/emulator/hipe/hipe_x86_bifs.m4 @@ -2,7 +2,7 @@ changecom(`/*', `*/')dnl /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2011. All Rights Reserved. + * Copyright Ericsson AB 2001-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/sys/common/erl_check_io.c b/erts/emulator/sys/common/erl_check_io.c index ce014c19c2..474408ae7c 100644 --- a/erts/emulator/sys/common/erl_check_io.c +++ b/erts/emulator/sys/common/erl_check_io.c @@ -200,17 +200,6 @@ static void event_large_fd_error(ErlDrvPort, ErtsSysFdType, ErlDrvEventData); #endif static void steal_pending_stop_select(erts_dsprintf_buf_t*, ErlDrvPort, ErtsDrvEventState*, int mode, int on); -static ERTS_INLINE Eterm -drvport2id(ErlDrvPort dp) -{ - Port *pp = erts_drvport2port(dp); - if (pp) - return pp->id; - else { - ASSERT(0); - return am_undefined; - } -} #ifdef ERTS_SMP ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(removed_fd, struct removed_fd, 64, ERTS_ALC_T_FD_LIST) @@ -378,7 +367,7 @@ abort_task(Eterm id, ErtsPortTaskHandle *pthp, EventStateType type) || !erts_port_task_is_scheduled(pthp)); } else if (erts_port_task_is_scheduled(pthp)) { - erts_port_task_abort(id, pthp); + erts_port_task_abort(pthp); ASSERT(erts_is_port_alive(id)); } } @@ -492,7 +481,7 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix, int on) { void (*stop_select_fn)(ErlDrvEvent, void*) = NULL; - Eterm id = drvport2id(ix); + Eterm id = erts_drvport2id(ix); ErtsSysFdType fd = (ErtsSysFdType) e; ErtsPollEvents ctl_events = (ErtsPollEvents) 0; ErtsPollEvents new_events, old_events; @@ -503,8 +492,8 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix, DTRACE_CHARBUF(name, 64); #endif - ERTS_SMP_LC_ASSERT(erts_drvport2port(ix) - && erts_lc_is_port_locked(erts_drvport2port(ix))); + ERTS_SMP_LC_ASSERT(erts_drvport2port(ix, NULL) + && erts_lc_is_port_locked(erts_drvport2port(ix, NULL))); #ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS if ((unsigned)fd >= (unsigned)erts_smp_atomic_read_nob(&drv_ev_state_len)) { @@ -530,9 +519,9 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix, if (!on && (mode&ERL_DRV_USE_NO_CALLBACK) == ERL_DRV_USE) { if (IS_FD_UNKNOWN(state)) { /* fast track to stop_select callback */ - stop_select_fn = erts_drvport2port(ix)->drv_ptr->stop_select; + stop_select_fn = erts_drvport2port(ix, NULL)->drv_ptr->stop_select; #ifdef USE_VM_PROBES - strncpy(name, erts_drvport2port(ix)->drv_ptr->name, sizeof(name)-1); + strncpy(name, erts_drvport2port(ix, NULL)->drv_ptr->name, sizeof(name)-1); name[sizeof(name)-1] = '\0'; #endif ret = 0; @@ -665,14 +654,14 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix, } } if ((mode & ERL_DRV_USE_NO_CALLBACK) == ERL_DRV_USE) { - erts_driver_t* drv_ptr = erts_drvport2port(ix)->drv_ptr; + erts_driver_t* drv_ptr = erts_drvport2port(ix, NULL)->drv_ptr; ASSERT(new_events==0); if (state->remove_cnt == 0 || !wake_poller) { /* Safe to close fd now as it is not in pollset or there was no need to eject fd (kernel poll) */ stop_select_fn = drv_ptr->stop_select; #ifdef USE_VM_PROBES - strncpy(name, erts_drvport2port(ix)->drv_ptr->name, sizeof(name)-1); + strncpy(name, erts_drvport2port(ix, NULL)->drv_ptr->name, sizeof(name)-1); name[sizeof(name)-1] = '\0'; #endif } @@ -719,13 +708,13 @@ ERTS_CIO_EXPORT(driver_event)(ErlDrvPort ix, ErtsPollEvents events; ErtsPollEvents add_events; ErtsPollEvents remove_events; - Eterm id = drvport2id(ix); + Eterm id = erts_drvport2id(ix); ErtsDrvEventState *state; int do_wake = 0; int ret; - ERTS_SMP_LC_ASSERT(erts_drvport2port(ix) - && erts_lc_is_port_locked(erts_drvport2port(ix))); + ERTS_SMP_LC_ASSERT(erts_drvport2port(ix, NULL) + && erts_lc_is_port_locked(erts_drvport2port(ix, NULL))); #ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS if ((unsigned)fd >= (unsigned)erts_smp_atomic_read_nob(&drv_ev_state_len)) { @@ -960,7 +949,7 @@ static void print_select_op(erts_dsprintf_buf_t *dsbufp, ErlDrvPort ix, ErtsSysFdType fd, int mode, int on) { - Port *pp = erts_drvport2port(ix); + Port *pp = erts_drvport2port(ix, NULL); erts_dsprintf(dsbufp, "driver_select(%p, %d,%s%s%s%s, %d) " "by ", @@ -971,8 +960,8 @@ print_select_op(erts_dsprintf_buf_t *dsbufp, mode & ERL_DRV_USE ? " ERL_DRV_USE" : "", mode & (ERL_DRV_USE_NO_CALLBACK & ~ERL_DRV_USE) ? "_NO_CALLBACK" : "", on); - print_driver_name(dsbufp, pp->id); - erts_dsprintf(dsbufp, "driver %T ", pp ? pp->id : NIL); + print_driver_name(dsbufp, pp->common.id); + erts_dsprintf(dsbufp, "driver %T ", pp ? pp->common.id : NIL); } static void @@ -1031,7 +1020,7 @@ steal_pending_stop_select(erts_dsprintf_buf_t *dsbufp, ErlDrvPort ix, state->driver.drv_ptr = NULL; } else if ((mode & ERL_DRV_USE_NO_CALLBACK) == ERL_DRV_USE) { - erts_driver_t* drv_ptr = erts_drvport2port(ix)->drv_ptr; + erts_driver_t* drv_ptr = erts_drvport2port(ix, NULL)->drv_ptr; if (drv_ptr != state->driver.drv_ptr) { /* Some other driver wants the stop_select callback */ if (state->driver.drv_ptr->handle) { @@ -1053,7 +1042,7 @@ static void print_event_op(erts_dsprintf_buf_t *dsbufp, ErlDrvPort ix, ErtsSysFdType fd, ErlDrvEventData event_data) { - Port *pp = erts_drvport2port(ix); + Port *pp = erts_drvport2port(ix, NULL); erts_dsprintf(dsbufp, "driver_event(%p, %d, ", ix, (int) fd); if (!event_data) erts_dsprintf(dsbufp, "NULL"); @@ -1062,8 +1051,8 @@ print_event_op(erts_dsprintf_buf_t *dsbufp, (unsigned int) event_data->events, (unsigned int) event_data->revents); erts_dsprintf(dsbufp, ") by "); - print_driver_name(dsbufp, pp->id); - erts_dsprintf(dsbufp, "driver %T ", pp ? pp->id : NIL); + print_driver_name(dsbufp, pp->common.id); + erts_dsprintf(dsbufp, "driver %T ", pp ? pp->common.id : NIL); } static void @@ -1100,8 +1089,7 @@ iready(Eterm id, ErtsDrvEventState *state) if (erts_port_task_schedule(id, &state->driver.select->intask, ERTS_PORT_TASK_INPUT, - (ErlDrvEvent) state->fd, - NULL) != 0) { + (ErlDrvEvent) state->fd) != 0) { stale_drv_select(id, state, ERL_DRV_READ); } } @@ -1112,8 +1100,7 @@ oready(Eterm id, ErtsDrvEventState *state) if (erts_port_task_schedule(id, &state->driver.select->outtask, ERTS_PORT_TASK_OUTPUT, - (ErlDrvEvent) state->fd, - NULL) != 0) { + (ErlDrvEvent) state->fd) != 0) { stale_drv_select(id, state, ERL_DRV_WRITE); } } diff --git a/erts/emulator/sys/unix/erl_unix_sys_ddll.c b/erts/emulator/sys/unix/erl_unix_sys_ddll.c index 336d9586c4..a35aec560a 100644 --- a/erts/emulator/sys/unix/erl_unix_sys_ddll.c +++ b/erts/emulator/sys/unix/erl_unix_sys_ddll.c @@ -101,7 +101,7 @@ void erl_sys_ddll_init(void) { /* * Open a shared object */ -int erts_sys_ddll_open2(char *full_name, void **handle, ErtsSysDdllError* err) +int erts_sys_ddll_open2(const char *full_name, void **handle, ErtsSysDdllError* err) { #if defined(HAVE_DLOPEN) char* dlname; @@ -153,7 +153,7 @@ int erts_sys_ddll_open_noext(char *dlname, void **handle, ErtsSysDdllError* err) /* * Find a symbol in the shared object */ -int erts_sys_ddll_sym2(void *handle, char *func_name, void **function, +int erts_sys_ddll_sym2(void *handle, const char *func_name, void **function, ErtsSysDdllError* err) { #if defined(HAVE_DLOPEN) diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c index b485dbf784..0b96eded76 100644 --- a/erts/emulator/sys/unix/sys.c +++ b/erts/emulator/sys/unix/sys.c @@ -123,7 +123,8 @@ struct ErtsSysReportExit_ { /* This data is shared by these drivers - initialized by spawn_init() */ static struct driver_data { - int port_num, ofd, packet_bytes; + ErlDrvPort port_num; + int ofd, packet_bytes; ErtsSysReportExit *report_exit; int pid; int alive; @@ -686,7 +687,7 @@ static RETSIGTYPE break_handler(int sig) } #endif /* 0 */ -static ERTS_INLINE void +static ERTS_INLINE int prepare_crash_dump(int secs) { #define NUFBUF (3) @@ -698,24 +699,41 @@ prepare_crash_dump(int secs) Eterm *hp = heap; Eterm list = NIL; int heart_fd[2] = {-1,-1}; + int has_heart = 0; UseTmpHeapNoproc(NUFBUF); if (ERTS_PREPARED_CRASH_DUMP) - return; /* We have already been called */ + return 0; /* We have already been called */ heart_port = erts_get_heart_port(); + + /* Positive secs means an alarm must be set + * 0 or negative means no alarm + * + * Set alarm before we try to write to a port + * we don't want to hang on a port write with + * no alarm. + * + */ + + if (secs >= 0) { + alarm((unsigned int)secs); + } + if (heart_port) { /* hearts input fd * We "know" drv_data is the in_fd since the port is started with read|write */ heart_fd[0] = (int)heart_port->drv_data; heart_fd[1] = (int)driver_data[heart_fd[0]].ofd; + has_heart = 1; list = CONS(hp, make_small(8), list); hp += 2; /* send to heart port, CMD = 8, i.e. prepare crash dump =o */ - erts_write_to_port(ERTS_INVALID_PID, heart_port, list); + erts_port_output(NULL, ERTS_PORT_SIG_FLG_FORCE_IMM_CALL, heart_port, + heart_port->common.id, list, NULL); } /* Make sure we unregister at epmd (unknown fd) and get at least @@ -752,20 +770,14 @@ prepare_crash_dump(int secs) erts_silence_warn_unused_result(nice(nice_val)); } - /* Positive secs means an alarm must be set - * 0 or negative means no alarm - */ - if (secs > 0) { - alarm((unsigned int)secs); - } UnUseTmpHeapNoproc(NUFBUF); #undef NUFBUF + return has_heart; } -void -erts_sys_prepare_crash_dump(int secs) +int erts_sys_prepare_crash_dump(int secs) { - prepare_crash_dump(secs); + return prepare_crash_dump(secs); } static ERTS_INLINE void @@ -802,12 +814,22 @@ static RETSIGTYPE request_break(int signum) static ERTS_INLINE void sigusr1_exit(void) { - /* We do this at interrupt level, since the main reason for - wanting to generate a crash dump in this way is that the emulator - is hung somewhere, so it won't be able to poll any flag we set here. - */ + char env[21]; /* enough to hold any 64-bit integer */ + size_t envsz; + int i, secs = -1; + + /* We do this at interrupt level, since the main reason for + * wanting to generate a crash dump in this way is that the emulator + * is hung somewhere, so it won't be able to poll any flag we set here. + */ ERTS_SET_GOT_SIGUSR1; - prepare_crash_dump((int)0); + + envsz = sizeof(env); + if ((i = erts_sys_getenv_raw("ERL_CRASH_DUMP_SECONDS", env, &envsz)) >= 0) { + secs = i != 0 ? 0 : atoi(env); + } + + prepare_crash_dump(secs); erl_exit(1, "Received SIGUSR1\n"); } @@ -1162,7 +1184,7 @@ static RETSIGTYPE onchld(int signum) #endif } -static int set_driver_data(int port_num, +static int set_driver_data(ErlDrvPort port_num, int ifd, int ofd, int packet_bytes, @@ -1170,6 +1192,7 @@ static int set_driver_data(int port_num, int exit_status, int pid) { + Port *prt; ErtsSysReportExit *report_exit; if (!exit_status) @@ -1178,7 +1201,7 @@ static int set_driver_data(int port_num, report_exit = erts_alloc(ERTS_ALC_T_PRT_REP_EXIT, sizeof(ErtsSysReportExit)); report_exit->next = report_exit_list; - report_exit->port = erts_port[port_num].id; + report_exit->port = erts_drvport2id(port_num); report_exit->pid = pid; report_exit->ifd = read_write & DO_READ ? ifd : -1; report_exit->ofd = read_write & DO_WRITE ? ofd : -1; @@ -1188,7 +1211,9 @@ static int set_driver_data(int port_num, report_exit_list = report_exit; } - erts_port[port_num].os_pid = pid; + prt = erts_drvport2port(port_num, NULL); + if (prt) + prt->os_pid = pid; if (read_write & DO_READ) { driver_data[ifd].packet_bytes = packet_bytes; @@ -1261,7 +1286,7 @@ static void close_pipes(int ifd[2], int ofd[2], int read_write) } } -static void init_fd_data(int fd, int prt) +static void init_fd_data(int fd, ErlDrvPort port_num) { fd_data[fd].buf = NULL; fd_data[fd].cpos = NULL; @@ -1951,7 +1976,7 @@ static void clear_fd_data(int fd) fd_data[fd].psz = 0; } -static void nbio_stop_fd(int prt, int fd) +static void nbio_stop_fd(ErlDrvPort prt, int fd) { driver_select(prt,fd,DO_READ|DO_WRITE,0); clear_fd_data(fd); @@ -1999,7 +2024,8 @@ static ErlDrvData vanilla_start(ErlDrvPort port_num, char* name, static void stop(ErlDrvData fd) { - int prt, ofd; + ErlDrvPort prt; + int ofd; prt = driver_data[(int)(long)fd].port_num; nbio_stop_fd(prt, (int)(long)fd); @@ -2012,7 +2038,7 @@ static void stop(ErlDrvData fd) CHLD_STAT_LOCK; - /* Mark as unused. Maybe resetting the 'port_num' slot is better? */ + /* Mark as unused. */ driver_data[(int)(long)fd].pid = -1; CHLD_STAT_UNLOCK; @@ -2028,7 +2054,7 @@ static void stop(ErlDrvData fd) static void outputv(ErlDrvData e, ErlIOVec* ev) { int fd = (int)(long)e; - int ix = driver_data[fd].port_num; + ErlDrvPort ix = driver_data[fd].port_num; int pb = driver_data[fd].packet_bytes; int ofd = driver_data[fd].ofd; ssize_t n; @@ -2078,7 +2104,7 @@ static void outputv(ErlDrvData e, ErlIOVec* ev) static void output(ErlDrvData e, char* buf, ErlDrvSizeT len) { int fd = (int)(long)e; - int ix = driver_data[fd].port_num; + ErlDrvPort ix = driver_data[fd].port_num; int pb = driver_data[fd].packet_bytes; int ofd = driver_data[fd].ofd; ssize_t n; @@ -2129,7 +2155,7 @@ static void output(ErlDrvData e, char* buf, ErlDrvSizeT len) return; /* 0; */ } -static int port_inp_failure(int port_num, int ready_fd, int res) +static int port_inp_failure(ErlDrvPort port_num, int ready_fd, int res) /* Result: 0 (eof) or -1 (error) */ { int err = errno; @@ -2179,7 +2205,7 @@ static int port_inp_failure(int port_num, int ready_fd, int res) static void ready_input(ErlDrvData e, ErlDrvEvent ready_fd) { int fd = (int)(long)e; - int port_num; + ErlDrvPort port_num; int packet_bytes; int res; Uint h; @@ -2302,7 +2328,7 @@ static void ready_input(ErlDrvData e, ErlDrvEvent ready_fd) static void ready_output(ErlDrvData e, ErlDrvEvent ready_fd) { int fd = (int)(long)e; - int ix = driver_data[fd].port_num; + ErlDrvPort ix = driver_data[fd].port_num; int n; struct iovec* iv; int vsize; @@ -2611,19 +2637,20 @@ report_exit_status(ErtsSysReportExit *rep, int status) Port *pp; #ifdef ERTS_SMP CHLD_STAT_UNLOCK; -#endif + pp = erts_thr_id2port_sflgs(rep->port, + ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP); + CHLD_STAT_LOCK; +#else pp = erts_id2port_sflgs(rep->port, NULL, 0, ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP); -#ifdef ERTS_SMP - CHLD_STAT_LOCK; #endif if (pp) { if (rep->ifd >= 0) { driver_data[rep->ifd].alive = 0; driver_data[rep->ifd].status = status; - (void) driver_select((ErlDrvPort) internal_port_index(pp->id), + (void) driver_select((ErlDrvPort) pp, rep->ifd, (ERL_DRV_READ|ERL_DRV_USE), 1); @@ -2631,12 +2658,16 @@ report_exit_status(ErtsSysReportExit *rep, int status) if (rep->ofd >= 0) { driver_data[rep->ofd].alive = 0; driver_data[rep->ofd].status = status; - (void) driver_select((ErlDrvPort) internal_port_index(pp->id), + (void) driver_select((ErlDrvPort) pp, rep->ofd, (ERL_DRV_WRITE|ERL_DRV_USE), 1); } +#ifdef ERTS_SMP + erts_thr_port_release(pp); +#else erts_port_release(pp); +#endif } erts_free(ERTS_ALC_T_PRT_REP_EXIT, rep); } diff --git a/erts/emulator/sys/win32/erl_win32_sys_ddll.c b/erts/emulator/sys/win32/erl_win32_sys_ddll.c index d00eed932b..54991a610c 100644 --- a/erts/emulator/sys/win32/erl_win32_sys_ddll.c +++ b/erts/emulator/sys/win32/erl_win32_sys_ddll.c @@ -58,7 +58,7 @@ void erl_sys_ddll_init(void) { /* * Open a shared object */ -int erts_sys_ddll_open2(char *full_name, void **handle, ErtsSysDdllError* err) +int erts_sys_ddll_open2(const char *full_name, void **handle, ErtsSysDdllError* err) { int len; char dlname[MAXPATHLEN + 1]; @@ -92,7 +92,7 @@ int erts_sys_ddll_open_noext(char *dlname, void **handle, ErtsSysDdllError* err) /* * Find a symbol in the shared object */ -int erts_sys_ddll_sym2(void *handle, char *func_name, void **function, +int erts_sys_ddll_sym2(void *handle, const char *func_name, void **function, ErtsSysDdllError* err) { FARPROC proc; diff --git a/erts/emulator/sys/win32/erl_win_dyn_driver.h b/erts/emulator/sys/win32/erl_win_dyn_driver.h index ec5141838a..8b6be2b2f1 100644 --- a/erts/emulator/sys/win32/erl_win_dyn_driver.h +++ b/erts/emulator/sys/win32/erl_win_dyn_driver.h @@ -74,7 +74,9 @@ WDD_TYPEDEF(ErlDrvTermData, driver_mk_port,(ErlDrvPort)); WDD_TYPEDEF(ErlDrvTermData, driver_connected,(ErlDrvPort)); WDD_TYPEDEF(ErlDrvTermData, driver_caller,(ErlDrvPort)); WDD_TYPEDEF(ErlDrvTermData, driver_mk_term_nil,(void)); +WDD_TYPEDEF(int, erl_drv_output_term, (ErlDrvTermData, ErlDrvTermData*, int)); WDD_TYPEDEF(int, driver_output_term, (ErlDrvPort, ErlDrvTermData*, int)); +WDD_TYPEDEF(int, erl_drv_send_term, (ErlDrvTermData, ErlDrvTermData, ErlDrvTermData*, int)); WDD_TYPEDEF(int, driver_send_term, (ErlDrvPort, ErlDrvTermData, ErlDrvTermData*, int)); WDD_TYPEDEF(long, driver_async, (ErlDrvPort,unsigned int*,void (*)(void*),void*,void (*)(void*))); WDD_TYPEDEF(int, driver_async_cancel, (unsigned int)); @@ -187,7 +189,9 @@ typedef struct { WDD_FTYPE(driver_connected) *driver_connected; WDD_FTYPE(driver_caller) *driver_caller; WDD_FTYPE(driver_mk_term_nil) *driver_mk_term_nil; + WDD_FTYPE(erl_drv_output_term) *erl_drv_output_term; WDD_FTYPE(driver_output_term) *driver_output_term; + WDD_FTYPE(erl_drv_send_term) *erl_drv_send_term; WDD_FTYPE(driver_send_term) *driver_send_term; WDD_FTYPE(driver_async) *driver_async; WDD_FTYPE(driver_async_cancel) *driver_async_cancel; @@ -294,7 +298,9 @@ extern TWinDynDriverCallbacks WinDynDriverCallbacks; #define driver_connected (WinDynDriverCallbacks.driver_connected) #define driver_caller (WinDynDriverCallbacks.driver_caller) #define driver_mk_term_nil (WinDynDriverCallbacks.driver_mk_term_nil) +#define erl_drv_output_term (WinDynDriverCallbacks.erl_drv_output_term) #define driver_output_term (WinDynDriverCallbacks.driver_output_term) +#define erl_drv_send_term (WinDynDriverCallbacks.erl_drv_send_term) #define driver_send_term (WinDynDriverCallbacks.driver_send_term) #define driver_async (WinDynDriverCallbacks.driver_async) #define driver_async_cancel (WinDynDriverCallbacks.driver_async_cancel) @@ -425,7 +431,9 @@ do { \ ((W).driver_connected) = driver_connected; \ ((W).driver_caller) = driver_caller; \ ((W).driver_mk_term_nil) = driver_mk_term_nil; \ +((W).erl_drv_output_term) = erl_drv_output_term; \ ((W).driver_output_term) = driver_output_term; \ +((W).erl_drv_send_term) = erl_drv_send_term; \ ((W).driver_send_term) = driver_send_term; \ ((W).driver_async) = driver_async; \ ((W).driver_async_cancel) = driver_async_cancel; \ diff --git a/erts/emulator/sys/win32/sys.c b/erts/emulator/sys/win32/sys.c index 47d12ed5fe..1cd9072cea 100755 --- a/erts/emulator/sys/win32/sys.c +++ b/erts/emulator/sys/win32/sys.c @@ -87,9 +87,6 @@ static erts_smp_tsd_key_t win32_errstr_key; static erts_smp_atomic_t pipe_creation_counter; -static erts_smp_mtx_t sys_driver_data_lock; - - /* Results from application_type(_w) is one of */ #define APPL_NONE 0 #define APPL_DOS 1 @@ -97,7 +94,6 @@ static erts_smp_mtx_t sys_driver_data_lock; #define APPL_WIN32 3 static int driver_write(long, HANDLE, byte*, int); -static void common_stop(int); static int create_file_thread(struct async_io* aio, int mode); #ifdef ERTS_SMP static void close_active_handle(ErlDrvPort, HANDLE handle); @@ -115,9 +111,6 @@ BOOL WINAPI ctrl_handler(DWORD dwCtrlType); #define PORT_BUFSIZ 4096 -#define PORT_FREE (-1) -#define PORT_EXITING (-2) - #define DRV_BUF_ALLOC(SZ) \ erts_alloc_fnf(ERTS_ALC_T_DRV_DATA_BUF, (SZ)) #define DRV_BUF_REALLOC(P, SZ) \ @@ -255,8 +248,7 @@ void erl_sys_args(int* argc, char** argv) #endif } -void -erts_sys_prepare_crash_dump(int secs) +int erts_sys_prepare_crash_dump(int secs) { Port *heart_port; Eterm heap[3]; @@ -270,11 +262,16 @@ erts_sys_prepare_crash_dump(int secs) list = CONS(hp, make_small(8), list); hp += 2; /* send to heart port, CMD = 8, i.e. prepare crash dump =o */ - erts_write_to_port(NIL, heart_port, list); + erts_port_output(NULL, ERTS_PORT_SIG_FLG_FORCE_IMM_CALL, heart_port, + heart_port->common.id, list, NULL); + + return 1; } /* Windows - free file descriptors are hopefully available */ /* Alarm not used on windows */ + + return 0; } static void @@ -471,7 +468,7 @@ typedef struct driver_data { byte *inbuf; /* Buffer to use for overlapped read. */ int outBufSize; /* Size of output buffer. */ byte *outbuf; /* Buffer to use for overlapped write. */ - ErlDrvPort port_num; /* The port number. */ + ErlDrvPort port_num; /* The port handle. */ int packet_bytes; /* 0: continous stream, 1, 2, or 4: the number * of bytes in the packet header. */ @@ -481,8 +478,6 @@ typedef struct driver_data { int report_exit; /* Do report exit status for the port */ } DriverData; -static DriverData* driver_data; /* Pointer to array of driver data. */ - /* Driver interfaces */ static ErlDrvData spawn_start(ErlDrvPort, char*, SysDriverOpts*); static ErlDrvData fd_start(ErlDrvPort, char*, SysDriverOpts*); @@ -594,67 +589,53 @@ struct erl_drv_entry vanilla_driver_entry = { */ static DriverData* -new_driver_data(int port_num, int packet_bytes, int wait_objs_required, int use_threads) +new_driver_data(ErlDrvPort port_num, int packet_bytes, int wait_objs_required, int use_threads) { DriverData* dp; - - erts_smp_mtx_lock(&sys_driver_data_lock); - DEBUGF(("new_driver_data(port_num %d, pb %d)\n", - port_num, packet_bytes)); + DEBUGF(("new_driver_data(%p, pb %d)\n", port_num, packet_bytes)); + dp = driver_alloc(sizeof(DriverData)); + if (!dp) + return NULL; /* * We used to test first at all that there is enough room in the * array used by WaitForMultipleObjects(), but that is not necessary * any more, since driver_select() can't fail. */ - /* - * Search for a free slot. - */ + dp->bytesInBuffer = 0; + dp->totalNeeded = packet_bytes; + dp->inBufSize = PORT_BUFSIZ; + dp->inbuf = DRV_BUF_ALLOC(dp->inBufSize); + if (dp->inbuf == NULL) + goto buf_alloc_error; + erts_smp_atomic_add_nob(&sys_misc_mem_sz, dp->inBufSize); + dp->outBufSize = 0; + dp->outbuf = NULL; + dp->port_num = port_num; + dp->packet_bytes = packet_bytes; + dp->port_pid = INVALID_HANDLE_VALUE; + if (init_async_io(&dp->in, use_threads) == -1) + goto async_io_error1; + if (init_async_io(&dp->out, use_threads) == -1) + goto async_io_error2; - for (dp = driver_data; dp < driver_data+max_files; dp++) { - if (dp->port_num == PORT_FREE) { - dp->bytesInBuffer = 0; - dp->totalNeeded = packet_bytes; - dp->inBufSize = PORT_BUFSIZ; - dp->inbuf = DRV_BUF_ALLOC(dp->inBufSize); - if (dp->inbuf == NULL) { - erts_smp_mtx_unlock(&sys_driver_data_lock); - return NULL; - } - erts_smp_atomic_add_nob(&sys_misc_mem_sz, dp->inBufSize); - dp->outBufSize = 0; - dp->outbuf = NULL; - dp->port_num = port_num; - dp->packet_bytes = packet_bytes; - dp->port_pid = INVALID_HANDLE_VALUE; - if (init_async_io(&dp->in, use_threads) == -1) - break; - if (init_async_io(&dp->out, use_threads) == -1) - break; - erts_smp_mtx_unlock(&sys_driver_data_lock); - return dp; - } - } + return dp; - /* - * Error or no free driver data. - */ +async_io_error2: + release_async_io(&dp->in, dp->port_num); +async_io_error1: + release_async_io(&dp->out, dp->port_num); - if (dp < driver_data+max_files) { - release_async_io(&dp->in, dp->port_num); - release_async_io(&dp->out, dp->port_num); - } - erts_smp_mtx_unlock(&sys_driver_data_lock); +buf_alloc_error: + driver_free(dp); return NULL; } static void release_driver_data(DriverData* dp) { - erts_smp_mtx_lock(&sys_driver_data_lock); - #ifdef ERTS_SMP #ifdef USE_CANCELIOEX if (fpCancelIoEx != NULL) { @@ -738,8 +719,7 @@ release_driver_data(DriverData* dp) * the exit thread. */ - dp->port_num = PORT_FREE; - erts_smp_mtx_unlock(&sys_driver_data_lock); + driver_free(dp); } #ifdef ERTS_SMP @@ -834,7 +814,6 @@ threaded_handle_closer(LPVOID param) static ErlDrvData set_driver_data(DriverData* dp, HANDLE ifd, HANDLE ofd, int read_write, int report_exit) { - int index = dp - driver_data; int result; dp->in.fd = ifd; @@ -853,13 +832,12 @@ set_driver_data(DriverData* dp, HANDLE ifd, HANDLE ofd, int read_write, int repo ERL_DRV_WRITE|ERL_DRV_USE, 1); ASSERT(result != -1); } - return (ErlDrvData)index; + return (ErlDrvData) dp; } static ErlDrvData reuse_driver_data(DriverData *dp, HANDLE ifd, HANDLE ofd, int read_write, ErlDrvPort port_num) { - int index = dp - driver_data; int result; dp->port_num = port_num; @@ -878,7 +856,7 @@ reuse_driver_data(DriverData *dp, HANDLE ifd, HANDLE ofd, int read_write, ErlDrv ERL_DRV_WRITE|ERL_DRV_USE, 1); ASSERT(result != -1); } - return (ErlDrvData)index; + return (ErlDrvData) dp; } /* @@ -1151,12 +1129,6 @@ spawn_init(void) ((module != NULL) ? GetProcAddress(module,"CancelIoEx") : NULL); DEBUGF(("fpCancelIoEx = %p\r\n", fpCancelIoEx)); #endif - driver_data = (struct driver_data *) - erts_alloc(ERTS_ALC_T_DRV_TAB, max_files * sizeof(struct driver_data)); - erts_smp_atomic_add_nob(&sys_misc_mem_sz, - max_files*sizeof(struct driver_data)); - for (i = 0; i < max_files; i++) - driver_data[i].port_num = PORT_FREE; return 0; } @@ -1287,9 +1259,12 @@ spawn_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) #endif retval = set_driver_data(dp, hFromChild, hToChild, opts->read_write, opts->exit_status); - if (retval != ERL_DRV_ERROR_GENERAL && retval != ERL_DRV_ERROR_ERRNO) - /* We assume that this cannot generate a negative number */ - erts_port[port_num].os_pid = (SWord) pid; + if (retval != ERL_DRV_ERROR_GENERAL && retval != ERL_DRV_ERROR_ERRNO) { + Port *prt = erts_drvport2port_raw(port_num); + /* We assume that this cannot generate a negative number */ + ASSERT(prt); + prt->os_pid = (SWord) pid; + } } if (retval != ERL_DRV_ERROR_GENERAL && retval != ERL_DRV_ERROR_ERRNO) @@ -2278,12 +2253,10 @@ fd_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) **/ if (!create_file_thread(&dp->in, DO_READ)) { - dp->port_num = PORT_FREE; return ERL_DRV_ERROR_GENERAL; } if (!create_file_thread(&dp->out, DO_WRITE)) { - dp->port_num = PORT_FREE; return ERL_DRV_ERROR_GENERAL; } @@ -2303,10 +2276,9 @@ fd_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) } } -static void fd_stop(ErlDrvData d) +static void fd_stop(ErlDrvData data) { - int fd = (int)d; - DriverData* dp = driver_data+fd; + DriverData * dp = (DriverData *) data; /* * There's no way we can terminate an fd port in a consistent way. * Instead we let it live until it's opened again (which it is, @@ -2369,16 +2341,10 @@ vanilla_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) } static void -stop(ErlDrvData index) +stop(ErlDrvData data) { - common_stop((int)index); -} - -static void common_stop(int index) -{ - DriverData* dp = driver_data+index; - - DEBUGF(("common_stop(%d)\n", index)); + DriverData *dp = (DriverData *) data; + DEBUGF(("stop(%p)\n", dp)); if (dp->in.ov.hEvent != NULL) { (void) driver_select(dp->port_num, @@ -2400,7 +2366,6 @@ static void common_stop(int index) */ HANDLE thread; DWORD tid; - dp->port_num = PORT_EXITING; thread = (HANDLE *) _beginthreadex(NULL, 0, threaded_exiter, dp, 0, &tid); CloseHandle(thread); } @@ -2525,22 +2490,17 @@ threaded_exiter(LPVOID param) static void output(ErlDrvData drv_data, char* buf, ErlDrvSizeT len) -/* long drv_data; /* The slot to use in the driver data table. +/* ErlDrvData drv_data; /* The slot to use in the driver data table. * For Windows NT, this is *NOT* a file handle. * The handle is found in the driver data. */ /* char *buf; /* Pointer to data to write to the port program. */ /* ErlDrvSizeT len; /* Number of bytes to write. */ { - DriverData* dp; + DriverData* dp = (DriverData *) drv_data; int pb; /* The header size for this port. */ - int port_num; /* The actual port number (for diagnostics). */ char* current; - dp = driver_data + (int)drv_data; - if ((port_num = dp->port_num) == -1) - return ; /*-1;*/ - pb = dp->packet_bytes; if ((pb+len) == 0) @@ -2551,7 +2511,7 @@ output(ErlDrvData drv_data, char* buf, ErlDrvSizeT len) */ if ((pb == 2 && len > 65535) || (pb == 1 && len > 255)) { - driver_failure_posix(port_num, EINVAL); + driver_failure_posix(dp->port_num, EINVAL); return ; /* -1; */ } @@ -2565,7 +2525,7 @@ output(ErlDrvData drv_data, char* buf, ErlDrvSizeT len) ASSERT(!dp->outbuf); dp->outbuf = DRV_BUF_ALLOC(pb+len); if (!dp->outbuf) { - driver_failure_posix(port_num, ENOMEM); + driver_failure_posix(dp->port_num, ENOMEM); return ; /* -1; */ } @@ -2595,7 +2555,7 @@ output(ErlDrvData drv_data, char* buf, ErlDrvSizeT len) memcpy(current, buf, len); if (!async_write_file(&dp->out, dp->outbuf, pb+len)) { - set_busy_port(port_num, 1); + set_busy_port(dp->port_num, 1); } else { dp->out.ov.Offset += pb+len; /* For vanilla driver. */ /* XXX OffsetHigh should be changed too. */ @@ -2630,10 +2590,9 @@ ready_input(ErlDrvData drv_data, ErlDrvEvent ready_event) { int error = 0; /* The error code (assume initially no errors). */ DWORD bytesRead; /* Number of bytes read. */ - DriverData* dp; + DriverData* dp = (DriverData *) drv_data; int pb; - dp = driver_data+(int)drv_data; pb = dp->packet_bytes; #ifdef ERTS_SMP if(dp->in.thread == (HANDLE) -1) { @@ -2801,7 +2760,7 @@ static void ready_output(ErlDrvData drv_data, ErlDrvEvent ready_event) { DWORD bytesWritten; - DriverData* dp = driver_data + (int)drv_data; + DriverData *dp = (DriverData *) drv_data; int error; #ifdef ERTS_SMP @@ -2809,7 +2768,7 @@ ready_output(ErlDrvData drv_data, ErlDrvEvent ready_event) dp->out.async_io_active = 0; } #endif - DEBUGF(("ready_output(%d, 0x%x)\n", drv_data, ready_event)); + DEBUGF(("ready_output(%p, 0x%x)\n", drv_data, ready_event)); set_busy_port(dp->port_num, 0); if (!(dp->outbuf)) { /* Happens because event sometimes get signalled during a successful @@ -2864,7 +2823,7 @@ sys_init_io(void) can change our view of the number of open files possible. We estimate the number to twice the amount of ports. We really dont know on windows, do we? */ - max_files = 2*erts_max_ports; + max_files = 2*erts_ptab_max(&erts_port); } #ifdef ERTS_SMP @@ -3319,9 +3278,6 @@ void erl_sys_init(void) noinherit_std_handle(STD_INPUT_HANDLE); noinherit_std_handle(STD_ERROR_HANDLE); - - erts_smp_mtx_init(&sys_driver_data_lock, "sys_driver_data_lock"); - #ifdef ERTS_SMP erts_smp_tsd_key_create(&win32_errstr_key); InitializeCriticalSection(&htbc_lock); diff --git a/erts/emulator/test/bs_bit_binaries_SUITE.erl b/erts/emulator/test/bs_bit_binaries_SUITE.erl index 1428387a65..7f2cd1e881 100644 --- a/erts/emulator/test/bs_bit_binaries_SUITE.erl +++ b/erts/emulator/test/bs_bit_binaries_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2011. All Rights Reserved. +%% Copyright Ericsson AB 2006-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/test/bs_construct_SUITE.erl b/erts/emulator/test/bs_construct_SUITE.erl index 9c88803fea..123952d01d 100644 --- a/erts/emulator/test/bs_construct_SUITE.erl +++ b/erts/emulator/test/bs_construct_SUITE.erl @@ -547,11 +547,47 @@ huge_float_check({'EXIT',{badarg,_}}) -> ok. huge_binary(Config) when is_list(Config) -> ?line 16777216 = size(<<0:(id(1 bsl 26)),(-1):(id(1 bsl 26))>>), ?line garbage_collect(), - ?line id(<<0:((1 bsl 32)-1)>>), + {Shift,Return} = case free_mem() of + undefined -> {32,ok}; + Mb when Mb > 600 -> {32,ok}; + Mb when Mb > 300 -> {31,"Limit huge binaries to 256 Mb"}; + _ -> {30,"Limit huge binary to 128 Mb"} + end, ?line garbage_collect(), - ?line id(<<0:(id((1 bsl 32)-1))>>), + ?line id(<<0:((1 bsl Shift)-1)>>), ?line garbage_collect(), - ok. + ?line id(<<0:(id((1 bsl Shift)-1))>>), + ?line garbage_collect(), + case Return of + ok -> ok; + Comment -> {comment, Comment} + end. + +free_mem() -> + Cmd = "uname; free", + Output = string:tokens(os:cmd(Cmd), "\n"), + io:format("Output from command ~p\n~p\n",[Cmd,Output]), + case Output of + [OS, ColumnNames, Values | _] -> + case string:str(OS,"Linux") of + 0 -> + io:format("Unknown OS\n",[]), + undefined; + _ -> + case {string:tokens(ColumnNames, " \t"), + string:tokens(Values, " \t")} of + {[_,_,"free"|_],["Mem:",_,_,FreeKb|_]} -> + list_to_integer(FreeKb) div 1024; + _ -> + io:format("Failed to parse output from 'free':\n",[]), + undefined + end + end; + _ -> + io:format("Too few lines in output\n",[]), + undefined + end. + system_limit(Config) when is_list(Config) -> WordSize = erlang:system_info(wordsize), diff --git a/erts/emulator/test/busy_port_SUITE.erl b/erts/emulator/test/busy_port_SUITE.erl index 3a29fd4d68..32e907ca69 100644 --- a/erts/emulator/test/busy_port_SUITE.erl +++ b/erts/emulator/test/busy_port_SUITE.erl @@ -148,9 +148,9 @@ message_order(Config) when is_list(Config) -> send_to_busy_1(Parent) -> {Owner, Slave} = get_slave(), - Slave ! {Owner, {command, "set_me_busy"}}, - Slave ! {Owner, {command, "hello"}}, - Slave ! {Owner, {command, "hello again"}}, + (catch port_command(Slave, "set_me_busy")), + (catch port_command(Slave, "hello")), + (catch port_command(Slave, "hello again")), receive Message -> Parent ! {self(), Message} @@ -193,10 +193,10 @@ system_monitor(Config) when is_list(Config) -> ?line Busy = spawn_link( fun() -> - Slave ! {Owner,{command,"set busy"}}, + (catch port_command(Slave, "set busy")), receive {Parent,alpha} -> ok end, - Slave ! {Owner,{command,"busy"}}, - Slave ! {Owner,{command,"free"}}, + (catch port_command(Slave, "busy")), + (catch port_command(Slave, "free")), Parent ! {self(),alpha}, command(lock), receive {Parent,beta} -> ok end, @@ -212,7 +212,7 @@ system_monitor(Config) when is_list(Config) -> ?line Void = rec(Void), ?line Busy ! {self(), beta}, ?line {monitor,Owner,busy_port,Slave} = rec(Void), - ?line Master ! {Owner, {command, "u"}}, + ?line port_command(Master, "u"), ?line {Busy,beta} = rec(Void), ?line Void = rec(Void), ?line _NewMonitor = erlang:system_monitor(OldMonitor), @@ -296,9 +296,9 @@ no_trap_exit_process(ResultTo, Link, Config) -> linked -> ok; unlink -> unlink(Slave) end, - ?line Slave ! {self(), {command, "lock port"}}, + ?line (catch port_command(Slave, "lock port")), ?line ResultTo ! {self(), port_created, Slave}, - ?line Slave ! {self(), {command, "suspend me"}}, + ?line (catch port_command(Slave, "suspend me")), ok. %% Assuming the following scenario, @@ -339,9 +339,9 @@ busy_port_exit_process(ResultTo, Config) -> ?line load_busy_driver(Config), ?line _Master = open_port({spawn, "busy_drv master"}, [eof]), ?line Slave = open_port({spawn, "busy_drv slave"}, [eof]), - ?line Slave ! {self(), {command, "lock port"}}, + ?line (catch port_command(Slave, "lock port")), ?line ResultTo ! {self(), port_created, Slave}, - ?line Slave ! {self(), {command, "suspend me"}}, + ?line (catch port_command(Slave, "suspend me")), receive {'EXIT', Slave, die} -> ResultTo ! {self(), ok}; @@ -383,8 +383,8 @@ multiple_writers(Config) when is_list(Config) -> quick_writer() -> {Owner, Port} = get_slave(), - Port ! {Owner, {command, "port to busy"}}, - Port ! {Owner, {command, "lock me"}}, + (catch port_command(Port, "port to busy")), + (catch port_command(Port, "lock me")), ok. hard_busy_driver(Config) when is_list(Config) -> @@ -644,11 +644,11 @@ loop(Master, Slave) -> Pid ! {busy_drv_reply, {self(), Slave}}, loop(Master, Slave); {Pid, unlock} -> - Master ! {self(), {command, "u"}}, + port_command(Master, "u"), Pid ! {busy_drv_reply, ok}, loop(Master, Slave); {Pid, lock} -> - Master ! {self(), {command, "l"}}, + port_command(Master, "l"), Pid ! {busy_drv_reply, ok}, loop(Master, Slave); {Pid, {port_command,Data}} -> diff --git a/erts/emulator/test/ddll_SUITE.erl b/erts/emulator/test/ddll_SUITE.erl index 6e15c228cd..4675cab15c 100644 --- a/erts/emulator/test/ddll_SUITE.erl +++ b/erts/emulator/test/ddll_SUITE.erl @@ -136,8 +136,8 @@ delayed_unload_with_ports(Config) when is_list(Config) -> ?line {ok,pending_driver,Ref} = erl_ddll:try_unload(echo_drv,[{monitor, pending_driver}]), ?line ok = receive _ -> false after 0 -> ok end, ?line Port ! {self(), close}, - ?line 1 = erl_ddll:info(echo_drv, port_count), ?line ok = receive {Port,closed} -> ok after 1000 -> false end, + ?line 1 = erl_ddll:info(echo_drv, port_count), ?line Port2 ! {self(), close}, ?line ok = receive {Port2,closed} -> ok after 1000 -> false end, ?line ok = receive {'DOWN', Ref, driver, echo_drv, unloaded} -> ok after 1000 -> false end, diff --git a/erts/emulator/test/driver_SUITE.erl b/erts/emulator/test/driver_SUITE.erl index 643357263c..13f18b4563 100644 --- a/erts/emulator/test/driver_SUITE.erl +++ b/erts/emulator/test/driver_SUITE.erl @@ -77,7 +77,8 @@ thread_mseg_alloc_cache_clean/1, otp_9302/1, thr_free_drv/1, - async_blast/1]). + async_blast/1, + thr_msg_blast/1]). -export([bin_prefix/2]). @@ -147,7 +148,8 @@ all() -> thread_mseg_alloc_cache_clean, otp_9302, thr_free_drv, - async_blast]. + async_blast, + thr_msg_blast]. groups() -> [{timer, [], @@ -1136,7 +1138,9 @@ check_driver_system_info_result(Result) -> {{1, 1}, _} -> ?line ExpNs = lists:sort(?EXPECTED_SYSTEM_INFO_NAMES -- ?EXPECTED_SYSTEM_INFO_NAMES2), - ?line ExpNs = lists:sort(Ns) + ?line ExpNs = lists:sort(Ns); + {{2, 0}, _} -> + ?line [] = Ns end. chk_sis(SIs, Ns) -> @@ -2010,7 +2014,64 @@ async_blast(Config) when is_list(Config) -> ?line erlang:display({async_blast_time, AsyncBlastTime}), ?line ok. +thr_msg_blast_receiver(_Port, N, N) -> + ok; +thr_msg_blast_receiver(Port, N, Max) -> + receive + {Port, hi} -> + thr_msg_blast_receiver(Port, N+1, Max) + end. + +thr_msg_blast_receiver_proc(Port, Max, Parent, Done) -> + case port_control(Port, 0, "") of + "receiver" -> + spawn(fun () -> + thr_msg_blast_receiver_proc(Port, Max+1, Parent, Done) + end), + thr_msg_blast_receiver(Port, 0, Max); + "done" -> + Parent ! Done + end. +thr_msg_blast(Config) when is_list(Config) -> + case erlang:system_info(smp_support) of + false -> + {skipped, "Non-SMP emulator; nothing to test..."}; + true -> + Path = ?config(data_dir, Config), + erl_ddll:start(), + ok = load_driver(Path, thr_msg_blast_drv), + MemBefore = driver_alloc_size(), + Start = os:timestamp(), + Port = open_port({spawn, thr_msg_blast_drv}, []), + true = is_port(Port), + Done = make_ref(), + Me = self(), + spawn(fun () -> + thr_msg_blast_receiver_proc(Port, 1, Me, Done) + end), + receive + Done -> ok + end, + ok = thr_msg_blast_receiver(Port, 0, 32*10000), + port_close(Port), + End = os:timestamp(), + receive + Garbage -> + ?t:fail({received_garbage, Port, Garbage}) + after 2000 -> + ok + end, + MemAfter = driver_alloc_size(), + io:format("MemBefore=~p, MemAfter=~p~n", + [MemBefore, MemAfter]), + ThrMsgBlastTime = timer:now_diff(End,Start)/1000000, + io:format("ThrMsgBlastTime=~p~n", [ThrMsgBlastTime]), + MemBefore = MemAfter, + Res = {thr_msg_blast_time, ThrMsgBlastTime}, + erlang:display(Res), + Res + end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Utilities diff --git a/erts/emulator/test/driver_SUITE_data/Makefile.src b/erts/emulator/test/driver_SUITE_data/Makefile.src index 9cc107cc66..b667dff6b6 100644 --- a/erts/emulator/test/driver_SUITE_data/Makefile.src +++ b/erts/emulator/test/driver_SUITE_data/Makefile.src @@ -14,7 +14,8 @@ MISC_DRVS = outputv_drv@dll@ \ thr_alloc_drv@dll@ \ otp_9302_drv@dll@ \ thr_free_drv@dll@ \ - async_blast_drv@dll@ + async_blast_drv@dll@ \ + thr_msg_blast_drv@dll@ SYS_INFO_DRVS = sys_info_base_drv@dll@ \ sys_info_prev_drv@dll@ \ diff --git a/erts/emulator/test/driver_SUITE_data/thr_msg_blast_drv.c b/erts/emulator/test/driver_SUITE_data/thr_msg_blast_drv.c new file mode 100644 index 0000000000..1070678d7b --- /dev/null +++ b/erts/emulator/test/driver_SUITE_data/thr_msg_blast_drv.c @@ -0,0 +1,178 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +#include "erl_driver.h" + +#define THR_MSG_BLAST_NO_PROCS 10 +#define THR_MSG_BLAST_NO_SENDS_PER_PROC 10000 + +#define THR_MSG_BLAST_THREADS 32 + +static void stop(ErlDrvData drv_data); +static ErlDrvData start(ErlDrvPort port, + char *command); +static ErlDrvSSizeT control(ErlDrvData drv_data, + unsigned int command, + char *buf, ErlDrvSizeT len, + char **rbuf, ErlDrvSizeT rlen); + +static ErlDrvEntry thr_msg_blast_drv_entry = { + NULL /* init */, + start, + stop, + NULL /* output */, + NULL /* ready_input */, + NULL /* ready_output */, + "thr_msg_blast_drv", + NULL /* finish */, + NULL /* handle */, + control, + NULL /* timeout */, + NULL /* outputv */, + NULL /* ready_async */, + NULL /* flush */, + NULL /* call */, + NULL /* event */, + ERL_DRV_EXTENDED_MARKER, + ERL_DRV_EXTENDED_MAJOR_VERSION, + ERL_DRV_EXTENDED_MINOR_VERSION, + ERL_DRV_FLAG_USE_PORT_LOCKING, + NULL /* handle2 */, + NULL /* handle_monitor */ +}; + +typedef struct { + ErlDrvPort port; + ErlDrvTermData td_port; + ErlDrvTermData hi; + ErlDrvTid tid[THR_MSG_BLAST_THREADS]; + int no_thrs; + ErlDrvTermData proc[THR_MSG_BLAST_NO_PROCS]; + int no_procs; +} thr_msg_blast_data_t; + + +DRIVER_INIT(thr_msg_blast_drv) +{ + return &thr_msg_blast_drv_entry; +} + +static void stop(ErlDrvData drv_data) +{ + int i; + thr_msg_blast_data_t *tmbd = (thr_msg_blast_data_t *) drv_data; + for (i = 0; i < tmbd->no_thrs; i++) + erl_drv_thread_join(tmbd->tid[i], NULL); + driver_free((void *) tmbd); +} + +static ErlDrvData start(ErlDrvPort port, + char *command) +{ + thr_msg_blast_data_t *tmbd; + + tmbd = driver_alloc(sizeof(thr_msg_blast_data_t)); + if (!tmbd) + return ERL_DRV_ERROR_GENERAL; + + tmbd->port = port; + tmbd->td_port = driver_mk_port(port); + tmbd->hi = driver_mk_atom("hi"); + tmbd->no_thrs = 0; + tmbd->no_procs = 1; + tmbd->proc[0] = driver_caller(port); + + return (ErlDrvData) tmbd; +} + +static void *thread(void *); + +static ErlDrvSSizeT control(ErlDrvData drv_data, + unsigned int command, + char *buf, ErlDrvSizeT len, + char **rbuf, ErlDrvSizeT rlen) +{ + thr_msg_blast_data_t *tmbd = (thr_msg_blast_data_t *) drv_data; + char *res_str = "error"; + + if (tmbd->no_procs >= THR_MSG_BLAST_NO_PROCS) { + int i; + for (i = 0; i < tmbd->no_thrs; i++) + erl_drv_thread_join(tmbd->tid[i], NULL); + tmbd->no_thrs = 0; + res_str = "done"; + } + else { + + tmbd->proc[tmbd->no_procs++] = driver_caller(tmbd->port); + + if (tmbd->no_procs == THR_MSG_BLAST_NO_PROCS) { + for (tmbd->no_thrs = 0; + tmbd->no_thrs < THR_MSG_BLAST_THREADS; + tmbd->no_thrs++) { + int res = erl_drv_thread_create("test", + &tmbd->tid[tmbd->no_thrs], + thread, + tmbd, + NULL); + if (res != 0) { + driver_failure_posix(tmbd->port, res); + goto done; + } + } + } + + res_str = "receiver"; + } + + done: { + ErlDrvSSizeT res_len = strlen(res_str); + if (res_len > rlen) { + char *abuf = driver_alloc(sizeof(char)*res_len); + if (!abuf) + return 0; + *rbuf = abuf; + } + + memcpy((void *) *rbuf, (void *) res_str, res_len); + + return res_len; + } +} + +static void *thread(void *varg) +{ + int s, p; + thr_msg_blast_data_t *tmbd = (thr_msg_blast_data_t *) varg; + ErlDrvTermData spec[] = { + ERL_DRV_PORT, tmbd->td_port, + ERL_DRV_ATOM, tmbd->hi, + ERL_DRV_TUPLE, 2 + }; + + for (s = 0; s < THR_MSG_BLAST_NO_SENDS_PER_PROC; s++) { + for (p = 0; p < THR_MSG_BLAST_NO_PROCS; p++) { + int res = driver_send_term(tmbd->port, tmbd->proc[p], + spec, sizeof(spec)/sizeof(spec[0])); + if (p == 0 && res <= 0) + abort(); /* Could not send to creator */ + } + } + return NULL; +} diff --git a/erts/emulator/test/fun_SUITE.erl b/erts/emulator/test/fun_SUITE.erl index ef06845cf2..36ba4e0f48 100644 --- a/erts/emulator/test/fun_SUITE.erl +++ b/erts/emulator/test/fun_SUITE.erl @@ -726,8 +726,8 @@ t_arity(Config) when is_list(Config) -> ok. t_is_function2(Config) when is_list(Config) -> - ?line true = is_function({a,b}, 0), - ?line true = is_function({a,b}, 234343434333433433), + false = is_function(id({a,b}), 0), + false = is_function(id({a,b}), 234343434333433433), ?line true = is_function(fun() -> ok end, 0), ?line true = is_function(fun(_) -> ok end, 1), ?line false = is_function(fun(_) -> ok end, 0), diff --git a/erts/emulator/test/port_SUITE.erl b/erts/emulator/test/port_SUITE.erl index 873601ddd1..13aa0f4c00 100644 --- a/erts/emulator/test/port_SUITE.erl +++ b/erts/emulator/test/port_SUITE.erl @@ -90,7 +90,7 @@ mix_up_ports/1, otp_5112/1, otp_5119/1, otp_6224/1, exit_status_multi_scheduling_block/1, ports/1, spawn_driver/1, spawn_executable/1, close_deaf_port/1, - unregister_name/1]). + unregister_name/1, parallelism_option/1]). -export([]). @@ -114,7 +114,8 @@ all() -> stderr_to_stdout, otp_3906, otp_4389, win_massive, mix_up_ports, otp_5112, otp_5119, exit_status_multi_scheduling_block, ports, spawn_driver, - spawn_executable, close_deaf_port, unregister_name]. + spawn_executable, close_deaf_port, unregister_name, + parallelism_option]. groups() -> [{stream, [], [stream_small, stream_big]}, @@ -159,11 +160,11 @@ win_massive(Config) when is_list(Config) -> do_win_massive() -> Dog = test_server:timetrap(test_server:seconds(360)), SuiteDir = filename:dirname(code:which(?MODULE)), - Env = " -env ERL_MAX_PORTS 8192", + Ports = " +Q 8192", {ok, Node} = test_server:start_node(win_massive, slave, - [{args, " -pa " ++ SuiteDir ++ Env}]), + [{args, " -pa " ++ SuiteDir ++ Ports}]), ok = rpc:call(Node,?MODULE,win_massive_client,[3000]), test_server:stop_node(Node), test_server:timetrap_cancel(Dog), @@ -1298,6 +1299,43 @@ spawn_driver(Config) when is_list(Config) -> test_server:timetrap_cancel(Dog), ok. +parallelism_option(suite) -> + []; +parallelism_option(doc) -> + ["Test parallelism option of open_port"]; +parallelism_option(Config) when is_list(Config) -> + ?line Dog = test_server:timetrap(test_server:seconds(10)), + ?line Path = ?config(data_dir, Config), + ?line ok = load_driver(Path, "echo_drv"), + ?line Port = erlang:open_port({spawn_driver, "echo_drv"}, + [{parallelism, true}]), + ?line {parallelism, true} = erlang:port_info(Port, parallelism), + ?line Port ! {self(), {command, "Hello port!"}}, + ?line receive + {Port, {data, "Hello port!"}} = Msg1 -> + io:format("~p~n", [Msg1]), + ok; + Other -> + test_server:fail({unexpected, Other}) + end, + ?line Port ! {self(), close}, + ?line receive {Port, closed} -> ok end, + + ?line Port2 = erlang:open_port({spawn_driver, "echo_drv -Hello port?"}, + [{parallelism, false}]), + ?line {parallelism, false} = erlang:port_info(Port2, parallelism), + ?line receive + {Port2, {data, "Hello port?"}} = Msg2 -> + io:format("~p~n", [Msg2]), + ok; + Other2 -> + test_server:fail({unexpected2, Other2}) + end, + ?line Port2 ! {self(), close}, + ?line receive {Port2, closed} -> ok end, + ?line test_server:timetrap_cancel(Dog), + ok. + spawn_executable(suite) -> []; spawn_executable(doc) -> @@ -1566,6 +1604,7 @@ otp_5112(Config) when is_list(Config) -> ?t:format("Links1: ~p~n",[Links1]), true = lists:member(Port, Links1), Port ! {self(), {command, ""}}, + ?line wait_until(fun () -> lists:member(Port, erlang:ports()) == false end), {links, Links2} = process_info(self(),links), ?t:format("Links2: ~p~n",[Links2]), false = lists:member(Port, Links2), %% This used to fail @@ -1636,38 +1675,8 @@ otp_5119_fill_empty_port_tab(Ports) -> LastPort end. --define(DEF_MAX_PORTS, 1024). - -max_ports_env() -> - case os:getenv("ERL_MAX_PORTS") of - EMP when is_list(EMP) -> - case catch list_to_integer(EMP) of - Int when is_integer(Int) -> Int; - _ -> false - end; - _ -> false - end. - max_ports() -> - PreMaxPorts - = case max_ports_env() of - Env when is_integer(Env) -> Env; - _ -> - case os:type() of - {unix, _} -> - UlimStr = string:strip(os:cmd("ulimit -n") - -- "\n"), - case catch list_to_integer(UlimStr) of - Ulim when is_integer(Ulim) -> Ulim; - _ -> ?DEF_MAX_PORTS - end; - _ -> ?DEF_MAX_PORTS - end - end, - case PreMaxPorts > ?DEF_MAX_PORTS of - true -> PreMaxPorts; - false -> ?DEF_MAX_PORTS - end. + erlang:system_info(port_limit). port_ix(Port) when is_port(Port) -> ["#Port",_,PortIxStr] = string:tokens(erlang:port_to_list(Port), @@ -2270,5 +2279,12 @@ close_deaf_port_1(N, Cmd) -> _:eagain -> {comment, "Could not spawn more than " ++ integer_to_list(N) ++ " OS processes."} end. - +wait_until(Fun) -> + case catch Fun() of + true -> + ok; + _ -> + receive after 100 -> ok end, + wait_until(Fun) + end. diff --git a/erts/emulator/test/process_SUITE.erl b/erts/emulator/test/process_SUITE.erl index 6509871a7d..898535693f 100644 --- a/erts/emulator/test/process_SUITE.erl +++ b/erts/emulator/test/process_SUITE.erl @@ -1384,7 +1384,7 @@ otp_6237_select_loop() -> -define(NoTestProcs, 10000). --record(processes_bif_info, {min_start_reds, +-record(ptab_list_bif_info, {min_start_reds, tab_chunks, tab_chunks_size, tab_indices_per_red, @@ -1419,11 +1419,11 @@ processes_large_tab(Config) when is_list(Config) -> %% the connection times out; therefore, shrink the test on %% high debug levels. ?line DbgLvl = case erts_debug:get_internal_state(processes_bif_info) of - #processes_bif_info{debug_level = Lvl} when Lvl > MaxDbgLvl -> + #ptab_list_bif_info{debug_level = Lvl} when Lvl > MaxDbgLvl -> 20; - #processes_bif_info{debug_level = Lvl} when Lvl < 0 -> + #ptab_list_bif_info{debug_level = Lvl} when Lvl < 0 -> ?line ?t:fail({debug_level, Lvl}); - #processes_bif_info{debug_level = Lvl} -> + #ptab_list_bif_info{debug_level = Lvl} -> Lvl end, ?line ProcTabSize3 = ProcTabSize2 - (1300000 * DbgLvl div MaxDbgLvl), @@ -1438,7 +1438,7 @@ processes_large_tab(Config) when is_list(Config) -> erts_debug, get_internal_state, [processes_bif_info]) of - #processes_bif_info{tab_chunks = Chunks} when is_integer(Chunks), + #ptab_list_bif_info{tab_chunks = Chunks} when is_integer(Chunks), Chunks > 1 -> ok; PBInfo -> ?t:fail(PBInfo) end, @@ -1460,14 +1460,14 @@ processes_small_tab(doc) -> processes_small_tab(suite) -> []; processes_small_tab(Config) when is_list(Config) -> - ?line {ok, SmallNode} = start_node(Config, "+P 500"), + ?line {ok, SmallNode} = start_node(Config, "+P 1024"), ?line Res = rpc:call(SmallNode, ?MODULE, processes_bif_test, []), ?line PBInfo = rpc:call(SmallNode, erts_debug, get_internal_state, [processes_bif_info]), ?line stop_node(SmallNode), - ?line 1 = PBInfo#processes_bif_info.tab_chunks, + ?line true = PBInfo#ptab_list_bif_info.tab_chunks < 10, ?line chk_processes_bif_test_res(Res). processes_this_tab(doc) -> @@ -1481,7 +1481,7 @@ chk_processes_bif_test_res(ok) -> ok; chk_processes_bif_test_res({comment, _} = Comment) -> Comment; chk_processes_bif_test_res(Failure) -> ?t:fail(Failure). -print_processes_bif_info(#processes_bif_info{min_start_reds = MinStartReds, +print_processes_bif_info(#ptab_list_bif_info{min_start_reds = MinStartReds, tab_chunks = TabChunks, tab_chunks_size = TabChunksSize, tab_indices_per_red = TabIndPerRed, @@ -1579,11 +1579,11 @@ processes_bif_test() -> ?line enable_internal_state(), ?line PBInfo = erts_debug:get_internal_state(processes_bif_info), ?line print_processes_bif_info(PBInfo), - ?line WantReds = PBInfo#processes_bif_info.min_start_reds + 10, + ?line WantReds = PBInfo#ptab_list_bif_info.min_start_reds + 10, ?line WillTrap = case PBInfo of - #processes_bif_info{tab_chunks = 1} -> - false; - #processes_bif_info{tab_chunks = Chunks, + #ptab_list_bif_info{tab_chunks = Chunks} when Chunks < 10 -> + false; %% Skip for small tables + #ptab_list_bif_info{tab_chunks = Chunks, tab_chunks_size = ChunksSize, tab_indices_per_red = IndiciesPerRed } -> @@ -1618,7 +1618,7 @@ processes_bif_test() -> ?line erlang:system_flag(multi_scheduling, unblock), ?line [{status,suspended}, - {current_function,{erlang,processes_trap,2}}] + {current_function,{erlang,ptab_list_continue,2}}] = process_info(Suspendee, [status, current_function]), ?line ok = do_processes_bif_test(WantReds, WillTrap, Processes), @@ -1785,7 +1785,7 @@ do_processes_bif_die_test(N, Processes) -> catch throw:{kill_in_trap, R} when N > 0 -> ?t:format("Failed to kill in trap: ~p~n", [R]), - ?t:format("Trying again~p~n", []), + ?t:format("Trying again~n", []), do_processes_bif_die_test(N-1, Processes) end. @@ -1848,7 +1848,7 @@ processes_last_call_trap(Config) when is_list(Config) -> ?line Processes = fun () -> processes() end, ?line PBInfo = erts_debug:get_internal_state(processes_bif_info), ?line print_processes_bif_info(PBInfo), - ?line WantReds = case PBInfo#processes_bif_info.min_start_reds of + ?line WantReds = case PBInfo#ptab_list_bif_info.min_start_reds of R when R > 10 -> R - 1; _R -> 9 end, @@ -1873,7 +1873,7 @@ processes_apply_trap(Config) when is_list(Config) -> ?line enable_internal_state(), ?line PBInfo = erts_debug:get_internal_state(processes_bif_info), ?line print_processes_bif_info(PBInfo), - ?line WantReds = case PBInfo#processes_bif_info.min_start_reds of + ?line WantReds = case PBInfo#ptab_list_bif_info.min_start_reds of R when R > 10 -> R - 1; _R -> 9 end, @@ -1893,7 +1893,7 @@ processes_gc_trap(Config) when is_list(Config) -> ?line enable_internal_state(), ?line PBInfo = erts_debug:get_internal_state(processes_bif_info), ?line print_processes_bif_info(PBInfo), - ?line WantReds = PBInfo#processes_bif_info.min_start_reds + 10, + ?line WantReds = PBInfo#ptab_list_bif_info.min_start_reds + 10, ?line Processes = fun () -> erts_debug:set_internal_state(reds_left,WantReds), processes() @@ -1911,7 +1911,7 @@ processes_gc_trap(Config) when is_list(Config) -> ?line erlang:suspend_process(Suspendee), ?line erlang:system_flag(multi_scheduling, unblock), - ?line [{status,suspended}, {current_function,{erlang,processes_trap,2}}] + ?line [{status,suspended}, {current_function,{erlang,ptab_list_continue,2}}] = process_info(Suspendee, [status, current_function]), ?line erlang:garbage_collect(Suspendee), @@ -1982,8 +1982,8 @@ chk_term_proc_list(Line, MustChk, ExpectBlks) -> not_enabled; {_, MS} -> {value, - {processes_term_proc_el, - DL}} = lists:keysearch(processes_term_proc_el, 1, MS), + {ptab_list_deleted_el, + DL}} = lists:keysearch(ptab_list_deleted_el, 1, MS), case lists:keysearch(blocks, 1, DL) of {value, {blocks, ExpectBlks, _, _}} -> ok; @@ -2001,8 +2001,8 @@ processes_term_proc_list_test(MustChk) -> ?line enable_internal_state(), ?line PBInfo = erts_debug:get_internal_state(processes_bif_info), ?line print_processes_bif_info(PBInfo), - ?line WantReds = PBInfo#processes_bif_info.min_start_reds + 10, - ?line #processes_bif_info{tab_chunks = Chunks, + ?line WantReds = PBInfo#ptab_list_bif_info.min_start_reds + 10, + ?line #ptab_list_bif_info{tab_chunks = Chunks, tab_chunks_size = ChunksSize, tab_indices_per_red = IndiciesPerRed } = PBInfo, @@ -2038,7 +2038,7 @@ processes_term_proc_list_test(MustChk) -> erlang:suspend_process(P), erlang:system_flag(multi_scheduling, unblock), [{status,suspended}, - {current_function,{erlang,processes_trap,2}}] + {current_function,{erlang,ptab_list_continue,2}}] = process_info(P, [status, current_function]), P end, diff --git a/erts/emulator/test/smoke_test_SUITE.erl b/erts/emulator/test/smoke_test_SUITE.erl index 71186d4942..6f5c2080c0 100644 --- a/erts/emulator/test/smoke_test_SUITE.erl +++ b/erts/emulator/test/smoke_test_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011. All Rights Reserved. +%% Copyright Ericsson AB 2011-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/test/tuple_SUITE.erl b/erts/emulator/test/tuple_SUITE.erl index bfc3910742..978d96a110 100644 --- a/erts/emulator/test/tuple_SUITE.erl +++ b/erts/emulator/test/tuple_SUITE.erl @@ -60,40 +60,40 @@ end_per_group(_GroupName, Config) -> build_and_match(Config) when is_list(Config) -> - ?line {} = id({}), - ?line {1} = id({1}), - ?line {1, 2} = id({1, 2}), - ?line {1, 2, 3} = id({1, 2, 3}), - ?line {1, 2, 3, 4} = id({1, 2, 3, 4}), - ?line {1, 2, 3, 4, 5} = id({1, 2, 3, 4, 5}), - ?line {1, 2, 3, 4, 5, 6} = id({1, 2, 3, 4, 5, 6}), - ?line {1, 2, 3, 4, 5, 6} = id({1, 2, 3, 4, 5, 6}), - ?line {1, 2, 3, 4, 5, 6, 7} = id({1, 2, 3, 4, 5, 6, 7}), - ?line {1, 2, 3, 4, 5, 6, 7, 8} = id({1, 2, 3, 4, 5, 6, 7, 8}), + {} = id({}), + {1} = id({1}), + {1, 2} = id({1, 2}), + {1, 2, 3} = id({1, 2, 3}), + {1, 2, 3, 4} = id({1, 2, 3, 4}), + {1, 2, 3, 4, 5} = id({1, 2, 3, 4, 5}), + {1, 2, 3, 4, 5, 6} = id({1, 2, 3, 4, 5, 6}), + {1, 2, 3, 4, 5, 6} = id({1, 2, 3, 4, 5, 6}), + {1, 2, 3, 4, 5, 6, 7} = id({1, 2, 3, 4, 5, 6, 7}), + {1, 2, 3, 4, 5, 6, 7, 8} = id({1, 2, 3, 4, 5, 6, 7, 8}), ok. %% Tests size(Tuple). t_size(Config) when is_list(Config) -> - ?line 0 = size({}), - ?line 1 = size({a}), - ?line 1 = size({{a}}), - ?line 2 = size({{a}, {b}}), - ?line 3 = size({1, 2, 3}), + 0 = size({}), + 1 = size({a}), + 1 = size({{a}}), + 2 = size({{a}, {b}}), + 3 = size({1, 2, 3}), ok. t_tuple_size(Config) when is_list(Config) -> - ?line 0 = tuple_size(id({})), - ?line 1 = tuple_size(id({a})), - ?line 1 = tuple_size(id({{a}})), - ?line 2 = tuple_size(id({{a},{b}})), - ?line 3 = tuple_size(id({1,2,3})), + 0 = tuple_size(id({})), + 1 = tuple_size(id({a})), + 1 = tuple_size(id({{a}})), + 2 = tuple_size(id({{a},{b}})), + 3 = tuple_size(id({1,2,3})), %% Error cases. - ?line {'EXIT',{badarg,_}} = (catch tuple_size([])), - ?line {'EXIT',{badarg,_}} = (catch tuple_size(<<1,2,3>>)), - ?line error = ludicrous_tuple_size({a,b,c}), - ?line error = ludicrous_tuple_size([a,b,c]), + {'EXIT',{badarg,_}} = (catch tuple_size([])), + {'EXIT',{badarg,_}} = (catch tuple_size(<<1,2,3>>)), + error = ludicrous_tuple_size({a,b,c}), + error = ludicrous_tuple_size([a,b,c]), ok. @@ -104,44 +104,44 @@ ludicrous_tuple_size(_) -> error. %% Tests element/2. t_element(Config) when is_list(Config) -> - ?line a = element(1, {a}), - ?line a = element(1, {a, b}), + a = element(1, {a}), + a = element(1, {a, b}), - ?line List = lists:seq(1, 4096), - ?line Tuple = list_to_tuple(lists:seq(1, 4096)), - ?line get_elements(List, Tuple, 1), + List = lists:seq(1, 4096), + Tuple = list_to_tuple(lists:seq(1, 4096)), + get_elements(List, Tuple, 1), - ?line {'EXIT', {badarg, _}} = (catch element(0, id({a,b}))), - ?line {'EXIT', {badarg, _}} = (catch element(3, id({a,b}))), - ?line {'EXIT', {badarg, _}} = (catch element(1, id({}))), - ?line {'EXIT', {badarg, _}} = (catch element(1, id([a,b]))), - ?line {'EXIT', {badarg, _}} = (catch element(1, id(42))), - ?line {'EXIT', {badarg, _}} = (catch element(id(1.5), id({a,b}))), + {'EXIT', {badarg, _}} = (catch element(0, id({a,b}))), + {'EXIT', {badarg, _}} = (catch element(3, id({a,b}))), + {'EXIT', {badarg, _}} = (catch element(1, id({}))), + {'EXIT', {badarg, _}} = (catch element(1, id([a,b]))), + {'EXIT', {badarg, _}} = (catch element(1, id(42))), + {'EXIT', {badarg, _}} = (catch element(id(1.5), id({a,b}))), ok. get_elements([Element|Rest], Tuple, Pos) -> - ?line Element = element(Pos, Tuple), - ?line get_elements(Rest, Tuple, Pos+1); + Element = element(Pos, Tuple), + get_elements(Rest, Tuple, Pos+1); get_elements([], _Tuple, _Pos) -> ok. %% Tests set_element/3. t_setelement(Config) when is_list(Config) -> - ?line {x} = setelement(1, id({1}), x), - ?line {x,2} = setelement(1, id({1,2}), x), - ?line {1,x} = setelement(2, id({1,2}), x), + {x} = setelement(1, id({1}), x), + {x,2} = setelement(1, id({1,2}), x), + {1,x} = setelement(2, id({1,2}), x), - ?line Tuple = list_to_tuple(lists:duplicate(2048, x)), - ?line NewTuple = set_all_elements(Tuple, 1), - ?line NewTuple = list_to_tuple(lists:seq(1+7, 2048+7)), + Tuple = list_to_tuple(lists:duplicate(2048, x)), + NewTuple = set_all_elements(Tuple, 1), + NewTuple = list_to_tuple(lists:seq(1+7, 2048+7)), - ?line {'EXIT', {badarg, _}} = (catch setelement(0, {a, b}, x)), - ?line {'EXIT', {badarg, _}} = (catch setelement(3, {a, b}, x)), - ?line {'EXIT', {badarg, _}} = (catch setelement(1, {}, x)), - ?line {'EXIT', {badarg, _}} = (catch setelement(1, [a, b], x)), - ?line {'EXIT', {badarg, _}} = (catch setelement(1.5, {a, b}, x)), + {'EXIT', {badarg, _}} = (catch setelement(0, {a, b}, x)), + {'EXIT', {badarg, _}} = (catch setelement(3, {a, b}, x)), + {'EXIT', {badarg, _}} = (catch setelement(1, {}, x)), + {'EXIT', {badarg, _}} = (catch setelement(1, [a, b], x)), + {'EXIT', {badarg, _}} = (catch setelement(1.5, {a, b}, x)), %% Nested setelement with literals. AnotherTuple = id({0,0,a,b,c}), @@ -159,52 +159,68 @@ set_all_elements(Tuple, Pos) when Pos > size(Tuple) -> %% Tests list_to_tuple/1. t_list_to_tuple(Config) when is_list(Config) -> - ?line {} = list_to_tuple([]), - ?line {a} = list_to_tuple([a]), - ?line {a, b} = list_to_tuple([a, b]), - ?line {a, b, c} = list_to_tuple([a, b, c]), - ?line {a, b, c, d} = list_to_tuple([a, b, c, d]), - ?line {a, b, c, d, e} = list_to_tuple([a, b, c, d, e]), - - ?line Size = 4096, - ?line Tuple = list_to_tuple(lists:seq(1, Size)), - ?line Size = size(Tuple), - - ?line {'EXIT', {badarg, _}} = (catch list_to_tuple(id({a,b}))), - ?line {'EXIT', {badarg, _}} = (catch list_to_tuple(id([a|b]))), - ?line {'EXIT', {badarg, _}} = (catch list_to_tuple(id([a|b]))), - + {} = list_to_tuple([]), + {a} = list_to_tuple([a]), + {a, b} = list_to_tuple([a, b]), + {a, b, c} = list_to_tuple([a, b, c]), + {a, b, c, d} = list_to_tuple([a, b, c, d]), + {a, b, c, d, e} = list_to_tuple([a, b, c, d, e]), + + Size = 4096, + Tuple = list_to_tuple(lists:seq(1, Size)), + Size = size(Tuple), + + {'EXIT', {badarg, _}} = (catch list_to_tuple(id({a,b}))), + {'EXIT', {badarg, _}} = (catch list_to_tuple(id([a|b]))), + {'EXIT', {badarg, _}} = (catch list_to_tuple(id([a|b]))), + + % test upper boundry, 16777215 elements + MaxSize = 1 bsl 24 - 1, + MaxTuple = list_to_tuple(lists:seq(1, MaxSize)), + MaxSize = size(MaxTuple), + + {'EXIT', {badarg,_}} = (catch list_to_tuple(lists:seq(1, 1 bsl 24))), ok. %% Tests tuple_to_list/1. t_tuple_to_list(Config) when is_list(Config) -> - ?line [] = tuple_to_list({}), - ?line [a] = tuple_to_list({a}), - ?line [a, b] = tuple_to_list({a, b}), - ?line [a, b, c] = tuple_to_list({a, b, c}), - ?line [a, b, c, d] = tuple_to_list({a, b, c, d}), - ?line [a, b, c, d] = tuple_to_list({a, b, c, d}), - - ?line Size = 4096, - ?line List = lists:seq(1, Size), - ?line Tuple = list_to_tuple(List), - ?line Size = size(Tuple), - ?line List = tuple_to_list(Tuple), - - ?line {'EXIT', {badarg,_}} = (catch tuple_to_list(id(a))), - ?line {'EXIT', {badarg,_}} = (catch tuple_to_list(id(42))), + [] = tuple_to_list({}), + [a] = tuple_to_list({a}), + [a, b] = tuple_to_list({a, b}), + [a, b, c] = tuple_to_list({a, b, c}), + [a, b, c, d] = tuple_to_list({a, b, c, d}), + [a, b, c, d] = tuple_to_list({a, b, c, d}), + + Size = 4096, + List = lists:seq(1, Size), + Tuple = list_to_tuple(List), + Size = size(Tuple), + List = tuple_to_list(Tuple), + + {'EXIT', {badarg,_}} = (catch tuple_to_list(id(a))), + {'EXIT', {badarg,_}} = (catch tuple_to_list(id(42))), ok. %% Tests the make_tuple/2 BIF. t_make_tuple_2(Config) when is_list(Config) -> - ?line t_make_tuple1([]), - ?line t_make_tuple1(42), - ?line t_make_tuple1(a), - ?line t_make_tuple1({}), - ?line t_make_tuple1({a}), - ?line t_make_tuple1(erlang:make_tuple(400, [])), + t_make_tuple1([]), + t_make_tuple1(42), + t_make_tuple1(a), + t_make_tuple1({}), + t_make_tuple1({a}), + t_make_tuple1(erlang:make_tuple(400, [])), + + % test upper boundry, 16777215 elements + t_make_tuple(1 bsl 24 - 1, a), + {'EXIT', {badarg,_}} = (catch erlang:make_tuple(1 bsl 24, a)), + + {'EXIT', {badarg,_}} = (catch erlang:make_tuple(-1, a)), + % 26 bits is the total header arity room (for now) + {'EXIT', {badarg,_}} = (catch erlang:make_tuple(1 bsl 26 + 3, a)), + % bignum + {'EXIT', {badarg,_}} = (catch erlang:make_tuple(1 bsl 65 + 3, a)), ok. t_make_tuple1(Element) -> @@ -222,29 +238,39 @@ t_make_tuple(Size, Element) -> %% Tests the erlang:make_tuple/3 BIF. t_make_tuple_3(Config) when is_list(Config) -> - ?line {} = erlang:make_tuple(0, def, []), - ?line {def} = erlang:make_tuple(1, def, []), - ?line {a} = erlang:make_tuple(1, def, [{1,a}]), - ?line {a,def,c,def,e} = erlang:make_tuple(5, def, [{5,e},{1,a},{3,c}]), - ?line {a,def,c,def,e} = erlang:make_tuple(5, def, - [{1,blurf},{5,e},{3,blurf}, - {1,a},{3,c}]), + {} = erlang:make_tuple(0, def, []), + {def} = erlang:make_tuple(1, def, []), + {a} = erlang:make_tuple(1, def, [{1,a}]), + + {a,def,c,def,e} = erlang:make_tuple(5, def, [{5,e},{1,a},{3,c}]), + {a,def,c,def,e} = erlang:make_tuple(5, def, [{1,blurf},{5,e},{3,blurf},{1,a},{3,c}]), + MaxSize = 1 bsl 16 - 1, + MaxTuple = erlang:make_tuple(MaxSize, def, [{1,blurf},{5,e},{3,blurf},{1,a},{3,c}]), + MaxSize = size(MaxTuple), %% Error cases. - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(0, def, [{1,a}])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{-1,a}])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{0,a}])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{6,z}])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(a, def, [{6,z}])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{1,a}|b])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [42])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [[a,b,c]])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, non_list)), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(0, def, [{1,a}])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{-1,a}])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{0,a}])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{6,z}])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(a, def, [{6,z}])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{1,a}|b])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [42])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [[a,b,c]])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, non_list)), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(1 bsl 24, def, [{5,e},{1,a},{3,c}])), + ok. %% Tests the append_element/2 BIF. t_append_element(Config) when is_list(Config) -> - t_append_element({}, 2048, 2048). + ok = t_append_element({}, 2048, 2048), + + % test upper boundry, 16777215 elements + MaxSize = 1 bsl 24 - 1, + MaxTuple = list_to_tuple(lists:seq(1, MaxSize)), + {'EXIT',{badarg,_}} = (catch erlang:append_element(MaxTuple, a)), + ok. t_append_element(_Tuple, 0, _High) -> ok; t_append_element(Tuple, N, High) -> @@ -261,7 +287,7 @@ verify_seq([High|T], High, Lower) -> %% (This is known to crash earlier versions of BEAM.) tuple_with_case(Config) when is_list(Config) -> - ?line {reply, true} = tuple_with_case(), + {reply, true} = tuple_with_case(), ok. tuple_with_case() -> @@ -280,21 +306,21 @@ foo() -> ignored. %% Test to build a tuple in a guard. tuple_in_guard(Config) when is_list(Config) -> - ?line Tuple1 = id({a,b}), - ?line Tuple2 = id({a,b,c}), - ?line if - Tuple1 == {element(1, Tuple2),element(2, Tuple2)} -> - ok; - true -> - ?line test_server:fail() - end, - ?line if - Tuple2 == {element(1, Tuple2),element(2, Tuple2), - element(3, Tuple2)} -> - ok; - true -> - ?line test_server:fail() - end, + Tuple1 = id({a,b}), + Tuple2 = id({a,b,c}), + if + Tuple1 == {element(1, Tuple2),element(2, Tuple2)} -> + ok; + true -> + test_server:fail() + end, + if + Tuple2 == {element(1, Tuple2),element(2, Tuple2), + element(3, Tuple2)} -> + ok; + true -> + test_server:fail() + end, ok. %% Use this function to avoid compile-time evaluation of an expression. diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c index 04d3425fe0..50c61f50eb 100644 --- a/erts/etc/common/erlexec.c +++ b/erts/etc/common/erlexec.c @@ -127,6 +127,7 @@ static char *pluss_val_switches[] = { "wt", "ws", "ss", + "pp", NULL }; /* +h arguments with values */ @@ -799,7 +800,9 @@ int main(int argc, char **argv) case 'A': case 'b': case 'i': + case 'n': case 'P': + case 'Q': case 'S': case 't': case 'T': @@ -1103,7 +1106,8 @@ usage_aux(void) "[-make] [-man [manopts] MANPAGE] [-x] [-emu_args] " "[-args_file FILENAME] [+A THREADS] [+a SIZE] [+B[c|d|i]] [+c] " "[+h HEAP_SIZE_OPTION] [+K BOOLEAN] " - "[+l] [+M<SUBSWITCH> <ARGUMENT>] [+P MAX_PROCS] [+R COMPAT_REL] " + "[+l] [+M<SUBSWITCH> <ARGUMENT>] [+P MAX_PROCS] [+Q MAX_PORTS] " + "[+R COMPAT_REL] " "[+r] [+rg READER_GROUPS_LIMIT] [+s SCHEDULER_OPTION] " "[+S NO_SCHEDULERS:NO_SCHEDULERS_ONLINE] [+T LEVEL] [+V] [+v] " "[+W<i|w>] [+z MISC_OPTION] [args ...]\n"); diff --git a/erts/etc/unix/cerl.src b/erts/etc/unix/cerl.src index 651452e531..cc7d77fd9a 100644 --- a/erts/etc/unix/cerl.src +++ b/erts/etc/unix/cerl.src @@ -304,10 +304,17 @@ else ;; esac - # Set annotation level for gdb in emacs 22 and higher. - emacs_major=`$EMACS --version | head -1 | sed 's,^[^0-9]*\([0-9]*\).*,\1,g'` - if [ '!' -z "$emacs_major" -a $emacs_major -gt 21 ]; then - GDBARGS="--annotate=1 " + if [ "$EMACS_ANNOTATE_LEVEL" != "" ]; then + GDBARGS="--annotate=$EMACS_ANNOTATE_LEVEL" + else + # Set annotation level for gdb in emacs 22 and higher. Seems to + # be working with level 1 for emacs 22 and level 3 for emacs 23... + emacs_major=`$EMACS --version | head -1 | sed 's,^[^0-9]*\([0-9]*\).*,\1,g'` + if [ '!' -z "$emacs_major" -a $emacs_major -gt 22 ]; then + GDBARGS="--annotate=3 " + elif [ '!' -z "$emacs_major" -a $emacs_major -gt 21 ]; then + GDBARGS="--annotate=1 " + fi fi gdbcmd="$gdbcmd $GDBBP \ (insert-string \"source $ROOTDIR/erts/etc/unix/etp-commands\") \ diff --git a/erts/etc/unix/etp-commands b/erts/etc/unix/etp-commands index d98505c0ff..f059662271 100644 --- a/erts/etc/unix/etp-commands +++ b/erts/etc/unix/etp-commands @@ -705,8 +705,6 @@ define etp-ct-name-1 end end - - define etp-pid-1 # Args: Eterm pid # @@ -714,9 +712,17 @@ define etp-pid-1 # set $etp_pid_1 = (Eterm)($arg0) if ($etp_pid_1 & 0xF) == 0x3 + if (etp_arch_bits == 64 && etp_halfword == 0) + if (etp_big_endian) + set $etp_pid_data = (unsigned) ((((Uint64) $etp_pid_1) >> 36) & 0x0fffffff) + else + set $etp_pid_data = (unsigned) ((((Uint64) $etp_pid_1) >> 4) & 0x0fffffff) + end + else + set $etp_pid_data = (unsigned) (((((Uint32) $etp_pid_1) >> 4) & ~erts_proc.r.o.pix_mask) | ((((Uint32) $etp_pid_1) >> (erts_proc.r.o.pix_cl_shift + 4)) & erts_proc.r.o.pix_cl_mask) | (((((Uint32) $etp_pid_1) >> 4) & erts_proc.r.o.pix_cli_mask) << erts_proc.r.o.pix_cli_shift)) + end # Internal pid - printf "<0.%u.%u>", (unsigned) ($etp_pid_1>>4)&0x7fff, \ - (unsigned) ($etp_pid_1>>19)&0x1fff + printf "<0.%u.%u>", $etp_pid_data & 0x7fff, ($etp_pid_data >> 15) & 0x1fff else printf "#NotPid<%#x>", ($arg0) end @@ -759,7 +765,6 @@ define etp-extpid-1 end - define etp-port-1 # Args: Eterm port # @@ -767,8 +772,17 @@ define etp-port-1 # set $etp_port_1 = (Eterm)($arg0) if ($etp_port_1 & 0xF) == 0x7 + if (etp_arch_bits == 64 && etp_halfword == 0) + if (etp_big_endian) + set $etp_port_data = (unsigned) ((((Uint64) $etp_port_1) >> 36) & 0x0fffffff) + else + set $etp_port_data = (unsigned) ((((Uint64) $etp_port_1) >> 4) & 0x0fffffff) + end + else + set $etp_port_data = (unsigned) (((((Uint32) $etp_port_1) >> 4) & ~erts_port.r.o.pix_mask) | ((((Uint32) $etp_port_1) >> (erts_port.r.o.pix_cl_shift + 4)) & erts_port.r.o.pix_cl_mask) | (((((Uint32) $etp_port_1) >> 4) & erts_port.r.o.pix_cli_mask) << erts_port.r.o.pix_cli_shift)) + end # Internal port - printf "#Port<0.%u>", (unsigned) ($etp_port_1>>4)&0x3ffff + printf "#Port<0.%u>", $etp_port_data else printf "#NotPort<%#x>", ($arg0) end @@ -1264,23 +1278,32 @@ document etpf-stackdump %--------------------------------------------------------------------------- end +define etp-pid2pix-1 +# Args: Eterm +# + if (etp_arch_bits == 64 && etp_halfword == 0) + if (etp_big_endian) + set $etp_pix = (int) (((Uint64) $arg0) & 0x0fffffff) + else + set $etp_pix = (int) ((((Uint64) $arg0) >> 32) & 0x0fffffff) + end + else + set $etp_pix = (int) ((((Uint32) $arg0) >> 4) & erts_proc.r.o.pix_mask) + end +end + define etp-pix2proc # Args: Eterm # - set $proc = (Process *) *((UWord *) &erts_proc.tab[((int) $arg0)]) + set $proc = (Process *) *((UWord *) &erts_proc.r.o.tab[((int) $arg0)]) printf "(Process *) %p\n", $proc end define etp-pid2proc-1 # Args: Eterm # - set $pix = (int) ($arg0 >> 4) - - if (erts_proc.pix_cl_mask != 0) - set $proc = (Process *) *((UWord *) &erts_proc.tab[(((($pix) & erts_proc.pix_cl_mask) << erts_proc.pix_cl_shift) + ((($pix) >> erts_proc.pix_cli_shift) & erts_proc.pix_cli_mask))]) - else - set $proc =(Process *) *((UWord *) &erts_proc.tab[((($pix) % erts_proc.max) % erts_proc.tab_cache_lines + (($pix) % erts_proc.max) / erts_proc.tab_cache_lines)]) - end + etp-pid2pix-1 $arg0 + set $proc = (Process *) *((UWord *) &erts_proc.r.o.tab[$etp_pix]) end define etp-pid2proc @@ -1369,13 +1392,15 @@ define etp-process-info # Args: Process* # printf " Pid: " - etp-1 $arg0->id + etp-1 $arg0->common.id printf "\n State: " etp-proc-state $arg0 - if ($arg0->reg) - printf " Registered name: " - etp-1 $arg0->reg->name - printf "\n" + if (*(((Uint32 *) &(((Process *) $arg0)->state))) & 0x4) == 0 + if ($arg0->common.u.alive.reg) + printf " Registered name: " + etp-1 $arg0->common.u.alive.reg->name + printf "\n" + end end if ($arg0->current) printf " Current function: " @@ -1400,7 +1425,7 @@ define etp-process-info end printf " Mbuf size: %ld\n", $arg0->mbuf_sz if (etp_smp_compiled) - printf " Msgq len: %ld (inner=%ld, outer=%ld)\n", ($arg0->msg.len + $arg0->u.alive.msg_inq.len), $arg0->msg.len, $arg0->u.alive.msg_inq.len + printf " Msgq len: %ld (inner=%ld, outer=%ld)\n", ($arg0->msg.len + $arg0->msg_inq.len), $arg0->msg.len, $arg0->msg_inq.len else printf " Msgq len: %d\n", $arg0->msg.len end @@ -1422,9 +1447,9 @@ define etp-processes printf "No processes, since system isn't initialized!\n" else set $proc_ix = 0 - while $proc_ix < erts_proc.max - set $proc = (Process *) *((UWord *) &erts_proc.tab[$proc_ix]) - if ($proc != (Process *) 0) + while $proc_ix < erts_proc.r.o.max + set $proc = (Process *) *((UWord *) &erts_proc.r.o.tab[$proc_ix]) + if ($proc != ((Process *) 0) && $proc != &erts_invalid_process) printf "---\n" printf " Pix: %d\n", $proc_ix etp-process-info $proc @@ -1443,8 +1468,108 @@ document etp-processes %--------------------------------------------------------------------------- end +define etp-port-id2pix-1 +# Args: Eterm +# + if (etp_arch_bits == 64 && etp_halfword == 0) + if (etp_big_endian) + set $etp_pix = (int) (((Uint64) $arg0) & 0x0fffffff) + elser + set $etp_pix = (int) ((((Uint64) $arg0) >> 32) & 0x0fffffff) + end + else + set $etp_pix = (int) ((((Uint32) $arg0) >> 4) & erts_port.r.o.pix_mask) + end +end + +define etp-pix2port +# Args: Eterm +# + set $port = (Port *) *((UWord *) &erts_port.r.o.tab[((int) $arg0)]) + printf "(Port *) %p\n", $port +end -define etp-port-status-int +define etp-id2port-1 +# Args: Eterm +# + etp-port-id2pix-1 $arg0 + set $port = (Port *) *((UWord *) &erts_port.r.o.tab[((int) $etp_pix)]) +end + +define etp-id2port +# Args: Eterm +# + etp-id2port-1 $arg0 + printf "(Port *) %p\n", $port +end + +define etp-port-sched-flags-int +# Args: int +# + if ($arg0 & 0x1) + printf " in-run-queue" + end + if ($arg0 & 0x2) + printf " executing" + end + if ($arg0 & 0x4) + printf " have-tasks" + end + if ($arg0 & 0x8) + printf " exited" + end + if ($arg0 & 0x10) + printf " busy-port" + end + if ($arg0 & 0x20) + printf " busy-port-q" + end + if ($arg0 & 0x40) + printf " chk-unset-busy-port-q" + end + if ($arg0 & 0x80) + printf " have-busy-tasks" + end + if ($arg0 & 0x100) + printf " have-nosuspend-tasks" + end + if ($arg0 & 0x200) + printf " parallelism" + end + if ($arg0 & 0x400) + printf " force-sched" + end + if ($arg0 & 0xfffff800) + printf " GARBAGE" + end + printf "\n" +end + +document etp-port-sched-flags-int +%--------------------------------------------------------------------------- +% etp-proc-sched-flags-int int +% +% Print port sched-flags +%--------------------------------------------------------------------------- +end + + +define etp-port-sched-flags +# Args: Port* +# + set $sched_flags_int = *(((Uint32 *) &(((Port *) $arg0)->sched.flags))) + etp-port-sched-flags-int $sched_flags_int +end + +document etp-port-sched-flags +%--------------------------------------------------------------------------- +% etp-proc-sched-flags-int Port * +% +% Print port sched-flags +%--------------------------------------------------------------------------- +end + +define etp-port-state-int # Args: int # if ($arg0 & 0x1) @@ -1463,68 +1588,62 @@ define etp-port-status-int printf " soft-eof" end if ($arg0 & 0x20) - printf " port-busy" + printf " closing" end if ($arg0 & 0x40) - printf " closing" + printf " send-closed" end if ($arg0 & 0x80) - printf " send-closed" + printf " linebuf-io" end if ($arg0 & 0x100) - printf " linebuf-io" + printf " free" end if ($arg0 & 0x200) - printf " immortal" + printf " initializing" end if ($arg0 & 0x400) - printf " free" + printf " port-specific-lock" end if ($arg0 & 0x800) - printf " free-scheduled" + printf " invalid" end if ($arg0 & 0x1000) - printf " initializing" - end - if ($arg0 & 0x2000) - printf " port-specific-lock" - end - if ($arg0 & 0x4000) - printf " invalid" + printf " halt" end if (etp_debug_compiled) - if ($arg0 & 0x7fff8000) + if ($arg0 & 0x7fffe000) printf " GARBAGE" end else - if ($arg0 & 0xffff8000) + if ($arg0 & 0xffffe000) printf " GARBAGE" end end printf "\n" end -document etp-port-status-int +document etp-port-state-int %--------------------------------------------------------------------------- % etp-proc-state-int int % -% Print port status +% Print port state %--------------------------------------------------------------------------- end -define etp-port-status +define etp-port-state # Args: Port* # - set $status_int = *(((Uint32 *) &(((Port *) $arg0)->status))) - etp-port-status-int $status_int + set $state_int = *(((Uint32 *) &(((Port *) $arg0)->state))) + etp-port-state-int $state_int end -document etp-port-status +document etp-port-state %--------------------------------------------------------------------------- % etp-proc-state-int Port * % -% Print port status +% Print port state %--------------------------------------------------------------------------- end @@ -1532,17 +1651,22 @@ define etp-port-info # Args: Port* # printf " Port: " - etp-1 $arg0->id + etp-1 $arg0->common.id printf "\n Name: %s\n", $arg0->name - printf " Status:" - etp-port-status $arg0 - if ($arg0->reg) - printf " Registered name: " - etp-1 $arg0->reg->name - printf "\n" + printf " State:" + etp-port-state $arg0 + printf " Scheduler flags:" + etp-port-sched-flags $arg0 + if (*(((Uint32 *) &(((Port *) $arg0)->state))) & 0x5C00) == 0 + if ($arg0->common.u.alive.reg) + printf " Registered name: " + etp-1 $arg0->common.u.alive.reg->name + printf "\n" + end end printf " Connected: " - etp-1 $arg0->connected + set $connected = *(((Eterm *) &(((Port *) $arg0)->connected))) + etp-1 $connected printf "\n Pointer: (Port *) %p\n", $arg0 end @@ -1560,13 +1684,15 @@ define etp-ports printf "No ports, since system isn't initialized!\n" else set $port_ix = 0 - while $port_ix < erts_max_ports - set $port = &erts_port[$port_ix] - if ($port->status & 0x400) == 0 - # I.e, not free - printf "---\n" - printf " Pix: %d\n", $port_ix - etp-port-info $port + while $port_ix < erts_port.r.o.max + set $port = (Port *) *((UWord *) &erts_port.r.o.tab[$port_ix]) + if ($port != ((Port *) 0) && $port != &erts_invalid_port) + if (*(((Uint32 *) &(((Port *) $port)->state))) & 0x100) == 0 + # I.e, not free + printf "---\n" + printf " Pix: %d\n", $port_ix + etp-port-info $port + end end set $port_ix++ end @@ -1866,6 +1992,12 @@ define etp-system-info printf "ERTS version: %s\n", etp_erts_version printf "Compile date: %s\n", etp_compile_date printf "Arch: %s\n", etp_arch + printf "Endianess: " + if (etp_big_endian) + printf "Big\n" + else + printf "Little\n" + end printf "Word size: %d-bit\n", etp_arch_bits printf "Halfword: " if (etp_halfword) @@ -2113,8 +2245,8 @@ define etp-search-heaps-1 end set $etp_search_heaps_i = 0 set $etp_search_heaps_found = 0 - while $etp_search_heaps_i < erts_proc.max - set $proc = (Process *) *((UWord *) &erts_proc.tab[$proc_ix]) + while $etp_search_heaps_i < erts_proc.r.o.max + set $proc = (Process *) *((UWord *) &erts_proc.r.o.tab[$proc_ix]) if $proc if ($proc->heap <= ($arg0)) && \ (($arg0) < $proc->hend) diff --git a/erts/etc/win32/nsis/Makefile b/erts/etc/win32/nsis/Makefile index a53ac708f8..7bcecaa264 100644 --- a/erts/etc/win32/nsis/Makefile +++ b/erts/etc/win32/nsis/Makefile @@ -62,9 +62,15 @@ else endif REDIST_FILE=$(shell (sh ./find_redist.sh || echo "")) +ifeq ($(MSYSTEM),MINGW32) + NICEREDISTFILE=$(shell (msys2win_path.sh -m "$(REDIST_FILE)" 2>/dev/null || echo "")) +else + NICEREDISTFILE=$(shell (cygpath -d -m "$(REDIST_FILE)" 2>/dev/null || echo "")) +endif + REDIST_TARGET=$(shell (sh ./find_redist.sh -n || echo "")) -REDIST_DLL_VERSION=$(shell (sh ./dll_version_helper.sh || echo "")) -REDIST_DLL_NAME=$(shell (sh ./dll_version_helper.sh -n || echo "")) +REDIST_DLL_VERSION=$(shell (sh ./dll_version_helper.sh $(NICEREDISTFILE) || echo "")) +REDIST_DLL_NAME=$(shell (sh ./dll_version_helper.sh -n $(NICEREDISTFILE) || echo "")) release_spec: @NSIS_VER=`makensis /hdrinfo | head -1 | awk '{print $$2}'`; \ diff --git a/erts/etc/win32/nsis/dll_version_helper.sh b/erts/etc/win32/nsis/dll_version_helper.sh index 96e4532b7a..0d9ba4248d 100755 --- a/erts/etc/win32/nsis/dll_version_helper.sh +++ b/erts/etc/win32/nsis/dll_version_helper.sh @@ -25,6 +25,13 @@ # echo "8.0.50727.763" # exit 0 +if [ "$1" = "-n" ]; then + SWITCH=$1 + shift +else + SWITCH="" +fi + cat > hello.c <<EOF #include <windows.h> #include <stdio.h> @@ -42,11 +49,16 @@ if [ '!' -f hello.exe.manifest ]; then # need another way of getting the version DLLNAME=`dumpbin.exe -imports hello.exe | egrep MSVCR.*dll` DLLNAME=`echo $DLLNAME` + if [ '!' -z "$1" ]; then + FILETOLOOKIN=$1 + else + FILETOLOOKIN=$DLLNAME + fi cat > helper.c <<EOF #include <windows.h> #include <stdio.h> -#define REQ_MODULE "$DLLNAME" +#define REQ_MODULE "$FILETOLOOKIN" int main(void) { @@ -100,7 +112,7 @@ else NAME=`grep '<assemblyIdentity' hello.exe.manifest | sed 's,.*name=.[A-Za-z\.]*\([0-9]*\).*,msvcr\1.dll,g' | grep -v '<'` fi #rm -f hello.c hello.obj hello.exe hello.exe.manifest helper.c helper.obj helper.exe helper.exe.manifest -if [ "$1" = "-n" ]; then +if [ "$SWITCH" = "-n" ]; then ASKEDFOR=$NAME else ASKEDFOR=$VERSION diff --git a/erts/etc/win32/nsis/erlang20.nsi b/erts/etc/win32/nsis/erlang20.nsi index c5ada9e3b3..3333c4a9aa 100644 --- a/erts/etc/win32/nsis/erlang20.nsi +++ b/erts/etc/win32/nsis/erlang20.nsi @@ -35,6 +35,7 @@ !include "MUI.nsh"
!include "WordFunc.nsh"
+ !include "WinVer.nsh"
;--------------------------------
;Configuration
@@ -341,6 +342,9 @@ FunctionEnd Function .onInit
Var /GLOBAL archprefix
Var /GLOBAL sysnativedir
+ Var /GLOBAL winvermajor
+ Var /GLOBAL winverminor
+
SectionGetFlags 0 $MYTEMP
StrCmpS ${WINTYPE} "win64" +1 +4
StrCpy $archprefix "amd64"
@@ -348,9 +352,16 @@ Function .onInit Goto +3
StrCpy $archprefix "x86"
StrCpy $sysnativedir $SYSDIR
- ;MessageBox MB_YESNO "Found $sysnativedir\${REDIST_DLL_NAME}" IDYES FoundLbl
+ ${WinVerGetMajor} $0
+ ${WinVerGetMinor} $1
+ StrCpy $winvermajor $0
+ StrCpy $winverminor $1
IfFileExists $sysnativedir\${REDIST_DLL_NAME} MaybeFoundInSystemLbl
SearchSxSLbl:
+ IntCmp $winvermajor 6 WVCheckMinorLbl WVCheckDoneLbl NotFoundLbl
+ WVCheckMinorLbl:
+ IntCmp $winverminor 1 WVCheckDoneLbl WVCheckDoneLbl NotFoundLbl
+ WVCheckDoneLbl:
FindFirst $0 $1 $WINDIR\WinSxS\$archprefix*
LoopLbl:
StrCmp $1 "" NotFoundLbl
diff --git a/erts/include/internal/ethread.h b/erts/include/internal/ethread.h index e1885c627a..aef31e282a 100644 --- a/erts/include/internal/ethread.h +++ b/erts/include/internal/ethread.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2011. All Rights Reserved. + * Copyright Ericsson AB 2004-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/lib_src/common/erl_misc_utils.c b/erts/lib_src/common/erl_misc_utils.c index 3b123063fa..1b49f69581 100644 --- a/erts/lib_src/common/erl_misc_utils.c +++ b/erts/lib_src/common/erl_misc_utils.c @@ -28,8 +28,6 @@ #include "erl_misc_utils.h" #if defined(__WIN32__) -#elif defined(VXWORKS) -# include <selectLib.h> #else /* UNIX */ # include <stdio.h> # include <sys/types.h> @@ -124,6 +122,12 @@ #include <sys/sysctl.h> #endif +/* Simplify include for static functions */ + +#if defined(__linux__) || defined(HAVE_KSTAT) || defined(__WIN32__) || defined(__FreeBSD__) +# define ERTS_CPU_TOPOLOGY_ENABLED (1) +#endif + static int read_topology(erts_cpu_info_t *cpuinfo); #if defined(ERTS_HAVE_MISC_UTIL_AFFINITY_MASK__) @@ -669,6 +673,7 @@ erts_unbind_from_cpu_str(char *str) } +#if defined(ERTS_CPU_TOPOLOGY_ENABLED) static int pn_cmp(const void *vx, const void *vy) { @@ -759,6 +764,7 @@ adjust_processor_nodes(erts_cpu_info_t *cpuinfo, int no_nodes) } } } +#endif #ifdef __linux__ diff --git a/erts/preloaded/ebin/erl_prim_loader.beam b/erts/preloaded/ebin/erl_prim_loader.beam Binary files differindex e46795e2b1..eb6b5259b5 100644 --- a/erts/preloaded/ebin/erl_prim_loader.beam +++ b/erts/preloaded/ebin/erl_prim_loader.beam diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex bf6d6e871b..074e31fb13 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/ebin/erts_internal.beam b/erts/preloaded/ebin/erts_internal.beam Binary files differnew file mode 100644 index 0000000000..c647fa5d4e --- /dev/null +++ b/erts/preloaded/ebin/erts_internal.beam diff --git a/erts/preloaded/ebin/init.beam b/erts/preloaded/ebin/init.beam Binary files differindex 1a250932da..4a34268f06 100644 --- a/erts/preloaded/ebin/init.beam +++ b/erts/preloaded/ebin/init.beam diff --git a/erts/preloaded/ebin/otp_ring0.beam b/erts/preloaded/ebin/otp_ring0.beam Binary files differindex 2fa007f683..6fcf515461 100644 --- a/erts/preloaded/ebin/otp_ring0.beam +++ b/erts/preloaded/ebin/otp_ring0.beam diff --git a/erts/preloaded/ebin/prim_file.beam b/erts/preloaded/ebin/prim_file.beam Binary files differindex 19cf856754..6d6d70920e 100644 --- a/erts/preloaded/ebin/prim_file.beam +++ b/erts/preloaded/ebin/prim_file.beam diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex b2d224e560..071dbb00f5 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/ebin/prim_zip.beam b/erts/preloaded/ebin/prim_zip.beam Binary files differindex 55697cfe11..c4487cc5fa 100644 --- a/erts/preloaded/ebin/prim_zip.beam +++ b/erts/preloaded/ebin/prim_zip.beam diff --git a/erts/preloaded/ebin/zlib.beam b/erts/preloaded/ebin/zlib.beam Binary files differindex d2f30fd9cd..d58261a57c 100644 --- a/erts/preloaded/ebin/zlib.beam +++ b/erts/preloaded/ebin/zlib.beam diff --git a/erts/preloaded/src/Makefile b/erts/preloaded/src/Makefile index 5bcc2eb6e4..a224b6a5d4 100644 --- a/erts/preloaded/src/Makefile +++ b/erts/preloaded/src/Makefile @@ -40,7 +40,8 @@ PRE_LOADED_MODULES = \ zlib \ prim_zip \ otp_ring0 \ - erlang + erlang \ + erts_internal RELSYSDIR = $(RELEASE_PATH)/lib/erts-$(VSN) # not $(RELEASE_PATH)/erts-$(VSN)/preloaded diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index 9e814ae54b..51a190b095 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -1075,57 +1075,6 @@ phash2(_Term, _Range) -> pid_to_list(_Pid) -> erlang:nif_error(undefined). -%% port_close/1 --spec port_close(Port) -> true when - Port :: port() | atom(). -port_close(_Port) -> - erlang:nif_error(undefined). - -%% port_command/2 --spec port_command(Port, Data) -> true when - Port :: port() | atom(), - Data :: iodata(). -port_command(_Port, _Data) -> - erlang:nif_error(undefined). - -%% port_command/3 --spec port_command(Port, Data, OptionList) -> boolean() when - Port :: port() | atom(), - Data :: iodata(), - OptionList :: [Option], - Option :: force | nosuspend. -port_command(_Port, _Data, _OptionList) -> - erlang:nif_error(undefined). - -%% port_connect/2 --spec port_connect(Port, Pid) -> true when - Port :: port() | atom(), - Pid :: pid(). -port_connect(_Port, _Pid) -> - erlang:nif_error(undefined). - -%% port_control/3 --spec port_control(Port, Operation, Data) -> Res when - Port :: port() | atom(), - Operation :: integer(), - Data :: iodata(), - Res :: string() | binary(). -port_control(_Port, _Operation, _Data) -> - erlang:nif_error(undefined). - -%% port_get_data/1 --spec erlang:port_get_data(P1) -> term() when - P1 :: port() | atom(). -port_get_data(_P1) -> - erlang:nif_error(undefined). - -%% port_set_data/2 --spec erlang:port_set_data(P1, P2) -> true when - P1 :: port() | atom(), - P2 :: term(). -port_set_data(_P1, _P2) -> - erlang:nif_error(undefined). - %% port_to_list/1 -spec erlang:port_to_list(Port) -> string() when Port :: port(). @@ -1700,60 +1649,11 @@ nodes(_Arg) -> | out | binary | eof + | {parallelism, Boolean :: boolean()} | hide. open_port(_PortName,_PortSettings) -> erlang:nif_error(undefined). -%% Shadowed by erl_bif_types: erlang:port_call/2 --spec erlang:port_call(Port, Data) -> term() when - Port :: port() | atom(), - Data :: term(). -port_call(_Port, _Data) -> - erlang:nif_error(undefined). - -%% Shadowed by erl_bif_types: erlang:port_call/3 --spec erlang:port_call(Port, Operation, Data) -> term() when - Port :: port() | atom(), - Operation :: integer(), - Data :: term(). -port_call(_Port, _Operation, _Data) -> - erlang:nif_error(undefined). - --type port_info_item() :: - registered_name | - id | - connected | - links | - name | - input | - output | - os_pid. - --type port_info_result_item() :: - {registered_name, RegName :: atom()} | - {id, Index :: non_neg_integer()} | - {connected, Pid :: pid()} | - {links, Pids :: [pid()]} | - {name, String :: string()} | - {input, Bytes :: non_neg_integer()} | - {output, Bytes :: non_neg_integer()} | - {os_pid, OsPid :: non_neg_integer() | 'undefined'}. - -%% Shadowed by erl_bif_types: erlang:port_info/1 --spec erlang:port_info(Port) -> Result when - Port :: port() | atom(), - Result :: [port_info_result_item()] | undefined. -port_info(_Result) -> - erlang:nif_error(undefined). - -%% Shadowed by erl_bif_types: erlang:port_info/2 --spec erlang:port_info(Port, Item) -> Result when - Port :: port() | atom(), - Item :: port_info_item(), - Result :: port_info_result_item() | undefined. -port_info(_Result, _Item) -> - erlang:nif_error(undefined). - -type priority_level() :: low | normal | high | max. @@ -2531,6 +2431,216 @@ suspend_process(P) -> end. %% +%% Port BIFs +%% +%% Currently all port BIFs calls the corresponding +%% erts_internal:port_*() native function which perform +%% most of the actual work. These native functions should +%% *never* be called directly by other functionality. The +%% native functions may be changed, or removed without any +%% notice whatsoever! +%% +%% IMPORTANT NOTE: +%% When the erts_internal:port_*() native functions return +%% a reference, they have also internally prepared the +%% message queue of the caller for a receive that will +%% unconditionally wait for a message containing this +%% reference. If the erlang code calling these native +%% functions do not do this, subsequent receives will not +%% work as expected! That is, it is of *vital importance* +%% that the receive is performed as described above! +%% + +-spec port_command(Port, Data) -> 'true' when + Port :: port() | atom(), + Data :: iodata(). + +port_command(Port, Data) -> + case case erts_internal:port_command(Port, Data, []) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + true -> true; + Error -> erlang:error(Error, [Port, Data]) + end. + +-spec port_command(Port, Data, OptionList) -> boolean() when + Port :: port() | atom(), + Data :: iodata(), + Option :: force | nosuspend, + OptionList :: [Option]. + +port_command(Port, Data, Flags) -> + case case erts_internal:port_command(Port, Data, Flags) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + Bool when Bool == true; Bool == false -> Bool; + Error -> erlang:error(Error, [Port, Data, Flags]) + end. + +-spec port_connect(Port, Pid) -> 'true' when + Port :: port() | atom(), + Pid :: pid(). + +port_connect(Port, Pid) -> + case case erts_internal:port_connect(Port, Pid) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + true -> true; + Error -> erlang:error(Error, [Port, Pid]) + end. + +-spec port_close(Port) -> 'true' when + Port :: port() | atom(). + +port_close(Port) -> + case case erts_internal:port_close(Port) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + true -> true; + Error -> erlang:error(Error, [Port]) + end. + +-spec port_control(Port, Operation, Data) -> iodata() | binary() when + Port :: port() | atom(), + Operation :: integer(), + Data :: iodata(). + +port_control(Port, Operation, Data) -> + case case erts_internal:port_control(Port, Operation, Data) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + badarg -> erlang:error(badarg, [Port, Operation, Data]); + Result -> Result + end. + +-spec erlang:port_call(Port, Data) -> term() when + Port :: port() | atom(), + Data :: term(). + +port_call(Port, Data) -> + case case erts_internal:port_call(Port, 0, Data) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + {ok, Result} -> Result; + Error -> erlang:error(Error, [Port, Data]) + end. + +-spec erlang:port_call(Port, Operation, Data) -> term() when + Port :: port() | atom(), + Operation :: integer(), + Data :: term(). + +port_call(Port, Operation, Data) -> + case case erts_internal:port_call(Port, Operation, Data) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + {ok, Result} -> Result; + Error -> erlang:error(Error, [Port, Operation, Data]) + end. + +-spec erlang:port_info(Port) -> Result when + Port :: port() | atom(), + ResultItem :: {registered_name, RegisteredName :: atom()} + | {id, Index :: non_neg_integer()} + | {connected, Pid :: pid()} + | {links, Pids :: [pid()]} + | {name, String :: string()} + | {input, Bytes :: non_neg_integer()} + | {output, Bytes :: non_neg_integer()} + | {os_pid, OsPid :: non_neg_integer() | 'undefined'}, + Result :: [ResultItem] | 'undefined'. + +port_info(Port) -> + case case erts_internal:port_info(Port) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + badarg -> erlang:error(badarg, [Port]); + Result -> Result + end. + +-spec erlang:port_info(Port, connected) -> {connected, Pid} | 'undefined' when + Port :: port() | atom(), + Pid :: pid(); + (Port, id) -> {id, Index} | 'undefined' when + Port :: port() | atom(), + Index :: non_neg_integer(); + (Port, input) -> {input, Bytes} | 'undefined' when + Port :: port() | atom(), + Bytes :: non_neg_integer(); + (Port, links) -> {links, Pids} | 'undefined' when + Port :: port() | atom(), + Pids :: [pid()]; + (Port, locking) -> {locking, Locking} | 'undefined' when + Port :: port() | atom(), + Locking :: 'false' | 'port_level' | 'driver_level'; + (Port, memory) -> {memory, Bytes} | 'undefined' when + Port :: port() | atom(), + Bytes :: non_neg_integer(); + (Port, monitors) -> {monitors, Monitors} | 'undefined' when + Port :: port() | atom(), + Monitors :: [{process, pid()}]; + (Port, name) -> {name, Name} | 'undefined' when + Port :: port() | atom(), + Name :: string(); + (Port, os_pid) -> {os_pid, OsPid} | 'undefined' when + Port :: port() | atom(), + OsPid :: non_neg_integer() | 'undefined'; + (Port, output) -> {output, Bytes} | 'undefined' when + Port :: port() | atom(), + Bytes :: non_neg_integer(); + (Port, parallelism) -> {parallelism, Boolean} | 'undefined' when + Port :: port() | atom(), + Boolean :: boolean(); + (Port, queue_size) -> {queue_size, Bytes} | 'undefined' when + Port :: port() | atom(), + Bytes :: non_neg_integer(); + (Port, registered_name) -> {registered_name, RegisteredName} | [] | 'undefined' when + Port :: port() | atom(), + RegisteredName :: atom(). + +port_info(Port, Item) -> + case case erts_internal:port_info(Port, Item) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + badarg -> erlang:error(badarg, [Port, Item]); + Result -> Result + end. + +-spec erlang:port_set_data(Port, Data) -> 'true' when + Port :: port() | atom(), + Data :: term(). + +port_set_data(Port, Data) -> + case case erts_internal:port_set_data(Port, Data) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + badarg -> erlang:error(badarg, [Port, Data]); + Result -> Result + end. + +-spec erlang:port_get_data(Port) -> term() when + Port :: port() | atom(). + +port_get_data(Port) -> + case case erts_internal:port_get_data(Port) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + {ok, Data} -> Data; + Error -> erlang:error(Error, [Port]) + end. + +%% %% If the emulator wants to perform a distributed command and %% a connection is not established to the actual node the following %% functions are called in order to set up the connection and then diff --git a/erts/preloaded/src/erts_internal.erl b/erts/preloaded/src/erts_internal.erl new file mode 100644 index 0000000000..f1c83f4518 --- /dev/null +++ b/erts/preloaded/src/erts_internal.erl @@ -0,0 +1,155 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%% +%% As the module name imply, this module is here for ERTS internal +%% functionality. As an application programmer you should *never* +%% call anything in this module directly. Functions exported by +%% this module may change behaviour or be removed at any time +%% without any notice whatsoever. Everything in this module is +%% intentionally left undocumented, and should remain so. +%% + +-module(erts_internal). + +-export([await_port_send_result/3]). + +-export([port_command/3, port_connect/2, port_close/1, + port_control/3, port_call/3, port_set_data/2, port_get_data/1, + port_info/1, port_info/2]). + +%% +%% Await result of send to port +%% + +await_port_send_result(Ref, Busy, Ok) -> + receive + {Ref, false} -> Busy; + {Ref, _} -> Ok + end. + +%% +%% Statically linked port NIFs +%% + +-spec erts_internal:port_command(Port, Data, OptionList) -> Result when + Port :: port() | atom(), + Data :: iodata(), + OptionList :: [Option], + Option :: force | nosuspend, + Result :: boolean() | reference() | badarg | notsup. +port_command(_Port, _Data, _OptionList) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_connect(Port, Pid) -> Result when + Port :: port() | atom(), + Pid :: pid(), + Result :: true | reference() | badarg. +port_connect(_Port, _Pid) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_close(Port) -> Result when + Port :: port() | atom(), + Result :: true | reference() | badarg. +port_close(_Port) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_control(Port, Operation, Data) -> Result when + Port :: port() | atom(), + Operation :: integer(), + Data :: iodata(), + Result :: string() | binary() | reference() | badarg. +port_control(_Port, _Operation, _Data) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_call(Port, Operation, Data) -> Result when + Port :: port() | atom(), + Operation :: integer(), + Data :: term(), + Result :: {ok, term()} | reference() | badarg. +port_call(_Port, _Operation, _Data) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_get_data(P1) -> Result when + P1 :: port() | atom(), + Result :: {ok, term()} | reference() | badarg. +port_get_data(_P1) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_set_data(P1, P2) -> Result when + P1 :: port() | atom(), + P2 :: term(), + Result :: true | reference() | badarg. +port_set_data(_P1, _P2) -> + erlang:nif_error(undefined). + +-type port_info_1_result_item() :: + {registered_name, RegName :: atom()} | + {id, Index :: non_neg_integer()} | + {connected, Pid :: pid()} | + {links, Pids :: [pid()]} | + {name, String :: string()} | + {input, Bytes :: non_neg_integer()} | + {output, Bytes :: non_neg_integer()} | + {os_pid, OsPid :: non_neg_integer() | 'undefined'}. + +-spec erts_internal:port_info(Port) -> Result when + Port :: port() | atom(), + Result :: [port_info_1_result_item()] | undefined | reference() | badarg | []. +port_info(_Result) -> + erlang:nif_error(undefined). + +-type port_info_2_item() :: + registered_name | + id | + connected | + links | + name | + input | + output | + os_pid | + monitors | + memory | + parallelism | + queue_size | + locking. + +-type port_info_2_result_item() :: + {registered_name, RegName :: atom()} | + [] | % No registered name + {id, Index :: non_neg_integer()} | + {connected, Pid :: pid()} | + {links, Pids :: [pid()]} | + {name, String :: string()} | + {input, Bytes :: non_neg_integer()} | + {output, Bytes :: non_neg_integer()} | + {os_pid, OsPid :: non_neg_integer() | 'undefined'} | + {monitors, Monitors :: [{process, pid()}]} | + {memory, MemSz :: non_neg_integer()} | + {parallelism, Boolean :: boolean()} | + {queue_size, QSz :: non_neg_integer()} | + {locking, Locking :: 'false' | 'port_level' | 'driver_level'}. + +-spec erts_internal:port_info(Port, Item) -> Result when + Port :: port() | atom(), + Item :: port_info_2_item(), + Result :: port_info_2_result_item() | undefined | reference() | badarg. + +port_info(_Result, _Item) -> + erlang:nif_error(undefined). diff --git a/erts/preloaded/src/prim_inet.erl b/erts/preloaded/src/prim_inet.erl index b9800a05da..21d23159f0 100644 --- a/erts/preloaded/src/prim_inet.erl +++ b/erts/preloaded/src/prim_inet.erl @@ -1072,6 +1072,8 @@ enc_opt(deliver) -> ?INET_LOPT_DELIVER; enc_opt(exit_on_close) -> ?INET_LOPT_EXITONCLOSE; enc_opt(high_watermark) -> ?INET_LOPT_TCP_HIWTRMRK; enc_opt(low_watermark) -> ?INET_LOPT_TCP_LOWTRMRK; +enc_opt(high_msgq_watermark) -> ?INET_LOPT_TCP_MSGQ_HIWTRMRK; +enc_opt(low_msgq_watermark) -> ?INET_LOPT_TCP_MSGQ_LOWTRMRK; enc_opt(send_timeout) -> ?INET_LOPT_TCP_SEND_TIMEOUT; enc_opt(send_timeout_close) -> ?INET_LOPT_TCP_SEND_TIMEOUT_CLOSE; enc_opt(delay_send) -> ?INET_LOPT_TCP_DELAY_SEND; @@ -1126,6 +1128,8 @@ dec_opt(?INET_LOPT_DELIVER) -> deliver; dec_opt(?INET_LOPT_EXITONCLOSE) -> exit_on_close; dec_opt(?INET_LOPT_TCP_HIWTRMRK) -> high_watermark; dec_opt(?INET_LOPT_TCP_LOWTRMRK) -> low_watermark; +dec_opt(?INET_LOPT_TCP_MSGQ_HIWTRMRK) -> high_msgq_watermark; +dec_opt(?INET_LOPT_TCP_MSGQ_LOWTRMRK) -> low_msgq_watermark; dec_opt(?INET_LOPT_TCP_SEND_TIMEOUT) -> send_timeout; dec_opt(?INET_LOPT_TCP_SEND_TIMEOUT_CLOSE) -> send_timeout_close; dec_opt(?INET_LOPT_TCP_DELAY_SEND) -> delay_send; @@ -1221,6 +1225,8 @@ type_opt_1(deliver) -> type_opt_1(exit_on_close) -> bool; type_opt_1(low_watermark) -> int; type_opt_1(high_watermark) -> int; +type_opt_1(low_msgq_watermark) -> int; +type_opt_1(high_msgq_watermark) -> int; type_opt_1(send_timeout) -> time; type_opt_1(send_timeout_close) -> bool; type_opt_1(delay_send) -> bool; diff --git a/erts/vsn.mk b/erts/vsn.mk index a420781e9f..a13326a02a 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -17,6 +17,7 @@ # %CopyrightEnd% # + VSN = 5.10 SYSTEM_VSN = R16B diff --git a/lib/asn1/doc/src/asn1_ug.xml b/lib/asn1/doc/src/asn1_ug.xml index 1b399fb641..6cb251c3e2 100644 --- a/lib/asn1/doc/src/asn1_ug.xml +++ b/lib/asn1/doc/src/asn1_ug.xml @@ -186,13 +186,21 @@ END </pre> The following shows how the compiler can be called from the Erlang shell:</p> <pre> -1><input>asn1ct:compile("People",[ber_bin]).</input> +1><input>asn1ct:compile("People", [ber]).</input> +ok +2> </pre> + + <p>The <c>verbose</c> option can be given to have information + about the generated files printed:</p> + <pre> +2><input>asn1ct:compile("People", [ber,verbose]).</input> Erlang ASN.1 compiling "People.asn" --{generated,"People.asn1db"}-- --{generated,"People.hrl"}-- --{generated,"People.erl"}-- ok -2> </pre> +3> </pre> + <p>The ASN.1 module People is now accepted and the abstract syntax tree is saved in the <c>People.asn1db</c> file, the generated Erlang code is compiled using the Erlang compiler and @@ -229,7 +237,7 @@ receive constructed and encoded using <c>'People':encode('Person',Answer)</c> which takes an instance of a defined ASN.1 type and transforms it to a - (possibly) nested list of bytes according to the BER or PER + binary according to the BER or PER encoding-rules. <br></br> The encoder and the decoder can also be run from @@ -239,24 +247,12 @@ The encoder and the decoder can also be run from <pre> 2> <input>Rockstar = {'Person',"Some Name",roving,50}.</input> {'Person',"Some Name",roving,50} -3> <input>{ok,Bytes} = asn1rt:encode('People','Person',Rockstar).</input> -{ok,[<<243>>, - [17], - [19,9,"Some Name"], - [2,1,[2]], - [2,1,"2"]]} -4> <input>Bin = list_to_binary(Bytes).</input> -<<243,17,19,9,83,111,109,101,32,78,97,109,101,2,1,2,2,1,50>> -5> <input>{ok,Person} = asn1rt:decode('People','Person',Bin).</input> +3> <input>{ok,Bin} = asn1rt:encode('People','Person',Rockstar).</input> +{ok,<<243,17,19,9,83,111,109,101,32,78,97,109,101,2,1,2, + 2,1,50>>} +4> <input>{ok,Person} = asn1rt:decode('People','Person',Bin).</input> {ok,{'Person',"Some Name",roving,50}} -6> </pre> - <p>Notice that the result from <c>encode</c> is a nested list which - must be turned into a binary before the call to <c>decode</c>. A - binary is necessary as input to decode since the module was compiled - with the <c>ber_bin</c> option - The reason for returning a nested list is that it is faster to produce - and the <c>list_to_binary</c> operation is - performed automatically when the list is sent via the Erlang port mechanism.</p> +5> </pre> </section> <section> @@ -305,17 +301,15 @@ The encoder and the decoder can also be run from ASN.1 compiler:</p> <pre> erlc Person.asn -erlc -bper_bin Person.asn -erlc -bber_bin +optimize ../Example.asn +erlc -bper Person.asn +erlc -bber ../Example.asn erlc -o ../asnfiles -I ../asnfiles -I /usr/local/standards/asn1 Person.asn </pre> <p>The useful options for the ASN.1 compiler are:</p> <taglist> - <tag><c>-b[ber | per | ber_bin | per_bin | uper_bin]</c></tag> + <tag><c>-b[ber | per | uper]</c></tag> <item> <p>Choice of encoding rules, if omitted <c>ber</c> is the - default. The <c>ber_bin</c> and <c>per_bin</c> options - allows for optimizations and are therefore recommended - instead of the <c>ber</c> and <c>per</c> options.</p> + default.</p> </item> <tag><c>-o OutDirectory</c></tag> <item> @@ -339,42 +333,12 @@ erlc -o ../asnfiles -I ../asnfiles -I /usr/local/standards/asn1 Person.asn </item> <tag><c>+der</c></tag> <item> - <p>DER encoding rule. Only when using <c>-ber</c> or - <c>-ber_bin</c> option.</p> - </item> - <tag><c>+optimize</c></tag> - <item> - <p>This flag has effect only when used together with one of - <c>per_bin</c> or <c>ber_bin</c> flags. It gives time optimized - code in the generated modules and it uses another runtime module. - In the <c>per_bin</c> case a nif is used. The - result from an encode is a binary.</p> - <p><em>When this flag is used you cannot use the old format</em><c>{TypeName,Value}</c> when you encode values. Since it is - an unnecessary construct it has been removed in favor of - performance. It - is neither admitted to construct SEQUENCE or SET component values - with the format <c>{ComponentName,Value}</c> since it also is - unnecessary. The only case were it is necessary is in a CHOICE, - were you have to pass values to the right component by specifying - <c>{ComponentName,Value}</c>. See also about - <seealso marker="#TypeNameValue">{Typename,Value}</seealso> below - and in the sections for each type.</p> - </item> - <tag><c>+driver</c></tag> - <item> - <p>As of R15B this means the same as the <c>nif</c> option. Kept for - backwards compatability reasons.</p> - </item> - <tag><c>+nif</c></tag> - <item> - <p>Together with the flags <c>ber_bin</c> - and <c>optimize</c> you choose to use a nif for considerable - faster encode and decode. </p> + <p>DER encoding rule. Only when using <c>-ber</c> option.</p> </item> <tag><c>+asn1config</c></tag> <item> <p>This functionality works together with the flags - <c>ber_bin</c> and <c>optimize</c>. You enables the + <c>ber</c>. It enables the specialized decodes, see the <seealso marker="asn1_spec">Specialized Decode</seealso> chapter. </p> </item> @@ -413,7 +377,6 @@ erlc -o ../asnfiles -I ../asnfiles -I /usr/local/standards/asn1 Person.asn </item> </taglist> <p>For a complete description of <c>erlc</c> see Erts Reference Manual.</p> - <p>For preferred option use see <seealso marker="#preferred option use">Preferred Option Use</seealso> section.</p> <p>The compiler and other compile-time functions can also be invoked from the Erlang shell. Below follows a brief description of the primary functions, for a @@ -429,9 +392,9 @@ asn1ct:compile("H323-MESSAGES.asn1"). </pre> <p>which equals:</p> <pre> asn1ct:compile("H323-MESSAGES.asn1",[ber]). </pre> - <p>If one wants PER encoding with optimizations:</p> + <p>If one wants PER encoding:</p> <pre> -asn1ct:compile("H323-MESSAGES.asn1",[per_bin,optimize]). </pre> +asn1ct:compile("H323-MESSAGES.asn1",[per]). </pre> <p>The generic encode and decode functions can be invoked like this:</p> <pre> asn1ct:encode('H323-MESSAGES','SomeChoiceType',{call,"octetstring"}). @@ -443,269 +406,6 @@ asn1ct:decode('H323-MESSAGES','SomeChoiceType',Bytes). </pre> </section> <section> - <marker id="preferred option use"></marker> - <title>Preferred Option Use</title> - <p> - It may not be obvious which compile options best fit a - situation. This section describes the format of the result of - encode and decode. It also gives some performance statistics - when using certain options. Finally there is a recommendation - which option combinations should be used. - </p> - <p> - The default option is <c>ber</c>. It is the same backend as - <c>ber_bin</c> except that the result of encode is transformed - to a flat list. Below is a table that gives the different - formats of input and output of encode and decode using the - <em>allowed combinations</em> of coding and optimization - options: (EAVF stands for how ASN1 values are represented in - Erlang which is described in the <seealso - marker="#ASN1Types">ASN1 Types chapter</seealso>) - </p> - <table> - <row> - <cell align="left" valign="middle"><em>Encoding Rule</em></cell> - <cell align="left" valign="middle"><em>Compile options, allowed combinations</em></cell> - <cell align="left" valign="middle"><em>encode input</em></cell> - <cell align="left" valign="middle"><em>encode output</em></cell> - <cell align="left" valign="middle"><em>decode input</em></cell> - <cell align="left" valign="middle"><em>decode output</em></cell> - </row> - <row> - <cell align="left" valign="middle">BER</cell> - <cell align="left" valign="middle">[ber] (default)</cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">flat list</cell> - <cell align="left" valign="middle">flat list / binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">BER</cell> - <cell align="left" valign="middle">[ber_bin]</cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">BER</cell> - <cell align="left" valign="middle"><em>[ber_bin, optimize]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">BER</cell> - <cell align="left" valign="middle"><em>[ber_bin, optimize, nif]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">iolist / binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">PER aligned variant</cell> - <cell align="left" valign="middle">[per]</cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">flat list</cell> - <cell align="left" valign="middle">flat list</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">PER aligned variant</cell> - <cell align="left" valign="middle"><em>[per_bin]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist / binary</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">PER aligned variant</cell> - <cell align="left" valign="middle"><em>[per_bin, optimize]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">PER unaligned variant</cell> - <cell align="left" valign="middle"><em>[uper_bin]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - - <row> - <cell align="left" valign="middle">DER</cell> - <cell align="left" valign="middle">[(ber), der]</cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">flat list</cell> - <cell align="left" valign="middle">flat list / binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">DER</cell> - <cell align="left" valign="middle">[ber_bin, der]</cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">DER</cell> - <cell align="left" valign="middle"><em>[ber_bin, optimize, der]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">DER</cell> - <cell align="left" valign="middle"><em>[ber_bin, optimize, nif, der]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - - - <tcaption>The output / input formats for different combinations of compile options.</tcaption> - </table> - <p> - Encode / decode speed comparison in one user case for the above - alternatives (except <c>DER</c>) is showed in the table below. The - <c>DER</c> alternatives are slower than their corresponding - <c>BER</c> alternative. - </p> - - <table> - <row> - <cell align="left" valign="middle"><em>compile options</em></cell> - <cell align="left" valign="middle"><em>encode time</em></cell> - <cell align="left" valign="middle"><em>decode time</em></cell> - </row> - <row> - <cell align="left" valign="middle">[ber]</cell> - <cell align="left" valign="middle">120</cell> - <cell align="left" valign="middle">162</cell> - </row> - <row> - <cell align="left" valign="middle">[ber_bin]</cell> - <cell align="left" valign="middle">124</cell> - <cell align="left" valign="middle">154</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize]</em></cell> - <cell align="left" valign="middle">50</cell> - <cell align="left" valign="middle">78</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize, driver]</em></cell> - <cell align="left" valign="middle">50</cell> - <cell align="left" valign="middle">62</cell> - </row> - <row> - <cell align="left" valign="middle">[per]</cell> - <cell align="left" valign="middle">141</cell> - <cell align="left" valign="middle">133</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[per_bin]</em></cell> - <cell align="left" valign="middle">125</cell> - <cell align="left" valign="middle">123</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[per_bin, optimize]</em></cell> - <cell align="left" valign="middle">77</cell> - <cell align="left" valign="middle">72</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[uper_bin]</em></cell> - <cell align="left" valign="middle">97</cell> - <cell align="left" valign="middle">104</cell> - </row> - <tcaption> - One example of difference in speed for the compile option alternatives. - </tcaption> - </table> - - <p> - The compile options <c>ber</c>, <c>per</c> and - <c>driver</c> are kept for backwards compatibility and should not be - used in new code. The nif implementation which replaces the linked-in - driver has been shown to be about 5-15% faster. - </p> - <p> - You are strongly recommended to use the appropriate alternative - of the bold typed options. The <c>optimize</c> and - <c>nif</c> options does not affect the encode or decode - result, just the time spent in run-time. When <c>ber_bin</c> and - <c>nif</c> or <c>per_bin</c> and <c>optimize</c> is - combined the C-code nif is used in chosen parts of encode / - decode procedure. - </p> - <table> - <row> - <cell align="left" valign="middle"><em>Compile options, allowed combinations</em></cell> - <cell align="left" valign="middle"><em>use of nif</em></cell> - </row> - <row> - <cell align="left" valign="middle">[ber]</cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle">[ber_bin]</cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize]</em></cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize, nif]</em></cell> - <cell align="left" valign="middle">yes</cell> - </row> - <row> - <cell align="left" valign="middle">[per]</cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[per_bin]</em></cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[per_bin, optimize]</em></cell> - <cell align="left" valign="middle">yes</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[uper_bin]</em></cell> - <cell align="left" valign="middle">no</cell> - </row> - - <row> - <cell align="left" valign="middle">[(ber), der]</cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle">[ber_bin, der]</cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize, der]</em></cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize, nif, der]</em></cell> - <cell align="left" valign="middle">yes</cell> - </row> - - - <tcaption>When the ASN1 nif is used.</tcaption> - </table> - - </section> - <section> <title>Run-time Functions</title> <p>A brief description of the major functions is given here. For a complete description of each function see @@ -719,9 +419,9 @@ asn1rt:decode('H323-MESSAGES','SomeChoiceType',Bytes). </pre> 'H323-MESSAGES':encode('SomeChoiceType',{call,"octetstring"}). 'H323-MESSAGES':decode('SomeChoiceType',Bytes). </pre> <p>The asn1 nif is enabled in two occasions: encoding of - asn1 values when the asn1 spec is compiled with <c>per_bin</c> and - <c>optimize</c> or decode of encoded asn1 values when the asn1 spec is - compiled with <c>ber_bin</c>, <c>optimize</c> and <c>nif</c>. In + asn1 values when the asn1 spec is compiled with <c>per</c> and + or decode of encoded asn1 values when the asn1 spec is + compiled with <c>ber</c>. In those cases the nif will be loaded automatically at the first call to <c>encode</c>/<c>decode</c>. If one doesn't want the performance overhead of the nif being loaded at the first call it is possible @@ -868,26 +568,6 @@ Operational ::= BOOLEAN --ASN.1 definition </pre> <pre> Val = true, {ok,Bytes}=asn1rt:encode(MyModule,'Operational',Val), </pre> - <p>For historical reasons it is also possible to assign ASN.1 values - in Erlang using a tuple notation - with type and value as this</p> - <pre> -Val = {'Operational',true} </pre> - <warning> - <marker id="warning"></marker> - <p>The tuple notation, <c>{Typename, Value}</c> is only kept - because of backward compatibility and may be withdrawn in a - future release. If the notation is used the <c>Typename</c> - element must be spelled correctly, otherwise a run-time error - will occur. - </p> - <p>If the ASN.1 module is compiled with the flags - <c>per_bin</c> or <c>ber_bin</c> and <c>optimize</c> it is not - allowed to use the tuple notation. That possibility has been - removed due to performance reasons. Neither is it allowed to - use the <c>{ComponentName,Value}</c> notation in case of a - SEQUENCE or SET type.</p> - </warning> <p>Below follows a description of how values of each type can be represented in Erlang. </p> @@ -1149,7 +829,7 @@ TextFileVal2 = [88,76,55,44,99,121 .......... a lot of characters here ....] The following example shows how it works:</p> <p>In a file <c>PrimStrings.asn1</c> the type <c>BMP</c> is defined as <br></br> -<c>BMP ::= BMPString</c> then using BER encoding (<c>ber_bin</c> +<c>BMP ::= BMPString</c> then using BER encoding (<c>ber</c> option)the input/output format will be:</p> <pre> 1> <input>{ok,Bytes1} = asn1rt:encode('PrimStrings','BMP',[{0,0,53,53},{0,0,45,56}]).</input> @@ -1174,9 +854,9 @@ TextFileVal2 = [88,76,55,44,99,121 .......... a lot of characters here ....] <c>utf8_list_to_binary</c>, are in the <c>asn1rt</c> module. In the example below we assume an asn1 definition <c>UTF ::= UTF8String</c> in a module <c>UTF.asn</c>:</p> <pre> -1> <input>asn1ct:compile('UTF',[ber_bin]).</input> +1> <input>asn1ct:compile('UTF',[ber]).</input> Erlang ASN.1 version "1.4.3.3" compiling "UTF.asn" -Compiler Options: [ber_bin] +Compiler Options: [ber] --{generated,"UTF.asn1db"}-- --{generated,"UTF.erl"}-- ok @@ -1287,14 +967,6 @@ Pdu ::= SEQUENCE { <p>Values can be assigned in Erlang as shown below:</p> <pre> MyPdu = #'Pdu'{a=22,b=77.99,c={0,1,2,3,4},d='NULL'}. </pre> -<note> - <p> - In very early versions of the asn1 compiler it was also possible to - specify the values of the components in - a SEQUENCE or a SET as a list of tuples <c>{ComponentName,Value}</c>. - This is no longer supported. - </p> -</note> <p>The decode functions will return a record as result when decoding a <c>SEQUENCE</c> or a <c>SET</c>. <marker id="DEFAULT"></marker> @@ -1739,12 +1411,9 @@ SS ::= SET { 1> <input>Val = 'Values':tt().</input> {'TT',77,["kalle","kula"]} 2> <input>{ok,Bytes} = 'Values':encode('TT',Val).</input> -{ok,["0", - [18], - [[[128],[1],"M"],["\\241","\\r",[[[4],[5],"kalle"],[[4],[4],"kula"]]]]]} -3> <input>FlatBytes = lists:flatten(Bytes).</input> -[48,18,128,1,77,161,13,4,5,107,97,108,108,101,4,4,107,117,108,97] -4> <input>'Values':decode('TT',FlatBytes).</input> +{ok,<<48,18,128,1,77,161,13,4,5,107,97,108,108,101,4,4, + 107,117,108,97>>} +4> <input>'Values':decode('TT',Bytes).</input> {ok,{'TT',77,["kalle","kula"]}} 5> </pre> diff --git a/lib/asn1/doc/src/asn1ct.xml b/lib/asn1/doc/src/asn1ct.xml index 3be58cbc8e..9a9bb7ec1b 100644 --- a/lib/asn1/doc/src/asn1ct.xml +++ b/lib/asn1/doc/src/asn1ct.xml @@ -41,6 +41,19 @@ encode/decode functions. There are also some generic functions which can be used in during development of applications which handles ASN.1 data (encoded as BER or PER).</p> + <note> + <p>In R16, the options have been simplified. The back-end is chosen + using one of the options <c>ber</c>, <c>per</c>, or <c>uper</c>. + The options <c>optimize</c>, <c>nif</c>, and <c>driver</c> options + are no longer necessary (and the ASN.1 compiler will print a + warning if they are used). The options <c>ber_bin</c>, <c>per_bin</c>, + and <c>uper_bin</c> options will still work, but will print a warning. + </p> + <p>Another change in R16 is that the generated <c>encode/2</c> + function (and <c>asn1rt:encode/3</c>) always returns a binary. + The <c>encode/2</c> function for the BER back-end used to return + an iolist.</p> + </note> </description> <funcs> <func> @@ -50,9 +63,9 @@ <type> <v>Asn1module = atom() | string()</v> <v>Options = [Option| OldOption]</v> - <v>Option = ber_bin | per_bin | uper_bin | der | compact_bit_string | - noobj | {n2n, EnumTypeName} |{outdir, Dir} | {i, IncludeDir} | optimize | - nif | asn1config | undec_rest | {inline, OutputName} | inline | + <v>Option = ber | per | uper | der | compact_bit_string | + noobj | {n2n, EnumTypeName} |{outdir, Dir} | {i, IncludeDir} | + asn1config | undec_rest | {inline, OutputName} | inline | {macro_name_prefix, Prefix} | {record_name_prefix, Prefix} | verbose | warnings_as_errors</v> <v>OldOption = ber | per</v> <v>Reason = term()</v> @@ -112,7 +125,7 @@ File3.asn </pre> section in users guide</seealso>. Available options are: </p> <taglist> - <tag><c>ber | ber_bin | per | per_bin | uper_bin</c></tag> + <tag><c>ber | per | uper</c></tag> <item> <p> The encoding rule to be used. The supported encoding rules @@ -120,23 +133,12 @@ File3.asn </pre> PER aligned (Packed Encoding Rules) and PER unaligned. If the encoding rule option is omitted <c>ber</c> is the default. - The <c>per_bin</c> option means the aligned - variant. To use the unaligned variant the <c>uper_bin</c> - option has to be used. </p> <p> The generated Erlang module always gets the same name as the ASN.1 module and as a consequence of this only one encoding rule per ASN.1 module can be used at runtime. </p> - <p> - The <c>ber_bin</c> and <c>per_bin</c> options are - equivalent with the <c>OldOptions</c> <c>ber</c> and <c>per</c> - with the difference that the generated encoding/decoding - functions take advantage of the bit syntax, which in most - cases increases the performance considerably. The result - from encoding is a binary or an iolist. - </p> </item> <tag><c>der</c></tag> <item> @@ -144,7 +146,7 @@ File3.asn </pre> By this option the Distinguished Encoding Rules (DER) is chosen. DER is regarded as a specialized variant of the BER encoding rule, therefore the <c>der</c> option only makes sense together - with the <c>ber</c> or <c>ber_bin</c> option. + with the <c>ber</c> option. This option sometimes adds sorting and value checks when encoding, which implies a slower encoding. The decoding routines are the same @@ -206,28 +208,6 @@ Binary = binary() shall be placed. If omitted the files are placed in the current directory.</p> </item> - <tag><c>optimize</c></tag> - <item> - <p>This option is only valid together with one of the - <c>per_bin</c> - or <c>ber_bin</c> option. It gives time optimized code - generated and it uses another runtime module and - in the <c>per_bin</c> case a nif. The result - in the <c>per_bin</c> case from an encode when compiled - with this option will be a binary.</p> - </item> - <tag><c>driver</c></tag> - <item> - <p>As of R15B this means the same as the <c>nif</c> option. Kept for - backwards compatability reasons.</p> - </item> - <tag><c>nif</c></tag> - <item> - <p>Option valid together with <c>ber_bin</c> and <c>optimize</c> - options. It enables the use of several nifs that gives faster - encode and decode. Nifs are only enabled by the explicit use of - the option <c>nif</c></p> - </item> <tag><c>asn1config</c></tag> <item> <p>When one of the specialized decodes, exclusive or @@ -270,10 +250,6 @@ Binary = binary() <c>{export, [atom()]}</c> or <c>{export_all, true}</c> option are provided. The list of atoms are names of chosen asn1 specs from the <c>.set.asn</c> file. </p> - <p>When used together with <c>nif</c> for <c>ber_bin</c>, the - asn1 nifs will be used if the <c>asn1rt_nif</c> module is - available. If it is not available, a slower erlang fallback - will be used.</p> </item> <tag><c>inline</c></tag> <item> @@ -327,13 +303,12 @@ Binary = binary() <type> <v>Module = Type = atom()</v> <v>Value = term()</v> - <v>Bytes = [Int] when integer(Int), Int >= 0, Int =< 255</v> + <v>Bytes = binary()</v> <v>Reason = term()</v> </type> <desc> <p>Encodes <c>Value</c> of <c>Type</c> defined in the ASN.1 module - <c>Module</c>. Returns a list of bytes if successful. To get as fast execution as - possible the + <c>Module</c>. To get as fast execution as possible the encode function only performs rudimentary tests that the input <c>Value</c> is a correct instance of <c>Type</c>. The length of strings is for example @@ -348,10 +323,10 @@ Binary = binary() <type> <v>Module = Type = atom()</v> <v>Value = Reason = term()</v> - <v>Bytes = [Int] when integer(Int), Int >= 0, Int =< 255</v> + <v>Bytes = binary()</v> </type> <desc> - <p>Decodes <c>Type</c> from <c>Module</c> from the list of bytes + <p>Decodes <c>Type</c> from <c>Module</c> from the binary <c>Bytes</c>. Returns <c>{ok, Value}</c> if successful.</p> </desc> </func> diff --git a/lib/asn1/doc/src/asn1rt.xml b/lib/asn1/doc/src/asn1rt.xml index 0c3c257189..f2cac0c9e7 100644 --- a/lib/asn1/doc/src/asn1rt.xml +++ b/lib/asn1/doc/src/asn1rt.xml @@ -47,35 +47,34 @@ <type> <v>Module = Type = atom()</v> <v>Value = Reason = term()</v> - <v>Bytes = binary | [Int] when integer(Int), Int >= 0, Int =< 255 | binary</v> + <v>Bytes = binary</v> </type> <desc> - <p>Decodes <c>Type</c> from <c>Module</c> from the list of bytes or - binary <c>Bytes</c>. If the module is compiled with <c>ber_bin</c> - or <c>per_bin</c> option <c>Bytes</c> must be a binary. + <p>Decodes <c>Type</c> from <c>Module</c> from the binary <c>Bytes</c>. Returns <c>{ok,Value}</c> if successful.</p> </desc> </func> <func> - <name>encode(Module,Type,Value)-> {ok,BinOrList} | {error,Reason}</name> + <name>encode(Module,Type,Value)-> {ok,Bytes} | {error,Reason}</name> <fsummary>Encode an ASN.1 value.</fsummary> <type> <v>Module = Type = atom()</v> <v>Value = term()</v> - <v>BinOrList = Bytes | binary()</v> - <v>Bytes = [Int|binary|Bytes] when integer(Int), Int >= 0, Int =< 255</v> + <v>Bytes = binary</v> <v>Reason = term()</v> </type> <desc> <p>Encodes <c>Value</c> of <c>Type</c> defined in the ASN.1 module <c>Module</c>. Returns a possibly nested list of bytes and or binaries - if successful. If <c>Module</c> was compiled with the options <c>per_bin</c> and <c>optimize</c> the result is a binary. To get as - fast execution as possible the + if successful. To get as fast execution as possible the encode function only performs rudimentary tests that the input <c>Value</c> is a correct instance of <c>Type</c>. The length of strings is for example not always checked. </p> + <note> + <p>Starting in R16, <c>Bytes</c> is always a binary.</p> + </note> </desc> </func> @@ -95,28 +94,6 @@ </func> <func> - <name>load_driver() -> ok | {error,Reason}</name> - <fsummary>Loads the linked-in driver. (deprecated)</fsummary> - <type> - <v>Reason = term()</v> - </type> - <desc> - <p>This function is obsolete and will be removed in R16A</p> - </desc> - </func> - - <func> - <name>unload_driver() -> ok | {error,Reason}</name> - <fsummary>Unloads the linked-in driver. (deprecated)</fsummary> - <type> - <v>Reason = term()</v> - </type> - <desc> - <p>This function is obsolete and will be removed in R16A</p> - </desc> - </func> - - <func> <name>utf8_binary_to_list(UTF8Binary) -> {ok,UnicodeList} | {error,Reason}</name> <fsummary>Transforms an utf8 encoded binary to a unicode list.</fsummary> <type> diff --git a/lib/asn1/doc/src/notes.xml b/lib/asn1/doc/src/notes.xml index 5ca86130a1..72b496caf7 100644 --- a/lib/asn1/doc/src/notes.xml +++ b/lib/asn1/doc/src/notes.xml @@ -483,7 +483,7 @@ ENUMERATION type, the compilation will be terminated with an error code.<br/> Below follows an example on how to use the option from the command line with <c>erlc</c>:<br/> - <c>erlc -bper_bin +optimize +driver +"{n2n,'CauseMisc'}" +"{n2n,'CausePcl'}" MyModyle.asn</c> + <c>erlc -bper+"{n2n,'CauseMisc'}" +"{n2n,'CausePcl'}" MyModyle.asn</c> </p> <p> Own Id: OTP-8136 Aux Id: seq11347 </p> diff --git a/lib/asn1/src/Makefile b/lib/asn1/src/Makefile index 4bd49aa93b..8d9422144e 100644 --- a/lib/asn1/src/Makefile +++ b/lib/asn1/src/Makefile @@ -52,10 +52,9 @@ CT_MODULES= \ asn1ct_gen_per_rt2ct \ asn1ct_name \ asn1ct_constructed_per \ - asn1ct_constructed_ber \ - asn1ct_gen_ber \ asn1ct_constructed_ber_bin_v2 \ asn1ct_gen_ber_bin_v2 \ + asn1ct_imm \ asn1ct_value \ asn1ct_tok \ asn1ct_parser2 \ @@ -63,7 +62,6 @@ CT_MODULES= \ RT_MODULES= \ asn1rt \ - asn1rt_per_bin \ asn1rt_ber_bin \ asn1rt_ber_bin_v2 \ asn1rt_per_bin_rt2ct \ @@ -166,11 +164,32 @@ release_spec: opt release_docs_spec: +# +# Dependencies +# - - - - - - - +$(EBIN)/asn1_app.beam: asn1_app.erl +$(EBIN)/asn1_db.beam: asn1_db.erl +$(EBIN)/asn1ct.beam: asn1ct.erl asn1_records.hrl +$(EBIN)/asn1ct_check.beam: asn1ct_check.erl asn1_records.hrl +$(EBIN)/asn1ct_constructed_ber_bin_v2.beam: asn1ct_constructed_ber_bin_v2.erl \ + asn1_records.hrl +$(EBIN)/asn1ct_constructed_per.beam: asn1ct_constructed_per.erl asn1_records.hrl +$(EBIN)/asn1ct_gen.beam: asn1ct_gen.erl asn1_records.hrl +$(EBIN)/asn1ct_gen_ber_bin_v2.beam: asn1ct_gen_ber_bin_v2.erl asn1_records.hrl +$(EBIN)/asn1ct_gen_per.beam: asn1ct_gen_per.erl asn1_records.hrl +$(EBIN)/asn1ct_gen_per_rt2ct.beam: asn1ct_gen_per_rt2ct.erl asn1_records.hrl +$(EBIN)/asn1ct_imm.beam: asn1ct_imm.erl +$(EBIN)/asn1ct_name.beam: asn1ct_name.erl +$(EBIN)/asn1ct_parser2.beam: asn1ct_parser2.erl asn1_records.hrl +$(EBIN)/asn1ct_pretty_format.beam: asn1ct_pretty_format.erl +$(EBIN)/asn1ct_table.beam: asn1ct_table.erl +$(EBIN)/asn1ct_tok.beam: asn1ct_tok.erl +$(EBIN)/asn1ct_value.beam: asn1ct_value.erl asn1_records.hrl +$(EBIN)/asn1rt.beam: asn1rt.erl +$(EBIN)/asn1rt_ber_bin.beam: asn1rt_ber_bin.erl asn1_records.hrl +$(EBIN)/asn1rt_ber_bin_v2.beam: asn1rt_ber_bin_v2.erl +$(EBIN)/asn1rt_check.beam: asn1rt_check.erl +$(EBIN)/asn1rt_nif.beam: asn1rt_nif.erl +$(EBIN)/asn1rt_per_bin_rt2ct.beam: asn1rt_per_bin_rt2ct.erl asn1_records.hrl +$(EBIN)/asn1rt_uper_bin.beam: asn1rt_uper_bin.erl asn1_records.hrl diff --git a/lib/asn1/src/asn1.app.src b/lib/asn1/src/asn1.app.src index 09144ba2f7..64b33a8a30 100644 --- a/lib/asn1/src/asn1.app.src +++ b/lib/asn1/src/asn1.app.src @@ -3,7 +3,6 @@ {vsn, "%VSN%"}, {modules, [ asn1rt, - asn1rt_per_bin, asn1rt_per_bin_rt2ct, asn1rt_uper_bin, asn1rt_ber_bin, diff --git a/lib/asn1/src/asn1ct.erl b/lib/asn1/src/asn1ct.erl index 8e971a1c76..90882462ac 100644 --- a/lib/asn1/src/asn1ct.erl +++ b/lib/asn1/src/asn1ct.erl @@ -85,14 +85,8 @@ compile(File) -> compile(File,[]). -compile(File,Options) when is_list(Options) -> - case lists:member(driver, Options) of %% remove me in R16A! - true -> - io:format("Warning: driver option is obsolete and will be removed in R16A, use nif instead!"); - false -> - ok - end, - Options1 = optimize_ber_bin(Options), +compile(File, Options0) when is_list(Options0) -> + Options1 = translate_options(Options0), Options2 = includes(File,Options1), Includes = strip_includes(Options2), in_process(fun() -> compile_proc(File, Includes, Options2) end). @@ -852,8 +846,8 @@ generate({true,{M,_Module,GenTOrV}},OutFile,EncodingRule,Options) -> debug_off(Options), put(compact_bit_string,false), erase(encoding_options), - erase(tlv_format), % used in ber_bin, optimize - erase(class_default_type),% used in ber_bin, optimize + erase(tlv_format), % used in ber + erase(class_default_type),% used in ber asn1ct_table:delete(check_functions), case Result of {error,_} -> @@ -876,14 +870,13 @@ parse_and_save(Module,S) -> Options = S#state.options, SourceDir = S#state.sourcedir, Includes = [I || {i,I} <-Options], - Options1 = optimize_ber_bin(Options), - + case get_input_file(Module,[SourceDir|Includes]) of %% search for asn1 source {file,SuffixedASN1source} -> - case dbfile_uptodate(SuffixedASN1source,Options1) of + case dbfile_uptodate(SuffixedASN1source,Options) of false -> - parse_and_save1(S,SuffixedASN1source,Options1,Includes); + parse_and_save1(S,SuffixedASN1source,Options,Includes); _ -> ok end; Err -> @@ -1065,9 +1058,9 @@ get_file_list1(Stream,Dir,Includes,Acc) -> end. get_rule(Options) -> - case [Rule ||Rule <-[per,ber,ber_bin,ber_bin_v2,per_bin,uper_bin], - Opt <- Options, - Rule==Opt] of + case [Rule || Rule <- [ber,per,uper], + Opt <- Options, + Rule =:= Opt] of [Rule] -> Rule; [Rule|_] -> @@ -1079,19 +1072,34 @@ get_rule(Options) -> get_runtime_mod(Options) -> RtMod1= case get_rule(Options) of - per -> ["asn1rt_per_bin.erl"]; - ber -> ["asn1rt_ber_bin.erl"]; - per_bin -> - case lists:member(optimize,Options) of - true -> ["asn1rt_per_bin_rt2ct.erl"]; - _ -> ["asn1rt_per_bin.erl"] - end; - ber_bin -> ["asn1rt_ber_bin.erl"]; - ber_bin_v2 -> ["asn1rt_ber_bin_v2.erl"]; - uper_bin -> ["asn1rt_uper_bin.erl"] + per -> "asn1rt_per_bin_rt2ct.erl"; + ber -> ["asn1rt_ber_bin_v2.erl"]; + uper -> ["asn1rt_uper_bin.erl"] end, RtMod1++["asn1rt_check.erl","asn1rt.erl"]. - + +%% translate_options(NewOptions) -> OldOptions +%% Translate the new option names to the old option name. +%% FIXME. We should rewrite all code to handle the new option names. + +translate_options([ber_bin|T]) -> + io:format("Warning: The option 'ber_bin' is now called 'ber'.\n"), + [ber|translate_options(T)]; +translate_options([per_bin|T]) -> + io:format("Warning: The option 'per_bin' is now called 'per'.\n"), + [per|translate_options(T)]; +translate_options([uper_bin|T]) -> + io:format("Warning: The option 'uper_bin' is now called 'uper'.\n"), + translate_options([uper|T]); +translate_options([nif|T]) -> + io:format("Warning: The option 'nif' is no longer needed.\n"), + translate_options(T); +translate_options([optimize|T]) -> + io:format("Warning: The option 'optimize' is no longer needed.\n"), + translate_options(T); +translate_options([H|T]) -> + [H|translate_options(T)]; +translate_options([]) -> []. erl_compile(OutFile,Options) -> % io:format("Options:~n~p~n",[Options]), @@ -1115,7 +1123,6 @@ remove_asn_flags(Options) -> X /= optimize, X /= compact_bit_string, X /= debug, - X /= keyed_list, X /= asn1config, X /= record_name_prefix]. @@ -1125,12 +1132,6 @@ debug_on(Options) -> put(asndebug,true); _ -> true - end, - case lists:member(keyed_list,Options) of - true -> - put(asn_keyed_list,true); - _ -> - true end. igorify_options(Options) -> @@ -1151,8 +1152,7 @@ generated_file(Name,Options) -> end. debug_off(_Options) -> - erase(asndebug), - erase(asn_keyed_list). + erase(asndebug). outfile(Base, Ext, Opts) -> @@ -1168,13 +1168,6 @@ outfile(Base, Ext, Opts) -> lists:concat([Obase,".",Ext]) end. -optimize_ber_bin(Options) -> - case {lists:member(optimize,Options),lists:member(ber_bin,Options)} of - {true,true} -> - [ber_bin_v2|Options--[ber_bin]]; - _ -> Options - end. - includes(File,Options) -> Options2 = include_append(".", Options), Options3 = include_append(filename:dirname(File), Options2), @@ -1284,12 +1277,7 @@ make_erl_options(Opts) -> Defines) ++ case OutputType of undefined -> [ber]; % temporary default (ber when it's ready) - ber -> [ber]; - ber_bin -> [ber_bin]; - ber_bin_v2 -> [ber_bin_v2]; - per -> [per]; - per_bin -> [per_bin]; - uper_bin -> [uper_bin] + _ -> [OutputType] % pass through end, Options++[errors, {cwd, Cwd}, {outdir, Outdir}| @@ -1400,8 +1388,7 @@ test_value(Module, Type, Value) -> in_process(fun() -> case catch encode(Module, Type, Value) of {ok, Bytes} -> - M = to_atom(Module), - NewBytes = prepare_bytes(M:encoding_rule(), Bytes), + NewBytes = prepare_bytes(Bytes), case decode(Module, Type, NewBytes) of {ok, Value} -> {ok, {Module, Type, Value}}; @@ -1452,18 +1439,8 @@ check(Module, Includes) -> end end. -to_atom(Term) when is_list(Term) -> list_to_atom(Term); -to_atom(Term) when is_atom(Term) -> Term. - -prepare_bytes(ber, Bytes) -> lists:flatten(Bytes); -prepare_bytes(ber_bin, Bytes) when is_binary(Bytes) -> Bytes; -prepare_bytes(ber_bin, Bytes) -> list_to_binary(Bytes); -prepare_bytes(ber_bin_v2, Bytes) when is_binary(Bytes) -> Bytes; -prepare_bytes(ber_bin_v2, Bytes) -> list_to_binary(Bytes); -prepare_bytes(per, Bytes) -> lists:flatten(Bytes); -prepare_bytes(per_bin, Bytes) when is_binary(Bytes) -> Bytes; -prepare_bytes(per_bin, Bytes) -> list_to_binary(Bytes); -prepare_bytes(uper_bin, Bytes) -> Bytes. +prepare_bytes(Bytes) when is_binary(Bytes) -> Bytes; +prepare_bytes(Bytes) -> list_to_binary(Bytes). vsn() -> ?vsn. @@ -1504,7 +1481,7 @@ specialized_decode_prepare(Erule,M,TsAndVs,Options) -> end. %% Reads the configuration file if it exists and stores information %% about partial decode and incomplete decode -partial_decode_prepare(ber_bin_v2,M,TsAndVs,Options) when is_tuple(TsAndVs) -> +partial_decode_prepare(ber,M,TsAndVs,Options) when is_tuple(TsAndVs) -> %% read configure file ModName = diff --git a/lib/asn1/src/asn1ct_check.erl b/lib/asn1/src/asn1ct_check.erl index 59e82b7a57..fe1b2e14a8 100644 --- a/lib/asn1/src/asn1ct_check.erl +++ b/lib/asn1/src/asn1ct_check.erl @@ -61,13 +61,13 @@ -define(TAG_PRIMITIVE(Num), case S#state.erule of - ber_bin_v2 -> + ber -> #tag{class='UNIVERSAL',number=Num,type='IMPLICIT',form=0}; _ -> [] end). -define(TAG_CONSTRUCTED(Num), case S#state.erule of - ber_bin_v2 -> + ber -> #tag{class='UNIVERSAL',number=Num,type='IMPLICIT',form=32}; _ -> [] end). @@ -3262,7 +3262,7 @@ check_type(S=#state{recordtopname=TopName},Type,Ts) when is_record(Ts,type) -> inlined=IsInlined}, TestFun = fun(Tref) -> - {_,MaybeChoice} = get_referenced_type(S,Tref), + MaybeChoice = get_non_typedef(S, Tref), case catch((MaybeChoice#typedef.typespec)#type.def) of {'CHOICE',_} -> maybe_illicit_implicit_tag(choice,Tag); @@ -3347,7 +3347,7 @@ check_type(S=#state{recordtopname=TopName},Type,Ts) when is_record(Ts,type) -> TempNewDef#newt{ type = check_externaltypereference(S,NewExt), tag = case S#state.erule of - ber_bin_v2 -> + ber -> merge_tags(Ct,RefType#type.tag); _ -> Ct @@ -3617,6 +3617,14 @@ check_type(S=#state{recordtopname=TopName},Type,Ts) when is_record(Ts,type) -> check_type(_S,Type,Ts) -> exit({error,{asn1,internal_error,Type,Ts}}). +get_non_typedef(S, Tref0) -> + case get_referenced_type(S, Tref0) of + {_,#typedef{typespec=#type{def=#'Externaltypereference'{}=Tref}}} -> + get_non_typedef(S, Tref); + {_,Type} -> + Type + end. + %% tablecinf_choose. A SEQUENCE or SET may be inserted in another %% SEQUENCE or SET by the COMPONENTS OF directive. If this inserted %% type is a referenced type that already has been checked it already @@ -5289,7 +5297,7 @@ iof_associated_type(S,[]) -> AssociateSeq = iof_associated_type1(S,[]), Tag = case S#state.erule of - ber_bin_v2 -> + ber -> [?TAG_CONSTRUCTED(?N_INSTANCE_OF)]; _ -> [] end, @@ -5320,7 +5328,7 @@ iof_associated_type1(S,C) -> end, {ObjIdTag,C1TypeTag}= case S#state.erule of - ber_bin_v2 -> + ber -> {[{'UNIVERSAL',8}], [#tag{class='UNIVERSAL', number=6, @@ -5551,8 +5559,9 @@ complist_as_tuple(_Per,[],Acc,Ext,_Acc2,ext) -> complist_as_tuple(_Per,[],Acc,Ext,Acc2,root2) -> {lists:reverse(Acc),lists:reverse(Ext),lists:reverse(Acc2)}. -is_erule_per(Erule) -> - lists:member(Erule,[per,per_bin,uper_bin]). +is_erule_per(per) -> true; +is_erule_per(uper) -> true; +is_erule_per(ber) -> false. expand_components(S, [{'COMPONENTS OF',Type}|T]) -> CompList = expand_components2(S,get_referenced_type(S,Type#type.def)), @@ -5641,7 +5650,7 @@ check_set(S,Type,Components) -> {true,_} -> {Sorted,SortedComponents} = sort_components(der,S,NewComponents), {Sorted,TableCInf,SortedComponents}; - {_,PER} when PER =:= per; PER =:= per_bin; PER =:= uper_bin -> + {_,PER} when PER =:= per; PER =:= uper -> {Sorted,SortedComponents} = sort_components(per,S,NewComponents), {Sorted,TableCInf,SortedComponents}; _ -> @@ -5765,7 +5774,7 @@ sort_universal_type(Components) -> decode_type(I) when is_integer(I) -> I; decode_type(T) -> - asn1ct_gen_ber:decode_type(T). + asn1ct_gen_ber_bin_v2:decode_type(T). untagged_choice(_S,[#'ComponentType'{typespec=#type{tag=[],def={'CHOICE',_}}}|_Rest]) -> true; @@ -6884,16 +6893,16 @@ get_taglist(S,{ObjCl,FieldNameList}) when is_record(ObjCl,objectclass), {TypeFieldName,_} when is_atom(TypeFieldName) -> []%should check if allowed end; get_taglist(S,Def) -> - case lists:member(S#state.erule,[ber_bin_v2]) of - false -> + case S#state.erule of + ber -> + []; + _ -> case Def of 'ASN1_OPEN_TYPE' -> % open_type has no UNIVERSAL tag as such []; _ -> [asn1ct_gen:def_to_tag(Def)] - end; - _ -> - [] + end end. get_taglist1(S,[#'ComponentType'{name=_Cname,tags=TagL}|Rest]) when is_list(TagL) -> diff --git a/lib/asn1/src/asn1ct_constructed_ber.erl b/lib/asn1/src/asn1ct_constructed_ber.erl deleted file mode 100644 index 360de77663..0000000000 --- a/lib/asn1/src/asn1ct_constructed_ber.erl +++ /dev/null @@ -1,1596 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1997-2011. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% -%% --module(asn1ct_constructed_ber). - --export([gen_encode_sequence/3]). --export([gen_decode_sequence/3]). --export([gen_encode_set/3]). --export([gen_decode_set/3]). --export([gen_encode_sof/4]). --export([gen_decode_sof/4]). --export([gen_encode_choice/3]). --export([gen_decode_choice/3]). - -%%%% Application internal exports --export([match_tag/2]). - --include("asn1_records.hrl"). - --import(asn1ct_gen, [emit/1,demit/1,get_record_name_prefix/0]). - -% the encoding of class of tag bits 8 and 7 --define(UNIVERSAL, 0). --define(APPLICATION, 16#40). --define(CONTEXT, 16#80). --define(PRIVATE, 16#C0). - -% primitive or constructed encoding % bit 6 --define(PRIMITIVE, 0). --define(CONSTRUCTED, 2#00100000). - - - - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Encode/decode SEQUENCE -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -gen_encode_sequence(Erules,Typename,D) when is_record(D,type) -> - asn1ct_name:start(), - asn1ct_name:new(term), - asn1ct_name:new(bytes), - - %% if EXTERNAL type the input value must be transformed to - %% ASN1 1990 format - case Typename of - ['EXTERNAL'] -> - emit([" NewVal = asn1rt_check:transform_to_EXTERNAL1990(Val),", - nl]); - _ -> - ok - end, - - {SeqOrSet,TableConsInfo,CompList0} = - case D#type.def of - #'SEQUENCE'{tablecinf=TCI,components=CL} -> - {'SEQUENCE',TCI,CL}; - #'SET'{tablecinf=TCI,components=CL} -> - {'SET',TCI,CL} - end, - %% filter away extensionAdditiongroup markers - CompList = filter_complist(CompList0), - Ext = extensible(CompList), - CompList1 = case CompList of - {Rl1,El,Rl2} -> Rl1 ++ El ++ Rl2; - {Rl,El} -> Rl ++ El; - _ -> CompList - end, - EncObj = - case TableConsInfo of - #simpletableattributes{usedclassfield=Used, - uniqueclassfield=Unique} when Used /= Unique -> - false; - %% ObjectSetRef, name of the object set in constraints - %% - %%{ObjectSetRef,AttrN,N,UniqueFieldName} - #simpletableattributes{objectsetname=ObjectSetRef, - c_name=AttrN, - c_index=N, - usedclassfield=UniqueFieldName, - uniqueclassfield=UniqueFieldName, - valueindex=ValueIndex - } -> - OSDef = - case ObjectSetRef of - {Module,OSName} -> - asn1_db:dbget(Module,OSName); - OSName -> - asn1_db:dbget(get(currmod),OSName) - end, -% io:format("currmod: ~p~nOSName: ~p~nAttrN: ~p~nN: ~p~nUniqueFieldName: ~p~n", -% [get(currmod),OSName,AttrN,N,UniqueFieldName]), - case (OSDef#typedef.typespec)#'ObjectSet'.gen of - true -> -% Val = lists:concat(["?RT_BER:cindex(", -% N+1,",Val,"]), - ObjectEncode = - asn1ct_gen:un_hyphen_var(lists:concat(['Obj', - AttrN])), - emit({ObjectEncode," = ",nl}), - {ObjSetMod,ObjSetName} = - case ObjectSetRef of - {M,O} -> - {{asis,M},O}; - O -> - {"?MODULE",O} - end, - emit({" ",ObjSetMod,":'getenc_",ObjSetName,"'(",{asis,UniqueFieldName}, - ", ",nl}), -% emit({indent(35),"?RT_BER:cindex(",N+1,", Val,", -% {asis,AttrN},")),",nl}), - Length = fun(X,_LFun) when is_atom(X) -> - length(atom_to_list(X)); - (X,_LFun) when is_list(X) -> - length(X); - ({X1,X2},LFun) -> - LFun(X1,LFun) + LFun(X2,LFun) - end, - emit([indent(10+Length(ObjectSetRef,Length)), - "value_match(",{asis,ValueIndex},",", - "?RT_BER:cindex(",N+1,",Val,", - {asis,AttrN},"))),",nl]), - notice_value_match(), - {AttrN,ObjectEncode}; - _ -> - false - end; - _ -> - case D#type.tablecinf of - [{objfun,_}|_] -> - %% when the simpletableattributes was at an - %% outer level and the objfun has been passed - %% through the function call - {"got objfun through args","ObjFun"}; - _ -> - false - end - end, - - gen_enc_sequence_call(Erules,Typename,CompList1,1,Ext,EncObj), - - MyTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- D#type.tag] - ++ - [#tag{class = asn1ct_gen_ber:decode_class('UNIVERSAL'), - number = asn1ct_gen_ber:decode_type(SeqOrSet), - form = ?CONSTRUCTED, - type = 'IMPLICIT'}], - emit([nl," BytesSoFar = "]), - case SeqOrSet of - 'SET' when (D#type.def)#'SET'.sorted == dynamic -> - emit("asn1rt_check:dynamicsort_SET_components(["), - mkvlist(asn1ct_name:all(encBytes)), - emit(["]),",nl]); - _ -> - emit("["), - mkvlist(asn1ct_name:all(encBytes)), - emit(["],",nl]) - end, - emit(" LenSoFar = "), - case asn1ct_name:all(encLen) of - [] -> emit("0"); - AllLengths -> - mkvplus(AllLengths) - end, - emit([",",nl]), -% emit(["{TagBytes,Len} = ?RT_BER:encode_tags(TagIn ++ ", - emit([" ?RT_BER:encode_tags(TagIn ++ ", - {asis,MyTag},", BytesSoFar, LenSoFar).",nl]). - - -gen_decode_sequence(Erules,Typename,D) when is_record(D,type) -> - asn1ct_name:start(), - asn1ct_name:new(tag), - #'SEQUENCE'{tablecinf=TableConsInfo,components=CList0} = D#type.def, - - %% filter away extensionAdditiongroup markers - CList = filter_complist(CList0), - - Ext = extensible(CList), - {CompList,CompList2} = case CList of - {Rl1,El,Rl2} -> {Rl1 ++ El ++ Rl2,CList}; - {Rl,El} -> {Rl ++ El, Rl ++ El}; - _ -> {CList,CList} - end, - - emit([" %%-------------------------------------------------",nl]), - emit([" %% decode tag and length ",nl]), - emit([" %%-------------------------------------------------",nl]), - - asn1ct_name:new(rb), - MyTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- D#type.tag] - ++ - [#tag{class = asn1ct_gen_ber:decode_class('UNIVERSAL'), - number = asn1ct_gen_ber:decode_type('SEQUENCE'), - form = ?CONSTRUCTED, - type = 'IMPLICIT'}], - emit([" {{_,",asn1ct_gen_ber:unused_var("Len",D#type.def),"},",{next,bytes},",",{curr,rb}, - "} = ?RT_BER:check_tags(TagIn ++ ",{asis,MyTag},", ", - {curr,bytes},", OptOrMand), ",nl]), - asn1ct_name:new(bytes), - asn1ct_name:new(len), - - case CompList of - [] -> true; - _ -> - emit({"{",{next,bytes}, - ",RemBytes} = ?RT_BER:split_list(", - {curr,bytes}, - ",", {prev,len},"),",nl}), - asn1ct_name:new(bytes) - end, - - {DecObjInf,UniqueFName,ValueIndex} = - case TableConsInfo of - #simpletableattributes{objectsetname=ObjectSet, - c_name=AttrN, - usedclassfield=UniqueFieldName, - uniqueclassfield=UniqueFieldName, - valueindex=ValIndex - } -> - F = fun(#'ComponentType'{typespec=CT})-> - case {asn1ct_gen:get_constraint(CT#type.constraint,componentrelation),CT#type.tablecinf} of -% case {CT#type.constraint,CT#type.tablecinf} of - {no,[{objfun,_}|_R]} -> true; - _ -> false - end - end, - case lists:any(F,CompList) of - %%AttributeName = asn1ct_gen:un_hyphen_var(AttrN), - true -> % when component relation constraint establish - %% relation from a component to another components - %% subtype component - {{AttrN,{deep,ObjectSet,UniqueFieldName, - ValIndex}}, - UniqueFieldName,ValIndex}; - false -> - {{AttrN,ObjectSet},UniqueFieldName,ValIndex} - end; - _ -> - {false,false,false} - end, - RecordName = lists:concat([get_record_name_prefix(),asn1ct_gen:list2rname(Typename)]), - case gen_dec_sequence_call(Erules,Typename,CompList2,Ext,DecObjInf) of - no_terms -> % an empty sequence - emit([nl,nl]), - demit({"Result = "}), %dbg - %% return value as record - asn1ct_name:new(rb), - emit([" {{'",RecordName,"'}, ",{curr,bytes},",",nl," "]), - asn1ct_gen_ber:add_removed_bytes(), - emit(["}.",nl]); - {LeadingAttrTerm,PostponedDecArgs} -> - emit([com,nl,nl]), - case {LeadingAttrTerm,PostponedDecArgs} of - {[],[]} -> - ok; - {_,[]} -> - ok; - {[{ObjSet,LeadingAttr,Term}],PostponedDecArgs} -> - DecObj = asn1ct_gen:un_hyphen_var(lists:concat(['DecObj',LeadingAttr,Term])), - ValueMatch = value_match(ValueIndex,Term), - {ObjSetMod,ObjSetName} = - case ObjSet of - {M,O} -> - {{asis,M},O}; - _ -> - {"?MODULE",ObjSet} - end, - emit([DecObj," =",nl," ",ObjSetMod,":'getdec_",ObjSetName,"'(", -% {asis,UniqueFName},", ",Term,"),",nl}), - {asis,UniqueFName},", ",ValueMatch,"),",nl]), - gen_dec_postponed_decs(DecObj,PostponedDecArgs) - end, - demit({"Result = "}), %dbg - %% return value as record - asn1ct_name:new(rb), - asn1ct_name:new(bytes), - ExtStatus = case Ext of - {ext,_,_} -> ext; - _ -> noext % noext | extensible - end, - emit([" {",{next,bytes},",",{curr,rb},"} = ?RT_BER:restbytes2(RemBytes, ", - {curr,bytes},",",ExtStatus,"),",nl]), - asn1ct_name:new(rb), - case Typename of - ['EXTERNAL'] -> - emit([" OldFormat={'",RecordName, - "', "]), - mkvlist(asn1ct_name:all(term)), - emit(["},",nl]), - emit([" ASN11994Format =",nl, - " asn1rt_check:transform_to_EXTERNAL1994", - "(OldFormat),",nl]), - emit([" {ASN11994Format,",{next,bytes},", "]); - _ -> - emit([" {{'",RecordName,"', "]), - mkvlist(asn1ct_name:all(term)), - emit(["}, ",{next,bytes},", "]) - end, - asn1ct_gen_ber:add_removed_bytes(), - emit(["}.",nl]) - end. - -gen_dec_postponed_decs(_,[]) -> - emit(nl); -gen_dec_postponed_decs(DecObj,[{_Cname,{FirstPFN,PFNList},Term,TmpTerm,_Tag,OptOrMand}|Rest]) -> -% asn1ct_name:new(term), - asn1ct_name:new(tmpterm), - asn1ct_name:new(reason), - - emit({"{",Term,", _, _} = ",nl}), - N = case OptOrMand of - mandatory -> 0; - 'OPTIONAL' -> - emit_opt_or_mand_check(asn1_NOVALUE,TmpTerm), - 6; - {'DEFAULT',Val} -> - emit_opt_or_mand_check(Val,TmpTerm), - 6 - end, - emit({indent(N+3),"case (catch ",DecObj,"(",{asis,FirstPFN}, -% ", ",TmpTerm,", ", {asis,Tag},", ",{asis,PFNList},")) of",nl}), - ", ",TmpTerm,", [], ",{asis,PFNList},")) of",nl}), - emit({indent(N+6),"{'EXIT', ",{curr,reason},"} ->",nl}), - emit({indent(N+9),"exit({'Type not compatible with table constraint',", - {curr,reason},"});",nl}), - emit({indent(N+6),{curr,tmpterm}," ->",nl}), - emit({indent(N+9),{curr,tmpterm},nl}), - - case OptOrMand of - mandatory -> emit([indent(N+3),"end,",nl]); - _ -> - emit([indent(N+3),"end",nl, - indent(3),"end,",nl]) - end, -% emit({indent(3),"end,",nl}), - gen_dec_postponed_decs(DecObj,Rest). - - -emit_opt_or_mand_check(Value,TmpTerm) -> - emit([indent(3),"case ",TmpTerm," of",nl, - indent(6),{asis,Value}," -> {",{asis,Value},",[],[]};",nl, - indent(6),"_ ->",nl]). - -%%============================================================================ -%% Encode/decode SET -%% -%%============================================================================ - -gen_encode_set(Erules,Typename,D) when is_record(D,type) -> - gen_encode_sequence(Erules,Typename,D). - -gen_decode_set(Erules,Typename,D) when is_record(D,type) -> - asn1ct_name:start(), - asn1ct_name:clear(), - asn1ct_name:new(term), - asn1ct_name:new(tag), - #'SET'{components=TCompList0} = D#type.def, - - %% filter away extensionAdditiongroup markers - TCompList = filter_complist(TCompList0), - Ext = extensible(TCompList), - ToOptional = fun(mandatory) -> - 'OPTIONAL'; - (X) -> X - end, - CompList = case TCompList of - {Rl1,El,Rl2} -> - Rl1 ++ [X#'ComponentType'{prop=ToOptional(Y)}||X = #'ComponentType'{prop=Y}<-El] ++ Rl2; - {Rl,El} -> Rl ++ El; - _ -> TCompList - end, - - emit([" %%-------------------------------------------------",nl]), - emit([" %% decode tag and length ",nl]), - emit([" %%-------------------------------------------------",nl]), - - asn1ct_name:new(rb), - MyTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- D#type.tag] - ++ - [#tag{class = asn1ct_gen_ber:decode_class('UNIVERSAL'), - number = asn1ct_gen_ber:decode_type('SET'), - form = ?CONSTRUCTED, - type = 'IMPLICIT'}], - emit([" {{_,Len},",{next,bytes},",",{curr,rb}, - "} = ?RT_BER:check_tags(TagIn ++ ",{asis,MyTag},", ", - {curr,bytes},", OptOrMand), ",nl]), - asn1ct_name:new(bytes), - asn1ct_name:new(len), - asn1ct_name:new(rb), - - emit([" {SetTerm, SetBytes, ",{curr,rb},"} = ?RT_BER:decode_set(0, Len, ", - {curr,bytes},", OptOrMand, ", - "fun 'dec_",asn1ct_gen:list2name(Typename),"_fun'/2, []),",nl]), - - asn1ct_name:new(rb), - {ExtFlatten1,ExtFlatten2} = - case Ext of - noext -> {"",""}; - _ -> {"lists:flatten(",")"} - end, - emit([" 'dec_",asn1ct_gen:list2name(Typename), - "__result__'(lists:sort(",ExtFlatten1,"SetTerm",ExtFlatten2,"), SetBytes, "]), - asn1ct_gen_ber:add_removed_bytes(), - emit([").",nl,nl,nl]), - - emit({"%%-------------------------------------------------",nl}), - emit({"%% Set loop fun for ",asn1ct_gen:list2name(Typename),nl}), - emit({"%%-------------------------------------------------",nl}), - - asn1ct_name:clear(), - asn1ct_name:new(term), - emit(["'dec_",asn1ct_gen:list2name(Typename),"_fun'(",{curr,bytes}, - ", OptOrMand) ->",nl]), - - asn1ct_name:new(bytes), - gen_dec_set(Erules,Typename,CompList,1,Ext), - - emit([" %% tag not found, if extensionmark we should skip bytes here",nl]), - emit([indent(6),"_ -> ",nl]), - case Ext of - noext -> - emit([indent(9),"{[], Bytes,0}",nl]); - _ -> - asn1ct_name:new(rbCho), - emit([indent(9),"{RestBytes, ",{curr,rbCho}, - "} = ?RT_BER:skipvalue(Bytes),",nl, - indent(9),"{[], RestBytes, ",{curr,rbCho},"}",nl]) - end, - emit([indent(3),"end.",nl,nl,nl]), - - - emit({"%%-------------------------------------------------",nl}), - emit({"%% Result ",asn1ct_gen:list2name(Typename),nl}), - emit({"%%-------------------------------------------------",nl}), - - asn1ct_name:clear(), - emit({"'dec_",asn1ct_gen:list2name(Typename),"__result__'(", - asn1ct_gen_ber:unused_var("TermList",D#type.def),", Bytes, Rb) ->",nl}), - RecordName = lists:concat([get_record_name_prefix(), - asn1ct_gen:list2rname(Typename)]), - case gen_dec_set_result(Erules,Typename,CompList) of - no_terms -> - %% return value as record - asn1ct_name:new(rb), - emit({" {{'",RecordName,"'}, Bytes, Rb}.",nl}); - _ -> - emit({nl," case ",{curr,termList}," of",nl}), - emit({" [] -> {{'",RecordName,"', "}), - mkvlist(asn1ct_name:all(term)), - emit({"}, Bytes, Rb};",nl}), - emit({" ExtraAtt -> exit({error,{asn1,{too_many_attributes, ExtraAtt}}})",nl}), - emit({" end.",nl}), - emit({nl,nl,nl}) - end. - - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Encode/decode SEQUENCE OF and SET OF -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -gen_encode_sof(Erules,Typename,_InnerTypename,D) when is_record(D,type) -> - asn1ct_name:start(), - {SeqOrSetOf, Cont} = D#type.def, - - Objfun = case D#type.tablecinf of - [{objfun,_}|_R] -> - ", ObjFun"; - _ -> - "" - end, - - emit({" {EncBytes,EncLen} = 'enc_",asn1ct_gen:list2name(Typename), - "_components'(Val",Objfun,",[],0),",nl}), - - MyTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- D#type.tag] - ++ - [#tag{class = asn1ct_gen_ber:decode_class('UNIVERSAL'), - number = asn1ct_gen_ber:decode_type(SeqOrSetOf), - form = ?CONSTRUCTED, - type = 'IMPLICIT'}], -% gen_encode_tags(Erules,MyTag,"EncLen","EncBytes"), - emit([" ?RT_BER:encode_tags(TagIn ++ ", - {asis,MyTag},", EncBytes, EncLen).",nl,nl]), - - gen_encode_sof_components(Erules,Typename,SeqOrSetOf,Cont). -% gen_enc_line(Erules,Typename,TypeNameSuffix,Cont,"H",0, -% mandatory,"{EncBytes,EncLen} = "), - - -gen_decode_sof(Erules,Typename,_InnerTypename,D) when is_record(D,type) -> - asn1ct_name:start(), - asn1ct_name:clear(), - {SeqOrSetOf, TypeTag, Cont} = - case D#type.def of - {'SET OF',_Cont} -> {'SET OF','SET',_Cont}; - {'SEQUENCE OF',_Cont} -> {'SEQUENCE OF','SEQUENCE',_Cont} - end, - TypeNameSuffix = asn1ct_gen:constructed_suffix(SeqOrSetOf,Cont#type.def), - - emit({" %%-------------------------------------------------",nl}), - emit({" %% decode tag and length ",nl}), - emit({" %%-------------------------------------------------",nl}), - - asn1ct_name:new(rb), - MyTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- D#type.tag] - ++ - [#tag{class = asn1ct_gen_ber:decode_class('UNIVERSAL'), - number = asn1ct_gen_ber:decode_type(TypeTag), - form = ?CONSTRUCTED, - type = 'IMPLICIT'}], - emit([" {{_,Len},",{next,bytes},",",{curr,rb}, - "} = ?RT_BER:check_tags(TagIn ++ ",{asis,MyTag},", ", - {curr,bytes},", OptOrMand), ",nl]), - - emit([" ?RT_BER:decode_components(",{curr,rb}]), - InnerType = asn1ct_gen:get_inner(Cont#type.def), - ContName = case asn1ct_gen:type(InnerType) of - Atom when is_atom(Atom) -> Atom; - _ -> TypeNameSuffix - end, - emit([", Len, ",{next,bytes},", "]), -% NewCont = -% case Cont#type.def of -% {'ENUMERATED',_,Components}-> -% Cont#type{def={'ENUMERATED',Components}}; -% _ -> Cont -% end, - ObjFun = - case D#type.tablecinf of - [{objfun,_}|_R] -> - ", ObjFun"; - _ -> - [] - end, - gen_dec_line_sof(Erules,Typename,ContName,Cont,ObjFun), - emit([", []).",nl,nl,nl]). - - -gen_encode_sof_components(Erules,Typename,SeqOrSetOf,Cont) - when is_record(Cont,type)-> - - {Objfun,ObjFun_novar,EncObj} = - case Cont#type.tablecinf of - [{objfun,_}|_R] -> - {", ObjFun",", _",{no_attr,"ObjFun"}}; - _ -> - {"","",false} - end, - emit(["'enc_",asn1ct_gen:list2name(Typename), - "_components'([]",ObjFun_novar,", AccBytes, AccLen) -> ",nl]), - - case catch lists:member(der,get(encoding_options)) of - true when SeqOrSetOf=='SET OF' -> - emit([indent(3), - "{asn1rt_check:dynamicsort_SETOF(AccBytes),AccLen};",nl,nl]); - _ -> - emit([indent(3),"{lists:reverse(AccBytes),AccLen};",nl,nl]) - end, - emit(["'enc_",asn1ct_gen:list2name(Typename), - "_components'([H|T]",Objfun,",AccBytes, AccLen) ->",nl]), - TypeNameSuffix = asn1ct_gen:constructed_suffix(SeqOrSetOf,Cont#type.def), - gen_enc_line(Erules,Typename,TypeNameSuffix,Cont,"H",3, -% mandatory,"{EncBytes,EncLen} = ",EncObj), - mandatory,EncObj), - emit([",",nl]), - emit([indent(3),"'enc_",asn1ct_gen:list2name(Typename), - "_components'(T",Objfun,","]), - emit(["[EncBytes|AccBytes], AccLen + EncLen).",nl,nl]). - -%%============================================================================ -%% Encode/decode CHOICE -%% -%%============================================================================ - -gen_encode_choice(Erules,Typename,D) when is_record(D,type) -> - ChoiceTag = D#type.tag, - {'CHOICE',CompList} = D#type.def, - Ext = extensible(CompList), - CompList1 = case CompList of - {Rl1,El,Rl2} -> Rl1 ++ El ++ Rl2; - {Rl,El} -> Rl ++ El; - _ -> CompList - end, - gen_enc_choice(Erules,Typename,ChoiceTag,CompList1,Ext), - emit({nl,nl}). - -gen_decode_choice(Erules,Typename,D) when is_record(D,type) -> - asn1ct_name:start(), - asn1ct_name:new(bytes), - ChoiceTag = D#type.tag, - {'CHOICE',CompList} = D#type.def, - Ext = extensible(CompList), - CompList1 = case CompList of - {Rl1,El,Rl2} -> Rl1 ++ El ++Rl2; - {Rl,El} -> Rl ++ El; - _ -> CompList - end, - gen_dec_choice(Erules,Typename,ChoiceTag,CompList1,Ext), - emit({".",nl}). - - -%%============================================================================ -%% Encode SEQUENCE -%% -%%============================================================================ - -gen_enc_sequence_call(Erules,TopType,[#'ComponentType'{name=Cname,typespec=Type,prop=Prop,textual_order=Order}|Rest],Pos,Ext,EncObj) -> - asn1ct_name:new(encBytes), - asn1ct_name:new(encLen), - CindexPos = - case Order of - undefined -> - Pos; - _ -> Order % der - end, - Element = - case TopType of - ['EXTERNAL'] -> - io_lib:format("?RT_BER:cindex(~w,NewVal,~w)",[CindexPos+1,Cname]); - _ -> - io_lib:format("?RT_BER:cindex(~w,Val,~w)",[CindexPos+1,Cname]) - end, - InnerType = asn1ct_gen:get_inner(Type#type.def), - print_attribute_comment(InnerType,Pos,Prop), - gen_enc_line(Erules,TopType,Cname,Type,Element,3,Prop,EncObj), - case Rest of - [] -> - emit({com,nl}); - _ -> - emit({com,nl}), - gen_enc_sequence_call(Erules,TopType,Rest,Pos+1,Ext,EncObj) - end; - -gen_enc_sequence_call(_Erules,_TopType,[],_Num,_,_) -> - true. - -%%============================================================================ -%% Decode SEQUENCE -%% -%%============================================================================ - -gen_dec_sequence_call(Erules,TopType,CompList,Ext,DecObjInf) - when is_list(CompList) -> - gen_dec_sequence_call1(Erules,TopType, CompList, 1, Ext,DecObjInf,[],[]); -gen_dec_sequence_call(Erules,TopType,CList,Ext,DecObjInf) -> - gen_dec_sequence_call2(Erules,TopType,CList,Ext,DecObjInf). - -gen_dec_sequence_call1(Erules,TopType,[#'ComponentType'{name=Cname,typespec=Type,prop=Prop,tags=Tags}|Rest],Num,Ext,DecObjInf,LeadingAttrAcc,ArgsAcc) -> - {LA,PostponedDec} = - gen_dec_component(Erules,TopType,Cname,Tags,Type,Num,Prop, - Ext,DecObjInf), - case Rest of - [] -> - {LA ++ LeadingAttrAcc,PostponedDec ++ ArgsAcc}; - _ -> - emit({com,nl}), -% asn1ct_name:new(term), - asn1ct_name:new(bytes), - gen_dec_sequence_call1(Erules,TopType,Rest,Num+1,Ext,DecObjInf, - LA++LeadingAttrAcc,PostponedDec++ArgsAcc) - end; - -gen_dec_sequence_call1(_Erules,_TopType,[],1,_,_,_,_) -> - no_terms. - -gen_dec_sequence_call2(_Erules,_TopType,{[],[],[]},_Ext,_DecObjInf) -> - no_terms; -gen_dec_sequence_call2(Erules,TopType,{Root1,EList,Root2},_Ext,DecObjInf) -> - {LA,ArgsAcc} = - case gen_dec_sequence_call1(Erules,TopType,Root1++EList,1, - extensible({Root1,EList}),DecObjInf,[],[]) of - no_terms -> - {[],[]}; - Res -> Res - end, - %% TagList is the tags of Root2 elements from the first up to and - %% including the first mandatory element. - TagList = get_root2_taglist(Root2,[]), - emit({com,nl}), - asn1ct_name:new(bytes), - emit([" {",{next,bytes},", ",{next,rb}, - "} = ?RT_BER:skip_ExtensionAdditions(", - {curr,bytes},", ",{asis,TagList},"),",nl]), - asn1ct_name:new(rb), - asn1ct_name:new(bytes), - gen_dec_sequence_call1(Erules,TopType,Root2, - length(Root1)+length(EList),noext, - DecObjInf,LA,ArgsAcc). - -%% returns a list of tags of the elements in the component (second -%% root) list up to and including the first mandatory tag. See 24.6 in -%% X.680 (7/2002) -get_root2_taglist([],Acc) -> - lists:reverse(Acc); -get_root2_taglist([#'ComponentType'{prop=Prop,typespec=Type}|Rest],Acc) -> - FirstTag = fun([])->[]; - ([H|_T])->H#tag{class=asn1ct_gen_ber:decode_class(H#tag.class)} - end(Type#type.tag), - case Prop of - mandatory -> - %% match_tags/ may be used - %% this is the last tag of interest -> return - lists:reverse([FirstTag|Acc]); - _ -> - get_root2_taglist(Rest,[FirstTag|Acc]) - end. - - -%%---------------------------- -%%SEQUENCE mandatory -%%---------------------------- - -gen_dec_component(Erules,TopType,Cname,CTags,Type,Pos,Prop,Ext,DecObjInf) -> - InnerType = - case Type#type.def of - #'ObjectClassFieldType'{type=OCFTType} -> OCFTType; - _ -> asn1ct_gen:get_inner(Type#type.def) - end, - - Prop1 = case {Prop,Ext} of - {_,{ext,Epos,_Root2pos}} when Pos < Epos -> - Prop; - {mandatory,{ext,Epos,_}} when Pos >= Epos -> - 'OPTIONAL'; - _ -> - Prop - end, - print_attribute_comment(InnerType,Pos,Prop1), - emit(" "), - - case {InnerType,DecObjInf} of - {{typefield,_},NotFalse} when NotFalse /= false -> - asn1ct_name:new(term), - asn1ct_name:new(tmpterm), - emit({"{",{curr,tmpterm},", ",{next,bytes},",",{next,rb},"} = "}); - {{objectfield,_,_},_} -> - asn1ct_name:new(term), - asn1ct_name:new(tmpterm), - emit({"{",{curr,tmpterm},", ",{next,bytes},",",{next,rb},"} = "}); - _ -> - asn1ct_name:new(term), - emit({"{",{curr,term},",",{next,bytes},",",{next,rb},"} = "}) - end, - asn1ct_name:new(rb), - PostponedDec = - gen_dec_line(Erules,TopType,Cname,CTags,Type,Prop1,DecObjInf), - asn1ct_name:new(form), - PostponedDec. - - -%%------------------------------------- -%% Decode SET -%%------------------------------------- - -gen_dec_set(Erules,TopType,CompList,Pos,Ext) -> - ExtCatch = case Ext of - noext ->""; - _ -> " catch" - end, - TagList = get_all_choice_tags(CompList), - emit({indent(3), - {curr,tagList}," = ",{asis,TagList},",",nl}), - emit({indent(3), - "case",ExtCatch," ?RT_BER:check_if_valid_tag(Bytes, ", - {curr,tagList},", OptOrMand) of",nl}), - asn1ct_name:new(tagList), - asn1ct_name:new(rbCho), - asn1ct_name:new(choTags), - gen_dec_set_cases(Erules,TopType,CompList,TagList,Pos), - asn1ct_name:new(tag), - asn1ct_name:new(bytes). - - - -gen_dec_set_cases(_,_,[],_,_) -> - ok; -gen_dec_set_cases(Erules,TopType,[H|T],List,Pos) -> - Name = H#'ComponentType'.name, - Type = H#'ComponentType'.typespec, - - emit({indent(6),"'",Name,"' ->",nl}), - case Type#type.def of - {'CHOICE',_NewCompList} -> - gen_dec_set_cases_choice(Erules,TopType,H,Pos); - _ -> - gen_dec_set_cases_type(Erules,TopType,H,Pos) - end, - gen_dec_set_cases(Erules,TopType,T,List,Pos+1). - - - -gen_dec_set_cases_choice(_Erules,TopType,H,Pos) -> - Cname = H#'ComponentType'.name, - Tag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)} - || X <- (H#'ComponentType'.typespec)#type.tag], - asn1ct_name:new(rbCho), - emit({indent(9),"{Dec, Rest, ",{curr,rbCho},"} = "}), - emit({"'dec_",asn1ct_gen:list2name([Cname|TopType]), - "'(Bytes,OptOrMand,",{asis,Tag},"),",nl}), - emit([" {{",Pos,",Dec}, Rest, ",{curr,rbCho},"}"]), - emit([";",nl,nl]). - - -gen_dec_set_cases_type(Erules,TopType,H,Pos) -> - Cname = H#'ComponentType'.name, - Type = H#'ComponentType'.typespec, - %% always use Prop = mandatory here Prop = H#'ComponentType'.prop, - - asn1ct_name:new(rbCho), - emit({indent(9),"{Dec, Rest, ",{curr,rbCho},"} = "}), - asn1ct_name:delete(bytes), - %% we have already seen the tag so now we must find the value - %% that why we always use 'mandatory' here - gen_dec_line(Erules,TopType,Cname,[],Type,mandatory,decObjInf), - asn1ct_name:new(bytes), - - emit([",",nl]), - emit(["{{",Pos,",Dec}, Rest, ",{curr,rbCho},"}"]), - emit([";",nl,nl]). - - -%%--------------------------------- -%% Decode SET result -%%--------------------------------- - -gen_dec_set_result(Erules,TopType,CompList) -> - gen_dec_set_result1(Erules,TopType, CompList, 1). - -gen_dec_set_result1(Erules,TopType, - [#'ComponentType'{name=Cname, - typespec=Type, - prop=Prop}|Rest],Num) -> - gen_dec_set_component(Erules,TopType,Cname,Type,Num,Prop), - case Rest of - [] -> - true; - _ -> - gen_dec_set_result1(Erules,TopType,Rest,Num+1) - end; - -gen_dec_set_result1(_Erules,_TopType,[],1) -> - no_terms; -gen_dec_set_result1(_Erules,_TopType,[],_Num) -> - true. - - -gen_dec_set_component(_Erules,_TopType,_Cname,Type,Pos,Prop) -> - InnerType = asn1ct_gen:get_inner(Type#type.def), - print_attribute_comment(InnerType,Pos,Prop), - emit({" {",{next,term},com,{next,termList},"} =",nl}), - emit({" case ",{curr,termList}," of",nl}), - emit({" [{",Pos,com,{curr,termTmp},"}|", - {curr,rest},"] -> "}), - emit({"{",{curr,termTmp},com, - {curr,rest},"};",nl}), - case Prop of - 'OPTIONAL' -> - emit([indent(10),"_ -> {asn1_NOVALUE, ",{curr,termList},"}",nl]); - {'DEFAULT', DefVal} -> - emit([indent(10), - "_ -> {",{asis,DefVal},", ",{curr,termList},"}",nl]); - mandatory -> - emit([indent(10), - "_ -> exit({error,{asn1,{mandatory_attribute_no, ", - Pos,", missing}}})",nl]) - end, - emit([indent(6),"end,",nl]), - asn1ct_name:new(rest), - asn1ct_name:new(term), - asn1ct_name:new(termList), - asn1ct_name:new(termTmp). - - -%%--------------------------------------------- -%% Encode CHOICE -%%--------------------------------------------- -%% for BER we currently do care (a little) if the choice has an EXTENSIONMARKER - - -gen_enc_choice(Erules,TopType,Tag,CompList,_Ext) -> - gen_enc_choice1(Erules,TopType,Tag,CompList,_Ext). - -gen_enc_choice1(Erules,TopType,Tag,CompList,_Ext) -> - asn1ct_name:clear(), - emit({" {EncBytes,EncLen} = case element(1,Val) of",nl}), - gen_enc_choice2(Erules,TopType,CompList), - emit([nl," end,",nl,nl]), - NewTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- Tag], -% gen_encode_tags(Erules,NewTag,"EncLen","EncBytes"). - emit(["?RT_BER:encode_tags(TagIn ++",{asis,NewTag},", EncBytes, EncLen).",nl]). - - - -gen_enc_choice2(Erules,TopType,[H1|T]) when is_record(H1,'ComponentType') -> - Cname = H1#'ComponentType'.name, - Type = H1#'ComponentType'.typespec, - emit({" ",{asis,Cname}," ->",nl}), - {Encobj,Assign} = -% case asn1ct_gen:get_constraint(Type#type.constraint, -% tableconstraint_info) of - case {Type#type.def,asn1ct_gen:get_constraint(Type#type.constraint, - componentrelation)} of - {#'ObjectClassFieldType'{},{componentrelation,_,_}} -> - asn1ct_name:new(tmpBytes), - asn1ct_name:new(encBytes), - asn1ct_name:new(encLen), - Emit = ["{",{curr,tmpBytes},", _} = "], - {{no_attr,"ObjFun"},Emit}; - _ -> - case Type#type.tablecinf of - [{objfun,_}] -> {{no_attr,"ObjFun"},[]}; - _-> {false,[]} - end - end, - gen_enc_line(Erules,TopType,Cname,Type,"element(2,Val)",9, - mandatory,Assign,Encobj), - case {Type#type.def,Encobj} of - {#'ObjectClassFieldType'{},{no_attr,"ObjFun"}} -> - emit({",",nl,indent(9),"{",{curr,encBytes},", ", - {curr,encLen},"}"}); - _ -> ok - end, - emit({";",nl}), - case T of - [] -> - emit([indent(6), "Else -> ",nl, - indent(9),"exit({error,{asn1,{invalid_choice_type,Else}}})"]); - _ -> - true - end, - gen_enc_choice2(Erules,TopType,T); - -gen_enc_choice2(_,_,[]) -> - true. - - - - -%%-------------------------------------------- -%% Decode CHOICE -%%-------------------------------------------- - -gen_dec_choice(Erules,TopType, ChTag, CompList, Ext) -> - asn1ct_name:delete(bytes), - Tags = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- ChTag], - - emit([" {{_,Len},",{next,bytes}, - ", RbExp} = ?RT_BER:check_tags(TagIn++", - {asis,Tags},", ", - {curr,bytes},", OptOrMand),",nl]), - asn1ct_name:new(bytes), - asn1ct_name:new(len), - gen_dec_choice_indef_funs(Erules), - case Erules of - ber_bin -> - emit([indent(3),"case ",{curr,bytes}," of",nl]); - ber -> - emit([indent(3), - "case (catch ?RT_BER:peek_tag(",{curr,bytes},")) of",nl]) - end, - asn1ct_name:new(tagList), - asn1ct_name:new(choTags), - gen_dec_choice_cases(Erules,TopType,CompList), - case Ext of - noext -> - emit([indent(6), {curr,else}," -> ",nl]), - emit([indent(9),"case OptOrMand of",nl, - indent(12),"mandatory ->","exit({error,{asn1,", - "{invalid_choice_tag,",{curr,else},"}}});",nl, - indent(12),"_ ->","exit({error,{asn1,{no_optional_tag,", - {curr,else},"}}})",nl, - indent(9),"end",nl]); - _ -> - emit([indent(6),"_ -> ",nl]), - emit([indent(9),"{{asn1_ExtAlt,",{curr,bytes},"},", - empty_lb(Erules),", RbExp}",nl]) - end, - emit([indent(3),"end"]), - asn1ct_name:new(tag), - asn1ct_name:new(else). - -gen_dec_choice_indef_funs(Erules) -> - emit({indent(3),"IndefEndBytes = fun(indefinite,",indefend_match(Erules,used_var), - ")-> R; (_,B)-> B end,",nl}), - emit({indent(3),"IndefEndRb = fun(indefinite,",indefend_match(Erules,unused_var), - ")-> 2; (_,_)-> 0 end,",nl}). - - -gen_dec_choice_cases(_,_, []) -> - ok; -gen_dec_choice_cases(Erules,TopType, [H|T]) -> - asn1ct_name:push(rbCho), - Name = H#'ComponentType'.name, - emit([nl,"%% '",Name,"'",nl]), - Fcases = fun([T1,T2|Tail],Fun) -> - emit([indent(6),match_tag(Erules,T1)," ->",nl]), - gen_dec_choice_cases_type(Erules,TopType, H), - Fun([T2|Tail],Fun); - ([T1],_) -> - emit([indent(6),match_tag(Erules,T1)," ->",nl]), - gen_dec_choice_cases_type(Erules,TopType, H) - end, - Fcases(H#'ComponentType'.tags,Fcases), - asn1ct_name:pop(rbCho), - gen_dec_choice_cases(Erules,TopType, T). - - - -gen_dec_choice_cases_type(Erules,TopType,H) -> - Cname = H#'ComponentType'.name, - Type = H#'ComponentType'.typespec, - Prop = H#'ComponentType'.prop, - emit({indent(9),"{Dec, Rest, ",{curr,rbCho},"} = "}), - gen_dec_line(Erules,TopType,Cname,[],Type,Prop,false), - emit([",",nl,indent(9),"{{",{asis,Cname}, - ", Dec}, IndefEndBytes(Len,Rest), RbExp + ", - {curr,rbCho}," + IndefEndRb(Len,Rest)};",nl,nl]). - -encode_tag_val(Erules,{Class,TagNo}) when is_integer(TagNo) -> - Rtmod = rtmod(Erules), - Rtmod:encode_tag_val({asn1ct_gen_ber:decode_class(Class), - 0,TagNo}); -encode_tag_val(Erules,{Class,TypeName}) -> - Rtmod = rtmod(Erules), - Rtmod:encode_tag_val({asn1ct_gen_ber:decode_class(Class), - 0,asn1ct_gen_ber:decode_type(TypeName)}). - - -match_tag(ber_bin,Arg) -> - match_tag_with_bitsyntax(Arg); -match_tag(Erules,Arg) -> - io_lib:format("~p",[encode_tag_val(Erules,Arg)]). - -match_tag_with_bitsyntax({Class,TagNo}) when is_integer(TagNo) -> - match_tag_with_bitsyntax1({asn1ct_gen_ber:decode_class(Class), - 0,TagNo}); -match_tag_with_bitsyntax({Class,TypeName}) -> - match_tag_with_bitsyntax1({asn1ct_gen_ber:decode_class(Class), - 0,asn1ct_gen_ber:decode_type(TypeName)}). - -match_tag_with_bitsyntax1({Class, _Form, TagNo}) when (TagNo =< 30) -> - io_lib:format("<<~p:2,_:1,~p:5,_/binary>>",[Class bsr 6,TagNo]); - -match_tag_with_bitsyntax1({Class, _Form, TagNo}) -> - {Octets,Len} = mk_object_val(TagNo), - OctForm = case Len of - 1 -> "~p"; - 2 -> "~p,~p"; - 3 -> "~p,~p,~p"; - 4 -> "~p,~p,~p,~p" - end, - io_lib:format("<<~p:2,_:1,31:5," ++ OctForm ++ ",_/binary>>", - [Class bsr 6] ++ Octets). - -%%%%%%%%%%% -%% mk_object_val(Value) -> {OctetList, Len} -%% returns a Val as a list of octets, the 8 bit is allways set to one except -%% for the last octet, where its 0 -%% - - -mk_object_val(Val) when Val =< 127 -> - {[255 band Val], 1}; -mk_object_val(Val) -> - mk_object_val(Val bsr 7, [Val band 127], 1). -mk_object_val(0, Ack, Len) -> - {Ack, Len}; -mk_object_val(Val, Ack, Len) -> - mk_object_val(Val bsr 7, [((Val band 127) bor 128) | Ack], Len + 1). - - -get_all_choice_tags(ComponentTypeList) -> - get_all_choice_tags(ComponentTypeList,[]). - -get_all_choice_tags([],TagList) -> - TagList; -get_all_choice_tags([H|T],TagList) -> - Tags = H#'ComponentType'.tags, - get_all_choice_tags(T, TagList ++ [{H#'ComponentType'.name, Tags}]). - - - -%%--------------------------------------- -%% Generate the encode/decode code -%%--------------------------------------- - -gen_enc_line(Erules,TopType,Cname, - Type=#type{constraint=C, - def=#'ObjectClassFieldType'{type={typefield,_}}}, - Element,Indent,OptOrMand=mandatory,EncObj) - when is_list(Element) -> - case asn1ct_gen:get_constraint(C,componentrelation) of - {componentrelation,_,_} -> - asn1ct_name:new(tmpBytes), - gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand, - ["{",{curr,tmpBytes},",_} = "],EncObj); - _ -> - gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand, - ["{",{curr,encBytes},",",{curr,encLen},"} = "], - EncObj) - end; - gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand,EncObj) - when is_list(Element) -> - gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand, - ["{",{curr,encBytes},",",{curr,encLen},"} = "],EncObj). - -gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand,Assign,EncObj) - when is_list(Element) -> - IndDeep = indent(Indent), - - Tag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)} - || X <- Type#type.tag], - InnerType = asn1ct_gen:get_inner(Type#type.def), - WhatKind = asn1ct_gen:type(InnerType), - emit(IndDeep), - emit(Assign), - gen_optormand_case(OptOrMand,Erules,TopType,Cname,Type,InnerType,WhatKind, - Element), - case {Type,asn1ct_gen:get_constraint(Type#type.constraint, - componentrelation)} of - {#type{def=#'ObjectClassFieldType'{type={typefield,_}, - fieldname=RefedFieldName}}, - {componentrelation,_,_}} -> - {_LeadingAttrName,Fun} = EncObj, - case RefedFieldName of - {Name,RestFieldNames} when is_atom(Name),Name =/= notype -> - case OptOrMand of - mandatory -> ok; - _ -> - emit(["{",{curr,tmpBytes},", _} = "]) - end, - emit({Fun,"(",{asis,Name},", ",Element,", [], ", - {asis,RestFieldNames},"),",nl}), - emit(IndDeep), - case OptOrMand of - mandatory -> - emit({"{",{curr,encBytes},", ",{curr,encLen},"} = "}), - emit({"?RT_BER:encode_open_type(",{curr,tmpBytes}, - ",",{asis,Tag},")"}); - _ -> - emit({"{",{next,tmpBytes},", ",{curr,tmpLen}, - "} = "}), - emit({"?RT_BER:encode_open_type(",{curr,tmpBytes}, - ",",{asis,Tag},"),",nl}), - emit(IndDeep), - emit({"{",{next,tmpBytes},", ",{curr,tmpLen},"}"}) - end; - Err -> - throw({asn1,{'internal error',Err}}) - end; - _ -> - case WhatKind of - {primitive,bif} -> - EncType = - case Type#type.def of - #'ObjectClassFieldType'{ - type={fixedtypevaluefield, - _,Btype}} -> - Btype; - _ -> - Type - end, - asn1ct_gen_ber:gen_encode_prim(ber,EncType,{asis,Tag}, - Element); - 'ASN1_OPEN_TYPE' -> - asn1ct_gen_ber:gen_encode_prim(ber,Type#type{def='ASN1_OPEN_TYPE'},{asis,Tag},Element); - _ -> - {EncFunName, _, _} = - mkfuncname(TopType,Cname,WhatKind,enc), - case {WhatKind,Type#type.tablecinf,EncObj} of - {{constructed,bif},[{objfun,_}|_R],{_,Fun}} -> - emit([EncFunName,"(",Element,", ",{asis,Tag}, - ", ",Fun,")"]); - _ -> - emit([EncFunName,"(",Element,", ",{asis,Tag},")"]) - end - end - end, - case OptOrMand of - mandatory -> true; - _ -> - emit({nl,indent(7),"end"}) - end. - - - -gen_optormand_case(mandatory,_,_,_,_,_,_, _) -> - ok; -gen_optormand_case('OPTIONAL',Erules,_,_,_,_,_,Element) -> - emit({" case ",Element," of",nl}), - emit({indent(9),"asn1_NOVALUE -> {", - empty_lb(Erules),",0};",nl}), - emit({indent(9),"_ ->",nl,indent(12)}); -gen_optormand_case({'DEFAULT',DefaultValue},Erules,TopType,Cname,Type, - InnerType,WhatKind,Element) -> - CurrMod = get(currmod), - case catch lists:member(der,get(encoding_options)) of - true -> - emit(" case catch "), - asn1ct_gen:gen_check_call(TopType,Cname,Type,InnerType, - WhatKind,{asis,DefaultValue}, - Element), - emit({" of",nl}), - emit({indent(12),"true -> {[],0};",nl}); - _ -> - emit({" case ",Element," of",nl}), - emit({indent(9),"asn1_DEFAULT -> {", - empty_lb(Erules), - ",0};",nl}), - case DefaultValue of - #'Externalvaluereference'{module=CurrMod, - value=V} -> - emit({indent(9),"?",{asis,V}," -> {", - empty_lb(Erules),",0};",nl}); - _ -> - emit({indent(9),{asis, - DefaultValue}," -> {", - empty_lb(Erules),",0};",nl}) - end - end, - emit({indent(9),"_ ->",nl,indent(12)}). - - - - -gen_dec_line_sof(Erules,TopType,Cname,Type,ObjFun) -> - - Tag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)} - || X <- Type#type.tag], - InnerType = asn1ct_gen:get_inner(Type#type.def), - WhatKind = asn1ct_gen:type(InnerType), - case WhatKind of - {primitive,bif} -> - asn1ct_name:delete(len), - - asn1ct_name:new(len), - emit(["fun(FBytes,_,_)->",nl]), - EncType = case Type#type.def of - #'ObjectClassFieldType'{ - type={fixedtypevaluefield, - _,Btype}} -> - Btype; - _ -> - Type - end, - asn1ct_gen_ber:gen_dec_prim(ber,EncType,"FBytes",Tag, - [],no_length,?PRIMITIVE, - mandatory), - emit([nl,"end, []"]); - _ -> - case ObjFun of - [] -> - {DecFunName, _, _} = - mkfunname(Erules,TopType,Cname,WhatKind,dec,3), - emit([DecFunName,", ",{asis,Tag}]); - _ -> - {DecFunName, _, _} = - mkfunname(Erules,TopType,Cname,WhatKind,dec,4), - emit([DecFunName,", ",{asis,Tag},", ObjFun"]) - end - end. - - -gen_dec_line(Erules,TopType,Cname,CTags,Type,OptOrMand,DecObjInf) -> - BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), - Tag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)} - || X <- Type#type.tag], - InnerType = - case Type#type.def of - #'ObjectClassFieldType'{type=OCFTType} -> - OCFTType; - _ -> - asn1ct_gen:get_inner(Type#type.def) - end, - PostpDec = - case OptOrMand of - mandatory -> - gen_dec_call(InnerType,Erules,TopType,Cname,Type, - BytesVar,Tag,mandatory,", mandatory, ", - DecObjInf,OptOrMand); - _ -> %optional or default - case {CTags,Erules} of - {[CTag],ber_bin} when CTag =/= [] -> % R9C-0.patch-34 - emit(["case ",{curr,bytes}," of",nl]), - emit([match_tag(Erules,CTag)," ->",nl]), - PostponedDec = - gen_dec_call(InnerType,Erules,TopType,Cname,Type, - BytesVar,Tag,mandatory, - ", opt_or_default, ",DecObjInf, - OptOrMand), - emit([";",nl]), - emit(["_ ->",nl]), - case OptOrMand of - {'DEFAULT', Def} -> - emit(["{",{asis,Def},",", - BytesVar,", 0 }",nl]); - 'OPTIONAL' -> - emit(["{ asn1_NOVALUE, ", - BytesVar,", 0 }",nl]) - end, - emit("end"), - PostponedDec; - _ -> - emit("case (catch "), - PostponedDec = - gen_dec_call(InnerType,Erules,TopType,Cname,Type, - BytesVar,Tag,OptOrMand, - ", opt_or_default, ",DecObjInf, - OptOrMand), - emit([") of",nl]), - case OptOrMand of - {'DEFAULT', Def} -> - emit(["{'EXIT',{error,{asn1,{no_optional_tag,_}}}}", - " -> {",{asis,Def},",", - BytesVar,", 0 };",nl]); - 'OPTIONAL' -> - emit(["{'EXIT',{error,{asn1,{no_optional_tag,_}}}}", - " -> { asn1_NOVALUE, ", - BytesVar,", 0 };",nl]) - end, - asn1ct_name:new(casetmp), - emit([{curr,casetmp},"-> ",{curr,casetmp},nl,"end"]), - PostponedDec - end - end, - case DecObjInf of - {Cname,ObjSet} -> % this must be the component were an object is - %% choosen from the object set according to the table - %% constraint. - ObjSetName = case ObjSet of - {deep,OSName,_,_} -> - OSName; - _ -> ObjSet - end, - {[{ObjSetName,Cname,asn1ct_gen:mk_var(asn1ct_name:curr(term))}], - PostpDec}; - _ -> {[],PostpDec} - end. - - -gen_dec_call({typefield,_},Erules,_,_,Type,_,Tag,_,_,false,_) -> - %% this in case of a choice with typefield components - asn1ct_name:new(reason), - {FirstPFName,RestPFName} = -% asn1ct_gen:get_constraint(Type#type.constraint, -% tableconstraint_info), - (Type#type.def)#'ObjectClassFieldType'.fieldname, - emit([nl,indent(6),"begin",nl]), - emit([indent(9),"{OpenDec,TmpRest,TmpRbCho} =",nl,indent(12), - "?RT_BER:decode_open_type(",Erules,",",{curr,bytes},",", - {asis,Tag},"),",nl]), - emit([indent(9),"case (catch ObjFun(",{asis,FirstPFName}, - ", OpenDec, [], ",{asis,RestPFName}, - ")) of", nl]),%% ??? What about Tag - emit([indent(12),"{'EXIT',",{curr,reason},"} ->",nl]), -%% emit({indent(15),"throw({runtime_error,{'Type not ", -%% "compatible with tableconstraint', OpenDec}});",nl}), - emit([indent(15),"exit({'Type not ", - "compatible with table constraint', ",{curr,reason},"});",nl]), - emit([indent(12),"{TmpDec,_ ,_} ->",nl]), - emit([indent(15),"{TmpDec, TmpRest, TmpRbCho}",nl]), - emit([indent(9),"end",nl,indent(6),"end",nl]), - []; -gen_dec_call({typefield,_},_Erules,_,Cname,Type,_BytesVar,Tag,_,_, - _DecObjInf,OptOrMandComp) -> - emit(["?RT_BER:decode_open_type(",{curr,bytes},",",{asis,Tag},")"]), - RefedFieldName = - (Type#type.def)#'ObjectClassFieldType'.fieldname, -% asn1ct_gen:get_constraint(Type#type.constraint, -% tableconstraint_info), - [{Cname,RefedFieldName, - asn1ct_gen:mk_var(asn1ct_name:curr(term)), -% asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)),[],OptOrMandComp}]; - asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)),Tag,OptOrMandComp}]; -gen_dec_call({objectfield,PrimFieldName,PFNList},_Erules,_,Cname,_,_,Tag,_,_,_, - OptOrMandComp) -> - emit(["?RT_BER:decode_open_type(",{curr,bytes},",",{asis,Tag},")"]), - [{Cname,{PrimFieldName,PFNList}, - asn1ct_gen:mk_var(asn1ct_name:curr(term)), -% asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)),[],OptOrMandComp}]; - asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)),Tag,OptOrMandComp}]; -gen_dec_call(InnerType,Erules,TopType,Cname,Type,BytesVar,Tag,PrimOptOrMand, - OptOrMand,DecObjInf,_) -> - WhatKind = asn1ct_gen:type(InnerType), - gen_dec_call1(WhatKind,InnerType,Erules,TopType,Cname,Type,BytesVar,Tag, - PrimOptOrMand,OptOrMand), - case DecObjInf of - {Cname,{_,OSet,UniqueFName,ValIndex}} -> - Term = asn1ct_gen:mk_var(asn1ct_name:curr(term)), - ValueMatch = value_match(ValIndex,Term), - {ObjSetMod,ObjSetName} = - case OSet of - {M,O} -> - {{asis,M},O}; - _ -> - {"?MODULE",OSet} - end, - emit({",",nl,"ObjFun = ",ObjSetMod,":'getdec_",ObjSetName,"'(", - {asis,UniqueFName},", ",ValueMatch,")"}); - _ -> - ok - end, - []. -gen_dec_call1({primitive,bif},InnerType,Erules,_,_,Type,BytesVar, - Tag,OptOrMand,_) -> - case InnerType of - {fixedtypevaluefield,_,Btype} -> - asn1ct_gen_ber:gen_dec_prim(Erules,Btype,BytesVar,Tag,[],no_length, - ?PRIMITIVE,OptOrMand); - _ -> - asn1ct_gen_ber:gen_dec_prim(Erules,Type,BytesVar,Tag,[],no_length, - ?PRIMITIVE,OptOrMand) - end; -gen_dec_call1('ASN1_OPEN_TYPE',_InnerType,Erules,_,_,Type,BytesVar, - Tag,OptOrMand,_) -> - asn1ct_gen_ber:gen_dec_prim(Erules,Type#type{def='ASN1_OPEN_TYPE'}, - BytesVar,Tag,[],no_length, - ?PRIMITIVE,OptOrMand); -gen_dec_call1(WhatKind,_,_Erules,TopType,Cname,Type,_,Tag,_,OptOrMand) -> - {DecFunName,_,_} = - mkfuncname(TopType,Cname,WhatKind,dec), - case {WhatKind,Type#type.tablecinf} of - {{constructed,bif},[{objfun,_}|_R]} -> - emit({DecFunName,"(",{curr,bytes},OptOrMand,{asis,Tag},", ObjFun)"}); - _ -> - emit({DecFunName,"(",{curr,bytes},OptOrMand,{asis,Tag},")"}) - end. - - -%%------------------------------------------------------ -%% General and special help functions (not exported) -%%------------------------------------------------------ - - -indent(N) -> - lists:duplicate(N,32). % 32 = space - - -mkvlist([H,T1|T], Sep) -> % Sep is a string e.g ", " or "+ " - emit([{var,H},Sep]), - mkvlist([T1|T], Sep); -mkvlist([H|T], Sep) -> - emit([{var,H}]), - mkvlist(T, Sep); -mkvlist([], _) -> - true. - -mkvlist(L) -> - mkvlist(L,", "). - -mkvplus(L) -> - mkvlist(L," + "). - -extensible(CompList) when is_list(CompList) -> - noext; -extensible({RootList,ExtList}) -> - {ext,length(RootList)+1,length(ExtList)}; -extensible({_Rl1,_ExtL,_Rl2}) -> - extensible. -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% filter away ExtensionAdditionGroup start and end marks since these -%% have no significance for the BER encoding -%% -filter_complist(CompList) when is_list(CompList) -> - lists:filter(fun(#'ExtensionAdditionGroup'{}) -> - false; - ('ExtensionAdditionGroupEnd') -> - false; - (_) -> - true - end, CompList); -filter_complist({Root,Ext}) -> - {Root,filter_complist(Ext)}; -filter_complist({Root1,Ext,Root2}) -> - {Root1,filter_complist(Ext),Root2}. - -print_attribute_comment(InnerType,Pos,Prop) -> - CommentLine = "%%-------------------------------------------------", - emit([nl,CommentLine]), - case InnerType of - {typereference,_,Name} -> - emit([nl,"%% attribute number ",Pos," with type ",Name]); - {'Externaltypereference',_,XModule,Name} -> - emit([nl,"%% attribute number ",Pos," External ",XModule,":",Name]); - _ -> - emit([nl,"%% attribute number ",Pos," with type ",InnerType]) - end, - case Prop of - mandatory -> - continue; - {'DEFAULT', Def} -> - emit([" DEFAULT = ",{asis,Def}]); - 'OPTIONAL' -> - emit([" OPTIONAL"]) - end, - emit([nl,CommentLine,nl]). - - -mkfuncname(TopType,Cname,WhatKind,DecOrEnc) -> - CurrMod = get(currmod), - case WhatKind of - #'Externaltypereference'{module=CurrMod,type=EType} -> - F = lists:concat(["'",DecOrEnc,"_",EType,"'"]), - {F, "?MODULE", F}; - #'Externaltypereference'{module=Mod,type=EType} -> - {lists:concat(["'",Mod,"':'",DecOrEnc,"_",EType,"'"]),Mod, - lists:concat(["'",DecOrEnc,"_",EType,"'"])}; - {constructed,bif} -> - F = lists:concat(["'",DecOrEnc,"_",asn1ct_gen:list2name([Cname|TopType]),"'"]), - {F, "?MODULE", F} - end. - -mkfunname(Erule,TopType,Cname,WhatKind,DecOrEnc,Arity) -> - CurrMod = get(currmod), - case WhatKind of - #'Externaltypereference'{module=CurrMod,type=EType} -> - F = lists:concat(["fun '",DecOrEnc,"_",EType,"'/",Arity]), - {F, "?MODULE", F}; - #'Externaltypereference'{module=Mod,type=EType} -> - {lists:concat(["fun '",Mod,"':'",DecOrEnc,"_",EType,"'/",Arity]),Mod, - lists:concat(["'",DecOrEnc,"_",EType,"'"])}; - {constructed,bif} -> - F = - lists:concat(["fun '",DecOrEnc,"_", - asn1ct_gen:list2name([Cname|TopType]),"'/", - Arity]), - {F, "?MODULE", F}; - 'ASN1_OPEN_TYPE' -> - case Arity of - 3 -> - F = lists:concat(["fun(A,_,C) -> ?RT_BER:decode_open_type(",Erule,",A,C) end"]), - {F, "?MODULE", F}; - 4 -> - F = lists:concat(["fun(A,_,C,_) -> ?RT_BER:decode_open_type(",Erule,",A,C) end"]), - {F, "?MODULE", F} - end - end. - -empty_lb(ber) -> - "[]"; -empty_lb(ber_bin) -> - "<<>>". - -rtmod(ber) -> - list_to_atom(?RT_BER_BIN); -rtmod(ber_bin) -> - list_to_atom(?RT_BER_BIN). - -indefend_match(ber,used_var) -> - "[0,0|R]"; -indefend_match(ber,unused_var) -> - "[0,0|_R]"; -indefend_match(ber_bin,used_var) -> - "<<0,0,R/binary>>"; -indefend_match(ber_bin,unused_var) -> - "<<0,0,_R/binary>>". - -notice_value_match() -> - Module = get(currmod), - put(value_match,{true,Module}). - -value_match(Index,Value) when is_atom(Value) -> - value_match(Index,atom_to_list(Value)); -value_match([],Value) -> - Value; -value_match([{VI,_Cname}|VIs],Value) -> - value_match1(Value,VIs,lists:concat(["element(",VI,","]),1). -value_match1(Value,[],Acc,Depth) -> - Acc ++ Value ++ lists:concat(lists:duplicate(Depth,")")); -value_match1(Value,[{VI,_Cname}|VIs],Acc,Depth) -> - value_match1(Value,VIs,Acc++lists:concat(["element(",VI,","]),Depth+1). diff --git a/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl b/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl index 2c4b44996d..78cb9297d8 100644 --- a/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl +++ b/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl @@ -32,7 +32,6 @@ -include("asn1_records.hrl"). -import(asn1ct_gen, [emit/1,demit/1,get_record_name_prefix/0]). --import(asn1ct_constructed_ber,[match_tag/2]). -define(ASN1CT_GEN_BER,asn1ct_gen_ber_bin_v2). @@ -913,7 +912,7 @@ gen_dec_choice_cases(Erules,TopType, [H|T]) -> [DecTag],Type}), asn1ct:update_gen_state(namelist,Names), emit([indent(4),{curr,res}," = ", - match_tag(ber_bin,{FirstT#tag.class,FirstT#tag.number}), + match_tag(FirstT#tag.class, FirstT#tag.number), " -> ",nl]), emit([indent(8),"{",{asis,Cname},", {'", asn1ct_gen:list2name([Cname|TopType]),"',", @@ -928,7 +927,25 @@ gen_dec_choice_cases(Erules,TopType, [H|T]) -> end, gen_dec_choice_cases(Erules,TopType, T). +match_tag(Class, TagNo) when is_integer(TagNo) -> + match_tag1(asn1ct_gen_ber_bin_v2:decode_class(Class), TagNo). +match_tag1(Class, TagNo) when TagNo =< 30 -> + io_lib:format("<<~p:2,_:1,~p:5,_/binary>>", [Class bsr 6,TagNo]); +match_tag1(Class, TagNo) -> + Octets = mk_object_val(TagNo), + io_lib:format("<<~p:2,_:1,31:5,~s,_/binary>>", [Class bsr 6,Octets]). + +mk_object_val(Val) when Val < 16#80 -> + integer_to_list(Val); +mk_object_val(Val) -> + mk_object_val(Val bsr 7, [integer_to_list(Val band 16#7F)]). + +mk_object_val(0, Acc) -> + Acc; +mk_object_val(Val, Acc) -> + I = integer_to_list((Val band 16#7F) bor 16#80), + mk_object_val(Val bsr 7, [I,","|Acc]). %%--------------------------------------- %% Generate the encode/decode code @@ -1493,10 +1510,6 @@ mkfuncname(TopType,Cname,WhatKind,Prefix,Suffix) -> end. empty_lb(ber) -> - "[]"; -empty_lb(ber_bin) -> - "<<>>"; -empty_lb(ber_bin_v2) -> "<<>>". value_match(Index,Value) when is_atom(Value) -> diff --git a/lib/asn1/src/asn1ct_constructed_per.erl b/lib/asn1/src/asn1ct_constructed_per.erl index 8de41a4dd4..b29a7b3048 100644 --- a/lib/asn1/src/asn1ct_constructed_per.erl +++ b/lib/asn1/src/asn1ct_constructed_per.erl @@ -76,13 +76,9 @@ gen_encode_constructed(Erule,Typename,D) when is_record(D,type) -> case {Optionals = optionals(to_textual_order(CompList)),CompList, is_optimized(Erule)} of {[],EmptyCL,_} when EmptyCL == {[],[],[]};EmptyCL == {[],[]};EmptyCL == [] -> - emit(["%%Variable setting just to eliminate ", - "compiler warning for unused vars!",nl, - "_Val = ",{curr,val},",",nl]); + ok; {[],_,_} -> - emit([{next,val}," = ?RT_PER:list_to_record("]), - emit(["'",asn1ct_gen:list2rname(Typename),"'"]), - emit([", ",{curr,val},"),",nl]); + emit([{next,val}," = ",{curr,val},",",nl]); {_,_,true} -> gen_fixoptionals(Optionals), FixOpts = param_map(fun(Var) -> @@ -155,7 +151,7 @@ gen_encode_constructed(Erule,Typename,D) when is_record(D,type) -> emit([ObjectEncode," = ",nl]), emit([" ",ObjSetMod,":'getenc_",ObjSetName,"'(", {asis,UniqueFieldName},", ",nl]), - El = make_element(N+1,asn1ct_gen:mk_var(asn1ct_name:curr(val)),AttrN), + El = make_element(N+1,asn1ct_gen:mk_var(asn1ct_name:curr(val))), Length = fun(X,_LFun) when is_atom(X) -> length(atom_to_list(X)); @@ -221,9 +217,74 @@ gen_decode_set(Erules,Typename,D) -> gen_decode_sequence(Erules,Typename,D) -> gen_decode_constructed(Erules,Typename,D). -gen_decode_constructed(Erules,Typename,D) when is_record(D,type) -> +gen_decode_constructed(Erule, Typename, #type{}=D) -> + Imm0 = gen_dec_constructed_imm(Erule, Typename, #type{}=D), + Imm = opt_imm(Imm0), asn1ct_name:start(), asn1ct_name:clear(), + emit_gen_dec_imm(Imm), + emit([".",nl,nl]). + +opt_imm(Imm0) -> + {Imm,_} = opt_imm_1(Imm0, unknown, []), + Imm. + +opt_imm_1([{imm,Imm0,F}|T], Al0, Acc) -> + {Imm,Al} = asn1ct_imm:optimize_alignment(Imm0, Al0), + opt_imm_1(T, Al, [{imm,Imm,F}|Acc]); +opt_imm_1([ignore|T], Al, Acc) -> + opt_imm_1(T, Al, Acc); +opt_imm_1([{ignore,_}=H|T], Al, Acc) -> + opt_imm_1(T, Al, [H|Acc]); +opt_imm_1([{safe,ignore}|T], Al, Acc) -> + opt_imm_1(T, Al, Acc); +opt_imm_1([{safe,_}=H|T], Al, Acc) -> + opt_imm_1(T, Al, [H|Acc]); +opt_imm_1([{group,G0}|T], Al0, Acc) -> + {G,Al} = opt_imm_1(G0, Al0, []), + opt_imm_1(T, Al, [{group,G}|Acc]); +opt_imm_1([Emit|T], _, Acc) when is_function(Emit, 1) -> + opt_imm_1(T, unknown, [Emit|Acc]); +opt_imm_1([], Al, Acc) -> + {lists:reverse(Acc),Al}. + +emit_gen_dec_imm(L) -> + emit_gen_dec_imm(L, "", []). + +emit_gen_dec_imm([{ignore,Fun}|T], Sep, St0) -> + St = Fun(St0), + emit_gen_dec_imm(T, Sep, St); +emit_gen_dec_imm([{group,L}|T], Sep, St0) -> + emit(Sep), + St = emit_gen_dec_imm_group(L, St0), + emit_gen_dec_imm(T, [com,nl], St); +emit_gen_dec_imm([{imm,Imm,Emit}|T], Sep, St0) -> + emit(Sep), + St = Emit(Imm, St0), + emit_gen_dec_imm(T, [com,nl], St); +emit_gen_dec_imm([{safe,Item}|T], Sep, St) -> + emit_gen_dec_imm([Item|T], Sep, St); +emit_gen_dec_imm([Emit|T], Sep, St0) -> + emit(Sep), + St = Emit(St0), + emit_gen_dec_imm(T, [com,nl], St); +emit_gen_dec_imm([], _, _) -> ok. + +emit_gen_dec_imm_group([H|T], St0) -> + St = emit_gen_dec_group_item(H, St0), + emit_gen_dec_imm_group(T, St); +emit_gen_dec_imm_group([], St) -> St. + +emit_gen_dec_group_item({ignore,Fun}, St) -> + Fun(St); +emit_gen_dec_group_item({imm,Imm,Fun}, St) -> + Fun(Imm, St); +emit_gen_dec_group_item({safe,Item}, St) -> + emit_gen_dec_group_item(Item, St); +emit_gen_dec_group_item(Emit, St) -> + Emit(St). + +gen_dec_constructed_imm(Erule, Typename, #type{}=D) -> {CompList,TableConsInfo} = case D#type.def of #'SEQUENCE'{tablecinf=TCI,components=CL} -> @@ -233,27 +294,19 @@ gen_decode_constructed(Erules,Typename,D) when is_record(D,type) -> {CL,TCI} % the textual order is already taken care of end, Ext = extensible_dec(CompList), - MaybeComma1 = case Ext of - {ext,_Pos,_NumExt} -> - gen_dec_extension_value("Bytes"), - {",",nl}; - _ -> - "" - end, + EmitExt = case Ext of + {ext,_Pos,_NumExt} -> + gen_dec_extension_value(); + _ -> ignore + end, Optionals = optionals(CompList), - MaybeComma2 = case Optionals of - [] -> MaybeComma1; - _ -> - Bcurr = asn1ct_name:curr(bytes), - Bnext = asn1ct_name:next(bytes), - emit(MaybeComma1), - GetoptCall = "} = ?RT_PER:getoptionals2(", - emit({"{Opt,",{var,Bnext},GetoptCall, - {var,Bcurr},",",{asis,length(Optionals)},")"}), - asn1ct_name:new(bytes), - ", " - end, - {DecObjInf,UniqueFName,ValueIndex} = + EmitOpt = case Optionals of + [] -> + ignore; + [_|_] -> + gen_dec_optionals(Optionals) + end, + ObjSetInfo = case TableConsInfo of %% {ObjectSet,AttrN,N,UniqueFieldName} ->%% N is index of attribute that determines constraint #simpletableattributes{objectsetname=ObjectSet, @@ -285,13 +338,19 @@ gen_decode_constructed(Erules,Typename,D) when is_record(D,type) -> {false,false,false} end end, -%% NewCompList = wrap_compList(CompList), - {AccTerm,AccBytes} = - gen_dec_components_call(Erules,Typename,CompList,MaybeComma2,DecObjInf,Ext,length(Optionals)), - case asn1ct_name:all(term) of - [] -> emit(MaybeComma2); % no components at all - _ -> emit({com,nl}) - end, + {DecObjInf,_,_} = ObjSetInfo, + EmitComp = gen_dec_components_call(Erule, Typename, CompList, + DecObjInf, Ext, length(Optionals)), + EmitRest = fun({AccTerm,AccBytes}) -> + gen_dec_constructed_imm_2(Typename, CompList, + ObjSetInfo, + AccTerm, AccBytes) + end, + [EmitExt,EmitOpt|EmitComp++[{safe,EmitRest}]]. + +gen_dec_constructed_imm_2(Typename, CompList, + ObjSetInfo, AccTerm, AccBytes) -> + {_,UniqueFName,ValueIndex} = ObjSetInfo, case {AccTerm,AccBytes} of {[],[]} -> ok; @@ -333,8 +392,7 @@ gen_decode_constructed(Erules,Typename,D) when is_record(D,type) -> mkvlist(textual_order(to_encoding_order(CompList),asn1ct_name:all(term))), emit("},") end, - emit({{curr,bytes},"}"}), - emit({".",nl,nl}). + emit({{curr,bytes},"}"}). textual_order([#'ComponentType'{textual_order=undefined}|_],TermList) -> TermList; @@ -516,10 +574,10 @@ gen_decode_sof(Erules,Typename,SeqOrSetOf,D) when is_record(D,type) -> _ -> "" end, - gen_decode_length(SizeConstraint, - is_optimized(Erules)), - emit({"'dec_",asn1ct_gen:list2name(Typename), - "_components'(Num, Bytes1, telltype",ObjFun,", []).",nl}), + {Num,Buf} = gen_decode_length(SizeConstraint, Erules), + emit([",",nl, + "'dec_",asn1ct_gen:list2name(Typename), + "_components'(",Num,", ",Buf,ObjFun,", []).",nl,nl]), NewComponentType = case ComponentType#type.def of {'ENUMERATED',_,Component}-> @@ -528,40 +586,13 @@ gen_decode_sof(Erules,Typename,SeqOrSetOf,D) when is_record(D,type) -> end, gen_decode_sof_components(Erules,Typename,SeqOrSetOf,NewComponentType). -%% Logic copied from asn1_per_bin_rt2ct:decode_constrained_number -gen_decode_length({Lb,Ub},true) when Ub =< 65535, Lb >= 0 -> - Range = Ub - Lb + 1, - Call = if - Range == 1 -> - "{0,Bytes}"; - Range == 2 -> - "?RT_PER:getbits(Bytes,1)"; - Range =< 4 -> - "?RT_PER:getbits(Bytes,2)"; - Range =< 8 -> - "?RT_PER:getbits(Bytes,3)"; - Range =< 16 -> - "?RT_PER:getbits(Bytes,4)"; - Range =< 32 -> - "?RT_PER:getbits(Bytes,5)"; - Range =< 64 -> - "?RT_PER:getbits(Bytes,6)"; - Range =< 128 -> - "?RT_PER:getbits(Bytes,7)"; - Range =< 255 -> - "?RT_PER:getbits(Bytes,8)"; - Range =< 256 -> - "?RT_PER:getoctets(Bytes,1)"; - Range =< 65536 -> - "?RT_PER:getoctets(Bytes,2)"; - true -> - ["exit({not_supported,{integer_range,",Range,"}}"] - end, - emit({nl,"{Val,Remain} = ",Call,",",nl}), - emit({nl,"{Num,Bytes1} = {Val+",Lb,",Remain},",nl}); -gen_decode_length(SizeConstraint,_) -> - emit({nl,"{Num,Bytes1} = ?RT_PER:decode_length(Bytes,", - {asis,SizeConstraint},"),",nl}). +is_aligned(per) -> true; +is_aligned(uper) -> false. + +gen_decode_length(Constraint, Erule) -> + emit(["%% Length with constraint ",{asis,Constraint},nl]), + Imm = asn1ct_imm:per_dec_length(Constraint, true, is_aligned(Erule)), + asn1ct_imm:dec_slim_cg(Imm, "Bytes"). gen_encode_sof_components(Erule,Typename,SeqOrSetOf,Cont) -> {ObjFun,ObjFun_Var} = @@ -583,8 +614,7 @@ gen_encode_sof_components(Erule,Typename,SeqOrSetOf,Cont) -> Conttype = asn1ct_gen:get_inner(Cont#type.def), Currmod = get(currmod), - Ctgenmod = list_to_atom(lists:concat(["asn1ct_gen_",per, - asn1ct_gen:rt2ct_suffix()])), + Ctgenmod = asn1ct_gen:ct_gen_module(Erule), case asn1ct_gen:type(Conttype) of {primitive,bif} -> gen_encode_prim_wrapper(Ctgenmod,Erule,Cont,false,"H"); @@ -614,16 +644,15 @@ gen_decode_sof_components(Erule,Typename,SeqOrSetOf,Cont) -> {"",""} end, emit({"'dec_",asn1ct_gen:list2name(Typename), - "_components'(0, Bytes, _",ObjFun_Var,", Acc) ->",nl, + "_components'(0, Bytes",ObjFun_Var,", Acc) ->",nl, indent(3),"{lists:reverse(Acc), Bytes};",nl}), emit({"'dec_",asn1ct_gen:list2name(Typename), - "_components'(Num, Bytes, _",ObjFun,", Acc) ->",nl}), + "_components'(Num, Bytes",ObjFun,", Acc) ->",nl}), emit({indent(3),"{Term,Remain} = "}), Constructed_Suffix = asn1ct_gen:constructed_suffix(SeqOrSetOf, Cont#type.def), Conttype = asn1ct_gen:get_inner(Cont#type.def), - Ctgenmod = list_to_atom(lists:concat(["asn1ct_gen_",per, - asn1ct_gen:rt2ct_suffix()])), + Ctgenmod = asn1ct_gen:ct_gen_module(Erule), CurrMod = get(currmod), case asn1ct_gen:type(Conttype) of {primitive,bif} -> @@ -647,7 +676,7 @@ gen_decode_sof_components(Erule,Typename,SeqOrSetOf,Cont) -> emit({"'dec_",Conttype,"'(Bytes,telltype),",nl}) end, emit({indent(3),"'dec_",asn1ct_gen:list2name(Typename), - "_components'(Num-1, Remain, telltype",ObjFun,", [Term|Acc]).",nl}). + "_components'(Num-1, Remain",ObjFun,", [Term|Acc]).",nl}). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -718,10 +747,25 @@ extgrouppos([_|T],ActualPos,VirtualPos,Len,Acc) -> extgrouppos(T,ActualPos,VirtualPos,Len+1,Acc). - -gen_dec_extension_value(_) -> - emit({"{Ext,",{next,bytes},"} = ?RT_PER:getext(",{curr,bytes},")"}), - asn1ct_name:new(bytes). +gen_dec_extension_value() -> + Imm0 = {get_bits,1,[1]}, + E = fun(Imm, _) -> + emit(["{Ext,",{next,bytes},"} = "]), + BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), + asn1ct_imm:dec_code_gen(Imm, BytesVar), + asn1ct_name:new(bytes) + end, + {imm,Imm0,E}. + +gen_dec_optionals(Optionals) -> + Imm0 = {get_bits,length(Optionals),[1]}, + E = fun(Imm, _) -> + BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), + emit(["{Opt,",{next,bytes},"} = "]), + asn1ct_imm:dec_code_gen(Imm, BytesVar), + asn1ct_name:new(bytes) + end, + {imm,Imm0,E}. gen_fixoptionals([{Pos,Def}|R]) -> asn1ct_name:new(fixopt), @@ -889,7 +933,7 @@ gen_enc_components_call1(_Erule,_TopType,[],Pos,_,_,_) -> Pos. gen_enc_component_default(Erule,TopType,Cname,Type,Pos,DynamicEnc,Ext,DefaultVal) -> - Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val)),Cname), + Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val))), emit({"case ",Element," of",nl}), % emit({"asn1_DEFAULT -> [];",nl}), emit({"DFLT when DFLT == asn1_DEFAULT; DFLT == ",{asis,DefaultVal}," -> [];",nl}), @@ -909,7 +953,7 @@ gen_enc_component_optional(Erule,TopType,Cname, components=_ExtGroupCompList}}, Pos,DynamicEnc,Ext) when is_integer(Number) -> - Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val)),Cname), + Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val))), emit({"case ",Element," of",nl}), emit({"asn1_NOVALUE -> [];",nl}), @@ -922,7 +966,7 @@ gen_enc_component_optional(Erule,TopType,Cname, gen_enc_line(Erule,TopType,Cname,Type,NextElement, Pos,DynamicEnc,Ext), emit({nl,"end"}); gen_enc_component_optional(Erule,TopType,Cname,Type,Pos,DynamicEnc,Ext) -> - Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val)),Cname), + Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val))), emit({"case ",Element," of",nl}), emit({"asn1_NOVALUE -> [];",nl}), @@ -942,11 +986,10 @@ gen_enc_component_mandatory(Erule,TopType,Cname,Type,Pos,DynamicEnc,Ext) -> gen_enc_line(Erule,TopType,Cname,Type,[],Pos,DynamicEnc,Ext). gen_enc_line(Erule,TopType, Cname, Type, [], Pos,DynamicEnc,Ext) -> - Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val)),Cname), + Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val))), gen_enc_line(Erule,TopType,Cname,Type,Element, Pos,DynamicEnc,Ext); gen_enc_line(Erule,TopType,Cname,Type,Element, _Pos,DynamicEnc,Ext) -> - Ctgenmod = list_to_atom(lists:concat(["asn1ct_gen_",per, - asn1ct_gen:rt2ct_suffix()])), + Ctgenmod = asn1ct_gen:ct_gen_module(Erule), Atype = case Type of #type{def=#'ObjectClassFieldType'{type=InnerType}} -> @@ -1036,53 +1079,79 @@ gen_enc_line(Erule,TopType,Cname,Type,Element, _Pos,DynamicEnc,Ext) -> emit("))"); _ -> true end. -gen_dec_components_call(Erule,TopType,{Root,ExtList},MaybeComma, + +gen_dec_components_call(Erule, TopType, {Root,ExtList}, + DecInfObj, Ext, NumberOfOptionals) -> + gen_dec_components_call(Erule,TopType,{Root,ExtList,[]}, + DecInfObj,Ext,NumberOfOptionals); +gen_dec_components_call(Erule,TopType,CL={Root1,ExtList,Root2}, DecInfObj,Ext,NumberOfOptionals) -> - gen_dec_components_call(Erule,TopType,{Root,ExtList,[]},MaybeComma,DecInfObj,Ext,NumberOfOptionals); -gen_dec_components_call(Erule,TopType,CL={Root1,ExtList,Root2},MaybeComma,DecInfObj,Ext,NumberOfOptionals) -> %% The type has extensionmarker - OptTable = create_optionality_table(Root1++Root2), - {Rpos,AccTerm,AccBytes} = - gen_dec_components_call1(Erule,TopType, Root1++Root2, 1, OptTable, - MaybeComma,DecInfObj,noext,[],[], - NumberOfOptionals), - emit([",",nl,"{Extensions,",{next,bytes},"} = "]), - emit(["?RT_PER:getextension(Ext,",{curr,bytes},"),",nl]), - asn1ct_name:new(bytes), + Init = {ignore,fun(_) -> {[],[]} end}, + {EmitRoot,Tpos} = + gen_dec_comp_calls(Root1++Root2, Erule, TopType, OptTable, + DecInfObj, noext, NumberOfOptionals, + 1, []), + EmitGetExt = gen_dec_get_extension(Erule), {extgrouppos,ExtGroupPosLen} = extgroup_pos_and_length(CL), - NewExtList = wrap_extensionAdditionGroups(ExtList,ExtGroupPosLen), - {_Epos,AccTermE,AccBytesE} = - gen_dec_components_call1(Erule,TopType,NewExtList,Rpos, OptTable, - "",DecInfObj,Ext,[],[],NumberOfOptionals), - case ExtList of - [] -> true; - _ -> emit([",",nl]) - end, - emit([{next,bytes},"= ?RT_PER:skipextensions(",{curr,bytes},",", - length(ExtList)+1,",Extensions)",nl]), - asn1ct_name:new(bytes), - {AccTerm++AccTermE,AccBytes++AccBytesE}; - -gen_dec_components_call(Erule,TopType,CompList,MaybeComma,DecInfObj, - Ext,NumberOfOptionals) -> + NewExtList = wrap_extensionAdditionGroups(ExtList, ExtGroupPosLen), + {EmitExts,_} = gen_dec_comp_calls(NewExtList, Erule, TopType, OptTable, + DecInfObj, Ext, NumberOfOptionals, + Tpos, []), + Finish = + fun(St) -> + emit([{next,bytes},"= ?RT_PER:skipextensions(",{curr,bytes},",", + length(ExtList)+1,",Extensions)"]), + asn1ct_name:new(bytes), + St + end, + [Init] ++ EmitRoot ++ [EmitGetExt|EmitExts] ++ [Finish]; +gen_dec_components_call(Erule, TopType, CompList, DecInfObj, + Ext, NumberOfOptionals) -> %% The type has no extensionmarker OptTable = create_optionality_table(CompList), - {_,AccTerm,AccBytes} = - gen_dec_components_call1(Erule,TopType, CompList, 1, OptTable, - MaybeComma,DecInfObj,Ext,[],[], - NumberOfOptionals), - {AccTerm,AccBytes}. - - -gen_dec_components_call1(Erule,TopType, - [C=#'ComponentType'{name=Cname,typespec=Type,prop=Prop,textual_order=TextPos}|Rest], - Tpos,OptTable,MaybeComma,DecInfObj,Ext,AccTerm,AccBytes,NumberOfOptionals) -> + Init = {ignore,fun(_) -> {[],[]} end}, + {Cs,_} = gen_dec_comp_calls(CompList, Erule, TopType, OptTable, + DecInfObj, Ext, NumberOfOptionals, + 1, []), + [Init|Cs]. + +gen_dec_get_extension(Erule) -> + Imm0 = asn1ct_imm:per_dec_extension_map(is_aligned(Erule)), + E = fun(Imm, St) -> + emit([nl,"%% Extensions", + nl, + "{Extensions,",{next,bytes},"} = ", + "case Ext of",nl, + "0 -> {<<>>,",{curr,bytes},"};",nl, + "1 ->",nl]), + BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), + {Dst,DstBuf} = asn1ct_imm:dec_slim_cg(Imm, BytesVar), + emit([com,nl, + "{",Dst,",",DstBuf,"}",nl, + "end"]), + asn1ct_name:new(bytes), + St + end, + {imm,Imm0,E}. + +gen_dec_comp_calls([C|Cs], Erule, TopType, OptTable, DecInfObj, + Ext, NumberOfOptionals, Tpos, Acc) -> + L = gen_dec_comp_call(C, Erule, TopType, Tpos, OptTable, DecInfObj, + Ext, NumberOfOptionals), + gen_dec_comp_calls(Cs, Erule, TopType, OptTable, DecInfObj, + Ext, NumberOfOptionals, Tpos+1, [L|Acc]); +gen_dec_comp_calls([], _, _, _, _, _, _, Tpos, Acc) -> + {lists:append(lists:reverse(Acc)),Tpos}. + +gen_dec_comp_call(Comp, Erule, TopType, Tpos, OptTable, DecInfObj, + Ext, NumberOfOptionals) -> + #'ComponentType'{typespec=Type,prop=Prop,textual_order=TextPos} = Comp, Pos = case Ext of noext -> Tpos; {ext,Epos,_Enum} -> Tpos - Epos + 1 end, - emit(MaybeComma), InnerType = case Type#type.def of #'ObjectClassFieldType'{type=InType} -> @@ -1091,109 +1160,128 @@ gen_dec_components_call1(Erule,TopType, asn1ct_gen:get_inner(Def) end, - case InnerType of - #'Externaltypereference'{type=T} -> - emit({nl,"%% attribute number ",TextPos," with type ", - T,nl}); - IT when is_tuple(IT) -> - emit({nl,"%% attribute number ",TextPos," with type ", - element(2,IT),nl}); - _ -> - emit({nl,"%% attribute number ",TextPos," with type ", - InnerType,nl}) - end, - - IsMandatoryAndPredefinedTableC = - fun(noext,mandatory,{"got objfun through args","ObjFun"}) -> - true; - (_,_,{"got objfun through args","ObjFun"}) -> - false; - (_,_,_) -> - true - end, - case {InnerType,IsMandatoryAndPredefinedTableC(Ext,Prop,DecInfObj)} of -%% {typefield,_} when Ext == noext, Prop == mandatory -> - {{typefield,_},true} -> - %% DecInfObj /= {"got objfun through args","ObjFun"} | - %% (DecInfObj == {"got objfun through args","ObjFun"} & - %% Ext == noext & Prop == mandatory) - asn1ct_name:new(term), - asn1ct_name:new(tmpterm), - emit({"{",{curr,tmpterm},", ",{next,bytes},"} = "}); + DispType = case InnerType of + #'Externaltypereference'{type=T} -> T; + IT when is_tuple(IT) -> element(2,IT); + _ -> InnerType + end, + Comment = fun(St) -> + emit([nl,"%% attribute number ",TextPos, + " with type ",DispType,nl]), + St + end, + + Preamble = + case {InnerType,is_mandatory_predef_tab_c(Ext, Prop, DecInfObj)} of + {{typefield,_},true} -> + %% DecInfObj /= {"got objfun through args","ObjFun"} | + %% (DecInfObj == {"got objfun through args","ObjFun"} & + %% Ext == noext & Prop == mandatory) + fun(St) -> + asn1ct_name:new(term), + asn1ct_name:new(tmpterm), + emit(["{",{curr,tmpterm},", ",{next,bytes},"} = "]), + St + end; %%{objectfield,_,_} when Ext == noext, Prop == mandatory -> - {{objectfield,_,_},true} -> - asn1ct_name:new(term), - asn1ct_name:new(tmpterm), - emit({"{",{curr,tmpterm},", ",{next,bytes},"} = "}); + {{objectfield,_,_},true} -> + fun(St) -> + asn1ct_name:new(term), + asn1ct_name:new(tmpterm), + emit(["{",{curr,tmpterm},", ",{next,bytes},"} = "]), + St + end; _ -> case Type of #type{def=#'SEQUENCE'{ extaddgroup=Number1, components=ExtGroupCompList1}} when is_integer(Number1)-> - emit({"{{_,"}), - emit_extaddgroupTerms(term,ExtGroupCompList1), - emit({"}"}); - _ -> - asn1ct_name:new(term), - emit({"{",{curr,term}}) - end, - emit({",",{next,bytes},"} = "}) - end, - - case {Ext,Prop,is_optimized(Erule)} of - {noext,mandatory,_} -> ok; % generate nothing - {noext,_,_} -> %% OPTIONAL or DEFAULT - OptPos = get_optionality_pos(TextPos,OptTable), - Element = io_lib:format("Opt band (1 bsl ~w)",[NumberOfOptionals - OptPos]), - emit(["case ",Element," of",nl]), - emit([" _Opt",TextPos," when _Opt",TextPos," > 0 ->"]); - {_,_,false} -> %% extension element, not bitstring - emit(["case Extensions of",nl]), - emit([" _ when size(Extensions) >= ",Pos,",element(",Pos,",Extensions) == 1 ->",nl]); - _ -> - emit(["case Extensions of",nl]), - emit([" <<_:",Pos-1,",1:1,_/bitstring>> when bit_size(Extensions) >= ",Pos," ->",nl]) - end, - put(component_type,{true,C}), - {TermVar,BytesVar} = gen_dec_line(Erule,TopType,Cname,Type,Tpos,DecInfObj,Ext,Prop), - erase(component_type), - case {Ext,Prop} of - {noext,mandatory} -> true; % generate nothing - {noext,_} -> - emit([";",nl,"0 ->"]), - emit(["{"]), - gen_dec_component_no_val(Ext,Prop), - emit({",",{curr,bytes},"}",nl}), - emit([nl,"end"]); - _ -> - emit([";",nl,"_ ->",nl]), - emit(["{"]), - case Type of - #type{def=#'SEQUENCE'{ - extaddgroup=Number2, - components=ExtGroupCompList2}} when is_integer(Number2)-> - emit({"{extAddGroup,"}), - gen_dec_extaddGroup_no_val(Ext,ExtGroupCompList2), - emit({"}"}); + fun(St) -> + emit(["{{_,"]), + emit_extaddgroupTerms(term,ExtGroupCompList1), + emit(["}"]), + emit([",",{next,bytes},"} = "]), + St + end; _ -> - gen_dec_component_no_val(Ext,Prop) - end, - emit({",",{curr,bytes},"}",nl}), - emit([nl,"end"]) - end, - asn1ct_name:new(bytes), - case Rest of - [] -> - {Tpos+1,AccTerm++TermVar,AccBytes++BytesVar}; - _ -> - emit({com,nl}), - gen_dec_components_call1(Erule,TopType,Rest,Tpos+1,OptTable, - "",DecInfObj,Ext, AccTerm++TermVar, - AccBytes++BytesVar,NumberOfOptionals) - end; + fun(St) -> + asn1ct_name:new(term), + emit(["{",{curr,term}]), + emit([",",{next,bytes},"} = "]), + St + end + end + end, -gen_dec_components_call1(_,_TopType,[],Pos,_OptTable,_,_,_,AccTerm,AccBytes,_NumberOfOptionals) -> - {Pos,AccTerm,AccBytes}. + OptOrDef = + case {Ext,Prop} of + {noext,mandatory} -> + ignore; + {noext,_} -> %% OPTIONAL or DEFAULT + OptPos = get_optionality_pos(TextPos, OptTable), + Element = io_lib:format("Opt band (1 bsl ~w)", + [NumberOfOptionals - OptPos]), + fun(St) -> + emit(["case ",Element," of",nl]), + emit([" _Opt",TextPos," when _Opt",TextPos," > 0 ->"]), + St + end; + {{ext,_,_},_} -> %Extension + fun(St) -> + emit(["case Extensions of",nl, + " <<_:",Pos-1,",1:1,_/bitstring>> ->",nl]), + St + end + end, + Lines = gen_dec_line_imm(Erule, TopType, Comp, Tpos, DecInfObj, Ext), + Postamble = + case {Ext,Prop} of + {noext,mandatory} -> + ignore; + {noext,_} -> + fun(St) -> + emit([";",nl,"0 ->"]), + emit(["{"]), + gen_dec_component_no_val(Ext,Prop), + emit({",",{curr,bytes},"}",nl}), + emit([nl,"end"]), + St + end; + _ -> + fun(St) -> + emit([";",nl,"_ ->",nl]), + emit(["{"]), + case Type of + #type{def=#'SEQUENCE'{ + extaddgroup=Number2, + components=ExtGroupCompList2}} + when is_integer(Number2)-> + emit({"{extAddGroup,"}), + gen_dec_extaddGroup_no_val(Ext,ExtGroupCompList2), + emit({"}"}); + _ -> + gen_dec_component_no_val(Ext, Prop) + end, + emit({",",{curr,bytes},"}",nl}), + emit([nl,"end"]), + St + end + end, + AdvBuffer = {ignore,fun(St) -> + asn1ct_name:new(bytes), + St + end}, + [{group,[{safe,Comment},{safe,Preamble}, + {safe,OptOrDef}|Lines]++ + [{safe,Postamble},{safe,AdvBuffer}]}]. + +is_mandatory_predef_tab_c(noext, mandatory, + {"got objfun through args","ObjFun"}) -> + true; +is_mandatory_predef_tab_c(_, _, {"got objfun through args","ObjFun"}) -> + false; +is_mandatory_predef_tab_c(_,_,_) -> + true. gen_dec_extaddGroup_no_val(Ext,[#'ComponentType'{prop=Prop}])-> gen_dec_component_no_val(Ext,Prop), @@ -1213,9 +1301,14 @@ gen_dec_component_no_val({ext,_,_},mandatory) -> emit({"asn1_NOVALUE"}). -gen_dec_line(Erule,TopType,Cname,Type,Pos,DecInfObj,Ext,Prop) -> - Ctgenmod = list_to_atom(lists:concat(["asn1ct_gen_",per, - asn1ct_gen:rt2ct_suffix()])), +gen_dec_line(Erule, TopType, Comp, Pos, DecInfObj, Ext) -> + Imm0 = gen_dec_line_imm(Erule, TopType, Comp, Pos, DecInfObj, Ext), + Init = {ignore,fun(_) -> {[],[]} end}, + Imm = [{group,[Init|Imm0]}], + emit_gen_dec_imm(Imm). + +gen_dec_line_imm(Erule, TopType, Comp, Pos, DecInfObj, Ext) -> + #'ComponentType'{name=Cname,typespec=Type} = Comp, Atype = case Type of #type{def=#'ObjectClassFieldType'{type=InnerType}} -> @@ -1224,175 +1317,235 @@ gen_dec_line(Erule,TopType,Cname,Type,Pos,DecInfObj,Ext,Prop) -> asn1ct_gen:get_inner(Type#type.def) end, - BytesVar0 = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), - BytesVar = case Ext of - {ext,Ep,_} when Pos >= Ep -> - emit(["begin",nl,"{TmpVal",Pos,",Trem",Pos, - "}=?RT_PER:decode_open_type(", - {curr,bytes},",[]),",nl, - "{TmpValx",Pos,",_}="]), - io_lib:format("TmpVal~p",[Pos]); - _ -> BytesVar0 - end, - SaveBytes = - case Atype of - {typefield,_} -> - case DecInfObj of - false -> % This is in a choice with typefield components - {Name,RestFieldNames} = - (Type#type.def)#'ObjectClassFieldType'.fieldname, - - asn1ct_name:new(tmpterm), - asn1ct_name:new(reason), - emit([indent(2),"{",{curr,tmpterm},", ",{next,bytes}, - "} = ?RT_PER:decode_open_type(",{curr,bytes}, - ", []),",nl]), - emit([indent(2),"case (catch ObjFun(", - {asis,Name},",",{curr,tmpterm},",telltype,", - {asis,RestFieldNames},")) of", nl]), - emit([indent(4),"{'EXIT',",{curr,reason},"} ->",nl]), - emit([indent(6),"exit({'Type not ", - "compatible with table constraint', ", - {curr,reason},"});",nl]), - asn1ct_name:new(tmpterm), - emit([indent(4),"{",{curr,tmpterm},", _} ->",nl]), - emit([indent(6),"{",{asis,Cname},", {",{curr,tmpterm},", ", - {next,bytes},"}}",nl]), - emit([indent(2),"end"]), - []; - {"got objfun through args","ObjFun"} -> - %% this is when the generated code gots the - %% objfun though arguments on function - %% invocation. - if - Ext == noext andalso Prop == mandatory -> - ok; - true -> - asn1ct_name:new(tmpterm), - asn1ct_name:new(tmpbytes), - emit([nl," {",{curr,tmpterm},", ",{curr,tmpbytes},"} ="]) - end, - {Name,RestFieldNames} = - (Type#type.def)#'ObjectClassFieldType'.fieldname, - emit(["?RT_PER:decode_open_type(",{curr,bytes}, - ", []),",nl]), - if - Ext == noext andalso Prop == mandatory -> - emit([{curr,term}," =",nl," "]); - true -> - emit([" {"]) - end, - emit(["case (catch ObjFun(",{asis,Name},",", - {curr,tmpterm},",telltype,", - {asis,RestFieldNames},")) of", nl]), - emit([" {'EXIT',",{curr,reason},"} ->",nl]), - emit([indent(6),"exit({'Type not ", - "compatible with table constraint', ", - {curr,reason},"});",nl]), - asn1ct_name:new(tmpterm), - emit([indent(4),"{",{curr,tmpterm},", _} ->",nl]), - emit([indent(6),{curr,tmpterm},nl]), - emit([indent(2),"end"]), - if - Ext == noext andalso Prop == mandatory -> - ok; - true -> - emit([",",nl,{curr,tmpbytes},"}"]) - end, - []; - _ -> - emit(["?RT_PER:decode_open_type(",{curr,bytes}, - ", [])"]), - RefedFieldName = - (Type#type.def)#'ObjectClassFieldType'.fieldname, - - [{Cname,RefedFieldName, - asn1ct_gen:mk_var(asn1ct_name:curr(term)), - asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), - get_components_prop()}] - end; - {objectfield,PrimFieldName1,PFNList} -> - emit(["?RT_PER:decode_open_type(",{curr,bytes},", [])"]), - [{Cname,{PrimFieldName1,PFNList}, - asn1ct_gen:mk_var(asn1ct_name:curr(term)), - asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), - get_components_prop()}]; - _ -> - CurrMod = get(currmod), - case asn1ct_gen:type(Atype) of - #'Externaltypereference'{module=CurrMod,type=EType} -> - emit({"'dec_",EType,"'(",BytesVar,",telltype)"}); - #'Externaltypereference'{module=Mod,type=EType} -> - emit({"'",Mod,"':'dec_",EType,"'(",BytesVar, - ",telltype)"}); - {primitive,bif} -> - case Atype of - {fixedtypevaluefield,_,Btype} -> - Ctgenmod:gen_dec_prim(Erule,Btype, - BytesVar); - _ -> - Ctgenmod:gen_dec_prim(Erule,Type, - BytesVar) - end; - 'ASN1_OPEN_TYPE' -> - case Type#type.def of - #'ObjectClassFieldType'{type=OpenType} -> - Ctgenmod:gen_dec_prim(Erule,#type{def=OpenType}, - BytesVar); - _ -> - Ctgenmod:gen_dec_prim(Erule,Type, - BytesVar) - end; - #typereference{val=Dname} -> - emit({"'dec_",Dname,"'(",BytesVar,",telltype)"}); - {notype,_} -> - emit({"'dec_",Atype,"'(",BytesVar,",telltype)"}); - {constructed,bif} -> - NewTypename = [Cname|TopType], - case Type#type.tablecinf of - [{objfun,_}|_R] -> - emit({"'dec_",asn1ct_gen:list2name(NewTypename), - "'(",BytesVar,", telltype, ObjFun)"}); - _ -> - emit({"'dec_",asn1ct_gen:list2name(NewTypename), - "'(",BytesVar,", telltype)"}) - end - end, + Pre = gen_dec_line_open_type(Erule, Ext, Pos), + Decode = gen_dec_line_special(Erule, Atype, TopType, Comp, DecInfObj, Ext), + Post = + fun({SaveBytes,Finish}) -> + {AccTerm,AccBytes} = Finish(), + #'ComponentType'{name=Cname} = Comp, case DecInfObj of - {Cname,{_,OSet,UniqueFName,ValIndex}} -> - Term = asn1ct_gen:mk_var(asn1ct_name:curr(term)), - ValueMatch = value_match(ValIndex,Term), - {ObjSetMod,ObjSetName} = - case OSet of - {M,O} -> {{asis,M},O}; - _ -> {"?MODULE",OSet} + {Cname,ObjSet} -> + ObjSetRef = + case ObjSet of + {deep,OSName,_,_} -> + OSName; + _ -> ObjSet end, - emit({",",nl,"ObjFun = ",ObjSetMod, - ":'getdec_",ObjSetName,"'(", - {asis,UniqueFName},", ",ValueMatch,")"}); + {AccTerm++[{ObjSetRef,Cname, + asn1ct_gen:mk_var(asn1ct_name:curr(term))}], + AccBytes++SaveBytes}; _ -> - ok - end, - [] + {AccTerm,AccBytes++SaveBytes} + end end, - case Ext of - {ext,Ep2,_} when Pos >= Ep2 -> - emit([", {TmpValx",Pos,",Trem",Pos,"}",nl,"end"]); - _ -> true - end, - %% Prepare return value + [Pre,Decode,{safe,Post}]. + +gen_dec_line_open_type(Erule, {ext,Ep,_}, Pos) when Pos >= Ep -> + Imm = asn1ct_imm:per_dec_open_type(is_aligned(Erule)), + {safe,fun(St) -> + emit(["begin",nl]), + BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), + {Dst,DstBuf} = asn1ct_imm:dec_slim_cg(Imm, BytesVar), + emit([",",nl,"{TmpValx",Pos,",_} = "]), + {Dst, + fun() -> + emit([",",nl, + "{TmpValx",Pos,",",DstBuf,"}",nl, + "end"]), + St + end} + end}; +gen_dec_line_open_type(_, _, _) -> + {safe,fun(St) -> + {asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), + fun() -> St end} + end}. + +gen_dec_line_special(_, {typefield,_}, _TopType, Comp, + DecInfObj, Ext) -> + #'ComponentType'{name=Cname,typespec=Type,prop=Prop} = Comp, + fun({_BytesVar,PrevSt}) -> + case DecInfObj of + false -> % This is in a choice with typefield components + {Name,RestFieldNames} = + (Type#type.def)#'ObjectClassFieldType'.fieldname, + + asn1ct_name:new(tmpterm), + asn1ct_name:new(reason), + emit([indent(2),"{",{curr,tmpterm},", ",{next,bytes}, + "} = ?RT_PER:decode_open_type(",{curr,bytes}, + ", []),",nl]), + emit([indent(2),"case (catch ObjFun(", + {asis,Name},",",{curr,tmpterm},",telltype,", + {asis,RestFieldNames},")) of", nl]), + emit([indent(4),"{'EXIT',",{curr,reason},"} ->",nl]), + emit([indent(6),"exit({'Type not ", + "compatible with table constraint', ", + {curr,reason},"});",nl]), + asn1ct_name:new(tmpterm), + emit([indent(4),"{",{curr,tmpterm},", _} ->",nl]), + emit([indent(6),"{",{asis,Cname},", {",{curr,tmpterm},", ", + {next,bytes},"}}",nl]), + emit([indent(2),"end"]), + {[],PrevSt}; + {"got objfun through args","ObjFun"} -> + %% this is when the generated code gots the + %% objfun though arguments on function + %% invocation. + if + Ext == noext andalso Prop == mandatory -> + ok; + true -> + asn1ct_name:new(tmpterm), + asn1ct_name:new(tmpbytes), + emit([nl," {",{curr,tmpterm},", ",{curr,tmpbytes},"} ="]) + end, + {Name,RestFieldNames} = + (Type#type.def)#'ObjectClassFieldType'.fieldname, + emit(["?RT_PER:decode_open_type(",{curr,bytes}, + ", []),",nl]), + if + Ext == noext andalso Prop == mandatory -> + emit([{curr,term}," =",nl," "]); + true -> + emit([" {"]) + end, + emit(["case (catch ObjFun(",{asis,Name},",", + {curr,tmpterm},",telltype,", + {asis,RestFieldNames},")) of", nl]), + emit([" {'EXIT',",{curr,reason},"} ->",nl]), + emit([indent(6),"exit({'Type not ", + "compatible with table constraint', ", + {curr,reason},"});",nl]), + asn1ct_name:new(tmpterm), + emit([indent(4),"{",{curr,tmpterm},", _} ->",nl]), + emit([indent(6),{curr,tmpterm},nl]), + emit([indent(2),"end"]), + if + Ext == noext andalso Prop == mandatory -> + ok; + true -> + emit([",",nl,{curr,tmpbytes},"}"]) + end, + {[],PrevSt}; + _ -> + emit(["?RT_PER:decode_open_type(",{curr,bytes}, + ", [])"]), + RefedFieldName = + (Type#type.def)#'ObjectClassFieldType'.fieldname, + + {[{Cname,RefedFieldName, + asn1ct_gen:mk_var(asn1ct_name:curr(term)), + asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), + Prop}],PrevSt} + end + end; +gen_dec_line_special(_, {objectfield,PrimFieldName1,PFNList}, _TopType, + Comp, _DecInfObj, _Ext) -> + fun({_BytesVar,PrevSt}) -> + emit(["?RT_PER:decode_open_type(",{curr,bytes},", [])"]), + #'ComponentType'{name=Cname,prop=Prop} = Comp, + SaveBytes = [{Cname,{PrimFieldName1,PFNList}, + asn1ct_gen:mk_var(asn1ct_name:curr(term)), + asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), + Prop}], + {SaveBytes,PrevSt} + end; +gen_dec_line_special(Erule, Atype, TopType, Comp, DecInfObj, _Ext) -> + case gen_dec_line_other(Erule, Atype, TopType, Comp) of + Fun when is_function(Fun, 1) -> + fun({BytesVar,PrevSt}) -> + Fun(BytesVar), + gen_dec_line_dec_inf(Comp, DecInfObj), + {[],PrevSt} + end; + Imm0 -> + {imm,Imm0, + fun(Imm, {BytesVar,PrevSt}) -> + asn1ct_imm:dec_code_gen(Imm, BytesVar), + gen_dec_line_dec_inf(Comp, DecInfObj), + {[],PrevSt} + end} + end. + +gen_dec_line_dec_inf(Comp, DecInfObj) -> + #'ComponentType'{name=Cname} = Comp, case DecInfObj of - {Cname,ObjSet} -> - ObjSetRef = - case ObjSet of - {deep,OSName,_,_} -> - OSName; - _ -> ObjSet + {Cname,{_,OSet,UniqueFName,ValIndex}} -> + Term = asn1ct_gen:mk_var(asn1ct_name:curr(term)), + ValueMatch = value_match(ValIndex,Term), + {ObjSetMod,ObjSetName} = + case OSet of + {M,O} -> {{asis,M},O}; + _ -> {"?MODULE",OSet} end, - {[{ObjSetRef,Cname,asn1ct_gen:mk_var(asn1ct_name:curr(term))}], - SaveBytes}; + emit({",",nl,"ObjFun = ",ObjSetMod, + ":'getdec_",ObjSetName,"'(", + {asis,UniqueFName},", ",ValueMatch,")"}); _ -> - {[],SaveBytes} + ok + end. + +gen_dec_line_other(Erule, Atype, TopType, Comp) -> + #'ComponentType'{name=Cname,typespec=Type} = Comp, + CurrMod = get(currmod), + Ctgenmod = asn1ct_gen:ct_gen_module(Erule), + case asn1ct_gen:type(Atype) of + #'Externaltypereference'{module=CurrMod,type=EType} -> + fun(BytesVar) -> + emit({"'dec_",EType,"'(",BytesVar,",telltype)"}) + end; + #'Externaltypereference'{module=Mod,type=EType} -> + fun(BytesVar) -> + emit({"'",Mod,"':'dec_",EType,"'(",BytesVar, + ",telltype)"}) + end; + {primitive,bif} -> + case Atype of + {fixedtypevaluefield,_,Btype} -> + gen_dec_prim(Ctgenmod, Erule, Btype); + _ -> + gen_dec_prim(Ctgenmod, Erule, Type) + end; + 'ASN1_OPEN_TYPE' -> + case Type#type.def of + #'ObjectClassFieldType'{type=OpenType} -> + gen_dec_prim(Ctgenmod, Erule, #type{def=OpenType}); + _ -> + gen_dec_prim(Ctgenmod, Erule, Type) + end; + #typereference{val=Dname} -> + fun(BytesVar) -> + emit({"'dec_",Dname,"'(",BytesVar,",telltype)"}) + end; + {notype,_} -> + fun(BytesVar) -> + emit({"'dec_",Atype,"'(",BytesVar,",telltype)"}) + end; + {constructed,bif} -> + NewTypename = [Cname|TopType], + case Type#type.tablecinf of + [{objfun,_}|_R] -> + fun(BytesVar) -> + emit({"'dec_",asn1ct_gen:list2name(NewTypename), + "'(",BytesVar,", telltype, ObjFun)"}) + end; + _ -> + fun(BytesVar) -> + emit({"'dec_",asn1ct_gen:list2name(NewTypename), + "'(",BytesVar,", telltype)"}) + end + end + end. + +gen_dec_prim(Ctgenmod, Erule, Type) -> + case asn1ct_gen_per:gen_dec_imm(Erule, Type) of + no -> + fun(BytesVar) -> + Ctgenmod:gen_dec_prim(Erule, Type, BytesVar) + end; + Imm -> + Imm end. gen_enc_choice(Erule,TopType,CompList,Ext) -> @@ -1578,22 +1731,17 @@ gen_encode_prim_wrapper(CtgenMod,Erule,Cont,DoTag,Value) -> make_elements(I,Val,ExtCnames) -> make_elements(I,Val,ExtCnames,[]). -make_elements(I,Val,[ExtCname],Acc)-> % the last one, no comma needed - Element = make_element(I,Val,ExtCname), +make_elements(I,Val,[_ExtCname],Acc)-> % the last one, no comma needed + Element = make_element(I, Val), make_elements(I+1,Val,[],[Element|Acc]); -make_elements(I,Val,[ExtCname|Rest],Acc)-> - Element = make_element(I,Val,ExtCname), +make_elements(I,Val,[_ExtCname|Rest],Acc)-> + Element = make_element(I, Val), make_elements(I+1,Val,Rest,[", ",Element|Acc]); make_elements(_I,_,[],Acc) -> lists:reverse(Acc). -make_element(I,Val,Cname) -> - case tuple_notation_allowed() of - true -> - io_lib:format("?RT_PER:cindex(~w,~s,~w)",[I,Val,Cname]); - _ -> - io_lib:format("element(~w,~s)",[I,Val]) - end. +make_element(I, Val) -> + io_lib:format("element(~w,~s)", [I,Val]). emit_extaddgroupTerms(VarSeries,[_]) -> asn1ct_name:new(VarSeries), @@ -1651,22 +1799,12 @@ wrap_extensionAdditionGroups([],_,Acc,_,_) -> lists:reverse(Acc). -tuple_notation_allowed() -> - Options = get(encoding_options), - not (lists:member(optimize,Options) orelse lists:member(uper_bin,Options)). - -wrap_gen_dec_line(Erule,C,TopType,Cname,Type,Pos,DIO,Ext) -> +wrap_gen_dec_line(Erule,C,TopType,_Cname,_Type,Pos,DIO,Ext) -> put(component_type,{true,C}), - gen_dec_line(Erule,TopType,Cname,Type,Pos,DIO,Ext,mandatory), + gen_dec_line(Erule, TopType, C#'ComponentType'{prop=mandatory}, + Pos, DIO, Ext), erase(component_type). -get_components_prop() -> - case get(component_type) of - undefined -> - mandatory; - {true,#'ComponentType'{prop=Prop}} -> Prop - end. - value_match(Index,Value) when is_atom(Value) -> value_match(Index,atom_to_list(Value)); @@ -1683,7 +1821,5 @@ notice_value_match() -> Module = get(currmod), put(value_match,{true,Module}). -is_optimized(per_bin) -> - lists:member(optimize,get(encoding_options)); -is_optimized(_Erule) -> - false. +is_optimized(per) -> true; +is_optimized(uper) -> false. diff --git a/lib/asn1/src/asn1ct_gen.erl b/lib/asn1/src/asn1ct_gen.erl index 64a3555f62..57b12ce186 100644 --- a/lib/asn1/src/asn1ct_gen.erl +++ b/lib/asn1/src/asn1ct_gen.erl @@ -37,8 +37,7 @@ gen_check_call/7, get_constraint/2, insert_once/2, - rt2ct_suffix/1, - rt2ct_suffix/0, + ct_gen_module/1, index2suffix/1, get_record_name_prefix/0]). -export([pgen/5, @@ -52,7 +51,7 @@ %% pgen(Outfile, Erules, Module, TypeOrVal, Options) %% Generate Erlang module (.erl) and (.hrl) file corresponding to an ASN.1 module %% .hrl file is only generated if necessary -%% Erules = per | ber | ber_bin | per_bin +%% Erules = per | ber %% Module = atom() %% TypeOrVal = {TypeList,ValueList} %% TypeList = ValueList = [atom()] @@ -83,7 +82,7 @@ pgen_module(OutFile,Erules,Module, pgen_exports(Erules,Module,TypeOrVal), pgen_dispatcher(Erules,Module,TypeOrVal), pgen_info(), - pgen_typeorval(wrap_ber(Erules),Module,N2nConvEnums,TypeOrVal), + pgen_typeorval(Erules,Module,N2nConvEnums,TypeOrVal), pgen_partial_incomplete_decode(Erules), % gen_vars(asn1_db:mod_to_vars(Module)), % gen_tag_table(AllTypes), @@ -92,8 +91,7 @@ pgen_module(OutFile,Erules,Module, pgen_typeorval(Erules,Module,N2nConvEnums,{Types,Values,_Ptypes,_Classes,Objects,ObjectSets}) -> - Rtmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Rtmod = ct_gen_module(Erules), pgen_types(Rtmod,Erules,N2nConvEnums,Module,Types), pgen_values(Erules,Module,Values), pgen_objects(Rtmod,Erules,Module,Objects), @@ -196,7 +194,7 @@ pgen_check_defaultval(Erules,Module) -> end, gen_check_defaultval(Erules,Module,CheckObjects). -pgen_partial_decode(Rtmod,Erule,Module) when Erule == ber_bin_v2 -> +pgen_partial_decode(Rtmod,Erule,Module) when Erule == ber -> pgen_partial_inc_dec(Rtmod,Erule,Module), pgen_partial_dec(Rtmod,Erule,Module); pgen_partial_decode(_,_,_) -> @@ -240,7 +238,7 @@ pgen_partial_inc_dec1(Rtmod,Erules,Module,[P|Ps]) -> pgen_partial_inc_dec1(_,_,_,[]) -> ok. -gen_partial_inc_dec_refed_funcs(Rtmod,Erule) when Erule == ber_bin_v2 -> +gen_partial_inc_dec_refed_funcs(Rtmod,Erule) when Erule == ber -> case asn1ct:next_refed_func() of [] -> ok; @@ -296,8 +294,7 @@ pgen_partial_types1(_,undefined) -> %% TypeList a decode function will be generated. traverse_type_structure(Erules,Type,[],FuncName,TopTypeName) -> %% this is the selected type - Ctmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Ctmod = ct_gen_module(Erules), TypeDef = case Type of #type{} -> @@ -457,7 +454,7 @@ pgen_partial_incomplete_decode(Erule) -> _ -> ok end. -pgen_partial_incomplete_decode1(ber_bin_v2) -> +pgen_partial_incomplete_decode1(ber) -> case asn1ct:read_config_data(partial_incomplete_decode) of undefined -> ok; @@ -552,20 +549,17 @@ gen_part_decode_funcs(WhatKind,_TypeName,{_,Directive,_,_}) -> gen_types(Erules,Tname,{RootL1,ExtList,RootL2}) when is_list(RootL1), is_list(RootL2) -> gen_types(Erules,Tname,RootL1), - Rtmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Rtmod = ct_gen_module(Erules), gen_types(Erules,Tname,Rtmod:extaddgroup2sequence(ExtList)), gen_types(Erules,Tname,RootL2); gen_types(Erules,Tname,{RootList,ExtList}) when is_list(RootList) -> gen_types(Erules,Tname,RootList), - Rtmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Rtmod = ct_gen_module(Erules), gen_types(Erules,Tname,Rtmod:extaddgroup2sequence(ExtList)); gen_types(Erules,Tname,[{'EXTENSIONMARK',_,_}|Rest]) -> gen_types(Erules,Tname,Rest); gen_types(Erules,Tname,[ComponentType|Rest]) -> - Rtmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Rtmod = ct_gen_module(Erules), asn1ct_name:clear(), Rtmod:gen_encode(Erules,Tname,ComponentType), asn1ct_name:clear(), @@ -574,8 +568,7 @@ gen_types(Erules,Tname,[ComponentType|Rest]) -> gen_types(_,_,[]) -> true; gen_types(Erules,Tname,Type) when is_record(Type,type) -> - Rtmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Rtmod = ct_gen_module(Erules), asn1ct_name:clear(), Rtmod:gen_encode(Erules,Tname,Type), asn1ct_name:clear(), @@ -754,8 +747,7 @@ gen_value(Value) when is_record(Value,valuedef) -> emit([{asis,V},".",nl,nl]). gen_encode_constructed(Erules,Typename,InnerType,D) when is_record(D,type) -> - - Rtmod = list_to_atom(lists:concat(["asn1ct_constructed_",erule(Erules)])), + Rtmod = ct_constructed_module(Erules), case InnerType of 'SET' -> Rtmod:gen_encode_set(Erules,Typename,D), @@ -787,7 +779,7 @@ gen_encode_constructed(Erules,Typename,InnerType,D) gen_encode_constructed(Erules,Typename,InnerType,D#typedef.typespec). gen_decode_constructed(Erules,Typename,InnerType,D) when is_record(D,type) -> - Rtmod = list_to_atom(lists:concat(["asn1ct_constructed_",erule(Erules)])), + Rtmod = ct_constructed_module(Erules), asn1ct:step_in_constructed(), %% updates namelist for exclusive decode case InnerType of 'SET' -> @@ -818,27 +810,11 @@ pgen_exports(Erules,_Module,{Types,Values,_,_,Objects,ObjectSets}) -> case Erules of ber -> gen_exports1(Types,"enc_",2); - ber_bin -> - gen_exports1(Types,"enc_",2); - ber_bin_v2 -> - gen_exports1(Types,"enc_",2); _ -> gen_exports1(Types,"enc_",1) end, emit({"-export([",nl}), - gen_exports1(Types,"dec_",2), - case Erules of - ber -> - emit({"-export([",nl}), - gen_exports1(Types,"dec_",3); - ber_bin -> - emit({"-export([",nl}), - gen_exports1(Types,"dec_",3); -% ber_bin_v2 -> -% emit({"-export([",nl}), -% gen_exports1(Types,"dec_",2); - _ -> ok - end + gen_exports1(Types,"dec_",2) end, case [X || {n2n,X} <- get(encoding_options)] of [] -> ok; @@ -863,16 +839,11 @@ pgen_exports(Erules,_Module,{Types,Values,_,_,Objects,ObjectSets}) -> gen_exports1(Objects,"enc_",3), emit({"-export([",nl}), gen_exports1(Objects,"dec_",4); - ber_bin_v2 -> + ber -> emit({"-export([",nl}), gen_exports1(Objects,"enc_",3), emit({"-export([",nl}), - gen_exports1(Objects,"dec_",3); - _ -> - emit({"-export([",nl}), - gen_exports1(Objects,"enc_",4), - emit({"-export([",nl}), - gen_exports1(Objects,"dec_",4) + gen_exports1(Objects,"dec_",3) end end, case ObjectSets of @@ -948,20 +919,17 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> emit(["encoding_rule() ->",nl]), emit([" ",{asis,Erules},".",nl,nl]), NoFinalPadding = lists:member(no_final_padding,get(encoding_options)), - Call = case Erules of - per -> "?RT_PER:complete(encode_disp(Type,Data))"; - per_bin -> ["?RT_PER:complete(encode_disp(Type,Data))"]; - ber -> "encode_disp(Type,Data)"; - ber_bin -> "encode_disp(Type,Data)"; - ber_bin_v2 -> "encode_disp(Type,Data)"; - uper_bin when NoFinalPadding == true -> - "?RT_PER:complete_NFP(encode_disp(Type,Data))"; - uper_bin -> ["?RT_PER:complete(encode_disp(Type,Data))"] - end, - EncWrap = case Erules of - ber -> "wrap_encode(Bytes)"; - _ -> "Bytes" - end, + {Call,BytesAsBinary} = + case Erules of + per -> + {["?RT_PER:complete(encode_disp(Type,Data))"],"Bytes"}; + ber -> + {"encode_disp(Type,Data)","iolist_to_binary(Bytes)"}; + uper when NoFinalPadding == true -> + {"?RT_PER:complete_NFP(encode_disp(Type,Data))","Bytes"}; + uper -> + {["?RT_PER:complete(encode_disp(Type,Data))"],"Bytes"} + end, emit(["encode(Type,Data) ->",nl, "case catch ",Call," of",nl, " {'EXIT',{error,Reason}} ->",nl, @@ -969,42 +937,25 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> " {'EXIT',Reason} ->",nl, " {error,{asn1,Reason}};",nl, " {Bytes,_Len} ->",nl, - " {ok,",EncWrap,"};",nl]), - case Erules of - per -> - emit([" Bytes when is_binary(Bytes) ->",nl, - " {ok,binary_to_list(Bytes)};",nl, - " Bytes ->",nl, - " {ok,binary_to_list(list_to_binary(Bytes))}",nl, - " end.",nl,nl]); - _ -> - emit([" Bytes ->",nl, - " {ok,",EncWrap,"}",nl, - "end.",nl,nl]) - end, - -% case Erules of -% ber_bin_v2 -> -% emit(["decode(Type,Data0) ->",nl]), -% emit(["{Data,_RestBin} = ?RT_BER:decode(Data0",nif_parameter(),"),",nl]); -% _ -> -% emit(["decode(Type,Data) ->",nl]) -% end, + " {ok,",BytesAsBinary,"};",nl, + " Bytes ->",nl, + " {ok,",BytesAsBinary,"}",nl, + "end.",nl,nl]), Return_rest = lists:member(undec_rest,get(encoding_options)), Data = case {Erules,Return_rest} of - {ber_bin_v2,true} -> "Data0"; + {ber,true} -> "Data0"; _ -> "Data" end, emit(["decode(Type,",Data,") ->",nl]), DecAnonymous = case {Erules,Return_rest} of - {ber_bin_v2,false} -> + {ber,false} -> io_lib:format("~s~s~s~n", ["element(1,?RT_BER:decode(Data", nif_parameter(),"))"]); - {ber_bin_v2,true} -> + {ber,true} -> emit(["{Data,Rest} = ?RT_BER:decode(Data0", nif_parameter(),"),",nl]), "Data"; @@ -1012,10 +963,8 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> "Data" end, DecWrap = case Erules of - ber -> "wrap_decode(Data)"; - ber_bin_v2 -> + ber -> DecAnonymous; - per -> "list_to_binary(Data)"; _ -> "Data" end, @@ -1025,32 +974,18 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> " {'EXIT',Reason} ->",nl, " {error,{asn1,Reason}};",nl]), case {Erules,Return_rest} of - {ber_bin_v2,false} -> + {ber,false} -> emit([" Result ->",nl, " {ok,Result}",nl]); - {ber_bin_v2,true} -> + {ber,true} -> emit([" Result ->",nl, " {ok,Result,Rest}",nl]); - {per,false} -> - emit([" {X,_Rest} ->",nl, - " {ok,if_binary2list(X)};",nl, - " {X,_Rest,_Len} ->",nl, - " {ok,if_binary2list(X)}",nl]); {_,false} -> emit([" {X,_Rest} ->",nl, " {ok,X};",nl, " {X,_Rest,_Len} ->",nl, " {ok,X}",nl]); - {per,true} -> - emit([" {X,{_,Rest}} ->",nl, - " {ok,if_binary2list(X),Rest};",nl, - " {X,{_,Rest},_Len} ->",nl, - " {ok,if_binary2list(X),Rest};",nl, - " {X,Rest} ->",nl, - " {ok,if_binary2list(X),Rest};",nl, - " {X,Rest,_Len} ->",nl, - " {ok,if_binary2list(X),Rest}",nl]); - {per_bin,true} -> + {per,true} -> emit([" {X,{_,Rest}} ->",nl, " {ok,X,Rest};",nl, " {X,{_,Rest},_Len} ->",nl, @@ -1059,7 +994,7 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> " {ok,X,Rest};",nl, " {X,Rest,_Len} ->",nl, " {ok,X,Rest}",nl]); - {uper_bin,true} -> + {uper,true} -> emit([" {X,{_,Rest}} ->",nl, " {ok,X,Rest};",nl, " {X,{_,Rest},_Len} ->",nl, @@ -1067,34 +1002,14 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> " {X,Rest} ->",nl, " {ok,X,Rest};",nl, " {X,Rest,_Len} ->",nl, - " {ok,X,Rest}",nl]); - _ -> - emit([" {X,Rest} ->",nl, - " {ok,X,Rest};",nl, - " {X,Rest,_Len} ->",nl, " {ok,X,Rest}",nl]) end, emit(["end.",nl,nl]), - case Erules of - per -> - emit(["if_binary2list(B) when is_binary(B) ->",nl, - " binary_to_list(B);",nl, - "if_binary2list(L) -> L.",nl,nl]); - _ -> - ok - end, - gen_decode_partial_incomplete(Erules), case Erules of ber -> - gen_dispatcher(Types,"encode_disp","enc_",",[]"), - gen_dispatcher(Types,"decode_disp","dec_",",mandatory"); - ber_bin -> - gen_dispatcher(Types,"encode_disp","enc_",",[]"), - gen_dispatcher(Types,"decode_disp","dec_",",mandatory"); - ber_bin_v2 -> gen_dispatcher(Types,"encode_disp","enc_",""), gen_dispatcher(Types,"decode_disp","dec_",""), gen_partial_inc_dispatcher(); @@ -1103,17 +1018,10 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> gen_dispatcher(Types,"decode_disp","dec_",",mandatory") end, emit([nl]), - - case Erules of - ber -> - gen_wrapper(); - _ -> ok - end, emit({nl,nl}). -gen_decode_partial_incomplete(Erule) when Erule == ber;Erule==ber_bin; - Erule==ber_bin_v2 -> +gen_decode_partial_incomplete(ber) -> case {asn1ct:read_config_data(partial_incomplete_decode), asn1ct:get_gen_state_field(inc_type_pattern)} of {undefined,_} -> @@ -1121,34 +1029,35 @@ gen_decode_partial_incomplete(Erule) when Erule == ber;Erule==ber_bin; {_,undefined} -> ok; _ -> - case Erule of - ber_bin_v2 -> - EmitCaseClauses = - fun() -> - emit([" {'EXIT',{error,Reason}} ->",nl, - " {error,Reason};",nl, - " {'EXIT',Reason} ->",nl, - " {error,{asn1,Reason}};",nl, - " Result ->",nl, - " {ok,Result}",nl, - " end.",nl,nl]) - end, - emit(["decode_partial_incomplete(Type,Data0,", - "Pattern) ->",nl]), - emit([" {Data,_RestBin} =",nl, - " ?RT_BER:decode_primitive_", - "incomplete(Pattern,Data0),",nl, - " case catch decode_partial_inc_disp(Type,", - "Data) of",nl]), - EmitCaseClauses(), - emit(["decode_part(Type,Data0) ->",nl]), - emit([" case catch decode_inc_disp(Type,element(1," - "?RT_BER:decode(Data0",nif_parameter(),"))) of",nl]), -% " {Data,_RestBin} = ?RT_BER:decode(Data0),",nl, -% " case catch decode_inc_disp(Type,Data) of",nl]), - EmitCaseClauses(); - _ -> ok % add later - end + EmitCaseClauses = + fun() -> + emit([" {'EXIT',{error,Reason}} ->",nl, + " {error,Reason};",nl, + " {'EXIT',Reason} ->",nl, + " {error,{asn1,Reason}};",nl, + " Result ->",nl, + " {ok,Result}",nl, + " end"]) + end, + emit(["decode_partial_incomplete(Type,Data0,", + "Pattern) ->",nl]), + emit([" {Data,_RestBin} =",nl, + " ?RT_BER:decode_primitive_", + "incomplete(Pattern,Data0),",nl, + " case catch decode_partial_inc_disp(Type,", + "Data) of",nl]), + EmitCaseClauses(), + emit([".",nl,nl]), + emit(["decode_part(Type, Data0) " + "when is_binary(Data0) ->",nl]), + emit([" case catch decode_inc_disp(Type,element(1," + "?RT_BER:decode(Data0",nif_parameter(),"))) of",nl]), + EmitCaseClauses(), + emit([";",nl]), + emit(["decode_part(Type, Data0) ->",nl]), + emit([" case catch decode_inc_disp(Type, Data0) of",nl]), + EmitCaseClauses(), + emit([".",nl,nl]) end; gen_decode_partial_incomplete(_Erule) -> ok. @@ -1187,23 +1096,8 @@ gen_partial_inc_dispatcher([],_) -> " exit({error,{asn1,{undefined_type,Type}}}).",nl]). nif_parameter() -> - Options = get(encoding_options), - case {lists:member(driver,Options),lists:member(nif,Options)} of - {true,_} -> ",nif"; - {_,true} -> ",nif"; - _ -> "" - end. + ",nif". -gen_wrapper() -> - emit(["wrap_encode(Bytes) when is_list(Bytes) ->",nl, - " binary_to_list(list_to_binary(Bytes));",nl, - "wrap_encode(Bytes) when is_binary(Bytes) ->",nl, - " binary_to_list(Bytes);",nl, - "wrap_encode(Bytes) -> Bytes.",nl,nl]), - emit(["wrap_decode(Bytes) when is_list(Bytes) ->",nl, - " list_to_binary(Bytes);",nl, - "wrap_decode(Bytes) -> Bytes.",nl]). - gen_dispatcher([F1,F2|T],FuncName,Prefix,ExtraArg) -> emit([FuncName,"('",F1,"',Data) -> '",Prefix,F1,"'(Data",ExtraArg,")",";",nl]), gen_dispatcher([F2|T],FuncName,Prefix,ExtraArg); @@ -1213,19 +1107,16 @@ gen_dispatcher([Flast|_T],FuncName,Prefix,ExtraArg) -> pgen_info() -> emit(["info() ->",nl, - " case ?MODULE:module_info() of",nl, - " MI when is_list(MI) ->",nl, - " case lists:keysearch(attributes,1,MI) of",nl, - " {value,{_,Attributes}} when is_list(Attributes) ->",nl, - " case lists:keysearch(asn1_info,1,Attributes) of",nl, - " {value,{_,Info}} when is_list(Info) ->",nl, - " Info;",nl, - " _ ->",nl, - " []",nl, - " end;",nl, - " _ ->",nl, - " []",nl, - " end",nl, + " case ?MODULE:module_info(attributes) of",nl, + " Attributes when is_list(Attributes) ->",nl, + " case lists:keyfind(asn1_info, 1, Attributes) of",nl, + " {_,Info} when is_list(Info) ->",nl, + " Info;",nl, + " _ ->",nl, + " []",nl, + " end;",nl, + " _ ->",nl, + " []",nl, " end.",nl]). open_hrl(OutFile,Module) -> @@ -1496,32 +1387,15 @@ gen_head(Erules,Mod,Hrl) -> {Rtmac,Rtmod} = case Erules of per -> emit({"%% Generated by the Erlang ASN.1 PER-" - "compiler version:",asn1ct:vsn(),nl}), - {"RT_PER",?RT_PER_BIN}; - ber -> - emit({"%% Generated by the Erlang ASN.1 BER-" - "compiler version:",asn1ct:vsn(),nl}), - {"RT_BER",?RT_BER_BIN}; - per_bin -> - emit({"%% Generated by the Erlang ASN.1 BER-" - "compiler version, utilizing bit-syntax:", - asn1ct:vsn(),nl}), - %% temporary code to enable rt2ct optimization - case lists:member(optimize,Options) of - true -> {"RT_PER","asn1rt_per_bin_rt2ct"}; - _ -> {"RT_PER",?RT_PER_BIN} - end; - ber_bin -> - emit({"%% Generated by the Erlang ASN.1 BER-" "compiler version, utilizing bit-syntax:", asn1ct:vsn(),nl}), - {"RT_BER",?RT_BER_BIN}; - ber_bin_v2 -> + {"RT_PER","asn1rt_per_bin_rt2ct"}; + ber -> emit({"%% Generated by the Erlang ASN.1 BER_V2-" "compiler version, utilizing bit-syntax:", asn1ct:vsn(),nl}), {"RT_BER","asn1rt_ber_bin_v2"}; - uper_bin -> + uper -> emit(["%% Generated by the Erlang ASN.1 UNALIGNED" " PER-compiler version, utilizing" " bit-syntax:", @@ -1531,7 +1405,7 @@ gen_head(Erules,Mod,Hrl) -> emit({"%% Purpose: encoder and decoder to the types in mod ",Mod,nl,nl}), emit({"-module('",Mod,"').",nl}), put(currmod,Mod), - %emit({"-compile(export_all).",nl}), + emit({"-compile(nowarn_unused_vars).",nl}), case {Hrl,lists:member(inline,get(encoding_options))} of {0,_} -> true; {_,true} -> true; @@ -1541,7 +1415,7 @@ gen_head(Erules,Mod,Hrl) -> emit(["-define('",Rtmac,"',",Rtmod,").",nl]), emit(["-asn1_info([{vsn,'",asn1ct:vsn(),"'},",nl, " {module,'",Mod,"'},",nl, - " {options,",io_lib:format("~w",[Options]),"}]).",nl,nl]). + " {options,",io_lib:format("~p",[Options]),"}]).",nl,nl]). gen_hrlhead(Mod) -> @@ -2019,43 +1893,29 @@ constructed_suffix('SEQUENCE OF',_) -> constructed_suffix('SET OF',_) -> 'SETOF'. -erule(ber) -> - ber; -erule(ber_bin) -> - ber; -erule(ber_bin_v2) -> - ber_bin_v2; -erule(per) -> - per; -erule(per_bin) -> - per; -erule(uper_bin) -> - per. - -wrap_ber(ber) -> - ber_bin; -wrap_ber(Erule) -> - Erule. - -rt2ct_suffix() -> - Options = get(encoding_options), - case {lists:member(optimize,Options),lists:member(per_bin,Options)} of - {true,true} -> "_rt2ct"; - _ -> "" - end. -rt2ct_suffix(per_bin) -> - Options = get(encoding_options), - case lists:member(optimize,Options) of - true -> "_rt2ct"; - _ -> "" - end; -rt2ct_suffix(_) -> "". +erule(ber) -> ber; +erule(per) -> per; +erule(uper) -> per. index2suffix(0) -> ""; index2suffix(N) -> lists:concat(["_",N]). +ct_gen_module(ber) -> + asn1ct_gen_ber_bin_v2; +ct_gen_module(per) -> + asn1ct_gen_per_rt2ct; +ct_gen_module(uper) -> + asn1ct_gen_per. + +ct_constructed_module(ber) -> + asn1ct_constructed_ber_bin_v2; +ct_constructed_module(per) -> + asn1ct_constructed_per; +ct_constructed_module(uper) -> + asn1ct_constructed_per. + get_constraint(C,Key) -> case lists:keysearch(Key,1,C) of false -> diff --git a/lib/asn1/src/asn1ct_gen_ber.erl b/lib/asn1/src/asn1ct_gen_ber.erl deleted file mode 100644 index 491ebcb8fd..0000000000 --- a/lib/asn1/src/asn1ct_gen_ber.erl +++ /dev/null @@ -1,1749 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1997-2010. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% -%% --module(asn1ct_gen_ber). - -%% Generate erlang module which handles (PER) encode and decode for -%% all types in an ASN.1 module - --include("asn1_records.hrl"). - --export([pgen/4]). --export([decode_class/1, decode_type/1]). --export([add_removed_bytes/0]). --export([gen_encode/2,gen_encode/3,gen_decode/2,gen_decode/3]). --export([gen_encode_prim/4]). --export([gen_dec_prim/8]). --export([gen_objectset_code/2, gen_obj_code/3]). --export([re_wrap_erule/1]). --export([unused_var/2]). --export([extaddgroup2sequence/1]). - --import(asn1ct_gen, [emit/1,demit/1]). - - % the encoding of class of tag bits 8 and 7 --define(UNIVERSAL, 0). --define(APPLICATION, 16#40). --define(CONTEXT, 16#80). --define(PRIVATE, 16#C0). - - % primitive or constructed encoding % bit 6 --define(PRIMITIVE, 0). --define(CONSTRUCTED, 2#00100000). - - --define(T_ObjectDescriptor, ?UNIVERSAL bor ?PRIMITIVE bor 7). - % restricted character string types --define(T_NumericString, ?UNIVERSAL bor ?PRIMITIVE bor 18). %can be constructed --define(T_PrintableString, ?UNIVERSAL bor ?PRIMITIVE bor 19). %can be constructed --define(T_TeletexString, ?UNIVERSAL bor ?PRIMITIVE bor 20). %can be constructed --define(T_VideotexString, ?UNIVERSAL bor ?PRIMITIVE bor 21). %can be constructed --define(T_IA5String, ?UNIVERSAL bor ?PRIMITIVE bor 22). %can be constructed --define(T_GraphicString, ?UNIVERSAL bor ?PRIMITIVE bor 25). %can be constructed --define(T_VisibleString, ?UNIVERSAL bor ?PRIMITIVE bor 26). %can be constructed --define(T_GeneralString, ?UNIVERSAL bor ?PRIMITIVE bor 27). %can be constructed - -%% pgen(Erules, Module, TypeOrVal) -%% Generate Erlang module (.erl) and (.hrl) file corresponding to an ASN.1 module -%% .hrl file is only generated if necessary -%% Erules = per | ber -%% Module = atom() -%% TypeOrVal = {TypeList,ValueList,PTypeList} -%% TypeList = ValueList = [atom()] - -pgen(OutFile,Erules,Module,TypeOrVal) -> - asn1ct_gen:pgen_module(OutFile,Erules,Module,TypeOrVal,[],true). - - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Generate ENCODING -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -%%=============================================================================== -%% encode #{typedef, {pos, name, typespec}} -%%=============================================================================== - -gen_encode(Erules,Type) when is_record(Type,typedef) -> - gen_encode_user(Erules,Type). - -%%=============================================================================== -%% encode #{type, {tag, def, constraint}} -%%=============================================================================== - -gen_encode(Erules,Typename,Type) when is_record(Type,type) -> - InnerType = asn1ct_gen:get_inner(Type#type.def), - ObjFun = - case lists:keysearch(objfun,1,Type#type.tablecinf) of - {value,{_,_Name}} -> - ", ObjFun"; - false -> - "" - end, - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> - emit([nl,nl,nl,"%%================================"]), - emit([nl,"%% ",asn1ct_gen:list2name(Typename)]), - emit([nl,"%%================================",nl]), - case lists:member(InnerType,['SET','SEQUENCE']) of - true -> - case get(asn_keyed_list) of - true -> - CompList = - case Type#type.def of - #'SEQUENCE'{components=Cl} -> Cl; - #'SET'{components=Cl} -> Cl - end, - emit([nl,"'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn",ObjFun, - ") when is_list(Val) ->",nl]), - emit([" 'enc_",asn1ct_gen:list2name(Typename), - "'(?RT_BER:fixoptionals(", - {asis,optionals(CompList)}, - ",Val), TagIn",ObjFun,");",nl,nl]); - _ -> true - end; - _ -> - emit([nl,"'enc_",asn1ct_gen:list2name(Typename), - "'({'",asn1ct_gen:list2name(Typename), - "',Val}, TagIn",ObjFun,") ->",nl]), - emit([" 'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn",ObjFun,");",nl,nl]) - end, - emit(["'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn",ObjFun,") ->",nl," "]), - asn1ct_gen:gen_encode_constructed(Erules,Typename,InnerType,Type); - _ -> - true - end; - -%%=============================================================================== -%% encode ComponentType -%%=============================================================================== - -gen_encode(Erules,Tname,#'ComponentType'{name=Cname,typespec=Type}) -> - NewTname = [Cname|Tname], - %% The tag is set to [] to avoid that it is - %% taken into account twice, both as a component/alternative (passed as - %% argument to the encode decode function and within the encode decode - %% function it self. - NewType = Type#type{tag=[]}, - gen_encode(Erules,NewTname,NewType). - -gen_encode_user(Erules,D) when is_record(D,typedef) -> - Typename = [D#typedef.name], - Type = D#typedef.typespec, - InnerType = asn1ct_gen:get_inner(Type#type.def), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit([nl,nl,"%%================================"]), - emit([nl,"%% ",Typename]), - emit([nl,"%%================================",nl]), - case lists:member(InnerType,['SET','SEQUENCE']) of - true -> - case get(asn_keyed_list) of - true -> - CompList = - case Type#type.def of - #'SEQUENCE'{components=Cl} -> Cl; - #'SET'{components=Cl} -> Cl - end, - - emit([nl,"'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn) when is_list(Val) ->",nl]), - emit([" 'enc_",asn1ct_gen:list2name(Typename), - "'(?RT_BER:fixoptionals(", - {asis,optionals(CompList)}, - ",Val), TagIn);",nl,nl]); - _ -> true - end; - _ -> - emit({nl,"'enc_",asn1ct_gen:list2name(Typename), - "'({'",asn1ct_gen:list2name(Typename),"',Val}, TagIn) ->",nl}), - emit({" 'enc_",asn1ct_gen:list2name(Typename),"'(Val, TagIn);",nl,nl}) - end, - emit({"'enc_",asn1ct_gen:list2name(Typename),"'(", - unused_var("Val",Type#type.def),", TagIn) ->",nl}), - CurrentMod = get(currmod), - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> - asn1ct_gen:gen_encode_constructed(Erules,Typename,InnerType,D); - {primitive,bif} -> - asn1ct_gen_ber:gen_encode_prim(ber,Type,["TagIn ++ ", - {asis,Tag}],"Val"), - emit([".",nl]); - #typereference{val=Ename} -> - emit([" 'enc_",Ename,"'(Val, TagIn ++ ",{asis,Tag},").",nl]); - #'Externaltypereference'{module=CurrentMod,type=Etype} -> - emit([" 'enc_",Etype,"'(Val, TagIn ++ ", - {asis,Tag},").",nl]); - #'Externaltypereference'{module=Emod,type=Etype} -> - emit([" '",Emod,"':'enc_",Etype,"'(Val, TagIn ++ ", - {asis,Tag},").",nl]); - 'ASN1_OPEN_TYPE' -> - emit(["%% OPEN TYPE",nl]), - asn1ct_gen_ber:gen_encode_prim(ber, - Type#type{def='ASN1_OPEN_TYPE'}, - ["TagIn ++ ", - {asis,Tag}],"Val"), - emit([".",nl]) - end. - -unused_var(Var,#'SEQUENCE'{components=Cl}) -> - unused_var1(Var,Cl); -unused_var(Var,#'SET'{components=Cl}) -> - unused_var1(Var,Cl); -unused_var(Var,_) -> - Var. -unused_var1(Var,Cs) when Cs == []; Cs == {[],[]} -> - lists:concat(["_",Var]); -unused_var1(Var,_) -> - Var. - -unused_optormand_var(Var,Def) -> - case asn1ct_gen:type(asn1ct_gen:get_inner(Def)) of - 'ASN1_OPEN_TYPE' -> - lists:concat(["_",Var]); - _ -> - Var - end. - - -gen_encode_prim(Erules,D,DoTag,Value) when is_record(D,type) -> - -%%% Currently not used for BER (except for BitString) and therefore replaced -%%% with [] as a placeholder - BitStringConstraint = D#type.constraint, - Constraint = [], - asn1ct_name:new(enumval), - case D#type.def of - 'BOOLEAN' -> - emit_encode_func('boolean',Value,DoTag); - 'INTEGER' -> - emit_encode_func('integer',Constraint,Value,DoTag); - {'INTEGER',NamedNumberList} -> - emit_encode_func('integer',Constraint,Value, - NamedNumberList,DoTag); - {'ENUMERATED',NamedNumberList={_,_}} -> - - emit(["case (case ",Value," of {_,",{curr,enumval},"}->", - {curr,enumval},";_->", Value," end) of",nl]), - asn1ct_name:new(enumval), - emit_enc_enumerated_cases(NamedNumberList,DoTag); - {'ENUMERATED',NamedNumberList} -> - - emit(["case (case ",Value," of {_,",{curr,enumval},"}->", - {curr,enumval},";_->", Value," end) of",nl]), - asn1ct_name:new(enumval), - emit_enc_enumerated_cases(NamedNumberList,DoTag); - - 'REAL' -> - emit_encode_func('real',Constraint,Value,DoTag); - - {'BIT STRING',NamedNumberList} -> - emit_encode_func('bit_string',BitStringConstraint,Value, - NamedNumberList,DoTag); - 'ANY' -> - emit_encode_func('open_type', Value,DoTag); - 'NULL' -> - emit_encode_func('null',Value,DoTag); - 'OBJECT IDENTIFIER' -> - emit_encode_func("object_identifier",Value,DoTag); - 'RELATIVE-OID' -> - emit_encode_func("relative_oid",Value,DoTag); - 'ObjectDescriptor' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_ObjectDescriptor,DoTag); - 'OCTET STRING' -> - emit_encode_func('octet_string',Constraint,Value,DoTag); - 'NumericString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_NumericString,DoTag); - TString when TString == 'TeletexString'; - TString == 'T61String' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_TeletexString,DoTag); - 'VideotexString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_VideotexString,DoTag); - 'GraphicString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_GraphicString,DoTag); - 'VisibleString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_VisibleString,DoTag); - 'GeneralString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_GeneralString,DoTag); - 'PrintableString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_PrintableString,DoTag); - 'IA5String' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_IA5String,DoTag); - 'UniversalString' -> - emit_encode_func('universal_string',Constraint,Value,DoTag); - 'UTF8String' -> - emit_encode_func('UTF8_string',Constraint,Value,DoTag); - 'BMPString' -> - emit_encode_func('BMP_string',Constraint,Value,DoTag); - 'UTCTime' -> - emit_encode_func('utc_time',Constraint,Value,DoTag); - 'GeneralizedTime' -> - emit_encode_func('generalized_time',Constraint,Value,DoTag); - 'ASN1_OPEN_TYPE' -> - emit_encode_func('open_type', Value,DoTag); - #'ObjectClassFieldType'{} -> - case asn1ct_gen:get_inner(D#type.def) of - {fixedtypevaluefield,_,InnerType} -> - gen_encode_prim(Erules,InnerType,DoTag,Value); - 'ASN1_OPEN_TYPE' -> - emit_encode_func('open_type', Value,DoTag); - XX -> - exit({'can not encode' ,XX}) - end; - XX -> - exit({'can not encode' ,XX}) - end. - - -emit_encode_func(Name,Value,Tags) when is_atom(Name) -> - emit_encode_func(atom_to_list(Name),Value,Tags); -emit_encode_func(Name,Value,Tags) -> - Fname = "?RT_BER:encode_" ++ Name, - emit([Fname,"(",Value,", ",Tags,")"]). - -emit_encode_func(Name,Constraint,Value,Tags) when is_atom(Name) -> - emit_encode_func(atom_to_list(Name),Constraint,Value,Tags); -emit_encode_func(Name,Constraint,Value,Tags) -> - Fname = "?RT_BER:encode_" ++ Name, - emit([Fname,"(",{asis,Constraint},", ",Value,", ",Tags,")"]). - -emit_encode_func(Name,Constraint,Value,Asis,Tags) when is_atom(Name) -> - emit_encode_func(atom_to_list(Name),Constraint,Value,Asis,Tags); -emit_encode_func(Name,Constraint,Value,Asis,Tags) -> - Fname = "?RT_BER:encode_" ++ Name, - emit([Fname,"(",{asis,Constraint},", ",Value, - ", ",{asis,Asis}, - ", ",Tags,")"]). - -emit_enc_enumerated_cases({L1,L2}, Tags) -> - emit_enc_enumerated_cases(L1++L2, Tags, ext); -emit_enc_enumerated_cases(L, Tags) -> - emit_enc_enumerated_cases(L, Tags, noext). - -emit_enc_enumerated_cases([{EnumName,EnumVal},H2|T], Tags, Ext) -> - emit([{asis,EnumName}," -> ?RT_BER:encode_enumerated(",EnumVal,",",Tags,");",nl]), -%% emit(["'",{asis,EnumName},"' -> ?RT_BER:encode_enumerated(",EnumVal,",",Tags,");",nl]), - emit_enc_enumerated_cases([H2|T], Tags, Ext); -emit_enc_enumerated_cases([{EnumName,EnumVal}], Tags, Ext) -> - emit([{asis,EnumName}," -> ?RT_BER:encode_enumerated(",EnumVal,",",Tags,")"]), -%% emit(["'",{asis,EnumName},"' -> ?RT_BER:encode_enumerated(",EnumVal,",",Tags,")"]), - case Ext of - noext -> emit([";",nl]); - ext -> -%% emit([";",nl,"{asn1_enum,",{curr,enumval},"} -> ", -%% "?RT_BER:encode_enumerated(",{curr,enumval},",",Tags,");",nl]), -%% asn1ct_name:new(enumval) - emit([";",nl]) - end, - emit([{curr,enumval}," -> exit({error,{asn1, {enumerated_not_in_range,",{curr, enumval},"}}})"]), - emit([nl,"end"]). - - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Generate DECODING -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -%%=============================================================================== -%% decode #{typedef, {pos, name, typespec}} -%%=============================================================================== - -gen_decode(Erules,Type) when is_record(Type,typedef) -> - D = Type, - emit({nl,nl}), - emit({"'dec_",Type#typedef.name,"'(Bytes, OptOrMand) ->",nl}), - emit({" 'dec_",Type#typedef.name,"'(Bytes, OptOrMand, []).",nl,nl}), - emit({"'dec_",Type#typedef.name,"'(Bytes, ", - unused_optormand_var("OptOrMand",(Type#typedef.typespec)#type.def),", TagIn) ->",nl}), - dbdec(Type#typedef.name), - gen_decode_user(Erules,D). - - -%%=============================================================================== -%% decode #{type, {tag, def, constraint}} -%%=============================================================================== - -gen_decode(Erules,Tname,Type) when is_record(Type,type) -> - Typename = Tname, - InnerType = asn1ct_gen:get_inner(Type#type.def), - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> - ObjFun = - case Type#type.tablecinf of - [{objfun,_}|_R] -> - ", ObjFun"; - _ -> - "" - end, - emit({"'dec_",asn1ct_gen:list2name(Typename),"'(Bytes, OptOrMand, TagIn",ObjFun,") ->",nl}), - dbdec(Typename), - asn1ct_gen:gen_decode_constructed(Erules,Typename,InnerType,Type); - _ -> - true - end; - - -%%=============================================================================== -%% decode ComponentType -%%=============================================================================== - -gen_decode(Erules,Tname,#'ComponentType'{name=Cname,typespec=Type}) -> - NewTname = [Cname|Tname], - %% The tag is set to [] to avoid that it is - %% taken into account twice, both as a component/alternative (passed as - %% argument to the encode decode function and within the encode decode - %% function it self. - NewType = Type#type{tag=[]}, - gen_decode(Erules,NewTname,NewType). - - -gen_decode_user(Erules,D) when is_record(D,typedef) -> - Typename = [D#typedef.name], - Def = D#typedef.typespec, - InnerType = asn1ct_gen:get_inner(Def#type.def), - InnerTag = Def#type.tag , - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- InnerTag], - case asn1ct_gen:type(InnerType) of - 'ASN1_OPEN_TYPE' -> - BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), - asn1ct_name:new(len), - gen_dec_prim(Erules, Def#type{def='ASN1_OPEN_TYPE'}, - BytesVar, Tag, "TagIn",no_length, - ?PRIMITIVE,"OptOrMand"), - emit({".",nl,nl}); - {primitive,bif} -> - BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), - asn1ct_name:new(len), - gen_dec_prim(Erules, Def, BytesVar, Tag, "TagIn",no_length, - ?PRIMITIVE,"OptOrMand"), - emit({".",nl,nl}); - {constructed,bif} -> - asn1ct_gen:gen_decode_constructed(Erules,Typename,InnerType,D); - TheType -> - DecFunName = mkfuncname(TheType,dec), - emit({DecFunName,"(",{curr,bytes}, - ", OptOrMand, TagIn++",{asis,Tag},")"}), - emit({".",nl,nl}) - end. - - -gen_dec_prim(Erules,Att,BytesVar,DoTag,TagIn,Length,Form,OptOrMand) -> - Typename = Att#type.def, -%% Currently not used for BER replaced with [] as place holder -%% Constraint = Att#type.constraint, -%% Constraint = [], - Constraint = - case get_constraint(Att#type.constraint,'SizeConstraint') of - no -> []; - Tc -> Tc - end, - ValueRange = - case get_constraint(Att#type.constraint,'ValueRange') of - no -> []; - Tv -> Tv - end, - SingleValue = - case get_constraint(Att#type.constraint,'SingleValue') of - no -> []; - Sv -> Sv - end, - AsBin = case get(binary_strings) of - true -> "_as_bin"; - _ -> "" - end, - NewTypeName = case Typename of - 'ANY' -> 'ASN1_OPEN_TYPE'; - _ -> Typename - end, - DoLength = - case NewTypeName of - 'BOOLEAN'-> - emit({"?RT_BER:decode_boolean(",BytesVar,","}), - false; - 'INTEGER' -> - emit({"?RT_BER:decode_integer(",BytesVar,",", - {asis,int_constr(SingleValue,ValueRange)},","}), - false; - {'INTEGER',NamedNumberList} -> - emit({"?RT_BER:decode_integer(",BytesVar,",", - {asis,int_constr(SingleValue,ValueRange)},",", - {asis,NamedNumberList},","}), - false; - {'ENUMERATED',NamedNumberList} -> - emit({"?RT_BER:decode_enumerated(",BytesVar,",", - {asis,Constraint},",", - {asis,NamedNumberList},","}), - false; - 'REAL' -> - emit({"?RT_BER:decode_real(",BytesVar,",", - {asis,Constraint},","}), - false; - {'BIT STRING',NamedNumberList} -> - case get(compact_bit_string) of - true -> - emit({"?RT_BER:decode_compact_bit_string(", - BytesVar,",",{asis,Constraint},",", - {asis,NamedNumberList},","}); - _ -> - emit({"?RT_BER:decode_bit_string(",BytesVar,",", - {asis,Constraint},",", - {asis,NamedNumberList},","}) - end, - true; - 'NULL' -> - emit({"?RT_BER:decode_null(",BytesVar,","}), - false; - 'OBJECT IDENTIFIER' -> - emit({"?RT_BER:decode_object_identifier(",BytesVar,","}), - false; - 'RELATIVE-OID' -> - emit({"?RT_BER:decode_relative_oid(",BytesVar,","}), - false; - 'ObjectDescriptor' -> - emit({"?RT_BER:decode_restricted_string(", - BytesVar,",",{asis,Constraint},",",{asis,?T_ObjectDescriptor},","}), - true; - 'OCTET STRING' -> - emit({"?RT_BER:decode_octet_string",AsBin,"(",BytesVar,",",{asis,Constraint},","}), - true; - 'NumericString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_NumericString},","}),true; - TString when TString == 'TeletexString'; - TString == 'T61String' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_TeletexString},","}), - true; - 'VideotexString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_VideotexString},","}), - true; - 'GraphicString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_GraphicString},","}) - ,true; - 'VisibleString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_VisibleString},","}), - true; - 'GeneralString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_GeneralString},","}), - true; - 'PrintableString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_PrintableString},","}), - true; - 'IA5String' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_IA5String},","}), - true; - 'UniversalString' -> - emit({"?RT_BER:decode_universal_string",AsBin,"(", - BytesVar,",",{asis,Constraint},","}), - true; - 'UTF8String' -> - emit({"?RT_BER:decode_UTF8_string",AsBin,"(", - BytesVar,","}), - false; - 'BMPString' -> - emit({"?RT_BER:decode_BMP_string",AsBin,"(", - BytesVar,",",{asis,Constraint},","}), - true; - 'UTCTime' -> - emit({"?RT_BER:decode_utc_time",AsBin,"(", - BytesVar,",",{asis,Constraint},","}), - true; - 'GeneralizedTime' -> - emit({"?RT_BER:decode_generalized_time",AsBin,"(", - BytesVar,",",{asis,Constraint},","}), - true; - 'ASN1_OPEN_TYPE' -> - emit(["?RT_BER:decode_open_type(",re_wrap_erule(Erules),",", - BytesVar,","]), - false; - #'ObjectClassFieldType'{} -> - case asn1ct_gen:get_inner(Att#type.def) of - {fixedtypevaluefield,_,InnerType} -> - gen_dec_prim(Erules,InnerType,BytesVar,DoTag,TagIn,Length,Form,OptOrMand), - false; - 'ASN1_OPEN_TYPE' -> - emit(["?RT_BER:decode_open_type(", - re_wrap_erule(Erules),",", - BytesVar,","]), - false; - XX -> - exit({'can not decode' ,XX}) - end; - Other -> - exit({'can not decode' ,Other}) - end, - - NewLength = case DoLength of - true -> [", ", Length]; - false -> "" - end, - NewOptOrMand = case OptOrMand of - _ when is_list(OptOrMand) -> OptOrMand; - mandatory -> {asis,mandatory}; - _ -> {asis,opt_or_default} - end, - case {TagIn,NewTypeName} of - {_,#'ObjectClassFieldType'{}} -> - case asn1ct_gen:get_inner(Att#type.def) of - 'ASN1_OPEN_TYPE' -> - emit([{asis,DoTag},")"]); - _ -> ok - end; - {[],'ASN1_OPEN_TYPE'} -> - emit([{asis,DoTag},")"]); - {_,'ASN1_OPEN_TYPE'} -> - emit([TagIn,"++",{asis,DoTag},")"]); - {[],_} -> - emit([{asis,DoTag},NewLength,", ",NewOptOrMand,")"]); - _ when is_list(TagIn) -> - emit([TagIn,"++",{asis,DoTag},NewLength,", ",NewOptOrMand,")"]) - end. - - -int_constr([],[]) -> - []; -int_constr([],ValueRange) -> - ValueRange; -int_constr(SingleValue,[]) -> - SingleValue; -int_constr(SV,VR) -> - [SV,VR]. - -%% Object code generating for encoding and decoding -%% ------------------------------------------------ - -gen_obj_code(Erules,_Module,Obj) when is_record(Obj,typedef) -> - ObjName = Obj#typedef.name, - Def = Obj#typedef.typespec, - #'Externaltypereference'{module=M,type=ClName} = Def#'Object'.classname, - Class = asn1_db:dbget(M,ClName), - - {object,_,Fields} = Def#'Object'.def, - emit({nl,nl,nl,"%%================================"}), - emit({nl,"%% ",ObjName}), - emit({nl,"%%================================",nl}), - EncConstructed = - gen_encode_objectfields(ClName,get_class_fields(Class), - ObjName,Fields,[]), - emit(nl), - gen_encode_constr_type(Erules,EncConstructed), - emit(nl), - DecConstructed = - gen_decode_objectfields(ClName,get_class_fields(Class), - ObjName,Fields,[]), - emit(nl), - gen_decode_constr_type(Erules,DecConstructed); -gen_obj_code(_Erules,_Module,Obj) when is_record(Obj,pobjectdef) -> - ok. - - -gen_encode_objectfields(ClassName,[{typefield,Name,OptOrMand}|Rest], - ObjName,ObjectFields,ConstrAcc) -> - EmitFuncClause = - fun(Args) -> - emit(["'enc_",ObjName,"'(",{asis,Name}, - ", ",Args,", _RestPrimFieldName) ->",nl]) - end, -% emit(["'enc_",ObjName,"'(",{asis,Name}, -% ", Val, TagIn, _RestPrimFieldName) ->",nl]), - MaybeConstr= - case {get_object_field(Name,ObjectFields),OptOrMand} of - {false,'MANDATORY'} -> %% this case is illegal - exit({error,{asn1,{"missing mandatory field in object", - ObjName}}}); - {false,'OPTIONAL'} -> %% OPTIONAL field in class - EmitFuncClause("Val, _"), %% Value must be anything - %% already encoded - emit([" {Val,0}"]), - []; - {false,{'DEFAULT',DefaultType}} -> - EmitFuncClause("Val, TagIn"), - gen_encode_default_call(ClassName,Name,DefaultType); - {{Name,TypeSpec},_} -> - %% A specified field owerwrites any 'DEFAULT' or - %% 'OPTIONAL' field in the class - EmitFuncClause("Val, TagIn"), - gen_encode_field_call(ObjName,Name,TypeSpec) - end, - case more_genfields(Rest) of - true -> - emit([";",nl]); - false -> - emit([".",nl]) - end, - gen_encode_objectfields(ClassName,Rest,ObjName,ObjectFields, - MaybeConstr++ConstrAcc); -gen_encode_objectfields(ClassName,[{objectfield,Name,_,_,OptOrMand}|Rest], - ObjName,ObjectFields,ConstrAcc) -> - CurrentMod = get(currmod), - EmitFuncClause = - fun(Args) -> - emit(["'enc_",ObjName,"'(",{asis,Name}, - ", ",Args,") ->",nl]) - end, -% emit(["'enc_",ObjName,"'(",{asis,Name}, -% ", Val, TagIn, [H|T]) ->",nl]), - case {get_object_field(Name,ObjectFields),OptOrMand} of - {false,'MANDATORY'} -> - exit({error,{asn1,{"missing mandatory field in object", - ObjName}}}); - {false,'OPTIONAL'} -> - EmitFuncClause("_,_,_"), - emit([" exit({error,{'use of missing field in object', ",{asis,Name}, - "}})"]); - {false,{'DEFAULT',_DefaultObject}} -> - exit({error,{asn1,{"not implemented yet",Name}}}); - {{Name,#'Externalvaluereference'{module=CurrentMod, - value=TypeName}},_} -> - EmitFuncClause(" Val, TagIn, [H|T]"), - emit({indent(3),"'enc_",TypeName,"'(H, Val, TagIn, T)"}); - {{Name,#'Externalvaluereference'{module=M,value=TypeName}},_} -> - EmitFuncClause(" Val, TagIn, [H|T]"), - emit({indent(3),"'",M,"':'enc_",TypeName,"'(H, Val, TagIn, T)"}); - {{Name,TypeSpec},_} -> - EmitFuncClause(" Val, TagIn, [H|T]"), - case TypeSpec#typedef.name of - {ExtMod,TypeName} -> - emit({indent(3),"'",ExtMod,"':'enc_",TypeName, - "'(H, Val, TagIn, T)"}); - TypeName -> - emit({indent(3),"'enc_",TypeName,"'(H, Val, TagIn, T)"}) - end - end, - case more_genfields(Rest) of - true -> - emit([";",nl]); - false -> - emit([".",nl]) - end, - gen_encode_objectfields(ClassName,Rest,ObjName,ObjectFields,ConstrAcc); -gen_encode_objectfields(ClassName,[_|Cs],O,OF,Acc) -> - gen_encode_objectfields(ClassName,Cs,O,OF,Acc); -gen_encode_objectfields(_,[],_,_,Acc) -> - Acc. - - -% gen_encode_objectfields(Class,ObjName,[{FieldName,Type}|Rest],ConstrAcc) -> -% Fields = Class#objectclass.fields, -% MaybeConstr= -% case is_typefield(Fields,FieldName) of -% true -> -% Def = Type#typedef.typespec, -% OTag = Def#type.tag, -% Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], -% emit({"'enc_",ObjName,"'(",{asis,FieldName}, -% ", Val, TagIn, RestPrimFieldName) ->",nl}), -% CAcc= -% case Type#typedef.name of -% {primitive,bif} -> -% gen_encode_prim(ber,Def,["TagIn ++ ",{asis,Tag}], -% "Val"), -% []; -% {constructed,bif} -> -% %%InnerType = asn1ct_gen:get_inner(Def#type.def), -% %%asn1ct_gen:gen_encode_constructed(ber,[ObjName], -% %% InnerType,Def); -% emit({" 'enc_",ObjName,'_',FieldName, -% "'(Val, TagIn ++ ",{asis,Tag},")"}), -% [{['enc_',ObjName,'_',FieldName],Def}]; -% {ExtMod,TypeName} -> -% emit({" '",ExtMod,"':'enc_",TypeName, -% "'(Val, TagIn ++ ",{asis,Tag},")"}), -% []; -% TypeName -> -% emit({" 'enc_",TypeName,"'(Val, TagIn ++ ", -% {asis,Tag},")"}), -% [] -% end, -% case more_genfields(Fields,Rest) of -% true -> -% emit({";",nl}); -% false -> -% emit({".",nl}) -% end, -% CAcc; -% {false,objectfield} -> -% emit({"'enc_",ObjName,"'(",{asis,FieldName}, -% ", Val, TagIn, [H|T]) ->",nl}), -% case Type#typedef.name of -% {ExtMod,TypeName} -> -% emit({indent(3),"'",ExtMod,"':'enc_",TypeName, -% "'(H, Val, TagIn, T)"}); -% TypeName -> -% emit({indent(3),"'enc_",TypeName,"'(H, Val, TagIn, T)"}) -% end, -% case more_genfields(Fields,Rest) of -% true -> -% emit({";",nl}); -% false -> -% emit({".",nl}) -% end, -% []; -% {false,_} -> [] -% end, -% gen_encode_objectfields(Class,ObjName,Rest,MaybeConstr ++ ConstrAcc); -% gen_encode_objectfields(C,O,[H|T],Acc) -> -% gen_encode_objectfields(C,O,T,Acc); -% gen_encode_objectfields(_,_,[],Acc) -> -% Acc. - -% gen_encode_constr_type([{Name,Def}|Rest]) -> -% emit({Name,"(Val,TagIn) ->",nl}), -% InnerType = asn1ct_gen:get_inner(Def#type.def), -% asn1ct_gen:gen_encode_constructed(ber,Name,InnerType,Def), -% gen_encode_constr_type(Rest); -gen_encode_constr_type(Erules,[TypeDef|Rest]) when is_record(TypeDef,typedef) -> - case is_already_generated(enc,TypeDef#typedef.name) of - true -> ok; - _ -> gen_encode_user(Erules,TypeDef) - end, - gen_encode_constr_type(Erules,Rest); -gen_encode_constr_type(_,[]) -> - ok. - -gen_encode_field_call(_ObjName,_FieldName, - #'Externaltypereference'{module=M,type=T}) -> - CurrentMod = get(currmod), - TDef = asn1_db:dbget(M,T), - Def = TDef#typedef.typespec, - OTag = Def#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - if - M == CurrentMod -> - emit({" 'enc_",T,"'(Val, TagIn ++ ",{asis,Tag},")"}), - []; - true -> - emit({" '",M,"':'enc_",T,"'(Val, TagIn ++ ",{asis,Tag},")"}), - [] - end; -gen_encode_field_call(ObjName,FieldName,Type) -> - Def = Type#typedef.typespec, - OTag = Def#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case Type#typedef.name of - {primitive,bif} -> %%tag should be the primitive tag - gen_encode_prim(ber,Def,["TagIn ++ ",{asis,Tag}], - "Val"), - []; - {constructed,bif} -> - emit({" 'enc_",ObjName,'_',FieldName, - "'(Val, TagIn ++",{asis,Tag},")"}), - [Type#typedef{name=list_to_atom(lists:concat([ObjName,'_',FieldName]))}]; - {ExtMod,TypeName} -> - emit({" '",ExtMod,"':'enc_",TypeName, - "'(Val, TagIn ++ ",{asis,Tag},")"}), - []; - TypeName -> - emit({" 'enc_",TypeName,"'(Val, TagIn ++ ",{asis,Tag},")"}), - [] - end. - -gen_encode_default_call(ClassName,FieldName,Type) -> - CurrentMod = get(currmod), - InnerType = asn1ct_gen:get_inner(Type#type.def), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> -%% asn1ct_gen:gen_encode_constructed(Erules,Typename,InnerType,Type); - emit([" 'enc_",ClassName,'_',FieldName,"'(Bytes, TagIn ++ ", - {asis,Tag},")"]), - [#typedef{name=list_to_atom(lists:concat([ClassName,'_',FieldName])), - typespec=Type}]; - {primitive,bif} -> - gen_encode_prim(ber,Type,["TagIn ++ ",{asis,Tag}],"Val"), - []; - #'Externaltypereference'{module=CurrentMod,type=Etype} -> - emit([" 'enc_",Etype,"'(Val, TagIn ++ ",{asis,Tag},")",nl]), - []; - #'Externaltypereference'{module=Emod,type=Etype} -> - emit([" '",Emod,"':'enc_",Etype,"'(Val, TagIn ++ ",{asis,Tag},")",nl]), - [] - end. - - - -gen_decode_objectfields(ClassName,[{typefield,Name,OptOrMand}|Rest], - ObjName,ObjectFields,ConstrAcc) -> - EmitFuncClause = - fun(Args) -> - emit(["'dec_",ObjName,"'(",{asis,Name}, - ", ",Args,"_) ->",nl]) - end, -% emit(["'dec_",ObjName,"'(",{asis,Name}, -% ", Bytes, TagIn, RestPrimFieldName) ->",nl]), - MaybeConstr= - case {get_object_field(Name,ObjectFields),OptOrMand} of - {false,'MANDATORY'} -> %% this case is illegal - exit({error,{asn1,{"missing mandatory field in object", - ObjName}}}); - {false,'OPTIONAL'} -> - EmitFuncClause("Bytes, _,"), -% emit([" asn1_NOVALUE"]), - emit([" {Bytes,[],0}"]), - []; - {false,{'DEFAULT',DefaultType}} -> - EmitFuncClause("Bytes, TagIn,"), - gen_decode_default_call(ClassName,Name,"Bytes",DefaultType); - {{Name,TypeSpec},_} -> - %% A specified field owerwrites any 'DEFAULT' or - %% 'OPTIONAL' field in the class - EmitFuncClause("Bytes, TagIn,"), - gen_decode_field_call(ObjName,Name,"Bytes",TypeSpec) - end, - case more_genfields(Rest) of - true -> - emit([";",nl]); - false -> - emit([".",nl]) - end, - gen_decode_objectfields(ClassName,Rest,ObjName,ObjectFields,MaybeConstr++ConstrAcc); -gen_decode_objectfields(ClassName,[{objectfield,Name,_,_,OptOrMand}|Rest], - ObjName,ObjectFields,ConstrAcc) -> - CurrentMod = get(currmod), - EmitFuncClause = - fun(Args) -> - emit(["'dec_",ObjName,"'(",{asis,Name}, - ", ",Args,") ->",nl]) - end, -% emit(["'dec_",ObjName,"'(",{asis,Name}, -% ", Bytes,TagIn,[H|T]) ->",nl]), - case {get_object_field(Name,ObjectFields),OptOrMand} of - {false,'MANDATORY'} -> - exit({error,{asn1,{"missing mandatory field in object", - ObjName}}}); - {false,'OPTIONAL'} -> - EmitFuncClause("_,_,_"), - emit([" exit({error,{'illegal use of missing field in object', ",{asis,Name}, - "}})"]); - {false,{'DEFAULT',_DefaultObject}} -> - exit({error,{asn1,{"not implemented yet",Name}}}); - {{Name,#'Externalvaluereference'{module=CurrentMod, - value=TypeName}},_} -> - EmitFuncClause("Bytes,TagIn,[H|T]"), - emit({indent(3),"'dec_",TypeName,"'(H, Bytes, TagIn, T)"}); - {{Name,#'Externalvaluereference'{module=M,value=TypeName}},_} -> - EmitFuncClause("Bytes,TagIn,[H|T]"), - emit({indent(3),"'",M,"':'dec_",TypeName, - "'(H, Bytes, TagIn, T)"}); - {{Name,TypeSpec},_} -> - EmitFuncClause("Bytes,TagIn,[H|T]"), - case TypeSpec#typedef.name of - {ExtMod,TypeName} -> - emit({indent(3),"'",ExtMod,"':'dec_",TypeName, - "'(H, Bytes, TagIn, T)"}); - TypeName -> - emit({indent(3),"'dec_",TypeName,"'(H, Bytes, TagIn, T)"}) - end - end, - case more_genfields(Rest) of - true -> - emit([";",nl]); - false -> - emit([".",nl]) - end, - gen_decode_objectfields(ClassName,Rest,ObjName,ObjectFields,ConstrAcc); -gen_decode_objectfields(CN,[_|Cs],O,OF,CAcc) -> - gen_decode_objectfields(CN,Cs,O,OF,CAcc); -gen_decode_objectfields(_,[],_,_,CAcc) -> - CAcc. - - -gen_decode_constr_type(Erules,[TypeDef|Rest]) when is_record(TypeDef,typedef) -> - case is_already_generated(dec,TypeDef#typedef.name) of - true -> ok; - _ -> - gen_decode(Erules,TypeDef) - end, - gen_decode_constr_type(Erules,Rest); -gen_decode_constr_type(_,[]) -> - ok. - -gen_decode_field_call(_ObjName,_FieldName,Bytes, - #'Externaltypereference'{module=M,type=T}) -> - CurrentMod = get(currmod), - TDef = asn1_db:dbget(M,T), - Def = TDef#typedef.typespec, - OTag = Def#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - if - M == CurrentMod -> - emit({" 'dec_",T,"'(",Bytes, - ", opt_or_default,TagIn ++ ",{asis,Tag},")"}), - []; - true -> - emit({" '",M,"':'dec_",T, - "'(",Bytes,", opt_or_default,TagIn ++ ",{asis,Tag},")"}), - [] - end; -gen_decode_field_call(ObjName,FieldName,Bytes,Type) -> - Def = Type#typedef.typespec, - OTag = Def#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case Type#typedef.name of - {primitive,bif} -> %%tag should be the primitive tag - gen_dec_prim(ber,Def,Bytes,Tag,"TagIn",no_length, - ?PRIMITIVE,opt_or_default), - []; - {constructed,bif} -> - emit({" 'dec_",ObjName,'_',FieldName, - "'(",Bytes,",opt_or_default, TagIn ++ ",{asis,Tag},")"}), - [Type#typedef{name=list_to_atom(lists:concat([ObjName,'_',FieldName]))}]; - {ExtMod,TypeName} -> - emit({" '",ExtMod,"':'dec_",TypeName, - "'(",Bytes,", opt_or_default,TagIn ++ ",{asis,Tag},")"}), - []; - TypeName -> - emit({" 'dec_",TypeName,"'(",Bytes, - ", opt_or_default,TagIn ++ ",{asis,Tag},")"}), - [] - end. - -gen_decode_default_call(ClassName,FieldName,Bytes,Type) -> - CurrentMod = get(currmod), - InnerType = asn1ct_gen:get_inner(Type#type.def), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> - emit([" 'dec_",ClassName,'_',FieldName,"'(",Bytes, - ",opt_or_default, TagIn ++ ",{asis,Tag},")"]), - [#typedef{name=list_to_atom(lists:concat([ClassName,'_',FieldName])), - typespec=Type}]; - {primitive,bif} -> - gen_dec_prim(ber,Type,Bytes,Tag,"TagIn",no_length, - ?PRIMITIVE,opt_or_default), - []; - #'Externaltypereference'{module=CurrentMod,type=Etype} -> - emit([" 'dec_",Etype,"'(",Bytes, - " ,opt_or_default, TagIn ++ ",{asis,Tag},")",nl]), - []; - #'Externaltypereference'{module=Emod,type=Etype} -> - emit([" '",Emod,"':'dec_",Etype,"'(",Bytes, - ", opt_or_defualt, TagIn ++ ",{asis,Tag},")",nl]), - [] - end. - - -more_genfields([]) -> - false; -more_genfields([Field|Fields]) -> - case element(1,Field) of - typefield -> - true; - objectfield -> - true; - _ -> - more_genfields(Fields) - end. - - - -%% Object Set code generating for encoding and decoding -%% ---------------------------------------------------- -gen_objectset_code(Erules,ObjSet) -> - ObjSetName = ObjSet#typedef.name, - Def = ObjSet#typedef.typespec, -% {ClassName,ClassDef} = Def#'ObjectSet'.class, - #'Externaltypereference'{module=ClassModule, - type=ClassName} = Def#'ObjectSet'.class, - ClassDef = asn1_db:dbget(ClassModule,ClassName), - UniqueFName = Def#'ObjectSet'.uniquefname, - Set = Def#'ObjectSet'.set, - emit({nl,nl,nl,"%%================================"}), - emit({nl,"%% ",ObjSetName}), - emit({nl,"%%================================",nl}), - case ClassName of - {_Module,ExtClassName} -> - gen_objset_code(Erules,ObjSetName,UniqueFName,Set, - ExtClassName,ClassDef); - _ -> - gen_objset_code(Erules,ObjSetName,UniqueFName,Set, - ClassName,ClassDef) - end, - emit(nl). - -gen_objset_code(Erules,ObjSetName,UniqueFName,Set,ClassName,ClassDef)-> - ClassFields = (ClassDef#classdef.typespec)#objectclass.fields, - InternalFuncs=gen_objset_enc(ObjSetName,UniqueFName,Set,ClassName,ClassFields,1,[]), - gen_objset_dec(Erules,ObjSetName,UniqueFName,Set,ClassName,ClassFields,1), - gen_internal_funcs(Erules,InternalFuncs). - - -%% gen_objset_enc iterates over the objects of the object set -gen_objset_enc(_,{unique,undefined},_,_,_,_,_) -> - %% There is no unique field in the class of this object set - %% don't bother about the constraint - []; -gen_objset_enc(ObjSName,UniqueName,[{_,no_unique_value,_},T|Rest], - ClName,ClFields,NthObj,Acc) -> - %% No need to check that this class has property OPTIONAL for the - %% unique field, it was detected in the previous phase - gen_objset_enc(ObjSName,UniqueName,[T|Rest],ClName,ClFields,NthObj,Acc); -gen_objset_enc(ObjSetName,UniqueName,[{_,no_unique_value,_}], - _ClName,_ClFields,_NthObj,Acc) -> - %% No need to check that this class has property OPTIONAL for the - %% unique field, it was detected in the previous phase - emit_default_getenc(ObjSetName,UniqueName), - emit({".",nl,nl}), - Acc; -gen_objset_enc(ObjSName,UniqueName, - [{ObjName,Val,Fields},T|Rest],ClName,ClFields,NthObj,Acc)-> - emit({"'getenc_",ObjSName,"'(",{asis,UniqueName},",",{asis,Val},") ->",nl}), - CurrMod = get(currmod), - {InternalFunc,NewNthObj}= - case ObjName of - {no_mod,no_name} -> - gen_inlined_enc_funs(Fields,ClFields,ObjSName,NthObj); - {CurrMod,Name} -> - emit({" fun 'enc_",Name,"'/4"}), - {[],NthObj}; - {ModuleName,Name} -> - emit_ext_fun(enc,ModuleName,Name), -% emit([" {'",ModuleName,"', 'enc_",Name,"'}"]), - {[],NthObj}; - _Other -> - emit({" fun 'enc_",ObjName,"'/4"}), - {[],NthObj} - end, - emit({";",nl}), - gen_objset_enc(ObjSName,UniqueName,[T|Rest],ClName,ClFields, - NewNthObj,InternalFunc ++ Acc); -gen_objset_enc(ObjSetName,UniqueName, - [{ObjName,Val,Fields}],_ClName,ClFields,NthObj,Acc) -> - emit({"'getenc_",ObjSetName,"'(",{asis,UniqueName},",",{asis,Val},") ->",nl}), - CurrMod = get(currmod), - {InternalFunc,_}= - case ObjName of - {no_mod,no_name} -> - gen_inlined_enc_funs(Fields,ClFields,ObjSetName,NthObj); - {CurrMod,Name} -> - emit({" fun 'enc_",Name,"'/4"}), - {[],NthObj}; - {ModuleName,Name} -> - emit_ext_fun(enc,ModuleName,Name), -% emit([" {'",ModuleName,"', 'enc_",Name,"'}"]), - {[],NthObj}; - _Other -> - emit({" fun 'enc_",ObjName,"'/4"}), - {[],NthObj} - end, - emit([";",nl]), - emit_default_getenc(ObjSetName,UniqueName), - emit({".",nl,nl}), - InternalFunc ++ Acc; -%% See X.681 Annex E for the following case -gen_objset_enc(ObjSetName,_UniqueName,['EXTENSIONMARK'], - _ClName,_ClFields,_NthObj,Acc) -> - emit({"'getenc_",ObjSetName,"'(_, _) ->",nl}), - emit({indent(3),"fun(_Attr, Val, _TagIn, _RestPrimFieldName) ->",nl}), - emit({indent(6),"Len = case Val of",nl,indent(9), - "Bin when is_binary(Bin) -> size(Bin);",nl,indent(9), - "_ -> length(Val)",nl,indent(6),"end,"}), - emit({indent(6),"{Val,Len}",nl}), - emit({indent(3),"end.",nl,nl}), - Acc; -gen_objset_enc(ObjSetName,UniqueName,['EXTENSIONMARK','EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj,Acc) -> - gen_objset_enc(ObjSetName,UniqueName,['EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj,Acc); -gen_objset_enc(ObjSetName,UniqueName,['EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj,Acc) -> - gen_objset_enc(ObjSetName,UniqueName,Rest++['EXTENSIONMARK'], - ClName,ClFields,NthObj,Acc); -gen_objset_enc(_,_,[],_,_,_,Acc) -> - Acc. - -emit_ext_fun(EncDec,ModuleName,Name) -> - emit([indent(3),"fun(T,V,_O1,_O2) -> '",ModuleName,"':'",EncDec,"_", - Name,"'(T,V,_O1,_O2) end"]). - -emit_default_getenc(ObjSetName,UniqueName) -> - emit(["'getenc_",ObjSetName,"'(",{asis,UniqueName},", ErrV) ->",nl]), - emit([indent(3),"fun(C,V,_,_) -> exit({'Type not compatible with table constraint',{component,C},{value,V}, {unique_name_and_value,",{asis,UniqueName},", ErrV}}) end"]). - -%% gen_inlined_enc_funs for each object iterates over all fields of a -%% class, and for each typefield it checks if the object has that -%% field and emits the proper code. -gen_inlined_enc_funs(Fields,[{typefield,Name,_}|Rest],ObjSetName, - NthObj) -> - CurrMod = get(currmod), - InternalDefFunName = asn1ct_gen:list2name([NthObj,Name,ObjSetName]), - case lists:keysearch(Name,1,Fields) of - {value,{_,Type}} when is_record(Type,type) -> - emit({indent(3),"fun(Type, Val, TagIn, _RestPrimFieldName) ->",nl, - indent(6),"case Type of",nl}), - {Ret,N} = emit_inner_of_fun(Type,InternalDefFunName), - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj+N,Ret); - {value,{_,Type}} when is_record(Type,typedef) -> - emit({indent(3),"fun(Type, Val, TagIn, _RestPrimFieldName) ->",nl, - indent(6),"case Type of",nl}), - emit({indent(9),{asis,Name}," ->",nl}), - {Ret,N} = emit_inner_of_fun(Type,InternalDefFunName), - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj+N,Ret); - {value,{_,#'Externaltypereference'{module=M,type=T}}} -> - #typedef{typespec=Type} = asn1_db:dbget(M,T), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit([indent(3),"fun(Type, Val, TagIn, _RestPrimFieldName) ->",nl, - indent(6),"case Type of",nl]), - emit([indent(9),{asis,Name}," ->",nl]), - if - M == CurrMod -> - emit([indent(12),"'enc_",T,"'(Val, TagIn ++ ", - {asis,Tag},")"]); - true -> - emit([indent(12),"'",M,"':'enc_",T,"'(Val,TagIn ++", - {asis,Tag},")"]) - end, - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj,[]); - false -> - %% This field was not present in the object thus there were no - %% type in the table and we therefore generate code that returns - %% the input for application treatment. - emit([indent(3),"fun(Type, Val, TagIn, _RestPrimFieldName) ->",nl, - indent(6),"case Type of",nl]), - emit([indent(9),{asis,Name}," ->",nl]), - emit([indent(12),"Len = case Val of",nl, - indent(15),"Bin when is_binary(Bin) -> size(Bin);",nl, - indent(15),"_ -> length(Val)",nl,indent(12),"end,",nl, - indent(12),"{Val,Len}"]), - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj,[]) - end; -gen_inlined_enc_funs(Fields,[_H|Rest],ObjSetName,NthObj) -> - gen_inlined_enc_funs(Fields,Rest,ObjSetName,NthObj); -gen_inlined_enc_funs(_,[],_,NthObj) -> - {[],NthObj}. - -gen_inlined_enc_funs1(Fields,[{typefield,Name,_}|Rest],ObjSetName, - NthObj,Acc) -> - CurrMod = get(currmod), - InternalDefFunName = asn1ct_gen:list2name([NthObj,Name,ObjSetName]), - {Acc2,NAdd}= - case lists:keysearch(Name,1,Fields) of - {value,{_,Type}} when is_record(Type,type) -> - emit({";",nl}), - {Ret,N}=emit_inner_of_fun(Type,InternalDefFunName), - {Ret++Acc,N}; - {value,{_,Type}} when is_record(Type,typedef) -> - emit({";",nl,indent(9),{asis,Name}," ->",nl}), - {Ret,N}=emit_inner_of_fun(Type,InternalDefFunName), - {Ret++Acc,N}; - {value,{_,#'Externaltypereference'{module=M,type=T}}} -> - #typedef{typespec=Type} = asn1_db:dbget(M,T), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit({";",nl,indent(9),{asis,Name}," ->",nl}), - if - M == CurrMod -> - emit([indent(12),"'enc_",T,"'(Val, TagIn ++ ", - {asis,Tag},")"]); - true -> - emit([indent(12),"'",M,"':'enc_",T,"'(Val,TagIn ++", - {asis,Tag},")"]) - end, - {Acc,0}; - false -> - %% This field was not present in the object thus there were no - %% type in the table and we therefore generate code that returns - %% the input for application treatment. - emit([";",nl,indent(9),{asis,Name}," ->",nl]), - emit([indent(12),"Len = case Val of",nl, - indent(15),"Bin when is_binary(Bin) -> size(Bin);",nl, - indent(15),"_ -> length(Val)",nl,indent(12),"end,",nl, - indent(12),"{Val,Len}"]), - {Acc,0} - end, - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj+NAdd,Acc2); -gen_inlined_enc_funs1(Fields,[_H|Rest],ObjSetName,NthObj,Acc)-> - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj,Acc); -gen_inlined_enc_funs1(_,[],_,NthObj,Acc) -> - emit({nl,indent(6),"end",nl}), - emit({indent(3),"end"}), - {Acc,NthObj}. - - -emit_inner_of_fun(TDef = #typedef{name={ExtMod,Name},typespec=Type}, - InternalDefFunName) -> - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case {ExtMod,Name} of - {primitive,bif} -> - emit(indent(12)), - gen_encode_prim(ber,Type,["TagIn ++ ",{asis,Tag}],"Val"), - {[],0}; - {constructed,bif} -> - emit([indent(12),"'enc_", - InternalDefFunName,"'(Val,TagIn ++ ", - {asis,Tag},")"]), - {[TDef#typedef{name=InternalDefFunName}],1}; - _ -> - emit({indent(12),"'",ExtMod,"':'enc_",Name,"'(Val, TagIn ++ ", - {asis,Tag},")"}), - {[],0} - end; -emit_inner_of_fun(#typedef{name=Name,typespec=Type},_) -> - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit({indent(12),"'enc_",Name,"'(Val, TagIn ++ ",{asis,Tag},")"}), - {[],0}; -emit_inner_of_fun(Type,_) when is_record(Type,type) -> - CurrMod = get(currmod), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case Type#type.def of - Def when is_atom(Def) -> - emit({indent(9),Def," ->",nl,indent(12)}), - gen_encode_prim(ber,Type,["TagIn ++ ",{asis,Tag}],"Val"); - TRef when is_record(TRef,typereference) -> - T = TRef#typereference.val, - emit({indent(9),T," ->",nl,indent(12),"'enc_",T, - "'(Val, TagIn ++ ",{asis,Tag},")"}); - #'Externaltypereference'{module=CurrMod,type=T} -> - emit({indent(9),T," ->",nl,indent(12),"'enc_",T, - "'(Val, TagIn ++ ",{asis,Tag},")"}); - #'Externaltypereference'{module=ExtMod,type=T} -> - emit({indent(9),T," ->",nl,indent(12),ExtMod,":'enc_", - T,"'(Val, TagIn ++ ",{asis,Tag},")"}) - end, - {[],0}. - -indent(N) -> - lists:duplicate(N,32). % 32 = space - - -gen_objset_dec(_,_,{unique,undefined},_,_,_,_) -> - %% There is no unique field in the class of this object set - %% don't bother about the constraint - ok; -gen_objset_dec(Erules,ObjSName,UniqueName,[{_,no_unique_value,_},T|Rest], - ClName,ClFields,NthObj)-> - gen_objset_dec(Erules,ObjSName,UniqueName,[T|Rest],ClName,ClFields, - NthObj); -gen_objset_dec(_Erules,ObjSetName,UniqueName,[{_,no_unique_value,_}], - _ClName,_ClFields,_NthObj)-> - emit_default_getdec(ObjSetName,UniqueName), - emit({".",nl,nl}); -gen_objset_dec(Erules,ObjSName,UniqueName,[{ObjName,Val,Fields},T|Rest], - ClName,ClFields,NthObj)-> - emit({"'getdec_",ObjSName,"'(",{asis,UniqueName},",",{asis,Val}, - ") ->",nl}), - CurrMod = get(currmod), - NewNthObj= - case ObjName of - {no_mod,no_name} -> - gen_inlined_dec_funs(Erules,Fields,ClFields,ObjSName, - NthObj); - {CurrMod,Name} -> - emit({" fun 'dec_",Name,"'/4"}), - NthObj; - {ModName,Name} -> - emit_ext_fun(dec,ModName,Name), -% emit([" {'",ModName,"', 'dec_",Name,"'}"]), - NthObj; - _Other -> - emit({" fun 'dec_",ObjName,"'/4"}), - NthObj - end, - emit({";",nl}), - gen_objset_dec(Erules,ObjSName,UniqueName,[T|Rest],ClName,ClFields, - NewNthObj); -gen_objset_dec(Erules,ObjSetName,UniqueName,[{ObjName,Val,Fields}],_ClName, - ClFields,NthObj) -> - emit({"'getdec_",ObjSetName,"'(",{asis,UniqueName},",",{asis,Val},") ->",nl}), - CurrMod = get(currmod), - case ObjName of - {no_mod,no_name} -> - gen_inlined_dec_funs(Erules,Fields,ClFields,ObjSetName, - NthObj); - {CurrMod,Name} -> - emit({" fun 'dec_",Name,"'/4"}); - {ModName,Name} -> - emit_ext_fun(dec,ModName,Name); -% emit([" {'",ModName,"', 'dec_",Name,"'}"]); - _Other -> - emit({" fun 'dec_",ObjName,"'/4"}) - end, - emit({";",nl}), - emit_default_getdec(ObjSetName,UniqueName), - emit({".",nl,nl}); -gen_objset_dec(_,ObjSetName,_UniqueName,['EXTENSIONMARK'],_ClName,_ClFields, - _NthObj) -> - emit({"'getdec_",ObjSetName,"'(_, _) ->",nl}), - emit({indent(3),"fun(_, Bytes, _, _) ->",nl}), - emit({indent(6),"Len = case Bytes of",nl,indent(9), - "Bin when is_binary(Bin) -> size(Bin);",nl,indent(9), - "_ -> length(Bytes)",nl,indent(6),"end,"}), - emit({indent(6),"{Bytes,[],Len}",nl}), - emit({indent(3),"end.",nl,nl}), - ok; -gen_objset_dec(Erule,ObjSetName,UniqueName, - ['EXTENSIONMARK','EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj) -> - gen_objset_dec(Erule,ObjSetName,UniqueName,['EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj); -gen_objset_dec(Erule,ObjSetName,UniqueName,['EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj) -> - gen_objset_dec(Erule,ObjSetName,UniqueName,Rest++['EXTENSIONMARK'], - ClName,ClFields,NthObj); -gen_objset_dec(_,_,_,[],_,_,_) -> - ok. - -emit_default_getdec(ObjSetName,UniqueName) -> - emit(["'getdec_",ObjSetName,"'(",{asis,UniqueName},", ErrV) ->",nl]), - emit([indent(3),"fun(C,V,_,_) -> exit({{component,C},{value,V}, {unique_name_and_value,",{asis,UniqueName},", ErrV}}) end"]). - -gen_inlined_dec_funs(Erules,Fields,[{typefield,Name,Prop}|Rest], - ObjSetName,NthObj) -> - DecProp = case Prop of - 'OPTIONAL' -> opt_or_default; - {'DEFAULT',_} -> opt_or_default; - _ -> mandatory - end, - CurrMod = get(currmod), - InternalDefFunName = [NthObj,Name,ObjSetName], - case lists:keysearch(Name,1,Fields) of - {value,{_,Type}} when is_record(Type,type) -> - emit({indent(3),"fun(Type, Bytes, TagIn, _RestPrimFieldName) ->", - nl,indent(6),"case Type of",nl}), - N=emit_inner_of_decfun(Erules,Type,DecProp,InternalDefFunName), - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj+N); - {value,{_,Type}} when is_record(Type,typedef) -> - emit({indent(3),"fun(Type, Bytes, TagIn, _RestPrimFieldName) ->", - nl,indent(6),"case Type of",nl}), - emit({indent(9),{asis,Name}," ->",nl}), - N=emit_inner_of_decfun(Erules,Type,DecProp,InternalDefFunName), - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj+N); - {value,{_,#'Externaltypereference'{module=M,type=T}}} -> - #typedef{typespec=Type} = asn1_db:dbget(M,T), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit({indent(3),"fun(Type, Bytes, TagIn, _RestPrimFieldName) ->", - nl,indent(6),"case Type of",nl}), - emit({indent(9),{asis,Name}," ->",nl}), - if - M == CurrMod -> - emit([indent(12),"'dec_",T,"'(Bytes, ",DecProp, - ", TagIn ++ ",{asis,Tag},")"]); - true -> - emit([indent(12),"'",M,"':'dec_",T,"'(Bytes, ", - DecProp,", TagIn ++ ",{asis,Tag},")"]) - end, - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj); - false -> - emit([indent(3),"fun(Type, Bytes, TagIn, _RestPrimFieldName) ->", - nl,indent(6),"case Type of",nl, - indent(9),{asis,Name}," ->",nl, - indent(12),"Len = case Bytes of",nl, - indent(15),"B when is_binary(B) -> size(B);",nl, - indent(15),"_ -> length(Bytes)",nl, - indent(12),"end,",nl, - indent(12),"{Bytes,[],Len}"]), - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj) - end; -gen_inlined_dec_funs(Erules,Fields,[_H|Rest],ObjSetName,NthObj) -> - gen_inlined_dec_funs(Erules,Fields,Rest,ObjSetName,NthObj); -gen_inlined_dec_funs(_,_,[],_,NthObj) -> - NthObj. - -gen_inlined_dec_funs1(Erules,Fields,[{typefield,Name,Prop}|Rest], - ObjSetName,NthObj) -> - DecProp = case Prop of - 'OPTIONAL' -> opt_or_default; - {'DEFAULT',_} -> opt_or_default; - _ -> mandatory - end, - CurrMod = get(currmod), - InternalDefFunName = [NthObj,Name,ObjSetName], - N= - case lists:keysearch(Name,1,Fields) of - {value,{_,Type}} when is_record(Type,type) -> - emit({";",nl}), - emit_inner_of_decfun(Erules,Type,DecProp,InternalDefFunName); - {value,{_,Type}} when is_record(Type,typedef) -> - emit({";",nl,indent(9),{asis,Name}," ->",nl}), - emit_inner_of_decfun(Erules,Type,DecProp,InternalDefFunName); - {value,{_,#'Externaltypereference'{module=M,type=T}}} -> - #typedef{typespec=Type} = asn1_db:dbget(M,T), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit({";",nl,indent(9),{asis,Name}," ->",nl}), - if - M == CurrMod -> - emit([indent(12),"'dec_",T,"'(Bytes, ",DecProp, - ", TagIn ++ ",{asis,Tag},")"]); - true -> - emit([indent(12),"'",M,"':'dec_",T,"'(Bytes, ", - DecProp,", TagIn ++ ",{asis,Tag},")"]) - end, - 0; - false -> - emit([";",nl, - indent(9),{asis,Name}," ->",nl, - indent(12),"Len = case Bytes of",nl, - indent(15),"B when is_binary(B) -> size(B);",nl, - indent(15),"_ -> length(Bytes)",nl, - indent(12),"end,",nl, - indent(12),"{Bytes,[],Len}"]), - 0 - end, - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj+N); -gen_inlined_dec_funs1(Erules,Fields,[_H|Rest],ObjSetName,NthObj)-> - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj); -gen_inlined_dec_funs1(_,_,[],_,NthObj) -> - emit({nl,indent(6),"end",nl}), - emit({indent(3),"end"}), - NthObj. - -emit_inner_of_decfun(Erules,#typedef{name={ExtName,Name},typespec=Type}, - Prop,InternalDefFunName) -> - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case {ExtName,Name} of - {primitive,bif} -> - emit(indent(12)), - gen_dec_prim(Erules,Type,"Bytes",Tag,"TagIn",no_length, - ?PRIMITIVE,Prop), - 0; - {constructed,bif} -> - emit({indent(12),"'dec_", - asn1ct_gen:list2name(InternalDefFunName),"'(Bytes, ",Prop, - ", TagIn ++ ",{asis,Tag},")"}), - 1; - _ -> - emit({indent(12),"'",ExtName,"':'dec_",Name,"'(Bytes, ",Prop, - ", TagIn ++ ",{asis,Tag},")"}), - 0 - end; -emit_inner_of_decfun(_,#typedef{name=Name,typespec=Type},Prop,_) -> - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit({indent(12),"'dec_",Name,"'(Bytes, ",Prop,", TagIn ++ ", - {asis,Tag},")"}), - 0; -emit_inner_of_decfun(Erules,Type,Prop,_) when is_record(Type,type) -> - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - CurrMod = get(currmod), - Def = Type#type.def, - InnerType = asn1ct_gen:get_inner(Def), - WhatKind = asn1ct_gen:type(InnerType), - case WhatKind of - {primitive,bif} -> - emit({indent(9),Def," ->",nl,indent(12)}), - gen_dec_prim(Erules,Type,"Bytes",Tag,"TagIn",no_length, - ?PRIMITIVE,Prop); -% TRef when is_record(TRef,typereference) -> -% T = TRef#typereference.val, -% emit({indent(9),T," ->",nl,indent(12),"'dec_",T,"'(Val)"}); - #'Externaltypereference'{module=CurrMod,type=T} -> - emit({indent(9),T," ->",nl,indent(12),"'dec_",T, - "'(Bytes, ",Prop,", TagIn ++ ",{asis,Tag},")"}); - #'Externaltypereference'{module=ExtMod,type=T} -> - emit({indent(9),T," ->",nl,indent(12),ExtMod,":'dec_", - T,"'(Bytes, ",Prop,", TagIn ++ ",{asis,Tag},")"}) - end, - 0. - - -gen_internal_funcs(_,[]) -> - ok; -gen_internal_funcs(Erules,[TypeDef|Rest]) -> - gen_encode_user(Erules,TypeDef), - emit({"'dec_",TypeDef#typedef.name,"'(Bytes, ", - unused_optormand_var("OptOrMand",(TypeDef#typedef.typespec)#type.def),", TagIn) ->",nl}), - gen_decode_user(Erules,TypeDef), - gen_internal_funcs(Erules,Rest). - - -dbdec(Type) -> - demit({"io:format(\"decoding: ",{asis,Type},"~w~n\",[Bytes]),",nl}). - - -decode_class('UNIVERSAL') -> - ?UNIVERSAL; -decode_class('APPLICATION') -> - ?APPLICATION; -decode_class('CONTEXT') -> - ?CONTEXT; -decode_class('PRIVATE') -> - ?PRIVATE. - -decode_type('BOOLEAN') -> 1; -decode_type('INTEGER') -> 2; -decode_type('BIT STRING') -> 3; -decode_type('OCTET STRING') -> 4; -decode_type('NULL') -> 5; -decode_type('OBJECT IDENTIFIER') -> 6; -decode_type('ObjectDescriptor') -> 7; -decode_type('EXTERNAL') -> 8; -decode_type('REAL') -> 9; -decode_type('ENUMERATED') -> 10; -decode_type('EMBEDDED_PDV') -> 11; -decode_type('UTF8String') -> 12; -decode_type('RELATIVE-OID') -> 13; -decode_type('SEQUENCE') -> 16; -decode_type('SEQUENCE OF') -> 16; -decode_type('SET') -> 17; -decode_type('SET OF') -> 17; -decode_type('NumericString') -> 18; -decode_type('PrintableString') -> 19; -decode_type('TeletexString') -> 20; -decode_type('T61String') -> 20; -decode_type('VideotexString') -> 21; -decode_type('IA5String') -> 22; -decode_type('UTCTime') -> 23; -decode_type('GeneralizedTime') -> 24; -decode_type('GraphicString') -> 25; -decode_type('VisibleString') -> 26; -decode_type('GeneralString') -> 27; -decode_type('UniversalString') -> 28; -decode_type('BMPString') -> 30; -decode_type('CHOICE') -> 'CHOICE'; % choice gets the tag from the actual alternative -decode_type(Else) -> exit({error,{asn1,{unrecognized_type,Else}}}). - -add_removed_bytes() -> - asn1ct_name:delete(rb), - add_removed_bytes(asn1ct_name:all(rb)). - -add_removed_bytes([H,T1|T]) -> - emit({{var,H},"+"}), - add_removed_bytes([T1|T]); -add_removed_bytes([H|T]) -> - emit({{var,H}}), - add_removed_bytes(T); -add_removed_bytes([]) -> - true. - -mkfuncname(WhatKind,DecOrEnc) -> - case WhatKind of - #'Externaltypereference'{module=Mod,type=EType} -> - CurrMod = get(currmod), - case CurrMod of - Mod -> - lists:concat(["'",DecOrEnc,"_",EType,"'"]); - _ -> -% io:format("CurrMod: ~p, Mod: ~p~n",[CurrMod,Mod]), - lists:concat(["'",Mod,"':'",DecOrEnc,"_",EType,"'"]) - end; - #'typereference'{val=EType} -> - lists:concat(["'",DecOrEnc,"_",EType,"'"]); - 'ASN1_OPEN_TYPE' -> - lists:concat(["'",DecOrEnc,"_",WhatKind,"'"]) - - end. - -optionals(L) -> optionals(L,[],1). - -optionals([{'EXTENSIONMARK',_,_}|Rest],Acc,Pos) -> - optionals(Rest,Acc,Pos); % optionals in extension are currently not handled -optionals([#'ComponentType'{name=Name,prop='OPTIONAL'}|Rest],Acc,Pos) -> - optionals(Rest,[{Name,Pos}|Acc],Pos+1); -optionals([#'ComponentType'{name=Name,prop={'DEFAULT',_}}|Rest],Acc,Pos) -> - optionals(Rest,[{Name,Pos}|Acc],Pos+1); -optionals([#'ComponentType'{}|Rest],Acc,Pos) -> - optionals(Rest,Acc,Pos+1); -optionals([],Acc,_) -> - lists:reverse(Acc). - -get_constraint(C,Key) -> - case lists:keysearch(Key,1,C) of - false -> - no; - {value,{_,V}} -> - V - end. - -%% if the original option was ber and it has been wrapped to ber_bin -%% turn it back to ber -re_wrap_erule(ber_bin) -> - case get(encoding_options) of - Options when is_list(Options) -> - case lists:member(ber,Options) of - true -> ber; - _ -> ber_bin - end; - _ -> ber_bin - end; -re_wrap_erule(Erule) -> - Erule. - -is_already_generated(Operation,Name) -> - case get(class_default_type) of - undefined -> - put(class_default_type,[{Operation,Name}]), - false; - GeneratedList -> - case lists:member({Operation,Name},GeneratedList) of - true -> - true; - false -> - put(class_default_type,[{Operation,Name}|GeneratedList]), - false - end - end. - -get_class_fields(#classdef{typespec=ObjClass}) -> - ObjClass#objectclass.fields; -get_class_fields(#objectclass{fields=Fields}) -> - Fields; -get_class_fields(_) -> - []. - -get_object_field(Name,ObjectFields) -> - case lists:keysearch(Name,1,ObjectFields) of - {value,Field} -> Field; - false -> false - end. - -%% For BER the ExtensionAdditionGroup notation has no impact on the encoding/decoding -%% and therefore we only filter away the ExtensionAdditionGroup start and end markers -%% -extaddgroup2sequence(ExtList) when is_list(ExtList) -> - lists:filter(fun(#'ExtensionAdditionGroup'{}) -> - false; - ('ExtensionAdditionGroupEnd') -> - false; - (_) -> - true - end, ExtList). diff --git a/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl b/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl index 3ccfca3784..00c3dd98b2 100644 --- a/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl +++ b/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl @@ -116,22 +116,7 @@ gen_encode(Erules,Typename,Type) when is_record(Type,type) -> end, case lists:member(InnerType,['SET','SEQUENCE']) of true -> - case get(asn_keyed_list) of - true -> - CompList = - case Type#type.def of - #'SEQUENCE'{components=Cl} -> Cl; - #'SET'{components=Cl} -> Cl - end, - emit([nl,"'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn",ObjFun, - ") when is_list(Val) ->",nl]), - emit([" 'enc_",asn1ct_gen:list2name(Typename), - "'(?RT_BER:fixoptionals(", - {asis,optionals(CompList)}, - ",Val), TagIn",ObjFun,");",nl,nl]); - _ -> true - end; + true; _ -> emit([nl,"'enc_",asn1ct_gen:list2name(Typename), "'({'",asn1ct_gen:list2name(Typename), @@ -175,22 +160,7 @@ gen_encode_user(Erules,D) when is_record(D,typedef) -> case lists:member(InnerType,['SET','SEQUENCE']) of true -> - case get(asn_keyed_list) of - true -> - CompList = - case Type#type.def of - #'SEQUENCE'{components=Cl} -> Cl; - #'SET'{components=Cl} -> Cl - end, - - emit([nl,"'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn) when is_list(Val) ->",nl]), - emit([" 'enc_",asn1ct_gen:list2name(Typename), - "'(?RT_BER:fixoptionals(", - {asis,optionals(CompList)}, - ",Val), TagIn);",nl,nl]); - _ -> true - end; + true; _ -> emit({nl,"'enc_",asn1ct_gen:list2name(Typename), "'({'",asn1ct_gen:list2name(Typename),"',Val}, TagIn) ->",nl}), @@ -1504,7 +1474,7 @@ gen_objset_dec(Erules,ObjSetName,_UniqueName,['EXTENSIONMARK'],_ClName, emit([indent(2),"fun(_,Bytes, _RestPrimFieldName) ->",nl]), case Erules of - ber_bin_v2 -> + ber -> emit([indent(4),"case Bytes of",nl, indent(6),"Bin when is_binary(Bin) -> ",nl, indent(8),"Bin;",nl, @@ -1772,19 +1742,6 @@ mkfuncname(WhatKind,DecOrEnc) -> end. -optionals(L) -> optionals(L,[],1). - -optionals([{'EXTENSIONMARK',_,_}|Rest],Acc,Pos) -> - optionals(Rest,Acc,Pos); % optionals in extension are currently not handled -optionals([#'ComponentType'{name=Name,prop='OPTIONAL'}|Rest],Acc,Pos) -> - optionals(Rest,[{Name,Pos}|Acc],Pos+1); -optionals([#'ComponentType'{name=Name,prop={'DEFAULT',_}}|Rest],Acc,Pos) -> - optionals(Rest,[{Name,Pos}|Acc],Pos+1); -optionals([#'ComponentType'{}|Rest],Acc,Pos) -> - optionals(Rest,Acc,Pos+1); -optionals([],Acc,_) -> - lists:reverse(Acc). - get_constraint(C,Key) -> case lists:keysearch(Key,1,C) of false -> diff --git a/lib/asn1/src/asn1ct_gen_per.erl b/lib/asn1/src/asn1ct_gen_per.erl index bd5b81991d..eb4cfdccc6 100644 --- a/lib/asn1/src/asn1ct_gen_per.erl +++ b/lib/asn1/src/asn1ct_gen_per.erl @@ -25,6 +25,7 @@ -include("asn1_records.hrl"). %-compile(export_all). +-export([gen_dec_imm/2]). -export([pgen/4,gen_dec_prim/3,gen_encode_prim/4]). -export([gen_obj_code/3,gen_objectset_code/2]). -export([gen_decode/2, gen_decode/3]). @@ -146,16 +147,18 @@ gen_encode_prim(Erules,D,DoTag,Value) when is_record(D,type) -> case D#type.def of 'INTEGER' -> emit({"?RT_PER:encode_integer(", %fel - {asis,effective_constraint(integer,Constraint)},",",Value,")"}); + {asis,asn1ct_imm:effective_constraint(integer,Constraint)}, + ",",Value,")"}); {'INTEGER',NamedNumberList} -> emit({"?RT_PER:encode_integer(", - {asis,effective_constraint(integer,Constraint)},",",Value,",", + {asis,asn1ct_imm:effective_constraint(integer,Constraint)}, + ",",Value,",", {asis,NamedNumberList},")"}); {'ENUMERATED',{Nlist1,Nlist2}} -> NewList = lists:concat([[{0,X}||{X,_} <- Nlist1],['EXT_MARK'],[{1,X}||{X,_} <- Nlist2]]), NewC = [{'ValueRange',{0,length(Nlist1)-1}}], case Erules of - uper_bin -> + uper -> emit(["case ",Value," of",nl]); _ -> emit(["case (case ",Value," of {_,",{curr,enumval},"}-> ", @@ -168,7 +171,7 @@ gen_encode_prim(Erules,D,DoTag,Value) when is_record(D,type) -> NewList = [X||{X,_} <- NamedNumberList], NewC = [{'ValueRange',{0,length(NewList)-1}}], case Erules of - uper_bin -> + uper -> emit(["case ",Value," of",nl]); _ -> emit(["case (case ",Value," of {_,",{curr,enumval}, @@ -184,7 +187,7 @@ gen_encode_prim(Erules,D,DoTag,Value) when is_record(D,type) -> {asis,Constraint},",",Value,",", {asis,NamedNumberList},")"}); 'NULL' -> - emit({"?RT_PER:encode_null(",Value,")"}); + emit("[]"); 'OBJECT IDENTIFIER' -> emit({"?RT_PER:encode_object_identifier(",Value,")"}); 'RELATIVE-OID' -> @@ -274,7 +277,7 @@ emit_enc_enumerated_cases(Erule, C, [H1,H2|T], Count) -> -emit_enc_enumerated_case(uper_bin,_C, {asn1_enum,High}, _) -> +emit_enc_enumerated_case(uper,_C, {asn1_enum,High}, _) -> emit([ "{asn1_enum,EnumV} when is_integer(EnumV), EnumV > ",High," -> ", "[<<1:1>>,?RT_PER:encode_small_number(EnumV)]"]); @@ -284,118 +287,27 @@ emit_enc_enumerated_case(_Per,_C, {asn1_enum,High}, _) -> "[{bit,1},?RT_PER:encode_small_number(EnumV)]"]); emit_enc_enumerated_case(_Erule, _C, 'EXT_MARK', _Count) -> true; -emit_enc_enumerated_case(uper_bin,_C, {1,EnumName}, Count) -> +emit_enc_enumerated_case(uper,_C, {1,EnumName}, Count) -> emit(["'",EnumName,"' -> [<<1:1>>,?RT_PER:encode_small_number(",Count,")]"]); emit_enc_enumerated_case(_Per,_C, {1,EnumName}, Count) -> emit(["'",EnumName,"' -> [{bit,1},?RT_PER:encode_small_number(",Count,")]"]); -emit_enc_enumerated_case(uper_bin,C, {0,EnumName}, Count) -> +emit_enc_enumerated_case(uper,C, {0,EnumName}, Count) -> emit(["'",EnumName,"' -> [<<0:1>>,?RT_PER:encode_integer(",{asis,C},", ",Count,")]"]); emit_enc_enumerated_case(_Per,C, {0,EnumName}, Count) -> emit(["'",EnumName,"' -> [{bit,0},?RT_PER:encode_integer(",{asis,C},", ",Count,")]"]); emit_enc_enumerated_case(_Erule, C, EnumName, Count) -> emit(["'",EnumName,"' -> ?RT_PER:encode_integer(",{asis,C},", ",Count,")"]). -%% effective_constraint(Type,C) -%% Type = atom() -%% C = [C1,...] -%% C1 = {'SingleValue',SV} | {'ValueRange',VR} | {atom(),term()} -%% SV = integer() | [integer(),...] -%% VR = {Lb,Ub} -%% Lb = 'MIN' | integer() -%% Ub = 'MAX' | integer() -%% Returns a single value if C only has a single value constraint, and no -%% value range constraints, that constrains to a single value, otherwise -%% returns a value range that has the lower bound set to the lowest value -%% of all single values and lower bound values in C and the upper bound to -%% the greatest value. -effective_constraint(integer,[C={{_,_},_}|_Rest]) -> % extension - [C]; %% [C|effective_constraint(integer,Rest)]; XXX what is possible ??? -effective_constraint(integer,C) -> - SVs = get_constraints(C,'SingleValue'), - SV = effective_constr('SingleValue',SVs), - VRs = get_constraints(C,'ValueRange'), - VR = effective_constr('ValueRange',VRs), - greatest_common_range(SV,VR). - -effective_constr(_,[]) -> - []; -effective_constr('SingleValue',List) -> - SVList = lists:flatten(lists:map(fun(X)->element(2,X)end,List)), - %% Sort and remove duplicates before generating SingleValue or ValueRange - %% In case of ValueRange, also check for 'MIN and 'MAX' - case lists:usort(SVList) of - [N] -> - [{'SingleValue',N}]; - L when is_list(L) -> - [{'ValueRange',{least_Lb(L),greatest_Ub(L)}}] - end; -effective_constr('ValueRange',List) -> - LBs = lists:map(fun({_,{Lb,_}})-> Lb end,List), - UBs = lists:map(fun({_,{_,Ub}})-> Ub end,List), - Lb = least_Lb(LBs), - [{'ValueRange',{Lb,lists:max(UBs)}}]. - -greatest_common_range([],VR) -> - VR; -greatest_common_range(SV,[]) -> - SV; -greatest_common_range(SV,VR) -> - greatest_common_range2(mk_vr(SV),mk_vr(VR)). -greatest_common_range2({_,Int},{'MIN',Ub}) when is_integer(Int), - Int > Ub -> - [{'ValueRange',{'MIN',Int}}]; -greatest_common_range2({_,Int},{Lb,Ub}) when is_integer(Int), - Int < Lb -> - [{'ValueRange',{Int,Ub}}]; -greatest_common_range2({_,Int},VR={_Lb,_Ub}) when is_integer(Int) -> - [{'ValueRange',VR}]; -greatest_common_range2({_,L},{Lb,Ub}) when is_list(L) -> - Min = least_Lb([Lb|L]), - Max = greatest_Ub([Ub|L]), - [{'ValueRange',{Min,Max}}]; -greatest_common_range2({Lb1,Ub1},{Lb2,Ub2}) -> - Min = least_Lb([Lb1,Lb2]), - Max = greatest_Ub([Ub1,Ub2]), - [{'ValueRange',{Min,Max}}]. - -mk_vr([{Type,I}]) when is_atom(Type), is_integer(I) -> - {I,I}; -mk_vr([{Type,{Lb,Ub}}]) when is_atom(Type) -> - {Lb,Ub}; -mk_vr(Other) -> - Other. - -least_Lb(L) -> - case lists:member('MIN',L) of - true -> 'MIN'; - _ -> lists:min(L) - end. - -greatest_Ub(L) -> - case lists:member('MAX',L) of - true -> 'MAX'; - _ -> lists:max(L) - end. - - -get_constraints(L=[{Key,_}],Key) -> - L; -get_constraints([],_) -> - []; -get_constraints(C,Key) -> - {value,L} = keysearch_allwithkey(Key,1,C,[]), - L. - -keysearch_allwithkey(Key,Ix,C,Acc) -> - case lists:keysearch(Key,Ix,C) of - false -> - {value,Acc}; - {value,T} -> - RestC = lists:delete(T,C), - keysearch_allwithkey(Key,Ix,RestC,[T|Acc]) +get_constraint([{Key,V}], Key) -> + V; +get_constraint([], _) -> + no; +get_constraint(C, Key) -> + case lists:keyfind(Key, 1, C) of + false -> no; + {Key,V} -> V end. - %% Object code generating for encoding and decoding %% ------------------------------------------------ @@ -442,7 +354,7 @@ gen_encode_objectfields(Erule,ClassName,[{typefield,Name,OptOrMand}|Rest], {false,'OPTIONAL'} -> EmitFuncClause("Val"), case Erule of - uper_bin -> + uper -> emit(" Val"); _ -> emit(" [{octets,Val}]") @@ -833,7 +745,7 @@ gen_objset_enc(Erule,ObjSetName,_UniqueName,['EXTENSIONMARK'],_ClName, emit({"'getenc_",ObjSetName,"'(_, _) ->",nl}), emit({indent(3),"fun(_, Val, _) ->",nl}), case Erule of - uper_bin -> + uper -> emit([indent(6),"Val",nl]); _ -> emit([indent(6),"[{octets,Val}]",nl]) @@ -883,7 +795,7 @@ gen_inlined_enc_funs(Erule,Fields,[{typefield,Name,_}|Rest],ObjSetName,NthObj) - emit({indent(9),{asis,Name}," ->",nl}), emit([indent(12),"'",M,"'",":'enc_",T,"'(Val)"]), gen_inlined_enc_funs1(Erule,Fields,Rest,ObjSetName,NthObj,[]); - false when Erule == uper_bin -> + false when Erule =:= uper -> emit([indent(3),"fun(Type,Val,_) ->",nl, indent(6),"case Type of",nl, indent(9),{asis,Name}," -> Val",nl]), @@ -921,7 +833,7 @@ gen_inlined_enc_funs1(Erule,Fields,[{typefield,Name,_}|Rest],ObjSetName, emit({";",nl,indent(9),{asis,Name}," ->",nl}), emit([indent(12),"'",M,"'",":'enc_",T,"'(Val)"]), {Acc,0}; - false when Erule == uper_bin -> + false when Erule =:= uper -> emit([";",nl, indent(9),{asis,Name}," -> ",nl, "Val",nl]), @@ -1228,19 +1140,43 @@ gen_decode_user(Erules,D) when is_record(D,typedef) -> exit({error,{asn1,{unknown,Other}}}) end. +gen_dec_imm(Erule, #type{def=Name,constraint=C}) -> + Aligned = case Erule of + uper -> false; + per -> true + end, + gen_dec_imm_1(Name, C, Aligned). + +gen_dec_imm_1('BOOLEAN', _Constr, _Aligned) -> + asn1ct_imm:per_dec_boolean(); +gen_dec_imm_1({'ENUMERATED',{Base,Ext}}, _Constr, Aligned) -> + asn1ct_imm:per_dec_enumerated(Base, Ext, Aligned); +gen_dec_imm_1({'ENUMERATED',NamedNumberList}, _Constr, Aligned) -> + asn1ct_imm:per_dec_enumerated(NamedNumberList, Aligned); +gen_dec_imm_1('INTEGER', Constr, Aligned) -> + asn1ct_imm:per_dec_integer(Constr, Aligned); +gen_dec_imm_1({'INTEGER',NamedNumberList}, Constraint, Aligned) -> + asn1ct_imm:per_dec_named_integer(Constraint, + NamedNumberList, + Aligned); +gen_dec_imm_1('OCTET STRING', Constraint, Aligned) -> + SzConstr = get_constraint(Constraint, 'SizeConstraint'), + Imm = asn1ct_imm:per_dec_octet_string(SzConstr, Aligned), + {convert,binary_to_list,Imm}; +gen_dec_imm_1(_, _, _) -> no. + +gen_dec_prim(Erule, Type, BytesVar) -> + case gen_dec_imm(Erule, Type) of + no -> + gen_dec_prim_1(Erule, Type, BytesVar); + Imm -> + asn1ct_imm:dec_code_gen(Imm, BytesVar) + end. -gen_dec_prim(Erules,Att,BytesVar) -> - Typename = Att#type.def, - Constraint = Att#type.constraint, +gen_dec_prim_1(Erule, + #type{def=Typename,constraint=Constraint}=Att, + BytesVar) -> case Typename of - 'INTEGER' -> - emit({"?RT_PER:decode_integer(",BytesVar,",", - {asis,effective_constraint(integer,Constraint)},")"}); - {'INTEGER',NamedNumberList} -> - emit({"?RT_PER:decode_integer(",BytesVar,",", - {asis,effective_constraint(integer,Constraint)},",", - {asis,NamedNumberList},")"}); - 'REAL' -> emit({"?RT_PER:decode_real(",BytesVar,")"}); @@ -1256,8 +1192,7 @@ gen_dec_prim(Erules,Att,BytesVar) -> {asis,NamedNumberList},")"}) end; 'NULL' -> - emit({"?RT_PER:decode_null(", - BytesVar,")"}); + emit({"{'NULL',",BytesVar,"}"}); 'OBJECT IDENTIFIER' -> emit({"?RT_PER:decode_object_identifier(", BytesVar,")"}); @@ -1267,24 +1202,6 @@ gen_dec_prim(Erules,Att,BytesVar) -> 'ObjectDescriptor' -> emit({"?RT_PER:decode_ObjectDescriptor(", BytesVar,")"}); - {'ENUMERATED',{NamedNumberList1,NamedNumberList2}} -> - NewTup = {list_to_tuple([X||{X,_} <- NamedNumberList1]), - list_to_tuple([X||{X,_} <- NamedNumberList2])}, - NewC = [{'ValueRange',{0,size(element(1,NewTup))-1}}], - emit({"?RT_PER:decode_enumerated(",BytesVar,",", - {asis,NewC},",", - {asis,NewTup},")"}); - {'ENUMERATED',NamedNumberList} -> - NewTup = list_to_tuple([X||{X,_} <- NamedNumberList]), - NewC = [{'ValueRange',{0,size(NewTup)-1}}], - emit({"?RT_PER:decode_enumerated(",BytesVar,",", - {asis,NewC},",", - {asis,NewTup},")"}); - 'BOOLEAN'-> - emit({"?RT_PER:decode_boolean(",BytesVar,")"}); - 'OCTET STRING' -> - emit({"?RT_PER:decode_octet_string(",BytesVar,",", - {asis,Constraint},")"}); 'NumericString' -> emit({"?RT_PER:decode_NumericString(",BytesVar,",", {asis,Constraint},")"}); @@ -1323,7 +1240,7 @@ gen_dec_prim(Erules,Att,BytesVar) -> 'UTF8String' -> emit({"?RT_PER:decode_UTF8String(",BytesVar,")"}); 'ANY' -> - case Erules of + case Erule of per -> emit(["fun() -> {XTerm,YTermXBytes} = ?RT_PER:decode_open_type(",BytesVar,",",{asis,Constraint}, "), {binary_to_list(XTerm),XBytes} end ()"]); _ -> @@ -1345,7 +1262,7 @@ gen_dec_prim(Erules,Att,BytesVar) -> emit([" {YTerm,_} = dec_",Tname,"(XTerm,mandatory),",nl]), emit([" {YTerm,XBytes} end(",BytesVar,")"]); _ -> - case Erules of + case Erule of per -> emit(["fun() -> {XTerm,XBytes} = ?RT_PER:decode_open_type(",BytesVar,", []), {binary_to_list(XTerm),XBytes} end()"]); _ -> @@ -1353,11 +1270,11 @@ gen_dec_prim(Erules,Att,BytesVar) -> end end; #'ObjectClassFieldType'{} -> - case asn1ct_gen:get_inner(Att#type.def) of + case asn1ct_gen:get_inner(Typename) of {fixedtypevaluefield,_,InnerType} -> - gen_dec_prim(Erules,InnerType,BytesVar); + gen_dec_prim(Erule, InnerType, BytesVar); T -> - gen_dec_prim(Erules,Att#type{def=T},BytesVar) + gen_dec_prim(Erule, Att#type{def=T}, BytesVar) end; Other -> exit({'cant decode' ,Other}) diff --git a/lib/asn1/src/asn1ct_gen_per_rt2ct.erl b/lib/asn1/src/asn1ct_gen_per_rt2ct.erl index 16eec92847..ecd212c3e3 100644 --- a/lib/asn1/src/asn1ct_gen_per_rt2ct.erl +++ b/lib/asn1/src/asn1ct_gen_per_rt2ct.erl @@ -176,7 +176,7 @@ gen_encode_prim(Erules,D,DoTag,Value) when is_record(D,type) -> {asis,NamedNumberList},")"}) end; 'NULL' -> - emit({"?RT_PER:encode_null(",Value,")"}); + emit("[]"); 'OBJECT IDENTIFIER' -> emit({"?RT_PER:encode_object_identifier(",Value,")"}); 'RELATIVE-OID' -> @@ -450,17 +450,6 @@ emit_enc_octet_string(_Erules,Constraint,Value) -> emit({" ?RT_PER:encode_octet_string(",{asis,C},",false,",Value,")",nl}) end. -emit_dec_octet_string(Constraint,BytesVar) -> - case get_constraint(Constraint,'SizeConstraint') of - 0 -> - emit({" {[],",BytesVar,"}",nl}); - {_,0} -> - emit({" {[],",BytesVar,"}",nl}); - C -> - emit({" ?RT_PER:decode_octet_string(",BytesVar,",", - {asis,C},",false)",nl}) - end. - emit_enc_integer_case(Value) -> case get(component_type) of {true,#'ComponentType'{prop=Prop}} -> @@ -624,23 +613,6 @@ get_constraint(C,Key) -> V end. -get_constraints(L=[{Key,_}],Key) -> - L; -get_constraints([],_) -> - []; -get_constraints(C,Key) -> - {value,L} = keysearch_allwithkey(Key,1,C,[]), - L. - -keysearch_allwithkey(Key,Ix,C,Acc) -> - case lists:keysearch(Key,Ix,C) of - false -> - {value,Acc}; - {value,T} -> - RestC = lists:delete(T,C), - keysearch_allwithkey(Key,Ix,RestC,[T|Acc]) - end. - %% effective_constraint(Type,C) %% Type = atom() %% C = [C1,...] @@ -657,69 +629,9 @@ keysearch_allwithkey(Key,Ix,C,Acc) -> effective_constraint(integer,[C={{_,_},_}|_Rest]) -> % extension [C]; %% [C|effective_constraint(integer,Rest)]; XXX what is possible ??? effective_constraint(integer,C) -> - SVs = get_constraints(C,'SingleValue'), - SV = effective_constr('SingleValue',SVs), - VRs = get_constraints(C,'ValueRange'), - VR = effective_constr('ValueRange',VRs), - CRange = greatest_common_range(SV,VR), - pre_encode(integer,CRange); + pre_encode(integer, asn1ct_imm:effective_constraint(integer, C)); effective_constraint(bitstring,C) -> - get_constraint(C,'SizeConstraint'). - -effective_constr(_,[]) -> - []; -effective_constr('SingleValue',List) -> - SVList = lists:flatten(lists:map(fun(X)->element(2,X)end,List)), - %% Sort and remove duplicates before generating SingleValue or ValueRange - %% In case of ValueRange, also check for 'MIN and 'MAX' - case lists:usort(SVList) of - [N] -> - [{'SingleValue',N}]; - L when is_list(L) -> - [{'ValueRange',{least_Lb(L),greatest_Ub(L)}}] - end; -effective_constr('ValueRange',List) -> - LBs = lists:map(fun({_,{Lb,_}})-> Lb end,List), - UBs = lists:map(fun({_,{_,Ub}})-> Ub end,List), - Lb = least_Lb(LBs), - [{'ValueRange',{Lb,lists:max(UBs)}}]. - -greatest_common_range([],VR) -> - VR; -greatest_common_range(SV,[]) -> - SV; -greatest_common_range([{_,Int}],[{_,{'MIN',Ub}}]) when is_integer(Int), - Int > Ub -> - [{'ValueRange',{'MIN',Int}}]; -greatest_common_range([{_,Int}],[{_,{Lb,Ub}}]) when is_integer(Int), - Int < Lb -> - [{'ValueRange',{Int,Ub}}]; -greatest_common_range([{_,Int}],VR=[{_,{_Lb,_Ub}}]) when is_integer(Int) -> - VR; -greatest_common_range([{_,L}],[{_,{Lb,Ub}}]) when is_list(L) -> - Min = least_Lb([Lb|L]), - Max = greatest_Ub([Ub|L]), - [{'ValueRange',{Min,Max}}]; -greatest_common_range([{_,{Lb1,Ub1}}],[{_,{Lb2,Ub2}}]) -> - Min = least_Lb([Lb1,Lb2]), - Max = greatest_Ub([Ub1,Ub2]), - [{'ValueRange',{Min,Max}}]. - - -least_Lb(L) -> - case lists:member('MIN',L) of - true -> 'MIN'; - _ -> lists:min(L) - end. - -greatest_Ub(L) -> - case lists:member('MAX',L) of - true -> 'MAX'; - _ -> lists:max(L) - end. - - - + asn1ct_imm:effective_constraint(bitstring, C). pre_encode(integer,[]) -> []; @@ -1586,17 +1498,9 @@ gen_dec_prim(Erules,Att,BytesVar) -> Constraint = Att#type.constraint, case Typename of 'INTEGER' -> - EffectiveConstr = effective_constraint(integer,Constraint), - emit_dec_integer(EffectiveConstr,BytesVar); -% emit({"?RT_PER:decode_integer(",BytesVar,",", -% {asis,EffectiveConstr},")"}); - {'INTEGER',NamedNumberList} -> - EffectiveConstr = effective_constraint(integer,Constraint), - emit_dec_integer(EffectiveConstr,BytesVar,NamedNumberList); -% emit({"?RT_PER:decode_integer(",BytesVar,",", -% {asis,EffectiveConstr},",", -% {asis,NamedNumberList},")"}); - + asn1ct_gen_per:gen_dec_prim(Erules, Att, BytesVar); + {'INTEGER',_NamedNumberList} -> + asn1ct_gen_per:gen_dec_prim(Erules, Att, BytesVar); 'REAL' -> emit(["?RT_PER:decode_real(",BytesVar,")"]); @@ -1612,8 +1516,7 @@ gen_dec_prim(Erules,Att,BytesVar) -> {asis,NamedNumberList},")"}) end; 'NULL' -> - emit({"?RT_PER:decode_null(", - BytesVar,")"}); + emit({"{'NULL',",BytesVar,"}"}); 'OBJECT IDENTIFIER' -> emit({"?RT_PER:decode_object_identifier(", BytesVar,")"}); @@ -1623,23 +1526,13 @@ gen_dec_prim(Erules,Att,BytesVar) -> 'ObjectDescriptor' -> emit({"?RT_PER:decode_ObjectDescriptor(", BytesVar,")"}); - {'ENUMERATED',{NamedNumberList1,NamedNumberList2}} -> - NewTup = {list_to_tuple([X||{X,_} <- NamedNumberList1]), - list_to_tuple([X||{X,_} <- NamedNumberList2])}, - NewC = [{'ValueRange',{0,size(element(1,NewTup))-1}}], - emit({"?RT_PER:decode_enumerated(",BytesVar,",", - {asis,NewC},",", - {asis,NewTup},")"}); - {'ENUMERATED',NamedNumberList} -> - NewNNL = [X||{X,_} <- NamedNumberList], - NewC = effective_constraint(integer, - [{'ValueRange',{0,length(NewNNL)-1}}]), - emit_dec_enumerated(BytesVar,NewC,NewNNL); + {'ENUMERATED',_} -> + asn1ct_gen_per:gen_dec_prim(Erules, Att, BytesVar); 'BOOLEAN'-> - emit({"?RT_PER:decode_boolean(",BytesVar,")"}); + asn1ct_gen_per:gen_dec_prim(Erules, Att, BytesVar); 'OCTET STRING' -> - emit_dec_octet_string(Constraint,BytesVar); + asn1ct_gen_per:gen_dec_prim(Erules, Att, BytesVar); 'NumericString' -> emit_dec_known_multiplier_string('NumericString', @@ -1716,88 +1609,6 @@ gen_dec_prim(Erules,Att,BytesVar) -> exit({'cant decode' ,Other}) end. - -emit_dec_integer(C,BytesVar,NNL) -> - asn1ct_name:new(tmpterm), - asn1ct_name:new(buffer), - Tmpterm = asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), - Buffer = asn1ct_gen:mk_var(asn1ct_name:curr(buffer)), - emit({" begin {",{curr,tmpterm},",",{curr,buffer},"} = ",nl}), - emit_dec_integer(C,BytesVar), - emit({",",nl," case ",Tmpterm," of",nl}), - lists:map(fun({Name,Int})->emit({" ",Int," -> {",{asis,Name},",", - Buffer,"};",nl}); - (_)-> exit({error,{asn1,{"error in named number list",NNL}}}) - end, - NNL), - emit({" _ -> {",Tmpterm,",",Buffer,"}",nl}), - emit({" end",nl}), % end of case - emit(" end"). % end of begin - -emit_dec_integer([{'SingleValue',Int}],BytesVar) when is_integer(Int) -> - emit(["{",Int,",",BytesVar,"}"]); -emit_dec_integer([{_,{Lb,_Ub},_Range,{BitsOrOctets,N}}],BytesVar) -> - GetBorO = - case BitsOrOctets of - bits -> "getbits"; - _ -> "getoctets" - end, - asn1ct_name:new(tmpterm), - asn1ct_name:new(tmpremain), - emit({" begin",nl," {",{curr,tmpterm},",",{curr,tmpremain},"}=", - "?RT_PER:",GetBorO,"(",BytesVar,",",N,"),",nl}), - emit({" {",{curr,tmpterm},"+",Lb,",",{curr,tmpremain},"}",nl, - " end"}); -emit_dec_integer([{_,{'MIN',_}}],BytesVar) -> - emit({"?RT_PER:decode_unconstrained_number(",BytesVar,")"}); -emit_dec_integer([{_,{Lb,'MAX'}}],BytesVar) -> - emit({"?RT_PER:decode_semi_constrained_number(",BytesVar,",",Lb,")"}); -emit_dec_integer([{'ValueRange',VR={Lb,Ub}}],BytesVar) -> - Range = Ub-Lb+1, - emit({"?RT_PER:decode_constrained_number(",BytesVar,",", - {asis,VR},",",Range,")"}); -emit_dec_integer(C=[{Rc,_}],BytesVar) when is_tuple(Rc) -> - emit({"?RT_PER:decode_integer(",BytesVar,",",{asis,C},")"}); -emit_dec_integer(_,BytesVar) -> - emit({"?RT_PER:decode_unconstrained_number(",BytesVar,")"}). - - -emit_dec_enumerated(BytesVar,C,NamedNumberList) -> - emit_dec_enumerated_begin(),% emits a begin if component - asn1ct_name:new(tmpterm), - Tmpterm = asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), - asn1ct_name:new(tmpremain), - Tmpremain = asn1ct_gen:mk_var(asn1ct_name:curr(tmpremain)), - emit({" {",{curr,tmpterm},",",{curr,tmpremain},"} =",nl}), - emit_dec_integer(C,BytesVar), - emit({",",nl," case ",Tmpterm," of "}), - - Cases=lists:flatten(dec_enumerated_cases(NamedNumberList,Tmpremain,0)), - emit({Cases++"_->exit({error,{asn1,{decode_enumerated,{",Tmpterm, - ",",{asis,NamedNumberList},"}}}}) end",nl}), - emit_dec_enumerated_end(). - -emit_dec_enumerated_begin() -> - case get(component_type) of - {true,_} -> - emit({" begin",nl}); - _ -> ok - end. - -emit_dec_enumerated_end() -> - case get(component_type) of - {true,_} -> - emit(" end"); - _ -> ok - end. - - -dec_enumerated_cases([Name|Rest],Tmpremain,No) -> - io_lib:format("~w->{~w,~s};",[No,Name,Tmpremain])++ - dec_enumerated_cases(Rest,Tmpremain,No+1); -dec_enumerated_cases([],_,_) -> - "". - %% For PER the ExtensionAdditionGroup notation has significance for the encoding and decoding %% the components within the ExtensionAdditionGroup is treated in a similar way as if they %% have been specified within a SEQUENCE, therefore we construct a fake sequence type here diff --git a/lib/asn1/src/asn1ct_imm.erl b/lib/asn1/src/asn1ct_imm.erl new file mode 100644 index 0000000000..34bb0b8714 --- /dev/null +++ b/lib/asn1/src/asn1ct_imm.erl @@ -0,0 +1,626 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(asn1ct_imm). +-export([per_dec_boolean/0,per_dec_enumerated/2,per_dec_enumerated/3, + per_dec_extension_map/1, + per_dec_integer/2,per_dec_length/3,per_dec_named_integer/3, + per_dec_octet_string/2,per_dec_open_type/1]). +-export([optimize_alignment/1,optimize_alignment/2, + dec_slim_cg/2,dec_code_gen/2]). +-export([effective_constraint/2]). +-import(asn1ct_gen, [emit/1]). + +-record(st, {var, + base}). + +dec_slim_cg(Imm0, BytesVar) -> + {Imm,_} = optimize_alignment(Imm0), + asn1ct_name:new(v), + [H|T] = atom_to_list(asn1ct_name:curr(v)) ++ "@", + VarBase = [H-($a-$A)|T], + St0 = #st{var=0,base=VarBase}, + {Res,Pre,_} = flatten(Imm, BytesVar, St0), + dcg_list_outside(Pre), + Res. + +dec_code_gen(Imm, BytesVar) -> + emit(["begin",nl]), + {Dst,DstBuf} = dec_slim_cg(Imm, BytesVar), + emit([",",nl, + "{",Dst,",",DstBuf,"}",nl, + "end"]), + ok. + +optimize_alignment(Imm) -> + opt_al(Imm, unknown). + +optimize_alignment(Imm, Al) -> + opt_al(Imm, Al). + + +per_dec_boolean() -> + {map,{get_bits,1,[1]},[{0,false},{1,true}]}. + +per_dec_enumerated(NamedList0, Aligned) -> + Constraint = [{'ValueRange',{0,length(NamedList0)-1}}], + NamedList = per_dec_enumerated_fix_list(NamedList0, [enum_error], 0), + Int = per_dec_integer(Constraint, Aligned), + {map,Int,NamedList}. + +per_dec_enumerated(BaseNamedList, NamedListExt0, Aligned) -> + Base = per_dec_enumerated(BaseNamedList, Aligned), + NamedListExt = per_dec_enumerated_fix_list(NamedListExt0, + [enum_default], 0), + Ext = {map,per_dec_normally_small_number(Aligned),NamedListExt}, + bit_case(Base, Ext). + +per_dec_extension_map(Aligned) -> + Len = {add,per_dec_normally_small_number(Aligned),1}, + {get_bits,Len,[1,bitstring]}. + +per_dec_integer(Constraint0, Aligned) -> + Constraint = effective_constraint(integer, Constraint0), + per_dec_integer_1(Constraint, Aligned). + +per_dec_length(SingleValue, _, _Aligned) when is_integer(SingleValue) -> + {value,SingleValue}; +per_dec_length({S,S}, _, _Aligned) when is_integer(S) -> + {value,S}; +per_dec_length({{_,_}=Constr,_}, AllowZero, Aligned) -> + bit_case(per_dec_length(Constr, AllowZero, Aligned), + per_dec_length(undefined, AllowZero, Aligned)); +per_dec_length({Lb,Ub}, _AllowZero, Aligned) when is_integer(Lb), + is_integer(Lb), + Ub =< 65535 -> + per_dec_constrained(Lb, Ub, Aligned); +per_dec_length({_,_}, AllowZero, Aligned) -> + decode_unconstrained_length(AllowZero, Aligned); +per_dec_length(undefined, AllowZero, Aligned) -> + decode_unconstrained_length(AllowZero, Aligned). + +per_dec_named_integer(Constraint, NamedList0, Aligned) -> + Int = per_dec_integer(Constraint, Aligned), + NamedList = [{K,V} || {V,K} <- NamedList0] ++ [integer_default], + {map,Int,NamedList}. + +per_dec_octet_string(Constraint, Aligned) -> + dec_string(Constraint, 8, Aligned). + +per_dec_open_type(Aligned) -> + {get_bits,decode_unconstrained_length(true, Aligned), + [8,binary,{align,Aligned}]}. + + +%%% +%%% Local functions. +%%% + +dec_string(Sv, U, _Aligned) when is_integer(Sv), U*Sv =< 16 -> + {get_bits,Sv,[U,binary]}; +dec_string(Sv, U, Aligned) when is_integer(Sv), Sv < 16#10000 -> + {get_bits,Sv,[U,binary,{align,Aligned}]}; +dec_string(C, U, Aligned) when is_list(C) -> + dec_string({hd(C),lists:max(C)}, U, Aligned); +dec_string({Sv,Sv}, U, Aligned) -> + dec_string(Sv, U, Aligned); +dec_string({{_,_}=C,_}, U, Aligned) -> + bit_case(dec_string(C, U, Aligned), + dec_string(no, U, Aligned)); +dec_string({Lb,Ub}, U, Aligned) when Ub < 16#10000 -> + Len = per_dec_constrained(Lb, Ub, Aligned), + {get_bits,Len,[U,binary,{align,Aligned}]}; +dec_string(_, U, Aligned) -> + Al = [{align,Aligned}], + DecRest = fun(V, Buf) -> + emit(["?RT_PER:decode_fragmented(",V,", ", + Buf,", ",U,")"]) + end, + {'case',[{test,{get_bits,1,[1|Al]},0, + {value,{get_bits, + {get_bits,7,[1]}, + [U,binary]}}}, + {test,{get_bits,1,[1|Al]},1, + {test,{get_bits,1,[1]},0, + {value,{get_bits, + {get_bits,14,[1]}, + [U,binary]}}}}, + {test,{get_bits,1,[1|Al]},1, + {test,{get_bits,1,[1]},1, + {value,{call,DecRest,{get_bits,6,[1]}}}}}]}. + +per_dec_enumerated_fix_list([{V,_}|T], Tail, N) -> + [{N,V}|per_dec_enumerated_fix_list(T, Tail, N+1)]; +per_dec_enumerated_fix_list([], Tail, _) -> Tail. + +per_dec_integer_1([{'SingleValue',Value}], _Aligned) -> + {value,Value}; +per_dec_integer_1([{'ValueRange',{Lb,'MAX'}}], Aligned) when is_integer(Lb) -> + per_dec_unconstrained(Aligned); +per_dec_integer_1([{'ValueRange',{Lb,Ub}}], Aligned) when is_integer(Lb), + is_integer(Ub) -> + per_dec_constrained(Lb, Ub, Aligned); +per_dec_integer_1([{{_,_}=Constr0,_}], Aligned) -> + Constr = effective_constraint(integer, [Constr0]), + bit_case(per_dec_integer(Constr, Aligned), + per_dec_unconstrained(Aligned)); +per_dec_integer_1([], Aligned) -> + per_dec_unconstrained(Aligned). + +per_dec_unconstrained(Aligned) -> + {get_bits,decode_unconstrained_length(false, Aligned),[8,signed]}. + +per_dec_constrained(Lb, Ub, false) -> + Range = Ub - Lb + 1, + Get = {get_bits,uper_num_bits(Range),[1]}, + add_lb(Lb, Get); +per_dec_constrained(Lb, Ub, true) -> + Range = Ub - Lb + 1, + Get = if + Range =< 255 -> + {get_bits,per_num_bits(Range),[1,unsigned]}; + Range == 256 -> + {get_bits,1,[8,unsigned,{align,true}]}; + Range =< 65536 -> + {get_bits,2,[8,unsigned,{align,true}]}; + true -> + RangeOctLen = byte_size(binary:encode_unsigned(Range - 1)), + {get_bits,per_dec_length({1,RangeOctLen}, false, true), + [8,unsigned,{align,true}]} + end, + add_lb(Lb, Get). + +add_lb(0, Get) -> Get; +add_lb(Lb, Get) -> {add,Get,Lb}. + +per_dec_normally_small_number(Aligned) -> + Small = {get_bits,6,[1]}, + Unlimited = per_decode_semi_constrained(0, Aligned), + bit_case(Small, Unlimited). + +per_decode_semi_constrained(Lb, Aligned) -> + add_lb(Lb, {get_bits,decode_unconstrained_length(false, Aligned),[8]}). + +bit_case(Base, Ext) -> + {'case',[{test,{get_bits,1,[1]},0,Base}, + {test,{get_bits,1,[1]},1,Ext}]}. + +decode_unconstrained_length(AllowZero, Aligned) -> + Al = [{align,Aligned}], + Zero = case AllowZero of + false -> [non_zero]; + true -> [] + end, + {'case',[{test,{get_bits,1,[1|Al]},0, + {value,{get_bits,7,[1|Zero]}}}, + {test,{get_bits,1,[1|Al]},1, + {test,{get_bits,1,[1]},0, + {value,{get_bits,14,[1|Zero]}}}}]}. + +uper_num_bits(N) -> + uper_num_bits(N, 1, 0). + +uper_num_bits(N, T, B) when N =< T -> B; +uper_num_bits(N, T, B) -> uper_num_bits(N, T bsl 1, B+1). + +per_num_bits(2) -> 1; +per_num_bits(N) when N =< 4 -> 2; +per_num_bits(N) when N =< 8 -> 3; +per_num_bits(N) when N =< 16 -> 4; +per_num_bits(N) when N =< 32 -> 5; +per_num_bits(N) when N =< 64 -> 6; +per_num_bits(N) when N =< 128 -> 7; +per_num_bits(N) when N =< 255 -> 8. + +%%% +%%% Remove unnecessary aligning to octet boundaries. +%%% + +opt_al({get_bits,E0,Opts0}, A0) -> + {E,A1} = opt_al(E0, A0), + Opts = opt_al_1(A1, Opts0), + A = update_al(A1, E, Opts), + {{get_bits,E,Opts},A}; +opt_al({call,Fun,E0}, A0) -> + {E,A} = opt_al(E0, A0), + {{call,Fun,E},A}; +opt_al({convert,Op,E0}, A0) -> + {E,A} = opt_al(E0, A0), + {{convert,Op,E},A}; +opt_al({value,E0}, A0) -> + {E,A} = opt_al(E0, A0), + {{value,E},A}; +opt_al({add,E0,I}, A0) when is_integer(I) -> + {E,A} = opt_al(E0, A0), + {{add,E,I},A}; +opt_al({test,E0,V,B0}, A0) -> + {E,A1} = opt_al(E0, A0), + {B,A2} = opt_al(B0, A1), + {{test,E,V,B},A2}; +opt_al({'case',Cs0}, A0) -> + {Cs,A} = opt_al_cs(Cs0, A0), + {{'case',Cs},A}; +opt_al({map,E0,Cs}, A0) -> + {E,A} = opt_al(E0, A0), + {{map,E,Cs},A}; +opt_al(I, A) when is_integer(I) -> + {I,A}. + +opt_al_cs([C0|Cs0], A0) -> + {C,A1} = opt_al(C0, A0), + {Cs,A2} = opt_al_cs(Cs0, A0), + {[C|Cs],merge_al(A1, A2)}; +opt_al_cs([], _) -> {[],none}. + +merge_al(unknown, _) -> unknown; +merge_al(Other, none) -> Other; +merge_al(_, unknown) -> unknown; +merge_al(I0, I1) -> + case {I0 rem 8,I1 rem 8} of + {I,I} -> I; + {_,_} -> unknown + end. + +opt_al_1(unknown, Opts) -> + Opts; +opt_al_1(A, Opts0) -> + case alignment(Opts0) of + none -> + Opts0; + full -> + case A rem 8 of + 0 -> + %% Already in alignment. + proplists:delete(align, Opts0); + Bits -> + %% Cheaper alignment with a constant padding. + Opts1 = proplists:delete(align, Opts0), + [{align,8-Bits }|Opts1] + end; + A -> %Assertion. + Opts0 + end. + +update_al(A0, E, Opts) -> + A = case alignment(Opts) of + none -> A0; + full -> 0; + Bits when is_integer(A0) -> + 0 = (A0 + Bits) rem 8; %Assertion. + _ -> + 0 + end, + [U] = [U || U <- Opts, is_integer(U)], + if + U rem 8 =:= 0 -> A; + is_integer(A), is_integer(E) -> A + U*E; + true -> unknown + end. + +%%% +%%% Flatten the intermediate format and assign temporaries. +%%% + +flatten({get_bits,I,U}, Buf0, St0) when is_integer(I) -> + {Dst,St} = new_var_pair(St0), + Gb = {get_bits,{I,Buf0},U,Dst}, + flatten_align(Gb, [], St); +flatten({get_bits,E0,U}, Buf0, St0) -> + {E,Pre,St1} = flatten(E0, Buf0, St0), + {Dst,St2} = new_var_pair(St1), + Gb = {get_bits,E,U,Dst}, + flatten_align(Gb, Pre, St2); +flatten({test,{get_bits,I,U},V,E0}, Buf0, St0) when is_integer(I) -> + {DstBuf0,St1} = new_var("Buf", St0), + Gb = {get_bits,{I,Buf0},U,{V,DstBuf0}}, + {{_Dst,DstBuf},Pre0,St2} = flatten_align(Gb, [], St1), + {E,Pre1,St3} = flatten(E0, DstBuf, St2), + {E,Pre0++Pre1,St3}; +flatten({add,E0,I}, Buf0, St0) -> + {{Src,Buf},Pre,St1} = flatten(E0, Buf0, St0), + {Dst,St} = new_var("Add", St1), + {{Dst,Buf},Pre++[{add,Src,I,Dst}],St}; +flatten({'case',Cs0}, Buf0, St0) -> + {Dst,St1} = new_var_pair(St0), + {Cs1,St} = flatten_cs(Cs0, Buf0, St1), + {Al,Cs2} = flatten_hoist_align(Cs1), + {Dst,Al++[{'case',Buf0,Cs2,Dst}],St}; +flatten({map,E0,Cs0}, Buf0, St0) -> + {{E,DstBuf},Pre,St1} = flatten(E0, Buf0, St0), + {Dst,St2} = new_var("Int", St1), + Cs = flatten_map_cs(Cs0, E), + {{Dst,DstBuf},Pre++[{'map',E,Cs,{Dst,DstBuf}}],St2}; +flatten({value,V0}, Buf0, St0) when is_integer(V0) -> + {{V0,Buf0},[],St0}; +flatten({value,V0}, Buf0, St0) -> + flatten(V0, Buf0, St0); +flatten({convert,Op,E0}, Buf0, St0) -> + {{E,Buf},Pre,St1} = flatten(E0, Buf0, St0), + {Dst,St2} = new_var("Conv", St1), + {{Dst,Buf},Pre++[{convert,Op,E,Dst}],St2}; +flatten({call,Fun,E0}, Buf0, St0) -> + {Src,Pre,St1} = flatten(E0, Buf0, St0), + {Dst,St2} = new_var_pair(St1), + {Dst,Pre++[{call,Fun,Src,Dst}],St2}. + +flatten_cs([C0|Cs0], Buf, St0) -> + {C,Pre,St1} = flatten(C0, Buf, St0), + {Cs,St2} = flatten_cs(Cs0, Buf, St0), + St3 = St2#st{var=max(St1#st.var, St2#st.var)}, + {[Pre++[{return,C}]|Cs],St3}; +flatten_cs([], _, St) -> {[],St}. + +flatten_map_cs(Cs, Var) -> + flatten_map_cs_1(Cs, {Var,Cs}). + +flatten_map_cs_1([{K,V}|Cs], DefData) -> + [{{asis,K},{asis,V}}|flatten_map_cs_1(Cs, DefData)]; +flatten_map_cs_1([integer_default], {Int,_}) -> + [{'_',Int}]; +flatten_map_cs_1([enum_default], {Int,_}) -> + [{'_',["{asn1_enum,",Int,"}"]}]; +flatten_map_cs_1([enum_error], {Var,Cs}) -> + Vs = [V || {_,V} <- Cs], + [{'_',["exit({error,{asn1,{decode_enumerated,{",Var,",", + {asis,Vs},"}}}})"]}]; +flatten_map_cs_1([], _) -> []. + +flatten_hoist_align([[{align_bits,_,_}=Ab|T]|Cs]) -> + flatten_hoist_align_1(Cs, Ab, [T]); +flatten_hoist_align(Cs) -> {[],Cs}. + +flatten_hoist_align_1([[Ab|T]|Cs], Ab, Acc) -> + flatten_hoist_align_1(Cs, Ab, [T|Acc]); +flatten_hoist_align_1([], Ab, Acc) -> + {[Ab],lists:reverse(Acc)}. + +flatten_align({get_bits,{SrcBits,SrcBuf},U,Dst}=Gb0, Pre, St0) -> + case alignment(U) of + none -> + flatten_align_1(U, Dst, Pre++[Gb0], St0); + full -> + {PadBits,St1} = new_var("Pad", St0), + {DstBuf,St2} = new_var("Buf", St1), + Ab = {align_bits,SrcBuf,PadBits}, + Agb = {get_bits,{PadBits,SrcBuf},[1],{'_',DstBuf}}, + Gb = {get_bits,{SrcBits,DstBuf},U,Dst}, + flatten_align_1(U, Dst, Pre++[Ab,Agb,Gb], St2); + PadBits when is_integer(PadBits), PadBits > 0 -> + {DstBuf,St1} = new_var("Buf", St0), + Agb = {get_bits,{PadBits,SrcBuf},[1],{'_',DstBuf}}, + Gb = {get_bits,{SrcBits,DstBuf},U,Dst}, + flatten_align_1(U, Dst, Pre++[Agb,Gb], St1) + end. + +flatten_align_1(U, {D,_}=Dst, Pre, St) -> + case is_non_zero(U) of + false -> + {Dst,Pre,St}; + true -> + {Dst,Pre++[{non_zero,D}],St} + end. + +new_var_pair(St0) -> + {Var,St1} = new_var("V", St0), + {Buf,St2} = new_var("Buf", St1), + {{Var,Buf},St2}. + +new_var(Tag, #st{base=VarBase,var=N}=St) -> + {VarBase++Tag++integer_to_list(N),St#st{var=N+1}}. + +alignment([{align,false}|_]) -> none; +alignment([{align,true}|_]) -> full; +alignment([{align,Bits}|_]) -> Bits; +alignment([_|T]) -> alignment(T); +alignment([]) -> none. + +is_non_zero(Fl) -> + lists:member(non_zero, Fl). + +%%% +%%% Generate Erlang code from the flattened intermediate format. +%%% + +dcg_list_outside([{align_bits,Buf,SzVar}|T]) -> + emit([SzVar," = bit_size(",Buf,") band 7"]), + iter_dcg_list_outside(T); +dcg_list_outside([{'case',Buf,Cs,Dst}|T]) -> + dcg_case(Buf, Cs, Dst), + iter_dcg_list_outside(T); +dcg_list_outside([{'map',Val,Cs,Dst}|T]) -> + dcg_map(Val, Cs, Dst), + iter_dcg_list_outside(T); +dcg_list_outside([{add,S1,S2,Dst}|T]) -> + emit([Dst," = ",S1," + ",S2]), + iter_dcg_list_outside(T); +dcg_list_outside([{return,{V,Buf}}|T]) -> + emit(["{",V,",",Buf,"}"]), + iter_dcg_list_outside(T); +dcg_list_outside([{call,Fun,{V,Buf},{Dst,DstBuf}}|T]) -> + emit(["{",Dst,",",DstBuf,"} = "]), + Fun(V, Buf), + iter_dcg_list_outside(T); +dcg_list_outside([{convert,Op,V,Dst}|T]) -> + emit([Dst," = ",Op,"(",V,")"]), + iter_dcg_list_outside(T); +dcg_list_outside([{get_bits,{_,Buf0},_,_}|_]=L0) -> + emit("<<"), + {L,Buf} = dcg_list_inside(L0, buf), + emit([Buf,"/bitstring>> = ",Buf0]), + iter_dcg_list_outside(L); +dcg_list_outside([]) -> + emit("ignore"), + ok. + +iter_dcg_list_outside([_|_]=T) -> + emit([",",nl]), + dcg_list_outside(T); +iter_dcg_list_outside([]) -> ok. + +dcg_case(Buf, Cs, {Dst,DstBuf}) -> + emit(["{",Dst,",",DstBuf,"} = case ",Buf," of",nl]), + dcg_case_cs(Cs), + emit("end"). + +dcg_case_cs([C|Cs]) -> + emit("<<"), + {T0,DstBuf} = dcg_list_inside(C, buf), + emit([DstBuf,"/bitstring>>"]), + T1 = dcg_guard(T0), + dcg_list_outside(T1), + case Cs of + [] -> emit([nl]); + [_|_] -> emit([";",nl]) + end, + dcg_case_cs(Cs); +dcg_case_cs([]) -> ok. + +dcg_guard([{non_zero,Src}|T]) -> + emit([" when ",Src," =/= 0 ->",nl]), + T; +dcg_guard(T) -> + emit([" ->",nl]), + T. + +dcg_map(Val, Cs, {Dst,_}) -> + emit([Dst," = case ",Val," of",nl]), + dcg_map_cs(Cs), + emit("end"). + +dcg_map_cs([{K,V}]) -> + emit([K," -> ",V,nl]); +dcg_map_cs([{K,V}|Cs]) -> + emit([K," -> ",V,";",nl]), + dcg_map_cs(Cs). + +dcg_list_inside([{get_bits,{Sz,_},Fl0,{Dst,DstBuf}}|T], _) -> + Fl = bit_flags(Fl0, []), + emit([mk_dest(Dst),":",Sz,Fl,","]), + dcg_list_inside(T, DstBuf); +dcg_list_inside(L, Dst) -> {L,Dst}. + +bit_flags([1|T], Acc) -> + bit_flags(T, Acc); +bit_flags([{align,_}|T], Acc) -> + bit_flags(T, Acc); +bit_flags([non_zero|T], Acc) -> + bit_flags(T, Acc); +bit_flags([U|T], Acc) when is_integer(U) -> + bit_flags(T, ["unit:"++integer_to_list(U)|Acc]); +bit_flags([H|T], Acc) -> + bit_flags(T, [atom_to_list(H)|Acc]); +bit_flags([], []) -> + ""; +bit_flags([], Acc) -> + "/" ++ bit_flags_1(Acc, ""). + +bit_flags_1([H|T], Sep) -> + Sep ++ H ++ bit_flags_1(T, "-"); +bit_flags_1([], _) -> []. + +mk_dest(I) when is_integer(I) -> + integer_to_list(I); +mk_dest(S) -> S. + +%% effective_constraint(Type,C) +%% Type = atom() +%% C = [C1,...] +%% C1 = {'SingleValue',SV} | {'ValueRange',VR} | {atom(),term()} +%% SV = integer() | [integer(),...] +%% VR = {Lb,Ub} +%% Lb = 'MIN' | integer() +%% Ub = 'MAX' | integer() +%% Returns a single value if C only has a single value constraint, and no +%% value range constraints, that constrains to a single value, otherwise +%% returns a value range that has the lower bound set to the lowest value +%% of all single values and lower bound values in C and the upper bound to +%% the greatest value. +effective_constraint(integer,[C={{_,_},_}|_Rest]) -> % extension + [C]; +effective_constraint(integer, C) -> + SVs = get_constraints(C, 'SingleValue'), + SV = effective_constr('SingleValue', SVs), + VRs = get_constraints(C, 'ValueRange'), + VR = effective_constr('ValueRange', VRs), + greatest_common_range(SV, VR); +effective_constraint(bitstring, C) -> + get_constraint(C, 'SizeConstraint'). + +effective_constr(_, []) -> []; +effective_constr('SingleValue', List) -> + SVList = lists:flatten(lists:map(fun(X) -> element(2, X) end, List)), + %% Sort and remove duplicates before generating SingleValue or ValueRange + %% In case of ValueRange, also check for 'MIN and 'MAX' + case lists:usort(SVList) of + [N] -> + [{'SingleValue',N}]; + [_|_]=L -> + [{'ValueRange',{least_Lb(L),greatest_Ub(L)}}] + end; +effective_constr('ValueRange', List) -> + LBs = lists:map(fun({_,{Lb,_}}) -> Lb end, List), + UBs = lists:map(fun({_,{_,Ub}}) -> Ub end, List), + Lb = least_Lb(LBs), + [{'ValueRange',{Lb,lists:max(UBs)}}]. + +greatest_common_range([], VR) -> + VR; +greatest_common_range(SV, []) -> + SV; +greatest_common_range([{_,Int}], [{_,{'MIN',Ub}}]) + when is_integer(Int), Int > Ub -> + [{'ValueRange',{'MIN',Int}}]; +greatest_common_range([{_,Int}],[{_,{Lb,Ub}}]) + when is_integer(Int), Int < Lb -> + [{'ValueRange',{Int,Ub}}]; +greatest_common_range([{_,Int}],VR=[{_,{_Lb,_Ub}}]) when is_integer(Int) -> + VR; +greatest_common_range([{_,L}],[{_,{Lb,Ub}}]) when is_list(L) -> + Min = least_Lb([Lb|L]), + Max = greatest_Ub([Ub|L]), + [{'ValueRange',{Min,Max}}]; +greatest_common_range([{_,{Lb1,Ub1}}], [{_,{Lb2,Ub2}}]) -> + Min = least_Lb([Lb1,Lb2]), + Max = greatest_Ub([Ub1,Ub2]), + [{'ValueRange',{Min,Max}}]. + + +least_Lb(L) -> + case lists:member('MIN', L) of + true -> 'MIN'; + false -> lists:min(L) + end. + +greatest_Ub(L) -> + case lists:member('MAX', L) of + true -> 'MAX'; + false -> lists:max(L) + end. + +get_constraint(C, Key) -> + case lists:keyfind(Key, 1, C) of + false -> no; + {_,V} -> V + end. + +get_constraints([{Key,_}=Pair|T], Key) -> + [Pair|get_constraints(T, Key)]; +get_constraints([_|T], Key) -> + get_constraints(T, Key); +get_constraints([], _) -> []. diff --git a/lib/asn1/src/asn1ct_value.erl b/lib/asn1/src/asn1ct_value.erl index 9013baef92..389642c446 100644 --- a/lib/asn1/src/asn1ct_value.erl +++ b/lib/asn1/src/asn1ct_value.erl @@ -54,7 +54,7 @@ from_type(M,Typename,Type) when is_record(Type,type) -> {notype,_} -> true; {primitive,bif} -> - from_type_prim(Type,get_encoding_rule(M)); + from_type_prim(Type); 'ASN1_OPEN_TYPE' -> case Type#type.constraint of [#'Externaltypereference'{type=TrefConstraint}] -> @@ -164,7 +164,7 @@ gen_list(_,_,_,0) -> gen_list(M,Typename,Oftype,N) -> [from_type(M,Typename,Oftype)|gen_list(M,Typename,Oftype,N-1)]. -from_type_prim(D,Erule) -> +from_type_prim(D) -> C = D#type.constraint, case D#type.def of 'INTEGER' -> @@ -303,12 +303,7 @@ from_type_prim(D,Erule) -> adjust_list(size_random(C),c_string(C,"BMPString")); 'UTF8String' -> {ok,Res}=asn1rt:utf8_list_to_binary(adjust_list(random(50),[$U,$T,$F,$8,$S,$t,$r,$i,$n,$g,16#ffff,16#fffffff,16#ffffff,16#fffff,16#fff])), - case Erule of - per -> - binary_to_list(Res); - _ -> - Res - end; + Res; 'UniversalString' -> adjust_list(size_random(C),c_string(C,"UniversalString")); XX -> @@ -440,20 +435,9 @@ get_encoding_rule(M) -> end. open_type_value(ber) -> - [4,9,111,112,101,110,95,116,121,112,101]; -open_type_value(ber_bin) -> -% [4,9,111,112,101,110,95,116,121,112,101]; - <<4,9,111,112,101,110,95,116,121,112,101>>; -open_type_value(ber_bin_v2) -> -% [4,9,111,112,101,110,95,116,121,112,101]; <<4,9,111,112,101,110,95,116,121,112,101>>; -open_type_value(per) -> - "\n\topen_type"; %octet string value "open_type" -open_type_value(per_bin) -> - <<"\n\topen_type">>; -% <<10,9,111,112,101,110,95,116,121,112,101>>; open_type_value(_) -> - [4,9,111,112,101,110,95,116,121,112,101]. + <<"\n\topen_type">>. %octet string value "open_type" to_textual_order({Root,Ext}) -> {to_textual_order(Root),Ext}; diff --git a/lib/asn1/src/asn1rt_ber_bin.erl b/lib/asn1/src/asn1rt_ber_bin.erl index 22f9f2ecfd..ec1549804b 100644 --- a/lib/asn1/src/asn1rt_ber_bin.erl +++ b/lib/asn1/src/asn1rt_ber_bin.erl @@ -19,337 +19,30 @@ %% -module(asn1rt_ber_bin). -%% encoding / decoding of BER - --export([decode/1]). --export([fixoptionals/2,split_list/2,cindex/3,restbytes2/3, - list_to_record/2, - encode_tag_val/1,decode_tag/1,peek_tag/1, - check_tags/3, encode_tags/3]). --export([encode_boolean/2,decode_boolean/3, - encode_integer/3,encode_integer/4, - decode_integer/4,decode_integer/5,encode_enumerated/2, - encode_enumerated/4,decode_enumerated/5, +-export([decode_length/1, encode_real/2, encode_real/3, decode_real/2, decode_real/4, - encode_bit_string/4,decode_bit_string/6, - decode_compact_bit_string/6, - encode_octet_string/3,decode_octet_string/5, - encode_null/2,decode_null/3, - encode_object_identifier/2,decode_object_identifier/3, - encode_relative_oid/2,decode_relative_oid/3, - encode_restricted_string/4,decode_restricted_string/6, - encode_universal_string/3,decode_universal_string/5, - encode_UTF8_string/3, decode_UTF8_string/3, - encode_BMP_string/3,decode_BMP_string/5, - encode_generalized_time/3,decode_generalized_time/5, - encode_utc_time/3,decode_utc_time/5, - encode_length/1,decode_length/1, - check_if_valid_tag/3, - decode_tag_and_length/1, decode_components/6, - decode_components/7, decode_set/6]). - --export([encode_open_type/1,encode_open_type/2,decode_open_type/1,decode_open_type/2,decode_open_type/3]). --export([skipvalue/1, skipvalue/2,skip_ExtensionAdditions/2]). + decode_tag/1]). -include("asn1_records.hrl"). -% the encoding of class of tag bits 8 and 7 +%% the encoding of class of tag bits 8 and 7 -define(UNIVERSAL, 0). --define(APPLICATION, 16#40). --define(CONTEXT, 16#80). --define(PRIVATE, 16#C0). %%% primitive or constructed encoding % bit 6 -define(PRIMITIVE, 0). -define(CONSTRUCTED, 2#00100000). %%% The tag-number for universal types --define(N_BOOLEAN, 1). --define(N_INTEGER, 2). --define(N_BIT_STRING, 3). --define(N_OCTET_STRING, 4). --define(N_NULL, 5). --define(N_OBJECT_IDENTIFIER, 6). --define(N_OBJECT_DESCRIPTOR, 7). --define(N_EXTERNAL, 8). -define(N_REAL, 9). --define(N_ENUMERATED, 10). --define(N_EMBEDDED_PDV, 11). --define(N_UTF8String, 12). --define('N_RELATIVE-OID',13). --define(N_SEQUENCE, 16). --define(N_SET, 17). --define(N_NumericString, 18). --define(N_PrintableString, 19). --define(N_TeletexString, 20). --define(N_VideotexString, 21). --define(N_IA5String, 22). --define(N_UTCTime, 23). --define(N_GeneralizedTime, 24). --define(N_GraphicString, 25). --define(N_VisibleString, 26). --define(N_GeneralString, 27). --define(N_UniversalString, 28). --define(N_BMPString, 30). - - -% the complete tag-word of built-in types --define(T_BOOLEAN, ?UNIVERSAL bor ?PRIMITIVE bor 1). --define(T_INTEGER, ?UNIVERSAL bor ?PRIMITIVE bor 2). --define(T_BIT_STRING, ?UNIVERSAL bor ?PRIMITIVE bor 3). % can be CONSTRUCTED --define(T_OCTET_STRING, ?UNIVERSAL bor ?PRIMITIVE bor 4). % can be CONSTRUCTED --define(T_NULL, ?UNIVERSAL bor ?PRIMITIVE bor 5). --define(T_OBJECT_IDENTIFIER,?UNIVERSAL bor ?PRIMITIVE bor 6). --define(T_OBJECT_DESCRIPTOR,?UNIVERSAL bor ?PRIMITIVE bor 7). --define(T_EXTERNAL, ?UNIVERSAL bor ?PRIMITIVE bor 8). --define(T_REAL, ?UNIVERSAL bor ?PRIMITIVE bor 9). --define(T_ENUMERATED, ?UNIVERSAL bor ?PRIMITIVE bor 10). --define(T_EMBEDDED_PDV, ?UNIVERSAL bor ?PRIMITIVE bor 11). --define(T_SEQUENCE, ?UNIVERSAL bor ?CONSTRUCTED bor 16). --define(T_SET, ?UNIVERSAL bor ?CONSTRUCTED bor 17). --define(T_NumericString, ?UNIVERSAL bor ?PRIMITIVE bor 18). %can be constructed --define(T_PrintableString, ?UNIVERSAL bor ?PRIMITIVE bor 19). %can be constructed --define(T_TeletexString, ?UNIVERSAL bor ?PRIMITIVE bor 20). %can be constructed --define(T_VideotexString, ?UNIVERSAL bor ?PRIMITIVE bor 21). %can be constructed --define(T_IA5String, ?UNIVERSAL bor ?PRIMITIVE bor 22). %can be constructed --define(T_UTCTime, ?UNIVERSAL bor ?PRIMITIVE bor 23). --define(T_GeneralizedTime, ?UNIVERSAL bor ?PRIMITIVE bor 24). --define(T_GraphicString, ?UNIVERSAL bor ?PRIMITIVE bor 25). %can be constructed --define(T_VisibleString, ?UNIVERSAL bor ?PRIMITIVE bor 26). %can be constructed --define(T_GeneralString, ?UNIVERSAL bor ?PRIMITIVE bor 27). %can be constructed --define(T_UniversalString, ?UNIVERSAL bor ?PRIMITIVE bor 28). %can be constructed --define(T_BMPString, ?UNIVERSAL bor ?PRIMITIVE bor 30). %can be constructed - - -decode(Bin) -> - decode_primitive(Bin). - -decode_primitive(Bin) -> - {Tlv = {Tag,Len,V},<<>>} = decode_tlv(Bin), - case element(2,Tag) of - ?CONSTRUCTED -> - {Tag,Len,decode_constructed(V)}; - _ -> - Tlv - end. - -decode_constructed(<<>>) -> - []; -decode_constructed(Bin) -> - {Tlv = {Tag,Len,V},Rest} = decode_tlv(Bin), - NewTlv = - case element(2,Tag) of - ?CONSTRUCTED -> - {Tag,Len,decode_constructed(V)}; - _ -> - Tlv - end, - [NewTlv|decode_constructed(Rest)]. - -decode_tlv(Bin) -> - {Tag,Bin1,_Rb1} = decode_tag(Bin), - {{Len,Bin2},_Rb2} = decode_length(Bin1), - <<V:Len/binary,Bin3/binary>> = Bin2, - {{Tag,Len,V},Bin3}. - - - -%%%%%%%%%%%%% -% split_list(List,HeadLen) -> {HeadList,TailList} -% -% splits List into HeadList (Length=HeadLen) and TailList -% if HeadLen == indefinite -> return {List,indefinite} -split_list(List,indefinite) -> - {List, indefinite}; -split_list(Bin, Len) when is_binary(Bin) -> - split_binary(Bin,Len); -split_list(List,Len) -> - {lists:sublist(List,Len),lists:nthtail(Len,List)}. - -%%% new function which fixes a bug regarding indefinite length decoding -restbytes2(indefinite,<<0,0,RemBytes/binary>>,_) -> - {RemBytes,2}; -restbytes2(indefinite,RemBytes,ext) -> - skipvalue(indefinite,RemBytes); -restbytes2(RemBytes,<<>>,_) -> - {RemBytes,0}; -restbytes2(_RemBytes,Bytes,noext) -> - exit({error,{asn1, {unexpected,Bytes}}}); -restbytes2(RemBytes,Bytes,ext) -> -%% {RemBytes,0}. - {RemBytes,byte_size(Bytes)}. - - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% skipvalue(Length, Bytes) -> {RemainingBytes, RemovedNumberOfBytes} -%% -%% skips the one complete (could be nested) TLV from Bytes -%% handles both definite and indefinite length encodings -%% - -skipvalue(L, Bytes) -> - skipvalue(L, Bytes, 0). - -skipvalue(L, Bytes, Rb) -> - skipvalue(L, Bytes, Rb, 0). - -skipvalue(indefinite, Bytes, Rb, IndefLevel) -> - {T,Bytes2,R2} = decode_tag(Bytes), - {{L,Bytes3},R3} = decode_length(Bytes2), - case {T,L} of - {_,indefinite} -> - skipvalue(indefinite,Bytes3,Rb+R2+R3,IndefLevel+1); - {{0,0,0},0} when IndefLevel =:= 0 -> - %% See X690 8.1.5 NOTE, end of indefinite content - {Bytes3,Rb+2}; - {{0,0,0},0} -> - skipvalue(indefinite,Bytes3,Rb+2,IndefLevel - 1); - _ -> - <<_:L/binary, RestBytes/binary>> = Bytes3, - skipvalue(indefinite,RestBytes,Rb+R2+R3+L, IndefLevel) - %%{RestBytes, R2+R3+L} - end; -%% case Bytes4 of -%% <<0,0,Bytes5/binary>> -> -%% {Bytes5,Rb+Rb4+2}; -%% _ -> skipvalue(indefinite,Bytes4,Rb+Rb4) -%% end; -skipvalue(L, Bytes, Rb, _) -> -% <<Skip:L/binary, RestBytes/binary>> = Bytes, - <<_:L/binary, RestBytes/binary>> = Bytes, - {RestBytes,Rb+L}. - - -skipvalue(Bytes) -> - {_T,Bytes2,R2} = decode_tag(Bytes), - {{L,Bytes3},R3} = decode_length(Bytes2), - skipvalue(L,Bytes3,R2+R3). - - -cindex(Ix,Val,Cname) -> - case element(Ix,Val) of - {Cname,Val2} -> Val2; - X -> X - end. - -%%% -%% skips byte sequence of Bytes that do not match a tag in Tags -skip_ExtensionAdditions(Bytes,Tags) -> - skip_ExtensionAdditions(Bytes,Tags,0). -skip_ExtensionAdditions(<<>>,_Tags,RmB) -> - {<<>>,RmB}; -skip_ExtensionAdditions(Bytes,Tags,RmB) -> - case catch decode_tag(Bytes) of - {'EXIT',_Reason} -> - tag_error(no_data,Tags,Bytes,'OPTIONAL'); - {_T={Class,_Form,TagNo},_Bytes2,_R2} -> - case [X||X=#tag{class=Cl,number=TN} <- Tags,Cl==Class,TN==TagNo] of - [] -> - %% skip this TLV and continue with next - {Bytes3,R3} = skipvalue(Bytes), - skip_ExtensionAdditions(Bytes3,Tags,RmB+R3); - _ -> - {Bytes,RmB} - end - end. - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Optionals, preset not filled optionals with asn1_NOVALUE -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -% converts a list to a record if necessary -list_to_record(Name,List) when is_list(List) -> - list_to_tuple([Name|List]); -list_to_record(_Name,Tuple) when is_tuple(Tuple) -> - Tuple. - - -fixoptionals(OptList,Val) when is_list(Val) -> - fixoptionals(OptList,Val,1,[],[]). - -fixoptionals([{Name,Pos}|Ot],[{Name,Val}|Vt],_Opt,Acc1,Acc2) -> - fixoptionals(Ot,Vt,Pos+1,[1|Acc1],[{Name,Val}|Acc2]); -fixoptionals([{_Name,Pos}|Ot],V,Pos,Acc1,Acc2) -> - fixoptionals(Ot,V,Pos+1,[0|Acc1],[asn1_NOVALUE|Acc2]); -fixoptionals(O,[Vh|Vt],Pos,Acc1,Acc2) -> - fixoptionals(O,Vt,Pos+1,Acc1,[Vh|Acc2]); -fixoptionals([],[Vh|Vt],Pos,Acc1,Acc2) -> - fixoptionals([],Vt,Pos+1,Acc1,[Vh|Acc2]); -fixoptionals([],[],_,_Acc1,Acc2) -> - % return Val as a record - list_to_tuple([asn1_RECORDNAME|lists:reverse(Acc2)]). - - -%%encode_tag(TagClass(?UNI, APP etc), Form (?PRIM etx), TagInteger) -> -%% 8bit Int | binary encode_tag_val({Class, Form, TagNo}) when (TagNo =< 30) -> <<(Class bsr 6):2,(Form bsr 5):1,TagNo:5>>; encode_tag_val({Class, Form, TagNo}) -> {Octets,_Len} = mk_object_val(TagNo), BinOct = list_to_binary(Octets), - <<(Class bsr 6):2, (Form bsr 5):1, 31:5,BinOct/binary>>; - -%% asumes whole correct tag bitpattern, multiple of 8 -encode_tag_val(Tag) when (Tag =< 255) -> Tag; %% anv�nds denna funktion??!! -%% asumes correct bitpattern of 0-5 -encode_tag_val(Tag) -> encode_tag_val2(Tag,[]). - -encode_tag_val2(Tag, OctAck) when (Tag =< 255) -> - [Tag | OctAck]; -encode_tag_val2(Tag, OctAck) -> - encode_tag_val2(Tag bsr 8, [255 band Tag | OctAck]). - - -%%%encode_tag(TagClass(?UNI, APP etc), Form (?PRIM etx), TagInteger) -> -%%% 8bit Int | [list of octets] -%encode_tag_val({Class, Form, TagNo}) when (TagNo =< 30) -> -%%% <<Class:2,Form:1,TagNo:5>>; -% [Class bor Form bor TagNo]; -%encode_tag_val({Class, Form, TagNo}) -> -% {Octets,L} = mk_object_val(TagNo), -% [Class bor Form bor 31 | Octets]; - - -%%============================================================================\%% Peek on the initial tag -%% peek_tag(Bytes) -> TagBytes -%% interprets the first byte and possible second, third and fourth byte as -%% a tag and returns all the bytes comprising the tag, the constructed/primitive bit (6:th bit of first byte) is normalised to 0 -%% - -peek_tag(<<B7_6:2,_:1,31:5,Buffer/binary>>) -> - Bin = peek_tag(Buffer, <<>>), - <<B7_6:2,31:6,Bin/binary>>; -%% single tag (tagno < 31) -peek_tag(<<B7_6:2,_:1,B4_0:5,_Buffer/binary>>) -> - <<B7_6:2,B4_0:6>>. - -peek_tag(<<0:1,PartialTag:7,_Buffer/binary>>, TagAck) -> - <<TagAck/binary,PartialTag>>; -peek_tag(<<PartialTag,Buffer/binary>>, TagAck) -> - peek_tag(Buffer,<<TagAck/binary,PartialTag>>); -peek_tag(_,TagAck) -> - exit({error,{asn1, {invalid_tag,TagAck}}}). -%%peek_tag([Tag|Buffer]) when (Tag band 31) =:= 31 -> -%% [Tag band 2#11011111 | peek_tag(Buffer,[])]; -%%%% single tag (tagno < 31) -%%peek_tag([Tag|Buffer]) -> -%% [Tag band 2#11011111]. - -%%peek_tag([PartialTag|Buffer], TagAck) when (PartialTag < 128 ) -> -%% lists:reverse([PartialTag|TagAck]); -%%peek_tag([PartialTag|Buffer], TagAck) -> -%% peek_tag(Buffer,[PartialTag|TagAck]); -%%peek_tag(Buffer,TagAck) -> -%% exit({error,{asn1, {invalid_tag,lists:reverse(TagAck)}}}). - + <<(Class bsr 6):2, (Form bsr 5):1, 31:5,BinOct/binary>>. %%=============================================================================== %% Decode a tag @@ -403,33 +96,11 @@ check_tags_i([Tag1|TagRest], Buffer, Rb, OptOrMand) -> _ -> check_tags_i(TagRest, Buffer2, Rb + Rb1, mandatory) end - end; - -check_tags_i([], Buffer, Rb, _) -> - {[],{{0,0},Buffer,Rb}}. + end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% This function is called from generated code -check_tags([Tag], Buffer, OptOrMand) -> % optimized very usual case - check_one_tag(Tag, Buffer, OptOrMand); -check_tags(Tags, Buffer, OptOrMand) -> - check_tags(Tags, Buffer, 0, OptOrMand). - -check_tags([Tag1,Tag2|TagRest], Buffer, Rb, OptOrMand) - when Tag1#tag.type == 'IMPLICIT' -> - check_tags([Tag1#tag{type=Tag2#tag.type}|TagRest], Buffer, Rb, OptOrMand); - -check_tags([Tag1|TagRest], Buffer, Rb, OptOrMand) -> - {Form_Length,Buffer2,Rb1} = check_one_tag(Tag1, Buffer, OptOrMand), - case TagRest of - [] -> {Form_Length, Buffer2, Rb + Rb1}; - _ -> check_tags(TagRest, Buffer2, Rb + Rb1, mandatory) - end; - -check_tags([], Buffer, Rb, _) -> - {{0,0},Buffer,Rb}. - check_one_tag(Tag=#tag{class=ExpectedClass,number=ExpectedNumber}, Buffer, OptOrMand) -> case catch decode_tag(Buffer) of {'EXIT',_Reason} -> @@ -491,382 +162,6 @@ encode_one_tag(#tag{class=Class,number=No,type=Type, form = Form}) -> Bytes = encode_tag_val({Class,NewForm,No}), {Bytes,size(Bytes)}. -%%=============================================================================== -%% Change the tag (used when an implicit tagged type has a reference to something else) -%% The constructed bit in the tag is taken from the tag to be replaced. -%% -%% change_tag(NewTag,[Tag,Buffer]) -> [NewTag,Buffer] -%%=============================================================================== - -%change_tag({NewClass,NewTagNr}, Buffer) -> -% {{OldClass, OldForm, OldTagNo}, Buffer1, RemovedBytes} = decode_tag(lists:flatten(Buffer)), -% [encode_tag_val({NewClass, OldForm, NewTagNr}) | Buffer1]. - - -%%=============================================================================== -%% -%% This comment is valid for all the encode/decode functions -%% -%% C = Constraint -> typically {'ValueRange',LowerBound,UpperBound} -%% used for PER-coding but not for BER-coding. -%% -%% Val = Value. If Val is an atom then it is a symbolic integer value -%% (i.e the atom must be one of the names in the NamedNumberList). -%% The NamedNumberList is used to translate the atom to an integer value -%% before encoding. -%% -%%=============================================================================== - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_open_type(Value) -> CompleteList -%% Value = list of bytes of an already encoded value (the list must be flat) -%% | binary - -%% This version does not consider Explicit tagging of the open type. It -%% is only left because of backward compatibility. -encode_open_type(Val) when is_list(Val) -> - {Val, byte_size(list_to_binary(Val))}; -encode_open_type(Val) -> - {Val, byte_size(Val)}. - -%% -encode_open_type(Val, []) when is_list(Val) -> - {Val, byte_size(list_to_binary(Val))}; -encode_open_type(Val, []) -> - {Val, byte_size(Val)}; -encode_open_type(Val, Tag) when is_list(Val) -> - encode_tags(Tag, Val, byte_size(list_to_binary(Val))); -encode_open_type(Val, Tag) -> - encode_tags(Tag, Val, byte_size(Val)). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_open_type(Buffer) -> Value -%% Bytes = [byte] with BER encoded data -%% Value = [byte] with decoded data (which must be decoded again as some type) -%% -decode_open_type(Bytes) -> -% {_Tag, Len, _RemainingBuffer, RemovedBytes} = decode_tag_and_length(Bytes), -% N = Len + RemovedBytes, - {_Tag, Len, RemainingBuffer, RemovedBytes} = decode_tag_and_length(Bytes), - {_RemainingBuffer2, RemovedBytes2} = skipvalue(Len, RemainingBuffer, RemovedBytes), - N = RemovedBytes2, - <<Val:N/binary, RemainingBytes/binary>> = Bytes, -% {Val, RemainingBytes, Len + RemovedBytes}. - {Val,RemainingBytes,N}. - -decode_open_type(<<>>,[]=ExplTag) -> % R9C-0.patch-40 - exit({error, {asn1,{no_optional_tag, ExplTag}}}); -decode_open_type(Bytes,ExplTag) -> - {Tag, Len, RemainingBuffer, RemovedBytes} = decode_tag_and_length(Bytes), - case {Tag,ExplTag} of -% {{Class,Form,32},[#tag{class=Class,number=No,form=32}]} -> -% {_Tag2, Len2, RemainingBuffer2, RemovedBytes2} = decode_tag_and_length(RemainingBuffer), -% {_RemainingBuffer3, RemovedBytes3} = skipvalue(Len2, RemainingBuffer2, RemovedBytes2), -% N = RemovedBytes3, -% <<_:RemovedBytes/unit:8,Val:N/binary,RemainingBytes/binary>> = Bytes, -% {Val, RemainingBytes, N + RemovedBytes}; - {{Class,Form,No},[#tag{class=Class,number=No,form=Form}]} -> - {_RemainingBuffer2, RemovedBytes2} = - skipvalue(Len, RemainingBuffer), - N = RemovedBytes2, - <<_:RemovedBytes/unit:8,Val:N/binary,RemainingBytes/binary>> = Bytes, - {Val, RemainingBytes, N + RemovedBytes}; - _ -> - {_RemainingBuffer2, RemovedBytes2} = - skipvalue(Len, RemainingBuffer, RemovedBytes), - N = RemovedBytes2, - <<Val:N/binary, RemainingBytes/binary>> = Bytes, - {Val, RemainingBytes, N} - end. - -decode_open_type(ber_bin,Bytes,ExplTag) -> - decode_open_type(Bytes,ExplTag); -decode_open_type(ber,Bytes,ExplTag) -> - {Val,RemBytes,Len}=decode_open_type(Bytes,ExplTag), - {binary_to_list(Val),RemBytes,Len}. - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Boolean, ITU_T X.690 Chapter 8.2 -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -%%=============================================================================== -%% encode_boolean(Integer, tag | notag) -> [octet list] -%%=============================================================================== - -encode_boolean({Name, Val}, DoTag) when is_atom(Name) -> - dotag(DoTag, ?N_BOOLEAN, encode_boolean(Val)); -encode_boolean(true,[]) -> - {[1,1,16#FF],3}; -encode_boolean(false,[]) -> - {[1,1,0],3}; -encode_boolean(Val, DoTag) -> - dotag(DoTag, ?N_BOOLEAN, encode_boolean(Val)). - -%% encode_boolean(Boolean) -> [Len, Boolean] = [1, $FF | 0] -encode_boolean(true) -> {[16#FF],1}; -encode_boolean(false) -> {[0],1}; -encode_boolean(X) -> exit({error,{asn1, {encode_boolean, X}}}). - - -%%=============================================================================== -%% decode_boolean(BuffList, HasTag, TotalLen) -> {true, Remain, RemovedBytes} | -%% {false, Remain, RemovedBytes} -%%=============================================================================== - -decode_boolean(Buffer, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_BOOLEAN}), - decode_boolean_notag(Buffer, NewTags, OptOrMand). - -decode_boolean_notag(Buffer, Tags, OptOrMand) -> - {RestTags, {FormLen,Buffer0,Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val,Buffer1,Rb1} = decode_boolean_notag(Buffer00, RestTags, OptOrMand), - {Buffer2, Rb2} = restbytes2(RestBytes,Buffer1,noext), - {Val, Buffer2, Rb0+Rb1+Rb2}; - {_,_} -> - decode_boolean2(Buffer0, Rb0) - end. - -decode_boolean2(<<0:8, Buffer/binary>>, RemovedBytes) -> - {false, Buffer, RemovedBytes + 1}; -decode_boolean2(<<_:8, Buffer/binary>>, RemovedBytes) -> - {true, Buffer, RemovedBytes + 1}; -decode_boolean2(Buffer, _) -> - exit({error,{asn1, {decode_boolean, Buffer}}}). - - -%%=========================================================================== -%% Integer, ITU_T X.690 Chapter 8.3 - -%% encode_integer(Constraint, Value, Tag) -> [octet list] -%% encode_integer(Constraint, Name, NamedNumberList, Tag) -> [octet list] -%% Value = INTEGER | {Name,INTEGER} -%% Tag = tag | notag -%%=========================================================================== - -encode_integer(C, Val, []) when is_integer(Val) -> - {EncVal,Len} = encode_integer(C, Val), - dotag_universal(?N_INTEGER,EncVal,Len); -encode_integer(C, Val, Tag) when is_integer(Val) -> - dotag(Tag, ?N_INTEGER, encode_integer(C, Val)); -encode_integer(C,{Name,Val},Tag) when is_atom(Name) -> - encode_integer(C,Val,Tag); -encode_integer(_, Val, _) -> - exit({error,{asn1, {encode_integer, Val}}}). - - -encode_integer(C, Val, NamedNumberList, Tag) when is_atom(Val) -> - case lists:keyfind(Val, 1, NamedNumberList) of - {_, NewVal} -> - dotag(Tag, ?N_INTEGER, encode_integer(C, NewVal)); - _ -> - exit({error,{asn1, {encode_integer_namednumber, Val}}}) - end; -encode_integer(C,{_,Val},NamedNumberList,Tag) -> - encode_integer(C,Val,NamedNumberList,Tag); -encode_integer(C, Val, _NamedNumberList, Tag) -> - dotag(Tag, ?N_INTEGER, encode_integer(C, Val)). - - -encode_integer(_C, Val) -> - Bytes = - if - Val >= 0 -> - encode_integer_pos(Val, []); - true -> - encode_integer_neg(Val, []) - end, - {Bytes,length(Bytes)}. - -encode_integer_pos(0, L=[B|_Acc]) when B < 128 -> - L; -encode_integer_pos(N, Acc) -> - encode_integer_pos((N bsr 8), [N band 16#ff| Acc]). - -encode_integer_neg(-1, L=[B1|_T]) when B1 > 127 -> - L; -encode_integer_neg(N, Acc) -> - encode_integer_neg(N bsr 8, [N band 16#ff|Acc]). - -%%=============================================================================== -%% decode integer -%% (Buffer, Range, HasTag, TotalLen) -> {Integer, Remain, RemovedBytes} -%% (Buffer, Range, NamedNumberList, HasTag, TotalLen) -> {Integer, Remain, RemovedBytes} -%%=============================================================================== - -decode_integer(Buffer, Range, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_INTEGER}), - decode_integer_notag(Buffer, Range, [], NewTags, OptOrMand). - -decode_integer(Buffer, Range, NamedNumberList, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_INTEGER}), - decode_integer_notag(Buffer, Range, NamedNumberList, NewTags, OptOrMand). - -decode_integer_notag(Buffer, Range, NamedNumberList, NewTags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(NewTags, Buffer, OptOrMand), -% Result = {Val, Buffer2, RemovedBytes} = - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00, RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_integer_notag(Buffer00, Range, NamedNumberList, - RestTags, OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_, Len} -> - Result = - decode_integer2(Len,Buffer0,Rb0+Len), - Result2 = check_integer_constraint(Result,Range), - resolve_named_value(Result2,NamedNumberList) - end. - -resolve_named_value(Result={Val,Buffer,RemBytes},NamedNumberList) -> - case NamedNumberList of - [] -> Result; - _ -> - NewVal = case lists:keyfind(Val, 2, NamedNumberList) of - {NamedVal, _} -> - NamedVal; - _ -> - Val - end, - {NewVal, Buffer, RemBytes} - end. - -check_integer_constraint(Result={Val, _Buffer,_},Range) -> - case Range of - [] -> % No length constraint - Result; - {Lb,Ub} when Val >= Lb, Ub >= Val -> % variable length constraint - Result; - Val -> % fixed value constraint - Result; - {_,_} -> - exit({error,{asn1,{integer_range,Range,Val}}}); - SingleValue when is_integer(SingleValue) -> - exit({error,{asn1,{integer_range,Range,Val}}}); - _ -> % some strange constraint that we don't support yet - Result - end. - -%%============================================================================ -%% Enumerated value, ITU_T X.690 Chapter 8.4 - -%% encode enumerated value -%%============================================================================ -encode_enumerated(Val, []) when is_integer(Val) -> - {EncVal,Len} = encode_integer(false,Val), - dotag_universal(?N_ENUMERATED,EncVal,Len); -encode_enumerated(Val, DoTag) when is_integer(Val) -> - dotag(DoTag, ?N_ENUMERATED, encode_integer(false,Val)); -encode_enumerated({Name,Val}, DoTag) when is_atom(Name) -> - encode_enumerated(Val, DoTag). - -%% The encode_enumerated functions below this line can be removed when the -%% new code generation is stable. (the functions might have to be kept here -%% a while longer for compatibility reasons) - -encode_enumerated(C, Val, {NamedNumberList,ExtList}, DoTag) when is_atom(Val) -> - case catch encode_enumerated(C, Val, NamedNumberList, DoTag) of - {'EXIT',_} -> encode_enumerated(C, Val, ExtList, DoTag); - Result -> Result - end; - -encode_enumerated(C, Val, NamedNumberList, DoTag) when is_atom(Val) -> - case lists:keyfind(Val, 1, NamedNumberList) of - {_, NewVal} when DoTag =:= [] -> - {EncVal,Len} = encode_integer(C,NewVal), - dotag_universal(?N_ENUMERATED,EncVal,Len); - {_, NewVal} -> - dotag(DoTag, ?N_ENUMERATED, encode_integer(C, NewVal)); - _ -> - exit({error,{asn1, {enumerated_not_in_range, Val}}}) - end; - -encode_enumerated(C, {asn1_enum, Val}, {_,_}, DoTag) when is_integer(Val) -> - dotag(DoTag, ?N_ENUMERATED, encode_integer(C,Val)); - -encode_enumerated(C, {Name,Val}, NamedNumberList, DoTag) when is_atom(Name) -> - encode_enumerated(C, Val, NamedNumberList, DoTag); - -encode_enumerated(_, Val, _, _) -> - exit({error,{asn1, {enumerated_not_namednumber, Val}}}). - - - -%%============================================================================ -%% decode enumerated value -%% (Buffer, Range, NamedNumberList, HasTag, TotalLen) -> -%% {Value, RemainingBuffer, RemovedBytes} -%%=========================================================================== -decode_enumerated(Buffer, Range, NamedNumberList, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_ENUMERATED}), - decode_enumerated_notag(Buffer, Range, NamedNumberList, - NewTags, OptOrMand). - -decode_enumerated_notag(Buffer, Range, NNList = {NamedNumberList,ExtList}, Tags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_enumerated_notag(Buffer00, Range, NNList, RestTags, OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - {Val01, Buffer01, Rb01} = - decode_integer2(Len, Buffer0, Rb0+Len), - case decode_enumerated1(Val01, NamedNumberList) of - {asn1_enum,Val01} -> - {decode_enumerated1(Val01,ExtList), Buffer01, Rb01}; - Result01 -> - {Result01, Buffer01, Rb01} - end - end; - -decode_enumerated_notag(Buffer, Range, NNList, Tags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_enumerated_notag(Buffer00, Range, NNList, RestTags, OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - {Val01, Buffer02, Rb02} = - decode_integer2(Len, Buffer0, Rb0+Len), - case decode_enumerated1(Val01, NNList) of - {asn1_enum,_} -> - exit({error,{asn1, {illegal_enumerated, Val01}}}); - Result01 -> - {Result01, Buffer02, Rb02} - end - end. - -decode_enumerated1(Val, NamedNumberList) -> - %% it must be a named integer - case lists:keyfind(Val, 2, NamedNumberList) of - {NamedVal, _} -> - NamedVal; - _ -> - {asn1_enum,Val} - end. - - %%============================================================================ %% %% Real value, ITU_T X.690 Chapter 8.5 @@ -1117,513 +412,15 @@ decode_real2(Buffer0, _C, Len, RemBytes1) -> {{Mantissa, Base, Exp}, Buffer4, RemBytes2+RemBytes3} end. +encode_integer_pos(0, L=[B|_Acc]) when B < 128 -> + L; +encode_integer_pos(N, Acc) -> + encode_integer_pos((N bsr 8), [N band 16#ff| Acc]). -%%============================================================================ -%% Bitstring value, ITU_T X.690 Chapter 8.6 -%% -%% encode bitstring value -%% -%% bitstring NamedBitList -%% Val can be of: -%% - [identifiers] where only named identifers are set to one, -%% the Constraint must then have some information of the -%% bitlength. -%% - [list of ones and zeroes] all bits -%% - integer value representing the bitlist -%% C is constrint Len, only valid when identifiers -%%============================================================================ - -encode_bit_string(C,Bin={Unused,BinBits},NamedBitList,DoTag) when is_integer(Unused), is_binary(BinBits) -> - encode_bin_bit_string(C,Bin,NamedBitList,DoTag); -encode_bit_string(C, [FirstVal | RestVal], NamedBitList, DoTag) when is_atom(FirstVal) -> - encode_bit_string_named(C, [FirstVal | RestVal], NamedBitList, DoTag); - -encode_bit_string(C, [{bit,X} | RestVal], NamedBitList, DoTag) -> - encode_bit_string_named(C, [{bit,X} | RestVal], NamedBitList, DoTag); - -encode_bit_string(C, [FirstVal| RestVal], NamedBitList, DoTag) when is_integer(FirstVal) -> - encode_bit_string_bits(C, [FirstVal | RestVal], NamedBitList, DoTag); - -encode_bit_string(_, 0, _, []) -> - {[?N_BIT_STRING,1,0],3}; - -encode_bit_string(_, 0, _, DoTag) -> - dotag(DoTag, ?N_BIT_STRING, {<<0>>,1}); - -encode_bit_string(_, [], _, []) -> - {[?N_BIT_STRING,1,0],3}; - -encode_bit_string(_, [], _, DoTag) -> - dotag(DoTag, ?N_BIT_STRING, {<<0>>,1}); - -encode_bit_string(C, IntegerVal, NamedBitList, DoTag) when is_integer(IntegerVal) -> - BitListVal = int_to_bitlist(IntegerVal), - encode_bit_string_bits(C, BitListVal, NamedBitList, DoTag); - -encode_bit_string(C, {Name,BitList}, NamedBitList, DoTag) when is_atom(Name) -> - encode_bit_string(C, BitList, NamedBitList, DoTag). - - - -int_to_bitlist(0) -> - []; -int_to_bitlist(Int) when is_integer(Int), Int >= 0 -> - [Int band 1 | int_to_bitlist(Int bsr 1)]. - - -%%================================================================= -%% Encode BIT STRING of the form {Unused,BinBits}. -%% Unused is the number of unused bits in the last byte in BinBits -%% and BinBits is a binary representing the BIT STRING. -%%================================================================= -encode_bin_bit_string(C,{Unused,BinBits},_NamedBitList,DoTag)-> - case get_constraint(C,'SizeConstraint') of - no -> - remove_unused_then_dotag(DoTag,?N_BIT_STRING,Unused,BinBits); - {_Min,Max} -> - BBLen = (size(BinBits)*8)-Unused, - if - BBLen > Max -> - exit({error,{asn1, - {bitstring_length, - {{was,BBLen},{maximum,Max}}}}}); - true -> - remove_unused_then_dotag(DoTag,?N_BIT_STRING, - Unused,BinBits) - end; - Size -> - case ((size(BinBits)*8)-Unused) of - BBSize when BBSize =< Size -> - remove_unused_then_dotag(DoTag,?N_BIT_STRING, - Unused,BinBits); - BBSize -> - exit({error,{asn1, - {bitstring_length, - {{was,BBSize},{should_be,Size}}}}}) - end - end. - -remove_unused_then_dotag(DoTag,StringType,Unused,BinBits) -> - case Unused of - 0 when (byte_size(BinBits) =:= 0), DoTag =:= [] -> - %% time optimization of next case - {[StringType,1,0],3}; - 0 when (byte_size(BinBits) =:= 0) -> - dotag(DoTag,StringType,{<<0>>,1}); - 0 when DoTag =:= [] -> % time optimization of next case - dotag_universal(StringType,[Unused|[BinBits]],size(BinBits)+1); -% {LenEnc,Len} = encode_legth(size(BinBits)+1), -% {[StringType,LenEnc,[Unused|BinBits]],size(BinBits)+1+Len+1}; - 0 -> - dotag(DoTag,StringType,<<Unused,BinBits/binary>>); - Num when DoTag =:= [] -> % time optimization of next case - N = byte_size(BinBits) - 1, - <<BBits:N/binary,LastByte>> = BinBits, - dotag_universal(StringType, - [Unused,BBits,(LastByte bsr Num) bsl Num], - byte_size(BinBits) + 1); -% {LenEnc,Len} = encode_legth(size(BinBits)+1), -% {[StringType,LenEnc,[Unused,BBits,(LastByte bsr Num) bsl Num], -% 1+Len+size(BinBits)+1}; - Num -> - N = byte_size(BinBits) - 1, - <<BBits:N/binary,LastByte>> = BinBits, - dotag(DoTag,StringType,{[Unused,binary_to_list(BBits) ++ - [(LastByte bsr Num) bsl Num]], - byte_size(BinBits) + 1}) - end. - - -%%================================================================= -%% Encode named bits -%%================================================================= - -encode_bit_string_named(C, [FirstVal | RestVal], NamedBitList, DoTag) -> - {Len,Unused,OctetList} = - case get_constraint(C,'SizeConstraint') of - no -> - ToSetPos = get_all_bitposes([FirstVal | RestVal], - NamedBitList, []), - BitList = make_and_set_list(lists:max(ToSetPos)+1, - ToSetPos, 0), - encode_bitstring(BitList); - {_Min,Max} -> - ToSetPos = get_all_bitposes([FirstVal | RestVal], - NamedBitList, []), - BitList = make_and_set_list(Max, ToSetPos, 0), - encode_bitstring(BitList); - Size -> - ToSetPos = get_all_bitposes([FirstVal | RestVal], - NamedBitList, []), - BitList = make_and_set_list(Size, ToSetPos, 0), - encode_bitstring(BitList) - end, - case DoTag of - [] -> - dotag_universal(?N_BIT_STRING,[Unused|OctetList],Len+1); -% {EncLen,LenLen} = encode_length(Len+1), -% {[?N_BIT_STRING,EncLen,Unused,OctetList],1+LenLen+Len+1}; - _ -> - dotag(DoTag, ?N_BIT_STRING, {[Unused|OctetList],Len+1}) - end. - - -%%---------------------------------------- -%% get_all_bitposes([list of named bits to set], named_bit_db, []) -> -%% [sorted_list_of_bitpositions_to_set] -%%---------------------------------------- - -get_all_bitposes([{bit,ValPos}|Rest], NamedBitList, Ack) -> - get_all_bitposes(Rest, NamedBitList, [ValPos | Ack ]); -get_all_bitposes([Val | Rest], NamedBitList, Ack) when is_atom(Val) -> - case lists:keyfind(Val, 1, NamedBitList) of - {_ValName, ValPos} -> - get_all_bitposes(Rest, NamedBitList, [ValPos | Ack]); - _ -> - exit({error,{asn1, {bitstring_namedbit, Val}}}) - end; -get_all_bitposes([], _NamedBitList, Ack) -> - lists:sort(Ack). - - -%%---------------------------------------- -%% make_and_set_list(Len of list to return, [list of positions to set to 1])-> -%% returns list of Len length, with all in SetPos set. -%% in positioning in list the first element is 0, the second 1 etc.., but -%% Len will make a list of length Len, not Len + 1. -%% BitList = make_and_set_list(C, ToSetPos, 0), -%%---------------------------------------- - -make_and_set_list(0, [], _) -> []; -make_and_set_list(0, _, _) -> - exit({error,{asn1,bitstring_sizeconstraint}}); -make_and_set_list(Len, [XPos|SetPos], XPos) -> - [1 | make_and_set_list(Len - 1, SetPos, XPos + 1)]; -make_and_set_list(Len, [Pos|SetPos], XPos) -> - [0 | make_and_set_list(Len - 1, [Pos | SetPos], XPos + 1)]; -make_and_set_list(Len, [], XPos) -> - [0 | make_and_set_list(Len - 1, [], XPos + 1)]. - - -%%================================================================= -%% Encode bit string for lists of ones and zeroes -%%================================================================= -encode_bit_string_bits(C, BitListVal, _NamedBitList, DoTag) when is_list(BitListVal) -> - {Len,Unused,OctetList} = - case get_constraint(C,'SizeConstraint') of - no -> - encode_bitstring(BitListVal); - Constr={Min,_Max} when is_integer(Min) -> - encode_constr_bit_str_bits(Constr,BitListVal,DoTag); - {Constr={_,_},[]} -> - %% constraint with extension mark - encode_constr_bit_str_bits(Constr,BitListVal,DoTag); - Constr={{_,_},{_,_}} ->%{{Min1,Max1},{Min2,Max2}} - %% constraint with extension mark - encode_constr_bit_str_bits(Constr,BitListVal,DoTag); - Size -> - case length(BitListVal) of - BitSize when BitSize =:= Size -> - encode_bitstring(BitListVal); - BitSize when BitSize < Size -> - PaddedList = - pad_bit_list(Size-BitSize,BitListVal), - encode_bitstring(PaddedList); - BitSize -> - exit({error, - {asn1, - {bitstring_length, - {{was,BitSize}, - {should_be,Size}}}}}) - end - end, - %%add unused byte to the Len - case DoTag of - [] -> - dotag_universal(?N_BIT_STRING,[Unused|OctetList],Len+1); -% {EncLen,LenLen}=encode_length(Len+1), -% {[?N_BIT_STRING,EncLen,Unused|OctetList],1+LenLen+Len+1}; - _ -> - dotag(DoTag, ?N_BIT_STRING, - {[Unused | OctetList],Len+1}) - end. - - -encode_constr_bit_str_bits({{_Min1,Max1},{Min2,Max2}},BitListVal,_DoTag) -> - BitLen = length(BitListVal), - case BitLen of - Len when Len > Max2 -> - exit({error,{asn1,{bitstring_length,{{was,BitLen}, - {maximum,Max2}}}}}); - Len when Len > Max1, Len < Min2 -> - exit({error,{asn1,{bitstring_length,{{was,BitLen}, - {not_allowed_interval, - Max1,Min2}}}}}); - _ -> - encode_bitstring(BitListVal) - end; -encode_constr_bit_str_bits({Min,Max},BitListVal,_DoTag) -> - BitLen = length(BitListVal), - if - BitLen > Max -> - exit({error,{asn1,{bitstring_length,{{was,BitLen}, - {maximum,Max}}}}}); - BitLen < Min -> - exit({error,{asn1,{bitstring_length,{{was,BitLen}, - {minimum,Min}}}}}); - true -> - encode_bitstring(BitListVal) - end. - - -%% returns a list of length Size + length(BitListVal), with BitListVal -%% as the most significant elements followed by padded zero elements -pad_bit_list(Size,BitListVal) -> - Tail = lists:duplicate(Size,0), - BitListVal ++ Tail. - -%%================================================================= -%% Do the actual encoding -%% ([bitlist]) -> {ListLen, UnusedBits, OctetList} -%%================================================================= - -encode_bitstring([B8, B7, B6, B5, B4, B3, B2, B1 | Rest]) -> - Val = (B8 bsl 7) bor (B7 bsl 6) bor (B6 bsl 5) bor (B5 bsl 4) bor - (B4 bsl 3) bor (B3 bsl 2) bor (B2 bsl 1) bor B1, - encode_bitstring(Rest, [Val], 1); -encode_bitstring(Val) -> - {Unused, Octet} = unused_bitlist(Val, 7, 0), - {1, Unused, [Octet]}. - -encode_bitstring([B8, B7, B6, B5, B4, B3, B2, B1 | Rest], Ack, Len) -> - Val = (B8 bsl 7) bor (B7 bsl 6) bor (B6 bsl 5) bor (B5 bsl 4) bor - (B4 bsl 3) bor (B3 bsl 2) bor (B2 bsl 1) bor B1, - encode_bitstring(Rest, [Ack | [Val]], Len + 1); -%%even multiple of 8 bits.. -encode_bitstring([], Ack, Len) -> - {Len, 0, Ack}; -%% unused bits in last octet -encode_bitstring(Rest, Ack, Len) -> -% io:format("uneven ~w ~w ~w~n",[Rest, Ack, Len]), - {Unused, Val} = unused_bitlist(Rest, 7, 0), - {Len + 1, Unused, [Ack | [Val]]}. - -%%%%%%%%%%%%%%%%%% -%% unused_bitlist([list of ones and zeros <= 7], 7, []) -> -%% {Unused bits, Last octet with bits moved to right} -unused_bitlist([], Trail, Ack) -> - {Trail + 1, Ack}; -unused_bitlist([Bit | Rest], Trail, Ack) -> -%% io:format("trail Bit: ~w Rest: ~w Trail: ~w Ack:~w~n",[Bit, Rest, Trail, Ack]), - unused_bitlist(Rest, Trail - 1, (Bit bsl Trail) bor Ack). - - -%%============================================================================ -%% decode bitstring value -%% (Buffer, Range, NamedNumberList, HasTag, TotalLen) -> {Integer, Remain, RemovedBytes} -%%============================================================================ - -decode_compact_bit_string(Buffer, Range, NamedNumberList, Tags, LenIn, OptOrMand) -> -% NewTags = new_tags(HasTag,#tag{class=?UNIVERSAL,number=?N_BIT_STRING}), - decode_restricted_string(Buffer, Range, ?N_BIT_STRING, Tags, LenIn, - NamedNumberList, OptOrMand,bin). - -decode_bit_string(Buffer, Range, NamedNumberList, Tags, LenIn, OptOrMand) -> -% NewTags = new_tags(HasTag,#tag{class=?UNIVERSAL,number=?N_BIT_STRING}), - decode_restricted_string(Buffer, Range, ?N_BIT_STRING, Tags, LenIn, - NamedNumberList, OptOrMand,old). - - -decode_bit_string2(1,<<0 ,Buffer/binary>>,_NamedNumberList,RemovedBytes,BinOrOld) -> - case BinOrOld of - bin -> - {{0,<<>>},Buffer,RemovedBytes}; - _ -> - {[], Buffer, RemovedBytes} - end; -decode_bit_string2(Len,<<Unused,Buffer/binary>>,NamedNumberList, - RemovedBytes,BinOrOld) -> - L = Len - 1, - <<Bits:L/binary,BufferTail/binary>> = Buffer, - case NamedNumberList of - [] -> - case BinOrOld of - bin -> - {{Unused,Bits},BufferTail,RemovedBytes}; - _ -> - BitString = decode_bitstring2(L, Unused, Buffer), - {BitString,BufferTail, RemovedBytes} - end; - _ -> - BitString = decode_bitstring2(L, Unused, Buffer), - {decode_bitstring_NNL(BitString,NamedNumberList), - BufferTail, - RemovedBytes} - end. - -%%---------------------------------------- -%% Decode the in buffer to bits -%%---------------------------------------- -decode_bitstring2(1,Unused,<<B7:1,B6:1,B5:1,B4:1,B3:1,B2:1,B1:1,B0:1,_/binary>>) -> - lists:sublist([B7,B6,B5,B4,B3,B2,B1,B0],8-Unused); -decode_bitstring2(Len, Unused, - <<B7:1,B6:1,B5:1,B4:1,B3:1,B2:1,B1:1,B0:1,Buffer/binary>>) -> - [B7, B6, B5, B4, B3, B2, B1, B0 | - decode_bitstring2(Len - 1, Unused, Buffer)]. - -%%decode_bitstring2(1, Unused, Buffer) -> -%% make_bits_of_int(hd(Buffer), 128, 8-Unused); -%%decode_bitstring2(Len, Unused, [BitVal | Buffer]) -> -%% [B7, B6, B5, B4, B3, B2, B1, B0] = make_bits_of_int(BitVal, 128, 8), -%% [B7, B6, B5, B4, B3, B2, B1, B0 | -%% decode_bitstring2(Len - 1, Unused, Buffer)]. - - -%%make_bits_of_int(_, _, 0) -> -%% []; -%%make_bits_of_int(BitVal, MaskVal, Unused) when Unused > 0 -> -%% X = case MaskVal band BitVal of -%% 0 -> 0 ; -%% _ -> 1 -%% end, -%% [X | make_bits_of_int(BitVal, MaskVal bsr 1, Unused - 1)]. - - - -%%---------------------------------------- -%% Decode the bitlist to names -%%---------------------------------------- - -decode_bitstring_NNL(BitList,NamedNumberList) -> - decode_bitstring_NNL(BitList,NamedNumberList,0,[]). - - -decode_bitstring_NNL([],_,_No,Result) -> - lists:reverse(Result); - -decode_bitstring_NNL([B|BitList],[{Name,No}|NamedNumberList],No,Result) -> - if - B =:= 0 -> - decode_bitstring_NNL(BitList,NamedNumberList,No+1,Result); - true -> - decode_bitstring_NNL(BitList,NamedNumberList,No+1,[Name|Result]) - end; -decode_bitstring_NNL([1|BitList],NamedNumberList,No,Result) -> - decode_bitstring_NNL(BitList,NamedNumberList,No+1,[{bit,No}|Result]); -decode_bitstring_NNL([0|BitList],NamedNumberList,No,Result) -> - decode_bitstring_NNL(BitList,NamedNumberList,No+1,Result). - - -%%============================================================================ -%% Octet string, ITU_T X.690 Chapter 8.7 -%% -%% encode octet string -%% The OctetList must be a flat list of integers in the range 0..255 -%% the function does not check this because it takes to much time -%%============================================================================ -encode_octet_string(_C, OctetList, []) when is_binary(OctetList) -> - dotag_universal(?N_OCTET_STRING,OctetList,byte_size(OctetList)); -encode_octet_string(_C, OctetList, DoTag) when is_binary(OctetList) -> - dotag(DoTag, ?N_OCTET_STRING, {OctetList,byte_size(OctetList)}); -encode_octet_string(_C, OctetList, DoTag) when is_list(OctetList) -> - case length(OctetList) of - Len when DoTag =:= [] -> - dotag_universal(?N_OCTET_STRING,OctetList,Len); - Len -> - dotag(DoTag, ?N_OCTET_STRING, {OctetList,Len}) - end; -%% encode_octet_string(C, OctetList, DoTag) when is_list(OctetList) -> -%% dotag(DoTag, ?N_OCTET_STRING, {OctetList,length(OctetList)}); -encode_octet_string(C, {Name,OctetList}, DoTag) when is_atom(Name) -> - encode_octet_string(C, OctetList, DoTag). - - -%%============================================================================ -%% decode octet string -%% (Buffer, Range, HasTag, TotalLen) -> {String, Remain, RemovedBytes} -%% -%% Octet string is decoded as a restricted string -%%============================================================================ -decode_octet_string(Buffer, Range, Tags, TotalLen, OptOrMand) -> -%% NewTags = new_tags(HasTag,#tag{class=?UNIVERSAL,number=?N_OCTET_STRING}), - decode_restricted_string(Buffer, Range, ?N_OCTET_STRING, - Tags, TotalLen, [], OptOrMand,old). - -%%============================================================================ -%% Null value, ITU_T X.690 Chapter 8.8 -%% -%% encode NULL value -%%============================================================================ - -encode_null(_, []) -> - {[?N_NULL,0],2}; -encode_null(_, DoTag) -> - dotag(DoTag, ?N_NULL, {[],0}). - -%%============================================================================ -%% decode NULL value -%% (Buffer, HasTag, TotalLen) -> {NULL, Remain, RemovedBytes} -%%============================================================================ -decode_null(Buffer, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_NULL}), - decode_null_notag(Buffer, NewTags, OptOrMand). - -decode_null_notag(Buffer, Tags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {_Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = decode_null_notag(Buffer0, RestTags, - OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,0} -> - {'NULL', Buffer0, Rb0}; - {_,Len} -> - exit({error,{asn1,{invalid_length,'NULL',Len}}}) - end. - - -%%============================================================================ -%% Object identifier, ITU_T X.690 Chapter 8.19 -%% -%% encode Object Identifier value -%%============================================================================ - -encode_object_identifier({Name,Val}, DoTag) when is_atom(Name) -> - encode_object_identifier(Val, DoTag); -encode_object_identifier(Val, []) -> - {EncVal,Len} = e_object_identifier(Val), - dotag_universal(?N_OBJECT_IDENTIFIER,EncVal,Len); -encode_object_identifier(Val, DoTag) -> - dotag(DoTag, ?N_OBJECT_IDENTIFIER, e_object_identifier(Val)). - -e_object_identifier({'OBJECT IDENTIFIER', V}) -> - e_object_identifier(V); -e_object_identifier({Cname, V}) when is_atom(Cname), is_tuple(V) -> - e_object_identifier(tuple_to_list(V)); -e_object_identifier({Cname, V}) when is_atom(Cname), is_list(V) -> - e_object_identifier(V); -e_object_identifier(V) when is_tuple(V) -> - e_object_identifier(tuple_to_list(V)); - -%%%%%%%%%%%%%%% -%% e_object_identifier([List of Obect Identifiers]) -> -%% {[Encoded Octetlist of ObjIds], IntLength} -%% -e_object_identifier([E1, E2 | Tail]) -> - Head = 40*E1 + E2, % wow! - {H,Lh} = mk_object_val(Head), - {R,Lr} = enc_obj_id_tail(Tail, [], 0), - {[H|R], Lh+Lr}. - -enc_obj_id_tail([], Ack, Len) -> - {lists:reverse(Ack), Len}; -enc_obj_id_tail([H|T], Ack, Len) -> - {B, L} = mk_object_val(H), - enc_obj_id_tail(T, [B|Ack], Len+L). +encode_integer_neg(-1, L=[B1|_T]) when B1 > 127 -> + L; +encode_integer_neg(N, Acc) -> + encode_integer_neg(N bsr 8, [N band 16#ff|Acc]). %%%%%%%%%%% @@ -1643,476 +440,6 @@ mk_object_val(Val, Ack, Len) -> mk_object_val(Val bsr 7, [((Val band 127) bor 128) | Ack], Len + 1). - -%%============================================================================ -%% decode Object Identifier value -%% (Buffer, HasTag, TotalLen) -> {{ObjId}, Remain, RemovedBytes} -%%============================================================================ - -decode_object_identifier(Buffer, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL, - number=?N_OBJECT_IDENTIFIER}), - decode_object_identifier_notag(Buffer, NewTags, OptOrMand). - -decode_object_identifier_notag(Buffer, Tags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_object_identifier_notag(Buffer00, - RestTags, OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - {[AddedObjVal|ObjVals],Buffer01} = - dec_subidentifiers(Buffer0,0,[],Len), - {Val1, Val2} = if - AddedObjVal < 40 -> - {0, AddedObjVal}; - AddedObjVal < 80 -> - {1, AddedObjVal - 40}; - true -> - {2, AddedObjVal - 80} - end, - {list_to_tuple([Val1, Val2 | ObjVals]), Buffer01, - Rb0+Len} - end. - -dec_subidentifiers(Buffer,_Av,Al,0) -> - {lists:reverse(Al),Buffer}; -dec_subidentifiers(<<1:1,H:7,T/binary>>,Av,Al,Len) -> - dec_subidentifiers(T,(Av bsl 7) + H,Al,Len-1); -dec_subidentifiers(<<H,T/binary>>,Av,Al,Len) -> - dec_subidentifiers(T,0,[((Av bsl 7) + H)|Al],Len-1). - -%%============================================================================ -%% RELATIVE-OID, ITU_T X.690 Chapter 8.20 -%% -%% encode Relative Object Identifier -%%============================================================================ -encode_relative_oid({Name,Val},TagIn) when is_atom(Name) -> - encode_relative_oid(Val,TagIn); -encode_relative_oid(Val,TagIn) when is_tuple(Val) -> - encode_relative_oid(tuple_to_list(Val),TagIn); -encode_relative_oid(Val,[]) -> - {EncVal,Len} = enc_relative_oid(Val), - dotag_universal(?'N_RELATIVE-OID',EncVal,Len); -encode_relative_oid(Val, DoTag) -> - dotag(DoTag, ?'N_RELATIVE-OID', enc_relative_oid(Val)). - -enc_relative_oid(Val) -> - lists:mapfoldl(fun(X,AccIn) -> - {SO,L}=mk_object_val(X), - {SO,L+AccIn} - end - ,0,Val). - -%%============================================================================ -%% decode Relative Object Identifier value -%% (Buffer, HasTag, TotalLen) -> {{ObjId}, Remain, RemovedBytes} -%%============================================================================ -decode_relative_oid(Buffer, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL, - number=?'N_RELATIVE-OID'}), - decode_relative_oid_notag(Buffer, NewTags, OptOrMand). - -decode_relative_oid_notag(Buffer, Tags, OptOrMand) -> - {_RestTags, {_FormLen={_,Len}, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - {ObjVals,Buffer01} = - dec_subidentifiers(Buffer0,0,[],Len), - {list_to_tuple(ObjVals), Buffer01, Rb0+Len}. - -%%============================================================================ -%% Restricted character string types, ITU_T X.690 Chapter 8.21 -%% -%% encode Numeric Printable Teletex Videotex Visible IA5 Graphic General strings -%%============================================================================ -encode_restricted_string(_C, OctetList, StringType, []) - when is_binary(OctetList) -> - dotag_universal(StringType, OctetList, byte_size(OctetList)); -encode_restricted_string(_C, OctetList, StringType, DoTag) - when is_binary(OctetList) -> - dotag(DoTag, StringType, {OctetList, byte_size(OctetList)}); -encode_restricted_string(_C, OctetList, StringType, []) - when is_list(OctetList) -> - dotag_universal(StringType, OctetList, length(OctetList)); -encode_restricted_string(_C, OctetList, StringType, DoTag) - when is_list(OctetList) -> - dotag(DoTag, StringType, {OctetList, length(OctetList)}); -encode_restricted_string(C,{Name,OctetL},StringType,DoTag) when is_atom(Name) -> - encode_restricted_string(C, OctetL, StringType, DoTag). - -%%============================================================================ -%% decode Numeric Printable Teletex Videotex Visible IA5 Graphic General strings -%% (Buffer, Range, StringType, HasTag, TotalLen) -> -%% {String, Remain, RemovedBytes} -%%============================================================================ - -decode_restricted_string(Buffer, Range, StringType, Tags, LenIn, OptOrMand) -> - {Val,Buffer2,Rb} = - decode_restricted_string_tag(Buffer, Range, StringType, Tags, - LenIn, [], OptOrMand,old), - {check_and_convert_restricted_string(Val,StringType,Range,[],old), - Buffer2,Rb}. - - -decode_restricted_string(Buffer, Range, StringType, Tags, LenIn, NNList, OptOrMand, BinOrOld ) -> - {Val,Buffer2,Rb} = - decode_restricted_string_tag(Buffer, Range, StringType, Tags, - LenIn, NNList, OptOrMand, BinOrOld), - {check_and_convert_restricted_string(Val,StringType,Range,NNList,BinOrOld), - Buffer2,Rb}. - -decode_restricted_string_tag(Buffer, Range, StringType, TagsIn, LenIn, NNList, OptOrMand, BinOrOld ) -> - NewTags = new_tags(TagsIn, #tag{class=?UNIVERSAL,number=StringType}), - decode_restricted_string_notag(Buffer, Range, StringType, NewTags, - LenIn, NNList, OptOrMand, BinOrOld). - - -check_and_convert_restricted_string(Val,StringType,Range,NamedNumberList,_BinOrOld) -> - {StrLen,NewVal} = case StringType of - ?N_BIT_STRING when NamedNumberList =/= [] -> - {no_check,Val}; - ?N_BIT_STRING when is_list(Val) -> - {length(Val),Val}; - ?N_BIT_STRING when is_tuple(Val) -> - {(size(element(2,Val))*8) - element(1,Val),Val}; - _ when is_binary(Val) -> - {byte_size(Val),binary_to_list(Val)}; - _ when is_list(Val) -> - {length(Val), Val} - end, - case Range of - _ when StrLen =:= no_check -> - NewVal; - [] -> % No length constraint - NewVal; - {Lb,Ub} when StrLen >= Lb, Ub >= StrLen -> % variable length constraint - NewVal; - {{Lb,_Ub},[]} when StrLen >= Lb -> - NewVal; - {{Lb,_Ub},_Ext=[MinExt|_]} when StrLen >= Lb; StrLen >= MinExt -> - NewVal; - {{Lb1,Ub1},{Lb2,Ub2}} when StrLen >= Lb1, StrLen =< Ub1; - StrLen =< Ub2, StrLen >= Lb2 -> - NewVal; - StrLen -> % fixed length constraint - NewVal; - {_,_} -> - exit({error,{asn1,{length,Range,Val}}}); - _Len when is_integer(_Len) -> - exit({error,{asn1,{length,Range,Val}}}); - _ -> % some strange constraint that we don't support yet - NewVal - end. - -%%============================================================================= -%% Common routines for several string types including bit string -%% handles indefinite length -%%============================================================================= - - -decode_restricted_string_notag(Buffer, _Range, StringType, TagsIn, - _, NamedNumberList, OptOrMand, BinOrOld) -> - %%----------------------------------------------------------- - %% Get inner (the implicit tag or no tag) and - %% outer (the explicit tag) lengths. - %%----------------------------------------------------------- - {RestTags, {FormLength={_,_Len01}, Buffer0, Rb0}} = - check_tags_i(TagsIn, Buffer, OptOrMand), - - case FormLength of - {?CONSTRUCTED,Len} -> - {Buffer00, RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_restricted_parts(Buffer00, RestBytes, [], StringType, - RestTags, - Len, NamedNumberList, - OptOrMand, - BinOrOld, 0, []), - {Val01, Buffer01, Rb0+Rb01}; - {_, Len} -> - {Val01, Buffer01, Rb01} = - decode_restricted(Buffer0, Len, StringType, - NamedNumberList, BinOrOld), - {Val01, Buffer01, Rb0+Rb01} - end. - - -decode_restricted_parts(Buffer, RestBytes, [], StringType, RestTags, Len, NNList, - OptOrMand, BinOrOld, AccRb, AccVal) -> - DecodeFun = case RestTags of - [] -> fun decode_restricted_string_tag/8; - _ -> fun decode_restricted_string_notag/8 - end, - {Val, Buffer1, Rb} = - DecodeFun(Buffer, [], StringType, RestTags, - no_length, NNList, - OptOrMand, BinOrOld), - {Buffer2,More} = - case Buffer1 of - <<0,0,Buffer10/binary>> when Len == indefinite -> - {Buffer10,false}; - <<>> -> - {RestBytes,false}; - _ -> - {Buffer1,true} - end, - {NewVal, NewRb} = - case StringType of - ?N_BIT_STRING when BinOrOld == bin -> - {concat_bit_binaries(AccVal, Val), AccRb+Rb}; - _ when is_binary(Val),is_binary(AccVal) -> - {<<AccVal/binary,Val/binary>>,AccRb+Rb}; - _ when is_binary(Val), AccVal =:= [] -> - {Val,AccRb+Rb}; - _ -> - {AccVal++Val, AccRb+Rb} - end, - case More of - false -> - {NewVal, Buffer2, NewRb}; - true -> - decode_restricted_parts(Buffer2, RestBytes, [], StringType, RestTags, Len, NNList, - OptOrMand, BinOrOld, NewRb, NewVal) - end. - - - -decode_restricted(Buffer, InnerLen, StringType, NamedNumberList,BinOrOld) -> - - case StringType of - ?N_BIT_STRING -> - decode_bit_string2(InnerLen,Buffer,NamedNumberList,InnerLen,BinOrOld); - - ?N_UniversalString -> - <<PreBuff:InnerLen/binary,RestBuff/binary>> = Buffer,%%added for binary - UniString = mk_universal_string(binary_to_list(PreBuff)), - {UniString,RestBuff,InnerLen}; - ?N_BMPString -> - <<PreBuff:InnerLen/binary,RestBuff/binary>> = Buffer,%%added for binary - BMP = mk_BMP_string(binary_to_list(PreBuff)), - {BMP,RestBuff,InnerLen}; - _ -> - <<PreBuff:InnerLen/binary,RestBuff/binary>> = Buffer,%%added for binary - {PreBuff, RestBuff, InnerLen} - end. - - - -%%============================================================================ -%% encode Universal string -%%============================================================================ - -encode_universal_string(C, {Name, Universal}, DoTag) when is_atom(Name) -> - encode_universal_string(C, Universal, DoTag); -encode_universal_string(_C, Universal, []) -> - OctetList = mk_uni_list(Universal), - dotag_universal(?N_UniversalString,OctetList,length(OctetList)); -encode_universal_string(_C, Universal, DoTag) -> - OctetList = mk_uni_list(Universal), - dotag(DoTag, ?N_UniversalString, {OctetList,length(OctetList)}). - -mk_uni_list(In) -> - mk_uni_list(In,[]). - -mk_uni_list([],List) -> - lists:reverse(List); -mk_uni_list([{A,B,C,D}|T],List) -> - mk_uni_list(T,[D,C,B,A|List]); -mk_uni_list([H|T],List) -> - mk_uni_list(T,[H,0,0,0|List]). - -%%=========================================================================== -%% decode Universal strings -%% (Buffer, Range, StringType, HasTag, LenIn) -> -%% {String, Remain, RemovedBytes} -%%=========================================================================== - -decode_universal_string(Buffer, Range, Tags, LenIn, OptOrMand) -> -% NewTags = new_tags(HasTag, #tag{class=?UNIVERSAL,number=?N_UniversalString}), - decode_restricted_string(Buffer, Range, ?N_UniversalString, - Tags, LenIn, [], OptOrMand,old). - - -mk_universal_string(In) -> - mk_universal_string(In,[]). - -mk_universal_string([],Acc) -> - lists:reverse(Acc); -mk_universal_string([0,0,0,D|T],Acc) -> - mk_universal_string(T,[D|Acc]); -mk_universal_string([A,B,C,D|T],Acc) -> - mk_universal_string(T,[{A,B,C,D}|Acc]). - - -%%============================================================================ -%% encode UTF8 string -%%============================================================================ -encode_UTF8_string(_,UTF8String,[]) when is_binary(UTF8String) -> - dotag_universal(?N_UTF8String,UTF8String,byte_size(UTF8String)); -encode_UTF8_string(_,UTF8String,DoTag) when is_binary(UTF8String) -> - dotag(DoTag,?N_UTF8String,{UTF8String,byte_size(UTF8String)}); -encode_UTF8_string(_,UTF8String,[]) -> - dotag_universal(?N_UTF8String,UTF8String,length(UTF8String)); -encode_UTF8_string(_,UTF8String,DoTag) -> - dotag(DoTag,?N_UTF8String,{UTF8String,length(UTF8String)}). - -%%============================================================================ -%% decode UTF8 string -%%============================================================================ - -decode_UTF8_string(Buffer, Tags, OptOrMand) -> - NewTags = new_tags(Tags, #tag{class=?UNIVERSAL,number=?N_UTF8String}), - decode_UTF8_string_notag(Buffer, NewTags, OptOrMand). - -decode_UTF8_string_notag(Buffer, Tags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - case FormLen of - {?CONSTRUCTED,Len} -> - %% an UTF8String may be encoded as a constructed type - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_UTF8_string_notag(Buffer00,RestTags,OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - <<Result:Len/binary,RestBuff/binary>> = Buffer0, - {Result,RestBuff,Rb0 + Len} - end. - - -%%============================================================================ -%% encode BMP string -%%============================================================================ - -encode_BMP_string(C, {Name,BMPString}, DoTag) when is_atom(Name) -> - encode_BMP_string(C, BMPString, DoTag); -encode_BMP_string(_C, BMPString, []) -> - OctetList = mk_BMP_list(BMPString), - dotag_universal(?N_BMPString,OctetList,length(OctetList)); -encode_BMP_string(_C, BMPString, DoTag) -> - OctetList = mk_BMP_list(BMPString), - dotag(DoTag, ?N_BMPString, {OctetList,length(OctetList)}). - -mk_BMP_list(In) -> - mk_BMP_list(In,[]). - -mk_BMP_list([],List) -> - lists:reverse(List); -mk_BMP_list([{0,0,C,D}|T],List) -> - mk_BMP_list(T,[D,C|List]); -mk_BMP_list([H|T],List) -> - mk_BMP_list(T,[H,0|List]). - -%%============================================================================ -%% decode (OctetList, Range(ignored), tag|notag) -> {ValList, RestList} -%% (Buffer, Range, StringType, HasTag, TotalLen) -> -%% {String, Remain, RemovedBytes} -%%============================================================================ -decode_BMP_string(Buffer, Range, Tags, LenIn, OptOrMand) -> -% NewTags = new_tags(HasTag, #tag{class=?UNIVERSAL,number=?N_BMPString}), - decode_restricted_string(Buffer, Range, ?N_BMPString, - Tags, LenIn, [], OptOrMand,old). - -mk_BMP_string(In) -> - mk_BMP_string(In,[]). - -mk_BMP_string([],US) -> - lists:reverse(US); -mk_BMP_string([0,B|T],US) -> - mk_BMP_string(T,[B|US]); -mk_BMP_string([C,D|T],US) -> - mk_BMP_string(T,[{0,0,C,D}|US]). - - -%%============================================================================ -%% Generalized time, ITU_T X.680 Chapter 39 -%% -%% encode Generalized time -%%============================================================================ - -encode_generalized_time(C, {Name,OctetList}, DoTag) when is_atom(Name) -> - encode_generalized_time(C, OctetList, DoTag); -encode_generalized_time(_C, OctetList, []) -> - dotag_universal(?N_GeneralizedTime,OctetList,length(OctetList)); -encode_generalized_time(_C, OctetList, DoTag) -> - dotag(DoTag, ?N_GeneralizedTime, {OctetList,length(OctetList)}). - -%%============================================================================ -%% decode Generalized time -%% (Buffer, Range, HasTag, TotalLen) -> {String, Remain, RemovedBytes} -%%============================================================================ - -decode_generalized_time(Buffer, Range, Tags, TotalLen, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL, - number=?N_GeneralizedTime}), - decode_generalized_time_notag(Buffer, Range, NewTags, TotalLen, OptOrMand). - -decode_generalized_time_notag(Buffer, Range, Tags, TotalLen, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_generalized_time_notag(Buffer00, Range, - RestTags, TotalLen, - OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - <<PreBuff:Len/binary,RestBuff/binary>> = Buffer0, - {binary_to_list(PreBuff), RestBuff, Rb0+Len} - end. - -%%============================================================================ -%% Universal time, ITU_T X.680 Chapter 40 -%% -%% encode UTC time -%%============================================================================ - -encode_utc_time(C, {Name,OctetList}, DoTag) when is_atom(Name) -> - encode_utc_time(C, OctetList, DoTag); -encode_utc_time(_C, OctetList, []) -> - dotag_universal(?N_UTCTime, OctetList,length(OctetList)); -encode_utc_time(_C, OctetList, DoTag) -> - dotag(DoTag, ?N_UTCTime, {OctetList,length(OctetList)}). - -%%============================================================================ -%% decode UTC time -%% (Buffer, Range, HasTag, TotalLen) -> {String, Remain, RemovedBytes} -%%============================================================================ - -decode_utc_time(Buffer, Range, Tags, TotalLen, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_UTCTime}), - decode_utc_time_notag(Buffer, Range, NewTags, TotalLen, OptOrMand). - -decode_utc_time_notag(Buffer, Range, Tags, TotalLen, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_utc_time_notag(Buffer00, Range, - RestTags, TotalLen, - OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - <<PreBuff:Len/binary,RestBuff/binary>> = Buffer0, - {binary_to_list(PreBuff), RestBuff, Rb0+Len} - end. - - %%============================================================================ %% Length handling %% @@ -2122,8 +449,6 @@ decode_utc_time_notag(Buffer, Range, Tags, TotalLen, OptOrMand) -> %% [<127]| [128 + Int (<127),OctetList] | [16#80] %%============================================================================ -encode_length(indefinite) -> - {[16#80],1}; % 128 encode_length(L) when L =< 16#7F -> {[L],1}; encode_length(L) -> @@ -2162,242 +487,12 @@ decode_length(<<1:1,LL:7,T/binary>>) -> <<Length:LL/unit:8,Rest/binary>> = T, {{Length,Rest}, LL+1}. -%decode_length([128 | T]) -> -% {{indefinite, T},1}; -%decode_length([H | T]) when H =< 127 -> -% {{H, T},1}; -%decode_length([H | T]) -> -% dec_long_length(H band 16#7F, T, 0, 1). - - -%%dec_long_length(0, Buffer, Acc, Len) -> -%% {{Acc, Buffer},Len}; -%%dec_long_length(Bytes, [H | T], Acc, Len) -> -%% dec_long_length(Bytes - 1, T, (Acc bsl 8) + H, Len+1). - -%%=========================================================================== -%% Decode tag and length -%% -%% decode_tag_and_length(Buffer) -> {Tag, Len, RemainingBuffer, RemovedBytes} -%% -%%=========================================================================== - -decode_tag_and_length(Buffer) -> - {Tag, Buffer2, RemBytesTag} = decode_tag(Buffer), - {{Len, Buffer3}, RemBytesLen} = decode_length(Buffer2), - {Tag, Len, Buffer3, RemBytesTag+RemBytesLen}. - - -%%============================================================================ -%% Check if valid tag -%% -%% check_if_valid_tag(Tag, List_of_valid_tags, OptOrMand) -> name of the tag -%%============================================================================ - -check_if_valid_tag(<<0,0,_/binary>>,_,_) -> - asn1_EOC; -check_if_valid_tag(<<>>, _, OptOrMand) -> - check_if_valid_tag2_error([], OptOrMand); -check_if_valid_tag(Bytes, ListOfTags, OptOrMand) when is_binary(Bytes) -> - {Tag, _, _} = decode_tag(Bytes), - check_if_valid_tag(Tag, ListOfTags, OptOrMand); - -%% This alternative should be removed in the near future -%% Bytes as input should be the only necessary call -check_if_valid_tag(Tag, ListOfTags, OptOrMand) -> - {Class, _Form, TagNo} = Tag, - C = code_class(Class), - T = case C of - 'UNIVERSAL' -> - code_type(TagNo); - _ -> - TagNo - end, - check_if_valid_tag2({C,T}, ListOfTags, Tag, OptOrMand). - -check_if_valid_tag2(_Class_TagNo, [], Tag, MandOrOpt) -> - check_if_valid_tag2_error(Tag,MandOrOpt); -check_if_valid_tag2(Class_TagNo, [{TagName,TagList}|T], Tag, OptOrMand) -> - case check_if_valid_tag_loop(Class_TagNo, TagList) of - true -> - TagName; - false -> - check_if_valid_tag2(Class_TagNo, T, Tag, OptOrMand) - end. - --spec check_if_valid_tag2_error(term(), atom()) -> no_return(). - -check_if_valid_tag2_error(Tag,mandatory) -> - exit({error,{asn1,{invalid_tag,Tag}}}); -check_if_valid_tag2_error(Tag,_) -> - exit({error,{asn1,{no_optional_tag,Tag}}}). - -check_if_valid_tag_loop(_Class_TagNo,[]) -> - false; -check_if_valid_tag_loop(Class_TagNo,[H|T]) -> - %% It is not possible to distinguish between SEQUENCE OF and SEQUENCE, and - %% between SET OF and SET because both are coded as 16 and 17, respectively. - H_without_OF = case H of - {C, 'SEQUENCE OF'} -> - {C, 'SEQUENCE'}; - {C, 'SET OF'} -> - {C, 'SET'}; - Else -> - Else - end, - - case H_without_OF of - Class_TagNo -> - true; - {_,_} -> - check_if_valid_tag_loop(Class_TagNo,T); - _ -> - check_if_valid_tag_loop(Class_TagNo,H), - check_if_valid_tag_loop(Class_TagNo,T) - end. - - - -code_class(0) -> 'UNIVERSAL'; -code_class(16#40) -> 'APPLICATION'; -code_class(16#80) -> 'CONTEXT'; -code_class(16#C0) -> 'PRIVATE'. - - -code_type(1) -> 'BOOLEAN'; -code_type(2) -> 'INTEGER'; -code_type(3) -> 'BIT STRING'; -code_type(4) -> 'OCTET STRING'; -code_type(5) -> 'NULL'; -code_type(6) -> 'OBJECT IDENTIFIER'; -code_type(7) -> 'ObjectDescriptor'; -code_type(8) -> 'EXTERNAL'; -code_type(9) -> 'REAL'; -code_type(10) -> 'ENUMERATED'; -code_type(11) -> 'EMBEDDED_PDV'; -code_type(16) -> 'SEQUENCE'; -% code_type(16) -> 'SEQUENCE OF'; -code_type(17) -> 'SET'; -% code_type(17) -> 'SET OF'; -code_type(18) -> 'NumericString'; -code_type(19) -> 'PrintableString'; -code_type(20) -> 'TeletexString'; -code_type(21) -> 'VideotexString'; -code_type(22) -> 'IA5String'; -code_type(23) -> 'UTCTime'; -code_type(24) -> 'GeneralizedTime'; -code_type(25) -> 'GraphicString'; -code_type(26) -> 'VisibleString'; -code_type(27) -> 'GeneralString'; -code_type(28) -> 'UniversalString'; -code_type(30) -> 'BMPString'; -code_type(Else) -> exit({error,{asn1,{unrecognized_type,Else}}}). - -%%------------------------------------------------------------------------- -%% decoding of the components of a SET -%%------------------------------------------------------------------------- - -decode_set(Rb, indefinite, <<0,0,Bytes/binary>>, _OptOrMand, _Fun3, Acc) -> - {lists:reverse(Acc),Bytes,Rb+2}; - -decode_set(Rb, indefinite, Bytes, OptOrMand, Fun3, Acc) -> - case Fun3(Bytes, OptOrMand) of - {_Term, _Remain, 0} -> - {lists:reverse(Acc),Bytes,Rb}; - {Term, Remain, Rb1} -> - Fun3(Bytes, OptOrMand), - decode_set(Rb+Rb1, indefinite, Remain, OptOrMand, Fun3, [Term|Acc]) - end; -%% {Term, Remain, Rb1} = Fun3(Bytes, OptOrMand), -%% decode_set(Rb+Rb1, indefinite, Remain, OptOrMand, Fun3, [Term|Acc]); - -decode_set(Rb, Num, Bytes, _OptOrMand, _Fun3, Acc) when Num == 0 -> - {lists:reverse(Acc), Bytes, Rb}; - -decode_set(_, Num, _, _, _, _) when Num < 0 -> - exit({error,{asn1,{length_error,'SET'}}}); - -decode_set(Rb, Num, Bytes, OptOrMand, Fun3, Acc) -> - case Fun3(Bytes, OptOrMand) of - {_Term, _Remain, 0} -> - {lists:reverse(Acc),Bytes,Rb}; - {Term, Remain, Rb1} -> - Fun3(Bytes, OptOrMand), - decode_set(Rb+Rb1, Num-Rb1, Remain, OptOrMand, Fun3, [Term|Acc]) - end. -%% {Term, Remain, Rb1} = Fun3(Bytes, OptOrMand), -%% decode_set(Rb+Rb1, Num-Rb1, Remain, OptOrMand, Fun3, [Term|Acc]). - - -%%------------------------------------------------------------------------- -%% decoding of SEQUENCE OF and SET OF -%%------------------------------------------------------------------------- - -decode_components(Rb, indefinite, <<0,0,Bytes/binary>>, _Fun3, _TagIn, Acc) -> - {lists:reverse(Acc),Bytes,Rb+2}; - -decode_components(Rb, indefinite, Bytes, Fun3, TagIn, Acc) -> - {Term, Remain, Rb1} = Fun3(Bytes, mandatory, TagIn), - decode_components(Rb+Rb1, indefinite, Remain, Fun3, TagIn, [Term|Acc]); - -decode_components(Rb, Num, Bytes, _Fun3, _TagIn, Acc) when Num == 0 -> - {lists:reverse(Acc), Bytes, Rb}; - -decode_components(_, Num, _, _, _, _) when Num < 0 -> - exit({error,{asn1,{length_error,'SET/SEQUENCE OF'}}}); - -decode_components(Rb, Num, Bytes, Fun3, TagIn, Acc) -> - {Term, Remain, Rb1} = Fun3(Bytes, mandatory, TagIn), - decode_components(Rb+Rb1, Num-Rb1, Remain, Fun3, TagIn, [Term|Acc]). - -%%decode_components(Rb, indefinite, [0,0|Bytes], _Fun3, _TagIn, Acc) -> -%% {lists:reverse(Acc),Bytes,Rb+2}; - -decode_components(Rb, indefinite, <<0,0,Bytes/binary>>, _Fun4, _TagIn, _Fun, Acc) -> - {lists:reverse(Acc),Bytes,Rb+2}; - -decode_components(Rb, indefinite, Bytes, _Fun4, TagIn, _Fun, Acc) -> - {Term, Remain, Rb1} = _Fun4(Bytes, mandatory, TagIn, _Fun), - decode_components(Rb+Rb1, indefinite, Remain, _Fun4, TagIn, _Fun, [Term|Acc]); - -decode_components(Rb, Num, Bytes, _Fun4, _TagIn, _Fun, Acc) when Num == 0 -> - {lists:reverse(Acc), Bytes, Rb}; - -decode_components(_, Num, _, _, _, _, _) when Num < 0 -> - exit({error,{asn1,{length_error,'SET/SEQUENCE OF'}}}); - -decode_components(Rb, Num, Bytes, _Fun4, TagIn, _Fun, Acc) -> - {Term, Remain, Rb1} = _Fun4(Bytes, mandatory, TagIn, _Fun), - decode_components(Rb+Rb1, Num-Rb1, Remain, _Fun4, TagIn, _Fun, [Term|Acc]). - - - -%%------------------------------------------------------------------------- -%% INTERNAL HELPER FUNCTIONS (not exported) -%%------------------------------------------------------------------------- - - -%%========================================================================== -%% Encode tag -%% -%% dotag(tag | notag, TagValpattern | TagValTuple, [Length, Value]) -> [Tag] -%% TagValPattern is a correct bitpattern for a tag -%% TagValTuple is a tuple of three bitpatterns, Class, Form and TagNo where -%% Class = UNIVERSAL | APPLICATION | CONTEXT | PRIVATE -%% Form = Primitive | Constructed -%% TagNo = Number of tag -%%========================================================================== - dotag([], Tag, {Bytes,Len}) -> dotag_universal(Tag,Bytes,Len); dotag(Tags, Tag, {Bytes,Len}) -> encode_tags(Tags ++ [#tag{class=?UNIVERSAL,number=Tag,form=?PRIMITIVE}], - Bytes, Len); - -dotag(Tags, Tag, Bytes) -> - encode_tags(Tags ++ [#tag{class=?UNIVERSAL,number=Tag,form=?PRIMITIVE}], - Bytes, size(Bytes)). + Bytes, Len). dotag_universal(UniversalTag,Bytes,Len) when Len =< 16#7F-> {[UniversalTag,Len,Bytes],2+Len}; @@ -2415,47 +510,6 @@ decode_integer2(Len,<<1:1,B2:7,Bs/binary>>,RemovedBytes) -> Int = N - (1 bsl (8 * Len - 1)), {Int,Buffer2,RemovedBytes}. -%%decode_integer2(Len,Buffer,Acc,RemovedBytes) when (hd(Buffer) band 16#FF) =< 16#7F -> -%% {decode_integer_pos(Buffer, 8 * (Len - 1)),skip(Buffer,Len),RemovedBytes}; -%%decode_integer2(Len,Buffer,Acc,RemovedBytes) -> -%% {decode_integer_neg(Buffer, 8 * (Len - 1)),skip(Buffer,Len),RemovedBytes}. - -%%decode_integer_pos([Byte|Tail], Shift) -> -%% (Byte bsl Shift) bor decode_integer_pos(Tail, Shift-8); -%%decode_integer_pos([], _) -> 0. - - -%%decode_integer_neg([Byte|Tail], Shift) -> -%% (-128 + (Byte band 127) bsl Shift) bor decode_integer_pos(Tail, Shift-8). - - -concat_bit_binaries([],Bin={_,_}) -> - Bin; -concat_bit_binaries({0,B1},{U2,B2}) -> - {U2,<<B1/binary,B2/binary>>}; -concat_bit_binaries({U1,B1},{U2,B2}) -> - S1 = (size(B1) * 8) - U1, - S2 = (size(B2) * 8) - U2, - PadBits = 8 - ((S1+S2) rem 8), - {PadBits, <<B1:S1/binary-unit:1,B2:S2/binary-unit:1,0:PadBits>>}; -concat_bit_binaries(L1,L2) when is_list(L1), is_list(L2) -> - %% this case occur when decoding with NNL - L1 ++ L2. - - -get_constraint(C,Key) -> - case lists:keyfind(Key,1,C) of - false -> - no; - {_, V} -> - V - end. - -%%skip(Buffer, 0) -> -%% Buffer; -%%skip([H | T], Len) -> -%% skip(T, Len-1). - new_tags([],LastTag) -> [LastTag]; new_tags(Tags = [#tag{type='IMPLICIT'}],_LastTag) -> diff --git a/lib/asn1/src/asn1rt_ber_bin_v2.erl b/lib/asn1/src/asn1rt_ber_bin_v2.erl index 9ff5017c68..92ca11cf89 100644 --- a/lib/asn1/src/asn1rt_ber_bin_v2.erl +++ b/lib/asn1/src/asn1rt_ber_bin_v2.erl @@ -22,8 +22,7 @@ %% encoding / decoding of BER -export([decode/1, decode/2, match_tags/2, encode/1, encode/2]). --export([fixoptionals/2, cindex/3, - list_to_record/2, +-export([fixoptionals/2, encode_tag_val/1, encode_tags/3, skip_ExtensionAdditions/2]). @@ -54,8 +53,6 @@ decode_open_type_as_binary/3]). -export([decode_primitive_incomplete/2,decode_selective/2]). - --export([is_nif_loadable/0]). % the encoding of class of tag bits 8 and 7 -define(UNIVERSAL, 0). @@ -135,12 +132,7 @@ encode(Tlv,_) when is_binary(Tlv) -> encode([Tlv],Method) -> encode(Tlv,Method); encode(Tlv, nif) -> - case is_nif_loadable() of - true -> - asn1rt_nif:encode_ber_tlv(Tlv); - false -> - encode_erl(Tlv) - end; + asn1rt_nif:encode_ber_tlv(Tlv); encode(Tlv, _) -> encode_erl(Tlv). @@ -178,36 +170,15 @@ decode(B) -> %% asn1-1.7 decode(B, nif) -> - case is_nif_loadable() of - true -> - case asn1rt_nif:decode_ber_tlv(B) of - {error, Reason} -> handle_error(Reason, B); - Else -> Else - end; - false -> - decode(B) + case asn1rt_nif:decode_ber_tlv(B) of + {error, Reason} -> handle_error(Reason, B); + Else -> Else end; decode(B,erlang) when is_binary(B) -> decode_primitive(B); decode(Tlv,erlang) -> {Tlv,<<>>}. -%% Have to check this since asn1 is not guaranteed to be available -is_nif_loadable() -> - case application:get_env(asn1, nif_loadable) of - {ok,R} -> - R; - undefined -> - case catch code:load_file(asn1rt_nif) of - {module, asn1rt_nif} -> - application:set_env(asn1, nif_loadable, true), - true; - _Else -> - application:set_env(asn1, nif_loadable, false), - false - end - end. - handle_error([],_)-> exit({error,{asn1,{"memory allocation problem"}}}); handle_error({$1,_},L) -> % error in nif @@ -612,13 +583,6 @@ match_tags(Tlv, []) -> Tlv; match_tags(Tlv = {Tag,_V},[T|_Tt]) -> exit({error,{asn1,{wrong_tag,{{expected,T},{got,Tag,Tlv}}}}}). - - -cindex(Ix,Val,Cname) -> - case element(Ix,Val) of - {Cname,Val2} -> Val2; - X -> X - end. %%% %% skips components that do not match a tag in Tags @@ -642,13 +606,6 @@ skip_ExtensionAdditions(TLV=[{Tag,_}|Rest],Tags) -> %%=============================================================================== %%=============================================================================== -% converts a list to a record if necessary -list_to_record(Name,List) when is_list(List) -> - list_to_tuple([Name|List]); -list_to_record(_Name,Tuple) when is_tuple(Tuple) -> - Tuple. - - fixoptionals(OptList,Val) when is_list(Val) -> fixoptionals(OptList,Val,1,[],[]). diff --git a/lib/asn1/src/asn1rt_per_bin.erl b/lib/asn1/src/asn1rt_per_bin.erl deleted file mode 100644 index 5772f09bf4..0000000000 --- a/lib/asn1/src/asn1rt_per_bin.erl +++ /dev/null @@ -1,2285 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2001-2012. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% -%% --module(asn1rt_per_bin). -%% encoding / decoding of PER aligned - --include("asn1_records.hrl"). - --export([dec_fixup/3, cindex/3, list_to_record/2]). --export([setchoiceext/1, setext/1, fixoptionals/2, fixoptionals/3, - fixextensions/2, - getext/1, getextension/2, skipextensions/3, getbit/1, getchoice/3 ]). --export([getoptionals/2, getoptionals2/2, set_choice/3, encode_integer/2, encode_integer/3 ]). --export([decode_integer/2, decode_integer/3, encode_small_number/1, encode_boolean/1, - decode_boolean/1, encode_length/2, decode_length/1, decode_length/2, - encode_small_length/1, decode_small_length/1, - decode_compact_bit_string/3]). --export([decode_enumerated/3, - encode_bit_string/3, decode_bit_string/3 ]). --export([encode_octet_string/2, decode_octet_string/2, - encode_null/1, decode_null/1, - encode_object_identifier/1, decode_object_identifier/1, - encode_real/1, decode_real/1, - encode_relative_oid/1, decode_relative_oid/1, - complete/1]). - - --export([encode_open_type/2, decode_open_type/2]). - --export([encode_UniversalString/2, decode_UniversalString/2, - encode_PrintableString/2, decode_PrintableString/2, - encode_GeneralString/2, decode_GeneralString/2, - encode_GraphicString/2, decode_GraphicString/2, - encode_TeletexString/2, decode_TeletexString/2, - encode_VideotexString/2, decode_VideotexString/2, - encode_VisibleString/2, decode_VisibleString/2, - encode_UTF8String/1, decode_UTF8String/1, - encode_BMPString/2, decode_BMPString/2, - encode_IA5String/2, decode_IA5String/2, - encode_NumericString/2, decode_NumericString/2, - encode_ObjectDescriptor/2, decode_ObjectDescriptor/1 - ]). --export([complete_bytes/1, getbits/2, getoctets/2, minimum_bits/1]). - --define('16K',16384). --define('32K',32768). --define('64K',65536). - -dec_fixup(Terms,Cnames,RemBytes) -> - dec_fixup(Terms,Cnames,RemBytes,[]). - -dec_fixup([novalue|T],[_Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,Acc); -dec_fixup([{_Name,novalue}|T],[_Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,Acc); -dec_fixup([H|T],[Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,[{Hc,H}|Acc]); -dec_fixup([],_Cnames,RemBytes,Acc) -> - {lists:reverse(Acc),RemBytes}. - -cindex(Ix,Val,Cname) -> - case element(Ix,Val) of - {Cname,Val2} -> Val2; - X -> X - end. - -%% converts a list to a record if necessary -list_to_record(_Name,Tuple) when is_tuple(Tuple) -> - Tuple; -list_to_record(Name,List) when is_list(List) -> - list_to_tuple([Name|List]). - -%%-------------------------------------------------------- -%% setchoiceext(InRootSet) -> [{bit,X}] -%% X is set to 1 when InRootSet==false -%% X is set to 0 when InRootSet==true -%% -setchoiceext(true) -> - [{debug,choiceext},{bits,1,0}]; -setchoiceext(false) -> - [{debug,choiceext},{bits,1,1}]. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% setext(true|false) -> CompleteList -%% - -setext(false) -> - [{debug,ext},{bits,1,0}]; -setext(true) -> - [{debug,ext},{bits,1,1}]. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% This version of fixoptionals/2 are left only because of -%% backward compatibility with older generates - -fixoptionals(OptList,Val) when is_tuple(Val) -> - fixoptionals1(OptList,Val,[]); - -fixoptionals(OptList,Val) when is_list(Val) -> - fixoptionals1(OptList,Val,1,[],[]). - -fixoptionals1([],Val,Acc) -> - %% return {Val,Opt} - {Val,lists:reverse(Acc)}; -fixoptionals1([{_,Pos}|Ot],Val,Acc) -> - case element(Pos+1,Val) of - asn1_NOVALUE -> fixoptionals1(Ot,Val,[0|Acc]); - asn1_DEFAULT -> fixoptionals1(Ot,Val,[0|Acc]); - _ -> fixoptionals1(Ot,Val,[1|Acc]) - end. - - -fixoptionals1([{Name,Pos}|Ot],[{Name,Val}|Vt],_Opt,Acc1,Acc2) -> - fixoptionals1(Ot,Vt,Pos+1,[1|Acc1],[{Name,Val}|Acc2]); -fixoptionals1([{_Name,Pos}|Ot],V,Pos,Acc1,Acc2) -> - fixoptionals1(Ot,V,Pos+1,[0|Acc1],[asn1_NOVALUE|Acc2]); -fixoptionals1(O,[Vh|Vt],Pos,Acc1,Acc2) -> - fixoptionals1(O,Vt,Pos+1,Acc1,[Vh|Acc2]); -fixoptionals1([],[Vh|Vt],Pos,Acc1,Acc2) -> - fixoptionals1([],Vt,Pos+1,Acc1,[Vh|Acc2]); -fixoptionals1([],[],_,Acc1,Acc2) -> - % return {Val,Opt} - {list_to_tuple([asn1_RECORDNAME|lists:reverse(Acc2)]),lists:reverse(Acc1)}. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% This is the new fixoptionals/3 which is used by the new generates -%% -fixoptionals(OptList,OptLength,Val) when is_tuple(Val) -> - Bits = fixoptionals(OptList,Val,0), - {Val,{bits,OptLength,Bits}}; - -fixoptionals([],_Val,Acc) -> - %% Optbits - Acc; -fixoptionals([{Pos,DefVal}|Ot],Val,Acc) -> - case element(Pos,Val) of - asn1_DEFAULT -> fixoptionals(Ot,Val,Acc bsl 1); - DefVal -> fixoptionals(Ot,Val,Acc bsl 1); - _ -> fixoptionals(Ot,Val,(Acc bsl 1) + 1) - end; -fixoptionals([Pos|Ot],Val,Acc) -> - case element(Pos,Val) of - asn1_NOVALUE -> fixoptionals(Ot,Val,Acc bsl 1); - asn1_DEFAULT -> fixoptionals(Ot,Val,Acc bsl 1); - _ -> fixoptionals(Ot,Val,(Acc bsl 1) + 1) - end. - - -getext(Bytes) when is_tuple(Bytes) -> - getbit(Bytes); -getext(Bytes) when is_binary(Bytes) -> - getbit({0,Bytes}). - -getextension(0, Bytes) -> - {{},Bytes}; -getextension(1, Bytes) -> - {Len,Bytes2} = decode_small_length(Bytes), - {Blist, Bytes3} = getbits_as_list(Len,Bytes2), - {list_to_tuple(Blist),Bytes3}. - -fixextensions({ext,ExtPos,ExtNum},Val) -> - case fixextensions(ExtPos,ExtNum+ExtPos,Val,0) of - 0 -> []; - ExtBits -> - [encode_small_length(ExtNum),{bits,ExtNum,ExtBits}] - end. - -fixextensions(Pos,MaxPos,_,Acc) when Pos >= MaxPos -> - Acc; -fixextensions(Pos,ExtPos,Val,Acc) -> - Bit = case catch(element(Pos+1,Val)) of - asn1_NOVALUE -> - 0; - asn1_NOEXTVALUE -> - 0; - {'EXIT',_} -> - 0; - _ -> - 1 - end, - fixextensions(Pos+1,ExtPos,Val,(Acc bsl 1)+Bit). - -skipextensions(Bytes,Nr,ExtensionBitPattern) -> - case (catch element(Nr,ExtensionBitPattern)) of - 1 -> - {_,Bytes2} = decode_open_type(Bytes,[]), - skipextensions(Bytes2, Nr+1, ExtensionBitPattern); - 0 -> - skipextensions(Bytes, Nr+1, ExtensionBitPattern); - {'EXIT',_} -> % badarg, no more extensions - Bytes - end. - - -getchoice(Bytes,1,0) -> % only 1 alternative is not encoded - {0,Bytes}; -getchoice(Bytes,_,1) -> - decode_small_number(Bytes); -getchoice(Bytes,NumChoices,0) -> - decode_constrained_number(Bytes,{0,NumChoices-1}). - -%% old version kept for backward compatibility with generates from R7B -getoptionals(Bytes,NumOpt) -> - {Blist,Bytes1} = getbits_as_list(NumOpt,Bytes), - {list_to_tuple(Blist),Bytes1}. - -%% new version used in generates from r8b_patch/3 and later -getoptionals2(Bytes,NumOpt) -> - getbits(Bytes,NumOpt). - - -%% getbits_as_binary(Num,Bytes) -> {{Unused,BinBits},RestBytes}, -%% Num = integer(), -%% Bytes = list() | tuple(), -%% Unused = integer(), -%% BinBits = binary(), -%% RestBytes = tuple() -getbits_as_binary(Num,Bytes) when is_binary(Bytes) -> - getbits_as_binary(Num,{0,Bytes}); -getbits_as_binary(0,Buffer) -> - {{0,<<>>},Buffer}; -getbits_as_binary(Num,{0,Bin}) when Num > 16 -> - Used = Num rem 8, - Pad = (8 - Used) rem 8, -% Nbytes = Num div 8, - <<Bits:Num,_:Pad,RestBin/binary>> = Bin, - {{Pad,<<Bits:Num,0:Pad>>},RestBin}; -getbits_as_binary(Num,Buffer={_Used,_Bin}) -> % Unaligned buffer - %% Num =< 16, - {Bits2,Buffer2} = getbits(Buffer,Num), - Pad = (8 - (Num rem 8)) rem 8, - {{Pad,<<Bits2:Num,0:Pad>>},Buffer2}. - - -% integer_from_list(Int,[],BigInt) -> -% BigInt; -% integer_from_list(Int,[H|T],BigInt) when Int < 8 -> -% (BigInt bsl Int) bor (H bsr (8-Int)); -% integer_from_list(Int,[H|T],BigInt) -> -% integer_from_list(Int-8,T,(BigInt bsl 8) bor H). - -getbits_as_list(Num,Bytes) when is_binary(Bytes) -> - getbits_as_list(Num,{0,Bytes},[]); -getbits_as_list(Num,Bytes) -> - getbits_as_list(Num,Bytes,[]). - -%% If buffer is empty and nothing more will be picked. -getbits_as_list(0, B, Acc) -> - {lists:reverse(Acc),B}; -%% If first byte in buffer is full and at least one byte will be picked, -%% then pick one byte. -getbits_as_list(N,{0,Bin},Acc) when N >= 8 -> - <<B7:1,B6:1,B5:1,B4:1,B3:1,B2:1,B1:1,B0:1,Rest/binary>> = Bin, - getbits_as_list(N-8,{0,Rest},[B0,B1,B2,B3,B4,B5,B6,B7|Acc]); -getbits_as_list(N,{Used,Bin},Acc) when N >= 4, Used =< 4 -> - NewUsed = Used + 4, - Rem = 8 - NewUsed, - <<_:Used,B3:1,B2:1,B1:1,B0:1,_:Rem, Rest/binary>> = Bin, - NewRest = case Rem of 0 -> Rest; _ -> Bin end, - getbits_as_list(N-4,{NewUsed rem 8,NewRest},[B0,B1,B2,B3|Acc]); -getbits_as_list(N,{Used,Bin},Acc) when N >= 2, Used =< 6 -> - NewUsed = Used + 2, - Rem = 8 - NewUsed, - <<_:Used,B1:1,B0:1,_:Rem, Rest/binary>> = Bin, - NewRest = case Rem of 0 -> Rest; _ -> Bin end, - getbits_as_list(N-2,{NewUsed rem 8,NewRest},[B0,B1|Acc]); -getbits_as_list(N,{Used,Bin},Acc) when Used =< 7 -> - NewUsed = Used + 1, - Rem = 8 - NewUsed, - <<_:Used,B0:1,_:Rem, Rest/binary>> = Bin, - NewRest = case Rem of 0 -> Rest; _ -> Bin end, - getbits_as_list(N-1,{NewUsed rem 8,NewRest},[B0|Acc]). - - -getbit({7,<<_:7,B:1,Rest/binary>>}) -> - {B,{0,Rest}}; -getbit({0,Buffer = <<B:1,_:7,_/binary>>}) -> - {B,{1,Buffer}}; -getbit({Used,Buffer}) -> - Unused = (8 - Used) - 1, - <<_:Used,B:1,_:Unused,_/binary>> = Buffer, - {B,{Used+1,Buffer}}; -getbit(Buffer) when is_binary(Buffer) -> - getbit({0,Buffer}). - - -getbits({0,Buffer},Num) when (Num rem 8) == 0 -> - <<Bits:Num,Rest/binary>> = Buffer, - {Bits,{0,Rest}}; -getbits({Used,Bin},Num) -> - NumPlusUsed = Num + Used, - NewUsed = NumPlusUsed rem 8, - Unused = (8-NewUsed) rem 8, - case Unused of - 0 -> - <<_:Used,Bits:Num,Rest/binary>> = Bin, - {Bits,{0,Rest}}; - _ -> - Bytes = NumPlusUsed div 8, - <<_:Used,Bits:Num,_UBits:Unused,_/binary>> = Bin, - <<_:Bytes/binary,Rest/binary>> = Bin, - {Bits,{NewUsed,Rest}} - end; -getbits(Bin,Num) when is_binary(Bin) -> - getbits({0,Bin},Num). - - - -% getoctet(Bytes) when is_list(Bytes) -> -% getoctet({0,Bytes}); -% getoctet(Bytes) -> -% %% io:format("getoctet:Buffer = ~p~n",[Bytes]), -% getoctet1(Bytes). - -% getoctet1({0,[H|T]}) -> -% {H,{0,T}}; -% getoctet1({Pos,[_,H|T]}) -> -% {H,{0,T}}. - -align({0,L}) -> - {0,L}; -align({_Pos,<<_H,T/binary>>}) -> - {0,T}; -align(Bytes) -> - {0,Bytes}. - -%% First align buffer, then pick the first Num octets. -%% Returns octets as an integer with bit significance as in buffer. -getoctets({0,Buffer},Num) -> - <<Val:Num/integer-unit:8,RestBin/binary>> = Buffer, - {Val,{0,RestBin}}; -getoctets({U,<<_Padding,Rest/binary>>},Num) when U /= 0 -> - getoctets({0,Rest},Num); -getoctets(Buffer,Num) when is_binary(Buffer) -> - getoctets({0,Buffer},Num). -% getoctets(Buffer,Num) -> -% %% io:format("getoctets:Buffer = ~p~nNum = ~p~n",[Buffer,Num]), -% getoctets(Buffer,Num,0). - -% getoctets(Buffer,0,Acc) -> -% {Acc,Buffer}; -% getoctets(Buffer,Num,Acc) -> -% {Oct,NewBuffer} = getoctet(Buffer), -% getoctets(NewBuffer,Num-1,(Acc bsl 8)+Oct). - -% getoctets_as_list(Buffer,Num) -> -% getoctets_as_list(Buffer,Num,[]). - -% getoctets_as_list(Buffer,0,Acc) -> -% {lists:reverse(Acc),Buffer}; -% getoctets_as_list(Buffer,Num,Acc) -> -% {Oct,NewBuffer} = getoctet(Buffer), -% getoctets_as_list(NewBuffer,Num-1,[Oct|Acc]). - -%% First align buffer, then pick the first Num octets. -%% Returns octets as a binary -getoctets_as_bin({0,Bin},Num)-> - <<Octets:Num/binary,RestBin/binary>> = Bin, - {Octets,{0,RestBin}}; -getoctets_as_bin({_U,Bin},Num) -> - <<_Padding,Octets:Num/binary,RestBin/binary>> = Bin, - {Octets,{0,RestBin}}; -getoctets_as_bin(Bin,Num) when is_binary(Bin) -> - getoctets_as_bin({0,Bin},Num). - -%% same as above but returns octets as a List -getoctets_as_list(Buffer,Num) -> - {Bin,Buffer2} = getoctets_as_bin(Buffer,Num), - {binary_to_list(Bin),Buffer2}. -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% set_choice(Alt,Choices,Altnum) -> ListofBitSettings -%% Alt = atom() -%% Altnum = integer() | {integer(),integer()}% number of alternatives -%% Choices = [atom()] | {[atom()],[atom()]} -%% When Choices is a tuple the first list is the Rootset and the -%% second is the Extensions and then Altnum must also be a tuple with the -%% lengths of the 2 lists -%% -set_choice(Alt,{L1,L2},{Len1,_Len2}) -> - case set_choice_tag(Alt,L1) of - N when is_integer(N), Len1 > 1 -> - [{bits,1,0}, % the value is in the root set - encode_integer([{'ValueRange',{0,Len1-1}}],N)]; - N when is_integer(N) -> - [{bits,1,0}]; % no encoding if only 0 or 1 alternative - false -> - [{bits,1,1}, % extension value - case set_choice_tag(Alt,L2) of - N2 when is_integer(N2) -> - encode_small_number(N2); - false -> - unknown_choice_alt - end] - end; -set_choice(Alt,L,Len) -> - case set_choice_tag(Alt,L) of - N when is_integer(N), Len > 1 -> - encode_integer([{'ValueRange',{0,Len-1}}],N); - N when is_integer(N) -> - []; % no encoding if only 0 or 1 alternative - false -> - [unknown_choice_alt] - end. - -set_choice_tag(Alt,Choices) -> - set_choice_tag(Alt,Choices,0). - -set_choice_tag(Alt,[Alt|_Rest],Tag) -> - Tag; -set_choice_tag(Alt,[_H|Rest],Tag) -> - set_choice_tag(Alt,Rest,Tag+1); -set_choice_tag(_Alt,[],_Tag) -> - false. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_fragmented_XXX; decode of values encoded fragmented according -%% to ITU-T X.691 clause 10.9.3.8. The unit (XXX) is either bits, octets, -%% characters or number of components (in a choice,sequence or similar). -%% Buffer is a buffer {Used, Bin}. -%% C is the constrained length. -%% If the buffer is not aligned, this function does that. -decode_fragmented_bits({0,Buffer},C) -> - decode_fragmented_bits(Buffer,C,[]); -decode_fragmented_bits({_N,<<_,Bs/binary>>},C) -> - decode_fragmented_bits(Bs,C,[]). - -decode_fragmented_bits(<<3:2,Len:6,Bin/binary>>,C,Acc) -> - {Value,Bin2} = split_binary(Bin, Len * ?'16K'), - decode_fragmented_bits(Bin2,C,[Value,Acc]); -decode_fragmented_bits(<<0:1,0:7,Bin/binary>>,C,Acc) -> - BinBits = list_to_binary(lists:reverse(Acc)), - case C of - Int when is_integer(Int),C == size(BinBits) -> - {BinBits,{0,Bin}}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,BinBits}}}) - end; -decode_fragmented_bits(<<0:1,Len:7,Bin/binary>>,C,Acc) -> - Result = {BinBits,{Used,_Rest}} = - case (Len rem 8) of - 0 -> - <<Value:Len/binary-unit:1,Bin2/binary>> = Bin, - {list_to_binary(lists:reverse([Value|Acc])),{0,Bin2}}; - Rem -> - Bytes = Len div 8, - U = 8 - Rem, - <<Value:Bytes/binary-unit:8,Bits1:Rem,Bits2:U,Bin2/binary>> = Bin, - {list_to_binary(lists:reverse([Bits1 bsl U,Value|Acc])), - {Rem,<<Bits2,Bin2/binary>>}} - end, - case C of - Int when is_integer(Int),C == (size(BinBits) - ((8 - Used) rem 8)) -> - Result; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,BinBits}}}) - end. - - -decode_fragmented_octets({0,Bin},C) -> - decode_fragmented_octets(Bin,C,[]). - -decode_fragmented_octets(<<3:2,Len:6,Bin/binary>>,C,Acc) -> - {Value,Bin2} = split_binary(Bin,Len * ?'16K'), - decode_fragmented_octets(Bin2,C,[Value,Acc]); -decode_fragmented_octets(<<0:1,0:7,Bin/binary>>,C,Acc) -> - Octets = list_to_binary(lists:reverse(Acc)), - case C of - Int when is_integer(Int), C == size(Octets) -> - {Octets,{0,Bin}}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,Octets}}}) - end; -decode_fragmented_octets(<<0:1,Len:7,Bin/binary>>,C,Acc) -> - <<Value:Len/binary-unit:8,Bin2/binary>> = Bin, - BinOctets = list_to_binary(lists:reverse([Value|Acc])), - case C of - Int when is_integer(Int),size(BinOctets) == Int -> - {BinOctets,Bin2}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,BinOctets}}}) - end. - - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_open_type(Constraint, Value) -> CompleteList -%% Value = list of bytes of an already encoded value (the list must be flat) -%% | binary -%% Contraint = not used in this version -%% -encode_open_type(_C, Val) when is_list(Val) -> - Bin = list_to_binary(Val), - [encode_length(undefined,size(Bin)),{octets,Bin}]; % octets implies align -encode_open_type(_C, Val) when is_binary(Val) -> - [encode_length(undefined,size(Val)),{octets,Val}]. % octets implies align -%% the binary_to_list is not optimal but compatible with the current solution - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_open_type(Buffer,Constraint) -> Value -%% Constraint is not used in this version -%% Buffer = [byte] with PER encoded data -%% Value = [byte] with decoded data (which must be decoded again as some type) -%% -decode_open_type(Bytes, _C) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - getoctets_as_bin(Bytes2,Len). - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_integer(Constraint,Value,NamedNumberList) -> CompleteList -%% encode_integer(Constraint,Value) -> CompleteList -%% encode_integer(Constraint,{Name,Value}) -> CompleteList -%% -%% -encode_integer(C,V,NamedNumberList) when is_atom(V) -> - case lists:keysearch(V,1,NamedNumberList) of - {value,{_,NewV}} -> - encode_integer(C,NewV); - _ -> - exit({error,{asn1,{namednumber,V}}}) - end; -encode_integer(C,V,_NamedNumberList) when is_integer(V) -> - encode_integer(C,V); -encode_integer(C,{Name,V},NamedNumberList) when is_atom(Name) -> - encode_integer(C,V,NamedNumberList). - -encode_integer(C,{Name,Val}) when is_atom(Name) -> - encode_integer(C,Val); - -encode_integer([{Rc,_Ec}],Val) when is_tuple(Rc) -> % XXX when is this invoked? First argument most often a list,...Ok this is the extension case...but it doesn't work. - case (catch encode_integer([Rc],Val)) of - {'EXIT',{error,{asn1,_}}} -> - [{bits,1,1},encode_unconstrained_number(Val)]; - Encoded -> - [{bits,1,0},Encoded] - end; -encode_integer(C,Val ) when is_list(C) -> - case get_constraint(C,'SingleValue') of - no -> - encode_integer1(C,Val); - V when is_integer(V),V == Val -> - []; % a type restricted to a single value encodes to nothing - V when is_list(V) -> - case lists:member(Val,V) of - true -> - encode_integer1(C,Val); - _ -> - exit({error,{asn1,{illegal_value,Val}}}) - end; - _ -> - exit({error,{asn1,{illegal_value,Val}}}) - end. - -encode_integer1(C, Val) -> - case VR = get_constraint(C,'ValueRange') of - no -> - encode_unconstrained_number(Val); - {Lb,'MAX'} -> - encode_semi_constrained_number(Lb,Val); - %% positive with range - {Lb,Ub} when Val >= Lb, - Ub >= Val -> - encode_constrained_number(VR,Val); - _ -> - exit({error,{asn1,{illegal_value,VR,Val}}}) - end. - -decode_integer(Buffer,Range,NamedNumberList) -> - {Val,Buffer2} = decode_integer(Buffer,Range), - case lists:keysearch(Val,2,NamedNumberList) of - {value,{NewVal,_}} -> {NewVal,Buffer2}; - _ -> {Val,Buffer2} - end. - -decode_integer(Buffer,[{Rc,_Ec}]) when is_tuple(Rc) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> decode_integer(Buffer2,[Rc]); - 1 -> decode_unconstrained_number(Buffer2) - end; -decode_integer(Buffer,undefined) -> - decode_unconstrained_number(Buffer); -decode_integer(Buffer,C) -> - case get_constraint(C,'SingleValue') of - V when is_integer(V) -> - {V,Buffer}; - V when is_list(V) -> - {Val,Buffer2} = decode_integer1(Buffer,C), - case lists:member(Val,V) of - true -> - {Val,Buffer2}; - _ -> - exit({error,{asn1,{illegal_value,Val}}}) - end; - _ -> - decode_integer1(Buffer,C) - end. - -decode_integer1(Buffer,C) -> - case VR = get_constraint(C,'ValueRange') of - no -> - decode_unconstrained_number(Buffer); - {Lb, 'MAX'} -> - decode_semi_constrained_number(Buffer,Lb); - {_,_} -> - decode_constrained_number(Buffer,VR) - end. - - % X.691:10.6 Encoding of a normally small non-negative whole number - % Use this for encoding of CHOICE index if there is an extension marker in - % the CHOICE -encode_small_number({Name,Val}) when is_atom(Name) -> - encode_small_number(Val); -encode_small_number(Val) when Val =< 63 -> -% [{bits,1,0},{bits,6,Val}]; - [{bits,7,Val}]; % same as above but more efficient -encode_small_number(Val) -> - [{bits,1,1},encode_semi_constrained_number(0,Val)]. - -decode_small_number(Bytes) -> - {Bit,Bytes2} = getbit(Bytes), - case Bit of - 0 -> - getbits(Bytes2,6); - 1 -> - decode_semi_constrained_number(Bytes2,0) - end. - -%% X.691:10.7 Encoding of a semi-constrained whole number -%% might be an optimization encode_semi_constrained_number(0,Val) -> -encode_semi_constrained_number(C,{Name,Val}) when is_atom(Name) -> - encode_semi_constrained_number(C,Val); -encode_semi_constrained_number({Lb,'MAX'},Val) -> - encode_semi_constrained_number(Lb,Val); -encode_semi_constrained_number(Lb,Val) -> - Val2 = Val - Lb, - Oct = eint_positive(Val2), - Len = length(Oct), - if - Len < 128 -> - {octets,[Len|Oct]}; % equiv with encode_length(undefined,Len) but faster - true -> - [encode_length(undefined,Len),{octets,Oct}] - end. - -decode_semi_constrained_number(Bytes,{Lb,_}) -> - decode_semi_constrained_number(Bytes,Lb); -decode_semi_constrained_number(Bytes,Lb) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {V,Bytes3} = getoctets(Bytes2,Len), - {V+Lb,Bytes3}. - -encode_constrained_number(Range,{Name,Val}) when is_atom(Name) -> - encode_constrained_number(Range,Val); -encode_constrained_number({Lb,Ub},Val) when Val >= Lb, Ub >= Val -> - Range = Ub - Lb + 1, - Val2 = Val - Lb, - if - Range == 1 -> - []; - Range == 2 -> - {bits,1,Val2}; - Range =< 4 -> - {bits,2,Val2}; - Range =< 8 -> - {bits,3,Val2}; - Range =< 16 -> - {bits,4,Val2}; - Range =< 32 -> - {bits,5,Val2}; - Range =< 64 -> - {bits,6,Val2}; - Range =< 128 -> - {bits,7,Val2}; - Range =< 255 -> - {bits,8,Val2}; - Range =< 256 -> - {octets,[Val2]}; - Range =< 65536 -> - {octets,<<Val2:16>>}; - Range =< (1 bsl (255*8)) -> - Octs = binary:encode_unsigned(Val2), - RangeOcts = binary:encode_unsigned(Range - 1), - OctsLen = erlang:byte_size(Octs), - RangeOctsLen = erlang:byte_size(RangeOcts), - LengthBitsNeeded = minimum_bits(RangeOctsLen - 1), - [{bits, LengthBitsNeeded, OctsLen - 1}, {octets, Octs}]; - true -> - exit({not_supported,{integer_range,Range}}) - end; -encode_constrained_number(Range,Val) -> - exit({error,{asn1,{integer_range,Range,value,Val}}}). - -%% For some reason the minimum bits needed in the length field in encoding of -%% constrained whole numbers must always be atleast 2? -minimum_bits(N) when N < 4 -> 2; -minimum_bits(N) when N < 8 -> 3; -minimum_bits(N) when N < 16 -> 4; -minimum_bits(N) when N < 32 -> 5; -minimum_bits(N) when N < 64 -> 6; -minimum_bits(N) when N < 128 -> 7; -minimum_bits(_N) -> 8. - -decode_constrained_number(Buffer,{Lb,Ub}) -> - Range = Ub - Lb + 1, - % Val2 = Val - Lb, - {Val,Remain} = - if - Range == 1 -> - {0,Buffer}; - Range == 2 -> - getbits(Buffer,1); - Range =< 4 -> - getbits(Buffer,2); - Range =< 8 -> - getbits(Buffer,3); - Range =< 16 -> - getbits(Buffer,4); - Range =< 32 -> - getbits(Buffer,5); - Range =< 64 -> - getbits(Buffer,6); - Range =< 128 -> - getbits(Buffer,7); - Range =< 255 -> - getbits(Buffer,8); - Range =< 256 -> - getoctets(Buffer,1); - Range =< 65536 -> - getoctets(Buffer,2); - Range =< (1 bsl (255*8)) -> - OList = binary:bin_to_list(binary:encode_unsigned(Range - 1)), - RangeOctLen = length(OList), - {Len, Bytes} = decode_length(Buffer, {1, RangeOctLen}), - {Octs, RestBytes} = getoctets_as_list(Bytes, Len), - {binary:decode_unsigned(binary:list_to_bin(Octs)), RestBytes}; - true -> - exit({not_supported,{integer_range,Range}}) - end, - {Val+Lb,Remain}. - -%% X.691:10.8 Encoding of an unconstrained whole number - -encode_unconstrained_number(Val) when Val >= 0 -> - Oct = eint(Val,[]), - Len = length(Oct), - if - Len < 128 -> - {octets,[Len|Oct]}; % equiv with encode_length(undefined,Len) but faster - true -> - [encode_length(undefined,Len),{octets,Oct}] - end; -encode_unconstrained_number(Val) -> % negative - Oct = enint(Val,[]), - Len = length(Oct), - if - Len < 128 -> - {octets,[Len|Oct]}; % equiv with encode_length(undefined,Len) but faster - true -> - [encode_length(undefined,Len),{octets,Oct}] - end. - - -%% used for positive Values which don't need a sign bit -%% returns a binary -eint_positive(Val) -> - case eint(Val,[]) of - [0,B1|T] -> - [B1|T]; - T -> - T - end. - - -eint(0, [B|Acc]) when B < 128 -> - [B|Acc]; -eint(N, Acc) -> - eint(N bsr 8, [N band 16#ff| Acc]). - -enint(-1, [B1|T]) when B1 > 127 -> - [B1|T]; -enint(N, Acc) -> - enint(N bsr 8, [N band 16#ff|Acc]). - -decode_unconstrained_number(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Ints,Bytes3} = getoctets_as_list(Bytes2,Len), - {dec_integer(Ints),Bytes3}. - -dec_integer(Ints) when hd(Ints) band 255 =< 127 -> %% Positive number - decpint(Ints, 8 * (length(Ints) - 1)); -dec_integer(Ints) -> %% Negative - decnint(Ints, 8 * (length(Ints) - 1)). - -decpint([Byte|Tail], Shift) -> - (Byte bsl Shift) bor decpint(Tail, Shift-8); -decpint([], _) -> 0. - -decnint([Byte|Tail], Shift) -> - (-128 + (Byte band 127) bsl Shift) bor decpint(Tail, Shift-8). - -% minimum_octets(Val) -> -% minimum_octets(Val,[]). - -% minimum_octets(Val,Acc) when Val > 0 -> -% minimum_octets((Val bsr 8),[Val band 16#FF|Acc]); -% minimum_octets(0,Acc) -> -% Acc. - - -%% X.691:10.9 Encoding of a length determinant -%%encode_small_length(undefined,Len) -> % null means no UpperBound -%% encode_small_number(Len). - -%% X.691:10.9.3.5 -%% X.691:10.9.3.7 -encode_length(undefined,Len) -> % un-constrained - if - Len < 128 -> - {octets,[Len]}; - Len < 16384 -> - {octets,<<2:2,Len:14>>}; - true -> % should be able to endode length >= 16384 - exit({error,{asn1,{encode_length,{nyi,above_16k}}}}) - end; - -encode_length({0,'MAX'},Len) -> - encode_length(undefined,Len); -encode_length(Vr={Lb,Ub},Len) when Ub =< 65535 ,Lb >= 0 -> % constrained - encode_constrained_number(Vr,Len); -encode_length({Lb,_Ub},Len) when is_integer(Lb), Lb >= 0 -> % Ub > 65535 - encode_length(undefined,Len); -encode_length({Vr={Lb,Ub},Ext},Len) - when Ub =< 65535 ,Lb >= 0, Len=<Ub, is_list(Ext) -> - %% constrained extensible - [{bits,1,0},encode_constrained_number(Vr,Len)]; -encode_length({{Lb,_Ub},Ext},Len) when is_list(Ext) -> - [{bits,1,1},encode_semi_constrained_number(Lb,Len)]; -encode_length(SingleValue,_Len) when is_integer(SingleValue) -> - []. - -%% X.691 10.9.3.4 (only used for length of bitmap that prefixes extension -%% additions in a sequence or set -encode_small_length(Len) when Len =< 64 -> -%% [{bits,1,0},{bits,6,Len-1}]; - {bits,7,Len-1}; % the same as above but more efficient -encode_small_length(Len) -> - [{bits,1,1},encode_length(undefined,Len)]. - -% decode_small_length({Used,<<_:Used,0:1,Num:6,_:((8-Used+1) rem 8),Rest/binary>>}) -> -% case Buffer of -% <<_:Used,0:1,Num:6,_:((8-Used+1) rem 8),Rest/binary>> -> -% {Num, -% case getbit(Buffer) of -% {0,Remain} -> -% {Bits,Remain2} = getbits(Remain,6), -% {Bits+1,Remain2}; -% {1,Remain} -> -% decode_length(Remain,undefined) -% end. - -decode_small_length(Buffer) -> - case getbit(Buffer) of - {0,Remain} -> - {Bits,Remain2} = getbits(Remain,6), - {Bits+1,Remain2}; - {1,Remain} -> - decode_length(Remain,undefined) - end. - -decode_length(Buffer) -> - decode_length(Buffer,undefined). - -decode_length(Buffer,undefined) -> % un-constrained - {0,Buffer2} = align(Buffer), - case Buffer2 of - <<0:1,Oct:7,Rest/binary>> -> - {Oct,{0,Rest}}; - <<2:2,Val:14,Rest/binary>> -> - {Val,{0,Rest}}; - <<3:2,_:14,_Rest/binary>> -> - %% this case should be fixed - exit({error,{asn1,{decode_length,{nyi,above_16k}}}}) - end; -%% {Bits,_} = getbits(Buffer2,2), -% case Bits of -% 2 -> -% {Val,Bytes3} = getoctets(Buffer2,2), -% {(Val band 16#3FFF),Bytes3}; -% 3 -> -% exit({error,{asn1,{decode_length,{nyi,above_16k}}}}); -% _ -> -% {Val,Bytes3} = getoctet(Buffer2), -% {Val band 16#7F,Bytes3} -% end; - -decode_length(Buffer,{Lb,Ub}) when Ub =< 65535 ,Lb >= 0 -> % constrained - decode_constrained_number(Buffer,{Lb,Ub}); -decode_length(Buffer,{Lb,_}) when is_integer(Lb), Lb >= 0 -> % Ub > 65535 - decode_length(Buffer,undefined); -decode_length(Buffer,{VR={_Lb,_Ub},Ext}) when is_list(Ext) -> - case getbit(Buffer) of - {0,Buffer2} -> - decode_length(Buffer2, VR); - {1,Buffer2} -> - decode_length(Buffer2, undefined) - end; -%% {0,Buffer2} = getbit(Buffer), -%% decode_length(Buffer2, VR); - - -%When does this case occur with {_,_Lb,Ub} ?? -% X.691:10.9.3.5 -decode_length({Used,Bin},{_,_Lb,_Ub}) -> %when Len =< 127 -> % Unconstrained or large Ub NOTE! this case does not cover case when Ub > 65535 - Unused = (8-Used) rem 8, - case Bin of - <<_:Used,0:1,Val:7,R:Unused,Rest/binary>> -> - {Val,{Used,<<R,Rest/binary>>}}; - <<_:Used,_:Unused,2:2,Val:14,Rest/binary>> -> - {Val, {0,Rest}}; - <<_:Used,_:Unused,3:2,_:14,_Rest/binary>> -> - exit({error,{asn1,{decode_length,{nyi,length_above_64K}}}}) - end; -% decode_length(Buffer,{_,_Lb,Ub}) -> %when Len =< 127 -> % Unconstrained or large Ub -% case getbit(Buffer) of -% {0,Remain} -> -% getbits(Remain,7); -% {1,Remain} -> -% {Val,Remain2} = getoctets(Buffer,2), -% {Val band 2#0111111111111111, Remain2} -% end; -decode_length(Buffer,SingleValue) when is_integer(SingleValue) -> - {SingleValue,Buffer}. - - - % X.691:11 -encode_boolean(true) -> - {bits,1,1}; -encode_boolean(false) -> - {bits,1,0}; -encode_boolean({Name,Val}) when is_atom(Name) -> - encode_boolean(Val); -encode_boolean(Val) -> - exit({error,{asn1,{encode_boolean,Val}}}). - -decode_boolean(Buffer) -> %when record(Buffer,buffer) - case getbit(Buffer) of - {1,Remain} -> {true,Remain}; - {0,Remain} -> {false,Remain} - end. - - -%% ENUMERATED with extension marker -decode_enumerated(Buffer,C,{Ntup1,Ntup2}) when is_tuple(Ntup1), is_tuple(Ntup2) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> % not an extension value - {Val,Buffer3} = decode_integer(Buffer2,C), - case catch (element(Val+1,Ntup1)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,[Ntup1,Ntup2]}}}}) - end; - 1 -> % this an extension value - {Val,Buffer3} = decode_small_number(Buffer2), - case catch (element(Val+1,Ntup2)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _ -> {{asn1_enum,Val},Buffer3} - end - end; - -decode_enumerated(Buffer,C,NamedNumberTup) when is_tuple(NamedNumberTup) -> - {Val,Buffer2} = decode_integer(Buffer,C), - case catch (element(Val+1,NamedNumberTup)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer2}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,NamedNumberTup}}}}) - end. - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Bitstring value, ITU_T X.690 Chapter 8.5 -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -%%=============================================================================== -%% encode bitstring value -%%=============================================================================== - - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% bitstring NamedBitList -%% Val can be of: -%% - [identifiers] where only named identifers are set to one, -%% the Constraint must then have some information of the -%% bitlength. -%% - [list of ones and zeroes] all bits -%% - integer value representing the bitlist -%% C is constraint Len, only valid when identifiers - - -%% when the value is a list of {Unused,BinBits}, where -%% Unused = integer(), -%% BinBits = binary(). - -encode_bit_string(C,Bin={Unused,BinBits},NamedBitList) when is_integer(Unused), - is_binary(BinBits) -> - encode_bin_bit_string(C,Bin,NamedBitList); - -%% when the value is a list of named bits -encode_bit_string(C, LoNB=[FirstVal | _RestVal], NamedBitList) when is_atom(FirstVal) -> - ToSetPos = get_all_bitposes(LoNB, NamedBitList, []), - BitList = make_and_set_list(ToSetPos,0), - encode_bit_string(C,BitList,NamedBitList); - -encode_bit_string(C, BL=[{bit,_No} | _RestVal], NamedBitList) -> - ToSetPos = get_all_bitposes(BL, NamedBitList, []), - BitList = make_and_set_list(ToSetPos,0), - encode_bit_string(C,BitList,NamedBitList); - -%% when the value is a list of ones and zeroes - -% encode_bit_string(C, BitListValue, NamedBitList) when is_list(BitListValue) -> -% Bl1 = -% case NamedBitList of -% [] -> % dont remove trailing zeroes -% BitListValue; -% _ -> % first remove any trailing zeroes -% lists:reverse(lists:dropwhile(fun(0)->true;(1)->false end, -% lists:reverse(BitListValue))) -% end, -% BitList = [{bit,X} || X <- Bl1], -% %% BListLen = length(BitList), -% case get_constraint(C,'SizeConstraint') of -% 0 -> % fixed length -% []; % nothing to encode -% V when is_integer(V),V=<16 -> % fixed length 16 bits or less -% pad_list(V,BitList); -% V when is_integer(V) -> % fixed length 16 bits or more -% [align,pad_list(V,BitList)]; % should be another case for V >= 65537 -% {Lb,Ub} when is_integer(Lb),is_integer(Ub) -> -% [encode_length({Lb,Ub},length(BitList)),align,BitList]; -% no -> -% [encode_length(undefined,length(BitList)),align,BitList]; -% Sc -> % extension marker -% [encode_length(Sc,length(BitList)),align,BitList] -% end; -encode_bit_string(C, BitListValue, NamedBitList) when is_list(BitListValue) -> - BitListToBinary = - %% fun that transforms a list of 1 and 0 to a tuple: - %% {UnusedBitsInLastByte, Binary} - fun([1|T],Acc,N,Fun) -> - Fun(T,(Acc bsl 1)+1,N+1,Fun); - ([0|T],Acc,N,Fun) -> - Fun(T,(Acc bsl 1),N+1,Fun); - ([_H|_T],_,_,_) -> - exit({error,{asn1,{bitstring_bitlist,BitListValue}}}); - ([],Acc,N,_) -> - Unused = (8 - (N rem 8)) rem 8, - {Unused,<<Acc:N,0:Unused>>} - end, - UnusedAndBin = - case NamedBitList of - [] -> % dont remove trailing zeroes - BitListToBinary(BitListValue,0,0,BitListToBinary); - _ -> - BitListToBinary(lists:reverse( - lists:dropwhile(fun(0)->true;(_)->false end, - lists:reverse(BitListValue))), - 0,0,BitListToBinary) - end, - encode_bin_bit_string(C,UnusedAndBin,NamedBitList); - -%% when the value is an integer -encode_bit_string(C, IntegerVal, NamedBitList) when is_integer(IntegerVal)-> - BitList = int_to_bitlist(IntegerVal), - encode_bit_string(C,BitList,NamedBitList); - -%% when the value is a tuple -encode_bit_string(C,{Name,Val}, NamedBitList) when is_atom(Name) -> - encode_bit_string(C,Val,NamedBitList). - - -%% encode_bin_bit_string/3, when value is a tuple of Unused and BinBits. -%% Unused = integer(),i.e. number unused bits in least sign. byte of -%% BinBits = binary(). - - -encode_bin_bit_string(C,UnusedAndBin={_Unused,_BinBits},NamedBitList) -> - Constr = get_constraint(C,'SizeConstraint'), - UnusedAndBin1 = {Unused1,Bin1} = - remove_trailing_bin(NamedBitList,UnusedAndBin,lower_bound(Constr)), - case Constr of - 0 -> - []; - V when is_integer(V),V=<16 -> - {Unused2,Bin2} = pad_list(V,UnusedAndBin1), - <<BitVal:V,_:Unused2>> = Bin2, - {bits,V,BitVal}; - V when is_integer(V) -> - [align, pad_list(V, UnusedAndBin1)]; - {Lb,Ub} when is_integer(Lb),is_integer(Ub) -> - [encode_length({Lb,Ub},size(Bin1)*8 - Unused1), - align,UnusedAndBin1]; - {{Fix,Fix},L} when is_integer(Fix),is_list(L) -> - %% X.691 � 15.6, the rest of this paragraph is covered by - %% the last, ie. Sc, clause in this case - case (size(Bin1)*8)-Unused1 of - Size when Size =< Fix, Fix =< 16 -> - {Unused2,Bin2} = pad_list(Fix,UnusedAndBin), - <<BitVal:Fix,_:Unused2>> = Bin2, - [{bits,1,0},{bits,Fix,BitVal}]; - Size when Size =< Fix -> - [{bits,1,0},align, pad_list(Fix, UnusedAndBin1)]; - Size -> - [{bits,1,1},encode_length(undefined,Size), - align,UnusedAndBin1] - end; - no -> - [encode_length(undefined,size(Bin1)*8 - Unused1), - align,UnusedAndBin1]; - Sc -> - [encode_length(Sc,size(Bin1)*8 - Unused1), - align,UnusedAndBin1] - end. - - -remove_trailing_bin([], {Unused,Bin},_) -> - {Unused,Bin}; -remove_trailing_bin(_NamedNumberList,{_Unused,<<>>},C) -> - case C of - Int when is_integer(Int),Int > 0 -> - %% this padding see OTP-4353 - pad_list(Int,{0,<<>>}); - _ -> {0,<<>>} - end; -remove_trailing_bin(NamedNumberList, {_Unused,Bin},C) -> - Size = size(Bin)-1, - <<Bfront:Size/binary, LastByte:8>> = Bin, - %% clear the Unused bits to be sure - Unused1 = trailingZeroesInNibble(LastByte band 15), - Unused2 = - case Unused1 of - 4 -> - 4 + trailingZeroesInNibble(LastByte bsr 4); - _ -> Unused1 - end, - case Unused2 of - 8 -> - remove_trailing_bin(NamedNumberList,{0,Bfront},C); - _ -> - case C of - Int when is_integer(Int),Int > ((size(Bin)*8)-Unused2) -> - %% this padding see OTP-4353 - pad_list(Int,{Unused2,Bin}); - _ -> {Unused2,Bin} - end - end. - - -trailingZeroesInNibble(0) -> - 4; -trailingZeroesInNibble(1) -> - 0; -trailingZeroesInNibble(2) -> - 1; -trailingZeroesInNibble(3) -> - 0; -trailingZeroesInNibble(4) -> - 2; -trailingZeroesInNibble(5) -> - 0; -trailingZeroesInNibble(6) -> - 1; -trailingZeroesInNibble(7) -> - 0; -trailingZeroesInNibble(8) -> - 3; -trailingZeroesInNibble(9) -> - 0; -trailingZeroesInNibble(10) -> - 1; -trailingZeroesInNibble(11) -> - 0; -trailingZeroesInNibble(12) -> %#1100 - 2; -trailingZeroesInNibble(13) -> - 0; -trailingZeroesInNibble(14) -> - 1; -trailingZeroesInNibble(15) -> - 0. - -lower_bound({{Lb,_},_}) when is_integer(Lb) -> - Lb; -lower_bound({Lb,_}) when is_integer(Lb) -> - Lb; -lower_bound(C) -> - C. - -%%%%%%%%%%%%%%% -%% The result is presented as a list of named bits (if possible) -%% else as a tuple {Unused,Bits}. Unused is the number of unused -%% bits, least significant bits in the last byte of Bits. Bits is -%% the BIT STRING represented as a binary. -%% -decode_compact_bit_string(Buffer, C, NamedNumberList) -> - case get_constraint(C,'SizeConstraint') of - 0 -> % fixed length - {{8,0},Buffer}; - V when is_integer(V),V=<16 -> %fixed length 16 bits or less - compact_bit_string(Buffer,V,NamedNumberList); - V when is_integer(V),V=<65536 -> %fixed length > 16 bits - Bytes2 = align(Buffer), - compact_bit_string(Bytes2,V,NamedNumberList); - V when is_integer(V) -> % V > 65536 => fragmented value - {Bin,Buffer2} = decode_fragmented_bits(Buffer,V), - case Buffer2 of - {0,_} -> {{0,Bin},Buffer2}; - {U,_} -> {{8-U,Bin},Buffer2} - end; - {Lb,Ub} when is_integer(Lb),is_integer(Ub) -> - %% This case may demand decoding of fragmented length/value - {Len,Bytes2} = decode_length(Buffer,{Lb,Ub}), - Bytes3 = align(Bytes2), - compact_bit_string(Bytes3,Len,NamedNumberList); - no -> - %% This case may demand decoding of fragmented length/value - {Len,Bytes2} = decode_length(Buffer,undefined), - Bytes3 = align(Bytes2), - compact_bit_string(Bytes3,Len,NamedNumberList); - {{Fix,Fix},L} = Sc when is_list(L), is_integer(Fix), Fix =< 16 -> - %% X.691 �15.6, special case of extension marker - case decode_length(Buffer,Sc) of - {Len,Bytes2} when Len > Fix -> - Bytes3 = align(Bytes2), - compact_bit_string(Bytes3,Len,NamedNumberList); - {Len,Bytes2} -> - compact_bit_string(Bytes2,Len,NamedNumberList) - end; - Sc -> - {Len,Bytes2} = decode_length(Buffer,Sc), - Bytes3 = align(Bytes2), - compact_bit_string(Bytes3,Len,NamedNumberList) - end. - - -%%%%%%%%%%%%%%% -%% The result is presented as a list of named bits (if possible) -%% else as a list of 0 and 1. -%% -decode_bit_string(Buffer, C, NamedNumberList) -> - case get_constraint(C,'SizeConstraint') of - {Lb,Ub} when is_integer(Lb),is_integer(Ub) -> - {Len,Bytes2} = decode_length(Buffer,{Lb,Ub}), - Bytes3 = align(Bytes2), - bit_list_or_named(Bytes3,Len,NamedNumberList); - no -> - {Len,Bytes2} = decode_length(Buffer,undefined), - Bytes3 = align(Bytes2), - bit_list_or_named(Bytes3,Len,NamedNumberList); - 0 -> % fixed length - {[],Buffer}; % nothing to encode - V when is_integer(V),V=<16 -> % fixed length 16 bits or less - bit_list_or_named(Buffer,V,NamedNumberList); - V when is_integer(V),V=<65536 -> - Bytes2 = align(Buffer), - bit_list_or_named(Bytes2,V,NamedNumberList); - V when is_integer(V) -> - Bytes2 = align(Buffer), - {BinBits,_} = decode_fragmented_bits(Bytes2,V), - bit_list_or_named(BinBits,V,NamedNumberList); - {{Fix,Fix},L} = Sc when is_list(L), is_integer(Fix), Fix =< 16 -> - %% X.691 �15.6, special case of extension marker - case decode_length(Buffer,Sc) of - {Len,Bytes2} when Len > Fix -> - Bytes3 = align(Bytes2), - bit_list_or_named(Bytes3,Len,NamedNumberList); - {Len,Bytes2} when Len > 16 -> - Bytes3 = align(Bytes2), - bit_list_or_named(Bytes3,Len,NamedNumberList); - {Len,Bytes2} -> - bit_list_or_named(Bytes2,Len,NamedNumberList) - end; - Sc -> %% X.691 �15.6, extension marker - {Len,Bytes2} = decode_length(Buffer,Sc), - Bytes3 = align(Bytes2), - bit_list_or_named(Bytes3,Len,NamedNumberList) - end. - - -%% if no named bits are declared we will return a -%% {Unused,Bits}. Unused = integer(), -%% Bits = binary(). -compact_bit_string(Buffer,Len,[]) -> - getbits_as_binary(Len,Buffer); % {{Unused,BinBits},NewBuffer} -compact_bit_string(Buffer,Len,NamedNumberList) -> - bit_list_or_named(Buffer,Len,NamedNumberList). - - -%% if no named bits are declared we will return a -%% BitList = [0 | 1] - -bit_list_or_named(Buffer,Len,[]) -> - getbits_as_list(Len,Buffer); - -%% if there are named bits declared we will return a named -%% BitList where the names are atoms and unnamed bits represented -%% as {bit,Pos} -%% BitList = [atom() | {bit,Pos}] -%% Pos = integer() - -bit_list_or_named(Buffer,Len,NamedNumberList) -> - {BitList,Rest} = getbits_as_list(Len,Buffer), - {bit_list_or_named1(0,BitList,NamedNumberList,[]), Rest}. - -bit_list_or_named1(Pos,[0|Bt],Names,Acc) -> - bit_list_or_named1(Pos+1,Bt,Names,Acc); -bit_list_or_named1(Pos,[1|Bt],Names,Acc) -> - case lists:keysearch(Pos,2,Names) of - {value,{Name,_}} -> - bit_list_or_named1(Pos+1,Bt,Names,[Name|Acc]); - _ -> - bit_list_or_named1(Pos+1,Bt,Names,[{bit,Pos}|Acc]) - end; -bit_list_or_named1(_,[],_,Acc) -> - lists:reverse(Acc). - - - -%%%%%%%%%%%%%%% -%% - -int_to_bitlist(Int) when is_integer(Int), Int > 0 -> - [Int band 1 | int_to_bitlist(Int bsr 1)]; -int_to_bitlist(0) -> - []. - - -%%%%%%%%%%%%%%%%%% -%% get_all_bitposes([list of named bits to set], named_bit_db, []) -> -%% [sorted_list_of_bitpositions_to_set] - -get_all_bitposes([{bit,ValPos}|Rest], NamedBitList, Ack) -> - get_all_bitposes(Rest, NamedBitList, [ValPos | Ack ]); - -get_all_bitposes([Val | Rest], NamedBitList, Ack) -> - case lists:keysearch(Val, 1, NamedBitList) of - {value, {_ValName, ValPos}} -> - get_all_bitposes(Rest, NamedBitList, [ValPos | Ack]); - _ -> - exit({error,{asn1, {bitstring_namedbit, Val}}}) - end; -get_all_bitposes([], _NamedBitList, Ack) -> - lists:sort(Ack). - -%%%%%%%%%%%%%%%%%% -%% make_and_set_list([list of positions to set to 1])-> -%% returns list with all in SetPos set. -%% in positioning in list the first element is 0, the second 1 etc.., but -%% - -make_and_set_list([XPos|SetPos], XPos) -> - [1 | make_and_set_list(SetPos, XPos + 1)]; -make_and_set_list([Pos|SetPos], XPos) -> - [0 | make_and_set_list([Pos | SetPos], XPos + 1)]; -make_and_set_list([], _) -> - []. - -%%%%%%%%%%%%%%%%% -%% pad_list(N,BitList) -> PaddedList -%% returns a padded (with trailing {bit,0} elements) list of length N -%% if Bitlist contains more than N significant bits set an exit asn1_error -%% is generated - -pad_list(N,In={Unused,Bin}) -> - pad_list(N, size(Bin)*8 - Unused, In). - -pad_list(N,Size,In={_,_}) when N < Size -> - exit({error,{asn1,{range_error,{bit_string,In}}}}); -pad_list(N,Size,{Unused,Bin}) when N > Size, Unused > 0 -> - pad_list(N,Size+1,{Unused-1,Bin}); -pad_list(N,Size,{_Unused,Bin}) when N > Size -> - pad_list(N,Size+1,{7,<<Bin/binary,0>>}); -pad_list(N,N,In={_,_}) -> - In. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% X.691:16 -%% encode_octet_string(Constraint,ExtensionMarker,Val) -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -encode_octet_string(C,Val) -> - encode_octet_string2(C,Val). - -encode_octet_string2(C,{_Name,Val}) -> - encode_octet_string2(C,Val); -encode_octet_string2(C,Val) -> - case get_constraint(C,'SizeConstraint') of - 0 -> - []; - 1 -> - [V] = Val, - {bits,8,V}; - 2 -> - [V1,V2] = Val, - [{bits,8,V1},{bits,8,V2}]; - Sv when Sv =<65535, Sv == length(Val) -> % fixed length - {octets,Val}; - {Lb,Ub} -> - [encode_length({Lb,Ub},length(Val)),{octets,Val}]; - Sv when is_list(Sv) -> - [encode_length({hd(Sv),lists:max(Sv)},length(Val)),{octets,Val}]; - no -> - [encode_length(undefined,length(Val)),{octets,Val}] - end. - -decode_octet_string(Bytes,Range) -> - decode_octet_string(Bytes,Range,false). - -decode_octet_string(Bytes,C,false) -> - case get_constraint(C,'SizeConstraint') of - 0 -> - {[],Bytes}; - 1 -> - {B1,Bytes2} = getbits(Bytes,8), - {[B1],Bytes2}; - 2 -> - {Bs,Bytes2}= getbits(Bytes,16), - {binary_to_list(<<Bs:16>>),Bytes2}; - {_,0} -> - {[],Bytes}; - Sv when is_integer(Sv), Sv =<65535 -> % fixed length - getoctets_as_list(Bytes,Sv); - Sv when is_integer(Sv) -> % fragmented encoding - Bytes2 = align(Bytes), - decode_fragmented_octets(Bytes2,Sv); - {Lb,Ub} -> - {Len,Bytes2} = decode_length(Bytes,{Lb,Ub}), - getoctets_as_list(Bytes2,Len); - Sv when is_list(Sv) -> - {Len,Bytes2} = decode_length(Bytes,{hd(Sv),lists:max(Sv)}), - getoctets_as_list(Bytes2,Len); - no -> - {Len,Bytes2} = decode_length(Bytes,undefined), - getoctets_as_list(Bytes2,Len) - end. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% Restricted char string types -%% (NumericString, PrintableString,VisibleString,IA5String,BMPString,UniversalString) -%% X.691:26 and X.680:34-36 -%%encode_restricted_string(aligned,'BMPString',Constraints,Extension,Val) - - -encode_restricted_string(aligned,{Name,Val}) when is_atom(Name) -> - encode_restricted_string(aligned,Val); - -encode_restricted_string(aligned,Val) when is_list(Val)-> - [encode_length(undefined,length(Val)),{octets,Val}]. - -encode_known_multiplier_string(aligned,StringType,C,_Ext,{Name,Val}) when is_atom(Name) -> - encode_known_multiplier_string(aligned,StringType,C,false,Val); - -encode_known_multiplier_string(aligned,StringType,C,_Ext,Val) -> - Result = chars_encode(C,StringType,Val), - NumBits = get_NumBits(C,StringType), - case get_constraint(C,'SizeConstraint') of - Ub when is_integer(Ub), Ub*NumBits =< 16 -> - Result; - 0 -> - []; - Ub when is_integer(Ub),Ub =<65535 -> % fixed length - [align,Result]; - {Ub,Lb} -> - [encode_length({Ub,Lb},length(Val)),align,Result]; - Vl when is_list(Vl) -> - [encode_length({lists:min(Vl),lists:max(Vl)},length(Val)),align,Result]; - no -> - [encode_length(undefined,length(Val)),align,Result] - end. - -decode_restricted_string(Bytes,aligned) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - getoctets_as_list(Bytes2,Len). - -decode_known_multiplier_string(Bytes,aligned,StringType,C,_Ext) -> - NumBits = get_NumBits(C,StringType), - case get_constraint(C,'SizeConstraint') of - Ub when is_integer(Ub), Ub*NumBits =< 16 -> - chars_decode(Bytes,NumBits,StringType,C,Ub); - Ub when is_integer(Ub),Ub =<65535 -> % fixed length - Bytes1 = align(Bytes), - chars_decode(Bytes1,NumBits,StringType,C,Ub); - 0 -> - {[],Bytes}; - Vl when is_list(Vl) -> - {Len,Bytes1} = decode_length(Bytes,{hd(Vl),lists:max(Vl)}), - Bytes2 = align(Bytes1), - chars_decode(Bytes2,NumBits,StringType,C,Len); - no -> - {Len,Bytes1} = decode_length(Bytes,undefined), - Bytes2 = align(Bytes1), - chars_decode(Bytes2,NumBits,StringType,C,Len); - {Lb,Ub}-> - {Len,Bytes1} = decode_length(Bytes,{Lb,Ub}), - Bytes2 = align(Bytes1), - chars_decode(Bytes2,NumBits,StringType,C,Len) - end. - - -encode_NumericString(C,Val) -> - encode_known_multiplier_string(aligned,'NumericString',C,false,Val). -decode_NumericString(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'NumericString',C,false). - -encode_PrintableString(C,Val) -> - encode_known_multiplier_string(aligned,'PrintableString',C,false,Val). -decode_PrintableString(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'PrintableString',C,false). - -encode_VisibleString(C,Val) -> % equivalent with ISO646String - encode_known_multiplier_string(aligned,'VisibleString',C,false,Val). -decode_VisibleString(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'VisibleString',C,false). - -encode_IA5String(C,Val) -> - encode_known_multiplier_string(aligned,'IA5String',C,false,Val). -decode_IA5String(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'IA5String',C,false). - -encode_BMPString(C,Val) -> - encode_known_multiplier_string(aligned,'BMPString',C,false,Val). -decode_BMPString(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'BMPString',C,false). - -encode_UniversalString(C,Val) -> - encode_known_multiplier_string(aligned,'UniversalString',C,false,Val). -decode_UniversalString(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'UniversalString',C,false). - - -%% end of known-multiplier strings for which PER visible constraints are -%% applied - -encode_GeneralString(_C,Val) -> - encode_restricted_string(aligned,Val). -decode_GeneralString(Bytes,_C) -> - decode_restricted_string(Bytes,aligned). - -encode_GraphicString(_C,Val) -> - encode_restricted_string(aligned,Val). -decode_GraphicString(Bytes,_C) -> - decode_restricted_string(Bytes,aligned). - -encode_ObjectDescriptor(_C,Val) -> - encode_restricted_string(aligned,Val). -decode_ObjectDescriptor(Bytes) -> - decode_restricted_string(Bytes,aligned). - -encode_TeletexString(_C,Val) -> % equivalent with T61String - encode_restricted_string(aligned,Val). -decode_TeletexString(Bytes,_C) -> - decode_restricted_string(Bytes,aligned). - -encode_VideotexString(_C,Val) -> - encode_restricted_string(aligned,Val). -decode_VideotexString(Bytes,_C) -> - decode_restricted_string(Bytes,aligned). - - - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% getBMPChars(Bytes,Len) ->{BMPcharList,RemainingBytes} -%% -getBMPChars(Bytes,1) -> - {O1,Bytes2} = getbits(Bytes,8), - {O2,Bytes3} = getbits(Bytes2,8), - if - O1 == 0 -> - {[O2],Bytes3}; - true -> - {[{0,0,O1,O2}],Bytes3} - end; -getBMPChars(Bytes,Len) -> - getBMPChars(Bytes,Len,[]). - -getBMPChars(Bytes,0,Acc) -> - {lists:reverse(Acc),Bytes}; -getBMPChars(Bytes,Len,Acc) -> - {Octs,Bytes1} = getoctets_as_list(Bytes,2), - case Octs of - [0,O2] -> - getBMPChars(Bytes1,Len-1,[O2|Acc]); - [O1,O2]-> - getBMPChars(Bytes1,Len-1,[{0,0,O1,O2}|Acc]) - end. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% chars_encode(C,StringType,Value) -> ValueList -%% -%% encodes chars according to the per rules taking the constraint PermittedAlphabet -%% into account. -%% This function does only encode the value part and NOT the length - -chars_encode(C,StringType,Value) -> - case {StringType,get_constraint(C,'PermittedAlphabet')} of - {'UniversalString',{_,_Sv}} -> - exit({error,{asn1,{'not implemented',"UniversalString with PermittedAlphabet constraint"}}}); - {'BMPString',{_,_Sv}} -> - exit({error,{asn1,{'not implemented',"BMPString with PermittedAlphabet constraint"}}}); - _ -> - {NumBits,CharOutTab} = {get_NumBits(C,StringType),get_CharOutTab(C,StringType)}, - chars_encode2(Value,NumBits,CharOutTab) - end. - -chars_encode2([H|T],NumBits,{Min,Max,notab}) when H =< Max, H >= Min -> - [{bits,NumBits,H-Min}|chars_encode2(T,NumBits,{Min,Max,notab})]; -chars_encode2([H|T],NumBits,{Min,Max,Tab}) when H =< Max, H >= Min -> - [{bits,NumBits,exit_if_false(H,element(H-Min+1,Tab))}|chars_encode2(T,NumBits,{Min,Max,Tab})]; -chars_encode2([{A,B,C,D}|T],NumBits,{Min,Max,notab}) -> - %% no value range check here (ought to be, but very expensive) -% [{bits,NumBits,(A*B*C*D)-Min}|chars_encode2(T,NumBits,{Min,Max,notab})]; - [{bits,NumBits,((((((A bsl 8)+B) bsl 8)+C) bsl 8)+D)-Min}|chars_encode2(T,NumBits,{Min,Max,notab})]; -chars_encode2([{A,B,C,D}|T],NumBits,{Min,Max,Tab}) -> - %% no value range check here (ought to be, but very expensive) -% [{bits,NumBits,element((A*B*C*D)-Min,Tab)}|chars_encode2(T,NumBits,{Min,Max,notab})]; - [{bits,NumBits,exit_if_false({A,B,C,D},element(((((((A bsl 8)+B) bsl 8)+C) bsl 8)+D)-Min,Tab))}|chars_encode2(T,NumBits,{Min,Max,notab})]; -chars_encode2([H|_T],_,{_,_,_}) -> - exit({error,{asn1,{illegal_char_value,H}}}); -chars_encode2([],_,_) -> - []. - -exit_if_false(V,false)-> - exit({error,{asn1,{"illegal value according to Permitted alphabet constraint",V}}}); -exit_if_false(_,V) ->V. - - -get_NumBits(C,StringType) -> - case get_constraint(C,'PermittedAlphabet') of - {'SingleValue',Sv} -> - charbits(length(Sv),aligned); - no -> - case StringType of - 'IA5String' -> - charbits(128,aligned); % 16#00..16#7F - 'VisibleString' -> - charbits(95,aligned); % 16#20..16#7E - 'PrintableString' -> - charbits(74,aligned); % [$\s,$',$(,$),$+,$,,$-,$.,$/,"0123456789",$:,$=,$?,$A..$Z,$a..$z - 'NumericString' -> - charbits(11,aligned); % $ ,"0123456789" - 'UniversalString' -> - 32; - 'BMPString' -> - 16 - end - end. - -%%Maybe used later -%%get_MaxChar(C,StringType) -> -%% case get_constraint(C,'PermittedAlphabet') of -%% {'SingleValue',Sv} -> -%% lists:nth(length(Sv),Sv); -%% no -> -%% case StringType of -%% 'IA5String' -> -%% 16#7F; % 16#00..16#7F -%% 'VisibleString' -> -%% 16#7E; % 16#20..16#7E -%% 'PrintableString' -> -%% $z; % [$\s,$',$(,$),$+,$,,$-,$.,$/,"0123456789",$:,$=,$?,$A..$Z,$a..$z -%% 'NumericString' -> -%% $9; % $ ,"0123456789" -%% 'UniversalString' -> -%% 16#ffffffff; -%% 'BMPString' -> -%% 16#ffff -%% end -%% end. - -%%Maybe used later -%%get_MinChar(C,StringType) -> -%% case get_constraint(C,'PermittedAlphabet') of -%% {'SingleValue',Sv} -> -%% hd(Sv); -%% no -> -%% case StringType of -%% 'IA5String' -> -%% 16#00; % 16#00..16#7F -%% 'VisibleString' -> -%% 16#20; % 16#20..16#7E -%% 'PrintableString' -> -%% $\s; % [$\s,$',$(,$),$+,$,,$-,$.,$/,"0123456789",$:,$=,$?,$A..$Z,$a..$z -%% 'NumericString' -> -%% $\s; % $ ,"0123456789" -%% 'UniversalString' -> -%% 16#00; -%% 'BMPString' -> -%% 16#00 -%% end -%% end. - -get_CharOutTab(C,StringType) -> - get_CharTab(C,StringType,out). - -get_CharInTab(C,StringType) -> - get_CharTab(C,StringType,in). - -get_CharTab(C,StringType,InOut) -> - case get_constraint(C,'PermittedAlphabet') of - {'SingleValue',Sv} -> - get_CharTab2(C,StringType,hd(Sv),lists:max(Sv),Sv,InOut); - no -> - case StringType of - 'IA5String' -> - {0,16#7F,notab}; - 'VisibleString' -> - get_CharTab2(C,StringType,16#20,16#7F,notab,InOut); - 'PrintableString' -> - Chars = lists:sort( - " '()+,-./0123456789:=?ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"), - get_CharTab2(C,StringType,hd(Chars),lists:max(Chars),Chars,InOut); - 'NumericString' -> - get_CharTab2(C,StringType,16#20,$9," 0123456789",InOut); - 'UniversalString' -> - {0,16#FFFFFFFF,notab}; - 'BMPString' -> - {0,16#FFFF,notab} - end - end. - -get_CharTab2(C,StringType,Min,Max,Chars,InOut) -> - BitValMax = (1 bsl get_NumBits(C,StringType))-1, - if - Max =< BitValMax -> - {0,Max,notab}; - true -> - case InOut of - out -> - {Min,Max,create_char_tab(Min,Chars)}; - in -> - {Min,Max,list_to_tuple(Chars)} - end - end. - -create_char_tab(Min,L) -> - list_to_tuple(create_char_tab(Min,L,0)). -create_char_tab(Min,[Min|T],V) -> - [V|create_char_tab(Min+1,T,V+1)]; -create_char_tab(_Min,[],_V) -> - []; -create_char_tab(Min,L,V) -> - [false|create_char_tab(Min+1,L,V)]. - -%% This very inefficient and should be moved to compiletime -charbits(NumOfChars,aligned) -> - case charbits(NumOfChars) of - 1 -> 1; - 2 -> 2; - B when B =< 4 -> 4; - B when B =< 8 -> 8; - B when B =< 16 -> 16; - B when B =< 32 -> 32 - end. - -charbits(NumOfChars) when NumOfChars =< 2 -> 1; -charbits(NumOfChars) when NumOfChars =< 4 -> 2; -charbits(NumOfChars) when NumOfChars =< 8 -> 3; -charbits(NumOfChars) when NumOfChars =< 16 -> 4; -charbits(NumOfChars) when NumOfChars =< 32 -> 5; -charbits(NumOfChars) when NumOfChars =< 64 -> 6; -charbits(NumOfChars) when NumOfChars =< 128 -> 7; -charbits(NumOfChars) when NumOfChars =< 256 -> 8; -charbits(NumOfChars) when NumOfChars =< 512 -> 9; -charbits(NumOfChars) when NumOfChars =< 1024 -> 10; -charbits(NumOfChars) when NumOfChars =< 2048 -> 11; -charbits(NumOfChars) when NumOfChars =< 4096 -> 12; -charbits(NumOfChars) when NumOfChars =< 8192 -> 13; -charbits(NumOfChars) when NumOfChars =< 16384 -> 14; -charbits(NumOfChars) when NumOfChars =< 32768 -> 15; -charbits(NumOfChars) when NumOfChars =< 65536 -> 16; -charbits(NumOfChars) when is_integer(NumOfChars) -> - 16 + charbits1(NumOfChars bsr 16). - -charbits1(0) -> - 0; -charbits1(NumOfChars) -> - 1 + charbits1(NumOfChars bsr 1). - - -chars_decode(Bytes,_,'BMPString',C,Len) -> - case get_constraint(C,'PermittedAlphabet') of - no -> - getBMPChars(Bytes,Len); - _ -> - exit({error,{asn1, - {'not implemented', - "BMPString with PermittedAlphabet constraint"}}}) - end; -chars_decode(Bytes,NumBits,StringType,C,Len) -> - CharInTab = get_CharInTab(C,StringType), - chars_decode2(Bytes,CharInTab,NumBits,Len). - - -chars_decode2(Bytes,CharInTab,NumBits,Len) -> - chars_decode2(Bytes,CharInTab,NumBits,Len,[]). - -chars_decode2(Bytes,_CharInTab,_NumBits,0,Acc) -> - {lists:reverse(Acc),Bytes}; -chars_decode2(Bytes,{Min,Max,notab},NumBits,Len,Acc) when NumBits > 8 -> - {Char,Bytes2} = getbits(Bytes,NumBits), - Result = - if - Char < 256 -> Char; - true -> - list_to_tuple(binary_to_list(<<Char:32>>)) - end, - chars_decode2(Bytes2,{Min,Max,notab},NumBits,Len -1,[Result|Acc]); -% chars_decode2(Bytes,{Min,Max,notab},NumBits,Len,Acc) when NumBits > 8 -> -% {Char,Bytes2} = getbits(Bytes,NumBits), -% Result = case minimum_octets(Char+Min) of -% [NewChar] -> NewChar; -% [C1,C2] -> {0,0,C1,C2}; -% [C1,C2,C3] -> {0,C1,C2,C3}; -% [C1,C2,C3,C4] -> {C1,C2,C3,C4} -% end, -% chars_decode2(Bytes2,{Min,Max,notab},NumBits,Len -1,[Result|Acc]); -chars_decode2(Bytes,{Min,Max,notab},NumBits,Len,Acc) -> - {Char,Bytes2} = getbits(Bytes,NumBits), - chars_decode2(Bytes2,{Min,Max,notab},NumBits,Len -1,[Char+Min|Acc]); - -%% BMPString and UniversalString with PermittedAlphabet is currently not supported -chars_decode2(Bytes,{Min,Max,CharInTab},NumBits,Len,Acc) -> - {Char,Bytes2} = getbits(Bytes,NumBits), - chars_decode2(Bytes2,{Min,Max,CharInTab},NumBits,Len -1,[element(Char+1,CharInTab)|Acc]). - - -%% UTF8String -encode_UTF8String(Val) when is_binary(Val) -> - [encode_length(undefined,size(Val)),{octets,Val}]; -encode_UTF8String(Val) -> - Bin = list_to_binary(Val), - encode_UTF8String(Bin). - -decode_UTF8String(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Octs,Bytes3} = getoctets_as_list(Bytes2,Len), - {list_to_binary(Octs),Bytes3}. - - - % X.691:17 -encode_null(_) -> []. % encodes to nothing -%encode_null({Name,Val}) when is_atom(Name) -> -% encode_null(Val). - -decode_null(Bytes) -> - {'NULL',Bytes}. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_object_identifier(Val) -> CompleteList -%% encode_object_identifier({Name,Val}) -> CompleteList -%% Val -> {Int1,Int2,...,IntN} % N >= 2 -%% Name -> atom() -%% Int1 -> integer(0..2) -%% Int2 -> integer(0..39) when Int1 (0..1) else integer() -%% Int3-N -> integer() -%% CompleteList -> [{bits,8,Val}|{octets,Ol}|align|...] -%% -encode_object_identifier({Name,Val}) when is_atom(Name) -> - encode_object_identifier(Val); -encode_object_identifier(Val) -> - OctetList = e_object_identifier(Val), - Octets = list_to_binary(OctetList), % performs a flatten at the same time - [{debug,object_identifier},encode_length(undefined,size(Octets)),{octets,Octets}]. - -%% This code is copied from asn1_encode.erl (BER) and corrected and modified - -e_object_identifier({'OBJECT IDENTIFIER',V}) -> - e_object_identifier(V); -e_object_identifier({Cname,V}) when is_atom(Cname),is_tuple(V) -> - e_object_identifier(tuple_to_list(V)); -e_object_identifier({Cname,V}) when is_atom(Cname),is_list(V) -> - e_object_identifier(V); -e_object_identifier(V) when is_tuple(V) -> - e_object_identifier(tuple_to_list(V)); - -%% E1 = 0|1|2 and (E2 < 40 when E1 = 0|1) -e_object_identifier([E1,E2|Tail]) when E1 >= 0, E1 < 2, E2 < 40 ; E1==2 -> - Head = 40*E1 + E2, % weird - e_object_elements([Head|Tail],[]); -e_object_identifier(Oid=[_,_|_Tail]) -> - exit({error,{asn1,{'illegal_value',Oid}}}). - -e_object_elements([],Acc) -> - lists:reverse(Acc); -e_object_elements([H|T],Acc) -> - e_object_elements(T,[e_object_element(H)|Acc]). - -e_object_element(Num) when Num < 128 -> - [Num]; -e_object_element(Num) -> - [e_o_e(Num bsr 7)|[Num band 2#1111111]]. -e_o_e(Num) when Num < 128 -> - Num bor 2#10000000; -e_o_e(Num) -> - [e_o_e(Num bsr 7)|[(Num band 2#1111111) bor 2#10000000]]. - - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_object_identifier(Bytes) -> {ObjId,RemainingBytes} -%% ObjId -> {integer(),integer(),...} % at least 2 integers -%% RemainingBytes -> [integer()] when integer() (0..255) -decode_object_identifier(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Octs,Bytes3} = getoctets_as_list(Bytes2,Len), - [First|Rest] = dec_subidentifiers(Octs,0,[]), - Idlist = if - First < 40 -> - [0,First|Rest]; - First < 80 -> - [1,First - 40|Rest]; - true -> - [2,First - 80|Rest] - end, - {list_to_tuple(Idlist),Bytes3}. - -dec_subidentifiers([H|T],Av,Al) when H >=16#80 -> - dec_subidentifiers(T,(Av bsl 7) + (H band 16#7F),Al); -dec_subidentifiers([H|T],Av,Al) -> - dec_subidentifiers(T,0,[(Av bsl 7) + H |Al]); -dec_subidentifiers([],_Av,Al) -> - lists:reverse(Al). - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_relative_oid(Val) -> CompleteList -%% encode_relative_oid({Name,Val}) -> CompleteList -encode_relative_oid({Name,Val}) when is_atom(Name) -> - encode_relative_oid(Val); -encode_relative_oid(Val) when is_tuple(Val) -> - encode_relative_oid(tuple_to_list(Val)); -encode_relative_oid(Val) when is_list(Val) -> - Octets = list_to_binary([e_object_element(X)||X <- Val]), - [encode_length(undefined,size(Octets)),{octets,Octets}]. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_relative_oid(Val) -> {ROID,Rest} -%% decode_relative_oid({Name,Val}) -> {ROID,Rest} -decode_relative_oid(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Octs,Bytes3} = getoctets_as_list(Bytes2,Len), - ObjVals = dec_subidentifiers(Octs,0,[]), - {list_to_tuple(ObjVals),Bytes3}. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_real(Val) -> CompleteList -%% encode_real({Name,Val}) -> CompleteList -encode_real({Name,Val}) when is_atom(Name) -> - encode_real(Val); -encode_real(Real) -> - {EncVal,Len} = ?RT_COMMON:encode_real([],Real), - [encode_length(undefined,Len),{octets,EncVal}]. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_real(Val) -> {REALvalue,Rest} -%% decode_real({Name,Val}) -> {REALvalue,Rest} -decode_real(Bytes) -> - {Len,{0,Bytes2}} = decode_length(Bytes,undefined), - {RealVal,Rest,Len} = ?RT_COMMON:decode_real(Bytes2,Len), - {RealVal,{0,Rest}}. - - -get_constraint([{Key,V}],Key) -> - V; -get_constraint([],_Key) -> - no; -get_constraint(C,Key) -> - case lists:keysearch(Key,1,C) of - false -> - no; - {value,{_,V}} -> - V - end. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% complete(InList) -> ByteList -%% Takes a coded list with bits and bytes and converts it to a list of bytes -%% Should be applied as the last step at encode of a complete ASN.1 type -%% - -% complete(L) -> -% case complete1(L) of -% {[],0} -> -% <<0>>; -% {Acc,0} -> -% lists:reverse(Acc); -% {[Hacc|Tacc],Acclen} -> % Acclen >0 -% Rest = 8 - Acclen, -% NewHacc = Hacc bsl Rest, -% lists:reverse([NewHacc|Tacc]) -% end. - - -% complete1(InList) when is_list(InList) -> -% complete1(InList,[]); -% complete1(InList) -> -% complete1([InList],[]). - -% complete1([{debug,_}|T], Acc) -> -% complete1(T,Acc); -% complete1([H|T],Acc) when is_list(H) -> -% {NewH,NewAcclen} = complete1(H,Acc), -% complete1(T,NewH,NewAcclen); - -% complete1([{0,Bin}|T],Acc,0) when is_binary(Bin) -> -% complete1(T,[Bin|Acc],0); -% complete1([{Unused,Bin}|T],Acc,0) when is_integer(Unused),is_binary(Bin) -> -% Size = size(Bin)-1, -% <<Bs:Size/binary,B>> = Bin, -% complete1(T,[(B bsr Unused),Bs|Acc],8-Unused); -% complete1([{Unused,Bin}|T],[Hacc|Tacc],Acclen) when is_integer(Unused),is_binary(Bin) -> -% Rest = 8 - Acclen, -% Used = 8 - Unused, -% case size(Bin) of -% 1 -> -% if -% Rest >= Used -> -% <<B:Used,_:Unused>> = Bin, -% complete1(T,[(Hacc bsl Used) + B|Tacc], -% (Acclen+Used) rem 8); -% true -> -% LeftOver = 8 - Rest - Unused, -% <<Val2:Rest,Val1:LeftOver,_:Unused>> = Bin, -% complete1(T,[Val1,(Hacc bsl Rest) + Val2|Tacc], -% (Acclen+Used) rem 8) -% end; -% N -> -% if -% Rest == Used -> -% N1 = N - 1, -% <<B:Rest,Bs:N1/binary,_:Unused>> = Bin, -% complete1(T,[Bs,(Hacc bsl Rest) + B|Tacc],0); -% Rest > Used -> -% N1 = N - 2, -% N2 = (8 - Rest) + Used, -% <<B1:Rest,Bytes:N1/binary,B2:N2,_:Unused>> = Bin, -% complete1(T,[B2,Bytes,(Hacc bsl Rest) + B1|Tacc], -% (Acclen + Used) rem 8); -% true -> % Rest < Used -% N1 = N - 1, -% N2 = Used - Rest, -% <<B1:Rest,Bytes:N1/binary,B2:N2,_:Unused>> = Bin, -% complete1(T,[B2,Bytes,(Hacc bsl Rest) + B1|Tacc], -% (Acclen + Used) rem 8) -% end -% end; - -% %complete1([{octets,N,Val}|T],Acc,Acclen) when N =< 4 ,is_integer(Val) -> -% % complete1([{octets,<<Val:N/unit:8>>}|T],Acc,Acclen); -% complete1([{octets,N,Val}|T],Acc,Acclen) when N =< 4 ,is_integer(Val) -> -% Newval = case N of -% 1 -> -% Val4 = Val band 16#FF, -% [Val4]; -% 2 -> -% Val3 = (Val bsr 8) band 16#FF, -% Val4 = Val band 16#FF, -% [Val3,Val4]; -% 3 -> -% Val2 = (Val bsr 16) band 16#FF, -% Val3 = (Val bsr 8) band 16#FF, -% Val4 = Val band 16#FF, -% [Val2,Val3,Val4]; -% 4 -> -% Val1 = (Val bsr 24) band 16#FF, -% Val2 = (Val bsr 16) band 16#FF, -% Val3 = (Val bsr 8) band 16#FF, -% Val4 = Val band 16#FF, -% [Val1,Val2,Val3,Val4] -% end, -% complete1([{octets,Newval}|T],Acc,Acclen); - -% complete1([{octets,Bin}|T],Acc,Acclen) when is_binary(Bin) -> -% Rest = 8 - Acclen, -% if -% Rest == 8 -> -% complete1(T,[Bin|Acc],0); -% true -> -% [Hacc|Tacc]=Acc, -% complete1(T,[Bin, Hacc bsl Rest|Tacc],0) -% end; - -% complete1([{octets,Oct}|T],Acc,Acclen) when is_list(Oct) -> -% Rest = 8 - Acclen, -% if -% Rest == 8 -> -% complete1(T,[list_to_binary(Oct)|Acc],0); -% true -> -% [Hacc|Tacc]=Acc, -% complete1(T,[list_to_binary(Oct), Hacc bsl Rest|Tacc],0) -% end; - -% complete1([{bit,Val}|T], Acc, Acclen) -> -% complete1([{bits,1,Val}|T],Acc,Acclen); -% complete1([{octet,Val}|T], Acc, Acclen) -> -% complete1([{octets,1,Val}|T],Acc,Acclen); - -% complete1([{bits,N,Val}|T], Acc, 0) when N =< 8 -> -% complete1(T,[Val|Acc],N); -% complete1([{bits,N,Val}|T], [Hacc|Tacc], Acclen) when N =< 8 -> -% Rest = 8 - Acclen, -% if -% Rest >= N -> -% complete1(T,[(Hacc bsl N) + Val|Tacc],(Acclen+N) rem 8); -% true -> -% Diff = N - Rest, -% NewHacc = (Hacc bsl Rest) + (Val bsr Diff), -% Mask = element(Diff,{1,3,7,15,31,63,127,255}), -% complete1(T,[(Val band Mask),NewHacc|Tacc],(Acclen+N) rem 8) -% end; -% complete1([{bits,N,Val}|T], Acc, Acclen) -> % N > 8 -% complete1([{bits,N-8,Val bsr 8},{bits,8,Val band 255}|T],Acc,Acclen); - -% complete1([align|T],Acc,0) -> -% complete1(T,Acc,0); -% complete1([align|T],[Hacc|Tacc],Acclen) -> -% Rest = 8 - Acclen, -% complete1(T,[Hacc bsl Rest|Tacc],0); -% complete1([{octets,N,Val}|T],Acc,Acclen) when is_list(Val) -> % no security check here -% complete1([{octets,Val}|T],Acc,Acclen); - -% complete1([],Acc,Acclen) -> -% {Acc,Acclen}. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% complete(InList) -> ByteList -%% Takes a coded list with bits and bytes and converts it to a list of bytes -%% Should be applied as the last step at encode of a complete ASN.1 type -%% - -complete(L) -> - case complete1(L) of - {[],[]} -> - <<0>>; - {Acc,[]} -> - Acc; - {Acc,Bacc} -> - [Acc|complete_bytes(Bacc)] - end. - -%% this function builds the ugly form of lists [E1|E2] to avoid having to reverse it at the end. -%% this is done because it is efficient and that the result always will be sent on a port or -%% converted by means of list_to_binary/1 -complete1(InList) when is_list(InList) -> - complete1(InList,[],[]); -complete1(InList) -> - complete1([InList],[],[]). - -complete1([],Acc,Bacc) -> - {Acc,Bacc}; -complete1([H|T],Acc,Bacc) when is_list(H) -> - {NewH,NewBacc} = complete1(H,Acc,Bacc), - complete1(T,NewH,NewBacc); - -complete1([{octets,Bin}|T],Acc,[]) -> - complete1(T,[Acc|Bin],[]); - -complete1([{octets,Bin}|T],Acc,Bacc) -> - complete1(T,[Acc|[complete_bytes(Bacc),Bin]],[]); - -complete1([{debug,_}|T], Acc,Bacc) -> - complete1(T,Acc,Bacc); - -complete1([{bits,N,Val}|T],Acc,Bacc) -> - complete1(T,Acc,complete_update_byte(Bacc,Val,N)); - -complete1([{bit,Val}|T],Acc,Bacc) -> - complete1(T,Acc,complete_update_byte(Bacc,Val,1)); - -complete1([align|T],Acc,[]) -> - complete1(T,Acc,[]); -complete1([align|T],Acc,Bacc) -> - complete1(T,[Acc|complete_bytes(Bacc)],[]); -complete1([{0,Bin}|T],Acc,[]) when is_binary(Bin) -> - complete1(T,[Acc|Bin],[]); -complete1([{Unused,Bin}|T],Acc,[]) when is_integer(Unused),is_binary(Bin) -> - Size = size(Bin)-1, - <<Bs:Size/binary,B>> = Bin, - NumBits = 8-Unused, - complete1(T,[Acc|Bs],[[B bsr Unused]|NumBits]); -complete1([{Unused,Bin}|T],Acc,Bacc) when is_integer(Unused),is_binary(Bin) -> - Size = size(Bin)-1, - <<Bs:Size/binary,B>> = Bin, - NumBits = 8 - Unused, - Bf = complete_bytes(Bacc), - complete1(T,[Acc|[Bf,Bs]],[[B bsr Unused]|NumBits]). - - -complete_update_byte([],Val,Len) -> - complete_update_byte([[0]|0],Val,Len); -complete_update_byte([[Byte|Bacc]|NumBits],Val,Len) when NumBits + Len == 8 -> - [[0,((Byte bsl Len) + Val) band 255|Bacc]|0]; -complete_update_byte([[Byte|Bacc]|NumBits],Val,Len) when NumBits + Len > 8 -> - Rem = 8 - NumBits, - Rest = Len - Rem, - complete_update_byte([[0,((Byte bsl Rem) + (Val bsr Rest)) band 255 |Bacc]|0],Val,Rest); -complete_update_byte([[Byte|Bacc]|NumBits],Val,Len) -> - [[((Byte bsl Len) + Val) band 255|Bacc]|NumBits+Len]. - - -complete_bytes([[_Byte|Bacc]|0]) -> - lists:reverse(Bacc); -complete_bytes([[Byte|Bacc]|NumBytes]) -> - lists:reverse([(Byte bsl (8-NumBytes)) band 255|Bacc]); -complete_bytes([]) -> - []. - -% complete_bytes(L) -> -% complete_bytes1(lists:reverse(L),[],[],0,0). - -% complete_bytes1([H={V,B}|T],Acc,ReplyAcc,NumBits,NumFields) when ((NumBits+B) rem 8) == 0 -> -% NewReplyAcc = [complete_bytes2([H|Acc],0)|ReplyAcc], -% complete_bytes1(T,[],NewReplyAcc,0,0); -% complete_bytes1([H={V,B}|T],Acc,ReplyAcc,NumBits,NumFields) when NumFields == 7; (NumBits+B) div 8 > 0 -> -% Rem = (NumBits+B) rem 8, -% NewReplyAcc = [complete_bytes2([{V bsr Rem,B - Rem}|Acc],0)|ReplyAcc], -% complete_bytes1([{V,Rem}|T],[],NewReplyAcc,0,0); -% complete_bytes1([H={V,B}|T],Acc,ReplyAcc,NumBits,NumFields) -> -% complete_bytes1(T,[H|Acc],ReplyAcc,NumBits+B,NumFields+1); -% complete_bytes1([],[],ReplyAcc,_,_) -> -% lists:reverse(ReplyAcc); -% complete_bytes1([],Acc,ReplyAcc,NumBits,_) -> -% PadBits = case NumBits rem 8 of -% 0 -> 0; -% Rem -> 8 - Rem -% end, -% lists:reverse([complete_bytes2(Acc,PadBits)|ReplyAcc]). - - -% complete_bytes2([{V1,B1}],PadBits) -> -% <<V1:B1,0:PadBits>>; -% complete_bytes2([{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,0:PadBits>>; -% complete_bytes2([{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,0:PadBits>>; -% complete_bytes2([{V4,B4},{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,V4:B4,0:PadBits>>; -% complete_bytes2([{V5,B5},{V4,B4},{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,V4:B4,V5:B5,0:PadBits>>; -% complete_bytes2([{V6,B6},{V5,B5},{V4,B4},{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,V4:B4,V5:B5,V6:B6,0:PadBits>>; -% complete_bytes2([{V7,B7},{V6,B6},{V5,B5},{V4,B4},{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,V4:B4,V5:B5,V6:B6,V7:B7,0:PadBits>>; -% complete_bytes2([{V8,B8},{V7,B7},{V6,B6},{V5,B5},{V4,B4},{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,V4:B4,V5:B5,V6:B6,V7:B7,V8:B8,0:PadBits>>. - - - - - - diff --git a/lib/asn1/src/asn1rt_per_bin_rt2ct.erl b/lib/asn1/src/asn1rt_per_bin_rt2ct.erl index 1df757a47f..5997232f13 100644 --- a/lib/asn1/src/asn1rt_per_bin_rt2ct.erl +++ b/lib/asn1/src/asn1rt_per_bin_rt2ct.erl @@ -22,25 +22,21 @@ -include("asn1_records.hrl"). --export([dec_fixup/3, cindex/3, list_to_record/2]). +-export([decode_fragmented/3]). -export([setchoiceext/1, setext/1, fixoptionals/3, fixextensions/2, - getext/1, getextension/2, skipextensions/3, getbit/1, getchoice/3 ]). --export([getoptionals/2, getoptionals2/2, - set_choice/3, encode_integer/2, encode_integer/3 ]). --export([decode_integer/2, decode_integer/3, encode_small_number/1, - decode_boolean/1, encode_length/2, decode_length/1, decode_length/2, - encode_small_length/1, decode_small_length/1, + skipextensions/3, getbit/1, getchoice/3 ]). +-export([set_choice/3, encode_integer/2, encode_integer/3 ]). +-export([encode_small_number/1, + encode_length/2, + encode_small_length/1, decode_compact_bit_string/3]). --export([decode_enumerated/3, - encode_bit_string/3, decode_bit_string/3 ]). --export([encode_octet_string/2, decode_octet_string/2, - encode_null/1, decode_null/1, +-export([encode_bit_string/3, decode_bit_string/3 ]). +-export([encode_octet_string/2, encode_object_identifier/1, decode_object_identifier/1, encode_real/1, decode_real/1, encode_relative_oid/1, decode_relative_oid/1, complete/1]). - -export([encode_open_type/2, decode_open_type/2]). -export([encode_GeneralString/2, decode_GeneralString/2, @@ -51,19 +47,10 @@ encode_UTF8String/1,decode_UTF8String/1 ]). --export([decode_constrained_number/2, - decode_constrained_number/3, - decode_unconstrained_number/1, - decode_semi_constrained_number/2, - encode_unconstrained_number/1, - decode_constrained_number/4, +-export([encode_unconstrained_number/1, encode_octet_string/3, - decode_octet_string/3, encode_known_multiplier_string/5, - decode_known_multiplier_string/5, - getoctets/2, getbits/2 -% start_drv/1,start_drv2/1,init_drv/1 - ]). + decode_known_multiplier_string/5]). -export([eint_positive/1]). @@ -73,32 +60,6 @@ -define('32K',32768). -define('64K',65536). -%%-define(nodriver,true). - -dec_fixup(Terms,Cnames,RemBytes) -> - dec_fixup(Terms,Cnames,RemBytes,[]). - -dec_fixup([novalue|T],[_Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,Acc); -dec_fixup([{_Name,novalue}|T],[_Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,Acc); -dec_fixup([H|T],[Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,[{Hc,H}|Acc]); -dec_fixup([],_Cnames,RemBytes,Acc) -> - {lists:reverse(Acc),RemBytes}. - -cindex(Ix,Val,Cname) -> - case element(Ix,Val) of - {Cname,Val2} -> Val2; - X -> X - end. - -%% converts a list to a record if necessary -list_to_record(_,Tuple) when is_tuple(Tuple) -> - Tuple; -list_to_record(Name,List) when is_list(List) -> - list_to_tuple([Name|List]). - %%-------------------------------------------------------- %% setchoiceext(InRootSet) -> [{bit,X}] %% X is set to 1 when InRootSet==false @@ -143,15 +104,6 @@ fixoptionals([Pos|Ot],Val,Acc) -> end. -getext(Bytes) when is_bitstring(Bytes) -> - getbit(Bytes). - -getextension(0, Bytes) -> - {<<>>,Bytes}; -getextension(1, Bytes) -> - {Len,Bytes2} = decode_small_length(Bytes), - getbits_as_binary(Len,Bytes2).% {Bin,Bytes3}. - fixextensions({ext,ExtPos,ExtNum},Val) -> case fixextensions(ExtPos,ExtNum+ExtPos,Val,0) of 0 -> []; @@ -194,14 +146,6 @@ getchoice(Bytes,_,1) -> getchoice(Bytes,NumChoices,0) -> decode_constrained_number(Bytes,{0,NumChoices-1}). -%% old version kept for backward compatibility with generates from R7B01 -getoptionals(Bytes,NumOpt) -> - getbits_as_binary(NumOpt,Bytes). - -%% new version used in generates from r8b_patch/3 and later -getoptionals2(Bytes,NumOpt) -> - {_,_} = getbits(Bytes,NumOpt). - %% getbits_as_binary(Num,Bytes) -> {Bin,Rest} %% Num = integer(), @@ -344,32 +288,6 @@ decode_fragmented_bits(<<0:1,Len:7,Bin/binary>>,C,Acc) -> exit({error,{asn1,{illegal_value,C,BinBits}}}) end. - -decode_fragmented_octets(Bin,C) -> - decode_fragmented_octets(Bin,C,[]). - -decode_fragmented_octets(<<3:2,Len:6,Bin/binary>>,C,Acc) -> - {Value,Bin2} = split_binary(Bin,Len * ?'16K'), - decode_fragmented_octets(Bin2,C,[Value|Acc]); -decode_fragmented_octets(<<0:1,0:7,Bin/binary>>,C,Acc) -> - Octets = list_to_binary(lists:reverse(Acc)), - case C of - Int when is_integer(Int), C == size(Octets) -> - {Octets,Bin}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,Octets}}}) - end; -decode_fragmented_octets(<<0:1,Len:7,Bin/binary>>,C,Acc) -> - <<Value:Len/binary-unit:8,Bin2/binary>> = Bin, - BinOctets = list_to_binary(lists:reverse([Value|Acc])), - case C of - Int when is_integer(Int),size(BinOctets) == Int -> - {BinOctets,Bin2}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,BinOctets}}}) - end. - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% encode_open_type(Constraint, Value) -> CompleteList @@ -454,40 +372,6 @@ encode_integer(_,Val) -> exit({error,{asn1,{illegal_value,Val}}}). - -decode_integer(Buffer,Range,NamedNumberList) -> - {Val,Buffer2} = decode_integer(Buffer,Range), - case lists:keysearch(Val,2,NamedNumberList) of - {value,{NewVal,_}} -> {NewVal,Buffer2}; - _ -> {Val,Buffer2} - end. - -decode_integer(Buffer,[{Rc,_Ec}]) when is_tuple(Rc) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> decode_integer(Buffer2,[Rc]); - 1 -> decode_unconstrained_number(Buffer2) - end; -decode_integer(Buffer,undefined) -> - decode_unconstrained_number(Buffer); -decode_integer(Buffer,C) -> - case get_constraint(C,'SingleValue') of - V when is_integer(V) -> - {V,Buffer}; - _ -> - decode_integer1(Buffer,C) - end. - -decode_integer1(Buffer,C) -> - case VR = get_constraint(C,'ValueRange') of - no -> - decode_unconstrained_number(Buffer); - {Lb, 'MAX'} -> - decode_semi_constrained_number(Buffer,Lb); - {_Lb,_Ub} -> - decode_constrained_number(Buffer,VR) - end. - %% X.691:10.6 Encoding of a normally small non-negative whole number %% Use this for encoding of CHOICE index if there is an extension marker in %% the CHOICE @@ -507,7 +391,7 @@ decode_small_number(Bytes) -> 0 -> getbits(Bytes2,6); 1 -> - decode_semi_constrained_number(Bytes2,0) + decode_semi_constrained_number(Bytes2) end. %% X.691:10.7 Encoding of a semi-constrained whole number @@ -530,12 +414,10 @@ encode_semi_constrained_number(Lb,Val) -> [encode_length(undefined,Len),[21,<<Len:16>>,Oct]] end. -decode_semi_constrained_number(Bytes,{Lb,_}) -> - decode_semi_constrained_number(Bytes,Lb); -decode_semi_constrained_number(Bytes,Lb) -> +decode_semi_constrained_number(Bytes) -> {Len,Bytes2} = decode_length(Bytes,undefined), {V,Bytes3} = getoctets(Bytes2,Len), - {V+Lb,Bytes3}. + {V,Bytes3}. encode_constrained_number({Lb,_Ub},_Range,{bits,N},Val) -> Val2 = Val-Lb, @@ -609,7 +491,7 @@ encode_constrained_number({Lb,Ub},Val) when Val >= Lb, Ub >= Val -> RangeOcts = binary:encode_unsigned(Range - 1), OctsLen = erlang:byte_size(Octs), RangeOctsLen = erlang:byte_size(RangeOcts), - LengthBitsNeeded = asn1rt_per_bin:minimum_bits(RangeOctsLen - 1), + LengthBitsNeeded = minimum_bits(RangeOctsLen - 1), [10,LengthBitsNeeded,OctsLen-1,20,OctsLen,Octs]; true -> exit({not_supported,{integer_range,Range}}) @@ -621,13 +503,6 @@ decode_constrained_number(Buffer,VR={Lb,Ub}) -> Range = Ub - Lb + 1, decode_constrained_number(Buffer,VR,Range). -decode_constrained_number(Buffer,{Lb,_Ub},_Range,{bits,N}) -> - {Val,Remain} = getbits(Buffer,N), - {Val+Lb,Remain}; -decode_constrained_number(Buffer,{Lb,_Ub},_Range,{octets,N}) -> - {Val,Remain} = getoctets(Buffer,N), - {Val+Lb,Remain}. - decode_constrained_number(Buffer,{Lb,_Ub},Range) -> % Val2 = Val - Lb, {Val,Remain} = @@ -665,6 +540,16 @@ decode_constrained_number(Buffer,{Lb,_Ub},Range) -> end, {Val+Lb,Remain}. +%% For some reason the minimum bits needed in the length field in +%% the encoding of constrained whole numbers must always be at least 2? +minimum_bits(N) when N < 4 -> 2; +minimum_bits(N) when N < 8 -> 3; +minimum_bits(N) when N < 16 -> 4; +minimum_bits(N) when N < 32 -> 5; +minimum_bits(N) when N < 64 -> 6; +minimum_bits(N) when N < 128 -> 7; +minimum_bits(_N) -> 8. + %% X.691:10.8 Encoding of an unconstrained whole number encode_unconstrained_number(Val) when Val >= 0 -> @@ -718,23 +603,6 @@ enint(-1, [B1|T]) when B1 > 127 -> enint(N, Acc) -> enint(N bsr 8, [N band 16#ff|Acc]). -decode_unconstrained_number(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Ints,Bytes3} = getoctets_as_bin(Bytes2,Len), - {dec_integer(Ints),Bytes3}. - -dec_integer(Bin = <<0:1,_:7,_/binary>>) -> - decpint(Bin); -dec_integer(<<_:1,B:7,BitStr/bitstring>>) -> - Size = bit_size(BitStr), - <<I:Size>> = BitStr, - (-128 + B) bsl bit_size(BitStr) bor I. - -decpint(Bin) -> - Size = bit_size(Bin), - <<Int:Size>> = Bin, - Int. - %% X.691:10.9 Encoding of a length determinant %%encode_small_length(undefined,Len) -> % null means no UpperBound %% encode_small_number(Len). @@ -779,18 +647,6 @@ encode_small_length(Len) -> [1,encode_length(undefined,Len)]. -decode_small_length(Buffer) -> - case getbit(Buffer) of - {0,Remain} -> - {Bits,Remain2} = getbits(Remain,6), - {Bits+1,Remain2}; - {1,Remain} -> - decode_length(Remain,undefined) - end. - -decode_length(Buffer) -> - decode_length(Buffer,undefined). - decode_length(Buffer,undefined) -> % un-constrained case align(Buffer) of <<0:1,Oct:7,Rest/binary>> -> @@ -833,38 +689,6 @@ decode_length(Buffer,SingleValue) when is_integer(SingleValue) -> {SingleValue,Buffer}. - % X.691:11 -decode_boolean(Buffer) -> %when record(Buffer,buffer) - case getbit(Buffer) of - {1,Remain} -> {true,Remain}; - {0,Remain} -> {false,Remain} - end. - - -%% ENUMERATED with extension marker -decode_enumerated(Buffer,C,{Ntup1,Ntup2}) when is_tuple(Ntup1), is_tuple(Ntup2) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> % not an extension value - {Val,Buffer3} = decode_integer(Buffer2,C), - case catch (element(Val+1,Ntup1)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,[Ntup1,Ntup2]}}}}) - end; - 1 -> % this an extension value - {Val,Buffer3} = decode_small_number(Buffer2), - case catch (element(Val+1,Ntup2)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _ -> {{asn1_enum,Val},Buffer3} - end - end; - -decode_enumerated(Buffer,C,NamedNumberTup) when is_tuple(NamedNumberTup) -> - {Val,Buffer2} = decode_integer(Buffer,C), - case catch (element(Val+1,NamedNumberTup)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer2}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,NamedNumberTup}}}}) - end. %%=============================================================================== %%=============================================================================== @@ -1315,49 +1139,74 @@ encode_octet_string(SZ={_,_},false,Val) -> % [encode_length(SZ,length(Val)),align, % {octets,Val}]; Len = length(Val), - [encode_length(SZ,Len),2, - octets_to_complete(Len,Val)]; + try + [encode_length(SZ,Len),2, + octets_to_complete(Len,Val)] + catch + exit:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end; encode_octet_string(SZ,false,Val) when is_list(SZ) -> Len = length(Val), - [encode_length({hd(SZ),lists:max(SZ)},Len),2, - octets_to_complete(Len,Val)]; + try + [encode_length({hd(SZ),lists:max(SZ)},Len),2, + octets_to_complete(Len,Val)] + catch + exit:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end; +encode_octet_string(Sv,false,Val) when is_integer(Sv) -> + encode_fragmented_octet_string(Val); encode_octet_string(no,false,Val) -> Len = length(Val), - [encode_length(undefined,Len),2, - octets_to_complete(Len,Val)]; + try + [encode_length(undefined,Len),2, + octets_to_complete(Len,Val)] + catch + exit:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end; encode_octet_string(C,_,_) -> exit({error,{not_implemented,C}}). - -decode_octet_string(Bytes,Range) -> - decode_octet_string(Bytes,Range,false). - -decode_octet_string(<<B1,Bytes/bitstring>>,1,false) -> -%% {B1,Bytes2} = getbits(Bytes,8), - {[B1],Bytes}; -decode_octet_string(<<B1,B2,Bytes/bitstring>>,2,false) -> -%% {Bs,Bytes2}= getbits(Bytes,16), -%% {binary_to_list(<<Bs:16>>),Bytes2}; - {[B1,B2],Bytes}; -decode_octet_string(Bytes,Sv,false) when is_integer(Sv),Sv=<65535 -> - %% Bytes2 = align(Bytes), - %% getoctets_as_list aligns buffer before it picks octets - getoctets_as_list(Bytes,Sv); -decode_octet_string(Bytes,Sv,false) when is_integer(Sv) -> - Bytes2 = align(Bytes), - decode_fragmented_octets(Bytes2,Sv); -decode_octet_string(Bytes,{Lb,Ub},false) -> - {Len,Bytes2} = decode_length(Bytes,{Lb,Ub}), -%% Bytes3 = align(Bytes2), - getoctets_as_list(Bytes2,Len); -decode_octet_string(Bytes,Sv,false) when is_list(Sv) -> - {Len,Bytes2} = decode_length(Bytes,{hd(Sv),lists:max(Sv)}), -%% Bytes3 = align(Bytes2), - getoctets_as_list(Bytes2,Len); -decode_octet_string(Bytes,no,false) -> - {Len,Bytes2} = decode_length(Bytes,undefined), -%% Bytes3 = align(Bytes2), - getoctets_as_list(Bytes2,Len). +encode_fragmented_octet_string(Val) -> + Bin = iolist_to_binary(Val), + efos_1(Bin). + +efos_1(<<B1:16#C000/binary,B2:16#4000/binary,T/binary>>) -> + [20,1,<<3:2,4:6>>, + octets_to_complete(16#C000, B1), + octets_to_complete(16#4000, B2)|efos_1(T)]; +efos_1(<<B:16#C000/binary,T/binary>>) -> + [20,1,<<3:2,3:6>>,octets_to_complete(16#C000, B)|efos_1(T)]; +efos_1(<<B:16#8000/binary,T/binary>>) -> + [20,1,<<3:2,2:6>>,octets_to_complete(16#8000, B)|efos_1(T)]; +efos_1(<<B:16#4000/binary,T/binary>>) -> + [20,1,<<3:2,1:6>>,octets_to_complete(16#4000, B)|efos_1(T)]; +efos_1(<<>>) -> + [20,1,0]; +efos_1(<<B/bitstring>>) -> + Len = byte_size(B), + [encode_length(undefined, Len),octets_to_complete(Len, B)]. + +decode_fragmented(SegSz0, Buf0, Unit) -> + SegSz = SegSz0 * Unit * ?'16K', + <<Res:SegSz/bitstring,Buf/bitstring>> = Buf0, + decode_fragmented_1(Buf, Unit, Res). + +decode_fragmented_1(<<0:1,N:7,Buf0/bitstring>>, Unit, Res) -> + Sz = N*Unit, + <<S:Sz/bitstring,Buf/bitstring>> = Buf0, + {<<Res/bitstring,S/bitstring>>,Buf}; +decode_fragmented_1(<<1:1,0:1,N:14,Buf0/bitstring>>, Unit, Res) -> + Sz = N*Unit, + <<S:Sz/bitstring,Buf/bitstring>> = Buf0, + {<<Res/bitstring,S/bitstring>>,Buf}; +decode_fragmented_1(<<1:1,1:1,SegSz0:6,Buf0/bitstring>>, Unit, Res0) -> + SegSz = SegSz0 * Unit * ?'16K', + <<Frag:SegSz/bitstring,Buf/bitstring>> = Buf0, + Res = <<Res0/bitstring,Frag/bitstring>>, + decode_fragmented_1(Buf, Unit, Res). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1545,16 +1394,6 @@ chars_decode2(Bytes,{Min,Max,CharInTab},NumBits,Len,Acc) -> chars_decode2(Bytes2,{Min,Max,CharInTab},NumBits,Len -1,[element(Char+1,CharInTab)|Acc]). - % X.691:17 -encode_null(_Val) -> []. % encodes to nothing -%encode_null({Name,Val}) when is_atom(Name) -> -% encode_null(Val). - -decode_null(Bytes) -> - {'NULL',Bytes}. - - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% encode_UTF8String(Val) -> CompleteList %% Val -> <<utf8encoded binary>> diff --git a/lib/asn1/src/asn1rt_uper_bin.erl b/lib/asn1/src/asn1rt_uper_bin.erl index abe178a69e..9410c3ef90 100644 --- a/lib/asn1/src/asn1rt_uper_bin.erl +++ b/lib/asn1/src/asn1rt_uper_bin.erl @@ -25,22 +25,20 @@ %%-compile(export_all). - -export([cindex/3, list_to_record/2]). - -export([setext/1, fixoptionals/3, +-export([decode_fragmented/3]). +-export([setext/1, fixoptionals/3, fixextensions/2, - getext/1, getextension/2, skipextensions/3, getbit/1, getchoice/3 ]). - -export([getoptionals2/2, set_choice/3, encode_integer/2, encode_integer/3 ]). - -export([decode_integer/2, decode_integer/3, encode_small_number/1, encode_boolean/1, - decode_boolean/1, encode_length/2, decode_length/1, decode_length/2, - encode_small_length/1, decode_small_length/1, + skipextensions/3, getbit/1, getchoice/3 ]). +-export([set_choice/3, encode_integer/2, encode_integer/3]). +-export([encode_small_number/1, encode_boolean/1, + encode_length/2, + encode_small_length/1, decode_compact_bit_string/3]). - -export([decode_enumerated/3, - encode_bit_string/3, decode_bit_string/3 ]). - -export([encode_octet_string/2, decode_octet_string/2, - encode_null/1, decode_null/1, - encode_relative_oid/1, decode_relative_oid/1, +-export([encode_bit_string/3, decode_bit_string/3 ]). +-export([encode_octet_string/2, + encode_relative_oid/1, decode_relative_oid/1, encode_object_identifier/1, decode_object_identifier/1, - encode_real/1, decode_real/1, + encode_real/1, decode_real/1, complete/1, complete_NFP/1]). @@ -65,19 +63,6 @@ -define('64K',65536). -cindex(Ix,Val,Cname) -> - case element(Ix,Val) of - {Cname,Val2} -> Val2; - X -> X - end. - -%% converts a list to a record if necessary -list_to_record(_Name,Tuple) when is_tuple(Tuple) -> - Tuple; -list_to_record(Name,List) when is_list(List) -> - list_to_tuple([Name|List]). - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% setext(true|false) -> CompleteList %% @@ -113,16 +98,6 @@ fixoptionals([Pos|Ot],Val,Acc) -> end. -getext(Bytes) when is_bitstring(Bytes) -> - getbit(Bytes). - -getextension(0, Bytes) -> - {{},Bytes}; -getextension(1, Bytes) -> - {Len,Bytes2} = decode_small_length(Bytes), - {Blist, Bytes3} = getbits_as_list(Len,Bytes2), - {list_to_tuple(Blist),Bytes3}. - fixextensions({ext,ExtPos,ExtNum},Val) -> case fixextensions(ExtPos,ExtNum+ExtPos,Val,0) of 0 -> []; @@ -165,11 +140,6 @@ getchoice(Bytes,NumChoices,0) -> decode_constrained_number(Bytes,{0,NumChoices-1}). -%%%%%%%%%%%%%%% -getoptionals2(Bytes,NumOpt) -> - getbits(Bytes,NumOpt). - - %% getbits_as_binary(Num,Bytes) -> {{Unused,BinBits},RestBytes}, %% Num = integer(), %% Bytes = list() | tuple(), @@ -289,33 +259,6 @@ decode_fragmented_bits(<<0:1,Len:7,BitStr/bitstring>>,C,Acc) -> exit({error,{asn1,{illegal_value,C,ResBitStr}}}) end. - -decode_fragmented_octets({0,Bin},C) -> - decode_fragmented_octets(Bin,C,[]). - -decode_fragmented_octets(<<3:2,Len:6,BitStr/bitstring>>,C,Acc) -> - FragLen = Len * ?'16K', - <<Value:FragLen/binary,Rest/bitstring>> = BitStr, - decode_fragmented_octets(Rest,C,[Value|Acc]); -decode_fragmented_octets(<<0:1,0:7,Bin/bitstring>>,C,Acc) -> - Octets = list_to_binary(lists:reverse(Acc)), - case C of - Int when is_integer(Int), C == size(Octets) -> - {Octets,Bin}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,Octets}}}) - end; -decode_fragmented_octets(<<0:1,Len:7,BitStr/bitstring>>,C,Acc) -> - <<Value:Len/binary-unit:8,BitStr2/binary>> = BitStr, - BinOctets = list_to_binary(lists:reverse([Value|Acc])), - case C of - Int when is_integer(Int),size(BinOctets) == Int -> - {BinOctets,BitStr2}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,BinOctets}}}) - end. - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% encode_open_type(Constraint, Value) -> CompleteList @@ -398,47 +341,6 @@ encode_integer1(C, Val) -> exit({error,{asn1,{illegal_value,VR,Val}}}) end. -decode_integer(Buffer,Range,NamedNumberList) -> - {Val,Buffer2} = decode_integer(Buffer,Range), - case lists:keysearch(Val,2,NamedNumberList) of - {value,{NewVal,_}} -> {NewVal,Buffer2}; - _ -> {Val,Buffer2} - end. - -decode_integer(Buffer,[{Rc,_Ec}]) when is_tuple(Rc) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> decode_integer(Buffer2,[Rc]); %% Value in root of constraint - 1 -> decode_unconstrained_number(Buffer2) - end; -decode_integer(Buffer,undefined) -> - decode_unconstrained_number(Buffer); -decode_integer(Buffer,C) -> - case get_constraint(C,'SingleValue') of - V when is_integer(V) -> - {V,Buffer}; - V when is_list(V) -> - {Val,Buffer2} = decode_integer1(Buffer,C), - case lists:member(Val,V) of - true -> - {Val,Buffer2}; - _ -> - exit({error,{asn1,{illegal_value,Val}}}) - end; - _ -> - decode_integer1(Buffer,C) - end. - -decode_integer1(Buffer,C) -> - case VR = get_constraint(C,'ValueRange') of - no -> - decode_unconstrained_number(Buffer); - {Lb, 'MAX'} -> - decode_semi_constrained_number(Buffer,Lb); - {_,_} -> - decode_constrained_number(Buffer,VR) - end. - %% X.691:10.6 Encoding of a normally small non-negative whole number %% Use this for encoding of CHOICE index if there is an extension marker in %% the CHOICE @@ -455,7 +357,7 @@ decode_small_number(Bytes) -> 0 -> getbits(Bytes2,6); 1 -> - decode_semi_constrained_number(Bytes2,0) + decode_semi_constrained_number(Bytes2) end. %% X.691:10.7 Encoding of a semi-constrained whole number @@ -479,12 +381,10 @@ encode_semi_constrained_number(Lb,Val) -> [encode_length(undefined,Size),Bin] end. -decode_semi_constrained_number(Bytes,{Lb,_}) -> - decode_semi_constrained_number(Bytes,Lb); -decode_semi_constrained_number(Bytes,Lb) -> +decode_semi_constrained_number(Bytes) -> {Len,Bytes2} = decode_length(Bytes,undefined), {V,Bytes3} = getoctets(Bytes2,Len), - {V+Lb,Bytes3}. + {V,Bytes3}. encode_constrained_number(Range,{Name,Val}) when is_atom(Name) -> encode_constrained_number(Range,Val); @@ -566,23 +466,6 @@ enint(-1, [B1|T]) when B1 > 127 -> enint(N, Acc) -> enint(N bsr 8, [N band 16#ff|Acc]). -decode_unconstrained_number(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Ints,Bytes3} = getoctets_as_bin(Bytes2,Len), - {dec_integer(Ints),Bytes3}. - -dec_integer(Bin = <<0:1,_:7,_/bitstring>>) -> - decpint(Bin); -dec_integer(<<_:1,B:7,BitStr/bitstring>>) -> - Size = bit_size(BitStr), - <<I:Size>> = BitStr, - (-128 + B) bsl bit_size(BitStr) bor I. - -decpint(Bin) -> - Size = bit_size(Bin), - <<Int:Size>> = Bin, - Int. - %% X.691:10.9 Encoding of a length determinant %%encode_small_length(undefined,Len) -> % null means no UpperBound @@ -597,7 +480,7 @@ encode_length(undefined,Len) -> % un-constrained Len < 16384 -> <<2:2,Len:14>>; true -> % should be able to endode length >= 16384 - exit({error,{asn1,{encode_length,{nyi,above_16k}}}}) + error({error,{asn1,{encode_length,{nyi,above_16k}}}}) end; encode_length({0,'MAX'},Len) -> @@ -623,18 +506,6 @@ encode_small_length(Len) -> [<<1:1>>,encode_length(undefined,Len)]. -decode_small_length(Buffer) -> - case getbit(Buffer) of - {0,Remain} -> - {Bits,Remain2} = getbits(Remain,6), - {Bits+1,Remain2}; - {1,Remain} -> - decode_length(Remain,undefined) - end. - -decode_length(Buffer) -> - decode_length(Buffer,undefined). - %% un-constrained decode_length(<<0:1,Oct:7,Rest/bitstring>>,undefined) -> {Oct,Rest}; @@ -677,38 +548,6 @@ encode_boolean({Name,Val}) when is_atom(Name) -> encode_boolean(Val) -> exit({error,{asn1,{encode_boolean,Val}}}). -decode_boolean(Buffer) -> %when record(Buffer,buffer) - case getbit(Buffer) of - {1,Remain} -> {true,Remain}; - {0,Remain} -> {false,Remain} - end. - - -%% ENUMERATED with extension marker -decode_enumerated(Buffer,C,{Ntup1,Ntup2}) when is_tuple(Ntup1), is_tuple(Ntup2) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> % not an extension value - {Val,Buffer3} = decode_integer(Buffer2,C), - case catch (element(Val+1,Ntup1)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,[Ntup1,Ntup2]}}}}) - end; - 1 -> % this an extension value - {Val,Buffer3} = decode_small_number(Buffer2), - case catch (element(Val+1,Ntup2)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _ -> {{asn1_enum,Val},Buffer3} - end - end; - -decode_enumerated(Buffer,C,NamedNumberTup) when is_tuple(NamedNumberTup) -> - {Val,Buffer2} = decode_integer(Buffer,C), - case catch (element(Val+1,NamedNumberTup)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer2}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,NamedNumberTup}}}}) - end. - %%============================================================================ %%============================================================================ @@ -1067,36 +906,71 @@ encode_octet_string(C,Val) -> list_to_binary(Val); 2 -> list_to_binary(Val); - Sv when Sv =<65535, Sv == length(Val) -> % fixed length - list_to_binary(Val); - VR = {_,_} -> - [encode_length(VR,length(Val)),list_to_binary(Val)]; + {_,_}=VR -> + try + [encode_length(VR, length(Val)),list_to_binary(Val)] + catch + error:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end; + Sv when is_integer(Sv), Sv =:= length(Val) -> % fixed length + if + Sv =< 65535 -> + list_to_binary(Val); + true -> + encode_fragmented_octet_string(Val) + end; Sv when is_list(Sv) -> - [encode_length({hd(Sv),lists:max(Sv)},length(Val)),list_to_binary(Val)]; + try + [encode_length({hd(Sv),lists:max(Sv)}, + length(Val)),list_to_binary(Val)] + catch + error:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end; no -> - [encode_length(undefined,length(Val)),list_to_binary(Val)] + try + [encode_length(undefined, length(Val)),list_to_binary(Val)] + catch + error:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end end. -decode_octet_string(Bytes,C) -> - decode_octet_string1(Bytes,get_constraint(C,'SizeConstraint')). -decode_octet_string1(<<B1,Bytes/bitstring>>,1) -> - {[B1],Bytes}; -decode_octet_string1(<<B1,B2,Bytes/bitstring>>,2) -> - {[B1,B2],Bytes}; -decode_octet_string1(Bytes,Sv) when is_integer(Sv),Sv=<65535 -> - getoctets_as_list(Bytes,Sv); -decode_octet_string1(Bytes,Sv) when is_integer(Sv) -> - decode_fragmented_octets(Bytes,Sv); -decode_octet_string1(Bytes,{Lb,Ub}) -> - {Len,Bytes2} = decode_length(Bytes,{Lb,Ub}), - getoctets_as_list(Bytes2,Len); -decode_octet_string1(Bytes,Sv) when is_list(Sv) -> - {Len,Bytes2} = decode_length(Bytes,{hd(Sv),lists:max(Sv)}), - getoctets_as_list(Bytes2,Len); -decode_octet_string1(Bytes,no) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - getoctets_as_list(Bytes2,Len). - +encode_fragmented_octet_string(Val) -> + Bin = list_to_binary(Val), + efos_1(Bin). + +efos_1(<<B:16#10000/binary,T/binary>>) -> + [<<3:2,4:6>>,B|efos_1(T)]; +efos_1(<<B:16#C000/binary,T/binary>>) -> + [<<3:2,3:6>>,B|efos_1(T)]; +efos_1(<<B:16#8000/binary,T/binary>>) -> + [<<3:2,2:6>>,B|efos_1(T)]; +efos_1(<<B:16#4000/binary,T/binary>>) -> + [<<3:2,1:6>>,B|efos_1(T)]; +efos_1(<<B/bitstring>>) -> + Len = byte_size(B), + [encode_length(undefined, Len),B]. + +decode_fragmented(SegSz0, Buf0, Unit) -> + SegSz = SegSz0 * Unit * ?'16K', + <<Res:SegSz/bitstring,Buf/bitstring>> = Buf0, + decode_fragmented_1(Buf, Unit, Res). + +decode_fragmented_1(<<0:1,N:7,Buf0/bitstring>>, Unit, Res) -> + Sz = N*Unit, + <<S:Sz/bitstring,Buf/bitstring>> = Buf0, + {<<Res/bitstring,S/bitstring>>,Buf}; +decode_fragmented_1(<<1:1,0:1,N:14,Buf0/bitstring>>, Unit, Res) -> + Sz = N*Unit, + <<S:Sz/bitstring,Buf/bitstring>> = Buf0, + {<<Res/bitstring,S/bitstring>>,Buf}; +decode_fragmented_1(<<1:1,1:1,SegSz0:6,Buf0/bitstring>>, Unit, Res0) -> + SegSz = SegSz0 * Unit * ?'16K', + <<Frag:SegSz/bitstring,Buf/bitstring>> = Buf0, + Res = <<Res0/bitstring,Frag/bitstring>>, + decode_fragmented_1(Buf, Unit, Res). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1441,12 +1315,6 @@ decode_UTF8String(Bytes) -> getoctets_as_bin(Bytes2,Len). - % X.691:17 -encode_null(_) -> []. % encodes to nothing - -decode_null(Bytes) -> - {'NULL',Bytes}. - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% encode_object_identifier(Val) -> CompleteList %% encode_object_identifier({Name,Val}) -> CompleteList diff --git a/lib/asn1/test/Makefile b/lib/asn1/test/Makefile index 6e6374baf1..1794d6bb71 100644 --- a/lib/asn1/test/Makefile +++ b/lib/asn1/test/Makefile @@ -83,6 +83,7 @@ MODULES= \ testInfObj \ testParameterizedInfObj \ testMergeCompile \ + testMultipleLevels \ testDeepTConstr \ testTimer \ testMegaco \ diff --git a/lib/asn1/test/asn1_SUITE.erl b/lib/asn1/test/asn1_SUITE.erl index 79c7bf1476..38ec72c473 100644 --- a/lib/asn1/test/asn1_SUITE.erl +++ b/lib/asn1/test/asn1_SUITE.erl @@ -21,27 +21,17 @@ -module(asn1_SUITE). -define(only_per(Func), - if Rule == per orelse Rule == per_bin -> Func; - true -> ok + if Rule =:= per -> Func; + true -> ok end). -define(only_ber(Func), - if Rule == ber orelse Rule == ber_bin orelse Rule == ber_bin_v2 -> Func; - true -> ok + if Rule =:= ber -> Func; + true -> ok end). -define(only_uper(Func), case Rule of - uper_bin -> Func; - _ -> ok - end). --define(only_per_nif(Func), - case {Rule, lists:member(optimize, Opts)} of - {per_bin, true} -> Func; - _ -> ok - end). --define(only_ber_nif(Func), - case {Rule, lists:member(nif, Opts)} of - {ber_bin_v2, true} -> Func; - _ -> ok + uper -> Func; + _ -> ok end). -compile(export_all). @@ -55,7 +45,8 @@ suite() -> [{ct_hooks, [ts_install_cth]}]. all() -> - [{group, parallel}, + [{group, compile}, + {group, parallel}, {group, app_test}, {group, appup_test}, @@ -76,16 +67,14 @@ groups() -> {ber, parallel([]), [ber_choiceinseq, % Uses 'SOpttest' - {group, [], [ber_optional, - ber_optional_keyed_list]}]}, + ber_optional]}, {app_test, [], [{asn1_app_test, all}]}, {appup_test, [], [{asn1_appup_test, all}]}, {parallel, parallel([]), - [{group, compile}, - {group, ber}, + [{group, ber}, % Uses 'P-Record', 'Constraints', 'MEDIA-GATEWAY-CONTROL'... {group, [], [parse, test_driver_load, @@ -116,6 +105,7 @@ groups() -> testChoTypeRefPrim, testChoTypeRefSeq, testChoTypeRefSet, + testMultipleLevels, testDef, testOpt, testSeqDefault, @@ -203,13 +193,8 @@ groups() -> {performance, [], [testTimer_ber, - testTimer_ber_bin, - testTimer_ber_bin_opt, - testTimer_ber_bin_opt_driver, testTimer_per, - testTimer_per_bin, - testTimer_per_bin_opt, - testTimer_uper_bin, + testTimer_uper, smp]}]. parallel(Options) -> @@ -257,14 +242,8 @@ end_per_testcase(_Func, Config) -> test(Config, TestF) -> test(Config, TestF, [per, - per_bin, - {per_bin, [optimize]}, - uper_bin, - ber, - ber_bin, - ber_bin_v2, - % TODO: {ber_bin_v2, [optimize, nif]} ? - {ber_bin_v2, [nif]}]). + uper, + ber]). test(Config, TestF, Rules) -> Fun = fun(C, R, O) -> @@ -344,10 +323,10 @@ testCompactBitString(Config, Rule, Opts) -> testCompactBitString:compact_bit_string(Rule), ?only_uper(testCompactBitString:bit_string_unnamed(Rule)), ?only_per(testCompactBitString:bit_string_unnamed(Rule)), - ?only_per_nif(testCompactBitString:ticket_7734(Rule)), - ?only_per_nif(asn1_test_lib:compile("Constraints", Config, + ?only_per(testCompactBitString:ticket_7734(Rule)), + ?only_per(asn1_test_lib:compile("Constraints", Config, [Rule, compact_bit_string|Opts])), - ?only_per_nif(testCompactBitString:otp_4869(Rule)). + ?only_per(testCompactBitString:otp_4869(Rule)). testPrimStrings(Config) -> test(Config, fun testPrimStrings/3). testPrimStrings(Config, Rule, Opts) -> @@ -371,10 +350,10 @@ testPrimExternal(Config, Rule, Opts) -> asn1_test_lib:compile_all(["External", "PrimExternal"], Config, [Rule|Opts]), testPrimExternal:external(Rule), - ?only_ber_nif(asn1_test_lib:compile_all(["PrimStrings", "BitStr"], Config, + ?only_ber(asn1_test_lib:compile_all(["PrimStrings", "BitStr"], Config, [Rule|Opts])), - ?only_ber_nif(testPrimStrings_cases(Rule)), - ?only_ber_nif(testPrimStrings:more_strings(Rule)). + ?only_ber(testPrimStrings_cases(Rule)), + ?only_ber(testPrimStrings:more_strings(Rule)). testChoPrim(Config) -> test(Config, fun testChoPrim/3). testChoPrim(Config, Rule, Opts) -> @@ -399,7 +378,7 @@ testChoOptional(Config, Rule, Opts) -> testChoOptionalImplicitTag(Config) -> test(Config, fun testChoOptionalImplicitTag/3, - [ber, ber_bin, ber_bin_v2]). + [ber]). testChoOptionalImplicitTag(Config, Rule, Opts) -> %% Only meaningful for ber & co asn1_test_lib:compile("ChoOptionalImplicitTag", Config, [Rule|Opts]), @@ -430,6 +409,11 @@ testChoTypeRefSet(Config, Rule, Opts) -> asn1_test_lib:compile("ChoTypeRefSet", Config, [Rule|Opts]), testChoTypeRefSet:set(Rule). +testMultipleLevels(Config) -> test(Config, fun testMultipleLevels/3). +testMultipleLevels(Config, Rule, Opts) -> + asn1_test_lib:compile("MultipleLevels", Config, [Rule|Opts]), + testMultipleLevels:main(Rule). + testDef(Config) -> test(Config, fun testDef/3). testDef(Config, Rule, Opts) -> asn1_test_lib:compile("Def", Config, [Rule|Opts]), @@ -515,8 +499,7 @@ testSeqOfCho(Config, Rule, Opts) -> testSeqOfCho:main(Rule). testSeqOfIndefinite(Config) -> - test(Config, fun testSeqOfIndefinite/3, - [ber, ber_bin, ber_bin_v2, {ber_bin_v2, [nif]}]). + test(Config, fun testSeqOfIndefinite/3, [ber]). testSeqOfIndefinite(Config, Rule, Opts) -> Files = ["Mvrasn-Constants-1", "Mvrasn-DataTypes-1", "Mvrasn-21-4", "Mvrasn-20-4", "Mvrasn-19-4", "Mvrasn-18-4", "Mvrasn-17-4", @@ -632,13 +615,12 @@ c_syntax(Config) -> "SeqBadComma"]]. c_string(Config) -> - test(Config, fun c_string/3, [per, per_bin, ber, ber_bin, ber_bin_v2]). + test(Config, fun c_string/3, [per, ber]). c_string(Config, Rule, Opts) -> asn1_test_lib:compile("String", Config, [Rule|Opts]). c_implicit_before_choice(Config) -> - test(Config, fun c_implicit_before_choice/3, - [ber, ber_bin, ber_bin_v2]). + test(Config, fun c_implicit_before_choice/3, [ber]). c_implicit_before_choice(Config, Rule, Opts) -> DataDir = ?config(data_dir, Config), CaseDir = ?config(case_dir, Config), @@ -649,12 +631,13 @@ parse(Config) -> [asn1_test_lib:compile(M, Config, [abs]) || M <- test_modules()]. per(Config) -> - test(Config, fun per/3, [per, per_bin, {per_bin, [optimize]}]). + test(Config, fun per/3, [per,uper]). per(Config, Rule, Opts) -> [module_test(M, Config, Rule, Opts) || M <- per_modules()]. ber_other(Config) -> - test(Config, fun ber_other/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun ber_other/3, [ber]). + ber_other(Config, Rule, Opts) -> [module_test(M, Config, Rule, Opts) || M <- ber_modules()]. @@ -669,12 +652,12 @@ module_test(M, Config, Rule, Opts) -> ber_choiceinseq(Config) -> - test(Config, fun ber_choiceinseq/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun ber_choiceinseq/3, [ber]). ber_choiceinseq(Config, Rule, Opts) -> asn1_test_lib:compile("ChoiceInSeq", Config, [Rule|Opts]). ber_optional(Config) -> - test(Config, fun ber_optional/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun ber_optional/3, [ber]). ber_optional(Config, Rule, Opts) -> asn1_test_lib:compile("SOpttest", Config, [Rule|Opts]), V = {'S', {'A', 10, asn1_NOVALUE, asn1_NOVALUE}, @@ -685,21 +668,6 @@ ber_optional(Config, Rule, Opts) -> V2 = asn1_wrapper:decode('SOpttest', 'S', Bytes), V = element(2, V2). -ber_optional_keyed_list(Config) -> - test(Config, fun ber_optional_keyed_list/3, [ber, ber_bin]). -ber_optional_keyed_list(Config, Rule, Opts) -> - asn1_test_lib:compile("SOpttest", Config, [Rule, keyed_list|Opts]), - Vrecord = {'S', {'A', 10, asn1_NOVALUE, asn1_NOVALUE}, - {'B', asn1_NOVALUE, asn1_NOVALUE, asn1_NOVALUE}, - {'C', asn1_NOVALUE, 111, asn1_NOVALUE}}, - V = [{a, [{scriptKey, 10}]}, - {b, []}, - {c, [{callingPartysCategory, 111}]}], - {ok, B} = asn1_wrapper:encode('SOpttest', 'S', V), - Bytes = lists:flatten(B), - V2 = asn1_wrapper:decode('SOpttest', 'S', Bytes), - Vrecord = element(2, V2). - %% records used by test-case default -record('Def1', {bool0, bool1 = asn1_DEFAULT, @@ -734,7 +702,7 @@ value_bad_enum_test(Config) -> end. constructed(Config) -> - test(Config, fun constructed/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun constructed/3, [ber]). constructed(Config, Rule, Opts) -> asn1_test_lib:compile("Constructed", Config, [Rule|Opts]), {ok, B} = asn1_wrapper:encode('Constructed', 'S', {'S', false}), @@ -745,7 +713,7 @@ constructed(Config, Rule, Opts) -> [136, 1, 10] = lists:flatten(B2). ber_decode_error(Config) -> - test(Config, fun ber_decode_error/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun ber_decode_error/3, [ber]). ber_decode_error(Config, Rule, Opts) -> asn1_test_lib:compile("Constructed", Config, [Rule|Opts]), ber_decode_error:run(Opts). @@ -758,14 +726,14 @@ h323test(Config, Rule, Opts) -> h323test:run(Rule). per_GeneralString(Config) -> - test(Config, fun per_GeneralString/3, [per, per_bin]). + test(Config, fun per_GeneralString/3, [per]). per_GeneralString(Config, Rule, Opts) -> asn1_test_lib:compile("MULTIMEDIA-SYSTEM-CONTROL", Config, [Rule|Opts]), UI = [109, 64, 1, 57], {ok, _V} = asn1_wrapper:decode('MULTIMEDIA-SYSTEM-CONTROL', 'MultimediaSystemControlMessage', UI). -per_open_type(Config) -> test(Config, fun per_open_type/3, [per, per_bin]). +per_open_type(Config) -> test(Config, fun per_open_type/3, [per]). per_open_type(Config, Rule, Opts) -> asn1_test_lib:compile("OpenType", Config, [Rule|Opts]), {ok, _} = asn1ct:test('OpenType', 'Ot', {'Stype', 10, true}). @@ -778,24 +746,24 @@ testConstraints(Config, Rule, Opts) -> testSeqIndefinite(Config) -> - test(Config, fun testSeqIndefinite/3, [ber, ber_bin, ber_bin_v2, - {ber_bin_v2, [nif]}]). + test(Config, fun testSeqIndefinite/3, [ber]). + testSeqIndefinite(Config, Rule, Opts) -> asn1_test_lib:compile("SeqSetIndefinite", Config, [Rule|Opts]), testSeqIndefinite:main(Rule). testSetIndefinite(Config) -> - test(Config, fun testSetIndefinite/3, [ber, ber_bin, ber_bin_v2, - {ber_bin_v2, [nif]}]). + test(Config, fun testSetIndefinite/3, [ber]). + testSetIndefinite(Config, Rule, Opts) -> asn1_test_lib:compile("SeqSetIndefinite", Config, [Rule|Opts]), testSetIndefinite:main(Rule). testChoiceIndefinite(Config) -> - test(Config, fun testChoiceIndefinite/3, [ber, ber_bin, ber_bin_v2, - {ber_bin_v2, [nif]}]). + test(Config, fun testChoiceIndefinite/3, [ber]). + testChoiceIndefinite(Config, Rule, Opts) -> asn1_test_lib:compile("ChoiceIndef", Config, [Rule|Opts]), testChoiceIndefinite:main(Rule). @@ -857,7 +825,7 @@ testExport(Config) -> end. testImport(Config) -> - test(Config, fun testImport/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun testImport/3, [ber]). testImport(Config, Rule, Opts) -> {error, _} = asn1ct:compile(filename:join(?config(data_dir, Config), "ImportsFrom"), @@ -906,8 +874,7 @@ duplicate_tags(Config) -> {skip, "Runs in asn1_SUITE only"} end. -rtUI(Config) -> test(Config, fun rtUI/3, [per, per_bin, ber, - ber_bin, ber_bin_v2]). +rtUI(Config) -> test(Config, fun rtUI/3, [per,ber]). rtUI(Config, Rule, Opts) -> asn1_test_lib:compile("Prim", Config, [Rule|Opts]), {ok, _} = asn1rt:info('Prim'). @@ -923,19 +890,19 @@ testINSTANCE_OF(Config, Rule, Opts) -> testINSTANCE_OF:main(Rule). testTCAP(Config) -> - test(Config, fun testTCAP/3, - [ber, ber_bin, ber_bin_v2, {ber_bin_v2, [nif]}]). + test(Config, fun testTCAP/3, [ber]). testTCAP(Config, Rule, Opts) -> testTCAP:compile(Config, [Rule|Opts]), testTCAP:test(Rule, Config), case Rule of - ber_bin_v2 -> testTCAP:compile_asn1config(Config, [Rule, asn1config]), - testTCAP:test_asn1config(); - _ -> ok + ber -> + testTCAP:compile_asn1config(Config, [Rule, asn1config]), + testTCAP:test_asn1config(); + _ -> ok end. testDER(Config) -> - test(Config, fun testDER/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun testDER/3, [ber]). testDER(Config, Rule, Opts) -> asn1_test_lib:compile("DERSpec", Config, [Rule, der|Opts]), testDER:test(), @@ -945,7 +912,7 @@ testDER(Config, Rule, Opts) -> testSeqSetDefaultVal:main(Rule). specialized_decodes(Config) -> - test(Config, fun specialized_decodes/3, [ber_bin_v2]). + test(Config, fun specialized_decodes/3, [ber]). specialized_decodes(Config, Rule, Opts) -> asn1_test_lib:compile_all(["PartialDecSeq.asn", "PartialDecSeq2.asn", @@ -953,13 +920,12 @@ specialized_decodes(Config, Rule, Opts) -> "PartialDecMyHTTP.asn", "MEDIA-GATEWAY-CONTROL.asn", "P-Record"], - Config, [Rule, optimize, asn1config|Opts]), + Config, [Rule, asn1config|Opts]), test_partial_incomplete_decode:test(Config), test_selective_decode:test(). special_decode_performance(Config) -> - test(Config, fun special_decode_performance/3, - [{ber_bin, [optimize]}, {ber_bin_v2, [optimize, nif]}]). + test(Config, fun special_decode_performance/3, [ber]). special_decode_performance(Config, Rule, Opts) -> Files = ["MEDIA-GATEWAY-CONTROL", "PartialDecSeq"], asn1_test_lib:compile_all(Files, Config, [Rule, asn1config|Opts]), @@ -967,19 +933,19 @@ special_decode_performance(Config, Rule, Opts) -> test_driver_load(Config) -> - test(Config, fun test_driver_load/3, [{per_bin, [optimize]}]). + test(Config, fun test_driver_load/3, [per]). test_driver_load(Config, Rule, Opts) -> asn1_test_lib:compile("P-Record", Config, [Rule|Opts]), test_driver_load:test(5). test_ParamTypeInfObj(Config) -> - asn1_test_lib:compile("IN-CS-1-Datatypes", Config, [ber_bin]). + asn1_test_lib:compile("IN-CS-1-Datatypes", Config, [ber]). test_WS_ParamClass(Config) -> - asn1_test_lib:compile("InformationFramework", Config, [ber_bin]). + asn1_test_lib:compile("InformationFramework", Config, [ber]). test_Defed_ObjectIdentifier(Config) -> - asn1_test_lib:compile("UsefulDefinitions", Config, [ber_bin]). + asn1_test_lib:compile("UsefulDefinitions", Config, [ber]). testSelectionType(Config) -> test(Config, fun testSelectionType/3). testSelectionType(Config, Rule, Opts) -> @@ -987,7 +953,7 @@ testSelectionType(Config, Rule, Opts) -> {ok, _} = testSelectionTypes:test(). testSSLspecs(Config) -> - test(Config, fun testSSLspecs/3, [ber, ber_bin, ber_bin_v2, {ber_bin_v2, [optimize]}]). + test(Config, fun testSSLspecs/3, [ber]). testSSLspecs(Config, Rule, Opts) -> ok = testSSLspecs:compile(Config, [Rule, compact_bit_string, der|Opts]), @@ -1011,12 +977,12 @@ test_undecoded_rest(Config, Rule, Opts) -> ok = test_undecoded_rest:test([], Config), asn1_test_lib:compile("P-Record", Config, [Rule,undec_rest|Opts]), case Rule of - ber_bin_v2 -> ok; + ber -> ok; _ -> test_undecoded_rest:test(undec_rest, Config) end. test_inline(Config) -> - test(Config, fun test_inline/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun test_inline/3, [ber]). test_inline(Config, Rule, Opts) -> case code:which(asn1ct) of cover_compiled -> @@ -1029,12 +995,11 @@ test_inline(Config, Rule, Opts) -> end. testTcapsystem(Config) -> - test(Config, fun testTcapsystem/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun testTcapsystem/3, [ber]). testTcapsystem(Config, Rule, Opts) -> testTcapsystem:compile(Config, [Rule|Opts]). -testNBAPsystem(Config) -> test(Config, fun testNBAPsystem/3, - [per, per_bin, {per_bin, [optimize]}]). +testNBAPsystem(Config) -> test(Config, fun testNBAPsystem/3, [per]). testNBAPsystem(Config, Rule, Opts) -> testNBAPsystem:compile(Config, [Rule|Opts]), testNBAPsystem:test(Rule, Config). @@ -1068,20 +1033,19 @@ test_modified_x420(Config) -> testX420() -> [{timetrap,{minutes,90}}]. testX420(Config) -> - test(Config, fun testX420/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun testX420/3, [ber]). testX420(Config, Rule, Opts) -> testX420:compile(Rule, [der|Opts], Config), ok = testX420:ticket7759(Rule, Config), testX420:compile(Rule, Opts, Config). test_x691(Config) -> - test(Config, fun test_x691/3, - [per, per_bin, uper_bin, {per_bin, [optimize]}]). + test(Config, fun test_x691/3, [per, uper]). test_x691(Config, Rule, Opts) -> Files = ["P-RecordA1", "P-RecordA2", "P-RecordA3"], asn1_test_lib:compile_all(Files, Config, [Rule|Opts]), test_x691:cases(Rule, case Rule of - uper_bin -> unaligned; + uper -> unaligned; _ -> aligned end), asn1_test_lib:ticket_7708(Config, []), @@ -1092,8 +1056,7 @@ ticket_6143(Config) -> testExtensionAdditionGroup(Config) -> %% FIXME problems with automatic tags [ber_bin], [ber_bin, optimize] - test(Config, fun testExtensionAdditionGroup/3, - [per_bin, {per_bin, [optimize]}, uper_bin]). + test(Config, fun testExtensionAdditionGroup/3, [per, uper]). testExtensionAdditionGroup(Config, Rule, Opts) -> asn1_test_lib:compile("Extension-Addition-Group", Config, [Rule|Opts]), asn1_test_lib:compile_erlang("extensionAdditionGroup", Config, @@ -1182,46 +1145,17 @@ timer_compile(Config, Rule, Opts) -> asn1_test_lib:compile_all(["H235-SECURITY-MESSAGES", "H323-MESSAGES"], Config, [Rule|Opts]). -testTimer_ber(suite) -> []; testTimer_ber(Config) -> timer_compile(Config,ber,[]), testTimer:go(Config,ber). -testTimer_ber_bin(suite) -> []; -testTimer_ber_bin(Config) -> - timer_compile(Config,ber_bin,[]), - testTimer:go(Config,ber_bin). - -testTimer_ber_bin_opt(suite) -> []; -testTimer_ber_bin_opt(Config) -> - timer_compile(Config,ber_bin,[optimize]), - testTimer:go(Config,ber_bin). - -testTimer_ber_bin_opt_driver(suite) -> []; -testTimer_ber_bin_opt_driver(Config) -> - timer_compile(Config,ber_bin,[optimize,driver]), - testTimer:go(Config,ber_bin). - -testTimer_per(suite) -> []; testTimer_per(Config) -> timer_compile(Config,per,[]), testTimer:go(Config,per). -testTimer_per_bin(suite) -> []; -testTimer_per_bin(Config) -> - timer_compile(Config,per_bin,[]), - testTimer:go(Config,per_bin). - -testTimer_per_bin_opt(suite) -> []; -testTimer_per_bin_opt(Config) -> - timer_compile(Config,per_bin,[optimize]), - testTimer:go(Config,per_bin). - - -testTimer_uper_bin(suite) -> []; -testTimer_uper_bin(Config) -> - timer_compile(Config,uper_bin,[]), - {comment,_} = testTimer:go(Config,uper_bin). +testTimer_uper(Config) -> + timer_compile(Config,uper,[]), + {comment,_} = testTimer:go(Config,uper). %% Test of multiple-line comment, OTP-8043 testComment(suite) -> []; @@ -1264,11 +1198,11 @@ testName2Number(Config) -> ok. ticket_7407(Config) -> - asn1_test_lib:compile("EUTRA-extract-7407", Config, [uper_bin]), + asn1_test_lib:compile("EUTRA-extract-7407", Config, [uper]), asn1_test_lib:ticket_7407_code(true), asn1_test_lib:compile("EUTRA-extract-7407", Config, - [uper_bin, no_final_padding]), + [uper, no_final_padding]), asn1_test_lib:ticket_7407_code(false). smp(suite) -> []; @@ -1279,7 +1213,7 @@ smp(Config) -> io:format("smp starting ~p workers\n",[NumOfProcs]), Msg = {initiatingMessage, testNBAPsystem:cell_setup_req_msg()}, - ok = testNBAPsystem:compile(Config, [per_bin, optimize]), + ok = testNBAPsystem:compile(Config, [per]), enc_dec(NumOfProcs,Msg,2), @@ -1288,7 +1222,7 @@ smp(Config) -> {Time1,ok} = timer:tc(?MODULE,enc_dec,[NumOfProcs,Msg, N]), {Time1S,ok} = timer:tc(?MODULE,enc_dec,[1, Msg, NumOfProcs * N]), - ok = testNBAPsystem:compile(Config, [ber_bin, optimize, nif]), + ok = testNBAPsystem:compile(Config, [ber]), {Time3,ok} = timer:tc(?MODULE,enc_dec,[NumOfProcs,Msg, N]), {Time3S,ok} = timer:tc(?MODULE,enc_dec,[1, Msg, NumOfProcs * N]), @@ -1309,10 +1243,8 @@ per_performance(Config) -> file:make_dir(NifDir),file:make_dir(ErlDir), Msg = {initiatingMessage, testNBAPsystem:cell_setup_req_msg()}, - ok = testNBAPsystem:compile([{priv_dir,NifDir}|Config], - [per_bin, optimize]), - ok = testNBAPsystem:compile([{priv_dir,ErlDir}|Config], - [per_bin]), + ok = testNBAPsystem:compile([{priv_dir,NifDir}|Config], [per]), + ok = testNBAPsystem:compile([{priv_dir,ErlDir}|Config], [per]), Modules = ['NBAP-CommonDataTypes', 'NBAP-Constants', @@ -1350,7 +1282,7 @@ per_performance(Config) -> ber_performance(Config) -> Msg = {initiatingMessage, testNBAPsystem:cell_setup_req_msg()}, - ok = testNBAPsystem:compile(Config, [ber_bin, optimize, nif]), + ok = testNBAPsystem:compile(Config, [ber]), BerFun = fun() -> @@ -1524,7 +1456,7 @@ pforeach(_Fun,[],[]) -> -record('Iu-ReleaseCommand',{first,second}). ticket7904(Config) -> - asn1_test_lib:compile("RANAPextract1", Config, [per_bin, optimize]), + asn1_test_lib:compile("RANAPextract1", Config, [per]), Val1 = #'InitiatingMessage'{procedureCode=1, criticality=ignore, diff --git a/lib/asn1/test/asn1_SUITE_data/MultipleLevels.asn b/lib/asn1/test/asn1_SUITE_data/MultipleLevels.asn new file mode 100644 index 0000000000..1824e1fa5a --- /dev/null +++ b/lib/asn1/test/asn1_SUITE_data/MultipleLevels.asn @@ -0,0 +1,19 @@ +MultipleLevels DEFINITIONS AUTOMATIC TAGS ::= + +BEGIN + +Top ::= SEQUENCE { + country CountryName, + region Name +} + +CountryName ::= CountryName0 + +CountryName0 ::= Name + +Name ::= CHOICE { + short PrintableString (SIZE (0..10)), + long PrintableString +} + +END diff --git a/lib/asn1/test/asn1_SUITE_data/PartialDecSeq.asn1config b/lib/asn1/test/asn1_SUITE_data/PartialDecSeq.asn1config index 19fa3c990e..d388f6cd02 100644 --- a/lib/asn1/test/asn1_SUITE_data/PartialDecSeq.asn1config +++ b/lib/asn1/test/asn1_SUITE_data/PartialDecSeq.asn1config @@ -17,6 +17,6 @@ {decode_D_incomplete,['D',[{a,undecoded}]]}, {decode_F_fb_exclusive2,['F',[{fb,[{b,parts},{d,[{da,parts}]}]}]]}, {decode_F_fb_exclusive3,['F',[{fb,[{b,parts},{d,[{da,parts},{dc,[{dcc,undecoded}]}]}]}]]}]}}. {module_name,'Seq.asn1'}. -{compile_options,[ber_bin,optimize,debug_info]}. +{compile_options,[ber,debug_info]}. {multifile_compile,['M1.asn','M2.asn']}. diff --git a/lib/asn1/test/asn1_SUITE_data/PrimStrings.asn1 b/lib/asn1/test/asn1_SUITE_data/PrimStrings.asn1 index d287840f30..7cb47e9792 100644 --- a/lib/asn1/test/asn1_SUITE_data/PrimStrings.asn1 +++ b/lib/asn1/test/asn1_SUITE_data/PrimStrings.asn1 @@ -55,6 +55,19 @@ BS1024 ::= BIT STRING (SIZE (1024)) OsExpCon ::= [60] EXPLICIT OCTET STRING OsExpPri ::= [PRIVATE 61] EXPLICIT OCTET STRING OsExpApp ::= [APPLICATION 62] EXPLICIT OCTET STRING + OsFrag ::= OCTET STRING (SIZE (0..100000)) + FixedOs65536 ::= OCTET STRING (SIZE (65536)) + FixedOs65537 ::= OCTET STRING (SIZE (65537)) + + OsAlignment ::= SEQUENCE { + b1 BOOLEAN, + s1 Os, + b2 BOOLEAN, + s2 OsFrag, + b3 BOOLEAN, + s3 FixedOs65536, + i INTEGER (0..63) + } Ns ::= NumericString NsCon ::= [70] NumericString diff --git a/lib/asn1/test/asn1_SUITE_data/testobj.erl b/lib/asn1/test/asn1_SUITE_data/testobj.erl index be7ceee7d1..80942f7e38 100644 --- a/lib/asn1/test/asn1_SUITE_data/testobj.erl +++ b/lib/asn1/test/asn1_SUITE_data/testobj.erl @@ -1420,24 +1420,7 @@ wrapper_encode(Module,Type,Value) -> Error end. -wrapper_decode(Module,Type,Bytes) -> - case Module:encoding_rule() of - ber -> - asn1rt:decode(Module,Type,Bytes); - ber_bin when binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - ber_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - ber_bin_v2 when binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - ber_bin_v2 -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - per -> - asn1rt:decode(Module,Type,Bytes); - per_bin when binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - per_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - uper_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)) - end. +wrapper_decode(Module, Type, Bytes) when is_binary(Bytes) -> + asn1rt:decode(Module, Type, Bytes); +wrapper_decode(Module, Type, Bytes) when is_list(Bytes) -> + asn1rt:decode(Module, Type, list_to_binary(Bytes)). diff --git a/lib/asn1/test/asn1_app_test.erl b/lib/asn1/test/asn1_app_test.erl index 2c31c3259d..9dbe1b50e0 100644 --- a/lib/asn1/test/asn1_app_test.erl +++ b/lib/asn1/test/asn1_app_test.erl @@ -138,7 +138,8 @@ check_asn1ct_modules(Extra) -> asn1ct_name,asn1ct_constructed_per,asn1ct_constructed_ber, asn1ct_gen_ber,asn1ct_constructed_ber_bin_v2, asn1ct_gen_ber_bin_v2,asn1ct_value, - asn1ct_tok,asn1ct_parser2,asn1ct_table], + asn1ct_tok,asn1ct_parser2,asn1ct_table, + asn1ct_imm], case Extra -- ASN1CTMods of [] -> ok; diff --git a/lib/asn1/test/asn1_test_lib.erl b/lib/asn1/test/asn1_test_lib.erl index 96c04a9436..fda635d0eb 100644 --- a/lib/asn1/test/asn1_test_lib.erl +++ b/lib/asn1/test/asn1_test_lib.erl @@ -87,14 +87,14 @@ ticket_7407_compile(Config,Option) -> ?line OutDir = ?config(priv_dir,Config), ?line ok = asn1ct:compile(DataDir ++ "EUTRA-extract-7407", - [uper_bin, {outdir,OutDir}]++Option). + [uper, {outdir,OutDir}]++Option). ticket_7708(Config,Option) -> ?line DataDir = ?config(data_dir,Config), ?line OutDir = ?config(priv_dir,Config), ?line ok = asn1ct:compile(DataDir ++ "EUTRA-extract-55", - [uper_bin, {outdir,OutDir}]++Option). + [uper, {outdir,OutDir}]++Option). ticket_7407_code(FinalPadding) -> @@ -154,7 +154,7 @@ ticket_7678(Config, Option) -> ?line OutDir = ?config(priv_dir,Config), ?line ok = asn1ct:compile(DataDir ++ "UPERDefault", - [uper_bin, {outdir,OutDir}]++Option), + [uper, {outdir,OutDir}]++Option), ?line Val = 'UPERDefault':seq(), ?line {ok,<<0,6,0>>} = 'UPERDefault':encode('Seq',Val), @@ -167,12 +167,12 @@ ticket_7763(Config) -> ?line OutDir = ?config(priv_dir,Config), ?line ok = asn1ct:compile(DataDir ++ "EUTRA-extract-55", - [uper_bin, {outdir,OutDir}]), + [uper, {outdir,OutDir}]), Val = {'Seq',15,lists:duplicate(8,0),[0],lists:duplicate(28,0),15,true}, ?line {ok,Bin} = 'EUTRA-extract-55':encode('Seq',Val), ?line ok = asn1ct:compile(DataDir ++ "EUTRA-extract-55", - [uper_bin,compact_bit_string,{outdir,OutDir}]), + [uper,compact_bit_string,{outdir,OutDir}]), CompactVal = {'Seq',15,{0,<<0>>},{7,<<0>>},{4,<<0,0,0,0>>},15,true}, {ok,CompactBin} = 'EUTRA-extract-55':encode('Seq',CompactVal), diff --git a/lib/asn1/test/asn1_wrapper.erl b/lib/asn1/test/asn1_wrapper.erl index d515b99ac2..e764d8b4ca 100644 --- a/lib/asn1/test/asn1_wrapper.erl +++ b/lib/asn1/test/asn1_wrapper.erl @@ -34,41 +34,16 @@ encode(Module,Type,Value) -> Error end. -decode(Module,Type,Bytes) -> - case Module:encoding_rule() of - ber -> - asn1rt:decode(Module,Type,Bytes); - ber_bin when is_binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - ber_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - ber_bin_v2 when is_binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - ber_bin_v2 -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - per -> - asn1rt:decode(Module,Type,Bytes); - per_bin when is_binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - per_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - uper_bin when is_binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - uper_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)) - end. +decode(Module, Type, Bytes) when is_binary(Bytes) -> + asn1rt:decode(Module, Type, Bytes); +decode(Module, Type, Bytes) when is_list(Bytes) -> + asn1rt:decode(Module, Type, list_to_binary(Bytes)). erule(ber) -> ber; -erule(ber_bin) -> - ber; -erule(ber_bin_v2) -> - ber; erule(per) -> per; -erule(per_bin) -> - per; -erule(uper_bin) -> +erule(uper) -> per. diff --git a/lib/asn1/test/h323test.erl b/lib/asn1/test/h323test.erl index b7a7d6e4df..426ae16994 100644 --- a/lib/asn1/test/h323test.erl +++ b/lib/asn1/test/h323test.erl @@ -22,7 +22,6 @@ -export([run/1]). -include_lib("test_server/include/test_server.hrl"). -run(per_bin) -> run(); run(per) -> run(); run(_Rules) -> ok. diff --git a/lib/asn1/test/testChoiceIndefinite.erl b/lib/asn1/test/testChoiceIndefinite.erl index 630efcf27a..b5832c985a 100644 --- a/lib/asn1/test/testChoiceIndefinite.erl +++ b/lib/asn1/test/testChoiceIndefinite.erl @@ -23,12 +23,7 @@ -include_lib("test_server/include/test_server.hrl"). -main(per_bin) -> ok; main(per) -> ok; -main(ber_bin_v2) -> - main(ber); -main(ber_bin) -> - main(ber); main(ber) -> %% Test case related to OTP-4358 %% normal encoding diff --git a/lib/asn1/test/testCompactBitString.erl b/lib/asn1/test/testCompactBitString.erl index 9563a31bf3..db102a3bda 100644 --- a/lib/asn1/test/testCompactBitString.erl +++ b/lib/asn1/test/testCompactBitString.erl @@ -233,7 +233,7 @@ compact_bit_string(Rules) -> ok. -ticket_7734(per_bin) -> +ticket_7734(per) -> ?line BS = {0,list_to_binary(lists:duplicate(128,0))}, ?line {ok,BSEnc} = asn1_wrapper:encode('PrimStrings','BS1024',BS), ?line {ok,BS} = asn1_wrapper:decode('PrimStrings','BS1024',BSEnc). @@ -251,7 +251,7 @@ bit_string_unnamed(Rules) -> lists:flatten(Bytes1)) end. -otp_4869(per_bin) -> +otp_4869(per) -> ?line Val1={'IP',[0],{0,<<62,235,90,50,0,0,0,0,0,0,0,0,0,0,0,0>>},asn1_NOVALUE}, ?line Val2 = {'IP',[0],[0,0,1,1,1,1,1,0,1,1,1,0,1,0,1,1,0,1,0,1,1,0,1,0,0,0,1,1,0,0,1,0] ++ lists:duplicate(128 - 32,0),asn1_NOVALUE}, diff --git a/lib/asn1/test/testEnumExt.erl b/lib/asn1/test/testEnumExt.erl index c97116413a..bb975a1d13 100644 --- a/lib/asn1/test/testEnumExt.erl +++ b/lib/asn1/test/testEnumExt.erl @@ -23,8 +23,8 @@ -include_lib("test_server/include/test_server.hrl"). -main(Rules) when Rules == per; Rules == per_bin; Rules == uper_bin -> - io:format("main(~p)~n",[Rules]), +main(Rule) when Rule =:= per; Rule =:= uper -> + io:format("main(~p)~n",[Rule]), B32=[32],B64=[64], %% ENUMERATED with extensionmark (value is in root set) ?line {ok,B32} = asn1_wrapper:encode('EnumExt','Ext',red), @@ -42,10 +42,6 @@ main(Rules) when Rules == per; Rules == per_bin; Rules == uper_bin -> ?line {ok,red} = asn1_wrapper:decode('EnumExt','Noext',B64), ok; -main(ber_bin_v2) -> - main(ber); -main(ber_bin) -> - main(ber); main(ber) -> io:format("main(ber)~n",[]), %% ENUMERATED with extensionmark (value is in root set) diff --git a/lib/asn1/test/testINSTANCE_OF.erl b/lib/asn1/test/testINSTANCE_OF.erl index 5986a00ec5..ce411beb92 100644 --- a/lib/asn1/test/testINSTANCE_OF.erl +++ b/lib/asn1/test/testINSTANCE_OF.erl @@ -26,7 +26,7 @@ main(Erule) -> ?line {ok,Integer} = asn1_wrapper:encode('INSTANCEOF','Int',3), - Int = wrap(Erule,Integer), + Int = list_to_binary(Integer), ValotherName = {otherName,{'INSTANCE OF',{2,4},Int}}, VallastName1 = {lastName,{'GeneralName_lastName',{2,4},12}}, VallastName2 = {lastName,{'GeneralName_lastName',{2,3,4}, @@ -61,18 +61,3 @@ test_encdec(_Erule,{lastName,{'GeneralName_lastName',{2,3,4}, ok; test_encdec(Erule,Res) -> {error,{Erule,Res}}. - -wrap(ber,Int) when is_list(Int) -> - binary_to_list(list_to_binary(Int)); -wrap(per,Int) when is_list(Int) -> - binary_to_list(list_to_binary(Int)); -wrap(ber_bin,Int) when is_list(Int) -> - list_to_binary(Int); -wrap(ber_bin_v2,Int) when is_list(Int) -> - list_to_binary(Int); -wrap(per_bin,Int) when is_list(Int) -> - list_to_binary(Int); -wrap(uper_bin,Int) when is_list(Int) -> - list_to_binary(Int); -wrap(_,Int) -> - Int. diff --git a/lib/asn1/test/testInfObjectClass.erl b/lib/asn1/test/testInfObjectClass.erl index e639066246..98408502c6 100644 --- a/lib/asn1/test/testInfObjectClass.erl +++ b/lib/asn1/test/testInfObjectClass.erl @@ -37,15 +37,12 @@ main(Rule) -> {component,'ArgumentType'}, {value,_},_}}} = asn1_wrapper:encode('InfClass','Seq', {'Seq',12,13,1}), - Bytes2 = - if - Rule==per;Rule==per_bin -> - [1,12,1,11,1,1]; - Rule == uper_bin -> - <<1,12,1,11,1,1>>; - true -> - [48,9,2,1,12,2,1,11,2,1,1] - end, + Bytes2 = case Rule of + ber -> + <<48,9,2,1,12,2,1,11,2,1,1>>; + _ -> + <<1,12,1,11,1,1>> + end, ?line {error,{asn1,{'Type not compatible with table constraint', {{component,_}, {value,_B},_}}}} = diff --git a/lib/asn1/test/testMergeCompile.erl b/lib/asn1/test/testMergeCompile.erl index 31aa3518f6..d63df28c31 100644 --- a/lib/asn1/test/testMergeCompile.erl +++ b/lib/asn1/test/testMergeCompile.erl @@ -43,17 +43,11 @@ main(Erule) -> ?line EncVal = case Erule of per -> - [1,100]; - per_bin -> <<1,100>>; - uper_bin -> + uper -> <<1,100>>; ber -> - [2,1,1]; - ber_bin -> - <<2,1,1>>; - ber_bin_v2 -> - <<2,1,1>> + [2,1,1] end, ?line PEVal2 = [{dummy,1,ignore,EncVal},{dummy,2,reject,EncVal}], ?line Val2 = @@ -76,7 +70,7 @@ main(Erule) -> mvrasn(Erule) -> case Erule of - Ber when Ber == ber;Ber == ber_bin -> + ber -> ?line ok = test(isd), ?line ok = test(isd2), ?line ok = test(dsd), diff --git a/lib/asn1/test/testMultipleLevels.erl b/lib/asn1/test/testMultipleLevels.erl new file mode 100644 index 0000000000..ff6d023440 --- /dev/null +++ b/lib/asn1/test/testMultipleLevels.erl @@ -0,0 +1,27 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(testMultipleLevels). +-export([main/1]). + +main(_) -> + Data = {'Top',{short,"abc"},{long,"a long string follows here"}}, + {ok,B} = 'MultipleLevels':encode('Top', Data), + {ok,Data} = 'MultipleLevels':decode('Top', iolist_to_binary(B)). diff --git a/lib/asn1/test/testParameterizedInfObj.erl b/lib/asn1/test/testParameterizedInfObj.erl index 68faf08a61..6f53595132 100644 --- a/lib/asn1/test/testParameterizedInfObj.erl +++ b/lib/asn1/test/testParameterizedInfObj.erl @@ -98,7 +98,7 @@ ranap(_Erule) -> ok. -open_type(uper_bin,Val) when is_list(Val) -> +open_type(uper,Val) when is_list(Val) -> list_to_binary(Val); open_type(_,Val) -> Val. diff --git a/lib/asn1/test/testPrimStrings.erl b/lib/asn1/test/testPrimStrings.erl index b1c5172b95..9a640a2cb1 100644 --- a/lib/asn1/test/testPrimStrings.erl +++ b/lib/asn1/test/testPrimStrings.erl @@ -338,69 +338,24 @@ octet_string(Rules) -> ok end, + roundtrip('Os', [47,23,99,255,1]), + roundtrip('OsCon', [47,23,99,255,1]), + roundtrip('OsPri', [47,23,99,255,1]), + roundtrip('OsApp', [47,23,99,255,1]), - - ?line {ok,Bytes4} = - asn1_wrapper:encode('PrimStrings','Os',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','Os',lists:flatten(Bytes4)), - - ?line {ok,Bytes5} = - asn1_wrapper:encode('PrimStrings','OsCon',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsCon',lists:flatten(Bytes5)), - - ?line {ok,Bytes6} = - asn1_wrapper:encode('PrimStrings','OsPri',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsPri',lists:flatten(Bytes6)), - - ?line {ok,Bytes7} = - asn1_wrapper:encode('PrimStrings','OsApp',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsApp',lists:flatten(Bytes7)), - - ?line {ok,Bytes8} = - asn1_wrapper:encode('PrimStrings','OsExpCon',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsExpCon',lists:flatten(Bytes8)), - - ?line {ok,Bytes9} = - asn1_wrapper:encode('PrimStrings','OsExpPri',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsExpPri',lists:flatten(Bytes9)), - - ?line {ok,Bytes10} = - asn1_wrapper:encode('PrimStrings','OsExpApp',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsExpApp',lists:flatten(Bytes10)), - - ?line {ok,Bytes11} = - asn1_wrapper:encode('PrimStrings','Os',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Os',lists:flatten(Bytes11)), - - ?line {ok,Bytes12} = - asn1_wrapper:encode('PrimStrings','OsApp',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','OsApp',lists:flatten(Bytes12)), - - ?line {ok,Bytes13} = - asn1_wrapper:encode('PrimStrings','OsExpApp',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','OsExpApp',lists:flatten(Bytes13)), - - - - - + roundtrip('OsExpCon', [47,23,99,255,1]), + roundtrip('OsExpPri', [47,23,99,255,1]), + roundtrip('OsExpApp', [47,23,99,255,1]), + roundtrip('Os', []), + roundtrip('OsApp', []), + roundtrip('OsExpApp',[]), OsR = "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890", - ?line {ok,Bytes21} = - asn1_wrapper:encode('PrimStrings','Os',OsR), - ?line {ok,Os1} = asn1_wrapper:decode('PrimStrings','Os',lists:flatten(Bytes21)), - ?line Os1 = OsR, - ?line {ok,Bytes22} = - asn1_wrapper:encode('PrimStrings','OsCon',OsR), - ?line {ok,Os2} = asn1_wrapper:decode('PrimStrings','OsCon',lists:flatten(Bytes22)), - ?line Os2 = OsR, - ?line {ok,Bytes23} = - asn1_wrapper:encode('PrimStrings','OsExpApp',OsR), - ?line {ok,Os3} = asn1_wrapper:decode('PrimStrings','OsExpApp',lists:flatten(Bytes23)), - ?line Os3 = OsR, - + roundtrip('Os', OsR), + roundtrip('OsCon', OsR), + roundtrip('OsExpApp', OsR), ?line case asn1_wrapper:erule(Rules) of @@ -416,21 +371,85 @@ octet_string(Rules) -> ok end, - + fragmented_octet_string(Rules), ok. +fragmented_octet_string(Erules) -> + K16 = 1 bsl 14, + K32 = K16 + K16, + K48 = K32 + K16, + K64 = K48 + K16, + Lens = [0,1,14,15,16,17,127,128, + K16-1,K16,K16+1,K16+(1 bsl 7)-1,K16+(1 bsl 7),K16+(1 bsl 7)+1, + K32-1,K32,K32+1,K32+(1 bsl 7)-1,K32+(1 bsl 7),K32+(1 bsl 7)+1, + K48-1,K48,K48+1,K48+(1 bsl 7)-1,K48+(1 bsl 7),K48+(1 bsl 7)+1, + K64-1,K64,K64+1,K64+(1 bsl 7)-1,K64+(1 bsl 7),K64+(1 bsl 7)+1, + K64+K16-1,K64+K16,K64+K16+1], + Types = ['Os','OsFrag'], + [fragmented_octet_string(Erules, Types, L) || L <- Lens], + fragmented_octet_string(Erules, ['FixedOs65536'], 65536), + fragmented_octet_string(Erules, ['FixedOs65537'], 65537), + + %% Make sure that octet alignment works. + roundtrip('OsAlignment', + {'OsAlignment',false,make_value(70000),true,make_value(66666), + false,make_value(65536),42}), + roundtrip('OsAlignment', + {'OsAlignment',false,make_value(0),true,make_value(0), + false,make_value(65536),42}), + ok. + +fragmented_octet_string(Erules, Types, L) -> + Value = make_value(L), + [begin + Encoded = enc_frag(Erules, Type, Value), + {ok,Value} = 'PrimStrings':decode(Type, Encoded) + end || Type <- Types], + ok. +enc_frag(Erules, Type, Value) -> + {ok,Encoded} = 'PrimStrings':encode(Type, Value), + case Erules of + ber -> + Encoded; + _ -> + %% Validate encoding with our own encoder. + Encoded = enc_frag_1(<<>>, list_to_binary(Value)) + end. + +enc_frag_1(Res, Bin0) -> + K16 = 1 bsl 14, + Sz = byte_size(Bin0), + if + Sz >= K16 -> + F = min(Sz div K16, 4), + FragSize = F * K16, + <<Frag:FragSize/binary-unit:8,Bin/binary>> = Bin0, + enc_frag_1(<<Res/binary,3:2,F:6,Frag/binary>>, Bin); + Sz >= 128 -> + <<Res/binary,1:1,0:1,Sz:14,Bin0/binary>>; + true -> + <<Res/binary,0:1,Sz:7,Bin0/binary>> + end. + +make_value(L) -> + make_value(L, 0, []). + +make_value(0, _, Acc) -> + Acc; +make_value(N, Byte, Acc) when Byte =< 255 -> + make_value(N-1, Byte+7, [Byte|Acc]); +make_value(N, Byte, Acc) -> + make_value(N, Byte band 16#FF, Acc). - numeric_string(Rules) -> %%========================================================== %% Ns ::= NumericString %%========================================================== - ?line {ok,BytesNs2} = asn1_wrapper:encode('PrimStrings','Ns',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Ns',lists:flatten(BytesNs2)), + roundtrip('Ns', []), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -455,10 +474,7 @@ numeric_string(Rules) -> %% NsCon ::= [70] NumericString %%========================================================== - ?line {ok,BytesNs12} = asn1_wrapper:encode('PrimStrings','NsCon',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','NsCon',lists:flatten(BytesNs12)), - - + roundtrip('NsCon', []), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -482,10 +498,7 @@ numeric_string(Rules) -> %% NsExpCon ::= [71] EXPLICIT NumericString %%========================================================== - ?line {ok,BytesNs22} = asn1_wrapper:encode('PrimStrings','NsExpCon',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','NsExpCon',lists:flatten(BytesNs22)), - - + roundtrip('NsExpCon', []), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -507,9 +520,6 @@ numeric_string(Rules) -> ok. - - - other_strings(_Rules) -> @@ -517,49 +527,27 @@ other_strings(_Rules) -> %% Ps ::= PrintableString %%========================================================== - ?line {ok,BytesPs1} = asn1_wrapper:encode('PrimStrings','Ps',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Ps',lists:flatten(BytesPs1)), - - ?line {ok,BytesPs2} = asn1_wrapper:encode('PrimStrings','Ps',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Ps',lists:flatten(BytesPs2)), - + roundtrip('Ps', [47,23,99,75,47]), + roundtrip('Ps', []), %%========================================================== %% Vis ::= VisibleString %%========================================================== - ?line {ok,BytesVis1} = asn1_wrapper:encode('PrimStrings','Vis',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Vis',lists:flatten(BytesVis1)), - - ?line {ok,BytesVis2} = asn1_wrapper:encode('PrimStrings','Vis',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Vis',lists:flatten(BytesVis2)), - + roundtrip('Vis', [47,23,99,75,47]), + roundtrip('Vis', []), %%========================================================== %% IA5 ::= IA5String %%========================================================== - ?line {ok,BytesIA51} = asn1_wrapper:encode('PrimStrings','IA5',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','IA5',lists:flatten(BytesIA51)), - - ?line {ok,BytesIA52} = asn1_wrapper:encode('PrimStrings','IA5',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','IA5',lists:flatten(BytesIA52)), + roundtrip('IA5', [47,23,99,75,47]), + roundtrip('IA5', []), - IA5_1 = "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890", - - ?line {ok,BytesIA53} = asn1_wrapper:encode('PrimStrings','IA5',IA5_1), - ?line {ok,IA5_1r} = asn1_wrapper:decode('PrimStrings','IA5',lists:flatten(BytesIA53)), - ?line IA5_1 = IA5_1r, - - - - + roundtrip('IA5', IA5_1), ok. @@ -568,94 +556,60 @@ more_strings(_Rules) -> %% Ts ::= TeletexString %%========================================================== - ?line {ok,BytesTs1} = asn1_wrapper:encode('PrimStrings','Ts',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Ts',lists:flatten(BytesTs1)), - - ?line {ok,BytesTs2} = asn1_wrapper:encode('PrimStrings','Ts',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Ts',lists:flatten(BytesTs2)), - + roundtrip('Ts', [47,23,99,75,47]), + roundtrip('Ts', []), %%========================================================== %% Vxs ::= VideotexString %%========================================================== - ?line {ok,BytesVxs1} = asn1_wrapper:encode('PrimStrings','Vxs',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Vxs',lists:flatten(BytesVxs1)), - - ?line {ok,BytesVxs2} = asn1_wrapper:encode('PrimStrings','Vxs',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Vxs',lists:flatten(BytesVxs2)), - + roundtrip('Vxs', [47,23,99,75,47]), + roundtrip('Vxs', []), %%========================================================== %% Grs ::= GraphicString %%========================================================== - ?line {ok,BytesGrs1} = asn1_wrapper:encode('PrimStrings','Grs',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Grs',lists:flatten(BytesGrs1)), - - ?line {ok,BytesGrs2} = asn1_wrapper:encode('PrimStrings','Grs',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Grs',lists:flatten(BytesGrs2)), + roundtrip('Grs',[47,23,99,75,47]), + roundtrip('Grs', []), %%========================================================== %% ODesc ::= ObjectDescriptor, test case for OTP-4161 %%========================================================== - ?line {ok,BytesODesc1} = asn1_wrapper:encode('PrimStrings','ODesc',[79,98,106,101,99,116,68,101,115,99,114,105,112,116,111,114]), - ?line {ok,[79,98,106,101,99,116,68,101,115,99,114,105,112,116,111,114]} = - asn1_wrapper:decode('PrimStrings','ODesc',lists:flatten(BytesODesc1)), - - ?line {ok,BytesODesc2} = asn1_wrapper:encode('PrimStrings','ODesc',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','ODesc',lists:flatten(BytesODesc2)), + roundtrip('ODesc', [79,98,106,101,99,116,68,101,115,99,114, + 105,112,116,111,114]), + roundtrip('ODesc', []), %%========================================================== %% Ges ::= GeneralString %%========================================================== - ?line {ok,BytesGes1} = asn1_wrapper:encode('PrimStrings','Ges',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Ges',lists:flatten(BytesGes1)), - - ?line {ok,BytesGes2} = asn1_wrapper:encode('PrimStrings','Ges',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Ges',lists:flatten(BytesGes2)), - - ok. + roundtrip('Ges', [47,23,99,75,47]), + roundtrip('Ges', []), + ok. universal_string(Rules) -> - %%========================================================== %% Us ::= UniversalString %%========================================================== - ?line {ok,Bytes1} = - asn1_wrapper:encode('PrimStrings','Us',[{47,23,99,47},{0,0,55,66}]), - ?line {ok,[{47,23,99,47},{0,0,55,66}]} = - asn1_wrapper:decode('PrimStrings','Us',lists:flatten(Bytes1)), + roundtrip('Us', [{47,23,99,47},{0,0,55,66}]), ?line {ok,Bytes2} = asn1_wrapper:encode('PrimStrings','Us',[{47,23,99,255},{0,0,0,201}]), ?line {ok,[{47,23,99,255},201]} = asn1_wrapper:decode('PrimStrings','Us',lists:flatten(Bytes2)), - ?line {ok,Bytes3} = asn1_wrapper:encode('PrimStrings','Us',"Universal String"), - ?line {ok,"Universal String"} = - asn1_wrapper:decode('PrimStrings','Us',lists:flatten(Bytes3)), - - ?line {ok,Bytes4} = asn1_wrapper:encode('PrimStrings','Us',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Us',lists:flatten(Bytes4)), - - ?line {ok,Bytes5} = asn1_wrapper:encode('PrimStrings','Us',[{47,23,99,47}]), - ?line {ok,[{47,23,99,47}]} = - asn1_wrapper:decode('PrimStrings','Us',lists:flatten(Bytes5)), - + roundtrip('Us', "Universal String"), + roundtrip('Us', []), + roundtrip('Us', [{47,23,99,47}]), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -670,32 +624,22 @@ universal_string(Rules) -> Us1 = "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890", - ?line {ok,Bytes15} = asn1_wrapper:encode('PrimStrings','IA5',Us1), - ?line {ok,Us1r} = asn1_wrapper:decode('PrimStrings','IA5',lists:flatten(Bytes15)), - ?line Us1 = Us1r, - + roundtrip('IA5', Us1), %%========================================================== %% UsCon ::= [70] UniversalString %%========================================================== - ?line {ok,Bytes11} = - asn1_wrapper:encode('PrimStrings','UsCon',[{47,23,99,255},{0,0,2,201}]), - ?line {ok,[{47,23,99,255},{0,0,2,201}]} = - asn1_wrapper:decode('PrimStrings','UsCon',lists:flatten(Bytes11)), + roundtrip('UsCon', [{47,23,99,255},{0,0,2,201}]), ?line {ok,Bytes12} = asn1_wrapper:encode('PrimStrings','UsCon',[{47,23,99,255},{0,0,0,201}]), ?line {ok,[{47,23,99,255},201]} = asn1_wrapper:decode('PrimStrings','UsCon',lists:flatten(Bytes12)), - ?line {ok,Bytes13} = asn1_wrapper:encode('PrimStrings','UsCon',"Universal String"), - ?line {ok,"Universal String"} = - asn1_wrapper:decode('PrimStrings','UsCon',lists:flatten(Bytes13)), - - ?line {ok,Bytes14} = asn1_wrapper:encode('PrimStrings','UsCon',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','UsCon',lists:flatten(Bytes14)), + roundtrip('UsCon', "Universal String"), + roundtrip('UsCon', []), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -712,25 +656,15 @@ universal_string(Rules) -> %% UsExpCon ::= [71] EXPLICIT UniversalString %%========================================================== - ?line {ok,Bytes21} = - asn1_wrapper:encode('PrimStrings','UsExpCon',[{47,23,99,255},{0,0,2,201}]), - ?line {ok,[{47,23,99,255},{0,0,2,201}]} = - asn1_wrapper:decode('PrimStrings','UsExpCon',lists:flatten(Bytes21)), + roundtrip('UsExpCon', [{47,23,99,255},{0,0,2,201}]), ?line {ok,Bytes22} = asn1_wrapper:encode('PrimStrings','UsExpCon',[{47,23,99,255},{0,0,0,201}]), ?line {ok,[{47,23,99,255},201]} = asn1_wrapper:decode('PrimStrings','UsExpCon',lists:flatten(Bytes22)), - ?line {ok,Bytes23} = - asn1_wrapper:encode('PrimStrings','UsExpCon',"Universal String"), - ?line {ok,"Universal String"} = - asn1_wrapper:decode('PrimStrings','UsExpCon',lists:flatten(Bytes23)), - - ?line {ok,Bytes24} = - asn1_wrapper:encode('PrimStrings','UsExpCon',[]), - ?line {ok,[]} = - asn1_wrapper:decode('PrimStrings','UsExpCon',lists:flatten(Bytes24)), + roundtrip('UsExpCon', "Universal String"), + roundtrip('UsExpCon', []), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -740,12 +674,8 @@ universal_string(Rules) -> asn1_wrapper:decode('PrimStrings','UsExpCon',lists:flatten([16#BF,16#47,16,60,16#80,28,4,47,23,99,255,28,4,0,0,2,201,0,0])); _ -> ok end, - - -ok. - - + ok. bmp_string(_Rules) -> @@ -754,29 +684,18 @@ bmp_string(_Rules) -> %% BMP ::= BMPString %%========================================================== - ?line {ok,Bytes1} = - asn1_wrapper:encode('PrimStrings','BMP',[{0,0,99,48},{0,0,2,201}]), - ?line {ok,[{0,0,99,48},{0,0,2,201}]} = - asn1_wrapper:decode('PrimStrings','BMP',lists:flatten(Bytes1)), + roundtrip('BMP', [{0,0,99,48},{0,0,2,201}]), ?line {ok,Bytes2} = asn1_wrapper:encode('PrimStrings','BMP',[{0,0,0,48},{0,0,2,201}]), ?line {ok,[48,{0,0,2,201}]} = asn1_wrapper:decode('PrimStrings','BMP',lists:flatten(Bytes2)), - - ?line {ok,Bytes3} = asn1_wrapper:encode('PrimStrings','BMP',"BMP String"), - ?line {ok,"BMP String"} = - asn1_wrapper:decode('PrimStrings','BMP',lists:flatten(Bytes3)), - - ?line {ok,Bytes4} = asn1_wrapper:encode('PrimStrings','BMP',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','BMP',lists:flatten(Bytes4)), + roundtrip('BMP', "BMP String"), + roundtrip('BMP', []), BMP1 = "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890", - ?line {ok,Bytes5} = asn1_wrapper:encode('PrimStrings','BMP',BMP1), - ?line {ok,BMP1r} = asn1_wrapper:decode('PrimStrings','BMP',lists:flatten(Bytes5)), - ?line BMP1 = BMP1r, - + roundtrip('BMP', BMP1), ok. @@ -790,35 +709,17 @@ times(_Rules) -> %% Gt ::= GeneralizedTime %%========================================================== - ?line {ok,Bytes1} = asn1_wrapper:encode('PrimStrings','Gt',"19970923110723.2"), - ?line {ok,"19970923110723.2"} = - asn1_wrapper:decode('PrimStrings','Gt',lists:flatten(Bytes1)), + roundtrip('Gt', "19970923110723.2"), + roundtrip('Gt', "19970923110723.2Z"), + roundtrip('Gt', "19970923110723.2-0500"), - ?line {ok,Bytes2} = asn1_wrapper:encode('PrimStrings','Gt',"19970923110723.2Z"), - ?line {ok,"19970923110723.2Z"} = - asn1_wrapper:decode('PrimStrings','Gt',lists:flatten(Bytes2)), - - ?line {ok,Bytes3} = asn1_wrapper:encode('PrimStrings','Gt',"19970923110723.2-0500"), - ?line {ok,"19970923110723.2-0500"} = - asn1_wrapper:decode('PrimStrings','Gt',lists:flatten(Bytes3)), - - - - - - %%========================================================== %% UTC ::= UTCTime %%========================================================== - ?line {ok,Bytes11} = asn1_wrapper:encode('PrimStrings','UTC',"9709211107Z"), - ?line {ok,"9709211107Z"} = - asn1_wrapper:decode('PrimStrings','UTC',lists:flatten(Bytes11)), - - ?line {ok,Bytes12} = asn1_wrapper:encode('PrimStrings','UTC',"9709211107-0500"), - ?line {ok,"9709211107-0500"} = - asn1_wrapper:decode('PrimStrings','UTC',lists:flatten(Bytes12)), + roundtrip('UTC', "9709211107Z"), + roundtrip('UTC', "9709211107-0500"), ok. @@ -917,3 +818,8 @@ wrapper_utf8_binary_to_list(L) when is_list(L) -> asn1rt:utf8_binary_to_list(list_to_binary(L)); wrapper_utf8_binary_to_list(B) -> asn1rt:utf8_binary_to_list(B). + +roundtrip(Type, Value) -> + {ok,Encoded} = 'PrimStrings':encode(Type, Value), + {ok,Value} = 'PrimStrings':decode(Type, Encoded), + ok. diff --git a/lib/asn1/test/testSSLspecs.erl b/lib/asn1/test/testSSLspecs.erl index 51ef134e5f..45c5da50f0 100644 --- a/lib/asn1/test/testSSLspecs.erl +++ b/lib/asn1/test/testSSLspecs.erl @@ -42,11 +42,11 @@ compile(Config, Options) -> asn1_test_lib:compile_all(["PKIX1Explicit93", "PKIX1Implicit93"], Config, NewOptions). -compile_inline(Config, Rule) when Rule == ber_bin; Rule == ber_bin_v2 -> +compile_inline(Config, ber=Rule) -> DataDir = ?config(data_dir, Config), CaseDir = ?config(case_dir, Config), Options = [{i, CaseDir}, {i, DataDir}, Rule, - der, compact_bit_string, optimize, asn1config, inline], + der, compact_bit_string, asn1config, inline], ok = remove_db_file_inline(CaseDir), asn1_test_lib:compile("OTP-PKIX.set.asn", Config, Options); compile_inline(_Config, _Rule) -> @@ -73,7 +73,7 @@ remove_db_file_inline(Dir) -> ?line ok = remove_db_file(Dir ++ "PKIX1Explicit88.asn1db"), ?line ok = remove_db_file(Dir ++ "PKIX1Implicit88.asn1db"). -run(BER) when BER==ber_bin;BER==ber_bin_v2 -> +run(ber) -> run1(1); run(_) -> ok. @@ -100,20 +100,20 @@ transform1(ATAV) -> ?line {ok, ATAVEnc} = 'PKIX1Explicit88':encode('AttributeTypeAndValue', ATAV), ?line {ok, _ATAVDec} = 'SSL-PKIX':decode('AttributeTypeAndValue', - list_to_binary(ATAVEnc)). + ATAVEnc). transform2(ATAV) -> ?line {ok, ATAVEnc} = 'PKIX1Explicit88':encode('AttributeTypeAndValue', ATAV), ?line {ok, _ATAVDec} = 'PKIX1Explicit88':decode('AttributeTypeAndValue', - list_to_binary(ATAVEnc)). + ATAVEnc). transform4(ATAV) -> ?line {ok, ATAVEnc} = 'PKIX1Explicit88':encode('Attribute', ATAV), ?line {ok, _ATAVDec} = 'PKIX1Explicit88':decode('Attribute', - list_to_binary(ATAVEnc)). + ATAVEnc). ex(1) -> @@ -146,7 +146,7 @@ ex(7) -> {1,2,840,113549,1,9,1}, [[19,5,111,116,112,67,65]]}. -run_inline(Rule) when Rule==ber_bin;Rule==ber_bin_v2 -> +run_inline(ber) -> Cert = cert(), ?line {ok,{'CertificatePKIX1Explicit88',{Type,UnDec},_,_}} = 'OTP-PKIX':decode_TBSCert_exclusive(Cert), ?line {ok,_} = 'OTP-PKIX':decode_part(Type,UnDec), diff --git a/lib/asn1/test/testSeqIndefinite.erl b/lib/asn1/test/testSeqIndefinite.erl index 25742474bb..c7b8aba523 100644 --- a/lib/asn1/test/testSeqIndefinite.erl +++ b/lib/asn1/test/testSeqIndefinite.erl @@ -23,13 +23,7 @@ -include_lib("test_server/include/test_server.hrl"). - -main(per_bin) -> ok; main(per) -> ok; -main(ber_bin_v2) -> - main(ber); -main(ber_bin) -> - main(ber); main(ber) -> %% normal encoding diff --git a/lib/asn1/test/testSeqOf.erl b/lib/asn1/test/testSeqOf.erl index 0c0bbc3e66..1aa1eab26d 100644 --- a/lib/asn1/test/testSeqOf.erl +++ b/lib/asn1/test/testSeqOf.erl @@ -198,19 +198,10 @@ main(Rules) -> ?line {ok,Bytes51} = asn1_wrapper:encode('SeqOf','SeqEmp',#'SeqEmp'{seq1 = [#'Empty'{}]}), ?line {ok,{'SeqEmp',[{'Empty'}]}} = asn1_wrapper:decode('SeqOf','SeqEmp',lists:flatten(Bytes51)), - - case Rules of - ber -> - ?line {ok,Bytes52} = asn1_wrapper:encode('SeqOfEnum','SeqOfEnum', - {'SeqOfEnum',[{'Enum',a},{'Enum',b}]}), - ?line {ok,[a,b]} = asn1_wrapper:decode('SeqOfEnum','SeqOfEnum', - lists:flatten(Bytes52)); - _ -> ok - end, %% tests of OTP-4590 case Rules of - PER when PER == per; PER == per_bin -> + per -> DayNames = ["Monday","Tuesday","Wednesday","Thursday","Friday","Saturday","Sunday"], ?line {ok,Bytes60} = asn1_wrapper:encode('XSeqOf','DayNames2',DayNames), ?line {ok,Bytes60} = asn1_wrapper:encode('XSeqOf','DayNames4',DayNames), diff --git a/lib/asn1/test/testSetIndefinite.erl b/lib/asn1/test/testSetIndefinite.erl index d8e2b6a9cf..73006da62b 100644 --- a/lib/asn1/test/testSetIndefinite.erl +++ b/lib/asn1/test/testSetIndefinite.erl @@ -24,12 +24,7 @@ -include_lib("test_server/include/test_server.hrl"). -main(per_bin) -> ok; main(per) -> ok; -main(ber_bin_v2) -> - main(ber); -main(ber_bin) -> - main(ber); main(ber) -> %% normal encoding diff --git a/lib/asn1/test/testSetOptional.erl b/lib/asn1/test/testSetOptional.erl index 4692941524..cef90bc843 100644 --- a/lib/asn1/test/testSetOptional.erl +++ b/lib/asn1/test/testSetOptional.erl @@ -181,7 +181,7 @@ main(_Rules) -> ok. -ticket_7533(Ber) when Ber == ber; Ber == ber_bin -> +ticket_7533(Ber) when Ber == ber -> Val = #'SetOpt1'{bool1 = true,int1=12,set1=#'SetIn'{boolIn=false,intIn=13}}, ?line {ok,B} = asn1_wrapper:encode('SetOptional','SetOpt1',Val), ?line {ok,Val} = asn1_wrapper:decode('SetOptional','SetOpt1',B), diff --git a/lib/asn1/test/testTCAP.erl b/lib/asn1/test/testTCAP.erl index 878ce7c070..b723995e40 100644 --- a/lib/asn1/test/testTCAP.erl +++ b/lib/asn1/test/testTCAP.erl @@ -37,7 +37,7 @@ compile_asn1config(Config, Options) -> asn1_test_lib:compile_all(Files, Config, Options), asn1_test_lib:compile_erlang("TCAPPackage_msg", Config, []). -test(Erule,_Config) when Erule==ber;Erule==ber_bin;Erule==ber_bin_v2 -> +test(ber=Erule,_Config) -> % ?line OutDir = ?config(priv_dir,Config), %% testing OTP-4798, open type encoded with indefinite length ?line {ok,_Res} = asn1_wrapper:decode('TCAPMessages-simple','MessageType', val_OTP_4798(Erule)), @@ -81,7 +81,7 @@ test_asn1config() -> ?line Val2 = 'TCAPPackage_msg':val('TransactionPDU'), ?line {ok,B2} = 'TCAPPackage':encode('TransactionPDU',Val2), - ?line {ok,ExMsg2}='TCAPPackage':decode_TransactionPDU(list_to_binary(B2)), + {ok,ExMsg2}='TCAPPackage':decode_TransactionPDU(B2), ?line {_,_,_,{Key2,ExVal2}}=ExMsg2, ?line {ok,_Parts2}='TCAPPackage':decode_part(Key2,ExVal2), diff --git a/lib/asn1/test/testTimer.erl b/lib/asn1/test/testTimer.erl index 2d3b777558..cd7ceb5630 100644 --- a/lib/asn1/test/testTimer.erl +++ b/lib/asn1/test/testTimer.erl @@ -133,23 +133,7 @@ go(Config,Enc) -> Module = 'H323-MESSAGES', Type = 'H323-UserInformation', Value = val(), -%% ok = asn1ct:compile(HelpModule,[Enc]), - -%% ok = asn1ct:compile(Module,[Enc]), - ?line {ok,B} = asn1rt:encode(Module,Type,Value), - Bytes = case Enc of - ber_bin -> - list_to_binary(B); - per_bin when is_list(B) -> - list_to_binary(B); - per_bin -> - B; - uper_bin -> - B; - _ -> - %%lists:flatten(B) - list_to_binary(B) - end, + {ok,Bytes} = asn1rt:encode(Module,Type,Value), CompileOptions = compile_options(), @@ -181,35 +165,18 @@ encode(N, Module,Type,Value) -> decode(0, _Module,_Type,_Value,_Erule) -> done; decode(N, Module,Type,Value,Erule) -> - case Erule of - ber -> - ?line {ok,_B} = asn1rt:decode(Module,Type,binary_to_list(Value)); - per -> - ?line {ok,_B} = asn1rt:decode(Module,Type,binary_to_list(Value)); - _ -> - ?line {ok,_B} = asn1rt:decode(Module,Type,Value) - end, + {ok,_B} = asn1rt:decode(Module,Type,Value), decode(N-1, Module,Type,Value,Erule). compile_options() -> - ?line {ok,Info} = asn1rt:info('H323-MESSAGES'), - case lists:keysearch(options,1,Info) of - {_,{_,Opts}} -> - Opts2 = - case lists:member(ber_bin_v2,Opts) of - true -> - [ber_bin,optimize] ++ lists:delete(optimize,Opts); - _ -> - Opts - end, - Opts3 = [X||X <- Opts2, - (X == ber orelse - X == ber_bin orelse - X == per orelse - X == per_bin orelse - X == optimize orelse - X == driver)], - lists:flatten(io_lib:format("~p",[Opts3])); + {ok,Info} = asn1rt:info('H323-MESSAGES'), + case lists:keyfind(options, 1, Info) of + {_,Opts0} -> + Opts1 = [X || X <- Opts0, + (X =:= ber orelse + X =:= per orelse + X =:= uper)], + lists:flatten(io_lib:format("~p", [Opts1])); _ -> "[]" end. diff --git a/lib/asn1/test/testTypeValueNotation.erl b/lib/asn1/test/testTypeValueNotation.erl index cd5223ef23..59f7385f08 100644 --- a/lib/asn1/test/testTypeValueNotation.erl +++ b/lib/asn1/test/testTypeValueNotation.erl @@ -21,11 +21,9 @@ -export([main/2]). --include_lib("test_server/include/test_server.hrl"). - -record('Seq', {octstr, int, bool, enum, bitstr, null, oid, vstr}). -main(Rule, Option) -> +main(_Rule, _Option) -> Value1 = #'Seq'{octstr = [1, 2, 3, 4], int = 12, bool = true, @@ -34,28 +32,5 @@ main(Rule, Option) -> null = 'NULL', oid = {1, 2, 55}, vstr = "Hello World"}, - Value2 = #'Seq'{octstr = {'OctStr', [1, 2, 3, 4]}, - int = {'Int', 12}, - bool = {'Bool', true}, - enum = {'Enum', a}, - bitstr = {'BitStr', [1, 0, 1, 0]}, - null = {'Null', 'NULL'}, - oid = {'OId', {1, 2, 55}}, - vstr = {'VStr', "Hello World"}}, - main(Rule, Option, Value1, Value2). - -%% Value2 will fail for ber_bin_v2, per_bin with nifs (optimize) and uper_bin -main(ber_bin_v2, _, Value1, Value2) -> encode_fail(Value1, Value2); -main(per_bin, [optimize], Value1, Value2) -> encode_fail(Value1, Value2); -main(uper_bin, [], Value1, Value2) -> encode_fail(Value1, Value2); -main(_, _, Value1, Value2) -> encode_normal(Value1, Value2). - -encode_normal(Value1, Value2) -> - {ok, Bytes} = asn1_wrapper:encode('SeqTypeRefPrim', 'Seq', Value1), - {ok, Bytes} = asn1_wrapper:encode('SeqTypeRefPrim', 'Seq', Value2), - {ok, Value1} = asn1_wrapper:decode('SeqTypeRefPrim', 'Seq', Bytes). - -encode_fail(Value1, Value2) -> - {ok, Bytes} = asn1_wrapper:encode('SeqTypeRefPrim', 'Seq', Value1), - {error, _Reason} = asn1_wrapper:encode('SeqTypeRefPrim', 'Seq', Value2), - {ok, Value1} = asn1_wrapper:decode('SeqTypeRefPrim', 'Seq', Bytes). + {ok, Bytes} = asn1_wrapper:encode('SeqTypeRefPrim', 'Seq', Value1), + {ok, Value1} = asn1_wrapper:decode('SeqTypeRefPrim', 'Seq', Bytes). diff --git a/lib/asn1/test/testX420.erl b/lib/asn1/test/testX420.erl index abdbbfe536..52b20a2c70 100644 --- a/lib/asn1/test/testX420.erl +++ b/lib/asn1/test/testX420.erl @@ -34,7 +34,7 @@ compile(Erule, Options, Config) -> compile_loop(_Erule, [], _Options, _Config) -> ok; compile_loop(Erule, [Spec|Specs], Options, Config) - when Erule == ber; Erule == ber_bin; Erule == ber_bin_v2; Erule == per -> + when Erule =:= ber; Erule =:= per -> CaseDir = ?config(case_dir, Config), asn1_test_lib:compile(filename:join([x420, Spec]), Config, [Erule, {i, CaseDir}]), diff --git a/lib/asn1/test/test_compile_options.erl b/lib/asn1/test/test_compile_options.erl index 4e732308d8..b973c5fbcc 100644 --- a/lib/asn1/test/test_compile_options.erl +++ b/lib/asn1/test/test_compile_options.erl @@ -92,7 +92,8 @@ noobj(Config) -> file:delete(filename:join([OutDir,'P-Record.beam'])), file:delete(filename:join([OutDir,'p_record.erl'])), file:delete(filename:join([OutDir,'p_record.beam'])), - ?line ok=asn1ct:compile(filename:join([DataDir,"p_record.set.asn"]),[asn1config,ber_bin,optimize,noobj,{outdir,OutDir}]), + ok = asn1ct:compile(filename:join([DataDir,"p_record.set.asn"]), + [asn1config,ber,noobj,{outdir,OutDir}]), %% ?line false = code:is_loaded('P-Record'), %% ?line false = code:is_loaded('p_record'), ?line {error,enoent} = diff --git a/lib/asn1/test/test_inline.erl b/lib/asn1/test/test_inline.erl index 62625572e3..e03ad739f9 100644 --- a/lib/asn1/test/test_inline.erl +++ b/lib/asn1/test/test_inline.erl @@ -41,16 +41,16 @@ inline1(Config, Rule, Opt) -> asn1_test_lib:compile("P-Record", Config, [{inline, 'inlined_P_Record'}|Opt]), test_inline1(), - ok=remove_inlined_files2(CaseDir, ber_bin_v2), + ok=remove_inlined_files2(CaseDir, ber), case Rule of - ber_bin_v2 -> + ber -> asn1_test_lib:compile("P-Record", Config, - [ber_bin, inline, asn1config, optimize|Opt]), + [ber, inline, asn1config|Opt]), test_inline2(Rule, 'P-Record'), remove_inlined_files3(CaseDir, Rule), asn1_test_lib:compile("p_record.set.asn", Config, - [ber_bin, inline, asn1config, optimize|Opt]), + [ber, inline, asn1config|Opt]), test_inline2(Rule, 'p_record'), remove_inlined_files4(CaseDir, Rule); _ -> @@ -71,12 +71,12 @@ test_inline1() -> ?line {ok,_}=asn1_wrapper:decode('inlined_P_Record', 'PersonnelRecord',Bytes). -test_inline2(ber_bin_v2,Mod) -> +test_inline2(ber,Mod) -> PRecMsg = {'PersonnelRecord',{'Name',"Sven","S","Svensson"}, "manager",123,"20000202",{'Name',"Inga","K","Svensson"}, asn1_DEFAULT}, ?line {ok,Bytes} = Mod:encode('PersonnelRecord',PRecMsg), - ?line {ok,_} = Mod:sel_dec(list_to_binary(Bytes)); + {ok,_} = Mod:sel_dec(Bytes); test_inline2(_,_) -> ok. @@ -243,7 +243,7 @@ remove_inlined_files2(Dir,Rule) -> ?line ok=file:delete(X) end,[TargetErl,TargetBeam]), ok. -remove_inlined_files3(Dir,ber_bin_v2) -> +remove_inlined_files3(Dir,ber) -> Erl=filename:join([Dir,"P-Record.erl"]), Beam=filename:join([Dir,"P-Record.beam"]), Asn1DB=filename:join([Dir,"P-Record.asn1db"]), @@ -255,7 +255,7 @@ remove_inlined_files3(Dir,ber_bin_v2) -> remove_inlined_files3(_,_) -> ok. -remove_inlined_files4(Dir,ber_bin_v2) -> +remove_inlined_files4(Dir,ber) -> Erl=filename:join([Dir,"p_record.erl"]), Beam=filename:join([Dir,"p_record.beam"]), Asn1DB=filename:join([Dir,"p_record.asn1db"]), diff --git a/lib/asn1/test/test_special_decode_performance.erl b/lib/asn1/test/test_special_decode_performance.erl index 4ac0ff2b27..dd56d29b28 100644 --- a/lib/asn1/test/test_special_decode_performance.erl +++ b/lib/asn1/test/test_special_decode_performance.erl @@ -33,7 +33,7 @@ go(all) -> go(N,Mod) -> ?line Val = val(Mod), ?line {ok,B} = Mod:encode(element(1,Val),Val), - ?line go(Mod,list_to_binary(B),N). + ?line go(Mod,B,N). go(Mod,Bin,N) -> ?line FsS = get_selective_funcs(Mod), diff --git a/lib/common_test/doc/src/ct_run.xml b/lib/common_test/doc/src/ct_run.xml index da18640df7..0750f560b3 100644 --- a/lib/common_test/doc/src/ct_run.xml +++ b/lib/common_test/doc/src/ct_run.xml @@ -90,7 +90,7 @@ <pre> ct_run [-dir TestDir1 TestDir2 .. TestDirN] | [[-dir TestDir] -suite Suite1 Suite2 .. SuiteN - [[-group Group1 Group2 .. GroupN] [-case Case1 Case2 .. CaseN]]] + [[-group Groups1 Groups2 .. GroupsN] [-case Case1 Case2 .. CaseN]]] [-step [config | keep_inactive]] [-config ConfigFile1 ConfigFile2 .. ConfigFileN] [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index abe8cb2041..7e33b71de1 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -32,6 +32,100 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.6.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The ct:run_test/1 option 'config' only worked with a + single config file, not a list of files. This has been + fixed.</p> + <p> + Own Id: OTP-10495</p> + </item> + <item> + <p> + ct_netconfc:close_session sometimes returned + {error,closed} because the ssh connection was closed + (from the server side) before the rpc-reply was received + by the client. This is normal and can not be helped. It + has been corrected so the return will be 'ok' in this + case. Other error situations will still give + {error,Reason}.</p> + <p> + Own Id: OTP-10510 Aux Id: kunagi-320 [231] </p> + </item> + <item> + <p> + ct_netconfc:close_session sometimes returned + {error,closed} or (if the connection was named) + {error,{process_down,Pid,normal}} because the ssh + connection was closed (from the server side) before the + rpc-reply was received by the client. This is normal and + can not be helped. It has been corrected so the return + will be 'ok' in this situation.</p> + <p> + Own Id: OTP-10570</p> + </item> + <item> + <p> + Fix bug where ct:require of same name with same config + would return name_in_use.</p> + <p> + Own Id: OTP-10572</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + A new test case group search functionality has been + implemented that makes Common Test search automatically + through the group definitions tree (the return value of + groups/0) and create tests for all paths of nested groups + that match the specification. It also allows for + specifying unique paths to sub groups in order to avoid + execution of unwanted tests. This new feature can be used + whenever starting a test run by means of the ct_run + program, the ct:run_test/1 API function, or a Test + Specification. Details can be found in the Test Case + Group Execution section in the Running Tests chapter.</p> + <p> + Own Id: OTP-10466 Aux Id: kunagi-276 [187] </p> + </item> + </list> + </section> + + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + Restore Config data if lost when test case fails.</p> + <p> + Own Id: OTP-10070 Aux Id: kunagi-175 [86] </p> + </item> + <item> + <p> + IO server error in test_server.</p> + <p> + Own Id: OTP-10125 Aux Id: OTP-10101, kunagi-177 [88] </p> + </item> + <item> + <p> + Faulty connection handling in common_test.</p> + <p> + Own Id: OTP-10126 Aux Id: kunagi-178 [89] </p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.6.2.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index b5b914d506..b804f134c6 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -119,7 +119,7 @@ <item><c><![CDATA[ct_run -userconfig <callbackmodulename> <configfilenames> -suite <suiteswithfullpath>]]></c> </item> <item><c><![CDATA[ct_run -config <configfilenames> -suite <suitewithfullpath> - -group <groupnames> -case <casenames>]]></c></item> + -group <groups> -case <casenames>]]></c></item> </list> <p>Examples:</p> <p><c>$ ct_run -config $CFGS/sys1.cfg $CFGS/sys2.cfg -dir $SYS1_TEST $SYS2_TEST</c></p> @@ -137,6 +137,8 @@ <p><c>$ ct_run -suite ./testdir/x_SUITE ./testdir/y_SUITE</c></p> + <p>For more details on <seealso marker="run_test_chapter#group_execution">test case group execution</seealso>, please see below.</p> + <p>Other flags that may be used with <c>ct_run</c>:</p> <list> <item><c><![CDATA[-logdir <dir>]]></c>, specifies where the HTML log files are to be written.</item> @@ -269,6 +271,163 @@ <c><seealso marker="ct#run_test-1">ct</seealso></c> manual page.</p> </section> + <marker id="group_execution"></marker> + <section> + <title>Test case group execution</title> + + <p>With the <c>ct_run</c> flag, or <c>ct:run_test/1</c> option <c>group</c>, + one or more test case groups can be specified, optionally in combination + with specific test cases. The syntax for specifying groups is as follows + (on the command line):</p> + + <pre> + <![CDATA[$ ct_run -group <group_names_or_paths> [-case <cases>]]]></pre> + <p>or (in the Erlang shell):</p> + <pre> + <![CDATA[1> ct:run_test([{group,GroupsNamesOrPaths}, {case,Cases}]).]]></pre> + + <p>The <c>group_names_or_paths</c> parameter specifies either one + or more group names and/or one or more group paths. At start up, + Common Test will search for matching groups in the group definitions + tree (i.e. the list returned from <c>Suite:groups/0</c>, please see the + <seealso marker="write_test_chapter#test_case_groups">Test case groups</seealso> + chapter for details). + Given a group name, say <c>g</c>, Common Test will search for all paths + that lead to <c>g</c>. By path here we mean a sequence of nested groups, + all of which have to be followed in order to get from the top level + group to <c>g</c>. Actually, what Common Test needs to do in order to + execute the test cases in group <c>g</c>, is to call the + <c>init_per_group/2</c> function for each group in the path to + <c>g</c>, as well as all corresponding <c>end_per_group/2</c> + functions afterwards. The obvious reason for this is that the configuration + of a test case in <c>g</c> (and its <c>Config</c> input data) depends on + <c>init_per_testcase(TestCase, Config)</c> and its return value, which + in turn depends on <c>init_per_group(g, Config)</c> and its return value, + which in turn depends on <c>init_per_group/2</c> of the group above + <c>g</c>, etc, all the way up to the top level group.</p> + + <p>As you may have already realized, this means that if there is more than + one way to locate a group (and its test cases) in a path, the result of the + group search operation is a number of tests, all of which will be performed. + Common Test actually interprets a group specification that consists of a + single name this way:</p> + + <p>"Search and find all paths in the group definitions tree that lead + to the specified group and, for each path, create a test which (1) executes + all configuration functions in the path to the specified group, then (2) + executes all - or all matching - test cases in this group, as well as (3) + all - or all matching - test cases in all sub groups of the group". + </p> + + <p>It is also possible for the user to specify a specific group path with + the <c>group_names_or_paths</c> parameter. With this type of specification it's + possible to avoid execution of unwanted groups (in otherwise matching paths), + and/or the execution of sub groups. The syntax of the group path is a list of + group names in the path, e.g. on the command line: + </p> + <p><c>$ ct_run -suite "./x_SUITE" -group [g1,g3,g4] -case tc1 tc5</c></p> + <p>or similarly in the Erlang shell (requires a list within the groups list):</p> + <p><c>1> ct:run_test([{suite,"./x_SUITE"}, {group,[[g1,g3,g4]]}, {testcase,[tc1,tc5]}]).</c></p> + + <p>The last group in the specified path will be the terminating group in + the test, i.e. no sub groups following this group will be executed. In the + example above, <c>g4</c> is the terminating group, hence Common Test will + execute a test that calls all init configuration functions in the path to + <c>g4</c>, i.e. <c>g1..g3..g4</c>. It will then call test cases <c>tc1</c> + and <c>tc5</c> in <c>g4</c> and finally all end configuration functions in order + <c>g4..g3..g1</c>.</p> + + <p>Note that the group path specification doesn't necessarily + have to include <em>all</em> groups in the path to the terminating group. + Common Test will search for all matching paths if given an incomplete group + path.</p> + + <p>Note also that it's possible to combine group names and group paths with the + <c>group_names_or_paths</c> parameter. Each element is treated as + an individual specification in combination with the <c>cases</c> parameter. + See examples below.</p> + + <p>Examples:</p> + <pre> + -module(x_SUITE). + ... + %% The group definitions: + groups() -> + [{top1,[],[tc11,tc12, + {sub11,[],[tc12,tc13]}, + {sub12,[],[tc14,tc15, + {sub121,[],[tc12,tc16]}]}]}, + + {top2,[],[{group,sub21},{group,sub22}]}, + {sub21,[],[tc21,{group,sub2X2}]}, + {sub22,[],[{group,sub221},tc21,tc22,{group,sub2X2}]}, + {sub221,[],[tc21,tc23]}, + {sub2X2,[],[tc21,tc24]}]. + </pre> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group all</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,all}]).</c></p> + <p>Two tests will be executed, one for all cases and all sub groups under <c>top1</c>, + and one for all under <c>top2</c>. (We would get the same result with + <c>-group top1 top2</c>, or <c>{group,[top1,top2]}</c>.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group top1</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[top1]}]).</c></p> + <p>This will execute one test for all cases and sub groups under <c>top1</c>.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group top1 -case tc12</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[top1]}, {testcase,[tc12]}]).</c></p> + <p>This will run a test that executes <c>tc12</c> in <c>top1</c> and any sub group + under <c>top1</c> where it can be found (<c>sub11</c> and <c>sub121</c>).</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group [top1] -case tc12</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[[top1]]}, {testcase,[tc12]}]).</c></p> + <p>This will execute <c>tc12</c> <em>only</em> in group <c>top1</c>.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group top1 -case tc16</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[top1]}, {testcase,[tc16]}]).</c></p> + <p>This will search <c>top1</c> and all its sub groups for <c>tc16</c> and the result + will be that this test case executes in group <c>sub121</c>. (The specific path: + <c>-group [sub121]</c> or <c>{group,[[sub121]]}</c>, would have given + us the same result in this example).</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group sub12 [sub12]</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[sub12,[sub12]]}]).</c></p> + <p>This will execute two tests, one that includes all cases and sub groups under + <c>sub12</c>, and one with <em>only</em> the test cases in <c>sub12</c>.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group sub2X2</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[sub2X2]}]).</c></p> + <p>In this example, Common Test will find and execute two tests, one for the path from + <c>top2</c> to <c>sub2X2</c> via <c>sub21</c>, and one from <c>top2</c> to <c>sub2X2</c> + via <c>sub22</c>.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group [sub21,sub2X2]</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[[sub21,sub2X2]]}]).</c></p> + <p>Here, by specifying the unique path: <c>top2 -> sub21 -> sub2X2</c>, only one test + is executed. The second possible path from <c>top2</c> to <c>sub2X2</c> (above) + will be discarded.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group [sub22] -case tc22 tc21</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[[sub22]]}, {testcase,[tc22,tc21]}]).</c></p> + <p>In this example only the test cases for <c>sub22</c> will be executed, and in + reverse order compared to the group definition.</p> + <br></br> + + <p>If a test case that belongs to a group (according to the group definition), is executed + without a group specification, i.e. simply by means of (command line):</p> + <p><c>$ ct_run -suite "my_SUITE" -case my_tc</c></p> + <p>or (Erlang shell):</p> + <p><c>1> ct:run_test([{suite,"my_SUITE"}, {testcase,my_tc}]).</c></p> + <p>then Common Test ignores the group definition and executes the test case in the scope of the + test suite only (no group configuration functions are called).</p> + + <p>The group specification feature, exactly as it has been presented in this section, can also + be used in <seealso marker="run_test_chapter#test_specifications">Test + Specifications</seealso> (with some extra features added). Please see below.</p> + </section> + + <section> <title>Running the interactive shell mode</title> @@ -400,8 +559,8 @@ terms (e.g. log directory, label, style sheet, auto compilation).</p> <p>With test specification terms it is possible to state exactly which tests should run and in which order. A test term specifies - either one or more suites, one or more test case groups, or one - or more test cases in a group or suite.</p> + either one or more suites, one or more test case groups (possibly nested), + or one or more test cases in a group (or in multiple groups) or in a suite.</p> <p>An arbitrary number of test terms may be declared in sequence. Common Test will by default compile the terms into one or more tests to be performed in one resulting test run. Note that a term that @@ -420,28 +579,32 @@ are not executed and show up in the HTML log files as SKIPPED.</p> <p>When a test case group is specified, the resulting test - executes the - <c>init_per_group</c> function, followed by all test cases and - sub groups (including their configuration functions), and + executes the <c>init_per_group</c> function, followed by all test + cases and sub groups (including their configuration functions), and finally the <c>end_per_group</c> function. Also if particular test cases in a group are specified, <c>init_per_group</c> and <c>end_per_group</c> for the group in question are called. If a group which is defined (in <c>Suite:group/0</c>) to - be a sub group of another group, is specified (or particular test + be a sub group of another group, is specified (or if particular test cases of a sub group are), Common Test will call the configuration functions for the top level groups as well as for the sub group in question (making it possible to pass configuration data all the way from <c>init_per_suite</c> down to the test cases in the sub group).</p> - - <p>With the <c>GroupSpec</c> element (below) it's possible to specify - group execution properties that will override those specified in the + <p>The test specification utilizes the same mechanism for specifying + test case groups by means of names and paths, as explained in the + <seealso marker="run_test_chapter#group_execution">Group Execution</seealso> + section above, with the addition of the <c>GroupSpec</c> element + described next.</p> + <p>The <c>GroupSpec</c> element makes it possible to specify + group execution properties that will override those in the group definition (i.e. in <c>groups/0</c>). Execution properties for sub-groups may be overridden as well. This feature makes it possible to change properties of groups at the time of execution, - without even having to edit the test suite. More detailed documentation, - and examples, can be found in the - <seealso marker="write_test_chapter#test_case_groups"> + without even having to edit the test suite. The very same + feature is available for <c>group</c> elements in the <c>Suite:all/0</c> + list. Therefore, more detailed documentation, and examples, can be + found in the <seealso marker="write_test_chapter#test_case_groups"> Test case groups</seealso> chapter.</p> <p>Below is the test specification syntax. Test specifications can @@ -546,8 +709,8 @@ {groups, Dir, Suite, Groups}. {groups, NodeRefs, Dir, Suite, Groups}. - {groups, Dir, Suite, GroupSpec, {cases,Cases}}. - {groups, NodeRefs, Dir, Suite, GroupSpec, {cases,Cases}}. + {groups, Dir, Suite, Groups, {cases,Cases}}. + {groups, NodeRefs, Dir, Suite, Groups, {cases,Cases}}. {cases, Dir, Suite, Cases}. {cases, NodeRefs, Dir, Suite, Cases}. @@ -595,7 +758,8 @@ Dir = string() Suites = atom() | [atom()] | all Suite = atom() - Groups = GroupSpec | [GroupSpec] | all + Groups = GroupPath | [GroupPath] | GroupSpec | [GroupSpec] | all + GroupPath = [GroupName] GroupSpec = GroupName | {GroupName,Properties} | {GroupName,Properties,GroupSpec} GroupName = atom() GroupNames = GroupName | [GroupName] diff --git a/lib/common_test/src/ct.erl b/lib/common_test/src/ct.erl index 6ecd0a022c..8eafdff29f 100644 --- a/lib/common_test/src/ct.erl +++ b/lib/common_test/src/ct.erl @@ -161,7 +161,8 @@ run(TestDirs) -> %%% TestDirs = [string()] | string() %%% Suites = [string()] | [atom()] | string() | atom() %%% Cases = [atom()] | atom() -%%% Groups = [atom()] | atom() +%%% Groups = GroupNameOrPath | [GroupNameOrPath] +%%% GroupNameOrPath = [atom()] | atom() | all %%% TestSpecs = [string()] | string() %%% Label = string() | atom() %%% CfgFiles = [string()] | string() diff --git a/lib/common_test/src/ct_groups.erl b/lib/common_test/src/ct_groups.erl index 24ca3826a8..74ab5e5439 100644 --- a/lib/common_test/src/ct_groups.erl +++ b/lib/common_test/src/ct_groups.erl @@ -176,7 +176,7 @@ find(Mod, GrNames, all, [{M,TC} | Gs], Known, %% Check if test case should be saved find(Mod, GrNames, TCs, [TC | Gs], Known, Defs, FindAll) when is_atom(TC) orelse - ((size(TC) == 2) and (hd(TC) /= group)) -> + ((size(TC) == 2) and (element(1,TC) /= group)) -> Case = if is_atom(TC) -> Tuple = {Mod,TC}, diff --git a/lib/common_test/src/ct_netconfc.erl b/lib/common_test/src/ct_netconfc.erl index 294b82bff6..11c8235040 100644 --- a/lib/common_test/src/ct_netconfc.erl +++ b/lib/common_test/src/ct_netconfc.erl @@ -307,7 +307,7 @@ -type option() :: {ssh,host()} | {port,inet:port_number()} | {user,string()} | {password,string()} | {user_dir,string()} | {timeout,timeout()}. --type host() :: inet:host_name() | inet:ip_address(). +-type host() :: inet:hostname() | inet:ip_address(). -type notification() :: {notification, xml_attributes(), notification_content()}. -type notification_content() :: [event_time()|simple_xml()]. diff --git a/lib/common_test/src/ct_slave.erl b/lib/common_test/src/ct_slave.erl index cb05423497..58633b7de6 100644 --- a/lib/common_test/src/ct_slave.erl +++ b/lib/common_test/src/ct_slave.erl @@ -37,7 +37,7 @@ -record(options, {username, password, boot_timeout, init_timeout, startup_timeout, startup_functions, monitor_master, - kill_if_fail, erl_flags}). + kill_if_fail, erl_flags, env}). %%%----------------------------------------------------------------- %%% @spec start(Node) -> Result @@ -85,7 +85,8 @@ start(Host, Node) -> %%% {startup_functions, StartupFunctions} | %%% {monitor_master, Monitor} | %%% {kill_if_fail, KillIfFail} | -%%% {erl_flags, ErlangFlags} +%%% {erl_flags, ErlangFlags} | +%%% {env, [{EnvVar,Value}]} %%% Username = string() %%% Password = string() %%% BootTimeout = integer() @@ -99,6 +100,8 @@ start(Host, Node) -> %%% Monitor = bool() %%% KillIfFail = bool() %%% ErlangFlags = string() +%%% EnvVar = string() +%%% Value = string() %%% Result = {ok, NodeName} | {error, already_started, NodeName} | %%% {error, started_not_connected, NodeName} | %%% {error, boot_timeout, NodeName} | @@ -152,6 +155,9 @@ start(Host, Node) -> %%% <p>Option <code>erlang_flags</code> specifies, which flags will be added %%% to the parameters of the <code>erl</code> executable.</p> %%% +%%% <p>Option <code>env</code> specifies a list of environment variables +%%% that will extended the environment.</p> +%%% %%% <p>Special return values are: %%% <list> %%% <item><code>{error, already_started, NodeName}</code> - if the node with @@ -233,10 +239,12 @@ fetch_options(Options) -> Monitor = get_option_value(monitor_master, Options, false), KillIfFail = get_option_value(kill_if_fail, Options, true), ErlFlags = get_option_value(erl_flags, Options, []), + EnvVars = get_option_value(env, Options, []), #options{username=UserName, password=Password, boot_timeout=BootTimeout, init_timeout=InitTimeout, startup_timeout=StartupTimeout, startup_functions=StartupFunctions, - monitor_master=Monitor, kill_if_fail=KillIfFail, erl_flags=ErlFlags}. + monitor_master=Monitor, kill_if_fail=KillIfFail, + erl_flags=ErlFlags, env=EnvVars}. % send a message when slave node is started % @hidden @@ -306,6 +314,7 @@ do_start(Host, Node, Options) -> true-> spawn_remote_node(Host, Node, Options) end, + BootTimeout = Options#options.boot_timeout, InitTimeout = Options#options.init_timeout, StartupTimeout = Options#options.startup_timeout, @@ -372,9 +381,9 @@ get_cmd(Node, Flags) -> % spawn node locally spawn_local_node(Node, Options) -> - ErlFlags = Options#options.erl_flags, + #options{env=Env,erl_flags=ErlFlags} = Options, Cmd = get_cmd(Node, ErlFlags), - open_port({spawn, Cmd}, [stream]). + open_port({spawn, Cmd}, [stream,{env,Env}]). % start crypto and ssh if not yet started check_for_ssh_running() -> @@ -393,9 +402,10 @@ check_for_ssh_running() -> % spawn node remotely spawn_remote_node(Host, Node, Options) -> - Username = Options#options.username, - Password = Options#options.password, - ErlFlags = Options#options.erl_flags, + #options{username=Username, + password=Password, + erl_flags=ErlFlags, + env=Env} = Options, SSHOptions = case {Username, Password} of {[], []}-> []; @@ -407,8 +417,17 @@ spawn_remote_node(Host, Node, Options) -> check_for_ssh_running(), {ok, SSHConnRef} = ssh:connect(atom_to_list(Host), 22, SSHOptions), {ok, SSHChannelId} = ssh_connection:session_channel(SSHConnRef, infinity), + ssh_setenv(SSHConnRef, SSHChannelId, Env), ssh_connection:exec(SSHConnRef, SSHChannelId, get_cmd(Node, ErlFlags), infinity). + +ssh_setenv(SSHConnRef, SSHChannelId, [{Var, Value} | Vars]) + when is_list(Var), is_list(Value) -> + success = ssh_connection:setenv(SSHConnRef, SSHChannelId, + Var, Value, infinity), + ssh_setenv(SSHConnRef, SSHChannelId, Vars); +ssh_setenv(_SSHConnRef, _SSHChannelId, []) -> ok. + % call functions on a remote Erlang node call_functions(_Node, []) -> ok; diff --git a/lib/common_test/src/cth_conn_log.erl b/lib/common_test/src/cth_conn_log.erl index 3af89db3a5..255f3ec78a 100644 --- a/lib/common_test/src/cth_conn_log.erl +++ b/lib/common_test/src/cth_conn_log.erl @@ -58,7 +58,7 @@ -spec init(Id, HookOpts) -> Result when Id :: term(), - HookOpts :: ct:hook_options(), + HookOpts :: ct_netconfc:hook_options(), Result :: {ok,[{ct_netconfc:conn_mod(), {ct_netconfc:log_type(),[ct_netconfc:key_or_name()]}}]}. init(_Id, HookOpts) -> diff --git a/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_1_SUITE.erl b/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_1_SUITE.erl index 6f6922e686..1c5b572f92 100644 --- a/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_1_SUITE.erl +++ b/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_1_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2011. All Rights Reserved. +%% Copyright Ericsson AB 2009-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_2_SUITE.erl b/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_2_SUITE.erl index ac3c000079..060012de29 100644 --- a/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_2_SUITE.erl +++ b/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_2_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2011. All Rights Reserved. +%% Copyright Ericsson AB 2009-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/common_test/test/ct_master_SUITE.erl b/lib/common_test/test/ct_master_SUITE.erl index 27243a0067..56a343a96f 100644 --- a/lib/common_test/test/ct_master_SUITE.erl +++ b/lib/common_test/test/ct_master_SUITE.erl @@ -117,14 +117,8 @@ ct_master_test(Config) when is_list(Config) -> reformat(Events, ?eh), PrivDir, []), - find_events(NodeNames, [{tc_start,{master_SUITE,init_per_suite}}, - {tc_start,{master_SUITE,first_testcase}}, - {tc_start,{master_SUITE,second_testcase}}, - {tc_start,{master_SUITE,third_testcase}}, - {tc_start,{master_SUITE,end_per_suite}}], - Events), - - ok. + TestEvents = events_to_check(ct_master_test), + ok = find_events(NodeNames, TestEvents, Events, Config). %%%----------------------------------------------------------------- %%% HELP FUNCTIONS @@ -153,13 +147,15 @@ make_spec(DataDir, FileName, NodeNames, Suites, Config) -> CM = [{config,master,filename:join(DataDir,"master/config.txt")}], + Env = [{"THIS_MUST_BE_SET","yes"}, + {"SO_MUST_THIS","value"}], NS = lists:map( fun(NodeName) -> {init,NodeName,[ {node_start,[{startup_functions,[]}, - {monitor_master,true}]}, - {eval,{erlang,nodes,[]}} - ] + {monitor_master,true}, + {env,Env}]}, + {eval,{erlang,nodes,[]}}] } end, NodeNames), @@ -199,7 +195,6 @@ run_test(_Name, FileName, Config) -> [{FileName,ok}] = ct_test_support:run({ct_master,run,[FileName]}, [{ct_master,basic_html,[true]}], Config), - timer:sleep(5000), [{FileName,ok}] = ct_test_support:run({ct_master,run,[FileName]}, [{ct_master,basic_html,[false]}], Config). @@ -210,28 +205,26 @@ reformat(Events, EH) -> %%%----------------------------------------------------------------- %%% TEST EVENTS %%%----------------------------------------------------------------- -find_events([], _CheckEvents, _) -> - ok; -find_events([NodeName|NodeNames],CheckEvents,AllEvents) -> - find_events(NodeNames, CheckEvents, - remove_events(add_host(NodeName),CheckEvents, AllEvents, [])). - -remove_events(Node,[{Name,Data} | RestChecks], - [{?eh,#event{ name = Name, node = Node, data = Data }}|RestEvs], - Acc) -> - remove_events(Node, RestChecks, RestEvs, Acc); -remove_events(Node, Checks, [Event|RestEvs], Acc) -> - remove_events(Node, Checks, RestEvs, [Event | Acc]); -remove_events(_Node, [], [], Acc) -> - lists:reverse(Acc); -remove_events(Node, Events, [], Acc) -> - test_server:format("Could not find events: ~p in ~p for node ~p", - [Events, lists:reverse(Acc), Node]), - exit(event_not_found). + +find_events(NodeNames, TestEvents, Events, Config) -> + [begin + Node = add_host(Node0), + io:format("Searching for events for node: ~s", [Node]), + ok = ct_test_support:verify_events(TestEvents, Events, Node, Config), + io:nl() + end || Node0 <- NodeNames], + ok. add_host(NodeName) -> {ok, HostName} = inet:gethostname(), list_to_atom(atom_to_list(NodeName)++"@"++HostName). -expected_events(_) -> - []. +events_to_check(_) -> + [{?eh,tc_start,{master_SUITE,first_testcase}}, + {?eh,tc_done,{master_SUITE,first_testcase,ok}}, + {?eh,tc_start,{master_SUITE,second_testcase}}, + {?eh,tc_done,{master_SUITE,second_testcase,ok}}, + {?eh,tc_start,{master_SUITE,third_testcase}}, + {?eh,tc_done,{master_SUITE,third_testcase,ok}}, + {?eh,tc_start,{master_SUITE,env_vars}}, + {?eh,tc_done,{master_SUITE,env_vars,ok}}]. diff --git a/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl b/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl index 032d69ad9f..8a5009ad62 100644 --- a/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl +++ b/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl @@ -39,7 +39,8 @@ init_per_suite(Config) -> end_per_suite(_) -> ok. -all() -> [first_testcase, second_testcase, third_testcase]. +all() -> [first_testcase, second_testcase, third_testcase, + env_vars]. init_per_testcase(_, Config) -> Config. @@ -56,3 +57,9 @@ second_testcase(_)-> third_testcase(_)-> A = 4, A = 2*2. + +env_vars(_) -> + io:format("~p\n", [os:getenv()]), + "yes" = os:getenv("THIS_MUST_BE_SET"), + "value" = os:getenv("SO_MUST_THIS"), + ok. diff --git a/lib/common_test/test/ct_test_support.erl b/lib/common_test/test/ct_test_support.erl index 901e53c71d..e5e2e68fcb 100644 --- a/lib/common_test/test/ct_test_support.erl +++ b/lib/common_test/test/ct_test_support.erl @@ -32,7 +32,7 @@ run/2, run/3, run/4, get_opts/1, wait_for_ct_stop/1]). -export([handle_event/2, start_event_receiver/1, get_events/2, - verify_events/3, reformat/2, log_events/4, + verify_events/3, verify_events/4, reformat/2, log_events/4, join_abs_dirs/2]). -export([ct_test_halt/1]). @@ -370,6 +370,14 @@ verify_events(TEvs, Evs, Config) -> ok end. +verify_events(TEvs, Evs, Node, Config) -> + case catch verify_events1(TEvs, Evs, Node, Config) of + {'EXIT',Reason} -> + Reason; + _ -> + ok + end. + verify_events1([TestEv|_], [{TEH,#event{name=stop_logging,node=Node,data=_}}|_], Node, _) when element(1,TestEv) == TEH, element(2,TestEv) =/= stop_logging -> test_server:format("Failed to find ~p in the list of events!~n", [TestEv]), diff --git a/lib/common_test/vsn.mk b/lib/common_test/vsn.mk index 5c9fdfc47e..f9bb22867e 100644 --- a/lib/common_test/vsn.mk +++ b/lib/common_test/vsn.mk @@ -1 +1 @@ -COMMON_TEST_VSN = 1.6.2.1 +COMMON_TEST_VSN = 1.6.3 diff --git a/lib/compiler/src/beam_bsm.erl b/lib/compiler/src/beam_bsm.erl index 37053e1cc4..02794a8e18 100644 --- a/lib/compiler/src/beam_bsm.erl +++ b/lib/compiler/src/beam_bsm.erl @@ -241,21 +241,45 @@ btb_reaches_match_2([{bif,_,{f,F},Ss,Dst}=I|Is], Regs0, D0) -> Regs = btb_kill([Dst], Regs0), D = btb_follow_branch(F, Regs, D0), btb_reaches_match_1(Is, Regs, D); -btb_reaches_match_2([{test,bs_start_match2,_,_,[Ctx,_],Ctx}|Is], Regs, D) -> - case btb_context_regs(Regs) of - [Ctx] -> - D; - CtxRegs -> - case member(Ctx, CtxRegs) of - false -> btb_reaches_match_2(Is, Regs, D); - true -> btb_error(unsuitable_bs_start_match) +btb_reaches_match_2([{test,bs_start_match2,{f,F},Live,[Ctx,_],Ctx}=I|Is], + Regs0, D0) -> + CtxRegs = btb_context_regs(Regs0), + case member(Ctx, CtxRegs) of + false -> + %% This bs_start_match2 instruction does not use "our" + %% match state. Therefore we can continue the search + %% for another bs_start_match2 instruction. + D = btb_follow_branch(F, Regs0, D0), + Regs = btb_kill_not_live(Live, Regs0), + btb_reaches_match_2(Is, Regs, D); + true -> + %% OK. This instruction will use "our" match state, + %% but we must make sure that all other copies of the + %% match state are killed in the code that follows + %% the instruction. (We know that the fail branch cannot + %% be taken in this case.) + OtherCtxRegs = CtxRegs -- [Ctx], + case btb_are_all_unused(OtherCtxRegs, Is, D0) of + false -> btb_error({OtherCtxRegs,not_all_unused_after,I}); + true -> D0 end end; -btb_reaches_match_2([{test,bs_start_match2,_,_,[Bin,_],Ctx}|Is], Regs, D) -> - CtxRegs = btb_context_regs(Regs), +btb_reaches_match_2([{test,bs_start_match2,{f,F},Live,[Bin,_],Ctx}|Is], + Regs0, D0) -> + CtxRegs = btb_context_regs(Regs0), case member(Bin, CtxRegs) orelse member(Ctx, CtxRegs) of - false -> btb_reaches_match_2(Is, Regs, D); - true -> btb_error(unsuitable_bs_start_match) + false -> + %% This bs_start_match2 does not reference any copy of the + %% match state. Therefore it can safely be passed on the + %% way to another (perhaps more suitable) bs_start_match2 + %% instruction. + D = btb_follow_branch(F, Regs0, D0), + Regs = btb_kill_not_live(Live, Regs0), + btb_reaches_match_2(Is, Regs, D); + true -> + %% This variant of the bs_start_match2 instruction does + %% not accept a match state as source. + btb_error(unsuitable_bs_start_match) end; btb_reaches_match_2([{test,_,{f,F},Ss}=I|Is], Regs, D0) -> btb_ensure_not_used(Ss, I, Regs), @@ -287,11 +311,11 @@ btb_reaches_match_2([{bs_restore2,Src,_}=I|Is], Regs0, D) -> btb_reaches_match_1(Is, Regs0, D); true -> %% Check that all other copies of the context registers - %% are killed by the following instructions. + %% are unused by the following instructions. Regs = btb_kill([Src], Regs0), CtxRegs = btb_context_regs(Regs), - case btb_are_all_killed(CtxRegs, Is, D) of - false -> btb_error({CtxRegs,not_all_killed_after,I}); + case btb_are_all_unused(CtxRegs, Is, D) of + false -> btb_error({CtxRegs,not_all_unused_after,I}); true -> D#btb{must_not_save=true} end end; @@ -301,11 +325,11 @@ btb_reaches_match_2([{bs_context_to_binary,Src}=I|Is], Regs0, D) -> btb_reaches_match_1(Is, Regs0, D); true -> %% Check that all other copies of the context registers - %% are killed by the following instructions. + %% are unused by the following instructions. Regs = btb_kill([Src], Regs0), CtxRegs = btb_context_regs(Regs), - case btb_are_all_killed(CtxRegs, Is, D) of - false -> btb_error({CtxRegs,not_all_killed_after,I}); + case btb_are_all_unused(CtxRegs, Is, D) of + false -> btb_error({CtxRegs,not_all_unused_after,I}); true -> D#btb{must_not_save=true} end end; @@ -343,7 +367,7 @@ btb_call(Arity, Lbl, Regs0, Is, D0) -> %% tucked away in a y register. RegList = btb_context_regs(Regs), YRegs = [R || {y,_}=R <- RegList], - case btb_are_all_killed(YRegs, Is, D) of + case btb_are_all_unused(YRegs, Is, D) of true -> D; false -> btb_error({multiple_uses,RegList}) end; @@ -426,11 +450,11 @@ btb_reaches_match_block([], Regs) -> Regs. %% btb_are_all_killed([Register], [Instruction], D) -> true|false -%% Test whether all of the register are killed in the instruction stream. +%% Test whether all of the register are unused in the instruction stream. -btb_are_all_killed(RegList, Is, #btb{index=Li}) -> +btb_are_all_unused(RegList, Is, #btb{index=Li}) -> all(fun(R) -> - beam_utils:is_killed(R, Is, Li) + beam_utils:is_not_used(R, Is, Li) end, RegList). %% btp_regs_from_list([Register]) -> RegisterSet. diff --git a/lib/compiler/src/beam_jump.erl b/lib/compiler/src/beam_jump.erl index b05d01b2a1..636c299e47 100644 --- a/lib/compiler/src/beam_jump.erl +++ b/lib/compiler/src/beam_jump.erl @@ -220,6 +220,8 @@ extract_seq([{line,_}=Line|Is], Acc) -> extract_seq(Is, [Line|Acc]); extract_seq([{block,_}=Bl|Is], Acc) -> extract_seq_1(Is, [Bl|Acc]); +extract_seq([{bs_context_to_binary,_}=I|Is], Acc) -> + extract_seq_1(Is, [I|Acc]); extract_seq([{label,_}|_]=Is, Acc) -> extract_seq_1(Is, Acc); extract_seq(_, _) -> no. diff --git a/lib/compiler/src/beam_utils.erl b/lib/compiler/src/beam_utils.erl index 8b661e6901..8af0447f63 100644 --- a/lib/compiler/src/beam_utils.erl +++ b/lib/compiler/src/beam_utils.erl @@ -263,19 +263,11 @@ check_liveness(R, [{test,_,{f,Fail},As}|Is], St0) -> {_,_}=Other -> Other end end; -check_liveness(R, [{test,_,{f,Fail},Live,Ss,_}|Is], St0) -> - case R of - {x,X} -> - case X < Live orelse member(R, Ss) of - true -> {used,St0}; - false -> check_liveness_at(R, Fail, St0) - end; - {y,_} -> - case check_liveness_at(R, Fail, St0) of - {killed,St} -> check_liveness(R, Is, St); - {_,_}=Other -> Other - end - end; +check_liveness(R, [{test,Op,Fail,Live,Ss,Dst}|Is], St) -> + %% Check this instruction as a block to get a less conservative + %% result if the caller is is_not_used/3. + Block = [{set,[Dst],Ss,{alloc,Live,{bif,Op,Fail}}}], + check_liveness(R, [{block,Block}|Is], St); check_liveness(R, [{select,_,R,_,_}|_], St) -> {used,St}; check_liveness(R, [{select,_,_,Fail,Branches}|_], St) -> diff --git a/lib/compiler/test/bs_match_SUITE.erl b/lib/compiler/test/bs_match_SUITE.erl index bcceb889ed..d63d2235d7 100644 --- a/lib/compiler/test/bs_match_SUITE.erl +++ b/lib/compiler/test/bs_match_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2011. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -33,7 +33,7 @@ matching_meets_construction/1,simon/1,matching_and_andalso/1, otp_7188/1,otp_7233/1,otp_7240/1,otp_7498/1, match_string/1,zero_width/1,bad_size/1,haystack/1, - cover_beam_bool/1,matched_out_size/1]). + cover_beam_bool/1,matched_out_size/1,follow_fail_branch/1]). -export([coverage_id/1,coverage_external_ignore/2]). @@ -57,7 +57,7 @@ groups() -> matching_meets_construction,simon, matching_and_andalso,otp_7188,otp_7233,otp_7240, otp_7498,match_string,zero_width,bad_size,haystack, - cover_beam_bool,matched_out_size]}]. + cover_beam_bool,matched_out_size,follow_fail_branch]}]. init_per_suite(Config) -> @@ -1108,6 +1108,32 @@ mos_bin(<<L,Bin:L/binary,"abcdefghij">>) -> L = byte_size(Bin), Bin. +follow_fail_branch(_) -> + 42 = ffb_1(<<0,1>>, <<0>>), + 8 = ffb_1(<<0,1>>, [a]), + 42 = ffb_2(<<0,1>>, <<0>>, 17), + 8 = ffb_2(<<0,1>>, [a], 0), + ok. + +ffb_1(<<_,T/bitstring>>, List) -> + case List of + <<_>> -> + 42; + [_|_] -> + %% The fail branch of the bs_start_match2 instruction + %% pointing to here would be ignored, making the compiler + %% incorrectly assume that the delayed sub-binary + %% optimization was safe. + bit_size(T) + end. + +ffb_2(<<_,T/bitstring>>, List, A) -> + case List of + <<_>> when A =:= 17 -> 42; + [_|_] -> bit_size(T) + end. + + check(F, R) -> R = F(). diff --git a/lib/crypto/c_src/Makefile.in b/lib/crypto/c_src/Makefile.in index ffd556ca1a..e19d6617f3 100644 --- a/lib/crypto/c_src/Makefile.in +++ b/lib/crypto/c_src/Makefile.in @@ -59,8 +59,6 @@ TYPE_FLAGS = $(CFLAGS) endif endif -ALL_CFLAGS = $(TYPE_FLAGS) $(INCLUDES) - # ---------------------------------------------------- # Release directory specification # ---------------------------------------------------- @@ -69,13 +67,16 @@ RELSYSDIR = $(RELEASE_PATH)/lib/crypto-$(VSN) # ---------------------------------------------------- # Misc Macros # ---------------------------------------------------- -OBJS = $(OBJDIR)/crypto$(TYPEMARKER).o +CRYPTO_OBJS = $(OBJDIR)/crypto$(TYPEMARKER).o +CALLBACK_OBJS = $(OBJDIR)/crypto_callback$(TYPEMARKER).o NIF_MAKEFILE = $(PRIVDIR)/Makefile ifeq ($(findstring win32,$(TARGET)), win32) NIF_LIB = $(LIBDIR)/crypto$(TYPEMARKER).dll +CALLBACK_LIB = $(LIBDIR)/crypto_callback$(TYPEMARKER).dll else NIF_LIB = $(LIBDIR)/crypto$(TYPEMARKER).so +CALLBACK_LIB = $(LIBDIR)/crypto_callback$(TYPEMARKER).so endif ifeq ($(HOST_OS),) @@ -86,43 +87,69 @@ DYNAMIC_CRYPTO_LIB=@SSL_DYNAMIC_ONLY@ ifeq ($(DYNAMIC_CRYPTO_LIB),yes) SSL_DED_LD_RUNTIME_LIBRARY_PATH = @SSL_DED_LD_RUNTIME_LIBRARY_PATH@ CRYPTO_LINK_LIB=$(SSL_DED_LD_RUNTIME_LIBRARY_PATH) -L$(SSL_LIBDIR) -l$(SSL_CRYPTO_LIBNAME) +EXTRA_FLAGS = -DHAVE_DYNAMIC_CRYPTO_LIB else SSL_DED_LD_RUNTIME_LIBRARY_PATH= CRYPTO_LINK_LIB=$(SSL_LIBDIR)/lib$(SSL_CRYPTO_LIBNAME).a +EXTRA_FLAGS = +CRYPTO_OBJS := $(CRYPTO_OBJS) $(CALLBACK_OBJS) +CALLBACK_OBJS = +CALLBACK_LIB = endif +ALL_CFLAGS = $(TYPE_FLAGS) $(EXTRA_FLAGS) $(INCLUDES) + # ---------------------------------------------------- # Targets # ---------------------------------------------------- _create_dirs := $(shell mkdir -p $(OBJDIR) $(LIBDIR)) -debug opt valgrind: $(NIF_LIB) +debug opt valgrind: $(NIF_LIB) $(CALLBACK_LIB) $(OBJDIR)/%$(TYPEMARKER).o: %.c $(INSTALL_DIR) $(OBJDIR) $(CC) -c -o $@ $(ALL_CFLAGS) $< -$(LIBDIR)/crypto$(TYPEMARKER).so: $(OBJS) - $(INSTALL_DIR) $(LIBDIR) +$(LIBDIR)/crypto$(TYPEMARKER).so: $(CRYPTO_OBJS) + $(INSTALL_DIR) $(LIBDIR) $(LD) $(LDFLAGS) -o $@ $^ $(LDLIBS) $(CRYPTO_LINK_LIB) -$(LIBDIR)/crypto$(TYPEMARKER).dll: $(OBJS) +$(LIBDIR)/crypto$(TYPEMARKER).dll: $(CRYPTO_OBJS) + $(INSTALL_DIR) $(LIBDIR) + $(LD) $(LDFLAGS) -o $@ $(SSL_DED_LD_RUNTIME_LIBRARY_PATH) -L$(SSL_LIBDIR) $(CRYPTO_OBJS) -l$(SSL_CRYPTO_LIBNAME) -l$(SSL_SSL_LIBNAME) + +ifeq ($(DYNAMIC_CRYPTO_LIB),yes) +$(LIBDIR)/crypto_callback$(TYPEMARKER).so: $(CALLBACK_OBJS) + $(INSTALL_DIR) $(LIBDIR) + $(LD) $(LDFLAGS) -o $@ $^ $(LDLIBS) + +$(LIBDIR)/crypto_callback$(TYPEMARKER).dll: $(CALLBACK_OBJS) $(INSTALL_DIR) $(LIBDIR) - $(LD) $(LDFLAGS) -o $@ $(SSL_DED_LD_RUNTIME_LIBRARY_PATH) -L$(SSL_LIBDIR) $(OBJS) -l$(SSL_CRYPTO_LIBNAME) -l$(SSL_SSL_LIBNAME) + $(LD) $(LDFLAGS) -o $@ $(CALLBACK_OBJS) +endif + clean: ifeq ($(findstring win32,$(TARGET)), win32) rm -f $(LIBDIR)/crypto.dll rm -f $(LIBDIR)/crypto.debug.dll + rm -f $(LIBDIR)/crypto_callback.dll + rm -f $(LIBDIR)/crypto_callback.debug.dll else rm -f $(LIBDIR)/crypto.so rm -f $(LIBDIR)/crypto.debug.so rm -f $(LIBDIR)/crypto.valgrind.so + rm -f $(LIBDIR)/crypto_callback.so + rm -f $(LIBDIR)/crypto_callback.debug.so + rm -f $(LIBDIR)/crypto_callback.valgrind.so endif rm -f $(OBJDIR)/crypto.o rm -f $(OBJDIR)/crypto.debug.o rm -f $(OBJDIR)/crypto.valgrind.o + rm -f $(OBJDIR)/crypto_callback.o + rm -f $(OBJDIR)/crypto_callback.debug.o + rm -f $(OBJDIR)/crypto_callback.valgrind.o rm -f core *~ docs: @@ -136,8 +163,12 @@ release_spec: opt $(INSTALL_DIR) "$(RELSYSDIR)/priv/obj" $(INSTALL_DIR) "$(RELSYSDIR)/priv/lib" $(INSTALL_DATA) $(NIF_MAKEFILE) "$(RELSYSDIR)/priv/obj" - $(INSTALL_PROGRAM) $(OBJS) "$(RELSYSDIR)/priv/obj" + $(INSTALL_PROGRAM) $(CRYPTO_OBJS) "$(RELSYSDIR)/priv/obj" $(INSTALL_PROGRAM) $(NIF_LIB) "$(RELSYSDIR)/priv/lib" +ifeq ($(DYNAMIC_CRYPTO_LIB),yes) + $(INSTALL_PROGRAM) $(CALLBACK_OBJS) "$(RELSYSDIR)/priv/obj" + $(INSTALL_PROGRAM) $(CALLBACK_LIB) "$(RELSYSDIR)/priv/lib" +endif release_docs_spec: diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c index 91ab244620..72c9e5b8e8 100644 --- a/lib/crypto/c_src/crypto.c +++ b/lib/crypto/c_src/crypto.c @@ -53,6 +53,8 @@ #include <openssl/evp.h> #include <openssl/hmac.h> +#include "crypto_callback.h" + #if OPENSSL_VERSION_NUMBER >= 0x00908000L && !defined(OPENSSL_NO_SHA224) && defined(NID_sha224)\ && !defined(OPENSSL_NO_SHA256) /* disabled like this in my sha.h (?) */ # define HAVE_SHA224 @@ -67,6 +69,9 @@ #if OPENSSL_VERSION_NUMBER >= 0x00908000L && !defined(OPENSSL_NO_SHA512) && defined(NID_sha512) # define HAVE_SHA512 #endif +#if OPENSSL_VERSION_NUMBER >= 0x0090705FL +# define HAVE_DES_ede3_cfb_encrypt +#endif #ifdef VALGRIND # include <valgrind/memcheck.h> @@ -125,7 +130,6 @@ /* NIF interface declarations */ static int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info); -static int reload(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info); static int upgrade(ErlNifEnv* env, void** priv_data, void** old_priv_data, ERL_NIF_TERM load_info); static void unload(ErlNifEnv* env, void* priv_data); @@ -172,7 +176,7 @@ static ERL_NIF_TERM des_cbc_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM a static ERL_NIF_TERM des_cfb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM des_ecb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM des_ede3_cbc_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM des_ede3_cfb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +static ERL_NIF_TERM des_ede3_cfb_crypt_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM aes_cfb_128_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM aes_ctr_encrypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM aes_ctr_stream_encrypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); @@ -204,17 +208,6 @@ static ERL_NIF_TERM bf_ecb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM ar static ERL_NIF_TERM blowfish_ofb64_encrypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); -/* openssl callbacks */ -#ifdef OPENSSL_THREADS -static void locking_function(int mode, int n, const char *file, int line); -static unsigned long id_function(void); -static struct CRYPTO_dynlock_value* dyn_create_function(const char *file, - int line); -static void dyn_lock_function(int mode, struct CRYPTO_dynlock_value* ptr, - const char *file, int line); -static void dyn_destroy_function(struct CRYPTO_dynlock_value *ptr, - const char *file, int line); -#endif /* OPENSSL_THREADS */ /* helpers */ static void init_digest_types(ErlNifEnv* env); @@ -291,7 +284,7 @@ static ErlNifFunc nif_funcs[] = { {"des_cfb_crypt", 4, des_cfb_crypt}, {"des_ecb_crypt", 3, des_ecb_crypt}, {"des_ede3_cbc_crypt", 6, des_ede3_cbc_crypt}, - {"des_ede3_cfb_crypt", 6, des_ede3_cfb_crypt}, + {"des_ede3_cfb_crypt_nif", 6, des_ede3_cfb_crypt_nif}, {"aes_cfb_128_crypt", 4, aes_cfb_128_crypt}, {"aes_ctr_encrypt", 3, aes_ctr_encrypt}, {"aes_ctr_decrypt", 3, aes_ctr_encrypt}, @@ -325,7 +318,7 @@ static ErlNifFunc nif_funcs[] = { {"blowfish_ofb64_encrypt", 3, blowfish_ofb64_encrypt} }; -ERL_NIF_INIT(crypto,nif_funcs,load,reload,upgrade,unload) +ERL_NIF_INIT(crypto,nif_funcs,load,NULL,upgrade,unload) #define MD5_CTX_LEN (sizeof(MD5_CTX)) @@ -347,7 +340,6 @@ ERL_NIF_INIT(crypto,nif_funcs,load,reload,upgrade,unload) #define HMAC_OPAD 0x5c -static ErlNifRWLock** lock_vec = NULL; /* Static locks used by openssl */ static ERL_NIF_TERM atom_true; static ERL_NIF_TERM atom_false; static ERL_NIF_TERM atom_sha; @@ -374,55 +366,60 @@ static ERL_NIF_TERM atom_none; static ERL_NIF_TERM atom_notsup; static ERL_NIF_TERM atom_digest; +/* +#define PRINTF_ERR0(FMT) enif_fprintf(stderr, FMT "\n") +#define PRINTF_ERR1(FMT, A1) enif_fprintf(stderr, FMT "\n", A1) +*/ +#define PRINTF_ERR0(FMT) +#define PRINTF_ERR1(FMT,A1) -static int is_ok_load_info(ErlNifEnv* env, ERL_NIF_TERM load_info) +#ifdef HAVE_DYNAMIC_CRYPTO_LIB +static int change_basename(char* buf, int bufsz, const char* newfile) { - int i; - return enif_get_int(env,load_info,&i) && i == 101; -} -static void* crypto_alloc(size_t size) -{ - return enif_alloc(size); + char* p = strrchr(buf, '/'); + p = (p == NULL) ? buf : p + 1; + + if ((p - buf) + strlen(newfile) >= bufsz) { + PRINTF_ERR0("CRYPTO: lib name too long"); + return 0; + } + strcpy(p, newfile); + return 1; } -static void* crypto_realloc(void* ptr, size_t size) + +static void error_handler(void* null, const char* errstr) { - return enif_realloc(ptr, size); -} -static void crypto_free(void* ptr) -{ - enif_free(ptr); + PRINTF_ERR1("CRYPTO LOADING ERROR: '%s'", errstr); } +#endif /* HAVE_DYNAMIC_CRYPTO_LIB */ -static int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) +static int init(ErlNifEnv* env, ERL_NIF_TERM load_info) { ErlNifSysInfo sys_info; - CRYPTO_set_mem_functions(crypto_alloc, crypto_realloc, crypto_free); - - if (!is_ok_load_info(env, load_info)) { - return -1; + get_crypto_callbacks_t* funcp; + struct crypto_callbacks* ccb; + int nlocks = 0; + int tpl_arity; + const ERL_NIF_TERM* tpl_array; + int vernum; + char lib_buf[1000]; + + /* load_info: {201, "/full/path/of/this/library"} */ + if (!enif_get_tuple(env, load_info, &tpl_arity, &tpl_array) + || tpl_arity != 2 + || !enif_get_int(env, tpl_array[0], &vernum) + || vernum != 201 + || enif_get_string(env, tpl_array[1], lib_buf, sizeof(lib_buf), ERL_NIF_LATIN1) <= 0) { + + PRINTF_ERR1("CRYPTO: Invalid load_info '%T'", load_info); + return 0; } - -#ifdef OPENSSL_THREADS - enif_system_info(&sys_info, sizeof(sys_info)); - - if (sys_info.scheduler_threads > 1) { - int i; - lock_vec = enif_alloc(CRYPTO_num_locks()*sizeof(*lock_vec)); - if (lock_vec==NULL) return -1; - memset(lock_vec,0,CRYPTO_num_locks()*sizeof(*lock_vec)); - - for (i=CRYPTO_num_locks()-1; i>=0; --i) { - lock_vec[i] = enif_rwlock_create("crypto_stat"); - if (lock_vec[i]==NULL) return -1; - } - CRYPTO_set_locking_callback(locking_function); - CRYPTO_set_id_callback(id_function); - CRYPTO_set_dynlock_create_callback(dyn_create_function); - CRYPTO_set_dynlock_lock_callback(dyn_lock_function); - CRYPTO_set_dynlock_destroy_callback(dyn_destroy_function); + if (library_refc > 0) { + /* Repeated loading of this library (module upgrade). + * Atoms and callbacks are already set, we are done. + */ + return 1; } - /* else no need for locks */ -#endif /* OPENSSL_THREADS */ atom_true = enif_make_atom(env,"true"); atom_false = enif_make_atom(env,"false"); @@ -451,37 +448,75 @@ static int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) init_digest_types(env); - *priv_data = NULL; - library_refc++; - return 0; -} - -static int reload(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) -{ - if (*priv_data != NULL) { - return -1; /* Don't know how to do that */ +#ifdef HAVE_DYNAMIC_CRYPTO_LIB + { + void* handle; + if (!change_basename(lib_buf, sizeof(lib_buf), "crypto_callback")) { + return 0; + } + if (!(handle = enif_dlopen(lib_buf, &error_handler, NULL))) { + return 0; + } + if (!(funcp = (get_crypto_callbacks_t*) enif_dlsym(handle, "get_crypto_callbacks", + &error_handler, NULL))) { + return 0; + } } - if (library_refc == 0) { - /* No support for real library upgrade. The tricky thing is to know - when to (re)set the callbacks for allocation and locking. */ - return -2; +#else /* !HAVE_DYNAMIC_CRYPTO_LIB */ + funcp = &get_crypto_callbacks; +#endif + +#ifdef OPENSSL_THREADS + enif_system_info(&sys_info, sizeof(sys_info)); + if (sys_info.scheduler_threads > 1) { + nlocks = CRYPTO_num_locks(); } - if (!is_ok_load_info(env, load_info)) { + /* else no need for locks */ +#endif + + ccb = (*funcp)(nlocks); + + if (!ccb || ccb->sizeof_me != sizeof(*ccb)) { + PRINTF_ERR0("Invalid 'crypto_callbacks'"); + return 0; + } + + CRYPTO_set_mem_functions(ccb->crypto_alloc, ccb->crypto_realloc, ccb->crypto_free); + +#ifdef OPENSSL_THREADS + if (nlocks > 0) { + CRYPTO_set_locking_callback(ccb->locking_function); + CRYPTO_set_id_callback(ccb->id_function); + CRYPTO_set_dynlock_create_callback(ccb->dyn_create_function); + CRYPTO_set_dynlock_lock_callback(ccb->dyn_lock_function); + CRYPTO_set_dynlock_destroy_callback(ccb->dyn_destroy_function); + } +#endif /* OPENSSL_THREADS */ + return 1; +} + +static int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) +{ + if (!init(env, load_info)) { return -1; } - return 0; + + *priv_data = NULL; + library_refc++; + return 0; } static int upgrade(ErlNifEnv* env, void** priv_data, void** old_priv_data, ERL_NIF_TERM load_info) { - int i; if (*old_priv_data != NULL) { return -1; /* Don't know how to do that */ } - i = reload(env,priv_data,load_info); - if (i != 0) { - return i; + if (*priv_data != NULL) { + return -1; /* Don't know how to do that */ + } + if (!init(env, load_info)) { + return -1; } library_refc++; return 0; @@ -489,20 +524,7 @@ static int upgrade(ErlNifEnv* env, void** priv_data, void** old_priv_data, static void unload(ErlNifEnv* env, void* priv_data) { - if (--library_refc <= 0) { - CRYPTO_cleanup_all_ex_data(); - - if (lock_vec != NULL) { - int i; - for (i=CRYPTO_num_locks()-1; i>=0; --i) { - if (lock_vec[i] != NULL) { - enif_rwlock_destroy(lock_vec[i]); - } - } - enif_free(lock_vec); - } - } - /*else NIF library still used by other (new) module code */ + --library_refc; } static ERL_NIF_TERM info_lib(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) @@ -514,12 +536,21 @@ static ERL_NIF_TERM info_lib(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[] const char* ver = SSLeay_version(SSLEAY_VERSION); unsigned ver_sz = strlen(ver); ERL_NIF_TERM name_term, ver_term; + int ver_num = OPENSSL_VERSION_NUMBER; + /* R16: + * Ignore library version number from SSLeay() and instead show header + * version. Otherwise user might try to call a function that is implemented + * by a newer library but not supported by the headers used at compile time. + * Example: DES_ede3_cfb_encrypt in 0.9.7i but not in 0.9.7d. + * + * Version string is still from library though. + */ memcpy(enif_make_new_binary(env, name_sz, &name_term), libname, name_sz); memcpy(enif_make_new_binary(env, ver_sz, &ver_term), ver, ver_sz); return enif_make_list1(env, enif_make_tuple3(env, name_term, - enif_make_int(env, SSLeay()), + enif_make_int(env, ver_num), ver_term)); } @@ -1199,8 +1230,9 @@ static ERL_NIF_TERM des_ede3_cbc_crypt(ErlNifEnv* env, int argc, const ERL_NIF_T return ret; } -static ERL_NIF_TERM des_ede3_cfb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +static ERL_NIF_TERM des_ede3_cfb_crypt_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (Key1, Key2, Key3, IVec, Text/Cipher, IsEncrypt) */ +#ifdef HAVE_DES_ede3_cfb_encrypt ErlNifBinary key1, key2, key3, ivec, text; DES_key_schedule schedule1, schedule2, schedule3; DES_cblock ivec_clone; /* writable copy */ @@ -1222,6 +1254,9 @@ static ERL_NIF_TERM des_ede3_cfb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_T 8, text.size, &schedule1, &schedule2, &schedule3, &ivec_clone, (argv[5] == atom_true)); return ret; +#else + return atom_notsup; +#endif } static ERL_NIF_TERM aes_cfb_128_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) @@ -2338,59 +2373,6 @@ static ERL_NIF_TERM blowfish_ofb64_encrypt(ErlNifEnv* env, int argc, const ERL_N -#ifdef OPENSSL_THREADS /* vvvvvvvvvvvvvvv OPENSSL_THREADS vvvvvvvvvvvvvvvv */ - -static INLINE void locking(int mode, ErlNifRWLock* lock) -{ - switch (mode) { - case CRYPTO_LOCK|CRYPTO_READ: - enif_rwlock_rlock(lock); - break; - case CRYPTO_LOCK|CRYPTO_WRITE: - enif_rwlock_rwlock(lock); - break; - case CRYPTO_UNLOCK|CRYPTO_READ: - enif_rwlock_runlock(lock); - break; - case CRYPTO_UNLOCK|CRYPTO_WRITE: - enif_rwlock_rwunlock(lock); - break; - default: - ASSERT(!"Invalid lock mode"); - } -} - -/* Callback from openssl for static locking - */ -static void locking_function(int mode, int n, const char *file, int line) -{ - ASSERT(n>=0 && n<CRYPTO_num_locks()); - - locking(mode, lock_vec[n]); -} - -/* Callback from openssl for thread id - */ -static unsigned long id_function(void) -{ - return(unsigned long) enif_thread_self(); -} - -/* Callbacks for dynamic locking, not used by current openssl version (0.9.8) - */ -static struct CRYPTO_dynlock_value* dyn_create_function(const char *file, int line) { - return(struct CRYPTO_dynlock_value*) enif_rwlock_create("crypto_dyn"); -} -static void dyn_lock_function(int mode, struct CRYPTO_dynlock_value* ptr,const char *file, int line) -{ - locking(mode, (ErlNifRWLock*)ptr); -} -static void dyn_destroy_function(struct CRYPTO_dynlock_value *ptr, const char *file, int line) -{ - enif_rwlock_destroy((ErlNifRWLock*)ptr); -} - -#endif /* ^^^^^^^^^^^^^^^^^^^^^^ OPENSSL_THREADS ^^^^^^^^^^^^^^^^^^^^^^ */ /* HMAC */ diff --git a/lib/crypto/c_src/crypto_callback.c b/lib/crypto/c_src/crypto_callback.c new file mode 100644 index 0000000000..81106b4cc2 --- /dev/null +++ b/lib/crypto/c_src/crypto_callback.c @@ -0,0 +1,165 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +#include <string.h> +#include <openssl/opensslconf.h> + +#include "erl_nif.h" +#include "crypto_callback.h" + +#ifdef DEBUG + # define ASSERT(e) \ + ((void) ((e) ? 1 : (fprintf(stderr,"Assert '%s' failed at %s:%d\n",\ + #e, __FILE__, __LINE__), abort(), 0))) +#else + # define ASSERT(e) ((void) 1) +#endif + +#ifdef __GNUC__ + # define INLINE __inline__ +#elif defined(__WIN32__) + # define INLINE __forceinline +#else + # define INLINE +#endif + +#ifdef __WIN32__ +# define DLLEXPORT __declspec(dllexport) +#else +# define DLLEXPORT +#endif + +/* to be dlsym'ed */ +DLLEXPORT struct crypto_callbacks* get_crypto_callbacks(int nlocks); + + +static ErlNifRWLock** lock_vec = NULL; /* Static locks used by openssl */ + +static void* crypto_alloc(size_t size) +{ + return enif_alloc(size); +} +static void* crypto_realloc(void* ptr, size_t size) +{ + return enif_realloc(ptr, size); +} +static void crypto_free(void* ptr) +{ + enif_free(ptr); +} + + +#ifdef OPENSSL_THREADS /* vvvvvvvvvvvvvvv OPENSSL_THREADS vvvvvvvvvvvvvvvv */ + +#include <openssl/crypto.h> + +static INLINE void locking(int mode, ErlNifRWLock* lock) +{ + switch (mode) { + case CRYPTO_LOCK|CRYPTO_READ: + enif_rwlock_rlock(lock); + break; + case CRYPTO_LOCK|CRYPTO_WRITE: + enif_rwlock_rwlock(lock); + break; + case CRYPTO_UNLOCK|CRYPTO_READ: + enif_rwlock_runlock(lock); + break; + case CRYPTO_UNLOCK|CRYPTO_WRITE: + enif_rwlock_rwunlock(lock); + break; + default: + ASSERT(!"Invalid lock mode"); + } +} + +static void locking_function(int mode, int n, const char *file, int line) +{ + ASSERT(n>=0 && n<CRYPTO_num_locks()); + + locking(mode, lock_vec[n]); +} + +static unsigned long id_function(void) +{ + return (unsigned long) enif_thread_self(); +} + +/* Dynamic locking, not used by current openssl version (0.9.8) + */ +static struct CRYPTO_dynlock_value* dyn_create_function(const char *file, int line) +{ + return (struct CRYPTO_dynlock_value*) enif_rwlock_create("crypto_dyn"); +} +static void dyn_lock_function(int mode, struct CRYPTO_dynlock_value* ptr,const char *file, int line) +{ + locking(mode, (ErlNifRWLock*)ptr); +} +static void dyn_destroy_function(struct CRYPTO_dynlock_value *ptr, const char *file, int line) +{ + enif_rwlock_destroy((ErlNifRWLock*)ptr); +} + +#endif /* ^^^^^^^^^^^^^^^^^^^^^^ OPENSSL_THREADS ^^^^^^^^^^^^^^^^^^^^^^ */ + +DLLEXPORT struct crypto_callbacks* get_crypto_callbacks(int nlocks) +{ + static int is_initialized = 0; + static struct crypto_callbacks the_struct = { + sizeof(struct crypto_callbacks), + + &crypto_alloc, + &crypto_realloc, + &crypto_free, + +#ifdef OPENSSL_THREADS + &locking_function, + &id_function, + &dyn_create_function, + &dyn_lock_function, + &dyn_destroy_function +#endif /* OPENSSL_THREADS */ + }; + + if (!is_initialized) { +#ifdef OPENSSL_THREADS + if (nlocks > 0) { + int i; + lock_vec = enif_alloc(nlocks*sizeof(*lock_vec)); + if (lock_vec==NULL) return NULL; + memset(lock_vec, 0, nlocks*sizeof(*lock_vec)); + + for (i=nlocks-1; i>=0; --i) { + lock_vec[i] = enif_rwlock_create("crypto_stat"); + if (lock_vec[i]==NULL) return NULL; + } + } +#endif + is_initialized = 1; + } + return &the_struct; +} + +#ifdef HAVE_DYNAMIC_CRYPTO_LIB +/* This is not really a NIF library, but we use ERL_NIF_INIT in order to + * get access to the erl_nif API (on Windows). + */ +ERL_NIF_INIT(dummy, (ErlNifFunc*)NULL , NULL, NULL, NULL, NULL) +#endif + diff --git a/lib/crypto/c_src/crypto_callback.h b/lib/crypto/c_src/crypto_callback.h new file mode 100644 index 0000000000..23ecba3e5d --- /dev/null +++ b/lib/crypto/c_src/crypto_callback.h @@ -0,0 +1,46 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +struct crypto_callbacks +{ + size_t sizeof_me; + + void* (*crypto_alloc)(size_t size); + void* (*crypto_realloc)(void* ptr, size_t size); + void (*crypto_free)(void* ptr); + + /* openssl callbacks */ + #ifdef OPENSSL_THREADS + void (*locking_function)(int mode, int n, const char *file, int line); + unsigned long (*id_function)(void); + struct CRYPTO_dynlock_value* (*dyn_create_function)(const char *file, + int line); + void (*dyn_lock_function)(int mode, struct CRYPTO_dynlock_value* ptr, + const char *file, int line); + void (*dyn_destroy_function)(struct CRYPTO_dynlock_value *ptr, + const char *file, int line); + #endif /* OPENSSL_THREADS */ +}; + +typedef struct crypto_callbacks* get_crypto_callbacks_t(int nlocks); + +#ifndef HAVE_DYNAMIC_CRYPTO_LIB +struct crypto_callbacks* get_crypto_callbacks(int nlocks); +#endif + diff --git a/lib/crypto/doc/src/crypto.xml b/lib/crypto/doc/src/crypto.xml index 4dcd6fc4ea..3e533158c8 100644 --- a/lib/crypto/doc/src/crypto.xml +++ b/lib/crypto/doc/src/crypto.xml @@ -115,6 +115,12 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]> > <input>info_lib().</input> [{<<"OpenSSL">>,9469983,<<"OpenSSL 0.9.8a 11 Oct 2005">>}] </pre> + <note><p> + From OTP R16 the <em>numeric version</em> represents the version of the OpenSSL + <em>header files</em> (<c>openssl/opensslv.h</c>) used when crypto was compiled. + The text variant represents the OpenSSL library used at runtime. + In earlier OTP versions both numeric and text was taken from the library. + </p></note> </desc> </func> <func> @@ -265,6 +271,8 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]> </type> <desc> <p>Computes a message digest of type <c>Type</c> from <c>Data</c>.</p> + <p>May throw exception <c>notsup</c> in case the chosen <c>Type</c> + is not supported by the underlying OpenSSL implementation.</p> </desc> </func> <func> @@ -277,6 +285,8 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]> <p>Initializes the context for streaming hash operations. <c>Type</c> determines which digest to use. The returned context should be used as argument to <seealso marker="#hash_update/2">hash_update</seealso>.</p> + <p>May throw exception <c>notsup</c> in case the chosen <c>Type</c> + is not supported by the underlying OpenSSL implementation.</p> </desc> </func> <func> @@ -548,6 +558,8 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]> keys, and <c>IVec</c> is an arbitrary initializing vector. The lengths of each of <c>Key1</c>, <c>Key2</c>, <c>Key3</c> and <c>IVec</c> must be 64 bits (8 bytes).</p> + <p>May throw exception <c>notsup</c> for old OpenSSL + versions (0.9.7) that does not support this encryption mode.</p> </desc> </func> <func> @@ -565,6 +577,8 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]> and <c>IVec</c> must have the same values as those used when encrypting. The lengths of <c>Key1</c>, <c>Key2</c>, <c>Key3</c>, and <c>IVec</c> must be 64 bits (8 bytes).</p> + <p>May throw exception <c>notsup</c> for old OpenSSL + versions (0.9.7) that does not support this encryption mode.</p> </desc> </func> diff --git a/lib/crypto/doc/src/crypto_app.xml b/lib/crypto/doc/src/crypto_app.xml index 1c01e3f099..6573a56f4c 100644 --- a/lib/crypto/doc/src/crypto_app.xml +++ b/lib/crypto/doc/src/crypto_app.xml @@ -62,7 +62,7 @@ <section> <title>OpenSSL libraries</title> <p>The current implementation of the Erlang Crypto application is - based on the <em>OpenSSL</em> package version 0.9.7 or higher. + based on the <em>OpenSSL</em> package version 0.9.8 or higher. There are source and binary releases on the web. </p> <p>Source releases of OpenSSL can be downloaded from the <url href="http://www.openssl.org">OpenSSL</url> project home page, diff --git a/lib/crypto/doc/src/notes.xml b/lib/crypto/doc/src/notes.xml index a9f8dd84af..4178ca2b08 100644 --- a/lib/crypto/doc/src/notes.xml +++ b/lib/crypto/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1999</year><year>2011</year> + <year>1999</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/crypto/src/crypto.erl b/lib/crypto/src/crypto.erl index 0089e79a4f..461558a79e 100644 --- a/lib/crypto/src/crypto.erl +++ b/lib/crypto/src/crypto.erl @@ -78,12 +78,11 @@ md5_mac, md5_mac_96, sha_mac, sha_mac_96, sha224_mac, sha256_mac, sha384_mac, sha512_mac, - sha_mac_init, sha_mac_update, sha_mac_final, des_cbc_encrypt, des_cbc_decrypt, des_cfb_encrypt, des_cfb_decrypt, des_ecb_encrypt, des_ecb_decrypt, - des_ede3_cbc_encrypt, des_ede3_cbc_decrypt, - des_ede3_cfb_encrypt, des_ede3_cfb_decrypt, + des3_cbc_encrypt, des3_cbc_decrypt, + des3_cfb_encrypt, des3_cfb_decrypt, aes_cfb_128_encrypt, aes_cfb_128_decrypt, rand_bytes, strong_rand_bytes, @@ -103,6 +102,13 @@ aes_cbc_256_encrypt, aes_cbc_256_decrypt, aes_ctr_encrypt, aes_ctr_decrypt, aes_ctr_stream_init, aes_ctr_stream_encrypt, aes_ctr_stream_decrypt, + aes_cbc_ivec, blowfish_cbc_encrypt, blowfish_cbc_decrypt, + blowfish_cfb64_encrypt, blowfish_cfb64_decrypt, + blowfish_ecb_encrypt, blowfish_ecb_decrypt, blowfish_ofb64_encrypt, + des_cbc_ivec, des_cfb_ivec, erlint, mpint, + hash, hash_init, hash_update, hash_final, + hmac_init, hmac_update, hmac_final, hmac_final_n, info, + rc2_cbc_encrypt, rc2_cbc_decrypt, info_lib]). -type rsa_digest_type() :: 'md5' | 'sha' | 'sha224' | 'sha256' | 'sha384' | 'sha512'. @@ -114,7 +120,7 @@ -on_load(on_load/0). --define(CRYPTO_NIF_VSN,101). +-define(CRYPTO_NIF_VSN,201). on_load() -> LibBaseName = "crypto", @@ -140,7 +146,7 @@ on_load() -> end end, Lib = filename:join([PrivDir, "lib", LibName]), - Status = case erlang:load_nif(Lib, ?CRYPTO_NIF_VSN) of + Status = case erlang:load_nif(Lib, {?CRYPTO_NIF_VSN,Lib}) of ok -> ok; {error, {load_failed, _}}=Error1 -> ArchLibDir = @@ -152,7 +158,7 @@ on_load() -> [] -> Error1; _ -> ArchLib = filename:join([ArchLibDir, LibName]), - erlang:load_nif(ArchLib, ?CRYPTO_NIF_VSN) + erlang:load_nif(ArchLib, {?CRYPTO_NIF_VSN,ArchLib}) end; Error1 -> Error1 end, @@ -597,12 +603,12 @@ des_ecb_crypt(_Key, _Data, _IsEncrypt) -> ?nif_stub. binary(). des3_cbc_encrypt(Key1, Key2, Key3, IVec, Data) -> - des_ede3_cbc_encrypt(Key1, Key2, Key3, IVec, Data). + des_ede3_cbc_crypt(Key1, Key2, Key3, IVec, Data, true). des_ede3_cbc_encrypt(Key1, Key2, Key3, IVec, Data) -> des_ede3_cbc_crypt(Key1, Key2, Key3, IVec, Data, true). des3_cbc_decrypt(Key1, Key2, Key3, IVec, Data) -> - des_ede3_cbc_decrypt(Key1, Key2, Key3, IVec, Data). + des_ede3_cbc_crypt(Key1, Key2, Key3, IVec, Data, false). des_ede3_cbc_decrypt(Key1, Key2, Key3, IVec, Data) -> des_ede3_cbc_crypt(Key1, Key2, Key3, IVec, Data, false). @@ -617,16 +623,18 @@ des_ede3_cbc_crypt(_Key1, _Key2, _Key3, _IVec, _Data, _IsEncrypt) -> ?nif_stub. binary(). des3_cfb_encrypt(Key1, Key2, Key3, IVec, Data) -> - des_ede3_cfb_encrypt(Key1, Key2, Key3, IVec, Data). -des_ede3_cfb_encrypt(Key1, Key2, Key3, IVec, Data) -> des_ede3_cfb_crypt(Key1, Key2, Key3, IVec, Data, true). des3_cfb_decrypt(Key1, Key2, Key3, IVec, Data) -> - des_ede3_cfb_decrypt(Key1, Key2, Key3, IVec, Data). -des_ede3_cfb_decrypt(Key1, Key2, Key3, IVec, Data) -> des_ede3_cfb_crypt(Key1, Key2, Key3, IVec, Data, false). -des_ede3_cfb_crypt(_Key1, _Key2, _Key3, _IVec, _Data, _IsEncrypt) -> ?nif_stub. +des_ede3_cfb_crypt(Key1, Key2, Key3, IVec, Data, IsEncrypt) -> + case des_ede3_cfb_crypt_nif(Key1,Key2,Key3,IVec,Data,IsEncrypt) of + notsup -> erlang:error(notsup); + Bin -> Bin + end. + +des_ede3_cfb_crypt_nif(_Key1, _Key2, _Key3, _IVec, _Data, _IsEncrypt) -> ?nif_stub. %% %% Blowfish diff --git a/lib/crypto/test/crypto_SUITE.erl b/lib/crypto/test/crypto_SUITE.erl index 7ac693f371..8965ab6b94 100644 --- a/lib/crypto/test/crypto_SUITE.erl +++ b/lib/crypto/test/crypto_SUITE.erl @@ -87,10 +87,12 @@ groups() -> {rest, [], [md5, md5_update, md4, md4_update, md5_mac, md5_mac_io, sha, sha_update, + sha256, sha256_update, sha512, sha512_update, hmac_update_sha, hmac_update_sha_n, hmac_update_sha256, hmac_update_sha512, hmac_update_md5_n, hmac_update_md5_io, hmac_update_md5, hmac_rfc4231, des_cbc, aes_cfb, aes_cbc, + des_cfb, des_cfb_iter, des3_cbc, des3_cfb, rc2_cbc, aes_cbc_iter, aes_ctr, aes_ctr_stream, des_cbc_iter, des_ecb, rand_uniform_test, strong_rand_test, rsa_verify_test, dsa_verify_test, rsa_sign_test, @@ -192,7 +194,16 @@ info(Config) when is_list(Config) -> {skip,"Missing crypto application"}; {_,_} -> ?line crypto:start(), - ?line crypto:info(), + ?line Info = crypto:info(), + ?line Exports = lists:usort([F || {F,_} <- crypto:module_info(exports)]), + ?line [] = Info -- Exports, + ?line NotInInfo = Exports -- Info, + io:format("NotInInfo = ~p\n", [NotInInfo]), + BlackList = lists:sort([des_ede3_cbc_decrypt, des_ede3_cbc_encrypt, + dh_check, dh_generate_parameters, + module_info, start, stop, version]), + ?line BlackList = NotInInfo, + ?line InfoLib = crypto:info_lib(), ?line [_|_] = InfoLib, F = fun([{Name,VerN,VerS}|T],Me) -> @@ -349,12 +360,8 @@ hmac_update_sha256(doc) -> hmac_update_sha256(suite) -> []; hmac_update_sha256(Config) when is_list(Config) -> - case openssl_version() of - V when V < 16#908000 -> - {skipped,"OpenSSL version too old"}; - _ -> - hmac_update_sha256_do() - end. + if_098(fun() -> hmac_update_sha256_do() end). + hmac_update_sha256_do() -> ?line Key = hexstr2bin("00010203101112132021222330313233" @@ -376,12 +383,7 @@ hmac_update_sha512(doc) -> hmac_update_sha512(suite) -> []; hmac_update_sha512(Config) when is_list(Config) -> - case openssl_version() of - V when V < 16#908000 -> - {skipped,"OpenSSL version too old"}; - _ -> - hmac_update_sha512_do() - end. + if_098(fun() -> hmac_update_sha512_do() end). hmac_update_sha512_do() -> ?line Key = hexstr2bin("00010203101112132021222330313233" @@ -422,12 +424,7 @@ hmac_rfc4231(doc) -> hmac_rfc4231(suite) -> []; hmac_rfc4231(Config) when is_list(Config) -> - case openssl_version() of - V when V < 16#908000 -> - {skipped,"OpenSSL version too old"}; - _ -> - hmac_rfc4231_do() - end. + if_098(fun() -> hmac_rfc4231_do() end). hmac_rfc4231_do() -> %% Test Case 1 @@ -746,6 +743,9 @@ sha256(doc) -> sha256(suite) -> []; sha256(Config) when is_list(Config) -> + if_098(fun() -> sha256_do() end). + +sha256_do() -> ?line m(crypto:sha256("abc"), hexstr2bin("BA7816BF8F01CFEA4141" "40DE5DAE2223B00361A396177A9CB410FF61F20015AD")), @@ -762,6 +762,9 @@ sha256_update(doc) -> sha256_update(suite) -> []; sha256_update(Config) when is_list(Config) -> + if_098(fun() -> sha256_update_do() end). + +sha256_update_do() -> ?line Ctx = crypto:sha256_init(), ?line Ctx1 = crypto:sha256_update(Ctx, "abcdbcdecdefdefgefghfghighi"), ?line Ctx2 = crypto:sha256_update(Ctx1, "jhijkijkljklmklmnlmnomnopnopq"), @@ -778,6 +781,9 @@ sha512(doc) -> sha512(suite) -> []; sha512(Config) when is_list(Config) -> + if_098(fun() -> sha512_do() end). + +sha512_do() -> ?line m(crypto:sha512("abc"), hexstr2bin("DDAF35A193617ABACC417349AE20413112E6FA4E89A97EA2" "0A9EEEE64B55D39A2192992A274FC1A836BA3C23A3FEEBBD" @@ -796,6 +802,9 @@ sha512_update(doc) -> sha512_update(suite) -> []; sha512_update(Config) when is_list(Config) -> + if_098(fun() -> sha512_update_do() end). + +sha512_update_do() -> ?line Ctx = crypto:sha512_init(), ?line Ctx1 = crypto:sha512_update(Ctx, "abcdefghbcdefghicdefghijdefghijkefghijklfghijklmghijklmn"), ?line Ctx2 = crypto:sha512_update(Ctx1, "hijklmnoijklmnopjklmnopqklmnopqrlmnopqrsmnopqrstnopqrstu"), @@ -996,6 +1005,12 @@ des3_cfb(doc) -> des3_cfb(suite) -> []; des3_cfb(Config) when is_list(Config) -> + case openssl_version() of + V when V < 16#90705F -> {skipped,"OpenSSL version too old"}; + _ -> des3_cfb_do() + end. + +des3_cfb_do() -> ?line Key1 = hexstr2bin("0123456789abcdef"), ?line Key2 = hexstr2bin("fedcba9876543210"), ?line Key3 = hexstr2bin("0f2d4b6987a5c3e1"), @@ -1959,3 +1974,10 @@ openssl_version() -> undefined end. +if_098(Fun) -> + case openssl_version() of + V when V < 16#908000 -> + {skipped,"OpenSSL version too old"}; + _ -> + Fun() + end. diff --git a/lib/debugger/src/dbg_wx_settings.erl b/lib/debugger/src/dbg_wx_settings.erl index 3be93c495c..94ecc5781d 100644 --- a/lib/debugger/src/dbg_wx_settings.erl +++ b/lib/debugger/src/dbg_wx_settings.erl @@ -49,31 +49,35 @@ save(Win, Pos, SFile) -> open_win(Win, Pos, SFile, Str, What) -> {SDir, SFileName} = - if - %% If settings are saved for the first time, and to - %% the default directory HOME/erlang.tools/debugger, - %% make sure the directory exists, or create it if - %% desired and possible - SFile==default -> {default_settings_dir(Win), "NoName.state"}; - true -> {filename:dirname(SFile), filename:basename(SFile)} - end, - + if + %% If settings are saved for the first time, and to + %% the default directory HOME/erlang.tools/debugger, + %% make sure the directory exists, or create it if + %% desired and possible + SFile==default -> {default_settings_dir(Win), "NoName.state"}; + true -> {filename:dirname(SFile), filename:basename(SFile)} + end, + FD = wxFileDialog:new(Win, [{message,Str},{pos, Pos}, - {defaultDir,SDir}, - {defaultFile,SFileName}, - {wildCard, "*.state"}, - {style,What}]), + {defaultDir,SDir}, + {defaultFile,SFileName}, + {wildCard, "*.state"}, + {style,What}]), case wxFileDialog:showModal(FD) of - ?wxID_OK -> - File = wxFileDialog:getFilename(FD), - Dir = wxFileDialog:getDirectory(FD), - wxFileDialog:destroy(FD), - {ok, filename:join(Dir,File)}; - _ -> - wxFileDialog:destroy(FD), - cancel + ?wxID_OK -> + case wxFileDialog:getPaths(FD) of + [NewFile] -> + wxFileDialog:destroy(FD), + {ok, NewFile}; + _ -> + wxFileDialog:destroy(FD), + cancel + end; + _ -> + wxFileDialog:destroy(FD), + cancel end. - + default_settings_dir(Win) -> {ok, [[Home]]} = init:get_argument(home), DefDir = filename:join([Home, ".erlang_tools", "debugger"]), diff --git a/lib/dialyzer/doc/src/notes.xml b/lib/dialyzer/doc/src/notes.xml index 0c1556308c..f485e4d03b 100644 --- a/lib/dialyzer/doc/src/notes.xml +++ b/lib/dialyzer/doc/src/notes.xml @@ -31,6 +31,24 @@ <p>This document describes the changes made to the Dialyzer application.</p> +<section><title>Dialyzer 2.5.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fix a crash in race condition detection</p> <p>Remove + old untested experimental extension</p> <p>Respect + {plt_check,false} option when using dialyzer:run/1</p> + <p>Fix handling of tuple set remote types appearing in + tuple sets</p> + <p> + Own Id: OTP-10464</p> + </item> + </list> + </section> + +</section> + <section><title>Dialyzer 2.5.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/dialyzer/vsn.mk b/lib/dialyzer/vsn.mk index 64d05156b1..f344fe0604 100644 --- a/lib/dialyzer/vsn.mk +++ b/lib/dialyzer/vsn.mk @@ -1 +1 @@ -DIALYZER_VSN = 2.5.2 +DIALYZER_VSN = 2.5.3 diff --git a/lib/diameter/doc/src/diameter.xml b/lib/diameter/doc/src/diameter.xml index bc42b75c7a..b7669b760b 100644 --- a/lib/diameter/doc/src/diameter.xml +++ b/lib/diameter/doc/src/diameter.xml @@ -1,5 +1,11 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY make_ref + '<seealso marker="erts:erlang#make_ref-0">erlang:make_ref/0</seealso>'> + <!ENTITY transport_module + '<seealso marker="diameter_transport">transport module</seealso>'> + <!ENTITY dictionary + '<seealso marker="diameter_dict">dictionary</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -50,7 +56,7 @@ under the License. <p> This module provides the interface with which a user can implement a Diameter node that sends and receives messages using the -Diameter protocol as defined in RFC 3588.</p> +Diameter protocol as defined in &the_rfc;.</p> <p> Basic usage consists of creating a representation of a @@ -67,7 +73,7 @@ Beware the difference between <em>diameter</em> (not capitalised) and <em>Diameter</em> (capitalised). The former refers to the Erlang application named diameter whose main api is defined here, the latter to Diameter protocol in the sense of -RFC 3588.</p> +&the_rfc;.</p> <p> The diameter application must be started before calling most functions @@ -91,7 +97,7 @@ in this module.</p> <tag><c>UTF8String()</c></tag> <item> <p> -Types corresponding to RFC 3588 AVP Data Formats. +Types corresponding to &the_rfc; AVP Data Formats. Defined in &dict_data_types;.</p> <marker id="application_alias"/> @@ -298,7 +304,7 @@ Has one of the following types.</p> <item> <p> An address list is available to the start function of a -<seealso marker="diameter_transport">transport module</seealso>, which +&transport_module;, which can return a new list for use in the subsequent CER or CEA. Host-IP-Address need not be specified if the transport start function returns an address list.</p> @@ -312,7 +318,7 @@ returns an address list.</p> Origin-State-Id is optional but will be included in outgoing messages sent by diameter itself: CER/CEA, DWR/DWA and DPR/DPA. Setting a value of <c>0</c> (zero) is equivalent to not setting a -value as documented in RFC 3588. +value as documented in &the_rfc;. The function &origin_state_id; can be used as to retrieve a value that is computed when the diameter application is started.</p> @@ -464,7 +470,7 @@ that matches no peer.</p> <p> The <c>host</c> and <c>realm</c> filters examine the outgoing request as passed to &call;, -assuming that this is a record- or list-valued <c>&app_message;</c>, +assuming that this is a record- or list-valued <c>&codec_message;</c>, and that the message contains at most one of each AVP. If this is not the case then the <c>{host|realm, &dict_DiameterIdentity;}</c> filters must be used to achieve the desired result. @@ -529,8 +535,7 @@ connectivity.</p> <p> Note that a single <c>up</c>/<c>down</c> event for a given peer -corresponds to one -<seealso marker="diameter_app#Mod:peer_up-3">peer_up/peer_down</seealso> +corresponds to one &app_peer_up;/&app_peer_down; callback for each of the Diameter applications negotiated during capablilities exchange. That is, the event communicates connectivity with the @@ -677,7 +682,7 @@ info fields of forms other than the above.</p> The name of a service as passed to &start_service; and with which the service is identified. There can be at most one service with a given name on a given node. -Note that <seealso marker="erts:erlang#make_ref-0">erlang:make_ref/0</seealso> +Note that &make_ref; can be used to generate a service name that is somewhat unique.</p> <marker id="service_opt"/> @@ -703,8 +708,7 @@ For an outgoing Diameter request, the relevant <c>&application_alias;</c> is passed to &call;, while for an incoming request the application identifier in the message header determines the application, the identifier being specified in -the application's <seealso marker="diameter_dict">dictionary</seealso> -file.</p> +the application's &dictionary; file.</p> </item> <tag><c>{restrict_connections, false @@ -749,7 +753,7 @@ as follows.</p> (H bsl N) bor (Id band ((1 bsl N) - 1)) </pre> <p> -Note that RFC 3588 requires that End-to-End identifiers remain unique +Note that &the_rfc; requires that End-to-End identifiers remain unique for a period of at least 4 minutes and that this and the call rate places a lower bound on the appropriate values of <c>N</c>: at a rate of <c>R</c> requests per second an <c>N</c>-bit counter @@ -951,7 +955,7 @@ Tc = &dict_Unsigned32; </pre> <p> -For a connecting transport, the RFC 3588 Tc timer, in milliseconds. +For a connecting transport, the &the_rfc; Tc timer, in milliseconds. Note that this timer determines the frequency with which a transport will attempt to establish a connection with its peer only <em>before</em> an initial connection is established: once there is an initial @@ -1125,7 +1129,7 @@ its transports.</p> <type> <v>SvcName = &service_name;</v> <v>App = &application_alias;</v> -<v>Request = &app_message;</v> +<v>Request = &codec_message;</v> <v>Answer = term()</v> <v>Opt = &call_opt;</v> </type> @@ -1618,7 +1622,7 @@ Return the list of started services.</p> Return a value for a Session-Id AVP.</p> <p> -The value has the form required by section 8.8 of RFC 3588. +The value has the form required by section 8.8 of &the_rfc;. Ident should be the Origin-Host of the peer from which the message containing the returned value will be sent.</p> diff --git a/lib/diameter/doc/src/diameter_app.xml b/lib/diameter/doc/src/diameter_app.xml index 304c69ebda..f4db625c71 100644 --- a/lib/diameter/doc/src/diameter_app.xml +++ b/lib/diameter/doc/src/diameter_app.xml @@ -1,5 +1,8 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY message '<seealso marker="#message">message()</seealso>'> + <!ENTITY dict + '<seealso marker="diameter_dict#MESSAGE_RECORDS">diameter_dict(4)</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -124,39 +127,17 @@ mandatory values as the bare value.</p> <marker id="message"/> -<tag><c>message() = record() | list()</c></tag> +<tag><c>message() = &codec_message;</c></tag> <item> <p> The representation of a Diameter message as passed to -&mod_call;. -The record representation is as outlined in -<seealso -marker="diameter_dict#MESSAGE_RECORDS">diameter_dict(4)</seealso>: -a message as defined in a dictionary file is encoded as a record with -one field for each component AVP. -Equivalently, a message can also be encoded as a list whose head is -the atom-valued message name (the record name minus any -prefix specified in the relevant dictionary file) and whose tail is a -list of <c>{FieldName, FieldValue}</c> pairs.</p> - -<p> -A third representation allows a message to be specified as a list -whose head is a <c>#diameter_header{}</c> record and whose tail is a list -of <c>#diameter_avp{}</c> records. -This representation is used by diameter itself when relaying requests -as directed by the return value of a -&handle_request; -callback. -It differs from the other other two in that it bypasses the checks for -messages that do not agree with their definitions in the dictionary in -question (since relays agents must handle arbitrary request): messages -are sent exactly as specified.</p> +&mod_call; or returned from a &handle_request; callback.</p> </item> <marker id="packet"/> -<tag><c>packet() = #diameter_packet{}</c></tag> +<tag><c>packet() = &codec_packet;</c></tag> <item> <p> A container for incoming and outgoing Diameter messages that's passed @@ -505,8 +486,7 @@ The application in which the callback takes place (that is, the callback module as configured with &mod_start_service;) is determined by the Application Identifier in the header of the incoming request message, the selected module being the one -whose corresponding <seealso -marker="diameter_dict#MESSAGE_RECORDS">dictionary</seealso> declares +whose corresponding dictionary declares itself as defining either the application in question or the Relay application.</p> @@ -526,8 +506,7 @@ The argument &packet; has the following signature.</p> The <c>msg</c> field will be <c>undefined</c> in case the request has been received in the relay application. Otherwise it contains the record representing the request as outlined -in <seealso -marker="diameter_dict#MESSAGE_RECORDS">diameter_dict(4)</seealso>.</p> +in &dict;.</p> <p> The <c>errors</c> field specifies any Result-Code's identifying errors @@ -579,7 +558,7 @@ where <c>Avps</c> sets the Origin-Host, Origin-Realm, the specified Result-Code and (if the request sent one) Session-Id AVP's.</p> <p> -Note that RFC 3588 mandates that only answers with a 3xxx series +Note that &the_rfc; mandates that only answers with a 3xxx series Result-Code (protocol errors) may set the E bit. Returning a non-3xxx value in a <c>protocol_error</c> tuple will cause the request process in question to fail.</p> diff --git a/lib/diameter/doc/src/diameter_codec.xml b/lib/diameter/doc/src/diameter_codec.xml new file mode 100644 index 0000000000..4a77d5435b --- /dev/null +++ b/lib/diameter/doc/src/diameter_codec.xml @@ -0,0 +1,389 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY records + '<seealso marker="diameter_dict#MESSAGE_RECORDS">diameter_dict(4)</seealso>'> + <!ENTITY types + '<seealso marker="diameter_dict#DATA_TYPES">diameter_dict(4)</seealso>'> + <!ENTITY % also SYSTEM "seealso.ent" > + <!ENTITY % here SYSTEM "seehere.ent" > + %also; + %here; +]> + +<erlref> +<header> +<copyright> +<year>2012</year> +<holder>Ericsson AB. All Rights Reserved.</holder> +</copyright> +<legalnotice> +The contents of this file are subject to the Erlang Public License, +Version 1.1, (the "License"); you may not use this file except in +compliance with the License. You should have received a copy of the +Erlang Public License along with this software. If not, it can be +retrieved online at http://www.erlang.org/. + +Software distributed under the License is distributed on an "AS IS" +basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +the License for the specific language governing rights and limitations +under the License. + +</legalnotice> + +<title>diameter_codec(3)</title> +<prepared>Anders Svensson</prepared> +<responsible></responsible> +<docno></docno> +<approved></approved> +<checked></checked> +<date></date> +<rev></rev> +<file>diameter_codec.xml</file> +</header> + +<module>diameter_codec</module> +<modulesummary>Decode and encode of Diameter messages.</modulesummary> + +<description> + +<p> +Incoming Diameter messages are decoded from binary() before being +communicated to &man_app; callbacks. +Similarly, outgoing Diameter messages are encoded into binary() before +being passed to the appropriate &man_transport; module for +transmission. +The functions in this module implement this encode/decode.</p> + +<note> +<p> +Calls to this module are made by diameter itself as a consequence of +configuration passed to &mod_start_service;. +The encode/decode functions may also be useful for other purposes (eg. +test) but the diameter user does not need to call them explicitly when +sending and receiving messages using &mod_call; and the callback +interface documented in &man_app;.</p> +</note> + +<p> +The &header; and &packet; records below +are defined in diameter.hrl, which can be included as follows.</p> + +<pre> +-include_lib("diameter/include/diameter.hrl"). +</pre> + +<p> +Application-specific records are definied in the hrl +files resulting from dictionary file compilation.</p> + +</description> + +<!-- ===================================================================== --> + +<section> +<title>DATA TYPES</title> + +<p></p> + +<taglist> + +<marker id="integers"/> + +<tag><c>uint8() = 0..255</c></tag> +<tag><c>uint24() = 0..16777215</c></tag> +<tag><c>uint32() = 0..4294967295</c></tag> +<item> +<p> +8-bit, 24-bit and 32-bit integers occurring in Diameter and AVP +headers.</p> +</item> + +<marker id="avp"/> + +<tag><c>avp() = #diameter_avp{}</c></tag> +<item> +<p> +The application-neutral representation of an AVP. +Primarily intended for use by relay applications that need to handle +arbitrary Diameter applications. +A service implementing a specific Diameter application +(for which it configures a dictionary) can manipulate values of type +&message; instead.</p> + +<p> +Fields have the following types.</p> + +<taglist> + +<tag><c>code = uint32()</c></tag> +<tag><c>is_mandatory = boolean()</c></tag> +<tag><c>need_encryption = boolean()</c></tag> +<tag><c>vendor_id = uint32() | undefined</c></tag> +<item> +<p> +Values in the AVP header, corresponding to AVP Code, the M flag, P +flags and Vendor-ID respectivelty. +A Vendor-ID other than <c>undefined</c> implies a set V flag.</p> +</item> + +<tag><c>data = iolist()</c></tag> +<item> +<p> +The data bytes of the AVP.</p> +</item> + +<tag><c>name = atom()</c></tag> +<item> +<p> +The name of the AVP as defined in the dictionary file in question, or +<c>undefined</c> if the AVP is unknown to the dictionary file in +question.</p> +</item> + +<tag><c>value = term()</c></tag> +<item> +<p> +The decoded value of an AVP. +Will be <c>undefined</c> on decode if the data bytes could +not be decoded or the AVP is unknown. +The type of a decoded value is as document in &types;.</p> +</item> + +<tag><c>type = atom()</c></tag> +<item> +<p> +The type of the AVP as specified in the dictionary file in question +(or one it inherits). +Possible types are <c>undefined</c> and the Diameter types: +<c>OctetString</c>, <c>Integer32</c>, <c>Integer64</c>, +<c>Unsigned32</c>, <c>Unsigned64</c>, <c>Float32</c>, <c>Float64</c>, +<c>Grouped</c>, <c>Enumerated</c>, <c>Address</c>, <c>Time</c>, +<c>UTF8String</c>, <c>DiameterIdentity</c>, <c>DiameterURI</c>, +<c>IPFilterRule</c> and <c>QoSFilterRule</c>.</p> +</item> + +</taglist> + +</item> + +<marker id="dictionary"/> + +<tag><c>dictionary() = module()</c></tag> +<item> + +<p> +The name of a generated dictionary module as generated by &man_compile; +or &make_codec;. +The interface provided by a dictionary module is an +implementation detail that may change.</p> +</item> + +<marker id="header"/> + +<tag><c>header() = #diameter_header{}</c></tag> +<item> +<p> +The record representation of the Diameter header. +Values in a &packet; returned by &decode; are as extracted from the +incoming message. +Values set in an &packet; passed to &encode; are preserved in the +encoded binary(), with the exception of <c>length</c>, <c>cmd_code</c> +and <c>application_id</c>, all of which are determined by the +&dictionary; in question.</p> + +<note> +<p> +It is not necessary to set header fields explicitly in outgoing +messages as diameter itself will set appropriate values. +Setting inappropriate values can be useful for test purposes.</p> +</note> + +<p> +Fields have the following types.</p> + +<taglist> + +<tag><c>version = uint8()</c></tag> +<tag><c>length = uint24()</c></tag> +<tag><c>cmd_code = uint24()</c></tag> +<tag><c>application_id = uint32()</c></tag> +<tag><c>hop_by_hop_id = uint32()</c></tag> +<tag><c>end_to_end_id = uint32()</c></tag> +<item> +<p> +Values of the Version, Message Length, Command-Code, Application-ID, +Hop-by-Hop Identifier and End-to-End Identifier fields of the Diameter +header.</p> +</item> + +<tag><c>is_request = boolean()</c></tag> +<tag><c>is_proxiable = boolean()</c></tag> +<tag><c>is_error = boolean()</c></tag> +<tag><c>is_retransmitted = boolean()</c></tag> +<item> +<p> +Values correspoding to the R(equest), P(roxiable), E(rror) +and T(Potentially re-transmitted message) flags of the Diameter +header.</p> +</item> + +</taglist> + +</item> + +<marker id="message"/> + +<tag><c>message() = record() | list()</c></tag> +<item> +<p> +The representation of a Diameter message as passed to +&mod_call; or returned from a &app_handle_request; callback. +The record representation is as outlined in &records;: +a message as defined in a dictionary file is encoded as a record with +one field for each component AVP. +Equivalently, a message can also be encoded as a list whose head is +the atom-valued message name (as specified in the relevant dictionary +file) and whose tail is a list of <c>{AvpName, AvpValue}</c> pairs.</p> + +<p> +Another list-valued representation allows a message to be specified +as a list whose head is a &header; and whose tail is an &avp; list. +This representation is used by diameter itself when relaying requests +as directed by the return value of a &app_handle_request; callback. +It differs from the other other two in that it bypasses the checks for +messages that do not agree with their definitions in the dictionary in +question: messages are sent exactly as specified.</p> + +</item> + +<marker id="packet"/> + +<tag><c>packet() = #diameter_packet{}</c></tag> +<item> +<p> +A container for incoming and outgoing Diameter messages. +Fields have the following types.</p> + +<taglist> + +<tag><c>header = &header; | undefined</c></tag> +<item> +<p> +The Diameter header of the message. +Can be (and typically should be) <c>undefined</c> for an outgoing +message in a non-relay application, in which case diameter provides +appropriate values.</p> +</item> + +<tag><c>avps = [&avp;] | undefined</c></tag> +<item> +<p> +The AVPs of the message. +Ignored for an outgoing message if the <c>msg</c> field is set to a +value other than <c>undefined</c>.</p> +</item> + +<tag><c>msg = &message; | undefined</c></tag> +<item> +<p> +The incoming/outgoing message. +For an incoming message, a record if the message can be +decoded in a non-relay application, <c>undefined</c> otherwise. +For an outgoing message, setting a <c>[&header; | &avp;]</c> list is +equivalent to setting the <c>header</c> and <c>avps</c> fields to the +corresponding values.</p> + +<warning> +<p> +A record-valued <c>msg</c> field does <b>not</b> imply an absence of +decode errors. +The <c>errors</c> field should also be examined.</p> +</warning> + +</item> + +<tag><c>bin = binary()</c></tag> +<item> +<p> +The incoming message prior to encode or the outgoing message after +encode.</p> +</item> + +<tag><c>errors = [5000..5999 | {5000..5999, avp()}]</c></tag> +<item> +<p> +Errors detected at decode of an incoming message, as identified by +a corresponding 5xxx series Result-Code (Permanent Failures). +For an incoming request, these should be used to formulate an +appropriate answer as documented for the &app_handle_request; +callback in &man_app;. +For an incoming answer, the &mod_application_opt; +<c>answer_errors</c> determines the behaviour.</p> +</item> + +<tag><c>transport_data = term()</c></tag> +<item> +<p> +An arbitrary term of meaning only to the transport process in +question, as documented in &man_transport;.</p> +</item> + +</taglist> + +</item> + +</taglist> + +</section> + +<!-- ===================================================================== --> + +<funcs> + +<func> +<name>decode(Mod, Bin) -> Pkt</name> +<fsummary>Decode a Diameter message.</fsummary> +<type> +<v>Mod = &dictionary;</v> +<v>Bin = binary()</v> +<v>Pkt = &packet;</v> +</type> +<desc> + +<p> +Decode a Diameter message.</p> + +</desc> +</func> + +<func> +<name>encode(Mod, Msg) -> Pkt</name> +<fsummary>Encode a Diameter message.</fsummary> +<type> +<v>Mod = &dictionary;</v> +<v>Msg = &message; | &packet;</v> +<v>Pkt = &packet;</v> +</type> +<desc> + +<p> +Encode a Diameter message. +</p> + +</desc> +</func> + +</funcs> + +<!-- ===================================================================== --> +<!-- ===================================================================== --> + +<section> +<title>SEE ALSO</title> + +<p> +&man_compile;, &man_app;, &man_dict;, &man_make;</p> + +</section> + +</erlref> diff --git a/lib/diameter/doc/src/diameter_compile.xml b/lib/diameter/doc/src/diameter_compile.xml index eb6de80c11..0bd7ad1789 100644 --- a/lib/diameter/doc/src/diameter_compile.xml +++ b/lib/diameter/doc/src/diameter_compile.xml @@ -1,5 +1,7 @@ <?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE comref SYSTEM "comref.dtd" [ + <!ENTITY dictionary + '<seealso marker="diameter_dict">dictionary file</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -36,12 +38,14 @@ supplied. <description> <p> -The diameterc utility is used to compile diameter -<seealso marker="diameter_dict">dictionary files</seealso> -into Erlang source. -The resulting source implements the interface diameter requires +The diameterc utility is used to compile a diameter +&dictionary; into Erlang source. +The resulting source implements the interface diameter required to encode and decode the dictionary's messages and AVP's.</p> +<p> +The module &man_make; provides an alternate compilation interface.</p> + </description> <section> @@ -52,53 +56,52 @@ to encode and decode the dictionary's messages and AVP's.</p> <tag><![CDATA[diameterc [<options>] <file>]]></tag> <item> <p> -Compiles a single dictionary file. Valid options are as follows.</p> - -<!-- Leave -h/d/v undocumented, except for the usage message from the - utility itself. --> - -<taglist> -<tag><![CDATA[-o <dir>]]></tag> -<item> -<p> -Specifies the directory into which the generated source should be written. -Defaults to the current working directory.</p> -</item> +Compile a single dictionary file to Erlang source. +Valid options are as follows.</p> <tag><![CDATA[-i <dir>]]></tag> <item> <p> -Specifies a directory to add to the code path. +Prepend the specified directory to the code path. Use to point at beam files compiled from inherited dictionaries, -<c>@inherits</c> in a dictionary file creating a beam dependency, not -an erl/hrl dependency.</p> +<c>&dict_inherits;</c> in a dictionary file creating a beam +dependency, not an erl/hrl dependency.</p> <p> Multiple <c>-i</c> options can be specified.</p> </item> +<taglist> +<tag><![CDATA[-o <dir>]]></tag> +<item> +<p> +Write generated source to the specified directory. +Defaults to the current working directory.</p> +</item> + <tag><![CDATA[-E]]></tag> <tag><![CDATA[-H]]></tag> <item> <p> -Supresses erl and hrl generation, respectively.</p> +Supress erl and hrl generation, respectively.</p> </item> <tag><![CDATA[--name <name>]]></tag> <tag><![CDATA[--prefix <prefix>]]></tag> <item> <p> -Set <c>@name</c> and <c>@prefix</c> in the dictionary, -respectively. +Set <c>&dict_name;</c> or <c>&dict_prefix;</c> to the specified +string. Overrides any setting in the file itself.</p> </item> <tag><![CDATA[--inherits <dict>]]></tag> <item> <p> -Append an <c>@inherits</c> to the dictionary before compiling. -Specifying <c>'-'</c> as the dictionary has the effect of clearing any -previous inherits, causing them to be ignored.</p> +Append &dict_inherits; of the specified module. +Specifying <c>"-"</c> has the effect of discarding clearing any +previous inherits, both in the dictionary file and on the options +list.</p> <p> Multiple <c>--inherits</c> options can be specified.</p> @@ -127,7 +130,7 @@ Returns 0 on success, non-zero on failure.</p> <title>SEE ALSO</title> <p> -&man_dict;</p> +&man_make;, &man_dict;</p> </section> diff --git a/lib/diameter/doc/src/diameter_dict.xml b/lib/diameter/doc/src/diameter_dict.xml index 4a6cccc276..8b0687a22e 100644 --- a/lib/diameter/doc/src/diameter_dict.xml +++ b/lib/diameter/doc/src/diameter_dict.xml @@ -1,5 +1,11 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "fileref.dtd" [ + <!ENTITY format + '<seealso marker="#FILE_FORMAT">FILE FORMAT</seealso>'> + <!ENTITY records + '<seealso marker="#MESSAGE_RECORDS">MESSAGE RECORDS</seealso>'> + <!ENTITY types + '<seealso marker="#DATA_TYPES">DATA TYPES</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -45,34 +51,33 @@ under the License. <description> <p> -A diameter service as configured with &mod_start_service; +A diameter service, as configured with &mod_start_service;, specifies one or more supported Diameter applications. Each Diameter application specifies a dictionary module that knows how to encode and decode its messages and AVPs. The dictionary module is in turn generated from a file that defines these messages and AVPs. -The format of such a file is defined in -<seealso marker="#FILE_FORMAT">FILE FORMAT</seealso> below. +The format of such a file is defined in &format; below. Users add support for their specific applications by creating dictionary files, compiling them to Erlang modules using -<seealso marker="diameterc">diameterc</seealso> and configuring the +either &man_compile; or &man_make; and configuring the resulting dictionaries modules on a service.</p> <p> -The codec generation also results in a hrl file that defines records -for the messages and grouped AVPs defined for the application, these -records being what a user of the diameter application sends and receives. -(Modulo other available formats as discussed in &man_app;.) +Dictionary module generation also results in a hrl file that defines +records for the messages and Grouped AVPs defined by the +dictionary, these records being what a user of the diameter +application sends and receives, modulo other possible formats as +discussed in &man_app;. These records and the underlying Erlang data types corresponding to -Diameter data formats are discussed in <seealso -marker="#MESSAGE_RECORDS">MESSAGE RECORDS</seealso> and <seealso -marker="#DATA_TYPES">DATA TYPES</seealso> respectively. -The generated hrl also contains defines for the possible values of +Diameter data formats are discussed in &records; and &types; +respectively. +The generated hrl also contains macro definitions for the possible values of AVPs of type Enumerated.</p> <p> The diameter application includes three dictionary modules -corresponding to applications defined in section 2.4 of RFC 3588: +corresponding to applications defined in section 2.4 of &the_rfc;: <c>diameter_gen_base_rfc3588</c> for the Diameter Common Messages application with application identifier 0, <c>diameter_gen_accounting</c> for the Diameter Base Accounting @@ -111,6 +116,8 @@ The order in which sections are specified is unimportant.</p> <taglist> +<marker id="id"/> + <tag><c>@id Number</c></tag> <item> <p> @@ -134,6 +141,8 @@ Example:</p> </item> +<marker id="name"/> + <tag><c>@name Mod</c></tag> <item> <p> @@ -155,6 +164,8 @@ Example:</p> </item> +<marker id="prefix"/> + <tag><c>@prefix Name</c></tag> <item> <p> @@ -178,6 +189,8 @@ Example:</p> </item> +<marker id="vendor"/> + <tag><c>@vendor Number Name</c></tag> <item> <p> @@ -198,6 +211,8 @@ Example:</p> </item> +<marker id="avp_vendor_id"/> + <tag><c>@avp_vendor_id Number</c></tag> <item> <p> @@ -218,6 +233,8 @@ Region-Set </item> +<marker id="inherits"/> + <tag><c>@inherits Mod</c></tag> <item> <p> @@ -241,7 +258,7 @@ is equivalent to using <c>@avp_vendor_id</c> with a copy of the dictionary's definitions but the former makes for easier reuse.</p> <p> -All dictionaries should typically inherit RFC3588 AVPs from +All dictionaries should typically inherit &the_rfc; AVPs from <c>diameter_gen_base_rfc3588</c>.</p> <p> @@ -253,6 +270,8 @@ Example:</p> </item> +<marker id="avp_types"/> + <tag><c>@avp_types</c></tag> <item> <p> @@ -263,7 +282,7 @@ The section consists of definitions of the form</p> <p> where Code is the integer AVP code, Type identifies an AVP Data Format -as defined in <seealso marker="#DATA_TYPES">DATA TYPES</seealso> below, +as defined in section &types; below, and Flags is a string of V, M and P characters indicating the flags to be set on an outgoing AVP or a single <c>'-'</c> (minus) character if none are to be set.</p> @@ -280,13 +299,13 @@ Requested-Information 353 Enumerated V <warning> <p> -The P flag has been deprecated by the Diameter Maintenance -and Extensions Working Group of the IETF and should be omitted -to conform to the current draft standard.</p> +The P flag has been deprecated by &the_rfc;.</p> </warning> </item> +<marker id="custom_types"/> + <tag><c>@custom_types Mod</c></tag> <item> <p> @@ -308,6 +327,8 @@ Framed-IP-Address </pre> </item> +<marker id="codecs"/> + <tag><c>@codecs Mod</c></tag> <item> <p> @@ -325,12 +346,14 @@ Framed-IP-Address </pre> </item> +<marker id="messages"/> + <tag><c>@messages</c></tag> <item> <p> Defines the messages of the application. The section content consists of definitions of the form specified in -section 3.2 of RFC 3588, "Command Code ABNF specification".</p> +section 3.2 of &the_rfc;, "Command Code Format Specification".</p> <!-- RFC 4740 RTR/RTA --> <pre> @@ -370,13 +393,15 @@ RTA ::= < Diameter Header: 287, PXY > </item> +<marker id="grouped"/> + <tag><c>@grouped</c></tag> <item> <p> Defines the contents of the AVPs of the application having type Grouped. The section content consists of definitions of the form specified in -section 4.4 of RFC 3588, "Grouped AVP Values".</p> +section 4.4 of &the_rfc;, "Grouped AVP Values".</p> <p> Example:</p> @@ -395,6 +420,8 @@ Specifying a Vendor-Id in the definition of a grouped AVP is equivalent to specifying it with <c>@avp_vendor_id</c>.</p> </item> +<marker id="enum"/> + <tag><c>@enum Name</c></tag> <item> <p> @@ -421,6 +448,8 @@ REMOVE_SIP_SERVER 3 </pre> </item> +<marker id="end"/> + <tag><c>@end</c></tag> <item> <p> @@ -447,8 +476,8 @@ The hrl generated from a dictionary specification defines records for the messages and grouped AVPs defined in <c>@messages</c> and <c>@grouped</c> sections. For each message or grouped AVP definition, a record is defined whose -name is the message or AVP name prefixed with any dictionary prefix -defined with <c>@prefix</c> and whose fields are the names of the AVPs +name is the message or AVP name, prefixed with any dictionary prefix +defined with <c>@prefix</c>, and whose fields are the names of the AVPs contained in the message or grouped AVP in the order specified in the definition in question. For example, the grouped AVP</p> @@ -476,7 +505,7 @@ type and number of times the AVP can occur. In particular, an AVP which is specified as occurring exactly once is encoded as a value of the AVP's type while an AVP with any other specification is encoded as a list of values of the AVP's type. -The AVP's type is as specified in the AVP definition, the RFC 3588 +The AVP's type is as specified in the AVP definition, the &the_rfc; types being described below.</p> <marker id="DATA_TYPES"/> @@ -489,7 +518,7 @@ types being described below.</p> <p> The data formats defined in sections 4.2 ("Basic AVP Data -Formats") and 4.3 ("Derived AVP Data Formats") of RFC 3588 are encoded +Formats") and 4.3 ("Derived AVP Data Formats") of &the_rfc; are encoded as values of the types defined here. Values are passed to &mod_call; in a request record when sending a request, returned in a resulting @@ -564,8 +593,8 @@ where <p> Additionally, values that can be encoded are -limited by way of their encoding as four octets as required by RFC -3588 with the required extension from RFC 2030. +limited by way of their encoding as four octets as required by +&the_rfc; with the required extension from RFC 2030. In particular, only values between <c>{{1968,1,20},{3,14,8}}</c> and <c>{{2104,2,26},{9,42,23}}</c> (both inclusive) can be encoded.</p> @@ -609,7 +638,7 @@ where On encode, fields port, transport and protocol default to 3868, sctp and diameter respectively. The grammar of an OctetString-valued DiameterURI() is as specified in -section 4.3 of RFC 3588. +section 4.3 of &the_rfc;. The record representation is used on decode.</p> <marker id="Enumerated"/> @@ -640,7 +669,7 @@ Values of these types are not currently parsed by diameter.</p> <title>SEE ALSO</title> <p> -&man_compile;, &man_main;, &man_app;</p> +&man_compile;, &man_main;, &man_app;, &man_codec;, &man_make;</p> </section> diff --git a/lib/diameter/doc/src/diameter_intro.xml b/lib/diameter/doc/src/diameter_intro.xml index bc2afbd453..fd578ccf45 100644 --- a/lib/diameter/doc/src/diameter_intro.xml +++ b/lib/diameter/doc/src/diameter_intro.xml @@ -1,5 +1,8 @@ <?xml version="1.0" encoding="latin1" ?> -<!DOCTYPE chapter SYSTEM "chapter.dtd"> +<!DOCTYPE chapter SYSTEM "chapter.dtd" [ + <!ENTITY % also SYSTEM "seealso.ent"> + %also; +]> <chapter> <header> @@ -35,7 +38,7 @@ under the License. <p> The diameter application is an implementation of the Diameter protocol -as defined by RFC 3588. +as defined by &the_rfc;. It supports arbitrary Diameter applications by way of a <em>dictionary</em> interface that allows messages and AVP's to be defined and input into diameter as configuration. @@ -86,9 +89,9 @@ dictionary module that provide encode/decode functionality for outgoing/incoming Diameter messages belonging to the application. A dictionary module is generated from a <seealso -marker="diameter_dict">specification file</seealso> using the <seealso +marker="diameter_dict">dictionary file</seealso> using the <seealso marker="diameterc">diameterc</seealso> utility. -Dictionaries for the RFC 3588 Diameter Common Messages, Base +Dictionaries for the &the_rfc; Diameter Common Messages, Base Accounting and Relay applications are provided with the diameter application.</p> diff --git a/lib/diameter/doc/src/diameter_make.xml b/lib/diameter/doc/src/diameter_make.xml new file mode 100644 index 0000000000..da6124310e --- /dev/null +++ b/lib/diameter/doc/src/diameter_make.xml @@ -0,0 +1,144 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY filename + '<seealso marker="kernel:file#type-name">file:name()</seealso>'> + <!ENTITY dictionary + '<seealso marker="diameter_dict">dictionary file</seealso>'> + <!ENTITY % also SYSTEM "seealso.ent" > + <!ENTITY % here SYSTEM "seehere.ent" > + %also; + %here; +]> + +<erlref> +<header> +<copyright> +<year>2012</year> +<holder>Ericsson AB. All Rights Reserved.</holder> +</copyright> +<legalnotice> +The contents of this file are subject to the Erlang Public License, +Version 1.1, (the "License"); you may not use this file except in +compliance with the License. You should have received a copy of the +Erlang Public License along with this software. If not, it can be +retrieved online at http://www.erlang.org/. + +Software distributed under the License is distributed on an "AS IS" +basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +the License for the specific language governing rights and limitations +under the License. + +</legalnotice> + +<title>diameter_make(3)</title> +<prepared>Anders Svensson</prepared> +<responsible></responsible> +<docno></docno> +<approved></approved> +<checked></checked> +<date></date> +<rev></rev> +<file>diameter_make.xml</file> +</header> + +<module>diameter_make</module> +<modulesummary>Diameter dictionary compilation.</modulesummary> + +<description> + +<p> +The function &codec; is used to compile a diameter +&dictionary; into Erlang source. +The resulting source implements the interface diameter required +to encode and decode the dictionary's messages and AVP's.</p> + +<p> +The utility &man_compile; provides an alternate compilation +interface.</p> + +</description> + +<!-- ===================================================================== --> + +<funcs> + +<func> +<name>codec(Path::string(), [Opt]) -> ok | {error, Reason}</name> +<fsummary>Compile a dictionary file into Erlang source.</fsummary> +<desc> + +<p> +Compile a single dictionary file to Erlang source. +<c>Opt</c> can have the following types.</p> + +<taglist> + +<tag><c>{include, Dir::string()}</c></tag> +<item> +<p> +Prepend the specified directory to the code path. +Use to point at beam files compiled from inherited dictionaries, +<c>&dict_inherits;</c> in a dictionary file creating a beam +dependency, not an erl/hrl dependency.</p> + +<p> +Multiple <c>include</c> options can be specified.</p> +</item> + +<tag><c>{outdir, Dir::string()}</c></tag> +<item> +<p> +Write generated source to the specified directory. +Defaults to the current working directory.</p> +</item> + +<tag><c>{name|prefix, string()}</c></tag> +<item> +<p> +Set <c>&dict_name;</c> or <c>&dict_prefix;</c> to the specified +string. +Overrides any setting in the file itself.</p> +</item> + +<tag><c>{inherits, Mod::string()}</c></tag> +<item> +<p> +Append &dict_inherits; of the specified module. +Specifying <c>"-"</c> has the effect of discarding clearing any +previous inherits, both in the dictionary file and on the options +list.</p> + +<p> +Multiple <c>inherits</c> options can be specified.</p> +</item> + +</taglist> + +</desc> +</func> + +</funcs> + +<!-- ===================================================================== --> + +<section> +<title>BUGS</title> + +<p> +All options are string-valued. +In particular, it is not currently possible to +an &dict_inherits; module as an atom() or a path as a &filename;</p> + +</section> + +<!-- ===================================================================== --> + +<section> +<title>SEE ALSO</title> + +<p> +&man_compile;, &man_dict;</p> + +</section> + +</erlref> diff --git a/lib/diameter/doc/src/diameter_sctp.xml b/lib/diameter/doc/src/diameter_sctp.xml index a023a9bc08..5e3fd5eaf1 100644 --- a/lib/diameter/doc/src/diameter_sctp.xml +++ b/lib/diameter/doc/src/diameter_sctp.xml @@ -1,5 +1,11 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY gen_sctp '<seealso marker="kernel:gen_sctp">gen_sctp(3)</seealso>'> + <!ENTITY gen_sctp_open1 + '<seealso marker="kernel:gen_sctp#open-1">gen_sctp:open/1</seealso>'> + <!ENTITY ip_address + '<seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>'> + <!ENTITY inet '<seealso marker="kernel:inet">inet(3)</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -43,8 +49,7 @@ under the License. <description> <p> -This module implements diameter transport over SCTP using <seealso -marker="kernel:gen_sctp">gen_sctp</seealso>. +This module implements diameter transport over SCTP using &gen_sctp;. It can be specified as the value of a transport_module option to &mod_add_transport; and implements the behaviour documented in @@ -65,9 +70,9 @@ and implements the behaviour documented in <v>Type = connect | accept</v> <v>Ref = &mod_transport_ref;</v> <v>Svc = #diameter_service{}</v> -<v>Opt = {raddr, <seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>} | {rport, integer()} | term()</v> +<v>Opt = {raddr, &ip_address;} | {rport, integer()} | term()</v> <v>Pid = pid()</v> -<v>LAddr = <seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso></v> +<v>LAddr = &ip_address;</v> <v>Reason = term()</v> </type> <desc> @@ -84,8 +89,7 @@ unspecified. More than one <c>raddr</c> option can be specified, in which case the connecting transport in question attempts each in sequence until an association is established. -Remaining options are any accepted by <seealso -marker="kernel:gen_sctp#open-1">gen_sctp:open/1</seealso>, with the exception +Remaining options are any accepted by &gen_sctp_open1;, with the exception of options <c>mode</c>, <c>binary</c>, <c>list</c>, <c>active</c> and <c>sctp_events</c>. Note that options <c>ip</c> and <c>port</c> specify the local address @@ -102,14 +106,12 @@ connecting transport.</p> <warning> <p> An insufficiently large receive buffer may result in a peer having to -resend incoming messages: set the <seealso -marker="kernel:inet">inet(3)</seealso> option <c>recbuf</c> to increase +resend incoming messages: set the &inet; option <c>recbuf</c> to increase the buffer size.</p> <p> An insufficiently large send buffer may result in outgoing messages -being discarded: set the <seealso -marker="kernel:inet">inet(3)</seealso> option <c>sndbuf</c> to increase +being discarded: set the &inet; option <c>sndbuf</c> to increase the buffer size.</p> </warning> @@ -145,10 +147,7 @@ outbound streams.</p> <title>SEE ALSO</title> <p> -&man_main;, -&man_transport;, -<seealso marker="kernel:gen_sctp">gen_sctp(3)</seealso>, -<seealso marker="kernel:inet">inet(3)</seealso></p> +&man_main;, &man_transport;, &gen_sctp;, &inet;</p> </section> diff --git a/lib/diameter/doc/src/diameter_soc.xml b/lib/diameter/doc/src/diameter_soc.xml index 6b9ef9f756..16f6b9d5bb 100644 --- a/lib/diameter/doc/src/diameter_soc.xml +++ b/lib/diameter/doc/src/diameter_soc.xml @@ -1,11 +1,15 @@ <?xml version="1.0" encoding="latin1" ?> -<!DOCTYPE chapter SYSTEM "chapter.dtd"> +<!DOCTYPE chapter SYSTEM "chapter.dtd" [ + <!ENTITY % also SYSTEM "seealso.ent" > + %also; +]> <chapter> <header> <copyright> <year>2011</year> +<year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> @@ -41,38 +45,20 @@ Known points of questionable or non-compliance.</p> <!-- ===================================================================== --> <section> -<title>RFC 3588</title> +<title>&the_rfc;</title> <list> <item> <p> -The End-to-End Security framework (section 2.9) isn't implemented -since it is largely unspecified. -The document that was to describe it -(reference [AAACMS]) was abandoned in an uncompleted state several -years ago and the current draft RFC deprecates the framework, -including the P Flag in the AVP header.</p> -</item> - -<item> -<p> -There is no TLS support over SCTP. -RFC 3588 requires that a Diameter server support TLS but in -practise this seems to mean TLS over SCTP since there are limitations -with running over SCTP: see RFC 6083 (DTLS over SCTP), which is a -response to RFC 3436 (TLS over SCTP). -The current RFC 3588 draft acknowledges this by equating -TLS with TLS/TCP and DTLS/SCTP but we do not yet support DTLS.</p> +There is no support for DTLS over SCTP.</p> </item> <item> <p> There is no explicit support for peer discovery (section 5.2). It can possibly be implemented on top of diameter as is but this is -probably something that diameter should do. -The current draft deprecates portions of the original RFC's mechanisms -however.</p> +probably something that diameter should do.</p> </item> <item> diff --git a/lib/diameter/doc/src/diameter_tcp.xml b/lib/diameter/doc/src/diameter_tcp.xml index be8a938115..e3b8c733b7 100644 --- a/lib/diameter/doc/src/diameter_tcp.xml +++ b/lib/diameter/doc/src/diameter_tcp.xml @@ -1,5 +1,22 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY gen_tcp_connect3 + '<seealso marker="kernel:gen_tcp#connect-3">gen_tcp:connect/3</seealso>'> + <!ENTITY gen_tcp_listen2 + '<seealso marker="kernel:gen_tcp#listen-2">gen_tcp:listen/2</seealso>'> + <!ENTITY ip_address + '<seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>'> + <!ENTITY ssl_connect2 + '<seealso marker="ssl:ssl#connect-2">ssl:connect/2</seealso>'> + <!ENTITY ssl_connect3 + '<seealso marker="ssl:ssl#connect-3">ssl:connect/3</seealso>'> + <!ENTITY ssl_accept2 + '<seealso marker="ssl:ssl#ssl_accept-2">ssl:ssl_accept/2</seealso>'> + <!ENTITY ssl_listen2 + '<seealso marker="ssl:ssl#listen-2">ssl:listen/2</seealso>'> + <!ENTITY gen_tcp '<seealso marker="kernel:gen_tcp">gen_tcp(3)</seealso>'> + <!ENTITY inet '<seealso marker="kernel:inet">inet(3)</seealso>'> + <!ENTITY ssl '<seealso marker="ssl:ssl">ssl(3)</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -43,14 +60,13 @@ under the License. <description> <p> -This module implements diameter transport over TCP using <seealso -marker="kernel:gen_tcp">gen_tcp</seealso>. +This module implements diameter transport over TCP using &gen_tcp;. It can be specified as the value of a <c>transport_module</c> option to &mod_add_transport; and implements the behaviour documented in &man_transport;. TLS security is supported, both as an upgrade following -capabilities exchange as specified by RFC 3588 and +capabilities exchange as specified by &the_rfc; and at connection establishment as in the current draft standard.</p> <p> @@ -74,9 +90,9 @@ before configuring TLS capability on diameter transports.</p> <v>Svc = #diameter_service{}</v> <v>Opt = OwnOpt | SslOpt | TcpOpt</v> <v>Pid = pid()</v> -<v>LAddr = <seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso></v> +<v>LAddr = &ip_address;</v> <v>Reason = term()</v> -<v>OwnOpt = {raddr, <seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>} +<v>OwnOpt = {raddr, &ip_address;} | {rport, integer()} | {port, integer()}</v> <v>SslOpt = {ssl_options, true | list()}</v> @@ -95,16 +111,12 @@ transport. Option <c>ssl_options</c> must be specified for a transport that should support TLS: a value of <c>true</c> results in a TLS handshake immediately upon connection establishment while -<c>list()</c> specifies options to be passed to <seealso -marker="ssl:ssl#connect-2">ssl:connect/2</seealso> or -<seealso marker="ssl:ssl#ssl_accept-2">ssl:ssl_accept/2</seealso> +<c>list()</c> specifies options to be passed to &ssl_connect2; or +&ssl_accept2; after capabilities exchange if TLS is negotiated. -Remaining options are any accepted by <seealso -marker="ssl:ssl#connect-3">ssl:connect/3</seealso> or <seealso -marker="kernel:gen_tcp#connect-3">gen_tcp:connect/3</seealso> for -a connecting transport, or <seealso -marker="ssl:ssl#listen-2">ssl:listen/2</seealso> or <seealso -marker="kernel:gen_tcp#listen-2">gen_tcp:listen/2</seealso> for +Remaining options are any accepted by &ssl_connect3; or +&gen_tcp_connect3; for +a connecting transport, or &ssl_listen2; or &gen_tcp_listen2; for a listening transport, depending on whether or not <c>{ssl_options, true}</c> has been specified. Options <c>binary</c>, @@ -150,11 +162,7 @@ The returned local address list has length one.</p> <title>SEE ALSO</title> <p> -&man_main;, -&man_transport;, -<seealso marker="kernel:gen_tcp">gen_tcp(3)</seealso>, -<seealso marker="kernel:inet">inet(3)</seealso>, -<seealso marker="ssl:ssl">ssl(3)</seealso></p> +&man_main;, &man_transport;, &gen_tcp;, &inet;, &ssl;</p> </section> diff --git a/lib/diameter/doc/src/diameter_transport.xml b/lib/diameter/doc/src/diameter_transport.xml index 0507af63a8..55b531155f 100644 --- a/lib/diameter/doc/src/diameter_transport.xml +++ b/lib/diameter/doc/src/diameter_transport.xml @@ -1,5 +1,8 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY message '<seealso marker="#message">message()</seealso>'> + <!ENTITY ip_address + '<seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -50,16 +53,49 @@ diameter starts a transport process and a message interface with which the transport process communicates with the process that starts it (aka its parent).</p> -<marker id="start"/> </description> <!-- ===================================================================== --> +<section> +<title>DATA TYPES</title> + +<taglist> + +<marker id="message"/> + +<tag><c>message() = binary() | &codec_packet;</c></tag> +<item> +<p> +A Diameter message as passed over the transport interface.</p> + +<p> +For an inbound message from a transport process, a &codec_packet; must +contain the received message in its <c>bin</c> field. +In the case of an inbound request, any value set in the +<c>transport_data</c> field will passed back to the transport module +in the corresponding answer message, unless the sender supplies +another value.</p> + +<p> +For an outbound message to a transport process, a &codec_packet; has a +value other than <c>undefined</c> in its <c>transport_data</c> field +and has the binary() to send in its <c>bin</c> field.</p> +</item> + +</taglist> + +</section> + +<!-- ===================================================================== --> + <funcs> <func> <name>Mod:start({Type, Ref}, Svc, Config) - -> {ok, Pid} | {ok, Pid, LAddrs} | {error, Reason}</name> + -> {ok, Pid} + | {ok, Pid, LAddrs} + | {error, Reason}</name> <fsummary>Start a transport process.</fsummary> <type> <v>Type = connect | accept</v> @@ -67,7 +103,7 @@ parent).</p> <v>Svc = #diameter_service{}</v> <v>Config = term()</v> <v>Pid = pid()</v> -<v>LAddrs = [<seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>]</v> +<v>LAddrs = [&ip_address;]</v> <v>Reason = term()</v> </type> <desc> @@ -79,8 +115,8 @@ A transport process maintains a connection with a single remote peer.</p> <p> <c>Type</c> indicates whether the transport process in question -is being started for a connecting (<c>connect</c>) or listening -(<c>accept</c>) transport. +is being started for a connecting (<c>Type=connect</c>) or listening +(<c>Type=accept</c>) transport. In the latter case, transport processes are started as required to accept connections from multiple peers.</p> @@ -90,13 +126,12 @@ that has lead to starting of a transport process.</p> <p> <c>Svc</c> contains the capabilities passed to &mod_start_service; and -&mod_add_transport;, -values passed to the latter overriding those passed to the former.</p> +&mod_add_transport;, values passed to the latter overriding those +passed to the former.</p> <p> <c>Config</c> is as passed in <c>transport_config</c> tuple in the -&mod_transport_opt; -list passed to &mod_add_transport;.</p> +&mod_transport_opt; list passed to &mod_add_transport;.</p> <p> The start function should use the <c>Host-IP-Address</c> list and/or @@ -114,13 +149,13 @@ it dies. It should not link to the parent. It should exit if its transport connection with its peer is lost.</p> -<marker id="MESSAGES"/> </desc> </func> </funcs> <!-- ===================================================================== --> +<marker id="MESSAGES"/> <section> <title>MESSAGES</title> @@ -130,19 +165,15 @@ All messages sent over the transport interface are of the form <c>{diameter, term()}</c>.</p> <p> -A transport process can expect the following messages from -diameter.</p> +A transport process can expect messages of the following types from +its parent.</p> <taglist> -<tag><c>{diameter, {send, Packet}}</c></tag> +<tag><c>{diameter, {send, &message;}}</c></tag> <item> <p> -An outbound Diameter message. -<c>Packet</c> can be either binary() (the message to be sent) -or a <c>#diameter_packet{}</c> record whose <c>transport_data</c> -field contains a value other than undefined and whose <c>bin</c> field -contains the binary to send.</p> +An outbound Diameter message.</p> </item> <tag><c>{diameter, {close, Pid}}</c></tag> @@ -185,7 +216,7 @@ TLS.</p> </taglist> <p> -A transport process should send the following messages +A transport process should send messages of the following types to its parent.</p> <taglist> @@ -208,19 +239,10 @@ Not sent if the transport process has <c>Type=accept</c>. endpoint to which the transport has connected.</p> </item> -<tag><c>{diameter, {recv, Packet}}</c></tag> +<tag><c>{diameter, {recv, &message;}}</c></tag> <item> <p> -An inbound Diameter message. -<c>Packet</c> can be either binary() (the received message) -or a <c>#diameter_packet{}</c> record -whose <c>bin</c> field contains the received binary(). -Any value (other than <c>undefined</c>) set in -the <c>transport_data</c> field will be passed back with a -corresponding answer message in the case that the inbound message is a -request unless the sender sets another value. -How <c>transport_data</c> is used/interpreted is up to the -transport module.</p> +An inbound Diameter message.</p> </item> <tag><c>{diameter, {tls, Ref}}</c></tag> diff --git a/lib/diameter/doc/src/files.mk b/lib/diameter/doc/src/files.mk index 89ec1031e6..00ced3d91e 100644 --- a/lib/diameter/doc/src/files.mk +++ b/lib/diameter/doc/src/files.mk @@ -26,6 +26,8 @@ XML_REF1_FILES = \ XML_REF3_FILES = \ diameter.xml \ diameter_app.xml \ + diameter_codec.xml \ + diameter_make.xml \ diameter_transport.xml \ diameter_tcp.xml \ diameter_sctp.xml diff --git a/lib/diameter/doc/src/notes.xml b/lib/diameter/doc/src/notes.xml index b89d84a4f6..d448b01eba 100644 --- a/lib/diameter/doc/src/notes.xml +++ b/lib/diameter/doc/src/notes.xml @@ -42,6 +42,163 @@ first.</p> <!-- ===================================================================== --> +<section><title>Diameter 1.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix faulty handling of Origin-State-Id and faulty config + values.</p> + <p> + The former was expected in a list despite the + documentation requiring (correctly) an integer. A bare + value for a list-valued capability was not handled.</p> + <p> + Own Id: OTP-10440</p> + </item> + <item> + <p> + Fix timing of up/down events.</p> + <p> + Previously, a call to diameter:call/4 following a peer_up + callback might incorrectly return {error, no_connection}, + depending on timing. Both events now follow the + corresponding callbacks.</p> + <p> + Own Id: OTP-10459</p> + </item> + <item> + <p> + Make diameter:service_info/2 usable in peer_up, peer_down + and pick_peer callbacks.</p> + <p> + Except for in pick_peer when {call_mutates_state, false}, + it would previously hang indefinitely.</p> + <p> + Own Id: OTP-10460</p> + </item> + <item> + <p> + Verify that End-to-End and Hop-by-Hop Identifiers in an + incoming CEA/DPA match those sent in the corresponding + CER/DPR.</p> + <p> + The values were previously ignored. Answers whose + identifiers do not match are handled as unexpected.</p> + <p> + Own Id: OTP-10565</p> + </item> + <item> + <p> + Fix formatting problems in PDF documentation.</p> + <p> + In particular, text corresponding to links in HTML was + omitted in preformatted blocks. There are still issues + with indentation but this is not diameter-specific.</p> + <p> + Own Id: OTP-10583</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Let prepare_request, prepare_retransmit and + handle_request callbacks return a function to be invoked + on outgoing messages after encode.</p> + <p> + This allows encoded messages to be logged for example.</p> + <p> + Own Id: OTP-10441</p> + </item> + <item> + <p> + Add service_opt() 'restrict_connections' to allow + multiple transport connections with the same peer.</p> + <p> + Own Id: OTP-10443</p> + </item> + <item> + <p> + Add service_opt() 'sequence' to allow the masking of a + constant onto the topmost bits of End-to-End and + Hop-by-Hop identifiers.</p> + <p> + This allows the same service on different nodes to use + distinct values in outgoing request messages.</p> + <p> + Own Id: OTP-10445</p> + </item> + <item> + <p> + Add diameter:service_info(PeerRef) to return the + transport_ref() and transport_opt() list of the + corresponding transport.</p> + <p> + This allows easy access to these from diameter_app + callbacks that only get peer_ref() as an argument.</p> + <p> + Own Id: OTP-10470</p> + </item> + <item> + <p> + Add reference pages diameter_codec(3) and + diameter_make(3).</p> + <p> + Own Id: OTP-10471</p> + </item> + <item> + <p> + Add events for service start and stop.</p> + <p> + Own Id: OTP-10492</p> + </item> + <item> + <p> + Add transport_opt() 'disconnect_cb' to make the sending + of DPR configurable.</p> + <p> + Whether or not DPR should be sent at application stop, + service stop or transport removal is determined by the + value returned by the callback, as is the + Disconnect-Cause and timeout if DPA is not received.</p> + <p> + Own Id: OTP-10493</p> + </item> + <item> + <p> + Add transport_opt() 'capx_timeout' for the timeout + associated with non-reception of CER/CEA.</p> + <p> + Own Id: OTP-10554</p> + </item> + <item> + <p> + Allow a handle_request callback to return a + #diameter_packet{}.</p> + <p> + This allows an answer to set transport_data and header + fields.</p> + <p> + Own Id: OTP-10566</p> + </item> + <item> + <p> + Update documentation for RFC 6733.</p> + <p> + RFC 3588 is now obsolete.</p> + <p> + Own Id: OTP-10568</p> + </item> + </list> + </section> + +</section> + <section><title>Diameter 1.2</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -428,7 +585,7 @@ Known issues or limitations:</p> Some agent-related functionality is not entirely complete. In particular, support for proxy agents, that advertise specific Diameter applications but otherwise relay messages in much the same -way as relay agents (for which a &handle_request; +way as relay agents (for which a handle_request callback can return a <c>relay</c> tuple), will be completed in an upcoming release. There may also be more explicit support for redirect agents, although @@ -460,7 +617,7 @@ could likely be expanded upon.</p> <item> <p> -The function &service_info; +The function diameter:service_info/2 can be used to retrieve information about a started service (statistics, information about connected peers, etc) but this is not yet documented and both the input and output may change diff --git a/lib/diameter/doc/src/ref_man.xml b/lib/diameter/doc/src/ref_man.xml index 137ce79094..4b99fe716d 100644 --- a/lib/diameter/doc/src/ref_man.xml +++ b/lib/diameter/doc/src/ref_man.xml @@ -6,6 +6,7 @@ <header> <copyright> <year>2011</year> +<year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -40,7 +41,9 @@ applications on top of the Diameter protocol. </p> <xi:include href="diameter.xml"/> <xi:include href="diameter_compile.xml"/> <xi:include href="diameter_app.xml"/> +<xi:include href="diameter_codec.xml"/> <xi:include href="diameter_dict.xml"/> +<xi:include href="diameter_make.xml"/> <xi:include href="diameter_transport.xml"/> <xi:include href="diameter_tcp.xml"/> <xi:include href="diameter_sctp.xml"/> diff --git a/lib/diameter/doc/src/seealso.ent b/lib/diameter/doc/src/seealso.ent index 6f67630220..4647c42f85 100644 --- a/lib/diameter/doc/src/seealso.ent +++ b/lib/diameter/doc/src/seealso.ent @@ -32,6 +32,8 @@ significant. --> +<!ENTITY the_rfc 'RFC 6733'> + <!-- diameter --> <!ENTITY mod_add_transport '<seealso marker="diameter#add_transport-2">diameter:add_transport/2</seealso>'> @@ -77,12 +79,21 @@ significant. <!ENTITY app_prepare_request '<seealso marker="diameter_app#Mod:prepare_request-3">prepare_request/3</seealso>'> <!ENTITY app_capabilities '<seealso marker="diameter_app#capabilities">diameter_app:capabilities()</seealso>'> -<!ENTITY app_message '<seealso marker="diameter_app#message">diameter_app:message()</seealso>'> -<!ENTITY app_packet '<seealso marker="diameter_app#packet">diameter_app:packet()</seealso>'> <!ENTITY app_peer '<seealso marker="diameter_app#peer">diameter_app:peer()</seealso>'> <!ENTITY app_peer_ref '<seealso marker="diameter_app#peer_ref">diameter_app:peer_ref()</seealso>'> <!ENTITY app_state '<seealso marker="diameter_app#state">diameter_app:state()</seealso>'> +<!-- diameter_codec --> + +<!ENTITY codec_encode '<seealso marker="diameter_codec#encode-2">diameter_codec:encode/2</seealso>'> +<!ENTITY codec_decode '<seealso marker="diameter_codec#decode-2">diameter_codec:decode/2</seealso>'> + +<!ENTITY codec_avp '<seealso marker="diameter_codec#avp">diameter_codec:avp()</seealso>'> +<!ENTITY codec_header '<seealso marker="diameter_codec#header">diameter_codec:header()</seealso>'> +<!ENTITY codec_dictionary '<seealso marker="diameter_codec#dictionary">diameter_codec:dictionary()</seealso>'> +<!ENTITY codec_message '<seealso marker="diameter_codec#message">diameter_codec:message()</seealso>'> +<!ENTITY codec_packet '<seealso marker="diameter_codec#packet">diameter_codec:packet()</seealso>'> + <!-- diameter_dict --> <!ENTITY dict_data_types '<seealso marker="diameter_dict#DATA_TYPES">diameter_dict(4)</seealso>'> @@ -95,6 +106,14 @@ significant. <!ENTITY dict_UTF8String '<seealso marker="diameter_dict#DATA_TYPES">UTF8String()</seealso>'> <!ENTITY dict_Unsigned32 '<seealso marker="diameter_dict#DATA_TYPES">Unsigned32()</seealso>'> +<!ENTITY dict_name '<seealso marker="diameter_dict#name">@name</seealso>'> +<!ENTITY dict_prefix '<seealso marker="diameter_dict#prefix">@prefix</seealso>'> +<!ENTITY dict_inherits '<seealso marker="diameter_dict#inherits">@inherits</seealso>'> + +<!-- diameter_make --> + +<!ENTITY make_codec '<seealso marker="diameter_make#codec-2">diameter_make:codec/2</seealso>'> + <!-- diameter_transport --> <!ENTITY transport_start @@ -105,8 +124,9 @@ significant. <!ENTITY man_compile '<seealso marker="diameterc">diameterc(1)</seealso>'> <!ENTITY man_main '<seealso marker="diameter">diameter(3)</seealso>'> <!ENTITY man_app '<seealso marker="diameter_app">diameter_app(3)</seealso>'> +<!ENTITY man_codec '<seealso marker="diameter_codec">diameter_codec(3)</seealso>'> <!ENTITY man_dict '<seealso marker="diameter_dict">diameter_dict(4)</seealso>'> -<!ENTITY man_transport - '<seealso marker="diameter_transport">diameter_transport(3)</seealso>'> +<!ENTITY man_make '<seealso marker="diameter_make">diameter_make(3)</seealso>'> +<!ENTITY man_transport '<seealso marker="diameter_transport">diameter_transport(3)</seealso>'> <!ENTITY man_sctp '<seealso marker="diameter_sctp">diameter_sctp(3)</seealso>'> <!ENTITY man_tcp '<seealso marker="diameter_tcp">diameter_tcp(3)</seealso>'> diff --git a/lib/diameter/doc/standard/draft-ietf-dime-capablities-update-07.txt b/lib/diameter/doc/standard/draft-ietf-dime-capablities-update-07.txt deleted file mode 100644 index bb7ec2d08c..0000000000 --- a/lib/diameter/doc/standard/draft-ietf-dime-capablities-update-07.txt +++ /dev/null @@ -1,392 +0,0 @@ -
-
-
-Network Working Group K. Jiao
-Internet-Draft Huawei
-Intended status: Standards Track G. Zorn
-Expires: April 27, 2011 Network Zen
- October 24, 2010
-
-
- The Diameter Capabilities Update Application
- draft-ietf-dime-capablities-update-07
-
-Abstract
-
- This document defines a new Diameter application and associated
- command codes. The Capabilities Update application is intended to
- allow the dynamic update of certain Diameter peer capabilities while
- the peer-to-peer connection is in the open state.
-
-Status of this Memo
-
- This Internet-Draft is submitted in full conformance with the
- provisions of BCP 78 and BCP 79.
-
- Internet-Drafts are working documents of the Internet Engineering
- Task Force (IETF). Note that other groups may also distribute
- working documents as Internet-Drafts. The list of current Internet-
- Drafts is at http://datatracker.ietf.org/drafts/current/.
-
- Internet-Drafts are draft documents valid for a maximum of six months
- and may be updated, replaced, or obsoleted by other documents at any
- time. It is inappropriate to use Internet-Drafts as reference
- material or to cite them other than as "work in progress."
-
- This Internet-Draft will expire on April 27, 2011.
-
-Copyright Notice
-
- Copyright (c) 2010 IETF Trust and the persons identified as the
- document authors. All rights reserved.
-
- This document is subject to BCP 78 and the IETF Trust's Legal
- Provisions Relating to IETF Documents
- (http://trustee.ietf.org/license-info) in effect on the date of
- publication of this document. Please review these documents
- carefully, as they describe your rights and restrictions with respect
- to this document. Code Components extracted from this document must
- include Simplified BSD License text as described in Section 4.e of
- the Trust Legal Provisions and are provided without warranty as
- described in the Simplified BSD License.
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 1]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
-Table of Contents
-
- 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
- 2. Specification of Requirements . . . . . . . . . . . . . . . . . 3
- 3. Diameter Protocol Considerations . . . . . . . . . . . . . . . 3
- 4. Capabilities Update . . . . . . . . . . . . . . . . . . . . . . 3
- 4.1. Command-Code Values . . . . . . . . . . . . . . . . . . . . 4
- 4.1.1. Capabilities-Update-Request . . . . . . . . . . . . . . 5
- 4.1.2. Capabilities-Update-Answer . . . . . . . . . . . . . . 5
- 5. Security Considerations . . . . . . . . . . . . . . . . . . . . 6
- 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 6
- 6.1. Application Identifier . . . . . . . . . . . . . . . . . . 6
- 6.2. Command Codes . . . . . . . . . . . . . . . . . . . . . . . 6
- 7. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 6
- 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 6
- 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 6
- 9.1. Normative References . . . . . . . . . . . . . . . . . . . 6
- 9.2. Informative References . . . . . . . . . . . . . . . . . . 7
- Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 7
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 2]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
-1. Introduction
-
- Capabilities exchange is an important component of the Diameter Base
- Protocol [I-D.ietf-dime-rfc3588bis], allowing peers to exchange
- identities and Diameter capabilities (protocol version number,
- supported Diameter applications, security mechanisms, etc.). As
- defined in RFC 3588, however, the capabilities exchange process takes
- place only once, at the inception of a transport connection between a
- given pair of peers. Therefore, if a peer's capabilities change (due
- to software update, for example), the existing connection(s) must be
- torn down (along with all of the associated user sessions) and
- restarted before the modified capabilities can be advertised.
-
- This document defines a new Diameter application intended to allow
- the dynamic update of a subset of Diameter peer capabilities over an
- existing connection. Because the Capabilities Update application
- specified herein operates over an existing transport connection,
- modification of certain capabilities is prohibited. Specifically,
- modifying the security mechanism in use is not allowed; if the
- security method used between a pair of peers is changed the affected
- connection MUST be restarted.
-
-
-2. Specification of Requirements
-
- The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
- "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
- document are to be interpreted as described in RFC 2119 [RFC2119].
-
-
-3. Diameter Protocol Considerations
-
- This section details the relationship of the Diameter Capabilities
- Update application to the Diameter Base Protocol.
-
- This document specifies Diameter Application-ID <TBD1>. Diameter
- nodes conforming to this specification MUST advertise support by
- including the value <TBD1> in the Auth-Application-Id of the
- Capabilities-Exchange-Request (CER) and Capabilities-Exchange-Answer
- (CEA) commands [I-D.ietf-dime-rfc3588bis].
-
-
-4. Capabilities Update
-
- When the capabilities of a Diameter node conforming to this
- specification change, it MUST notify all of the nodes with which it
- has an open transport connection and which have also advertised
- support for the Capabilities Update application using the
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 3]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
- Capabilities-Update-Request (CUR) message (Section 4.1.1). This
- message allows the update of a peer's capabilities (supported
- Diameter applications, etc.).
-
- A Diameter node only issues a given command to those peers that have
- advertised support for the Diameter application that defines the
- command; a Diameter node must cache the supported applications in
- order to ensure that unrecognized commands and/or Attribute-Value
- Pairs (AVPs) are not unnecessarily sent to a peer.
-
- The receiver of the CUR MUST determine common applications by
- computing the intersection of its own set of supported Application Id
- against all of the application identifier AVPs (Auth-Application-Id,
- Acct-Application-Id and Vendor-Specific- Application-Id) present in
- the CUR. The value of the Vendor-Id AVP in the Vendor-Specific-
- Application-Id MUST NOT be used during computation.
-
- If the receiver of a CUR does not have any applications in common
- with the sender then it MUST return a Capabilities-Update-Answer
- (CUA) (Section 4.1.2) with the Result-Code AVP set to
- DIAMETER_NO_COMMON_APPLICATION [I-D.ietf-dime-rfc3588bis], and SHOULD
- disconnect the transport layer connection; however, if active
- sessions are using the connection, peers MAY delay disconnection
- until the sessions can be redirected or gracefully terminated. Note
- that receiving a CUA from a peer advertising itself as a Relay (see
- [I-D.ietf-dime-rfc3588bis], Section 2.4) MUST be interpreted as
- having common applications with the peer.
-
- As for CER/CEA messages, the CUR and CUA messages MUST NOT be
- proxied, redirected or relayed.
-
- Even though the CUR/CUA messages cannot be proxied, it is still
- possible for an upstream agent to receive a message for which there
- are no peers available to handle the application that corresponds to
- the Command-Code. This could happen if, for example, the peers are
- too busy or down. In such instances, the 'E' bit MUST be set in the
- answer message with the Result-Code AVP set to
- DIAMETER_UNABLE_TO_DELIVER to inform the downstream peer to take
- action (e.g., re-routing requests to an alternate peer).
-
-4.1. Command-Code Values
-
- This section defines Command-Code [I-D.ietf-dime-rfc3588bis] values
- that MUST be supported by all Diameter implementations conforming to
- this specification. The following Command Codes are defined (using
- modified ABNF [I-D.ietf-dime-rfc3588bis]) in this document:
- Capabilities-Update-Request (CUR, Section 4.1.1) and Capabilities-
- Update-Answer (CUA, Section 4.1.2).
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 4]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
-4.1.1. Capabilities-Update-Request
-
- The Capabilities-Update-Request (CUR), indicated by the Command-Code
- set to <CUCC> and the Command Flags' 'R' bit set, is sent to update
- local capabilities. Upon detection of a transport failure, this
- message MUST NOT be sent to an alternate peer.
-
- When Diameter is run over the Stream Control Transmission Protocol
- [RFC4960], which allows connections to span multiple interfaces and
- multiple IP addresses, the Capabilities-Update-Request message MUST
- contain one Host-IP-Address AVP for each potential IP address that
- may be locally used when transmitting Diameter messages.
-
- Message Format
-
- <CUR> ::= < Diameter Header: <CUCC>, REQ >
- { Origin-Host }
- { Origin-Realm }
- 1* { Host-IP-Address }
- { Vendor-Id }
- { Product-Name }
- [ Origin-State-Id ]
- * [ Supported-Vendor-Id ]
- * [ Auth-Application-Id ]
- * [ Acct-Application-Id ]
- * [ Vendor-Specific-Application-Id ]
- [ Firmware-Revision ]
- * [ AVP ]
-
-4.1.2. Capabilities-Update-Answer
-
- The Capabilities-Update-Answer, indicated by the Command-Code set to
- <CUCC> and the Command Flags' 'R' bit cleared, is sent in response to
- a CUR message.
-
- Message Format
-
- <CUA> ::= < Diameter Header: <CUCC> >
- { Origin-Host }
- { Origin-Realm }
- { Result-Code }
- [ Error-Message ]
- * [ AVP ]
-
-
-
-
-
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 5]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
-5. Security Considerations
-
- The security considerations applicable to the Diameter Base Protocol
- [I-D.ietf-dime-rfc3588bis] are also applicable to this document.
-
-
-6. IANA Considerations
-
- This section explains the criteria to be used by the IANA for
- assignment of numbers within namespaces used within this document.
-
-6.1. Application Identifier
-
- This specification assigns the value <TBD1> from the Application
- Identifiers namespace [I-D.ietf-dime-rfc3588bis]. See Section 3 for
- the assignment of the namespace in this specification.
-
-6.2. Command Codes
-
- This specification assigns the value <CUCC> from the Command Codes
- namespace [I-D.ietf-dime-rfc3588bis]. See Section 4.1 for the
- assignment of the namespace in this specification.
-
-
-7. Contributors
-
- This document is based upon work done by Tina Tsou.
-
-
-8. Acknowledgements
-
- Thanks to Sebastien Decugis, Niklas Neumann, Subash Comerica, Lionel
- Morand, Dan Romascanu, Dan Harkins and Ravi for helpful review and
- discussion.
-
-
-9. References
-
-9.1. Normative References
-
- [I-D.ietf-dime-rfc3588bis]
- Fajardo, V., Arkko, J., Loughney, J., and G. Zorn,
- "Diameter Base Protocol", draft-ietf-dime-rfc3588bis-25
- (work in progress), September 2010.
-
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
- Requirement Levels", BCP 14, RFC 2119, March 1997.
-
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 6]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
-9.2. Informative References
-
- [RFC4960] Stewart, R., "Stream Control Transmission Protocol",
- RFC 4960, September 2007.
-
-
-Authors' Addresses
-
- Jiao Kang
- Huawei Technologies
- Section B1, Huawei Industrial Base
- Bantian, Longgang District
- Shenzhen 518129
- P.R. China
-
- Phone: +86 755 2878-6690
- Email: [email protected]
-
-
- Glen Zorn
- Network Zen
- 227/358 Thanon Sanphawut
- Bang Na, Bangkok 10260
- Thailand
-
- Phone: +66 (0) 87-040-4617
- Email: [email protected]
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 7]
-
diff --git a/lib/diameter/doc/standard/draft-ietf-dime-rfc3588bis-26.txt b/lib/diameter/doc/standard/rfc6733.txt index 87b9562f93..2f5a477347 100644 --- a/lib/diameter/doc/standard/draft-ietf-dime-rfc3588bis-26.txt +++ b/lib/diameter/doc/standard/rfc6733.txt @@ -1,62 +1,72 @@ -DIME V. Fajardo, Ed. -Internet-Draft Telcordia Technologies -Obsoletes: 3588 (if approved) J. Arkko -Intended status: Standards Track Ericsson Research -Expires: July 24, 2011 J. Loughney + + + +Internet Engineering Task Force (IETF) V. Fajardo, Ed. +Request for Comments: 6733 Telcordia Technologies +Obsoletes: 3588, 5719 J. Arkko +Category: Standards Track Ericsson Research +ISSN: 2070-1721 J. Loughney Nokia Research Center - G. Zorn + G. Zorn, Ed. Network Zen - January 20, 2011 + October 2012 Diameter Base Protocol - draft-ietf-dime-rfc3588bis-26.txt Abstract The Diameter base protocol is intended to provide an Authentication, - Authorization and Accounting (AAA) framework for applications such as - network access or IP mobility in both local and roaming situations. - This document specifies the message format, transport, error - reporting, accounting and security services used by all Diameter - applications. The Diameter base protocol as defined in this document - must be supported by all Diameter implementations. + Authorization, and Accounting (AAA) framework for applications such + as network access or IP mobility in both local and roaming + situations. This document specifies the message format, transport, + error reporting, accounting, and security services used by all + Diameter applications. The Diameter base protocol as defined in this + document obsoletes RFC 3588 and RFC 5719, and it must be supported by + all new Diameter implementations. -Status of this Memo +Status of This Memo - This Internet-Draft is submitted in full conformance with the - provisions of BCP 78 and BCP 79. + This is an Internet Standards Track document. - Internet-Drafts are working documents of the Internet Engineering - Task Force (IETF). Note that other groups may also distribute - working documents as Internet-Drafts. The list of current Internet- - Drafts is at http://datatracker.ietf.org/drafts/current/. + This document is a product of the Internet Engineering Task Force + (IETF). It represents the consensus of the IETF community. It has + received public review and has been approved for publication by the + Internet Engineering Steering Group (IESG). Further information on + Internet Standards is available in Section 2 of RFC 5741. + + Information about the current status of this document, any errata, + and how to provide feedback on it may be obtained at + http://www.rfc-editor.org/info/rfc6733. - Internet-Drafts are draft documents valid for a maximum of six months - and may be updated, replaced, or obsoleted by other documents at any - time. It is inappropriate to use Internet-Drafts as reference - material or to cite them other than as "work in progress." - This Internet-Draft will expire on July 24, 2011. -Copyright Notice - Copyright (c) 2011 IETF Trust and the persons identified as the - document authors. All rights reserved. - This document is subject to BCP 78 and the IETF Trust's Legal - Provisions Relating to IETF Documents -Fajardo, et al. Expires July 24, 2011 [Page 1] + + + + + + +Fajardo, et al. Standards Track [Page 1] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 +Copyright Notice + + Copyright (c) 2012 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect @@ -65,256 +75,303 @@ Internet-Draft Diameter Base Protocol January 2011 the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. + This document may contain material from IETF Documents or IETF + Contributions published or made publicly available before November + 10, 2008. The person(s) controlling the copyright in some of this + material may not have granted the IETF Trust the right to allow + modifications of such material outside the IETF Standards Process. + Without obtaining an adequate license from the person(s) controlling + the copyright in such materials, this document may not be modified + outside the IETF Standards Process, and derivative works of it may + not be created outside the IETF Standards Process, except to format + it for publication as an RFC or to translate it into languages other + than English. Table of Contents - 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 6 - 1.1. Diameter Protocol . . . . . . . . . . . . . . . . . . . . 8 - 1.1.1. Description of the Document Set . . . . . . . . . . . 9 - 1.1.2. Conventions Used in This Document . . . . . . . . . . 10 - 1.1.3. Changes from RFC3588 . . . . . . . . . . . . . . . . 10 - 1.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 12 - 1.3. Approach to Extensibility . . . . . . . . . . . . . . . . 17 - 1.3.1. Defining New AVP Values . . . . . . . . . . . . . . . 18 - 1.3.2. Creating New AVPs . . . . . . . . . . . . . . . . . . 18 - 1.3.3. Creating New Commands . . . . . . . . . . . . . . . . 18 - 1.3.4. Creating New Diameter Applications . . . . . . . . . 19 - 2. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . 21 - 2.1. Transport . . . . . . . . . . . . . . . . . . . . . . . . 22 - 2.1.1. SCTP Guidelines . . . . . . . . . . . . . . . . . . . 23 - 2.2. Securing Diameter Messages . . . . . . . . . . . . . . . 24 - 2.3. Diameter Application Compliance . . . . . . . . . . . . . 24 - 2.4. Application Identifiers . . . . . . . . . . . . . . . . . 24 - 2.5. Connections vs. Sessions . . . . . . . . . . . . . . . . 25 - 2.6. Peer Table . . . . . . . . . . . . . . . . . . . . . . . 26 - 2.7. Routing Table . . . . . . . . . . . . . . . . . . . . . . 27 - 2.8. Role of Diameter Agents . . . . . . . . . . . . . . . . . 28 - 2.8.1. Relay Agents . . . . . . . . . . . . . . . . . . . . 29 - 2.8.2. Proxy Agents . . . . . . . . . . . . . . . . . . . . 30 - 2.8.3. Redirect Agents . . . . . . . . . . . . . . . . . . . 31 - 2.8.4. Translation Agents . . . . . . . . . . . . . . . . . 32 - 2.9. Diameter Path Authorization . . . . . . . . . . . . . . . 33 - 3. Diameter Header . . . . . . . . . . . . . . . . . . . . . . . 35 - 3.1. Command Codes . . . . . . . . . . . . . . . . . . . . . . 38 - 3.2. Command Code ABNF specification . . . . . . . . . . . . . 38 - 3.3. Diameter Command Naming Conventions . . . . . . . . . . . 41 - 4. Diameter AVPs . . . . . . . . . . . . . . . . . . . . . . . . 42 - 4.1. AVP Header . . . . . . . . . . . . . . . . . . . . . . . 42 - 4.1.1. Optional Header Elements . . . . . . . . . . . . . . 43 - 4.2. Basic AVP Data Formats . . . . . . . . . . . . . . . . . 44 - 4.3. Derived AVP Data Formats . . . . . . . . . . . . . . . . 45 - 4.3.1. Common Derived AVPs . . . . . . . . . . . . . . . . . 45 - 4.4. Grouped AVP Values . . . . . . . . . . . . . . . . . . . 52 - - - -Fajardo, et al. Expires July 24, 2011 [Page 2] - -Internet-Draft Diameter Base Protocol January 2011 - - - 4.4.1. Example AVP with a Grouped Data type . . . . . . . . 53 - 4.5. Diameter Base Protocol AVPs . . . . . . . . . . . . . . . 56 - 5. Diameter Peers . . . . . . . . . . . . . . . . . . . . . . . 59 - 5.1. Peer Connections . . . . . . . . . . . . . . . . . . . . 59 - 5.2. Diameter Peer Discovery . . . . . . . . . . . . . . . . . 60 - 5.3. Capabilities Exchange . . . . . . . . . . . . . . . . . . 61 - 5.3.1. Capabilities-Exchange-Request . . . . . . . . . . . . 63 - 5.3.2. Capabilities-Exchange-Answer . . . . . . . . . . . . 63 - 5.3.3. Vendor-Id AVP . . . . . . . . . . . . . . . . . . . . 64 - 5.3.4. Firmware-Revision AVP . . . . . . . . . . . . . . . . 64 - 5.3.5. Host-IP-Address AVP . . . . . . . . . . . . . . . . . 64 - 5.3.6. Supported-Vendor-Id AVP . . . . . . . . . . . . . . . 65 - 5.3.7. Product-Name AVP . . . . . . . . . . . . . . . . . . 65 - 5.4. Disconnecting Peer connections . . . . . . . . . . . . . 65 - 5.4.1. Disconnect-Peer-Request . . . . . . . . . . . . . . . 66 - 5.4.2. Disconnect-Peer-Answer . . . . . . . . . . . . . . . 66 - 5.4.3. Disconnect-Cause AVP . . . . . . . . . . . . . . . . 66 - 5.5. Transport Failure Detection . . . . . . . . . . . . . . . 67 - 5.5.1. Device-Watchdog-Request . . . . . . . . . . . . . . . 67 - 5.5.2. Device-Watchdog-Answer . . . . . . . . . . . . . . . 67 - 5.5.3. Transport Failure Algorithm . . . . . . . . . . . . . 68 - 5.5.4. Failover and Failback Procedures . . . . . . . . . . 68 - 5.6. Peer State Machine . . . . . . . . . . . . . . . . . . . 69 - 5.6.1. Incoming connections . . . . . . . . . . . . . . . . 71 - 5.6.2. Events . . . . . . . . . . . . . . . . . . . . . . . 72 - 5.6.3. Actions . . . . . . . . . . . . . . . . . . . . . . . 73 - 5.6.4. The Election Process . . . . . . . . . . . . . . . . 75 - 6. Diameter message processing . . . . . . . . . . . . . . . . . 76 - 6.1. Diameter Request Routing Overview . . . . . . . . . . . . 76 - 6.1.1. Originating a Request . . . . . . . . . . . . . . . . 77 - 6.1.2. Sending a Request . . . . . . . . . . . . . . . . . . 77 - 6.1.3. Receiving Requests . . . . . . . . . . . . . . . . . 78 - 6.1.4. Processing Local Requests . . . . . . . . . . . . . . 78 - 6.1.5. Request Forwarding . . . . . . . . . . . . . . . . . 78 - 6.1.6. Request Routing . . . . . . . . . . . . . . . . . . . 78 - 6.1.7. Predictive Loop Avoidance . . . . . . . . . . . . . . 79 - 6.1.8. Redirecting Requests . . . . . . . . . . . . . . . . 79 - 6.1.9. Relaying and Proxying Requests . . . . . . . . . . . 80 - 6.2. Diameter Answer Processing . . . . . . . . . . . . . . . 82 - 6.2.1. Processing received Answers . . . . . . . . . . . . . 82 - 6.2.2. Relaying and Proxying Answers . . . . . . . . . . . . 82 - 6.3. Origin-Host AVP . . . . . . . . . . . . . . . . . . . . . 83 - 6.4. Origin-Realm AVP . . . . . . . . . . . . . . . . . . . . 83 - 6.5. Destination-Host AVP . . . . . . . . . . . . . . . . . . 83 - 6.6. Destination-Realm AVP . . . . . . . . . . . . . . . . . . 84 - 6.7. Routing AVPs . . . . . . . . . . . . . . . . . . . . . . 84 - 6.7.1. Route-Record AVP . . . . . . . . . . . . . . . . . . 84 - 6.7.2. Proxy-Info AVP . . . . . . . . . . . . . . . . . . . 84 - - - -Fajardo, et al. Expires July 24, 2011 [Page 3] - -Internet-Draft Diameter Base Protocol January 2011 - - - 6.7.3. Proxy-Host AVP . . . . . . . . . . . . . . . . . . . 85 - 6.7.4. Proxy-State AVP . . . . . . . . . . . . . . . . . . . 85 - 6.8. Auth-Application-Id AVP . . . . . . . . . . . . . . . . . 85 - 6.9. Acct-Application-Id AVP . . . . . . . . . . . . . . . . . 85 - 6.10. Inband-Security-Id AVP . . . . . . . . . . . . . . . . . 85 - 6.11. Vendor-Specific-Application-Id AVP . . . . . . . . . . . 86 - 6.12. Redirect-Host AVP . . . . . . . . . . . . . . . . . . . . 87 - 6.13. Redirect-Host-Usage AVP . . . . . . . . . . . . . . . . . 87 - 6.14. Redirect-Max-Cache-Time AVP . . . . . . . . . . . . . . . 88 - 7. Error Handling . . . . . . . . . . . . . . . . . . . . . . . 90 - 7.1. Result-Code AVP . . . . . . . . . . . . . . . . . . . . . 92 - 7.1.1. Informational . . . . . . . . . . . . . . . . . . . . 92 - 7.1.2. Success . . . . . . . . . . . . . . . . . . . . . . . 93 - 7.1.3. Protocol Errors . . . . . . . . . . . . . . . . . . . 93 - 7.1.4. Transient Failures . . . . . . . . . . . . . . . . . 94 - 7.1.5. Permanent Failures . . . . . . . . . . . . . . . . . 95 - 7.2. Error Bit . . . . . . . . . . . . . . . . . . . . . . . . 98 - 7.3. Error-Message AVP . . . . . . . . . . . . . . . . . . . . 99 - 7.4. Error-Reporting-Host AVP . . . . . . . . . . . . . . . . 99 - 7.5. Failed-AVP AVP . . . . . . . . . . . . . . . . . . . . . 99 - 7.6. Experimental-Result AVP . . . . . . . . . . . . . . . . . 100 - 7.7. Experimental-Result-Code AVP . . . . . . . . . . . . . . 100 - 8. Diameter User Sessions . . . . . . . . . . . . . . . . . . . 101 - 8.1. Authorization Session State Machine . . . . . . . . . . . 102 - 8.2. Accounting Session State Machine . . . . . . . . . . . . 107 - 8.3. Server-Initiated Re-Auth . . . . . . . . . . . . . . . . 112 - 8.3.1. Re-Auth-Request . . . . . . . . . . . . . . . . . . . 112 - 8.3.2. Re-Auth-Answer . . . . . . . . . . . . . . . . . . . 113 - 8.4. Session Termination . . . . . . . . . . . . . . . . . . . 114 - 8.4.1. Session-Termination-Request . . . . . . . . . . . . . 115 - 8.4.2. Session-Termination-Answer . . . . . . . . . . . . . 115 - 8.5. Aborting a Session . . . . . . . . . . . . . . . . . . . 116 - 8.5.1. Abort-Session-Request . . . . . . . . . . . . . . . . 116 - 8.5.2. Abort-Session-Answer . . . . . . . . . . . . . . . . 117 - 8.6. Inferring Session Termination from Origin-State-Id . . . 118 - 8.7. Auth-Request-Type AVP . . . . . . . . . . . . . . . . . . 118 - 8.8. Session-Id AVP . . . . . . . . . . . . . . . . . . . . . 119 - 8.9. Authorization-Lifetime AVP . . . . . . . . . . . . . . . 120 - 8.10. Auth-Grace-Period AVP . . . . . . . . . . . . . . . . . . 121 - 8.11. Auth-Session-State AVP . . . . . . . . . . . . . . . . . 121 - 8.12. Re-Auth-Request-Type AVP . . . . . . . . . . . . . . . . 121 - 8.13. Session-Timeout AVP . . . . . . . . . . . . . . . . . . . 122 - 8.14. User-Name AVP . . . . . . . . . . . . . . . . . . . . . . 122 - 8.15. Termination-Cause AVP . . . . . . . . . . . . . . . . . . 122 - 8.16. Origin-State-Id AVP . . . . . . . . . . . . . . . . . . . 124 - 8.17. Session-Binding AVP . . . . . . . . . . . . . . . . . . . 124 - 8.18. Session-Server-Failover AVP . . . . . . . . . . . . . . . 125 - 8.19. Multi-Round-Time-Out AVP . . . . . . . . . . . . . . . . 126 - - - -Fajardo, et al. Expires July 24, 2011 [Page 4] - -Internet-Draft Diameter Base Protocol January 2011 - - - 8.20. Class AVP . . . . . . . . . . . . . . . . . . . . . . . . 126 - 8.21. Event-Timestamp AVP . . . . . . . . . . . . . . . . . . . 126 - 9. Accounting . . . . . . . . . . . . . . . . . . . . . . . . . 127 - 9.1. Server Directed Model . . . . . . . . . . . . . . . . . . 127 - 9.2. Protocol Messages . . . . . . . . . . . . . . . . . . . . 128 - 9.3. Accounting Application Extension and Requirements . . . . 128 - 9.4. Fault Resilience . . . . . . . . . . . . . . . . . . . . 129 - 9.5. Accounting Records . . . . . . . . . . . . . . . . . . . 129 - 9.6. Correlation of Accounting Records . . . . . . . . . . . . 130 - 9.7. Accounting Command-Codes . . . . . . . . . . . . . . . . 131 - 9.7.1. Accounting-Request . . . . . . . . . . . . . . . . . 131 - 9.7.2. Accounting-Answer . . . . . . . . . . . . . . . . . . 132 - 9.8. Accounting AVPs . . . . . . . . . . . . . . . . . . . . . 133 - 9.8.1. Accounting-Record-Type AVP . . . . . . . . . . . . . 133 - 9.8.2. Acct-Interim-Interval AVP . . . . . . . . . . . . . . 134 - 9.8.3. Accounting-Record-Number AVP . . . . . . . . . . . . 135 - 9.8.4. Acct-Session-Id AVP . . . . . . . . . . . . . . . . . 135 - 9.8.5. Acct-Multi-Session-Id AVP . . . . . . . . . . . . . . 135 - 9.8.6. Accounting-Sub-Session-Id AVP . . . . . . . . . . . . 135 - 9.8.7. Accounting-Realtime-Required AVP . . . . . . . . . . 136 - 10. AVP Occurrence Table . . . . . . . . . . . . . . . . . . . . 137 - 10.1. Base Protocol Command AVP Table . . . . . . . . . . . . . 137 - 10.2. Accounting AVP Table . . . . . . . . . . . . . . . . . . 138 - 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 140 - 11.1. Changes to AVP Header Allocation . . . . . . . . . . . . 140 - 11.2. Diameter Header . . . . . . . . . . . . . . . . . . . . . 140 - 11.3. AVP Values . . . . . . . . . . . . . . . . . . . . . . . 140 - 11.3.1. Experimental-Result-Code AVP . . . . . . . . . . . . 140 - 11.4. Diameter TCP, SCTP, TLS/TCP and DTLS/SCTP Port Numbers . 141 - 11.5. S-NAPTR Parameters . . . . . . . . . . . . . . . . . . . 141 - 12. Diameter protocol related configurable parameters . . . . . . 142 - 13. Security Considerations . . . . . . . . . . . . . . . . . . . 143 - 13.1. TLS/TCP and DTLS/SCTP Usage . . . . . . . . . . . . . . . 143 - 13.2. Peer-to-Peer Considerations . . . . . . . . . . . . . . . 144 - 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 145 - 14.1. Normative References . . . . . . . . . . . . . . . . . . 145 - 14.2. Informational References . . . . . . . . . . . . . . . . 147 - Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 149 - A.1. RFC3588bis . . . . . . . . . . . . . . . . . . . . . . . 149 - A.2. RFC3588 . . . . . . . . . . . . . . . . . . . . . . . . . 150 - Appendix B. S-NAPTR Example . . . . . . . . . . . . . . . . . . 151 - Appendix C. Duplicate Detection . . . . . . . . . . . . . . . . 152 - Appendix D. Internationalized Domain Names . . . . . . . . . . . 154 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 155 - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 5] - -Internet-Draft Diameter Base Protocol January 2011 + 1. Introduction ....................................................7 + 1.1. Diameter Protocol ..........................................9 + 1.1.1. Description of the Document Set ....................10 + 1.1.2. Conventions Used in This Document ..................11 + 1.1.3. Changes from RFC 3588 ..............................11 + 1.2. Terminology ...............................................12 + 1.3. Approach to Extensibility .................................17 + 1.3.1. Defining New AVP Values ............................18 + 1.3.2. Creating New AVPs ..................................18 + 1.3.3. Creating New Commands ..............................18 + 1.3.4. Creating New Diameter Applications .................19 + 2. Protocol Overview ..............................................20 + 2.1. Transport .................................................22 + 2.1.1. SCTP Guidelines ....................................23 + 2.2. Securing Diameter Messages ................................24 + 2.3. Diameter Application Compliance ...........................24 + 2.4. Application Identifiers ...................................24 + 2.5. Connections vs. Sessions ..................................25 + 2.6. Peer Table ................................................26 + + + +Fajardo, et al. Standards Track [Page 2] + +RFC 6733 Diameter Base Protocol October 2012 + + + 2.7. Routing Table .............................................27 + 2.8. Role of Diameter Agents ...................................28 + 2.8.1. Relay Agents .......................................30 + 2.8.2. Proxy Agents .......................................31 + 2.8.3. Redirect Agents ....................................31 + 2.8.4. Translation Agents .................................32 + 2.9. Diameter Path Authorization ...............................33 + 3. Diameter Header ................................................34 + 3.1. Command Codes .............................................37 + 3.2. Command Code Format Specification .........................38 + 3.3. Diameter Command Naming Conventions .......................40 + 4. Diameter AVPs ..................................................40 + 4.1. AVP Header ................................................41 + 4.1.1. Optional Header Elements ...........................42 + 4.2. Basic AVP Data Formats ....................................43 + 4.3. Derived AVP Data Formats ..................................44 + 4.3.1. Common Derived AVP Data Formats ....................44 + 4.4. Grouped AVP Values ........................................51 + 4.4.1. Example AVP with a Grouped Data Type ...............52 + 4.5. Diameter Base Protocol AVPs ...............................55 + 5. Diameter Peers .................................................58 + 5.1. Peer Connections ..........................................58 + 5.2. Diameter Peer Discovery ...................................59 + 5.3. Capabilities Exchange .....................................60 + 5.3.1. Capabilities-Exchange-Request ......................62 + 5.3.2. Capabilities-Exchange-Answer .......................63 + 5.3.3. Vendor-Id AVP ......................................63 + 5.3.4. Firmware-Revision AVP ..............................64 + 5.3.5. Host-IP-Address AVP ................................64 + 5.3.6. Supported-Vendor-Id AVP ............................64 + 5.3.7. Product-Name AVP ...................................64 + 5.4. Disconnecting Peer Connections ............................64 + 5.4.1. Disconnect-Peer-Request ............................65 + 5.4.2. Disconnect-Peer-Answer .............................65 + 5.4.3. Disconnect-Cause AVP ...............................66 + 5.5. Transport Failure Detection ...............................66 + 5.5.1. Device-Watchdog-Request ............................67 + 5.5.2. Device-Watchdog-Answer .............................67 + 5.5.3. Transport Failure Algorithm ........................67 + 5.5.4. Failover and Failback Procedures ...................67 + 5.6. Peer State Machine ........................................68 + 5.6.1. Incoming Connections ...............................71 + 5.6.2. Events .............................................71 + 5.6.3. Actions ............................................72 + 5.6.4. The Election Process ...............................74 + + + + + + +Fajardo, et al. Standards Track [Page 3] + +RFC 6733 Diameter Base Protocol October 2012 + + + 6. Diameter Message Processing ....................................74 + 6.1. Diameter Request Routing Overview .........................74 + 6.1.1. Originating a Request ..............................75 + 6.1.2. Sending a Request ..................................76 + 6.1.3. Receiving Requests .................................76 + 6.1.4. Processing Local Requests ..........................76 + 6.1.5. Request Forwarding .................................77 + 6.1.6. Request Routing ....................................77 + 6.1.7. Predictive Loop Avoidance ..........................77 + 6.1.8. Redirecting Requests ...............................78 + 6.1.9. Relaying and Proxying Requests .....................79 + 6.2. Diameter Answer Processing ................................80 + 6.2.1. Processing Received Answers ........................81 + 6.2.2. Relaying and Proxying Answers ......................81 + 6.3. Origin-Host AVP ...........................................81 + 6.4. Origin-Realm AVP ..........................................82 + 6.5. Destination-Host AVP ......................................82 + 6.6. Destination-Realm AVP .....................................82 + 6.7. Routing AVPs ..............................................83 + 6.7.1. Route-Record AVP ...................................83 + 6.7.2. Proxy-Info AVP .....................................83 + 6.7.3. Proxy-Host AVP .....................................83 + 6.7.4. Proxy-State AVP ....................................83 + 6.8. Auth-Application-Id AVP ...................................83 + 6.9. Acct-Application-Id AVP ...................................84 + 6.10. Inband-Security-Id AVP ...................................84 + 6.11. Vendor-Specific-Application-Id AVP .......................84 + 6.12. Redirect-Host AVP ........................................85 + 6.13. Redirect-Host-Usage AVP ..................................85 + 6.14. Redirect-Max-Cache-Time AVP ..............................87 + 7. Error Handling .................................................87 + 7.1. Result-Code AVP ...........................................89 + 7.1.1. Informational ......................................90 + 7.1.2. Success ............................................90 + 7.1.3. Protocol Errors ....................................90 + 7.1.4. Transient Failures .................................92 + 7.1.5. Permanent Failures .................................92 + 7.2. Error Bit .................................................95 + 7.3. Error-Message AVP .........................................96 + 7.4. Error-Reporting-Host AVP ..................................96 + 7.5. Failed-AVP AVP ............................................96 + 7.6. Experimental-Result AVP ...................................97 + 7.7. Experimental-Result-Code AVP ..............................97 + 8. Diameter User Sessions .........................................98 + 8.1. Authorization Session State Machine .......................99 + 8.2. Accounting Session State Machine .........................104 + + + + + +Fajardo, et al. Standards Track [Page 4] + +RFC 6733 Diameter Base Protocol October 2012 + + + 8.3. Server-Initiated Re-Auth .................................110 + 8.3.1. Re-Auth-Request ...................................110 + 8.3.2. Re-Auth-Answer ....................................110 + 8.4. Session Termination ......................................111 + 8.4.1. Session-Termination-Request .......................112 + 8.4.2. Session-Termination-Answer ........................113 + 8.5. Aborting a Session .......................................113 + 8.5.1. Abort-Session-Request .............................114 + 8.5.2. Abort-Session-Answer ..............................114 + 8.6. Inferring Session Termination from Origin-State-Id .......115 + 8.7. Auth-Request-Type AVP ....................................116 + 8.8. Session-Id AVP ...........................................116 + 8.9. Authorization-Lifetime AVP ...............................117 + 8.10. Auth-Grace-Period AVP ...................................118 + 8.11. Auth-Session-State AVP ..................................118 + 8.12. Re-Auth-Request-Type AVP ................................118 + 8.13. Session-Timeout AVP .....................................119 + 8.14. User-Name AVP ...........................................119 + 8.15. Termination-Cause AVP ...................................120 + 8.16. Origin-State-Id AVP .....................................120 + 8.17. Session-Binding AVP .....................................120 + 8.18. Session-Server-Failover AVP .............................121 + 8.19. Multi-Round-Time-Out AVP ................................122 + 8.20. Class AVP ...............................................122 + 8.21. Event-Timestamp AVP .....................................122 + 9. Accounting ....................................................123 + 9.1. Server Directed Model ....................................123 + 9.2. Protocol Messages ........................................124 + 9.3. Accounting Application Extension and Requirements ........124 + 9.4. Fault Resilience .........................................125 + 9.5. Accounting Records .......................................125 + 9.6. Correlation of Accounting Records ........................126 + 9.7. Accounting Command Codes .................................127 + 9.7.1. Accounting-Request ................................127 + 9.7.2. Accounting-Answer .................................128 + 9.8. Accounting AVPs ..........................................129 + 9.8.1. Accounting-Record-Type AVP ........................129 + 9.8.2. Acct-Interim-Interval AVP .........................130 + 9.8.3. Accounting-Record-Number AVP ......................131 + 9.8.4. Acct-Session-Id AVP ...............................131 + 9.8.5. Acct-Multi-Session-Id AVP .........................131 + 9.8.6. Accounting-Sub-Session-Id AVP .....................131 + 9.8.7. Accounting-Realtime-Required AVP ..................132 + 10. AVP Occurrence Tables ........................................132 + 10.1. Base Protocol Command AVP Table .........................133 + 10.2. Accounting AVP Table ....................................134 + + + + + +Fajardo, et al. Standards Track [Page 5] + +RFC 6733 Diameter Base Protocol October 2012 + + + 11. IANA Considerations ..........................................135 + 11.1. AVP Header ..............................................135 + 11.1.1. AVP Codes ........................................136 + 11.1.2. AVP Flags ........................................136 + 11.2. Diameter Header .........................................136 + 11.2.1. Command Codes ....................................136 + 11.2.2. Command Flags ....................................137 + 11.3. AVP Values ..............................................137 + 11.3.1. Experimental-Result-Code AVP .....................137 + 11.3.2. Result-Code AVP Values ...........................137 + 11.3.3. Accounting-Record-Type AVP Values ................137 + 11.3.4. Termination-Cause AVP Values .....................137 + 11.3.5. Redirect-Host-Usage AVP Values ...................137 + 11.3.6. Session-Server-Failover AVP Values ...............137 + 11.3.7. Session-Binding AVP Values .......................137 + 11.3.8. Disconnect-Cause AVP Values ......................138 + 11.3.9. Auth-Request-Type AVP Values .....................138 + 11.3.10. Auth-Session-State AVP Values ...................138 + 11.3.11. Re-Auth-Request-Type AVP Values .................138 + 11.3.12. Accounting-Realtime-Required AVP Values .........138 + 11.3.13. Inband-Security-Id AVP (code 299) ...............138 + 11.4. _diameters Service Name and Port Number Registration ....138 + 11.5. SCTP Payload Protocol Identifiers .......................139 + 11.6. S-NAPTR Parameters ......................................139 + 12. Diameter Protocol-Related Configurable Parameters ............139 + 13. Security Considerations ......................................140 + 13.1. TLS/TCP and DTLS/SCTP Usage .............................140 + 13.2. Peer-to-Peer Considerations .............................141 + 13.3. AVP Considerations ......................................141 + 14. References ...................................................142 + 14.1. Normative References ....................................142 + 14.2. Informative References ..................................144 + Appendix A. Acknowledgements .....................................147 + A.1. This Document .............................................147 + A.2. RFC 3588 ..................................................148 + Appendix B. S-NAPTR Example ......................................148 + Appendix C. Duplicate Detection ..................................149 + Appendix D. Internationalized Domain Names .......................151 + + + + + + + + + + + + + +Fajardo, et al. Standards Track [Page 6] + +RFC 6733 Diameter Base Protocol October 2012 1. Introduction - Authentication, Authorization and Accounting (AAA) protocols such as + Authentication, Authorization, and Accounting (AAA) protocols such as TACACS [RFC1492] and RADIUS [RFC2865] were initially deployed to provide dial-up PPP [RFC1661] and terminal server access. Over time, AAA support was needed on many new access technologies, the scale and complexity of AAA networks grew, and AAA was also used on new - applications (such as voice over IP). This lead to new demands on - AAA protocols. + applications (such as voice over IP). This led to new demands on AAA + protocols. Network access requirements for AAA protocols are summarized in - [RFC2989]. These include: - + Aboba, et al. [RFC2989]. These include: Failover - [RFC2865] does not define failover mechanisms, and as a result, + [RFC2865] does not define failover mechanisms and, as a result, failover behavior differs between implementations. In order to provide well-defined failover behavior, Diameter supports - application-layer acknowledgements, and defines failover - algorithms and the associated state machine. This is described in - Section 5.5 and [RFC3539]. + application-layer acknowledgements and defines failover algorithms + and the associated state machine. Transmission-level security - [RFC2865] defines an application-layer authentication and - integrity scheme that is required only for use with Response + RADIUS [RFC2865] defines an application-layer authentication and + integrity scheme that is required only for use with response packets. While [RFC2869] defines an additional authentication and integrity mechanism, use is only required during Extensible - Authentication Protocol (EAP) sessions. While attribute-hiding is - supported, [RFC2865] does not provide support for per-packet - confidentiality. In accounting, [RFC2866] assumes that replay - protection is provided by the backend billing server, rather than - within the protocol itself. + Authentication Protocol (EAP) [RFC3748] sessions. While attribute + hiding is supported, [RFC2865] does not provide support for per- + packet confidentiality. In accounting, [RFC2866] assumes that + replay protection is provided by the backend billing server rather + than within the protocol itself. While [RFC3162] defines the use of IPsec with RADIUS, support for IPsec is not required. In order to provide universal support for @@ -322,50 +379,48 @@ Internet-Draft Diameter Base Protocol January 2011 domain AAA deployments, Diameter provides support for TLS/TCP and DTLS/SCTP. Security is discussed in Section 13. - Reliable transport - RADIUS runs over UDP, and does not define retransmission behavior; as a result, reliability varies between implementations. As described in [RFC2975], this is a major issue in accounting, where + packet loss may translate directly into revenue loss. In order to -Fajardo, et al. Expires July 24, 2011 [Page 6] - -Internet-Draft Diameter Base Protocol January 2011 - packet loss may translate directly into revenue loss. In order to - provide well defined transport behavior, Diameter runs over - reliable transport mechanisms (TCP, SCTP) as defined in [RFC3539]. +Fajardo, et al. Standards Track [Page 7] + +RFC 6733 Diameter Base Protocol October 2012 - Agent support - [RFC2865] does not provide for explicit support for agents, - including Proxies, Redirects and Relays. Since the expected - behavior is not defined, it varies between implementations. - Diameter defines agent behavior explicitly; this is described in - Section 2.8. + provide well-defined transport behavior, Diameter runs over + reliable transport mechanisms (TCP, Stream Control Transmission + Protocol (SCTP)) as defined in [RFC3539]. + + Agent support + RADIUS does not provide for explicit support for agents, including + proxies, redirects, and relays. Since the expected behavior is + not defined, it varies between implementations. Diameter defines + agent behavior explicitly; this is described in Section 2.8. Server-initiated messages - While RADIUS server-initiated messages are defined in [RFC5176], + While server-initiated messages are defined in RADIUS [RFC5176], support is optional. This makes it difficult to implement features such as unsolicited disconnect or re-authentication/ re-authorization on demand across a heterogeneous deployment. To - tackle this issue, support for server-initiated messages is + address this issue, support for server-initiated messages is mandatory in Diameter. - Transition support While Diameter does not share a common protocol data unit (PDU) with RADIUS, considerable effort has been expended in enabling - backward compatibility with RADIUS, so that the two protocols may + backward compatibility with RADIUS so that the two protocols may be deployed in the same network. Initially, it is expected that Diameter will be deployed within new network devices, as well as within gateways enabling communication between legacy RADIUS @@ -373,9 +428,8 @@ Internet-Draft Diameter Base Protocol January 2011 support to be added to legacy networks, by addition of a gateway or server speaking both RADIUS and Diameter. - In addition to addressing the above requirements, Diameter also - provides support for the following: - + In addition to addressing the above requirements, Diameter also + provides support for the following: Capability negotiation @@ -383,19 +437,19 @@ Internet-Draft Diameter Base Protocol January 2011 a mandatory/non-mandatory flag for attributes. Since RADIUS clients and servers are not aware of each other's capabilities, they may not be able to successfully negotiate a mutually - acceptable service, or in some cases, even be aware of what + acceptable service or, in some cases, even be aware of what service has been implemented. Diameter includes support for error + handling (Section 7), capability negotiation (Section 5.3), and + mandatory/non-mandatory Attribute-Value Pairs (AVPs) + (Section 4.1). -Fajardo, et al. Expires July 24, 2011 [Page 7] - -Internet-Draft Diameter Base Protocol January 2011 - handling (Section 7), capability negotiation (Section 5.3), and - mandatory/non-mandatory Attribute-Value Pairs (AVPs) (Section - 4.1). +Fajardo, et al. Standards Track [Page 8] + +RFC 6733 Diameter Base Protocol October 2012 Peer discovery and configuration @@ -403,7 +457,7 @@ Internet-Draft Diameter Base Protocol January 2011 RADIUS implementations typically require that the name or address of servers or clients be manually configured, along with the corresponding shared secrets. This results in a large - administrative burden, and creates the temptation to reuse the + administrative burden and creates the temptation to reuse the RADIUS shared secret, which can result in major security vulnerabilities if the Request Authenticator is not globally and temporally unique as required in [RFC2865]. Through DNS, Diameter @@ -411,7 +465,6 @@ Internet-Draft Diameter Base Protocol January 2011 of dynamic session keys is enabled via transmission-level security. - Over time, the capabilities of Network Access Server (NAS) devices have increased substantially. As a result, while Diameter is a considerably more sophisticated protocol than RADIUS, it remains @@ -427,42 +480,41 @@ Internet-Draft Diameter Base Protocol January 2011 o Error notification - o Extensibility, through addition of new applications, commands and - AVPs (required in [RFC2989]). + o Extensibility, required in [RFC2989], through addition of new + applications, commands, and AVPs - o Basic services necessary for applications, such as handling of + o Basic services necessary for applications, such as the handling of user sessions or accounting All data delivered by the protocol is in the form of AVPs. Some of these AVP values are used by the Diameter protocol itself, while others deliver data associated with particular applications that employ Diameter. AVPs may be arbitrarily added to Diameter messages, - the only restriction being that the Augmented Backus-Naur Form (ABNF, - [RFC5234]) Command Code syntax specification (Section 3.2) is - satisfied. AVPs are used by the base Diameter protocol to support - the following required features: + the only restriction being that the Command Code Format (CCF) + specification (Section 3.2) be satisfied. AVPs are used by the base + Diameter protocol to support the following required features: + o Transporting of user authentication information, for the purposes + of enabling the Diameter server to authenticate the user + o Transporting of service-specific authorization information, + between client and servers, allowing the peers to decide whether a + user's access request should be granted -Fajardo, et al. Expires July 24, 2011 [Page 8] - -Internet-Draft Diameter Base Protocol January 2011 - o Transporting of user authentication information, for the purposes - of enabling the Diameter server to authenticate the user. +Fajardo, et al. Standards Track [Page 9] + +RFC 6733 Diameter Base Protocol October 2012 - o Transporting of service-specific authorization information, - between client and servers, allowing the peers to decide whether a - user's access request should be granted. o Exchanging resource usage information, which may be used for accounting purposes, capacity planning, etc. - o Routing, relaying, proxying and redirecting of Diameter messages - through a server hierarchy. + o Routing, relaying, proxying, and redirecting of Diameter messages + through a server hierarchy - The Diameter base protocol satisfies the minimum requirements for an + The Diameter base protocol satisfies the minimum requirements for a AAA protocol, as specified by [RFC2989]. The base protocol may be used by itself for accounting purposes only, or it may be used with a Diameter application, such as Mobile IPv4 [RFC4004], or network @@ -473,16 +525,16 @@ Internet-Draft Diameter Base Protocol January 2011 many applications might provide functionality not provided by Diameter. Therefore, it is imperative that the designers of new applications understand their requirements before using Diameter. - See Section 2.4 for more information on Diameter applications. + See Section 1.3.4 for more information on Diameter applications. Any node can initiate a request. In that sense, Diameter is a peer- - to-peer protocol. In this document, a Diameter Client is a device at + to-peer protocol. In this document, a Diameter client is a device at the edge of the network that performs access control, such as a Network Access Server (NAS) or a Foreign Agent (FA). A Diameter client generates Diameter messages to request authentication, authorization, and accounting services for the user. A Diameter agent is a node that does not provide local user authentication or - authorization services; agents include proxies, redirects and relay + authorization services; agents include proxies, redirects, and relay agents. A Diameter server performs authentication and/or authorization of the user. A Diameter node may act as an agent for certain requests while acting as a server for others. @@ -494,33 +546,33 @@ Internet-Draft Diameter Base Protocol January 2011 The Diameter specification consists of an updated version of the base protocol specification (this document) and the Transport Profile - [RFC3539]. This document obsoletes RFC 3588. A summary of the base - protocol updates included in this document can be found in - Section 1.1.3. - - - -Fajardo, et al. Expires July 24, 2011 [Page 9] - -Internet-Draft Diameter Base Protocol January 2011 - + [RFC3539]. This document obsoletes both RFC 3588 and RFC 5719. A + summary of the base protocol updates included in this document can be + found in Section 1.1.3. This document defines the base protocol specification for AAA, which includes support for accounting. There are also a myriad of applications documents describing applications that use this base - specification for Authentication, Authorization and Accounting. + specification for Authentication, Authorization, and Accounting. These application documents specify how to use the Diameter protocol within the context of their application. + + +Fajardo, et al. Standards Track [Page 10] + +RFC 6733 Diameter Base Protocol October 2012 + + The Transport Profile document [RFC3539] discusses transport layer issues that arise with AAA protocols and recommendations on how to overcome these issues. This document also defines the Diameter failover algorithm and state machine. - Clarifications on the Routing of Diameter Request based on Username - and the Realm [RFC5729] defines specific behavior on how to route - request based on the content of the User-Name AVP (Attribute Value - Pair). + "Clarifications on the Routing of Diameter Request Based on the + Username and the Realm" [RFC5729] defines specific behavior on how to + route requests based on the content of the User-Name AVP (Attribute + Value Pair). 1.1.2. Conventions Used in This Document @@ -528,100 +580,100 @@ Internet-Draft Diameter Base Protocol January 2011 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. -1.1.3. Changes from RFC3588 +1.1.3. Changes from RFC 3588 This document obsoletes RFC 3588 but is fully backward compatible with that document. The changes introduced in this document focus on - fixing issues that have surfaced during implementation of [RFC3588]. - An overview of some the major changes are given below. - - o Deprecated the use of Inband-Security AVP for negotiating - transport layer security. It has been generally considered that - bootstrapping of TLS via Inband-Security AVP creates certain - security risk because it does not completely protect the - information carried in the CER (Capabilities Exchange Request)/CEA - (Capabilities Exchange Answer). This version of Diameter adopted - a common approach of defining a well-known secured port that peers - should use when communicating via TLS/TCP and DTLS/SCTP. This new - approach augments the existing Inband-Security negotiation but - does not completely replace it. The old method is kept for - backwards compatibility reasons. + fixing issues that have surfaced during the implementation of + Diameter (RFC 3588). An overview of some the major changes are given + below. + + o Deprecated the use of the Inband-Security AVP for negotiating + Transport Layer Security (TLS) [RFC5246]. It has been generally + considered that bootstrapping of TLS via Inband-Security AVP + creates certain security risks because it does not completely + protect the information carried in the CER/CEA (Capabilities- + Exchange-Request/Capabilities-Exchange-Answer). This version of + Diameter adopts the common approach of defining a well-known + secured port that peers should use when communicating via TLS/TCP + and DTLS/SCTP. This new approach augments the existing in-band + security negotiation, but it does not completely replace it. The + old method is kept for backward compatibility reasons. o Deprecated the exchange of CER/CEA messages in the open state. - This feature was implied in the peer state machine table of - [RFC3588] but it was not clearly defined anywhere else in that + This feature was implied in the peer state machine table of RFC + 3588, but it was not clearly defined anywhere else in that document. As work on this document progressed, it became clear - that the multiplicity of meaning and use of Application Id AVPs in + that the multiplicity of meaning and use of Application-Id AVPs in the CER/CEA messages (and the messages themselves) is seen as an + abuse of the Diameter extensibility rules and thus required + simplification. Capabilities exchange in the open state has been + re-introduced in a separate specification [RFC6737], which clearly + defines new commands for this feature. -Fajardo, et al. Expires July 24, 2011 [Page 10] - -Internet-Draft Diameter Base Protocol January 2011 - abuse of the Diameter extensibility rules and thus required - simplification. It is assumed that the capabilities exchange in - the open state will be re-introduced in a separate specification - which clearly defines new commands for this feature. +Fajardo, et al. Standards Track [Page 11] + +RFC 6733 Diameter Base Protocol October 2012 + - o Simplified Security Requirements. The use of a secured transport + o Simplified security requirements. The use of a secured transport for exchanging Diameter messages remains mandatory. However, TLS/ - TCP and DTLS/SCTP has become the primary method of securing - Diameter and IPsec is a secondary alternative. See Section 13 for - details. The support for the End-to-End security framework - (E2ESequence AVP and 'P'-bit in the AVP header) has also been + TCP and DTLS/SCTP have become the primary methods of securing + Diameter with IPsec as a secondary alternative. See Section 13 + for details. The support for the End-to-End security framework + (E2E-Sequence AVP and 'P'-bit in the AVP header) has also been deprecated. - o Diameter Extensibility Changes. This includes fixes to the + o Changed Diameter extensibility. This includes fixes to the Diameter extensibility description (Section 1.3 and others) to better aid Diameter application designers; in addition, the new specification relaxes the policy with respect to the allocation of - command codes for vendor-specific uses. - - o Application Id Usage. Clarify the proper use of Application Id - information which can be found in multiple places within a - Diameter message. This includes correlating Application Ids found - in the message headers and AVPs. These changes also clearly - specify the proper Application Id value to use for specific base - protocol messages (ASR/ASA, STR/STA) as well as clarifying the - content and use of Vendor-Specific-Application-Id. - - o Routing Fixes. This document more clearly specifies what - information (AVPs and Application Id) can be used for making + Command Codes for vendor-specific uses. + + o Clarified Application Id usage. Clarify the proper use of + Application Id information, which can be found in multiple places + within a Diameter message. This includes correlating Application + Ids found in the message headers and AVPs. These changes also + clearly specify the proper Application Id value to use for + specific base protocol messages (ASR/ASA, STR/STA) as well as + clarify the content and use of Vendor-Specific-Application-Id. + + o Clarified routing fixes. This document more clearly specifies + what information (AVPs and Application Ids) can be used for making general routing decisions. A rule for the prioritization of redirect routing criteria when multiple route entries are found - via redirects has also been added (See Section 6.13 for details). + via redirects has also been added (see Section 6.13). - o Simplification of Diameter Peer Discovery. The Diameter discovery + o Simplified Diameter peer discovery. The Diameter discovery process now supports only widely used discovery schemes; the rest have been deprecated (see Section 5.2 for details). - There are many other many miscellaneous fixes that have been - introduced in this document that may not be considered significant - but they are important nonetheless. Examples are removal of obsolete - types, fixes to command ABNFs, fixes to the state machine, - clarification of the election process, message validation, fixes to - Failed-AVP and Result-Code AVP values, etc. A comprehensive list of - changes is not shown here for practical reasons. - - - + There are many other miscellaneous fixes that have been introduced in + this document that may not be considered significant, but they have + value nonetheless. Examples are removal of obsolete types, fixes to + the state machine, clarification of the election process, message + validation, fixes to Failed-AVP and Result-Code AVP values, etc. All + of the errata filed against RFC 3588 prior to the publication of this + document have been addressed. A comprehensive list of changes is not + shown here for practical reasons. +1.2. Terminology + AAA + Authentication, Authorization, and Accounting. -Fajardo, et al. Expires July 24, 2011 [Page 11] - -Internet-Draft Diameter Base Protocol January 2011 -1.2. Terminology - AAA - Authentication, Authorization and Accounting. +Fajardo, et al. Standards Track [Page 12] + +RFC 6733 Diameter Base Protocol October 2012 ABNF @@ -631,14 +683,12 @@ Internet-Draft Diameter Base Protocol January 2011 is used to define message exchanges in a bi-directional communications protocol. - Accounting The act of collecting information on resource usage for the - purpose of capacity planning, auditing, billing or cost + purpose of capacity planning, auditing, billing, or cost allocation. - Accounting Record An accounting record represents a summary of the resource @@ -647,114 +697,102 @@ Internet-Draft Diameter Base Protocol January 2011 accounting events or accounting events from several devices serving the same user. - Authentication The act of verifying the identity of an entity (subject). - Authorization The act of determining whether a requesting entity (subject) will be allowed access to a resource (object). - - AVP + Attribute-Value Pair (AVP) The Diameter protocol consists of a header followed by one or more Attribute-Value-Pairs (AVPs). An AVP includes a header and is used to encapsulate protocol-specific data (e.g., routing - information) as well as authentication, authorization or + information) as well as authentication, authorization, or + accounting information. + Command Code Format (CCF) + A modified form of ABNF used to define Diameter commands (see + Section 3.2). -Fajardo, et al. Expires July 24, 2011 [Page 12] - -Internet-Draft Diameter Base Protocol January 2011 + Diameter Agent + A Diameter Agent is a Diameter node that provides relay, proxy, + redirect, or translation services. - accounting information. - Diameter Agent - A Diameter Agent is a Diameter Node that provides either relay, - proxy, redirect or translation services. +Fajardo, et al. Standards Track [Page 13] + +RFC 6733 Diameter Base Protocol October 2012 Diameter Client - A Diameter Client is a Diameter Node that supports Diameter client - applications as well as the base protocol. Diameter Clients are + A Diameter client is a Diameter node that supports Diameter client + applications as well as the base protocol. Diameter clients are often implemented in devices situated at the edge of a network and provide access control services for that network. Typical - examples of Diameter Clients include the Network Access Server + examples of Diameter clients include the Network Access Server (NAS) and the Mobile IP Foreign Agent (FA). - Diameter Node - A Diameter Node is a host process that implements the Diameter - protocol, and acts either as a Client, Agent or Server. - + A Diameter node is a host process that implements the Diameter + protocol and acts as either a client, an agent, or a server. Diameter Peer - If a Diameter Node shares a direct transport connection with - another Diameter Node, it is a Diameter Peer to that Diameter - Node. - + Two Diameter nodes sharing a direct TCP or SCTP transport + connection are called Diameter peers. Diameter Server - A Diameter Server is a Diameter Node that handles authentication, - authorization and accounting requests for a particular realm. By - its very nature, a Diameter Server must support Diameter server + A Diameter server is a Diameter node that handles authentication, + authorization, and accounting requests for a particular realm. By + its very nature, a Diameter server must support Diameter server applications in addition to the base protocol. - Downstream Downstream is used to identify the direction of a particular - Diameter message from the Home Server towards the Diameter Client. - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 13] - -Internet-Draft Diameter Base Protocol January 2011 - + Diameter message from the home server towards the Diameter client. Home Realm A Home Realm is the administrative domain with which the user maintains an account relationship. - Home Server - A Diameter Server which serves the Home Realm. - + A Diameter server that serves the Home Realm. - Interim accounting + Interim Accounting An interim accounting message provides a snapshot of usage during - a user's session. It is typically implemented in order to provide - for partial accounting of a user's session in the case a device - reboot or other network problem prevents the delivery of a session - summary message or session record. + a user's session. Typically, it is implemented in order to + provide for partial accounting of a user's session in case a + device reboot or other network problem prevents the delivery of a + session summary message or session record. + + + + +Fajardo, et al. Standards Track [Page 14] + +RFC 6733 Diameter Base Protocol October 2012 Local Realm A local realm is the administrative domain providing services to a user. An administrative domain may act as a local realm for - certain users, while being a home realm for others. - + certain users while being a home realm for others. Multi-session @@ -764,60 +802,56 @@ Internet-Draft Diameter Base Protocol January 2011 leg of the bundle would be a session while the entire bundle would be a multi-session. - Network Access Identifier The Network Access Identifier, or NAI [RFC4282], is used in the Diameter protocol to extract a user's identity and realm. The identity is used to identify the user during authentication and/or - authorization, while the realm is used for message routing + authorization while the realm is used for message routing purposes. - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 14] - -Internet-Draft Diameter Base Protocol January 2011 - - Proxy Agent or Proxy In addition to forwarding requests and responses, proxies make policy decisions relating to resource usage and provisioning. - This is typically accomplished by tracking the state of NAS - devices. While proxies typically do not respond to client - Requests prior to receiving a Response from the server, they may - originate Reject messages in cases where policies are violated. - As a result, proxies need to understand the semantics of the - messages passing through them, and may not support all Diameter + Typically, this is accomplished by tracking the state of NAS + devices. While proxies usually do not respond to client requests + prior to receiving a response from the server, they may originate + Reject messages in cases where policies are violated. As a + result, proxies need to understand the semantics of the messages + passing through them, and they may not support all Diameter applications. - Realm The string in the NAI that immediately follows the '@' character. - NAI realm names are required to be unique, and are piggybacked on + NAI realm names are required to be unique and are piggybacked on the administration of the DNS namespace. Diameter makes use of the realm, also loosely referred to as domain, to determine - whether messages can be satisfied locally, or whether they must be + whether messages can be satisfied locally or whether they must be routed or redirected. In RADIUS, realm names are not necessarily piggybacked on the DNS namespace but may be independent of it. - Real-time Accounting + + + + + + +Fajardo, et al. Standards Track [Page 15] + +RFC 6733 Diameter Base Protocol October 2012 + + + Real-Time Accounting Real-time accounting involves the processing of information on - resource usage within a defined time window. Time constraints are - typically imposed in order to limit financial risk. The Diameter - Credit Control Application [RFC4006] is an example of an + resource usage within a defined time window. Typically, time + constraints are imposed in order to limit financial risk. The + Diameter Credit-Control Application [RFC4006] is an example of an application that defines real-time accounting functionality. - Relay Agent or Relay Relays forward requests and responses based on routing-related @@ -827,20 +861,9 @@ Internet-Draft Diameter Base Protocol January 2011 the semantics of messages or non-routing AVPs, and are capable of handling any Diameter application or message type. Since relays make decisions based on information in routing AVPs and realm - forwarding tables they do not keep state on NAS resource usage or + forwarding tables, they do not keep state on NAS resource usage or sessions in progress. - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 15] - -Internet-Draft Diameter Base Protocol January 2011 - - Redirect Agent Rather than forwarding requests and responses between clients and @@ -850,11 +873,10 @@ Internet-Draft Diameter Base Protocol January 2011 client and server. Redirect agents do not originate messages and are capable of handling any message type, although they may be configured only to redirect messages of certain types, while - acting as relay or proxy agents for other types. As with proxy + acting as relay or proxy agents for other types. As with relay agents, redirect agents do not keep state with respect to sessions or NAS resources. - Session A session is a related progression of events devoted to a @@ -863,14 +885,19 @@ Internet-Draft Diameter Base Protocol January 2011 packets with the same Session-Id are considered to be part of the same session. - Stateful Agent A stateful agent is one that maintains session state information, by keeping track of all authorized active sessions. Each authorized session is bound to a particular service, and its state - is considered active either until it is notified otherwise, or by - expiration. + is considered active either until it is notified otherwise or + until expiration. + + + +Fajardo, et al. Standards Track [Page 16] + +RFC 6733 Diameter Base Protocol October 2012 Sub-session @@ -881,53 +908,33 @@ Internet-Draft Diameter Base Protocol January 2011 during the same session) or serially. These changes in sessions are tracked with the Accounting-Sub-Session-Id. - - Transaction state + Transaction State The Diameter protocol requires that agents maintain transaction state, which is used for failover purposes. Transaction state - implies that upon forwarding a request, the Hop-by-Hop identifier + implies that upon forwarding a request, the Hop-by-Hop Identifier is saved; the field is replaced with a locally unique identifier, which is restored to its original value when the corresponding - - - -Fajardo, et al. Expires July 24, 2011 [Page 16] - -Internet-Draft Diameter Base Protocol January 2011 - - answer is received. The request's state is released upon receipt of the answer. A stateless agent is one that only maintains transaction state. - Translation Agent - A translation agent is a stateful Diameter node that performs - protocol translation between Diameter and another AAA protocol, - such as RADIUS. - - - Transport Connection - - A transport connection is a TCP or SCTP connection existing - directly between two Diameter peers, otherwise known as a Peer-to- - Peer Connection. - + A translation agent (TLA in Figure 4) is a stateful Diameter node + that performs protocol translation between Diameter and another + AAA protocol, such as RADIUS. Upstream Upstream is used to identify the direction of a particular - Diameter message from the Diameter Client towards the Home Server. - + Diameter message from the Diameter client towards the home server. User The entity or device requesting or using some resource, in support of which a Diameter client has generated a request. - 1.3. Approach to Extensibility The Diameter protocol is designed to be extensible, using several @@ -941,107 +948,108 @@ Internet-Draft Diameter Base Protocol January 2011 o Creating new applications - From the point of view of extensibility Diameter authentication, - authorization and accounting applications are treated in the same - way. - -Fajardo, et al. Expires July 24, 2011 [Page 17] +Fajardo, et al. Standards Track [Page 17] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - Note: Protocol designers should try to re-use existing functionality, + From the point of view of extensibility, Diameter authentication, + authorization, and accounting applications are treated in the same + way. + + Note: Protocol designers should try to reuse existing functionality, namely AVP values, AVPs, commands, and Diameter applications. Reuse simplifies standardization and implementation. To avoid potential - interoperability issues it is important to ensure that the semantics - of the re-used features are well understood. Given that Diameter can - also carry RADIUS attributes as Diameter AVPs, such re-use - considerations apply also to existing RADIUS attributes that may be + interoperability issues, it is important to ensure that the semantics + of the reused features are well understood. Given that Diameter can + also carry RADIUS attributes as Diameter AVPs, such reuse + considerations also apply to existing RADIUS attributes that may be useful in a Diameter application. -1.3.1. Defining New AVP Values +1.3.1. Defining New AVP Values In order to allocate a new AVP value for AVPs defined in the Diameter - Base protocol, the IETF needs to approve a new RFC that describes the + base protocol, the IETF needs to approve a new RFC that describes the AVP value. IANA considerations for these AVP values are discussed in - Section 11.4. + Section 11.3. The allocation of AVP values for other AVPs is guided by the IANA considerations of the document that defines those AVPs. Typically, - allocation of new values for an AVP defined in an IETF RFC should - require IETF Review [RFC5226], whereas values for vendor-specific - AVPs can be allocated by the vendor. + allocation of new values for an AVP defined in an RFC would require + IETF Review [RFC5226], whereas values for vendor-specific AVPs can be + allocated by the vendor. -1.3.2. Creating New AVPs +1.3.2. Creating New AVPs A new AVP being defined MUST use one of the data types listed in - Section 4.2 or Section 4.3. If an appropriate derived data type is - already defined, it SHOULD be used instead of a base data type to - encourage reusability and good design practice. + Sections 4.2 or 4.3. If an appropriate derived data type is already + defined, it SHOULD be used instead of a base data type to encourage + reusability and good design practice. In the event that a logical grouping of AVPs is necessary, and multiple "groups" are possible in a given command, it is recommended that a Grouped AVP be used (see Section 4.4). The creation of new AVPs can happen in various ways. The recommended - approach is to define a new general-purpose AVP in a standards track - RFC approved by the IETF. However, as described in Section 11.1.1 - there are also other mechanisms. + approach is to define a new general-purpose AVP in a Standards Track + RFC approved by the IETF. However, as described in Section 11.1.1, + there are other mechanisms. -1.3.3. Creating New Commands +1.3.3. Creating New Commands A new Command Code MUST be allocated when required AVPs (those - indicated as {AVP} in the ABNF definition) are added to, deleted from + indicated as {AVP} in the CCF definition) are added to, deleted from, or redefined in (for example, by changing a required AVP into an optional one) an existing command. - Furthermore, if the transport characteristics of a command are - changed (for example, with respect to the number of round trips - required) a new Command Code MUST be registered. - -Fajardo, et al. Expires July 24, 2011 [Page 18] +Fajardo, et al. Standards Track [Page 18] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + Furthermore, if the transport characteristics of a command are + changed (for example, with respect to the number of round trips + required), a new Command Code MUST be registered. - A change to the ABNF of a command, such as described above, MUST + A change to the CCF of a command, such as described above, MUST result in the definition of a new Command Code. This subsequently - leads to the need to define a new Diameter Application for any - application that will use that new Command. - - The IANA considerations for commands are discussed in Section 11.2.1. - -1.3.4. Creating New Diameter Applications - - Every Diameter application specification MUST have an IANA assigned - Application Id (see Section 2.4 and Section 11.3). The managed - Application Id space is flat and there is no relationship between - different Diameter applications with respect to their Application - Ids. As such, there is no versioning support provided by these - application Ids itself; every Diameter application is a standalone - application. If the application has a relationship with other - Diameter applications, such a relationship is not known to Diameter. - - Before describing the rules for creating new Diameter applications it - is important to discuss the semantics of the AVPs occurrences as - stated in the ABNF and the M-bit flag (Section 4.1) for an AVP. - There is no relationship imposed between the two; they are set + leads to the need to define a new Diameter application for any + application that will use that new command. + + The IANA considerations for Command Codes are discussed in + Section 3.1. + +1.3.4. Creating New Diameter Applications + + Every Diameter application specification MUST have an IANA-assigned + Application Id (see Section 2.4). The managed Application ID space + is flat, and there is no relationship between different Diameter + applications with respect to their Application Ids. As such, there + is no versioning support provided by these Application Ids + themselves; every Diameter application is a standalone application. + If the application has a relationship with other Diameter + applications, such a relationship is not known to Diameter. + + Before describing the rules for creating new Diameter applications, + it is important to discuss the semantics of the AVP occurrences as + stated in the CCF and the M-bit flag (Section 4.1) for an AVP. There + is no relationship imposed between the two; they are set independently. - o The ABNF indicates what AVPs are placed into a Diameter Command by - the sender of that Command. Often, since there are multiple modes - of protocol interactions many of the AVPs are indicated as + o The CCF indicates what AVPs are placed into a Diameter command by + the sender of that command. Often, since there are multiple modes + of protocol interactions, many of the AVPs are indicated as optional. o The M-bit allows the sender to indicate to the receiver whether or not understanding the semantics of an AVP and its content is mandatory. If the M-bit is set by the sender and the receiver - does not understand the AVP or the values carried within that AVP + does not understand the AVP or the values carried within that AVP, then a failure is generated (see Section 7). It is the decision of the protocol designer when to develop a new @@ -1050,77 +1058,48 @@ Internet-Draft Diameter Base Protocol January 2011 of the following criteria are met: - M-bit Setting - An AVP with the M-bit in the MUST column of the AVP flag table is - added to an existing Command/Application. - An AVP with the M-bit in the MAY column of the AVP flag table is - added to an existing Command/Application. -Fajardo, et al. Expires July 24, 2011 [Page 19] +Fajardo, et al. Standards Track [Page 19] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + M-bit Setting + + An AVP with the M-bit in the MUST column of the AVP flag table is + added to an existing Command/Application. An AVP with the M-bit + in the MAY column of the AVP flag table is added to an existing + Command/Application. Note: The M-bit setting for a given AVP is relevant to an - Application and each command within that application which - includes the AVP. That is, if an AVP appears in two commands for + Application and each command within that application that includes + the AVP. That is, if an AVP appears in two commands for application Foo and the M-bit settings are different in each command, then there should be two AVP flag tables describing when to set the M-bit. Commands - A new command is used within the existing application either - because an additional command is added, an existing command has + A new command is used within the existing application because + either an additional command is added, an existing command has been modified so that a new Command Code had to be registered, or a command has been deleted. - If the ABNF definition of a command allows it, an implementation may + AVP Flag bits + + If an existing application changes the meaning/semantics of its + AVP Flags or adds new flag bits, then a new Diameter application + MUST be created. + + If the CCF definition of a command allows it, an implementation may add arbitrary optional AVPs with the M-bit cleared (including vendor- specific AVPs) to that command without needing to define a new application. Please refer to Section 11.1.1 for details. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 20] - -Internet-Draft Diameter Base Protocol January 2011 - - 2. Protocol Overview The base Diameter protocol concerns itself with establishing @@ -1137,9 +1116,17 @@ Internet-Draft Diameter Base Protocol January 2011 The initial request for authentication and/or authorization of a user would include the Session-Id AVP. The Session-Id is then used in all subsequent messages to identify the user's session (see Section 8 for - more information). The communicating party may accept the request, - or reject it by returning an answer message with the Result-Code AVP - set to indicate an error occurred. The specific behavior of the + + + +Fajardo, et al. Standards Track [Page 20] + +RFC 6733 Diameter Base Protocol October 2012 + + + more information). The communicating party may accept the request or + reject it by returning an answer message with the Result-Code AVP set + to indicate that an error occurred. The specific behavior of the Diameter server or client receiving a request depends on the Diameter application employed. @@ -1153,30 +1140,23 @@ Internet-Draft Diameter Base Protocol January 2011 applications. For authentication and authorization, it is always extended for a particular application. - Diameter Clients MUST support the base protocol, which includes + Diameter clients MUST support the base protocol, which includes accounting. In addition, they MUST fully support each Diameter application that is needed to implement the client's service, e.g., - NASREQ and/or Mobile IPv4. A Diameter Client MUST be referred to as - "Diameter X Client" where X is the application which it supports, and - not a "Diameter Client". + Network Access Server Requirements (NASREQ) [RFC2881] and/or Mobile + IPv4. A Diameter client MUST be referred to as "Diameter X Client" + where X is the application that it supports and not a "Diameter + Client". - Diameter Servers MUST support the base protocol, which includes + Diameter servers MUST support the base protocol, which includes accounting. In addition, they MUST fully support each Diameter application that is needed to implement the intended service, e.g., - NASREQ and/or Mobile IPv4. A Diameter Server MUST be referred to as - "Diameter X Server" where X is the application which it supports, and + NASREQ and/or Mobile IPv4. A Diameter server MUST be referred to as + "Diameter X Server" where X is the application that it supports, and not a "Diameter Server". - Diameter Relays and redirect agents are transparent to the Diameter - applications but they MUST support the Diameter base protocol, which - - - -Fajardo, et al. Expires July 24, 2011 [Page 21] - -Internet-Draft Diameter Base Protocol January 2011 - - + Diameter relays and redirect agents are transparent to the Diameter + applications, but they MUST support the Diameter base protocol, which includes accounting, and all Diameter applications. Diameter proxies MUST support the base protocol, which includes @@ -1186,55 +1166,75 @@ Internet-Draft Diameter Base Protocol January 2011 "Diameter X Proxy" where X is the application which it supports, and not a "Diameter Proxy". + + + + + + + + + +Fajardo, et al. Standards Track [Page 21] + +RFC 6733 Diameter Base Protocol October 2012 + + 2.1. Transport The Diameter Transport profile is defined in [RFC3539]. - The base Diameter protocol is run on port 3868 for both TCP [RFC793] - and SCTP [RFC4960]. For TLS [RFC5246] and DTLS [RFC4347], a Diameter - node that initiate a connection prior to any message exchanges MUST - run on port [TBD]. It is assumed that TLS is run on top of TCP when - it is used and DTLS is run on top of SCTP when it is used. + The base Diameter protocol is run on port 3868 for both TCP [RFC0793] + and SCTP [RFC4960]. For TLS [RFC5246] and Datagram Transport Layer + Security (DTLS) [RFC6347], a Diameter node that initiates a + connection prior to any message exchanges MUST run on port 5658. It + is assumed that TLS is run on top of TCP when it is used, and DTLS is + run on top of SCTP when it is used. If the Diameter peer does not support receiving TLS/TCP and DTLS/SCTP - connections on port [TBD], i.e. the peer complies only with - [RFC3588], then the initiator MAY revert to using TCP or SCTP and on - port 3868. Note that this scheme is kept for the purpose of - backwards compatibility only and that there are inherent security - vulnerabilities when the initial CER/CEA messages are sent un- - protected (see Section 5.6). + connections on port 5658 (i.e., the peer complies only with RFC + 3588), then the initiator MAY revert to using TCP or SCTP on port + 3868. Note that this scheme is kept only for the purpose of backward + compatibility and that there are inherent security vulnerabilities + when the initial CER/CEA messages are sent unprotected (see + Section 5.6). - Diameter clients MUST support either TCP or SCTP, while agents and - servers SHOULD support both. + Diameter clients MUST support either TCP or SCTP; agents and servers + SHOULD support both. A Diameter node MAY initiate connections from a source port other than the one that it declares it accepts incoming connections on, and - MUST be prepared to receive connections on port 3868 for TCP or SCTP - and port [TBD] for TLS/TCP and DTLS/SCTP connections. A given - Diameter instance of the peer state machine MUST NOT use more than - one transport connection to communicate with a given peer, unless - multiple instances exist on the peer in which case a separate - connection per process is allowed. + it MUST always be prepared to receive connections on port 3868 for + TCP or SCTP and port 5658 for TLS/TCP and DTLS/SCTP connections. + When DNS-based peer discovery (Section 5.2) is used, the port numbers + received from SRV records take precedence over the default ports + (3868 and 5658). + + A given Diameter instance of the peer state machine MUST NOT use more + than one transport connection to communicate with a given peer, + unless multiple instances exist on the peer, in which, case a + separate connection per process is allowed. When no transport connection exists with a peer, an attempt to - connect SHOULD be periodically made. This behavior is handled via + connect SHOULD be made periodically. This behavior is handled via the Tc timer (see Section 12 for details), whose recommended value is 30 seconds. There are certain exceptions to this rule, such as when a peer has terminated the transport connection stating that it does not wish to communicate. When connecting to a peer and either zero or more transports are - specified, TLS SHOULD be tried first, followed by DTLS, then by TCP + specified, TLS SHOULD be tried first, followed by DTLS, then by TCP, + and finally by SCTP. See Section 5.2 for more information on peer + discovery. -Fajardo, et al. Expires July 24, 2011 [Page 22] - -Internet-Draft Diameter Base Protocol January 2011 - and finally by SCTP. See Section 5.2 for more information on peer - discovery. +Fajardo, et al. Standards Track [Page 22] + +RFC 6733 Diameter Base Protocol October 2012 + Diameter implementations SHOULD be able to interpret ICMP protocol port unreachable messages as explicit indications that the server is @@ -1253,18 +1253,18 @@ Internet-Draft Diameter Base Protocol January 2011 Diameter messages SHOULD be mapped into SCTP streams in a way that avoids head-of-the-line (HOL) blocking. Among different ways of performing the mapping that fulfill this requirement it is - RECOMMENDED that a Diameter node sends every Diameter message - (request or response) over the stream zero with the unordered flag - set. However, Diameter nodes MAY select and implement other design - alternatives for avoiding HOL blocking such as using multiple streams - with the unordered flag cleared (as originally instructed in - RFC3588). On the receiving side, a Diameter entity MUST be ready to - receive Diameter messages over any stream and it is free to return - responses over a different stream. This way, both sides manage the - available streams in the sending direction, independently of the - streams chosen by the other side to send a particular Diameter - message. These messages can be out-of-order and belong to different - Diameter sessions. + RECOMMENDED that a Diameter node send every Diameter message (request + or response) over stream zero with the unordered flag set. However, + Diameter nodes MAY select and implement other design alternatives for + avoiding HOL blocking such as using multiple streams with the + unordered flag cleared (as originally instructed in RFC 3588). On + the receiving side, a Diameter entity MUST be ready to receive + Diameter messages over any stream, and it is free to return responses + over a different stream. This way, both sides manage the available + streams in the sending direction, independently of the streams chosen + by the other side to send a particular Diameter message. These + messages can be out-of-order and belong to different Diameter + sessions. Out-of-order delivery has special concerns during a connection establishment and termination. When a connection is established, the @@ -1275,30 +1275,37 @@ Internet-Draft Diameter Base Protocol January 2011 the connection. In order to avoid this race condition, the receiver side SHOULD NOT use out-of-order delivery methods until the first message has been received from the initiator, proving that it has - moved to I-Open state. To trigger such message, the receiver side - could send a DWR immediatly after sending CEA. Upon reception of the - corresponding DWA, the receiver side should start using out-of-order - delivery methods to counter the HOL blocking. + moved to I-Open state. To trigger such a message, the receiver side + could send a DWR immediately after sending a CEA. Upon reception of + the corresponding DWA, the receiver side should start using out-of- + order delivery methods to counter the HOL blocking. Another race condition may occur when DPR and DPA messages are used. + Both DPR and DPA are small in size; thus, they may be delivered to + the peer faster than application messages when an out-of-order + delivery mechanism is used. Therefore, it is possible that a DPR/DPA -Fajardo, et al. Expires July 24, 2011 [Page 23] +Fajardo, et al. Standards Track [Page 23] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + exchange completes while application messages are still in transit, + resulting in a loss of these messages. An implementation could + mitigate this race condition, for example, using timers, and wait for + a short period of time for pending application level messages to + arrive before proceeding to disconnect the transport connection. + Eventually, lost messages are handled by the retransmission mechanism + described in Section 5.5.4. - Both DPR and DPA are small in size, thus they may be delivered faster - to the peer than application messages when out-of-order delivery - mechanism is used. Therefore, it is possible that a DPR/DPA exchange - completes while application messages are still in transit, resulting - to a loss of these messages. An implementation could mitigate this - race condition, for example, using timers and wait for a short period - of time for pending application level messages to arrive before - proceeding to disconnect the transport connection. Eventually, lost - messages are handled by the retransmission mechanism described in - Section 5.5.4. + A Diameter agent SHOULD use dedicated payload protocol identifiers + (PPIDs) for clear text and encrypted SCTP DATA chunks instead of only + using the unspecified payload protocol identifier (value 0). For + this purpose, two PPID values are allocated: the PPID value 46 is for + Diameter messages in clear text SCTP DATA chunks, and the PPID value + 47 is for Diameter messages in protected DTLS/SCTP DATA chunks. 2.2. Securing Diameter Messages @@ -1307,7 +1314,7 @@ Internet-Draft Diameter Base Protocol January 2011 the use of TLS/TCP and DTLS/SCTP. If desired, alternative security mechanisms that are independent of Diameter, such as IPsec [RFC4301], can be deployed to secure connections between peers. The Diameter - protocol MUST NOT be used without any security mechanism. + protocol MUST NOT be used without one of TLS, DTLS, or IPsec. 2.3. Diameter Application Compliance @@ -1318,37 +1325,40 @@ Internet-Draft Diameter Base Protocol January 2011 Implementations MAY add arbitrary optional AVPs with the M-bit cleared (including vendor-specific AVPs) to a command defined in an - application, but only if the command's ABNF syntax specification + application, but only if the command's CCF syntax specification allows for it. Please refer to Section 11.1.1 for details. 2.4. Application Identifiers - Each Diameter application MUST have an IANA assigned Application Id - (see Section 11.3). The base protocol does not require an - Application Id since its support is mandatory. During the - capabilities exchange, Diameter nodes inform their peers of locally - supported applications. Furthermore, all Diameter messages contain - an Application Id, which is used in the message forwarding process. + Each Diameter application MUST have an IANA-assigned Application ID. + The base protocol does not require an Application Id since its + support is mandatory. During the capabilities exchange, Diameter + nodes inform their peers of locally supported applications. + Furthermore, all Diameter messages contain an Application Id, which + is used in the message forwarding process. - The following Application Id values are defined: - Diameter Common Messages 0 - Diameter Base Accounting 3 - Relay 0xffffffff - Relay and redirect agents MUST advertise the Relay Application -Fajardo, et al. Expires July 24, 2011 [Page 24] + +Fajardo, et al. Standards Track [Page 24] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + The following Application Id values are defined: + + Diameter common message 0 + Diameter base accounting 3 + Relay 0xffffffff - Identifier, while all other Diameter nodes MUST advertise locally - supported applications. The receiver of a Capabilities Exchange - message advertising Relay service MUST assume that the sender - supports all current and future applications. + Relay and redirect agents MUST advertise the Relay Application ID, + while all other Diameter nodes MUST advertise locally supported + applications. The receiver of a Capabilities Exchange message + advertising relay service MUST assume that the sender supports all + current and future applications. Diameter relay and proxy agents are responsible for finding an upstream server that supports the application of a particular @@ -1358,16 +1368,15 @@ Internet-Draft Diameter Base Protocol January 2011 2.5. Connections vs. Sessions This section attempts to provide the reader with an understanding of - the difference between connection and session, which are terms used - extensively throughout this document. + the difference between "connection" and "session", which are terms + used extensively throughout this document. - A connection refers to a transport level connection between two peers + A connection refers to a transport-level connection between two peers that is used to send and receive Diameter messages. A session is a - logical concept at the application layer existing between the + logical concept at the application layer that exists between the Diameter client and the Diameter server; it is identified via the Session-Id AVP. - +--------+ +-------+ +--------+ | Client | | Relay | | Server | +--------+ +-------+ +--------+ @@ -1377,100 +1386,93 @@ Internet-Draft Diameter Base Protocol January 2011 <-----------------------------> User session x - Figure 1: Diameter connections and sessions + Figure 1: Diameter Connections and Sessions In the example provided in Figure 1, peer connection A is established - between the Client and the Relay. Peer connection B is established - between the Relay and the Server. User session X spans from the - Client via the Relay to the Server. Each "user" of a service causes + between the client and the relay. Peer connection B is established + between the relay and the server. User session X spans from the + client via the relay to the server. Each "user" of a service causes an auth request to be sent, with a unique session identifier. Once accepted by the server, both the client and the server are aware of the session. - It is important to note that there is no relationship between a - connection and a session, and that Diameter messages for multiple - sessions are all multiplexed through a single connection. Also note - that Diameter messages pertaining to the session, both application - specific and those that are defined in this document such as ASR/ASA, - RAR/RAA and STR/STA MUST carry the Application Id of the application. -Fajardo, et al. Expires July 24, 2011 [Page 25] +Fajardo, et al. Standards Track [Page 25] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - Diameter messages pertaining to peer connection establishment and - maintenance such as CER/CEA, DWR/DWA and DPR/DPA MUST carry an - Application Id of zero (0). + It is important to note that there is no relationship between a + connection and a session, and that Diameter messages for multiple + sessions are all multiplexed through a single connection. Also, note + that Diameter messages pertaining to the session, both application- + specific and those that are defined in this document such as ASR/ASA, + RAR/RAA, and STR/STA, MUST carry the Application Id of the + application. Diameter messages pertaining to peer connection + establishment and maintenance such as CER/CEA, DWR/DWA, and DPR/DPA + MUST carry an Application Id of zero (0). 2.6. Peer Table - The Diameter Peer Table is used in message forwarding, and referenced - by the Routing Table. A Peer Table entry contains the following - fields: + The Diameter peer table is used in message forwarding and is + referenced by the routing table. A peer table entry contains the + following fields: - Host identity + Host Identity - Following the conventions described for the DiameterIdentity - derived AVP data format in Section 4.3. This field contains the + Following the conventions described for the DiameterIdentity- + derived AVP data format in Section 4.3.1, this field contains the contents of the Origin-Host (Section 6.3) AVP found in the CER or CEA message. - StatusT - This is the state of the peer entry, and MUST match one of the + This is the state of the peer entry, and it MUST match one of the values listed in Section 5.6. - Static or Dynamic Specifies whether a peer entry was statically configured or dynamically discovered. - - Expiration time + Expiration Time Specifies the time at which dynamically discovered peer table - entries are to be either refreshed, or expired. - + entries are to be either refreshed or expired. If public key + certificates are used for Diameter security (e.g., with TLS), this + value MUST NOT be greater than the expiry times in the relevant + certificates. TLS/TCP and DTLS/SCTP Enabled Specifies whether TLS/TCP and DTLS/SCTP is to be used when communicating with the peer. - Additional security information, when needed (e.g., keys, - certificates) - - + certificates). - - -Fajardo, et al. Expires July 24, 2011 [Page 26] +Fajardo, et al. Standards Track [Page 26] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 2.7. Routing Table All Realm-Based routing lookups are performed against what is - commonly known as the Routing Table (see Section 12). A Routing - Table Entry contains the following fields: + commonly known as the routing table (see Section 12). Each routing + table entry contains the following fields: Realm Name - This is the field that is MUST be used as a primary key in the + This is the field that MUST be used as a primary key in the routing table lookups. Note that some implementations perform their lookups based on longest-match-from-the-right on the realm rather than requiring an exact match. - Application Identifier An application is identified by an Application Id. A route entry @@ -1478,21 +1480,19 @@ Internet-Draft Diameter Base Protocol January 2011 the message header. This field MUST be used as a secondary key field in routing table lookups. - Local Action The Local Action field is used to identify how a message should be treated. The following actions are supported: - - 1. LOCAL - Diameter messages that can be satisfied locally, and - do not need to be routed to another Diameter entity. + 1. LOCAL - Diameter messages that can be satisfied locally and do + not need to be routed to another Diameter entity. 2. RELAY - All Diameter messages that fall within this category - MUST be routed to a next hop Diameter entity that is indicated + MUST be routed to a next-hop Diameter entity that is indicated by the identifier described below. Routing is done without modifying any non-routing AVPs. See Section 6.1.9 for - relaying guidelines + relaying guidelines. 3. PROXY - All Diameter messages that fall within this category MUST be routed to a next Diameter entity that is indicated by @@ -1504,47 +1504,53 @@ Internet-Draft Diameter Base Protocol January 2011 4. REDIRECT - Diameter messages that fall within this category MUST have the identity of the home Diameter server(s) appended, and returned to the sender of the message. See - Section 6.1.8 for redirect guidelines. + Section 6.1.8 for redirection guidelines. + + + -Fajardo, et al. Expires July 24, 2011 [Page 27] +Fajardo, et al. Standards Track [Page 27] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Server Identifier - One or more servers to which the message is to be routed. These - servers MUST also be present in the Peer table. When the Local - Action is set to RELAY or PROXY, this field contains the identity - of the server(s) the message MUST be routed to. When the Local + The identity of one or more servers to which the message is to be + routed. This identity MUST also be present in the Host Identity + field of the peer table (Section 2.6). When the Local Action is + set to RELAY or PROXY, this field contains the identity of the + server(s) to which the message MUST be routed. When the Local Action field is set to REDIRECT, this field contains the identity - of one or more servers the message MUST be redirected to. + of one or more servers to which the message MUST be redirected. Static or Dynamic Specifies whether a route entry was statically configured or dynamically discovered. - Expiration time + Expiration Time Specifies the time at which a dynamically discovered route table - entry expires. + entry expires. If public key certificates are used for Diameter + security (e.g., with TLS), this value MUST NOT be greater than the + expiry time in the relevant certificates. It is important to note that Diameter agents MUST support at least - one of the LOCAL, RELAY, PROXY or REDIRECT modes of operation. + one of the LOCAL, RELAY, PROXY, or REDIRECT modes of operation. Agents do not need to support all modes of operation in order to - conform with the protocol specification, but MUST follow the protocol - compliance guidelines in Section 2. Relay agents and proxies MUST - NOT reorder AVPs. + conform with the protocol specification, but they MUST follow the + protocol compliance guidelines in Section 2. Relay agents and + proxies MUST NOT reorder AVPs. The routing table MAY include a default entry that MUST be used for any requests not matching any of the other entries. The routing table MAY consist of only such an entry. When a request is routed, the target server MUST have advertised the - Application Id (see Section 2.4) for the given message, or have + Application Id (see Section 2.4) for the given message or have advertised itself as a relay or proxy agent. Otherwise, an error is returned with the Result-Code AVP set to DIAMETER_UNABLE_TO_DELIVER. @@ -1552,23 +1558,24 @@ Internet-Draft Diameter Base Protocol January 2011 In addition to clients and servers, the Diameter protocol introduces relay, proxy, redirect, and translation agents, each of which is - defined in Section 1.3. These Diameter agents are useful for several + defined in Section 1.2. Diameter agents are useful for several reasons: o They can distribute administration of systems to a configurable grouping, including the maintenance of security associations. - o They can be used for concentration of requests from an number of - co-located or distributed NAS equipment sets to a set of like user - groups. -Fajardo, et al. Expires July 24, 2011 [Page 28] +Fajardo, et al. Standards Track [Page 28] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + o They can be used for concentration of requests from a number of + co-located or distributed NAS equipment sets to a set of like user + groups. + o They can do value-added processing to the requests or responses. o They can be used for load balancing. @@ -1578,7 +1585,7 @@ Internet-Draft Diameter Base Protocol January 2011 The Diameter protocol requires that agents maintain transaction state, which is used for failover purposes. Transaction state - implies that upon forwarding a request, its Hop-by-Hop identifier is + implies that upon forwarding a request, its Hop-by-Hop Identifier is saved; the field is replaced with a locally unique identifier, which is restored to its original value when the corresponding answer is received. The request's state is released upon receipt of the @@ -1593,7 +1600,7 @@ Internet-Draft Diameter Base Protocol January 2011 A stateful agent is one that maintains session state information by keeping track of all authorized active sessions. Each authorized session is bound to a particular service, and its state is considered - active either until the agent is notified otherwise, or the session + active until either the agent is notified otherwise or the session expires. Each authorized session has an expiration, which is communicated by Diameter servers via the Session-Timeout AVP. @@ -1604,40 +1611,43 @@ Internet-Draft Diameter Base Protocol January 2011 o Limiting resources authorized to a particular user - o Per user or transaction auditing + o Per-user or per-transaction auditing A Diameter agent MAY act in a stateful manner for some requests and be stateless for others. A Diameter implementation MAY act as one - type of agent for some requests, and as another type of agent for + type of agent for some requests and as another type of agent for others. -2.8.1. Relay Agents - Relay Agents are Diameter agents that accept requests and route - messages to other Diameter nodes based on information found in the - messages (e.g., Destination-Realm). This routing decision is - performed using a list of supported realms, and known peers. This is -Fajardo, et al. Expires July 24, 2011 [Page 29] + +Fajardo, et al. Standards Track [Page 29] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + +2.8.1. Relay Agents - known as the Routing Table, as is defined further in Section 2.7. + Relay agents are Diameter agents that accept requests and route + messages to other Diameter nodes based on information found in the + messages (e.g., the value of the Destination-Realm AVP Section 6.6). + This routing decision is performed using a list of supported realms + and known peers. This is known as the routing table, as is defined + further in Section 2.7. Relays may, for example, be used to aggregate requests from multiple Network Access Servers (NASes) within a common geographical area - (POP). The use of Relays is advantageous since it eliminates the - need for NASes to be configured with the necessary security - information they would otherwise require to communicate with Diameter - servers in other realms. Likewise, this reduces the configuration - load on Diameter servers that would otherwise be necessary when NASes - are added, changed or deleted. + (Point of Presence, POP). The use of relays is advantageous since it + eliminates the need for NASes to be configured with the necessary + security information they would otherwise require to communicate with + Diameter servers in other realms. Likewise, this reduces the + configuration load on Diameter servers that would otherwise be + necessary when NASes are added, changed, or deleted. Relays modify Diameter messages by inserting and removing routing - information, but do not modify any other portion of a message. + information, but they do not modify any other portion of a message. Relays SHOULD NOT maintain session state but MUST maintain transaction state. @@ -1650,44 +1660,46 @@ Internet-Draft Diameter Base Protocol January 2011 Figure 2: Relaying of Diameter messages - The example provided in Figure 2 depicts a request issued from NAS, + The example provided in Figure 2 depicts a request issued from a NAS, which is an access device, for the user [email protected]. Prior to - issuing the request, NAS performs a Diameter route lookup, using + issuing the request, the NAS performs a Diameter route lookup, using "example.com" as the key, and determines that the message is to be - relayed to DRL, which is a Diameter Relay. DRL performs the same - route lookup as NAS, and relays the message to HMS, which is - example.com's Home Diameter Server. HMS identifies that the request - can be locally supported (via the realm), processes the + relayed to a DRL, which is a Diameter relay. The DRL performs the + same route lookup as the NAS, and relays the message to the HMS, + which is example.com's home server. The HMS identifies that the + request can be locally supported (via the realm), processes the authentication and/or authorization request, and replies with an - answer, which is routed back to NAS using saved transaction state. - - Since Relays do not perform any application level processing, they - provide relaying services for all Diameter applications, and - therefore MUST advertise the Relay Application Id. + answer, which is routed back to the NAS using saved transaction + state. -2.8.2. Proxy Agents + Since relays do not perform any application-level processing, they + provide relaying services for all Diameter applications; therefore, + they MUST advertise the Relay Application Id. - Similarly to relays, proxy agents route Diameter messages using the - Diameter Routing Table. However, they differ since they modify - messages to implement policy enforcement. This requires that proxies - maintain the state of their downstream peers (e.g., access devices) - to enforce resource usage, provide admission control, and - provisioning. -Fajardo, et al. Expires July 24, 2011 [Page 30] +Fajardo, et al. Standards Track [Page 30] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + +2.8.2. Proxy Agents + + Similar to relays, proxy agents route Diameter messages using the + Diameter routing table. However, they differ since they modify + messages to implement policy enforcement. This requires that proxies + maintain the state of their downstream peers (e.g., access devices) + to enforce resource usage, provide admission control, and provide + provisioning. Proxies may, for example, be used in call control centers or access - ISPs that provide outsourced connections, they can monitor the number - and types of ports in use, and make allocation and admission - decisions according to their configuration. + ISPs that provide outsourced connections; they can monitor the number + and type of ports in use and make allocation and admission decisions + according to their configuration. Since enforcing policies requires an understanding of the service - being provided, Proxies MUST only advertise the Diameter applications + being provided, proxies MUST only advertise the Diameter applications they support. 2.8.3. Redirect Agents @@ -1709,18 +1721,12 @@ Internet-Draft Diameter Base Protocol January 2011 The example provided in Figure 3 depicts a request issued from the access device, NAS, for the user [email protected]. The message is forwarded by the NAS to its relay, DRL, which does not have a routing - entry in its Diameter Routing Table for example.com. DRL has a + entry in its Diameter routing table for example.com. The DRL has a default route configured to DRD, which is a redirect agent that - returns a redirect notification to DRL, as well as HMS' contact - information. Upon receipt of the redirect notification, DRL - establishes a transport connection with HMS, if one doesn't already - exist, and forwards the request to it. - - - - - - + returns a redirect notification to DRL, as well as the HMS' contact + information. Upon receipt of the redirect notification, the DRL + establishes a transport connection with the HMS, if one doesn't + already exist, and forwards the request to it. @@ -1729,12 +1735,9 @@ Internet-Draft Diameter Base Protocol January 2011 - - - -Fajardo, et al. Expires July 24, 2011 [Page 31] +Fajardo, et al. Standards Track [Page 31] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 +------+ @@ -1755,10 +1758,10 @@ Internet-Draft Diameter Base Protocol January 2011 Figure 3: Redirecting a Diameter Message - Since redirect agents do not perform any application level + Since redirect agents do not perform any application-level processing, they provide relaying services for all Diameter - applications, and therefore MUST advertise the Relay Application - Identifier. + applications; therefore, they MUST advertise the Relay Application + ID. 2.8.4. Translation Agents @@ -1773,7 +1776,7 @@ Internet-Draft Diameter Base Protocol January 2011 MUST maintain transaction state. Translation of messages can only occur if the agent recognizes the - application of a particular request, and therefore translation agents + application of a particular request; therefore, translation agents MUST only advertise their locally supported applications. +------+ ---------> +------+ ---------> +------+ @@ -1788,23 +1791,24 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 32] +Fajardo, et al. Standards Track [Page 32] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 2.9. Diameter Path Authorization - As noted in Section 2.2, Diameter provides transmission level + As noted in Section 2.2, Diameter provides transmission-level security for each connection using TLS/TCP and DTLS/SCTP. Therefore, - each connection can be authenticated, replay and integrity protected. + each connection can be authenticated and can be replay and integrity + protected. - In addition to authenticating each connection, each connection as - well as the entire session MUST also be authorized. Before - initiating a connection, a Diameter Peer MUST check that its peers - are authorized to act in their roles. For example, a Diameter peer - may be authentic, but that does not mean that it is authorized to act - as a Diameter Server advertising a set of Diameter applications. + In addition to authenticating each connection, the entire session + MUST also be authorized. Before initiating a connection, a Diameter + peer MUST check that its peers are authorized to act in their roles. + For example, a Diameter peer may be authentic, but that does not mean + that it is authorized to act as a Diameter server advertising a set + of Diameter applications. Prior to bringing up a connection, authorization checks are performed at each connection along the path. Diameter capabilities negotiation @@ -1815,7 +1819,7 @@ Internet-Draft Diameter Base Protocol January 2011 As noted in Section 6.1.9, a relay or proxy agent MUST append a Route-Record AVP to all requests forwarded. The AVP contains the - identity of the peer the request was received from. + identity of the peer from which the request was received. The home Diameter server, prior to authorizing a session, MUST check the Route-Record AVPs to make sure that the route traversed by the @@ -1823,7 +1827,7 @@ Internet-Draft Diameter Base Protocol January 2011 realm may not wish to honor requests that have been routed through an untrusted realm. By authorizing a request, the home Diameter server is implicitly indicating its willingness to engage in the business - transaction as specified by the contractual relationship between the + transaction as specified by any contractual relationship between the server and the previous hop. A DIAMETER_AUTHORIZATION_REJECTED error message (see Section 7.1.5) is sent if the route traversed by the request is unacceptable. @@ -1839,72 +1843,24 @@ Internet-Draft Diameter Base Protocol January 2011 willingness to take on financial risk relative to the session. A local realm may wish to limit this exposure, for example, by establishing credit limits for intermediate realms and refusing to - accept responses which would violate those limits. By issuing an - accounting request corresponding to the authorization response, the + accept responses that would violate those limits. By issuing an -Fajardo, et al. Expires July 24, 2011 [Page 33] +Fajardo, et al. Standards Track [Page 33] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + accounting request corresponding to the authorization response, the local realm implicitly indicates its agreement to provide the service indicated in the authorization response. If the service cannot be provided by the local realm, then a DIAMETER_UNABLE_TO_COMPLY error message MUST be sent within the accounting request; a Diameter client receiving an authorization response for a service that it cannot - perform MUST NOT substitute an alternate service, and then send + perform MUST NOT substitute an alternate service and then send accounting requests for the alternate service instead. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 34] - -Internet-Draft Diameter Base Protocol January 2011 - - 3. Diameter Header A summary of the Diameter header format is shown below. The fields @@ -1915,7 +1871,7 @@ Internet-Draft Diameter Base Protocol January 2011 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | Message Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | command flags | Command-Code | + | Command Flags | Command Code | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Application-ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ @@ -1935,13 +1891,23 @@ Internet-Draft Diameter Base Protocol January 2011 The Message Length field is three octets and indicates the length of the Diameter message including the header fields and the padded - AVPs. Thus the message length field is always a multiple of 4. + AVPs. Thus, the Message Length field is always a multiple of 4. Command Flags The Command Flags field is eight bits. The following bits are assigned: + + + + + +Fajardo, et al. Standards Track [Page 34] + +RFC 6733 Diameter Base Protocol October 2012 + + 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+-+ |R P E T r r r r| @@ -1952,127 +1918,113 @@ Internet-Draft Diameter Base Protocol January 2011 If set, the message is a request. If cleared, the message is an answer. - - - - -Fajardo, et al. Expires July 24, 2011 [Page 35] - -Internet-Draft Diameter Base Protocol January 2011 - - P(roxiable) - If set, the message MAY be proxied, relayed or redirected. If + If set, the message MAY be proxied, relayed, or redirected. If cleared, the message MUST be locally processed. - E(rror) If set, the message contains a protocol error, and the message - will not conform to the ABNF described for this command. + will not conform to the CCF described for this command. Messages with the 'E' bit set are commonly referred to as error - messages. This bit MUST NOT be set in request messages. See - Section 7.2. + messages. This bit MUST NOT be set in request messages (see + Section 7.2). - - T(Potentially re-transmitted message) + T(Potentially retransmitted message) This flag is set after a link failover procedure, to aid the removal of duplicate requests. It is set when resending requests not yet acknowledged, as an indication of a possible duplicate due to a link failure. This bit MUST be cleared when - sending a request for the first time, otherwise the sender MUST - set this flag. Diameter agents only need to be concerned about - the number of requests they send based on a single received - request; retransmissions by other entities need not be tracked. - Diameter agents that receive a request with the T flag set, - MUST keep the T flag set in the forwarded request. This flag - MUST NOT be set if an error answer message (e.g., a protocol - error) has been received for the earlier message. It can be - set only in cases where no answer has been received from the - server for a request and the request is sent again. This flag - MUST NOT be set in answer messages. - + sending a request for the first time; otherwise, the sender + MUST set this flag. Diameter agents only need to be concerned + about the number of requests they send based on a single + received request; retransmissions by other entities need not be + tracked. Diameter agents that receive a request with the T + flag set, MUST keep the T flag set in the forwarded request. + This flag MUST NOT be set if an error answer message (e.g., a + protocol error) has been received for the earlier message. It + can be set only in cases where no answer has been received from + the server for a request, and the request has been sent again. + This flag MUST NOT be set in answer messages. r(eserved) - These flag bits are reserved for future use, and MUST be set to - zero, and ignored by the receiver. + These flag bits are reserved for future use; they MUST be set + to zero and ignored by the receiver. - Command-Code - The Command-Code field is three octets, and is used in order to - communicate the command associated with the message. The 24-bit - address space is managed by IANA (see Section 11.2.1). - Command-Code values 16,777,214 and 16,777,215 (hexadecimal values - FFFFFE -FFFFFF) are reserved for experimental use (See Section - 11.3). -Fajardo, et al. Expires July 24, 2011 [Page 36] +Fajardo, et al. Standards Track [Page 35] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + + Command Code + The Command Code field is three octets and is used in order to + communicate the command associated with the message. The 24-bit + address space is managed by IANA (see Section 3.1). Command Code + values 16,777,214 and 16,777,215 (hexadecimal values FFFFFE- + FFFFFF) are reserved for experimental use (see Section 11.2). - Application-ID + Application-ID - Application-ID is four octets and is used to identify to which - application the message is applicable for. The application can be - an authentication application, an accounting application or a - vendor specific application. See Section 11.3 for the possible - values that the application-id may use. + Application-ID is four octets and is used to identify for which + application the message is applicable. The application can be an + authentication application, an accounting application, or a + vendor-specific application. - The value of the application-id field in the header MUST be the - same as any relevant application-id AVPs contained in the message. + The value of the Application-ID field in the header MUST be the + same as any relevant Application-Id AVPs contained in the message. Hop-by-Hop Identifier The Hop-by-Hop Identifier is an unsigned 32-bit integer field (in - network byte order) and aids in matching requests and replies. - The sender MUST ensure that the Hop-by-Hop identifier in a request - is unique on a given connection at any given time, and MAY attempt - to ensure that the number is unique across reboots. The sender of - an Answer message MUST ensure that the Hop-by-Hop Identifier field - contains the same value that was found in the corresponding - request. The Hop-by-Hop identifier is normally a monotonically - increasing number, whose start value was randomly generated. An - answer message that is received with an unknown Hop-by-Hop - Identifier MUST be discarded. - + network byte order) that aids in matching requests and replies. + The sender MUST ensure that the Hop-by-Hop Identifier in a request + is unique on a given connection at any given time, and it MAY + attempt to ensure that the number is unique across reboots. The + sender of an answer message MUST ensure that the Hop-by-Hop + Identifier field contains the same value that was found in the + corresponding request. The Hop-by-Hop Identifier is normally a + monotonically increasing number, whose start value was randomly + generated. An answer message that is received with an unknown + Hop-by-Hop Identifier MUST be discarded. End-to-End Identifier The End-to-End Identifier is an unsigned 32-bit integer field (in - network byte order) and is used to detect duplicate messages. - Upon reboot implementations MAY set the high order 12 bits to + network byte order) that is used to detect duplicate messages. + Upon reboot, implementations MAY set the high order 12 bits to contain the low order 12 bits of current time, and the low order 20 bits to a random value. Senders of request messages MUST insert a unique identifier on each message. The identifier MUST remain locally unique for a period of at least 4 minutes, even - across reboots. The originator of an Answer message MUST ensure + across reboots. The originator of an answer message MUST ensure that the End-to-End Identifier field contains the same value that was found in the corresponding request. The End-to-End Identifier MUST NOT be modified by Diameter agents of any kind. The - combination of the Origin-Host (see Section 6.3) and this field is + combination of the Origin-Host AVP (Section 6.3) and this field is used to detect duplicates. Duplicate requests SHOULD cause the - same answer to be transmitted (modulo the hop-by-hop Identifier - field and any routing AVPs that may be present), and MUST NOT - affect any state that was set when the original request was - processed. Duplicate answer messages that are to be locally - consumed (see Section 6.2) SHOULD be silently discarded. - - + same answer to be transmitted (modulo the Hop-by-Hop Identifier -Fajardo, et al. Expires July 24, 2011 [Page 37] +Fajardo, et al. Standards Track [Page 36] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + field and any routing AVPs that may be present), and they MUST NOT + affect any state that was set when the original request was + processed. Duplicate answer messages that are to be locally + consumed (see Section 6.2) SHOULD be silently discarded. + AVPs AVPs are a method of encapsulating information relevant to the @@ -2080,17 +2032,17 @@ Internet-Draft Diameter Base Protocol January 2011 3.1. Command Codes - Each command Request/Answer pair is assigned a command code, and the + Each command Request/Answer pair is assigned a Command Code, and the sub-type (i.e., request or answer) is identified via the 'R' bit in the Command Flags field of the Diameter header. - - Every Diameter message MUST contain a command code in its header's - Command-Code field, which is used to determine the action that is to + Every Diameter message MUST contain a Command Code in its header's + Command Code field, which is used to determine the action that is to be taken for a particular message. The following Command Codes are defined in the Diameter base protocol: - Command-Name Abbrev. Code Reference + Section + Command Name Abbrev. Code Reference -------------------------------------------------------- Abort-Session-Request ASR 274 8.5.1 Abort-Session-Answer ASA 274 8.5.2 @@ -2111,140 +2063,142 @@ Internet-Draft Diameter Base Protocol January 2011 Session-Termination- STA 275 8.4.2 Answer -3.2. Command Code ABNF specification - Every Command Code defined MUST include a corresponding ABNF - specification, which is used to define the AVPs that MUST or MAY be - present when sending the message. The following format is used in - the definition: - command-def = <command-name> "::=" diameter-message - command-name = diameter-name -Fajardo, et al. Expires July 24, 2011 [Page 38] - -Internet-Draft Diameter Base Protocol January 2011 - diameter-name = ALPHA *(ALPHA / DIGIT / "-") +Fajardo, et al. Standards Track [Page 37] + +RFC 6733 Diameter Base Protocol October 2012 - diameter-message = header [ *fixed] [ *required] [ *optional] - header = "<" "Diameter Header:" command-id - [r-bit] [p-bit] [e-bit] [application-id] ">" +3.2. Command Code Format Specification - application-id = 1*DIGIT + Every Command Code defined MUST include a corresponding Command Code + Format (CCF) specification, which is used to define the AVPs that + MUST or MAY be present when sending the message. The following ABNF + specifies the CCF used in the definition: - command-id = 1*DIGIT - ; The Command Code assigned to the command + command-def = "<" command-name ">" "::=" diameter-message - r-bit = ", REQ" - ; If present, the 'R' bit in the Command - ; Flags is set, indicating that the message - ; is a request, as opposed to an answer. + command-name = diameter-name - p-bit = ", PXY" - ; If present, the 'P' bit in the Command - ; Flags is set, indicating that the message - ; is proxiable. + diameter-name = ALPHA *(ALPHA / DIGIT / "-") - e-bit = ", ERR" - ; If present, the 'E' bit in the Command - ; Flags is set, indicating that the answer - ; message contains a Result-Code AVP in - ; the "protocol error" class. + diameter-message = header *fixed *required *optional - fixed = [qual] "<" avp-spec ">" - ; Defines the fixed position of an AVP + header = "<Diameter-Header:" command-id + [r-bit] [p-bit] [e-bit] [application-id]">" - required = [qual] "{" avp-spec "}" - ; The AVP MUST be present and can appear - ; anywhere in the message. + application-id = 1*DIGIT + command-id = 1*DIGIT + ; The Command Code assigned to the command. - optional = [qual] "[" avp-name "]" - ; The avp-name in the 'optional' rule cannot - ; evaluate to any AVP Name which is included - ; in a fixed or required rule. The AVP can - ; appear anywhere in the message. - ; - ; NOTE: "[" and "]" have a slightly different - ; meaning than in ABNF (RFC 5234]). These braces - ; cannot be used to express optional fixed rules - ; (such as an optional ICV at the end). To do this, - ; the convention is '0*1fixed'. + r-bit = ", REQ" + ; If present, the 'R' bit in the Command + ; Flags is set, indicating that the message + ; is a request as opposed to an answer. + p-bit = ", PXY" + ; If present, the 'P' bit in the Command + ; Flags is set, indicating that the message + ; is proxiable. + e-bit = ", ERR" + ; If present, the 'E' bit in the Command + ; Flags is set, indicating that the answer + ; message contains a Result-Code AVP in + ; the "protocol error" class. + fixed = [qual] "<" avp-spec ">" + ; Defines the fixed position of an AVP. -Fajardo, et al. Expires July 24, 2011 [Page 39] - -Internet-Draft Diameter Base Protocol January 2011 + required = [qual] "{" avp-spec "}" + ; The AVP MUST be present and can appear + ; anywhere in the message. - qual = [min] "*" [max] - ; See ABNF conventions, RFC 5234 Section 4. - ; The absence of any qualifiers depends on - ; whether it precedes a fixed, required, or - ; optional rule. If a fixed or required rule has - ; no qualifier, then exactly one such AVP MUST - ; be present. If an optional rule has no - ; qualifier, then 0 or 1 such AVP may be - ; present. If an optional rule has a qualifier, - ; then the value of min MUST be 0 if present. - min = 1*DIGIT - ; The minimum number of times the element may - ; be present. If absent, the default value is zero - ; for fixed and optional rules and one for required - ; rules. The value MUST be at least one for for - ; required rules. - max = 1*DIGIT - ; The maximum number of times the element may - ; be present. If absent, the default value is - ; infinity. A value of zero implies the AVP MUST - ; NOT be present. - avp-spec = diameter-name - ; The avp-spec has to be an AVP Name, defined - ; in the base or extended Diameter - ; specifications. - - avp-name = avp-spec / "AVP" - ; The string "AVP" stands for *any* arbitrary AVP - ; Name, not otherwise listed in that command code - ; definition. Addition this AVP is recommended for - ; all command ABNFs to allow for extensibility. +Fajardo, et al. Standards Track [Page 38] + +RFC 6733 Diameter Base Protocol October 2012 - The following is a definition of a fictitious command code: + optional = [qual] "[" avp-name "]" + ; The avp-name in the 'optional' rule cannot + ; evaluate to any AVP Name that is included + ; in a fixed or required rule. The AVP can + ; appear anywhere in the message. + ; + ; NOTE: "[" and "]" have a slightly different + ; meaning than in ABNF. These braces + ; cannot be used to express optional fixed rules + ; (such as an optional ICV at the end). To do + ; this, the convention is '0*1fixed'. - Example-Request ::= < Diameter Header: 9999999, REQ, PXY > - { User-Name } - * { Origin-Host } - * [ AVP ] + qual = [min] "*" [max] + ; See ABNF conventions, RFC 5234, Section 4. + ; The absence of any qualifier depends on + ; whether it precedes a fixed, required, or + ; optional rule. If a fixed or required rule has + ; no qualifier, then exactly one such AVP MUST + ; be present. If an optional rule has no + ; qualifier, then 0 or 1 such AVP may be + ; present. If an optional rule has a qualifier, + ; then the value of min MUST be 0 if present. + min = 1*DIGIT + ; The minimum number of times the element may + ; be present. If absent, the default value is 0 + ; for fixed and optional rules and 1 for + ; required rules. The value MUST be at least 1 + ; for required rules. + max = 1*DIGIT + ; The maximum number of times the element may + ; be present. If absent, the default value is + ; infinity. A value of 0 implies the AVP MUST + ; NOT be present. + avp-spec = diameter-name + ; The avp-spec has to be an AVP Name, defined + ; in the base or extended Diameter + ; specifications. + avp-name = avp-spec / "AVP" + ; The string "AVP" stands for *any* arbitrary AVP + ; Name, not otherwise listed in that Command Code + ; definition. The inclusion of this string + ; is recommended for all CCFs to allow for + ; extensibility. -Fajardo, et al. Expires July 24, 2011 [Page 40] +Fajardo, et al. Standards Track [Page 39] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + + The following is a definition of a fictitious Command Code: + Example-Request ::= < Diameter Header: 9999999, REQ, PXY > + { User-Name } + 1* { Origin-Host } + * [ AVP ] 3.3. Diameter Command Naming Conventions Diameter command names typically includes one or more English words - followed by the verb Request or Answer. Each English word is + followed by the verb "Request" or "Answer". Each English word is delimited by a hyphen. A three-letter acronym for both the request and answer is also normally provided. @@ -2253,10 +2207,10 @@ Internet-Draft Diameter Base Protocol January 2011 the acronyms are STR and STA, respectively. Both the request and the answer for a given command share the same - command code. The request is identified by the R(equest) bit in the + Command Code. The request is identified by the R(equest) bit in the Diameter header set to one (1), to ask that a particular action be performed, such as authorizing a user or terminating a session. Once - the receiver has completed the request it issues the corresponding + the receiver has completed the request, it issues the corresponding answer, which includes a result code that communicates one of the following: @@ -2274,38 +2228,25 @@ Internet-Draft Diameter Base Protocol January 2011 Additional information, encoded within AVPs, may also be included in answer messages. +4. Diameter AVPs + Diameter AVPs carry specific authentication, accounting, + authorization, and routing information as well as configuration + details for the request and reply. - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 41] +Fajardo, et al. Standards Track [Page 40] -Internet-Draft Diameter Base Protocol January 2011 - - -4. Diameter AVPs +RFC 6733 Diameter Base Protocol October 2012 - Diameter AVPs carry specific authentication, accounting, - authorization and routing information as well as configuration - details for the request and reply. Each AVP of type OctetString MUST be padded to align on a 32-bit boundary, while other AVP types align naturally. A number of zero- - valued bytes are added to the end of the AVP Data field till a word + valued bytes are added to the end of the AVP Data field until a word boundary is reached. The length of the padding is not reflected in the AVP Length field. @@ -2330,92 +2271,96 @@ Internet-Draft Diameter Base Protocol January 2011 The AVP Code, combined with the Vendor-Id field, identifies the attribute uniquely. AVP numbers 1 through 255 are reserved for - re-use of RADIUS attributes, without setting the Vendor-Id field. + reuse of RADIUS attributes, without setting the Vendor-Id field. AVP numbers 256 and above are used for Diameter, which are - allocated by IANA (see Section 11.1). - + allocated by IANA (see Section 11.1.1). AVP Flags The AVP Flags field informs the receiver how each attribute must - be handled. The 'r' (reserved) bits are unused and SHOULD be set - to 0. Note that subsequent Diameter applications MAY define - additional bits within the AVP Header, and an unrecognized bit - SHOULD be considered an error. The 'P' bit has been reserved for - future usage of end-to-end security. At the time of writing there - are no end-to-end security mechanisms specified therefore the 'P' - bit SHOULD be set to 0. + be handled. New Diameter applications SHOULD NOT define + additional AVP Flag bits. However, note that new Diameter + applications MAY define additional bits within the AVP header, and + an unrecognized bit SHOULD be considered an error. The sender of + the AVP MUST set 'R' (reserved) bits to 0 and the receiver SHOULD + ignore all 'R' (reserved) bits. The 'P' bit has been reserved for + future usage of end-to-end security. At the time of writing, + there are no end-to-end security mechanisms specified; therefore, + the 'P' bit SHOULD be set to 0. + + The 'M' bit, known as the Mandatory bit, indicates whether the + receiver of the AVP MUST parse and understand the semantics of the + AVP including its content. The receiving entity MUST return an + appropriate error message if it receives an AVP that has the M-bit -Fajardo, et al. Expires July 24, 2011 [Page 42] +Fajardo, et al. Standards Track [Page 41] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - The 'M' Bit, known as the Mandatory bit, indicates whether the - receiver of the AVP MUST parse and understand the semantic of the - AVP including its content. The receiving entity MUST return an - appropriate error message if it receives an AVP that has the M-bit set but does not understand it. An exception applies when the AVP is embedded within a Grouped AVP. See Section 4.4 for details. - Diameter Relay and redirect agents MUST NOT reject messages with + Diameter relay and redirect agents MUST NOT reject messages with unrecognized AVPs. The 'M' bit MUST be set according to the rules defined in the - application specification which introduces or re-uses this AVP. - Within a given application, the M-bit setting for an AVP is either - defined for all command types or for each command type. + application specification that introduces or reuses this AVP. + Within a given application, the M-bit setting for an AVP is + defined either for all command types or for each command type. - AVPs with the 'M' bit cleared are informational only and a - receiver that receives a message with such an AVP that is not - supported, or whose value is not supported, MAY simply ignore the - AVP. + AVPs with the 'M' bit cleared are informational only; a receiver + that receives a message with such an AVP that is not supported, or + whose value is not supported, MAY simply ignore the AVP. The 'V' bit, known as the Vendor-Specific bit, indicates whether the optional Vendor-ID field is present in the AVP header. When - set the AVP Code belongs to the specific vendor code address + set, the AVP Code belongs to the specific vendor code address space. AVP Length The AVP Length field is three octets, and indicates the number of - octets in this AVP including the AVP Code, AVP Length, AVP Flags, - Vendor-ID field (if present) and the AVP data. If a message is - received with an invalid attribute length, the message MUST be - rejected. + octets in this AVP including the AVP Code field, AVP Length field, + AVP Flags field, Vendor-ID field (if present), and the AVP Data + field. If a message is received with an invalid attribute length, + the message MUST be rejected. 4.1.1. Optional Header Elements - The AVP Header contains one optional field. This field is only + The AVP header contains one optional field. This field is only present if the respective bit-flag is enabled. - Vendor-ID The Vendor-ID field is present if the 'V' bit is set in the AVP Flags field. The optional four-octet Vendor-ID field contains the - IANA assigned "SMI Network Management Private Enterprise Codes" - [RFC3232] value, encoded in network byte order. Any vendor or - standardization organization that are also treated like vendors in - the IANA managed "SMI Network Management Private Enterprise Codes" - space wishing to implement a vendor-specific Diameter AVP MUST use - their own Vendor-ID along with their privately managed AVP address + IANA-assigned "SMI Network Management Private Enterprise Codes" + [ENTERPRISE] value, encoded in network byte order. Any vendors or + standardization organizations that are also treated like vendors + in the IANA-managed "SMI Network Management Private Enterprise + Codes" space wishing to implement a vendor-specific Diameter AVP + MUST use their own Vendor-ID along with their privately managed + AVP address space, guaranteeing that they will not collide with + any other vendor's vendor-specific AVP(s) or with future IETF + AVPs. -Fajardo, et al. Expires July 24, 2011 [Page 43] - -Internet-Draft Diameter Base Protocol January 2011 - space, guaranteeing that they will not collide with any other - vendor's vendor-specific AVP(s), nor with future IETF AVPs. - A vendor ID value of zero (0) corresponds to the IETF adopted AVP - values, as managed by the IANA. Since the absence of the vendor - ID field implies that the AVP in question is not vendor specific, - implementations MUST NOT use the zero (0) vendor ID. +Fajardo, et al. Standards Track [Page 42] + +RFC 6733 Diameter Base Protocol October 2012 + + + A Vendor-ID value of zero (0) corresponds to the IETF-adopted AVP + values, as managed by IANA. Since the absence of the Vendor-ID + field implies that the AVP in question is not vendor specific, + implementations MUST NOT use the value of zero (0) for the + Vendor-ID field. 4.2. Basic AVP Data Formats @@ -2426,49 +2371,45 @@ Internet-Draft Diameter Base Protocol January 2011 type derived from the base data types. In the event that a new Basic AVP Data Format is needed, a new version of this RFC MUST be created. - OctetString The data contains arbitrary data of variable length. Unless otherwise noted, the AVP Length field MUST be set to at least 8 - (12 if the 'V' bit is enabled). AVP Values of this type that are - not a multiple of four-octets in length is followed by the - necessary padding so that the next AVP (if any) will start on a - 32-bit boundary. - + (12 if the 'V' bit is enabled). AVP values of this type that are + not a multiple of 4 octets in length are followed by the necessary + padding so that the next AVP (if any) will start on a 32-bit + boundary. Integer32 - 32 bit signed value, in network byte order. The AVP Length field + 32-bit signed value, in network byte order. The AVP Length field MUST be set to 12 (16 if the 'V' bit is enabled). - Integer64 - 64 bit signed value, in network byte order. The AVP Length field + 64-bit signed value, in network byte order. The AVP Length field MUST be set to 16 (20 if the 'V' bit is enabled). - Unsigned32 - 32 bit unsigned value, in network byte order. The AVP Length + 32-bit unsigned value, in network byte order. The AVP Length field MUST be set to 12 (16 if the 'V' bit is enabled). + Unsigned64 + 64-bit unsigned value, in network byte order. The AVP Length + field MUST be set to 16 (20 if the 'V' bit is enabled). -Fajardo, et al. Expires July 24, 2011 [Page 44] - -Internet-Draft Diameter Base Protocol January 2011 - Unsigned64 - 64 bit unsigned value, in network byte order. The AVP Length - field MUST be set to 16 (20 if the 'V' bit is enabled). +Fajardo, et al. Standards Track [Page 43] + +RFC 6733 Diameter Base Protocol October 2012 Float32 @@ -2478,7 +2419,6 @@ Internet-Draft Diameter Base Protocol January 2011 network byte order. The AVP Length field MUST be set to 12 (16 if the 'V' bit is enabled). - Float64 This represents floating point values of double precision as @@ -2486,73 +2426,69 @@ Internet-Draft Diameter Base Protocol January 2011 network byte order. The AVP Length field MUST be set to 16 (20 if the 'V' bit is enabled). - Grouped - The Data field is specified as a sequence of AVPs. Each of these - AVPs follows - in the order in which they are specified - - including their headers and padding. The AVP Length field is set - to 8 (12 if the 'V' bit is enabled) plus the total length of all - included AVPs, including their headers and padding. Thus the AVP - length field of an AVP of type Grouped is always a multiple of 4. - + The Data field is specified as a sequence of AVPs. These AVPs are + concatenated -- including their headers and padding -- in the + order in which they are specified and the result encapsulated in + the Data field. The AVP Length field is set to 8 (12 if the 'V' + bit is enabled) plus the total length of all included AVPs, + including their headers and padding. Thus, the AVP Length field + of an AVP of type Grouped is always a multiple of 4. 4.3. Derived AVP Data Formats In addition to using the Basic AVP Data Formats, applications may define data formats derived from the Basic AVP Data Formats. An application that defines new Derived AVP Data Formats MUST include - them in a section entitled "Derived AVP Data Formats", using the same + them in a section titled "Derived AVP Data Formats", using the same format as the definitions below. Each new definition MUST be either defined or listed with a reference to the RFC that defines the format. -4.3.1. Common Derived AVPs +4.3.1. Common Derived AVP Data Formats The following are commonly used Derived AVP Data Formats. + Address + The Address format is derived from the OctetString Basic AVP + Format. It is a discriminated union representing, for example, a + 32-bit (IPv4) [RFC0791] or 128-bit (IPv6) [RFC4291] address, most + significant octet first. The first two octets of the Address AVP + represent the AddressType, which contains an Address Family, + defined in [IANAADFAM]. The AddressType is used to discriminate + the content and format of the remaining octets. -Fajardo, et al. Expires July 24, 2011 [Page 45] +Fajardo, et al. Standards Track [Page 44] -Internet-Draft Diameter Base Protocol January 2011 - - - Address - - The Address format is derived from the OctetString AVP Base - Format. It is a discriminated union, representing, for example a - 32-bit (IPv4) [RFC791] or 128-bit (IPv6) [RFC4291] address, most - significant octet first. The first two octets of the Address AVP - represents the AddressType, which contains an Address Family - defined in [IANAADFAM]. The AddressType is used to discriminate - the content and format of the remaining octets. +RFC 6733 Diameter Base Protocol October 2012 Time - The Time format is derived from the OctetString AVP Base Format. + The Time format is derived from the OctetString Basic AVP Format. The string MUST contain four octets, in the same format as the first four bytes are in the NTP timestamp format. The NTP - Timestamp format is defined in Chapter 3 of [RFC5905]. + timestamp format is defined in Section 3 of [RFC5905]. This represents the number of seconds since 0h on 1 January 1900 with respect to the Coordinated Universal Time (UTC). - On 6h 28m 16s UTC, 7 February 2036 the time value will overflow. - SNTP [RFC5905] describes a procedure to extend the time to 2104. - This procedure MUST be supported by all Diameter nodes. - + On 6h 28m 16s UTC, 7 February 2036, the time value will overflow. + Simple Network Time Protocol (SNTP) [RFC5905] describes a + procedure to extend the time to 2104. This procedure MUST be + supported by all Diameter nodes. UTF8String - The UTF8String format is derived from the OctetString AVP Base - Format. This is a human readable string represented using the + The UTF8String format is derived from the OctetString Basic AVP + Format. This is a human-readable string represented using the ISO/IEC IS 10646-1 character set, encoded as an OctetString using - the UTF-8 [RFC3629] transformation format described in RFC 3629. + the UTF-8 transformation format [RFC3629]. Since additional code points are added by amendments to the 10646 standard from time to time, implementations MUST be prepared to @@ -2570,78 +2506,79 @@ Internet-Draft Diameter Base Protocol January 2011 or software, an alternative means of entry and display, such as hexadecimal, MAY be provided. - - -Fajardo, et al. Expires July 24, 2011 [Page 46] - -Internet-Draft Diameter Base Protocol January 2011 - - For information encoded in 7-bit US-ASCII, the UTF-8 charset is identical to the US-ASCII charset. UTF-8 may require multiple bytes to represent a single character / - code point; thus the length of an UTF8String in octets may be + code point; thus, the length of a UTF8String in octets may be different from the number of characters encoded. Note that the AVP Length field of an UTF8String is measured in - octets, not characters. + octets not characters. + + + + +Fajardo, et al. Standards Track [Page 45] + +RFC 6733 Diameter Base Protocol October 2012 + DiameterIdentity - The DiameterIdentity format is derived from the OctetString AVP - Base Format. + The DiameterIdentity format is derived from the OctetString Basic + AVP Format. DiameterIdentity = FQDN/Realm - - DiameterIdentity value is used to uniquely identify either: + The DiameterIdentity value is used to uniquely identify either: * A Diameter node for purposes of duplicate connection and routing loop detection. - * A Realm to determine whether messages can be satisfied locally, + * A Realm to determine whether messages can be satisfied locally or whether they must be routed or redirected. + When a DiameterIdentity value is used to identify a Diameter node, + the contents of the string MUST be the Fully Qualified Domain Name + (FQDN) of the Diameter node. If multiple Diameter nodes run on + the same host, each Diameter node MUST be assigned a unique + DiameterIdentity. If a Diameter node can be identified by several + FQDNs, a single FQDN should be picked at startup and used as the + only DiameterIdentity for that node, whatever the connection on + which it is sent. In this document, note that DiameterIdentity is + in ASCII form in order to be compatible with existing DNS + infrastructure. See Appendix D for interactions between the + Diameter protocol and Internationalized Domain Names (IDNs). - When a DiameterIdentity is used to identify a Diameter node the - contents of the string MUST be the FQDN of the Diameter node. If - multiple Diameter nodes run on the same host, each Diameter node - MUST be assigned a unique DiameterIdentity. If a Diameter node - can be identified by several FQDNs, a single FQDN should be picked - at startup, and used as the only DiameterIdentity for that node, - whatever the connection it is sent on. Note that in this - document, DiameterIdentity is in ASCII form in order to be - compatible with existing DNS infrastructure. See Appendix D for - interactions between the Diameter protocol and Internationalized - Domain Name (IDNs). + DiameterURI + The DiameterURI MUST follow the Uniform Resource Identifiers (RFC + 3986) syntax [RFC3986] rules specified below: - DiameterURI + "aaa://" FQDN [ port ] [ transport ] [ protocol ] - The DiameterURI MUST follow the Uniform Resource Identifiers (URI) - syntax [RFC3986] rules specified below: + ; No transport security + "aaas://" FQDN [ port ] [ transport ] [ protocol ] + ; Transport security used + FQDN = < Fully Qualified Domain Name > -Fajardo, et al. Expires July 24, 2011 [Page 47] - -Internet-Draft Diameter Base Protocol January 2011 - "aaa://" FQDN [ port ] [ transport ] [ protocol ] - ; No transport security - "aaas://" FQDN [ port ] [ transport ] [ protocol ] - ; Transport security used - FQDN = Fully Qualified Host Name +Fajardo, et al. Standards Track [Page 46] + +RFC 6733 Diameter Base Protocol October 2012 + port = ":" 1*DIGIT @@ -2649,8 +2586,9 @@ Internet-Draft Diameter Base Protocol January 2011 ; incoming connections. ; If absent, the default Diameter port ; (3868) is assumed if no transport - ; security is used and port (TBD) when - ; transport security (TLS/TCP and DTLS/SCTP) is used. + ; security is used and port 5658 when + ; transport security (TLS/TCP and DTLS/SCTP) + ; is used. transport = ";transport=" transport-protocol @@ -2660,46 +2598,47 @@ Internet-Draft Diameter Base Protocol January 2011 ; UDP MUST NOT be used when the aaa-protocol ; field is set to diameter. - transport-protocol = ( "tcp" / "sctp" / "udp" ) + transport-protocol = ( "tcp" / "sctp" / "udp" ) - protocol = ";protocol=" aaa-protocol + protocol = ";protocol=" aaa-protocol ; If absent, the default AAA protocol ; is Diameter. - aaa-protocol = ( "diameter" / "radius" / "tacacs+" ) + aaa-protocol = ( "diameter" / "radius" / "tacacs+" ) - The following are examples of valid Diameter host identities: + The following are examples of valid Diameter host identities: - aaa://host.example.com;transport=tcp - aaa://host.example.com:6666;transport=tcp - aaa://host.example.com;protocol=diameter - aaa://host.example.com:6666;protocol=diameter - aaa://host.example.com:6666;transport=tcp;protocol=diameter - aaa://host.example.com:1813;transport=udp;protocol=radius + aaa://host.example.com;transport=tcp + aaa://host.example.com:6666;transport=tcp + aaa://host.example.com;protocol=diameter + aaa://host.example.com:6666;protocol=diameter + aaa://host.example.com:6666;transport=tcp;protocol=diameter + aaa://host.example.com:1813;transport=udp;protocol=radius + Enumerated + The Enumerated format is derived from the Integer32 Basic AVP + Format. The definition contains a list of valid values and their + interpretation and is described in the Diameter application + introducing the AVP. -Fajardo, et al. Expires July 24, 2011 [Page 48] - -Internet-Draft Diameter Base Protocol January 2011 - Enumerated - Enumerated is derived from the Integer32 AVP Base Format. The - definition contains a list of valid values and their - interpretation and is described in the Diameter application - introducing the AVP. + +Fajardo, et al. Standards Track [Page 47] + +RFC 6733 Diameter Base Protocol October 2012 IPFilterRule - The IPFilterRule format is derived from the OctetString AVP Base + The IPFilterRule format is derived from the OctetString Basic AVP Format and uses the ASCII charset. The rule syntax is a modified subset of ipfw(8) from FreeBSD. Packets may be filtered based on the following information that is associated with it: @@ -2713,13 +2652,13 @@ Internet-Draft Diameter Base Protocol January 2011 IP options ICMP types - Rules for the appropriate direction are evaluated in order, with - the first matched rule terminating the evaluation. Each packet is - evaluated once. If no rule matches, the packet is dropped if the - last rule evaluated was a permit, and passed if the last rule was - a deny. + Rules for the appropriate direction are evaluated in order, with the + first matched rule terminating the evaluation. Each packet is + evaluated once. If no rule matches, the packet is dropped if the + last rule evaluated was a permit, and passed if the last rule was a + deny. - IPFilterRule filters MUST follow the format: + IPFilterRule filters MUST follow the format: action dir proto from src to dst [options] @@ -2737,22 +2676,28 @@ Internet-Draft Diameter Base Protocol January 2011 The <address/mask> may be specified as: ipno An IPv4 or IPv6 number in dotted- quad or canonical IPv6 form. Only + this exact IP number will match the + rule. + + + + + + -Fajardo, et al. Expires July 24, 2011 [Page 49] +Fajardo, et al. Standards Track [Page 48] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - this exact IP number will match the - rule. ipno/bits An IP number as above with a mask width of the form 192.0.2.10/24. In this case, all IP numbers from 192.0.2.0 to 192.0.2.255 will match. The bit width MUST be valid for the - IP version and the IP number MUST + IP version, and the IP number MUST NOT have bits set beyond the mask. For a match to occur, the same IP version must be present in the @@ -2765,7 +2710,7 @@ Internet-Draft Diameter Base Protocol January 2011 is the address or set of addresses assigned to the terminal. For IPv4, a typical first rule is often "deny - in ip! assigned" + in ip! assigned". The sense of the match can be inverted by preceding an address with the not modifier (!), @@ -2773,7 +2718,7 @@ Internet-Draft Diameter Base Protocol January 2011 instead. This does not affect the selection of port numbers. - With the TCP, UDP and SCTP protocols, optional + With the TCP, UDP, and SCTP protocols, optional ports may be specified as: {port/port-port}[,ports[,...]] @@ -2796,33 +2741,35 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 50] + + +Fajardo, et al. Standards Track [Page 49] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - ipoptions spec - Match if the IP header contains the comma - separated list of options specified in spec. The + ipoptions spec + Match if the IP header contains the comma-separated + list of options specified in spec. The supported IP options are: ssrr (strict source route), lsrr (loose source - route), rr (record packet route) and ts + route), rr (record packet route), and ts (timestamp). The absence of a particular option may be denoted with a '!'. tcpoptions spec - Match if the TCP header contains the comma - separated list of options specified in spec. The + Match if the TCP header contains the comma-separated + list of options specified in spec. The supported TCP options are: mss (maximum segment size), window (tcp window advertisement), sack (selective ack), ts (rfc1323 - timestamp) and cc (rfc1644 t/tcp connection + timestamp), and cc (rfc1644 t/tcp connection count). The absence of a particular option may be denoted with a '!'. - established + established TCP packets only. Match packets that have the RST or ACK bits set. @@ -2832,10 +2779,10 @@ Internet-Draft Diameter Base Protocol January 2011 tcpflags spec TCP packets only. Match if the TCP header - contains the comma separated list of flags + contains the comma-separated list of flags specified in spec. The supported TCP flags are: - fin, syn, rst, psh, ack and urg. The absence of a + fin, syn, rst, psh, ack, and urg. The absence of a particular flag may be denoted with a '!'. A rule that contains a tcpflags specification can never match a fragmented packet that has a non-zero @@ -2852,9 +2799,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 51] +Fajardo, et al. Standards Track [Page 50] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 echo reply (0), destination unreachable (3), @@ -2863,21 +2810,20 @@ Internet-Draft Diameter Base Protocol January 2011 solicitation (10), time-to-live exceeded (11), IP header bad (12), timestamp request (13), timestamp reply (14), information request (15), - information reply (16), address mask request (17) + information reply (16), address mask request (17), and address mask reply (18). - There is one kind of packet that the access device MUST always - discard, that is an IP fragment with a fragment offset of one. - This is a valid packet, but it only has one use, to try to - circumvent firewalls. - - An access device that is unable to interpret or apply a deny rule - MUST terminate the session. An access device that is unable to - interpret or apply a permit rule MAY apply a more restrictive - rule. An access device MAY apply deny rules of its own before the - supplied rules, for example to protect the access device owner's - infrastructure. + There is one kind of packet that the access device MUST always + discard, that is an IP fragment with a fragment offset of one. This + is a valid packet, but it only has one use, to try to circumvent + firewalls. + An access device that is unable to interpret or apply a deny rule + MUST terminate the session. An access device that is unable to + interpret or apply a permit rule MAY apply a more restrictive rule. + An access device MAY apply deny rules of its own before the supplied + rules, for example to protect the access device owner's + infrastructure. 4.4. Grouped AVP Values @@ -2885,75 +2831,76 @@ Internet-Draft Diameter Base Protocol January 2011 implies that the Data field is actually a sequence of AVPs. It is possible to include an AVP with a Grouped type within a Grouped type, that is, to nest them. AVPs within an AVP of type Grouped have the - same padding requirements as non-Grouped AVPs, as defined in Section - 4. + same padding requirements as non-Grouped AVPs, as defined in + Section 4.4. The AVP Code numbering space of all AVPs included in a Grouped AVP is - the same as for non-grouped AVPs. Receivers of a Grouped AVP that + the same as for non-Grouped AVPs. Receivers of a Grouped AVP that does not have the 'M' (mandatory) bit set and one or more of the encapsulated AVPs within the group has the 'M' (mandatory) bit set MAY simply be ignored if the Grouped AVP itself is unrecognized. The rule applies even if the encapsulated AVP with its 'M' (mandatory) - bit set is further encapsulated within other sub-groups; i.e. other + bit set is further encapsulated within other sub-groups, i.e., other Grouped AVPs embedded within the Grouped AVP. - Every Grouped AVP defined MUST include a corresponding grammar, using - ABNF [RFC5234] (with modifications), as defined below. - + Every Grouped AVP definition MUST include a corresponding grammar, + using ABNF [RFC5234] (with modifications), as defined below. + grouped-avp-def = "<" name ">" "::=" avp + name-fmt = ALPHA *(ALPHA / DIGIT / "-") -Fajardo, et al. Expires July 24, 2011 [Page 52] +Fajardo, et al. Standards Track [Page 51] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - grouped-avp-def = <name> "::=" avp - - name-fmt = ALPHA *(ALPHA / DIGIT / "-") - name = name-fmt ; The name has to be the name of an AVP, ; defined in the base or extended Diameter ; specifications. - avp = header [ *fixed] [ *required] [ *optional] + avp = header *fixed *required *optional header = "<" "AVP-Header:" avpcode [vendor] ">" avpcode = 1*DIGIT - ; The AVP Code assigned to the Grouped AVP + ; The AVP Code assigned to the Grouped AVP. vendor = 1*DIGIT ; The Vendor-ID assigned to the Grouped AVP. ; If absent, the default value of zero is ; used. -4.4.1. Example AVP with a Grouped Data type +4.4.1. Example AVP with a Grouped Data Type The Example-AVP (AVP Code 999999) is of type Grouped and is used to clarify how Grouped AVP values work. The Grouped Data field has the - following ABNF grammar: - - - - - - - - + following CCF grammar: + Example-AVP ::= < AVP Header: 999999 > + { Origin-Host } + 1*{ Session-Id } + *[ AVP ] + An Example-AVP with Grouped Data follows. + The Origin-Host AVP (Section 6.3) is required. In this case: + Origin-Host = "example.com". + One or more Session-Ids must follow. Here there are two: + Session-Id = + "grump.example.com:33041;23432;893;0AF3B81" + Session-Id = + "grump.example.com:33054;23561;2358;0AF3B82" @@ -2964,31 +2911,11 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 53] +Fajardo, et al. Standards Track [Page 52] -Internet-Draft Diameter Base Protocol January 2011 - - - Example-AVP ::= < AVP Header: 999999 > - { Origin-Host } - 1*{ Session-Id } - *[ AVP ] - - An Example-AVP with Grouped Data follows. - - The Origin-Host AVP is required (Section 6.3). In this case: - - Origin-Host = "example.com". +RFC 6733 Diameter Base Protocol October 2012 - One or more Session-Ids must follow. Here there are two: - - Session-Id = - "grump.example.com:33041;23432;893;0AF3B81" - - Session-Id = - "grump.example.com:33054;23561;2358;0AF3B82" - optional AVPs included are Recovery-Policy = <binary> @@ -3007,25 +2934,42 @@ Internet-Draft Diameter Base Protocol January 2011 41d018d56fe938f3cbf089aac12a912a2f0d1923a9390e5f789cb2e5067 d3427475e49968f841 - The data for the optional AVPs is represented in hex since the format - of these AVPs is neither known at the time of definition of the - Example-AVP group, nor (likely) at the time when the example instance - of this AVP is interpreted - except by Diameter implementations which - support the same set of AVPs. The encoding example illustrates how - padding is used and how length fields are calculated. Also note that - AVPs may be present in the Grouped AVP value which the receiver - cannot interpret (here, the Recover-Policy and Futuristic-Acct-Record - AVPs). The length of the Example-AVP is the sum of all the length of - the member AVPs including their padding plus the Example-AVP header + The data for the optional AVPs is represented in hexadecimal form + since the format of these AVPs is not known at the time of definition + of the Example-AVP group nor (likely) at the time when the example + instance of this AVP is interpreted -- except by Diameter + implementations that support the same set of AVPs. The encoding + example illustrates how padding is used and how length fields are + calculated. Also, note that AVPs may be present in the Grouped AVP + value that the receiver cannot interpret (here, the Recover-Policy + and Futuristic-Acct-Record AVPs). The length of the Example-AVP is + the sum of all the length of the member AVPs, including their + padding, plus the Example-AVP header size. + + + + + + + + + + + + + + + + -Fajardo, et al. Expires July 24, 2011 [Page 54] - -Internet-Draft Diameter Base Protocol January 2011 - size. + +Fajardo, et al. Standards Track [Page 53] + +RFC 6733 Diameter Base Protocol October 2012 This AVP would be encoded as follows: @@ -3076,15 +3020,18 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 55] + + + +Fajardo, et al. Standards Track [Page 54] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 4.5. Diameter Base Protocol AVPs The following table describes the Diameter AVPs defined in the base - protocol, their AVP Code values, types, possible flag values. + protocol, their AVP Code values, types, and possible flag values. Due to space constraints, the short form DiamIdent is used to represent DiameterIdentity. @@ -3132,9 +3079,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 56] +Fajardo, et al. Standards Track [Page 55] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 +----------+ @@ -3154,7 +3101,7 @@ Internet-Draft Diameter Base Protocol January 2011 Record-Number | | | Accounting- 480 9.8.1 Enumerated | M | V | Record-Type | | | - Accounting- 44 9.8.4 OctetString| M | V | + Acct- 44 9.8.4 OctetString| M | V | Session-Id | | | Accounting- 287 9.8.6 Unsigned64 | M | V | Sub-Session-Id | | | @@ -3188,9 +3135,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 57] +Fajardo, et al. Standards Track [Page 56] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 +----------+ @@ -3244,9 +3191,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 58] +Fajardo, et al. Standards Track [Page 57] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 5. Diameter Peers @@ -3258,25 +3205,25 @@ Internet-Draft Diameter Base Protocol January 2011 Connections between diameter peers are established using their valid DiameterIdentity. A Diameter node initiating a connection to a peer - MUST know the peers DiameterIdentity. Methods for discovering a + MUST know the peer's DiameterIdentity. Methods for discovering a Diameter peer can be found in Section 5.2. - Although a Diameter node may have many possible peers that it is able - to communicate with, it may not be economical to have an established - connection to all of them. At a minimum, a Diameter node SHOULD have - an established connection with two peers per realm, known as the - primary and secondary peers. Of course, a node MAY have additional - connections, if it is deemed necessary. Typically, all messages for - a realm are sent to the primary peer, but in the event that failover - procedures are invoked, any pending requests are sent to the - secondary peer. However, implementations are free to load balance - requests between a set of peers. - - Note that a given peer MAY act as a primary for a given realm, while + Although a Diameter node may have many possible peers with which it + is able to communicate, it may not be economical to have an + established connection to all of them. At a minimum, a Diameter node + SHOULD have an established connection with two peers per realm, known + as the primary and secondary peers. Of course, a node MAY have + additional connections, if it is deemed necessary. Typically, all + messages for a realm are sent to the primary peer but, in the event + that failover procedures are invoked, any pending requests are sent + to the secondary peer. However, implementations are free to load + balance requests between a set of peers. + + Note that a given peer MAY act as a primary for a given realm while acting as a secondary for another realm. When a peer is deemed suspect, which could occur for various reasons, - including not receiving a DWA within an allotted timeframe, no new + including not receiving a DWA within an allotted time frame, no new requests should be forwarded to the peer, but failover procedures are invoked. When an active peer is moved to this mode, additional connections SHOULD be established to ensure that the necessary number @@ -3284,15 +3231,14 @@ Internet-Draft Diameter Base Protocol January 2011 There are two ways that a peer is removed from the suspect peer list: - 1. The peer is no longer reachable, causing the transport connection - to be shutdown. The peer is moved to the closed state. + to be shut down. The peer is moved to the closed state. - 2. Three watchdog messages are exchanged with accepted round trip + 2. Three watchdog messages are exchanged with accepted round-trip times, and the connection to the peer is considered stabilized. In the event the peer being removed is either the primary or - secondary, an alternate peer SHOULD replace the deleted peer, and + secondary, an alternate peer SHOULD replace the deleted peer and assume the role of either primary or secondary. @@ -3300,99 +3246,106 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 59] + +Fajardo, et al. Standards Track [Page 58] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 5.2. Diameter Peer Discovery - Allowing for dynamic Diameter agent discovery will make it possible - for simpler and more robust deployment of Diameter services. In - order to promote interoperable implementations of Diameter peer - discovery, the following mechanisms are described. These are based - on existing IETF standards. The first option (manual configuration) - MUST be supported by all Diameter nodes, while the latter option - (DNS) MAY be supported. + Allowing for dynamic Diameter agent discovery makes possible simpler + and more robust deployment of Diameter services. In order to promote + interoperable implementations of Diameter peer discovery, the + following mechanisms (manual configuration and DNS) are described. + These are based on existing IETF standards. Both mechanisms MUST be + supported by all Diameter implementations; either MAY be used. There are two cases where Diameter peer discovery may be performed. The first is when a Diameter client needs to discover a first-hop Diameter agent. The second case is when a Diameter agent needs to - discover another agent - for further handling of a Diameter - operation. In both cases, the following 'search order' is - recommended: - + discover another agent for further handling of a Diameter operation. + In both cases, the following 'search order' is recommended: - 1. The Diameter implementation consults its list of static + 1. The Diameter implementation consults its list of statically (manually) configured Diameter agent locations. These will be used if they exist and respond. - 2. The Diameter implementation performs a NAPTR query for a server - in a particular realm. The Diameter implementation has to know - in advance which realm to look for a Diameter agent. This could - be deduced, for example, from the 'realm' in a NAI that a - Diameter implementation needed to perform a Diameter operation - on. + in a particular realm. The Diameter implementation has to know, + in advance, in which realm to look for a Diameter agent. This + could be deduced, for example, from the 'realm' in an NAI on + which a Diameter implementation needed to perform a Diameter + operation. The NAPTR usage in Diameter follows the S-NAPTR DDDS application [RFC3958] in which the SERVICE field includes tags for the desired application and supported application protocol. The application service tag for a Diameter application is 'aaa' and the supported application protocol tags are 'diameter.tcp', - 'diameter.sctp', 'diameter.dtls' or 'diameter.tls.tcp'. + 'diameter.sctp', 'diameter.dtls', or 'diameter.tls.tcp' + [RFC6408]. The client can follow the resolution process defined by the - S-NAPTR DDDS [RFC3958] application to find a matching SRV, A or + S-NAPTR DDDS [RFC3958] application to find a matching SRV, A, or AAAA record of a suitable peer. The domain suffixes in the NAPTR replacement field SHOULD match the domain of the original query. An example can be found in Appendix B. 3. If no NAPTR records are found, the requester directly queries for - SRV records '_diameter._sctp'.realm, '_diameter._dtls'.realm, - '_diameter._tcp'.realm and '_diameter._tls'.realm depending on - the requesters network protocol capabilities. If SRV records are - found then the requester can perform address record query (A RR's + one of the following SRV records: for Diameter over TCP, use + "_diameter._tcp.realm"; for Diameter over TLS, use + "_diameters._tcp.realm"; for Diameter over SCTP, use + "_diameter._sctp.realm"; for Diameter over DTLS, use + "_diameters._sctp.realm". If SRV records are found, then the + requester can perform address record query (A RR's and/or AAAA + -Fajardo, et al. Expires July 24, 2011 [Page 60] +Fajardo, et al. Standards Track [Page 59] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - and/or AAAA RR's) for the target hostname specified in the SRV - records. If no SRV records are found, the requester gives up. + RR's) for the target hostname specified in the SRV records + following the rules given in [RFC2782]. If no SRV records are + found, the requester gives up. If the server is using a site certificate, the domain name in the NAPTR query and the domain name in the replacement field MUST both be valid based on the site certificate handed out by the server in the - TLS/TCP and DTLS/SCTP or IKE exchange. Similarly, the domain name in - the SRV query and the domain name in the target in the SRV record - MUST both be valid based on the same site certificate. Otherwise, an - attacker could modify the DNS records to contain replacement values - in a different domain, and the client could not validate that this - was the desired behavior, or the result of an attack. - - Also, the Diameter Peer MUST check to make sure that the discovered + TLS/TCP and DTLS/SCTP or Internet Key Exchange Protocol (IKE) + exchange. Similarly, the domain name in the SRV query and the domain + name in the target in the SRV record MUST both be valid based on the + same site certificate. Otherwise, an attacker could modify the DNS + records to contain replacement values in a different domain, and the + client could not validate whether this was the desired behavior or + the result of an attack. + + Also, the Diameter peer MUST check to make sure that the discovered peers are authorized to act in its role. Authentication via IKE or TLS/TCP and DTLS/SCTP, or validation of DNS RRs via DNSSEC is not sufficient to conclude this. For example, a web server may have obtained a valid TLS/TCP and DTLS/SCTP certificate, and secured RRs may be included in the DNS, but this does not imply that it is - authorized to act as a Diameter Server. - - Authorization can be achieved for example, by configuration of a - Diameter Server CA. Alternatively this can be achieved by definition - of OIDs within TLS/TCP and DTLS/SCTP or IKE certificates so as to - signify Diameter Server authorization. - - A dynamically discovered peer causes an entry in the Peer Table (see + authorized to act as a Diameter server. + + Authorization can be achieved, for example, by the configuration of a + Diameter server Certification Authority (CA). The server CA issues a + certificate to the Diameter server, which includes an Object + Identifier (OID) to indicate the subject is a Diameter server in the + Extended Key Usage extension [RFC5280]. This certificate is then + used during TLS/TCP, DTLS/SCTP, or IKE security negotiation. + However, note that, at the time of writing, no Diameter server + Certification Authorities exist. + + A dynamically discovered peer causes an entry in the peer table (see Section 2.6) to be created. Note that entries created via DNS MUST - expire (or be refreshed) within the DNS TTL. If a peer is discovered - outside of the local realm, a routing table entry (see Section 2.7) - for the peer's realm is created. The routing table entry's - expiration MUST match the peer's expiration value. + expire (or be refreshed) within the DNS Time to Live (TTL). If a + peer is discovered outside of the local realm, a routing table entry + (see Section 2.7) for the peer's realm is created. The routing table + entry's expiration MUST match the peer's expiration value. 5.3. Capabilities Exchange @@ -3400,35 +3353,36 @@ Internet-Draft Diameter Base Protocol January 2011 exchange the Capabilities Exchange messages, as specified in the peer state machine (see Section 5.6). This message allows the discovery of a peer's identity and its capabilities (protocol version number, - supported Diameter applications, security mechanisms, etc.) + the identifiers of supported Diameter applications, security + mechanisms, etc.). - The receiver only issues commands to its peers that have advertised - support for the Diameter application that defines the command. A - Diameter node MUST cache the supported applications in order to - ensure that unrecognized commands and/or AVPs are not unnecessarily - sent to a peer. - - A receiver of a Capabilities-Exchange-Req (CER) message that does not -Fajardo, et al. Expires July 24, 2011 [Page 61] +Fajardo, et al. Standards Track [Page 60] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - have any applications in common with the sender MUST return a + The receiver only issues commands to its peers that have advertised + support for the Diameter application that defines the command. A + Diameter node MUST cache the supported Application Ids in order to + ensure that unrecognized commands and/or AVPs are not unnecessarily + sent to a peer. + + A receiver of a Capabilities-Exchange-Request (CER) message that does + not have any applications in common with the sender MUST return a Capabilities-Exchange-Answer (CEA) with the Result-Code AVP set to - DIAMETER_NO_COMMON_APPLICATION, and SHOULD disconnect the transport + DIAMETER_NO_COMMON_APPLICATION and SHOULD disconnect the transport layer connection. Note that receiving a CER or CEA from a peer - advertising itself as a Relay (see Section 2.4) MUST be interpreted + advertising itself as a relay (see Section 2.4) MUST be interpreted as having common applications with the peer. The receiver of the Capabilities-Exchange-Request (CER) MUST determine common applications by computing the intersection of its - own set of supported Application Id against all of the application - identifier AVPs (Auth-Application-Id, Acct-Application-Id and Vendor- - Specific-Application-Id) present in the CER. The value of the + own set of supported Application Ids against all of the + Application-Id AVPs (Auth-Application-Id, Acct-Application-Id, and + Vendor-Specific-Application-Id) present in the CER. The value of the Vendor-Id AVP in the Vendor-Specific-Application-Id MUST NOT be used during computation. The sender of the Capabilities-Exchange-Answer (CEA) SHOULD include all of its supported applications as a hint to @@ -3438,12 +3392,13 @@ Internet-Draft Diameter Base Protocol January 2011 and DTLS/SCTP connection prior to the CER/CEA exchange. This protects the capabilities information of both peers. To support older Diameter implementations that do not fully conform to this - document, the transport security MAY still be negotiated via Inband- - Security AVP. In this case, the receiver of a Capabilities-Exchange- - Req (CER) message that does not have any security mechanisms in - common with the sender MUST return a Capabilities-Exchange-Answer - (CEA) with the Result-Code AVP set to DIAMETER_NO_COMMON_SECURITY, - and SHOULD disconnect the transport layer connection. + document, the transport security MAY still be negotiated via an + Inband-Security AVP. In this case, the receiver of a Capabilities- + Exchange-Request (CER) message that does not have any security + mechanisms in common with the sender MUST return a Capabilities- + Exchange-Answer (CEA) with the Result-Code AVP set to + DIAMETER_NO_COMMON_SECURITY and SHOULD disconnect the transport layer + connection. CERs received from unknown peers MAY be silently discarded, or a CEA MAY be issued with the Result-Code AVP set to DIAMETER_UNKNOWN_PEER. @@ -3455,34 +3410,34 @@ Internet-Draft Diameter Base Protocol January 2011 failure, all the pending transactions destined to the unknown peer can be discarded. - The CER and CEA messages MUST NOT be proxied, redirected or relayed. + The CER and CEA messages MUST NOT be proxied, redirected, or relayed. - Since the CER/CEA messages cannot be proxied, it is still possible - that an upstream agent receives a message for which it has no - available peers to handle the application that corresponds to the - Command-Code. In such instances, the 'E' bit is set in the answer - message (see Section 7.) with the Result-Code AVP set to - DIAMETER_UNABLE_TO_DELIVER to inform the downstream to take action - (e.g., re-routing request to an alternate peer). - -Fajardo, et al. Expires July 24, 2011 [Page 62] +Fajardo, et al. Standards Track [Page 61] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + Since the CER/CEA messages cannot be proxied, it is still possible + that an upstream agent will receive a message for which it has no + available peers to handle the application that corresponds to the + Command Code. In such instances, the 'E' bit is set in the answer + message (Section 7) with the Result-Code AVP set to + DIAMETER_UNABLE_TO_DELIVER to inform the downstream agent to take + action (e.g., re-routing request to an alternate peer). + With the exception of the Capabilities-Exchange-Request message, a message of type Request that includes the Auth-Application-Id or Acct-Application-Id AVPs, or a message with an application-specific - command code, MAY only be forwarded to a host that has explicitly + Command Code MAY only be forwarded to a host that has explicitly advertised support for the application (or has advertised the Relay Application Id). 5.3.1. Capabilities-Exchange-Request - The Capabilities-Exchange-Request (CER), indicated by the Command- + The Capabilities-Exchange-Request (CER), indicated by the Command Code set to 257 and the Command Flags' 'R' bit set, is sent to exchange local capabilities. Upon detection of a transport failure, this message MUST NOT be sent to an alternate peer. @@ -3490,7 +3445,7 @@ Internet-Draft Diameter Base Protocol January 2011 When Diameter is run over SCTP [RFC4960] or DTLS/SCTP [RFC6083], which allow for connections to span multiple interfaces and multiple IP addresses, the Capabilities-Exchange-Request message MUST contain - one Host-IP- Address AVP for each potential IP address that MAY be + one Host-IP-Address AVP for each potential IP address that MAY be locally used when transmitting Diameter messages. Message Format @@ -3510,9 +3465,20 @@ Internet-Draft Diameter Base Protocol January 2011 [ Firmware-Revision ] * [ AVP ] + + + + + + +Fajardo, et al. Standards Track [Page 62] + +RFC 6733 Diameter Base Protocol October 2012 + + 5.3.2. Capabilities-Exchange-Answer - The Capabilities-Exchange-Answer (CEA), indicated by the Command-Code + The Capabilities-Exchange-Answer (CEA), indicated by the Command Code set to 257 and the Command Flags' 'R' bit cleared, is sent in response to a CER message. @@ -3522,13 +3488,6 @@ Internet-Draft Diameter Base Protocol January 2011 one Host-IP-Address AVP for each potential IP address that MAY be locally used when transmitting Diameter messages. - - -Fajardo, et al. Expires July 24, 2011 [Page 63] - -Internet-Draft Diameter Base Protocol January 2011 - - Message Format <CEA> ::= < Diameter Header: 257 > @@ -3552,22 +3511,34 @@ Internet-Draft Diameter Base Protocol January 2011 5.3.3. Vendor-Id AVP The Vendor-Id AVP (AVP Code 266) is of type Unsigned32 and contains - the IANA "SMI Network Management Private Enterprise Codes" [RFC3232] - value assigned to the vendor of the Diameter device. It is + the IANA "SMI Network Management Private Enterprise Codes" + [ENTERPRISE] value assigned to the Diameter Software vendor. It is envisioned that the combination of the Vendor-Id, Product-Name - (Section 5.3.7) and the Firmware-Revision (Section 5.3.4) AVPs may + (Section 5.3.7), and Firmware-Revision (Section 5.3.4) AVPs may provide useful debugging information. - A Vendor-Id value of zero in the CER or CEA messages is reserved and + A Vendor-Id value of zero in the CER or CEA message is reserved and indicates that this field is ignored. + + + + + + + +Fajardo, et al. Standards Track [Page 63] + +RFC 6733 Diameter Base Protocol October 2012 + + 5.3.4. Firmware-Revision AVP The Firmware-Revision AVP (AVP Code 267) is of type Unsigned32 and is used to inform a Diameter peer of the firmware revision of the issuing device. - For devices that do not have a firmware revision (general purpose + For devices that do not have a firmware revision (general-purpose computers running Diameter software modules, for instance), the revision of the Diameter software module may be reported instead. @@ -3577,77 +3548,70 @@ Internet-Draft Diameter Base Protocol January 2011 to inform a Diameter peer of the sender's IP address. All source addresses that a Diameter node expects to use with SCTP [RFC4960] or DTLS/SCTP [RFC6083] MUST be advertised in the CER and CEA messages by - - - -Fajardo, et al. Expires July 24, 2011 [Page 64] - -Internet-Draft Diameter Base Protocol January 2011 - - including a Host-IP-Address AVP for each address. 5.3.6. Supported-Vendor-Id AVP The Supported-Vendor-Id AVP (AVP Code 265) is of type Unsigned32 and contains the IANA "SMI Network Management Private Enterprise Codes" - [RFC3232] value assigned to a vendor other than the device vendor but - including the application vendor. This is used in the CER and CEA - messages in order to inform the peer that the sender supports (a - subset of) the vendor-specific AVPs defined by the vendor identified + [ENTERPRISE] value assigned to a vendor other than the device vendor + but including the application vendor. This is used in the CER and + CEA messages in order to inform the peer that the sender supports (a + subset of) the Vendor-Specific AVPs defined by the vendor identified in this AVP. The value of this AVP MUST NOT be set to zero. Multiple instances of this AVP containing the same value SHOULD NOT be sent. 5.3.7. Product-Name AVP - The Product-Name AVP (AVP Code 269) is of type UTF8String, and - contains the vendor assigned name for the product. The Product-Name + The Product-Name AVP (AVP Code 269) is of type UTF8String and + contains the vendor-assigned name for the product. The Product-Name AVP SHOULD remain constant across firmware revisions for the same product. -5.4. Disconnecting Peer connections +5.4. Disconnecting Peer Connections When a Diameter node disconnects one of its transport connections, - its peer cannot know the reason for the disconnect, and will most - likely assume that a connectivity problem occurred, or that the peer + its peer cannot know the reason for the disconnect and will most + likely assume that a connectivity problem occurred or that the peer has rebooted. In these cases, the peer may periodically attempt to reconnect, as stated in Section 2.1. In the event that the - disconnect was a result of either a shortage of internal resources, - or simply that the node in question has no intentions of forwarding - any Diameter messages to the peer in the foreseeable future, a - periodic connection request would not be welcomed. The - Disconnection-Reason AVP contains the reason the Diameter node issued - the Disconnect-Peer-Request message. + disconnect was a result of either a shortage of internal resources or + simply that the node in question has no intentions of forwarding any + Diameter messages to the peer in the foreseeable future, a periodic - The Disconnect-Peer-Request message is used by a Diameter node to - inform its peer of its intent to disconnect the transport layer, and - that the peer shouldn't reconnect unless it has a valid reason to do - so (e.g., message to be forwarded). Upon receipt of the message, the - Disconnect-Peer-Answer is returned, which SHOULD contain an error if - messages have recently been forwarded, and are likely in flight, - which would otherwise cause a race condition. - The receiver of the Disconnect-Peer-Answer initiates the transport - disconnect. The sender of the Disconnect-Peer-Answer should be able - to detect the transport closure and cleanup the connection. +Fajardo, et al. Standards Track [Page 64] + +RFC 6733 Diameter Base Protocol October 2012 + connection request would not be welcomed. The Disconnection-Reason + AVP contains the reason the Diameter node issued the Disconnect-Peer- + Request message. -Fajardo, et al. Expires July 24, 2011 [Page 65] - -Internet-Draft Diameter Base Protocol January 2011 + The Disconnect-Peer-Request message is used by a Diameter node to + inform its peer of its intent to disconnect the transport layer and + that the peer shouldn't reconnect unless it has a valid reason to do + so (e.g., message to be forwarded). Upon receipt of the message, the + Disconnect-Peer-Answer message is returned, which SHOULD contain an + error if messages have recently been forwarded, and are likely in + flight, which would otherwise cause a race condition. + The receiver of the Disconnect-Peer-Answer message initiates the + transport disconnect. The sender of the Disconnect-Peer-Answer + message should be able to detect the transport closure and clean up + the connection. 5.4.1. Disconnect-Peer-Request - The Disconnect-Peer-Request (DPR), indicated by the Command-Code set + The Disconnect-Peer-Request (DPR), indicated by the Command Code set to 282 and the Command Flags' 'R' bit set, is sent to a peer to - inform its intentions to shutdown the transport connection. Upon - detection of a transport failure, this message MUST NOT be sent to an - alternate peer. + inform it of its intentions to shut down the transport connection. + Upon detection of a transport failure, this message MUST NOT be sent + to an alternate peer. Message Format @@ -3659,28 +3623,14 @@ Internet-Draft Diameter Base Protocol January 2011 5.4.2. Disconnect-Peer-Answer - The Disconnect-Peer-Answer (DPA), indicated by the Command-Code set + The Disconnect-Peer-Answer (DPA), indicated by the Command Code set to 282 and the Command Flags' 'R' bit cleared, is sent as a response to the Disconnect-Peer-Request message. Upon receipt of this - message, the transport connection is shutdown. + message, the transport connection is shut down. - Message Format - <DPA> ::= < Diameter Header: 282 > - { Result-Code } - { Origin-Host } - { Origin-Realm } - [ Error-Message ] - [ Failed-AVP ] - * [ AVP ] -5.4.3. Disconnect-Cause AVP - The Disconnect-Cause AVP (AVP Code 273) is of type Enumerated. A - Diameter node MUST include this AVP in the Disconnect-Peer-Request - message to inform the peer of the reason for its intention to - shutdown the transport connection. The following values are - supported: @@ -3689,29 +3639,44 @@ Internet-Draft Diameter Base Protocol January 2011 +Fajardo, et al. Standards Track [Page 65] + +RFC 6733 Diameter Base Protocol October 2012 + Message Format + + <DPA> ::= < Diameter Header: 282 > + { Result-Code } + { Origin-Host } + { Origin-Realm } + [ Error-Message ] + [ Failed-AVP ] + * [ AVP ] -Fajardo, et al. Expires July 24, 2011 [Page 66] - -Internet-Draft Diameter Base Protocol January 2011 +5.4.3. Disconnect-Cause AVP + + The Disconnect-Cause AVP (AVP Code 273) is of type Enumerated. A + Diameter node MUST include this AVP in the Disconnect-Peer-Request + message to inform the peer of the reason for its intention to shut + down the transport connection. The following values are supported: - REBOOTING 0 - A scheduled reboot is imminent. Receiver of DPR with above result - code MAY attempt reconnection. + REBOOTING 0 + A scheduled reboot is imminent. A receiver of a DPR with + above result code MAY attempt reconnection. - BUSY 1 - The peer's internal resources are constrained, and it has - determined that the transport connection needs to be closed. - Receiver of DPR with above result code SHOULD NOT attempt - reconnection. + BUSY 1 + The peer's internal resources are constrained, and it has + determined that the transport connection needs to be closed. + A receiver of a DPR with above result code SHOULD NOT attempt + reconnection. - DO_NOT_WANT_TO_TALK_TO_YOU 2 - The peer has determined that it does not see a need for the - transport connection to exist, since it does not expect any - messages to be exchanged in the near future. Receiver of DPR - with above result code SHOULD NOT attempt reconnection. + DO_NOT_WANT_TO_TALK_TO_YOU 2 + The peer has determined that it does not see a need for the + transport connection to exist, since it does not expect any + messages to be exchanged in the near future. A receiver of a + DPR with above result code SHOULD NOT attempt reconnection. 5.5. Transport Failure Detection @@ -3723,9 +3688,21 @@ Internet-Draft Diameter Base Protocol January 2011 Watchdog-Answer messages, defined in this section, are used to pro- actively detect transport failures. + + + + + + + +Fajardo, et al. Standards Track [Page 66] + +RFC 6733 Diameter Base Protocol October 2012 + + 5.5.1. Device-Watchdog-Request - The Device-Watchdog-Request (DWR), indicated by the Command-Code set + The Device-Watchdog-Request (DWR), indicated by the Command Code set to 280 and the Command Flags' 'R' bit set, is sent to a peer when no traffic has been exchanged between two peers (see Section 5.5.3). Upon detection of a transport failure, this message MUST NOT be sent @@ -3741,18 +3718,10 @@ Internet-Draft Diameter Base Protocol January 2011 5.5.2. Device-Watchdog-Answer - The Device-Watchdog-Answer (DWA), indicated by the Command-Code set + The Device-Watchdog-Answer (DWA), indicated by the Command Code set to 280 and the Command Flags' 'R' bit cleared, is sent as a response to the Device-Watchdog-Request message. - - - -Fajardo, et al. Expires July 24, 2011 [Page 67] - -Internet-Draft Diameter Base Protocol January 2011 - - Message Format <DWA> ::= < Diameter Header: 280 > @@ -3767,7 +3736,7 @@ Internet-Draft Diameter Base Protocol January 2011 5.5.3. Transport Failure Algorithm The transport failure algorithm is defined in [RFC3539]. All - Diameter implementations MUST support the algorithm defined in the + Diameter implementations MUST support the algorithm defined in that specification in order to be compliant to the Diameter base protocol. 5.5.4. Failover and Failback Procedures @@ -3775,7 +3744,17 @@ Internet-Draft Diameter Base Protocol January 2011 In the event that a transport failure is detected with a peer, it is necessary for all pending request messages to be forwarded to an alternate agent, if possible. This is commonly referred to as - failover. + "failover". + + + + + + +Fajardo, et al. Standards Track [Page 67] + +RFC 6733 Diameter Base Protocol October 2012 + In order for a Diameter node to perform failover procedures, it is necessary for the node to maintain a pending message queue for a @@ -3783,16 +3762,16 @@ Internet-Draft Diameter Base Protocol January 2011 request is removed from the queue. The Hop-by-Hop Identifier field is used to match the answer with the queued request. - When a transport failure is detected, if possible all messages in the - queue are sent to an alternate agent with the T flag set. On booting - a Diameter client or agent, the T flag is also set on any records - still remaining to be transmitted in non-volatile storage. An - example of a case where it is not possible to forward the message to - an alternate server is when the message has a fixed destination, and - the unavailable peer is the message's final destination (see - Destination-Host AVP). Such an error requires that the agent return - an answer message with the 'E' bit set and the Result-Code AVP set to - DIAMETER_UNABLE_TO_DELIVER. + When a transport failure is detected, if possible, all messages in + the queue are sent to an alternate agent with the T flag set. On + booting a Diameter client or agent, the T flag is also set on any + remaining records in non-volatile storage that are still waiting to + be transmitted. An example of a case where it is not possible to + forward the message to an alternate server is when the message has a + fixed destination, and the unavailable peer is the message's final + destination (see Destination-Host AVP). Such an error requires that + the agent return an answer message with the 'E' bit set and the + Result-Code AVP set to DIAMETER_UNABLE_TO_DELIVER. It is important to note that multiple identical requests or answers MAY be received as a result of a failover. The End-to-End Identifier @@ -3801,17 +3780,9 @@ Internet-Draft Diameter Base Protocol January 2011 As described in Section 2.1, a connection request should be periodically attempted with the failed peer in order to re-establish - - - -Fajardo, et al. Expires July 24, 2011 [Page 68] - -Internet-Draft Diameter Base Protocol January 2011 - - the transport connection. Once a connection has been successfully established, messages can once again be forwarded to the peer. This - is commonly referred to as failback. + is commonly referred to as "failback". 5.6. Peer State Machine @@ -3824,16 +3795,24 @@ Internet-Draft Diameter Base Protocol January 2011 This state machine is closely coupled with the state machine described in [RFC3539], which is used to open, close, failover, - probe, and reopen transport connections. Note in particular that + probe, and reopen transport connections. In particular, note that [RFC3539] requires the use of watchdog messages to probe connections. For Diameter, DWR and DWA messages are to be used. - I- is used to represent the initiator (connecting) connection, while - the R- is used to represent the responder (listening) connection. - The lack of a prefix indicates that the event or action is the same - regardless of the connection on which the event occurred. + The I- prefix is used to represent the initiator (connecting) + connection, while the R- prefix is used to represent the responder + (listening) connection. The lack of a prefix indicates that the + event or action is the same regardless of the connection on which the + event occurred. + + + +Fajardo, et al. Standards Track [Page 68] + +RFC 6733 Diameter Base Protocol October 2012 + - The stable states that a state machine may be in are Closed, I-Open + The stable states that a state machine may be in are Closed, I-Open, and R-Open; all other states are intermediate. Note that I-Open and R-Open are equivalent except for whether the initiator or responder transport connection is used for communication. @@ -3848,23 +3827,15 @@ Internet-Draft Diameter Base Protocol January 2011 the results of an election on one peer are guaranteed to be the inverse of the results on the other. - For TLS/TCP and DTLS/SCTP usage, TLS/TCP and DTLS/SCTP handshake + For TLS/TCP and DTLS/SCTP usage, a TLS/TCP and DTLS/SCTP handshake SHOULD begin when both ends are in the closed state prior to any Diameter message exchanges. The TLS/TCP and DTLS/SCTP connection SHOULD be established before sending any CER or CEA message to secure and protect the capabilities information of both peers. The TLS/TCP and DTLS/SCTP connection SHOULD be disconnected when the state machine moves to the closed state. When connecting to responders - that do not conform to this document (i.e. older Diameter + that do not conform to this document (i.e., older Diameter implementations that are not prepared to received TLS/TCP and DTLS/ - - - -Fajardo, et al. Expires July 24, 2011 [Page 69] - -Internet-Draft Diameter Base Protocol January 2011 - - SCTP connections in the closed state), the initial TLS/TCP and DTLS/ SCTP connection attempt will fail. The initiator MAY then attempt to connect via TCP or SCTP and initiate the TLS/TCP and DTLS/SCTP @@ -3878,6 +3849,25 @@ Internet-Draft Diameter Base Protocol January 2011 Any implementation that produces equivalent results is considered compliant. + + + + + + + + + + + + + + +Fajardo, et al. Standards Track [Page 69] + +RFC 6733 Diameter Base Protocol October 2012 + + state event action next state ----------------------------------------------------------------- Closed Start I-Snd-Conn-Req Wait-Conn-Ack @@ -3914,13 +3904,6 @@ Internet-Draft Diameter Base Protocol January 2011 R-Conn-CER R-Reject Wait-Returns Timeout Error Closed - - -Fajardo, et al. Expires July 24, 2011 [Page 70] - -Internet-Draft Diameter Base Protocol January 2011 - - R-Open Send-Message R-Snd-Message R-Open R-Rcv-Message Process R-Open R-Rcv-DWR Process-DWR, R-Open @@ -3928,10 +3911,19 @@ Internet-Draft Diameter Base Protocol January 2011 R-Rcv-DWA Process-DWA R-Open R-Conn-CER R-Reject R-Open Stop R-Snd-DPR Closing - R-Rcv-DPR R-Snd-DPA, Closed - R-Disc + R-Rcv-DPR R-Snd-DPA Closing R-Peer-Disc R-Disc Closed + + + + + +Fajardo, et al. Standards Track [Page 70] + +RFC 6733 Diameter Base Protocol October 2012 + + I-Open Send-Message I-Snd-Message I-Open I-Rcv-Message Process I-Open I-Rcv-DWR Process-DWR, I-Open @@ -3939,8 +3931,7 @@ Internet-Draft Diameter Base Protocol January 2011 I-Rcv-DWA Process-DWA I-Open R-Conn-CER R-Reject I-Open Stop I-Snd-DPR Closing - I-Rcv-DPR I-Snd-DPA, Closed - I-Disc + I-Rcv-DPR I-Snd-DPA Closing I-Peer-Disc I-Disc Closed Closing I-Rcv-DPA I-Disc Closed @@ -3949,88 +3940,44 @@ Internet-Draft Diameter Base Protocol January 2011 I-Peer-Disc I-Disc Closed R-Peer-Disc R-Disc Closed -5.6.1. Incoming connections +5.6.1. Incoming Connections When a connection request is received from a Diameter peer, it is not, in the general case, possible to know the identity of that peer until a CER is received from it. This is because host and port - determine the identity of a Diameter peer; and the source port of an - incoming connection is arbitrary. Upon receipt of CER, the identity - of the connecting peer can be uniquely determined from Origin-Host. + determine the identity of a Diameter peer; the source port of an + incoming connection is arbitrary. Upon receipt of a CER, the + identity of the connecting peer can be uniquely determined from the + Origin-Host. For this reason, a Diameter peer must employ logic separate from the state machine to receive connection requests, accept them, and await - CER. Once CER arrives on a new connection, the Origin-Host that - identifies the peer is used to locate the state machine associated - with that peer, and the new connection and CER are passed to the - state machine as an R-Conn-CER event. + the CER. Once the CER arrives on a new connection, the Origin-Host + that identifies the peer is used to locate the state machine + associated with that peer, and the new connection and CER are passed + to the state machine as an R-Conn-CER event. The logic that handles incoming connections SHOULD close and discard - the connection if any message other than CER arrives, or if an + the connection if any message other than a CER arrives or if an implementation-defined timeout occurs prior to receipt of CER. - - - -Fajardo, et al. Expires July 24, 2011 [Page 71] - -Internet-Draft Diameter Base Protocol January 2011 - - Because handling of incoming connections up to and including receipt - of CER requires logic, separate from that of any individual state + of a CER requires logic, separate from that of any individual state machine associated with a particular peer, it is described separately in this section rather than in the state machine above. 5.6.2. Events Transitions and actions in the automaton are caused by events. In - this section, we will ignore the -I and -R prefix, since the actual - event would be identical, but would occur on one of two possible + this section, we will ignore the I- and R- prefixes, since the actual + event would be identical, but it would occur on one of two possible connections. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 72] +Fajardo, et al. Standards Track [Page 71] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Start The Diameter application has signaled that a @@ -4053,7 +4000,8 @@ Internet-Draft Diameter Base Protocol January 2011 Rcv-CEA A CEA message from the peer was received. - Rcv-Non-CEA A message other than CEA from the peer was received. + Rcv-Non-CEA A message, other than a CEA, from the peer was + received. Peer-Disc A disconnection indication from the peer was received. @@ -4066,7 +4014,7 @@ Internet-Draft Diameter Base Protocol January 2011 Send-Message A message is to be sent. - Rcv-Message A message other than CER, CEA, DPR, DPA, DWR or DWA + Rcv-Message A message other than CER, CEA, DPR, DPA, DWR, or DWA was received. Stop The Diameter application has signaled that a @@ -4077,16 +4025,15 @@ Internet-Draft Diameter Base Protocol January 2011 Actions in the automaton are caused by events and typically indicate the transmission of packets and/or an action to be taken on the - connection. In this section we will ignore the I- and R-prefix, - since the actual action would be identical, but would occur on one of - two possible connections. - + connection. In this section, we will ignore the I- and R- prefixes, + since the actual action would be identical, but it would occur on one + of two possible connections. -Fajardo, et al. Expires July 24, 2011 [Page 73] +Fajardo, et al. Standards Track [Page 72] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Snd-Conn-Req A transport connection is initiated with the peer. @@ -4098,11 +4045,12 @@ Internet-Draft Diameter Base Protocol January 2011 is disconnected. Process-CER The CER associated with the R-Conn-CER is processed. + Snd-CER A CER message is sent to the peer. Snd-CEA A CEA message is sent to the peer. - Cleanup If necessary, the connection is shutdown, and any + Cleanup If necessary, the connection is shut down, and any local resources are freed. Error The transport layer connection is disconnected, @@ -4139,10 +4087,9 @@ Internet-Draft Diameter Base Protocol January 2011 - -Fajardo, et al. Expires July 24, 2011 [Page 74] +Fajardo, et al. Standards Track [Page 73] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 5.6.4. The Election Process @@ -4150,134 +4097,97 @@ Internet-Draft Diameter Base Protocol January 2011 The election is performed on the responder. The responder compares the Origin-Host received in the CER with its own Origin-Host as two streams of octets. If the local Origin-Host lexicographically - succeeds the received Origin-Host a Win-Election event is issued - locally. Diameter identities are in ASCII form therefore the lexical - comparison is consistent with DNS case insensitivity where octets - that fall in the ASCII range 'a' through 'z' MUST compare equally to - their upper-case counterparts between 'A' and 'Z'. See Appendix D - for interactions between the Diameter protocol and Internationalized - Domain Name (IDNs). + succeeds the received Origin-Host, a Win-Election event is issued + locally. Diameter identities are in ASCII form; therefore, the + lexical comparison is consistent with DNS case insensitivity, where + octets that fall in the ASCII range 'a' through 'z' MUST compare + equally to their uppercase counterparts between 'A' and 'Z'. See + Appendix D for interactions between the Diameter protocol and + Internationalized Domain Name (IDNs). The winner of the election MUST close the connection it initiated. Historically, maintaining the responder side of a connection was more efficient than maintaining the initiator side. However, current practices makes this distinction irrelevant. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 75] - -Internet-Draft Diameter Base Protocol January 2011 - - -6. Diameter message processing +6. Diameter Message Processing This section describes how Diameter requests and answers are created and processed. 6.1. Diameter Request Routing Overview - A request is sent towards its final destination using a combination - of the Destination-Realm and Destination-Host AVPs, in one of these - three combinations: + A request is sent towards its final destination using one of the + following three combinations of the Destination-Realm and + Destination-Host AVPs: - o a request that is not able to be proxied (such as CER) MUST NOT + o A request that is not able to be proxied (such as a CER) MUST NOT contain either Destination-Realm or Destination-Host AVPs. - o a request that needs to be sent to a home server serving a + o A request that needs to be sent to a home server serving a specific realm, but not to a specific server (such as the first - request of a series of round-trips), MUST contain a Destination- - Realm AVP, but MUST NOT contain a Destination-Host AVP. For + request of a series of round trips), MUST contain a Destination- + Realm AVP but MUST NOT contain a Destination-Host AVP. For Diameter clients, the value of the Destination-Realm AVP MAY be extracted from the User-Name AVP, or other methods. - o otherwise, a request that needs to be sent to a specific home - server among those serving a given realm, MUST contain both the + o Otherwise, a request that needs to be sent to a specific home + server among those serving a given realm MUST contain both the Destination-Realm and Destination-Host AVPs. The Destination-Host AVP is used as described above when the destination of the request is fixed, which includes: - o Authentication requests that span multiple round trips + o Authentication requests that span multiple round trips. + + + + +Fajardo, et al. Standards Track [Page 74] + +RFC 6733 Diameter Base Protocol October 2012 + o A Diameter message that uses a security mechanism that makes use of a pre-established session key shared between the source and the final destination of the message. - o Server initiated messages that MUST be received by a specific + o Server-initiated messages that MUST be received by a specific Diameter client (e.g., access device), such as the Abort-Session- Request message, which is used to request that a particular user's session be terminated. - Note that an agent can forward a request to a host described in the - Destination-Host AVP only if the host in question is included in its - peer table (see Section 2.7). Otherwise, the request is routed based - on the Destination-Realm only (see Sections 6.1.6). + Note that an agent can only forward a request to a host described in + the Destination-Host AVP if the host in question is included in its + peer table (see Section 2.6). Otherwise, the request is routed based + on the Destination-Realm only (see Section 6.1.6). When a message is received, the message is processed in the following order: - - - - -Fajardo, et al. Expires July 24, 2011 [Page 76] - -Internet-Draft Diameter Base Protocol January 2011 - - o If the message is destined for the local host, the procedures listed in Section 6.1.4 are followed. o If the message is intended for a Diameter peer with whom the local host is able to directly communicate, the procedures listed in - Section 6.1.5 are followed. This is known as Request Forwarding. + Section 6.1.5 are followed. This is known as "Request + Forwarding". - o The procedures listed in Section 6.1.6 are followed, which is - known as Request Routing. + o The procedure listed in Section 6.1.6 is followed, which is known + as "Request Routing". - o If none of the above is successful, an answer is returned with the - Result-Code set to DIAMETER_UNABLE_TO_DELIVER, with the E-bit set. + o If none of the above are successful, an answer is returned with + the Result-Code set to DIAMETER_UNABLE_TO_DELIVER, with the 'E' + bit set. For routing of Diameter messages to work within an administrative domain, all Diameter nodes within the realm MUST be peers. - Note the processing rules contained in this section are intended to - be used as general guidelines to Diameter developers. Certain - implementations MAY use different methods than the ones described - here, and still comply with the protocol specification. See Section - 7 for more detail on error handling. + The overview contained in this section (6.1) is intended to provide + general guidelines to Diameter developers. Implementations are free + to use different methods than the ones described here as long as they + conform to the requirements specified in Sections 6.1.1 through + 6.1.9. See Section 7 for more details on error handling. 6.1.1. Originating a Request @@ -4285,33 +4195,35 @@ Internet-Draft Diameter Base Protocol January 2011 described in the application definition for that specific request, the following procedures MUST be followed: - o the Command-Code is set to the appropriate value - o the 'R' bit is set - o the End-to-End Identifier is set to a locally unique value - o the Origin-Host and Origin-Realm AVPs MUST be set to the - appropriate values, used to identify the source of the message - o the Destination-Host and Destination-Realm AVPs MUST be set to the - appropriate values as described in Section 6.1. +Fajardo, et al. Standards Track [Page 75] + +RFC 6733 Diameter Base Protocol October 2012 -6.1.2. Sending a Request - When sending a request, originated either locally, or as the result - of a forwarding or routing operation, the following procedures SHOULD - be followed: + o the Command Code is set to the appropriate value; - o The Hop-by-Hop Identifier SHOULD be set to a locally unique value. + o the 'R' bit is set; + o the End-to-End Identifier is set to a locally unique value; + o the Origin-Host and Origin-Realm AVPs MUST be set to the + appropriate values, used to identify the source of the message; + and + o the Destination-Host and Destination-Realm AVPs MUST be set to the + appropriate values, as described in Section 6.1. -Fajardo, et al. Expires July 24, 2011 [Page 77] - -Internet-Draft Diameter Base Protocol January 2011 +6.1.2. Sending a Request + When sending a request, originated either locally or as the result of + a forwarding or routing operation, the following procedures SHOULD be + followed: + + o The Hop-by-Hop Identifier SHOULD be set to a locally unique value. o The message SHOULD be saved in the list of pending requests. @@ -4328,25 +4240,34 @@ Internet-Draft Diameter Base Protocol January 2011 6.1.4. Processing Local Requests A request is known to be for local consumption when one of the - following conditions occur: + following conditions occurs: - o The Destination-Host AVP contains the local host's identity, + o The Destination-Host AVP contains the local host's identity; o The Destination-Host AVP is not present, the Destination-Realm AVP contains a realm the server is configured to process locally, and - the Diameter application is locally supported, or + the Diameter application is locally supported; or o Both the Destination-Host and the Destination-Realm are not present. + + + + +Fajardo, et al. Standards Track [Page 76] + +RFC 6733 Diameter Base Protocol October 2012 + + When a request is locally processed, the rules in Section 6.2 should be used to generate the corresponding answer. 6.1.5. Request Forwarding - Request forwarding is done using the Diameter Peer Table. The - Diameter peer table contains all of the peers that the local node is - able to directly communicate with. + Request forwarding is done using the Diameter peer table. The + Diameter peer table contains all of the peers with which the local + node is able to directly communicate. When a request is received, and the host encoded in the Destination- Host AVP is one that is present in the peer table, the message SHOULD @@ -4354,29 +4275,21 @@ Internet-Draft Diameter Base Protocol January 2011 6.1.6. Request Routing - Diameter request message routing is done via realms and application - identifiers. A Diameter message that may be forwarded by Diameter - agents (proxies, redirect or relay agents) MUST include the target + Diameter request message routing is done via realms and Application + Ids. A Diameter message that may be forwarded by Diameter agents + (proxies, redirect agents, or relay agents) MUST include the target realm in the Destination-Realm AVP. Request routing SHOULD rely on the Destination-Realm AVP and the Application Id present in the request message header to aid in the routing decision. The realm MAY be retrieved from the User-Name AVP, which is in the form of a - - - -Fajardo, et al. Expires July 24, 2011 [Page 78] - -Internet-Draft Diameter Base Protocol January 2011 - - Network Access Identifier (NAI). The realm portion of the NAI is inserted in the Destination-Realm AVP. Diameter agents MAY have a list of locally supported realms and - applications, and MAY have a list of externally supported realms and - applications. When a request is received that includes a realm + applications, and they MAY have a list of externally supported realms + and applications. When a request is received that includes a realm and/or application that is not locally supported, the message is - routed to the peer configured in the Routing Table (see Section 2.7). + routed to the peer configured in the routing table (see Section 2.7). Realm names and Application Ids are the minimum supported routing criteria, additional information may be needed to support redirect @@ -4385,13 +4298,23 @@ Internet-Draft Diameter Base Protocol January 2011 6.1.7. Predictive Loop Avoidance Before forwarding or routing a request, Diameter agents, in addition - to processing done in Section 6.1.3, SHOULD check for the presence of - candidate route's peer identity in any of the Route-Record AVPs. In - an event of the agent detecting the presence of a candidate route's - peer identity in a Route-Record AVP, the agent MUST ignore such route - for the Diameter request message and attempt alternate routes if any. - In case all the candidate routes are eliminated by the above - criteria, the agent SHOULD return DIAMETER_UNABLE_TO_DELIVER message. + to performing the processing described in Section 6.1.3, SHOULD check + for the presence of a candidate route's peer identity in any of the + Route-Record AVPs. In the event of the agent detecting the presence + of a candidate route's peer identity in a Route-Record AVP, the agent + MUST ignore such a route for the Diameter request message and attempt + alternate routes if any exist. In case all the candidate routes are + eliminated by the above criteria, the agent SHOULD return a + DIAMETER_UNABLE_TO_DELIVER message. + + + + + +Fajardo, et al. Standards Track [Page 77] + +RFC 6733 Diameter Base Protocol October 2012 + 6.1.8. Redirecting Requests @@ -4399,7 +4322,7 @@ Internet-Draft Diameter Base Protocol January 2011 to REDIRECT, it MUST reply with an answer message with the 'E' bit set, while maintaining the Hop-by-Hop Identifier in the header, and include the Result-Code AVP to DIAMETER_REDIRECT_INDICATION. Each of - the servers associated with the routing entry are added in separate + the servers associated with the routing entry are added in a separate Redirect-Host AVP. +------------------+ @@ -4417,21 +4340,13 @@ Internet-Draft Diameter Base Protocol January 2011 | Agent |<-------------| Server | +-------------+ 4. Answer +-------------+ - - - -Fajardo, et al. Expires July 24, 2011 [Page 79] - -Internet-Draft Diameter Base Protocol January 2011 - - Figure 5: Diameter Redirect Agent - The receiver of the answer message with the 'E' bit set, and the - Result-Code AVP set to DIAMETER_REDIRECT_INDICATION uses the hop-by- - hop field in the Diameter header to identify the request in the - pending message queue (see Section 5.3) that is to be redirected. If - no transport connection exists with the new agent, one is created, + The receiver of an answer message with the 'E' bit set and the + Result-Code AVP set to DIAMETER_REDIRECT_INDICATION uses the Hop-by- + Hop Identifier in the Diameter header to identify the request in the + pending message queue (see Section 5.5.4) that is to be redirected. + If no transport connection exists with the new peer, one is created, and the request is sent directly to it. Multiple Redirect-Host AVPs are allowed. The receiver of the answer @@ -4440,17 +4355,26 @@ Internet-Draft Diameter Base Protocol January 2011 When the Redirect-Host-Usage AVP included in the answer message has a non-zero value, a route entry for the redirect indications is created - and cached by the receiver. The redirect usage for such route entry - is set by the value of Redirect-Host-Usage AVP and the lifetime of - the cached route entry is set by Redirect-Max-Cache-Time AVP value. + and cached by the receiver. The redirect usage for such a route + entry is set by the value of Redirect-Host-Usage AVP and the lifetime + of the cached route entry is set by Redirect-Max-Cache-Time AVP + value. It is possible that multiple redirect indications can create multiple cached route entries differing only in their redirect usage and the peer to forward messages to. As an example, two(2) route entries that are created by two(2) redirect indications results in two(2) + + + +Fajardo, et al. Standards Track [Page 78] + +RFC 6733 Diameter Base Protocol October 2012 + + cached routes for the same realm and Application Id. However, one - has a redirect usage of ALL_SESSION where matching request will be - forwarded to one peer and the other has a redirect usage of ALL_REALM + has a redirect usage of ALL_SESSION, where matching requests will be + forwarded to one peer; the other has a redirect usage of ALL_REALM, where request are forwarded to another peer. Therefore, an incoming request that matches the realm and Application Id of both routes will need additional resolution. In such a case, a routing precedence @@ -4460,32 +4384,24 @@ Internet-Draft Diameter Base Protocol January 2011 6.1.9. Relaying and Proxying Requests A relay or proxy agent MUST append a Route-Record AVP to all requests - forwarded. The AVP contains the identity of the peer the request was - received from. + forwarded. The AVP contains the identity of the peer from which the + request was received. - The Hop-by-Hop identifier in the request is saved, and replaced with - a locally unique value. The source of the request is also saved, - which includes the IP address, port and protocol. + The Hop-by-Hop Identifier in the request is saved and replaced with a + locally unique value. The source of the request is also saved, which + includes the IP address, port, and protocol. A relay or proxy agent MAY include the Proxy-Info AVP in requests if it requires access to any local state information when the corresponding response is received. The Proxy-Info AVP has security - implications as state information is distribute to other entities. - As such, it is RECOMMMENDED to protect the content of the Proxy-Info - AVP with cryptographic mechanisms, for example by using a keyed - - - -Fajardo, et al. Expires July 24, 2011 [Page 80] - -Internet-Draft Diameter Base Protocol January 2011 - - - message digest. Such a mechanism, however, requires the management - of keys, although only locally at the Diameter server. Still, a full - description of the management of the keys used to protect the Proxy- - Info AVP is beyond the scope of this document. Below is a list of - commonly recommended: + implications as state information is distributed to other entities. + As such, it is RECOMMENDED that the content of the Proxy-Info AVP be + protected with cryptographic mechanisms, for example, by using a + keyed message digest such as HMAC-SHA1 [RFC2104]. Such a mechanism, + however, requires the management of keys, although only locally at + the Diameter server. Still, a full description of the management of + the keys used to protect the Proxy-Info AVP is beyond the scope of + this document. Below is a list of common recommendations: o The keys should be generated securely following the randomness recommendations in [RFC4086]. @@ -4494,32 +4410,39 @@ Internet-Draft Diameter Base Protocol January 2011 least 128 bits in strength. o The keys should not be used for any other purpose than generating - and verifying tickets. + and verifying instances of the Proxy-Info AVP. o The keys should be changed regularly. - o The keys should be changed if the ticket format or cryptographic + o The keys should be changed if the AVP format or cryptographic protection algorithms change. The message is then forwarded to the next hop, as identified in the - Routing Table. + routing table. + + + + +Fajardo, et al. Standards Track [Page 79] + +RFC 6733 Diameter Base Protocol October 2012 + Figure 6 provides an example of message routing using the procedures listed in these sections. - (Origin-Host=nas.example.net) (Origin-Host=nas.example.net) - (Origin-Realm=example.net) (Origin-Realm=example.net) - (Destination-Realm=example.com) (Destination- - Realm=example.com) + (Origin-Host=nas.example.net) (Origin-Host=nas.example.net) + (Origin-Realm=example.net) (Origin-Realm=example.net) + (Destination-Realm=example.com) (Destination-Realm=example.com) (Route-Record=nas.example.net) - +------+ ------> +------+ ------> +------+ - | | (Request) | | (Request) | | - | NAS +-------------------+ DRL +-------------------+ HMS | - | | | | | | - +------+ <------ +------+ <------ +------+ - example.net (Answer) example.net (Answer) example.com - (Origin-Host=hms.example.com) (Origin-Host=hms.example.com) - (Origin-Realm=example.com) (Origin-Realm=example.com) + +------+ ------> +------+ ------> +------+ + | | (Request) | | (Request) | | + | NAS +-------------------+ DRL +-------------------+ HMS | + | | | | | | + +------+ <------ +------+ <------ +------+ + example.net (Answer) example.net (Answer) example.com + (Origin-Host=hms.example.com) (Origin-Host=hms.example.com) + (Origin-Realm=example.com) (Origin-Realm=example.com) Figure 6: Routing of Diameter messages @@ -4527,15 +4450,7 @@ Internet-Draft Diameter Base Protocol January 2011 incoming messages. At a minimum, validation of the message header and relevant routing AVPs has to be done when relaying messages. Proxy agents may optionally perform more in-depth message validation - for applications it is interested in. - - - - -Fajardo, et al. Expires July 24, 2011 [Page 81] - -Internet-Draft Diameter Base Protocol January 2011 - + for applications in which it is interested. 6.2. Diameter Answer Processing @@ -4544,7 +4459,7 @@ Internet-Draft Diameter Base Protocol January 2011 additional procedures that MAY be discussed in the Diameter application defining the command: - o The same Hop-by-Hop identifier in the request is used in the + o The same Hop-by-Hop Identifier in the request is used in the answer. o The local host's identity is encoded in the Origin-Host AVP. @@ -4561,15 +4476,23 @@ Internet-Draft Diameter Base Protocol January 2011 o Any Proxy-Info AVPs in the request MUST be added to the answer message, in the same order they were present in the request. + + + +Fajardo, et al. Standards Track [Page 80] + +RFC 6733 Diameter Base Protocol October 2012 + + o The 'P' bit is set to the same value as the one in the request. o The same End-to-End identifier in the request is used in the answer. - Note that the error messages (see Section 7.3) are also subjected to + Note that the error messages (see Section 7) are also subjected to the above processing rules. -6.2.1. Processing received Answers +6.2.1. Processing Received Answers A Diameter client or proxy MUST match the Hop-by-Hop Identifier in an answer received against the list of pending requests. The @@ -4579,31 +4502,23 @@ Internet-Draft Diameter Base Protocol January 2011 6.2.2. Relaying and Proxying Answers - If the answer is for a request which was proxied or relayed, the - agent MUST restore the original value of the Diameter header's Hop- - by-Hop Identifier field. + If the answer is for a request that was proxied or relayed, the agent + MUST restore the original value of the Diameter header's Hop-by-Hop + Identifier field. If the last Proxy-Info AVP in the message is targeted to the local Diameter server, the AVP MUST be removed before the answer is - - - -Fajardo, et al. Expires July 24, 2011 [Page 82] - -Internet-Draft Diameter Base Protocol January 2011 - - forwarded. If a relay or proxy agent receives an answer with a Result-Code AVP indicating a failure, it MUST NOT modify the contents of the AVP. - Any additional local errors detected SHOULD be logged, but not + Any additional local errors detected SHOULD be logged but not reflected in the Result-Code AVP. If the agent receives an answer message with a Result-Code AVP indicating success, and it wishes to modify the AVP to indicate an error, it MUST modify the Result-Code AVP to contain the appropriate error in the message destined towards - the access device as well as include the Error-Reporting-Host AVP and - it MUST issue an STR on behalf of the access device towards the + the access device as well as include the Error-Reporting-Host AVP; it + MUST also issue an STR on behalf of the access device towards the Diameter server. The agent MUST then send the answer to the host that it received the @@ -4612,10 +4527,19 @@ Internet-Draft Diameter Base Protocol January 2011 6.3. Origin-Host AVP The Origin-Host AVP (AVP Code 264) is of type DiameterIdentity, and - MUST be present in all Diameter messages. This AVP identifies the + it MUST be present in all Diameter messages. This AVP identifies the endpoint that originated the Diameter message. Relay agents MUST NOT modify this AVP. + + + + +Fajardo, et al. Standards Track [Page 81] + +RFC 6733 Diameter Base Protocol October 2012 + + The value of the Origin-Host AVP is guaranteed to be unique within a single host. @@ -4638,17 +4562,9 @@ Internet-Draft Diameter Base Protocol January 2011 The Destination-Host AVP (AVP Code 293) is of type DiameterIdentity. This AVP MUST be present in all unsolicited agent initiated messages, - MAY be present in request messages, and MUST NOT be present in Answer + MAY be present in request messages, and MUST NOT be present in answer messages. - - - -Fajardo, et al. Expires July 24, 2011 [Page 83] - -Internet-Draft Diameter Base Protocol January 2011 - - The absence of the Destination-Host AVP will cause a message to be sent to any Diameter server supporting the application within the realm specified in Destination-Realm AVP. @@ -4658,20 +4574,28 @@ Internet-Draft Diameter Base Protocol January 2011 6.6. Destination-Realm AVP - The Destination-Realm AVP (AVP Code 283) is of type DiameterIdentity, - and contains the realm the message is to be routed to. The - Destination-Realm AVP MUST NOT be present in Answer messages. - Diameter Clients insert the realm portion of the User-Name AVP. + The Destination-Realm AVP (AVP Code 283) is of type DiameterIdentity + and contains the realm to which the message is to be routed. The + Destination-Realm AVP MUST NOT be present in answer messages. + Diameter clients insert the realm portion of the User-Name AVP. Diameter servers initiating a request message use the value of the Origin-Realm AVP from a previous message received from the intended target host (unless it is known a priori). When present, the Destination-Realm AVP is used to perform message routing decisions. - An ABNF for a request message that includes the Destination-Realm AVP + The CCF for a request message that includes the Destination-Realm AVP SHOULD list the Destination-Realm AVP as a required AVP (an AVP - indicated as {AVP}) otherwise the message is inherently a non- + indicated as {AVP}); otherwise, the message is inherently a non- routable message. + + + +Fajardo, et al. Standards Track [Page 82] + +RFC 6733 Diameter Base Protocol October 2012 + + This AVP SHOULD be placed as close to the Diameter header as possible. @@ -4692,19 +4616,11 @@ Internet-Draft Diameter Base Protocol January 2011 The Proxy-Info AVP (AVP Code 284) is of type Grouped. This AVP contains the identity and local state information of the Diameter node that creates and adds it to a message. The Grouped Data field - has the following ABNF grammar: + has the following CCF grammar: Proxy-Info ::= < AVP Header: 284 > { Proxy-Host } { Proxy-State } - - - -Fajardo, et al. Expires July 24, 2011 [Page 84] - -Internet-Draft Diameter Base Protocol January 2011 - - * [ AVP ] 6.7.3. Proxy-Host AVP @@ -4728,10 +4644,18 @@ Internet-Draft Diameter Base Protocol January 2011 Application-Id AVP MUST match the Application Id present in the Diameter message header. + + + +Fajardo, et al. Standards Track [Page 83] + +RFC 6733 Diameter Base Protocol October 2012 + + 6.9. Acct-Application-Id AVP The Acct-Application-Id AVP (AVP Code 259) is of type Unsigned32 and - is used in order to advertise support of the Accounting portion of an + is used in order to advertise support of the accounting portion of an application (see Section 2.4). If present in a message other than CER and CEA, the value of the Acct-Application-Id AVP MUST match the Application Id present in the Diameter message header. @@ -4740,10 +4664,10 @@ Internet-Draft Diameter Base Protocol January 2011 The Inband-Security-Id AVP (AVP Code 299) is of type Unsigned32 and is used in order to advertise support of the security portion of the - application. The use of this AVP in CER and CEA messages is no - longer recommended. Instead, discovery of a Diameter entities - security capabilities can be done either through static configuration - or via Diameter Peer Discovery described in Section 5.2. + application. The use of this AVP in CER and CEA messages is NOT + RECOMMENDED. Instead, discovery of a Diameter entity's security + capabilities can be done either through static configuration or via + Diameter Peer Discovery as described in Section 5.2. The following values are supported: @@ -4753,36 +4677,37 @@ Internet-Draft Diameter Base Protocol January 2011 This peer does not support TLS/TCP and DTLS/SCTP. This is the default value, if the AVP is omitted. - - - -Fajardo, et al. Expires July 24, 2011 [Page 85] - -Internet-Draft Diameter Base Protocol January 2011 - - TLS 1 - This node supports TLS/TCP and DTLS/SCTP security, as defined by - [RFC5246]. + This node supports TLS/TCP [RFC5246] and DTLS/SCTP [RFC6083] + security. 6.11. Vendor-Specific-Application-Id AVP The Vendor-Specific-Application-Id AVP (AVP Code 260) is of type Grouped and is used to advertise support of a vendor-specific - Diameter Application. Exactly one instance of either Auth- + Diameter application. Exactly one instance of either Auth- Application-Id or Acct-Application-Id AVP MUST be present. The Application Id carried by either Auth-Application-Id or Acct- - Application-Id AVP MUST comply with vendor specific Application Id - assignment described in Sec 11.3. It MUST also match the Application - Id present in the Diameter header except when used in a CER or CEA - message. + Application-Id AVP MUST comply with vendor-specific Application Id + assignment described in Section 11.3. It MUST also match the + Application Id present in the Diameter header except when used in a + CER or CEA message. The Vendor-Id AVP is an informational AVP pertaining to the vendor who may have authorship of the vendor-specific Diameter application. It MUST NOT be used as a means of defining a completely separate vendor-specific Application Id space. + + + + +Fajardo, et al. Standards Track [Page 84] + +RFC 6733 Diameter Base Protocol October 2012 + + The Vendor-Specific-Application-Id AVP SHOULD be placed as close to the Diameter header as possible. @@ -4795,9 +4720,9 @@ Internet-Draft Diameter Base Protocol January 2011 A Vendor-Specific-Application-Id AVP MUST contain exactly one of either Auth-Application-Id or Acct-Application-Id. If a Vendor- - Specific-Application-Id is received without any of these two AVPs, + Specific-Application-Id is received without one of these two AVPs, then the recipient SHOULD issue an answer with a Result-Code set to - DIAMETER_MISSING_AVP. The answer SHOULD also include a Failed-AVP + DIAMETER_MISSING_AVP. The answer SHOULD also include a Failed-AVP, which MUST contain an example of an Auth-Application-Id AVP and an Acct-Application-Id AVP. @@ -4805,22 +4730,13 @@ Internet-Draft Diameter Base Protocol January 2011 Auth-Application-Id and Acct-Application-Id, then the recipient MUST issue an answer with Result-Code set to DIAMETER_AVP_OCCURS_TOO_MANY_TIMES. The answer MUST also include a - Failed-AVP which MUST contain the received Auth-Application-Id AVP + Failed-AVP, which MUST contain the received Auth-Application-Id AVP and Acct-Application-Id AVP. - - - - -Fajardo, et al. Expires July 24, 2011 [Page 86] - -Internet-Draft Diameter Base Protocol January 2011 - - 6.12. Redirect-Host AVP The Redirect-Host AVP (AVP Code 292) is of type DiameterURI. One or - more of instances of this AVP MUST be present if the answer message's + more instances of this AVP MUST be present if the answer message's 'E' bit is set and the Result-Code AVP is set to DIAMETER_REDIRECT_INDICATION. @@ -4836,63 +4752,55 @@ Internet-Draft Diameter Base Protocol January 2011 This AVP MAY be present in answer messages whose 'E' bit is set and the Result-Code AVP is set to DIAMETER_REDIRECT_INDICATION. - When present, this AVP provides a hints about how the routing entry + When present, this AVP provides hints about how the routing entry resulting from the Redirect-Host is to be used. The following values are supported: + + +Fajardo, et al. Standards Track [Page 85] + +RFC 6733 Diameter Base Protocol October 2012 + + DONT_CACHE 0 The host specified in the Redirect-Host AVP SHOULD NOT be cached. This is the default value. - ALL_SESSION 1 All messages within the same session, as defined by the same value of the Session-ID AVP SHOULD be sent to the host specified in the Redirect-Host AVP. - ALL_REALM 2 All messages destined for the realm requested SHOULD be sent to the host specified in the Redirect-Host AVP. - REALM_AND_APPLICATION 3 All messages for the application requested to the realm specified SHOULD be sent to the host specified in the Redirect-Host AVP. - - - -Fajardo, et al. Expires July 24, 2011 [Page 87] - -Internet-Draft Diameter Base Protocol January 2011 - - ALL_APPLICATION 4 All messages for the application requested SHOULD be sent to the host specified in the Redirect-Host AVP. - ALL_HOST 5 All messages that would be sent to the host that generated the Redirect-Host SHOULD be sent to the host specified in the - Redirect- Host AVP. - + Redirect-Host AVP. ALL_USER 6 All messages for the user requested SHOULD be sent to the host specified in the Redirect-Host AVP. - - When multiple cached routes are created by redirect indications and they differ only in redirect usage and peers to forward requests to (see Section 6.1.8), a precedence rule MUST be applied to the @@ -4902,6 +4810,16 @@ Internet-Draft Diameter Base Protocol January 2011 other as they appear. The order is as follows: + + + + + +Fajardo, et al. Standards Track [Page 86] + +RFC 6733 Diameter Base Protocol October 2012 + + 1. ALL_SESSION 2. ALL_USER @@ -4917,58 +4835,31 @@ Internet-Draft Diameter Base Protocol January 2011 6.14. Redirect-Max-Cache-Time AVP The Redirect-Max-Cache-Time AVP (AVP Code 262) is of type Unsigned32. - This AVP MUST be present in answer messages whose 'E' bit is set, the - Result-Code AVP is set to DIAMETER_REDIRECT_INDICATION and the - Redirect-Host-Usage AVP set to a non-zero value. - - - - -Fajardo, et al. Expires July 24, 2011 [Page 88] - -Internet-Draft Diameter Base Protocol January 2011 - + This AVP MUST be present in answer messages whose 'E' bit is set, + whose Result-Code AVP is set to DIAMETER_REDIRECT_INDICATION, and + whose Redirect-Host-Usage AVP set to a non-zero value. This AVP contains the maximum number of seconds the peer and route table entries, created as a result of the Redirect-Host, SHOULD be cached. Note that once a host is no longer reachable, any associated - cache, peer and routing table entries MUST be deleted. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + cache, peer, and routing table entries MUST be deleted. +7. Error Handling + There are two different types of errors in Diameter; protocol errors + and application errors. A protocol error is one that occurs at the + base protocol level and MAY require per-hop attention (e.g., a + message routing error). Application errors, on the other hand, + generally occur due to a problem with a function specified in a + Diameter application (e.g., user authentication, missing AVP). + Result-Code AVP values that are used to report protocol errors MUST + only be present in answer messages whose 'E' bit is set. When a + request message is received that causes a protocol error, an answer + message is returned with the 'E' bit set, and the Result-Code AVP is + set to the appropriate protocol error value. As the answer is sent + back towards the originator of the request, each proxy or relay agent + MAY take action on the message. @@ -4980,27 +4871,10 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 89] +Fajardo, et al. Standards Track [Page 87] -Internet-Draft Diameter Base Protocol January 2011 - - -7. Error Handling +RFC 6733 Diameter Base Protocol October 2012 - There are two different types of errors in Diameter; protocol and - application errors. A protocol error is one that occurs at the base - protocol level, and MAY require per hop attention (e.g., message - routing error). Application errors, on the other hand, generally - occur due to a problem with a function specified in a Diameter - application (e.g., user authentication, missing AVP). - - Result-Code AVP values that are used to report protocol errors MUST - only be present in answer messages whose 'E' bit is set. When a - request message is received that causes a protocol error, an answer - message is returned with the 'E' bit set, and the Result-Code AVP is - set to the appropriate protocol error value. As the answer is sent - back towards the originator of the request, each proxy or relay agent - MAY take action on the message. 1. Request +---------+ Link Broken +-------------------------->|Diameter |----///----+ @@ -5014,7 +4888,7 @@ Internet-Draft Diameter Base Protocol January 2011 | Relay 3 |-----------+ +---------+ - Figure 7: Example of Protocol Error causing answer message + Figure 7: Example of Protocol Error Causing Answer Message Figure 7 provides an example of a message forwarded upstream by a Diameter relay. When the message is received by Relay 2, and it @@ -5032,40 +4906,40 @@ Internet-Draft Diameter Base Protocol January 2011 +---------+ 4. Answer +---------+ 3. Answer +---------+ (Missing AVP) (Missing AVP) - Figure 8: Example of Application Error Answer message - - - -Fajardo, et al. Expires July 24, 2011 [Page 90] - -Internet-Draft Diameter Base Protocol January 2011 - + Figure 8: Example of Application Error Answer Message Figure 8 provides an example of a Diameter message that caused an application error. When application errors occur, the Diameter - entity reporting the error clears the 'R' bit in the Command Flags, + entity reporting the error clears the 'R' bit in the Command Flags and adds the Result-Code AVP with the proper value. Application - errors do not require any proxy or relay agent involvement, and - therefore the message would be forwarded back to the originator of + errors do not require any proxy or relay agent involvement; + therefore, the message would be forwarded back to the originator of the request. In the case where the answer message itself contains errors, any related session SHOULD be terminated by sending an STR or ASR message. The Termination-Cause AVP in the STR MAY be filled with the appropriate value to indicate the cause of the error. An application - MAY also send an application-specific request instead of STR or ASR - to signal the error in the case where no state is maintained or to - allow for some form of error recovery with the corresponding Diameter - entity. + MAY also send an application-specific request instead of an STR or + ASR message to signal the error in the case where no state is + maintained or to allow for some form of error recovery with the + corresponding Diameter entity. + + + +Fajardo, et al. Standards Track [Page 88] + +RFC 6733 Diameter Base Protocol October 2012 + There are certain Result-Code AVP application errors that require additional AVPs to be present in the answer. In these cases, the Diameter node that sets the Result-Code AVP to indicate the error - MUST add the AVPs. Examples are: + MUST add the AVPs. Examples are as follows: o A request with an unrecognized AVP is received with the 'M' bit - (Mandatory bit) set, causes an answer to be sent with the Result- - Code AVP set to DIAMETER_AVP_UNSUPPORTED, and the Failed-AVP AVP + (Mandatory bit) set causes an answer to be sent with the Result- + Code AVP set to DIAMETER_AVP_UNSUPPORTED and the Failed-AVP AVP containing the offending AVP. o A request with an AVP that is received with an unrecognized value @@ -5073,46 +4947,46 @@ Internet-Draft Diameter Base Protocol January 2011 DIAMETER_INVALID_AVP_VALUE, with the Failed-AVP AVP containing the AVP causing the error. - o A received command which is missing AVP(s) that are defined as - required in the commands ABNF; examples are AVPs indicated as + o A received command that is missing AVPs that are defined as + required in the commands CCF; examples are AVPs indicated as {AVP}. The receiver issues an answer with the Result-Code set to - DIAMETER_MISSING_AVP, and creates an AVP with the AVP Code and + DIAMETER_MISSING_AVP and creates an AVP with the AVP Code and other fields set as expected in the missing AVP. The created AVP - is then added to the Failed- AVP AVP. + is then added to the Failed-AVP AVP. The Result-Code AVP describes the error that the Diameter node encountered in its processing. In case there are multiple errors, the Diameter node MUST report only the first error it encountered - (detected possibly in some implementation dependent order). The + (detected possibly in some implementation-dependent order). The specific errors that can be described by this AVP are described in the following section. +7.1. Result-Code AVP + The Result-Code AVP (AVP Code 268) is of type Unsigned32 and + indicates whether a particular request was completed successfully or + an error occurred. All Diameter answer messages in IETF-defined + Diameter application specifications MUST include one Result-Code AVP. + A non-successful Result-Code AVP (one containing a non-2xxx value + other than DIAMETER_REDIRECT_INDICATION) MUST include the Error- + Reporting-Host AVP if the host setting the Result-Code AVP is + different from the identity encoded in the Origin-Host AVP. + The Result-Code data field contains an IANA-managed 32-bit address + space representing errors (see Section 11.3.2). Diameter provides + the following classes of errors, all identified by the thousands + digit in the decimal notation: -Fajardo, et al. Expires July 24, 2011 [Page 91] - -Internet-Draft Diameter Base Protocol January 2011 -7.1. Result-Code AVP - The Result-Code AVP (AVP Code 268) is of type Unsigned32 and - indicates whether a particular request was completed successfully or - whether an error occurred. All Diameter answer messages in IETF - defined Diameter application specification MUST include one Result- - Code AVP. A non-successful Result-Code AVP (one containing a non - 2xxx value other than DIAMETER_REDIRECT_INDICATION) MUST include the - Error-Reporting-Host AVP if the host setting the Result-Code AVP is - different from the identity encoded in the Origin-Host AVP. +Fajardo, et al. Standards Track [Page 89] + +RFC 6733 Diameter Base Protocol October 2012 - The Result-Code data field contains an IANA-managed 32-bit address - space representing errors (see Section 11.4). Diameter provides the - following classes of errors, all identified by the thousands digit in - the decimal notation: o 1xxx (Informational) @@ -5124,7 +4998,7 @@ Internet-Draft Diameter Base Protocol January 2011 o 5xxx (Permanent Failure) - A non-recognized class (one whose first digit is not defined in this + An unrecognized class (one whose first digit is not defined in this section) MUST be handled as a permanent failure. 7.1.1. Informational @@ -5133,7 +5007,6 @@ Internet-Draft Diameter Base Protocol January 2011 requester that a request could not be satisfied, and additional action is required on its part before access is granted. - DIAMETER_MULTI_ROUND_AUTH 1001 This informational error is returned by a Diameter server to @@ -5141,24 +5014,11 @@ Internet-Draft Diameter Base Protocol January 2011 used requires multiple round trips, and a subsequent request needs to be issued in order for access to be granted. - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 92] - -Internet-Draft Diameter Base Protocol January 2011 - - 7.1.2. Success Errors that fall within the Success category are used to inform a peer that a request has been successfully completed. - DIAMETER_SUCCESS 2001 The request was successfully completed. @@ -5174,27 +5034,33 @@ Internet-Draft Diameter Base Protocol January 2011 Errors that fall within the Protocol Error category SHOULD be treated on a per-hop basis, and Diameter proxies MAY attempt to correct the error, if it is possible. Note that these errors MUST only be used - in answer messages whose 'E' bit is set. This document omits some - error codes defined in [RFC3588]. To provide backward compatibility - with [RFC3588] implementations these error code values are not re- - used and hence the error codes values enumerated below are non- - sequential. + in answer messages whose 'E' bit is set. + + +Fajardo, et al. Standards Track [Page 90] + +RFC 6733 Diameter Base Protocol October 2012 + + + DIAMETER_COMMAND_UNSUPPORTED 3001 + + This error code is used when a Diameter entity receives a message + with a Command Code that it does not support. + DIAMETER_UNABLE_TO_DELIVER 3002 - This error is given when Diameter can not deliver the message to + This error is given when Diameter cannot deliver the message to the destination, either because no host within the realm supporting the required application was available to process the - request, or because Destination-Host AVP was given without the + request or because the Destination-Host AVP was given without the associated Destination-Realm AVP. - DIAMETER_REALM_NOT_SERVED 3003 The intended realm of the request is not recognized. - DIAMETER_TOO_BUSY 3004 When returned, a Diameter node SHOULD attempt to send the message @@ -5202,13 +5068,6 @@ Internet-Draft Diameter Base Protocol January 2011 specific server is requested, and it cannot provide the requested service. - - -Fajardo, et al. Expires July 24, 2011 [Page 93] - -Internet-Draft Diameter Base Protocol January 2011 - - DIAMETER_LOOP_DETECTED 3005 An agent detected a loop while trying to get the message to the @@ -5216,121 +5075,110 @@ Internet-Draft Diameter Base Protocol January 2011 if one is available, but the peer reporting the error has identified a configuration problem. - DIAMETER_REDIRECT_INDICATION 3006 A redirect agent has determined that the request could not be - satisfied locally and the initiator of the request SHOULD direct + satisfied locally, and the initiator of the request SHOULD direct the request directly to the server, whose contact information has been added to the response. When set, the Redirect-Host AVP MUST be present. - DIAMETER_APPLICATION_UNSUPPORTED 3007 A request was sent for an application that is not supported. + DIAMETER_INVALID_HDR_BITS 3008 - DIAMETER_INVALID_BIT_IN_HEADER 3011 + A request was received whose bits in the Diameter header were set + either to an invalid combination or to a value that is + inconsistent with the Command Code's definition. - This error is returned when a reserved bit in the Diameter header - is set to one (1) or the bits in the Diameter header defined in - Section 3 are set incorrectly. - DIAMETER_INVALID_MESSAGE_LENGTH 3012 +Fajardo, et al. Standards Track [Page 91] + +RFC 6733 Diameter Base Protocol October 2012 - This error is returned when a request is received with an invalid - message length. + DIAMETER_INVALID_AVP_BITS 3009 + + A request was received that included an AVP whose flag bits are + set to an unrecognized value or that is inconsistent with the + AVP's definition. + + DIAMETER_UNKNOWN_PEER 3010 + + A CER was received from an unknown peer. 7.1.4. Transient Failures Errors that fall within the transient failures category are used to inform a peer that the request could not be satisfied at the time it - was received, but MAY be able to satisfy the request in the future. + was received but MAY be able to satisfy the request in the future. Note that these errors MUST be used in answer messages whose 'E' bit is not set. - DIAMETER_AUTHENTICATION_REJECTED 4001 The authentication process for the user failed, most likely due to an invalid password used by the user. Further attempts MUST only - - - -Fajardo, et al. Expires July 24, 2011 [Page 94] - -Internet-Draft Diameter Base Protocol January 2011 - - be tried after prompting the user for a new password. - DIAMETER_OUT_OF_SPACE 4002 A Diameter node received the accounting request but was unable to commit it to stable storage due to a temporary lack of space. - ELECTION_LOST 4003 The peer has determined that it has lost the election process and has therefore disconnected the transport connection. - 7.1.5. Permanent Failures Errors that fall within the permanent failures category are used to - inform the peer that the request failed, and should not be attempted + inform the peer that the request failed and should not be attempted again. Note that these errors SHOULD be used in answer messages whose 'E' bit is not set. In error conditions where it is not - possible or efficient to compose application-specific answer grammar - then answer messages with E-bit set and complying to the grammar - described in 7.2 MAY also be used for permanent errors. + possible or efficient to compose application-specific answer grammar, + answer messages with the 'E' bit set and which comply to the grammar + described in Section 7.2 MAY also be used for permanent errors. - To provide backward compatibility with existing implementations that - follow [RFC3588], some of the error values that have previously been - used in this category by [RFC3588] will not be re-used. Therefore - the error values enumerated here may be non-sequential. - DIAMETER_AVP_UNSUPPORTED 5001 - The peer received a message that contained an AVP that is not - recognized or supported and was marked with the Mandatory bit. A - Diameter message with this error MUST contain one or more Failed- - AVP AVP containing the AVPs that caused the failure. - DIAMETER_UNKNOWN_SESSION_ID 5002 - The request contained an unknown Session-Id. +Fajardo, et al. Standards Track [Page 92] + +RFC 6733 Diameter Base Protocol October 2012 - DIAMETER_AUTHORIZATION_REJECTED 5003 - A request was received for which the user could not be authorized. - This error could occur if the service requested is not permitted + DIAMETER_AVP_UNSUPPORTED 5001 + The peer received a message that contained an AVP that is not + recognized or supported and was marked with the 'M' (Mandatory) + bit. A Diameter message with this error MUST contain one or more + Failed-AVP AVPs containing the AVPs that caused the failure. + DIAMETER_UNKNOWN_SESSION_ID 5002 -Fajardo, et al. Expires July 24, 2011 [Page 95] - -Internet-Draft Diameter Base Protocol January 2011 + The request contained an unknown Session-Id. + DIAMETER_AUTHORIZATION_REJECTED 5003 + A request was received for which the user could not be authorized. + This error could occur if the service requested is not permitted to the user. - DIAMETER_INVALID_AVP_VALUE 5004 The request contained an AVP with an invalid value in its data portion. A Diameter message indicating this error MUST include the offending AVPs within a Failed-AVP AVP. - DIAMETER_MISSING_AVP 5005 The request did not contain an AVP that is required by the Command @@ -5340,21 +5188,28 @@ Internet-Draft Diameter Base Protocol January 2011 Vendor-Id if applicable. The value field of the missing AVP should be of correct minimum length and contain zeroes. - DIAMETER_RESOURCES_EXCEEDED 5006 A request was received that cannot be authorized because the user has already expended allowed resources. An example of this error - condition is a user that is restricted to one dial-up PPP port, - attempts to establish a second PPP connection. - + condition is when a user that is restricted to one dial-up PPP + port attempts to establish a second PPP connection. DIAMETER_CONTRADICTING_AVPS 5007 The Home Diameter server has detected AVPs in the request that - contradicted each other, and is not willing to provide service to - the user. The Failed-AVP AVPs MUST be present which contains the - AVPs that contradicted each other. + contradicted each other, and it is not willing to provide service + to the user. The Failed-AVP AVP MUST be present, which contain + the AVPs that contradicted each other. + + + + + + +Fajardo, et al. Standards Track [Page 93] + +RFC 6733 Diameter Base Protocol October 2012 DIAMETER_AVP_NOT_ALLOWED 5008 @@ -5363,22 +5218,12 @@ Internet-Draft Diameter Base Protocol January 2011 Failed-AVP AVP MUST be included and contain a copy of the offending AVP. - DIAMETER_AVP_OCCURS_TOO_MANY_TIMES 5009 A message was received that included an AVP that appeared more often than permitted in the message definition. The Failed-AVP AVP MUST be included and contain a copy of the first instance of - - - -Fajardo, et al. Expires July 24, 2011 [Page 96] - -Internet-Draft Diameter Base Protocol January 2011 - - - the offending AVP that exceeded the maximum number of occurrences - + the offending AVP that exceeded the maximum number of occurrences. DIAMETER_NO_COMMON_APPLICATION 5010 @@ -5386,18 +5231,21 @@ Internet-Draft Diameter Base Protocol January 2011 whereby no applications are common between the CER sending peer and the CER receiving peer. - DIAMETER_UNSUPPORTED_VERSION 5011 This error is returned when a request was received, whose version number is unsupported. - DIAMETER_UNABLE_TO_COMPLY 5012 This error is returned when a request is rejected for unspecified reasons. + DIAMETER_INVALID_BIT_IN_HEADER 5013 + + This error is returned when a reserved bit in the Diameter header + is set to one (1) or the bits in the Diameter header are set + incorrectly. DIAMETER_INVALID_AVP_LENGTH 5014 @@ -5407,56 +5255,41 @@ Internet-Draft Diameter Base Protocol January 2011 value exceeds the message length or is less than the minimum AVP header length, it is sufficient to include the offending AVP header and a zero filled payload of the minimum required length - for the payloads data type. If the AVP is a grouped AVP, the - grouped AVP header with an empty payload would be sufficient to + for the payloads data type. If the AVP is a Grouped AVP, the + Grouped AVP header with an empty payload would be sufficient to indicate the offending AVP. In the case where the offending AVP header cannot be fully decoded when the AVP length is less than - the minimum AVP header length, it is sufficient to include an - offending AVP header that is formulated by padding the incomplete - AVP header with zero up to the minimum AVP header length. - - - DIAMETER_NO_COMMON_SECURITY 5017 - - This error is returned when a CER message is received, and there - are no common security mechanisms supported between the peers. A - Capabilities-Exchange-Answer (CEA) MUST be returned with the - Result-Code AVP set to DIAMETER_NO_COMMON_SECURITY. - - -Fajardo, et al. Expires July 24, 2011 [Page 97] +Fajardo, et al. Standards Track [Page 94] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - DIAMETER_UNKNOWN_PEER 5018 - - A CER was received from an unknown peer. - - - DIAMETER_COMMAND_UNSUPPORTED 5019 - - This error code is used when a Diameter entity receives a message - with a Command Code that it does not support. - + the minimum AVP header length, it is sufficient to include an + offending AVP header that is formulated by padding the incomplete + AVP header with zero up to the minimum AVP header length. - DIAMETER_INVALID_HDR_BITS 5020 + DIAMETER_INVALID_MESSAGE_LENGTH 5015 - A request was received whose bits in the Diameter header were - either set to an invalid combination, or to a value that is - inconsistent with the command code's definition. + This error is returned when a request is received with an invalid + message length. + DIAMETER_INVALID_AVP_BIT_COMBO 5016 - DIAMETER_INVALID_AVP_BITS 5021 + The request contained an AVP with which is not allowed to have the + given value in the AVP Flags field. A Diameter message indicating + this error MUST include the offending AVPs within a Failed-AVP + AVP. - A request was received that included an AVP whose flag bits are - set to an unrecognized value, or that is inconsistent with the - AVP's definition. + DIAMETER_NO_COMMON_SECURITY 5017 + This error is returned when a CER message is received, and there + are no common security mechanisms supported between the peers. A + Capabilities-Exchange-Answer (CEA) message MUST be returned with + the Result-Code AVP set to DIAMETER_NO_COMMON_SECURITY. 7.2. Error Bit @@ -5465,12 +5298,12 @@ Internet-Draft Diameter Base Protocol January 2011 the 'E' bit MUST NOT be sent as a response to an answer message. Note that a message with the 'E' bit set is still subjected to the processing rules defined in Section 6.2. When set, the answer - message will not conform to the ABNF specification for the command, - and will instead conform to the following ABNF: + message will not conform to the CCF specification for the command; + instead, it and will conform to the following CCF: Message Format - <answer-message> ::= < Diameter Header: code, ERR [PXY] > + <answer-message> ::= < Diameter Header: code, ERR [, PXY] > 0*1< Session-Id > { Origin-Host } { Origin-Realm } @@ -5481,16 +5314,16 @@ Internet-Draft Diameter Base Protocol January 2011 [ Failed-AVP ] [ Experimental-Result ] * [ Proxy-Info ] + * [ AVP ] -Fajardo, et al. Expires July 24, 2011 [Page 98] + +Fajardo, et al. Standards Track [Page 95] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - * [ AVP ] - Note that the code used in the header is the same than the one found in the request message, but with the 'R' bit cleared and the 'E' bit set. The 'P' bit in the header is set to the same value as the one @@ -5499,10 +5332,10 @@ Internet-Draft Diameter Base Protocol January 2011 7.3. Error-Message AVP The Error-Message AVP (AVP Code 281) is of type UTF8String. It MAY - accompany a Result-Code AVP as a human readable error message. The + accompany a Result-Code AVP as a human-readable error message. The Error-Message AVP is not intended to be useful in an environment where error messages are processed automatically. It SHOULD NOT be - expected that the content of this AVP is parsed by network entities. + expected that the content of this AVP be parsed by network entities. 7.4. Error-Reporting-Host AVP @@ -5511,8 +5344,8 @@ Internet-Draft Diameter Base Protocol January 2011 host that sent the Result-Code AVP to a value other than 2001 (Success), only if the host setting the Result-Code is different from the one encoded in the Origin-Host AVP. This AVP is intended to be - used for troubleshooting purposes, and MUST be set when the Result- - Code AVP indicates a failure. + used for troubleshooting purposes, and it MUST be set when the + Result-Code AVP indicates a failure. 7.5. Failed-AVP AVP @@ -5520,43 +5353,44 @@ Internet-Draft Diameter Base Protocol January 2011 debugging information in cases where a request is rejected or not fully processed due to erroneous information in a specific AVP. The value of the Result-Code AVP will provide information on the reason - for the Failed-AVP AVP. A Diameter message SHOULD contain only one - Failed-AVP that corresponds to the error indicated by the Result-Code - AVP. For practical purposes, this Failed-AVP would typically refer - to the first AVP processing error that a Diameter node encounters. + for the Failed-AVP AVP. A Diameter answer message SHOULD contain an + instance of the Failed-AVP AVP that corresponds to the error + indicated by the Result-Code AVP. For practical purposes, this + Failed-AVP would typically refer to the first AVP processing error + that a Diameter node encounters. The possible reasons for this AVP are the presence of an improperly constructed AVP, an unsupported or unrecognized AVP, an invalid AVP value, the omission of a required AVP, the presence of an explicitly - excluded AVP (see tables in Section 10), or the presence of two or - more occurrences of an AVP which is restricted to 0, 1, or 0-1 + excluded AVP (see tables in Section 10) or the presence of two or + more occurrences of an AVP that is restricted to 0, 1, or 0-1 occurrences. A Diameter message SHOULD contain one Failed-AVP AVP, containing the entire AVP that could not be processed successfully. If the failure reason is omission of a required AVP, an AVP with the missing AVP - code, the missing vendor id, and a zero filled payload of the minimum + code, the missing Vendor-Id, and a zero-filled payload of the minimum required length for the omitted AVP will be added. If the failure + reason is an invalid AVP length where the reported length is less -Fajardo, et al. Expires July 24, 2011 [Page 99] +Fajardo, et al. Standards Track [Page 96] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - reason is an invalid AVP length where the reported length is less than the minimum AVP header length or greater than the reported - message length, a copy of the offending AVP header and a zero filled + message length, a copy of the offending AVP header and a zero-filled payload of the minimum required length SHOULD be added. - In the case where the offending AVP is embedded within a grouped AVP, - the Failed-AVP MAY contain the grouped AVP which in turn contains the - single offending AVP. The same method MAY be employed if the grouped - AVP itself is embedded in yet another grouped AVP and so on. In this - case, the Failed-AVP MAY contain the grouped AVP hierarchy up to the - single offending AVP. This enables the recipient to detect the - location of the offending AVP when embedded in a group. + In the case where the offending AVP is embedded within a Grouped AVP, + the Failed-AVP MAY contain the grouped AVP, which in turn contains + the single offending AVP. The same method MAY be employed if the + grouped AVP itself is embedded in yet another grouped AVP and so on. + In this case, the Failed-AVP MAY contain the grouped AVP hierarchy up + to the single offending AVP. This enables the recipient to detect + the location of the offending AVP when embedded in a group. AVP Format @@ -5577,7 +5411,7 @@ Internet-Draft Diameter Base Protocol January 2011 { Experimental-Result-Code } The Vendor-Id AVP (see Section 5.3.3) in this grouped AVP identifies - the vendor responsible for the assignment of the result code which + the vendor responsible for the assignment of the result code that follows. All Diameter answer messages defined in vendor-specific applications MUST include either one Result-Code AVP or one Experimental-Result AVP. @@ -5590,23 +5424,24 @@ Internet-Draft Diameter Base Protocol January 2011 It is recommended that vendor-specific result codes follow the same conventions given for the Result-Code AVP regarding the different - types of result codes and the handling of errors (for non 2xxx + types of result codes and the handling of errors (for non-2xxx values). -Fajardo, et al. Expires July 24, 2011 [Page 100] + +Fajardo, et al. Standards Track [Page 97] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 8. Diameter User Sessions In general, Diameter can provide two different types of services to applications. The first involves authentication and authorization, - and can optionally make use of accounting. The second only makes use - of accounting. + and it can optionally make use of accounting. The second only makes + use of accounting. When a service makes use of the authentication and/or authorization portion of an application, and a user requests access to the network, @@ -5614,27 +5449,28 @@ Internet-Draft Diameter Base Protocol January 2011 auth request is defined in a service-specific Diameter application (e.g., NASREQ). The request contains a Session-Id AVP, which is used in subsequent messages (e.g., subsequent authorization, accounting, - etc) relating to the user's session. The Session-Id AVP is a means + etc.) relating to the user's session. The Session-Id AVP is a means for the client and servers to correlate a Diameter message with a user session. - When a Diameter server authorizes a user to use network resources for - a finite amount of time, and it is willing to extend the - authorization via a future request, it MUST add the Authorization- - Lifetime AVP to the answer message. The Authorization-Lifetime AVP - defines the maximum number of seconds a user MAY make use of the - resources before another authorization request is expected by the - server. The Auth-Grace-Period AVP contains the number of seconds - following the expiration of the Authorization-Lifetime, after which - the server will release all state information related to the user's - session. Note that if payment for services is expected by the - serving realm from the user's home realm, the Authorization-Lifetime - AVP, combined with the Auth-Grace-Period AVP, implies the maximum - length of the session the home realm is willing to be fiscally - responsible for. Services provided past the expiration of the - Authorization-Lifetime and Auth-Grace-Period AVPs are the - responsibility of the access device. Of course, the actual cost of - services rendered is clearly outside the scope of the protocol. + When a Diameter server authorizes a user to implement network + resources for a finite amount of time, and it is willing to extend + the authorization via a future request, it MUST add the + Authorization- Lifetime AVP to the answer message. The + Authorization-Lifetime AVP defines the maximum number of seconds a + user MAY make use of the resources before another authorization + request is expected by the server. The Auth-Grace-Period AVP + contains the number of seconds following the expiration of the + Authorization-Lifetime, after which the server will release all state + information related to the user's session. Note that if payment for + services is expected by the serving realm from the user's home realm, + the Authorization-Lifetime AVP, combined with the Auth-Grace-Period + AVP, implies the maximum length of the session for which the home + realm is willing to be fiscally responsible. Services provided past + the expiration of the Authorization-Lifetime and Auth-Grace-Period + AVPs are the responsibility of the access device. Of course, the + actual cost of services rendered is clearly outside the scope of the + protocol. An access device that does not expect to send a re-authorization or a session termination request to the server MAY include the Auth- @@ -5648,32 +5484,33 @@ Internet-Draft Diameter Base Protocol January 2011 value NO_STATE_MAINTAINED MUST NOT be set in subsequent re- authorization requests and answers. - The base protocol does not include any authorization request -Fajardo, et al. Expires July 24, 2011 [Page 101] +Fajardo, et al. Standards Track [Page 98] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + The base protocol does not include any authorization request messages, since these are largely application-specific and are defined in a Diameter application document. However, the base protocol does define a set of messages that are used to terminate user sessions. These are used to allow servers that maintain state information to free resources. - When a service only makes use of the Accounting portion of the + When a service only makes use of the accounting portion of the Diameter protocol, even in combination with an application, the Session-Id is still used to identify user sessions. However, the session termination messages are not used, since a session is signaled as being terminated by issuing an accounting stop message. Diameter may also be used for services that cannot be easily - categorized as authentication, authorization or accounting (e.g., - certain 3GPP IMS interfaces). In such cases, the finite state + categorized as authentication, authorization, or accounting (e.g., + certain Third Generation Partnership Project Internet Multimedia + System (3GPP IMS) interfaces). In such cases, the finite state machine defined in subsequent sections may not be applicable. - Therefore, the applications itself MAY need to define its own finite + Therefore, the application itself MAY need to define its own finite state machine. However, such application-specific state machines SHOULD follow the general state machine framework outlined in this document such as the use of Session-Id AVPs and the use of STR/STA, @@ -5681,12 +5518,12 @@ Internet-Draft Diameter Base Protocol January 2011 8.1. Authorization Session State Machine - This section contains a set of finite state machines, representing - the life cycle of Diameter sessions, and which MUST be observed by - all Diameter implementations that make use of the authentication - and/or authorization portion of a Diameter application. The term - Service-Specific below refers to a message defined in a Diameter - application (e.g., Mobile IPv4, NASREQ). + This section contains a set of finite state machines, which represent + the life cycle of Diameter sessions and which MUST be observed by all + Diameter implementations that make use of the authentication and/or + authorization portion of a Diameter application. The term "Service- + Specific" below refers to a message defined in a Diameter application + (e.g., Mobile IPv4, NASREQ). There are four different authorization session state machines supported in the Diameter base protocol. The first two describe a @@ -5700,26 +5537,25 @@ Internet-Draft Diameter Base Protocol January 2011 When a session is moved to the Idle state, any resources that were allocated for the particular session must be released. Any event not - listed in the state machines MUST be considered as an error - condition, and an answer, if applicable, MUST be returned to the - originator of the message. + listed in the state machines MUST be considered an error condition, + and an answer, if applicable, MUST be returned to the originator of + the message. - In the case that an application does not support re-auth, the state - -Fajardo, et al. Expires July 24, 2011 [Page 102] +Fajardo, et al. Standards Track [Page 99] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - transitions related to server-initiated re-auth when both client and - server session maintains state (e.g., Send RAR, Pending, Receive RAA) - MAY be ignored. + In the case that an application does not support re-auth, the state + transitions related to server-initiated re-auth, when both client and + server sessions maintain state (e.g., Send RAR, Pending, Receive + RAA), MAY be ignored. - In the state table, the event 'Failure to send X' means that the + In the state table, the event "Failure to send X" means that the Diameter agent is unable to send command X to the desired - destination. This could be due to the peer being down, or due to the + destination. This could be due to the peer being down or due to the peer sending back a transient failure or temporary protocol error notification DIAMETER_TOO_BUSY or DIAMETER_LOOP_DETECTED in the Result-Code AVP of the corresponding Answer command. The event 'X @@ -5731,8 +5567,8 @@ Internet-Draft Diameter Base Protocol January 2011 CLIENT, STATEFUL State Event Action New State --------------------------------------------------------------- - Idle Client or Device Requests Send Pending - access service + Idle Client or device requests Send Pending + access service- specific auth req @@ -5748,39 +5584,38 @@ Internet-Draft Diameter Base Protocol January 2011 UNKNOWN_ SESSION_ID - Pending Successful Service-specific Grant Open + Pending Successful service-specific Grant Open authorization answer Access received with default Auth-Session-State value - Pending Successful Service-specific Sent STR Discon - authorization answer received + Pending Successful service-specific Sent STR Discon + authorization answer received, but service not provided Pending Error processing successful Sent STR Discon - Service-specific authorization + service-specific authorization answer - -Fajardo, et al. Expires July 24, 2011 [Page 103] +Fajardo, et al. Standards Track [Page 100] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - Pending Failed Service-specific Cleanup Idle + Pending Failed service-specific Clean up Idle authorization answer received Open User or client device Send Open - requests access to service service + requests access to service service- specific auth req - Open Successful Service-specific Provide Open - authorization answer received Service + Open Successful service-specific Provide Open + authorization answer received service - Open Failed Service-specific Discon. Idle + Open Failed service-specific Discon. Idle authorization answer user/device received. @@ -5796,10 +5631,10 @@ Internet-Draft Diameter Base Protocol January 2011 Discon. user/device - Open Session-Timeout Expires on Send STR Discon - Access Device + Open Session-Timeout expires on Send STR Discon + access device - Open ASR Received, Send ASA Discon + Open ASR received, Send ASA Discon client will comply with with request to end the Result-Code = session = SUCCESS, @@ -5814,17 +5649,18 @@ Internet-Draft Diameter Base Protocol January 2011 Auth-Grace-Period expires on access device - Discon ASR Received Send ASA Discon + Discon ASR received Send ASA Discon + - Discon STA Received Discon. Idle -Fajardo, et al. Expires July 24, 2011 [Page 104] +Fajardo, et al. Standards Track [Page 101] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + Discon STA received Discon. Idle user/device The following state machine is observed by a server when it is @@ -5835,31 +5671,34 @@ Internet-Draft Diameter Base Protocol January 2011 --------------------------------------------------------------- Idle Service-specific authorization Send Open request received, and successful - user is authorized serv. + user is authorized service- specific answer Idle Service-specific authorization Send Idle - request received, and failed serv. - user is not authorized specific + request received, and failed + user is not authorized service- + specific answer Open Service-specific authorization Send Open request received, and user successful - is authorized serv. specific + is authorized service- + specific answer Open Service-specific authorization Send Idle - request received, and user failed serv. - is not authorized specific + request received, and user failed + is not authorized service- + specific answer, - Cleanup + Clean up Open Home server wants to confirm Send RAR Pending authentication and/or authorization of the user - Pending Received RAA with a failed Cleanup Idle + Pending Received RAA with a failed Clean up Idle Result-Code Pending Received RAA with Result-Code Update Open @@ -5868,32 +5707,33 @@ Internet-Draft Diameter Base Protocol January 2011 Open Home server wants to Send ASR Discon terminate the service - Open Authorization-Lifetime (and Cleanup Idle - Auth-Grace-Period) expires - on home server. - Open Session-Timeout expires on Cleanup Idle -Fajardo, et al. Expires July 24, 2011 [Page 105] +Fajardo, et al. Standards Track [Page 102] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + Open Authorization-Lifetime (and Clean up Idle + Auth-Grace-Period) expires + on home server + + Open Session-Timeout expires on Clean up Idle home server Discon Failure to send ASR Wait, Discon resend ASR - Discon ASR successfully sent and Cleanup Idle + Discon ASR successfully sent and Clean up Idle ASA Received with Result-Code - Not ASA Received None No Change. + Not ASA Received None No Change Discon Any STR Received Send STA, Idle - Cleanup. + Clean up The following state machine is observed by a client when state is not maintained on the server: @@ -5901,48 +5741,47 @@ Internet-Draft Diameter Base Protocol January 2011 CLIENT, STATELESS State Event Action New State --------------------------------------------------------------- - Idle Client or Device Requests Send Pending - access service + Idle Client or device requests Send Pending + access service- specific auth req - Pending Successful Service-specific Grant Open - authorization answer Access + Pending Successful service-specific Grant Open + authorization answer access received with Auth-Session- State set to NO_STATE_MAINTAINED - Pending Failed Service-specific Cleanup Idle + Pending Failed service-specific Clean up Idle authorization answer received - Open Session-Timeout Expires on Discon. Idle - Access Device user/device + Open Session-Timeout expires on Discon. Idle + access device user/device Open Service to user is terminated Discon. Idle user/device - The following state machine is observed by a server when it is not - maintaining state for the session: - - - -Fajardo, et al. Expires July 24, 2011 [Page 106] +Fajardo, et al. Standards Track [Page 103] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + The following state machine is observed by a server when it is not + maintaining state for the session: SERVER, STATELESS State Event Action New State --------------------------------------------------------------- - Idle Service-specific authorization Send serv. Idle - request received, and specific - successfully processed answer + Idle Service-specific authorization Send Idle + request received, and service- + successfully processed specific + answer 8.2. Accounting Session State Machine @@ -5960,8 +5799,8 @@ Internet-Draft Diameter Base Protocol January 2011 machine in this section described below. The default server side state machine requires the reception of - accounting records in any order and at any time, and does not place - any standards requirement on the processing of these records. + accounting records in any order and at any time, and it does not + place any standards requirement on the processing of these records. Implementations of Diameter may perform checking, ordering, correlation, fraud detection, and other tasks based on these records. AVPs may need to be inspected as a part of these tasks. The tasks @@ -5970,7 +5809,7 @@ Internet-Draft Diameter Base Protocol January 2011 or even policy dependent, they are not standardized by the Diameter specifications. Applications MAY define requirements on when to accept accounting records based on the used value of Accounting- - Realtime-Required AVP, credit limits checks, and so on. + Realtime-Required AVP, credit-limit checks, and so on. However, the Diameter base protocol defines one optional server side state machine that MAY be followed by applications that require @@ -5978,31 +5817,31 @@ Internet-Draft Diameter Base Protocol January 2011 that such tracking is incompatible with the ability to sustain long duration connectivity problems. Therefore, the use of this state machine is recommended only in applications where the value of the - Accounting-Realtime-Required AVP is DELIVER_AND_GRANT, and hence + Accounting-Realtime-Required AVP is DELIVER_AND_GRANT; hence, accounting connectivity problems are required to cause the serviced user to be disconnected. Otherwise, records produced by the client - may be lost by the server which no longer accepts them after the - connectivity is re-established. This state machine is the third - state machine in this section. The state machine is supervised by a - supervision session timer Ts, which the value should be reasonably -Fajardo, et al. Expires July 24, 2011 [Page 107] +Fajardo, et al. Standards Track [Page 104] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - higher than the Acct_Interim_Interval value. Ts MAY be set to two - times the value of the Acct_Interim_Interval so as to avoid the - accounting session in the Diameter server to change to Idle state in - case of short transient network failure. + may be lost by the server, which no longer accepts them after the + connectivity is re-established. This state machine is the third + state machine in this section. The state machine is supervised by a + supervision session timer Ts, whose value should be reasonably higher + than the Acct_Interim_Interval value. Ts MAY be set to two times the + value of the Acct_Interim_Interval so as to avoid the accounting + session in the Diameter server to change to Idle state in case of + short transient network failure. Any event not listed in the state machines MUST be considered as an error condition, and a corresponding answer, if applicable, MUST be returned to the originator of the message. - In the state table, the event 'Failure to send' means that the + In the state table, the event "Failure to send" means that the Diameter client is unable to communicate with the desired destination. This could be due to the peer being down, or due to the peer sending back a transient failure or temporary protocol error @@ -6010,17 +5849,17 @@ Internet-Draft Diameter Base Protocol January 2011 DIAMETER_LOOP_DETECTED in the Result-Code AVP of the Accounting Answer command. - The event 'Failed answer' means that the Diameter client received a + The event "Failed answer" means that the Diameter client received a non-transient failure notification in the Accounting Answer command. - Note that the action 'Disconnect user/dev' MUST have an effect also - to the authorization session state table, e.g., cause the STR message + Note that the action "Disconnect user/dev" MUST also have an effect + on the authorization session state table, e.g., cause the STR message to be sent, if the given application has both authentication/ authorization and accounting portions. - The states PendingS, PendingI, PendingL, PendingE and PendingB stand + The states PendingS, PendingI, PendingL, PendingE, and PendingB stand for pending states to wait for an answer to an accounting request - related to a Start, Interim, Stop, Event or buffered record, + related to a Start, Interim, Stop, Event, or buffered record, respectively. CLIENT, ACCOUNTING @@ -6037,36 +5876,36 @@ Internet-Draft Diameter Base Protocol January 2011 Idle Records in storage Send PendingB record - PendingS Successful accounting Open - start answer received - - PendingS Failure to send and buffer Store Open -Fajardo, et al. Expires July 24, 2011 [Page 108] +Fajardo, et al. Standards Track [Page 105] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - space available and realtime Start + PendingS Successful accounting Open + start answer received + + PendingS Failure to send and buffer Store Open + space available and real time Start not equal to DELIVER_AND_GRANT Record PendingS Failure to send and no buffer Open - space available and realtime + space available and real time equal to GRANT_AND_LOSE PendingS Failure to send and no Disconnect Idle buffer space available and user/dev - realtime not equal to + real time not equal to GRANT_AND_LOSE PendingS Failed accounting start answer Open - received and realtime equal + received and real time equal to GRANT_AND_LOSE PendingS Failed accounting start answer Disconnect Idle - received and realtime not user/dev + received and real time not user/dev equal to GRANT_AND_LOSE PendingS User service terminated Store PendingS @@ -6077,6 +5916,7 @@ Internet-Draft Diameter Base Protocol January 2011 accounting interim record + Open User service terminated Send PendingL accounting stop req. @@ -6087,34 +5927,35 @@ Internet-Draft Diameter Base Protocol January 2011 PendingI Failure to send and (buffer Store Open space available or old interim record can be overwritten) record - and realtime not equal to + and real time not equal to DELIVER_AND_GRANT - PendingI Failure to send and no buffer Open - space available and realtime - equal to GRANT_AND_LOSE - PendingI Failure to send and no Disconnect Idle - buffer space available and user/dev -Fajardo, et al. Expires July 24, 2011 [Page 109] +Fajardo, et al. Standards Track [Page 106] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + PendingI Failure to send and no buffer Open + space available and real time + equal to GRANT_AND_LOSE - realtime not equal to + PendingI Failure to send and no Disconnect Idle + buffer space available and user/dev + real time not equal to GRANT_AND_LOSE PendingI Failed accounting interim Open - answer received and realtime + answer received and real time equal to GRANT_AND_LOSE PendingI Failed accounting interim Disconnect Idle answer received and user/dev - realtime not equal to + real time not equal to GRANT_AND_LOSE PendingI User service terminated Store PendingI @@ -6148,6 +5989,13 @@ Internet-Draft Diameter Base Protocol January 2011 space available stop record + + +Fajardo, et al. Standards Track [Page 107] + +RFC 6733 Diameter Base Protocol October 2012 + + PendingL Failure to send and no buffer Idle space available @@ -6155,124 +6003,133 @@ Internet-Draft Diameter Base Protocol January 2011 received - -Fajardo, et al. Expires July 24, 2011 [Page 110] - -Internet-Draft Diameter Base Protocol January 2011 - - SERVER, STATELESS ACCOUNTING State Event Action New State --------------------------------------------------------------- Idle Accounting start request Send Idle - received, and successfully accounting + received and successfully accounting processed. start answer Idle Accounting event request Send Idle - received, and successfully accounting + received and successfully accounting processed. event answer - Idle Interim record received, Send Idle + Idle Interim record received Send Idle and successfully processed. accounting interim answer Idle Accounting stop request Send Idle - received, and successfully accounting + received and successfully accounting processed stop answer - Idle Accounting request received, Send Idle + Idle Accounting request received; Send Idle no space left to store accounting - records answer, + records answer; Result-Code = OUT_OF_ SPACE + + + + + + + + + + + + + + +Fajardo, et al. Standards Track [Page 108] + +RFC 6733 Diameter Base Protocol October 2012 + + SERVER, STATEFUL ACCOUNTING State Event Action New State --------------------------------------------------------------- Idle Accounting start request Send Open - received, and successfully accounting + received and successfully accounting processed. start - answer, + answer; Start Ts Idle Accounting event request Send Idle - received, and successfully accounting + received and successfully accounting processed. event answer - - Idle Accounting request received, Send Idle + Idle Accounting request received; Send Idle no space left to store accounting - records answer, - - - -Fajardo, et al. Expires July 24, 2011 [Page 111] - -Internet-Draft Diameter Base Protocol January 2011 - - + records answer; Result-Code = OUT_OF_ SPACE - Open Interim record received, Send Open + Open Interim record received Send Open and successfully processed. accounting interim - answer, + answer; Restart Ts Open Accounting stop request Send Idle - received, and successfully accounting - processed stop answer, + received and successfully accounting + processed stop answer; Stop Ts - Open Accounting request received, Send Idle + Open Accounting request received; Send Idle no space left to store accounting - records answer, + records answer; Result-Code = OUT_OF_ - SPACE, + SPACE; Stop Ts Open Session supervision timer Ts Stop Ts Idle expired + + + + + + + + +Fajardo, et al. Standards Track [Page 109] + +RFC 6733 Diameter Base Protocol October 2012 + + 8.3. Server-Initiated Re-Auth A Diameter server may initiate a re-authentication and/or re- authorization service for a particular session by issuing a Re-Auth- Request (RAR). - For example, for pre-paid services, the Diameter server that + For example, for prepaid services, the Diameter server that originally authorized a session may need some confirmation that the user is still using the services. - An access device that receives a RAR message with Session-Id equal to - a currently active session MUST initiate a re-auth towards the user, - if the service supports this particular feature. Each Diameter - application MUST state whether server-initiated re-auth is supported, - since some applications do not allow access devices to prompt the - user for re-auth. + An access device that receives an RAR message with the Session-Id + equal to a currently active session MUST initiate a re-auth towards + the user, if the service supports this particular feature. Each + Diameter application MUST state whether server-initiated re-auth is + supported, since some applications do not allow access devices to + prompt the user for re-auth. 8.3.1. Re-Auth-Request - The Re-Auth-Request (RAR), indicated by the Command-Code set to 258 + The Re-Auth-Request (RAR), indicated by the Command Code set to 258 and the message flags' 'R' bit set, may be sent by any server to the access device that is providing session service, to request that the - - - -Fajardo, et al. Expires July 24, 2011 [Page 112] - -Internet-Draft Diameter Base Protocol January 2011 - - user be re-authenticated and/or re-authorized. @@ -6294,15 +6151,22 @@ Internet-Draft Diameter Base Protocol January 2011 8.3.2. Re-Auth-Answer - The Re-Auth-Answer (RAA), indicated by the Command-Code set to 258 + The Re-Auth-Answer (RAA), indicated by the Command Code set to 258 and the message flags' 'R' bit clear, is sent in response to the RAR. - The Result-Code AVP MUST be present, and indicates the disposition of - the request. + The Result-Code AVP MUST be present, and it indicates the disposition + of the request. + + + + +Fajardo, et al. Standards Track [Page 110] + +RFC 6733 Diameter Base Protocol October 2012 + A successful RAA message MUST be followed by an application-specific authentication and/or authorization message. - Message Format <RAA> ::= < Diameter Header: 258, PXY > @@ -6321,14 +6185,6 @@ Internet-Draft Diameter Base Protocol January 2011 * [ Proxy-Info ] * [ AVP ] - - - -Fajardo, et al. Expires July 24, 2011 [Page 113] - -Internet-Draft Diameter Base Protocol January 2011 - - 8.4. Session Termination It is necessary for a Diameter server that authorized a session, for @@ -6356,6 +6212,14 @@ Internet-Draft Diameter Base Protocol January 2011 It is also possible that a session that was authorized is never actually started due to action of a proxy. For example, a proxy may + + + +Fajardo, et al. Standards Track [Page 111] + +RFC 6733 Diameter Base Protocol October 2012 + + modify an authorization answer, converting the result from success to failure, prior to forwarding the message to the access device. If the answer did not contain an Auth-Session-State AVP with the value @@ -6366,36 +6230,27 @@ Internet-Draft Diameter Base Protocol January 2011 A Diameter server that receives an STR message MUST clean up resources (e.g., session state) associated with the Session-Id - specified in the STR, and return a Session-Termination-Answer. + specified in the STR and return a Session-Termination-Answer. A Diameter server also MUST clean up resources when the Session- Timeout expires, or when the Authorization-Lifetime and the Auth- - Grace-Period AVPs expires without receipt of a re-authorization + Grace-Period AVPs expire without receipt of a re-authorization request, regardless of whether an STR for that session is received. The access device is not expected to provide service beyond the expiration of these timers; thus, expiration of either of these timers implies that the access device may have unexpectedly shut down. - - - -Fajardo, et al. Expires July 24, 2011 [Page 114] - -Internet-Draft Diameter Base Protocol January 2011 - - 8.4.1. Session-Termination-Request - The Session-Termination-Request (STR), indicated by the Command-Code + The Session-Termination-Request (STR), indicated by the Command Code set to 275 and the Command Flags' 'R' bit set, is sent by a Diameter - client or by a Diameter proxy to inform the Diameter Server that an + client or by a Diameter proxy to inform the Diameter server that an authenticated and/or authorized session is being terminated. + Message Format - Message Format - - <STR> ::= < Diameter Header: 275, REQ, PXY > + <STR> ::= < Diameter Header: 275, REQ, PXY > < Session-Id > { Origin-Host } { Origin-Realm } @@ -6410,40 +6265,33 @@ Internet-Draft Diameter Base Protocol January 2011 * [ Route-Record ] * [ AVP ] -8.4.2. Session-Termination-Answer - - The Session-Termination-Answer (STA), indicated by the Command-Code - set to 275 and the message flags' 'R' bit clear, is sent by the - Diameter Server to acknowledge the notification that the session has - been terminated. The Result-Code AVP MUST be present, and MAY - contain an indication that an error occurred while servicing the STR. - - Upon sending or receipt of the STA, the Diameter Server MUST release - all resources for the session indicated by the Session-Id AVP. Any - intermediate server in the Proxy-Chain MAY also release any - resources, if necessary. - - - - - +Fajardo, et al. Standards Track [Page 112] + +RFC 6733 Diameter Base Protocol October 2012 +8.4.2. Session-Termination-Answer -Fajardo, et al. Expires July 24, 2011 [Page 115] - -Internet-Draft Diameter Base Protocol January 2011 + The Session-Termination-Answer (STA), indicated by the Command Code + set to 275 and the message flags' 'R' bit clear, is sent by the + Diameter server to acknowledge the notification that the session has + been terminated. The Result-Code AVP MUST be present, and it MAY + contain an indication that an error occurred while servicing the STR. + Upon sending or receipt of the STA, the Diameter server MUST release + all resources for the session indicated by the Session-Id AVP. Any + intermediate server in the Proxy-Chain MAY also release any + resources, if necessary. - Message Format + Message Format - <STA> ::= < Diameter Header: 275, PXY > + <STA> ::= < Diameter Header: 275, PXY > < Session-Id > { Result-Code } { Origin-Host } @@ -6473,31 +6321,29 @@ Internet-Draft Diameter Base Protocol January 2011 An access device that receives an ASR with Session-ID equal to a currently active session MAY stop the session. Whether the access - device stops the session or not is implementation- and/or - configuration-dependent. For example, an access device may honor + device stops the session or not is implementation and/or + configuration dependent. For example, an access device may honor ASRs from certain agents only. In any case, the access device MUST + + + +Fajardo, et al. Standards Track [Page 113] + +RFC 6733 Diameter Base Protocol October 2012 + + respond with an Abort-Session-Answer, including a Result-Code AVP to indicate what action it took. 8.5.1. Abort-Session-Request - The Abort-Session-Request (ASR), indicated by the Command-Code set to + The Abort-Session-Request (ASR), indicated by the Command Code set to 274 and the message flags' 'R' bit set, may be sent by any Diameter server or any Diameter proxy to the access device that is providing session service, to request that the session identified by the Session-Id be stopped. - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 116] - -Internet-Draft Diameter Base Protocol January 2011 - - - Message Format + Message Format <ASR> ::= < Diameter Header: 274, REQ, PXY > < Session-Id > @@ -6514,20 +6360,35 @@ Internet-Draft Diameter Base Protocol January 2011 8.5.2. Abort-Session-Answer - The Abort-Session-Answer (ASA), indicated by the Command-Code set to + The Abort-Session-Answer (ASA), indicated by the Command Code set to 274 and the message flags' 'R' bit clear, is sent in response to the - ASR. The Result-Code AVP MUST be present, and indicates the + ASR. The Result-Code AVP MUST be present and indicates the disposition of the request. If the session identified by Session-Id in the ASR was successfully - terminated, Result-Code is set to DIAMETER_SUCCESS. If the session - is not currently active, Result-Code is set to + terminated, the Result-Code is set to DIAMETER_SUCCESS. If the + session is not currently active, the Result-Code is set to DIAMETER_UNKNOWN_SESSION_ID. If the access device does not stop the - session for any other reason, Result-Code is set to + session for any other reason, the Result-Code is set to DIAMETER_UNABLE_TO_COMPLY. - Message Format + + + + + + + + + + +Fajardo, et al. Standards Track [Page 114] + +RFC 6733 Diameter Base Protocol October 2012 + + + Message Format <ASA> ::= < Diameter Header: 274, PXY > < Session-Id > @@ -6545,14 +6406,6 @@ Internet-Draft Diameter Base Protocol January 2011 * [ Proxy-Info ] * [ AVP ] - - - -Fajardo, et al. Expires July 24, 2011 [Page 117] - -Internet-Draft Diameter Base Protocol January 2011 - - 8.6. Inferring Session Termination from Origin-State-Id The Origin-State-Id is used to allow detection of terminated sessions @@ -6560,94 +6413,96 @@ Internet-Draft Diameter Base Protocol January 2011 shutdown of an access device. A Diameter client or access device increments the value of the - Origin-State-Id every time it is started or powered-up. The new + Origin-State-Id every time it is started or powered up. The new Origin-State-Id is then sent in the CER/CEA message immediately upon connection to the server. The Diameter server receiving the new Origin-State-Id can determine whether the sending Diameter client had - abruptly shutdown by comparing the old value of the Origin-State-Id + abruptly shut down by comparing the old value of the Origin-State-Id it has kept for that specific client is less than the new value and whether it has un-terminated sessions originating from that client. An access device can also include the Origin-State-Id in request - messages other than CER if there are relays or proxies in between the - access device and the server. In this case, however, the server + messages other than the CER if there are relays or proxies in between + the access device and the server. In this case, however, the server cannot discover that the access device has been restarted unless and - until it receives a new request from it. Therefore this mechanism is - more opportunistic across proxies and relays. + until it receives a new request from it. Therefore, this mechanism + is more opportunistic across proxies and relays. The Diameter server may assume that all sessions that were active prior to detection of a client restart have been terminated. The Diameter server MAY clean up all session state associated with such - lost sessions, and MAY also issues STRs for all such lost sessions + lost sessions, and it MAY also issue STRs for all such lost sessions that were authorized on upstream servers, to allow session state to be cleaned up globally. + + + + +Fajardo, et al. Standards Track [Page 115] + +RFC 6733 Diameter Base Protocol October 2012 + + 8.7. Auth-Request-Type AVP The Auth-Request-Type AVP (AVP Code 274) is of type Enumerated and is included in application-specific auth requests to inform the peers - whether a user is to be authenticated only, authorized only or both. + whether a user is to be authenticated only, authorized only, or both. Note any value other than both MAY cause RADIUS interoperability issues. The following values are defined: - AUTHENTICATE_ONLY 1 - The request being sent is for authentication only, and MUST - contain the relevant application specific authentication AVPs that + The request being sent is for authentication only, and it MUST + contain the relevant application-specific authentication AVPs that are needed by the Diameter server to authenticate the user. - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 118] - -Internet-Draft Diameter Base Protocol January 2011 - - AUTHORIZE_ONLY 2 - The request being sent is for authorization only, and MUST contain - the application-specific authorization AVPs that are necessary to - identify the service being requested/offered. - + The request being sent is for authorization only, and it MUST + contain the application-specific authorization AVPs that are + necessary to identify the service being requested/offered. AUTHORIZE_AUTHENTICATE 3 The request contains a request for both authentication and authorization. The request MUST include both the relevant - application-specific authentication information, and authorization + application-specific authentication information and authorization information necessary to identify the service being requested/ offered. - 8.8. Session-Id AVP The Session-Id AVP (AVP Code 263) is of type UTF8String and is used to identify a specific session (see Section 8). All messages - pertaining to a specific session MUST include only one Session-Id AVP - and the same value MUST be used throughout the life of a session. - When present, the Session-Id SHOULD appear immediately following the - Diameter Header (see Section 3). + pertaining to a specific session MUST include only one Session-Id + AVP, and the same value MUST be used throughout the life of a + session. When present, the Session-Id SHOULD appear immediately + following the Diameter header (see Section 3). The Session-Id MUST be globally and eternally unique, as it is meant to uniquely identify a user session without reference to any other - information, and may be needed to correlate historical authentication - information with accounting information. The Session-Id includes a - mandatory portion and an implementation-defined portion; a - recommended format for the implementation-defined portion is outlined - below. + information, and it may be needed to correlate historical + authentication information with accounting information. The + Session-Id includes a mandatory portion and an implementation-defined + portion; a recommended format for the implementation-defined portion + is outlined below. The Session-Id MUST begin with the sender's identity encoded in the - DiameterIdentity type (see Section 4.4). The remainder of the - Session-Id is delimited by a ";" character, and MAY be any sequence - that the client can guarantee to be eternally unique; however, the - following format is recommended, (square brackets [] indicate an - optional element): + DiameterIdentity type (see Section 4.3.1). The remainder of the + Session-Id is delimited by a ";" character, and it MAY be any + + + +Fajardo, et al. Standards Track [Page 116] + +RFC 6733 Diameter Base Protocol October 2012 + + + sequence that the client can guarantee to be eternally unique; + however, the following format is recommended, (square brackets [] + indicate an optional element): <DiameterIdentity>;<high 32 bits>;<low 32 bits>[;<optional value>] @@ -6657,22 +6512,14 @@ Internet-Draft Diameter Base Protocol January 2011 processors. At startup, the high 32 bits of the 64-bit value MAY be initialized to the time in NTP format [RFC5905], and the low 32 bits MAY be initialized to zero. This will for practical purposes - - - -Fajardo, et al. Expires July 24, 2011 [Page 119] - -Internet-Draft Diameter Base Protocol January 2011 - - eliminate the possibility of overlapping Session-Ids after a reboot, assuming the reboot process takes longer than a second. Alternatively, an implementation MAY keep track of the increasing value in non-volatile memory. - <optional value> is implementation specific but may include a modem's - device Id, a layer 2 address, timestamp, etc. + <optional value> is implementation specific, but it may include a + modem's device Id, a Layer 2 address, timestamp, etc. Example, in which there is no optional value: @@ -6683,45 +6530,44 @@ Internet-Draft Diameter Base Protocol January 2011 accesspoint7.example.com;1876543210;523;[email protected] The Session-Id is created by the Diameter application initiating the - session, which in most cases is done by the client. Note that a - Session-Id MAY be used for both the authentication, authorization and - accounting commands of a given application. + session, which, in most cases, is done by the client. Note that a + Session-Id MAY be used for both the authentication, authorization, + and accounting commands of a given application. 8.9. Authorization-Lifetime AVP The Authorization-Lifetime AVP (AVP Code 291) is of type Unsigned32 and contains the maximum number of seconds of service to be provided to the user before the user is to be re-authenticated and/or re- - authorized. Care should be taken when the Authorization- Lifetime - value is determined, since a low, non-zero, value could create + authorized. Care should be taken when the Authorization-Lifetime + value is determined, since a low, non-zero value could create significant Diameter traffic, which could congest both the network and the agents. A value of zero (0) means that immediate re-auth is necessary by the access device. The absence of this AVP, or a value of all ones - (meaning all bits in the 32 bit field are set to one) means no re- + (meaning all bits in the 32-bit field are set to one) means no re- auth is expected. + + +Fajardo, et al. Standards Track [Page 117] + +RFC 6733 Diameter Base Protocol October 2012 + + If both this AVP and the Session-Timeout AVP are present in a message, the value of the latter MUST NOT be smaller than the Authorization-Lifetime AVP. An Authorization-Lifetime AVP MAY be present in re-authorization - messages, and contains the number of seconds the user is authorized - to receive service from the time the re-auth answer message is - received by the access device. + messages, and it contains the number of seconds the user is + authorized to receive service from the time the re-auth answer + message is received by the access device. This AVP MAY be provided by the client as a hint of the maximum lifetime that it is willing to accept. The server MUST return a - - - -Fajardo, et al. Expires July 24, 2011 [Page 120] - -Internet-Draft Diameter Base Protocol January 2011 - - - value that is equal to, or smaller, than the one provided by the + value that is equal to, or smaller than, the one provided by the client. 8.10. Auth-Grace-Period AVP @@ -6739,7 +6585,6 @@ Internet-Draft Diameter Base Protocol January 2011 the value in the server's answer message is binding. The following values are supported: - STATE_MAINTAINED 0 This value is used to specify that session state is being @@ -6747,46 +6592,42 @@ Internet-Draft Diameter Base Protocol January 2011 message when service to the user is terminated. This is the default value. - NO_STATE_MAINTAINED 1 This value is used to specify that no session termination messages will be sent by the access device upon expiration of the Authorization-Lifetime. - 8.12. Re-Auth-Request-Type AVP The Re-Auth-Request-Type AVP (AVP Code 285) is of type Enumerated and is included in application-specific auth answers to inform the client of the action expected upon expiration of the Authorization-Lifetime. - If the answer message contains an Authorization-Lifetime AVP with a - positive value, the Re-Auth-Request-Type AVP MUST be present in an - answer message. The following values are defined: - AUTHORIZE_ONLY 0 - An authorization only re-auth is expected upon expiration of the - Authorization-Lifetime. This is the default value if the AVP is +Fajardo, et al. Standards Track [Page 118] + +RFC 6733 Diameter Base Protocol October 2012 -Fajardo, et al. Expires July 24, 2011 [Page 121] - -Internet-Draft Diameter Base Protocol January 2011 + If the answer message contains an Authorization-Lifetime AVP with a + positive value, the Re-Auth-Request-Type AVP MUST be present in an + answer message. The following values are defined: + AUTHORIZE_ONLY 0 + An authorization only re-auth is expected upon expiration of the + Authorization-Lifetime. This is the default value if the AVP is not present in answer messages that include the Authorization- Lifetime. - AUTHORIZE_AUTHENTICATE 1 An authentication and authorization re-auth is expected upon expiration of the Authorization-Lifetime. - 8.13. Session-Timeout AVP The Session-Timeout AVP (AVP Code 27) [RFC2865] is of type Unsigned32 @@ -6799,10 +6640,10 @@ Internet-Draft Diameter Base Protocol January 2011 A session that terminates on an access device due to the expiration of the Session-Timeout MUST cause an STR to be issued, unless both the access device and the home server had previously agreed that no - session termination messages would be sent (see Section 8.11). + session termination messages would be sent (see Section 8). A Session-Timeout AVP MAY be present in a re-authorization answer - message, and contains the remaining number of seconds from the + message, and it contains the remaining number of seconds from the beginning of the re-auth. A value of zero, or the absence of this AVP, means that this session @@ -6810,7 +6651,7 @@ Internet-Draft Diameter Base Protocol January 2011 This AVP MAY be provided by the client as a hint of the maximum timeout that it is willing to accept. However, the server MAY return - a value that is equal to, or smaller, than the one provided by the + a value that is equal to, or smaller than, the one provided by the client. 8.14. User-Name AVP @@ -6819,83 +6660,29 @@ Internet-Draft Diameter Base Protocol January 2011 contains the User-Name, in a format consistent with the NAI specification [RFC4282]. -8.15. Termination-Cause AVP - - The Termination-Cause AVP (AVP Code 295) is of type Enumerated, and - is used to indicate the reason why a session was terminated on the - access device. The following values are defined: - -Fajardo, et al. Expires July 24, 2011 [Page 122] +Fajardo, et al. Standards Track [Page 119] -Internet-Draft Diameter Base Protocol January 2011 - - - DIAMETER_LOGOUT 1 - - The user initiated a disconnect - - - DIAMETER_SERVICE_NOT_PROVIDED 2 - - This value is used when the user disconnected prior to the receipt - of the authorization answer message. - - - DIAMETER_BAD_ANSWER 3 - - This value indicates that the authorization answer received by the - access device was not processed successfully. - - - DIAMETER_ADMINISTRATIVE 4 - - The user was not granted access, or was disconnected, due to - administrative reasons, such as the receipt of a Abort-Session- - Request message. - - - DIAMETER_LINK_BROKEN 5 - - The communication to the user was abruptly disconnected. - - - DIAMETER_AUTH_EXPIRED 6 - - The user's access was terminated since its authorized session time - has expired. - - - DIAMETER_USER_MOVED 7 - - The user is receiving services from another access device. - - - DIAMETER_SESSION_TIMEOUT 8 +RFC 6733 Diameter Base Protocol October 2012 - The user's session has timed out, and service has been terminated. +8.15. Termination-Cause AVP - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 123] - -Internet-Draft Diameter Base Protocol January 2011 - + The Termination-Cause AVP (AVP Code 295) is of type Enumerated, and + is used to indicate the reason why a session was terminated on the + access device. The currently assigned values for this AVP can be + found in the IANA registry for Termination-Cause AVP Values + [IANATCV]. 8.16. Origin-State-Id AVP The Origin-State-Id AVP (AVP Code 278), of type Unsigned32, is a monotonically increasing value that is advanced whenever a Diameter - entity restarts with loss of previous state, for example upon reboot. - Origin-State-Id MAY be included in any Diameter message, including - CER. + entity restarts with loss of previous state, for example, upon + reboot. Origin-State-Id MAY be included in any Diameter message, + including CER. A Diameter entity issuing this AVP MUST create a higher value for this AVP each time its state is reset. A Diameter entity MAY set @@ -6911,20 +6698,34 @@ Internet-Draft Diameter Base Protocol January 2011 message, it allows other Diameter entities to infer that sessions associated with a lower Origin-State-Id are no longer active. If an access device does not intend for such inferences to be made, it MUST - either not include Origin-State-Id in any message, or set its value - to 0. + either not include Origin-State-Id in any message or set its value to + 0. 8.17. Session-Binding AVP - The Session-Binding AVP (AVP Code 270) is of type Unsigned32, and MAY - be present in application-specific authorization answer messages. If - present, this AVP MAY inform the Diameter client that all future + The Session-Binding AVP (AVP Code 270) is of type Unsigned32, and it + MAY be present in application-specific authorization answer messages. + If present, this AVP MAY inform the Diameter client that all future application-specific re-auth and Session-Termination-Request messages for this session MUST be sent to the same authorization server. - This field is a bit mask, and the following bits have been defined: + + + + + + + + +Fajardo, et al. Standards Track [Page 120] + +RFC 6733 Diameter Base Protocol October 2012 + + + This field is a bit mask, and the following bits have been defined: + RE_AUTH 1 When set, future re-auth messages for this session MUST NOT @@ -6932,23 +6733,13 @@ Internet-Draft Diameter Base Protocol January 2011 value, the Destination-Host AVP MUST be present in all re-auth messages for this session. - STR 2 When set, the STR message for this session MUST NOT include the Destination-Host AVP. When cleared, the default value, the - - - -Fajardo, et al. Expires July 24, 2011 [Page 124] - -Internet-Draft Diameter Base Protocol January 2011 - - Destination-Host AVP MUST be present in the STR message for this session. - ACCOUNTING 4 When set, all accounting messages for this session MUST NOT @@ -6956,10 +6747,9 @@ Internet-Draft Diameter Base Protocol January 2011 value, the Destination-Host AVP, if known, MUST be present in all accounting messages for this session. - 8.18. Session-Server-Failover AVP - The Session-Server-Failover AVP (AVP Code 271) is of type Enumerated, + The Session-Server-Failover AVP (AVP Code 271) is of type Enumerated and MAY be present in application-specific authorization answer messages that either do not include the Session-Binding AVP or include the Session-Binding AVP with any of the bits set to a zero @@ -6970,11 +6760,24 @@ Internet-Draft Diameter Base Protocol January 2011 The following values are supported: - REFUSE_SERVICE 0 If either the re-auth or the STR message delivery fails, terminate - service with the user, and do not attempt any subsequent attempts. + service with the user and do not attempt any subsequent attempts. + + + + + + + + + + + +Fajardo, et al. Standards Track [Page 121] + +RFC 6733 Diameter Base Protocol October 2012 TRY_AGAIN 1 @@ -6982,34 +6785,23 @@ Internet-Draft Diameter Base Protocol January 2011 If either the re-auth or the STR message delivery fails, resend the failed message without the Destination-Host AVP present. - ALLOW_SERVICE 2 If re-auth message delivery fails, assume that re-authorization succeeded. If STR message delivery fails, terminate the session. - TRY_AGAIN_ALLOW_SERVICE 3 If either the re-auth or the STR message delivery fails, resend the failed message without the Destination-Host AVP present. If - - - -Fajardo, et al. Expires July 24, 2011 [Page 125] - -Internet-Draft Diameter Base Protocol January 2011 - - the second delivery fails for re-auth, assume re-authorization succeeded. If the second delivery fails for STR, terminate the session. - 8.19. Multi-Round-Time-Out AVP - The Multi-Round-Time-Out AVP (AVP Code 272) is of type Unsigned32, - and SHOULD be present in application-specific authorization answer + The Multi-Round-Time-Out AVP (AVP Code 272) is of type Unsigned32 and + SHOULD be present in application-specific authorization answer messages whose Result-Code AVP is set to DIAMETER_MULTI_ROUND_AUTH. This AVP contains the maximum number of seconds that the access device MUST provide the user in responding to an authentication @@ -7031,37 +6823,24 @@ Internet-Draft Diameter Base Protocol January 2011 8.21. Event-Timestamp AVP - The Event-Timestamp (AVP Code 55) is of type Time, and MAY be - included in an Accounting-Request and Accounting-Answer messages to - record the time that the reported event occurred, in seconds since - January 1, 1900 00:00 UTC. - - + The Event-Timestamp (AVP Code 55) is of type Time and MAY be included + in an Accounting-Request and Accounting-Answer messages to record the + time that the reported event occurred, in seconds since January 1, + 1900 00:00 UTC. - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 126] +Fajardo, et al. Standards Track [Page 122] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 9. Accounting This accounting protocol is based on a server directed model with capabilities for real-time delivery of accounting information. - Several fault resilience methods [RFC2975] have been built in to the + Several fault resilience methods [RFC2975] have been built into the protocol in order minimize loss of accounting data in various fault situations and under different assumptions about the capabilities of the used devices. @@ -7075,15 +6854,16 @@ Internet-Draft Diameter Base Protocol January 2011 timeliness requirements. As discussed in [RFC2975], real-time transfer of accounting records - is a requirement, such as the need to perform credit limit checks and + is a requirement, such as the need to perform credit-limit checks and fraud detection. Note that batch accounting is not a requirement, and is therefore not supported by Diameter. Should batched accounting be required in the future, a new Diameter application will need to be created, or it could be handled using another protocol. - Note, however, that even if at the Diameter layer accounting requests - are processed one by one, transport protocols used under Diameter - typically batch several requests in the same packet under heavy - traffic conditions. This may be sufficient for many applications. + Note, however, that even if at the Diameter layer, accounting + requests are processed one by one; transport protocols used under + Diameter typically batch several requests in the same packet under + heavy traffic conditions. This may be sufficient for many + applications. The authorization server (chain) directs the selection of proper transfer strategy, based on its knowledge of the user and @@ -7097,7 +6877,7 @@ Internet-Draft Diameter Base Protocol January 2011 records from the Diameter client is delayed or unsuccessful. The Diameter accounting server MAY override the interim interval or - the realtime requirements by including the Acct-Interim-Interval or + the real-time requirements by including the Acct-Interim-Interval or Accounting-Realtime-Required AVP in the Accounting-Answer message. When one of these AVPs is present, the latest value received SHOULD be used in further accounting activities for the same session. @@ -7107,16 +6887,15 @@ Internet-Draft Diameter Base Protocol January 2011 - -Fajardo, et al. Expires July 24, 2011 [Page 127] +Fajardo, et al. Standards Track [Page 123] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 9.2. Protocol Messages A Diameter node that receives a successful authentication and/or - authorization messages from the Diameter server SHOULD collect + authorization message from the Diameter server SHOULD collect accounting information for the session. The Accounting-Request message is used to transmit the accounting information to the Diameter server, which MUST reply with the Accounting-Answer message @@ -7129,12 +6908,12 @@ Internet-Draft Diameter Base Protocol January 2011 9.3. Accounting Application Extension and Requirements - Each Diameter application (e.g., NASREQ, MobileIP), SHOULD define - their Service-Specific AVPs that MUST be present in the Accounting- - Request message in a section entitled "Accounting AVPs". The - application MUST assume that the AVPs described in this document will - be present in all Accounting messages, so only their respective - service-specific AVPs need to be defined in that section. + Each Diameter application (e.g., NASREQ, Mobile IP) SHOULD define its + service-specific AVPs that MUST be present in the Accounting-Request + message in a section titled "Accounting AVPs". The application MUST + assume that the AVPs described in this document will be present in + all Accounting messages, so only their respective service-specific + AVPs need to be defined in that section. Applications have the option of using one or both of the following accounting application extension models: @@ -7150,40 +6929,37 @@ Internet-Draft Diameter Base Protocol January 2011 advertise the Diameter base accounting Application Id during capabilities exchange. - Coupled Accounting Service - The accounting messages will carry the Application Id of the + The accounting message will carry the Application Id of the application that is using it. The application itself will process the received accounting records or forward them to an accounting server. There is no accounting application advertisement required - during capabilities exchange and the accounting messages will be - routed the same as any of the other application messages. - + during capabilities exchange, and the accounting messages will be + routed the same way as any of the other application messages. + In cases where an application does not define its own accounting + service, it is preferred that the split accounting model be used. -Fajardo, et al. Expires July 24, 2011 [Page 128] +Fajardo, et al. Standards Track [Page 124] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - In cases where an application does not define its own accounting - service, it is preferred that the split accounting model be used. - 9.4. Fault Resilience - Diameter Base protocol mechanisms are used to overcome small message - loss and network faults of temporary nature. + Diameter base protocol mechanisms are used to overcome small message + loss and network faults of a temporary nature. Diameter peers acting as clients MUST implement the use of failover to guard against server failures and certain network failures. Diameter peers acting as agents or related off-line processing systems MUST detect duplicate accounting records caused by the - sending of same record to several servers and duplication of messages - in transit. This detection MUST be based on the inspection of the - Session-Id and Accounting-Record-Number AVP pairs. Appendix C + sending of the same record to several servers and duplication of + messages in transit. This detection MUST be based on the inspection + of the Session-Id and Accounting-Record-Number AVP pairs. Appendix C discusses duplicate detection needs and implementation issues. Diameter clients MAY have non-volatile memory for the safe storage of @@ -7191,23 +6967,23 @@ Internet-Draft Diameter Base Protocol January 2011 partitions, and server failures. If such memory is available, the client SHOULD store new accounting records there as soon as the records are created and until a positive acknowledgement of their - reception from the Diameter Server has been received. Upon a reboot, - the client MUST starting sending the records in the non-volatile - memory to the accounting server with appropriate modifications in + reception from the Diameter server has been received. Upon a reboot, + the client MUST start sending the records in the non-volatile memory + to the accounting server with the appropriate modifications in termination cause, session length, and other relevant information in the records. A further application of this protocol may include AVPs to control - how many accounting records may at most be stored in the Diameter - client without committing them to the non-volatile memory or + the maximum number of accounting records that may be stored in the + Diameter client without committing them to the non-volatile memory or transferring them to the Diameter server. The client SHOULD NOT remove the accounting data from any of its memory areas before the correct Accounting-Answer has been received. - The client MAY remove oldest, undelivered or yet unacknowledged - accounting data if it runs out of resources such as memory. It is an - implementation dependent matter for the client to accept new sessions - under this condition. + The client MAY remove the oldest, undelivered, or as yet + unacknowledged accounting data if it runs out of resources such as + memory. It is an implementation-dependent matter for the client to + accept new sessions under this condition. 9.5. Accounting Records @@ -7217,15 +6993,17 @@ Internet-Draft Diameter Base Protocol January 2011 Different types of accounting records are sent depending on the actual type of accounted service and the authorization server's + directions for interim accounting. If the accounted service is a + + -Fajardo, et al. Expires July 24, 2011 [Page 129] +Fajardo, et al. Standards Track [Page 125] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - directions for interim accounting. If the accounted service is a one-time event, meaning that the start and stop of the event are simultaneous, then the Accounting-Record-Type AVP MUST be present and set to the value EVENT_RECORD. @@ -7251,11 +7029,11 @@ Internet-Draft Diameter Base Protocol January 2011 session. A particular value of Accounting-Sub-Session-Id MUST appear only in - one sequence of accounting records from a DIAMETER client, except for + one sequence of accounting records from a Diameter client, except for the purposes of retransmission. The one sequence that is sent MUST be either one record with Accounting-Record-Type AVP set to the value - EVENT_RECORD, or several records starting with one having the value - START_RECORD, followed by zero or more INTERIM_RECORD and a single + EVENT_RECORD or several records starting with one having the value + START_RECORD, followed by zero or more INTERIM_RECORDs and a single STOP_RECORD. A particular Diameter application specification MUST define the type of sequences that MUST be used. @@ -7265,20 +7043,21 @@ Internet-Draft Diameter Base Protocol January 2011 accounting records with a specific application session by using the Session-Id of the particular application session in the accounting messages. Accounting messages MAY also use a different Session-Id - from that of the application sessions in which case other session + from that of the application sessions, in which case, other session- related information is needed to perform correlation. In cases where an application requires multiple accounting sub- - session, an Accounting-Sub-Session-Id AVP is used to differentiate + sessions, an Accounting-Sub-Session-Id AVP is used to differentiate each sub-session. The Session-Id would remain constant for all sub- - sessions and is be used to correlate all the sub-sessions to a + sessions and is used to correlate all the sub-sessions to a particular application session. Note that receiving a STOP_RECORD -Fajardo, et al. Expires July 24, 2011 [Page 130] + +Fajardo, et al. Standards Track [Page 126] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 with no Accounting-Sub-Session-Id AVP when sub-sessions were @@ -7290,36 +7069,36 @@ Internet-Draft Diameter Base Protocol January 2011 record may span multiple different Diameter applications and sessions used by the same user at a given time. In such cases, the Acct- Multi-Session-Id AVP is used. The Acct-Multi-Session-Id AVP SHOULD - be signaled by the server to the access device (typically during + be signaled by the server to the access device (typically, during authorization) when it determines that a request belongs to an existing session. The access device MUST then include the Acct- Multi-Session-Id AVP in all subsequent accounting messages. The Acct-Multi-Session-Id AVP MAY include the value of the original - Session-Id. It's contents are implementation specific, but MUST be - globally unique across other Acct-Multi-Session-Id, and MUST NOT + Session-Id. Its contents are implementation specific, but the MUST + be globally unique across other Acct-Multi-Session-Ids and MUST NOT change during the life of a session. A Diameter application document MUST define the exact concept of a - session that is being accounted, and MAY define the concept of a + session that is being accounted, and it MAY define the concept of a multi-session. For instance, the NASREQ DIAMETER application treats - a single PPP connection to a Network Access Server as one session, - and a set of Multilink PPP sessions as one multi-session. + a single PPP connection to a Network Access Server as one session and + a set of Multilink PPP sessions as one multi-session. -9.7. Accounting Command-Codes +9.7. Accounting Command Codes - This section defines Command-Code values that MUST be supported by - all Diameter implementations that provide Accounting services. + This section defines Command Code values that MUST be supported by + all Diameter implementations that provide accounting services. 9.7.1. Accounting-Request - The Accounting-Request (ACR) command, indicated by the Command-Code + The Accounting-Request (ACR) command, indicated by the Command Code field set to 271 and the Command Flags' 'R' bit set, is sent by a Diameter node, acting as a client, in order to exchange accounting information with a peer. - The AVP listed below SHOULD include service-specific accounting AVPs, - as described in Section 9.3. + In addition to the AVPs listed below, Accounting-Request messages + SHOULD include service-specific accounting AVPs. @@ -7332,9 +7111,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 131] +Fajardo, et al. Standards Track [Page 127] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Message Format @@ -7363,16 +7142,16 @@ Internet-Draft Diameter Base Protocol January 2011 9.7.2. Accounting-Answer - The Accounting-Answer (ACA) command, indicated by the Command-Code + The Accounting-Answer (ACA) command, indicated by the Command Code field set to 271 and the Command Flags' 'R' bit cleared, is used to acknowledge an Accounting-Request command. The Accounting-Answer command contains the same Session-Id as the corresponding request. - Only the target Diameter Server, known as the home Diameter Server, + Only the target Diameter server, known as the home Diameter server, SHOULD respond with the Accounting-Answer command. - The AVP listed below SHOULD include service-specific accounting AVPs, - as described in Section 9.3. + In addition to the AVPs listed below, Accounting-Answer messages + SHOULD include service-specific accounting AVPs. @@ -7388,9 +7167,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 132] +Fajardo, et al. Standards Track [Page 128] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Message Format @@ -7429,13 +7208,12 @@ Internet-Draft Diameter Base Protocol January 2011 and contains the type of accounting record being sent. The following values are currently defined for the Accounting-Record-Type AVP: - EVENT_RECORD 1 An Accounting Event Record is used to indicate that a one-time event has occurred (meaning that the start and end of the event are simultaneous). This record contains all information relevant - to the service, and is the only record of the service. + to the service, and it is the only record of the service. @@ -7444,20 +7222,20 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 133] + +Fajardo, et al. Standards Track [Page 129] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 START_RECORD 2 - An Accounting Start, Interim, and Stop Records are used to - indicate that a service of a measurable length has been given. An - Accounting Start Record is used to initiate an accounting session, + Accounting Start, Interim, and Stop Records are used to indicate + that a service of a measurable length has been given. An + Accounting Start Record is used to initiate an accounting session and contains accounting information that is relevant to the initiation of the session. - INTERIM_RECORD 3 An Interim Accounting Record contains cumulative accounting @@ -7468,14 +7246,12 @@ Internet-Draft Diameter Base Protocol January 2011 applications. The selection of whether to use INTERIM_RECORD records is done by the Acct-Interim-Interval AVP. - STOP_RECORD 4 An Accounting Stop Record is sent to terminate an accounting session and contains cumulative accounting information relevant to the existing session. - 9.8.2. Acct-Interim-Interval AVP The Acct-Interim-Interval AVP (AVP Code 85) is of type Unsigned32 and @@ -7483,31 +7259,31 @@ Internet-Draft Diameter Base Protocol January 2011 client. The client uses information in this AVP to decide how and when to produce accounting records. With different values in this AVP, service sessions can result in one, two, or two+N accounting - records, based on the needs of the home-organization. The following + records, based on the needs of the home organization. The following accounting record production behavior is directed by the inclusion of this AVP: - 1. The omission of the Acct-Interim-Interval AVP or its inclusion with Value field set to 0 means that EVENT_RECORD, START_RECORD, and STOP_RECORD are produced, as appropriate for the service. - 2. The inclusion of the AVP with Value field set to a non-zero value means that INTERIM_RECORD records MUST be produced between the START_RECORD and STOP_RECORD records. The Value field of this AVP is the nominal interval between these records in seconds. + The Diameter node that originates the accounting information, + known as the client, MUST produce the first INTERIM_RECORD record + roughly at the time when this nominal interval has elapsed from + + -Fajardo, et al. Expires July 24, 2011 [Page 134] +Fajardo, et al. Standards Track [Page 130] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - The Diameter node that originates the accounting information, - known as the client, MUST produce the first INTERIM_RECORD record - roughly at the time when this nominal interval has elapsed from the START_RECORD, the next one again as the interval has elapsed once more, and so on until the session ends and a STOP_RECORD record is produced. @@ -7522,10 +7298,10 @@ Internet-Draft Diameter Base Protocol January 2011 The Accounting-Record-Number AVP (AVP Code 485) is of type Unsigned32 and identifies this record within one session. As Session-Id AVPs are globally unique, the combination of Session-Id and Accounting- - Record-Number AVPs is also globally unique, and can be used in + Record-Number AVPs is also globally unique and can be used in matching accounting records with confirmations. An easy way to produce unique numbers is to set the value to 0 for records of type - EVENT_RECORD and START_RECORD, and set the value to 1 for the first + EVENT_RECORD and START_RECORD and set the value to 1 for the first INTERIM_RECORD, 2 for the second, and so on until the value for STOP_RECORD is one more than for the last INTERIM_RECORD. @@ -7539,10 +7315,10 @@ Internet-Draft Diameter Base Protocol January 2011 The Acct-Multi-Session-Id AVP (AVP Code 50) is of type UTF8String, following the format specified in Section 8.8. The Acct-Multi- - Session-Id AVP is used to link together multiple related accounting - sessions, where each session would have a unique Session-Id, but the - same Acct-Multi-Session-Id AVP. This AVP MAY be returned by the - Diameter server in an authorization answer, and MUST be used in all + Session-Id AVP is used to link multiple related accounting sessions, + where each session would have a unique Session-Id but the same Acct- + Multi-Session-Id AVP. This AVP MAY be returned by the Diameter + server in an authorization answer, and it MUST be used in all accounting messages for the given session. 9.8.6. Accounting-Sub-Session-Id AVP @@ -7553,18 +7329,17 @@ Internet-Draft Diameter Base Protocol January 2011 session, and the value of this AVP MUST be monotonically increased by one for all new sub-sessions. The absence of this AVP implies no sub-sessions are in use, with the exception of an Accounting-Request + whose Accounting-Record-Type is set to STOP_RECORD. A STOP_RECORD + message with no Accounting-Sub-Session-Id AVP present will signal the + termination of all sub-sessions for a given Session-Id. -Fajardo, et al. Expires July 24, 2011 [Page 135] +Fajardo, et al. Standards Track [Page 131] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - whose Accounting-Record-Type is set to STOP_RECORD. A STOP_RECORD - message with no Accounting-Sub-Session-Id AVP present will signal the - termination of all sub-sessions for a given Session-Id. - 9.8.7. Accounting-Realtime-Required AVP The Accounting-Realtime-Required AVP (AVP Code 483) is of type @@ -7574,7 +7349,6 @@ Internet-Draft Diameter Base Protocol January 2011 if the sending of accounting records to the accounting server has been temporarily prevented due to, for instance, a network problem. - DELIVER_AND_GRANT 1 The AVP with Value field set to DELIVER_AND_GRANT means that the @@ -7584,7 +7358,6 @@ Internet-Draft Diameter Base Protocol January 2011 the accounting record stream to a backup server is not a reason to discontinue the service to the user. - GRANT_AND_STORE 2 The AVP with Value field set to GRANT_AND_STORE means that service @@ -7594,60 +7367,51 @@ Internet-Draft Diameter Base Protocol January 2011 This is the default behavior if the AVP isn't included in the reply from the authorization server. - GRANT_AND_LOSE 3 The AVP with Value field set to GRANT_AND_LOSE means that service SHOULD be granted even if the records cannot be delivered or stored. +10. AVP Occurrence Tables + The following tables present the AVPs defined in this document and + specify in which Diameter messages they MAY or MAY NOT be present. + AVPs that occur only inside a Grouped AVP are not shown in these + tables. + The tables use the following symbols: + 0 The AVP MUST NOT be present in the message. + 0+ Zero or more instances of the AVP MAY be present in the + message. - - -Fajardo, et al. Expires July 24, 2011 [Page 136] +Fajardo, et al. Standards Track [Page 132] -Internet-Draft Diameter Base Protocol January 2011 - - -10. AVP Occurrence Table - - The following tables presents the AVPs defined in this document, and - specifies in which Diameter messages they MAY be present or not. - AVPs that occur only inside a Grouped AVP are not shown in this - table. +RFC 6733 Diameter Base Protocol October 2012 - The table uses the following symbols: + 0-1 Zero or one instance of the AVP MAY be present in the message. + It is considered an error if there are more than one instance + of the AVP. - 0 The AVP MUST NOT be present in the message. + 1 One instance of the AVP MUST be present in the message. - 0+ Zero or more instances of the AVP MAY be present in the - message. - - 0-1 Zero or one instance of the AVP MAY be present in the message. - It is considered an error if there are more than one instance of - the AVP. - - 1 One instance of the AVP MUST be present in the message. - - 1+ At least one instance of the AVP MUST be present in the - message. + 1+ At least one instance of the AVP MUST be present in the + message. 10.1. Base Protocol Command AVP Table - The table in this section is limited to the non-accounting Command + The table in this section is limited to the non-Accounting Command Codes defined in this specification. +-----------------------------------------------+ - | Command-Code | + | Command Code | +---+---+---+---+---+---+---+---+---+---+---+---+ Attribute Name |CER|CEA|DPR|DPA|DWR|DWA|RAR|RAA|ASR|ASA|STR|STA| --------------------+---+---+---+---+---+---+---+---+---+---+---+---+ @@ -7665,22 +7429,29 @@ Internet-Draft Diameter Base Protocol January 2011 Class |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0+ |0+ | Destination-Host |0 |0 |0 |0 |0 |0 |1 |0 |1 |0 |0-1|0 | Destination-Realm |0 |0 |0 |0 |0 |0 |1 |0 |1 |0 |1 |0 | - - - -Fajardo, et al. Expires July 24, 2011 [Page 137] - -Internet-Draft Diameter Base Protocol January 2011 - - Disconnect-Cause |0 |0 |1 |0 |0 |0 |0 |0 |0 |0 |0 |0 | Error-Message |0 |0-1|0 |0-1|0 |0-1|0 |0-1|0 |0-1|0 |0-1| Error-Reporting-Host|0 |0 |0 |0 |0 |0 |0 |0-1|0 |0-1|0 |0-1| - Failed-AVP |0 |0+ |0 |0+ |0 |0+ |0 |0+ |0 |0+ |0 |0+ | + Failed-AVP |0 |0-1|0 |0-1|0 |0-1|0 |0-1|0 |0-1|0 |0-1| Firmware-Revision |0-1|0-1|0 |0 |0 |0 |0 |0 |0 |0 |0 |0 | Host-IP-Address |1+ |1+ |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 | Inband-Security-Id |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 | Multi-Round-Time-Out|0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 | + + + + + + + + + + +Fajardo, et al. Standards Track [Page 133] + +RFC 6733 Diameter Base Protocol October 2012 + + Origin-Host |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 | Origin-Realm |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 | Origin-State-Id |0-1|0-1|0 |0 |0-1|0-1|0-1|0-1|0-1|0-1|0-1|0-1| @@ -7724,9 +7495,17 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 138] + + + + + + + + +Fajardo, et al. Standards Track [Page 134] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 +-----------+ @@ -7749,6 +7528,7 @@ Internet-Draft Diameter Base Protocol January 2011 Destination-Realm | 1 | 0 | Error-Reporting-Host | 0 | 0+ | Event-Timestamp | 0-1 | 0-1 | + Failed-AVP | 0 | 0-1 | Origin-Host | 1 | 1 | Origin-Realm | 1 | 1 | Proxy-Info | 0+ | 0+ | @@ -7760,144 +7540,220 @@ Internet-Draft Diameter Base Protocol January 2011 Vendor-Specific-Application-Id| 0-1 | 0-1 | ------------------------------+-----+-----+ +11. IANA Considerations + This section provides guidance to the Internet Assigned Numbers + Authority (IANA) regarding registration of values related to the + Diameter protocol, in accordance with [RFC5226]. Existing IANA + registries and assignments put in place by RFC 3588 remain the same + unless explicitly updated or deprecated in this section. +11.1. AVP Header + As defined in Section 4, the AVP header contains three fields that + require IANA namespace management: the AVP Code, Vendor-ID, and Flags + fields. - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 139] +Fajardo, et al. Standards Track [Page 135] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 -11. IANA Considerations +11.1.1. AVP Codes - This section provides guidance to the Internet Assigned Numbers - Authority (IANA) regarding registration of values related to the - Diameter protocol, in accordance with BCP 26 [RFC5226]. The policies - and procedures for the IANA put in place by [RFC3588] applies here. - The criteria used by the IANA for assignment of numbers within this - namespace remains the same unless otherwise stated in this section. - Existing assignments remains the same unless explicitly updated or - deprecated in this secion. + There are multiple namespaces. Vendors can have their own AVP Codes + namespace that will be identified by their Vendor-ID (also known as + Enterprise-Number), and they control the assignments of their vendor- + specific AVP Codes within their own namespace. The absence of a + Vendor-ID or a Vendor-ID value of zero (0) identifies the IETF AVP + Codes namespace, which is under IANA control. The AVP Codes and + sometimes possible values in an AVP are controlled and maintained by + IANA. AVP Code 0 is not used. AVP Codes 1-255 are managed + separately as RADIUS Attribute Types. Where a Vendor-Specific AVP is + implemented by more than one vendor, allocation of global AVPs should + be encouraged instead. + + AVPs may be allocated following Expert Review (by a Designated + Expert) with Specification Required [RFC5226]. A block allocation + (release of more than three AVPs at a time for a given purpose) + requires IETF Review [RFC5226]. -11.1. Changes to AVP Header Allocation +11.1.2. AVP Flags - For AVP Headers, the only change is the AVP code block allocations. - Block allocation (release of more than 3 at a time for a given - purpose) now only require IETF Review as opposed to an IETF - Consensus. + Section 4.1 describes the existing AVP Flags. The remaining bits can + only be assigned via a Standards Action [RFC5226]. 11.2. Diameter Header - For the Diameter Header, the command code namespace allocation has +11.2.1. Command Codes + + For the Diameter header, the Command Code namespace allocation has changed. The new allocation rules are as follows: - The command code values 256 - 8,388,607 (0x100 to 0x7fffff) are + The Command Code values 256 - 8,388,607 (0x100 to 0x7fffff) are for permanent, standard commands, allocated by IETF Review [RFC5226]. The values 8,388,608 - 16,777,213 (0x800000 - 0xfffffd) are - reserved for vendor-specific command codes, to be allocated on a + reserved for vendor-specific Command Codes, to be allocated on a First Come, First Served basis by IANA [RFC5226]. The request to IANA for a Vendor-Specific Command Code SHOULD include a reference - to a publicly available specification which documents the command + to a publicly available specification that documents the command in sufficient detail to aid in interoperability between independent implementations. If the specification cannot be made - publicly available, the request for a vendor-specific command code + publicly available, the request for a vendor-specific Command Code MUST include the contact information of persons and/or entities responsible for authoring and maintaining the command. + + + + + +Fajardo, et al. Standards Track [Page 136] + +RFC 6733 Diameter Base Protocol October 2012 + + + The values 16,777,214 and 16,777,215 (hexadecimal values 0xfffffe + - 0xffffff) are reserved for experimental commands. As these + codes are only for experimental and testing purposes, no guarantee + is made for interoperability between Diameter peers using + experimental commands. + +11.2.2. Command Flags + + Section 3 describes the existing Command Flags field. The remaining + bits can only be assigned via a Standards Action [RFC5226]. + 11.3. AVP Values For AVP values, the Experimental-Result-Code AVP value allocation has - been added. The new rule is as follows: + been added; see Section 11.3.1. The old AVP value allocation rule, + IETF Consensus, has been updated to IETF Review as per [RFC5226], and + affected AVPs are listed as reminders. 11.3.1. Experimental-Result-Code AVP Values for this AVP are purely local to the indicated vendor, and no IANA registry is maintained for them. +11.3.2. Result-Code AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.3. Accounting-Record-Type AVP Values -Fajardo, et al. Expires July 24, 2011 [Page 140] - -Internet-Draft Diameter Base Protocol January 2011 + New values are available for assignment via IETF Review [RFC5226]. + +11.3.4. Termination-Cause AVP Values + New values are available for assignment via IETF Review [RFC5226]. -11.4. Diameter TCP, SCTP, TLS/TCP and DTLS/SCTP Port Numbers +11.3.5. Redirect-Host-Usage AVP Values - Updated port number assignments are described in this section. The - IANA has assigned port number 3868 for TCP and SCTP. The port number - [TBD] has been assigned for TLS/TCP and DTLS/SCTP. + New values are available for assignment via IETF Review [RFC5226]. -11.5. S-NAPTR Parameters +11.3.6. Session-Server-Failover AVP Values - This document registers a new S-NAPTR Application Service Tag value - of "aaa". + New values are available for assignment via IETF Review [RFC5226]. - This document also registers the following S-NAPTR Application - Protocol Tags: +11.3.7. Session-Binding AVP Values - Tag | Protocol - -------------------|--------- - diameter.tcp | TCP - diameter.sctp | SCTP - diameter.tls.tcp | TLS/TCP - diameter.dtls.sctp | DTLS/SCTP + New values are available for assignment via IETF Review [RFC5226]. +Fajardo, et al. Standards Track [Page 137] + +RFC 6733 Diameter Base Protocol October 2012 + + +11.3.8. Disconnect-Cause AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.9. Auth-Request-Type AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.10. Auth-Session-State AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.11. Re-Auth-Request-Type AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.12. Accounting-Realtime-Required AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.13. Inband-Security-Id AVP (code 299) + The use of this AVP has been deprecated. +11.4. _diameters Service Name and Port Number Registration + IANA has registered the "_diameters" service name and assigned port + numbers for TLS/TCP and DTLS/SCTP according to the guidelines given + in [RFC6335]. + Service Name: _diameters + Transport Protocols: TCP, SCTP + Assignee: IESG <[email protected]> + Contact: IETF Chair <[email protected]> + Description: Diameter over TLS/TCP and DTLS/SCTP + Reference: RFC 6733 + Port Number: 5868, from the User Range -Fajardo, et al. Expires July 24, 2011 [Page 141] + + +Fajardo, et al. Standards Track [Page 138] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + +11.5. SCTP Payload Protocol Identifiers + + Two SCTP payload protocol identifiers have been registered in the + SCTP Payload Protocol Identifiers registry: + + + Value | SCTP Payload Protocol Identifier + -------|----------------------------------- + 46 | Diameter in a SCTP DATA chunk + 47 | Diameter in a DTLS/SCTP DATA chunk + +11.6. S-NAPTR Parameters -12. Diameter protocol related configurable parameters + The following tag has been registered in the S-NAPTR Application + Protocol Tags registry: + + Tag | Protocol + -------------------|--------- + diameter.dtls.sctp | DTLS/SCTP + +12. Diameter Protocol-Related Configurable Parameters This section contains the configurable parameters that are found throughout this document: @@ -7915,8 +7771,8 @@ Internet-Draft Diameter Base Protocol January 2011 A Diameter proxy server routes messages based on the realm portion of a Network Access Identifier (NAI). The server MUST have a table of Realm Names, and the address of the peer to which the - message must be forwarded to. The routing table MAY also include - a "default route", which is typically used for all messages that + message must be forwarded. The routing table MAY also include a + "default route", which is typically used for all messages that cannot be locally processed. Tc timer @@ -7927,30 +7783,9 @@ Internet-Draft Diameter Base Protocol January 2011 - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 142] +Fajardo, et al. Standards Track [Page 139] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 13. Security Considerations @@ -7959,34 +7794,34 @@ Internet-Draft Diameter Base Protocol January 2011 [RFC5246] or DTLS/SCTP [RFC6083]. Additional security mechanisms such as IPsec [RFC4301] MAY also be deployed to secure connections between peers. However, all Diameter base protocol implementations - MUST support the use of TLS/TCP and DTLS/SCTP and the Diameter - protocol MUST NOT be used without any security mechanism. + MUST support the use of TLS/TCP and DTLS/SCTP, and the Diameter + protocol MUST NOT be used without one of TLS, DTLS, or IPsec. If a Diameter connection is to be protected via TLS/TCP and DTLS/SCTP or IPsec, then TLS/TCP and DTLS/SCTP or IPsec/IKE SHOULD begin prior to any Diameter message exchange. All security parameters for TLS/ TCP and DTLS/SCTP or IPsec are configured independent of the Diameter - protocol. All Diameter message will be sent through the TLS/TCP and + protocol. All Diameter messages will be sent through the TLS/TCP and DTLS/SCTP or IPsec connection after a successful setup. For TLS/TCP and DTLS/SCTP connections to be established in the open state, the CER/CEA exchange MUST include an Inband-Security-ID AVP with a value of TLS/TCP and DTLS/SCTP. The TLS/TCP and DTLS/SCTP - handshake will begin when both ends successfully reached the open + handshake will begin when both ends successfully reach the open state, after completion of the CER/CEA exchange. If the TLS/TCP and DTLS/SCTP handshake is successful, all further messages will be sent - via TLS/TCP and DTLS/SCTP. If the handshake fails, both ends move to - the closed state. See Sections 13.1 for more details. + via TLS/TCP and DTLS/SCTP. If the handshake fails, both ends MUST + move to the closed state. See Section 13.1 for more details. 13.1. TLS/TCP and DTLS/SCTP Usage Diameter nodes using TLS/TCP and DTLS/SCTP for security MUST mutually authenticate as part of TLS/TCP and DTLS/SCTP session establishment. In order to ensure mutual authentication, the Diameter node acting as - TLS/TCP and DTLS/SCTP server MUST request a certificate from the + the TLS/TCP and DTLS/SCTP server MUST request a certificate from the Diameter node acting as TLS/TCP and DTLS/SCTP client, and the - Diameter node acting as TLS/TCP and DTLS/SCTP client MUST be prepared - to supply a certificate on request. + Diameter node acting as the TLS/TCP and DTLS/SCTP client MUST be + prepared to supply a certificate on request. Diameter nodes MUST be able to negotiate the following TLS/TCP and DTLS/SCTP cipher suites: @@ -8000,16 +7835,24 @@ Internet-Draft Diameter Base Protocol January 2011 TLS_RSA_WITH_AES_128_CBC_SHA - Diameter nodes MAY negotiate other TLS/TCP and DTLS/SCTP cipher -Fajardo, et al. Expires July 24, 2011 [Page 143] + +Fajardo, et al. Standards Track [Page 140] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + Note that it is quite possible that support for the + TLS_RSA_WITH_AES_128_CBC_SHA cipher suite will be REQUIRED at some + future date. Diameter nodes MAY negotiate other TLS/TCP and DTLS/ + SCTP cipher suites. - suites. + If public key certificates are used for Diameter security (for + example, with TLS), the value of the expiration times in the routing + and peer tables MUST NOT be greater than the expiry time in the + relevant certificates. 13.2. Peer-to-Peer Considerations @@ -8027,67 +7870,105 @@ Internet-Draft Diameter Base Protocol January 2011 peers may not be known beforehand and therefore peer discovery may be required. +13.3. AVP Considerations + Diameter AVPs often contain security-sensitive data; for example, + user passwords and location data, network addresses and cryptographic + keys. The following AVPs defined in this document are considered to + be security-sensitive: + o Acct-Interim-Interval + o Accounting-Realtime-Required + o Acct-Multi-Session-Id + o Accounting-Record-Number + o Accounting-Record-Type + o Accounting-Session-Id + o Accounting-Sub-Session-Id + o Class +Fajardo, et al. Standards Track [Page 141] + +RFC 6733 Diameter Base Protocol October 2012 + o Session-Id + o Session-Binding + o Session-Server-Failover + o User-Name + Diameter messages containing these or any other AVPs considered to be + security-sensitive MUST only be sent protected via mutually + authenticated TLS or IPsec. In addition, those messages MUST NOT be + sent via intermediate nodes unless there is end-to-end security + between the originator and recipient or the originator has locally + trusted configuration that indicates that end-to-end security is not + needed. For example, end-to-end security may not be required in the + case where an intermediary node is known to be operated as part of + the same administrative domain as the endpoints so that an ability to + successfully compromise the intermediary would imply a high + probability of being able to compromise the endpoints as well. Note + that no end-to-end security mechanism is specified in this document. +14. References +14.1. Normative References + [FLOATPOINT] + Institute of Electrical and Electronics Engineers, "IEEE + Standard for Binary Floating-Point Arithmetic, ANSI/IEEE + Standard 754-1985", August 1985. + [IANAADFAM] + IANA, "Address Family Numbers", + <http://www.iana.org/assignments/address-family-numbers>. + [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, + September 1981. + [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, + RFC 793, September 1981. + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + [RFC3492] Costello, A., "Punycode: A Bootstring encoding of Unicode + for Internationalized Domain Names in Applications + (IDNA)", RFC 3492, March 2003. - - -Fajardo, et al. Expires July 24, 2011 [Page 144] +Fajardo, et al. Standards Track [Page 142] -Internet-Draft Diameter Base Protocol January 2011 - - -14. References - -14.1. Normative References - - [FLOATPOINT] - Institute of Electrical and Electronics Engineers, "IEEE - Standard for Binary Floating-Point Arithmetic, ANSI/IEEE - Standard 754-1985", August 1985. +RFC 6733 Diameter Base Protocol October 2012 - [IANAADFAM] - IANA,, "Address Family Numbers", - http://www.iana.org/assignments/address-family-numbers. - [RADTYPE] IANA,, "RADIUS Types", - http://www.iana.org/assignments/radius-types. + [RFC3539] Aboba, B. and J. Wood, "Authentication, Authorization and + Accounting (AAA) Transport Profile", RFC 3539, June 2003. - [RFC791] Postel, J., "Internet Protocol", RFC 791, September 1981. + [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO + 10646", STD 63, RFC 3629, November 2003. - [RFC793] Postel, J., "Transmission Control Protocol", RFC 793, - January 1981. + [RFC3958] Daigle, L. and A. Newton, "Domain-Based Application + Service Location Using SRV RRs and the Dynamic Delegation + Discovery Service (DDDS)", RFC 3958, January 2005. - [RFC3539] Aboba, B. and J. Wood, "Authentication, Authorization and - Accounting (AAA) Transport Profile", RFC 3539, June 2003. + [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform + Resource Identifier (URI): Generic Syntax", STD 66, + RFC 3986, January 2005. [RFC4004] Calhoun, P., Johansson, T., Perkins, C., Hiller, T., and P. McCann, "Diameter Mobile IPv4 Application", RFC 4004, @@ -8101,51 +7982,44 @@ Internet-Draft Diameter Base Protocol January 2011 Loughney, "Diameter Credit-Control Application", RFC 4006, August 2005. - [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax - Specifications: ABNF", STD 68, RFC 5234, January 2008. - - [RFC3588] Calhoun, P., Loughney, J., Guttman, E., Zorn, G., and J. - Arkko, "Diameter Base Protocol", RFC 3588, September 2003. + [RFC4086] Eastlake, D., Schiller, J., and S. Crocker, "Randomness + Requirements for Security", BCP 106, RFC 4086, June 2005. - [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an - IANA Considerations Section in RFCs", BCP 26, RFC 5226, - May 2008. + [RFC4282] Aboba, B., Beadles, M., Arkko, J., and P. Eronen, "The + Network Access Identifier", RFC 4282, December 2005. [RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture", RFC 4291, February 2006. + [RFC4960] Stewart, R., "Stream Control Transmission Protocol", + RFC 4960, September 2007. + [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an + IANA Considerations Section in RFCs", BCP 26, RFC 5226, + May 2008. -Fajardo, et al. Expires July 24, 2011 [Page 145] - -Internet-Draft Diameter Base Protocol January 2011 - + [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax + Specifications: ABNF", STD 68, RFC 5234, January 2008. - [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate - Requirement Levels", BCP 14, RFC 2119, March 1997. + [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security + (TLS) Protocol Version 1.2", RFC 5246, August 2008. - [RFC4282] Aboba, B., Beadles, M., Arkko, J., and P. Eronen, "The - Network Access Identifier", RFC 4282, December 2005. - [RFC4086] Eastlake, D., Schiller, J., and S. Crocker, "Randomness - Requirements for Security", BCP 106, RFC 4086, June 2005. - [RFC4960] Stewart, R., "Stream Control Transmission Protocol", - RFC 4960, September 2007. - [RFC3958] Daigle, L. and A. Newton, "Domain-Based Application - Service Location Using SRV RRs and the Dynamic Delegation - Discovery Service (DDDS)", RFC 3958, January 2005. +Fajardo, et al. Standards Track [Page 143] + +RFC 6733 Diameter Base Protocol October 2012 - [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security - (TLS) Protocol Version 1.2", RFC 5246, August 2008. - [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform - Resource Identifier (URI): Generic Syntax", STD 66, - RFC 3986, January 2005. + [RFC5280] Cooper, D., Santesson, S., Farrell, S., Boeyen, S., + Housley, R., and W. Polk, "Internet X.509 Public Key + Infrastructure Certificate and Certificate Revocation List + (CRL) Profile", RFC 5280, May 2008. - [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO - 10646", STD 63, RFC 3629, November 2003. + [RFC5729] Korhonen, J., Jones, M., Morand, L., and T. Tsou, + "Clarifications on the Routing of Diameter Requests Based + on the Username and the Realm", RFC 5729, December 2009. [RFC5890] Klensin, J., "Internationalized Domain Names for Applications (IDNA): Definitions and Document Framework", @@ -8154,103 +8028,120 @@ Internet-Draft Diameter Base Protocol January 2011 [RFC5891] Klensin, J., "Internationalized Domain Names in Applications (IDNA): Protocol", RFC 5891, August 2010. - [RFC3492] Costello, A., "Punycode: A Bootstring encoding of Unicode - for Internationalized Domain Names in Applications - (IDNA)", RFC 3492, March 2003. - - [RFC5729] Korhonen, J., Jones, M., Morand, L., and T. Tsou, - "Clarifications on the Routing of Diameter Requests Based - on the Username and the Realm", RFC 5729, December 2009. - - [RFC4347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer - Security", RFC 4347, April 2006. - [RFC6083] Tuexen, M., Seggelmann, R., and E. Rescorla, "Datagram Transport Layer Security (DTLS) for Stream Control Transmission Protocol (SCTP)", RFC 6083, January 2011. + [RFC6347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer + Security Version 1.2", RFC 6347, January 2012. + [RFC6408] Jones, M., Korhonen, J., and L. Morand, "Diameter + Straightforward-Naming Authority Pointer (S-NAPTR) Usage", + RFC 6408, November 2011. +14.2. Informative References -Fajardo, et al. Expires July 24, 2011 [Page 146] - -Internet-Draft Diameter Base Protocol January 2011 + [ENTERPRISE] IANA, "SMI Network Management Private Enterprise + Codes", + <http://www.iana.org/assignments/enterprise-numbers>. + [IANATCV] IANA, "Termination-Cause AVP Values (code 295)", + <http://www.iana.org/assignments/aaa-parameters/ + aaa-parameters.xml#aaa-parameters-16>. -14.2. Informational References + [RFC1492] Finseth, C., "An Access Control Protocol, Sometimes + Called TACACS", RFC 1492, July 1993. - [RFC2989] Aboba, B., Calhoun, P., Glass, S., Hiller, T., McCann, P., - Shiino, H., Walsh, P., Zorn, G., Dommety, G., Perkins, C., - Patil, B., Mitton, D., Manning, S., Beadles, M., Chen, X., - Sivalingham, S., Hameed, A., Munson, M., Jacobs, S., Lim, - B., Hirschman, B., Hsu, R., Koo, H., Lipford, M., - Campbell, E., Xu, Y., Baba, S., and E. Jaques, "Criteria - for Evaluating AAA Protocols for Network Access", - RFC 2989, November 2000. + [RFC1661] Simpson, W., "The Point-to-Point Protocol (PPP)", + STD 51, RFC 1661, July 1994. - [RFC2975] Aboba, B., Arkko, J., and D. Harrington, "Introduction to - Accounting Management", RFC 2975, October 2000. + [RFC2104] Krawczyk, H., Bellare, M., and R. Canetti, "HMAC: + Keyed-Hashing for Message Authentication", RFC 2104, + February 1997. - [RFC3232] Reynolds, J., "Assigned Numbers: RFC 1700 is Replaced by - an On-line Database", RFC 3232, January 2002. - [RFC5176] Chiba, M., Dommety, G., Eklund, M., Mitton, D., and B. - Aboba, "Dynamic Authorization Extensions to Remote - Authentication Dial In User Service (RADIUS)", RFC 5176, - January 2008. - [RFC1661] Simpson, W., "The Point-to-Point Protocol (PPP)", STD 51, - RFC 1661, July 1994. - [RFC2866] Rigney, C., "RADIUS Accounting", RFC 2866, June 2000. - [RFC2869] Rigney, C., Willats, W., and P. Calhoun, "RADIUS - Extensions", RFC 2869, June 2000. +Fajardo, et al. Standards Track [Page 144] + +RFC 6733 Diameter Base Protocol October 2012 - [RFC2865] Rigney, C., Willens, S., Rubens, A., and W. Simpson, - "Remote Authentication Dial In User Service (RADIUS)", - RFC 2865, June 2000. - [RFC3162] Aboba, B., Zorn, G., and D. Mitton, "RADIUS and IPv6", - RFC 3162, August 2001. + [RFC2782] Gulbrandsen, A., Vixie, P., and L. Esibov, "A DNS RR + for specifying the location of services (DNS SRV)", + RFC 2782, February 2000. - [RFC4301] Kent, S. and K. Seo, "Security Architecture for the - Internet Protocol", RFC 4301, December 2005. + [RFC2865] Rigney, C., Willens, S., Rubens, A., and W. Simpson, + "Remote Authentication Dial In User Service (RADIUS)", + RFC 2865, June 2000. - [RFC5905] Mills, D., Martin, J., Burbank, J., and W. Kasch, "Network - Time Protocol Version 4: Protocol and Algorithms - Specification", RFC 5905, June 2010. + [RFC2866] Rigney, C., "RADIUS Accounting", RFC 2866, June 2000. - [RFC1492] Finseth, C., "An Access Control Protocol, Sometimes Called - TACACS", RFC 1492, July 1993. + [RFC2869] Rigney, C., Willats, W., and P. Calhoun, "RADIUS + Extensions", RFC 2869, June 2000. - [RFC4690] Klensin, J., Faltstrom, P., Karp, C., and IAB, "Review and + [RFC2881] Mitton, D. and M. Beadles, "Network Access Server + Requirements Next Generation (NASREQNG) NAS Model", + RFC 2881, July 2000. + [RFC2975] Aboba, B., Arkko, J., and D. Harrington, "Introduction + to Accounting Management", RFC 2975, October 2000. + [RFC2989] Aboba, B., Calhoun, P., Glass, S., Hiller, T., McCann, + P., Shiino, H., Walsh, P., Zorn, G., Dommety, G., + Perkins, C., Patil, B., Mitton, D., Manning, S., + Beadles, M., Chen, X., Sivalingham, S., Hameed, A., + Munson, M., Jacobs, S., Lim, B., Hirschman, B., Hsu, + R., Koo, H., Lipford, M., Campbell, E., Xu, Y., Baba, + S., and E. Jaques, "Criteria for Evaluating AAA + Protocols for Network Access", RFC 2989, November 2000. -Fajardo, et al. Expires July 24, 2011 [Page 147] - -Internet-Draft Diameter Base Protocol January 2011 + [RFC3162] Aboba, B., Zorn, G., and D. Mitton, "RADIUS and IPv6", + RFC 3162, August 2001. + [RFC3748] Aboba, B., Blunk, L., Vollbrecht, J., Carlson, J., and + H. Levkowetz, "Extensible Authentication Protocol + (EAP)", RFC 3748, June 2004. - Recommendations for Internationalized Domain Names - (IDNs)", RFC 4690, September 2006. + [RFC4301] Kent, S. and K. Seo, "Security Architecture for the + Internet Protocol", RFC 4301, December 2005. - [RFC5461] Gont, F., "TCP's Reaction to Soft Errors", RFC 5461, - February 2009. + [RFC4690] Klensin, J., Faltstrom, P., Karp, C., and IAB, "Review + and Recommendations for Internationalized Domain Names + (IDNs)", RFC 4690, September 2006. - [RFC5927] Gont, F., "ICMP Attacks against TCP", RFC 5927, July 2010. + [RFC5176] Chiba, M., Dommety, G., Eklund, M., Mitton, D., and B. + Aboba, "Dynamic Authorization Extensions to Remote + Authentication Dial In User Service (RADIUS)", + RFC 5176, January 2008. - [RFC3692] Narten, T., "Assigning Experimental and Testing Numbers - Considered Useful", BCP 82, RFC 3692, January 2004. +Fajardo, et al. Standards Track [Page 145] + +RFC 6733 Diameter Base Protocol October 2012 + + [RFC5461] Gont, F., "TCP's Reaction to Soft Errors", RFC 5461, + February 2009. + [RFC5905] Mills, D., Martin, J., Burbank, J., and W. Kasch, + "Network Time Protocol Version 4: Protocol and + Algorithms Specification", RFC 5905, June 2010. + [RFC5927] Gont, F., "ICMP Attacks against TCP", RFC 5927, + July 2010. + [RFC6335] Cotton, M., Eggert, L., Touch, J., Westerlund, M., and + S. Cheshire, "Internet Assigned Numbers Authority + (IANA) Procedures for the Management of the Service + Name and Transport Protocol Port Number Registry", + BCP 165, RFC 6335, August 2011. + [RFC6737] Kang, J. and G. Zorn, "The Diameter Capabilities Update + Application", RFC 6737, October 2012. @@ -8284,80 +8175,83 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 148] +Fajardo, et al. Standards Track [Page 146] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Appendix A. Acknowledgements -A.1. RFC3588bis +A.1. This Document The authors would like to thank the following people that have provided proposals and contributions to this document: To Vishnu Ram and Satendra Gera for their contributions on - Capabilities Updates, Predictive Loop Avoidance as well as many other - technical proposals. To Tolga Asveren for his insights and + capabilities updates, predictive loop avoidance, as well as many + other technical proposals. To Tolga Asveren for his insights and contributions on almost all of the proposed solutions incorporated - into this document. To Timothy Smith for helping on the Capabilities - Updates and other topics. To Tony Zhang for providing fixes to loop - holes on composing Failed-AVPs as well as many other issues and + into this document. To Timothy Smith for helping on the capabilities + Update and other topics. To Tony Zhang for providing fixes to + loopholes on composing Failed-AVPs as well as many other issues and topics. To Jan Nordqvist for clearly stating the usage of - Application Ids. To Anders Kristensen for providing needed technical + Application Ids. To Anders Kristensen for providing needed technical opinions. To David Frascone for providing invaluable review of the document. To Mark Jones for providing clarifying text on vendor - command codes and other vendor specific indicators. + command codes and other vendor-specific indicators. To Victor + Pascual and Sebastien Decugis for new text and recommendations on + SCTP/DTLS. To Jouni Korhonen for taking over the editing task and + resolving last bits from versions 27 through 29. - Special thanks to the Diameter extensibility design team which helped - resolve the tricky question of mandatory AVPs and ABNF semantics. - The members of this team are as follows: + Special thanks to the Diameter extensibility design team, which + helped resolve the tricky question of mandatory AVPs and ABNF + semantics. The members of this team are as follows: Avi Lior, Jari Arkko, Glen Zorn, Lionel Morand, Mark Jones, Tolga - Asveren Jouni Korhonen, Glenn McGregor. + Asveren, Jouni Korhonen, and Glenn McGregor. Special thanks also to people who have provided invaluable comments and inputs especially in resolving controversial issues: - Glen Zorn, Yoshihiro Ohba, Marco Stura, and Pasi Eronen. + Glen Zorn, Yoshihiro Ohba, Marco Stura, Stephen Farrel, Pete Resnick, + Peter Saint-Andre, Robert Sparks, Krishna Prasad, Sean Turner, Barry + Leiba, and Pasi Eronen. Finally, we would like to thank the original authors of this document: - Pat Calhoun, John Loughney, Jari Arkko, Erik Guttman and Glen Zorn. + Pat Calhoun, John Loughney, Jari Arkko, Erik Guttman, and Glen Zorn. Their invaluable knowledge and experience has given us a robust and flexible AAA protocol that many people have seen great value in adopting. We greatly appreciate their support and stewardship for the continued improvements of Diameter as a protocol. We would also like to extend our gratitude to folks aside from the authors who have - assisted and contributed to the original version of this document. - Their efforts significantly contributed to the success of Diameter. - - - -Fajardo, et al. Expires July 24, 2011 [Page 149] +Fajardo, et al. Standards Track [Page 147] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 -A.2. RFC3588 + assisted and contributed to the original version of this document. + Their efforts significantly contributed to the success of Diameter. + +A.2. RFC 3588 The authors would like to thank Nenad Trifunovic, Tony Johansson and Pankaj Patel for their participation in the pre-IETF Document Reading - Party. Allison Mankin, Jonathan Wood and Bernard Aboba provided - invaluable assistance in working out transport issues, and similarly - with Steven Bellovin in the security area. + Party. Allison Mankin, Jonathan Wood, and Bernard Aboba provided + invaluable assistance in working out transport issues and this was + also the case with Steven Bellovin in the security area. Paul Funk and David Mitton were instrumental in getting the Peer State Machine correct, and our deep thanks go to them for their time. Text in this document was also provided by Paul Funk, Mark Eklund, - Mark Jones and Dave Spence. Jacques Caron provided many great + Mark Jones, and Dave Spence. Jacques Caron provided many great comments as a result of a thorough review of the spec. The authors would also like to acknowledge the following people for @@ -8367,96 +8261,57 @@ A.2. RFC3588 David Frascone, Daniel C. Fox, Lol Grant, Ignacio Goyret, Nancy Greene, Peter Heitman, Fredrik Johansson, Mark Jones, Martin Julien, Bob Kopacz, Paul Krumviede, Fergal Ladley, Ryan Moats, Victor Muslin, - Kenneth Peirce, John Schnizlein, Sumit Vakil, John R. Vollbrecht and + Kenneth Peirce, John Schnizlein, Sumit Vakil, John R. Vollbrecht, and Jeff Weisberg. Finally, Pat Calhoun would like to thank Sun Microsystems since most of the effort put into this document was done while he was in their employ. - - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 150] - -Internet-Draft Diameter Base Protocol January 2011 - - Appendix B. S-NAPTR Example As an example, consider a client that wishes to resolve aaa: - example1.com. The client performs a NAPTR query for that domain, and - the following NAPTR records are returned: + ex1.example.com. The client performs a NAPTR query for that domain, + and the following NAPTR records are returned: ;; order pref flags service regexp replacement IN NAPTR 50 50 "s" "aaa:diameter.tls.tcp" "" - _diameter._tls.example1.com + _diameter._tls.ex1.example.com IN NAPTR 100 50 "s" "aaa:diameter.tcp" "" - _aaa._tcp.example1.com + _aaa._tcp.ex1.example.com IN NAPTR 150 50 "s" "aaa:diameter.sctp" "" - _diameter._sctp.example1.com + _diameter._sctp.ex1.example.com - This indicates that the server supports TLS, TCP and SCTP in that + This indicates that the server supports TLS, TCP, and SCTP in that order. If the client supports TLS, TLS will be used, targeted to a - host determined by an SRV lookup of _diameter._tls.example1.com. + + + +Fajardo, et al. Standards Track [Page 148] + +RFC 6733 Diameter Base Protocol October 2012 + + + host determined by an SRV lookup of _diameter._tls.ex1.example.com. That lookup would return: ;; Priority Weight Port Target - IN SRV 0 1 5060 server1.example1.com - IN SRV 0 2 5060 server2.example1.com + IN SRV 0 1 5060 server1.ex1.example.com + IN SRV 0 2 5060 server2.ex1.example.com As an alternative example, a client that wishes to resolve aaa: - example2.com. The client performs a NAPTR query for that domain, and - the following NAPTR records are returned: + ex2.example.com. The client performs a NAPTR query for that domain, + and the following NAPTR records are returned: ;; order pref flags service regexp replacement IN NAPTR 150 50 "a" "aaa:diameter.tls.tcp" "" - server1.example2.com + server1.ex2.example.com IN NAPTR 150 50 "a" "aaa:diameter.tls.tcp" "" - server2.example2.com + server2.ex2.example.com This indicates that the server supports TCP available at the returned host names. - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 151] - -Internet-Draft Diameter Base Protocol January 2011 - - Appendix C. Duplicate Detection As described in Section 9.4, accounting record duplicate detection is @@ -8464,14 +8319,14 @@ Appendix C. Duplicate Detection reasons: o Failover to an alternate server. Where close to real-time - performance is required, failover thresholds need to be kept low - and this may lead to an increased likelihood of duplicates. - Failover can occur at the client or within Diameter agents. - - o Failure of a client or agent after sending of a record from non- - volatile memory, but prior to receipt of an application layer ACK - and deletion of the record. record to be sent. This will result - in retransmission of the record soon after the client or agent has + performance is required, failover thresholds need to be kept low. + This may lead to an increased likelihood of duplicates. Failover + can occur at the client or within Diameter agents. + + o Failure of a client or agent after sending a record from non- + volatile memory, but prior to receipt of an application-layer ACK + and deletion of the record to be sent. This will result in + retransmission of the record soon after the client or agent has rebooted. o Duplicates received from RADIUS gateways. Since the @@ -8481,48 +8336,48 @@ Appendix C. Duplicate Detection o Implementation problems and misconfiguration. - The T flag is used as an indication of an application layer + The T flag is used as an indication of an application-layer retransmission event, e.g., due to failover to an alternate server. It is defined only for request messages sent by Diameter clients or agents. For instance, after a reboot, a client may not know whether + + + +Fajardo, et al. Standards Track [Page 149] + +RFC 6733 Diameter Base Protocol October 2012 + + it has already tried to send the accounting records in its non- volatile memory before the reboot occurred. Diameter servers MAY use the T flag as an aid when processing requests and detecting duplicate messages. However, servers that do this MUST ensure that duplicates are found even when the first transmitted request arrives at the server after the retransmitted request. It can be used only in cases - where no answer has been received from the Server for a request and + where no answer has been received from the server for a request and the request is sent again, (e.g., due to a failover to an alternate peer, due to a recovered primary peer or due to a client re-sending a stored record from non-volatile memory such as after reboot of a client or agent). - In some cases the Diameter accounting server can delay the duplicate + In some cases, the Diameter accounting server can delay the duplicate detection and accounting record processing until a post-processing phase takes place. At that time records are likely to be sorted according to the included User-Name and duplicate elimination is easy - in this case. In other situations it may be necessary to perform + in this case. In other situations, it may be necessary to perform real-time duplicate detection, such as when credit limits are imposed or real-time fraud detection is desired. - - - -Fajardo, et al. Expires July 24, 2011 [Page 152] - -Internet-Draft Diameter Base Protocol January 2011 - - In general, only generation of duplicates due to failover or re- sending of records in non-volatile storage can be reliably detected - by Diameter clients or agents. In such cases the Diameter client or - agents can mark the message as possible duplicate by setting the T + by Diameter clients or agents. In such cases, the Diameter client or + agents can mark the message as a possible duplicate by setting the T flag. Since the Diameter server is responsible for duplicate - detection, it can choose to make use of the T flag or not, in order - to optimize duplicate detection. Since the T flag does not affect - interoperability, and may not be needed by some servers, generation - of the T flag is REQUIRED for Diameter clients and agents, but MAY be - implemented by Diameter servers. + detection, it can choose whether or not to make use of the T flag, in + order to optimize duplicate detection. Since the T flag does not + affect interoperability, and it may not be needed by some servers, + generation of the T flag is REQUIRED for Diameter clients and agents, + but it MAY be implemented by Diameter servers. As an example, it can be usually be assumed that duplicates appear within a time window of longest recorded network partition or device @@ -8535,40 +8390,39 @@ Internet-Draft Diameter Base Protocol January 2011 The following is an example of how the T flag may be used by the server to detect duplicate requests. - A Diameter server MAY check the T flag of the received message to determine if the record is a possible duplicate. If the T flag is set in the request message, the server searches for a duplicate within a configurable duplication time window backward and forward. This limits database searching to those records where - the T flag is set. In a well run network, network partitions and + the T flag is set. In a well-run network, network partitions and + + + +Fajardo, et al. Standards Track [Page 150] + +RFC 6733 Diameter Base Protocol October 2012 + + device faults will presumably be rare events, so this approach represents a substantial optimization of the duplicate detection process. During failover, it is possible for the original record - to be received after the T flag marked record, due to differences + to be received after the T-flag-marked record, due to differences in network delays experienced along the path by the original and duplicate transmissions. The likelihood of this occurring increases as the failover interval is decreased. In order to be - able to detect out of order duplicates, the Diameter server should - use backward and forward time windows when performing duplicate - checking for the T flag marked request. For example, in order to - allow time for the original record to exit the network and be - recorded by the accounting server, the Diameter server can delay - processing records with the T flag set until a time period - TIME_WAIT + RECORD_PROCESSING_TIME has elapsed after the closing - of the original transport connection. After this time period has - expired, then it may check the T flag marked records against the + able to detect duplicates that are out of order, the Diameter + server should use backward and forward time windows when + performing duplicate checking for the T-flag-marked request. For + example, in order to allow time for the original record to exit + the network and be recorded by the accounting server, the Diameter + server can delay processing records with the T flag set until a + time period TIME_WAIT + RECORD_PROCESSING_TIME has elapsed after + the closing of the original transport connection. After this time + period, it may check the T-flag-marked records against the database with relative assurance that the original records, if sent, have been received and recorded. - - - -Fajardo, et al. Expires July 24, 2011 [Page 153] - -Internet-Draft Diameter Base Protocol January 2011 - - Appendix D. Internationalized Domain Names To be compatible with the existing DNS infrastructure and simplify @@ -8579,14 +8433,7 @@ Appendix D. Internationalized Domain Names recommendations in [RFC4690] and [RFC5890]. Applications that provide support for IDNs outside of the Diameter protocol but interacting with it SHOULD use the representation and conversion - framework described in [RFC5890], [RFC5891] and [RFC3492]. - - - - - - - + framework described in [RFC5890], [RFC5891], and [RFC3492]. @@ -8608,21 +8455,9 @@ Appendix D. Internationalized Domain Names - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 154] +Fajardo, et al. Standards Track [Page 151] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Authors' Addresses @@ -8634,7 +8469,7 @@ Authors' Addresses USA Phone: +1-908-421-1845 - Email: [email protected] + EMail: [email protected] Jari Arkko @@ -8643,7 +8478,7 @@ Authors' Addresses Finland Phone: +358 40 5079256 - Email: [email protected] + EMail: [email protected] John Loughney @@ -8653,17 +8488,17 @@ Authors' Addresses US Phone: +1-650-283-8068 - Email: [email protected] + EMail: [email protected] - Glenn Zorn + Glen Zorn (editor) Network Zen - 1310 East Thomas Street - Seattle, WA 98102 - US + 227/358 Thanon Sanphawut + Bang Na, Bangkok 10260 + Thailand - Phone: - Email: [email protected] + Phone: +66 (0) 87-0404617 + EMail: [email protected] @@ -8676,6 +8511,5 @@ Authors' Addresses -Fajardo, et al. Expires July 24, 2011 [Page 155] +Fajardo, et al. Standards Track [Page 152] - diff --git a/lib/diameter/doc/standard/rfc6737.txt b/lib/diameter/doc/standard/rfc6737.txt new file mode 100644 index 0000000000..50aa33e98f --- /dev/null +++ b/lib/diameter/doc/standard/rfc6737.txt @@ -0,0 +1,339 @@ + + + + + + +Internet Engineering Task Force (IETF) K. Jiao +Request for Comments: 6737 Huawei +Category: Standards Track G. Zorn +ISSN: 2070-1721 Network Zen + October 2012 + + + The Diameter Capabilities Update Application + +Abstract + + This document defines a new Diameter application and associated + Command Codes. The Capabilities Update application is intended to + allow the dynamic update of certain Diameter peer capabilities while + the peer-to-peer connection is in the open state. + +Status of This Memo + + This is an Internet Standards Track document. + + This document is a product of the Internet Engineering Task Force + (IETF). It represents the consensus of the IETF community. It has + received public review and has been approved for publication by the + Internet Engineering Steering Group (IESG). Further information on + Internet Standards is available in Section 2 of RFC 5741. + + Information about the current status of this document, any errata, + and how to provide feedback on it may be obtained at + http://www.rfc-editor.org/info/rfc6737. + +Copyright Notice + + Copyright (c) 2012 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents + (http://trustee.ietf.org/license-info) in effect on the date of + publication of this document. Please review these documents + carefully, as they describe your rights and restrictions with respect + to this document. Code Components extracted from this document must + include Simplified BSD License text as described in Section 4.e of + the Trust Legal Provisions and are provided without warranty as + described in the Simplified BSD License. + + + + + + + +Jiao & Zorn Standards Track [Page 1] + +RFC 6737 Diameter Capabilities Update October 2012 + + +Table of Contents + + 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2 + 2. Specification of Requirements . . . . . . . . . . . . . . . . . 2 + 3. Diameter Protocol Considerations . . . . . . . . . . . . . . . 3 + 4. Capabilities Update . . . . . . . . . . . . . . . . . . . . . . 3 + 4.1. Command Code Values . . . . . . . . . . . . . . . . . . . . 4 + 4.1.1. Capabilities-Update-Request . . . . . . . . . . . . . . 4 + 4.1.2. Capabilities-Update-Answer . . . . . . . . . . . . . . 5 + 5. Security Considerations . . . . . . . . . . . . . . . . . . . . 5 + 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 5 + 6.1. Application Identifier . . . . . . . . . . . . . . . . . . 5 + 6.2. Command Codes . . . . . . . . . . . . . . . . . . . . . . . 5 + 7. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 5 + 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 5 + 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 6 + 9.1. Normative References . . . . . . . . . . . . . . . . . . . 6 + 9.2. Informative References . . . . . . . . . . . . . . . . . . 6 + +1. Introduction + + Capabilities exchange is an important component of the Diameter base + protocol [RFC6733], allowing peers to exchange identities and + Diameter capabilities (protocol version number, supported Diameter + applications, security mechanisms, etc.). As defined in RFC 3588, + however, the capabilities exchange process takes place only once, at + the inception of a transport connection between a given pair of + peers. Therefore, if a peer's capabilities change (due to a software + update, for example), the existing connection(s) must be torn down + (along with all of the associated user sessions) and restarted before + the modified capabilities can be advertised. + + This document defines a new Diameter application intended to allow + the dynamic update of a subset of Diameter peer capabilities over an + existing connection. Because the Capabilities Update application + specified herein operates over an existing transport connection, + modification of certain capabilities is prohibited. Specifically, + modifying the security mechanism in use is not allowed; if the + security method used between a pair of peers is changed, the affected + connection MUST be restarted. + +2. Specification of Requirements + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in RFC 2119 [RFC2119]. + + + + + +Jiao & Zorn Standards Track [Page 2] + +RFC 6737 Diameter Capabilities Update October 2012 + + +3. Diameter Protocol Considerations + + This section details the relationship of the Diameter Capabilities + Update application to the Diameter base protocol. + + This document specifies Diameter Application-Id 10. Diameter nodes + conforming to this specification MUST advertise support by including + the value 10 in the Auth-Application-Id of the Capabilities-Exchange- + Request (CER) and Capabilities-Exchange-Answer (CEA) commands + [RFC6733]. + +4. Capabilities Update + + When the capabilities of a Diameter node conforming to this + specification change, the node MUST notify all of the nodes with + which it has an open transport connection and which have also + advertised support for the Capabilities Update application using the + Capabilities-Update-Request (CUR) message (Section 4.1.1). This + message allows the update of a peer's capabilities (supported + Diameter applications, etc.). + + A Diameter node only issues a given command to those peers that have + advertised support for the Diameter application that defines the + command; a Diameter node must cache the supported applications in + order to ensure that unrecognized commands and/or Attribute-Value + Pairs (AVPs) are not unnecessarily sent to a peer. + + The receiver of the CUR MUST determine common applications by + computing the intersection of its own set of supported Application + Ids against all of the Application-Id AVPs (Auth-Application-Id, + Acct-Application-Id, and Vendor-Specific-Application-Id) present in + the CUR. The value of the Vendor-Id AVP in the Vendor-Specific- + Application-Id MUST NOT be used during computation. + + If the receiver of a CUR does not have any applications in common + with the sender, then it MUST return a Capabilities-Update-Answer + (CUA) (Section 4.1.2) with the Result-Code AVP set to + DIAMETER_NO_COMMON_APPLICATION [RFC6733], and it SHOULD disconnect + the transport-layer connection. However, if active sessions are + using the connection, peers MAY delay disconnection until the + sessions can be redirected or gracefully terminated. Note that + receiving a CUA from a peer advertising itself as a relay (see + [RFC6733], Section 2.4) MUST be interpreted as having common + applications with the peer. + + As for CER/CEA messages, the CUR and CUA messages MUST NOT be + proxied, redirected, or relayed. + + + + +Jiao & Zorn Standards Track [Page 3] + +RFC 6737 Diameter Capabilities Update October 2012 + + + Even though the CUR/CUA messages cannot be proxied, it is still + possible for an upstream agent to receive a message for which there + are no peers available to handle the application that corresponds to + the Command Code. This could happen if, for example, the peers are + too busy or down. In such instances, the 'E' bit MUST be set in the + answer message with the Result-Code AVP set to + DIAMETER_UNABLE_TO_DELIVER to inform the downstream peer to take + action (e.g., re-routing requests to an alternate peer). + +4.1. Command Code Values + + This section defines Command Code [RFC6733] values that MUST be + supported by all Diameter implementations conforming to this + specification. The following Command Codes are defined in this + document: Capabilities-Update-Request (CUR, Section 4.1.1), and + Capabilities-Update-Answer (CUA, Section 4.1.2). The Diameter + Command Code Format (CCF) ([RFC6733], Section 3.2) is used in the + definitions. + +4.1.1. Capabilities-Update-Request + + The Capabilities-Update-Request (CUR), indicated by the Command Code + set to 328 and the Command Flags' 'R' bit set, is sent to update + local capabilities. Upon detection of a transport failure, this + message MUST NOT be sent to an alternate peer. + + When Diameter is run over the Stream Control Transmission Protocol + (SCTP) [RFC4960], which allows connections to span multiple + interfaces and multiple IP addresses, the Capabilities-Update-Request + message MUST contain one Host-IP-Address AVP for each potential IP + address that may be locally used when transmitting Diameter messages. + + Message Format + + <CUR> ::= < Diameter Header: 328, REQ > + { Origin-Host } + { Origin-Realm } + 1* { Host-IP-Address } + { Vendor-Id } + { Product-Name } + [ Origin-State-Id ] + * [ Supported-Vendor-Id ] + * [ Auth-Application-Id ] + * [ Acct-Application-Id ] + * [ Vendor-Specific-Application-Id ] + [ Firmware-Revision ] + * [ AVP ] + + + + +Jiao & Zorn Standards Track [Page 4] + +RFC 6737 Diameter Capabilities Update October 2012 + + +4.1.2. Capabilities-Update-Answer + + The Capabilities-Update-Answer, indicated by the Command Code set to + 328 and the Command Flags' 'R' bit cleared, is sent in response to a + CUR message. + + Message Format + + <CUA> ::= < Diameter Header: 328 > + { Origin-Host } + { Origin-Realm } + { Result-Code } + [ Error-Message ] + * [ AVP ] + +5. Security Considerations + + The security considerations applicable to the Diameter base protocol + [RFC6733] are also applicable to this document. + +6. IANA Considerations + + This section explains the criteria to be used by the IANA for + assignment of numbers within namespaces used within this document. + +6.1. Application Identifier + + This specification assigns the value 10 (Diameter Capabilities + Update) from the Application Identifiers namespace [RFC6733]. See + Section 3 for the assignment of the namespace in this specification. + +6.2. Command Codes + + This specification assigns the value 328 (Capabilities-Update- + Request/Capabilities-Update-Answer (CUR/CUA)) from the Command Codes + namespace [RFC6733]. See Section 4.1 for the assignment of the + namespace in this specification. + +7. Contributors + + This document is based upon work done by Tina Tsou. + +8. Acknowledgements + + Thanks to Sebastien Decugis, Niklas Neumann, Subash Comerica, Lionel + Morand, Dan Romascanu, Dan Harkins, and Ravi for helpful review and + discussion. + + + + +Jiao & Zorn Standards Track [Page 5] + +RFC 6737 Diameter Capabilities Update October 2012 + + +9. References + +9.1. Normative References + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + + [RFC6733] Fajardo, V., Arkko, J., Loughney, J., and G. Zorn, + "Diameter Base Protocol", RFC 6733, October 2012. + +9.2. Informative References + + [RFC4960] Stewart, R., "Stream Control Transmission Protocol", + RFC 4960, September 2007. + +Authors' Addresses + + Jiao Kang + Huawei Technologies + Section F1, Huawei Industrial Base + Bantian, Longgang District + Shenzhen 518129 + P.R. China + + EMail: [email protected] + + + Glen Zorn + Network Zen + 227/358 Thanon Sanphawut + Bang Na, Bangkok 10260 + Thailand + + Phone: +66 (0) 909-201060 + EMail: [email protected] + + + + + + + + + + + + + + + + +Jiao & Zorn Standards Track [Page 6] + diff --git a/lib/diameter/src/base/diameter_capx.erl b/lib/diameter/src/base/diameter_capx.erl index 190d37262b..c6c3d2934d 100644 --- a/lib/diameter/src/base/diameter_capx.erl +++ b/lib/diameter/src/base/diameter_capx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2011. All Rights Reserved. +%% Copyright Ericsson AB 2010-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/diameter/src/base/diameter_service.erl b/lib/diameter/src/base/diameter_service.erl index a4a0b80348..91384b8b91 100644 --- a/lib/diameter/src/base/diameter_service.erl +++ b/lib/diameter/src/base/diameter_service.erl @@ -1703,16 +1703,20 @@ send(Pid, Pkt) -> %% retransmit/4 retransmit({TPid, Caps, #diameter_app{alias = Alias} = App} = T, - #request{app = Alias, packet = Pkt} + #request{app = Alias, packet = Pkt0} = Req, SvcName, Timeout) -> - have_request(Pkt, TPid) %% Don't failover to a peer we've + have_request(Pkt0, TPid) %% Don't failover to a peer we've andalso ?THROW(timeout), %% already sent to. + #diameter_packet{header = Hdr0} = Pkt0, + Hdr = Hdr0#diameter_header{is_retransmitted = true}, + Pkt = Pkt0#diameter_packet{header = Hdr}, + resend_req(cb(App, prepare_retransmit, [Pkt, SvcName, {TPid, Caps}]), T, - Req, + Req#request{packet = Pkt}, Timeout, []). diff --git a/lib/diameter/test/diameter_failover_SUITE.erl b/lib/diameter/test/diameter_failover_SUITE.erl index ed31670031..bb820a8bf2 100644 --- a/lib/diameter/test/diameter_failover_SUITE.erl +++ b/lib/diameter/test/diameter_failover_SUITE.erl @@ -18,19 +18,19 @@ %% %% -%% Tests of traffic between six Diameter nodes in three realms, +%% Tests of traffic between seven Diameter nodes in four realms, %% connected as follows. %% -%% ----- SERVER1.REALM2 -%% / -%% / ----- SERVER2.REALM2 -%% | / -%% CLIENT.REALM1 ------ SERVER3.REALM2 -%% | \ -%% | \ -%% \ ---- SERVER1.REALM3 -%% \ -%% ----- SERVER2.REALM3 +%% ----- SERVER1.REALM2 ----- +%% / \ +%% / ----- SERVER2.REALM2 ----- \ +%% | / \ | +%% CLIENT.REALM1 ------ SERVER3.REALM2 ------ CLIENT.REALM4 +%% | \ / | +%% | \ / | +%% \ ---- SERVER1.REALM3 ----- / +%% \ / +%% ----- SERVER2.REALM3 ----- %% -module(diameter_failover_SUITE). @@ -44,12 +44,16 @@ connect/1, send_ok/1, send_nok/1, + send_discard_1/1, + send_discard_2/1, stop_services/1, stop/1]). %% diameter callbacks -export([pick_peer/4, prepare_request/3, + prepare_retransmit/3, + handle_error/4, handle_answer/4, handle_request/3]). @@ -62,14 +66,18 @@ -define(ADDR, {127,0,0,1}). --define(CLIENT, "CLIENT.REALM1"). +-define(CLIENT1, "CLIENT.REALM1"). +-define(CLIENT2, "CLIENT.REALM4"). -define(SERVER1, "SERVER1.REALM2"). -define(SERVER2, "SERVER2.REALM2"). -define(SERVER3, "SERVER3.REALM2"). -define(SERVER4, "SERVER1.REALM3"). -define(SERVER5, "SERVER2.REALM3"). --define(SERVICES, [?CLIENT, ?SERVER1, ?SERVER2, ?SERVER3, ?SERVER4, ?SERVER5]). +-define(IS_CLIENT(Svc), Svc == ?CLIENT1; Svc == ?CLIENT2). + +-define(CLIENTS, [?CLIENT1, ?CLIENT2]). +-define(SERVERS, [?SERVER1, ?SERVER2, ?SERVER3, ?SERVER4, ?SERVER5]). -define(DICT_COMMON, ?DIAMETER_DICT_COMMON). @@ -77,26 +85,27 @@ -define(APP_ID, ?DICT_COMMON:id()). %% Config for diameter:start_service/2. --define(SERVICE(Host, Dict), +-define(SERVICE(Host), [{'Origin-Host', Host}, {'Origin-Realm', realm(Host)}, {'Host-IP-Address', [?ADDR]}, {'Vendor-Id', 12345}, {'Product-Name', "OTP/diameter"}, - {'Acct-Application-Id', [Dict:id()]}, + {'Acct-Application-Id', [?APP_ID]}, {application, [{alias, ?APP_ALIAS}, - {dictionary, Dict}, + {dictionary, ?DICT_COMMON}, {module, #diameter_callback {peer_up = false, peer_down = false, - handle_error = false, - prepare_retransmit = false, default = ?MODULE}}, {answer_errors, callback}]}]). -define(SUCCESS, 2001). --define(LOGOUT, ?'DIAMETER_BASE_TERMINATION-CAUSE_DIAMETER_LOGOUT'). +%% Value of Termination-Cause determines client/server behaviour. +-define(LOGOUT, ?'DIAMETER_BASE_TERMINATION-CAUSE_DIAMETER_LOGOUT'). +-define(MOVED, ?'DIAMETER_BASE_TERMINATION-CAUSE_DIAMETER_USER_MOVED'). +-define(TIMEOUT, ?'DIAMETER_BASE_TERMINATION-CAUSE_DIAMETER_SESSION_TIMEOUT'). %% =========================================================================== @@ -109,6 +118,8 @@ all() -> connect, send_ok, send_nok, + send_discard_1, + send_discard_2, stop_services, stop]. @@ -119,19 +130,20 @@ start(_Config) -> ok = diameter:start(). start_services(_Config) -> - S = [server(N, ?DICT_COMMON) || N <- tl(?SERVICES)], - - ok = diameter:start_service(?CLIENT, ?SERVICE(?CLIENT, ?DICT_COMMON)), + Servers = [server(N) || N <- ?SERVERS], + [] = [T || C <- ?CLIENTS, + T <- [diameter:start_service(C, ?SERVICE(C))], + T /= ok], - {save_config, [{?CLIENT, S}]}. + {save_config, Servers}. connect(Config) -> - {_, Conns} = proplists:get_value(saved_config, Config), + {start_services, Servers} = proplists:get_value(saved_config, Config), - lists:foreach(fun({CN,Ss}) -> connect(CN, Ss) end, Conns). + lists:foreach(fun(C) -> connect(C, Servers) end, ?CLIENTS). stop_services(_Config) -> - [] = [{H,T} || H <- ?SERVICES, + [] = [{H,T} || H <- ?CLIENTS ++ ?SERVERS, T <- [diameter:stop_service(H)], T /= ok]. @@ -140,8 +152,8 @@ stop(_Config) -> %% ---------------------------------------- -server(Name, Dict) -> - ok = diameter:start_service(Name, ?SERVICE(Name, Dict)), +server(Name) -> + ok = diameter:start_service(Name, ?SERVICE(Name)), {Name, ?util:listen(Name, tcp)}. connect(Name, Refs) -> @@ -153,30 +165,39 @@ connect(Name, Refs) -> %% Send an STR and expect success after SERVER3 answers after a couple %% of failovers. send_ok(_Config) -> - Req = ['STR', {'Destination-Realm', realm(?SERVER1)}, - {'Termination-Cause', ?LOGOUT}, - {'Auth-Application-Id', ?APP_ID}], + Req = #diameter_base_STR{'Destination-Realm' = realm(?SERVER1), + 'Termination-Cause' = ?LOGOUT, + 'Auth-Application-Id' = ?APP_ID}, #diameter_base_STA{'Result-Code' = ?SUCCESS, 'Origin-Host' = ?SERVER3} - = call(Req, [{filter, realm}]). + = call(?CLIENT1, Req). %% Send an STR and expect failure when both servers fail. send_nok(_Config) -> - Req = ['STR', {'Destination-Realm', realm(?SERVER4)}, - {'Termination-Cause', ?LOGOUT}, - {'Auth-Application-Id', ?APP_ID}], - {error, failover} = call(Req, [{filter, realm}]). + Req = #diameter_base_STR{'Destination-Realm' = realm(?SERVER4), + 'Termination-Cause' = ?LOGOUT, + 'Auth-Application-Id' = ?APP_ID}, + {failover, ?LOGOUT} = call(?CLIENT1, Req). + +%% Send an STR and have prepare_retransmit discard it. +send_discard_1(_Config) -> + Req = #diameter_base_STR{'Destination-Realm' = realm(?SERVER1), + 'Termination-Cause' = ?TIMEOUT, + 'Auth-Application-Id' = ?APP_ID}, + {rejected, ?TIMEOUT} = call(?CLIENT2, Req). +send_discard_2(_Config) -> + Req = #diameter_base_STR{'Destination-Realm' = realm(?SERVER4), + 'Termination-Cause' = ?MOVED, + 'Auth-Application-Id' = ?APP_ID}, + {discarded, ?MOVED} = call(?CLIENT2, Req). %% =========================================================================== realm(Host) -> tl(lists:dropwhile(fun(C) -> C /= $. end, Host)). -call(Req, Opts) -> - diameter:call(?CLIENT, ?APP_ALIAS, Req, Opts). - -set([H|T], Vs) -> - [H | Vs ++ T]. +call(Svc, Req) -> + diameter:call(Svc, ?APP_ALIAS, Req, [{filter, realm}]). %% =========================================================================== %% diameter callbacks @@ -184,7 +205,8 @@ set([H|T], Vs) -> %% pick_peer/4 %% Choose a server other than SERVER3 or SERVER5 if possible. -pick_peer(Peers, _, ?CLIENT, _State) -> +pick_peer(Peers, _, Svc, _State) + when ?IS_CLIENT(Svc) -> case lists:partition(fun({_, #diameter_caps{origin_host = {_, OH}}}) -> OH /= ?SERVER3 andalso OH /= ?SERVER5 end, @@ -198,20 +220,41 @@ pick_peer(Peers, _, ?CLIENT, _State) -> %% prepare_request/3 -prepare_request(Pkt, ?CLIENT, {_Ref, Caps}) -> +prepare_request(Pkt, Svc, {_Ref, Caps}) + when ?IS_CLIENT(Svc) -> {send, prepare(Pkt, Caps)}. prepare(#diameter_packet{msg = Req}, Caps) -> #diameter_caps{origin_host = {OH, _}, origin_realm = {OR, _}} = Caps, - set(Req, [{'Session-Id', diameter:session_id(OH)}, - {'Origin-Host', OH}, - {'Origin-Realm', OR}]). + Req#diameter_base_STR{'Origin-Host' = OH, + 'Origin-Realm' = OR, + 'Session-Id' = diameter:session_id(OH)}. + +%% prepare_retransmit/3 + +prepare_retransmit(#diameter_packet{header = H} = P, Svc, {_,_}) + when ?IS_CLIENT(Svc) -> + #diameter_header{is_retransmitted = true} = H, %% assert + prepare(P). + +prepare(#diameter_packet{msg = M} = P) -> + case M#diameter_base_STR.'Termination-Cause' of + ?LOGOUT -> {send, P}; + ?MOVED -> discard; + ?TIMEOUT -> {discard, rejected} + end. + +%% handle_error/4 + +handle_error(Reason, Req, _, _) -> + {Reason, Req#diameter_base_STR.'Termination-Cause'}. %% handle_answer/4 -handle_answer(Pkt, _Req, ?CLIENT, _Peer) -> +handle_answer(Pkt, _Req, Svc, _Peer) + when ?IS_CLIENT(Svc) -> #diameter_packet{msg = Rec, errors = []} = Pkt, Rec. @@ -219,8 +262,8 @@ handle_answer(Pkt, _Req, ?CLIENT, _Peer) -> %% Only SERVER3 actually answers. handle_request(Pkt, ?SERVER3, {_, Caps}) -> - #diameter_packet{msg = #diameter_base_STR{'Session-Id' = SId, - 'Origin-Host' = ?CLIENT}} + #diameter_packet{header = #diameter_header{is_retransmitted = true}, + msg = #diameter_base_STR{'Session-Id' = SId}} = Pkt, #diameter_caps{origin_host = {OH, _}, origin_realm = {OR, _}} diff --git a/lib/edoc/doc/src/notes.xml b/lib/edoc/doc/src/notes.xml index e01a6d6675..472b3bfc34 100644 --- a/lib/edoc/doc/src/notes.xml +++ b/lib/edoc/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2007</year><year>2011</year> + <year>2007</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/eldap/src/Makefile b/lib/eldap/src/Makefile index 39a41d08e2..46fb805bcc 100644 --- a/lib/eldap/src/Makefile +++ b/lib/eldap/src/Makefile @@ -88,7 +88,7 @@ $(TARGET_FILES): $(HRL_FILES) # Special Build Targets # ---------------------------------------------------- $(ASN1_HRL): ../asn1/$(ASN1_FILES) - $(ERLC) -o $(EBIN) -bber_bin +optimize +nif $(ERL_COMPILE_FLAGS) ../asn1/ELDAPv3.asn1 + $(ERLC) -o $(EBIN) -bber $(ERL_COMPILE_FLAGS) ../asn1/ELDAPv3.asn1 # ---------------------------------------------------- # Release Target diff --git a/lib/eldap/src/eldap.erl b/lib/eldap/src/eldap.erl index 699dfc8791..b3249d4f56 100644 --- a/lib/eldap/src/eldap.erl +++ b/lib/eldap/src/eldap.erl @@ -700,20 +700,22 @@ recv_response(S, Data) -> Error -> throw(Error) end; {error,Reason} -> - throw({gen_tcp_error, Reason}); - Error -> - throw(Error) + throw({gen_tcp_error, Reason}) end. %%% Sanity check of received packet check_tag(Data) -> - case asn1rt_ber_bin:decode_tag(l2b(Data)) of - {_Tag, Data1, _Rb} -> - case asn1rt_ber_bin:decode_length(l2b(Data1)) of - {{_Len, _Data2}, _Rb2} -> ok; - _ -> throw({error,decoded_tag_length}) - end; - _ -> throw({error,decoded_tag}) + try + {_Tag, Data1, _Rb} = asn1rt_ber_bin:decode_tag(l2b(Data)), + try + {{_Len, _Data2}, _Rb2} = asn1rt_ber_bin:decode_length(l2b(Data1)), + ok + catch + _ -> throw({error,decoded_tag_length}) + end + catch + _ -> + throw({error, decoded_tag}) end. %%% Check for expected kind of reply diff --git a/lib/eldap/vsn.mk b/lib/eldap/vsn.mk index c9d6e4e324..4d05d3d1e3 100644 --- a/lib/eldap/vsn.mk +++ b/lib/eldap/vsn.mk @@ -1 +1 @@ -ELDAP_VSN = 1.0 +ELDAP_VSN = 1.0.1 diff --git a/lib/erl_docgen/doc/src/notes.xml b/lib/erl_docgen/doc/src/notes.xml index ee62de86b1..9775909d76 100644 --- a/lib/erl_docgen/doc/src/notes.xml +++ b/lib/erl_docgen/doc/src/notes.xml @@ -30,7 +30,23 @@ </header> <p>This document describes the changes made to the <em>erl_docgen</em> application.</p> - <section><title>Erl_Docgen 0.3.2</title> + <section><title>Erl_Docgen 0.3.3</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> A possibility to configure erl_docgen so it can + generate documentation for other products than + Erlang/OTP. </p> + <p> + Own Id: OTP-9040</p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Docgen 0.3.2</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/erl_interface/configure.in b/lib/erl_interface/configure.in index f1c9ebbb6f..97f1cff345 100644 --- a/lib/erl_interface/configure.in +++ b/lib/erl_interface/configure.in @@ -1,7 +1,7 @@ # -*- Autoconf -*- # %CopyrightBegin% # -# Copyright Ericsson AB 2000-2011. All Rights Reserved. +# Copyright Ericsson AB 2000-2012. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/erl_interface/doc/src/notes.xml b/lib/erl_interface/doc/src/notes.xml index 6791c39c7d..f0a9b336ff 100644 --- a/lib/erl_interface/doc/src/notes.xml +++ b/lib/erl_interface/doc/src/notes.xml @@ -30,6 +30,24 @@ </header> <p>This document describes the changes made to the Erl_interface application.</p> +<section><title>Erl_Interface 3.7.9</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Teach lib/erl_interface/configure.in to look for + pthreads support in libc (where it can be found on + QNX)</p> <p>A minor tweak such that this configure + *fails* if you pass --enable-threads and no pthreads + support can be found.</p> (Thanks to Per Hedeland) + <p> + Own Id: OTP-10581</p> + </item> + </list> + </section> + +</section> + <section><title>Erl_Interface 3.7.8</title> <section><title>Improvements and New Features</title> diff --git a/lib/erl_interface/src/connect/ei_resolve.c b/lib/erl_interface/src/connect/ei_resolve.c index ba8f8fbce3..79d259b92d 100644 --- a/lib/erl_interface/src/connect/ei_resolve.c +++ b/lib/erl_interface/src/connect/ei_resolve.c @@ -186,11 +186,11 @@ static int verify_dns_configuration(void) * advance: increment buf by n bytes, reduce len by same amount . */ #if defined SIZEOF_VOID_P -#define ALIGNBYTES (SIZEOF_VOID_P - 1) +#define EI_ALIGNBYTES (SIZEOF_VOID_P - 1) #else -#define ALIGNBYTES (sizeof(void*) - 1) +#define EI_ALIGNBYTES (sizeof(void*) - 1) #endif -#define align_buf(buf,len) for (;(((unsigned)buf) & ALIGNBYTES); (buf)++, len--) +#define align_buf(buf,len) for (;(((unsigned)buf) & EI_ALIGNBYTES); (buf)++, len--) #define advance_buf(buf,len,n) ((buf)+=(n),(len)-=(n)) /* "and now the tricky part..." */ @@ -282,6 +282,8 @@ static int copy_hostent(struct hostent *dest, const struct hostent *src, char *b return 0; } +#undef EI_ALIGNBYTES + /* This function is a pseudo-reentrant version of gethostbyname(). It * uses locks to serialize the call to the regular (non-reentrant) * gethostbyname() and then copies the data into the user-provided diff --git a/lib/erl_interface/vsn.mk b/lib/erl_interface/vsn.mk index 4e401c874e..1718f38069 100644 --- a/lib/erl_interface/vsn.mk +++ b/lib/erl_interface/vsn.mk @@ -1 +1 @@ -EI_VSN = 3.7.8 +EI_VSN = 3.7.9 diff --git a/lib/eunit/doc/src/notes.xml b/lib/eunit/doc/src/notes.xml index 814580e228..b797be0ccb 100644 --- a/lib/eunit/doc/src/notes.xml +++ b/lib/eunit/doc/src/notes.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2011</year> + <year>2012</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/hipe/cerl/erl_bif_types.erl b/lib/hipe/cerl/erl_bif_types.erl index 776e336aea..98d65abba1 100644 --- a/lib/hipe/cerl/erl_bif_types.erl +++ b/lib/hipe/cerl/erl_bif_types.erl @@ -766,37 +766,6 @@ type(erlang, node, 0, _) -> t_node(); type(erlang, node, 1, Xs) -> strict(arg_types(erlang, node, 1), Xs, fun (_) -> t_node() end); type(erlang, nodes, 0, _) -> t_list(t_node()); -type(erlang, port_call, Arity, Xs) when Arity =:= 2; Arity =:= 3 -> - strict(arg_types(erlang, port_call, Arity), Xs, fun (_) -> t_any() end); -type(erlang, port_info, 1, Xs) -> - strict(arg_types(erlang, port_info, 1), Xs, - fun (_) -> t_sup(t_atom('undefined'), t_list()) end); -type(erlang, port_info, 2, Xs) -> - strict(arg_types(erlang, port_info, 2), Xs, - fun ([_Port, Item]) -> - t_sup(t_atom('undefined'), - case t_atom_vals(Item) of - ['connected'] -> t_tuple([Item, t_pid()]); - ['id'] -> t_tuple([Item, t_integer()]); - ['input'] -> t_tuple([Item, t_integer()]); - ['links'] -> t_tuple([Item, t_list(t_pid())]); - ['name'] -> t_tuple([Item, t_string()]); - ['output'] -> t_tuple([Item, t_integer()]); - ['os_pid'] -> t_tuple([Item, t_sup(t_non_neg_integer(),t_atom('undefined'))]); - ['registered_name'] -> t_tuple([Item, t_atom()]); - List when is_list(List) -> - t_tuple([t_sup([t_atom(A) || A <- List]), - t_sup([t_atom(), t_integer(), - t_pid(), t_list(t_pid()), - t_string()])]); - unknown -> - [_, PosItem] = arg_types(erlang, port_info, 2), - t_tuple([PosItem, - t_sup([t_atom(), t_integer(), - t_pid(), t_list(t_pid()), - t_string()])]) - end) - end); %% Guard bif, needs to be here. type(erlang, round, 1, Xs) -> strict(arg_types(erlang, round, 1), Xs, fun (_) -> t_integer() end); @@ -922,6 +891,14 @@ type(erlang, system_info, 1, Xs) -> t_non_neg_fixnum(), t_non_neg_fixnum()]), t_string()); + ['otp_release'] -> + t_string(); + ['port_parallelism'] -> + t_boolean(); + ['port_count'] -> + t_non_neg_fixnum(); + ['port_limit'] -> + t_non_neg_fixnum(); ['process_count'] -> t_non_neg_fixnum(); ['process_limit'] -> @@ -2275,16 +2252,6 @@ arg_types(erlang, node, 1) -> [t_identifier()]; arg_types(erlang, nodes, 0) -> []; -arg_types(erlang, port_call, 2) -> - [t_sup(t_port(), t_atom()), t_any()]; -arg_types(erlang, port_call, 3) -> - [t_sup(t_port(), t_atom()), t_integer(), t_any()]; -arg_types(erlang, port_info, 1) -> - [t_sup(t_port(), t_atom())]; -arg_types(erlang, port_info, 2) -> - [t_sup(t_port(), t_atom()), - t_atoms(['registered_name', 'id', 'connected', - 'links', 'name', 'input', 'output', 'os_pid'])]; %% Guard bif, needs to be here. arg_types(erlang, round, 1) -> [t_number()]; diff --git a/lib/hipe/doc/src/notes.xml b/lib/hipe/doc/src/notes.xml index 1841c7d9de..cfd22a9d8d 100644 --- a/lib/hipe/doc/src/notes.xml +++ b/lib/hipe/doc/src/notes.xml @@ -30,6 +30,22 @@ </header> <p>This document describes the changes made to HiPE.</p> +<section><title>Hipe 3.9.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A faulty spec for process_info/2 could cause false + dialyzer warnings. The spec is corrected.</p> + <p> + Own Id: OTP-10584</p> + </item> + </list> + </section> + +</section> + <section><title>Hipe 3.9.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/hipe/vsn.mk b/lib/hipe/vsn.mk index 144167f5d1..f3e2e695b5 100644 --- a/lib/hipe/vsn.mk +++ b/lib/hipe/vsn.mk @@ -1 +1 @@ -HIPE_VSN = 3.9.2 +HIPE_VSN = 3.9.3 diff --git a/lib/ic/test/java_client_erl_server_SUITE.erl b/lib/ic/test/java_client_erl_server_SUITE.erl index c2bec94697..9e49305c3c 100644 --- a/lib/ic/test/java_client_erl_server_SUITE.erl +++ b/lib/ic/test/java_client_erl_server_SUITE.erl @@ -62,9 +62,14 @@ init_per_suite(Config) when is_list(Config) -> case case code:priv_dir(jinterface) of {error,bad_name} -> false; - P -> - filelib:is_dir(P) - end + P -> + case filelib:wildcard(filename:join(P, "*.jar")) of + [_|_] -> + true; + [] -> + false + end + end of true -> case find_executable(["java"]) of diff --git a/lib/ic/test/java_client_erl_server_SUITE_data/Makefile.src b/lib/ic/test/java_client_erl_server_SUITE_data/Makefile.src index ac8f2e619f..3143ab427b 100644 --- a/lib/ic/test/java_client_erl_server_SUITE_data/Makefile.src +++ b/lib/ic/test/java_client_erl_server_SUITE_data/Makefile.src @@ -68,7 +68,7 @@ EBINS = $(ERL_FILES:.erl=.@EMULATOR@) @IFEQ@ (@jinterface_classpath@,) all: -@ELSE +@ELSE@ all: $(CLASS_FILES) $(EBINS) @ENDIF@ diff --git a/lib/inets/doc/src/httpd.xml b/lib/inets/doc/src/httpd.xml index 8497d91549..3fced5dfcd 100644 --- a/lib/inets/doc/src/httpd.xml +++ b/lib/inets/doc/src/httpd.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1997</year><year>2011</year> + <year>1997</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml index 3aae1ff70a..e0d6ae3454 100644 --- a/lib/inets/doc/src/notes.xml +++ b/lib/inets/doc/src/notes.xml @@ -33,7 +33,27 @@ </header> - <section> + <section><title>Inets 5.9.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Minimum bytes per second</p> + <p> + New option to http server, {minimum_bytes_per_second, + integer()}, for a connection, if it is not reached the + socket will close for that specific connection. Can be + used to prevent hanging requests from faulty clients.</p> + <p> + Own Id: OTP-10392</p> + </item> + </list> + </section> + +</section> + +<section> <title>Inets 5.9.1</title> <section> diff --git a/lib/inets/src/http_server/httpd_request_handler.erl b/lib/inets/src/http_server/httpd_request_handler.erl index 5e0bd39cb3..0f47d785ef 100644 --- a/lib/inets/src/http_server/httpd_request_handler.erl +++ b/lib/inets/src/http_server/httpd_request_handler.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2011. All Rights Reserved. +%% Copyright Ericsson AB 1997-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/inets/test/erl_make_certs.erl b/lib/inets/test/erl_make_certs.erl index 254aa6d2f9..d6bdd05d01 100644 --- a/lib/inets/test/erl_make_certs.erl +++ b/lib/inets/test/erl_make_certs.erl @@ -137,10 +137,10 @@ decode_key(PemBin, Pw) -> encode_key(Key = #'RSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('RSAPrivateKey', Key), - {'RSAPrivateKey', list_to_binary(Der), not_encrypted}; + {'RSAPrivateKey', Der, not_encrypted}; encode_key(Key = #'DSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('DSAPrivateKey', Key), - {'DSAPrivateKey', list_to_binary(Der), not_encrypted}. + {'DSAPrivateKey', Der, not_encrypted}. make_tbs(SubjectKey, Opts) -> Version = list_to_atom("v"++integer_to_list(proplists:get_value(version, Opts, 3))), diff --git a/lib/inets/test/inets_app_test.erl b/lib/inets/test/inets_app_test.erl index d32f7e290b..eabfa69f7c 100644 --- a/lib/inets/test/inets_app_test.erl +++ b/lib/inets/test/inets_app_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2011. All Rights Reserved. +%% Copyright Ericsson AB 2002-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/jinterface/test/jitu.erl b/lib/jinterface/test/jitu.erl index fb262cf9d7..571a2dc9c7 100644 --- a/lib/jinterface/test/jitu.erl +++ b/lib/jinterface/test/jitu.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2010. All Rights Reserved. +%% Copyright Ericsson AB 2004-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/kernel/doc/src/heart.xml b/lib/kernel/doc/src/heart.xml index 2826d3d00a..2856d84dcf 100644 --- a/lib/kernel/doc/src/heart.xml +++ b/lib/kernel/doc/src/heart.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2011</year> + <year>1996</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/kernel/doc/src/inet.xml b/lib/kernel/doc/src/inet.xml index f498a88c5e..a1590c2dce 100644 --- a/lib/kernel/doc/src/inet.xml +++ b/lib/kernel/doc/src/inet.xml @@ -371,7 +371,51 @@ fe80::204:acff:fe17:bf38 </taglist> </desc> </func> - + <func> + <name name="parse_ipv4_address" arity="1" /> + <fsummary>Parse an IPv4 address</fsummary> + <desc> + <p>Parses an IPv4 address string and returns an <a href="#type-ip4_address">ip4_address()</a>. + Accepts a shortened IPv4 shortened address string.</p> + </desc> + </func> + <func> + <name name="parse_ipv4strict_address" arity="1" /> + <fsummary>Parse an IPv4 address strict.</fsummary> + <desc> + <p>Parses an IPv4 address string containing four fields, i.e <b>not</b> shortened, and returns an <a href="#type-ip4_adress">ip4_address()</a>.</p> + </desc> + </func> + <func> + <name name="parse_ipv6_address" arity="1" /> + <fsummary>Parse an IPv6 address</fsummary> + <desc> + <p>Parses an IPv6 address string and returns an <a href="#type-ip6_address">ip6_address()</a>. + If an IPv4 address string is passed, an IPv4-mapped IPv6 address is returned.</p> + </desc> + </func> + <func> + <name name="parse_ipv6strict_address" arity="1" /> + <fsummary>Parse an IPv6 address strict.</fsummary> + <desc> + <p>Parses an IPv6 address string and returns an <a href="#type-ip6_address">ip6_address()</a>. + Does <b>not</b> accept IPv4 adresses.</p> + </desc> + </func> + <func> + <name name="parse_address" arity="1" /> + <fsummary>Parse an IPv4 or IPv6 address.</fsummary> + <desc> + <p>Parses an IPv4 or IPv6 address string and returns an <a href="#type-ip4_address">ip4_address()</a> or <a href="#type-ip6_address">ip6_address()</a>. Accepts a shortened IPv4 address string.</p> + </desc> + </func> + <func> + <name name="parse_strict_address" arity="1" /> + <fsummary>Parse an IPv4 or IPv6 address strict.</fsummary> + <desc> + <p>Parses an IPv4 or IPv6 address string and returns an <a href="#type-ip4_address">ip4_address()</a> or <a href="#type-ip6_adress">ip6_address()</a>. Does <b>not</b> accept a shortened IPv4 address string.</p> + </desc> + </func> <func> <name name="peername" arity="1"/> <fsummary>Return the address and port for the other end of a connection</fsummary> @@ -512,13 +556,37 @@ fe80::204:acff:fe17:bf38 <c>[Byte1,Byte2|Binary]</c>.</p> </item> - <tag><c>{high_watermark, Size}</c></tag> - <item> <p> - Sender is forced busy if sent and enqueued data - reaches the highwater mark. - <br /> Default: 8192 kB. - </p> - </item> + <tag><c>{high_msgq_watermark, Size}</c> (TCP/IP sockets)</tag> + <item> + <p>The socket message queue will be set into a busy + state when the amount of data queued on the message + queue reaches this limit. Note that this limit only + concerns data that have not yet reached the ERTS internal + socket implementation. Default value used is 8 kB.</p> + <p>Senders of data to the socket will be suspended if + either the socket message queue is busy, or the socket + itself is busy.</p> + <p>For more information see the <c>low_msgq_watermark</c>, + <c>high_watermark</c>, and <c>low_watermark</c> options.</p> + <p>Note that distribution sockets will disable the use of + <c>high_msgq_watermark</c> and <c>low_msgq_watermark</c>, + and will instead use the + <seealso marker="erts:erlang#system_info_dist_buf_busy_limit">distribution + buffer busy limit</seealso> which is a similar feature.</p> + </item> + + <tag><c>{high_watermark, Size}</c> (TCP/IP sockets)</tag> + <item> + <p>The socket will be set into a busy state when the amount + of data queued internally by the ERTS socket implementation + reaches this limit. Default value used is 8 kB.</p> + <p>Senders of data to the socket will be suspended if + either the socket message queue is busy, or the socket + itself is busy.</p> + <p>For more information see the <c>low_watermark</c>, + <c>high_msgq_watermark</c>, and <c>low_msqg_watermark</c> + options.</p> + </item> <tag><c>{ipv6_v6only, Boolean}</c></tag> <item> @@ -603,14 +671,41 @@ fe80::204:acff:fe17:bf38 the flushing time-out in seconds.</p> </item> - <tag><c>{low_watermark, Size}</c></tag> - <item> <p> - If the port has reached its <c>high_watermark</c> it will - force busy onto senders. When the port data queue reaches the - <c>low_watermark</c> callers are no longer forced busy. - <br /> Default: 4096 kB. - </p> - </item> + <tag><c>{low_msgq_watermark, Size}</c> (TCP/IP sockets)</tag> + <item> + <p>If the socket message queue is in a busy state, the + socket message queue will be set in a not busy state when + the amount of data queued in the message queue falls + below this limit. Note that this limit only concerns data + that have not yet reached the ERTS internal socket + implementation. Default value used is 4 kB.</p> + <p>Senders that have been suspended due to either a + busy message queue or a busy socket, will be resumed + when neither the socket message queue, nor the socket + are busy.</p> + <p>For more information see the <c>high_msgq_watermark</c>, + <c>high_watermark</c>, and <c>low_watermark</c> options.</p> + <p>Note that distribution sockets will disable the use of + <c>high_msgq_watermark</c> and <c>low_msgq_watermark</c>, + and will instead use the + <seealso marker="erts:erlang#system_info_dist_buf_busy_limit">distribution + buffer busy limit</seealso> which is a similar feature.</p> + </item> + + <tag><c>{low_watermark, Size}</c> (TCP/IP sockets)</tag> + <item> + <p>If the socket is in a busy state, the socket will + be set in a not busy state when the amount of data + queued internally by the ERTS socket implementation + falls below this limit. Default value used is 4 kB.</p> + <p>Senders that have been suspended due to either a + busy message queue or a busy socket, will be resumed + when neither the socket message queue, nor the socket + are busy.</p> + <p>For more information see the <c>high_watermark</c>, + <c>high_msgq_watermark</c>, and <c>low_msgq_watermark</c> + options.</p> + </item> <tag><c>{mode, Mode :: binary | list}</c></tag> <item> diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index 8e911a406b..78bc533464 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -30,6 +30,44 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 2.15.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Ensure 'erl_crash.dump' when asked for it. This will + change erl_crash.dump behaviour.</p> + <p> + * Not setting ERL_CRASH_DUMP_SECONDS will now terminate + beam immediately on a crash without writing a crash dump + file.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to 0 will also terminate + beam immediately on a crash without writing a crash dump + file, i.e. same as not setting ERL_CRASH_DUMP_SECONDS + environment variable.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to a negative value will + let the beam wait indefinitely on the crash dump file + being written.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to a positive value will + let the beam wait that many seconds on the crash dump + file being written.</p> + <p> + A positive value will set an alarm/timeout for restart + both in beam and in heart if heart is running.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-10422 Aux Id: kunagi-250 [161] </p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 2.15.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/src/application_controller.erl b/lib/kernel/src/application_controller.erl index 68cd26ec10..75ce852001 100644 --- a/lib/kernel/src/application_controller.erl +++ b/lib/kernel/src/application_controller.erl @@ -1960,5 +1960,5 @@ to_string(Term) -> true -> Term; false -> - lists:flatten(io_lib:write(Term)) + lists:flatten(io_lib:format("~134217728p", [Term])) end. diff --git a/lib/kernel/src/gen_tcp.erl b/lib/kernel/src/gen_tcp.erl index 22e6aa5bc8..ec13ab6d2e 100644 --- a/lib/kernel/src/gen_tcp.erl +++ b/lib/kernel/src/gen_tcp.erl @@ -37,9 +37,11 @@ {dontroute, boolean()} | {exit_on_close, boolean()} | {header, non_neg_integer()} | + {high_msgq_watermark, pos_integer()} | {high_watermark, non_neg_integer()} | {keepalive, boolean()} | {linger, {boolean(), non_neg_integer()}} | + {low_msgq_watermark, pos_integer()} | {low_watermark, non_neg_integer()} | {mode, list | binary} | list | binary | {nodelay, boolean()} | @@ -67,9 +69,11 @@ dontroute | exit_on_close | header | + high_msgq_watermark | high_watermark | keepalive | linger | + low_msgq_watermark | low_watermark | mode | nodelay | diff --git a/lib/kernel/src/heart.erl b/lib/kernel/src/heart.erl index de287bfa43..87cb9d7f51 100644 --- a/lib/kernel/src/heart.erl +++ b/lib/kernel/src/heart.erl @@ -46,6 +46,7 @@ -define(TIMEOUT, 5000). -define(CYCLE_TIMEOUT, 10000). +-define(HEART_PORT_NAME, heart_port). %%--------------------------------------------------------------------- @@ -132,7 +133,7 @@ start_portprogram() -> case wait_ack(Port) of ok -> %% register port so the vm can find it if need be - register(heart_port, Port), + register(?HEART_PORT_NAME, Port), {ok, Port}; {error, Reason} -> report_problem({{port_problem, Reason}, @@ -228,6 +229,7 @@ no_reboot_shutdown(Port) -> end. do_cycle_port_program(Caller, Parent, Port, Cmd) -> + unregister(?HEART_PORT_NAME), case catch start_portprogram() of {ok, NewPort} -> send_shutdown(Port), diff --git a/lib/kernel/src/inet.erl b/lib/kernel/src/inet.erl index 92c1802a86..9670271b2e 100644 --- a/lib/kernel/src/inet.erl +++ b/lib/kernel/src/inet.erl @@ -30,7 +30,9 @@ ifget/3, ifget/2, ifset/3, ifset/2, getstat/1, getstat/2, ip/1, stats/0, options/0, - pushf/3, popf/1, close/1, gethostname/0, gethostname/1]). + pushf/3, popf/1, close/1, gethostname/0, gethostname/1, + parse_ipv4_address/1, parse_ipv6_address/1, parse_ipv4strict_address/1, + parse_ipv6strict_address/1, parse_address/1, parse_strict_address/1]). -export([connect_options/2, listen_options/2, udp_options/2, sctp_options/2]). @@ -527,6 +529,48 @@ getservbyname(Name, Protocol) when is_atom(Name) -> Error -> Error end. +-spec parse_ipv4_address(Address) -> + {ok, IPv4Address} | {error, einval} when + Address :: string(), + IPv4Address :: ip_address(). +parse_ipv4_address(Addr) -> + inet_parse:ipv4_address(Addr). + +-spec parse_ipv6_address(Address) -> + {ok, IPv6Address} | {error, einval} when + Address :: string(), + IPv6Address :: ip_address(). +parse_ipv6_address(Addr) -> + inet_parse:ipv6_address(Addr). + +-spec parse_ipv4strict_address(Address) -> + {ok, IPv4Address} | {error, einval} when + Address :: string(), + IPv4Address :: ip_address(). +parse_ipv4strict_address(Addr) -> + inet_parse:ipv4strict_address(Addr). + +-spec parse_ipv6strict_address(Address) -> + {ok, IPv6Address} | {error, einval} when + Address :: string(), + IPv6Address :: ip_address(). +parse_ipv6strict_address(Addr) -> + inet_parse:ipv6strict_address(Addr). + +-spec parse_address(Address) -> + {ok, IPAddress} | {error, einval} when + Address :: string(), + IPAddress :: ip_address(). +parse_address(Addr) -> + inet_parse:address(Addr). + +-spec parse_strict_address(Address) -> + {ok, IPAddress} | {error, einval} when + Address :: string(), + IPAddress :: ip_address(). +parse_strict_address(Addr) -> + inet_parse:strict_address(Addr). + %% Return a list of available options options() -> [ @@ -535,6 +579,7 @@ options() -> buffer, header, active, packet, deliver, mode, multicast_if, multicast_ttl, multicast_loop, exit_on_close, high_watermark, low_watermark, + high_msgq_watermark, low_msgq_watermark, send_timeout, send_timeout_close ]. @@ -552,8 +597,8 @@ stats() -> connect_options() -> [tos, priority, reuseaddr, keepalive, linger, sndbuf, recbuf, nodelay, header, active, packet, packet_size, buffer, mode, deliver, - exit_on_close, high_watermark, low_watermark, send_timeout, - send_timeout_close, delay_send,raw]. + exit_on_close, high_watermark, low_watermark, high_msgq_watermark, + low_msgq_watermark, send_timeout, send_timeout_close, delay_send, raw]. connect_options(Opts, Family) -> BaseOpts = @@ -608,8 +653,9 @@ con_add(Name, Val, R, Opts, AllOpts) -> listen_options() -> [tos, priority, reuseaddr, keepalive, linger, sndbuf, recbuf, nodelay, header, active, packet, buffer, mode, deliver, backlog, ipv6_v6only, - exit_on_close, high_watermark, low_watermark, send_timeout, - send_timeout_close, delay_send, packet_size,raw]. + exit_on_close, high_watermark, low_watermark, high_msgq_watermark, + low_msgq_watermark, send_timeout, send_timeout_close, delay_send, + packet_size, raw]. listen_options(Opts, Family) -> BaseOpts = diff --git a/lib/kernel/src/inet_int.hrl b/lib/kernel/src/inet_int.hrl index 6d808b54cd..000119bc74 100644 --- a/lib/kernel/src/inet_int.hrl +++ b/lib/kernel/src/inet_int.hrl @@ -141,6 +141,8 @@ -define(INET_LOPT_READ_PACKETS, 33). -define(INET_OPT_RAW, 34). -define(INET_LOPT_TCP_SEND_TIMEOUT_CLOSE, 35). +-define(INET_LOPT_TCP_MSGQ_HIWTRMRK, 36). +-define(INET_LOPT_TCP_MSGQ_LOWTRMRK, 37). % Specific SCTP options: separate range: -define(SCTP_OPT_RTOINFO, 100). -define(SCTP_OPT_ASSOCINFO, 101). diff --git a/lib/kernel/src/inet_parse.erl b/lib/kernel/src/inet_parse.erl index ba62a59068..3551e701b6 100644 --- a/lib/kernel/src/inet_parse.erl +++ b/lib/kernel/src/inet_parse.erl @@ -36,7 +36,7 @@ -export([ipv4_address/1, ipv6_address/1]). -export([ipv4strict_address/1, ipv6strict_address/1]). --export([address/1]). +-export([address/1, strict_address/1]). -export([visible_string/1, domain/1]). -export([ntoa/1, dots/1]). -export([split_line/1]). @@ -456,6 +456,17 @@ address(Cs) when is_list(Cs) -> address(_) -> {error, einval}. +%%Parse ipv4 strict address or ipv6 strict address +strict_address(Cs) when is_list(Cs) -> + case ipv4strict_address(Cs) of + {ok,IP} -> + {ok,IP}; + _ -> + ipv6strict_address(Cs) + end; +strict_address(Cs) -> + {error, einval}. + %% %% Parse IPv4 address: %% d1.d2.d3.d4 diff --git a/lib/kernel/test/global_SUITE.erl b/lib/kernel/test/global_SUITE.erl index 6eb2134644..b40c50f79f 100644 --- a/lib/kernel/test/global_SUITE.erl +++ b/lib/kernel/test/global_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2011. All Rights Reserved. +%% Copyright Ericsson AB 1997-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -3821,7 +3821,7 @@ start_node_rel(Name0, Rel, Config) -> Name = node_name(Name0, Config), {Release, Compat} = case Rel of this -> - {[this], "+R8"}; + {[this], ""}; Rel when is_atom(Rel) -> {[{release, atom_to_list(Rel)}], ""}; RelList -> diff --git a/lib/kernel/test/heart_SUITE.erl b/lib/kernel/test/heart_SUITE.erl index 970a03cfd5..2ec3b7c297 100644 --- a/lib/kernel/test/heart_SUITE.erl +++ b/lib/kernel/test/heart_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/kernel/test/inet_SUITE.erl b/lib/kernel/test/inet_SUITE.erl index ce138b6804..e5e1794514 100644 --- a/lib/kernel/test/inet_SUITE.erl +++ b/lib/kernel/test/inet_SUITE.erl @@ -37,7 +37,8 @@ gethostnative_soft_restart/0, gethostnative_soft_restart/1, gethostnative_debug_level/0, gethostnative_debug_level/1, getif/1, - getif_ifr_name_overflow/1,getservbyname_overflow/1, getifaddrs/1]). + getif_ifr_name_overflow/1,getservbyname_overflow/1, getifaddrs/1, + parse_strict_address/1]). -export([get_hosts/1, get_ipv6_hosts/1, parse_hosts/1, parse_address/1, kill_gethost/0, parallell_gethost/0]). @@ -52,7 +53,7 @@ all() -> t_gethostnative, gethostnative_parallell, cname_loop, gethostnative_debug_level, gethostnative_soft_restart, getif, getif_ifr_name_overflow, getservbyname_overflow, - getifaddrs]. + getifaddrs, parse_strict_address]. groups() -> [{parse, [], [parse_hosts, parse_address]}]. @@ -582,16 +583,16 @@ parse_address(Config) when is_list(Config) -> "fe80::198.168.0.", "fec0::fFfF:127.0.0.1."], t_parse_address - (ipv6_address, + (parse_ipv6_address, V6Strict++V6Sloppy++V6Err++V4Err), t_parse_address - (ipv6strict_address, + (parse_ipv6strict_address, V6Strict++V6Err++V4Err++[S || {_,S} <- V6Sloppy]), t_parse_address - (ipv4_address, + (parse_ipv4_address, V4Strict++V4Sloppy++V4Err++V6Err++[S || {_,S} <- V6Strict]), t_parse_address - (ipv4strict_address, + (parse_ipv4strict_address, V4Strict++V4Err++V6Err++[S || {_,S} <- V4Sloppy++V6Strict]). t_parse_address(Func, []) -> @@ -599,14 +600,16 @@ t_parse_address(Func, []) -> ok; t_parse_address(Func, [{Addr,String}|L]) -> io:format("~p = ~p.~n", [Addr,String]), - {ok,Addr} = inet_parse:Func(String), + {ok,Addr} = inet:Func(String), t_parse_address(Func, L); t_parse_address(Func, [String|L]) -> io:format("~p.~n", [String]), - {error,einval} = inet_parse:Func(String), + {error,einval} = inet:Func(String), t_parse_address(Func, L). - +parse_strict_address(Config) when is_list(Config) -> + {ok, Ipv4} = inet:parse_strict_address("127.0.0.1"), + {ok, Ipv6} = inet:parse_strict_address("c11:0c22:5c33:c440:55c0:c66c:77:0088"). t_gethostnative(suite) ->[]; t_gethostnative(doc) ->[]; diff --git a/lib/kernel/test/wrap_log_reader_SUITE.erl b/lib/kernel/test/wrap_log_reader_SUITE.erl index 6c47fda9c5..16b3a7cc1e 100644 --- a/lib/kernel/test/wrap_log_reader_SUITE.erl +++ b/lib/kernel/test/wrap_log_reader_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2011. All Rights Reserved. +%% Copyright Ericsson AB 1998-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/megaco/doc/src/megaco_encode.xml b/lib/megaco/doc/src/megaco_encode.xml index 410e4f3b31..4a9d63c2e3 100644 --- a/lib/megaco/doc/src/megaco_encode.xml +++ b/lib/megaco/doc/src/megaco_encode.xml @@ -224,22 +224,11 @@ messages.</p> </item> <item> - <p>megaco_ber_bin_encoder - encode/decode ASN.1 BER - messages. This encoder uses ASN.1 ber_bin which - has been optimized using the bit syntax.</p> - </item> - <item> <p>megaco_per_encoder - encode/decode ASN.1 PER messages. N.B. that this format is not included in the Megaco standard.</p> </item> <item> - <p>megaco_per_bin_encoder - encode/decode ASN.1 PER - messages. N.B. that this format is not included in the - Megaco standard. This encoder uses ASN.1 per_bin which - has been optimized using the bit syntax.</p> - </item> - <item> <p>megaco_erl_dist_encoder - encodes messages into Erlangs distribution format. It is rather verbose but encoding and decoding is blinding fast. N.B. that this format is not @@ -370,18 +359,6 @@ needs to be specified.</p> <list type="bulleted"> <item> - <p><c><![CDATA[[driver|_]]]></c> - make use of the asn1 driver for decode - (ber_bin) and encode (per_bin). This option is only available for - encoding modules: <c><![CDATA[megaco_binary_encoder]]></c>, - <c><![CDATA[megaco_ber_bin_encoder]]></c> and <c><![CDATA[megaco_per_bin_encoder]]></c>.</p> - <p>If this option is present in the encoding config, it <em>must</em> - to be the <em>first</em>, unless the - <seealso marker="#handling_versions">version3</seealso> encoding - config is present, in which case it must come second, after - the version3 encoding config, - e.g. <c><![CDATA[[{version3,prev3b},driver]]]></c>.</p> - </item> - <item> <p><c><![CDATA[[native]]]></c> - skips the transformation phase, i.e. the decoded message(s) will not be transformed into our internal form.</p> diff --git a/lib/megaco/examples/meas/megaco_codec_meas.erl b/lib/megaco/examples/meas/megaco_codec_meas.erl index 51ee396338..65b986ccd1 100644 --- a/lib/megaco/examples/meas/megaco_codec_meas.erl +++ b/lib/megaco/examples/meas/megaco_codec_meas.erl @@ -176,7 +176,7 @@ display_megaco_info() -> io:format("Megaco version: ~s (~s)~n", [Ver, FlexStr]). display_asn1_info() -> - AI = megaco_ber_bin_drv_media_gateway_control_v1:info(), + AI = megaco_ber_media_gateway_control_v1:info(), Vsn = case lists:keysearch(vsn, 1, AI) of {value, {vsn, V}} when is_atom(V) -> @@ -281,15 +281,11 @@ expand_codec(Codec) -> [{Codec, megaco_compact_text_encoder, [flex_scanner], 3000}, {Codec, megaco_compact_text_encoder, [], 1500}]; ber -> - [{Codec, megaco_ber_bin_encoder, [driver,native], 4000}, - {Codec, megaco_ber_bin_encoder, [native], 3000}, - {Codec, megaco_ber_bin_encoder, [driver], 3000}, - {Codec, megaco_ber_bin_encoder, [], 1000}]; + [{Codec, megaco_ber_encoder, [native], 3000}, + {Codec, megaco_ber_encoder, [], 1000}]; per -> - [{Codec, megaco_per_bin_encoder, [driver,native], 4000}, - {Codec, megaco_per_bin_encoder, [native], 3000}, - {Codec, megaco_per_bin_encoder, [driver], 3000}, - {Codec, megaco_per_bin_encoder, [], 1000}]; + [{Codec, megaco_per_encoder, [native], 3000}, + {Codec, megaco_per_encoder, [], 1000}]; erlang -> [ {Codec, megaco_erl_dist_encoder, [megaco_compressed,compressed], 500}, diff --git a/lib/megaco/examples/meas/megaco_codec_mstone_lib.erl b/lib/megaco/examples/meas/megaco_codec_mstone_lib.erl index 9af88d9f50..b527ff2e89 100644 --- a/lib/megaco/examples/meas/megaco_codec_mstone_lib.erl +++ b/lib/megaco/examples/meas/megaco_codec_mstone_lib.erl @@ -268,7 +268,7 @@ display_megaco_info() -> io:format("Megaco version: ~s~n", [Ver]). display_asn1_info() -> - AI = megaco_ber_bin_drv_media_gateway_control_v1:info(), + AI = megaco_ber__media_gateway_control_v1:info(), Vsn = case lists:keysearch(vsn, 1, AI) of {value, {vsn, V}} when is_atom(V) -> @@ -361,15 +361,11 @@ expand_codec(Codec, only_drv) -> [{Codec, megaco_compact_text_encoder, [flex_scanner]}, {Codec, megaco_compact_text_encoder, [flex_scanner]}]; ber -> - [{Codec, megaco_ber_bin_encoder, [driver,native]}, - {Codec, megaco_ber_bin_encoder, [driver]}, - {Codec, megaco_ber_bin_encoder, [driver,native]}, - {Codec, megaco_ber_bin_encoder, [driver]}]; + [{Codec, megaco_ber_encoder, [native]}, + {Codec, megaco_ber_encoder, []}]; per -> - [{Codec, megaco_per_bin_encoder, [driver,native]}, - {Codec, megaco_per_bin_encoder, [native]}, - {Codec, megaco_per_bin_encoder, [driver,native]}, - {Codec, megaco_per_bin_encoder, [native]}]; + [{Codec, megaco_per_encoder, [native]}, + {Codec, megaco_per_encoder, []}]; erlang -> Encoder = megaco_erl_dist_encoder, [ @@ -390,15 +386,11 @@ expand_codec(Codec, no_drv) -> [{Codec, megaco_compact_text_encoder, []}, {Codec, megaco_compact_text_encoder, []}]; ber -> - [{Codec, megaco_ber_bin_encoder, [native]}, - {Codec, megaco_ber_bin_encoder, []}, - {Codec, megaco_ber_bin_encoder, [native]}, - {Codec, megaco_ber_bin_encoder, []}]; + [{Codec, megaco_ber_encoder, [native]}, + {Codec, megaco_ber_encoder, []}]; per -> - [{Codec, megaco_per_bin_encoder, [native]}, - {Codec, megaco_per_bin_encoder, []}, - {Codec, megaco_per_bin_encoder, [native]}, - {Codec, megaco_per_bin_encoder, []}]; + [{Codec, megaco_per_encoder, [native]}, + {Codec, megaco_per_encoder, []}]; erlang -> Encoder = megaco_erl_dist_encoder, [ @@ -419,15 +411,11 @@ expand_codec(Codec, _) -> [{Codec, megaco_compact_text_encoder, [flex_scanner]}, {Codec, megaco_compact_text_encoder, []}]; ber -> - [{Codec, megaco_ber_bin_encoder, [driver,native]}, - {Codec, megaco_ber_bin_encoder, [native]}, - {Codec, megaco_ber_bin_encoder, [driver]}, - {Codec, megaco_ber_bin_encoder, []}]; + [{Codec, megaco_ber_encoder, [native]}, + {Codec, megaco_ber_encoder, []}]; per -> - [{Codec, megaco_per_bin_encoder, [driver,native]}, - {Codec, megaco_per_bin_encoder, [native]}, - {Codec, megaco_per_bin_encoder, [driver]}, - {Codec, megaco_per_bin_encoder, []}]; + [{Codec, megaco_per_encoder, [native]}, + {Codec, megaco_per_encoder, []}]; erlang -> Encoder = megaco_erl_dist_encoder, [ diff --git a/lib/megaco/examples/meas/megaco_codec_transform.erl b/lib/megaco/examples/meas/megaco_codec_transform.erl index cfe832ff26..15db165566 100644 --- a/lib/megaco/examples/meas/megaco_codec_transform.erl +++ b/lib/megaco/examples/meas/megaco_codec_transform.erl @@ -213,11 +213,11 @@ decode_message(compact, BinMsg) -> Conf = [{version3,?V3}], do_decode(Mod, Conf, BinMsg); decode_message(ber, BinMsg) -> - Mod = megaco_ber_bin_encoder, + Mod = megaco_ber_encoder, Conf = [{version3,?V3}], do_decode(Mod, Conf, BinMsg); decode_message(per, BinMsg) -> - Mod = megaco_per_bin_encoder, + Mod = megaco_per_encoder, Conf = [{version3,?V3}], do_decode(Mod, Conf, BinMsg); decode_message(erlang, BinMsg) -> diff --git a/lib/megaco/src/app/megaco.app.src b/lib/megaco/src/app/megaco.app.src index c0d8218ac8..0ba2a866f9 100644 --- a/lib/megaco/src/app/megaco.app.src +++ b/lib/megaco/src/app/megaco.app.src @@ -23,19 +23,6 @@ {modules, [ megaco, - megaco_ber_bin_encoder, - megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3a, - megaco_ber_bin_drv_media_gateway_control_prev3b, - megaco_ber_bin_drv_media_gateway_control_prev3c, - megaco_ber_bin_drv_media_gateway_control_v3, - megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3a, - megaco_ber_bin_media_gateway_control_prev3b, - megaco_ber_bin_media_gateway_control_prev3c, - megaco_ber_bin_media_gateway_control_v3, megaco_ber_encoder, megaco_ber_media_gateway_control_v1, megaco_ber_media_gateway_control_v2, @@ -87,19 +74,6 @@ megaco_per_media_gateway_control_prev3b, megaco_per_media_gateway_control_prev3c, megaco_per_media_gateway_control_v3, - megaco_per_bin_encoder, - megaco_per_bin_drv_media_gateway_control_v1, - megaco_per_bin_drv_media_gateway_control_v2, - megaco_per_bin_drv_media_gateway_control_prev3a, - megaco_per_bin_drv_media_gateway_control_prev3b, - megaco_per_bin_drv_media_gateway_control_prev3c, - megaco_per_bin_drv_media_gateway_control_v3, - megaco_per_bin_media_gateway_control_v1, - megaco_per_bin_media_gateway_control_v2, - megaco_per_bin_media_gateway_control_prev3a, - megaco_per_bin_media_gateway_control_prev3b, - megaco_per_bin_media_gateway_control_prev3c, - megaco_per_bin_media_gateway_control_v3, megaco_pretty_text_encoder, megaco_pretty_text_encoder_v1, megaco_pretty_text_encoder_v2, diff --git a/lib/megaco/src/binary/Makefile b/lib/megaco/src/binary/Makefile index 695599b9dc..660713605e 100644 --- a/lib/megaco/src/binary/Makefile +++ b/lib/megaco/src/binary/Makefile @@ -50,46 +50,22 @@ ASN1_SPECS = $(ASN1_V1_SPEC) \ ASN1_FILES = $(ASN1_SPECS:%=%.asn) V1_SPECS = $(BER_ASN1_V1_SPEC) \ - $(BER_BIN_ASN1_V1_SPEC) \ - $(BER_BIN_DRV_ASN1_V1_SPEC) \ - $(PER_ASN1_V1_SPEC) \ - $(PER_BIN_ASN1_V1_SPEC) \ - $(PER_BIN_DRV_ASN1_V1_SPEC) + $(PER_ASN1_V1_SPEC) V2_SPECS = $(BER_ASN1_V2_SPEC) \ - $(BER_BIN_ASN1_V2_SPEC) \ - $(BER_BIN_DRV_ASN1_V2_SPEC) \ - $(PER_ASN1_V2_SPEC) \ - $(PER_BIN_ASN1_V2_SPEC) \ - $(PER_BIN_DRV_ASN1_V2_SPEC) + $(PER_ASN1_V2_SPEC) PREV3A_SPECS = $(BER_ASN1_PREV3A_SPEC) \ - $(BER_BIN_ASN1_PREV3A_SPEC) \ - $(BER_BIN_DRV_ASN1_PREV3A_SPEC) \ - $(PER_ASN1_PREV3A_SPEC) \ - $(PER_BIN_ASN1_PREV3A_SPEC) \ - $(PER_BIN_DRV_ASN1_PREV3A_SPEC) + $(PER_ASN1_PREV3A_SPEC) PREV3B_SPECS = $(BER_ASN1_PREV3B_SPEC) \ - $(BER_BIN_ASN1_PREV3B_SPEC) \ - $(BER_BIN_DRV_ASN1_PREV3B_SPEC) \ - $(PER_ASN1_PREV3B_SPEC) \ - $(PER_BIN_ASN1_PREV3B_SPEC) \ - $(PER_BIN_DRV_ASN1_PREV3B_SPEC) + $(PER_ASN1_PREV3B_SPEC) PREV3C_SPECS = $(BER_ASN1_PREV3C_SPEC) \ - $(BER_BIN_ASN1_PREV3C_SPEC) \ - $(BER_BIN_DRV_ASN1_PREV3C_SPEC) \ - $(PER_ASN1_PREV3C_SPEC) \ - $(PER_BIN_ASN1_PREV3C_SPEC) \ - $(PER_BIN_DRV_ASN1_PREV3C_SPEC) + $(PER_ASN1_PREV3C_SPEC) V3_SPECS = $(BER_ASN1_V3_SPEC) \ - $(BER_BIN_ASN1_V3_SPEC) \ - $(BER_BIN_DRV_ASN1_V3_SPEC) \ $(PER_ASN1_V3_SPEC) \ - $(PER_BIN_ASN1_V3_SPEC) \ - $(PER_BIN_DRV_ASN1_V3_SPEC) \ $(PREV3A_SPECS) $(PREV3B_SPECS) $(PREV3C_SPECS) SPECS = $(V1_SPECS) $(V2_SPECS) $(V3_SPECS) diff --git a/lib/megaco/src/binary/depend.mk b/lib/megaco/src/binary/depend.mk index c9ca34bcf6..a1318079be 100644 --- a/lib/megaco/src/binary/depend.mk +++ b/lib/megaco/src/binary/depend.mk @@ -19,17 +19,8 @@ # Flag description: # -# +optimize -# For ber_bin this means "optimize" (whatever that is), -# but for per_bin it means that a stage in the encode -# is done in the asn1 driver. -# -# +nif -# For ber_bin this means that part of the decode is done -# in the asn1 nif. -# # +asn1config -# This is only used by the ber_bin, and means that +# This is only used by the ber, and means that # some partial decode functions will be created # (as described by the asn1config file). # @@ -43,42 +34,18 @@ ifeq ($(MEGACO_INLINE_ASN1_RT),true) ASN1_CT_OPTS += +inline endif -BER_V1_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_V1_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_V1_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif -BER_V2_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_V2_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_V2_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif -BER_PREV3A_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_PREV3A_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_PREV3A_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif -BER_PREV3B_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_PREV3B_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_PREV3B_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif -BER_PREV3C_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_PREV3C_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_PREV3C_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif -BER_V3_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_V3_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_V3_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif +BER_V1_FLAGS = $(ASN1_CT_OPTS) +asn1config +BER_V2_FLAGS = $(ASN1_CT_OPTS) +asn1config +BER_PREV3A_FLAGS = $(ASN1_CT_OPTS) +asn1config +BER_PREV3B_FLAGS = $(ASN1_CT_OPTS) +asn1config +BER_PREV3C_FLAGS = $(ASN1_CT_OPTS) +asn1config +BER_V3_FLAGS = $(ASN1_CT_OPTS) +asn1config PER_V1_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_V1_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_V1_FLAGS = $(ASN1_CT_OPTS) +optimize PER_V2_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_V2_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_V2_FLAGS = $(ASN1_CT_OPTS) +optimize PER_PREV3A_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_PREV3A_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_PREV3A_FLAGS = $(ASN1_CT_OPTS) +optimize PER_PREV3B_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_PREV3B_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_PREV3B_FLAGS = $(ASN1_CT_OPTS) +optimize PER_PREV3C_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_PREV3C_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_PREV3C_FLAGS = $(ASN1_CT_OPTS) +optimize PER_V3_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_V3_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_V3_FLAGS = $(ASN1_CT_OPTS) +optimize # --- Version 1 --- @@ -92,26 +59,6 @@ $(BER_ASN1_V1_SPEC).erl: \ $(EBIN)/$(BER_ASN1_V1_SPEC).$(EMULATOR): \ $(BER_ASN1_V1_SPEC).erl -$(BER_BIN_ASN1_V1_SPEC).erl: \ - $(BER_BIN_ASN1_V1_SPEC).set.asn \ - $(BER_BIN_ASN1_V1_SPEC).asn1config \ - $(ASN1_V1_SPEC).asn - @echo "$(BER_BIN_ASN1_V1_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_V1_FLAGS) $(BER_BIN_ASN1_V1_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_V1_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_V1_SPEC).erl - -$(BER_BIN_DRV_ASN1_V1_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_V1_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_V1_SPEC).asn1config \ - $(ASN1_V1_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_V1_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_V1_FLAGS) $(BER_BIN_DRV_ASN1_V1_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_V1_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_V1_SPEC).erl - $(PER_ASN1_V1_SPEC).erl: \ $(PER_ASN1_V1_SPEC).set.asn \ $(ASN1_V1_SPEC).asn @@ -121,24 +68,6 @@ $(PER_ASN1_V1_SPEC).erl: \ $(EBIN)/$(PER_ASN1_V1_SPEC).$(EMULATOR): \ $(PER_ASN1_V1_SPEC).erl -$(PER_BIN_ASN1_V1_SPEC).erl: \ - $(PER_BIN_ASN1_V1_SPEC).set.asn \ - $(ASN1_V1_SPEC).asn - @echo "$(PER_BIN_ASN1_V1_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_V1_FLAGS) $(PER_BIN_ASN1_V1_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_V1_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_V1_SPEC).erl - -$(PER_BIN_DRV_ASN1_V1_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_V1_SPEC).set.asn \ - $(ASN1_V1_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_V1_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_V1_FLAGS) $(PER_BIN_DRV_ASN1_V1_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_V1_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_V1_SPEC).erl - # --- Version 2 --- @@ -151,26 +80,6 @@ $(BER_ASN1_V2_SPEC).erl: \ $(EBIN)/$(BER_ASN1_V2_SPEC).$(EMULATOR): \ $(BER_ASN1_V2_SPEC).erl -$(BER_BIN_ASN1_V2_SPEC).erl: \ - $(BER_BIN_ASN1_V2_SPEC).set.asn \ - $(BER_BIN_ASN1_V2_SPEC).asn1config \ - $(ASN1_V2_SPEC).asn - @echo "$(BER_BIN_ASN1_V2_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_V2_FLAGS) $(BER_BIN_ASN1_V2_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_V2_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_V2_SPEC).erl - -$(BER_BIN_DRV_ASN1_V2_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_V2_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_V2_SPEC).asn1config \ - $(ASN1_V2_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_V2_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_V2_FLAGS) $(BER_BIN_DRV_ASN1_V2_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_V2_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_V2_SPEC).erl - $(PER_ASN1_V2_SPEC).erl: \ $(PER_ASN1_V2_SPEC).set.asn \ $(ASN1_V2_SPEC).asn @@ -180,25 +89,6 @@ $(PER_ASN1_V2_SPEC).erl: \ $(EBIN)/$(PER_ASN1_V2_SPEC).$(EMULATOR): \ $(PER_ASN1_V2_SPEC).erl -$(PER_BIN_ASN1_V2_SPEC).erl: \ - $(PER_BIN_ASN1_V2_SPEC).set.asn \ - $(ASN1_V2_SPEC).asn - @echo "$(PER_BIN_ASN1_V2_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_V2_FLAGS) $(PER_BIN_ASN1_V2_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_V2_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_V2_SPEC).erl - -$(PER_BIN_DRV_ASN1_V2_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_V2_SPEC).set.asn \ - $(ASN1_V2_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_V2_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_V2_FLAGS) $(PER_BIN_DRV_ASN1_V2_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_V2_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_V2_SPEC).erl - - # --- Version 3 --- # -- (prev3a) -- @@ -212,26 +102,6 @@ $(BER_ASN1_PREV3A_SPEC).erl: \ $(EBIN)/$(BER_ASN1_PREV3A_SPEC).$(EMULATOR): \ $(BER_ASN1_PREV3A_SPEC).erl -$(BER_BIN_ASN1_PREV3A_SPEC).erl: \ - $(BER_BIN_ASN1_PREV3A_SPEC).set.asn \ - $(BER_BIN_ASN1_PREV3A_SPEC).asn1config \ - $(ASN1_PREV3A_SPEC).asn - @echo "$(BER_BIN_ASN1_PREV3A_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_PREV3A_FLAGS) $(BER_BIN_ASN1_PREV3A_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_PREV3A_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_PREV3A_SPEC).erl - -$(BER_BIN_DRV_ASN1_PREV3A_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_PREV3A_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_PREV3A_SPEC).asn1config \ - $(ASN1_PREV3A_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_PREV3A_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_PREV3A_FLAGS) $(BER_BIN_DRV_ASN1_PREV3A_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_PREV3A_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_PREV3A_SPEC).erl - $(PER_ASN1_PREV3A_SPEC).erl: \ $(PER_ASN1_PREV3A_SPEC).set.asn \ $(ASN1_PREV3A_SPEC).asn @@ -241,23 +111,6 @@ $(PER_ASN1_PREV3A_SPEC).erl: \ $(EBIN)/$(PER_ASN1_PREV3A_SPEC).$(EMULATOR): \ $(PER_ASN1_PREV3A_SPEC).erl -$(PER_BIN_ASN1_PREV3A_SPEC).erl: \ - $(PER_BIN_ASN1_PREV3A_SPEC).set.asn \ - $(ASN1_PREV3A_SPEC).asn - @echo "$(PER_BIN_ASN1_PREV3A_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_PREV3A_FLAGS) $(PER_BIN_ASN1_PREV3A_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_PREV3A_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_PREV3A_SPEC).erl - -$(PER_BIN_DRV_ASN1_PREV3A_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_PREV3A_SPEC).set.asn \ - $(ASN1_PREV3A_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_PREV3A_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_PREV3A_FLAGS) $(PER_BIN_DRV_ASN1_PREV3A_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_PREV3A_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_PREV3A_SPEC).erl # -- (prev3b) -- @@ -270,26 +123,6 @@ $(BER_ASN1_PREV3B_SPEC).erl: \ $(EBIN)/$(BER_ASN1_PREV3B_SPEC).$(EMULATOR): \ $(BER_ASN1_PREV3B_SPEC).erl -$(BER_BIN_ASN1_PREV3B_SPEC).erl: \ - $(BER_BIN_ASN1_PREV3B_SPEC).set.asn \ - $(BER_BIN_ASN1_PREV3B_SPEC).asn1config \ - $(ASN1_PREV3B_SPEC).asn - @echo "$(BER_BIN_ASN1_PREV3B_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_PREV3B_FLAGS) $(BER_BIN_ASN1_PREV3B_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_PREV3B_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_PREV3B_SPEC).erl - -$(BER_BIN_DRV_ASN1_PREV3B_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_PREV3B_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_PREV3B_SPEC).asn1config \ - $(ASN1_PREV3B_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_PREV3B_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_PREV3B_FLAGS) $(BER_BIN_DRV_ASN1_PREV3B_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_PREV3B_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_PREV3B_SPEC).erl - $(PER_ASN1_PREV3B_SPEC).erl: \ $(PER_ASN1_PREV3B_SPEC).set.asn \ $(ASN1_PREV3B_SPEC).asn @@ -299,24 +132,6 @@ $(PER_ASN1_PREV3B_SPEC).erl: \ $(EBIN)/$(PER_ASN1_PREV3B_SPEC).$(EMULATOR): \ $(PER_ASN1_PREV3B_SPEC).erl -$(PER_BIN_ASN1_PREV3B_SPEC).erl: \ - $(PER_BIN_ASN1_PREV3B_SPEC).set.asn \ - $(ASN1_PREV3B_SPEC).asn - @echo "$(PER_BIN_ASN1_PREV3B_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_PREV3B_FLAGS) $(PER_BIN_ASN1_PREV3B_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_PREV3B_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_PREV3B_SPEC).erl - -$(PER_BIN_DRV_ASN1_PREV3B_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_PREV3B_SPEC).set.asn \ - $(ASN1_PREV3B_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_PREV3B_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_PREV3B_FLAGS) $(PER_BIN_DRV_ASN1_PREV3B_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_PREV3B_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_PREV3B_SPEC).erl - # -- (prev3c) -- @@ -329,26 +144,6 @@ $(BER_ASN1_PREV3C_SPEC).erl: \ $(EBIN)/$(BER_ASN1_PREV3C_SPEC).$(EMULATOR): \ $(BER_ASN1_PREV3C_SPEC).erl -$(BER_BIN_ASN1_PREV3C_SPEC).erl: \ - $(BER_BIN_ASN1_PREV3C_SPEC).set.asn \ - $(BER_BIN_ASN1_PREV3C_SPEC).asn1config \ - $(ASN1_PREV3C_SPEC).asn - @echo "$(BER_BIN_ASN1_PREV3C_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_PREV3C_FLAGS) $(BER_BIN_ASN1_PREV3C_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_PREV3C_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_PREV3C_SPEC).erl - -$(BER_BIN_DRV_ASN1_PREV3C_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_PREV3C_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_PREV3C_SPEC).asn1config \ - $(ASN1_PREV3C_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_PREV3C_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_PREV3C_FLAGS) $(BER_BIN_DRV_ASN1_PREV3C_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_PREV3C_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_PREV3C_SPEC).erl - $(PER_ASN1_PREV3C_SPEC).erl: \ $(PER_ASN1_PREV3C_SPEC).set.asn \ $(ASN1_PREV3C_SPEC).asn @@ -358,24 +153,6 @@ $(PER_ASN1_PREV3C_SPEC).erl: \ $(EBIN)/$(PER_ASN1_PREV3C_SPEC).$(EMULATOR): \ $(PER_ASN1_PREV3C_SPEC).erl -$(PER_BIN_ASN1_PREV3C_SPEC).erl: \ - $(PER_BIN_ASN1_PREV3C_SPEC).set.asn \ - $(ASN1_PREV3C_SPEC).asn - @echo "$(PER_BIN_ASN1_PREV3C_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_PREV3C_FLAGS) $(PER_BIN_ASN1_PREV3C_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_PREV3C_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_PREV3C_SPEC).erl - -$(PER_BIN_DRV_ASN1_PREV3C_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_PREV3C_SPEC).set.asn \ - $(ASN1_PREV3C_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_PREV3C_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_PREV3C_FLAGS) $(PER_BIN_DRV_ASN1_PREV3C_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_PREV3C_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_PREV3C_SPEC).erl - # -- (v3) -- @@ -388,26 +165,6 @@ $(BER_ASN1_V3_SPEC).erl: \ $(EBIN)/$(BER_ASN1_V3_SPEC).$(EMULATOR): \ $(BER_ASN1_V3_SPEC).erl -$(BER_BIN_ASN1_V3_SPEC).erl: \ - $(BER_BIN_ASN1_V3_SPEC).set.asn \ - $(BER_BIN_ASN1_V3_SPEC).asn1config \ - $(ASN1_V3_SPEC).asn - @echo "$(BER_BIN_ASN1_V3_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_V3_FLAGS) $(BER_BIN_ASN1_V3_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_V3_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_V3_SPEC).erl - -$(BER_BIN_DRV_ASN1_V3_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_V3_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_V3_SPEC).asn1config \ - $(ASN1_V3_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_V3_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_V3_FLAGS) $(BER_BIN_DRV_ASN1_V3_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_V3_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_V3_SPEC).erl - $(PER_ASN1_V3_SPEC).erl: \ $(PER_ASN1_V3_SPEC).set.asn \ $(ASN1_V3_SPEC).asn @@ -417,39 +174,15 @@ $(PER_ASN1_V3_SPEC).erl: \ $(EBIN)/$(PER_ASN1_V3_SPEC).$(EMULATOR): \ $(PER_ASN1_V3_SPEC).erl -$(PER_BIN_ASN1_V3_SPEC).erl: \ - $(PER_BIN_ASN1_V3_SPEC).set.asn \ - $(ASN1_V3_SPEC).asn - @echo "$(PER_BIN_ASN1_V3_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_V3_FLAGS) $(PER_BIN_ASN1_V3_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_V3_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_V3_SPEC).erl - -$(PER_BIN_DRV_ASN1_V3_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_V3_SPEC).set.asn \ - $(ASN1_V3_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_V3_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_V3_FLAGS) $(PER_BIN_DRV_ASN1_V3_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_V3_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_V3_SPEC).erl - # ------------- $(EBIN)/megaco_ber_encoder.$(EMULATOR): megaco_ber_encoder.erl \ $(MEGACO_ENGINEDIR)/megaco_message_internal.hrl -$(EBIN)/megaco_ber_bin_encoder.$(EMULATOR): megaco_ber_bin_encoder.erl \ - $(MEGACO_ENGINEDIR)/megaco_message_internal.hrl - $(EBIN)/megaco_per_encoder.$(EMULATOR): megaco_per_encoder.erl \ $(MEGACO_ENGINEDIR)/megaco_message_internal.hrl -$(EBIN)/megaco_per_bin_encoder.$(EMULATOR): megaco_per_bin_encoder.erl \ - $(MEGACO_ENGINEDIR)/megaco_message_internal.hrl - $(EBIN)/megaco_binary_encoder_lib.$(EMULATOR): megaco_binary_encoder_lib.erl \ $(MEGACO_ENGINEDIR)/megaco_message_internal.hrl diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.asn1config b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.asn1config deleted file mode 100644 index 179473717d..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_prev3a', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.set.asn deleted file mode 100644 index b9ba7ffdb4..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3a.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.asn1config b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.asn1config deleted file mode 100644 index ceda97fbd1..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_prev3b', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.set.asn deleted file mode 100644 index 0437bde310..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3b.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.asn1config b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.asn1config deleted file mode 100644 index d181ef44bd..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_prev3c', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.set.asn deleted file mode 100644 index e78055fbad..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3c.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v1.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v1.set.asn deleted file mode 100644 index 0f5a92dba1..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v1.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v1.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.asn1config b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.asn1config deleted file mode 100644 index 3d0cb9a019..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_v2', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.set.asn deleted file mode 100644 index 7fc82b127f..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v2.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.asn1config b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.asn1config deleted file mode 100644 index cc662c0145..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_v3', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.set.asn deleted file mode 100644 index 1d7950a283..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v3.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_encoder.erl b/lib/megaco/src/binary/megaco_ber_bin_encoder.erl deleted file mode 100644 index bf9926c7e5..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_encoder.erl +++ /dev/null @@ -1,716 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2000-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% - -%% -%%---------------------------------------------------------------------- -%% Purpose : Handle ASN.1 BER encoding of Megaco/H.248 -%%---------------------------------------------------------------------- - --module(megaco_ber_bin_encoder). - --behaviour(megaco_encoder). - --export([encode_message/3, decode_message/3, - decode_mini_message/3, - - encode_transaction/3, - encode_action_requests/3, - encode_action_request/3, - encode_action_reply/3, - - version_of/2]). - -%% Backward compatible functions: --export([encode_message/2, decode_message/2]). - --include_lib("megaco/src/engine/megaco_message_internal.hrl"). - --define(V1_ASN1_MOD, megaco_ber_bin_media_gateway_control_v1). --define(V2_ASN1_MOD, megaco_ber_bin_media_gateway_control_v2). --define(V3_ASN1_MOD, megaco_ber_bin_media_gateway_control_v3). --define(PREV3A_ASN1_MOD, megaco_ber_bin_media_gateway_control_prev3a). --define(PREV3B_ASN1_MOD, megaco_ber_bin_media_gateway_control_prev3b). --define(PREV3C_ASN1_MOD, megaco_ber_bin_media_gateway_control_prev3c). --define(V1_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_v1). --define(V2_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_v2). --define(V3_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_v3). --define(PREV3A_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_prev3a). --define(PREV3B_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_prev3b). --define(PREV3C_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_prev3c). - --define(V1_TRANS_MOD, megaco_binary_transformer_v1). --define(V2_TRANS_MOD, megaco_binary_transformer_v2). --define(V3_TRANS_MOD, megaco_binary_transformer_v3). --define(PREV3A_TRANS_MOD, megaco_binary_transformer_prev3a). --define(PREV3B_TRANS_MOD, megaco_binary_transformer_prev3b). --define(PREV3C_TRANS_MOD, megaco_binary_transformer_prev3c). - --define(BIN_LIB, megaco_binary_encoder_lib). - - -%%---------------------------------------------------------------------- -%% Detect (check/get) message version -%% Return {ok, Version} | {error, Reason} -%%---------------------------------------------------------------------- - -version_of([{version3,v3},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?V3_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3c},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3C_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3b},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3B_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3a},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3A_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,v3}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?V3_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3c}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3C_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3b}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3B_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3a}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3A_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?V3_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -version_of(EC, Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?V3_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders). - - -%%---------------------------------------------------------------------- -%% Convert a 'MegacoMessage' record into a binary -%% Return {ok, Binary} | {error, Reason} -%%---------------------------------------------------------------------- - - -encode_message(EC, - #'MegacoMessage'{mess = #'Message'{version = V}} = MegaMsg) -> - encode_message(EC, V, MegaMsg). - - -%% -- Version 1 -- - -encode_message([{version3, _},driver|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,_}|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - - -%% -- Version 2 -- - -encode_message([{version3,_},driver|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,_}|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - - -%% -- Version 3 -- - -encode_message([{version3,v3},driver|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3c},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3C_ASN1_MOD_DRV, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3b},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3B_ASN1_MOD_DRV, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3a},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3A_ASN1_MOD_DRV, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,v3}|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3c}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3C_ASN1_MOD, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3b}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3B_ASN1_MOD, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3a}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3A_ASN1_MOD, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list). - - -%%---------------------------------------------------------------------- -%% Convert a transaction (or transactions in the case of ack) record(s) -%% into a binary -%% Return {ok, Binary} | {error, Reason} -%%---------------------------------------------------------------------- - -%% encode_transaction([] = EC, 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD_DRV, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction(_EC, 1, _Trans) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([] = EC, 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD_DRV, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%%encode_transaction(_EC, 2, _Trans) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list). -%% encode_transaction([] = EC, 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD_DRV, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%%encode_transaction(_EC, 3, _Trans) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list). -encode_transaction(_EC, _V, _Trans) -> - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a list of ActionRequest record's into a binary -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- -%% encode_action_requests([] = EC, 1, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([native] = EC, 1, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([driver|EC], 1, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V1_ASN1_MOD_DRV, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests(_EC, 1, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([] = EC, 2, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([native] = EC, 2, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([driver|EC], 2, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V2_ASN1_MOD_DRV, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests(_EC, 2, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([] = EC, 3, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([native] = EC, 3, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([driver|EC], 3, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V3_ASN1_MOD_DRV, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%%encode_action_requests(_EC, 3, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests(_EC, V, _ActReqs) -> -%% {error, {bad_version, V}}. -encode_action_requests(_EC, _V, _ActReqs) -> - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a ActionRequest record into a binary -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- -%% encode_action_request([] = EC, 1, ActReq) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([native] = EC, 1, ActReq) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([driver|EC], 1, ActReq) -> -%% AsnMod = ?V1_ASN1_MOD_DRV, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request(_EC, 1, ActReq) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([] = EC, 2, ActReq) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([native] = EC, 2, ActReq) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([driver|EC], 2, ActReq) -> -%% AsnMod = ?V2_ASN1_MOD_DRV, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request(_EC, 2, ActReq) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([] = EC, 3, ActReq) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([native] = EC, 3, ActReq) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([driver|EC], 3, ActReq) -> -%% AsnMod = ?V3_ASN1_MOD_DRV, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request(_EC, 3, ActReq) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -encode_action_request(_EC, _V, _ActReq) -> - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a action reply into a deep io list -%% Not yest supported by this binary codec! -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- - -encode_action_reply(_EC, _V, _AcionReply) -> - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a binary into a 'MegacoMessage' record -%% Return {ok, MegacoMessageRecord} | {error, Reason} -%%---------------------------------------------------------------------- - -%% Old decode function -decode_message(EC, Binary) -> - decode_message(EC, 1, Binary). - -%% -- Dynamic version detection -- - -%% Select from message -decode_message([{version3,v3},driver|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD_DRV, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD_DRV, ?V2_TRANS_MOD}, - {?V3_ASN1_MOD_DRV, ?V3_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3c},driver|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD_DRV, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD_DRV, ?V2_TRANS_MOD}, - {?PREV3C_ASN1_MOD_DRV, ?PREV3C_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3b},driver|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD_DRV, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD_DRV, ?V2_TRANS_MOD}, - {?PREV3B_ASN1_MOD_DRV, ?PREV3B_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3a},driver|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD_DRV, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD_DRV, ?V2_TRANS_MOD}, - {?PREV3A_ASN1_MOD_DRV, ?PREV3A_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,v3}|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD, ?V2_TRANS_MOD}, - {?V3_ASN1_MOD, ?V3_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3c}|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD, ?V2_TRANS_MOD}, - {?PREV3C_ASN1_MOD, ?PREV3C_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3b}|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD, ?V2_TRANS_MOD}, - {?PREV3B_ASN1_MOD, ?PREV3B_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3a}|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD, ?V2_TRANS_MOD}, - {?PREV3A_ASN1_MOD, ?PREV3A_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([driver|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD_DRV, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD_DRV, ?V2_TRANS_MOD}, - {?V3_ASN1_MOD_DRV, ?V3_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD, ?V2_TRANS_MOD}, - {?V3_ASN1_MOD, ?V3_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); - - -%% -- Version 1 -- - -decode_message([{version3,_},driver|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,_}|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 1, Binary) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - - -%% -- Version 2 -- - -decode_message([{version3,_},driver|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,_}|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 2, Binary) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - - -%% -- Version 3 -- - -decode_message([{version3,v3},driver|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3c},driver|EC], 3, Binary) -> - AsnMod = ?PREV3C_ASN1_MOD_DRV, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3b},driver|EC], 3, Binary) -> - AsnMod = ?PREV3B_ASN1_MOD_DRV, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3a},driver|EC], 3, Binary) -> - AsnMod = ?PREV3A_ASN1_MOD_DRV, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,v3}|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3c}|EC], 3, Binary) -> - AsnMod = ?PREV3C_ASN1_MOD, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3b}|EC], 3, Binary) -> - AsnMod = ?PREV3B_ASN1_MOD, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3a}|EC], 3, Binary) -> - AsnMod = ?PREV3A_ASN1_MOD, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 3, Binary) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary). - - -decode_mini_message([{version3,v3},driver|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD_DRV, - ?V2_ASN1_MOD_DRV, - ?V3_ASN1_MOD_DRV], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3c},driver|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD_DRV, - ?V2_ASN1_MOD_DRV, - ?PREV3C_ASN1_MOD_DRV], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3b},driver|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD_DRV, - ?V2_ASN1_MOD_DRV, - ?PREV3B_ASN1_MOD_DRV], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3a},driver|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD_DRV, - ?V2_ASN1_MOD_DRV, - ?PREV3A_ASN1_MOD_DRV], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,v3}|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD, - ?V2_ASN1_MOD, - ?V3_ASN1_MOD], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3c}|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD, - ?V2_ASN1_MOD, - ?PREV3C_ASN1_MOD], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3b}|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD, - ?V2_ASN1_MOD, - ?PREV3B_ASN1_MOD], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3a}|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD, - ?V2_ASN1_MOD, - ?PREV3A_ASN1_MOD], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([driver|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD_DRV, - ?V2_ASN1_MOD_DRV, - ?V3_ASN1_MOD_DRV], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message(EC, dynamic, Bin) -> - Mods = [?V1_ASN1_MOD, - ?V2_ASN1_MOD, - ?V3_ASN1_MOD], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,_},driver|EC], 1, Bin) -> - AsnMod = ?V1_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,_}|EC], 1, Bin) -> - AsnMod = ?V1_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 1, Bin) -> - AsnMod = ?V1_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message(EC, 1, Bin) -> - AsnMod = ?V1_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,_},driver|EC], 2, Bin) -> - AsnMod = ?V2_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,_}|EC], 2, Bin) -> - AsnMod = ?V2_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 2, Bin) -> - AsnMod = ?V2_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message(EC, 2, Bin) -> - AsnMod = ?V2_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,v3},driver|EC], 3, Bin) -> - AsnMod = ?V3_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3c},driver|EC], 3, Bin) -> - AsnMod = ?PREV3C_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3b},driver|EC], 3, Bin) -> - AsnMod = ?PREV3B_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3a},driver|EC], 3, Bin) -> - AsnMod = ?PREV3A_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,v3}|EC], 3, Bin) -> - AsnMod = ?V3_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3c}|EC], 3, Bin) -> - AsnMod = ?PREV3C_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3b}|EC], 3, Bin) -> - AsnMod = ?PREV3B_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3a}|EC], 3, Bin) -> - AsnMod = ?PREV3A_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 3, Bin) -> - AsnMod = ?V3_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message(EC, 3, Bin) -> - AsnMod = ?V3_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary). diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3a.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3a.set.asn deleted file mode 100644 index b9ba7ffdb4..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3a.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3a.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3b.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3b.set.asn deleted file mode 100644 index 0437bde310..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3b.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3b.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.asn1config b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.asn1config deleted file mode 100644 index c74422b9a2..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_media_gateway_control_prev3c', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.set.asn deleted file mode 100644 index e78055fbad..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3c.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.asn1config b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.asn1config deleted file mode 100644 index e815e90948..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.asn1config +++ /dev/null @@ -1,44 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_media_gateway_control_v1', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.set.asn deleted file mode 100644 index 0f5a92dba1..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v1.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v2.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v2.set.asn deleted file mode 100644 index 7fc82b127f..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v2.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v2.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v3.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v3.set.asn deleted file mode 100644 index 1d7950a283..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v3.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v3.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v2.asn1config b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3a.asn1config index cc072b30ee..da67561f1b 100644 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v2.asn1config +++ b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3a.asn1config @@ -1,5 +1,5 @@ {exclusive_decode, - {'megaco_ber_bin_media_gateway_control_v2', + {'megaco_ber_media_gateway_control_prev3a', [ {decode_message_trans_partial, [ @@ -14,7 +14,7 @@ {actions,parts} ] }, - {transactionReply, + {transactionReply, [ {transactionResult, [{actionReplies,parts}]} ] diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v3.asn1config b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3b.asn1config index deeb2b2da9..2f25f03d97 100644 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v3.asn1config +++ b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3b.asn1config @@ -1,5 +1,5 @@ {exclusive_decode, - {'megaco_ber_bin_media_gateway_control_v3', + {'megaco_ber_media_gateway_control_prev3b', [ {decode_message_trans_partial, [ @@ -14,7 +14,7 @@ {actions,parts} ] }, - {transactionReply, + {transactionReply, [ {transactionResult, [{actionReplies,parts}]} ] diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3a.asn1config b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3c.asn1config index 456ce750ad..23c343eed0 100644 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3a.asn1config +++ b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3c.asn1config @@ -1,5 +1,5 @@ {exclusive_decode, - {'megaco_ber_bin_media_gateway_control_prev3a', + {'megaco_ber_media_gateway_control_prev3c', [ {decode_message_trans_partial, [ @@ -14,7 +14,7 @@ {actions,parts} ] }, - {transactionReply, + {transactionReply, [ {transactionResult, [{actionReplies,parts}]} ] diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v1.asn1config b/lib/megaco/src/binary/megaco_ber_media_gateway_control_v1.asn1config index ea10a7d527..951825c0aa 100644 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v1.asn1config +++ b/lib/megaco/src/binary/megaco_ber_media_gateway_control_v1.asn1config @@ -1,5 +1,5 @@ {exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_v1', + {'megaco_ber_media_gateway_control_v1', [ {decode_message_trans_partial, [ @@ -14,7 +14,7 @@ {actions,parts} ] }, - {transactionReply, + {transactionReply, [ {transactionResult, [{actionReplies,parts}]} ] @@ -38,7 +38,7 @@ ] ] } + ] } }. - diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3b.asn1config b/lib/megaco/src/binary/megaco_ber_media_gateway_control_v3.asn1config index fa5cd80baf..e4b1f9ece9 100644 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3b.asn1config +++ b/lib/megaco/src/binary/megaco_ber_media_gateway_control_v3.asn1config @@ -1,5 +1,5 @@ {exclusive_decode, - {'megaco_ber_bin_media_gateway_control_prev3b', + {'megaco_ber_media_gateway_control_v3', [ {decode_message_trans_partial, [ @@ -14,7 +14,7 @@ {actions,parts} ] }, - {transactionReply, + {transactionReply, [ {transactionResult, [{actionReplies,parts}]} ] diff --git a/lib/megaco/src/binary/megaco_binary_encoder.erl b/lib/megaco/src/binary/megaco_binary_encoder.erl index f825f91a45..51e167590d 100644 --- a/lib/megaco/src/binary/megaco_binary_encoder.erl +++ b/lib/megaco/src/binary/megaco_binary_encoder.erl @@ -241,55 +241,30 @@ encode_action_reply(_EC, _V, _AcionReply) -> %% Return {ok, Version} | {error, Reason} %%---------------------------------------------------------------------- -version_of([{version3,v3},driver|EC], Binary) -> - Decoders = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_v3], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3c},driver|EC], Binary) -> - Decoders = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3c], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3b},driver|EC], Binary) -> - Decoders = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3b], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3a},driver|EC], Binary) -> - Decoders = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3a], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([driver|EC], Binary) -> - Decoders = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_v3], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); version_of([{version3,v3}|EC], Binary) -> - Decoders = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_v3], + Decoders = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_v3], ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); version_of([{version3,prev3c}|EC], Binary) -> - Decoders = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3c], + Decoders = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3c], ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); version_of([{version3,prev3b}|EC], Binary) -> - Decoders = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3b], + Decoders = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3b], ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); version_of([{version3,prev3a}|EC], Binary) -> - Decoders = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3a], + Decoders = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3a], ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); version_of(EC, Binary) -> - Decoders = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_v3], + Decoders = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_v3], ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders). @@ -301,287 +276,153 @@ version_of(EC, Binary) -> decode_message(EC, Binary) -> decode_message(EC, 1, Binary). -decode_message([{version3,v3},driver|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_drv_media_gateway_control_v1, - megaco_binary_transformer_v1}, - {megaco_ber_bin_drv_media_gateway_control_v2, - megaco_binary_transformer_v2}, - {megaco_ber_bin_drv_media_gateway_control_v3, - megaco_binary_transformer_v3}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); -decode_message([{version3,prev3c},driver|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_drv_media_gateway_control_v1, - megaco_binary_transformer_v1}, - {megaco_ber_bin_drv_media_gateway_control_v2, - megaco_binary_transformer_v2}, - {megaco_ber_bin_drv_media_gateway_control_prev3c, - megaco_binary_transformer_prev3c}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); -decode_message([{version3,prev3b},driver|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_drv_media_gateway_control_v1, - megaco_binary_transformer_v1}, - {megaco_ber_bin_drv_media_gateway_control_v2, - megaco_binary_transformer_v2}, - {megaco_ber_bin_drv_media_gateway_control_prev3b, - megaco_binary_transformer_prev3b}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); -decode_message([{version3,prev3a},driver|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_drv_media_gateway_control_v1, - megaco_binary_transformer_v1}, - {megaco_ber_bin_drv_media_gateway_control_v2, - megaco_binary_transformer_v2}, - {megaco_ber_bin_drv_media_gateway_control_prev3a, - megaco_binary_transformer_prev3a}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); -decode_message([driver|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_drv_media_gateway_control_v1, - megaco_binary_transformer_v1}, - {megaco_ber_bin_drv_media_gateway_control_v2, - megaco_binary_transformer_v2}, - {megaco_ber_bin_drv_media_gateway_control_v3, - megaco_binary_transformer_v3}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); decode_message([{version3,v3}|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_media_gateway_control_v1, + Decoders = [{megaco_ber_media_gateway_control_v1, megaco_binary_transformer_v1}, - {megaco_ber_bin_media_gateway_control_v2, + {megaco_ber_media_gateway_control_v2, megaco_binary_transformer_v2}, - {megaco_ber_bin_media_gateway_control_v3, + {megaco_ber_media_gateway_control_v3, megaco_binary_transformer_v3}], ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); decode_message([{version3,prev3c}|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_media_gateway_control_v1, + Decoders = [{megaco_ber_media_gateway_control_v1, megaco_binary_transformer_v1}, - {megaco_ber_bin_media_gateway_control_v2, + {megaco_ber_media_gateway_control_v2, megaco_binary_transformer_v2}, - {megaco_ber_bin_media_gateway_control_prev3c, + {megaco_ber_media_gateway_control_prev3c, megaco_binary_transformer_prev3c}], ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); decode_message([{version3,prev3b}|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_media_gateway_control_v1, + Decoders = [{megaco_ber_media_gateway_control_v1, megaco_binary_transformer_v1}, - {megaco_ber_bin_media_gateway_control_v2, + {megaco_ber_media_gateway_control_v2, megaco_binary_transformer_v2}, - {megaco_ber_bin_media_gateway_control_prev3b, + {megaco_ber_media_gateway_control_prev3b, megaco_binary_transformer_prev3b}], ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); decode_message([{version3,prev3a}|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_media_gateway_control_v1, + Decoders = [{megaco_ber_media_gateway_control_v1, megaco_binary_transformer_v1}, - {megaco_ber_bin_media_gateway_control_v2, + {megaco_ber_media_gateway_control_v2, megaco_binary_transformer_v2}, - {megaco_ber_bin_media_gateway_control_prev3a, + {megaco_ber_media_gateway_control_prev3a, megaco_binary_transformer_prev3a}], ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); decode_message(EC, dynamic, Binary) -> - Decoders = [{megaco_ber_bin_media_gateway_control_v1, + Decoders = [{megaco_ber_media_gateway_control_v1, megaco_binary_transformer_v1}, - {megaco_ber_bin_media_gateway_control_v2, + {megaco_ber_media_gateway_control_v2, megaco_binary_transformer_v2}, - {megaco_ber_bin_media_gateway_control_v3, + {megaco_ber_media_gateway_control_v3, megaco_binary_transformer_v3}], ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); %% -- Version 1 -- -decode_message([{version3,_},driver|EC], 1, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v1, - TransMod = megaco_binary_transformer_v1, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -decode_message([driver|EC], 1, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v1, - TransMod = megaco_binary_transformer_v1, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - decode_message([{version3,_}|EC], 1, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v1, + AsnMod = megaco_ber_media_gateway_control_v1, TransMod = megaco_binary_transformer_v1, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message(EC, 1, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v1, + AsnMod = megaco_ber_media_gateway_control_v1, TransMod = megaco_binary_transformer_v1, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); %% -- Version 2 -- -decode_message([{version3,_},driver|EC], 2, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v2, - TransMod = megaco_binary_transformer_v2, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -decode_message([driver|EC], 2, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v2, - TransMod = megaco_binary_transformer_v2, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - decode_message([{version3,_}|EC], 2, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v2, + AsnMod = megaco_ber_media_gateway_control_v2, TransMod = megaco_binary_transformer_v2, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message(EC, 2, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v2, + AsnMod = megaco_ber_media_gateway_control_v2, TransMod = megaco_binary_transformer_v2, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); %% -- Version 3 -- -decode_message([{version3,v3},driver|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v3, - TransMod = megaco_binary_transformer_v3, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3c},driver|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3c, - TransMod = megaco_binary_transformer_prev3c, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3b},driver|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3b, - TransMod = megaco_binary_transformer_prev3b, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3a},driver|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3a, - TransMod = megaco_binary_transformer_prev3a, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -decode_message([driver|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v3, - TransMod = megaco_binary_transformer_v3, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - decode_message([{version3,v3}|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v3, + AsnMod = megaco_ber_media_gateway_control_v3, TransMod = megaco_binary_transformer_v3, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message([{version3,prev3c}|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3c, + AsnMod = megaco_ber_media_gateway_control_prev3c, TransMod = megaco_binary_transformer_prev3c, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message([{version3,prev3b}|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3b, + AsnMod = megaco_ber_media_gateway_control_prev3b, TransMod = megaco_binary_transformer_prev3b, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message([{version3,prev3a}|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3a, + AsnMod = megaco_ber_media_gateway_control_prev3a, TransMod = megaco_binary_transformer_prev3a, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message(EC, 3, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v3, + AsnMod = megaco_ber_media_gateway_control_v3, TransMod = megaco_binary_transformer_v3, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary). -decode_mini_message([{version3,v3},driver|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_v3], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3c},driver|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3c], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3b},driver|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3b], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3a},driver|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3a], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([driver|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_v3], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); decode_mini_message([{version3,v3}|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_v3], + Mods = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_v3], ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); decode_mini_message([{version3,prev3c}|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3c], + Mods = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3c], ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); decode_mini_message([{version3,prev3b}|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3b], + Mods = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3b], ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); decode_mini_message([{version3,prev3a}|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3a], + Mods = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3a], ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); decode_mini_message(EC, dynamic, Bin) -> - Mods = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_v3], + Mods = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_v3], ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,_},driver|EC], 1, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v1, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 1, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v1, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,_}|EC], 1, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v1, + AsnMod = megaco_ber_media_gateway_control_v1, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message(EC, 1, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v1, + AsnMod = megaco_ber_media_gateway_control_v1, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,_},driver|EC], 2, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v2, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 2, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v2, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,_}|EC], 2, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v2, + AsnMod = megaco_ber_media_gateway_control_v2, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message(EC, 2, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v2, + AsnMod = megaco_ber_media_gateway_control_v2, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,v3},driver|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v3, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3c},driver|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3c, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3b},driver|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3b, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3a},driver|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3a, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v3, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,v3}|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v3, + AsnMod = megaco_ber_media_gateway_control_v3, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,prev3c}|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3c, + AsnMod = megaco_ber_media_gateway_control_prev3c, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,prev3b}|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3b, + AsnMod = megaco_ber_media_gateway_control_prev3b, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,prev3a}|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3a, + AsnMod = megaco_ber_media_gateway_control_prev3a, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message(EC, 3, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v3, + AsnMod = megaco_ber_media_gateway_control_v3, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary). diff --git a/lib/megaco/src/binary/megaco_binary_encoder_lib.erl b/lib/megaco/src/binary/megaco_binary_encoder_lib.erl index 967ee93935..262889db39 100644 --- a/lib/megaco/src/binary/megaco_binary_encoder_lib.erl +++ b/lib/megaco/src/binary/megaco_binary_encoder_lib.erl @@ -275,7 +275,7 @@ decode_message_dynamic(_EC, _BadBin, _Mods, _Type) -> {error, no_binary}. -decode_message(EC, Bin, AsnMod, TransMod, binary) -> +decode_message(EC, Bin, AsnMod, TransMod, _) -> case asn1rt:decode(AsnMod, 'MegacoMessage', Bin) of {ok, MegaMsg} -> case EC of @@ -286,19 +286,6 @@ decode_message(EC, Bin, AsnMod, TransMod, binary) -> end; {error, Reason} -> {error, Reason} - end; -decode_message(EC, Bin, AsnMod, TransMod, io_list) -> - ShallowIoList = erlang:binary_to_list(Bin), - case asn1rt:decode(AsnMod, 'MegacoMessage', ShallowIoList) of - {ok, MegaMsg} -> - case EC of - [native] -> - {ok, MegaMsg}; - _ -> - {ok, TransMod:tr_message(MegaMsg, decode, EC)} - end; - {error, Reason} -> - {error, Reason} end. diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3a.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3a.set.asn deleted file mode 100644 index b9ba7ffdb4..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3a.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3a.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3b.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3b.set.asn deleted file mode 100644 index 0437bde310..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3b.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3b.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3c.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3c.set.asn deleted file mode 100644 index e78055fbad..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3c.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3c.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v1.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v1.set.asn deleted file mode 100644 index 0f5a92dba1..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v1.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v1.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v2.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v2.set.asn deleted file mode 100644 index 7fc82b127f..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v2.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v2.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v3.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v3.set.asn deleted file mode 100644 index 1d7950a283..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v3.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v3.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_encoder.erl b/lib/megaco/src/binary/megaco_per_bin_encoder.erl deleted file mode 100644 index f7280f4e04..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_encoder.erl +++ /dev/null @@ -1,447 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% - -%% -%%---------------------------------------------------------------------- -%% Purpose : Handle ASN.1 PER encoding of Megaco/H.248 -%%---------------------------------------------------------------------- - --module(megaco_per_bin_encoder). - --behaviour(megaco_encoder). - --export([encode_message/3, decode_message/3, - decode_mini_message/3, - - encode_transaction/3, - encode_action_requests/3, - encode_action_request/3, - encode_action_reply/3, - - version_of/2]). - -%% Backward compatible functions: --export([encode_message/2, decode_message/2]). - --include_lib("megaco/src/engine/megaco_message_internal.hrl"). - --define(V1_ASN1_MOD, megaco_per_bin_media_gateway_control_v1). --define(V2_ASN1_MOD, megaco_per_bin_media_gateway_control_v2). --define(V3_ASN1_MOD, megaco_per_bin_media_gateway_control_v3). --define(PREV3A_ASN1_MOD, megaco_per_bin_media_gateway_control_prev3a). --define(PREV3B_ASN1_MOD, megaco_per_bin_media_gateway_control_prev3b). --define(PREV3C_ASN1_MOD, megaco_per_bin_media_gateway_control_prev3c). --define(V1_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_v1). --define(V2_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_v2). --define(V3_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_v3). --define(PREV3A_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_prev3a). --define(PREV3B_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_prev3b). --define(PREV3C_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_prev3c). - --define(V1_TRANS_MOD, megaco_binary_transformer_v1). --define(V2_TRANS_MOD, megaco_binary_transformer_v2). --define(V3_TRANS_MOD, megaco_binary_transformer_v3). --define(PREV3A_TRANS_MOD, megaco_binary_transformer_prev3a). --define(PREV3B_TRANS_MOD, megaco_binary_transformer_prev3b). --define(PREV3C_TRANS_MOD, megaco_binary_transformer_prev3c). - --define(BIN_LIB, megaco_binary_encoder_lib). - - -%%---------------------------------------------------------------------- -%% Detect (check/get) message version -%% Return {ok, Version} | {error, Reason} -%%---------------------------------------------------------------------- - -version_of([{version3,v3},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?V3_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3c},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3C_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3b},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3B_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3a},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3A_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?V3_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,v3}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?V3_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3c}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3C_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3b}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3B_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3a}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3A_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -version_of(EC, Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?V3_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders). - - -%%---------------------------------------------------------------------- -%% Convert a 'MegacoMessage' record into a binary -%% Return {ok, Binary} | {error, Reason} -%%---------------------------------------------------------------------- - -encode_message(EC, - #'MegacoMessage'{mess = #'Message'{version = V}} = MegaMsg) -> - encode_message(EC, V, MegaMsg). - - -%% -- Version 1 -- - -encode_message([{version3, _},driver|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,_}|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - - -%% -- Version 2 -- - -encode_message([{version3,_},driver|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,_}|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - - -%% -- Version 3 -- - -encode_message([{version3,v3},driver|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3c},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3C_ASN1_MOD_DRV, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3b},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3B_ASN1_MOD_DRV, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3a},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3A_ASN1_MOD_DRV, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,v3}|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3c}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3C_ASN1_MOD, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3b}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3B_ASN1_MOD, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3a}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3A_ASN1_MOD, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list). - - -%%---------------------------------------------------------------------- -%% Convert a transaction (or transactions in the case of ack) record(s) -%% into a binary -%% Return {ok, Binary} | {error, Reason} -%%---------------------------------------------------------------------- - -%% encode_transaction([] = EC, 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD_DRV, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -encode_transaction(_EC, 1, _Trans) -> - %% AsnMod = ?V1_ASN1_MOD, - %% TransMod = ?V1_TRANS_MOD, - %% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, - %% io_list); - {error, not_implemented}; - -%% encode_transaction([] = EC, 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD_DRV, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -encode_transaction(_EC, 2, _Trans) -> - %% AsnMod = ?V2_ASN1_MOD, - %% TransMod = ?V2_TRANS_MOD, - %% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, - %% io_list). - {error, not_implemented}; - -%% encode_transaction([] = EC, 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD_DRV, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -encode_transaction(_EC, 3, _Trans) -> - %% AsnMod = ?V3_ASN1_MOD, - %% TransMod = ?V3_TRANS_MOD, - %% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, %% io_list). - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a list of ActionRequest record's into a binary -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- -encode_action_requests(_EC, 1, ActReqs) when is_list(ActReqs) -> - %% ?BIN_LIB:encode_action_requests(EC, ActReqs, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list); - {error, not_implemented}; -encode_action_requests(_EC, 2, ActReqs) when is_list(ActReqs) -> - %% ?BIN_LIB:encode_action_requests(EC, ActReqs, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list). - {error, not_implemented}; -encode_action_requests(_EC, 3, ActReqs) when is_list(ActReqs) -> - %% ?BIN_LIB:encode_action_requests(EC, ActReqs, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list). - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a ActionRequest record into a binary -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- -encode_action_request(_EC, 1, _ActReq) -> - %% ?BIN_LIB:encode_action_request(EC, ActReq, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list); - {error, not_implemented}; -encode_action_request(_EC, 2, _ActReq) -> - %% ?BIN_LIB:encode_action_request(EC, ActReq, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list). - {error, not_implemented}; -encode_action_request(_EC, 3, _ActReq) -> - %% ?BIN_LIB:encode_action_request(EC, ActReq, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list). - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a action reply into a deep io list -%% Not yest supported by this binary codec! -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- - -encode_action_reply(_EC, _V, _AcionReply) -> - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a binary into a 'MegacoMessage' record -%% Return {ok, MegacoMessageRecord} | {error, Reason} -%%---------------------------------------------------------------------- - -decode_message(EC, Binary) -> - decode_message(EC, 1, Binary). - -%% PER does not support partial decode, so this means V1 -decode_message(EC, dynamic, Binary) -> - decode_message(EC, 1, Binary); - - -%% -- Version 1 -- - -decode_message([{version3,_},driver|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,_}|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 1, Binary) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - - -%% -- Version 2 -- - -decode_message([{version3,_},driver|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,_}|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 2, Binary) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - - -%% -- Version 3 -- - -decode_message([{version3,v3},driver|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3c},driver|EC], 3, Binary) -> - AsnMod = ?PREV3C_ASN1_MOD_DRV, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3b},driver|EC], 3, Binary) -> - AsnMod = ?PREV3B_ASN1_MOD_DRV, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3a},driver|EC], 3, Binary) -> - AsnMod = ?PREV3A_ASN1_MOD_DRV, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,v3}|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3c}|EC], 3, Binary) -> - AsnMod = ?PREV3C_ASN1_MOD, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3b}|EC], 3, Binary) -> - AsnMod = ?PREV3B_ASN1_MOD, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3a}|EC], 3, Binary) -> - AsnMod = ?PREV3A_ASN1_MOD, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 3, Binary) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary). - - -decode_mini_message(_EC, _Vsn, _Bin) -> - {error, not_implemented}. diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3a.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3a.set.asn deleted file mode 100644 index b9ba7ffdb4..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3a.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3a.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3b.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3b.set.asn deleted file mode 100644 index 0437bde310..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3b.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3b.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3c.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3c.set.asn deleted file mode 100644 index e78055fbad..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3c.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3c.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v1.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v1.set.asn deleted file mode 100644 index 0f5a92dba1..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v1.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v1.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v2.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v2.set.asn deleted file mode 100644 index 7fc82b127f..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v2.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v2.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v3.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v3.set.asn deleted file mode 100644 index 1d7950a283..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v3.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v3.asn diff --git a/lib/megaco/src/binary/modules.mk b/lib/megaco/src/binary/modules.mk index a86ce2aecc..bbaf087ceb 100644 --- a/lib/megaco/src/binary/modules.mk +++ b/lib/megaco/src/binary/modules.mk @@ -27,19 +27,6 @@ MODULES = \ megaco_ber_media_gateway_control_prev3b \ megaco_ber_media_gateway_control_prev3c \ megaco_ber_media_gateway_control_v3 \ - megaco_ber_bin_encoder \ - megaco_ber_bin_media_gateway_control_v1 \ - megaco_ber_bin_media_gateway_control_v2 \ - megaco_ber_bin_media_gateway_control_prev3a \ - megaco_ber_bin_media_gateway_control_prev3b \ - megaco_ber_bin_media_gateway_control_prev3c \ - megaco_ber_bin_media_gateway_control_v3 \ - megaco_ber_bin_drv_media_gateway_control_v1 \ - megaco_ber_bin_drv_media_gateway_control_v2 \ - megaco_ber_bin_drv_media_gateway_control_prev3a \ - megaco_ber_bin_drv_media_gateway_control_prev3b \ - megaco_ber_bin_drv_media_gateway_control_prev3c \ - megaco_ber_bin_drv_media_gateway_control_v3 \ megaco_per_encoder \ megaco_per_media_gateway_control_v1 \ megaco_per_media_gateway_control_v2 \ @@ -47,19 +34,6 @@ MODULES = \ megaco_per_media_gateway_control_prev3b \ megaco_per_media_gateway_control_prev3c \ megaco_per_media_gateway_control_v3 \ - megaco_per_bin_encoder \ - megaco_per_bin_media_gateway_control_v1 \ - megaco_per_bin_media_gateway_control_v2 \ - megaco_per_bin_media_gateway_control_prev3a \ - megaco_per_bin_media_gateway_control_prev3b \ - megaco_per_bin_media_gateway_control_prev3c \ - megaco_per_bin_media_gateway_control_v3 \ - megaco_per_bin_drv_media_gateway_control_v1 \ - megaco_per_bin_drv_media_gateway_control_v2 \ - megaco_per_bin_drv_media_gateway_control_prev3a \ - megaco_per_bin_drv_media_gateway_control_prev3b \ - megaco_per_bin_drv_media_gateway_control_prev3c \ - megaco_per_bin_drv_media_gateway_control_v3 \ megaco_binary_name_resolver_v1 \ megaco_binary_name_resolver_v2 \ megaco_binary_name_resolver_prev3a \ @@ -85,44 +59,20 @@ ASN1_PREV3C_SPEC = MEDIA-GATEWAY-CONTROL-prev3c ASN1_V3_SPEC = MEDIA-GATEWAY-CONTROL-v3 BER_ASN1_V1_SPEC = megaco_ber_media_gateway_control_v1 -BER_BIN_ASN1_V1_SPEC = megaco_ber_bin_media_gateway_control_v1 -BER_BIN_DRV_ASN1_V1_SPEC = megaco_ber_bin_drv_media_gateway_control_v1 PER_ASN1_V1_SPEC = megaco_per_media_gateway_control_v1 -PER_BIN_ASN1_V1_SPEC = megaco_per_bin_media_gateway_control_v1 -PER_BIN_DRV_ASN1_V1_SPEC = megaco_per_bin_drv_media_gateway_control_v1 BER_ASN1_V2_SPEC = megaco_ber_media_gateway_control_v2 -BER_BIN_ASN1_V2_SPEC = megaco_ber_bin_media_gateway_control_v2 -BER_BIN_DRV_ASN1_V2_SPEC = megaco_ber_bin_drv_media_gateway_control_v2 PER_ASN1_V2_SPEC = megaco_per_media_gateway_control_v2 -PER_BIN_ASN1_V2_SPEC = megaco_per_bin_media_gateway_control_v2 -PER_BIN_DRV_ASN1_V2_SPEC = megaco_per_bin_drv_media_gateway_control_v2 BER_ASN1_PREV3A_SPEC = megaco_ber_media_gateway_control_prev3a -BER_BIN_ASN1_PREV3A_SPEC = megaco_ber_bin_media_gateway_control_prev3a -BER_BIN_DRV_ASN1_PREV3A_SPEC = megaco_ber_bin_drv_media_gateway_control_prev3a PER_ASN1_PREV3A_SPEC = megaco_per_media_gateway_control_prev3a -PER_BIN_ASN1_PREV3A_SPEC = megaco_per_bin_media_gateway_control_prev3a -PER_BIN_DRV_ASN1_PREV3A_SPEC = megaco_per_bin_drv_media_gateway_control_prev3a BER_ASN1_PREV3B_SPEC = megaco_ber_media_gateway_control_prev3b -BER_BIN_ASN1_PREV3B_SPEC = megaco_ber_bin_media_gateway_control_prev3b -BER_BIN_DRV_ASN1_PREV3B_SPEC = megaco_ber_bin_drv_media_gateway_control_prev3b PER_ASN1_PREV3B_SPEC = megaco_per_media_gateway_control_prev3b -PER_BIN_ASN1_PREV3B_SPEC = megaco_per_bin_media_gateway_control_prev3b -PER_BIN_DRV_ASN1_PREV3B_SPEC = megaco_per_bin_drv_media_gateway_control_prev3b BER_ASN1_PREV3C_SPEC = megaco_ber_media_gateway_control_prev3c -BER_BIN_ASN1_PREV3C_SPEC = megaco_ber_bin_media_gateway_control_prev3c -BER_BIN_DRV_ASN1_PREV3C_SPEC = megaco_ber_bin_drv_media_gateway_control_prev3c PER_ASN1_PREV3C_SPEC = megaco_per_media_gateway_control_prev3c -PER_BIN_ASN1_PREV3C_SPEC = megaco_per_bin_media_gateway_control_prev3c -PER_BIN_DRV_ASN1_PREV3C_SPEC = megaco_per_bin_drv_media_gateway_control_prev3c BER_ASN1_V3_SPEC = megaco_ber_media_gateway_control_v3 -BER_BIN_ASN1_V3_SPEC = megaco_ber_bin_media_gateway_control_v3 -BER_BIN_DRV_ASN1_V3_SPEC = megaco_ber_bin_drv_media_gateway_control_v3 PER_ASN1_V3_SPEC = megaco_per_media_gateway_control_v3 -PER_BIN_ASN1_V3_SPEC = megaco_per_bin_media_gateway_control_v3 -PER_BIN_DRV_ASN1_V3_SPEC = megaco_per_bin_drv_media_gateway_control_v3 diff --git a/lib/megaco/test/megaco_actions_test.erl b/lib/megaco/test/megaco_actions_test.erl index 2efb6e834a..6d0e80281d 100644 --- a/lib/megaco/test/megaco_actions_test.erl +++ b/lib/megaco/test/megaco_actions_test.erl @@ -80,8 +80,7 @@ end_per_testcase(Case, Config) -> all() -> [pretty_text, flex_pretty_text, compact_text, - flex_compact_text, erl_dist, erl_dist_mc, ber_bin, - ber_bin_drv, ber_bin_native, ber_bin_drv_native]. + flex_compact_text, erl_dist, erl_dist_mc]. groups() -> []. @@ -170,39 +169,6 @@ erl_dist_mc(Config) when is_list(Config) -> req_and_rep(Config, Codec, Version, EncodingConfig). -ber_bin(suite) -> - []; -ber_bin(doc) -> - []; -ber_bin(Config) when is_list(Config) -> - ?SKIP(currently_not_supported_by_asn1). - - -ber_bin_drv(suite) -> - []; -ber_bin_drv(doc) -> - []; -ber_bin_drv(Config) when is_list(Config) -> - ?SKIP(currently_not_supported_by_asn1). - - -ber_bin_native(suite) -> - []; -ber_bin_native(doc) -> - []; -ber_bin_native(Config) when is_list(Config) -> - ?SKIP(currently_not_supported_by_asn1). - - -ber_bin_drv_native(suite) -> - []; -ber_bin_drv_native(doc) -> - []; -ber_bin_drv_native(Config) when is_list(Config) -> - ?SKIP(currently_not_supported_by_asn1). - - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% req_and_rep(Config, Codec, _Version, EC) when is_list(Config) -> diff --git a/lib/megaco/test/megaco_call_flow_test.erl b/lib/megaco/test/megaco_call_flow_test.erl index b9d64ca8b2..d5888018cd 100644 --- a/lib/megaco/test/megaco_call_flow_test.erl +++ b/lib/megaco/test/megaco_call_flow_test.erl @@ -25,7 +25,6 @@ %% megaco_call_flow_test:compact_text(). %% megaco_call_flow_test:bin(). %% megaco_call_flow_test:asn1_ber(). -%% megaco_call_flow_test:asn1_ber_bin(). %% megaco_call_flow_test:asn1_per(). %% megaco_call_flow_test:erl_dist(). %% megaco_call_flow_test:compressed_erl_dist(). @@ -62,7 +61,7 @@ all() -> groups() -> [{text, [], [pretty, compact]}, {flex, [], [pretty_flex, compact_flex]}, - {binary, [], [bin, ber, ber_bin, per]}]. + {binary, [], [bin, ber, per]}]. init_per_group(_GroupName, Config) -> Config. @@ -106,12 +105,6 @@ ber(Config) when is_list(Config) -> ?ACQUIRE_NODES(1, Config), asn1_ber(). -ber_bin(suite) -> - []; -ber_bin(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - asn1_ber_bin(). - per(suite) -> []; per(Config) when is_list(Config) -> @@ -1198,8 +1191,7 @@ encoders() -> {megaco_pretty_text_encoder, [], []}, {megaco_compact_text_encoder, [], []}, {megaco_binary_encoder, [], [native]}, - %% {megaco_ber_encoder, [], [native]}, - %% {megaco_ber_bin_encoder, [], [native]}, + {megaco_ber_encoder, [], [native]}, {megaco_per_encoder, [], [native]}, {megaco_erl_dist_encoder, [], []}, {megaco_erl_dist_encoder, [compressed], [compressed]} @@ -1214,7 +1206,6 @@ pretty_mod({Mod, Opt, _Opt2}) -> megaco_compact_text_encoder -> compact_text; megaco_binary_encoder -> asn1_ber; megaco_ber_encoder -> asn1_ber_old; - megaco_ber_bin_encoder -> asn1_ber_bin; megaco_per_encoder -> asn1_per; megaco_erl_dist_encoder when Opt == [] -> standard_erl; megaco_erl_dist_encoder when Opt == [compressed] -> compressed_erl; @@ -1263,13 +1254,6 @@ asn1_ber() -> All = [encode(Slogan, Msg, Encoder) || {Slogan, Msg} <- messages()], compute_res(All). -asn1_ber_bin() -> - Default = [], - Native = [native], - Encoder = {megaco_ber_bin_encoder, Default, Native}, - All = [encode(Slogan, Msg, Encoder) || {Slogan, Msg} <- messages()], - compute_res(All). - asn1_per() -> Default = [], Native = [native], @@ -1634,7 +1618,7 @@ gen_ber_header() -> %% Generate headerfile for asn.1 BER test in C %%---------------------------------------------------------------------- gen_ber_bin_header() -> - Encoder = {megaco_ber_bin_encoder, [], []}, + Encoder = {megaco_ber_encoder, [], []}, L = [{S, gen_byte_msg(Msg, Encoder)} || {S, Msg} <- messages()], gen_header_file_binary(L). diff --git a/lib/megaco/test/megaco_codec_prev3a_test.erl b/lib/megaco/test/megaco_codec_prev3a_test.erl index d50e72aef1..4f1160b93f 100644 --- a/lib/megaco/test/megaco_codec_prev3a_test.erl +++ b/lib/megaco/test/megaco_codec_prev3a_test.erl @@ -63,12 +63,8 @@ ber_test_msgs/1, - ber_bin_test_msgs/1, - per_test_msgs/1, - per_bin_test_msgs/1, - erl_dist_m_test_msgs/1, tickets/0, @@ -280,17 +276,14 @@ groups() -> [{group, pretty}, {group, flex_pretty}, {group, compact}, {group, flex_compact}]}, {binary, [], - [{group, bin}, {group, ber}, {group, ber_bin}, - {group, per}, {group, per_bin}]}, + [{group, bin}, {group, ber}, {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, {flex_pretty, [], flex_pretty_cases()}, {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, @@ -1106,17 +1099,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(binary) ++ msgs4(binary) ++ msgs5(binary) ++ msgs6(binary), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - per_test_msgs(suite) -> []; per_test_msgs(Config) when is_list(Config) -> @@ -1128,17 +1110,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(binary) ++ msgs4(binary) ++ msgs5(binary) ++ msgs6(binary), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> diff --git a/lib/megaco/test/megaco_codec_prev3b_test.erl b/lib/megaco/test/megaco_codec_prev3b_test.erl index eaab8f37c1..4bd49365e7 100644 --- a/lib/megaco/test/megaco_codec_prev3b_test.erl +++ b/lib/megaco/test/megaco_codec_prev3b_test.erl @@ -63,12 +63,8 @@ ber_test_msgs/1, - ber_bin_test_msgs/1, - per_test_msgs/1, - per_bin_test_msgs/1, - erl_dist_m_test_msgs/1, tickets/0, @@ -296,17 +292,14 @@ groups() -> [{group, pretty}, {group, flex_pretty}, {group, compact}, {group, flex_compact}]}, {binary, [], - [{group, bin}, {group, ber}, {group, ber_bin}, - {group, per}, {group, per_bin}]}, + [{group, bin}, {group, ber}, {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, {flex_pretty, [], flex_pretty_cases()}, {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, @@ -1171,16 +1164,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(binary) ++ msgs4(binary) ++ msgs5(binary) ++ msgs6(binary), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% per_test_msgs(suite) -> []; @@ -1193,17 +1176,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(binary) ++ msgs4(binary) ++ msgs5(binary) ++ msgs6(binary), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> diff --git a/lib/megaco/test/megaco_codec_prev3c_test.erl b/lib/megaco/test/megaco_codec_prev3c_test.erl index 7f9c0fe4e7..baa1c7f547 100644 --- a/lib/megaco/test/megaco_codec_prev3c_test.erl +++ b/lib/megaco/test/megaco_codec_prev3c_test.erl @@ -65,12 +65,8 @@ ber_test_msgs/1, - ber_bin_test_msgs/1, - per_test_msgs/1, - per_bin_test_msgs/1, - erl_dist_m_test_msgs/1, tickets/0, @@ -301,17 +297,14 @@ groups() -> [{group, pretty}, {group, flex_pretty}, {group, compact}, {group, flex_compact}]}, {binary, [], - [{group, bin}, {group, ber}, {group, ber_bin}, - {group, per}, {group, per_bin}]}, + [{group, bin}, {group, ber}, {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, {flex_pretty, [], flex_pretty_cases()}, {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, @@ -823,21 +816,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = - msgs1a(binary) ++ - msgs5(binary) ++ - msgs6(binary) ++ - msgs7(binary), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - per_test_msgs(suite) -> []; per_test_msgs(Config) when is_list(Config) -> @@ -853,21 +831,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = - msgs1a(binary) ++ - msgs5(binary) ++ - msgs6(binary) ++ - msgs7(binary), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> diff --git a/lib/megaco/test/megaco_codec_v1_test.erl b/lib/megaco/test/megaco_codec_v1_test.erl index e9c19605dd..86f332fde0 100644 --- a/lib/megaco/test/megaco_codec_v1_test.erl +++ b/lib/megaco/test/megaco_codec_v1_test.erl @@ -66,12 +66,8 @@ ber_test_msgs/1, - ber_bin_test_msgs/1, - per_test_msgs/1, - per_bin_test_msgs/1, - erl_dist_m_test_msgs/1, tickets/0, @@ -476,9 +472,7 @@ groups() -> {group, flex_compact}]}, {binary, [], [{group, bin}, {group, ber}, - {group, ber_bin}, - {group, per}, - {group, per_bin}]}, + {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, @@ -486,9 +480,7 @@ groups() -> {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, {group, pretty_tickets}, @@ -1266,17 +1258,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, [], Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - per_test_msgs(suite) -> []; per_test_msgs(Config) when is_list(Config) -> @@ -1288,17 +1269,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, [], Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> diff --git a/lib/megaco/test/megaco_codec_v2_test.erl b/lib/megaco/test/megaco_codec_v2_test.erl index a44f74166c..e8db9e1cb1 100644 --- a/lib/megaco/test/megaco_codec_v2_test.erl +++ b/lib/megaco/test/megaco_codec_v2_test.erl @@ -64,12 +64,8 @@ ber_test_msgs/1, - ber_bin_test_msgs/1, - per_test_msgs/1, - per_bin_test_msgs/1, - erl_dist_m_test_msgs/1, tickets/0, @@ -447,17 +443,14 @@ groups() -> [{group, pretty}, {group, flex_pretty}, {group, compact}, {group, flex_compact}]}, {binary, [], - [{group, bin}, {group, ber}, {group, ber_bin}, - {group, per}, {group, per_bin}]}, + [{group, bin}, {group, ber}, {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, {flex_pretty, [], flex_pretty_cases()}, {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, {group, pretty_tickets}, @@ -1285,17 +1278,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1() ++ msgs4(), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, [], Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - per_test_msgs(suite) -> []; per_test_msgs(Config) when is_list(Config) -> @@ -1307,17 +1289,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1() ++ msgs4(), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, [], Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> diff --git a/lib/megaco/test/megaco_codec_v3_test.erl b/lib/megaco/test/megaco_codec_v3_test.erl index 2c35ce13b3..67805479f9 100644 --- a/lib/megaco/test/megaco_codec_v3_test.erl +++ b/lib/megaco/test/megaco_codec_v3_test.erl @@ -56,9 +56,7 @@ flex_compact_dm_timers8/1, bin_test_msgs/1, ber_test_msgs/1, - ber_bin_test_msgs/1, per_test_msgs/1, - per_bin_test_msgs/1, erl_dist_m_test_msgs/1, tickets/0, @@ -288,17 +286,14 @@ groups() -> [{group, pretty}, {group, flex_pretty}, {group, compact}, {group, flex_compact}]}, {binary, [], - [{group, bin}, {group, ber}, {group, ber_bin}, - {group, per}, {group, per_bin}]}, + [{group, bin}, {group, ber}, {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, {flex_pretty, [], flex_pretty_cases()}, {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, @@ -823,22 +818,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = - msgs1a(binary) ++ - msgs5(binary) ++ - msgs6(binary) ++ - msgs7(binary) ++ - msgs8(binary), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - per_test_msgs(suite) -> []; per_test_msgs(Config) when is_list(Config) -> @@ -855,22 +834,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = - msgs1a(binary) ++ - msgs5(binary) ++ - msgs6(binary) ++ - msgs7(binary) ++ - msgs8(binary), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> diff --git a/lib/megaco/test/megaco_mess_test.erl b/lib/megaco/test/megaco_mess_test.erl index 663ac8c329..f8be96c254 100644 --- a/lib/megaco/test/megaco_mess_test.erl +++ b/lib/megaco/test/megaco_mess_test.erl @@ -1393,7 +1393,7 @@ rarpaop_mgc_event_sequence(text, tcp) -> rarpaop_mgc_event_sequence(binary, tcp) -> Port = 2945, TranspMod = megaco_tcp, - EncMod = megaco_ber_bin_encoder, + EncMod = megaco_ber_encoder, EncConf = [], rarpaop_mgc_event_sequence(Port, TranspMod, EncMod, EncConf). @@ -1680,7 +1680,7 @@ rarpaop_mg_event_sequence(text, tcp) -> rarpaop_mg_event_sequence(Port, EncMod, EncConf); rarpaop_mg_event_sequence(binary, tcp) -> Port = 2945, - EncMod = megaco_ber_bin_encoder, + EncMod = megaco_ber_encoder, EncConf = [], rarpaop_mg_event_sequence(Port, EncMod, EncConf). diff --git a/lib/megaco/test/megaco_mib_test.erl b/lib/megaco/test/megaco_mib_test.erl index 52d99d1442..ddc74ab741 100644 --- a/lib/megaco/test/megaco_mib_test.erl +++ b/lib/megaco/test/megaco_mib_test.erl @@ -647,13 +647,13 @@ mk_recv_info([{text,udp}|ET], Acc) -> {port, 2944}], mk_recv_info(ET, [RI|Acc]); mk_recv_info([{binary,tcp}|ET], Acc) -> - RI = [{encoding_module, megaco_ber_bin_encoder}, + RI = [{encoding_module, megaco_ber_encoder}, {encoding_config, []}, {transport_module, megaco_tcp}, {port, 2945}], mk_recv_info(ET, [RI|Acc]); mk_recv_info([{binary,udp}|ET], Acc) -> - RI = [{encoding_module, megaco_ber_bin_encoder}, + RI = [{encoding_module, megaco_ber_encoder}, {encoding_config, []}, {transport_module, megaco_udp}, {port, 2945}], @@ -1013,7 +1013,7 @@ start_mg(Node, Mid, Encoding, Transport, Verbosity) -> {encoding_config, []}, {port,2944}]; binary -> - [{encoding_module, megaco_ber_bin_encoder}, + [{encoding_module, megaco_ber_encoder}, {encoding_config, []}, {port,2945}] end, diff --git a/lib/megaco/test/megaco_test_mg.erl b/lib/megaco/test/megaco_test_mg.erl index ecb3cedc83..947f0eebbb 100644 --- a/lib/megaco/test/megaco_test_mg.erl +++ b/lib/megaco/test/megaco_test_mg.erl @@ -158,7 +158,7 @@ select_encoding(pretty_text) -> select_encoding(compact_text) -> {megaco_compact_text_encoder, 2944}; select_encoding(binary) -> - {megaco_ber_bin_encoder, 2945}; + {megaco_ber_encoder, 2945}; select_encoding(erl_dist) -> {megaco_erl_dist_encoder, 2946}; select_encoding(Encoding) -> diff --git a/lib/megaco/test/megaco_test_mgc.erl b/lib/megaco/test/megaco_test_mgc.erl index 13c1cebe56..a964983861 100644 --- a/lib/megaco/test/megaco_test_mgc.erl +++ b/lib/megaco/test/megaco_test_mgc.erl @@ -135,7 +135,7 @@ select_encoding(pretty_text) -> select_encoding(compact_text) -> {megaco_compact_text_encoder, 2944}; select_encoding(binary) -> - {megaco_ber_bin_encoder, 2945}; + {megaco_ber_encoder, 2945}; select_encoding(erl_dist) -> {megaco_erl_dist_encoder, 2946}; select_encoding(Encoding) -> diff --git a/lib/observer/doc/src/notes.xml b/lib/observer/doc/src/notes.xml index ea665eee88..4ec03782a7 100644 --- a/lib/observer/doc/src/notes.xml +++ b/lib/observer/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2011</year> + <year>2004</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/odbc/c_src/odbcserver.c b/lib/odbc/c_src/odbcserver.c index fe81d1dd3a..4a7a5224e5 100644 --- a/lib/odbc/c_src/odbcserver.c +++ b/lib/odbc/c_src/odbcserver.c @@ -153,7 +153,7 @@ static db_result_msg db_describe_table(byte *sql, db_state *state); /* ------------- Encode/decode functions -------- ------------------------*/ static db_result_msg encode_empty_message(void); -static db_result_msg encode_error_message(char *reason); +static db_result_msg encode_error_message(char *reason, char *errCode, SQLINTEGER nativeError); static db_result_msg encode_atom_message(char *atom); static db_result_msg encode_result(db_state *state); static db_result_msg encode_result_set(SQLSMALLINT num_of_columns, @@ -230,7 +230,7 @@ static void init_param_statement(int cols, static void map_dec_num_2_c_column(col_type *type, int precision, int scale); -static db_result_msg map_sql_2_c_column(db_column* column); +static db_result_msg map_sql_2_c_column(db_column* column, db_state *state); static param_array * bind_parameter_arrays(byte *buffer, int *index, @@ -246,7 +246,7 @@ static db_result_msg retrive_scrollable_cursor_support_info(db_state static int num_out_params(int num_of_params, param_array* params); /* ------------- Error handling functions --------------------------------*/ -static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle); +static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle, Boolean extendedErrors); /* ------------- Boolean functions ---------------------------------------*/ @@ -349,7 +349,7 @@ DWORD WINAPI database_handler(const char *port) byte *request_buffer = NULL; db_state state = {NULL, NULL, NULL, NULL, 0, {NULL, 0, 0}, - FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; byte request_id; #ifdef WIN32 SOCKET socket; @@ -438,14 +438,16 @@ static db_result_msg db_connect(byte *args, db_state *state) diagnos diagnos; byte *connStrIn; int erl_auto_commit_mode, erl_trace_driver, - use_srollable_cursors, tuple_row_state, binary_strings; + use_srollable_cursors, tuple_row_state, binary_strings, + extended_errors; erl_auto_commit_mode = args[0]; erl_trace_driver = args[1]; use_srollable_cursors = args[2]; tuple_row_state = args[3]; binary_strings = args[4]; - connStrIn = args + 5 * sizeof(byte); + extended_errors = args[5]; + connStrIn = args + 6 * sizeof(byte); if(tuple_row_state == ON) { tuple_row(state) = TRUE; @@ -465,6 +467,12 @@ static db_result_msg db_connect(byte *args, db_state *state) use_srollable_cursors(state) = FALSE; } + if(extended_errors == ON) { + extended_errors(state) = TRUE; + } else { + extended_errors(state) = FALSE; + } + init_driver(erl_auto_commit_mode, erl_trace_driver, state); connlen = (SQLSMALLINT)strlen((const char*)connStrIn); @@ -475,10 +483,10 @@ static db_result_msg db_connect(byte *args, db_state *state) &stringlength2ptr, SQL_DRIVER_NOPROMPT); if (!sql_success(result)) { - diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state), extended_errors(state)); strcat((char *)diagnos.error_msg, " Connection to database failed."); - msg = encode_error_message(diagnos.error_msg); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError ); if(!sql_success(SQLFreeHandle(SQL_HANDLE_DBC, connection_handle(state)))) @@ -509,8 +517,8 @@ static db_result_msg db_close_connection(db_state *state) result = SQLDisconnect(connection_handle(state)); if (!sql_success(result)) { - diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state)); - return encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state), extended_errors(state)); + return encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); } if(!sql_success(SQLFreeHandle(SQL_HANDLE_DBC, @@ -536,8 +544,8 @@ static db_result_msg db_end_tran(byte compleationtype, db_state *state) (SQLSMALLINT)compleationtype); if (!sql_success(result)) { - diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state)); - return encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state), extended_errors(state)); + return encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); } else { return encode_atom_message("ok"); } @@ -572,7 +580,7 @@ static db_result_msg db_query(byte *sql, db_state *state) /* SQL_SUCCESS_WITH_INFO at this point may indicate an error in user input. */ if (result != SQL_SUCCESS && result != SQL_NO_DATA_FOUND) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); if(strcmp((char *)diagnos.sqlState, INFO) == 0) { is_error[0] = 0; strncat((char *)is_error, (char *)diagnos.error_msg, @@ -583,12 +591,12 @@ static db_result_msg db_query(byte *sql, db_state *state) it as we want a nice and clean Erlang API */ if((strcmp((char *)is_error, "error") == 0)) { - msg = encode_error_message((char *)diagnos.error_msg); + msg = encode_error_message((char *)diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } } else { - msg = encode_error_message((char *)diagnos.error_msg); + msg = encode_error_message((char *)diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } @@ -661,9 +669,9 @@ static db_result_msg db_select_count(byte *sql, db_state *state) } if(!sql_success(SQLExecDirect(statement_handle(state), sql, SQL_NTS))) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); clean_state(state); - return encode_error_message(diagnos.error_msg); + return encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); } if(!sql_success(SQLNumResultCols(statement_handle(state), @@ -803,13 +811,13 @@ static db_result_msg db_param_query(byte *buffer, db_state *state) result = SQLExecDirect(statement_handle(state), sql, SQL_NTS); if (!sql_success(result) || result == SQL_NO_DATA) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); } /* SQL_NO_DATA and SQLSTATE 00000 indicate success for updates/deletes that affect no rows */ if(!sql_success(result) && !(result == SQL_NO_DATA && !strcmp((char *)diagnos.sqlState, INFO))) { - msg = encode_error_message(diagnos.error_msg); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); } else { for (i = 0; i < param_status.params_processed; i++) { switch (param_status.param_status_array[i]) { @@ -823,8 +831,8 @@ static db_result_msg db_param_query(byte *buffer, db_state *state) break; default: diagnos = - get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); - msg = encode_error_message(diagnos.error_msg); + get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); i = param_status.params_processed; break; } @@ -893,16 +901,16 @@ static db_result_msg db_describe_table(byte *sql, db_state *state) DO_EXIT(EXIT_ALLOC); if (!sql_success(SQLPrepare(statement_handle(state), sql, SQL_NTS))){ - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); - msg = encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } if(!sql_success(SQLNumResultCols(statement_handle(state), &num_of_columns))) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); - msg = encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } @@ -951,7 +959,7 @@ static db_result_msg encode_empty_message(void) } /* Description: Encode an error-message to send back to erlang*/ -static db_result_msg encode_error_message(char *reason) +static db_result_msg encode_error_message(char *reason, char *errCode, SQLINTEGER nativeError ) { int index; db_result_msg msg; @@ -960,6 +968,12 @@ static db_result_msg encode_error_message(char *reason) ei_encode_version(NULL, &index); ei_encode_tuple_header(NULL, &index, 2); ei_encode_atom(NULL, &index, "error"); + if (errCode) + { + ei_encode_tuple_header(NULL, &index, 3); + ei_encode_string(NULL, &index, errCode); + ei_encode_long(NULL, &index, nativeError); + } ei_encode_string(NULL, &index, reason); msg.length = index; @@ -970,6 +984,12 @@ static db_result_msg encode_error_message(char *reason) ei_encode_version((char *)msg.buffer, &index); ei_encode_tuple_header((char *)msg.buffer, &index, 2); ei_encode_atom((char *)msg.buffer, &index, "error"); + if (errCode) + { + ei_encode_tuple_header((char *)msg.buffer, &index, 3); + ei_encode_string((char *)msg.buffer, &index, errCode); + ei_encode_long((char *)msg.buffer, &index, nativeError); + } ei_encode_string((char *)msg.buffer, &index, reason); return msg; @@ -1013,8 +1033,8 @@ static db_result_msg encode_result(db_state *state) if(!sql_success(SQLNumResultCols(statement_handle(state), &num_of_columns))) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); - msg = encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } @@ -1030,8 +1050,8 @@ static db_result_msg encode_result(db_state *state) } if(!sql_success(SQLRowCount(statement_handle(state), &RowCountPtr))) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); - msg = encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } @@ -1239,7 +1259,7 @@ static db_result_msg encode_column_name_list(SQLSMALLINT num_of_columns, (columns(state)[i]).type.sql = sql_type; (columns(state)[i]).type.col_size = size; - msg = map_sql_2_c_column(&columns(state)[i]); + msg = map_sql_2_c_column(&columns(state)[i], state); if (msg.length > 0) { return msg; /* An error has occurred */ } else { @@ -2346,7 +2366,7 @@ static void map_dec_num_2_c_column(col_type *type, int precision, int scale) /* Description: Transform SQL columntype to C columntype. Returns a dummy db_result_msg with length 0 on success and an errormessage otherwise.*/ -static db_result_msg map_sql_2_c_column(db_column* column) +static db_result_msg map_sql_2_c_column(db_column* column, db_state *state) { db_result_msg msg; @@ -2415,10 +2435,10 @@ static db_result_msg map_sql_2_c_column(db_column* column) column -> type.strlen_or_indptr = (SQLLEN)NULL; break; case SQL_UNKNOWN_TYPE: - msg = encode_error_message("Unknown column type"); + msg = encode_error_message("Unknown column type", extended_error(state, ""), 0); break; default: - msg = encode_error_message("Column type not supported"); + msg = encode_error_message("Column type not supported", extended_error(state, ""), 0); break; } return msg; @@ -2527,7 +2547,7 @@ static db_column retrive_binary_data(db_column column, int column_nr, while (result == SQL_SUCCESS_WITH_INFO) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); if(strcmp((char *)diagnos.sqlState, TRUNCATED) == 0) { outputlen = column.type.len - 1; @@ -2614,10 +2634,10 @@ static db_result_msg more_result_sets(db_state *state) /* As we found an error we do not care about any potential more result sets */ exists_more_result_sets(state) = FALSE; - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); strcat((char *)diagnos.error_msg, "Failed to create on of the result sets"); - msg = encode_error_message(diagnos.error_msg); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); return msg; } } @@ -2634,7 +2654,7 @@ static Boolean sql_success(SQLRETURN result) diagnostic records scaning for error messages and the sqlstate. If this function is called when no error has ocurred only the sqlState field may be referenced.*/ -static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle) +static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle, Boolean extendedErrors) { diagnos diagnos; SQLINTEGER nativeError; @@ -2658,17 +2678,16 @@ static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle) result = SQLGetDiagRec(handleType, handle, record_nr, current_sql_state, &nativeError, current_errmsg_pos, (SQLSMALLINT)errmsg_buffer_size, &errmsg_size); - if(result != SQL_SUCCESS && result != SQL_NO_DATA) { - - - break; - } else { + if(result == SQL_SUCCESS) { /* update the sqlstate in the diagnos record, because the SQLGetDiagRec call succeeded */ memcpy(diagnos.sqlState, current_sql_state, SQL_STATE_SIZE); + diagnos.nativeError = nativeError; errmsg_buffer_size = errmsg_buffer_size - errmsg_size; acc_errmsg_size = acc_errmsg_size + errmsg_size; current_errmsg_pos = current_errmsg_pos + errmsg_size; + } else { + break; } } @@ -2676,7 +2695,7 @@ static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle) strcat((char *)diagnos.error_msg, "No SQL-driver information available."); } - else { + else if (!extendedErrors){ strcat(strcat((char *)diagnos.error_msg, " SQLSTATE IS: "), (char *)diagnos.sqlState); } diff --git a/lib/odbc/c_src/odbcserver.h b/lib/odbc/c_src/odbcserver.h index a76cedf1af..71760189e7 100644 --- a/lib/odbc/c_src/odbcserver.h +++ b/lib/odbc/c_src/odbcserver.h @@ -145,6 +145,7 @@ typedef struct { typedef struct { SQLCHAR sqlState[SQL_STATE_SIZE]; + SQLINTEGER nativeError; byte error_msg[MAX_ERR_MSG]; } diagnos; @@ -179,6 +180,7 @@ typedef struct { Boolean exists_more_result_sets; Boolean param_query; Boolean out_params; + Boolean extended_errors; } db_state; typedef enum { @@ -198,3 +200,5 @@ typedef enum { #define exists_more_result_sets(db_state) (db_state -> exists_more_result_sets) #define param_query(db_state) (db_state -> param_query) #define out_params(db_state) (db_state -> out_params) +#define extended_errors(db_state) (db_state -> extended_errors) +#define extended_error(db_state, errorcode) ( extended_errors(state) ? errorcode : NULL ) diff --git a/lib/odbc/doc/src/notes.xml b/lib/odbc/doc/src/notes.xml index 5f6cf91961..7ba0307a45 100644 --- a/lib/odbc/doc/src/notes.xml +++ b/lib/odbc/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2011</year> + <year>2004</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/odbc/doc/src/odbc.xml b/lib/odbc/doc/src/odbc.xml index 8a58dc2848..0e3386b11f 100644 --- a/lib/odbc/doc/src/odbc.xml +++ b/lib/odbc/doc/src/odbc.xml @@ -74,8 +74,12 @@ <code type="none"> milliseconds() = integer() >= 0 </code> <code type="none"> - common_reason() = connection_closed | term() - some kind of - explanation of what went wrong </code> + common_reason() = connection_closed | extended_error() | term() - some kind of + explanation of what went wrong </code> + <code type="none"> + extended_error() = {string(), integer(), Reason} - extended error type with ODBC + and native database error codes, as well as the base reason that would have been + returned had extended_errors not been enabled. </code> <code type="none"> string() = list of ASCII characters </code> <code type="none"> @@ -143,7 +147,7 @@ <d>All options has default values. </d> <v>option() = {auto_commit, on | off} | {timeout, milliseconds()} | {binary_strings, on | off} | {tuple_row, on | off} | {scrollable_cursors, on | off} | - {trace_driver, on | off} </v> + {trace_driver, on | off} | {extended_errors, on | off} </v> <v>Ref = connection_reference() - should be used to access the connection. </v> <v>Reason = port_program_executable_not_found | common_reason()</v> </type> @@ -196,6 +200,19 @@ <p>For more information about the <c>ConnectStr</c> see description of the function SQLDriverConnect in [1].</p> </note> + + <p>The <c>extended_errors</c> option enables extended ODBC error + information when an operation fails. Rather than returning <c>{error, Reason}</c>, + the failing function will reutrn <c>{error, {ODBCErrorCode, NativeErrorCode, Reason}}</c>. + Note that this information is probably of little use when writing database-independent code, + but can be of assistance in providing more sophisticated error handling when dealing with + a known underlying database. + <list type="bulleted"> + <item><c>ODBCErrorCode</c> is the ODBC error string returned by the ODBC driver.</item> + <item><c>NativeErrorCode</c> is the numberic error code returned by the underlying database. The possible values + and their meanings are dependent on the database being used.</item> + <item><c>Reason</c> is as per the <c>Reason</c> field when extended errors are not enabled.</item> + </list></p> </desc> </func> <func> @@ -203,7 +220,7 @@ <fsummary>Closes a connection to a database. </fsummary> <type> <v>Ref = connection_reference()</v> - <v>Reason = process_not_owner_of_odbc_connection</v> + <v>Reason = process_not_owner_of_odbc_connection | extended_error()</v> </type> <desc> <p>Closes a connection to a database. This will also diff --git a/lib/odbc/src/odbc.erl b/lib/odbc/src/odbc.erl index 16fdb4aabd..3eabec9ec3 100644 --- a/lib/odbc/src/odbc.erl +++ b/lib/odbc/src/odbc.erl @@ -810,10 +810,11 @@ connect(ConnectionReferense, ConnectionStr, Options) -> {C_TupleRow, _} = connection_config(tuple_row, Options), {BinaryStrings, _} = connection_config(binary_strings, Options), + {ExtendedErrors, _} = connection_config(extended_errors, Options), ODBCCmd = [?OPEN_CONNECTION, C_AutoCommitMode, C_TraceDriver, - C_SrollableCursors, C_TupleRow, BinaryStrings, ConnectionStr], + C_SrollableCursors, C_TupleRow, BinaryStrings, ExtendedErrors, ConnectionStr], %% Send request, to open a database connection, to the control process. case call(ConnectionReferense, @@ -860,6 +861,8 @@ connection_default(trace_driver) -> connection_default(scrollable_cursors) -> {?ON, on}; connection_default(binary_strings) -> + {?OFF, off}; +connection_default(extended_errors) -> {?OFF, off}. %%------------------------------------------------------------------------- diff --git a/lib/odbc/test/odbc_connect_SUITE.erl b/lib/odbc/test/odbc_connect_SUITE.erl index a076c4dfff..7d732f20f7 100644 --- a/lib/odbc/test/odbc_connect_SUITE.erl +++ b/lib/odbc/test/odbc_connect_SUITE.erl @@ -51,7 +51,7 @@ all() -> {group, client_dies}, connect_timeout, timeout, many_timeouts, timeout_reset, disconnect_on_timeout, connection_closed, disable_scrollable_cursors, - return_rows_as_lists, api_missuse]; + return_rows_as_lists, api_missuse, extended_errors]; Other -> {skip, Other} end. @@ -838,3 +838,33 @@ transaction_support_str(mysql) -> "ENGINE = InnoDB"; transaction_support_str(_) -> "". + + +%%------------------------------------------------------------------------- +extended_errors(doc)-> + ["Test the extended errors connection option: When off; the old behaviour of just an error " + "string is returned on error. When on, the error string is replaced by a 3 element tuple " + "that also exposes underlying ODBC provider error codes."]; +extended_errors(suite) -> []; +extended_errors(Config) when is_list(Config)-> + Table = ?config(tableName, Config), + {ok, Ref} = odbc:connect(?RDBMS:connection_string(), odbc_test_lib:platform_options()), + {updated, _} = odbc:sql_query(Ref, "create table " ++ Table ++" ( id integer, data varchar(10))"), + + % Error case WITHOUT extended errors on... + case odbc:sql_query(Ref, "create table " ++ Table ++" ( id integer, data varchar(10))") of + {error, ErrorString} when is_list(ErrorString) -> ok + end, + + % Now the test case with extended errors on - This should return a tuple, not a list/string now. + % The first element is a string that is the ODBC error string; the 2nd element is a native integer error + % code passed from the underlying provider driver. The last is the familiar old error string. + % We can't check the actual error code; as each different underlying provider will return + % a different value - So we just check the return types at least. + {ok, RefExtended} = odbc:connect(?RDBMS:connection_string(), odbc_test_lib:platform_options() ++ [{extended_errors, on}]), + case odbc:sql_query(RefExtended, "create table " ++ Table ++" ( id integer, data varchar(10))") of + {error, {ODBCCodeString, NativeCodeNum, ShortErrorString}} when is_list(ODBCCodeString), is_number(NativeCodeNum), is_list(ShortErrorString) -> ok + end, + + ok = odbc:disconnect(Ref), + ok = odbc:disconnect(RefExtended). diff --git a/lib/odbc/vsn.mk b/lib/odbc/vsn.mk index 3bb2fe5bce..585b92b2d2 100644 --- a/lib/odbc/vsn.mk +++ b/lib/odbc/vsn.mk @@ -1 +1 @@ -ODBC_VSN = 2.10.13 +ODBC_VSN = 2.10.14 diff --git a/lib/orber/src/Makefile b/lib/orber/src/Makefile index 72610def2b..6eb659407d 100644 --- a/lib/orber/src/Makefile +++ b/lib/orber/src/Makefile @@ -200,8 +200,7 @@ ERL_COMPILE_FLAGS += $(ERL_IDL_FLAGS) \ +'{attribute,insert,app_vsn,"orber_$(ORBER_VSN)"}' \ -D'ORBVSN="$(ORBER_VSN)"' -ASN_FLAGS = -bber_bin +der +compact_bit_string +optimize \ - +nowarn_unused_record +ASN_FLAGS = -bber +der +compact_bit_string +nowarn_unused_record # ---------------------------------------------------- # Targets diff --git a/lib/percept/doc/src/notes.xml b/lib/percept/doc/src/notes.xml index 1ac1a5ab62..dd70cfe994 100644 --- a/lib/percept/doc/src/notes.xml +++ b/lib/percept/doc/src/notes.xml @@ -32,6 +32,21 @@ </header> <p>This document describes the changes made to the Percept application.</p> +<section><title>Percept 0.8.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Add missing modules in app-file</p> + <p> + Own Id: OTP-10439</p> + </item> + </list> + </section> + +</section> + <section><title>Percept 0.8.6.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/percept/src/percept.app.src b/lib/percept/src/percept.app.src index 7b20093ece..cf4a9fc438 100644 --- a/lib/percept/src/percept.app.src +++ b/lib/percept/src/percept.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2009. All Rights Reserved. +%% Copyright Ericsson AB 2007-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/percept/vsn.mk b/lib/percept/vsn.mk index 9267a41055..f868246f7b 100644 --- a/lib/percept/vsn.mk +++ b/lib/percept/vsn.mk @@ -1 +1 @@ -PERCEPT_VSN = 0.8.6.1 +PERCEPT_VSN = 0.8.7 diff --git a/lib/public_key/asn1/Makefile b/lib/public_key/asn1/Makefile index 957c332cad..763b788e53 100644 --- a/lib/public_key/asn1/Makefile +++ b/lib/public_key/asn1/Makefile @@ -66,7 +66,7 @@ EBIN = ../ebin EXTRA_ERLC_FLAGS = ERL_COMPILE_FLAGS += $(EXTRA_ERLC_FLAGS) -ASN_FLAGS = -bber_bin +der +compact_bit_string +optimize +noobj +asn1config +inline +nif +ASN_FLAGS = -bber +der +compact_bit_string +noobj +asn1config +inline # ---------------------------------------------------- # Targets diff --git a/lib/public_key/doc/src/notes.xml b/lib/public_key/doc/src/notes.xml index d895042570..a5e8beedf0 100644 --- a/lib/public_key/doc/src/notes.xml +++ b/lib/public_key/doc/src/notes.xml @@ -34,6 +34,41 @@ <file>notes.xml</file> </header> +<section><title>Public_Key 0.17</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + ssh_decode now handles comments, at the end of the line, + containing withe spaces correctly</p> + <p> + Own Id: OTP-9361</p> + </item> + <item> + <p> + Add missing references to sha224 and sha384</p> + <p> + Own Id: OTP-9362 Aux Id: seq12116 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + public_key now supports PKCS-10 and includes exprimental + support for PKCS-7</p> + <p> + Own Id: OTP-10509 Aux Id: kunagi-291 [202] </p> + </item> + </list> + </section> + +</section> + <section><title>Public_Key 0.16</title> <section><title>Improvements and New Features</title> diff --git a/lib/public_key/include/public_key.hrl b/lib/public_key/include/public_key.hrl index 2dfdbbb8f3..90ca7256ea 100644 --- a/lib/public_key/include/public_key.hrl +++ b/lib/public_key/include/public_key.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2011. All Rights Reserved. +%% Copyright Ericsson AB 2008-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/public_key/src/pubkey_cert_records.erl b/lib/public_key/src/pubkey_cert_records.erl index 33fe940ea2..98004c71a3 100644 --- a/lib/public_key/src/pubkey_cert_records.erl +++ b/lib/public_key/src/pubkey_cert_records.erl @@ -119,7 +119,7 @@ encode_supportedPublicKey(#'OTPSubjectPublicKeyInfo'{algorithm= PA = subjectPublicKey = SPK0}) -> Type = supportedPublicKeyAlgorithms(Algo), {ok, SPK} = 'OTP-PUB-KEY':encode(Type, SPK0), - #'OTPSubjectPublicKeyInfo'{subjectPublicKey = {0,list_to_binary(SPK)}, algorithm=PA}. + #'OTPSubjectPublicKeyInfo'{subjectPublicKey = {0,SPK}, algorithm=PA}. %%% Extensions @@ -161,7 +161,7 @@ decode_extensions(Exts) -> case extension_id(Id) of undefined -> Ext; Type -> - {ok, Value} = 'OTP-PUB-KEY':decode(Type, list_to_binary(Value0)), + {ok, Value} = 'OTP-PUB-KEY':decode(Type, iolist_to_binary(Value0)), Ext#'Extension'{extnValue=transform(Value,decode)} end end, Exts). @@ -176,7 +176,7 @@ encode_extensions(Exts) -> Type -> Value1 = transform(Value0,encode), {ok, Value} = 'OTP-PUB-KEY':encode(Type, Value1), - Ext#'Extension'{extnValue=list_to_binary(Value)} + Ext#'Extension'{extnValue=Value} end end, Exts). diff --git a/lib/public_key/test/erl_make_certs.erl b/lib/public_key/test/erl_make_certs.erl index 254aa6d2f9..d6bdd05d01 100644 --- a/lib/public_key/test/erl_make_certs.erl +++ b/lib/public_key/test/erl_make_certs.erl @@ -137,10 +137,10 @@ decode_key(PemBin, Pw) -> encode_key(Key = #'RSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('RSAPrivateKey', Key), - {'RSAPrivateKey', list_to_binary(Der), not_encrypted}; + {'RSAPrivateKey', Der, not_encrypted}; encode_key(Key = #'DSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('DSAPrivateKey', Key), - {'DSAPrivateKey', list_to_binary(Der), not_encrypted}. + {'DSAPrivateKey', Der, not_encrypted}. make_tbs(SubjectKey, Opts) -> Version = list_to_atom("v"++integer_to_list(proplists:get_value(version, Opts, 3))), diff --git a/lib/sasl/src/systools_make.erl b/lib/sasl/src/systools_make.erl index e8b28998c1..29c4a0d168 100644 --- a/lib/sasl/src/systools_make.erl +++ b/lib/sasl/src/systools_make.erl @@ -1485,7 +1485,8 @@ mandatory_modules() -> preloaded() -> %% Sorted - [erl_prim_loader,erlang,init,otp_ring0,prim_file,prim_inet, prim_zip,zlib]. + [erl_prim_loader,erlang,erts_internal,init,otp_ring0,prim_file,prim_inet, + prim_zip,zlib]. %%______________________________________________________________________ %% Kernel processes; processes that are specially treated by the init diff --git a/lib/snmp/doc/src/snmpm.xml b/lib/snmp/doc/src/snmpm.xml index 8ab3be8e18..e6e188d5d5 100644 --- a/lib/snmp/doc/src/snmpm.xml +++ b/lib/snmp/doc/src/snmpm.xml @@ -367,7 +367,7 @@ sec_level = noAuthNoPriv | authNoPriv | authPriv <p>Update agent config. The function <c>update_agent_info/3</c> should be used when several values needs to be updated atomically. </p> <p>See function - <seealso marker="#register_agent">register_agent</seealso>) + <seealso marker="#register_agent">register_agent</seealso> for more info about what kind of items are allowed. </p> <marker id="which_agents"></marker> diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml index 36010da07a..d4acb2ef1a 100644 --- a/lib/ssh/doc/src/notes.xml +++ b/lib/ssh/doc/src/notes.xml @@ -29,6 +29,50 @@ <file>notes.xml</file> </header> +<section><title>Ssh 2.1.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + SSH quiet mode</p> + <p> + A new option to ssh:connect/3,4, quiet_mode. If true, the + client will not print out anything on authorization.</p> + <p> + Own Id: OTP-10429 Aux Id: kunagi-273 [184] </p> + </item> + <item> + <p> + Restrict which key algorithms to use</p> + <p> + A new option to ssh:connect/3,4 is introduced, + public_key_algs, where you can restrict which key + algorithms to use and in which order to try them.</p> + <p> + Own Id: OTP-10498 Aux Id: kunagi-289 [200] </p> + </item> + <item> + <p> + Confidentiality of client password</p> + <p> + Unsets clients password after authentication.</p> + <p> + Own Id: OTP-10511 Aux Id: kunagi-292 [203] </p> + </item> + <item> + <p> + Fixed user interaction for SSH</p> + <p> + It's now available to accept hosts and input password</p> + <p> + Own Id: OTP-10513 Aux Id: kunagi-293 [204] </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 2.1.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssh/doc/src/ssh.xml b/lib/ssh/doc/src/ssh.xml index aac4b462a2..04b7a2ae56 100644 --- a/lib/ssh/doc/src/ssh.xml +++ b/lib/ssh/doc/src/ssh.xml @@ -308,6 +308,18 @@ <item> <p>Determines if SSH shall use IPv6 or not (only used when HostAddress is set to any).</p></item> + <tag><c><![CDATA[{failfun, fun()}]]></c></tag> + <item> + <p>Provide a fun() to implement your own logging when a user fails to authenticate.</p> + </item> + <tag><c><![CDATA[{connectfun, fun()}]]></c></tag> + <item> + <p>Provide a fun() to implement your own logging when a user authenticates to the server.</p> + </item> + <tag><c><![CDATA[{disconnectfun, fun()}]]></c></tag> + <item> + <p>Provide a fun() to implement your own logging when a user disconnects from the server.</p> + </item> </taglist> </desc> </func> diff --git a/lib/ssh/src/ssh_io.erl b/lib/ssh/src/ssh_io.erl index 17a7cebb4a..01fc713569 100644 --- a/lib/ssh/src/ssh_io.erl +++ b/lib/ssh/src/ssh_io.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2011. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 6c01954010..49bbd5d27d 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -30,7 +30,54 @@ </header> <p>This document describes the changes made to the SSL application.</p> - <section><title>SSL 5.1</title> + <section><title>SSL 5.1.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + ssl:ssl_accept/2 timeout is no longer ignored</p> + <p> + Own Id: OTP-10600</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 5.1.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + ssl:recv/3 could "loose" data when the timeout occurs. If + the timout in ssl:connect or ssl:ssl_accept expired the + ssl connection process was not terminated as it should, + this due to gen_fsm:send_all_state_event timout is a + client side time out. These timouts are now handled by + the gen_fsm-procss instead.</p> + <p> + Own Id: OTP-10569</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Better termination handling that avoids hanging.</p> + <p> + Own Id: OTP-10574</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 5.1</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/ssl/src/Makefile b/lib/ssl/src/Makefile index c5c5bf593a..6be8a1456e 100644 --- a/lib/ssl/src/Makefile +++ b/lib/ssl/src/Makefile @@ -130,3 +130,23 @@ release_spec: opt release_docs_spec: +# ---------------------------------------------------- +# Dependencies +# ---------------------------------------------------- +$(EBIN)/inet_tls_dist.$(EMULATOR): ../../kernel/include/net_address.hrl ../../kernel/include/dist.hrl ../../kernel/include/dist_util.hrl +$(EBIN)/ssl.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl ssl_handshake.hrl ../../public_key/include/public_key.hrl +$(EBIN)/ssl_alert.$(EMULATOR): ssl_alert.hrl ssl_record.hrl +$(EBIN)/ssl_certificate.$(EMULATOR): ssl_internal.hrl ssl_alert.hrl ssl_handshake.hrl ../../public_key/include/public_key.hrl +$(EBIN)/ssl_certificate_db.$(EMULATOR): ssl_internal.hrl ../../public_key/include/public_key.hrl ../../kernel/include/file.hrl +$(EBIN)/ssl_cipher.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl ssl_handshake.hrl ssl_alert.hrl ../../public_key/include/public_key.hrl +$(EBIN)/ssl_connection.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl ssl_handshake.hrl ssl_alert.hrl ../../public_key/include/public_key.hrl +$(EBIN)/ssl_handshake.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl ssl_handshake.hrl ssl_alert.hrl ../../public_key/include/public_key.hrl +$(EBIN)/ssl_manager.$(EMULATOR): ssl_internal.hrl ssl_handshake.hrl ../../kernel/include/file.hrl +$(EBIN)/ssl_record.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl ssl_handshake.hrl ssl_alert.hrl +$(EBIN)/ssl_session.$(EMULATOR): ssl_internal.hrl ssl_handshake.hrl +$(EBIN)/ssl_session_cache.$(EMULATOR): ssl_internal.hrl ssl_handshake.hrl +$(EBIN)/ssl_session_cache_api.$(EMULATOR): ssl_internal.hrl ssl_handshake.hrl +$(EBIN)/ssl_ssl3.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl +$(EBIN)/ssl_tls1.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl + + diff --git a/lib/ssl/src/ssl.appup.src b/lib/ssl/src/ssl.appup.src index c118c129e8..9b1227fa7f 100644 --- a/lib/ssl/src/ssl.appup.src +++ b/lib/ssl/src/ssl.appup.src @@ -1,6 +1,8 @@ %% -*- erlang -*- {"%VSN%", [ + {"5.1.1", [{restart_application, ssl}] + }, {"5.1", [ {load_module, ssl_connection, soft_purge, soft_purge, []} ] @@ -10,6 +12,8 @@ {<<"3\\.*">>, [{restart_application, ssl}]} ], [ + {"5.1.1", [{restart_application, ssl}] + }, {"5.1", [ {load_module, ssl_connection, soft_purge, soft_purge, []} ] diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index 771bfa5739..6224334a6e 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -47,7 +47,7 @@ -export_type([connect_option/0, listen_option/0, ssl_option/0, transport_option/0, erl_cipher_suite/0, %% From ssl_cipher.hrl tls_atom_version/0, %% From ssl_internal.hrl - prf_random/0]). + prf_random/0, sslsocket/0]). -record(config, {ssl, %% SSL parameters inet_user, %% User set inet options @@ -55,6 +55,8 @@ inet_ssl, %% inet options for internal ssl socket cb %% Callback info }). + +-type sslsocket() :: #sslsocket{}. -type connect_option() :: socket_connect_option() | ssl_option() | transport_option(). -type socket_connect_option() :: gen_tcp:connect_option(). -type listen_option() :: socket_listen_option() | ssl_option() | transport_option(). diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index 87cf49d07d..cde13069b5 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -89,6 +89,7 @@ log_alert, % boolean() renegotiation, % {boolean(), From | internal | peer} start_or_recv_from, % "gen_fsm From" + timer, % start_or_recv_timer send_queue, % queue() terminated = false, % allow_renegotiate = true, @@ -775,8 +776,9 @@ handle_sync_event({application_data, Data}, From, StateName, get_timeout(State)}; handle_sync_event({start, Timeout}, StartFrom, hello, State) -> - start_or_recv_cancel_timer(Timeout, StartFrom), - hello(start, State#state{start_or_recv_from = StartFrom}); + Timer = start_or_recv_cancel_timer(Timeout, StartFrom), + hello(start, State#state{start_or_recv_from = StartFrom, + timer = Timer}); %% The two clauses below could happen if a server upgrades a socket in %% active mode. Note that in this case we are lucky that @@ -792,8 +794,9 @@ handle_sync_event({start,_}, _From, error, {Error, State = #state{}}) -> {stop, {shutdown, Error}, {error, Error}, State}; handle_sync_event({start, Timeout}, StartFrom, StateName, State) -> - start_or_recv_cancel_timer(Timeout, StartFrom), - {next_state, StateName, State#state{start_or_recv_from = StartFrom}, get_timeout(State)}; + Timer = start_or_recv_cancel_timer(Timeout, StartFrom), + {next_state, StateName, State#state{start_or_recv_from = StartFrom, + timer = Timer}, get_timeout(State)}; handle_sync_event(close, _, StateName, State) -> %% Run terminate before returning @@ -825,14 +828,16 @@ handle_sync_event({shutdown, How0}, _, StateName, end; handle_sync_event({recv, N, Timeout}, RecvFrom, connection = StateName, State0) -> - start_or_recv_cancel_timer(Timeout, RecvFrom), - passive_receive(State0#state{bytes_to_read = N, start_or_recv_from = RecvFrom}, StateName); + Timer = start_or_recv_cancel_timer(Timeout, RecvFrom), + passive_receive(State0#state{bytes_to_read = N, + start_or_recv_from = RecvFrom, timer = Timer}, StateName); %% Doing renegotiate wait with handling request until renegotiate is %% finished. Will be handled by next_state_is_connection/2. handle_sync_event({recv, N, Timeout}, RecvFrom, StateName, State) -> - start_or_recv_cancel_timer(Timeout, RecvFrom), - {next_state, StateName, State#state{bytes_to_read = N, start_or_recv_from = RecvFrom}, + Timer = start_or_recv_cancel_timer(Timeout, RecvFrom), + {next_state, StateName, State#state{bytes_to_read = N, start_or_recv_from = RecvFrom, + timer = Timer}, get_timeout(State)}; handle_sync_event({new_user, User}, _From, StateName, @@ -996,13 +1001,20 @@ handle_info({'DOWN', MonitorRef, _, _, _}, _, handle_info(allow_renegotiate, StateName, State) -> {next_state, StateName, State#state{allow_renegotiate = true}, get_timeout(State)}; - -handle_info({cancel_start_or_recv, RecvFrom}, connection = StateName, #state{start_or_recv_from = RecvFrom} = State) -> + +handle_info({cancel_start_or_recv, StartFrom}, StateName, + #state{renegotiation = {false, first}} = State) when StateName =/= connection -> + gen_fsm:reply(StartFrom, {error, timeout}), + {stop, {shutdown, user_timeout}, State#state{timer = undefined}}; + +handle_info({cancel_start_or_recv, RecvFrom}, StateName, #state{start_or_recv_from = RecvFrom} = State) -> gen_fsm:reply(RecvFrom, {error, timeout}), - {next_state, StateName, State#state{start_or_recv_from = undefined}, get_timeout(State)}; + {next_state, StateName, State#state{start_or_recv_from = undefined, + bytes_to_read = undefined, + timer = undefined}, get_timeout(State)}; handle_info({cancel_start_or_recv, _RecvFrom}, StateName, State) -> - {next_state, StateName, State, get_timeout(State)}; + {next_state, StateName, State#state{timer = undefined}, get_timeout(State)}; handle_info(Msg, StateName, State) -> Report = io_lib:format("SSL: Got unexpected info: ~p ~n", [Msg]), @@ -1770,6 +1782,7 @@ read_application_data(Data, #state{user_application = {_Mon, Pid}, socket_options = SOpts, bytes_to_read = BytesToRead, start_or_recv_from = RecvFrom, + timer = Timer, user_data_buffer = Buffer0} = State0) -> Buffer1 = if Buffer0 =:= <<>> -> Data; @@ -1779,9 +1792,11 @@ read_application_data(Data, #state{user_application = {_Mon, Pid}, case get_data(SOpts, BytesToRead, Buffer1) of {ok, ClientData, Buffer} -> % Send data SocketOpt = deliver_app_data(Socket, SOpts, ClientData, Pid, RecvFrom), + cancel_timer(Timer), State = State0#state{user_data_buffer = Buffer, start_or_recv_from = undefined, - bytes_to_read = 0, + timer = undefined, + bytes_to_read = undefined, socket_options = SocketOpt }, if @@ -1794,6 +1809,8 @@ read_application_data(Data, #state{user_application = {_Mon, Pid}, end; {more, Buffer} -> % no reply, we need more data next_record(State0#state{user_data_buffer = Buffer}); + {passive, Buffer} -> + next_record_if_active(State0#state{user_data_buffer = Buffer}); {error,_Reason} -> %% Invalid packet in packet mode deliver_packet_error(Socket, SOpts, Buffer1, Pid, RecvFrom), {stop, normal, State0} @@ -1835,6 +1852,9 @@ is_time_to_renegotiate(_,_) -> %% Picks ClientData get_data(_, _, <<>>) -> {more, <<>>}; +%% Recv timed out save buffer data until next recv +get_data(#socket_options{active=false}, undefined, Buffer) -> + {passive, Buffer}; get_data(#socket_options{active=Active, packet=Raw}, BytesToRead, Buffer) when Raw =:= raw; Raw =:= 0 -> %% Raw Mode if @@ -2140,7 +2160,6 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions}, User, tls_record_buffer = <<>>, tls_cipher_texts = [], user_application = {Monitor, User}, - bytes_to_read = 0, user_data_buffer = <<>>, log_alert = true, session_cache_cb = SessionCacheCb, @@ -2363,9 +2382,11 @@ ack_connection(#state{renegotiation = {true, From}} = State) -> gen_fsm:reply(From, ok), State#state{renegotiation = undefined}; ack_connection(#state{renegotiation = {false, first}, - start_or_recv_from = StartFrom} = State) when StartFrom =/= undefined -> + start_or_recv_from = StartFrom, + timer = Timer} = State) when StartFrom =/= undefined -> gen_fsm:reply(StartFrom, connected), - State#state{renegotiation = undefined, start_or_recv_from = undefined}; + cancel_timer(Timer), + State#state{renegotiation = undefined, start_or_recv_from = undefined, timer = undefined}; ack_connection(State) -> State. @@ -2503,10 +2524,15 @@ default_hashsign(_Version, KeyExchange) {null, anon}. start_or_recv_cancel_timer(infinity, _RecvFrom) -> - ok; + undefined; start_or_recv_cancel_timer(Timeout, RecvFrom) -> erlang:send_after(Timeout, self(), {cancel_start_or_recv, RecvFrom}). +cancel_timer(undefined) -> + ok; +cancel_timer(Timer) -> + erlang:cancel_timer(Timer). + handle_unrecv_data(StateName, #state{socket = Socket, transport_cb = Transport} = State) -> inet:setopts(Socket, [{active, false}]), case Transport:recv(Socket, 0, 0) of diff --git a/lib/ssl/src/ssl_manager.erl b/lib/ssl/src/ssl_manager.erl index 0cf4f2ce33..13689ce7d8 100644 --- a/lib/ssl/src/ssl_manager.erl +++ b/lib/ssl/src/ssl_manager.erl @@ -24,8 +24,6 @@ -module(ssl_manager). -behaviour(gen_server). --include("ssl_internal.hrl"). - %% Internal application API -export([start_link/1, start_link_dist/1, connection_init/2, cache_pem_file/2, diff --git a/lib/ssl/src/ssl_session.erl b/lib/ssl/src/ssl_session.erl index 2ad422fc03..a24b2d9444 100644 --- a/lib/ssl/src/ssl_session.erl +++ b/lib/ssl/src/ssl_session.erl @@ -72,15 +72,12 @@ valid_session(#session{time_stamp = TimeStamp}, LifeTime) -> server_id(Port, <<>>, _SslOpts, _Cert, _, _) -> {ssl_manager:new_session_id(Port), undefined}; -server_id(Port, SuggestedId, - #ssl_options{reuse_sessions = ReuseEnabled, - reuse_session = ReuseFun}, - Cert, Cache, CacheCb) -> +server_id(Port, SuggestedId, Options, Cert, Cache, CacheCb) -> LifeTime = case application:get_env(ssl, session_lifetime) of {ok, Time} when is_integer(Time) -> Time; _ -> ?'24H_in_sec' end, - case is_resumable(SuggestedId, Port, ReuseEnabled,ReuseFun, + case is_resumable(SuggestedId, Port, Options, Cache, CacheCb, LifeTime, Cert) of {true, Resumed} -> @@ -112,9 +109,9 @@ select_session(Sessions, #ssl_options{ciphers = Ciphers}, OwnCert) -> [[Id, _]|_] -> Id end. -is_resumable(_, _, false, _, _, _, _, _) -> +is_resumable(_, _, #ssl_options{reuse_sessions = false}, _, _, _, _) -> {false, undefined}; -is_resumable(SuggestedSessionId, Port, true, ReuseFun, Cache, +is_resumable(SuggestedSessionId, Port, #ssl_options{reuse_session = ReuseFun} = Options, Cache, CacheCb, SecondLifeTime, OwnCert) -> case CacheCb:lookup(Cache, {Port, SuggestedSessionId}) of #session{cipher_suite = CipherSuite, @@ -125,6 +122,7 @@ is_resumable(SuggestedSessionId, Port, true, ReuseFun, Cache, case resumable(IsResumable) andalso (OwnCert == SessionOwnCert) andalso valid_session(Session, SecondLifeTime) + andalso reusable_options(Options, Session) andalso ReuseFun(SuggestedSessionId, PeerCert, Compression, CipherSuite) of @@ -139,3 +137,9 @@ resumable(new) -> false; resumable(IsResumable) -> IsResumable. + +reusable_options(#ssl_options{fail_if_no_peer_cert = true, + verify = verify_peer}, Session) -> + (Session#session.peer_certificate =/= undefined); +reusable_options(_,_) -> + true. diff --git a/lib/ssl/test/erl_make_certs.erl b/lib/ssl/test/erl_make_certs.erl index 254aa6d2f9..d6bdd05d01 100644 --- a/lib/ssl/test/erl_make_certs.erl +++ b/lib/ssl/test/erl_make_certs.erl @@ -137,10 +137,10 @@ decode_key(PemBin, Pw) -> encode_key(Key = #'RSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('RSAPrivateKey', Key), - {'RSAPrivateKey', list_to_binary(Der), not_encrypted}; + {'RSAPrivateKey', Der, not_encrypted}; encode_key(Key = #'DSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('DSAPrivateKey', Key), - {'DSAPrivateKey', list_to_binary(Der), not_encrypted}. + {'DSAPrivateKey', Der, not_encrypted}. make_tbs(SubjectKey, Opts) -> Version = list_to_atom("v"++integer_to_list(proplists:get_value(version, Opts, 3))), diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index a202aca943..faed91e559 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -248,6 +248,7 @@ api_tests() -> [connection_info, peername, peercert, + peercert_with_client_cert, sockname, versions, controlling_process, @@ -258,7 +259,9 @@ api_tests() -> shutdown_both, shutdown_error, hibernate, - listen_socket + listen_socket, + ssl_accept_timeout, + ssl_recv_timeout ]. certificate_verify_tests() -> @@ -274,6 +277,7 @@ certificate_verify_tests() -> server_verify_client_once_passive, server_verify_client_once_active, server_verify_client_once_active_once, + new_server_wants_peer_cert, client_verify_none_passive, client_verify_none_active, client_verify_none_active_once, @@ -788,6 +792,43 @@ peercert(Config) when is_list(Config) -> peercert_result(Socket) -> ssl:peercert(Socket). +%%-------------------------------------------------------------------- + +peercert_with_client_cert(doc) -> + [""]; +peercert_with_client_cert(suite) -> + []; +peercert_with_client_cert(Config) when is_list(Config) -> + ClientOpts = ?config(client_dsa_opts, Config), + ServerOpts = ?config(server_dsa_verify_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = ssl_test_lib:start_server([{node, ClientNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, peercert_result, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client([{node, ServerNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, peercert_result, []}}, + {options, ClientOpts}]), + + ServerCertFile = proplists:get_value(certfile, ServerOpts), + [{'Certificate', ServerBinCert, _}]= ssl_test_lib:pem_to_der(ServerCertFile), + ClientCertFile = proplists:get_value(certfile, ClientOpts), + [{'Certificate', ClientBinCert, _}]= ssl_test_lib:pem_to_der(ClientCertFile), + + ServerMsg = {ok, ClientBinCert}, + ClientMsg = {ok, ServerBinCert}, + + test_server:format("Testcase ~p, Client ~p Server ~p ~n", + [self(), Client, Server]), + + ssl_test_lib:check_result(Server, ServerMsg, Client, ClientMsg), + + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). %%-------------------------------------------------------------------- sockname(doc) -> @@ -3610,9 +3651,14 @@ no_reuses_session_server_restart_new_cert(Config) when is_list(Config) -> %% Make sure session is registered test_server:sleep(?SLEEP), + Monitor = erlang:monitor(process, Server), ssl_test_lib:close(Server), ssl_test_lib:close(Client0), - + receive + {'DOWN', Monitor, _, _, _} -> + ok + end, + Server1 = ssl_test_lib:start_server([{node, ServerNode}, {port, Port}, {from, self()}, @@ -3719,10 +3765,14 @@ reuseaddr(Config) when is_list(Config) -> {from, self()}, {mfa, {ssl_test_lib, no_result, []}}, {options, [{active, false} | ClientOpts]}]), - test_server:sleep(?SLEEP), + Monitor = erlang:monitor(process, Server), ssl_test_lib:close(Server), ssl_test_lib:close(Client), - + receive + {'DOWN', Monitor, _, _, _} -> + ok + end, + Server1 = ssl_test_lib:start_server([{node, ServerNode}, {port, Port}, {from, self()}, @@ -3805,6 +3855,61 @@ listen_socket(Config) -> {error, enotconn} = ssl:shutdown(ListenSocket, read_write), ok = ssl:close(ListenSocket). +%%-------------------------------------------------------------------- +ssl_accept_timeout(doc) -> + ["Test ssl:ssl_accept timeout"]; +ssl_accept_timeout(suite) -> + []; +ssl_accept_timeout(Config) -> + process_flag(trap_exit, true), + ServerOpts = ?config(server_opts, Config), + {_, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {timeout, 5000}, + {mfa, {ssl_test_lib, + no_result_msg, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + {ok, CSocket} = gen_tcp:connect(Hostname, Port, [binary, {active, true}]), + + receive + {tcp_closed, CSocket} -> + ssl_test_lib:check_result(Server, {error, timeout}), + receive + {'EXIT', Server, _} -> + [] = supervisor:which_children(ssl_connection_sup) + end + end. + +%%-------------------------------------------------------------------- +ssl_recv_timeout(doc) -> + ["Test ssl:ssl_accept timeout"]; +ssl_recv_timeout(suite) -> + []; +ssl_recv_timeout(Config) -> + ServerOpts = ?config(server_opts, Config), + ClientOpts = ?config(client_opts, Config), + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, send_recv_result_timeout_server, []}}, + {options, [{active, false} | ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, + send_recv_result_timeout_client, []}}, + {options, [{active, false} | ClientOpts]}]), + + ssl_test_lib:check_result(Client, ok, Server, ok), + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). %%-------------------------------------------------------------------- @@ -4041,6 +4146,67 @@ client_server_opts({KeyAlgo,_,_}, Config) when KeyAlgo == dss orelse KeyAlgo == {?config(client_dsa_opts, Config), ?config(server_dsa_opts, Config)}. + +%%-------------------------------------------------------------------- + +new_server_wants_peer_cert(doc) -> + ["Test that server configured to do client certification does" + " not reuse session without a client certificate."]; +new_server_wants_peer_cert(suite) -> + []; +new_server_wants_peer_cert(Config) when is_list(Config) -> + ServerOpts = ?config(server_opts, Config), + VServerOpts = [{verify, verify_peer}, {fail_if_no_peer_cert, true} + | ?config(server_verification_opts, Config)], + ClientOpts = ?config(client_verification_opts, Config), + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, peercert_result, []}}, + {options, [ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + Client = + ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, ClientOpts}]), + + Monitor = erlang:monitor(process, Server), + ssl_test_lib:close(Server), + ssl_test_lib:close(Client), + receive + {'DOWN', Monitor, _, _, _} -> + ok + end, + + Server1 = ssl_test_lib:start_server([{node, ServerNode}, {port, Port}, + {from, self()}, + {mfa, {?MODULE, peercert_result, []}}, + {options, VServerOpts}]), + Client1 = + ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, [ClientOpts]}]), + + CertFile = proplists:get_value(certfile, ClientOpts), + [{'Certificate', BinCert, _}]= ssl_test_lib:pem_to_der(CertFile), + + ServerMsg = {error, no_peercert}, + Sever1Msg = {ok, BinCert}, + + ssl_test_lib:check_result(Server, ServerMsg, Server1, Sever1Msg), + + ssl_test_lib:close(Server1), + ssl_test_lib:close(Client), + ssl_test_lib:close(Client1). + + %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- @@ -4049,6 +4215,23 @@ send_recv_result(Socket) -> {ok,"Hello world"} = ssl:recv(Socket, 11), ok. +send_recv_result_timeout_client(Socket) -> + {error, timeout} = ssl:recv(Socket, 11, 500), + ssl:send(Socket, "Hello world"), + receive + Msg -> + io:format("Msg ~p~n",[Msg]) + after 500 -> + ok + end, + {ok, "Hello world"} = ssl:recv(Socket, 11, 500), + ok. +send_recv_result_timeout_server(Socket) -> + ssl:send(Socket, "Hello"), + {ok, "Hello world"} = ssl:recv(Socket, 11), + ssl:send(Socket, " world"), + ok. + recv_close(Socket) -> {error, closed} = ssl:recv(Socket, 11), receive diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 63731ee25c..f1f5b9ae0a 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -72,7 +72,13 @@ run_server(Opts) -> run_server(ListenSocket, Opts). run_server(ListenSocket, Opts) -> - AcceptSocket = connect(ListenSocket, Opts), + do_run_server(ListenSocket, connect(ListenSocket, Opts), Opts). + +do_run_server(_, {error, timeout} = Result, Opts) -> + Pid = proplists:get_value(from, Opts), + Pid ! {self(), Result}; + +do_run_server(ListenSocket, AcceptSocket, Opts) -> Node = proplists:get_value(node, Opts), Pid = proplists:get_value(from, Opts), {Module, Function, Args} = proplists:get_value(mfa, Opts), @@ -102,7 +108,8 @@ run_server(ListenSocket, Opts) -> connect(ListenSocket, Opts) -> Node = proplists:get_value(node, Opts), ReconnectTimes = proplists:get_value(reconnect_times, Opts, 0), - AcceptSocket = connect(ListenSocket, Node, 1 + ReconnectTimes, dummy), + Timeout = proplists:get_value(timeout, Opts, infinity), + AcceptSocket = connect(ListenSocket, Node, 1 + ReconnectTimes, dummy, Timeout), case ReconnectTimes of 0 -> AcceptSocket; @@ -111,15 +118,21 @@ connect(ListenSocket, Opts) -> AcceptSocket end. -connect(_, _, 0, AcceptSocket) -> +connect(_, _, 0, AcceptSocket, _) -> AcceptSocket; -connect(ListenSocket, Node, N, _) -> +connect(ListenSocket, Node, N, _, Timeout) -> test_server:format("ssl:transport_accept(~p)~n", [ListenSocket]), {ok, AcceptSocket} = rpc:call(Node, ssl, transport_accept, [ListenSocket]), - test_server:format("ssl:ssl_accept(~p)~n", [AcceptSocket]), - ok = rpc:call(Node, ssl, ssl_accept, [AcceptSocket]), - connect(ListenSocket, Node, N-1, AcceptSocket). + test_server:format("ssl:ssl_accept(~p, ~p)~n", [AcceptSocket, Timeout]), + + case rpc:call(Node, ssl, ssl_accept, [AcceptSocket, Timeout]) of + ok -> + connect(ListenSocket, Node, N-1, AcceptSocket, Timeout); + Result -> + Result + end. + remove_close_msg(0) -> ok; diff --git a/lib/ssl/test/ssl_to_openssl_SUITE.erl b/lib/ssl/test/ssl_to_openssl_SUITE.erl index 98ef050b14..f4e19b3f87 100644 --- a/lib/ssl/test/ssl_to_openssl_SUITE.erl +++ b/lib/ssl/test/ssl_to_openssl_SUITE.erl @@ -1080,14 +1080,13 @@ ssl2_erlang_server_openssl_client(Config) when is_list(Config) -> OpenSslPort = open_port({spawn, Cmd}, [stderr_to_stdout]), port_command(OpenSslPort, Data), - + receive + {'EXIT', OpenSslPort, _} -> + ok + + end, ssl_test_lib:check_result(Server, {error,"protocol version"}), - - %% Clean close down! Server needs to be closed first !! - ssl_test_lib:close(Server), - close_port(OpenSslPort), - process_flag(trap_exit, false), - ok. + process_flag(trap_exit, false). %%-------------------------------------------------------------------- erlang_client_openssl_server_npn(doc) -> diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index bc8b8fd039..adfb29e639 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1 +1 @@ -SSL_VSN = 5.1.1 +SSL_VSN = 5.1.2 diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index b4d9f9a444..2a308cbe09 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -30,6 +30,21 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 1.18.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Minor test updates</p> + <p> + Own Id: OTP-10591</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 1.18.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/src/erl_expand_records.erl b/lib/stdlib/src/erl_expand_records.erl index 9759a8f001..85defacc43 100644 --- a/lib/stdlib/src/erl_expand_records.erl +++ b/lib/stdlib/src/erl_expand_records.erl @@ -381,9 +381,6 @@ expr({call,Line,{record_field,_,_,_}=M,As0}, St0) -> expr({call,Line,{remote,Lr,M,F},As0}, St0) -> {[M1,F1 | As1],St1} = expr_list([M,F | As0], St0), {{call,Line,{remote,Lr,M1,F1},As1},St1}; -expr({call,Line,{tuple,Lt,[{atom,_,_}=M,{atom,_,_}=F]},As0}, St0) -> - {As,St1} = expr_list(As0, St0), - {{call,Line,{tuple,Lt,[M,F]},As},St1}; expr({call,Line,F,As0}, St0) -> {[Fun1 | As1],St1} = expr_list([F | As0], St0), {{call,Line,Fun1,As1},St1}; diff --git a/lib/test_server/doc/src/notes.xml b/lib/test_server/doc/src/notes.xml index 3b109193a0..6a9add044a 100644 --- a/lib/test_server/doc/src/notes.xml +++ b/lib/test_server/doc/src/notes.xml @@ -32,6 +32,47 @@ <file>notes.xml</file> </header> +<section><title>Test_Server 3.5.3</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + test_server_h will now recognize info_reports written by + ct connection handlers (according to the description in + cth_conn_log) and ignore them as they will be completely + handled by by ct_conn_log_h.</p> + <p> + Earlier test_server_h would print a tag (testcase name) + before forwarding the report to error_logger_tty_h. This + would cause lots of tags in the log with no info report + following (since error_logger_tty_h did not handle them).</p> + <p> + Own Id: OTP-10571</p> + </item> + </list> + </section> + + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + Restore Config data if lost when test case fails.</p> + <p> + Own Id: OTP-10070 Aux Id: kunagi-175 [86] </p> + </item> + <item> + <p> + IO server error in test_server.</p> + <p> + Own Id: OTP-10125 Aux Id: OTP-10101, kunagi-177 [88] </p> + </item> + </list> + </section> + +</section> + <section><title>Test_Server 3.5.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/test_server/src/test_server_io.erl b/lib/test_server/src/test_server_io.erl index e960b3087a..777b377201 100644 --- a/lib/test_server/src/test_server_io.erl +++ b/lib/test_server/src/test_server_io.erl @@ -307,8 +307,10 @@ do_print_buffered(Q0, St) -> gc(#st{gls=Gls0}) -> InUse0 = [begin - {group_leader,GL} = process_info(P, group_leader), - GL + case process_info(P, group_leader) of + {group_leader,GL} -> GL; + undefined -> undefined + end end || P <- processes()], InUse = ordsets:from_list(InUse0), Gls = gb_sets:to_list(Gls0), diff --git a/lib/test_server/src/ts.erl b/lib/test_server/src/ts.erl index 3ddc58fdbc..115e783070 100644 --- a/lib/test_server/src/ts.erl +++ b/lib/test_server/src/ts.erl @@ -259,21 +259,43 @@ run(List, Opts) when is_list(List), is_list(Opts) -> run(Testspec, Config) when is_atom(Testspec), is_list(Config) -> Options=check_test_get_opts(Testspec, Config), File=atom_to_list(Testspec), - Spec = case code:lib_dir(Testspec) of - _ when Testspec == emulator; - Testspec == system; - Testspec == epmd -> - File++".spec"; - {error, bad_name} -> - create_skip_spec(Testspec, tests(Testspec)); - Path -> - case file:read_file_info(filename:join(Path,"ebin")) of - {ok,_} -> - File++".spec"; - _ -> - create_skip_spec(Testspec, tests(Testspec)) - end - end, + WhatToDo = + case Testspec of + %% Known to exist but fails generic tests below + emulator -> test; + system -> test; + erl_interface -> test; + epmd -> test; + _ -> + case code:lib_dir(Testspec) of + {error,bad_name} -> + %% Application does not exist + skip; + Path -> + case file:read_file_info(filename:join(Path,"ebin")) of + {ok,#file_info{type=directory}} -> + %% Erlang application is built + test; + _ -> + case filelib:wildcard( + filename:join([Path,"priv","*.jar"])) of + [] -> + %% The application is not built + skip; + [_|_] -> + %% Java application is built + test + end + end + end + end, + Spec = + case WhatToDo of + skip -> + create_skip_spec(Testspec, tests(Testspec)); + test -> + File++".spec" + end, run_test(File, [{spec,[Spec]}], Options); %% Runs one module in a spec (interactive) run(Testspec, Mod) when is_atom(Testspec), is_atom(Mod) -> diff --git a/lib/test_server/vsn.mk b/lib/test_server/vsn.mk index aecf595f3f..b956ebb2b3 100644 --- a/lib/test_server/vsn.mk +++ b/lib/test_server/vsn.mk @@ -1 +1 @@ -TEST_SERVER_VSN = 3.5.2 +TEST_SERVER_VSN = 3.5.3 diff --git a/lib/tools/doc/src/xref.xml b/lib/tools/doc/src/xref.xml index 17de66bb22..9706ae6746 100644 --- a/lib/tools/doc/src/xref.xml +++ b/lib/tools/doc/src/xref.xml @@ -1453,7 +1453,7 @@ Evaluates a predefined analysis. Note however that the code path will be traversed once for each used <seealso marker="#library_module">library module</seealso> while setting up module data. On the other hand, if there are only a few modules that are - used by not analyzed, using <c>code_path</c> may be faster + used but not analyzed, using <c>code_path</c> may be faster than setting the library path to <c>code:get_path()</c>. </p> <p>If the library path is set to <c>code_path</c>, the set of diff --git a/lib/tools/src/tools.app.src b/lib/tools/src/tools.app.src index 94998fb763..553c5eb96b 100644 --- a/lib/tools/src/tools.app.src +++ b/lib/tools/src/tools.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2009. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/system/doc/efficiency_guide/advanced.xml b/system/doc/efficiency_guide/advanced.xml index 821175bb09..ac35a37bc4 100644 --- a/system/doc/efficiency_guide/advanced.xml +++ b/system/doc/efficiency_guide/advanced.xml @@ -123,12 +123,11 @@ On 64-bit architectures: 4 words for a reference from the current local node, an <tag><em>Processes</em></tag> <item> <p>The maximum number of simultaneously alive Erlang processes is - by default 32768. This limit can be raised up to at most 268435456 - processes at startup (see documentation of the system flag - <seealso marker="erts:erl#max_processes">+P</seealso> in the - <seealso marker="erts:erl">erl(1)</seealso> documentation). - The maximum limit of 268435456 processes will at least on a 32-bit - architecture be impossible to reach due to memory shortage.</p> + by default 32768. This limit can be configured at startup, + for more information see the + <seealso marker="erts:erl#max_processes"><c>+P</c></seealso> + command line flag of + <seealso marker="erts:erl"><c>erl(1)</c></seealso>.</p> </item> <tag><em>Distributed nodes</em></tag> <item> @@ -184,13 +183,12 @@ On 64-bit architectures: 4 words for a reference from the current local node, an <tag><em>Open ports</em></tag> <item> <marker id="ports"></marker> - <p>The maximum number of simultaneously open Erlang ports is - by default 1024. This limit can be raised up to at most 268435456 - at startup (see environment variable - <seealso marker="erts:erlang#ERL_MAX_PORTS">ERL_MAX_PORTS</seealso> - in <seealso marker="erts:erlang">erlang(3)</seealso>) - The maximum limit of 268435456 open ports will at least on a 32-bit - architecture be impossible to reach due to memory shortage.</p> + <p>The maximum number of simultaneously oper Erlang ports is + often by default 16384. This limit can be configured at startup, + for more information see the + <seealso marker="erts:erl#max_ports"><c>+Q</c></seealso> + command line flag of + <seealso marker="erts:erl"><c>erl(1)</c></seealso>.</p> </item> <tag><em>Open files, and sockets</em></tag> <item> <marker id="files_sockets"></marker> diff --git a/system/doc/efficiency_guide/drivers.xml b/system/doc/efficiency_guide/drivers.xml index fec68ca059..b10595ea4d 100644 --- a/system/doc/efficiency_guide/drivers.xml +++ b/system/doc/efficiency_guide/drivers.xml @@ -105,9 +105,9 @@ client_port() -> <p>If you know that the binaries you return are always small, you should use driver API calls that do not require a pre-allocated binary, for instance - <seealso marker="erts:erl_driver#int driver_output-3">driver_output()</seealso> + <seealso marker="erts:erl_driver#driver_output">driver_output()</seealso> or - <seealso marker="erts:erl_driver#int driver_output_term-3">driver_output_term()</seealso> + <seealso marker="erts:erl_driver#erl_drv_output_term">erl_drv_output_term()</seealso> using the <c>ERL_DRV_BUF2BINARY</c> format, to allow the run-time to construct a heap binary.</p> @@ -120,7 +120,7 @@ client_port() -> the driver to an Erlang process, the driver must first allocate the binary and then send it to an Erlang process in some way.</p> - <p>Use <seealso marker="erts:erl_driver#ErlDrvBinary* driver_alloc_binary-1">driver_alloc_binary()</seealso> to allocate a binary.</p> + <p>Use <seealso marker="erts:erl_driver#driver_alloc_binary">driver_alloc_binary()</seealso> to allocate a binary.</p> <p>There are several ways to send a binary created with <c>driver_alloc_binary()</c>.</p> @@ -128,17 +128,17 @@ client_port() -> <list type="bulleted"> <item><p>From the <c>control</c> callback, a binary can be returned provided that - <seealso marker="erts:erl_driver#void set_port_control_flags-2">set_port_control()</seealso> + <seealso marker="erts:erl_driver#set_port_control_flags">set_port_control_flags()</seealso> has been called with the flag value <c>PORT_CONTROL_FLAG_BINARY</c>.</p> </item> <item><p>A single binary can be sent with - <seealso marker="erts:erl_driver#int driver_output_binary-6">driver_output_binary()</seealso>.</p></item> + <seealso marker="erts:erl_driver#driver_output_binary">driver_output_binary()</seealso>.</p></item> <item><p>Using - <seealso marker="erts:erl_driver#int driver_output_term-3">driver_output_term()</seealso> + <seealso marker="erts:erl_driver#erl_drv_output_term">erl_drv_output_term()</seealso> or - <seealso marker="erts:erl_driver#int driver_send_term-4">driver_send_term()</seealso>, + <seealso marker="erts:erl_driver#erl_drv_send_term">erl_drv_send_term()</seealso>, a binary can be included in an Erlang term.</p> </item> </list> diff --git a/system/doc/reference_manual/ports.xml b/system/doc/reference_manual/ports.xml index 4847dd67cd..c4e4ef1d35 100644 --- a/system/doc/reference_manual/ports.xml +++ b/system/doc/reference_manual/ports.xml @@ -87,8 +87,14 @@ of bytes, the option <c>binary</c> must be included.</p> <p>The port owner <c>Pid</c> can communicate with the port <c>Port</c> by sending and receiving messages. (In fact, any - process can send the messages to the port, but the messages from - the port always go to the port owner).</p> + process can send the messages to the port, but the port owner must + be identified in the message).</p> + <p>As of OTP-R16 messages sent to ports are delivered truly + asynchronously. The underlying implementation previously + delivered messages to ports synchronously. Message passing has + however always been documented as an asynchronous operation, so + this should not be an issue for an Erlang program communicating + with ports, unless false assumptions about ports has been made.</p> <p>Below, <c>Data</c> must be an I/O list. An I/O list is a binary or a (possibly deep) list of binaries or integers in the range 0..255.</p> @@ -127,8 +133,7 @@ <tcaption>Messages Received From a Port.</tcaption> </table> <p>Instead of sending and receiving messages, there are also a - number of BIFs that can be used. These can be called by any - process, not only the port owner.</p> + number of BIFs that can be used.</p> <table> <row> <cell align="left" valign="middle"><c>port_command(Port,Data)</c></cell> |