Merge branch 'bmk/megaco/flex_buffer_overrun_while_scanning_prop_parms/OTP-10998/r15' into bmk/megaco/flex_buffer_overrun_while_scanning_prop_parms/OTP-10998/r16

Conflicts:
	lib/megaco/doc/src/notes.xml
	lib/megaco/src/flex/megaco_flex_scanner_drv.flex.src
	lib/megaco/test/megaco_codec_prev3a_test.erl
	lib/megaco/test/megaco_codec_prev3b_test.erl
	lib/megaco/test/megaco_codec_prev3c_test.erl
	lib/megaco/test/megaco_codec_v1_test.erl
	lib/megaco/test/megaco_codec_v2_test.erl
	lib/megaco/test/megaco_codec_v3_test.erl
	lib/megaco/vsn.mk

1 files changed, 124 insertions, 657 deletions

diff --git a/lib/megaco/doc/src/notes.xml b/lib/megaco/doc/src/notes.xml
index bb30ce4c2a..91905b76f7 100644
--- a/lib/megaco/doc/src/notes.xml
+++ b/lib/megaco/doc/src/notes.xml
@@ -36,24 +36,138 @@
     section is the version number of Megaco.</p>
 
 
+  <section><title>Megaco 3.17</title>
+
+    <p>Version 3.17 supports code replacement in runtime from/to
+      version 3.16.0.3 and and 3.16.0.2. </p>
+
+    <section>
+      <title>Improvements and new features</title>
+
+      <p>-</p>
+
+<!--
+      <list type="bulleted">
+        <item>
+          <p>Allow whitespaces in installation path. </p>
+          <p>It is now possible to give configure and make an 
+	  installation/release path with whitespaces in it. </p>
+          <p>Own Id: OTP-10107</p>
+        </item>
+
+     </list>
+-->
+
+    </section>
+
+    <section>
+      <title>Fixed bugs and malfunctions</title>
+      
+ <!--
+      <p>-</p>
+ -->
+      
+      <list type="bulleted">
+	<item>
+          <p>Buffer overrun error while flex scanner processing 
+	  property parm groups. </p>
+          <p>This error occured only for large messages if a 
+	  buffer realloc was needed while processing the 
+	  property parm groups. </p>
+          <p>Own Id: OTP-10998</p>
+          <p>Aux Id: Seq 12263</p>
+	</item>
+	
+      </list>
+      
+    </section>
+
+    <section>
+      <title>Incompatibilities</title>
+      <p>-</p>
+
+<!--
+      <list type="bulleted">
+        <item>
+	  <p>A number of binary encoding alternatives has been removed. 
+	  The binary encoding option <c>driver</c> has been removed
+	  since this (the use of the asn1 linked in driver) is 
+	  now default and there is now way to <em>not</em> use it. 
+	  See <seealso marker="megaco_encode#binary_config">configuration of binary encoding</seealso> for more info. </p>
+        </item>
+
+      </list>
+-->
+
+    </section>
+
+  </section> <!-- 3.17 -->
+
+
   <section><title>Megaco 3.16.0.3</title>
 
-    <section><title>Improvements and New Features</title>
-      <list>
+    <p>Version 3.16.0.2 supports code replacement in runtime from/to
+      version 3.16.0.1, 3.16, 3.15.1.1, 3.15.1 and 3.15.</p>
+
+    <section>
+      <title>Improvements and new features</title>
+
+<!--
+      <p>-</p>
+-->
+
+      <list type="bulleted">
         <item>
-	    <p>Where necessary a comment stating encoding has been
-	    added to Erlang files. The comment is meant to be removed
-	    in Erlang/OTP R17B when UTF-8 becomes the default
-	    encoding. </p>
-          <p>
-	    Own Id: OTP-10630</p>
+	  <p>Where necessary, a comment stating encoding has been
+	  added to Erlang files. The comment is meant to be removed
+	  in Erlang/OTP R17B when UTF-8 becomes the default encoding. </p>
+	  <p>Own Id: OTP-10630</p>
         </item>
+
+     </list>
+
+    </section>
+
+    <section>
+      <title>Fixed bugs and malfunctions</title>
+      
+      <p>-</p>
+      
+ <!--
+      <list type="bulleted">
+	<item>
+	  <p>Fixing miscellaneous things detected by dialyzer. </p>
+	  <p>Own Id: OTP-9075</p>
+	</item>
+	
       </list>
+ -->
+      
     </section>
 
-</section>
+    <section>
+      <title>Incompatibilities</title>
+<!--
+      <p>-</p>
+-->
+
+      <list type="bulleted">
+        <item>
+	  <p>A number of binary encoding alternatives has been removed. 
+	  The binary encoding option <c>driver</c> has been removed
+	  since this (the use of the asn1 linked in driver) is 
+	  now default and there is now way to <em>not</em> use it. 
+	  See <seealso marker="megaco_encode#binary_config">configuration of binary encoding</seealso> for more info. </p>
+        </item>
+
+      </list>
 
-<section><title>Megaco 3.16.0.2</title>
+    </section>
+
+  </section> <!-- 3.16.0.3 -->
+
+
+  <section><title>Megaco 3.16.0.2</title>
 
     <p>Version 3.16.0.2 supports code replacement in runtime from/to
       version 3.16.0.1, 3.16, 3.15.1.1, 3.15.1 and 3.15.</p>
@@ -657,653 +771,6 @@
   </section> <!-- 3.13 -->
 
 
-  <section>
-    <title>Megaco 3.12</title>
-
-<!--
-    <p>Version 3.12 supports code replacement in runtime from/to
-      version 3.11.3.</p>
--->
-
-    <section>
-      <title>Improvements and new features</title>
-<!--
-      <p>-</p>
--->
-
-      <list type="bulleted">
-        <item>
-          <p>Improve handling of async transaction reply. </p>
-          <p>For asynchronous requests, issued using 
-            <seealso marker="megaco#cast">megaco:cast/3</seealso>, 
-            the reply will be delivered using the 
-            <seealso marker="megaco_user#trans_reply">handle_trans_reply/4,5</seealso> 
-            callback function. </p>
-          <p>If a receiver of a request, issued using 
-            <seealso marker="megaco#cast">megaco:cast/3</seealso>, 
-            does not reply in time, megaco re-sends the request. 
-            If the receiver of the request sends the reply at the same 
-            time as megaco re-sends, it may also send a reply to the 
-            resent request (thinking the first reply got lost). These 
-            two replies may arrive more or less at the same time, 
-            causing confusion. </p>
-          <p>In order to improve this situation, a number of 
-            improvements have been done: </p>
-          <list type="bulleted">
-            <item>
-              <p>When the first reply arrives, a timer, request-keep-alive, 
-                is started. This timer is used to decide when to stop 
-                accepting replies as legitimate. </p>
-              <p>The timeout time for the timer is specified by the
-                config option <em>request_keep_alive_timout</em>, 
-                which can be set per 
-                <seealso marker="megaco#ui_request_keep_alive_timeout">user</seealso> 
-                or per 
-                <seealso marker="megaco#ci_request_keep_alive_timeout">connection</seealso>. </p>
-	    </item>
-            <item>
-              <p>We also keep track of how many replies has been received
-                (we do this as long as the request-keep-alive timer is 
-                running). </p>
-            </item>
-            <item>
-              <p>Each reply that arrives while the request-keep-alive timer 
-                is running (including the first) will be delivered using the 
-                <seealso marker="megaco_user#trans_reply">handle_trans_reply/4,5</seealso> 
-                callback function, but with the UserReply augmented to
-                include a serial number indicating which reply number this
-                is. 
-                The <em>first</em> reply to arrive, 
-                will be numbered <em>one (1)</em>. </p>
-            </item>
-            <item>
-              <p>Replies arriving after the timer has expired will be delivered
-                in the same way as before, using the 
-                <seealso marker="megaco_user#unexpected_trans">handle_unexpected_trans/3,4</seealso> 
-                callback function. </p>
-	     </item>  
-            <item>
-              <p>Note that if the timer was <em>not</em> configured, 
-                megaco will act exactly as before! </p>
-            </item>
-          </list>
-          <p>Own Id: OTP-8183</p>
-	  <p>Aux Id: Seq 11393</p>
-        </item>
-
-      </list>
-
-    </section>
-
-    <section>
-      <title>Fixed bugs and malfunctions</title>
-<!--
-      <p>-</p>
--->
-
-      <list type="bulleted">
-        <item>
-          <p>If the megaco app received a transaction reply, for a request 
-            issued using the
-	    <seealso marker="megaco#call">call/3</seealso> function, from 
-            the wrong remote entity (wrong MId)), megaco would still deliver 
-            the reply (<seealso marker="megaco#call">call/3</seealso> 
-            returnes) as if from the correct remote entity (right MId). </p>
-          <p>This has been changed so that the function now returns with
-            an error reason. </p>
-          <p>See <seealso marker="megaco#call">call/3</seealso> for more 
-            info. </p>
-          <p>*** POTENTIAL INCOMPATIBILITY ***</p>
-          <p>Own Id: OTP-8212</p>
-	  <p>Aux Id: Seq 11305</p>
-        </item>
-
-      </list>
-
-    </section>
-
-  </section> <!-- 3.12 -->
-
-
-  <section>
-    <title>Megaco 3.11.3</title>
-
-<!--
-    <p>Version 3.11.3 supports code replacement in runtime from/to
-      version 3.11.2, 3.11.1 and 3.11.</p>
--->
-
-    <section>
-      <title>Improvements and new features</title>
-<!--
-      <p>-</p>
--->
-
-      <list type="bulleted">
-        <item>
-          <p>Replacing obsolete guard tests.</p>
-          <p>Own Id: OTP-8164</p>
-	  <!-- <p>Aux Id: Seq 11332</p> -->
-        </item>
-
-        <item>
-          <p>Added the config option 
-            <seealso marker="megaco#ui_call_proxy_gc_timeout">call_proxy_gc_timeout</seealso>
-            to be able to control the way unexpected replies (when requests issued 
-            via calls to <seealso marker="megaco#call">call/3</seealso>)
-            are handled. </p>
-          <p>See
-            <seealso marker="megaco#user_info">user_info/2</seealso>, 
-            <seealso marker="megaco#conn_info">conn_info/2</seealso> and
-            <seealso marker="megaco#call">call/3</seealso> for more info. </p>
-          <p>Own Id: OTP-8167</p>
-	  <p>Aux Id: Seq 11393</p>
-        </item>
-
-        <item>
-          <p>Make flex scanner c89 compiler compliant.</p>
-	  <p>Akira Kitada</p>
-          <p>Own Id: OTP-8191</p>
-	  <!-- <p>Aux Id: Seq 11332</p> -->
-        </item>
-
-      </list>
-
-    </section>
-
-    <section>
-      <title>Fixed bugs and malfunctions</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>Replacing obsolete guard tests.</p>
-          <p>Own Id: OTP-8164</p>
-	  <p>Aux Id: Seq 11332</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-
-    <section>
-      <title>Incompatibilities</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>For those implementing their own codec's, the new megaco_encoder
-            behaviour will require three more functions. See above for more
-            info. </p>
-          <p>Own Id: OTP-7168</p>
-	  <p>Aux Id: Seq 10867</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-  </section> <!-- 3.11.3 -->
-
-
-  <section>
-    <title>Megaco 3.11.2</title>
-
-    <p>Version 3.11.2 supports code replacement in runtime from/to
-      version 3.11.1 and 3.11.</p>
-
-    <section>
-      <title>Improvements and new features</title>
-<!--
-      <p>-</p>
--->
-
-      <list type="bulleted">
-        <item>
-          <p>Megaco was unnecessarily strict when parsing the SDP 
-            attribute <c>maxptime</c> (leading or trailing spaces
-            cased the value parse to fail). </p>
-          <p>This has been improved so that leading and trailing 
-            spaces are stripped before parsing the value. 
-            The same has been done for the attribute <c>ptime</c>.</p>
-          <p>Own Id: OTP-8123</p>
-	  <p>Aux Id: Seq 11364</p>
-        </item>
-
-      </list>
-
-    </section>
-
-    <section>
-      <title>Fixed bugs and malfunctions</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>[text] The <em>unquoted</em> string BOTH was interpreted as the 
-            <c>'BothToken'</c> token. This was a version 3 (prev3a, prev3b, 
-            prev3c and v3) only. </p>
-          <p>Own Id: OTP-8114</p>
-	  <p>Aux Id: Seq 11353</p>
-        </item>
-
-        <item>
-          <p>The reply proxy could crash if the timeout time calculation
-            results in a negative number. This will result in a function 
-            clause with resulting error report.</p>
-          <p>Own Id: OTP-8081</p>
-	  <p>Aux Id: Seq 11332</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-
-    <section>
-      <title>Incompatibilities</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>For those implementing their own codec's, the new megaco_encoder
-            behaviour will require three more functions. See above for more
-            info. </p>
-          <p>Own Id: OTP-7168</p>
-	  <p>Aux Id: Seq 10867</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-  </section> <!-- 3.11.2 -->
-
-
-  <section>
-    <title>Megaco 3.11.1</title>
-
-    <p>Version 3.11.1 supports code replacement in runtime from/to
-      version 3.11.</p>
-
-    <section>
-      <title>Improvements and new features</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>In order to better utilize multi-core procs, the 
-            <c>flex</c> (text) scanner has been improved. </p>
-          <p>The <c>flex</c> (text) scanner has been made reentrant,
-            <em>if</em> the flex utility supports this. Note that the version
-            of <c>flex</c> supplied with some OS/distros (Solaris 10, 
-            FreeBSD and OpenBSD to mention a few) may not support this, in which
-            case the flex scanner will be non-reentrant, just as before. </p>
-          <p>Own Id: OTP-7302</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-
-    <section>
-      <title>Fixed bugs and malfunctions</title>
-<!--
-      <p>-</p>
--->
-
-      <list type="bulleted">
-        <item>
-          <p>[text] The <em>unquoted</em> string BOTH was interpreted as the 
-            <c>'BothToken'</c> token. This was a version 3 (prev3a, prev3b, 
-            prev3c and v3) only. </p>
-          <p>Own Id: OTP-8114</p>
-	  <p>Aux Id: Seq 11353</p>
-        </item>
-
-        <item>
-          <p>The reply proxy could crash if the timeout time calculation
-            results in a negative number. This will result in a function 
-            clause with resulting error report.</p>
-          <p>Own Id: OTP-8081</p>
-	  <p>Aux Id: Seq 11332</p>
-        </item>
-
-      </list>
-
-    </section>
-
-    <section>
-      <title>Incompatibilities</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>For those implementing their own codec's, the new megaco_encoder
-            behaviour will require three more functions. See above for more
-            info. </p>
-          <p>Own Id: OTP-7168</p>
-	  <p>Aux Id: Seq 10867</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-  </section> <!-- 3.11.1 -->
-
-
-  <section>
-    <title>Megaco 3.11</title>
-
-<!--
-    <p>Version 3.11 supports code replacement in runtime from/to
-      version 3.10.1 and 3.10.0.1.</p>
--->
-
-    <section>
-      <title>Improvements and new features</title>
-<!--
-      <p>-</p>
--->
-
-      <list type="bulleted">
-        <item>
-          <p>In order to better utilize multi-core procs, the 
-            <c>flex</c> (text) scanner has been improved. </p>
-          <p>The <c>flex</c> (text) scanner has been made reentrant,
-            <em>if</em> the flex utility supports this. Note that the version
-            of <c>flex</c> supplied with some OS/distros (Solaris 10, 
-            FreeBSD and OpenBSD to mention a few) may not support this, in which
-            case the flex scanner will be non-reentrant, just as before. </p>
-          <p>Own Id: OTP-7302</p>
-        </item>
-
-      </list>
-    </section>
-
-    <section>
-      <title>Fixed bugs and malfunctions</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>The (plain) text scanner could incorrectly identify
-            character strings (any 17 char long string with the
-            char t in the middle) as a TimeStampToken.</p>
-          <p>Own Id: OTP-7249</p>
-	  <p>Aux Id: Seq 10917</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-
-    <section>
-      <title>Incompatibilities</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>For those implementing their own codec's, the new megaco_encoder
-            behaviour will require three more functions. See above for more
-            info. </p>
-          <p>Own Id: OTP-7168</p>
-	  <p>Aux Id: Seq 10867</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-  </section> <!-- 3.11 -->
-
-
-  <section>
-    <title>Megaco 3.10.1</title>
-
-    <p>Version 3.10.1 supports code replacement in runtime from/to
-      version 3.10.0.1, 3.10 and 3.9.4.</p>
-
-    <section>
-      <title>Improvements and new features</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>Updated file headers.</p>
-          <p>Own Id: OTP-7851</p>
-	  <p>Aux Id: Seq 11140</p>
-        </item>
-
-      </list>
--->
-    </section>
-
-    <section>
-      <title>Fixed bugs and malfunctions</title>
-<!--
-      <p>-</p>
--->
-
-      <list type="bulleted">
-        <item>
-          <p>Unexpected 
-            <seealso marker="megaco_user#unexpected_trans">handle_unexpected_reply</seealso> 
-            callbacks. </p>
-          <p>The <seealso marker="megaco_user">megaco_user</seealso> callback 
-            function 
-            <seealso marker="megaco_user#unexpected_trans">handle_unexpected_reply</seealso> 
-            could during high load be called with unexpected values for the Trans
-            argument, such as an <c>TransactionReply</c> where 
-            <c>transactionResult</c> had the value <c>{error, timeout}</c>. 
-            This was a result of a race condition and has now been fixed. </p>
-          <p>Own Id: OTP-7926</p>
-	  <p>Aux Id: Seq 11255</p>
-        </item>
-
-        <item>
-          <p>[text] PropertyParm values cannot be quoted. </p>
-          <p>It was not possible to encode a PropertyParm value as a quoted string 
-            (unless it *had* to (has at least one RestChar)). The megaco text codec's 
-            now also accepts quoted strings as PropertyParm values. </p>
-          <p>Own Id: OTP-7936</p>
-	  <p>Aux Id: Seq 11258</p>
-        </item>
-
-      </list>
-
-    </section>
-
-    <section>
-      <title>Incompatibilities</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>If the transport module calls the 
-            <seealso marker="megaco#process_received_message">process_received_message/5</seealso> 
-            or 
-            <seealso marker="megaco#receive_message">receive_message/5</seealso> 
-            function(s) for the initial message, then the 
-            <seealso marker="megaco_user#connect">handle_connect/3</seealso> 
-            function will now be called and not the
-            <seealso marker="megaco_user#connect">handle_connect/2</seealso> 
-            function. </p>
-          <p>Own Id: OTP-7713</p>
-	  <p>Aux Id: Seq 11140</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-  </section> <!-- 3.10.1 -->
-
-
-  <section>
-    <title>Megaco 3.10.0.1</title>
-
-    <p>Version 3.10.0.1 supports code replacement in runtime from/to
-      version 3.10 and 3.9.4 except 
-      when using any of the drivers (flex for text or asn1 for binary).</p>
-
-    <section>
-      <title>Improvements and new features</title>
-<!--
-      <p>-</p>
--->
-
-      <list type="bulleted">
-        <item>
-          <p>Updated file headers.</p>
-          <p>Own Id: OTP-7851</p>
-	  <!-- <p>Aux Id: Seq 11140</p> -->
-        </item>
-
-      </list>
-    </section>
-
-    <section>
-      <title>Fixed bugs and malfunctions</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>Memory leak in the flex scanner. There was a memory
-            leak in the flex scanner function handling 
-            Property Parameters. </p>
-          <p>Own Id: OTP-7700</p>
-	  <p>Aux Id: Seq 11126</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-
-    <section>
-      <title>Incompatibilities</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>If the transport module calls the 
-            <seealso marker="megaco#process_received_message">process_received_message/5</seealso> 
-            or 
-            <seealso marker="megaco#receive_message">receive_message/5</seealso> 
-            function(s) for the initial message, then the 
-            <seealso marker="megaco_user#connect">handle_connect/3</seealso> 
-            function will now be called and not the
-            <seealso marker="megaco_user#connect">handle_connect/2</seealso> 
-            function. </p>
-          <p>Own Id: OTP-7713</p>
-	  <p>Aux Id: Seq 11140</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-  </section> <!-- 3.10.0.1 -->
-
-
-  <section>
-    <title>Megaco 3.10</title>
-
-    <p>Version 3.10 supports code replacement in runtime from/to
-      version 3.9.4, 3.9.3, 3.9.2, 3.9.1.1, 3.9.1, 3.9, 3.8.2, 3.8.1 and 3.8 except 
-      when using any of the drivers (flex for text or asn1 for binary).</p>
-
-    <section>
-      <title>Improvements and new features</title>
-<!--
-      <p>-</p>
--->
-
-      <list type="bulleted">
-        <item>
-          <p>Added new API function 
-            <seealso marker="megaco#connect">megaco:connect/5</seealso> and
-            the corresponding new <c>megaco_user</c> callback function 
-            <seealso marker="megaco_user#connect">handle_connect/3</seealso>. 
-            The purpose of this is to be able to pass information to the
-            <seealso marker="megaco_user#connect">handle_connect/3</seealso> 
-            function by calling the 
-            <seealso marker="megaco#connect">megaco:connect/5</seealso> 
-            function. </p>
-          <p>Own Id: OTP-7713</p>
-	  <p>Aux Id: Seq 11140</p>
-        </item>
-
-        <item>
-          <p>Update file headers with new copyright notice. </p>
-          <p>Own Id: OTP-7743</p>
-        </item>
-
-      </list>
-    </section>
-
-    <section>
-      <title>Fixed bugs and malfunctions</title>
-      <p>-</p>
-
-<!--
-      <list type="bulleted">
-        <item>
-          <p>Memory leak in the flex scanner. There was a memory
-            leak in the flex scanner function handling 
-            Property Parameters. </p>
-          <p>Own Id: OTP-7700</p>
-	  <p>Aux Id: Seq 11126</p>
-        </item>
-
-      </list>
--->
-
-    </section>
-
-    <section>
-      <title>Incompatibilities</title>
-<!--
-      <p>-</p>
--->
-
-      <list type="bulleted">
-        <item>
-          <p>If the transport module calls the 
-            <seealso marker="megaco#process_received_message">process_received_message/5</seealso> 
-            or 
-            <seealso marker="megaco#receive_message">receive_message/5</seealso> 
-            function(s) for the initial message, then the 
-            <seealso marker="megaco_user#connect">handle_connect/3</seealso> 
-            function will now be called and not the
-            <seealso marker="megaco_user#connect">handle_connect/2</seealso> 
-            function. </p>
-          <p>Own Id: OTP-7713</p>
-	  <p>Aux Id: Seq 11140</p>
-        </item>
-
-      </list>
-
-    </section>
-  </section> <!-- 3.10 -->
-
   <!-- section>
     <title>Release notes history</title>
     <p>For information about older versions see