1 files changed, 263 insertions, 211 deletions

diff --git a/articles/erlang-validate-utf8/index.html b/articles/erlang-validate-utf8/index.html
index a96aba26..3eceb840 100644
--- a/articles/erlang-validate-utf8/index.html
+++ b/articles/erlang-validate-utf8/index.html
@@ -7,7 +7,7 @@
     <meta name="description" content="">
     <meta name="author" content="Loïc Hoguin based on a design from (Soft10) Pol Cámara">
 
-    <meta name="generator" content="Hugo 0.17" />
+    <meta name="generator" content="Hugo 0.26" />
 
     <title>Nine Nines: Validating UTF-8 binaries with Erlang</title>
 
@@ -74,191 +74,191 @@
 	</p>
 </header>
 
-<div class="paragraph"><p>Yesterday I pushed Websocket permessage-deflate to

-Cowboy master. I also pushed

-<a href="https://github.com/ninenines/cowlib/commit/7e4983b70ddf8cedb967e36fba6a600731bdad5d">a

-change in the way the code validates UTF-8 data</a>

-(required for text and close frames as per the spec).</p></div>

-<div class="paragraph"><p>When looking into why the permessage-deflate tests

-in autobahntestsuite were taking such a long time, I

-found that autobahn is using an adaptation of the

-algorithm named <a href="http://bjoern.hoehrmann.de/utf-8/decoder/dfa/">Flexible

-and Economical UTF-8 Decoder</a>. This is the C99

-implementation:</p></div>

-<div class="listingblock">

-<div class="content"><!-- Generator: GNU source-highlight 3.1.8

-by Lorenzo Bettini

-http://www.lorenzobettini.it

-http://www.gnu.org/software/src-highlite -->

-<pre><tt><span style="font-style: italic"><span style="color: #9A1900">// Copyright (c) 2008-2009 Bjoern Hoehrmann &lt;[email protected]&gt;</span></span>

-<span style="font-style: italic"><span style="color: #9A1900">// See http://bjoern.hoehrmann.de/utf-8/decoder/dfa/ for details.</span></span>

-

-<span style="font-weight: bold"><span style="color: #000080">#define</span></span> UTF8_ACCEPT <span style="color: #993399">0</span>

-<span style="font-weight: bold"><span style="color: #000080">#define</span></span> UTF8_REJECT <span style="color: #993399">1</span>

-

-<span style="font-weight: bold"><span style="color: #0000FF">static</span></span> <span style="font-weight: bold"><span style="color: #0000FF">const</span></span> <span style="color: #008080">uint8_t</span> utf8d<span style="color: #990000">[]</span> <span style="color: #990000">=</span> <span style="color: #FF0000">{</span>

-  <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 00..1f</span></span>

-  <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 20..3f</span></span>

-  <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 40..5f</span></span>

-  <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 60..7f</span></span>

-  <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 80..9f</span></span>

-  <span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// a0..bf</span></span>

-  <span style="color: #993399">8</span><span style="color: #990000">,</span><span style="color: #993399">8</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// c0..df</span></span>

-  <span style="color: #993399">0xa</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x4</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// e0..ef</span></span>

-  <span style="color: #993399">0xb</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x5</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// f0..ff</span></span>

-  <span style="color: #993399">0x0</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x2</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x5</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x7</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x4</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s0..s0</span></span>

-  <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s1..s2</span></span>

-  <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s3..s4</span></span>

-  <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s5..s6</span></span>

-  <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s7..s8</span></span>

-<span style="color: #FF0000">}</span><span style="color: #990000">;</span>

-

-uint32_t inline

-<span style="font-weight: bold"><span style="color: #000000">decode</span></span><span style="color: #990000">(</span>uint32_t<span style="color: #990000">*</span> state<span style="color: #990000">,</span> uint32_t<span style="color: #990000">*</span> codep<span style="color: #990000">,</span> <span style="color: #008080">uint32_t</span> byte<span style="color: #990000">)</span> <span style="color: #FF0000">{</span>

-  <span style="color: #008080">uint32_t</span> type <span style="color: #990000">=</span> utf8d<span style="color: #990000">[</span>byte<span style="color: #990000">];</span>

-

-  <span style="color: #990000">*</span>codep <span style="color: #990000">=</span> <span style="color: #990000">(*</span>state <span style="color: #990000">!=</span> UTF8_ACCEPT<span style="color: #990000">)</span> <span style="color: #990000">?</span>

-    <span style="color: #990000">(</span>byte <span style="color: #990000">&amp;</span> <span style="color: #993399">0x3fu</span><span style="color: #990000">)</span> <span style="color: #990000">|</span> <span style="color: #990000">(*</span>codep <span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">6</span><span style="color: #990000">)</span> <span style="color: #990000">:</span>

-    <span style="color: #990000">(</span><span style="color: #993399">0xff</span> <span style="color: #990000">&gt;&gt;</span> type<span style="color: #990000">)</span> <span style="color: #990000">&amp;</span> <span style="color: #990000">(</span>byte<span style="color: #990000">);</span>

-

-  <span style="color: #990000">*</span>state <span style="color: #990000">=</span> utf8d<span style="color: #990000">[</span><span style="color: #993399">256</span> <span style="color: #990000">+</span> <span style="color: #990000">*</span>state<span style="color: #990000">*</span><span style="color: #993399">16</span> <span style="color: #990000">+</span> type<span style="color: #990000">];</span>

-  <span style="font-weight: bold"><span style="color: #0000FF">return</span></span> <span style="color: #990000">*</span>state<span style="color: #990000">;</span>

-<span style="color: #FF0000">}</span></tt></pre></div></div>

-<div class="paragraph"><p>And this is the Erlang implementation I came up with:</p></div>

-<div class="listingblock">

-<div class="content"><!-- Generator: GNU source-highlight 3.1.8

-by Lorenzo Bettini

-http://www.lorenzobettini.it

-http://www.gnu.org/software/src-highlite -->

-<pre><tt><span style="font-style: italic"><span style="color: #9A1900">%% This function returns 0 on success, 1 on error, and 2..8 on incomplete data.</span></span>

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;&gt;&gt;</span>, <span style="color: #009900">State</span>) <span style="color: #990000">-&gt;</span> <span style="color: #009900">State</span>;

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">128</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">8</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">4</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">194</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">224</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">224</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">4</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">225</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">237</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">237</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">5</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">238</span>; <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">239</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">240</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">6</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">241</span>; <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">242</span>; <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">243</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">7</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">244</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">8</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">_</span>, <span style="color: #990000">_</span>) <span style="color: #990000">-&gt;</span> <span style="color: #993399">1</span><span style="color: #990000">.</span></tt></pre></div></div>

-<div class="paragraph"><p>Does it look similar to you? So how did we get there?</p></div>

-<div class="paragraph"><p>I started with a naive implementation of the original. First, we

-don&#8217;t need the codepoint calculated and extracted for our validation

-function. We just want to know the data is valid, so we only need to

-calculate the next state. Then, the only thing we needed to be careful

-about was that tuples are 1-based, and that we need to stop processing

-the binary when we get the state 1 or when the binary is empty.</p></div>

-<div class="listingblock">

-<div class="content"><!-- Generator: GNU source-highlight 3.1.8

-by Lorenzo Bettini

-http://www.lorenzobettini.it

-http://www.gnu.org/software/src-highlite -->

-<pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;&gt;&gt;</span>, <span style="color: #009900">State</span>) <span style="color: #990000">-&gt;</span> <span style="color: #009900">State</span>;

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">_</span>, <span style="color: #993399">1</span>) <span style="color: #990000">-&gt;</span> <span style="color: #993399">1</span>;

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #009900">State</span>) <span style="color: #990000">-&gt;</span>

-        <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">257</span> <span style="color: #990000">+</span> <span style="color: #009900">State</span> <span style="color: #990000">*</span> <span style="color: #993399">16</span> <span style="color: #990000">+</span> <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">1</span> <span style="color: #990000">+</span> <span style="color: #009900">C</span>, <span style="font-weight: bold"><span style="color: #000080">?UTF8D</span></span>), <span style="font-weight: bold"><span style="color: #000080">?UTF8D</span></span>))<span style="color: #990000">.</span></tt></pre></div></div>

-<div class="paragraph"><p>The macro <code>?UTF8D</code> is the tuple equivalent of the C array

-in the original code.</p></div>

-<div class="paragraph"><p>Compared to our previous algorithm, this performed about the same.

-In some situations a little faster, in some a little slower. In other words,

-not good enough. But because this new algorithm allows us to avoid a binary

-concatenation this warranted looking further.</p></div>

-<div class="paragraph"><p>It was time to step into crazy land.</p></div>

-<div class="paragraph"><p>Erlang is very good at pattern matching, even more so than doing some

-arithmetic coupled by fetching elements from a tuple. So I decided I was

-going to write all possible clauses for all combinations of <code>C</code>

-and <code>State</code>. And by write I mean generate.</p></div>

-<div class="paragraph"><p>So I opened my Erlang shell, defined the variable <code>D</code> to be

-the tuple <code>?UTF8D</code> with its 400 elements, and then ran the

-following expression (after a bit of trial and error):</p></div>

-<div class="listingblock">

-<div class="content"><!-- Generator: GNU source-highlight 3.1.8

-by Lorenzo Bettini

-http://www.lorenzobettini.it

-http://www.gnu.org/software/src-highlite -->

-<pre><tt><span style="color: #993399">16</span><span style="color: #990000">&gt;</span> <span style="font-weight: bold"><span style="color: #000000">file:write_file</span></span>(<span style="color: #FF0000">"out.txt"</span>,

-        [<span style="font-weight: bold"><span style="color: #000000">io_lib:format</span></span>(<span style="color: #FF0000">"validate_utf8(&lt;&lt; ~p, Rest/bits &gt;&gt;, ~p) -&gt; ~p;~n"</span>,

-                [<span style="color: #009900">C</span>, <span style="color: #009900">S</span>, <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">257</span> <span style="color: #990000">+</span> <span style="color: #009900">S</span> <span style="color: #990000">*</span> <span style="color: #993399">16</span> <span style="color: #990000">+</span> <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">1</span> <span style="color: #990000">+</span> <span style="color: #009900">C</span>, <span style="color: #009900">D</span>), <span style="color: #009900">D</span>)])

-                        || <span style="color: #009900">C</span> <span style="color: #990000">&lt;-</span> <span style="font-weight: bold"><span style="color: #000000">lists:seq</span></span>(<span style="color: #993399">0</span>,<span style="color: #993399">255</span>), <span style="color: #009900">S</span> <span style="color: #990000">&lt;-</span> <span style="font-weight: bold"><span style="color: #000000">lists:seq</span></span>(<span style="color: #993399">0</span>,<span style="color: #993399">8</span>)])<span style="color: #990000">.</span>

-<span style="color: #FF6600">ok</span></tt></pre></div></div>

-<div class="paragraph"><p>The result is a 2304 lines long file, containing 2304 clauses.

-People who pay attention to what I say on Twitter will remember

-I said something around 3000 clauses, but that was just me not

-using the right number of states in my estimate.</p></div>

-<div class="paragraph"><p>There was a little more work to be done on this generated

-code that I did using regular expressions. We need to recurse

-when the resulting state is not 1. We also need to stop when

-the binary is empty, making it the 2305th clause.</p></div>

-<div class="paragraph"><p>Still, 2305 is a lot. But hey, the code did work, and faster

-than the previous implementation too! But hey, perhaps I could

-find a way to reduce its size.</p></div>

-<div class="paragraph"><p>Removing all the clauses that return 1 and putting a catch-all

-clause at the end instead reduced the number to about 500, and

-showed that many clauses were similar:</p></div>

-<div class="listingblock">

-<div class="content"><!-- Generator: GNU source-highlight 3.1.8

-by Lorenzo Bettini

-http://www.lorenzobettini.it

-http://www.gnu.org/software/src-highlite -->

-<pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">0</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">1</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">2</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">3</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">4</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">5</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">6</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">7</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);</tt></pre></div></div>

-<div class="paragraph"><p>But also:</p></div>

-<div class="listingblock">

-<div class="content"><!-- Generator: GNU source-highlight 3.1.8

-by Lorenzo Bettini

-http://www.lorenzobettini.it

-http://www.gnu.org/software/src-highlite -->

-<pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);</tt></pre></div></div>

-<div class="paragraph"><p>Patterns, my favorites!</p></div>

-<div class="paragraph"><p>A little more time was spent to edit the 500 or so clauses into

-smaller equivalents, testing that performance was not impacted, and

-comitting the result.</p></div>

-<div class="paragraph"><p>The patterns above can be found here in the resulting function:</p></div>

-<div class="listingblock">

-<div class="content"><!-- Generator: GNU source-highlight 3.1.8

-by Lorenzo Bettini

-http://www.lorenzobettini.it

-http://www.gnu.org/software/src-highlite -->

-<pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">128</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="color: #990000">...</span>

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);

-<span style="color: #990000">...</span></tt></pre></div></div>

-<div class="paragraph"><p>I hope you enjoyed this post.</p></div>

+<div class="paragraph"><p>Yesterday I pushed Websocket permessage-deflate to
+Cowboy master. I also pushed
+<a href="https://github.com/ninenines/cowlib/commit/7e4983b70ddf8cedb967e36fba6a600731bdad5d">a
+change in the way the code validates UTF-8 data</a>
+(required for text and close frames as per the spec).</p></div>
+<div class="paragraph"><p>When looking into why the permessage-deflate tests
+in autobahntestsuite were taking such a long time, I
+found that autobahn is using an adaptation of the
+algorithm named <a href="http://bjoern.hoehrmann.de/utf-8/decoder/dfa/">Flexible
+and Economical UTF-8 Decoder</a>. This is the C99
+implementation:</p></div>
+<div class="listingblock">
+<div class="content"><!-- Generator: GNU source-highlight 3.1.8
+by Lorenzo Bettini
+http://www.lorenzobettini.it
+http://www.gnu.org/software/src-highlite -->
+<pre><tt><span style="font-style: italic"><span style="color: #9A1900">// Copyright (c) 2008-2009 Bjoern Hoehrmann &lt;[email protected]&gt;</span></span>
+<span style="font-style: italic"><span style="color: #9A1900">// See http://bjoern.hoehrmann.de/utf-8/decoder/dfa/ for details.</span></span>
+
+<span style="font-weight: bold"><span style="color: #000080">#define</span></span> UTF8_ACCEPT <span style="color: #993399">0</span>
+<span style="font-weight: bold"><span style="color: #000080">#define</span></span> UTF8_REJECT <span style="color: #993399">1</span>
+
+<span style="font-weight: bold"><span style="color: #0000FF">static</span></span> <span style="font-weight: bold"><span style="color: #0000FF">const</span></span> <span style="color: #008080">uint8_t</span> utf8d<span style="color: #990000">[]</span> <span style="color: #990000">=</span> <span style="color: #FF0000">{</span>
+  <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 00..1f</span></span>
+  <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 20..3f</span></span>
+  <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 40..5f</span></span>
+  <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 60..7f</span></span>
+  <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 80..9f</span></span>
+  <span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// a0..bf</span></span>
+  <span style="color: #993399">8</span><span style="color: #990000">,</span><span style="color: #993399">8</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// c0..df</span></span>
+  <span style="color: #993399">0xa</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x4</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// e0..ef</span></span>
+  <span style="color: #993399">0xb</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x5</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// f0..ff</span></span>
+  <span style="color: #993399">0x0</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x2</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x5</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x7</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x4</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s0..s0</span></span>
+  <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s1..s2</span></span>
+  <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s3..s4</span></span>
+  <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s5..s6</span></span>
+  <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s7..s8</span></span>
+<span style="color: #FF0000">}</span><span style="color: #990000">;</span>
+
+uint32_t inline
+<span style="font-weight: bold"><span style="color: #000000">decode</span></span><span style="color: #990000">(</span>uint32_t<span style="color: #990000">*</span> state<span style="color: #990000">,</span> uint32_t<span style="color: #990000">*</span> codep<span style="color: #990000">,</span> <span style="color: #008080">uint32_t</span> byte<span style="color: #990000">)</span> <span style="color: #FF0000">{</span>
+  <span style="color: #008080">uint32_t</span> type <span style="color: #990000">=</span> utf8d<span style="color: #990000">[</span>byte<span style="color: #990000">];</span>
+
+  <span style="color: #990000">*</span>codep <span style="color: #990000">=</span> <span style="color: #990000">(*</span>state <span style="color: #990000">!=</span> UTF8_ACCEPT<span style="color: #990000">)</span> <span style="color: #990000">?</span>
+    <span style="color: #990000">(</span>byte <span style="color: #990000">&amp;</span> <span style="color: #993399">0x3fu</span><span style="color: #990000">)</span> <span style="color: #990000">|</span> <span style="color: #990000">(*</span>codep <span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">6</span><span style="color: #990000">)</span> <span style="color: #990000">:</span>
+    <span style="color: #990000">(</span><span style="color: #993399">0xff</span> <span style="color: #990000">&gt;&gt;</span> type<span style="color: #990000">)</span> <span style="color: #990000">&amp;</span> <span style="color: #990000">(</span>byte<span style="color: #990000">);</span>
+
+  <span style="color: #990000">*</span>state <span style="color: #990000">=</span> utf8d<span style="color: #990000">[</span><span style="color: #993399">256</span> <span style="color: #990000">+</span> <span style="color: #990000">*</span>state<span style="color: #990000">*</span><span style="color: #993399">16</span> <span style="color: #990000">+</span> type<span style="color: #990000">];</span>
+  <span style="font-weight: bold"><span style="color: #0000FF">return</span></span> <span style="color: #990000">*</span>state<span style="color: #990000">;</span>
+<span style="color: #FF0000">}</span></tt></pre></div></div>
+<div class="paragraph"><p>And this is the Erlang implementation I came up with:</p></div>
+<div class="listingblock">
+<div class="content"><!-- Generator: GNU source-highlight 3.1.8
+by Lorenzo Bettini
+http://www.lorenzobettini.it
+http://www.gnu.org/software/src-highlite -->
+<pre><tt><span style="font-style: italic"><span style="color: #9A1900">%% This function returns 0 on success, 1 on error, and 2..8 on incomplete data.</span></span>
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;&gt;&gt;</span>, <span style="color: #009900">State</span>) <span style="color: #990000">-&gt;</span> <span style="color: #009900">State</span>;
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">128</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">8</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">4</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">194</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">224</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">224</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">4</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">225</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">237</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">237</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">5</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">238</span>; <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">239</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">240</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">6</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">241</span>; <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">242</span>; <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">243</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">7</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">244</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">8</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">_</span>, <span style="color: #990000">_</span>) <span style="color: #990000">-&gt;</span> <span style="color: #993399">1</span><span style="color: #990000">.</span></tt></pre></div></div>
+<div class="paragraph"><p>Does it look similar to you? So how did we get there?</p></div>
+<div class="paragraph"><p>I started with a naive implementation of the original. First, we
+don&#8217;t need the codepoint calculated and extracted for our validation
+function. We just want to know the data is valid, so we only need to
+calculate the next state. Then, the only thing we needed to be careful
+about was that tuples are 1-based, and that we need to stop processing
+the binary when we get the state 1 or when the binary is empty.</p></div>
+<div class="listingblock">
+<div class="content"><!-- Generator: GNU source-highlight 3.1.8
+by Lorenzo Bettini
+http://www.lorenzobettini.it
+http://www.gnu.org/software/src-highlite -->
+<pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;&gt;&gt;</span>, <span style="color: #009900">State</span>) <span style="color: #990000">-&gt;</span> <span style="color: #009900">State</span>;
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">_</span>, <span style="color: #993399">1</span>) <span style="color: #990000">-&gt;</span> <span style="color: #993399">1</span>;
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #009900">State</span>) <span style="color: #990000">-&gt;</span>
+        <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">257</span> <span style="color: #990000">+</span> <span style="color: #009900">State</span> <span style="color: #990000">*</span> <span style="color: #993399">16</span> <span style="color: #990000">+</span> <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">1</span> <span style="color: #990000">+</span> <span style="color: #009900">C</span>, <span style="font-weight: bold"><span style="color: #000080">?UTF8D</span></span>), <span style="font-weight: bold"><span style="color: #000080">?UTF8D</span></span>))<span style="color: #990000">.</span></tt></pre></div></div>
+<div class="paragraph"><p>The macro <code>?UTF8D</code> is the tuple equivalent of the C array
+in the original code.</p></div>
+<div class="paragraph"><p>Compared to our previous algorithm, this performed about the same.
+In some situations a little faster, in some a little slower. In other words,
+not good enough. But because this new algorithm allows us to avoid a binary
+concatenation this warranted looking further.</p></div>
+<div class="paragraph"><p>It was time to step into crazy land.</p></div>
+<div class="paragraph"><p>Erlang is very good at pattern matching, even more so than doing some
+arithmetic coupled by fetching elements from a tuple. So I decided I was
+going to write all possible clauses for all combinations of <code>C</code>
+and <code>State</code>. And by write I mean generate.</p></div>
+<div class="paragraph"><p>So I opened my Erlang shell, defined the variable <code>D</code> to be
+the tuple <code>?UTF8D</code> with its 400 elements, and then ran the
+following expression (after a bit of trial and error):</p></div>
+<div class="listingblock">
+<div class="content"><!-- Generator: GNU source-highlight 3.1.8
+by Lorenzo Bettini
+http://www.lorenzobettini.it
+http://www.gnu.org/software/src-highlite -->
+<pre><tt><span style="color: #993399">16</span><span style="color: #990000">&gt;</span> <span style="font-weight: bold"><span style="color: #000000">file:write_file</span></span>(<span style="color: #FF0000">"out.txt"</span>,
+        [<span style="font-weight: bold"><span style="color: #000000">io_lib:format</span></span>(<span style="color: #FF0000">"validate_utf8(&lt;&lt; ~p, Rest/bits &gt;&gt;, ~p) -&gt; ~p;~n"</span>,
+                [<span style="color: #009900">C</span>, <span style="color: #009900">S</span>, <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">257</span> <span style="color: #990000">+</span> <span style="color: #009900">S</span> <span style="color: #990000">*</span> <span style="color: #993399">16</span> <span style="color: #990000">+</span> <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">1</span> <span style="color: #990000">+</span> <span style="color: #009900">C</span>, <span style="color: #009900">D</span>), <span style="color: #009900">D</span>)])
+                        || <span style="color: #009900">C</span> <span style="color: #990000">&lt;-</span> <span style="font-weight: bold"><span style="color: #000000">lists:seq</span></span>(<span style="color: #993399">0</span>,<span style="color: #993399">255</span>), <span style="color: #009900">S</span> <span style="color: #990000">&lt;-</span> <span style="font-weight: bold"><span style="color: #000000">lists:seq</span></span>(<span style="color: #993399">0</span>,<span style="color: #993399">8</span>)])<span style="color: #990000">.</span>
+<span style="color: #FF6600">ok</span></tt></pre></div></div>
+<div class="paragraph"><p>The result is a 2304 lines long file, containing 2304 clauses.
+People who pay attention to what I say on Twitter will remember
+I said something around 3000 clauses, but that was just me not
+using the right number of states in my estimate.</p></div>
+<div class="paragraph"><p>There was a little more work to be done on this generated
+code that I did using regular expressions. We need to recurse
+when the resulting state is not 1. We also need to stop when
+the binary is empty, making it the 2305th clause.</p></div>
+<div class="paragraph"><p>Still, 2305 is a lot. But hey, the code did work, and faster
+than the previous implementation too! But hey, perhaps I could
+find a way to reduce its size.</p></div>
+<div class="paragraph"><p>Removing all the clauses that return 1 and putting a catch-all
+clause at the end instead reduced the number to about 500, and
+showed that many clauses were similar:</p></div>
+<div class="listingblock">
+<div class="content"><!-- Generator: GNU source-highlight 3.1.8
+by Lorenzo Bettini
+http://www.lorenzobettini.it
+http://www.gnu.org/software/src-highlite -->
+<pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">0</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">1</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">2</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">3</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">4</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">5</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">6</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">7</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);</tt></pre></div></div>
+<div class="paragraph"><p>But also:</p></div>
+<div class="listingblock">
+<div class="content"><!-- Generator: GNU source-highlight 3.1.8
+by Lorenzo Bettini
+http://www.lorenzobettini.it
+http://www.gnu.org/software/src-highlite -->
+<pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);</tt></pre></div></div>
+<div class="paragraph"><p>Patterns, my favorites!</p></div>
+<div class="paragraph"><p>A little more time was spent to edit the 500 or so clauses into
+smaller equivalents, testing that performance was not impacted, and
+comitting the result.</p></div>
+<div class="paragraph"><p>The patterns above can be found here in the resulting function:</p></div>
+<div class="listingblock">
+<div class="content"><!-- Generator: GNU source-highlight 3.1.8
+by Lorenzo Bettini
+http://www.lorenzobettini.it
+http://www.gnu.org/software/src-highlite -->
+<pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">128</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="color: #990000">...</span>
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);
+<span style="color: #990000">...</span></tt></pre></div></div>
+<div class="paragraph"><p>I hope you enjoyed this post.</p></div>
 
 </article>
 </div>
@@ -267,55 +267,107 @@ http://www.gnu.org/software/src-highlite -->
 <h3>More articles</h3>
 <ul id="articles-nav" class="extra_margin">
 	
-		<li><a href="https://ninenines.eu/articles/cowboy-2.0.0-rc.2/">Cowboy 2.0 release candidate 2</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/cowboy-2.0.0-rc.2/">Cowboy 2.0 release candidate 2</a></li>
+		
+	
+		
+			<li><a href="https://ninenines.eu/articles/cowboy-2.0.0-rc.1/">Cowboy 2.0 release candidate 1</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/cowboy-2.0.0-rc.1/">Cowboy 2.0 release candidate 1</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/the-elephant-in-the-room/">The elephant in the room</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/the-elephant-in-the-room/">The elephant in the room</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/dont-let-it-crash/">Don&#39;t let it crash</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/dont-let-it-crash/">Don&#39;t let it crash</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/cowboy-2.0.0-pre.4/">Cowboy 2.0 pre-release 4</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/cowboy-2.0.0-pre.4/">Cowboy 2.0 pre-release 4</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/ranch-1.3/">Ranch 1.3</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/ranch-1.3/">Ranch 1.3</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/ml-archives/">Mailing list archived</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/ml-archives/">Mailing list archived</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/website-update/">Website update</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/website-update/">Website update</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/erlanger-playbook-september-2015-update/">The Erlanger Playbook September 2015 Update</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/erlanger-playbook-september-2015-update/">The Erlanger Playbook September 2015 Update</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/erlanger-playbook/">The Erlanger Playbook</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/erlanger-playbook/">The Erlanger Playbook</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/erlang-validate-utf8/">Validating UTF-8 binaries with Erlang</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/erlang-validate-utf8/">Validating UTF-8 binaries with Erlang</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/on-open-source/">On open source</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/on-open-source/">On open source</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/the-story-so-far/">The story so far</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/the-story-so-far/">The story so far</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/cowboy2-qs/">Cowboy 2.0 and query strings</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/cowboy2-qs/">Cowboy 2.0 and query strings</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/january-2014-status/">January 2014 status</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/january-2014-status/">January 2014 status</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/farwest-funded/">Farwest got funded!</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/farwest-funded/">Farwest got funded!</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/erlang.mk-and-relx/">Build Erlang releases with Erlang.mk and Relx</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/erlang.mk-and-relx/">Build Erlang releases with Erlang.mk and Relx</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/xerl-0.5-intermediate-module/">Xerl: intermediate module</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/xerl-0.5-intermediate-module/">Xerl: intermediate module</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/xerl-0.4-expression-separator/">Xerl: expression separator</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/xerl-0.4-expression-separator/">Xerl: expression separator</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/erlang-scalability/">Erlang Scalability</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/erlang-scalability/">Erlang Scalability</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/xerl-0.3-atomic-expressions/">Xerl: atomic expressions</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/xerl-0.3-atomic-expressions/">Xerl: atomic expressions</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/xerl-0.2-two-modules/">Xerl: two modules</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/xerl-0.2-two-modules/">Xerl: two modules</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/xerl-0.1-empty-modules/">Xerl: empty modules</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/xerl-0.1-empty-modules/">Xerl: empty modules</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/ranch-ftp/">Build an FTP Server with Ranch in 30 Minutes</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/ranch-ftp/">Build an FTP Server with Ranch in 30 Minutes</a></li>
+		
+			<li><a href="https://ninenines.eu/articles/tictactoe/">Erlang Tic Tac Toe</a></li>
+		
 	
-		<li><a href="https://ninenines.eu/articles/tictactoe/">Erlang Tic Tac Toe</a></li>
+		
 	
 </ul>