Nine Nines https://ninenines.eu/index.xml Recent content on Nine Nines Hugo -- gohugo.io en-us Sun, 26 Mar 2017 00:00:00 +0100 The elephant in the room https://ninenines.eu/articles/the-elephant-in-the-room/ Sun, 26 Mar 2017 00:00:00 +0100 https://ninenines.eu/articles/the-elephant-in-the-room/ <div class="paragraph"><p>Have you ever tried telling someone why they should use Erlang? You boast the smaller code size, the auto healing mechanisms, the distribution and they seem really excited. They wonder why they never heard about Erlang before. And then you show them what the code looks like. All excitement goes away. The smiles disappear. Their face starts becoming really serious.</p></div> <div class="paragraph"><p>You lost them. You know you lost them. They comment on the syntax, or perhaps you do, already admitting defeat. It&#8217;s unlike anything they have ever used before. And they will most likely end up not using it.</p></div> <div class="paragraph"><p>What about people who already know what the syntax looks like? As soon as you mention Erlang, the topic of the syntax comes in. It&#8217;s like nothing else matters.</p></div> <div class="paragraph"><p>Perhaps the topic of syntax didn&#8217;t come up. But they&#8217;re still not going to try Erlang because of it.</p></div> <div class="paragraph"><p>You&#8217;re probably not having these kinds of interactions at Erlang conferences. This doesn&#8217;t happen with people who are already somewhat interested in, or need, the features that Erlang provides. With them the syntax is at worst a minor inconvenience.</p></div> <div class="paragraph"><p>This happens because most developers are familiar with syntaxes that look nothing like Erlang. To be clear, I include language features and other concepts like objects as part of "syntax" here. Familiarity is a very important factor to drive adoption.</p></div> <div class="paragraph"><p>You can see an example of that in the Elixir world, where the majority of people come from Ruby or already knew and liked Ruby. The 2016 survey tells us that 59% of Elixir developers were using Ruby primarily before. That&#8217;s in large part because of the syntax. They will deny it of course and find other reasons. And yet, we don&#8217;t see such a strong adoption of Erlang from Ruby developers, before or after Elixir appeared.</p></div> <div class="paragraph"><p>Side note: have you ever wondered why the Elixir community is, I quote, much friendlier than the Ruby community? Despite having much of the same people?</p></div> <div class="paragraph"><p>Before we continue, let me be clear. I love the Erlang syntax. It is simple and explicit. It is powerful, especially when dealing with binary data. It has very few quirks. It has little to no ambiguity. It&#8217;s great. Except for persuading people to use it.</p></div> <div class="paragraph"><p>Over the years I have been writing Erlang, I have seen very few people point out that the syntax slows down adoption. We have no problem with it, so why would others? At the same time, people coming to Erlang come to solve a real problem they&#8217;re having, so the syntax is fairly secondary. Even if they hate it at first, they know they can solve their problems despite the syntax.</p></div> <div class="paragraph"><p>You don&#8217;t build a popular product or language by solving people&#8217;s problems though. In general you end up solving some problems and creating new problems. No, you build a popular product by <strong>convincing people to use it</strong>. And you make them stay with your product by making them <strong>commit</strong> to using it.</p></div> <div class="paragraph"><p>Take MongoDB for example. It didn&#8217;t become popular by working, or even by being practical. It wasn&#8217;t performing its primary function and was losing people&#8217;s data. That didn&#8217;t stop it from becoming popular. Smart people would knowingly use a database that was losing data. Think about that for a minute.</p></div> <div class="paragraph"><p>MongoDB of course had a huge marketing machine, and they focused on that. They helped organize many meetups all over the world, complete with various swag items given for free, including a small handbook about MongoDB. All people had to do was show up.</p></div> <div class="paragraph"><p>They didn&#8217;t go tell people to look at all the weaknesses their product had. They focused on the strengths. On what would convince people to try it. They would go to meetups, discuss with others, commit to try it (or try it at meetups directly), and by doing so sell MongoDB to themselves.</p></div> <div class="paragraph"><p>How do we get people to meetups though? That&#8217;d be the first step: you need to <strong>catch their attention</strong>. I believe MongoDB did this using benchmark results. Ironic isn&#8217;t it? MongoDB gets fast benchmark results because they lose data, and this gets everyone to buy into the product.</p></div> <div class="paragraph"><p>The key points to remember about this are:</p></div> <div class="ulist"><ul> <li> <p> catch people&#8217;s attention </p> </li> <li> <p> show your product&#8217;s strengths </p> </li> <li> <p> make people take a commitment </p> </li> </ul></div> <div class="paragraph"><p>Once they commit to something, you win. Everyone will not end up ultimately using your product of course, but it&#8217;s at the very least become a consideration. It&#8217;s on their mind. Their resolve will be stronger when they ultimately try it and inevitably run into issues.</p></div> <div class="paragraph"><p>Erlang&#8217;s syntax is a weakness. Almost nobody looks at the Erlang syntax and falls in love with it at first sight. No, it takes time to learn it and understand how good it is. You need to sell Erlang to people without showing the Erlang syntax. If you do show it, then you need to hide the parts that feel alien. Function calls are OK. Recursion, not so much. Maps are OK. Records, not.</p></div> <div class="paragraph"><p>Avoiding code is not always possible when you try to sell it, especially to developers. You can however prepare them to accept the alien syntax by admitting that the syntax is not perfect before you show it. You can do this while praising it at the same time. For example, "the syntax is a little out there, but it matches the concepts perfectly, it will all make sense when you start learning".</p></div> <div class="paragraph"><p>This might not be the best introduction. Someone will need to A/B test it to find the one that gives the best results. But that should give you ideas.</p></div> <div class="paragraph"><p>When something terrible happens, mentioning that this isn&#8217;t the end of the world <strong>before</strong> you tell others what happened will soften their reaction. When someone breaks your favorite item and cries over it calling themselves stupid, it&#8217;s harder to get mad at them, compared to the same event with no emotional reaction.</p></div> <div class="paragraph"><p>Our behavior is largely dependent on what&#8217;s at the top of our mind, so it&#8217;s up to you to take advantage of this to make your case in the best conditions.</p></div> <div class="paragraph"><p>Next time you try to make someone use Erlang, remember that you should aim for getting a spoken commitment out of them, if possible before you show the syntax. If that&#8217;s not possible, then prepare them to accept the flaws or the weirdness before they see them.</p></div> Don't let it crash https://ninenines.eu/articles/dont-let-it-crash/ Sun, 22 Jan 2017 00:00:00 +0100 https://ninenines.eu/articles/dont-let-it-crash/ <div class="paragraph"><p>We have a specific mindset when writing Erlang programs. We focus on the normal execution of the program and don&#8217;t handle most of the errors that may occur. We sometimes call this normal execution the <em>happy path</em>.</p></div> <div class="paragraph"><p>The general pattern behind writing only for the <em>happy path</em>, letting the VM catch errors (writing them to a log for future consumption) and then having a supervisor restart the processes that failed from a clean state, has a name. We call it <em>let it crash</em>; and it drives many of our design decisions.</p></div> <div class="paragraph"><p>It&#8217;s a really great way to program and the results are fantastic compared to most other programming languages. And yet, <em>let it crash</em> barely convinced anyone that they should use Erlang. Why would that be?</p></div> <div class="paragraph"><p>You may already know that Cowboy is capable of handling at least 2 million Websocket connections on a single server. This is in large part thanks to the capabilities of the VM. Still, 2 million is good, much better than most other servers can do.</p></div> <div class="paragraph"><p>Cowboy is not just a Websocket server; it&#8217;s also an HTTP and HTTP/2 server, and it handles many related features like long polling or the parsing of most request headers.</p></div> <div class="paragraph"><p>Can you guess how large the Cowboy codebase is, without looking at the source?</p></div> <div class="paragraph"><p>Do make sure you have a clear answer in your mind before you go check.</p></div> <div class="paragraph"><p>Good, you are back. Now what were the results? If I am correct, you overestimated the size of Cowboy. Cowboy is in fact about five thousand lines of code. You probably thought it was at least ten thousand. About eighty percent of readers will have overestimated the size of Cowboy. And you did only because I mentioned it can handle millions of Websocket connections.</p></div> <div class="paragraph"><p>Numerous studies show this effect. Just mentioning the large number already prepared your mind to think in that direction. Repeating the number made you focus even more on it. Then the question asked for a number, which ended up larger than the reality.</p></div> <div class="paragraph"><p>The same effect can be applied to negotiation for example. You generally want to start by giving your offer (and not let the other party initiate) and you want to give a really large number first. You can also prepare your customer by mentioning an even larger number in the previous discussion.</p></div> <div class="paragraph"><p>And it&#8217;s not just numbers either. An experiment showed that just by looking at an image of clouds, customers of a pillow store were buying pillows more comfortable (and more expensive) than those who didn&#8217;t see that image.</p></div> <div class="paragraph"><p>This is the power of associations. It is covered in much larger detail in the books <a href="https://www.amazon.com/Influence-Psychology-Persuasion-Robert-Cialdini/dp/006124189X">Influence</a> and <a href="https://www.amazon.com/Pre-Suasion-Revolutionary-Way-Influence-Persuade/dp/1501109790">Pre-suasion</a>. I highly recommend reading those and applying what you learn to your daily life. I&#8217;m definitely not a professional psychologist so take this post with a grain of salt.</p></div> <div class="paragraph"><p>When selling Erlang, whether we are selling it to a customer or trying to convince a developer friend to start using it, we often talk about how Erlang <em>lets you sleep at night</em>, that it is auto healing and always gets fantastic uptimes.</p></div> <div class="paragraph"><p>And then we talk about <em>let it crash</em>.</p></div> <div class="paragraph"><p>And we describe what it means.</p></div> <div class="paragraph"><p>We might as well just say that Erlang crashes a lot and then take the door. It would have the same effect. It doesn&#8217;t even stop at programs crashing. You know what else crashes? Cars, planes, trains. Often with disastrous consequences. Is that really the message we want to convey?</p></div> <div class="paragraph"><p>They even <a href="https://img.youtube.com/vi/oEUBW2lCkIk/0.jpg">printed it on a t-shirt</a>! Keep calm and let it crash. It&#8217;s the kind of t-shirt you probably shouldn&#8217;t wear in an airport, and for good reasons. A few people did, then realized what they were wearing and were not too smug about it.</p></div> <div class="paragraph"><p>And yet this is how we sell Erlang.</p></div> <div class="paragraph"><p>A better way would be to focus on the positives, of course, but also to make sure that those positives are phrased in a way that prevents bad associations to be formed in people&#8217;s minds.</p></div> <div class="paragraph"><p>Instead of <em>let it crash</em>, you can say that Erlang has <em>auto healing mechanisms</em>. Healing is a good thing and accurately describes what happens in the system.</p></div> <div class="paragraph"><p>Should you need to go into more details, you will probably want to avoid <em>recover from crashes</em> and instead say <em>recover from exceptions</em>. Exceptions are a pretty neutral word and, should you explain what you mean by that, you can talk about exceptions that occur for reasons unrelated to Erlang, like hardware failure or network instability.</p></div> <div class="paragraph"><p>The trick is to always use positive words and phrases to describe Erlang, and to use external factors to explain how Erlang deals with failures. Never mention the failures internal to Erlang systems unless you are asked specifically, in which case you can say that the auto healing applies to all exceptions.</p></div> <div class="paragraph"><p>The <em>let it crash</em> philosophy is great when learning Erlang or when writing fault-tolerant systems. But it&#8217;s not going to convince anyone to use it unless they were already looking for it.</p></div> <div class="paragraph"><p>Do you like this post? Tell me on Twitter. I might make more.</p></div> Cowboy 2.0 pre-release 4 https://ninenines.eu/articles/cowboy-2.0.0-pre.4/ Tue, 03 Jan 2017 00:00:00 +0100 https://ninenines.eu/articles/cowboy-2.0.0-pre.4/ <div class="paragraph"><p>Cowboy <code>2.0.0-pre.4</code> has been released!</p></div> <div class="paragraph"><p>This is the new recommended version of Cowboy. While I would not recommend putting it in production just yet, I do recommend you start writing new applications with this Cowboy version.</p></div> <div class="paragraph"><p>The most significant changes in the pre-release are:</p></div> <div class="ulist"><ul> <li> <p> A new architecture: there now is one process per connection and one process per request. This was done because HTTP/2 allows running requests concurrently. </p> </li> <li> <p> Stream handlers. Every request, response and data goes through stream handlers. They are meant to replace hooks and more. They will be documented in a future pre-release. Check <code>cowboy_stream</code> and <code>cowboy_stream_h</code> if interested. </p> </li> <li> <p> Numerous changes to the <code>cowboy_req</code> interface. This is very close to final. Check the manual for what changed. </p> </li> <li> <p> The Req object is no longer passed in Websocket callbacks. </p> </li> <li> <p> It is now possible to send frames directly from <code>websocket_init/1</code>. </p> </li> <li> <p> SPDY support was removed, now that we have HTTP/2. </p> </li> <li> <p> Update Ranch to 1.3. We still depend on Cowlib master for the time being. </p> </li> <li> <p> A much improved manual. </p> </li> </ul></div> <div class="paragraph"><p>The manual received a lot of love. It now has one page per function with a detailed description, arguments list, return value, changelog and examples. It also links to the other relevant manual pages: <a href="https://ninenines.eu/docs/en/cowboy/2.0/manual/">https://ninenines.eu/docs/en/cowboy/2.0/manual/</a></p></div> <div class="paragraph"><p>I am quite proud of the manual right now. While more improvements can be made, what we have now is way better than before. Feedback for further improvements is welcome!</p></div> <div class="paragraph"><p>This is a significant step toward Cowboy 2.0. Almost all the breaking changes are in. A few more pre-releases are planned and will be released on a weekly basis (with exceptions).</p></div> <div class="paragraph"><p>Cowboy is now tested and supported with Erlang/OTP 18.0 or above on Arch Linux, FreeBSD, OSX, Ubuntu and Windows 7. Contact me if you can provide permanent access to another platform for the purposes of testing.</p></div> <div class="paragraph"><p>Cowboy is now available from four locations:</p></div> <div class="ulist"><ul> <li> <p> <a href="https://git.ninenines.eu/cowboy.git">https://git.ninenines.eu/cowboy.git</a> </p> </li> <li> <p> <a href="https://github.com/ninenines/cowboy.git">https://github.com/ninenines/cowboy.git</a> </p> </li> <li> <p> <a href="https://bitbucket.org/ninenines/cowboy.git">https://bitbucket.org/ninenines/cowboy.git</a> </p> </li> <li> <p> <a href="https://gitlab.com/ninenines/cowboy.git">https://gitlab.com/ninenines/cowboy.git</a> </p> </li> </ul></div> <div class="paragraph"><p>They are updated at the same time so there is no real difference.</p></div> <div class="paragraph"><p>Cowboy 2.0 will be released once all the breaking changes are completed and the temporarily removed features are added back.</p></div> <div class="paragraph"><p>Thanks for your patience. I know it took a long time.</p></div> <hr/> <div class="paragraph"><p>Half-price on all donations because I need a new hat:</p></div> <form action="https://www.paypal.com/cgi-bin/webscr" method="post" style="display:inline"> <input type="hidden" name="cmd" value="_donations"> <input type="hidden" name="business" value="essen@ninenines.eu"> <input type="hidden" name="lc" value="FR"> <input type="hidden" name="item_name" value="Loic Hoguin"> <input type="hidden" name="item_number" value="99s"> <input type="hidden" name="currency_code" value="EUR"> <input type="hidden" name="bn" value="PP-DonationsBF:btn_donate_LG.gif:NonHosted"> <input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif" border="0" name="submit" alt="PayPal - The safer, easier way to pay online!"> <img alt="" border="0" src="https://www.paypalobjects.com/fr_FR/i/scr/pixel.gif" width="1" height="1"> </form> Ranch 1.3 https://ninenines.eu/articles/ranch-1.3/ Mon, 28 Nov 2016 00:00:00 +0100 https://ninenines.eu/articles/ranch-1.3/ <div class="paragraph"><p>Ranch <code>1.3.0</code> has been released!</p></div> <div class="paragraph"><p>This release fixes a number of long standing issues and adds a small number of features:</p></div> <div class="paragraph"><p>The <code>ssl</code> application has been added to the list of dependencies. If you don&#8217;t need it, you can remove it automatically when fetching Ranch or when building the release. If you do need it, you will no longer have issues shutting down a node because of Ranch.</p></div> <div class="paragraph"><p>The <code>ranch:info/0</code> and <code>ranch:procs/2</code> can be used to retrieve information about Ranch&#8217;s state. Use it for diagnostic and discovery purposes.</p></div> <div class="paragraph"><p>SSL listeners can now be configured without a certificate, for setups that make use of the SNI extension.</p></div> <div class="paragraph"><p>Transport options are now a blacklist, meaning all unknown options will be accepted. However Dialyzer will warn if said option is not defined in Ranch&#8217;s type specifications. Please send a patch when that happens!</p></div> <div class="paragraph"><p>Various bugs have been fixed, including the bug where the number of active connections could become negative. Common errors at listener startup should be easier to read (for example when the port is already in use).</p></div> <div class="paragraph"><p>See the <a href="https://git.ninenines.eu/ranch.git/plain/CHANGELOG.asciidoc">CHANGELOG</a> for more details.</p></div> <div class="paragraph"><p>Ranch is now tested and supported with Erlang/OTP R16B or above on Arch Linux, FreeBSD, OSX, Ubuntu and Windows 7. Contact me if you can provide permanent access to another platform for the purposes of testing.</p></div> <div class="paragraph"><p>Ranch is now available from four locations:</p></div> <div class="ulist"><ul> <li> <p> <a href="https://git.ninenines.eu/ranch.git">https://git.ninenines.eu/ranch.git</a> </p> </li> <li> <p> <a href="https://github.com/ninenines/ranch.git">https://github.com/ninenines/ranch.git</a> </p> </li> <li> <p> <a href="https://bitbucket.org/ninenines/ranch.git">https://bitbucket.org/ninenines/ranch.git</a> </p> </li> <li> <p> <a href="https://gitlab.com/ninenines/ranch.git">https://gitlab.com/ninenines/ranch.git</a> </p> </li> </ul></div> <div class="paragraph"><p>They are updated at the same time so there is no real difference.</p></div> <div class="paragraph"><p>The most recent Ranch commit is now always signed. You can import the <a href="https://pgp.mit.edu/pks/lookup?op=vindex&amp;fingerprint=on&amp;exact=on&amp;search=0xF19F189CECC7439699CEDD7A6EF7A77066CCCC8A">signing key for Loïc Hoguin</a> with:</p></div> <div class="listingblock"> <div class="content"><!-- Generator: GNU source-highlight 3.1.8 by Lorenzo Bettini http://www.lorenzobettini.it http://www.gnu.org/software/src-highlite --> <pre><tt>$ gpg --keyserver hkp<span style="color: #990000">:</span>//keys<span style="color: #990000">.</span>gnupg<span style="color: #990000">.</span>net --recv-key 66CCCC8A</tt></pre></div></div> <div class="paragraph"><p>The primary key fingerprint is <code>F19F 189C ECC7 4396 99CE DD7A 6EF7 A770 66CC CC8A</code>.</p></div> <div class="paragraph"><p>When verifying signatures in git, the following should appear:</p></div> <div class="listingblock"> <div class="content"><!-- Generator: GNU source-highlight 3.1.8 by Lorenzo Bettini http://www.lorenzobettini.it http://www.gnu.org/software/src-highlite --> <pre><tt>gpg<span style="color: #990000">:</span> Signature made Sat <span style="color: #993399">26</span> Nov <span style="color: #993399">2016</span> <span style="color: #993399">12</span><span style="color: #990000">:</span><span style="color: #993399">58</span><span style="color: #990000">:</span><span style="color: #993399">35</span> PM CET gpg<span style="color: #990000">:</span> using RSA key 71366FF21851DF03 gpg<span style="color: #990000">:</span> Good signature from <span style="color: #FF0000">"Loïc Hoguin &lt;essen@ninenines.eu&gt;"</span> <span style="color: #990000">[</span>unknown<span style="color: #990000">]</span> gpg<span style="color: #990000">:</span> WARNING<span style="color: #990000">:</span> This key is not certified with a trusted signature<span style="color: #990000">!</span> gpg<span style="color: #990000">:</span> There is no indication that the signature belongs to the owner<span style="color: #990000">.</span> Primary key fingerprint<span style="color: #990000">:</span> F19F 189C ECC7 <span style="color: #993399">4396</span> 99CE DD7A 6EF7 A770 66CC CC8A Subkey fingerprint<span style="color: #990000">:</span> FEDA <span style="color: #993399">6E41</span> B390 F745 A385 5CDC <span style="color: #993399">7136</span> 6FF2 <span style="color: #993399">1851</span> DF03</tt></pre></div></div> <div class="paragraph"><p>You can safely ignore the warning if you don&#8217;t know what it means, as long as everything else is correct.</p></div> <div class="paragraph"><p>Mirrors and signature verification will soon be implemented directly in Erlang.mk. In the meantime, you will need to set them up manually.</p></div> <div class="paragraph"><p>Most of this work was done to fix issues in RabbitMQ. Paid customers get priority; contact me if you have some issues that need fixing sooner rather than later.</p></div> <div class="paragraph"><p>Expect future releases to be announced in this space.</p></div> <div class="paragraph"><p>Thanks for reading!</p></div> Mailing list archived https://ninenines.eu/articles/ml-archives/ Mon, 29 Aug 2016 00:00:00 +0100 https://ninenines.eu/articles/ml-archives/ <div class="paragraph"><p>The <a href="https://ninenines.eu/archives/extend">old mailing list archives</a> have been added to the site, mainly for referencing purposes.</p></div> <div class="paragraph"><p>The mailing list has been shut down and all personal information has been deleted.</p></div> <div class="paragraph"><p>If you need help with a project, consider either opening a ticket on that project&#8217;s issues tracker or going through the community channels (erlang-questions, #ninenines or #erlang on Freenode).</p></div> <div class="paragraph"><p>Prefer tickets; often when people have issues it highlights an underlying problem in the project or its documentation.</p></div> <div class="paragraph"><p>Thanks.</p></div> Website update https://ninenines.eu/articles/website-update/ Sat, 02 Apr 2016 00:00:00 +0100 https://ninenines.eu/articles/website-update/ <div class="paragraph"><p>Last week-end I updated the Nine Nines website.</p></div> <div class="paragraph"><p>I switched to <a href="http://gohugo.io/">Hugo</a>. The site is now built from <a href="http://asciidoc.org/">Asciidoc</a> documents. You probably saw me switch to Asciidoc for documentation this past year. This is the natural conclusion to that story. The great thing is that with a little bit of Makefile magic I can just copy the documentation files into Hugo and poof, they appear on the website.</p></div> <div class="paragraph"><p>I am very happy with that new setup. I can now post my thoughts again. Woo! Expect regular posts from now on. I will try to replace my long series of tweets with posts.</p></div> <div class="paragraph"><p>The sections have been rearranged. There used to be a separate training section; now <a href="https://ninenines.eu/services">all my services</a> are described in one page. I have also clarified my areas of expertise. There used to be confusion in the past, so now it should be clearer that I am not a distributed systems expert.</p></div> <div class="paragraph"><p>On that note, if you are looking for my services right now, I&#8217;m not available. I&#8217;ll have to work 7 days a week for a while. Try again in a couple months. More on that in a future post.</p></div> <div class="paragraph"><p>The <a href="https://ninenines.eu/docs">documentation</a> becomes a first class citizen. Bullet and Cowlib don&#8217;t have proper documentation&#8230; yet. I have started working on the Cowlib documentation, and Bullet shouldn&#8217;t take too long. All these projects will be documented when Cowboy gets to 2.0, and will all be supported equally. Note that the Cowboy 1.0 documentation still has the old website templates and links. Don&#8217;t worry about it.</p></div> <div class="paragraph"><p>The mailing lists link has been removed. I did announce a few months back that mailing lists were going to go. They&#8217;re still up right now, but not for long. I am planning to put the archives read-only, link to them from a future post and be done with it. If you have a question, open a ticket on Github. Then I can just decide to leave the ticket open if I want to do improvements based on your feedback.</p></div> <div class="paragraph"><p>I have replaced most of the "we" by "I". I am a one-man company right now. Have been for a while. Doesn&#8217;t make sense to keep a facade. I want to be close to users, not put a barrier between us.</p></div> <div class="paragraph"><p>The RSS changed. The old link doesn&#8217;t work anymore. The new link is at <a href="https://ninenines.eu/index.xml">/index.xml</a>, or <a href="https://ninenines.eu/articles/index.xml">/articles/index.xml</a> if you only care about my posts. I guess that&#8217;s the one most people want.</p></div> <div class="paragraph"><p>I still have some tweaks to do, but it will take a while. My long term plan is to remove Bootstrap, use vanilla CSS and as little JS as possible. The reason for that is that it&#8217;s cheaper than upgrading libraries every few years. Life is too short to spend it upgrading JS libraries.</p></div> The Erlanger Playbook September 2015 Update https://ninenines.eu/articles/erlanger-playbook-september-2015-update/ Wed, 02 Sep 2015 00:00:00 +0100 https://ninenines.eu/articles/erlanger-playbook-september-2015-update/ <div class="paragraph"><p>An update to The Erlanger Playbook is now available!</p></div> <div class="paragraph"><p>The Erlanger Playbook is a book about software development using Erlang. It currently covers all areas from the conception, design, the writing of code, documentation and tests.</p></div> <div class="paragraph"><p>The book is still a work in progress. Future topics will include refactoring, debugging and tracing, benchmarking, releases, community management (for open source projects).</p></div> <div class="paragraph"><p>This update fixes a number of things and adds two chapters: IOlists and Erlang building blocks.</p></div> <div class="paragraph"><p>Learn more about <a href="https://ninenines.eu/articles/erlanger-playbook">The Erlanger Playbook</a>!</p></div> <div class="paragraph"><p>This is a self-published ebook. The base price is 50€. All proceeds will be used to allow me to work on open source full time.</p></div> <div class="paragraph"><p>Thank you for helping me helping you help us all!</p></div> Consulting & Training https://ninenines.eu/services/ Wed, 01 Jul 2015 00:00:00 +0100 https://ninenines.eu/services/ <div class="paragraph"><p>If you are interested by any of these opportunities, <a href="mailto:contact@ninenines.eu">send me an email</a>.</p></div> <div class="sect1"> <h2 id="_consulting">Consulting</h2> <div class="sectionbody"> <div class="paragraph"><p>You can get me, Loïc Hoguin, author of Cowboy, to help you solve a problem or work on a particular project.</p></div> <div class="paragraph"><p>My area of expertise is Erlang; HTTP, Websocket and REST APIs; design and implementation of protocols; and messaging systems.</p></div> <div class="paragraph"><p>I can also be helpful with testing or code reviews.</p></div> <div class="paragraph"><p>I offer both hourly and daily rates:</p></div> <div class="ulist"><ul> <li> <p> 200€ hourly rate (remote) </p> </li> <li> <p> 1000€ daily rate (remote and on-site) </p> </li> </ul></div> <div class="paragraph"><p>For remote consulting, the work can be done by phone, email, IRC, GitHub and/or any other platform for collaborative work.</p></div> <div class="paragraph"><p>For on-site consulting, the travel expenses and accomodations are to be paid by the customer. I will also ask for a higher rate if forced to stay on-site for more than a week.</p></div> <div class="paragraph"><p>Note that my expertise does not cover all areas where Erlang is used. My help will be limited in the areas of distributed databases, or large distributed systems.</p></div> </div> </div> <div class="sect1"> <h2 id="_sponsoring">Sponsoring</h2> <div class="sectionbody"> <div class="paragraph"><p>You can sponsor one of my projects.</p></div> <div class="paragraph"><p>Sponsoring gives you:</p></div> <div class="ulist"><ul> <li> <p> a direct, private line of communication </p> </li> <li> <p> the power to make me maintain older versions of my projects (as long as they are sponsoring) </p> </li> <li> <p> priority when adding features or fixing bugs </p> </li> <li> <p> advertisement space on this website and in the README file of the project of your choice </p> </li> </ul></div> <div class="paragraph"><p>Sponsors may choose to benefit from any of these perks.</p></div> <div class="paragraph"><p>In exchange sponsors must contribute financially. A minimum of 200€ per month is required. Sponsors may give as much as they want. Payment can be monthly or one-time. Invoices are of course provided.</p></div> </div> </div> <div class="sect1"> <h2 id="_erlang_beginner_training">Erlang beginner training</h2> <div class="sectionbody"> <div class="paragraph"><p>I would be happy to introduce more people to Erlang. I have a 1-day Erlang training readily available for consumption. The goal of this training is to teach the basics of Erlang systems and programming. It&#8217;s a kind of "Getting started" for Erlang.</p></div> <div class="paragraph"><p>You can review the <a href="https://ninenines.eu/talks/thinking-in-erlang/thinking-in-erlang.html">training slides</a>.</p></div> <div class="paragraph"><p>This training is meant to be given to a large number of people interested in Erlang, as part of a public event, where anyone interested can come.</p></div> <div class="paragraph"><p>Another important aspect of this training is that it is meant to be affordable. We want the most people to learn Erlang as possible.</p></div> <div class="paragraph"><p>If you have room, think you can gather 20+ people and are interested in sponsoring a training session, then we should talk.</p></div> </div> </div> <div class="sect1"> <h2 id="_custom_training">Custom training</h2> <div class="sectionbody"> <div class="paragraph"><p>I can also provide custom training, tailored to your level and your needs. It can take the form of a class, Q&amp;A or a code review/writing session. I need to know your expectations to prepare an appropriate training.</p></div> <div class="paragraph"><p>Custom training rates are the same as consulting rates and the same restrictions apply.</p></div> </div> </div> Documentation https://ninenines.eu/docs/ Wed, 01 Jul 2015 00:00:00 +0100 https://ninenines.eu/docs/ <div class="sect2"> <h3 id="_contribute">Contribute</h3> <div class="paragraph"><p>Do you have examples, tutorials, videos about one or more of my projects? I would happily include them on this page.</p></div> <div class="paragraph"><p><a href="mailto:contact@ninenines.eu">Send me an email with the details</a>.</p></div> </div> Donate https://ninenines.eu/donate/ Wed, 01 Jul 2015 00:00:00 +0100 https://ninenines.eu/donate/ <div class="sect2"> <h3 id="_like_my_work_donate">Like my work? Donate!</h3> <div class="paragraph"><p>Donate to Loïc Hoguin because his work on Cowboy and Erlang.mk is fantastic:</p></div> <form action="https://www.paypal.com/cgi-bin/webscr" method="post" style="display:inline"> <input type="hidden" name="cmd" value="_donations"> <input type="hidden" name="business" value="essen@ninenines.eu"> <input type="hidden" name="lc" value="FR"> <input type="hidden" name="item_name" value="Loic Hoguin"> <input type="hidden" name="item_number" value="99s"> <input type="hidden" name="currency_code" value="EUR"> <input type="hidden" name="bn" value="PP-DonationsBF:btn_donate_LG.gif:NonHosted"> <input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif" border="0" name="submit" alt="PayPal - The safer, easier way to pay online!"> <img alt="" border="0" src="https://www.paypalobjects.com/fr_FR/i/scr/pixel.gif" width="1" height="1"> </form> </div> Public talks https://ninenines.eu/talks/ Wed, 01 Jul 2015 00:00:00 +0100 https://ninenines.eu/talks/ <div class="sect2"> <h3 id="_talk_requests">Talk requests</h3> <div class="paragraph"><p>Organizing a conference and in need of a speaker for a talk about Erlang and the Web? Need an introduction to Erlang/OTP for your company? Looking for a cool subject for a user group meeting?</p></div> <div class="paragraph"><p><a href="mailto:contact@ninenines.eu">Send me an email with the details</a>.</p></div> </div> Slogan https://ninenines.eu/slogan/ Wed, 01 Jul 2015 00:00:00 +0100 https://ninenines.eu/slogan/ <div class="paragraph"><p>The Erlanger Playbook is now available!<br /> <a href="https://ninenines.eu/articles/erlanger-playbook">Buy now</a> — <a href="https://ninenines.eu/services">Become a Cowboy project sponsor</a></p></div> The Erlanger Playbook https://ninenines.eu/articles/erlanger-playbook/ Thu, 18 Jun 2015 00:00:00 +0100 https://ninenines.eu/articles/erlanger-playbook/ <div class="paragraph"><p>I am proud to announce the pre-release of The Erlanger Playbook.</p></div> <div class="paragraph"><p>The Erlanger Playbook is a book about software development using Erlang. It currently covers all areas from the conception, design, the writing of code, documentation and tests.</p></div> <div class="paragraph"><p>The book is still a work in progress. Future topics will include refactoring, debugging and tracing, benchmarking, releases, community management (for open source projects).</p></div> <div class="paragraph"><p>The following sections are currently available:</p></div> <div class="ulist"><ul> <li> <p> About this book; Future additions </p> </li> <li> <p> <em>Workflow:</em> Think; Write; Stay productive </p> </li> <li> <p> <em>Documentation:</em> On documentation; Tutorials; User guide; Manual </p> </li> <li> <p> <em>Code:</em> Starting a project; Version control; Project structure; Code style; Best practices; Special processes </p> </li> <li> <p> <em>Tests:</em> On testing; Success typing analysis; Manual testing; Unit testing; Functional testing </p> </li> </ul></div> <div class="paragraph"><p>Read a preview: <a href="https://ninenines.eu/res/erlanger-preview.pdf">Special processes</a></p></div> <div class="paragraph"><p>The book is currently just shy of 100 pages. The final version of the book is planned to be between 200 and 250 pages. A print version of the book will be considered once the final version gets released. The printed book is <strong>not</strong> included in the price.</p></div> <div class="paragraph"><p>This is a self-published book. The base price is 50€. All proceeds will be used to allow me to work on open source full time.</p></div> <form action="https://www.paypal.com/cgi-bin/webscr" method="post" target="_top"> <input type="hidden" name="cmd" value="_s-xclick"> <input type="hidden" name="hosted_button_id" value="9M44HJCGX3GVN"> <input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_buynowCC_LG.gif" border="0" name="submit" alt="PayPal - The safer, easier way to pay online!"> <img alt="" border="0" src="https://www.paypalobjects.com/fr_FR/i/scr/pixel.gif" width="1" height="1"> </form> <div class="paragraph"><p>You are more than welcome to pay extra by using this second button. It allows you to set the price you want. Make sure to set it to at least 50€ to receive the book.</p></div> <form action="https://www.paypal.com/cgi-bin/webscr" method="post" target="_top"> <input type="hidden" name="cmd" value="_s-xclick"> <input type="hidden" name="hosted_button_id" value="BBW9TR9LBK8C2"> <input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_buynowCC_LG.gif" border="0" name="submit" alt="PayPal - The safer, easier way to pay online!"> <img alt="" border="0" src="https://www.paypalobjects.com/fr_FR/i/scr/pixel.gif" width="1" height="1"> </form> <div class="paragraph"><p>Make sure to provide a valid email address.</p></div> <div class="paragraph"><p>There will be a <strong>delay</strong> between payment and sending of the book. This process is currently manual.</p></div> <div class="paragraph"><p>As the book is a pre-release, feedback is more than welcome. You can send your comments to erlanger@ this website.</p></div> <div class="paragraph"><p>The plan is to add about 20 pages every month until it is completed. You will receive updates to the book for free as soon as they are available.</p></div> <div class="paragraph"><p>Huge thanks for your interest in buying this book!</p></div> Validating UTF-8 binaries with Erlang https://ninenines.eu/articles/erlang-validate-utf8/ Fri, 06 Mar 2015 00:00:00 +0100 https://ninenines.eu/articles/erlang-validate-utf8/ <div class="paragraph"><p>Yesterday I pushed Websocket permessage-deflate to Cowboy master. I also pushed <a href="https://github.com/ninenines/cowlib/commit/7e4983b70ddf8cedb967e36fba6a600731bdad5d">a change in the way the code validates UTF-8 data</a> (required for text and close frames as per the spec).</p></div> <div class="paragraph"><p>When looking into why the permessage-deflate tests in autobahntestsuite were taking such a long time, I found that autobahn is using an adaptation of the algorithm named <a href="http://bjoern.hoehrmann.de/utf-8/decoder/dfa/">Flexible and Economical UTF-8 Decoder</a>. This is the C99 implementation:</p></div> <div class="listingblock"> <div class="content"><!-- Generator: GNU source-highlight 3.1.8 by Lorenzo Bettini http://www.lorenzobettini.it http://www.gnu.org/software/src-highlite --> <pre><tt><span style="font-style: italic"><span style="color: #9A1900">// Copyright (c) 2008-2009 Bjoern Hoehrmann &lt;bjoern@hoehrmann.de&gt;</span></span> <span style="font-style: italic"><span style="color: #9A1900">// See http://bjoern.hoehrmann.de/utf-8/decoder/dfa/ for details.</span></span> <span style="font-weight: bold"><span style="color: #000080">#define</span></span> UTF8_ACCEPT <span style="color: #993399">0</span> <span style="font-weight: bold"><span style="color: #000080">#define</span></span> UTF8_REJECT <span style="color: #993399">1</span> <span style="font-weight: bold"><span style="color: #0000FF">static</span></span> <span style="font-weight: bold"><span style="color: #0000FF">const</span></span> <span style="color: #008080">uint8_t</span> utf8d<span style="color: #990000">[]</span> <span style="color: #990000">=</span> <span style="color: #FF0000">{</span> <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 00..1f</span></span> <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 20..3f</span></span> <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 40..5f</span></span> <span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 60..7f</span></span> <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span><span style="color: #993399">9</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// 80..9f</span></span> <span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span><span style="color: #993399">7</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// a0..bf</span></span> <span style="color: #993399">8</span><span style="color: #990000">,</span><span style="color: #993399">8</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// c0..df</span></span> <span style="color: #993399">0xa</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x4</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// e0..ef</span></span> <span style="color: #993399">0xb</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x5</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// f0..ff</span></span> <span style="color: #993399">0x0</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x2</span><span style="color: #990000">,</span><span style="color: #993399">0x3</span><span style="color: #990000">,</span><span style="color: #993399">0x5</span><span style="color: #990000">,</span><span style="color: #993399">0x8</span><span style="color: #990000">,</span><span style="color: #993399">0x7</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x4</span><span style="color: #990000">,</span><span style="color: #993399">0x6</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span><span style="color: #993399">0x1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s0..s0</span></span> <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">0</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s1..s2</span></span> <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s3..s4</span></span> <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">2</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s5..s6</span></span> <span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">3</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span><span style="color: #993399">1</span><span style="color: #990000">,</span> <span style="font-style: italic"><span style="color: #9A1900">// s7..s8</span></span> <span style="color: #FF0000">}</span><span style="color: #990000">;</span> uint32_t inline <span style="font-weight: bold"><span style="color: #000000">decode</span></span><span style="color: #990000">(</span>uint32_t<span style="color: #990000">*</span> state<span style="color: #990000">,</span> uint32_t<span style="color: #990000">*</span> codep<span style="color: #990000">,</span> <span style="color: #008080">uint32_t</span> byte<span style="color: #990000">)</span> <span style="color: #FF0000">{</span> <span style="color: #008080">uint32_t</span> type <span style="color: #990000">=</span> utf8d<span style="color: #990000">[</span>byte<span style="color: #990000">];</span> <span style="color: #990000">*</span>codep <span style="color: #990000">=</span> <span style="color: #990000">(*</span>state <span style="color: #990000">!=</span> UTF8_ACCEPT<span style="color: #990000">)</span> <span style="color: #990000">?</span> <span style="color: #990000">(</span>byte <span style="color: #990000">&amp;</span> <span style="color: #993399">0x3fu</span><span style="color: #990000">)</span> <span style="color: #990000">|</span> <span style="color: #990000">(*</span>codep <span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">6</span><span style="color: #990000">)</span> <span style="color: #990000">:</span> <span style="color: #990000">(</span><span style="color: #993399">0xff</span> <span style="color: #990000">&gt;&gt;</span> type<span style="color: #990000">)</span> <span style="color: #990000">&amp;</span> <span style="color: #990000">(</span>byte<span style="color: #990000">);</span> <span style="color: #990000">*</span>state <span style="color: #990000">=</span> utf8d<span style="color: #990000">[</span><span style="color: #993399">256</span> <span style="color: #990000">+</span> <span style="color: #990000">*</span>state<span style="color: #990000">*</span><span style="color: #993399">16</span> <span style="color: #990000">+</span> type<span style="color: #990000">];</span> <span style="font-weight: bold"><span style="color: #0000FF">return</span></span> <span style="color: #990000">*</span>state<span style="color: #990000">;</span> <span style="color: #FF0000">}</span></tt></pre></div></div> <div class="paragraph"><p>And this is the Erlang implementation I came up with:</p></div> <div class="listingblock"> <div class="content"><!-- Generator: GNU source-highlight 3.1.8 by Lorenzo Bettini http://www.lorenzobettini.it http://www.gnu.org/software/src-highlite --> <pre><tt><span style="font-style: italic"><span style="color: #9A1900">%% This function returns 0 on success, 1 on error, and 2..8 on incomplete data.</span></span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;&gt;&gt;</span>, <span style="color: #009900">State</span>) <span style="color: #990000">-&gt;</span> <span style="color: #009900">State</span>; <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">128</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">8</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">128</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">144</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">4</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">160</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">192</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">194</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">224</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">224</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">4</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">225</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">237</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">237</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">5</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">238</span>; <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">239</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">240</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">6</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">241</span>; <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">242</span>; <span style="color: #009900">C</span> <span style="color: #990000">=:=</span> <span style="color: #993399">243</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">7</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">244</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">8</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">_</span>, <span style="color: #990000">_</span>) <span style="color: #990000">-&gt;</span> <span style="color: #993399">1</span><span style="color: #990000">.</span></tt></pre></div></div> <div class="paragraph"><p>Does it look similar to you? So how did we get there?</p></div> <div class="paragraph"><p>I started with a naive implementation of the original. First, we don&#8217;t need the codepoint calculated and extracted for our validation function. We just want to know the data is valid, so we only need to calculate the next state. Then, the only thing we needed to be careful about was that tuples are 1-based, and that we need to stop processing the binary when we get the state 1 or when the binary is empty.</p></div> <div class="listingblock"> <div class="content"><!-- Generator: GNU source-highlight 3.1.8 by Lorenzo Bettini http://www.lorenzobettini.it http://www.gnu.org/software/src-highlite --> <pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;&gt;&gt;</span>, <span style="color: #009900">State</span>) <span style="color: #990000">-&gt;</span> <span style="color: #009900">State</span>; <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">_</span>, <span style="color: #993399">1</span>) <span style="color: #990000">-&gt;</span> <span style="color: #993399">1</span>; <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #009900">State</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">257</span> <span style="color: #990000">+</span> <span style="color: #009900">State</span> <span style="color: #990000">*</span> <span style="color: #993399">16</span> <span style="color: #990000">+</span> <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">1</span> <span style="color: #990000">+</span> <span style="color: #009900">C</span>, <span style="font-weight: bold"><span style="color: #000080">?UTF8D</span></span>), <span style="font-weight: bold"><span style="color: #000080">?UTF8D</span></span>))<span style="color: #990000">.</span></tt></pre></div></div> <div class="paragraph"><p>The macro <code>?UTF8D</code> is the tuple equivalent of the C array in the original code.</p></div> <div class="paragraph"><p>Compared to our previous algorithm, this performed about the same. In some situations a little faster, in some a little slower. In other words, not good enough. But because this new algorithm allows us to avoid a binary concatenation this warranted looking further.</p></div> <div class="paragraph"><p>It was time to step into crazy land.</p></div> <div class="paragraph"><p>Erlang is very good at pattern matching, even more so than doing some arithmetic coupled by fetching elements from a tuple. So I decided I was going to write all possible clauses for all combinations of <code>C</code> and <code>State</code>. And by write I mean generate.</p></div> <div class="paragraph"><p>So I opened my Erlang shell, defined the variable <code>D</code> to be the tuple <code>?UTF8D</code> with its 400 elements, and then ran the following expression (after a bit of trial and error):</p></div> <div class="listingblock"> <div class="content"><!-- Generator: GNU source-highlight 3.1.8 by Lorenzo Bettini http://www.lorenzobettini.it http://www.gnu.org/software/src-highlite --> <pre><tt><span style="color: #993399">16</span><span style="color: #990000">&gt;</span> <span style="font-weight: bold"><span style="color: #000000">file:write_file</span></span>(<span style="color: #FF0000">"out.txt"</span>, [<span style="font-weight: bold"><span style="color: #000000">io_lib:format</span></span>(<span style="color: #FF0000">"validate_utf8(&lt;&lt; ~p, Rest/bits &gt;&gt;, ~p) -&gt; ~p;~n"</span>, [<span style="color: #009900">C</span>, <span style="color: #009900">S</span>, <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">257</span> <span style="color: #990000">+</span> <span style="color: #009900">S</span> <span style="color: #990000">*</span> <span style="color: #993399">16</span> <span style="color: #990000">+</span> <span style="font-weight: bold"><span style="color: #000080">element</span></span>(<span style="color: #993399">1</span> <span style="color: #990000">+</span> <span style="color: #009900">C</span>, <span style="color: #009900">D</span>), <span style="color: #009900">D</span>)]) || <span style="color: #009900">C</span> <span style="color: #990000">&lt;-</span> <span style="font-weight: bold"><span style="color: #000000">lists:seq</span></span>(<span style="color: #993399">0</span>,<span style="color: #993399">255</span>), <span style="color: #009900">S</span> <span style="color: #990000">&lt;-</span> <span style="font-weight: bold"><span style="color: #000000">lists:seq</span></span>(<span style="color: #993399">0</span>,<span style="color: #993399">8</span>)])<span style="color: #990000">.</span> <span style="color: #FF6600">ok</span></tt></pre></div></div> <div class="paragraph"><p>The result is a 2304 lines long file, containing 2304 clauses. People who pay attention to what I say on Twitter will remember I said something around 3000 clauses, but that was just me not using the right number of states in my estimate.</p></div> <div class="paragraph"><p>There was a little more work to be done on this generated code that I did using regular expressions. We need to recurse when the resulting state is not 1. We also need to stop when the binary is empty, making it the 2305th clause.</p></div> <div class="paragraph"><p>Still, 2305 is a lot. But hey, the code did work, and faster than the previous implementation too! But hey, perhaps I could find a way to reduce its size.</p></div> <div class="paragraph"><p>Removing all the clauses that return 1 and putting a catch-all clause at the end instead reduced the number to about 500, and showed that many clauses were similar:</p></div> <div class="listingblock"> <div class="content"><!-- Generator: GNU source-highlight 3.1.8 by Lorenzo Bettini http://www.lorenzobettini.it http://www.gnu.org/software/src-highlite --> <pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">0</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">1</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">2</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">3</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">4</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">5</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">6</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">7</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>);</tt></pre></div></div> <div class="paragraph"><p>But also:</p></div> <div class="listingblock"> <div class="content"><!-- Generator: GNU source-highlight 3.1.8 by Lorenzo Bettini http://www.lorenzobettini.it http://www.gnu.org/software/src-highlite --> <pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">157</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #993399">158</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>);</tt></pre></div></div> <div class="paragraph"><p>Patterns, my favorites!</p></div> <div class="paragraph"><p>A little more time was spent to edit the 500 or so clauses into smaller equivalents, testing that performance was not impacted, and comitting the result.</p></div> <div class="paragraph"><p>The patterns above can be found here in the resulting function:</p></div> <div class="listingblock"> <div class="content"><!-- Generator: GNU source-highlight 3.1.8 by Lorenzo Bettini http://www.lorenzobettini.it http://www.gnu.org/software/src-highlite --> <pre><tt><span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">0</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">128</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="color: #990000">...</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">2</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">0</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">3</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">5</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">2</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">6</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #990000">&lt;&lt;</span> <span style="color: #009900">C</span>, <span style="color: #009900">Rest</span><span style="color: #990000">/</span><span style="color: #FF6600">bits</span> <span style="color: #990000">&gt;&gt;</span>, <span style="color: #993399">7</span>) <span style="font-weight: bold"><span style="color: #0000FF">when</span></span> <span style="color: #009900">C</span> <span style="color: #990000">&gt;=</span> <span style="color: #993399">144</span>, <span style="color: #009900">C</span> <span style="color: #990000">&lt;</span> <span style="color: #993399">160</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">validate_utf8</span></span>(<span style="color: #009900">Rest</span>, <span style="color: #993399">3</span>); <span style="color: #990000">...</span></tt></pre></div></div> <div class="paragraph"><p>I hope you enjoyed this post.</p></div> On open source https://ninenines.eu/articles/on-open-source/ Fri, 05 Sep 2014 00:00:00 +0100 https://ninenines.eu/articles/on-open-source/ <div class="paragraph"><p>Last week I read a great article <a href="http://videlalvaro.github.io/2014/08/on-contributing-to-opensource.html">on contributing to open source</a> by Alvaro Videla. He makes many great points and I am in agreement with most of it. This made me want to properly explain my point of view with regard to open source and contributions. Unlike most open source evangelism articles I will not talk about ideals or any of that crap, but rather my personal feelings and experience.</p></div> <div class="paragraph"><p>I have been doing open source work for quite some time. My very first open source project was a graphics driver for (the very early version of) the PCSX2 emulator. That was more than ten years ago, and there <a href="http://ngemu.com/threads/gstaris-0-6.30469/">isn&#8217;t much left to look at today</a>. This was followed by a <a href="https://github.com/extend/wee">PHP framework</a> (started long before Zend Framework was even a thing) and a few other small projects. None of them really took off. It&#8217;s alright, that&#8217;s pretty much the fate of most open source projects. You spend a lot of work and sweat and get very little in return from others.</p></div> <div class="paragraph"><p>This sounds harsh but this is the reality of all open source projects. If you are thinking of building a project and releasing it as open source, you should be prepared for that. This is how most of your projects will feel like. Don&#8217;t release a project as open source thinking everyone will pat you on the back and cheer, this won&#8217;t happen. In fact if your project is a too small improvement over existing software, what many people will do is say you have NIH syndrome, regardless of the improvement you bring. So you need not to rely on other people in order to get your enjoyment out of building open source software.</p></div> <div class="paragraph"><p>In my case I get enjoyment from thinking about problems that need solving. Often times the problems are already solved, but nevermind that, I still think about them and sometimes come up with something I feel is better and then write code for it. Writing code is also fun, but not as fun as using my brain to imagine solutions.</p></div> <div class="paragraph"><p>You don&#8217;t need thousands of users to do that. So are users worthless to me then? No, of course not. In fact they are an important component: they bring me problems that need solving. So users are very important to me. But that&#8217;s not the only reason.</p></div> <div class="paragraph"><p>I got lucky that the Cowboy project became popular. And seeing it be this popular, and some of my other projects also do quite well, made me believe I could perhaps work full time on open source. If I can work full time then I can produce better software. What I had one hour to work on before I can now spend a day on, and experiment until I am satisfied. This is very useful because that means I can get it almost right from the beginning, and avoid the million API breaking changes that occured before Cowboy 1.0 was released.</p></div> <div class="paragraph"><p>To be able to work full time on open source however, I need money. This is a largely unspoken topic of open source work. The work is never free. You can download the product for free, but someone has to pay for the work itself. Life is unfortunately not free.</p></div> <div class="paragraph"><p>Large projects and some lucky people have their work sponsored by their employers. Everyone else has to deal with it differently. In my case I was sponsored for a while by the <a href="http://leo-project.net/leofs/">LeoFS</a> project, but that ended. I also had the Farwest fundraiser, which was a success, although the project stalled after that. (Fear not, as Farwest will make a comeback as a conglomerate of Web development projects in the future.) After that I set up the <a href="http://ninenines.eu/support/">sponsoring scheme</a>, which I can proudly say today brings in enough money to cover my food and shelter. Great!</p></div> <div class="paragraph"><p>This is a start, but it&#8217;s of course not enough. Life is a little more than food and shelter, and so I am still looking for sponsors. This is not a very glorious experience, as I am essentially looking for scraps that companies can throw away. Still, if a handful more companies were doing that, not only would I be able to live comfortably, but I would also be able to stop worrying about the future as I could put money on the side for when it gets rough.</p></div> <div class="paragraph"><p>A few companies giving me some scrap money so I could live and work independently is by far the most important thing anyone can do to help my projects, including Cowboy. Yes, they&#8217;re even more important than code contributions, bug reports and feedback. Because this money gives me the time I need to handle the code contributions, bug reports and feedback.</p></div> <div class="paragraph"><p>If Cowboy or another project is a large part of your product or infrastructure, then the best thing you can do is become a sponsor. The second best is opening tickets and/or providing feedback. The third best is providing good code contributions.</p></div> <div class="paragraph"><p>I will not expand on the feedback part. Feedback is very important, and even just a high five or a retweet is already good feedback. It&#8217;s not very complicated.</p></div> <div class="paragraph"><p>I want to expand a little on code contributions however. Not long ago I ran across the term "patch bomb" which means dropping patches and expecting the project maintainers to merge them and maintain them. I receive a lot of patches, and often have to refuse them. Causes for refusal vary. Some patches only benefit the people who submitted them (or a very small number of people). Some patches are not refined enough to be included. Others are out of scope of the project. These are some of the reasons why I refuse patches. Having limited time and resources, I have to focus my efforts on the code used by the larger number of users. I have to prioritize patches from submitters who are reactive and address the issues pointed out. And I have to plainly refuse other patches.</p></div> <div class="paragraph"><p>I believe this wraps up my thoughts on open source. Overall I had a great experience, the Erlang community being nice and understanding of the issues at hand in general. And if the money problem could be solved soon, then I would be one of the luckiest and happiest open source developer on Earth.</p></div> <div class="paragraph"><p>Think about it the next time you see a donation button or a request for funds or sponsoring. You can considerably improve an open source developer&#8217;s life with very little of your company&#8217;s money.</p></div>