From 7bb98d0476dbe4ee5e9317e42ca17e4a7d717e0d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Lo=C3=AFc=20Hoguin?= Date: Sun, 22 Jan 2017 13:51:38 +0100 Subject: Publish the "Don't let it crash" article --- index.xml | 281 +++++++++++++++++++++++++++----------------------------------- 1 file changed, 122 insertions(+), 159 deletions(-) (limited to 'index.xml') diff --git a/index.xml b/index.xml index 6d4b70a1..31704d83 100644 --- a/index.xml +++ b/index.xml @@ -6,9 +6,130 @@ Recent content on Nine Nines Hugo -- gohugo.io en-us - Tue, 03 Jan 2017 00:00:00 +0100 + Sun, 22 Jan 2017 00:00:00 +0100 + + Don't let it crash + https://ninenines.eu/articles/dont-let-it-crash/ + Sun, 22 Jan 2017 00:00:00 +0100 + + https://ninenines.eu/articles/dont-let-it-crash/ + <div class="paragraph"><p>We have a specific mindset when writing Erlang +programs. We focus on the normal execution of the +program and don&#8217;t handle most of the errors that may +occur. We sometimes call this normal execution the +<em>happy path</em>.</p></div> +<div class="paragraph"><p>The general pattern behind writing only for the +<em>happy path</em>, letting the VM catch errors (writing +them to a log for future consumption) and then +having a supervisor restart the processes that +failed from a clean state, has a name. We call it +<em>let it crash</em>; and it drives many of our design +decisions.</p></div> +<div class="paragraph"><p>It&#8217;s a really great way to program and the results +are fantastic compared to most other programming +languages. And yet, <em>let it crash</em> barely convinced +anyone that they should use Erlang. Why would that +be?</p></div> +<div class="paragraph"><p>You may already know that Cowboy is capable of +handling at least 2 million Websocket connections +on a single server. This is in large part thanks +to the capabilities of the VM. Still, 2 million +is good, much better than most other servers can +do.</p></div> +<div class="paragraph"><p>Cowboy is not just a Websocket server; it&#8217;s also +an HTTP and HTTP/2 server, and it handles many +related features like long polling or the parsing +of most request headers.</p></div> +<div class="paragraph"><p>Can you guess how large the Cowboy codebase is, +without looking at the source?</p></div> +<div class="paragraph"><p>Do make sure you have a clear answer in your mind +before you go check.</p></div> +<div class="paragraph"><p>Good, you are back. Now what were the results? If +I am correct, you overestimated the size of Cowboy. +Cowboy is in fact about five thousand lines of code. +You probably thought it was at least ten thousand. +About eighty percent of readers will have +overestimated the size of Cowboy. And you did only +because I mentioned it can handle millions of +Websocket connections.</p></div> +<div class="paragraph"><p>Numerous studies show this effect. Just mentioning +the large number already prepared your mind to think +in that direction. Repeating the number made you +focus even more on it. Then the question asked for +a number, which ended up larger than the reality.</p></div> +<div class="paragraph"><p>The same effect can be applied to negotiation for +example. You generally want to start by giving your +offer (and not let the other party initiate) and +you want to give a really large number first. You +can also prepare your customer by mentioning an even +larger number in the previous discussion.</p></div> +<div class="paragraph"><p>And it&#8217;s not just numbers either. An experiment +showed that just by looking at an image of clouds, +customers of a pillow store were buying pillows +more comfortable (and more expensive) than those +who didn&#8217;t see that image.</p></div> +<div class="paragraph"><p>This is the power of associations. It is covered in +much larger detail in the books +<a href="https://www.amazon.com/Influence-Psychology-Persuasion-Robert-Cialdini/dp/006124189X">Influence</a> +and +<a href="https://www.amazon.com/Pre-Suasion-Revolutionary-Way-Influence-Persuade/dp/1501109790">Pre-suasion</a>. +I highly recommend reading those and applying what +you learn to your daily life. I&#8217;m definitely not +a professional psychologist so take this post with +a grain of salt.</p></div> +<div class="paragraph"><p>When selling Erlang, whether we are selling it to +a customer or trying to convince a developer friend +to start using it, we often talk about how Erlang +<em>lets you sleep at night</em>, that it is auto healing +and always gets fantastic uptimes.</p></div> +<div class="paragraph"><p>And then we talk about <em>let it crash</em>.</p></div> +<div class="paragraph"><p>And we describe what it means.</p></div> +<div class="paragraph"><p>We might as well just say that Erlang crashes a lot +and then take the door. It would have the same effect. +It doesn&#8217;t even stop at programs crashing. You know +what else crashes? Cars, planes, trains. Often with +disastrous consequences. Is that really the message +we want to convey?</p></div> +<div class="paragraph"><p>They even <a href="https://img.youtube.com/vi/oEUBW2lCkIk/0.jpg">printed it on a t-shirt</a>! +Keep calm and let it crash. It&#8217;s the kind of t-shirt +you probably shouldn&#8217;t wear in an airport, and for good +reasons. A few people did, then realized what they were +wearing and were not too smug about it.</p></div> +<div class="paragraph"><p>And yet this is how we sell Erlang.</p></div> +<div class="paragraph"><p>A better way would be to focus on the positives, of +course, but also to make sure that those positives +are phrased in a way that prevents bad associations +to be formed in people&#8217;s minds.</p></div> +<div class="paragraph"><p>Instead of <em>let it crash</em>, you can say that Erlang +has <em>auto healing mechanisms</em>. Healing is a good +thing and accurately describes what happens in the +system.</p></div> +<div class="paragraph"><p>Should you need to go into more details, you will +probably want to avoid <em>recover from crashes</em> and +instead say <em>recover from exceptions</em>. Exceptions +are a pretty neutral word and, should you explain +what you mean by that, you can talk about exceptions +that occur for reasons unrelated to Erlang, like +hardware failure or network instability.</p></div> +<div class="paragraph"><p>The trick is to always use positive words and +phrases to describe Erlang, and to use external +factors to explain how Erlang deals with failures. +Never mention the failures internal to Erlang +systems unless you are asked specifically, in +which case you can say that the auto healing +applies to all exceptions.</p></div> +<div class="paragraph"><p>The <em>let it crash</em> philosophy is great when +learning Erlang or when writing fault-tolerant +systems. But it&#8217;s not going to convince anyone +to use it unless they were already looking for +it.</p></div> +<div class="paragraph"><p>Do you like this post? Tell me on Twitter. I might +make more.</p></div> + + + Cowboy 2.0 pre-release 4 https://ninenines.eu/articles/cowboy-2.0.0-pre.4/ @@ -1129,164 +1250,6 @@ giving the same bad advice instead. Fix the code or the documentation that led to this mistake. Slowly improve the project and make sure it doesn&#8217;t happen again.</p></div> <div class="paragraph"><p>This is my story. So far, anyway.</p></div> - - - - - Cowboy 2.0 and query strings - https://ninenines.eu/articles/cowboy2-qs/ - Wed, 20 Aug 2014 00:00:00 +0100 - - https://ninenines.eu/articles/cowboy2-qs/ - <div class="paragraph"><p>Now that Cowboy 1.0 is out, I can spend some of my time thinking -about Cowboy 2.0 that will be released soon after Erlang/OTP 18.0. -This entry discusses the proposed changes to query string handling -in Cowboy.</p></div> -<div class="paragraph"><p>Cowboy 2.0 will respond to user wishes by simplifying the interface -of the <code>cowboy_req</code> module. Users want two things: less -juggling with the Req variable, and more maps. Maps is the only -dynamic key/value data structure in Erlang that we can match directly -to extract values, allowing users to greatly simplify their code as -they don&#8217;t need to call functions to do everything anymore.</p></div> -<div class="paragraph"><p>Query strings are a good candidate for maps. It&#8217;s a list of -key/values, so it&#8217;s pretty obvious we can win a lot by using maps. -However query strings have one difference with maps: they can have -duplicate keys.</p></div> -<div class="paragraph"><p>How are we expected to handle duplicate keys? There&#8217;s no standard -behavior. It&#8217;s up to applications. And looking at what is done in -the wild, there&#8217;s no de facto standard either. While some ignore -duplicate keys (keeping the first or the last they find), others -require duplicate keys to end with <code>[]</code> to automatically -put the values in a list, or even worse, languages like PHP even -allow you to do things like <code>key[something][other]</code> and -create a deep structure for it. Finally some allow any key to have -duplicates and just gives you lists of key/values.</p></div> -<div class="paragraph"><p>Cowboy so far had functions to retrieve query string values one -value at a time, and if there were duplicates it would return the -first it finds. It also has a function returning the entire list -with all duplicates, allowing you to filter it to get all of them, -and another function that returns the raw query string.</p></div> -<div class="paragraph"><p>What are duplicates used for? Not that many things actually.</p></div> -<div class="paragraph"><p>One use of duplicate keys is with HTML forms. It is common practice -to give all related checkboxes the same name so you get a list of -what&#8217;s been checked. When nothing is checked, nothing is sent at all, -the key is not in the list.</p></div> -<div class="paragraph"><p>Another use of duplicate keys is when generating forms. A good -example of that would be a form that allows uploading any number -of files. When you add a file, client-side code adds another field -to the form. Repeat up to a certain limit.</p></div> -<div class="paragraph"><p>And that&#8217;s about it. Of note is that HTML radio elements share -the same name too, but only one key/value is sent, so they are not -relevant here.</p></div> -<div class="paragraph"><p>Normally this would be the part where I tell you how we solve -this elegantly. But I had doubts. Why? Because there&#8217;s no good -solutions to solving only this particular problem.</p></div> -<div class="paragraph"><p>I then stopped thinking about duplicate keys for a minute and -started to think about the larger problem.</p></div> -<div class="paragraph"><p>Query strings are input data. They take a particular form, -and may be sent as part of the URI or as part of the request -body. We have other kinds of input data. We have headers and -cookies and the request body in various forms. We also have -path segments in URIs.</p></div> -<div class="paragraph"><p>What do you do with input data? Well you use it to do -something. But there is one thing that you almost always do -(and if you don&#8217;t, you really should): you validate it and -you map it into Erlang terms.</p></div> -<div class="paragraph"><p>Cowboy left the user take care of validation and conversion -into Erlang terms so far. Rather, it left the user take care -of it everywhere except one place. Guess where? That&#8217;s right, -bindings.</p></div> -<div class="paragraph"><p>If you define routes with bindings then you have the option -to provide constraints. Constraints can be used to do two things: -validate the data and convert it in a more appropriate term. For -example if you use the <code>int</code> constraint, Cowboy will -make sure the binding is an integer, and will replace the value -with the integer representation so that you can use it directly. -In this particular case it not only routes the URI, but also -validates and converts the bindings directly.</p></div> -<div class="paragraph"><p>This is very relevant in the case of our duplicate keys, -because if we have a list with duplicates of a key, chances -are we want to convert that into a list of Erlang terms, and -also make sure that all the elements in this list are expected.</p></div> -<div class="paragraph"><p>The answer to this particular problem is simple. We need a -function that will parse the query string and apply constraints. -But this is not all, there is one other problem to be solved.</p></div> -<div class="paragraph"><p>The other problem is that for the user some keys are mandatory -and some are optional. Optional keys include the ones that -correspond to HTML checkboxes: if the key for one or more -checkbox is missing from the query string, we still want to -have an empty list in our map so we can easily match. Matching -maps is great, but not so much when values might be missing, -so we have to normalize this data a little.</p></div> -<div class="paragraph"><p>This problem is solved by allowing a default value. If the -key is missing and a default exists, set it. If no default -exists, then the key was mandatory and we want to crash.</p></div> -<div class="paragraph"><p>I therefore make a proposal for changing the query string -interface to three functions.</p></div> -<div class="paragraph"><p>The first function already exists, it is <code>cowboy_req:qs(Req)</code> -and it returns only the query string binary. No more Req returned.</p></div> -<div class="paragraph"><p>The second function is a renaming of <code>cowboy_req:qs_vals(Req)</code> -to something more explicit: <code>cowboy_req:parse_qs(Req)</code>. -The new name implies that a parsing operation is done. It was implicit -and cached before. It will be explicit and not cached anymore now. -Again, no more Req returned.</p></div> -<div class="paragraph"><p>The third function is the one I mentioned above. I think -the interface <code>cowboy_req:match_qs(Req, Fields)</code> is -most appropriate. It returns a normalized map that is the same -regardless of optional fields being provided with the request, -allowing for easy matching. It crashes if something went wrong. -Still no Req returned.</p></div> -<div class="paragraph"><p>I feel that this three function interface provides everything -one would need to comfortably write applications. You can get -low level and get the query string directly; you can get a list -of key/value binaries without any additional processing and do it -on your own; or you can get a processed map that contains Erlang -terms ready to be used.</p></div> -<div class="paragraph"><p>I strongly believe that by democratizing the constraints to -more than just bindings, but also to query string, cookies and -other key/values in Cowboy, we can allow the developer to quickly -and easily go from HTTP request to Erlang function calls. The -constraints are reusable functions that can serve as guards -against unwanted data, providing convenience in the process.</p></div> -<div class="paragraph"><p>Your handlers will not look like an endless series of calls -to get and convert the input data, they will instead be just -one call at the beginning followed by the actual application -logic, thanks to constraints and maps.</p></div> -<div class="listingblock"> -<div class="content"><!-- Generator: GNU source-highlight 3.1.8 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><tt><span style="font-weight: bold"><span style="color: #000000">handle</span></span>(<span style="color: #009900">Req</span>, <span style="color: #009900">State</span>) <span style="color: #990000">-&gt;</span> - #{<span style="color: #FF6600">name</span><span style="color: #990000">:=</span><span style="color: #009900">Name</span>, <span style="color: #FF6600">email</span><span style="color: #990000">:=</span><span style="color: #009900">Email</span>, <span style="color: #FF6600">choices</span><span style="color: #990000">:=</span><span style="color: #009900">ChoicesList</span>, <span style="color: #FF6600">remember_me</span><span style="color: #990000">:=</span><span style="color: #009900">RememberMe</span>} <span style="color: #990000">=</span> - <span style="font-weight: bold"><span style="color: #000000">cowboy_req:match_qs</span></span>(<span style="color: #009900">Req</span>, [ - <span style="font-weight: bold"><span style="color: #000080">name</span></span>, {<span style="color: #FF6600">email</span>, <span style="color: #FF6600">email</span>}, - {<span style="color: #FF6600">choices</span>, <span style="font-weight: bold"><span style="color: #0000FF">fun</span></span> <span style="font-weight: bold"><span style="color: #000000">check_choices</span></span><span style="color: #990000">/</span><span style="color: #993399">1</span>, []}, - {<span style="color: #FF6600">remember_me</span>, <span style="color: #FF6600">boolean</span>, <span style="color: #000080">false</span>}]), - <span style="font-weight: bold"><span style="color: #000000">save_choices</span></span>(<span style="color: #009900">Name</span>, <span style="color: #009900">Email</span>, <span style="color: #009900">ChoicesList</span>), - <span style="font-weight: bold"><span style="color: #0000FF">if</span></span> <span style="color: #009900">RememberMe</span> <span style="color: #990000">-&gt;</span> <span style="font-weight: bold"><span style="color: #000000">create_account</span></span>(<span style="color: #009900">Name</span>, <span style="color: #009900">Email</span>); <span style="color: #000080">true</span> <span style="color: #990000">-&gt;</span> <span style="color: #FF6600">ok</span> <span style="font-weight: bold"><span style="color: #0000FF">end</span></span>, - {<span style="color: #FF6600">ok</span>, <span style="color: #009900">Req</span>, <span style="color: #009900">State</span>}<span style="color: #990000">.</span> - -<span style="font-weight: bold"><span style="color: #000000">check_choices</span></span>(<span style="color: #990000">&lt;&lt;</span><span style="color: #FF0000">"blue"</span><span style="color: #990000">&gt;&gt;</span>) <span style="color: #990000">-&gt;</span> {<span style="color: #000080">true</span>, <span style="color: #FF6600">blue</span>}; -<span style="font-weight: bold"><span style="color: #000000">check_choices</span></span>(<span style="color: #990000">&lt;&lt;</span><span style="color: #FF0000">"red"</span><span style="color: #990000">&gt;&gt;</span>) <span style="color: #990000">-&gt;</span> {<span style="color: #000080">true</span>, <span style="color: #FF6600">red</span>}; -<span style="font-weight: bold"><span style="color: #000000">check_choices</span></span>(<span style="color: #990000">_</span>) <span style="color: #990000">-&gt;</span> <span style="color: #000080">false</span>;</tt></pre></div></div> -<div class="paragraph"><p>(Don&#8217;t look too closely at the structure yet.)</p></div> -<div class="paragraph"><p>As you can see in the above snippet, it becomes really easy -to go from query string to values. You can also use the map -directly as it is guaranteed to only contain the keys you -specified, any extra key is not returned.</p></div> -<div class="paragraph"><p>This would I believe be a huge step up as we can now -focus on writing applications instead of translating HTTP -calls. Cowboy can now take care of it.</p></div> -<div class="paragraph"><p>And to conclude, this also solves our duplicate keys -dilemma, as they now automatically become a list of binaries, -and this list is then checked against constraints that -will fail if they were not expecting a list. And in the -example above, it even converts the values to atoms for -easier manipulation.</p></div> -<div class="paragraph"><p>As usual, feedback is more than welcome, and I apologize -for the rocky structure of this post as it contains all the -thoughts that went into this rather than just the conclusion.</p></div> -- cgit v1.2.3