https://ninenines.eu/articles/ Recent content in Articles on Nine Nines Hugo -- gohugo.io en-us Tue, 25 Dec 2018 07:00:00 +0100 https://ninenines.eu/articles/merry-christmas-2018/ Tue, 25 Dec 2018 07:00:00 +0100 https://ninenines.eu/articles/merry-christmas-2018/ As the year 2018 ends it's time for a short recap and a look forward for the next year. Overall, more than half of all open tickets have been closed. Most tickets were opened since 2015 and I couldn't get to those, but now that pre-school started I have a lot more time! I'm aiming to keep the number of tickets below 100 across all my projects. Cowboy 2.x is now mature. https://ninenines.eu/articles/cowboy-2.6.0/ Mon, 19 Nov 2018 07:00:00 +0100 https://ninenines.eu/articles/cowboy-2.6.0/ Cowboy 2.6.0 has been released! Cowboy 2.6 greatly refactored the HTTP/2 code, a large part of which was moved to Cowlib and is now used by both the Cowboy server and the Gun client. A large number of tickets were also closed which resulted in many bugs fixed and many features and options added, although some of them are still experimental. Of note is the support for the PROXY protocol header built directly into Cowboy; the ability to use the sendfile tuple to send files while streaming a response body (for example you could build a tar file on the fly); and experimental support for range requests in the REST and static file handlers, including an automatic mode that lets you enable byte range requests to existing handlers with what's basically a one-liner. https://ninenines.eu/articles/ranch-1.7.0/ Wed, 14 Nov 2018 10:00:00 +0100 https://ninenines.eu/articles/ranch-1.7.0/ Ranch 1.7.0 has been released! This release adds built-in support for the PROXY protocol. The PROXY protocol is a simple and efficient way for proxies to transmit information about the client. While a third-party library already existed, it was not entirely compatible with the Ranch interface, in particular when socket active mode was involved. This new implementation fixes that and supports the full protocol with as little overhead as possible compared to normal operations: just one extra function call. https://ninenines.eu/articles/cowboy-2.5.0/ Wed, 03 Oct 2018 07:00:00 +0100 https://ninenines.eu/articles/cowboy-2.5.0/ Cowboy 2.5.0 has been released! Cowboy 2.5 focused on making the test suites pass. It is now possible to get all the Cowboy tests to pass successfully, at least on Linux and on the more recent Erlang/OTP versions. HTTP/1.1 has been improved with a fix for the TCP reset problem and the ability to stream a response body without using chunked transfer-encoding. Two functions have been added: cowboy_req:stream_events/3 encodes and streams one or more text/event-stream events, and cowboy_req:read_and_match_urlencoded_body/2,3 reads, parses and matches application/x-www-form-urlencoded request bodies. https://ninenines.eu/articles/gun-1.3.0/ Mon, 01 Oct 2018 10:00:00 +0100 https://ninenines.eu/articles/gun-1.3.0/ Gun 1.3.0 has been released! Gun is an HTTP/1.1, HTTP/2 and Websocket client for Erlang/OTP. This release improves the CONNECT support introduced in the previous version and adds built-in Websocket protocol negotiation. A complete list of changes can be found in the migration guide: Migrating from Gun 1.2 to 1.3. You can donate to this project via BountySource. These funds are used to pay for additional servers for testing. And healthy food. https://ninenines.eu/articles/gun-1.2.0/ Mon, 17 Sep 2018 10:00:00 +0100 https://ninenines.eu/articles/gun-1.2.0/ Gun 1.2.0 has been released! Gun is an HTTP/1.1, HTTP/2 and Websocket client for Erlang/OTP. Gun now supports issuing CONNECT requests to HTTP proxies in order to establish tunnels to origin servers. Gun can establish tunnels over one or more proxies as necessary. All existing protocols can be used inside the tunnel, including HTTP/1.1, HTTP/2 and Websocket over both TCP and TLS connections. Note that it is currently not possible to tunnel a TLS connection via an HTTPS proxy due to limitations in the current version of Erlang/OTP. https://ninenines.eu/articles/ranch-1.6.0/ Wed, 01 Aug 2018 10:00:00 +0100 https://ninenines.eu/articles/ranch-1.6.0/ Ranch 1.6.0 has been released! This release sees the introduction of a suspend/resume mechanism for the listeners which makes the listener close the listening socket and stop accepting new connections. Existing connections continue uninterrupted. This can be used to update the socket options of the listener, or to implement a graceful shutdown. To that end a function has also been added which allows waiting until connections reach a certain number. https://ninenines.eu/articles/gun-1.0.0/ Fri, 27 Jul 2018 10:00:00 +0100 https://ninenines.eu/articles/gun-1.0.0/ Gun 1.0.0 has been released! Gun is an HTTP/1.1, HTTP/2 and Websocket client for Erlang/OTP. Gun provides an asynchronous interface and will keep the connection open to the server, reconnecting as necessary. Gun has existed for many years as the test client for Cowboy and is now mature enough to receive a proper version. Gun is battle tested by customers and other users but is not the most well tested client there is. https://ninenines.eu/articles/asciideck/ Wed, 13 Jun 2018 07:00:00 +0100 https://ninenines.eu/articles/asciideck/ Asciideck is a new project I have been working on in my spare time that implements an Asciidoc parser and translation of Asciidoc documents into various output formats. The Asciideck parser returns an AST for the document. That AST can be further manipulated should it be necessary: for example you may need to rewrite some relative links if you are not keeping the same file directory structure as the original Asciidoc documents. https://ninenines.eu/articles/gun-1.0.0-rc.1/ Mon, 04 Jun 2018 10:00:00 +0100 https://ninenines.eu/articles/gun-1.0.0-rc.1/ Gun 1.0.0-rc.1 has been released! Gun is an HTTP/1.1, HTTP/2 and Websocket client for Erlang/OTP. Gun provides an asynchronous interface and will keep the connection open to the server, reconnecting as necessary. Gun has existed for many years as the test client for Cowboy and is now mature enough to receive a proper version. Gun is battle tested by customers and other users but is not the most well tested client there is. https://ninenines.eu/articles/cowboy-2.4.0/ Wed, 02 May 2018 10:00:00 +0100 https://ninenines.eu/articles/cowboy-2.4.0/ Cowboy 2.4.0 has been released! Numerous HTTP/2 options have been added to control the HTTP/2 SETTINGS and general behavior of HTTP/2 connections. The options for initial window sizes, maximum frame sizes or compression table sizes might be of interest for optimizing the performance of HTTP/2 connections. Experimental support for Websocket over HTTP/2 was added. Note that browsers do not currently support it. The only browser with partial support is Google Chrome 67 (dev build) started with a specific flag. https://ninenines.eu/articles/cowboy-2.3.0/ Wed, 04 Apr 2018 12:00:00 +0100 https://ninenines.eu/articles/cowboy-2.3.0/ Cowboy 2.3.0 has been released! This release focused on adding support for the functions from the sys module for introspecting Cowboy processes. Many bugs have also been fixed. A more complete list of changes can be found in the migration guide: Migrating from Cowboy 2.2 to 2.3. You can donate to this project via BountySource because I need to eat snacks when I write code. Thanks in advance! As usual, feedback is appreciated, and issues should be reported by opening a ticket. https://ninenines.eu/articles/cowboy-2.2.0/ Wed, 13 Dec 2017 12:00:00 +0100 https://ninenines.eu/articles/cowboy-2.2.0/ Cowboy 2.2.0 has been released! This release focused on adding features required for writing gRPC servers and on completing test suites for the core HTTP RFCs. The cowboy_req:stream_trailers/2 function has been added. It terminates the streamed response by adding some trailer field values. This feature is required for gRPC. The max_skip_body_length option was added. It controls how much of the request body we are willing to skip to get to the next request for HTTP/1. https://ninenines.eu/articles/cowboy-2.1.0/ Thu, 09 Nov 2017 12:00:00 +0100 https://ninenines.eu/articles/cowboy-2.1.0/ Cowboy 2.1.0 has been released! This release focused on adding features that were temporarily removed during the 2.0 release process: The client TLS certificate can now be obtained. The 100 Continue response is now sent automatically again when necessary. NEW: It is now possible to send informational responses (1XX) directly from user code via the cowboy_req:inform/2,3 functions. NEW: cowboy_rest handlers can now switch to any other type of handler from almost any callback. https://ninenines.eu/articles/cowboy-2.0.0/ Wed, 04 Oct 2017 12:00:00 +0100 https://ninenines.eu/articles/cowboy-2.0.0/ Cowboy 2.0.0 has been released! This is the new stable version of Cowboy. There are no new releases planned for the 1.x version of Cowboy. The highlights from the release are: HTTP/2 support! Websocket compression! Much simpler, cleaner interface. No more weird errors just because you discard the Req object. A new low-level interface that receives all events from every set of request and response. This replaces the awkward hooks from 1. https://ninenines.eu/articles/cowboy-2.0.0-rc.2/ Wed, 23 Aug 2017 18:00:00 +0100 https://ninenines.eu/articles/cowboy-2.0.0-rc.2/ Cowboy 2.0.0-rc.2 has been released! This is the new recommended version of Cowboy. Its API should not change before release. While you probably should not use it in production yet, many do successfully. Use at your own risk. This new version contains fixes for the following issues: HTTP/2 server push was using the wrong header compression context. HTTP/2 flow control could end up queueing data in the wrong order when resuming the sending of data. https://ninenines.eu/articles/cowboy-2.0.0-rc.1/ Mon, 24 Jul 2017 18:00:00 +0100 https://ninenines.eu/articles/cowboy-2.0.0-rc.1/ Cowboy 2.0.0-rc.1 has been released! This is the new recommended version of Cowboy. Its API should not change before release. While you probably should not use it in production yet, many do successfully. Use at your own risk. The plan is to have a new RC version every couple weeks until the summer ends or later if there are still blocking issues open. Only issues that can't be fixed without making breaking changes to the interface may block the release. https://ninenines.eu/articles/the-elephant-in-the-room/ Sun, 26 Mar 2017 00:00:00 +0100 https://ninenines.eu/articles/the-elephant-in-the-room/ Have you ever tried telling someone why they should use Erlang? You boast the smaller code size, the auto healing mechanisms, the distribution and they seem really excited. They wonder why they never heard about Erlang before. And then you show them what the code looks like. All excitement goes away. The smiles disappear. Their face starts becoming really serious. You lost them. You know you lost them. They comment on the syntax, or perhaps you do, already admitting defeat. https://ninenines.eu/articles/dont-let-it-crash/ Sun, 22 Jan 2017 00:00:00 +0100 https://ninenines.eu/articles/dont-let-it-crash/ We have a specific mindset when writing Erlang programs. We focus on the normal execution of the program and don't handle most of the errors that may occur. We sometimes call this normal execution the happy path. The general pattern behind writing only for the happy path, letting the VM catch errors (writing them to a log for future consumption) and then having a supervisor restart the processes that failed from a clean state, has a name. https://ninenines.eu/articles/cowboy-2.0.0-pre.4/ Tue, 03 Jan 2017 00:00:00 +0100 https://ninenines.eu/articles/cowboy-2.0.0-pre.4/ Cowboy 2.0.0-pre.4 has been released! This is the new recommended version of Cowboy. While I would not recommend putting it in production just yet, I do recommend you start writing new applications with this Cowboy version. The most significant changes in the pre-release are: A new architecture: there now is one process per connection and one process per request. This was done because HTTP/2 allows running requests concurrently. Stream handlers. https://ninenines.eu/articles/ranch-1.3/ Mon, 28 Nov 2016 00:00:00 +0100 https://ninenines.eu/articles/ranch-1.3/ Ranch 1.3.0 has been released! This release fixes a number of long standing issues and adds a small number of features: The ssl application has been added to the list of dependencies. If you don't need it, you can remove it automatically when fetching Ranch or when building the release. If you do need it, you will no longer have issues shutting down a node because of Ranch. The ranch:info/0 and ranch:procs/2 can be used to retrieve information about Ranch's state. https://ninenines.eu/articles/ml-archives/ Mon, 29 Aug 2016 00:00:00 +0100 https://ninenines.eu/articles/ml-archives/ The old mailing list archives have been added to the site, mainly for referencing purposes. The mailing list has been shut down and all personal information has been deleted. If you need help with a project, consider either opening a ticket on that project's issues tracker or going through the community channels (erlang-questions, #ninenines or #erlang on Freenode). Prefer tickets; often when people have issues it highlights an underlying problem in the project or its documentation. https://ninenines.eu/articles/website-update/ Sat, 02 Apr 2016 00:00:00 +0100 https://ninenines.eu/articles/website-update/ Last week-end I updated the Nine Nines website. I switched to Hugo. The site is now built from Asciidoc documents. You probably saw me switch to Asciidoc for documentation this past year. This is the natural conclusion to that story. The great thing is that with a little bit of Makefile magic I can just copy the documentation files into Hugo and poof, they appear on the website. I am very happy with that new setup. https://ninenines.eu/articles/erlanger-playbook-september-2015-update/ Wed, 02 Sep 2015 00:00:00 +0100 https://ninenines.eu/articles/erlanger-playbook-september-2015-update/ An update to The Erlanger Playbook is now available! The Erlanger Playbook is a book about software development using Erlang. It currently covers all areas from the conception, design, the writing of code, documentation and tests. The book is still a work in progress. Future topics will include refactoring, debugging and tracing, benchmarking, releases, community management (for open source projects). This update fixes a number of things and adds two chapters: IOlists and Erlang building blocks. https://ninenines.eu/articles/erlanger-playbook/ Thu, 18 Jun 2015 00:00:00 +0100 https://ninenines.eu/articles/erlanger-playbook/ I am proud to announce the pre-release of The Erlanger Playbook. The Erlanger Playbook is a book about software development using Erlang. It currently covers all areas from the conception, design, the writing of code, documentation and tests. The book is still a work in progress. Future topics will include refactoring, debugging and tracing, benchmarking, releases, community management (for open source projects). The following sections are currently available: About this book; Changelog; Future additions Erlang: Building blocks; Patterns Workflow: Think; Write; Stay productive Documentation: On documentation; Tutorials; User guide; Manual; README files Design: RESTful APIs; Lessons learned Code: Starting a project; Version control; Project structure; Code style; Best practices; Special processes; IOLists; The process dictionary Tests: On testing; Success typing analysis; Manual testing; Unit testing; Functional testing Selling Erlang: On persuasion; Don't let it crash Read a preview: Special processes https://ninenines.eu/articles/erlang-validate-utf8/ Fri, 06 Mar 2015 00:00:00 +0100 https://ninenines.eu/articles/erlang-validate-utf8/ Yesterday I pushed Websocket permessage-deflate to Cowboy master. I also pushed a change in the way the code validates UTF-8 data (required for text and close frames as per the spec). When looking into why the permessage-deflate tests in autobahntestsuite were taking such a long time, I found that autobahn is using an adaptation of the algorithm named Flexible and Economical UTF-8 Decoder. This is the C99 implementation: // Copyright (c) 2008-2009 Bjoern Hoehrmann <bjoern@hoehrmann. https://ninenines.eu/articles/on-open-source/ Fri, 05 Sep 2014 00:00:00 +0100 https://ninenines.eu/articles/on-open-source/ Last week I read a great article on contributing to open source by Alvaro Videla. He makes many great points and I am in agreement with most of it. This made me want to properly explain my point of view with regard to open source and contributions. Unlike most open source evangelism articles I will not talk about ideals or any of that crap, but rather my personal feelings and experience. https://ninenines.eu/articles/the-story-so-far/ Sat, 23 Aug 2014 00:00:00 +0100 https://ninenines.eu/articles/the-story-so-far/ As I am away from home with little to do (some call this a vacation) I wanted to reflect a little on the story so far, or how I arrived to Erlang and got to where I am now. The raw personal experience. It'll be an article that's more about social aspect, communities and marketing a project than technical considerations. As a period piece, it will also allow me to reflect on the evolution of Erlang in recent years. https://ninenines.eu/articles/cowboy2-qs/ Wed, 20 Aug 2014 00:00:00 +0100 https://ninenines.eu/articles/cowboy2-qs/ Now that Cowboy 1.0 is out, I can spend some of my time thinking about Cowboy 2.0 that will be released soon after Erlang/OTP 18.0. This entry discusses the proposed changes to query string handling in Cowboy. Cowboy 2.0 will respond to user wishes by simplifying the interface of the cowboy_req module. Users want two things: less juggling with the Req variable, and more maps. Maps is the only dynamic key/value data structure in Erlang that we can match directly to extract values, allowing users to greatly simplify their code as they don't need to call functions to do everything anymore. https://ninenines.eu/articles/january-2014-status/ Tue, 07 Jan 2014 00:00:00 +0100 https://ninenines.eu/articles/january-2014-status/ I will now be regularly writing posts about project status, plans and hopes for the future. Before that though, there's one important news to share. Until a year ago all development was financed through consulting and development services. This worked alright but too much time was spent doing things that didn't benefit the open source projects. And that didn't make me happy at all. Because I like being happy I stopped that for the most part and spent the year figuring things out, experimenting and discussing with people about it. https://ninenines.eu/articles/farwest-funded/ Thu, 27 Jun 2013 00:00:00 +0100 https://ninenines.eu/articles/farwest-funded/ This was a triumph! I'm making a note here: HUGE SUCCESS!! It's hard to overstate my satisfaction. Thanks to everyone who made this possible. If you have backed this fundraiser, and haven't provided your personal details yet, please do so quickly so that your rewards can be sent! I am hoping that we will be able to make good use of all that money. The details of the expenses will be published regularly on the 2013 Fundraiser wiki page, giving you full disclosure as to how your money is used. https://ninenines.eu/articles/erlang.mk-and-relx/ Tue, 28 May 2013 00:00:00 +0100 https://ninenines.eu/articles/erlang.mk-and-relx/ Building OTP releases has always been a difficult task. Tools like Reltool or Rebar have made this simpler, but it's no panacea. This article will show you an alternative and hopefully much simpler solution. There is two steps to building a release. First you need to build the various OTP applications you want to include in the release. Once done, you need to create the release itself, by including the Erlang runtime system alongside the applications, a boot script to start the node and all its applications, and some configuration files. https://ninenines.eu/articles/xerl-0.5-intermediate-module/ Mon, 25 Mar 2013 00:00:00 +0100 https://ninenines.eu/articles/xerl-0.5-intermediate-module/ Today we will start the work on the intermediate module that will be used to run the code for the expressions found in our file's body, replacing our interpreter. This is what we want to have when all the work is done: xerl -> tokens -> AST -> intermediate -> cerl Today we will perform this work only on the atomic integer expression however, so we will not build any module at the end. https://ninenines.eu/articles/xerl-0.4-expression-separator/ Fri, 01 Mar 2013 00:00:00 +0100 https://ninenines.eu/articles/xerl-0.4-expression-separator/ As promised we are adding an expression separator this time. This will be short and easy. In the tokenizer we only need to add a line recognizing the comma as a valid token. , : {token, {',', TokenLine}}. Then we need to change the following lines in the parser: exprs -> expr : ['$1']. exprs -> expr exprs : ['$1' | '$2']. And add a comma between the expressions on the second line: https://ninenines.eu/articles/erlang-scalability/ Mon, 18 Feb 2013 00:00:00 +0100 https://ninenines.eu/articles/erlang-scalability/ I would like to share some experience and theories on Erlang scalability. This will be in the form of a series of hints, which may or may not be accompanied with explanations as to why things are this way, or how they improve or reduce the scalability of a system. I will try to do my best to avoid giving falsehoods, even if that means a few things won't be explained. https://ninenines.eu/articles/xerl-0.3-atomic-expressions/ Mon, 18 Feb 2013 00:00:00 +0100 https://ninenines.eu/articles/xerl-0.3-atomic-expressions/ We will be adding atomic integer expressions to our language. These look as follow in Erlang: 42. And the result of this expression is of course 42. We will be running this expression at compile time, since we don't have the means to run code at runtime yet. This will of course result in no module being compiled, but that's OK, it will allow us to discuss a few important things we'll have to plan for later on. https://ninenines.eu/articles/xerl-0.2-two-modules/ Sun, 03 Feb 2013 00:00:00 +0100 https://ninenines.eu/articles/xerl-0.2-two-modules/ Everything is an expression. This sentence carries profound meaning. We will invoke it many times over the course of these articles. If everything is an expression, then the language shouldn't have any problem with me defining two modules in the same source file. mod first_module begin end mod second_module begin end Likewise, it shouldn't have any problem with me defining a module inside another module. mod out_module begin mod in_module begin end end Of course, in the context of the Erlang VM, these two snippets are equivalent; there is nothing preventing you from calling the in_module module from any other module. https://ninenines.eu/articles/xerl-0.1-empty-modules/ Wed, 30 Jan 2013 00:00:00 +0100 https://ninenines.eu/articles/xerl-0.1-empty-modules/ Let's build a programming language. I call it Xerl: eXtended ERLang. It'll be an occasion for us to learn a few things, especially me. Unlike in Erlang, in this language, everything is an expression. This means that modules and functions are expression, and indeed that you can have more than one module per file. We are just starting, so let's no go ahead of ourselves here. We'll begin with writing the code allowing us to compile an empty module. https://ninenines.eu/articles/ranch-ftp/ Wed, 14 Nov 2012 00:00:00 +0100 https://ninenines.eu/articles/ranch-ftp/ Last week I was speaking at the London Erlang Factory Lite where I presented a live demonstration of building an FTP server using Ranch. As there was no slide, you should use this article as a reference instead. The goal of this article is to showcase how to use Ranch for writing a network protocol implementation, how Ranch gets out of the way to let you write the code that matters, and the common techniques used when writing servers. https://ninenines.eu/articles/tictactoe/ Wed, 17 Oct 2012 00:00:00 +0100 https://ninenines.eu/articles/tictactoe/ Everyone knows Tic Tac Toe, right? Players choose either to be the Xs or the Os, then place their symbol on a 3x3 board one after another, trying to create a line of 3 of them. Writing an algorithm to check for victory sounds easy, right? It's easily tested, considering there's only 8 possible winning rows (3 horizontal, 3 vertical and 2 diagonal). In Erlang though, you probably wouldn't want an algorithm.