Docs on Nine Nines

Architecture

Mon, 01 Jan 0001 00:00:00 +0000

Cowboy is a lightweight HTTP server.

It is built on top of Ranch. Please see the Ranch guide for more +information.

One process per connection

It uses only one process per connection. The process where your +code runs is the process controlling the socket. Using one process +instead of two allows for lower memory usage.

Because there can be more than one request per connection with the +keepalive feature of HTTP/1.1, that means the same process will be +used to handle many requests.

Because of this, you are expected to make sure your process cleans +up before terminating the handling of the current request. This may +include cleaning up the process dictionary, timers, monitoring and +more.

Binaries

It uses binaries. Binaries are more efficient than lists for +representing strings because they take less memory space. Processing +performance can vary depending on the operation. Binaries are known +for generally getting a great boost if the code is compiled natively. +Please see the HiPE documentation for more details.

Date header

Because querying for the current date and time can be expensive, +Cowboy generates one Date header value every second, shares it +to all other processes, which then simply copy it in the response. +This allows compliance with HTTP/1.1 with no actual performance loss.

Max connections

By default the maximum number of active connections is set to a +generally accepted big enough number. This is meant to prevent having +too many processes performing potentially heavy work and slowing +everything else down, or taking up all the memory.

Disabling this feature, by setting the {max_connections, infinity} +protocol option, would give you greater performance when you are +only processing short-lived requests.

AsciiDoc documentation

Mon, 01 Jan 0001 00:00:00 +0000

Erlang.mk provides rules for generating documentation from +AsciiDoc files. It can automatically build a user guide PDF, +chunked HTML documentation and Unix manual pages.

Requirements

It is necessary to have AsciiDoc, +xsltproc and +dblatex installed on your +system for Erlang.mk to generate documentation from AsciiDoc sources.

Writing AsciiDoc documentation

AsciiDoc is a text document format for +writing notes, documentation, articles, books, ebooks, slideshows, +web pages, man pages and blogs. AsciiDoc files can be translated +to many formats including HTML, PDF, EPUB, man page.

The AsciiDoc user guide +describes the AsciiDoc syntax.

The Erlang.mk user guide +is written in AsciiDoc and can be used as an example. The entry +file is book.asciidoc.

Erlang.mk expects you to put your documentation in a specific +location. This is doc/src/guide/ for the user guide, and +doc/src/manual/ for the function reference. In the case of +the user guide, the entry point is always doc/src/guide/book.asciidoc.

For manual pages, it is good practice to use section 3 for +modules, and section 7 for the application itself.

Configuration

All of the AsciiDoc related configuration can be done directly +inside the files themselves.

Usage

To build all documentation:

$ make docs

To build only the AsciiDoc documentation:

$ make asciidoc

To build only the user guide:

$ make asciidoc-guide

To build only the manual:

$ make asciidoc-manual

To install man pages on Unix:

$ make install-docs

Erlang.mk allows customizing the installation path and sections +of the man pages to be installed. The MAN_INSTALL_PATH variable +defines where man pages will be installed. It defaults to +/usr/local/share/man. The MAN_SECTIONS variable defines +which manual sections are to be installed. It defaults to 3 7.

To install man pages to a custom location:

$ make install-docs MAN_INSTALL_PATH=/opt/share/man

Note that you may need to run the install commands using +sudo or equivalent if the location is not writeable by +your user.

Building

Mon, 01 Jan 0001 00:00:00 +0000

Erlang.mk can do a lot of things, but it is, first and +foremost, a build tool. In this chapter we will cover +the basics of building a project with Erlang.mk.

For most of this chapter, we will assume that you are +using a project generated by Erlang.mk.

How to build

To build a project, all you have to do is type make:

$ make

It will work regardless of your project: OTP applications, +library applications, NIFs, port drivers or even releases. +Erlang.mk also automatically downloads and compiles the +dependencies for your project.

All this is possible thanks to a combination of configuration +and conventions. Most of the conventions come from Erlang/OTP +itself so any seasoned Erlang developers should feel right at +home.

What to build

Erlang.mk gives you control over three steps of the build +process, allowing you to do a partial build if needed.

A build has three phases: first any dependency is fetched +and built, then the project itself is built and finally a +release may be generated when applicable. A release is only +generated for projects specifically configured to do so.

Erlang.mk handles those three phases automatically when you +type make. But sometimes you just want to repeat one or +two of them.

The commands detailed in this section are most useful after +you have a successful build as they allow you to quickly +redo a step instead of going through everything. This is +especially useful for large projects or projects that end +up generating releases.

Application

You can build your application and dependencies without +generating a release by running the following command:

$ make app

To build your application without touching dependencies +at all, you can use the SKIP_DEPS variable:

$ make app SKIP_DEPS=1

This command is very useful if you have a lot of dependencies +and develop on a machine with slow file access, like the +Raspberry Pi and many other embedded devices.

Note that this command may fail if a required dependency +is missing.

Dependencies

You can build all dependencies, and nothing else, by +running the following command:

$ make deps

This will fetch and compile all dependencies and their +dependencies, recursively.

Packages and dependencies are covered +in the next chapter.

Release

It is not possible to build the release without at least +building the application itself, unless of course if there’s +no application to begin with.

To generate the release, make will generally suffice with +a normal Erlang.mk. A separate target is however available, +and will take care of building the release, after building +the application and all dependencies:

$ make rel

Consult the Releases chapter for more +information about what releases are and how they are generated.

Application resource file

When building your application, Erlang.mk will generate the +application resource file. +This file is mandatory for all Erlang applications and is +found in ebin/$(PROJECT).app.

PROJECT is a variable defined in your Makefile and taken +from the name of the directory when Erlang.mk bootstraps +your project.

Erlang.mk can build the ebin/$(PROJECT).app in two different +ways: from the configuration found in the Makefile, or from +the src/$(PROJECT).app.src file.

Application configuration

Erlang.mk automatically fills the PROJECT variable when +bootstrapping a new project, but everything else is up to +you. None of the values are required to build your project, +although it is recommended to fill everything relevant to +your situation.

+PROJECT +: +
+ The name of the OTP application or library. +
+
+PROJECT_DESCRIPTION +: +
+ Short description of the project. +
+
+PROJECT_VERSION +: +
+ Current version of the project. +
+
+PROJECT_REGISTERED +: +
+ List of the names of all registered processes. +
+
+LOCAL_DEPS +: +
+ List of Erlang/OTP applications this project depends on, + excluding erts, kernel and stdlib, or list of + dependencies local to this repository (in APPS_DIR). +
+
+DEPS +: +
+ List of applications this project depends on that need + to be fetched by Erlang.mk. +
+

There’s no need for quotes or anything. The relevant part of +the Cowboy Makefile follows, if you need an example:

PROJECT = cowboy
+PROJECT_DESCRIPTION = Small, fast, modular HTTP server.
+PROJECT_VERSION = 2.0.0-pre.2
+PROJECT_REGISTERED = cowboy_clock
+
+LOCAL_DEPS = crypto
+DEPS = cowlib ranch

Any space before and after the value is dropped.

Dependencies are covered in details in +the next chapter.

Legacy method

The src/$(PROJECT).app.src file is a legacy method of +building Erlang applications. It was introduced by the original +rebar build tool, of which Erlang.mk owes a great deal as it +is its main inspiration.

The .app.src file serves as a template to generate the .app +file. Erlang.mk will take it, fill in the modules value +dynamically, and save the result in ebin/$(PROJECT).app.

When using this method, Erlang.mk cannot fill the applications +key from dependencies automatically, which means you need to +add them to Erlang.mk and to the .app.src at the same time, +duplicating the work.

If you really can’t live without the legacy method, for one +reason or another, worry not; Erlang.mk will support it. And +if you need to create a new project that uses this method, you +just have to say so when bootstrapping:

$ make -f erlang.mk bootstrap-lib LEGACY=1

Automatic application resource file values

When building the application resource file, Erlang.mk may +automatically add an id key with information about the +Git commit (if using Git), or an empty string otherwise. +It will only do this under specific conditions:

+
+The application was built as a dependency of another, or +
+
+
+The legacy method was used, and the .app.src file contained {id, "git"} +
+

This value is most useful when you need to help your users, +as it allows you to know which version they run exactly by +asking them to look in the file, or by running a simple +command on their production server:

1> application:get_all_key(cowboy).
+{ok,[{description,"Small, fast, modular HTTP server."},
+     {id,"2.0.0-pre.2-25-g0ffde50-dirty"},

File formats

Erlang.mk supports a variety of different source file formats. +The following formats are supported natively:

+ +++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Extension	Location	Description	Output
.erl	src/	Erlang source	ebin/*.beam
.core	src/	Core Erlang source	ebin/*.beam
.xrl	src/	Leex source	src/*.erl
.yrl	src/	Yecc source	src/*.erl
.asn1	asn1/	ASN.1 files	include/.hrl include/.asn1db src/*.erl
.mib	mibs/	SNMP MIB files	include/.hrl priv/mibs/.bin

Files are always searched recursively.

The build is ordered, so that files that generate Erlang source +files are run before, and the resulting Erlang source files are +then built normally.

In addition, Erlang.mk keeps track of header files (.hrl) +as described at the end of this chapter. It can also compile +C code, as described in the NIFs and port drivers +chapter.

Erlang.mk also comes with plugins for the following formats:

+ +++++ + + + + + + + + + + + + + + + + + + + + + +

Extension	Location	Description	Output
.dtl	templates/	Django templates	ebin/*.beam
.proto	src/	Protocol buffers	ebin/*.beam

Compilation options

Erlang.mk provides a few variables that you can use to customize +the build process and the resulting files.

ERLC_OPTS

ERLC_OPTS can be used to pass some options to erlc, the Erlang +compiler. Erlang.mk does not restrict any option. Please refer to +the erlc Manual for the +full list.

By default, Erlang.mk will set the following options:

ERLC_OPTS = -Werror +debug_info +warn_export_vars +warn_shadow_vars +warn_obsolete_guard

In other words: warnings as errors, debug info (recommended) and +enable warnings for exported variables, shadow variables and +obsolete guard functions.

You can redefine this variable in your Makefile to change it +completely, either before or after including Erlang.mk:

ERLC_OPTS = +debug_info

You can also filter out some options from the defaults Erlang.mk +sets, by defining ERLC_OPTS after including Erlang.mk using the +:= operator.

include erlang.mk
+
+ERLC_OPTS := $(filter-out -Werror,$(ERLC_OPTS))

ERLC_EXCLUDE

ERLC_EXCLUDE can be used to exclude some modules from the +compilation. It’s there for handling special cases, you should +not normally need it.

To exclude a module, simply list it in the variable, either +before or after including Erlang.mk:

ERLC_EXCLUDE = cowboy_http2

Cold and hot builds

The first time you run make, Erlang.mk will build everything.

The second time you run make, and all subsequent times, Erlang.mk +will only rebuild what changed. Erlang.mk has been optimized for +this use case, as it is the most common during development.

Erlang.mk figures out what changed by using the dependency tracking +feature of Make. Make automatically rebuilds a target if one of its +dependency has changed (for example if a header file has changed, +all the source files that include it will be rebuilt), and Erlang.mk +leverages this feature to cut down on rebuild times.

Note that this applies only to building; some other features of +Erlang.mk will run every time they are called regardless of files +changed.

Dependency tracking

+ + + +

Note

This section is about the dependency tracking between files +inside your project, not application dependencies.

Erlang.mk keeps track of the dependencies between the different +files in your project. This information is kept in the $(PROJECT).d +file in your directory. It is generated if missing, and will be +generated again after every file change, by default.

Dependency tracking is what allows Erlang.mk to know when to +rebuild Erlang files when header files, behaviors or parse +transforms have changed. Erlang.mk also automatically keeps +track of which files should be compiled first, for example +when you have behaviors used by other modules in your project.

If your project is stable, you may want to disable generating +the dependency tracking file every time you compile. You can +do this by adding the following line to your Makefile:

NO_MAKEDEP ?= 1

As you can see, the snippet above uses ?= instead of a +simple equal sign. This is to allow you to temporarily override +this value when you do make substantial changes to your project +(including a new header file, new module with dependencies, etc.) +and want to rebuild the dependency tracking file. You’ll be +able to use the following command:

$ NO_MAKEDEP= make

Otherwise, make clean app will of course force the +recompilation of your project.

Erlang.mk can also keep track of the source files generated +by other means, for example if you generate code from a data +file in your repository.

Generating Erlang source

Erlang.mk provides hooks at different stages of the build process. +When your goal is to generate Erlang source files, you can +add your own rules before or after the dependency tracking +file is generated. To do this, you would add your hook before +or after including the erlang.mk file.

The easiest way is after:

PROJECT = example
+
+include erlang.mk
+
+$(PROJECT).d:: src/generated_mod.erl
+
+src/generated_mod.erl:: gen-mod.sh
+        $(gen_verbose) ./gen-mod.sh $@

In this case we use $(gen_verbose) to hide the details of +the build by default. Erlang.mk will simply say what file +is it currently generating.

When using an external script to generate the Erlang source +file, it is recommended to depend on that script, so that +the source file gets generated again when the script gets +modified.

If for whatever reason you prefer to hook before including +Erlang.mk, don’t forget to set the .DEFAULT_GOAL variable, +otherwise nothing will get built:

PROJECT = example
+
+.DEFAULT_GOAL = all
+
+$(PROJECT).d:: src/generated_mod.erl
+
+include erlang.mk
+
+src/generated_mod.erl:: gen-mod.sh
+        $(gen_verbose) ./gen-mod.sh $@

Cleaning

Building typically involves creating a lot of new files. Some +are reused in rebuilds, some are simply replaced. All can be +removed safely.

Erlang.mk provides two commands to remove them: clean and +distclean. clean removes all the intermediate files that +were created as a result of building, including the BEAM files, +the dependency tracking file and the generated documentation. +distclean removes these and more, including the downloaded +dependencies, Dialyzer’s PLT file and the generated release, +putting your directory back to the state it was before you +started working on it.

To clean:

$ make clean

Or distclean:

$ make distclean

That is the question.

Note that Erlang.mk will automatically clean some files as +part of other targets, but it will never run distclean if +you don’t explicitly use it.

Code coverage

Mon, 01 Jan 0001 00:00:00 +0000

Placeholder chapter.

Common Test

Mon, 01 Jan 0001 00:00:00 +0000

Common Test is Erlang’s functional testing framework. +Erlang.mk automates the discovery and running of Common +Test suites.

Writing tests

The Common Test user guide +is the best place to learn how to write tests. Erlang.mk +requires that file names for test suites end with _SUITE.erl +and that the files be located in the $(TEST_DIR) directory. +This defaults to test/.

Configuration

The CT_OPTS variable allows you to set extra Common Test +options. Options are documented in the +Common Test user guide. +You can use it to set Common Test hooks, for example:

CT_OPTS = -ct_hooks cowboy_ct_hook

The CT_SUITES variable can be used to override what +Common Test suites Erlang.mk will be aware of. It does +not normally need to be set as Erlang.mk will find the +test suites automatically.

The name of the suite is the part before _SUITE.erl. +If the file is named http_SUITE.erl, the test suite +is http:

CT_SUITES = http ws

Usage

To run all tests (including Common Test):

$ make tests

To run all tests and static checks (including Common Test):

$ make check

You can also run Common Test separately:

$ make ct

Erlang.mk will create targets for all test suites it finds. +If you have a file named test/http_SUITE.erl, then the +target ct-http will run that specific test suite:

$ make ct-http

Erlang.mk provides a convenient way to run a specific +group or a specific test case within a specific group, +using the variable t. Note that this only applies to +suite-specific targets, like the ct-http example above.

To run all tests from the http_compress group in the +http_SUITE test suite, write:

$ make ct-http t=http_compress

Similarly, to run a specific test case in that group:

$ make ct-http t=http_compress:headers_dupe

To do the same against a multi-application repository, +you can use the -C option:

$ make -C apps/my_app ct-http t=my_group:my_case

Note that this also applies to dependencies. When using Cowboy +as a dependency, you can run the following directly:

$ make -C deps/cowboy ct-http t=http_compress

Finally, code coverage is available, +but covered in its own chapter.

Compatibility with other build tools

Mon, 01 Jan 0001 00:00:00 +0000

Erlang.mk tries its best to be compatible with the other Erlang +build tools. It can use dependencies written with other build +tools in mind, and can also make your projects usable by those +build tools as well. Erlang.mk is like the cool kid that gets +along with everybody.

In this chapter I will use the term Rebar project to refer +to a project built using Rebar 2, Rebar 3 or Mad. These three +build tools are very similar and share the same configuration +file.

Rebar projects as Erlang.mk dependencies

Erlang.mk comes with a feature called Autoload which will +use Rebar 2 to patch any Rebar project and make it compatible +with Erlang.mk. This feature essentially patches Rebar out +and adds a Makefile to the project that Erlang.mk can then +use for building:

Autoload is documented in more details in the +Packages and dependencies chapter.

Erlang.mk projects as Rebar dependencies

Erlang.mk projects can be made compatible with the Rebar family +of build tools pretty easily, as Erlang.mk will generate +all the files they require for building.

The Rebar family requires two files: a rebar.config file +containing compilation options and the list of dependencies, +and the application resource file, found either at +ebin/$(PROJECT).app or at src/$(PROJECT).app.src.

Rebar configuration

Erlang.mk comes with a target that generates a rebar.config +file when invoked:

$ make rebar.config

Careful! This will build the file even if it already existed +before.

To build this file, Erlang.mk uses information it finds in +the DEPS and ERLC_OPTS variables, among others. This +means that the Rebar family builds your project much the +same way as Erlang.mk.

Careful though! Different build tools have different fetching +strategies. If some applications provide differing dependencies, +they might be fetched differently by other build tools. Check +the upcoming Sanity check chapter to find out how to detect such +issues.

You can automatically generate this file when you build +your application, by making it a dependency of the app +target:

app:: rebar.config

Don’t forget to commit the file when it changes!

If you run into other issues, it’s probably because you use a +feature specific to Erlang.mk, like the cp fetch method. +It could also be that we forgot to handle something! Sorry. +We are of course interested to hear about any compatibility +problems you may have, just open a ticket!

Application resource file

Erlang.mk has two ways to generate an application resource +file: from the information found in the Makefile, or from +the information found in the src/$(PROJECT).app.src file. +Needless to say, if you have this file in your repository, +then you don’t need to worry about compatibility with other +build tools.

If you don’t, however, it’s not much harder. Every time +Erlang.mk will compile your application, it will produce +a new ebin/$(PROJECT).app file. Simply commit this file +when it changes. It will only change when you modify the +configuration, add or remove modules.

Connection

Mon, 01 Jan 0001 00:00:00 +0000

This chapter describes how to open, monitor and close +a connection using the Gun client.

Gun connections

Gun is designed with the SPDY and Websocket protocols in mind. +They are built for long-running connections that allow concurrent +exchange of data, either in the form of request/responses for +SPDY or in the form of messages for Websocket.

A Gun connection is an Erlang process that manages a socket to +a remote endpoint. This Gun connection is owned by a user +process that is called the owner of the connection, and is +managed by the supervision tree of the gun application.

The owner process communicates with the Gun connection +by calling functions from the module gun. All functions +perform their respective operations asynchronously. The Gun +connection will send Erlang messages to the owner process +whenever needed.

When the remote endpoint closes the connection, Gun attempts +to reconnect automatically.

Opening a new connection

The gun:open/{2,3} function must be used to open a connection.

Opening a connection to example.org on port 443

{ok, ConnPid} = gun:open("example.org", 443).

If the port given is 443, Gun will attempt to connect using +SSL. The protocol will be selected automatically using the +NPN extension for TLS. By default Gun supports SPDY/3.1, +SPDY/3 and HTTP/1.1 when connecting using SSL.

For any other port, Gun will attempt to connect using TCP +and will use the HTTP/1.1 protocol.

The transport and protocol used can be overriden using +options. The manual documents all available options.

Options can be provided as a third argument, and take the +form of a map.

Opening an SSL connection to example.org on port 8443

{ok, ConnPid} = gun:open("example.org", 8443, #{transport=>ssl}).

Waiting for the connection to be established

When Gun successfully connects to the server, it sends a +gun_up message with the protocol that has been selected +for the connection.

Gun provides the functions gun:await_up/{1,2,3} that wait +for the gun_up message. They can optionally take a monitor +reference and/or timeout value. If no monitor is provided, +one will be created for the duration of the function call.

Synchronous opening of a connection

{ok, ConnPid} = gun:open("example.org", 443),
+{ok, Protocol} = gun:await_up(ConnPid).

Handling connection loss

When the connection is lost, Gun will send a gun_down +message indicating the current protocol, the reason the +connection was lost and two list of stream references.

The first list indicates open streams that may have been +processed by the server. The second list indicates open +streams that the server did not process.

Monitoring the connection process

@todo Gun should detect the owner process being killed

Because software errors are unavoidable, it is important to +detect when the Gun process crashes. It is also important +to detect when it exits normally. Erlang provides two ways +to do that: links and monitors.

Gun leaves you the choice as to which one will be used. +However, if you use the gun:await/{2,3} or gun:await_body/{2,3} +functions, a monitor may be used for you to avoid getting +stuck waiting for a message that will never come.

If you choose to monitor yourself you can do it on a permanent +basis rather than on every message you will receive, saving +resources. Indeed, the gun:await/{3,4} and gun:await_body/{3,4} +functions both accept a monitor argument if you have one already.

Monitoring the connection process

{ok, ConnPid} = gun:open("example.org", 443).
+MRef = monitor(process, ConnPid).

This monitor reference can be kept and used until the connection +process exits.

Handling DOWN messages

receive
+        %% Receive Gun messages here...
+        {'DOWN', Mref, process, ConnPid, Reason} ->
+                error_logger:error_msg("Oops!"),
+                exit(Reason);
+end.

What to do when you receive a DOWN message is entirely up to you.

Closing the connection abruptly

The connection can be stopped abruptly at any time by calling +the gun:close/1 function.

Immediate closing of the connection

gun:close(ConnPid).

The process is stopped immediately without having a chance to +perform the protocol’s closing handshake, if any.

Closing the connection gracefully

The connection can also be stopped gracefully by calling the +gun:shutdown/1 function.

Graceful shutdown of the connection

gun:shutdown(ConnPid).

Gun will refuse any new requests or messages after you call +this function. It will however continue to send you messages +for existing streams until they are all completed.

For example if you performed a GET request just before calling +gun:shutdown/1, you will still receive the response before +Gun closes the connection.

If you set a monitor beforehand, you will receive a message +when the connection has been closed.

Constraints

Mon, 01 Jan 0001 00:00:00 +0000

Cowboy provides an optional constraints based validation feature +when interacting with user input.

Constraints are first used during routing. The router uses +constraints to more accurately match bound values, allowing +to create routes where a segment is an integer for example, +and rejecting the others.

Constraints are also used when performing a match operation +on input data, like the query string or cookies. There, a +default value can also be provided for optional values.

Finally, constraints can be used to not only validate input, +but also convert said input into proper Erlang terms, all in +one step.

Structure

Constraints are provided as a list of fields and for each +field a list of constraints for that field can be provided.

Fields are either the name of the field; the name and +one or more constraints; or the name, one or more constraints +and a default value.

When no default value is provided then the field is required. +Otherwise the default value is used.

All constraints for a field will be used to match its value +in the order they are given. If the value is modified by a +constraint, the next constraint receives the updated value.

Built-in constraints

+ +++ + + + + + + + + + + + + + + + +

Constraint	Description
int	Convert binary value to integer.
nonempty	Ensures the binary value is non-empty.

Custom constraint

In addition to the predefined constraints, Cowboy will accept +a fun. This fun must accept one argument and return one of +true, {true, NewValue} or false. The result indicates +whether the value matches the constraint, and if it does it +can optionally be modified. This allows converting the value +to a more appropriate Erlang term.

Note that constraint functions SHOULD be pure and MUST NOT crash.

Continuous integration

Mon, 01 Jan 0001 00:00:00 +0000

Placeholder chapter.

Contributing

Mon, 01 Jan 0001 00:00:00 +0000

You are welcome and encouraged to contribute.

This is how.

Priorities

From the most important to the least important:

+
+Bugs +
+
+
+Package issues/additions +
+
+
+Refactoring +
+
+
+Features +
+

Bugs

If you have found a bug, you should open a ticket. Include +everything relevant including the command you used, output, +a link to the code that triggers the issue, why you think +this is a bug, etc.

If you think you have found a bug but you are not sure, you +should open a ticket as previously explained.

If you have found a bug and you need it to be solved RIGHT +NOW, open a ticket as previously explained.

Once you have opened a ticket, be patient, try to answer +questions in a timely manner and confirm that the bug was +indeed fixed when it is.

If you can’t be patient, either try to solve the bug and +contribute the fix back or become a paying customer.

Code

The code is located in the core/*.mk and plugins/*.mk files. +The tests are located in the test/Makefile and test/*.mk files.

If you have a fix or a hack for a bug, you should open a +pull request. Any fix should include a test case that fails +before the fix and is working after.

If you have a test case that reproduces a bug, but no fix for +it, you should open a pull request.

Changes need to be tested with at least the make check +command. A specific test case can be tested using make check c=CASE +with CASE the name of the target to run. Output can be +modulated using the V variable, which is an integer +from 0 to 4. A typical use would be make check c=dialyzer V=3. +The value 4 is particular and shows expanded commands right +before they are executed.

To run tests in parallel, use the -j option. It is generally +a good idea to also use the -k option to run all tests even +if one fails. For example: make check -j 32 -k.

Some changes should be tested against all packages. Continue +reading for more details on testing them.

Packages

You can search existing packages using the make search q=STRING +command. This can be done both from an Erlang.mk project or +directly from the Erlang.mk repository.

Packages can be added to the index using the pkg_add.sh script.

$ git clone https://github.com/$YOURUSERNAME/erlang.mk
+$ cd erlang.mk
+$ ./pkg_add.sh cowboy git https://github.com/ninenines/cowboy 1.0.0
+  http://ninenines.eu "Small, fast and modular HTTP server."
+$ git push origin master

Before sending a pull request, you should test your package. +You can use the following command: make check p=PACKAGE, +where PACKAGE is the name of the package, for example +cowboy.

To test all packages, the make packages command can be used. +This can take a long time. Some packages will fail with certain +versions of Erlang, or if a prerequisite is missing from your system. +You can of course speed things up using the -j and -k flags.

After all packages have been tested, you can run the command +make summary to know what changed since the previous run.

Documentation

The documentation is always right.

If you think you have found a mistake in the documentation, +this is a bug. You can either open a ticket or send a pull +request.

To make sure that the documentation changes work, install +the listed Requirements on your system and +run make docs.

Feature requests

If you have an awesome idea or need something that Erlang.mk +doesn’t provide yet, open a ticket. Provide as much detail as +possible.

If you have code, great! Open a pull request as previously +explained.

If not, you can still improve your feature request by writing +the related documentation.

Cowboy Function Reference

Mon, 01 Jan 0001 00:00:00 +0000

Cowboy User Guide

Mon, 01 Jan 0001 00:00:00 +0000

Rationale

+
+The modern Web +
+
+
+Erlang and the Web +
+

Introduction

Configuration

+
+routing +
+
+
+Constraints +
+
+
+Static files +
+

Request and response

+
+Handlers +
+
+
+Loop handlers +
+
+
+The Req object +
+
+
+Reading the request body +
+
+
+Sending a response +
+
+
+Using cookies +
+
+
+Multipart +
+

REST

Websocket

+
+The Websocket protocol +
+
+
+Handling Websocket connections +
+

Internals

+
+Architecture +
+
+
+Dealing with broken clients +
+
+
+Middlewares +
+
+
+Sub protocols +
+
+
+Hooks +
+

Dealing with broken clients

Mon, 01 Jan 0001 00:00:00 +0000

There exists a very large number of implementations for the +HTTP protocol. Most widely used clients, like browsers, +follow the standard quite well, but others may not. In +particular custom enterprise clients tend to be very badly +written.

Cowboy tries to follow the standard as much as possible, +but is not trying to handle every possible special cases. +Instead Cowboy focuses on the cases reported in the wild, +on the public Web.

That means clients that ignore the HTTP standard completely +may fail to understand Cowboy’s responses. There are of +course workarounds. This chapter aims to cover them.

Lowercase headers

Cowboy converts all headers it receives to lowercase, and +similarly sends back headers all in lowercase. Some broken +HTTP clients have issues with that.

A simple way to solve this is to create an onresponse hook +that will format the header names with the expected case.

capitalize_hook(Status, Headers, Body, Req) ->
+    Headers2 = [{cowboy_bstr:capitalize_token(N), V}
+        || {N, V} <- Headers],
+    cowboy_req:reply(Status, Headers2, Body, Req).

Note that HTTP/2 clients do not have that particular issue +because the specification explicitly says all headers are +lowercase, unlike HTTP which allows any case but treats +them as case insensitive.

Camel-case headers

Sometimes it is desirable to keep the actual case used by +clients, for example when acting as a proxy between two broken +implementations. There is no easy solution for this other than +forking the project and editing the cowboy_protocol file +directly.

Chunked transfer-encoding

Sometimes an HTTP client advertises itself as HTTP/1.1 but +does not support chunked transfer-encoding. This is invalid +behavior, as HTTP/1.1 clients are required to support it.

A simple workaround exists in these cases. By changing the +Req object response state to waiting_stream, Cowboy will +understand that it must use the identity transfer-encoding +when replying, just like if it was an HTTP/1.0 client.

Req2 = cowboy_req:set(resp_state, waiting_stream).

Designing a resource handler

Mon, 01 Jan 0001 00:00:00 +0000

This chapter aims to provide you with a list of questions +you must answer in order to write a good resource handler. +It is meant to be usable as a step by step guide.

The service

Can the service become unavailable, and when it does, can +we detect it? For example, database connectivity problems +may be detected early. We may also have planned outages +of all or parts of the system. Implement the +service_available callback.

What HTTP methods does the service implement? Do we need +more than the standard OPTIONS, HEAD, GET, PUT, POST, +PATCH and DELETE? Are we not using one of those at all? +Implement the known_methods callback.

Type of resource handler

Am I writing a handler for a collection of resources, +or for a single resource?

The semantics for each of these are quite different. +You should not mix collection and single resource in +the same handler.

Collection handler

Skip this section if you are not doing a collection.

Is the collection hardcoded or dynamic? For example, +if you use the route /users for the collection of +users then the collection is hardcoded; if you use +/forums/:category for the collection of threads +then it isn’t. When the collection is hardcoded you +can safely assume the resource always exists.

What methods should I implement?

OPTIONS is used to get some information about the +collection. It is recommended to allow it even if you +do not implement it, as Cowboy has a default +implementation built-in.

HEAD and GET are used to retrieve the collection. +If you allow GET, also allow HEAD as there’s no extra +work required to make it work.

POST is used to create a new resource inside the +collection. Creating a resource by using POST on +the collection is useful when resources may be +created before knowing their URI, usually because +parts of it are generated dynamically. A common +case is some kind of auto incremented integer +identifier.

The next methods are more rarely allowed.

PUT is used to create a new collection (when +the collection isn’t hardcoded), or replace +the entire collection.

DELETE is used to delete the entire collection.

PATCH is used to modify the collection using +instructions given in the request body. A PATCH +operation is atomic. The PATCH operation may +be used for such things as reordering; adding, +modifying or deleting parts of the collection.

Single resource handler

Skip this section if you are doing a collection.

What methods should I implement?

OPTIONS is used to get some information about the +resource. It is recommended to allow it even if you +do not implement it, as Cowboy has a default +implementation built-in.

HEAD and GET are used to retrieve the resource. +If you allow GET, also allow HEAD as there’s no extra +work required to make it work.

POST is used to update the resource.

PUT is used to create a new resource (when it doesn’t +already exist) or replace the resource.

DELETE is used to delete the resource.

PATCH is used to modify the resource using +instructions given in the request body. A PATCH +operation is atomic. The PATCH operation may +be used for adding, removing or modifying specific +values in the resource.

The resource

Following the above discussion, implement the +allowed_methods callback.

Does the resource always exist? If it may not, implement +the resource_exists callback.

Do I need to authenticate the client before they can +access the resource? What authentication mechanisms +should I provide? This may include form-based, token-based +(in the URL or a cookie), HTTP basic, HTTP digest, +SSL certificate or any other form of authentication. +Implement the is_authorized callback.

Do I need fine-grained access control? How do I determine +that they are authorized access? Handle that in your +is_authorized callback.

Can access to a resource be forbidden regardless of access +being authorized? A simple example of that is censorship +of a resource. Implement the forbidden callback.

Are there any constraints on the length of the resource URI? +For example, the URI may be used as a key in storage and may +have a limit in length. Implement uri_too_long.

Representations

What media types do I provide? If text based, what charsets +are provided? What languages do I provide?

Implement the mandatory content_types_provided. Prefix +the callbacks with to_ for clarity. For example, to_html +or to_text.

Implement the languages_provided or charsets_provided +callbacks if applicable.

Is there any other header that may make the representation +of the resource vary? Implement the variances callback.

Depending on your choices for caching content, you may +want to implement one or more of the generate_etag, +last_modified and expires callbacks.

Do I want the user or user agent to actively choose a +representation available? Send a list of available +representations in the response body and implement +the multiple_choices callback.

Redirections

Do I need to keep track of what resources were deleted? +For example, you may have a mechanism where moving a +resource leaves a redirect link to its new location. +Implement the previously_existed callback.

Was the resource moved, and is the move temporary? If +it is explicitly temporary, for example due to maintenance, +implement the moved_temporarily callback. Otherwise, +implement the moved_permanently callback.

The request

Do we need to perform extra checks to make sure the request +is valid? Cowboy will do many checks when receiving the +request already, do we need more? Note that this only +applies to the request-line and headers of the request, +and not the body. Implement malformed_request.

May there be a request body? Will I know its size? +What’s the maximum size of the request body I’m willing +to accept? Implement valid_entity_length.

Finally, take a look at the sections corresponding to the +methods you are implementing.

OPTIONS method

Cowboy by default will send back a list of allowed methods. +Do I need to add more information to the response? Implement +the options method.

GET and HEAD methods

If you implement the methods GET and/or HEAD, you must +implement one ProvideResource callback for each +content-type returned by the content_types_provided +callback.

PUT, POST and PATCH methods

If you implement the methods PUT, POST and/or PATCH, +you must implement the content_types_accepted callback, +and one AcceptResource callback for each content-type +it returns. Prefix the AcceptResource callback names +with from_ for clarity. For example, from_html or +from_json.

Do we want to allow the POST method to create individual +resources directly through their URI (like PUT)? Implement +the allow_missing_post callback. It is recommended to +explicitly use PUT in these cases instead.

May there be conflicts when using PUT to create or replace +a resource? Do we want to make sure that two updates around +the same time are not cancelling one another? Implement the +is_conflict callback.

DELETE methods

If you implement the method DELETE, you must implement +the delete_resource callback.

When delete_resource returns, is the resource completely +removed from the server, including from any caching service? +If not, and/or if the deletion is asynchronous and we have +no way of knowing it has been completed yet, implement the +delete_completed callback.

Dialyzer

Mon, 01 Jan 0001 00:00:00 +0000

Dialyzer is a tool that will detect discrepancies in your +program. It does so using a technique known as success +typing analysis which has the advantage of providing no +false positives. Dialyzer is able to detect type errors, +dead code and more.

Erlang.mk provides a wrapper around Dialyzer.

How it works

Dialyzer requires a PLT file to work. The PLT file contains +the analysis information from all applications which are not +expected to change, or rarely do. These would be all the +dependencies of the application or applications you are +currently working on, including standard applications in +Erlang/OTP itself.

Dialyzer can generate this PLT file. Erlang.mk includes rules +to automatically generate the PLT file when it is missing.

Once the PLT file is generated, Dialyzer can perform the +analysis in record time.

Configuration

In a typical usage scenario, no variable needs to be set. +The defaults should be enough. Do note however that the +dependencies need to be set properly using the DEPS and +LOCAL_DEPS variables.

The DIALYZER_PLT file indicates where the PLT file will +be written to (and read from). By default this is +$(PROJECT).plt in the project’s directory. Note that +the DIALYZER_PLT variable is exported and is understood +by Dialyzer directly.

The PLT_APPS variable can be used to add additional +applications to the PLT. You can either list application +names or paths to these applications.

Erlang.mk defines two variables for specifying options +for the analysis: DIALYZER_DIRS and DIALYZER_OPTS. +The former one defines which directories should be part +of the analysis. The latter defines what extra warnings +Dialyzer should report.

Note that Erlang.mk enables the race condition warnings +by default. As it can take considerably large resources +to run, you may want to disable it on larger projects.

Usage

To perform an analysis, run the following command:

$ make dialyze

This will create the PLT file if it doesn’t exist.

The analysis will also be performed when you run the +following command, alongside tests:

$ make check

You can use the plt target to create the PLT file if +it doesn’t exist. This is normally not necessary as +Dialyzer creates it automatically.

The PLT file will be removed when you run make distclean.