[[building]]
== Building
Erlang.mk can do a lot of things, but it is, first and
foremost, a build tool. In this chapter we will cover
the basics of building a project with Erlang.mk.
For most of this chapter, we will assume that you are
using a project xref:getting_started[generated by Erlang.mk].
=== How to build
To build a project, all you have to do is type `make`:
[source,bash]
$ make
It will work regardless of your project: OTP applications,
library applications, NIFs, port drivers or even releases.
Erlang.mk also automatically downloads and compiles the
dependencies for your project.
All this is possible thanks to a combination of configuration
and conventions. Most of the conventions come from Erlang/OTP
itself so any seasoned Erlang developers should feel right at
home.
Erlang.mk supports multi-threaded building. Parallel execution
is supported for all targets. To execute Erlang.mk in parallel
the `-j` option must be used:
[source,bash]
$ make -j8
The `MAKEFLAGS` variable can be used to enable parallel
building permanently on your system. It can be set in
your `.zshrc`, `.bashrc` or equivalent file.
[source,bash]
MAKEFLAGS="-j8"
=== What to build
Erlang.mk gives you control over three steps of the build
process, allowing you to do a partial build if needed.
A build has three phases: first any dependency is fetched
and built, then the project itself is built and finally a
release may be generated when applicable. A release is only
generated for projects specifically configured to do so.
Erlang.mk handles those three phases automatically when you
type `make`. But sometimes you just want to repeat one or
two of them.
The commands detailed in this section are most useful after
you have a successful build as they allow you to quickly
redo a step instead of going through everything. This is
especially useful for large projects or projects that end
up generating releases.
==== Application
You can build your application and dependencies without
generating a release by running the following command:
[source,bash]
$ make app
To build your application without touching dependencies
at all, you can use the `SKIP_DEPS` variable:
[source,bash]
$ make app SKIP_DEPS=1
This command is very useful if you have a lot of dependencies
and develop on a machine with slow file access, like the
Raspberry Pi and many other embedded devices.
Note that this command may fail if a required dependency
is missing.
==== Dependencies
You can build all dependencies, and nothing else, by
running the following command:
[source,bash]
$ make deps
This will fetch and compile all dependencies and their
dependencies, recursively.
xref:deps[Packages and dependencies] are covered
in the next chapter.
==== Release
It is not possible to build the release without at least
building the application itself, unless of course if there's
no application to begin with.
To generate the release, `make` will generally suffice with
a normal Erlang.mk. A separate target is however available,
and will take care of building the release, after building
the application and all dependencies:
[source,bash]
$ make rel
Consult the xref:relx[Releases] chapter for more
information about what releases are and how they are generated.
=== Application resource file
When building your application, Erlang.mk will generate the
http://www.erlang.org/doc/man/app.html[application resource file].
This file is mandatory for all Erlang applications and is
found in 'ebin/$(PROJECT).app'.
`PROJECT` is a variable defined in your Makefile and taken
from the name of the directory when Erlang.mk bootstraps
your project.
Erlang.mk can build the 'ebin/$(PROJECT).app' in two different
ways: from the configuration found in the Makefile, or from
the 'src/$(PROJECT).app.src' file.
==== Application configuration
Erlang.mk automatically fills the `PROJECT` variable when
bootstrapping a new project, but everything else is up to
you. None of the values are required to build your project,
although it is recommended to fill everything relevant to
your situation.
`PROJECT`::
The name of the OTP application or library.
`PROJECT_DESCRIPTION`::
Short description of the project.
`PROJECT_VERSION`::
Current version of the project.
`PROJECT_MOD`::
The application callback module.
`PROJECT_REGISTERED`::
List of the names of all registered processes.
`PROJECT_ENV`::
Configuration parameters used by the application.
`PROJECT_APP_EXTRA_KEYS`::
Other keys you want to add to the application `.app` file.
The variable content is written as-is to the `.app` file,
so be sure to format valid Erlang terms. For example:
`PROJECT_APP_EXTRA_KEYS = {maxT, 10000}, {start_phases, [...]}`.
`LOCAL_DEPS`::
List of Erlang/OTP applications this project depends on,
excluding `erts`, `kernel` and `stdlib`, or list of
dependencies local to this repository (in `APPS_DIR`).
`DEPS`::
List of applications this project depends on that need
to be fetched by Erlang.mk.
There's no need for quotes or anything. The relevant part of
the Cowboy Makefile follows, if you need an example:
[source,make]
----
PROJECT = cowboy
PROJECT_DESCRIPTION = Small, fast, modular HTTP server.
PROJECT_VERSION = 2.0.0-pre.2
PROJECT_REGISTERED = cowboy_clock
LOCAL_DEPS = crypto
DEPS = cowlib ranch
----
Any space before and after the value is dropped.
xref:deps[Dependencies] are covered in details in
the next chapter.
==== Application environment
The `PROJECT_ENV` variable is used to set the application
environment:
[source,make]
----
define PROJECT_ENV
[
{chips, [currysauce,{mushypeas,false}]},
{pizza, [{size,large},{toppings,[anchovies]}]}
]
endef
----
If you have a large set of environment variables, you may find it
easier to use a separate file. Do this by including the following
in your Makefile:
[source,make]
----
PROJECT_ENV_FILE = src/env.src
PROJECT_ENV = $(subst \n,$(newline),$(shell cat $(PROJECT_ENV_FILE) | sed -e 's/$$/\\n/;'))
ebin/$(PROJECT).app:: $(PROJECT_ENV_FILE)
----
The file has the same contents as the `PROJECT_ENV` variable:
[source,erlang]
----
[
{chips, [currysauce,{mushypeas,false}]},
{pizza, [{size,large},{toppings,[anchovies]}]}
]
----
==== Legacy method
The 'src/$(PROJECT).app.src' file is a legacy method of
building Erlang applications. It was introduced by the original
`rebar` build tool, of which Erlang.mk owes a great deal as it
is its main inspiration.
The '.app.src' file serves as a template to generate the '.app'
file. Erlang.mk will take it, fill in the `modules` value
dynamically, and save the result in 'ebin/$(PROJECT).app'.
When using this method, Erlang.mk cannot fill the `applications`
key from dependencies automatically, which means you need to
add them to Erlang.mk and to the '.app.src' at the same time,
duplicating the work.
If you really can't live without the legacy method, for one
reason or another, worry not; Erlang.mk will support it. And
if you need to create a new project that uses this method, you
just have to say so when bootstrapping:
[source,bash]
$ make -f erlang.mk bootstrap-lib LEGACY=1
=== Automatic application resource file values
When building the application resource file, Erlang.mk may
automatically add an `id` key with information about the
Git commit (if using Git), or an empty string otherwise.
It will only do this under specific conditions:
* The application was built as a dependency of another, or
* The legacy method was used, and the '.app.src' file contained `{id, "git"}`
This value is most useful when you need to help your users,
as it allows you to know which version they run exactly by
asking them to look in the file, or by running a simple
command on their production server:
[source,erlang]
----
1> application:get_all_key(cowboy).
{ok,[{description,"Small, fast, modular HTTP server."},
{id,"2.0.0-pre.2-25-g0ffde50-dirty"},
----
=== File formats
Erlang.mk supports a variety of different source file formats.
The following formats are supported natively:
[cols="<,3*^",options="header"]
|===
| Extension | Location | Description | Output
| .erl | src/ | Erlang source | ebin/*.beam
| .core | src/ | Core Erlang source | ebin/*.beam
| .xrl | src/ | Leex source | src/*.erl
| .yrl | src/ | Yecc source | src/*.erl
| .asn1 | asn1/ | ASN.1 files | include/*.hrl include/*.asn1db src/*.erl
| .mib | mibs/ | SNMP MIB files | include/*.hrl priv/mibs/*.bin
|===
Files are always searched recursively.
The build is ordered, so that files that generate Erlang source
files are run before, and the resulting Erlang source files are
then built normally.
In addition, Erlang.mk keeps track of header files (`.hrl`)
as described at the end of this chapter. It can also compile
C code, as described in the xref:ports[NIFs and port drivers]
chapter.
Erlang.mk also comes with plugins for the following formats:
[cols="<,3*^",options="header"]
|===
| Extension | Location | Description | Output
| .dtl | templates/ | Django templates | ebin/*.beam
| .proto | src/ | Protocol buffers | ebin/*.beam
|===
=== Compilation options
Erlang.mk provides a few variables that you can use to customize
the build process and the resulting files.
==== ERLC_OPTS
`ERLC_OPTS` can be used to pass some options to `erlc`, the Erlang
compiler. Erlang.mk does not restrict any option. Please refer to
the http://www.erlang.org/doc/man/erlc.html[erlc Manual] for the
full list.
By default, Erlang.mk will set the following options:
[source,make]
ERLC_OPTS = -Werror +debug_info +warn_export_vars +warn_shadow_vars +warn_obsolete_guard
In other words: warnings as errors, debug info (recommended) and
enable warnings for exported variables, shadow variables and
obsolete guard functions.
You can redefine this variable in your Makefile to change it
completely, either before or after including Erlang.mk:
[source,make]
ERLC_OPTS = +debug_info
You can also filter out some options from the defaults Erlang.mk
sets, by defining ERLC_OPTS after including Erlang.mk using the
`:=` operator.
[source,make]
----
include erlang.mk
ERLC_OPTS := $(filter-out -Werror,$(ERLC_OPTS))
----
==== ERLC_ASN1_OPTS
`ERLC_ASN1_OPTS` can be used to pass compiler options when compiling
ASN.1 files. Please refer to the
http://erlang.org/doc/man/asn1ct.html[asn1ct manual] for the full list.
By default, Erlang.mk will leave this empty.
You can redefine this variable in your Makefile.
Please see the `ERLC_OPTS` section for instructions.
==== ERLC_EXCLUDE
`ERLC_EXCLUDE` can be used to exclude some modules from the
compilation. It's there for handling special cases, you should
not normally need it.
To exclude a module, simply list it in the variable, either
before or after including Erlang.mk:
[source,make]
ERLC_EXCLUDE = cowboy_http2
=== Cold and hot builds
The first time you run `make`, Erlang.mk will build everything.
The second time you run `make`, and all subsequent times, Erlang.mk
will only rebuild what changed. Erlang.mk has been optimized for
this use case, as it is the most common during development.
Erlang.mk figures out what changed by using the dependency tracking
feature of Make. Make automatically rebuilds a target if one of its
dependency has changed (for example if a header file has changed,
all the source files that include it will be rebuilt), and Erlang.mk
leverages this feature to cut down on rebuild times.
Note that this applies only to building; some other features of
Erlang.mk will run every time they are called regardless of files
changed.
=== Dependency tracking
NOTE: This section is about the dependency tracking between files
inside your project, not application dependencies.
Erlang.mk keeps track of the dependencies between the different
files in your project. This information is kept in the '$(PROJECT).d'
file in your directory. It is generated if missing, and will be
generated again after every file change, by default.
Dependency tracking is what allows Erlang.mk to know when to
rebuild Erlang files when header files, behaviors or parse
transforms have changed. Erlang.mk also automatically keeps
track of which files should be compiled first, for example
when you have behaviors used by other modules in your project.
If your project is stable, you may want to disable generating
the dependency tracking file every time you compile. You can
do this by adding the following line to your 'Makefile':
[source,make]
NO_MAKEDEP ?= 1
As you can see, the snippet above uses `?=` instead of a
simple equal sign. This is to allow you to temporarily override
this value when you do make substantial changes to your project
(including a new header file, new module with dependencies, etc.)
and want to rebuild the dependency tracking file. You'll be
able to use the following command:
[source,bash]
$ NO_MAKEDEP= make
Otherwise, `make clean app` will of course force the
recompilation of your project.
Erlang.mk can also keep track of the source files generated
by other means, for example if you generate code from a data
file in your repository.
=== Generating Erlang source
Erlang.mk provides hooks at different stages of the build process.
When your goal is to generate Erlang source files, you can
add your own rules before or after the dependency tracking
file is generated. To do this, you would add your hook before
or after including the 'erlang.mk' file.
The easiest way is after:
[source,make]
----
PROJECT = example
include erlang.mk
$(PROJECT).d:: src/generated_mod.erl
src/generated_mod.erl:: gen-mod.sh
$(gen_verbose) ./gen-mod.sh $@
----
In this case we use `$(gen_verbose)` to hide the details of
the build by default. Erlang.mk will simply say what file
is it currently generating.
When using an external script to generate the Erlang source
file, it is recommended to depend on that script, so that
the source file gets generated again when the script gets
modified.
If for whatever reason you prefer to hook before including
Erlang.mk, don't forget to set the `.DEFAULT_GOAL` variable,
otherwise nothing will get built:
[source,make]
----
PROJECT = example
.DEFAULT_GOAL = all
$(PROJECT).d:: src/generated_mod.erl
include erlang.mk
src/generated_mod.erl:: gen-mod.sh
$(gen_verbose) ./gen-mod.sh $@
----
=== Cleaning
Building typically involves creating a lot of new files. Some
are reused in rebuilds, some are simply replaced. All can be
removed safely.
Erlang.mk provides two commands to remove them: `clean` and
`distclean`. `clean` removes all the intermediate files that
were created as a result of building, including the BEAM files,
the dependency tracking file and the generated documentation.
`distclean` removes these and more, including the downloaded
dependencies, Dialyzer's PLT file and the generated release,
putting your directory back to the state it was before you
started working on it.
To clean:
[source,bash]
$ make clean
Or distclean:
[source,bash]
$ make distclean
That is the question.
Note that Erlang.mk will automatically clean some files as
part of other targets, but it will never run `distclean` if
you don't explicitly use it.
