diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/src/guide/book.asciidoc | 2 | ||||
| -rw-r--r-- | doc/src/guide/deps.asciidoc | 463 |
2 files changed, 465 insertions, 0 deletions
diff --git a/doc/src/guide/book.asciidoc b/doc/src/guide/book.asciidoc index 2056dad..8533997 100644 --- a/doc/src/guide/book.asciidoc +++ b/doc/src/guide/book.asciidoc @@ -18,6 +18,8 @@ include::limitations.asciidoc[Limitations] include::app.asciidoc[Building] +include::deps.asciidoc[Packages and dependencies] + include::compat.asciidoc[Compatibility with other build tools] = Advanced diff --git a/doc/src/guide/deps.asciidoc b/doc/src/guide/deps.asciidoc new file mode 100644 index 0000000..edf94c2 --- /dev/null +++ b/doc/src/guide/deps.asciidoc @@ -0,0 +1,463 @@ +== Packages and dependencies + +Erlang.mk can fetch and compile the dependencies that your +project requires. Erlang.mk improves upon the concepts +introduced by Rebar, so they should be familiar to many +seasoned Erlang developers. + +Erlang.mk is not a package manager, nor is it trying to be, +but it does include an index of Erlang packages to make +discovering useful projects easier. + +This chapter will explain how to use packages, add +dependencies to your project or bundle them directly +in a single repository. + +=== Searching packages + +Erlang.mk gives you access to nearly 500 packages, with more +being added regularly. + +To find a package, search for it: + +[source,bash] +$ make search q=pool + +This will return all packages matching this word, like worker +pool and acceptor pool projects. + +You can also list everything and use regular command line +tools to find what you need, for example: + +[source,bash] +$ make search | less + +// @todo Simplify adding packages, add a new chapter explaining +// everything, then link to this new chapter from here. + +=== Adding dependencies to your project + +Once you find the package you need, adding it as a dependency +to your project is a one-liner: + +[source,make] +DEPS = cowboy + +And that's it! The next time you run `make`, Erlang.mk will +fetch and compile Cowboy. Erlang.mk will also ensure Cowboy +is available whenever you use the shell, run tests and any +other operations. + +Erlang.mk will fill in the application resource file with +all applications found in `DEPS`. But not all dependencies +are Erlang applications, and not all dependencies need to +be a runtime dependency. That's where the `BUILD_DEPS` +variable comes in: it works just like `DEPS`, except the +dependencies listed there will not be added as runtime +dependencies. + +For example, you could add a parse transform project like +this to make it available only at build time: + +[source,make] +BUILD_DEPS = erlando + +Or you could depend on a C project directly, if you are +building a NIF: + +[source,make] +BUILD_DEPS = leveldb +dep_leveldb = git https://github.com/basho/leveldb 2.1.3 + +This dependency will be built before your application, so +you could easily copy the resulting shared file into your +'priv/' directory as part of the build process. More information +about that in the link:ports.asciidoc[NIFs and port drivers] +chapter. + +Another variable, `LOCAL_DEPS`, allows specifying runtime +dependencies which are part of Erlang/OTP itself, but also +dependencies that are included in the repository. Since they +are already on your system, there is no need to fetch them. +Do note that there is no way to choose the version, the +application used will be the one already on your system. + +You could depend on the Crypto application, for example: + +[source,make] +LOCAL_DEPS = crypto + +Erlang.mk comes with additional types of dependencies. +It has `TEST_DEPS` for dependencies used only for testing: + +[source,make] +TEST_DEPS = ct_helper +dep_ct_helper = git https://github.com/ninenines/ct_helper master + +`DOC_DEPS` for dependencies used only when building documentation: + +[source,make] +DOC_DEPS = edown + +`REL_DEPS` for dependencies required to build the release, +or to include extra applications in the release: + +[source,make] +REL_DEPS = recon + +And `SHELL_DEPS` for dependencies to make available when running +the `make shell` command: + +[source,make] +SHELL_DEPS = tddreloader + +All these will be documented in more details in their respective +chapters. + +==== Modifying the dependency source or version + +By default, Erlang.mk will look into its package index to +find the project you are looking for, if you only provide +its name. This is this case: + +[source,make] +DEPS = cowboy + +If you need a different version, you need to define another +variable. There are two ways to do this, each being useful +for different reasons. + +If you simply want to change the commit number, all you +need to do is to define the `dep_$(DEP_NAME)_commit` +variable. In the case of Cowboy, this would look like this: + +[source,make] +DEPS = cowboy +dep_cowboy_commit = 2.0.0-pre.2 + +Erlang.mk will use the package index to get all information +about Cowboy, except the commit number which will be overriden. + +If you need to set the fetch method or repository information +too, for example because you want to use your own fork, or +simply because the project is missing from the index, you +can define the `dep_$(DEP_NAME)` variable with everything: + +[source,make] +DEPS = cowboy +dep_cowboy = git https://github.com/essen/cowboy 2.0.0-pre.2 + +This will fetch Cowboy from your fork at the given commit. + +==== Fetch methods + +Erlang.mk comes with a number of different fetch methods. +You can fetch from Git, Mercurial, SVN, to name a few. +There are fetch methods that will work everywhere, and +fetch methods that will only work in a given environment. + +The following table lists all existing methods: + +[cols="<,3*^",options="header"] +|=== +| Name | Format | Description +| git | git repo commit | Clone the Git repository and checkout the given version +| hg | hg repo commit | Clone the Mercurial repository and update to the given version +| svn | svn repo | Checkout the given SVN repository +| cp | cp path/to/repo | Recursively copy a local directory +| hex | hex version | Download the given project version from hex.pm +| fail | N/A | Always fail, reserved for internal use +| legacy | N/A | Legacy Erlang.mk fetcher, reserved for internal use +|=== + +The `git` and `hg` methods both have a repository and commit. +You can use any valid commit, tag or branch in that repository +for the commit value. + +For example, to fetch Cowboy with tag 2.0.0-pre.2 from Git: + +[source,make] +dep_cowboy = git https://github.com/ninenines/cowboy 2.0.0-pre.2 + +Or to fetch Ehsa tag 4.0.3 from Mercurial: + +[source,make] +dep_ehsa = hg https://bitbucket.org/a12n/ehsa 4.0.3 + +The `svn` method only has a repository value, but that's +simply because the SVN repository URL can also contain +the path and commit. + +This would fetch an example project from the trunk: + +[source,make] +dep_ex1 = svn https://example.com/svn/trunk/project/ex1 + +And this would fetch a separate example project from a +specific commit: + +[source,make] +dep_ex2 = svn svn://example.com/svn/branches/erlang-proj/ex2@264 + +You can copy a directory from your machine using the `cp` method. +It only takes the path to copy from: + +[source,make] +dep_cowboy = cp $(HOME)/ninenines/cowboy + +Finally, you can use a package from the +link:https://hex.pm/[Hex repository]: + +[source,make] +dep_cowboy = hex 1.0.3 + +==== Custom fetch methods + +If none of the existing methods fit your use, you can simply +define your own. Erlang.mk will consider all variables that +are named as `dep_fetch_$(METHOD)` to be available fetch +methods. You can do anything inside this variable, as long +as you create a folder named '$(DEPS_DIR)/$(call dep_name,$1)'. +Or in layman terms, if your dependency is Cowboy, this would +become 'deps/cowboy'. + +To give an example, this is what the Git method does: + +[source,make] +---- +define dep_fetch_git + git clone -q -n -- $(call dep_repo,$1) $(DEPS_DIR)/$(call dep_name,$1); \ + cd $(DEPS_DIR)/$(call dep_name,$1) && git checkout -q $(call dep_commit,$1); +endef +---- + +Note that, like dependency information, this custom fetch method +must be written before including 'erlang.mk'. + +=== How deps are fetched and built + +The order in which dependencies are fetched and built is well +defined. This means that Erlang.mk will get the same applications +regardless of the command or options being used. + +In tree traversal terms, where the list of dependencies is a +tree, Erlang.mk fetches everything using the pre-order traversal +method. The steps can be summarized like this, starting from +the root application: + +. Fetch all dependencies for the application +. Build first dependency +. Build Nth dependency +. Build last dependency + +Every time a dependency is built, these same steps are followed, +recursively. + +Do note that the first step, fetching all dependencies of +an application, is not guaranteed to be ordered. The reason +for this is that it is not possible to have the same dependency +listed twice in a single application, and therefore there can +be no conflicts. Remember, this step only fetches, at no point +are different applications built in parallel. + +What about conflicts between the dependencies of different +applications? Simple. Since builds are ordered, this means +that the first version of an application that is fetched +will be the one that wins. + +This means that if project A depends on projects B and C, +in this order, and that both B and C depend on a different +version of D, it will always be B's version of D that wins, +because we fetch the dependencies of B before fetching +those from C. + +Similarly, if project A depends on projects B, C and D, +regardless of the order, and A, B and C depend on a +different version of D, it will always be A's version +that wins, because we fetch all dependencies of A before +fetching those from B or C. + +=== Ignoring unwanted dependencies + +Sometimes, you may want to ignore dependencies entirely. +Not even fetch them. You may want to do this because a +project you depend on depends on an application you do +not need (like a dependency for building documentation +or testing). Or maybe the dependency is already installed +on your system. + +To ignore a dependency, simply add it to the `IGNORE_DEPS` +variable: + +[source,make] +IGNORE_DEPS += edown proper + +This will only ignore dependencies that are needed for +building. It is therefore safe to write: + +[source,make] +IGNORE_DEPS += edown proper +TEST_DEPS = proper + +The PropEr application will be fetched as intended when +running `make tests` or `make check`. It will however +not be fetched when running `make` or `make deps`. + +=== Dependencies directory + +Dependencies are fetched in '$(DEPS_DIR)'. By default this is +the 'deps' directory. You can change this default, but you +should only do so if it was not defined previously. Erlang.mk +uses this variable to tell dependencies where to fetch their +own dependencies. + +You will therefore need to use `?=` instead of `=`. Of course, +if you know you will never use this project as a dependency, +`=` will work. But to avoid it biting you later on, do this: + +[source,make] +DEPS_DIR ?= $(CURDIR)/libs + +The `$(CURDIR)` part is important, otherwise dependencies of +dependencies will be fetched in the wrong directory. + +Erlang.mk will also export the `REBAR_DEPS_DIR` variable for +compatibility with Rebar build tools, as long as they are +recent enough. + +=== Dependencies local to the repository + +In addition to the dependencies that are fetched, Erlang.mk +also allows you to have dependencies local to your repository. +This kind of layout is sometimes called multi-application +repositories, or repositories with multiple applications. + +They work exactly the same as remote dependencies, except: + +* They are not fetched +* They are not autopatched +* They are not deleted on `make distclean` +* They are not automatically added to the application resource file + +To properly fill the application resource file, you will +need to define the `LOCAL_DEPS` variable for each relevant +application, the same as for OTP applications. + +If there is a conflict between a local dependency and a +remote dependency, then the local dependency always wins; +an error will be triggered when trying to fetch the +conflicting remote dependency. + +To start using dependencies local to the repository, simply +create a folder named '$(APPS_DIR)'. By default, this folder +is the 'apps/' directory. + +You can use Erlang.mk to bootstrap local dependencies by +using the command `make new-app` or `make new-lib`. This +command will create the necessary directories and bootstrap +the application. + +For example, to create a full fledged OTP application as +a local dependency: + +[source,bash] +$ make new-app in=webchat + +Or, the same as an OTP library: + +[source,bash] +$ make new-lib in=webchat + +Templates also work with local dependencies, from the root +directory of the project. You do need however to tell +Erlang.mk to create the files in the correct application: + +[source,bash] +$ make new t=gen_server n=my_server in=webchat + +=== Repositories with no application at the root level + +It's possible to use Erlang.mk with only applications in +'$(APPS_DIR)', and nothing at the root of the repository. +Just create a folder, put the 'erlang.mk' file in it, +write a Makefile that includes it, and start creating +your applications. + +Similarly, it's possible to have a repository with only +dependencies found in '$(DEPS_DIR)'. You just need to +create a Makefile and specify the dependencies you want. +This allows you to create a repository for handling the +building of releases, for example. + +=== Autopatch + +Erlang.mk will automatically patch all the dependencies it +fetches. It needs to do this to ensure that the dependencies +become compatible with not only Erlang.mk, but also with +the version of Erlang.mk that is currently used. + +When fetching a dependency, the following operations are +performed: + +* Fetch the dependency using the configured fetch method +* If it contains a 'configure.ac' or 'configure.in' file, run `autoreconf -Wall -vif -I m4` +* If it contains a 'configure' script, run it +* Run autopatch on the project + +Autopatch first checks if there is any project-specific patch +enabled. There are currently two: `RABBITMQ_CLIENT_PATCH` for +the `amqp_client` dependency, and `RABBITMQ_SERVER_PATCH` for +the `rabbit` dependency. These are needed only for RabbitMQ +versions before 3.6.0 (assuming you are using upstream RabbitMQ, +and not a fork). + +Otherwise, autopatch performs different operations depending +on the kind of project it finds the dependency to be. + +* Rebar projects are automatically converted to use Erlang.mk +as their build tool. This essentially patches Rebar out, and +fixes and converts the project to be compatible with Erlang.mk. + +* Erlang.mk projects have their 'erlang.mk' file redirect to +the top-level project's Erlang.mk. This is to ensure that +functionality works across all dependencies, even if the +dependency's Erlang.mk is outdated. + +* Other Erlang projects get a small Erlang.mk Makefile +generated automatically. + +* Projects with no source directory and no Makefile get an +empty Makefile generated, for compatibility purposes. + +* Other projects with no Makefile are left untouched. + +You can disable the replacing of the 'erlang.mk' file by +defining the `NO_AUTOPATCH_ERLANG_MK` variable: + +[source,make] +NO_AUTOPATCH_ERLANG_MK = 1 + +You can also disable autopatch entirely for a few select +projects using the `NO_AUTOPATCH` variable: + +[source,make] +NO_AUTOPATCH = cowboy ranch cowlib + +=== Skipping deps + +It is possible to temporarily skip all dependency operations. +This is done by defining the `SKIP_DEPS` variable. Use cases +include being somewhere with no connection to download them, +or perhaps a peculiar setup. + +A typical usage would be: + +[source,bash] +$ make SKIP_DEPS=1 + +When the variable is defined: + +* Dependencies will not be compiled or downloaded when required +* The dependency directory '$(DEPS_DIR)' will not be removed on `make distclean` + +This variable only applies to remote dependencies. |