diff options
Diffstat (limited to 'HOWTO')
| -rw-r--r-- | HOWTO/BENCHMARKS.md | 73 | ||||
| -rw-r--r-- | HOWTO/BOOTSTRAP.md | 61 | ||||
| -rw-r--r-- | HOWTO/DTRACE.md | 393 | ||||
| -rw-r--r-- | HOWTO/INSTALL-CROSS.md | 560 | ||||
| -rw-r--r-- | HOWTO/INSTALL-WIN32.md | 1047 | ||||
| -rw-r--r-- | HOWTO/INSTALL.md | 843 | ||||
| -rw-r--r-- | HOWTO/MARKDOWN.md | 266 | ||||
| -rw-r--r-- | HOWTO/SYSTEMTAP.md | 75 | ||||
| -rw-r--r-- | HOWTO/TESTING.md | 150 |
9 files changed, 3468 insertions, 0 deletions
diff --git a/HOWTO/BENCHMARKS.md b/HOWTO/BENCHMARKS.md new file mode 100644 index 0000000000..28a590dfd2 --- /dev/null +++ b/HOWTO/BENCHMARKS.md @@ -0,0 +1,73 @@ +Benchmarking Erlang/OTP +======================= + +The Erlang/OTP source tree contains a number of benchmarks. The same framework +is used to run these benchmarks as is used to run tests. Therefore in order to +run benchmarks you have to [release the tests][] just as you normally would. + +Note that many of these benchmarks were developed to test a specific feature +under a specific setting. We strive to keep the benchmarks up-to-date, but alas +time is not an endless resource so some benchmarks will be outdated and +irrelevant. + +Running the benchmarks +---------------------- + +As with testing, `ts` is used to run the benchmarks. Before running any +benchmarks you have to [install the tests][]. To get a listing of all +benchmarks you have available call `ts:benchmarks()`. + +To run all benchmarks call `ts:bench()`. This will run all benchmarks using +the emulator which is in your `$PATH` (Note that this does not have to be the +same as from which the benchmarks were built from). All the results of the +benchmarks are put in a folder in `$TESTROOT/test_server/` called +`YYYY_MO_DDTHH_MI_SS`. + +Each benchmark is run multiple times and the data for all runs is collected in +the files within the benchmark folder. All benchmarks are written so that +higher values are better. + +Writing benchmarks +------------------ + +Benchmarks are just normal testcases in Common Test suites. They are marked as +benchmarks by being included in the `AppName_bench.spec` which is located in +`lib/AppName/test/` for the applications which have benchmarks. Note that you +might want to add a skip clause to `AppName.spec` for the benchmarks if you do +not want them to be run in the nightly tests. + +Results of benchmarks are sent using the ct_event mechanism and automatically +collected and formatted by ts. + + ct_event:notify( + #event{name = benchmark_data, + data = [{value,TPS}]}). + +The application, suite and testcase associated with the value is automatically +detected. If you want to supply your own you can include `suite` andor `name` +with the data. i.e. + + ct_event:notify( + #event{name = benchmark_data, + data = [{suite,"erts_bench"}, + {name,"ets_transactions_per_sec"}, + {value,TPS}]}). + +The reason for using the internal ct_event and not ct is because the benchmark +code has to be backwards compatible with at least R14. + +The value which is reported should be as raw as possible. i.e. you should not +do any averaging of the value before reporting. The tools we use to collect the +benchmark data over time will do averages, means, stddev and more with the data. +So the more data which is sent using `ct_event` the better. + +Viewing benchmarks +------------------ + +At the moment of writing this HOWTO the tool for viewing benchmark results is +not available as opensource. This will hopefully change in the near future. + + + [release the tests]: TESTING.md#releasing-tests + [install the tests]: TESTING.md#configuring-the-test-environment + diff --git a/HOWTO/BOOTSTRAP.md b/HOWTO/BOOTSTRAP.md new file mode 100644 index 0000000000..59e31165cf --- /dev/null +++ b/HOWTO/BOOTSTRAP.md @@ -0,0 +1,61 @@ +Notes about prebuilt beam files under version control +===================================================== + +This information applies mostly to developers, some parts only +to developers of the main branch i.e. Ericsson and HiPE personel. + +There are two types of derived code under version control, namely: + +primary bootstrap - Resides in the `$ERL_TOP/bootstrap/{lib,bin}` directories. +preloaded code - Resides in the `$ERL_TOP/erts/preloaded` directory. + +Primary bootstrap +----------------- + +The two types of version controlled code are fundamentally +different. The primary bootstrap is code compiled from source files in +the lib/{kernel,stdlib,compiler} (and header files from +lib/orber/include. They are checked in in the version control system +to make it possible to build directly from the code base tree without +the need for an earlier version of the compiler. When a new version of +OTP is released, these files are updated manually (or rather, by using +the `$ERL_TOP/otp_build` script) and checked in. The files can also be +updated due to changes in the compiler during the development +process. The primary bootstrap is always updated as a separate +deliberate process, never during a normal development build. + +If a prebuilt open source version of erlang is used, the directory +bootstrap initially does not contain any beam files, the directory is +instead populated by copying beam files from the +`$ERL_TOP/lib/{kernel,stdlib,compiler}/ebin` directories. This +construction is to save space in the distribution, but the result +would be the same. Open source developers need not provide patches for +the precompiled beam files in the primary bootstrap, the bootstrap +update is always performed by the main developers. + +Preloaded code +-------------- + +The directory `$ERL_TOP/preloaded` contains both src and ebin +subdirectories. The preloaded code is compiled into the virtual +machine and always present. When compiling the virtual machine, those +beam files need to be present and they are considered a part of the +virtual machine rather than a part of the kernel application. When +preloaded files are to be updated, the source code is built using a +special Makefile in the `$ERL_TOP/preloaded/src` directory, which +creates beam files in the same directory. When they seem to compile +successfully, they can be used in an emulator build by being copied +to the ebin directory. `otp_build update_preloaded` can be used to +ease the process (there are also similar targets in the +`$ERL_TOP/preloaded/src/Makefile`). + +In prebuilt open source distributions, these beam files are also +present, but to update them one might need to change permission on the +`$ERL_TOP/preloaded/ebin` directory, then build and then manually copy +the beam files from the source directory to ../ebin. If patches are +created that involve the source files used to build preloaded code, +always note this specially as the preloaded/ebin directory needs +updating, or provide the new derived files in the patch or as complete +binaries. + +/Patrik, OTP diff --git a/HOWTO/DTRACE.md b/HOWTO/DTRACE.md new file mode 100644 index 0000000000..8fa2fd9d50 --- /dev/null +++ b/HOWTO/DTRACE.md @@ -0,0 +1,393 @@ +DTrace and Erlang/OTP +===================== + +History +------- + +The first implementation of DTrace probes for the Erlang virtual +machine was presented at the [2008 Erlang User Conference] [1]. That +work, based on the Erlang/OTP R12 release, was discontinued due to +what appears to be miscommunication with the original developers. + +Several users have created Erlang port drivers, linked-in drivers, or +NIFs that allow Erlang code to try to activate a probe, +e.g. `foo_module:dtrace_probe("message goes here!")`. + +Goals +----- + +* Annotate as much of the Erlang VM as is practical. + * The initial goal is to trace file I/O operations. +* Support all platforms that implement DTrace: OS X, Solaris, + and (I hope) FreeBSD and NetBSD. +* To the extent that it's practical, support SystemTap on Linux + via DTrace provider compatibility. +* Allow Erlang code to supply annotations. + +Supported platforms +------------------- + +* OS X 10.6.x / Snow Leopard. It should also work for 10.7 / Lion, + but I haven't personally tested it. +* Solaris 10. I have done limited testing on Solaris 11 and + OpenIndiana release 151a, and both appear to work. +* FreeBSD 9.0, though please see the "FreeBSD 9.0 Release Notes" + section below! +* Linux via SystemTap compatibility. Please see + [$ERL_TOP/HOWTO/SYSTEMTAP.md][] for more details. + +Just add the `--with-dynamic-trace=dtrace` option to your command when you +run the `configure` script. If you are using systemtap, the configure option +is `--with-dynamic-trace=systemtap` + +Status +------ + +As of R15B01, the dynamic trace code is included in the OTP source distribution, +although it's considered experimental. The main development of the dtrace code +still happens outside of Ericsson, but there is no need to fetch a patched +version of the OTP source to get the basic funtionality. + +Implementation summary +---------------------- + +So far, most effort has been focused on the `efile_drv.c` code, +which implements most file I/O on behalf of the Erlang virtual +machine. This driver also presents a big challenge: its use of an I/O +worker pool (enabled by using the `erl +A 8` flag, for example) makes +it much more difficult to trace I/O activity because each of the +following may be executed in a different Pthread: + +* I/O initiation (Erlang code) +* I/O proxy process handling, e.g. read/write when file is not opened + in `raw` mode, operations executed by the code & file server processes. + (Erlang code) +* `efile_drv` command setup (C code) +* `efile_drv` command execution (C code) +* `efile_drv` status return (C code) + +Example output from `lib/runtime_tools/examples/efile_drv.d` while executing +`file:rename("old-name", "new-name")`: + + efile_drv enter tag={3,84} user tag some-user-tag | RENAME (12) | args: old-name new-name ,\ + 0 0 (port #Port<0.59>) + async I/O worker tag={3,83} | RENAME (12) | efile_drv-int_entry + async I/O worker tag={3,83} | RENAME (12) | efile_drv-int_return + efile_drv return tag={3,83} user tag | RENAME (12) | errno 2 + +... where the following key can help decipher the output: + +* `{3,83}` is the Erlang scheduler thread number (3) and operation + counter number (83) assigned to this I/O operation. Together, + these two numbers form a unique ID for the I/O operation. +* `12` is the command number for the rename operation. See the + definition for `FILE_RENAME` in the source code file `efile_drv.c` + or the `BEGIN` section of the D script `lib/runtime_tools/examples/efile_drv.d`. +* `old-name` and `new-name` are the two string arguments for the + source and destination of the `rename(2)` system call. + The two integer arguments are unused; the simple formatting code + prints the arguments anyway, 0 and 0. +* The worker pool code was called on behalf of Erlang port `#Port<0.59>`. +* The system call failed with a POSIX errno value of 2: `ENOENT`, + because the path `old-name` does not exist. +* The `efile_drv-int_entry` and `efile_drv_int_return` probes are + provided in case the user is + interested in measuring only the latency of code executed by + `efile_drv` asynchronous functions by I/O worker pool threads + and the OS system call that they encapsulate. + +So, where does the `some-user-tag` string come from? + +At the moment, the user tag comes from code like the following: + + dyntrace:put_tag("some-user-tag"), + file:rename("old-name", "new-name"), + +This method of tagging I/O at the Erlang level is subject to change. + +Example DTrace probe specification +---------------------------------- + + /** + * Fired when a message is sent from one local process to another. + * + * NOTE: The 'size' parameter is in machine-dependent words and + * that the actual size of any binary terms in the message + * are not included. + * + * @param sender the PID (string form) of the sender + * @param receiver the PID (string form) of the receiver + * @param size the size of the message being delivered (words) + * @param token_label for the sender's sequential trace token + * @param token_previous count for the sender's sequential trace token + * @param token_current count for the sender's sequential trace token + */ + probe message__send(char *sender, char *receiver, uint32_t size, + int token_label, int token_previous, int token_current); + + /** + * Fired when a message is sent from a local process to a remote process. + * + * NOTE: The 'size' parameter is in machine-dependent words and + * that the actual size of any binary terms in the message + * are not included. + * + * @param sender the PID (string form) of the sender + * @param node_name the Erlang node name (string form) of the receiver + * @param receiver the PID/name (string form) of the receiver + * @param size the size of the message being delivered (words) + * @param token_label for the sender's sequential trace token + * @param token_previous count for the sender's sequential trace token + * @param token_current count for the sender's sequential trace token + */ + probe message__send__remote(char *sender, char *node_name, char *receiver, + uint32_t size, + int token_label, int token_previous, int token_current); + + /** + * Fired when a message is queued to a local process. This probe + * will not fire if the sender's pid == receiver's pid. + * + * NOTE: The 'size' parameter is in machine-dependent words and + * that the actual size of any binary terms in the message + * are not included. + * + * @param receiver the PID (string form) of the receiver + * @param size the size of the message being delivered (words) + * @param queue_len length of the queue of the receiving process + * @param token_label for the sender's sequential trace token + * @param token_previous count for the sender's sequential trace token + * @param token_current count for the sender's sequential trace token + */ + probe message__queued(char *receiver, uint32_t size, uint32_t queue_len, + int token_label, int token_previous, int token_current); + + /** + * Fired when a message is 'receive'd by a local process and removed + * from its mailbox. + * + * NOTE: The 'size' parameter is in machine-dependent words and + * that the actual size of any binary terms in the message + * are not included. + * + * @param receiver the PID (string form) of the receiver + * @param size the size of the message being delivered (words) + * @param queue_len length of the queue of the receiving process + * @param token_label for the sender's sequential trace token + * @param token_previous count for the sender's sequential trace token + * @param token_current count for the sender's sequential trace token + */ + probe message__receive(char *receiver, uint32_t size, uint32_t queue_len, + int token_label, int token_previous, int token_current); + + /* ... */ + + /* Async driver pool */ + + /** + * Show the post-add length of the async driver thread pool member's queue. + * + * NOTE: The port name is not available: additional lock(s) must + * be acquired in order to get the port name safely in an SMP + * environment. The same is true for the aio__pool_get probe. + * + * @param port the Port (string form) + * @param new queue length + */ + probe aio_pool__add(char *, int); + + /** + * Show the post-get length of the async driver thread pool member's queue. + * + * @param port the Port (string form) + * @param new queue length + */ + probe aio_pool__get(char *, int); + + /* Probes for efile_drv.c */ + + /** + * Entry into the efile_drv.c file I/O driver + * + * For a list of command numbers used by this driver, see the section + * "Guide to probe arguments" in ../../../README.md. That section + * also contains explanation of the various integer and string + * arguments that may be present when any particular probe fires. + * + * TODO: Adding the port string, args[10], is a pain. Making that + * port string available to all the other efile_drv.c probes + * will be more pain. Is the pain worth it? If yes, then + * add them everywhere else and grit our teeth. If no, then + * rip it out. + * + * @param thread-id number of the scheduler Pthread arg0 + * @param tag number: {thread-id, tag} uniquely names a driver operation + * @param user-tag string arg2 + * @param command number arg3 + * @param string argument 1 arg4 + * @param string argument 2 arg5 + * @param integer argument 1 arg6 + * @param integer argument 2 arg7 + * @param integer argument 3 arg8 + * @param integer argument 4 arg9 + * @param port the port ID of the busy port args[10] + */ + probe efile_drv__entry(int, int, char *, int, char *, char *, + int64_t, int64_t, int64_t, int64_t, char *); + + /** + * Entry into the driver's internal work function. Computation here + * is performed by a async worker pool Pthread. + * + * @param thread-id number + * @param tag number + * @param command number + */ + probe efile_drv__int_entry(int, int, int); + + /** + * Return from the driver's internal work function. + * + * @param thread-id number + * @param tag number + * @param command number + */ + probe efile_drv__int_return(int, int, int); + + /** + * Return from the efile_drv.c file I/O driver + * + * @param thread-id number arg0 + * @param tag number arg1 + * @param user-tag string arg2 + * @param command number arg3 + * @param Success? 1 is success, 0 is failure arg4 + * @param If failure, the errno of the error. arg5 + */ + probe efile_drv__return(int, int, char *, int, int, int); + +Guide to efile_drv.c probe arguments +------------------------------------ + + /* Driver op code: used by efile_drv-entry arg3 */ + /* used by efile_drv-int_entry arg3 */ + /* used by efile_drv-int_return arg3 */ + /* used by efile_drv-return arg3 */ + + #define FILE_OPEN 1 (probe arg3) + probe arg6 = C driver dt_i1 = flags; + probe arg4 = C driver dt_s1 = path; + + #define FILE_READ 2 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + probe arg7 = C driver dt_i2 = flags; + probe arg8 = C driver dt_i3 = size; + + #define FILE_LSEEK 3 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + probe arg7 = C driver dt_i2 = offset; + probe arg8 = C driver dt_i3 = origin; + + #define FILE_WRITE 4 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + probe arg7 = C driver dt_i2 = flags; + probe arg8 = C driver dt_i3 = size; + + #define FILE_FSTAT 5 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + + #define FILE_PWD 6 (probe arg3) + none + + #define FILE_READDIR 7 (probe arg3) + probe arg4 = C driver dt_s1 = path; + + #define FILE_CHDIR 8 (probe arg3) + probe arg4 = C driver dt_s1 = path; + + #define FILE_FSYNC 9 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + + #define FILE_MKDIR 10 (probe arg3) + probe arg4 = C driver dt_s1 = path; + + #define FILE_DELETE 11 (probe arg3) + probe arg4 = C driver dt_s1 = path; + + #define FILE_RENAME 12 (probe arg3) + probe arg4 = C driver dt_s1 = old_name; + probe arg5 = C driver dt_s2 = new_name; + + #define FILE_RMDIR 13 (probe arg3) + probe arg4 = C driver dt_s1 = path; + + #define FILE_TRUNCATE 14 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + probe arg7 = C driver dt_i2 = flags; + + #define FILE_READ_FILE 15 (probe arg3) + probe arg4 = C driver dt_s1 = path; + + #define FILE_WRITE_INFO 16 (probe arg3) + probe arg6 = C driver dt_i1 = mode; + probe arg7 = C driver dt_i2 = uid; + probe arg8 = C driver dt_i3 = gid; + + #define FILE_LSTAT 19 (probe arg3) + probe arg4 = C driver dt_s1 = path; + + #define FILE_READLINK 20 (probe arg3) + probe arg4 = C driver dt_s1 = path; + + #define FILE_LINK 21 (probe arg3) + probe arg4 = C driver dt_s1 = existing_path; + probe arg5 = C driver dt_s2 = new_path; + + #define FILE_SYMLINK 22 (probe arg3) + probe arg4 = C driver dt_s1 = existing_path; + probe arg5 = C driver dt_s2 = new_path; + + #define FILE_CLOSE 23 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + probe arg7 = C driver dt_i2 = flags; + + #define FILE_PWRITEV 24 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + probe arg7 = C driver dt_i2 = flags; + probe arg8 = C driver dt_i3 = size; + + #define FILE_PREADV 25 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + probe arg7 = C driver dt_i2 = flags; + probe arg8 = C driver dt_i3 = size; + + #define FILE_SETOPT 26 (probe arg3) + probe arg6 = C driver dt_i1 = opt_name; + probe arg7 = C driver dt_i2 = opt_specific_value; + + #define FILE_IPREAD 27 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + probe arg7 = C driver dt_i2 = flags; + probe arg8 = C driver dt_i3 = offsets[0]; + probe arg9 = C driver dt_i4 = size; + + #define FILE_ALTNAME 28 (probe arg3) + probe arg4 = C driver dt_s1 = path; + + #define FILE_READ_LINE 29 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + probe arg7 = C driver dt_i2 = flags; + probe arg8 = C driver dt_i3 = read_offset; + probe arg9 = C driver dt_i4 = read_ahead; + + #define FILE_FDATASYNC 30 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + + #define FILE_FADVISE 31 (probe arg3) + probe arg6 = C driver dt_i1 = fd; + probe arg7 = C driver dt_i2 = offset; + probe arg8 = C driver dt_i3 = length; + probe arg9 = C driver dt_i4 = advise_type; + + [1]: http://www.erlang.org/euc/08/ + [$ERL_TOP/HOWTO/SYSTEMTAP.md]: SYSTEMTAP.md diff --git a/HOWTO/INSTALL-CROSS.md b/HOWTO/INSTALL-CROSS.md new file mode 100644 index 0000000000..8a2b1bc8ad --- /dev/null +++ b/HOWTO/INSTALL-CROSS.md @@ -0,0 +1,560 @@ +Cross Compiling Erlang/OTP +========================== + +Introduction +------------ + +This document describes how to cross compile Erlang/OTP-%OTP-REL%. Note that +the support for cross compiling Erlang/OTP should be considered as +experimental. As far as we know, the %OTP-REL% release should cross compile +fine, but since we currently have a very limited set of cross compilation +environments to test with we cannot be sure. The cross compilation support +will remain in an experimental state until we get a lot more cross compilation +environments to test with. + +You are advised to read the whole document before attempting to cross +compile Erlang/OTP. However, before reading this document, you should read +the [$ERL_TOP/HOWTO/INSTALL.md][] document which describes building and installing +Erlang/OTP in general. `$ERL_TOP` is the top directory in the source tree. + +### otp\_build Versus configure/make ### + +Building Erlang/OTP can be done either by using the `$ERL_TOP/otp_build` +script, or by invoking `$ERL_TOP/configure` and `make` directly. Building using +`otp_build` is easier since it involves fewer steps, but the `otp_build` build +procedure is not as flexible as the `configure`/`make` build procedure. Note +that `otp_build configure` will produce a default configuration that differs +from what `configure` will produce by default. For example, currently +`--disable-dynamic-ssl-lib` is added to the `configure` command line arguments +unless `--enable-dynamic-ssl-lib` has been explicitly passed. The binary +releases that we deliver are built using `otp_build`. The defaults used by +`otp_build configure` may change at any time without prior notice. + +### Cross Configuration ### + +The `$ERL_TOP/xcomp/erl-xcomp.conf.template` file contains all available cross +configuration variables and can be used as a template when creating a cross +compilation configuration. All [cross configuration variables][] are also +listed at the end of this document. For examples of working cross +configurations see the `$ERL_TOP/xcomp/erl-xcomp-TileraMDE2.0-tilepro.conf` +file and the `$ERL_TOP/xcomp/erl-xcomp-x86_64-saf-linux-gnu.conf` file. If the +default behavior of a variable is satisfactory, the variable does not need to +be set. However, the `configure` script will issue a warning when a default +value is used. When a variable has been set, no warning will be issued. + +A cross configuration file can be passed to `otp_build configure` using the +`--xcomp-conf` command line argument. Note that `configure` does not accept +this command line argument. When using the `configure` script directly, pass +the configuration variables as arguments to `configure` using a +`<VARIABLE>=<VALUE>` syntax. Variables can also be passed as environment +variables to `configure`. However, if you pass the configuration in the +environment, make sure to unset all of these environment variables before +invoking `make`; otherwise, the environment variables might set make variables +in some applications, or parts of some applications, and you may end up with +an erroneously configured build. + +### What can be Cross Compiled? ### + +All Erlang/OTP applications except the `wx` application can be cross compiled. +The build of the `wx` driver will currently be automatically disabled when +cross compiling. + +### Compatibility ### + +The build system, including cross compilation configuration variables used, +may be subject to non backward compatible changes without prior notice. +Current cross build system has been tested when cross compiling some Linux/GNU +systems, but has only been partly tested for more esoteric platforms. The +VxWorks example file is highly dependent on our environment and is here more +or less only for internal use. + +### Patches ### + +Please submit any patches for cross compiling in a way consistent with this +system. All input is welcome as we have a very limited set of cross compiling +environments to test with. If a new configuration variable is needed, add it +to `$ERL_TOP/xcomp/erl-xcomp.conf.template`, and use it in `configure.in`. +Other files that might need to be updated are: + +- `$ERL_TOP/xcomp/erl-xcomp-vars.sh` +- `$ERL_TOP/erl-build-tool-vars.sh` +- `$ERL_TOP/erts/aclocal.m4` +- `$ERL_TOP/xcomp/README.md` +- `$ERL_TOP/xcomp/erl-xcomp-*.conf` + +Note that this might be an incomplete list of files that need to be updated. + +General information on how to submit patches can be found at: + <http://wiki.github.com/erlang/otp/submitting-patches> + +Build and Install Procedure +--------------------------- + +If you are building in Git, you want to read the [Building in Git][] section +of [$ERL_TOP/HOWTO/INSTALL.md][] before proceeding. + +We will first go through the `configure`/`make` build procedure which people +probably are most familiar with. + +### Building With configure/make Directly ### + + (1) + +Change directory into the top directory of the Erlang/OTP source tree. + + $ cd $ERL_TOP + +In order to compile Erlang code, a small Erlang bootstrap system has to be +built, or an Erlang/OTP system of the same release as the one being built +has to be provided in the `$PATH`. The Erlang/OTP for the target system will +be built using this Erlang system, together with the cross compilation tools +provided. + +If you want to build the documentation out of the same source tree as you are +cross compiling in, you currently need a full Erlang/OTP system of the same +release as the one being built for the build machine. If this is the case, +build and install one for the build machine (or use one already built) and add +it to the `$PATH` before cross building, and building the documentation. See +the [How to Build the Documentation][] section in the [$ERL_TOP/HOWTO/INSTALL.md][] +document for information on how to build the documentation. + +If you want to build using a compatible Erlang/OTP system in the `$PATH`, +jump to (3). + +#### Building a Bootstrap System #### + + (2) + + $ ./configure --enable-bootstrap-only + $ make + +The `--enable-bootstrap-only` argument to `configure` isn't strictly necessary, +but will speed things up. It will only run `configure` in applications +necessary for the bootstrap, and will disable a lot of things not needed by +the bootstrap system. If you run `configure` without `--enable-boostrap-only` +you also have to run make as `make bootstrap`; otherwise, the whole system will +be built. + +#### Cross Building the System #### + + (3) + + $ ./configure --host=<HOST> --build=<BUILD> [Other Config Args] + $ make + +`<HOST>` is the host/target system that you build for. It does not have to be +a full `CPU-VENDOR-OS` triplet, but can be. The full `CPU-VENDOR-OS` triplet +will be created by executing `$ERL_TOP/erts/autoconf/config.sub <HOST>`. If +`config.sub` fails, you need to be more specific. + +`<BUILD>` should equal the `CPU-VENDOR-OS` triplet of the system that you +build on. If you execute `$ERL_TOP/erts/autoconf/config.guess`, it will in +most cases print the triplet you want to use for this. + +Pass the cross compilation variables as command line arguments to `configure` +using a `<VARIABLE>=<VALUE>` syntax. + +> *NOTE*: You can *not* pass a configuration file using the `--xcomp-conf` +> argument when you invoke `configure` directly. The `--xcomp-conf` argument +> can only be passed to `otp_build configure`. + +`make` will verify that the Erlang/OTP system used when building is of the +same release as the system being built, and will fail if this is not the case. +It is possible, however not recommended, to force the cross compilation even +though the wrong Erlang/OTP system is used. This by invoking `make` like this: +`make ERL_XCOMP_FORCE_DIFFERENT_OTP=yes`. + +> *WARNING*: Invoking `make ERL_XCOMP_FORCE_DIFFERENT_OTP=yes` might fail, +> silently produce suboptimal code, or silently produce erroneous code. + +#### Installing #### + +You can either install using the installation paths determined by `configure` +(4), or install manually using (5). + +##### Installing Using Paths Determined by configure ##### + + (4) + + $ make install DESTDIR=<TEMPORARY_PREFIX> + +`make install` will install at a location specified when doing `configure`. +`configure` arguments specifying where the installation should reside are for +example: `--prefix`, `--exec-prefix`, `--libdir`, `--bindir`, etc. By default +it will install under `/usr/local`. You typically do not want to install your +cross build under `/usr/local` on your build machine. Using [DESTDIR][] +will cause the installation paths to be prefixed by `$DESTDIR`. This makes it +possible to install and package the installation on the build machine without +having to place the installation in the same directory on the build machine as +it should be executed from on the target machine. + +When `make install` has finished, change directory into `$DESTDIR`, package +the system, move it to the target machine, and unpack it. Note that the +installation will only be working on the target machine at the location +determined by `configure`. + +##### Installing Manually ##### + + (5) + + $ make release RELEASE_ROOT=<RELEASE_DIR> + +`make release` will copy what you have built for the target machine to +`<RELEASE_DIR>`. The `Install` script will not be run. The content of +`<RELEASE_DIR>` is what by default ends up in `/usr/local/lib/erlang`. + +The `Install` script used when installing Erlang/OTP requires common Unix +tools such as `sed` to be present in your `$PATH`. If your target system +does not have such tools, you need to run the `Install` script on your +build machine before packaging Erlang/OTP. The `Install` script should +currently be invoked as follows in the directory where it resides +(the top directory): + + $ ./Install [-cross] [-minimal|-sasl] <ERL_ROOT> + +where: + +* `-minimal` Creates an installation that starts up a minimal amount + of applications, i.e., only `kernel` and `stdlib` are started. The + minimal system is normally enough, and is what `make install` uses. +* `-sasl` Creates an installation that also starts up the `sasl` + application. +* `-cross` For cross compilation. Informs the install script that it + is run on the build machine. +* `<ERL_ROOT>` - The absolute path to the Erlang installation to use + at run time. This is often the same as the current working directory, + but does not have to be. It can follow any other path through the file + system to the same directory. + +If neither `-minimal`, nor `-sasl` is passed as argument you will be +prompted. + +You can now either do: + + (6) + +* Decide where the installation should be located on the target machine, + run the `Install` script on the build machine, and package the installed + installation. The installation just need to be unpacked at the right + location on the target machine: + + $ cd <RELEASE_DIR> + $ ./Install -cross [-minimal|-sasl] <ABSOLUTE_INSTALL_DIR_ON_TARGET> + +or: + + (7) + +* Package the installation in `<RELEASE_DIR>`, place it wherever you want + on your target machine, and run the `Install` script on your target + machine: + + $ cd <ABSOLUTE_INSTALL_DIR_ON_TARGET> + $ ./Install [-minimal|-sasl] <ABSOLUTE_INSTALL_DIR_ON_TARGET> + +### Building With the otp\_build Script ### + + (8) + + $ cd $ERL_TOP + + (9) + + $ ./otp_build configure --xcomp-conf=<FILE> [Other Config Args] + +alternatively: + + $ ./otp_build configure --host=<HOST> --build=<BUILD> [Other Config Args] + +If you have your cross compilation configuration in a file, pass it using the +`--xcomp-conf=<FILE>` command line argument. If not, pass `--host=<HOST>`, +`--build=<BUILD>`, and the configuration variables using a `<VARIABLE>=<VALUE>` +syntax on the command line (same as in (3)). Note that `<HOST>` and `<BUILD>` +have to be passed one way or the other; either by using `erl_xcomp_host=<HOST>` +and `erl_xcomp_build=<BUILD>` in the configuration file, or by using the +`--host=<HOST>`, and `--build=<BUILD>` command line arguments. + +`otp_build configure` will configure both for the boostrap system on the +build machine and the cross host system. + + (10) + + $ ./otp_build boot -a + +`otp_build boot -a` will first build a bootstrap system for the build machine +and then do the cross build of the system. + + (11) + + $ ./otp_build release -a <RELEASE_DIR> + +`otp_build release -a` will do the same as (5), and you will after this have +to do a manual install either by doing (6), or (7). + +Testing the cross compiled system +--------------------------------- +Some of the tests that come with erlang use native code to test. This means +that when cross compiling erlang you also have to cross compile test suites +in order to run tests on the target host. To do this you first have to release +the tests as usual. + + $ make release_tests + +or + + $ ./otp_build tests + +The tests will be released into `$ERL_TOP/release/tests`. After releasing the +tests you have to install the tests on the build machine. You supply the same +xcomp file as to `./otp_build` in (9). + + $ cd $ERL_TOP/release/tests/test_server/ + $ $ERL_TOP/bootstrap/bin/erl -eval 'ts:install([{xcomp,"<FILE>"}])' -s ts compile_testcases -s init stop + +You should get a lot of printouts as the testcases are compiled. Once done you +should copy the entire `$ERL_TOP/release/tests` folder to the cross host system. + +Then go to the cross host system and setup the erlang installed in (4) or (5) +to be in your `$PATH`. Then go to what previously was +`$ERL_TOP/release/tests/test_server` and issue the following command. + + $ erl -s ts install -s ts run all_tests -s init stop + +The configure should be skipped and all tests should hopefully pass. For more +details about how to use ts run `erl -s ts help -s init stop` + +Currently Used Configuration Variables +-------------------------------------- + +Note that you cannot define arbitrary variables in a cross compilation +configuration file. Only the ones listed below will be guaranteed to be +visible throughout the whole execution of all `configure` scripts. Other +variables needs to be defined as arguments to `configure` or exported in +the environment. + +### Variables for otp\_build Only ### + +Variables in this section are only used, when configuring Erlang/OTP for +cross compilation using `$ERL_TOP/otp_build configure`. + +> *NOTE*: These variables currently have *no* effect if you configure using +> the `configure` script directly. + +* `erl_xcomp_build` - The build system used. This value will be passed as + `--build=$erl_xcomp_build` argument to the `configure` script. It does + not have to be a full `CPU-VENDOR-OS` triplet, but can be. The full + `CPU-VENDOR-OS` triplet will be created by + `$ERL_TOP/erts/autoconf/config.sub $erl_xcomp_build`. If set to `guess`, + the build system will be guessed using + `$ERL_TOP/erts/autoconf/config.guess`. + +* `erl_xcomp_host` - Cross host/target system to build for. This value will + be passed as `--host=$erl_xcomp_host` argument to the `configure` script. + It does not have to be a full `CPU-VENDOR-OS` triplet, but can be. The + full `CPU-VENDOR-OS` triplet will be created by + `$ERL_TOP/erts/autoconf/config.sub $erl_xcomp_host`. + +* `erl_xcomp_configure_flags` - Extra configure flags to pass to the + `configure` script. + +### Cross Compiler and Other Tools ### + +If the cross compilation tools are prefixed by `<HOST>-` you probably do +not need to set these variables (where `<HOST>` is what has been passed as +`--host=<HOST>` argument to `configure`). + +All variables in this section can also be used when native compiling. + +* `CC` - C compiler. + +* `CFLAGS` - C compiler flags. + +* `STATIC_CFLAGS` - Static C compiler flags. + +* `CFLAG_RUNTIME_LIBRARY_PATH` - This flag should set runtime library + search path for the shared libraries. Note that this actually is a + linker flag, but it needs to be passed via the compiler. + +* `CPP` - C pre-processor. + +* `CPPFLAGS` - C pre-processor flags. + +* `CXX` - C++ compiler. + +* `CXXFLAGS` - C++ compiler flags. + +* `LD` - Linker. + +* `LDFLAGS` - Linker flags. + +* `LIBS` - Libraries. + +#### Dynamic Erlang Driver Linking #### + +> *NOTE*: Either set all or none of the `DED_LD*` variables. + +* `DED_LD` - Linker for Dynamically loaded Erlang Drivers. + +* `DED_LDFLAGS` - Linker flags to use with `DED_LD`. + +* `DED_LD_FLAG_RUNTIME_LIBRARY_PATH` - This flag should set runtime library + search path for shared libraries when linking with `DED_LD`. + +#### Large File Support #### + +> *NOTE*: Either set all or none of the `LFS_*` variables. + +* `LFS_CFLAGS` - Large file support C compiler flags. + +* `LFS_LDFLAGS` - Large file support linker flags. + +* `LFS_LIBS` - Large file support libraries. + +#### Other Tools #### + +* `RANLIB` - `ranlib` archive index tool. + +* `AR` - `ar` archiving tool. + +* `GETCONF` - `getconf` system configuration inspection tool. `getconf` is + currently used for finding out large file support flags to use, and + on Linux systems for finding out if we have an NPTL thread library or + not. + +### Cross System Root Locations ### + +* `erl_xcomp_sysroot` - The absolute path to the system root of the cross + compilation environment. Currently, the `crypto`, `odbc`, `ssh` and + `ssl` applications need the system root. These applications will be + skipped if the system root has not been set. The system root might be + needed for other things too. If this is the case and the system root + has not been set, `configure` will fail and request you to set it. + +* `erl_xcomp_isysroot` - The absolute path to the system root for includes + of the cross compilation environment. If not set, this value defaults + to `$erl_xcomp_sysroot`, i.e., only set this value if the include system + root path is not the same as the system root path. + +### Optional Feature, and Bug Tests ### + +These tests cannot (always) be done automatically when cross compiling. You +usually do not need to set these variables. + +> *WARNING*: Setting these variables wrong may cause hard to detect +> runtime errors. If you need to change these values, *really* make sure +> that the values are correct. + +> *NOTE*: Some of these values will override results of tests performed +> by `configure`, and some will not be used until `configure` is sure that +> it cannot figure the result out. + +The `configure` script will issue a warning when a default value is used. +When a variable has been set, no warning will be issued. + +* `erl_xcomp_after_morecore_hook` - `yes|no`. Defaults to `no`. If `yes`, + the target system must have a working `__after_morecore_hook` that can be + used for tracking used `malloc()` implementations core memory usage. + This is currently only used by unsupported features. + +* `erl_xcomp_bigendian` - `yes|no`. No default. If `yes`, the target system + must be big endian. If `no`, little endian. This can often be + automatically detected, but not always. If not automatically detected, + `configure` will fail unless this variable is set. Since no default + value is used, `configure` will try to figure this out automatically. + +* `erl_xcomp_double_middle` - `yes|no`. Defaults to `no`. + If `yes`, the target system must have doubles in "middle-endian" format. If + `no`, it has "regular" endianness. + +* `erl_xcomp_clock_gettime_cpu_time` - `yes|no`. Defaults to `no`. If `yes`, + the target system must have a working `clock_gettime()` implementation + that can be used for retrieving process CPU time. + +* `erl_xcomp_getaddrinfo` - `yes|no`. Defaults to `no`. If `yes`, the target + system must have a working `getaddrinfo()` implementation that can + handle both IPv4 and IPv6. + +* `erl_xcomp_gethrvtime_procfs_ioctl` - `yes|no`. Defaults to `no`. If `yes`, + the target system must have a working `gethrvtime()` implementation and + is used with procfs `ioctl()`. + +* `erl_xcomp_dlsym_brk_wrappers` - `yes|no`. Defaults to `no`. If `yes`, the + target system must have a working `dlsym(RTLD_NEXT, <S>)` implementation + that can be used on `brk` and `sbrk` symbols used by the `malloc()` + implementation in use, and by this track the `malloc()` implementations + core memory usage. This is currently only used by unsupported features. + +* `erl_xcomp_kqueue` - `yes|no`. Defaults to `no`. If `yes`, the target + system must have a working `kqueue()` implementation that returns a file + descriptor which can be used by `poll()` and/or `select()`. If `no` and + the target system has not got `epoll()` or `/dev/poll`, the kernel-poll + feature will be disabled. + +* `erl_xcomp_linux_clock_gettime_correction` - `yes|no`. Defaults to `yes` on + Linux; otherwise, `no`. If `yes`, `clock_gettime(CLOCK_MONOTONIC, _)` on + the target system must work. This variable is recommended to be set to + `no` on Linux systems with kernel versions less than 2.6. + +* `erl_xcomp_linux_nptl` - `yes|no`. Defaults to `yes` on Linux; otherwise, + `no`. If `yes`, the target system must have NPTL (Native POSIX Thread + Library). Older Linux systems have LinuxThreads instead of NPTL (Linux + kernel versions typically less than 2.6). + +* `erl_xcomp_linux_usable_sigaltstack` - `yes|no`. Defaults to `yes` on Linux; + otherwise, `no`. If `yes`, `sigaltstack()` must be usable on the target + system. `sigaltstack()` on Linux kernel versions less than 2.4 are + broken. + +* `erl_xcomp_linux_usable_sigusrx` - `yes|no`. Defaults to `yes`. If `yes`, + the `SIGUSR1` and `SIGUSR2` signals must be usable by the ERTS. Old + LinuxThreads thread libraries (Linux kernel versions typically less than + 2.2) used these signals and made them unusable by the ERTS. + +* `erl_xcomp_poll` - `yes|no`. Defaults to `no` on Darwin/MacOSX; otherwise, + `yes`. If `yes`, the target system must have a working `poll()` + implementation that also can handle devices. If `no`, `select()` will be + used instead of `poll()`. + +* `erl_xcomp_putenv_copy` - `yes|no`. Defaults to `no`. If `yes`, the target + system must have a `putenv()` implementation that stores a copy of the + key/value pair. + +* `erl_xcomp_reliable_fpe` - `yes|no`. Defaults to `no`. If `yes`, the target + system must have reliable floating point exceptions. + +Copyright and License +--------------------- + +%CopyrightBegin% + +Copyright Ericsson AB 2009-2010. All Rights Reserved. + +The contents of this file are subject to the Erlang Public License, +Version 1.1, (the "License"); you may not use this file except in +compliance with the License. You should have received a copy of the +Erlang Public License along with this software. If not, it can be +retrieved online at http://www.erlang.org/. + +Software distributed under the License is distributed on an "AS IS" +basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +the License for the specific language governing rights and limitations +under the License. + +%CopyrightEnd% + +Modifying This Document +----------------------- + +Before modifying this document you need to have a look at the +[$ERL_TOP/HOWTO/MARKDOWN.md][] document. + + + + [$ERL_TOP/HOWTO/INSTALL.md]: INSTALL.md + [Building in Git]: INSTALL.md#How-to-Build-and-Install-ErlangOTP_Building-in-Git + [How to Build the Documentation]: INSTALL.md#The-ErlangOTP-Documentation_How-to-Build-the-Documentation + [cross configuration variables]: #Currently-Used-Configuration-Variables + [DESTDIR]: http://www.gnu.org/prep/standards/html_node/DESTDIR.html + [$ERL_TOP/HOWTO/MARKDOWN.md]: MARKDOWN.md + + [?TOC]: true diff --git a/HOWTO/INSTALL-WIN32.md b/HOWTO/INSTALL-WIN32.md new file mode 100644 index 0000000000..94d3688f23 --- /dev/null +++ b/HOWTO/INSTALL-WIN32.md @@ -0,0 +1,1047 @@ +How to Build Erlang/OTP on Windows +================================== + +Introduction +------------ + +This file describes how to build the Erlang emulator and the OTP +libraries on Windows. The instructions apply to versions of Windows +supporting the Cygwin emulated gnuish environment for Windows or the +Msys ditto. We've built on the following platforms: Windows 2003 +server, Windows XP Home/Professional, Windows Vista and Windows 7 (32 +and 64 bit). You can probably build on Windows 2000, but you will not +be able to install the latest Microsoft SDK, so you have to go back to +some earlier compiler. Any Windows95'ish platform will surely get you +into trouble, what I'm not sure of, but it certainly will... + +The procedure described uses either Cygwin or Msys as a build +environment, you run the bash shell in Cygwin/Msys and use gnu +make/configure/autoconf etc to do the build. The emulator C-source +code is, however, mostly compiled with Microsoft Visual C++™, +producing a native Windows binary. This is the same procedure as we +use to build the pre-built binaries. The fact that we use VC++ and not +gcc is explained further in the FAQ section. + +I describe the build procedure to make it possible for open source +customers to build the emulator, given that they have the needed +tools. The binary Windows releases is still a preferred alternative if +one does not have Microsoft's development tools and/or don't want to +install Cygwin or Msys. + +To use Cygwin/Msys, one needs basic experience from a Unix environment, if +one does not know how to set environment variables, run programs etc +in a Unix environment, one will be quite lost in the Cygwin os Msys +ditto. I can unfortunately not teach all the world how to use +Cygwin and bash, neither how to install Cygwin nor perform basic tasks +on a computer. Please refer to other documentation on the net for +help, or use the binary release instead if you have problems using the +tools. + +However, if you feel comfortable with the environment and build +system, and have all the necessary tools, you have a great opportunity +to make the Erlang/OTP distribution for Windows better. Please submit +any suggestions and patches to the appropriate [mailing lists] [1] to let +them find their way into the next version of Erlang. If making changes +to the build system (like makefiles etc) please bear in mind that the +same makefiles are used on Unix/VxWorks, so that your changes +don't break other platforms. That of course goes for C-code too, system +specific code resides in the `$ERL_TOP/erts/emulator/sys/win32` and +`$ERL_TOP/erts/etc/win32` directories mostly. The +`$ERL_TOP/erts/emulator/beam` directory is for common code. + +Before the R9C release of Erlang/OTP, the Windows release was built +partly on a Unix (Solaris) box and partly on a Windows box, using Perl +hacks to communicate and sync between the two machines. R9C was the +first release ever built solely on Windows, where no Unix machine is +needed at all. Now we've used this build procedure for a couple of +releases, and it has worked fine for us. Still, there might be all +sorts of troubles on different machines and with different +setups. I'll try to give hints wherever I've encountered difficulties, +but please share your experiences by using the [erlang-questions] [1] +mailing list. I cannot of course help everyone with all +their problems, please try to solve the problems and submit +solutions/workarounds. Remember, it's all about sharing, not about +demanding... + +Starting with R15B, our build system runs both on Cygwin and Msys +(MinGW's fork of an early cygwin version). Msys is a smaller package +to install and may on some machines run slightly faster. If Cygwin +gives you trouble, try Msys instead, and v.v. Beginning with R15B +there is also a native 64bit version of Erlang for 64bit Windows 7 +(only). These instructions apply to both the 32bit VM and the 64bit +ditto. + +Note that even if you build a 64bit VM, most of the directories and +files involved are still named win32. You can view the name win32 as +meaning any windows version not beeing 16bit. A few occurences of the +name Win64 are however present in the system, for example the +installation file for a 64 bit windows version of Erlang is by default +named `otp_win64_<version>.exe`. + +Lets go then, I'll start with a little FAQ, based on in house questions +and misunderstandings. + + +Frequently Asked Questions +-------------------------- + +* Q: So, now I can build Erlang using GCC on Windows? + + A: No, unfortunately not. You'll need Microsoft's Visual C++ + still, a Bourne-shell script (cc.sh) wraps the Visual C++ compiler + and runs it from within the Cygwin environment. All other tools + needed to build Erlang are free-ware/open source, but not the C + compiler. The Windows SDK is however enough to build Erlang, you + do not need to buy Visual C++, just download the SDK (SDK version + 7.1 == Visual studio 2010). + +* Q: Why haven't you got rid of VC++ then, you \*\*\*\*\*\*? + + A: Well, partly because it's a good compiler - really! Actually it's + been possible in late R11-releases to build using mingw instead of + visual C++ (you might see the remnants of that in some scripts and + directories). Unfortunately the development of the SMP version for + Windows broke the mingw build and we chose to focus on the VC++ build + as the performance has been much better in the VC++ versions. The + mingw build will possibly be back, but as long as VC++ gives better + performance, the commercial build will be a VC++ one. + +* Q: OK, you need VC++, but now you've started to demand a very recent + (and expensive) version of Visual studio, not the old and stable VC++ + 6.0 that was used in earlier versions. Why? + + A: Well, it's not expensive, it's free (as in free beer). Just + download and install the latest Windows SDK from Microsoft and all + the tools you need are there. The included debugger (WinDbg) is + also quite usable, it's what I used when porting Erlang to 64bit + Windows. Another reason to use the latest Microsoft compilers is + DLL compatibility. DLL's using a new version of the standard + library might not load if the VM is compiled with an old VC++ + version, why we should aim to use the latest freely available SDK + and compiler. + +* Q: Can/will I build a Cygwin binary with the procedure you describe? + + A: No, the result will be a pure Windows binary, and as far as I know, + it's not possible to make a Cygwin binary yet. That is of course + something desirable, but there are still some problems with the + dynamic linking (dynamic Erlang driver loading) as well as the TCP/IP + emulation in Cygwin, which, I'm sure of, will improve, but still has + some problems. Fixing those problems might be easy or might be hard. + I suggest you try yourself and share your experience. No one would be + happier if a simple `./configure && make` would produce a fully fledged + Cygwin binary. Ericsson does however not pay me to do a Cygwin port, so + such a port would have to happen in spare time, which is a limited + resource... + +* Q: Hah, I saw you, you used GCC even though you said you didn't! + + A: OK, I admit, one of the files is compiled using Cygwin's or + MinGW's GCC and the resulting object code is then converted to MS + VC++ compatible coff using a small C hack. It's because that + particular file, `beam_emu.c` benefits immensely from being able + to use the GCC labels-as-values extension, which boosts emulator + performance by up to 50%. That does unfortunately not (yet) mean + that all of OTP could be compiled using GCC, that particular + source code does not do anything system specific and actually is + adopted to the fact that GCC is used to compile it on Windows. + +* Q: So now there's a MS VC++ project file somewhere and I can build OTP + using the nifty VC++ GUI? + + A: No, never. The hassle of keeping the project files up to date and + do all the steps that constitute an OTP build from within the VC++ GUI + is simply not worth it, maybe even impossible. A VC++ project + file for Erlang/OTP will never happen, at least I will never make + one. Clicking around in super-multi-tab'd dialogs to add a file or + compiler option when it's so much easier in a makefile is simply not + my style. + +* Q: So how does it all work then? + + A: Cygwin or Msys is the environment, which closely resembles the + environments found on any Unix machine. It's almost like you had a + virtual Unix machine inside Windows. Configure, given certain + parameters, then creates makefiles that are used by the + Cygwin/Msys gnu-make to built the system. Most of the actual + compilers etc are not, however, Cygwin/Msys tools, so I've written + a couple of wrappers (Bourne-shell scripts), which reside in + `$ERL_TOP/etc/win32/cygwin_tools` and + `$ERL_TOP/etc/win32/msys_tools`. They all do conversion of + parameters and switches common in the Unix environment to fit the + native Windows tools. Most notable is of course the paths, which + in Cygwin/Msys are Unix-like paths with "forward slashes" (/) and + no drive letters, the Cygwin specific command `cygpath` is used + for most of the path conversions in a Cygwin environment, other + tools are used (when needed) in the corresponding Msys + environment. Luckily most compilers accept forward slashes instead + of backslashes as path separators, but one still have to get the drive + letters etc right, though. The wrapper scripts are not general in + the sense that, for example, cc.sh would understand and translates + every possible gcc option and passes correct options to + cl.exe. The principle is that the scripts are powerful enough to + allow building of Erlang/OTP, no more, no less. They might need + extensions to cope with changes during the development of Erlang, + that's one of the reasons I made them into shell-scripts and not + Perl-scripts, I believe they are easier to understand and change + that way. I might be wrong though, cause another reason I didn't + write them in Perl is because I've never liked Perl and my Perl + code is no pleasant reading... + + In `$ERL_TOP`, there is a script called `otp_build`. That script handles + the hassle of giving all the right parameters to `configure`/`make` and + also helps you set up the correct environment variables to work with + the Erlang source under Cygwin. + +* Q: You use and need Cygwin, but then you haven't taken the time to + port Erlang to the Cygwin environment but instead focus on your + commercial release, is that really ethical? + + A: No, not really, but see this as a step in the right direction. I'm + aiming at GCC compiled emulators and a Cygwin version, but I really + need to do other things as well... In time, but don't hold your + breath... + +* Q: Can I build something that looks exactly as the commercial release? + + A: Yes, we use the exactly same build procedure. + +* Q: Which version of Cygwin/Msys and other tools do you use then? + + A: For Cygwin and Msys alike, we try to use the latest releases + available when building. What versions you use shouldn't really + matter, I try to include workarounds for the bugs I've found in + different Cygwin/Msys releases, please help me add workarounds + for new Cygwin/Msys-related bugs as soon as you encounter + them. Also please do submit bug reports to the appropriate Cygwin + and/or Msys developers. The GCC we used for %OTP-REL% was version + 4.7.0 (MinGW 64bit) and 4.3.4 (Cygwin 32bit). We used VC++ 10.0 + (i.e. Visual studio 2010), Sun's JDK 1.5.0\_17 (32bit) and Sun's + JDK 1.7.0\_1 (64bit), NSIS 2.46, and Win32 OpenSSL 0.9.8r. Please + read the next section for details on what you need. + +* Q: Can you help me setup X in Cygwin? + + A: No, unfortunately I haven't got time to help with Cygwin related + user problems, please read Cygwin related web sites, newsgroups and + mailing lists. + +* Q: Why is the instruction so long? Is it really that complicated? + + A: Partly it's long because I babble too much, partly because I've + described as much as I could about the installation of the needed + tools. Once the tools are installed, building is quite easy. I also + have tried to make this instruction understandable for people with + limited Unix experience. Cygwin/Msys is a whole new environment to some + Windows users, why careful explanation of environment variables etc + seemed to be in place. The short story, for the experienced and + impatient is: + + * Get and install complete Cygwin (latest) or complete MinGW with msys + + * Install Microsofts Windows SDK 7.1 (and .Net 4) + + * Get and install Sun's JDK 1.5.0 or higher + + * Get and install NSIS 2.01 or higher (up to 2.46 tried and working) + + * Get, build and install OpenSSL 0.9.8r or higher (up to 1.0.0a + tried & working) with static libs. + + * Get the Erlang source distribution (from + <http://www.erlang.org/download.html>) and unpack with Cygwin's `tar`. + + * Set `ERL_TOP` to where you unpacked the source distribution + + * `$ cd $ERL_TOP` + + * Get (from <http://www.erlang.org/download/tcltk85_win32_bin.tar.gz>) + and unpack the prebuilt TCL/TK binaries for windows with cygwin tar, + standing in `$ERL_TOP` + + * Modify PATH and other environment variables so that all these tools + are runnable from a bash shell. Still standing in `$ERL_TOP`, issue + the following commands: + + $ eval `./otp_build env_win32` + $ ./otp_build autoconf + $ ./otp_build configure + $ ./otp_build boot -a + $ ./otp_build release -a + $ ./otp_build installer_win32 + $ release/win32/otp_win32_%OTP-REL% /S + + Voila! `Start->Programs->Erlang OTP %OTP-REL%->Erlang` starts the Erlang + Windows shell. + + +Tools you Need and Their Environment +------------------------------------ + +You need some tools to be able to build Erlang/OTP on Windows. Most +notably you'll need Cygwin or Msys and Microsofts Windows SDK, but +you also might want a Java compiler, the NSIS install system and +OpenSSL. Well' here's the list: + +* Cygwin, the very latest is usually best. Get all the development + tools and of course all the basic ditto. In fact getting the complete + package might be a good idea, as you'll start to love Cygwin after a + while if you're accustomed to Unix. Make sure to get jar and also make + sure *not* to install a Cygwin'ish Java... The Cygwin jar command is + used but Sun's Java compiler and virtual machine... + + If you are going to build a 64bit Windows version, you should make + sure to get MinGW's 64bit gcc installed with cygwin. It's in one of + the development packages. + + URL: <http://www.cygwin.com> + + Get the installer from the web site and use that to install + Cygwin. Be sure to have fair privileges. If you're on a NT domain you + should consider running `mkpasswd -d` and `mkgroup -d` after the + installation to get the user databases correct. See their respective + manual pages. + + When you start you first bash shell, you will get an awful prompt. You + might also have a `PATH` environment variable that contains backslashes + and such. Edit `$HOME/.profile` and `$HOME/.bashrc` to set fair prompts + and set a correct PATH. Also do a `export SHELL` in `.profile`. For some + non-obvious reason the environment variable `$SHELL` is not exported in + bash. Also note that `.profile` is run at login time and `.bashrc` when + sub shells are created. You'll need to explicitly source `.bashrc` from + `.profile` if you want the commands there to be run at login time (like + setting up aliases, shell functions and the like). I personally + usually do like this at the end of `.profile`: + + ENV=$HOME/.bashrc + export ENV + . $ENV + + You might also, if you're a hard core type of person at least, want to + setup X-windows (XFree86), that might be as easy as running startx + from the command prompt and it might be much harder. Use Google to + find help... + + If you don't use X-windows, you might want to setup the Windows + console window by selecting properties in the console system menu + (upper left corner of the window, the Cygwin icon in the title + bar). Especially setting a larger screen buffer size (lines) is useful + as it gets you a scrollbar so you can see whatever error messages + that might appear... + + If you want to use (t)csh instead of bash you're on your own, I + haven't tried and know of no one that has. I expect + that you use bash in all shell examples. + +* Alternatively you download MinGW and Msys. You'll find the latest + installer at: + + URL: <http://sourceforge.net/projects/mingw/files/Installer/mingw-get-inst/> + + Make sure to install everything they've got. + + To be able to build the 64bit VM, you will also need the 64bit + MinGW compiler from: + + URL: <http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Automated%20Builds/> + + The latest version should do it. Make sure you download the + `mingw-w64-bin_i686-mingw_<something>.zip`, not a linux + version. You unzip the package on top of your MinGW installation + (`c:\MinGW`) and that's it. + + Setting up your environment in Msys is similar to setting it up in + Cygwin. + +* Microsofts Windows SDK version 7.1 (corresponding to VC++ 10.0 and + Visual Studio 2010). You'll find it here: + + URL: <http://www.microsoft.com/download/en/details.aspx?id=8279> + + but before you install that, you need to have .Net 4 installed, + you'll find that here: + + URL: <http://www.microsoft.com/download/en/details.aspx?id=17851> + + Use the web installer for the SDK, at least when I tried + downloading the whole package as an image, I got SDK 7.0 instead, + which is not what you want... + + There will be a Windows command file in `%PROGRAMFILES%\Mirosoft + SDKs\Windows\v7.1\Bin\SetEnv.cmd` that set's the appropriate + environment for a Windows command prompt. This is not appropriate + for bash, so you'll need to convert it to bash-style environments + by editing your `.bash_profile`. In my case, where the SDK is + installed in the default directory and `%PROGRAMFILES%` is + `C:\Program Files`, the commands for setting up a 32bit build + environment (on a 64bit or 32bit machine) look like this (in cygwin): + + # Some common paths + C_DRV=/cygdrive/c + PRG_FLS=$C_DRV/Program\ Files + + # nsis + NSIS_BIN=$PRG_FLS/NSIS + # java + JAVA_BIN=$PRG_FLS/Java/jdk1.6.0_16/bin + + ## + ## MS SDK + ## + + CYGWIN=nowinsymlinks + MVS10="$PRG_FILES/Microsoft Visual Studio 10.0" + WIN_MVS10="C:\\Program Files\\Microsoft Visual Studio 10.0" + SDK10="$PRG_FILES/Microsoft SDKs/Windows/v7.1" + WIN_SDK10="C:\\Program Files\\Microsoft SDKs\\Windows\\v7.1" + + PATH="$NSIS_BIN:\ + $MVS10/Common7/IDE:\ + $MVS10/Common7/Tools:\ + $MVS10/VC/Bin:\ + $MVS10/VC/Bin/VCPackages:\ + $SDK10/Bin/NETFX 4.0 Tools:\ + $SDK10/Bin:\ + /usr/local/bin:/usr/bin:/bin:\ + /cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:\ + /cygdrive/c/WINDOWS/system32/Wbem:\ + $JAVA_BIN" + + LIBPATH="$WIN_MVS10\\VC\\LIB" + + LIB="$WIN_MVS10\\VC\\LIB;$WIN_SDK10\\LIB" + + INCLUDE="$WIN_MVS10\\VC\\INCLUDE;$WIN_SDK10\\INCLUDE;$WIN_SDK10\\INCLUDE\\gl" + + export CYGWIN PATH LIBPATH LIB INCLUDE + + If you're using Msys instead, the only thing you need to change is + the `C_DRV` setting, which would read: + + C_DRV=/c + + And of course you might need to change `C:\Program Files` etc if + you're using a non-english version of Windows (XP). Note that in + later versions of Windows, the national adoptions of the program + files directories etc are not on the file system but only in the + explorer, so even if explorer says that your programs reside in + e.g. `C:\Program`, they might still reside in `C:\Program Files` + in reality... + + If you are building a 64 bit version of Erlang, you should set up + PATHs etc a little differently. I use the following script to + make things work in both Cygwin and Msys: + + make_winpath() + { + P=$1 + if [ "$IN_CYGWIN" = "true" ]; then + cygpath -d "$P" + else + (cd "$P" && /bin/cmd //C "for %i in (".") do @echo %~fsi") + fi + } + + make_upath() + { + P=$1 + if [ "$IN_CYGWIN" = "true" ]; then + cygpath "$P" + else + echo "$P" | /bin/sed 's,^\([a-zA-Z]\):\\,/\L\1/,;s,\\,/,g' + fi + } + + # Some common paths + if [ -x /usr/bin/msysinfo ]; then + # Without this the path conversion won't work + COMSPEC='C:\Windows\SysWOW64\cmd.exe' + MSYSTEM=MINGW32 + export MSYSTEM COMSPEC + IN_CYGWIN=false + else + CYGWIN=nowinsymlinks + export CYGWIN + IN_CYGWIN=true + fi + + if [ "$IN_CYGWIN" = "true" ]; then + PATH=/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:\ + /cygdrive/c/windows/system32:/cygdrive/c/windows:/cygdrive/c/windows/system32/Wbem + else + PATH=/usr/local/bin:/mingw/bin:/bin:/c/Windows/system32:/c/Windows:\ + /c/Windows/System32/Wbem + fi + + if [ "$IN_CYGWIN" = "true" ]; then + C_DRV=/cygdrive/c + else + C_DRV=/c + fi + + PRG_FLS64=$C_DRV/Program\ Files + PRG_FLS32=$C_DRV/Program\ Files\ \(x86\) + VISUAL_STUDIO_ROOT32=$PRG_FLS32/Microsoft\ Visual\ Studio\ 10.0 + MS_SDK_ROOT64=$PRG_FLS64/Microsoft\ SDKs/Windows/v7.1 + + # Okay, now mangle the paths and get rid of spaces by using short names + WIN_VCROOT32=`make_winpath "$VISUAL_STUDIO_ROOT32"` + VCROOT32=`make_upath $WIN_VCROOT32` + WIN_SDKROOT64=`make_winpath "$MS_SDK_ROOT64"` + SDKROOT64=`make_upath $WIN_SDKROOT64` + WIN_PROGRAMFILES32=`make_winpath "$PRG_FLS32"` + PROGRAMFILES32=`make_upath $WIN_PROGRAMFILES32` + + WIN_PROGRAMFILES64=`make_winpath "$PRG_FLS64"` + PROGRAMFILES64=`make_upath $WIN_PROGRAMFILES64` + + # nsis + NSIS_BIN=$PROGRAMFILES32/NSIS + # java + JAVA_BIN=$PROGRAMFILES64/Java/jdk1.7.0_01/bin + + ## The PATH variable should be Unix'ish + VCPATH=$VCROOT32/Common7/IDE:$VCROOT32/VC/BIN/amd64:$VCROOT32/Common7/Tools:\ + $VCROOT32/VC/VCPackages:$SDKROOT64/bin/NETFX4~1.0TO/x64:$SDKROOT64/bin/x64:\ + $SDKROOT64/bin + + ## Microsoft SDK libs + + LIBPATH=$WIN_VCROOT32\\VC\\LIB\\amd64 + LIB=$WIN_VCROOT32\\VC\\LIB\\amd64\;$WIN_SDKROOT64\\LIB\\X64 + INCLUDE=$WIN_VCROOT32\\VC\\INCLUDE\;$WIN_SDKROOT64\\include\;\ + $WIN_SDKROOT64\\include\\gl + + # Put nsis, c compiler and java in path + PATH=$NSIS_BIN:$VCPATH:$PATH:$JAVA_BIN + + # Make sure LIB and INCLUDE is available for others + export PATH LIBPATH LIB INCLUDE + + All this is derived from the SetEnv.cmd command file mentioned + earlier. The bottom line is to set the PATH so that NSIS and + Microsoft SDK is found before the Msys/Cygwin tools and that Java + is last in the PATH. + + Make a simple hello world (maybe one that prints out + `sizeof(void *)`) and try to compile it with the `cl` command from within + bash. If that does not work, your environment needs fixing. Also + remember to fix up the PATH environment, especially old Erlang + installations might have inserted quoted paths that Cygwin/Msys + does not understand. Remove or correct such paths. There should be + no backslashes in your path environment variable in Cygwin bash, + but LIB and INCLUDE should contain Windows style paths with + semicolon, drive letters and backslashes. + +* Sun's Java JDK 1.5.0 or higher. Our Java code (jinterface, ic) is + written for JDK 1.5.0. Get it for Windows and install it, the JRE is + not enough. If you don't care about Java, you can skip this step, the + result will be that jinterface is not built. + + URL: <http://java.sun.com> + + Add javac *LAST* to your path environment in bash, in my case this means: + + `PATH="$PATH:/cygdrive/c/Program Files/Java/jdk1.5.0_17/bin"` + + No `CLASSPATH` or anything is needed. Type `javac` at the bash prompt + and you should get a list of available Java options. Make sure by + typing `type java` that you use the Java you installed. Note however that + Cygwin's `jar.exe` is used, that's why the JDK bin-directory should be + added last in the `PATH`. + +* Nullsoft NSIS installer system. You need this to build the self + installing package. It's a free open source installer that's much + nicer to use than the commercial Wise and Install shield + installers. This is the installer we use for commercial releases as + well from R9C an on. + + URL: <http://www.nullsoft.com/free/nsis> + + Install the lot, especially the modern user interface components, as + it's definitely needed. Put `makensis` in your path, in my case: + + PATH=/cygdrive/c/Program\ Files/NSIS:$PATH + + type makensis at the bash prompt and you should get a list of options + if everything is OK. + +* OpenSSL. This is if you want the SSL and crypto applications to + compile (and run). There are prebuilt binaries available, but I + strongly recommend building this yourself. It's quite easy. + + First get the source from + + URL: <http://openssl.org/source/> + + I would recommend using 0.9.8r. + + Download the tar file and unpack it (using your bash prompt) into + a directory of your choise. + + You will need a Windowish Perl for the build. ActiveState has one: + + URL: <http://www.activestate.com/activeperl/downloads> + + Download and install that. Disable options to associate it with + the .pl suffix and/or adding things to PATH, they are not needed. + + Now fire up the Microsoft Windows SDK command prompt in RELEASE + mode for the architecture you are going to build. The easiest is + to copy the shortcut from the SDKs start menu item and edit the + command line in the shortcut (Right click->Properties) to end with + `/Release`. Make sure the banner when you double click your + shortcut (the text in the resulting command window) says + `Targeting Windows XP x64 Release` if you are going to do a 64 bit + build and `Targeting Windows XP x86 Release` if you are building a + 32 bit version. + + Now cd to where you unpacked the OpenSSL source using your Release + Windows command prompt (it should be on the same drive as where + you are going to install it if everything is to work smothly). + + C:\> cd <some dir> + + Add ActiveState (or some other windows perl, not cygwins) to your PATH: + + C:\...\> set PATH=C:\Perl\bin;%PATH% + + Or if you installed the 64bit perl: + + C:\...\> set PATH=C:\Perl64\bin;%PATH% + + Configure OpenSSL for 32 bit: + + C:\...\> perl Configure VC-WIN32 --prefix=/OpenSSL + + Or for 64 bit: + + C:\...\> perl Configure VC-WIN64A --prefix=/OpenSSL-Win64 + + Do some setup (for 32 bit): + + C:\...\> ms\do_ms + + The same for 64 bit: + + C:\...\> ms\do_win64a + + Then build static libraries and install: + + C:\...\> nmake -f ms\nt.mak + C:\...\> nmake -f ms\nt.mak install + + That's it - you now have your perfectly consistent static build of + openssl. If you want to get rid of any possibly patented + algorithms in the lib, just read up on the OpenSSL FAQ and follow + the instructions. + + The installation locations chosen are where configure will look + for OpenSSL, so try to keep them as is. + +* Building with wxWidgets. Download wxWidgets-2.8.9 or higher patch + release (2.9.\* is a developer release which currently does not work + with wxErlang). + + Install or unpack it to `DRIVE:/PATH/cygwin/opt/local/pgm`. + + edit: `C:\cygwin\opt\local\pgm\wxMSW-2.8.11\include\wx\msw\setup.h` + enable `wxUSE_GLCANVAS`, `wxUSE_POSTSCRIPT` and `wxUSE_GRAPHICS_CONTEXT` + + build: From a command prompt with the VC tools available (See the + instructions for OpenSSL build above for help on starting the + proper command prompt in RELEASE mode): + + C:\...\> cd C:\cygwin\opt\local\pgm\wxMSW-2.8.11\build\msw + C:\...\> nmake BUILD=release SHARED=0 UNICODE=1 USE_OPENGL=1 USE_GDIPLUS=1 DIR_SUFFIX_CPU= -f makefile.vc + C:\...\> cd C:\cygwin\opt\local\pgm\wxMSW-2.8.11\contrib\build\stc + C:\...\> nmake BUILD=release SHARED=0 UNICODE=1 USE_OPENGL=1 USE_GDIPLUS=1 DIR_SUFFIX_CPU= -f makefile.vc + + Or - if building a 64bit version: + + C:\...\> cd C:\cygwin\opt\local\pgm\wxMSW-2.8.11\build\msw + C:\...\> nmake TARGET_CPU=amd64 BUILD=release SHARED=0 UNICODE=1 USE_OPENGL=1 USE_GDIPLUS=1 DIR_SUFFIX_CPU= -f makefile.vc + C:\...\> cd C:\cygwin\opt\local\pgm\wxMSW-2.8.11\contrib\build\stc + C:\...\> nmake TARGET_CPU=amd64 BUILD=release SHARED=0 UNICODE=1 USE_OPENGL=1 USE_GDIPLUS=1 DIR_SUFFIX_CPU= -f makefile.vc + +* The Erlang source distribution (from <http://www.erlang.org/download.html>). + The same as for Unix platforms. Preferably use tar from within Cygwin to + unpack the source tar.gz (`tar zxf otp_src_%OTP-REL%.tar.gz`). + + set the environment `ERL_TOP` to point to the root directory of the + source distribution. Let's say I stood in `$HOME/src` and unpacked + `otp_src_%OTP-REL%.tar.gz`, I then add the following to `.profile`: + + ERL_TOP=$HOME/src/otp_src_%OTP-REL% + export $ERL_TOP + +* The TCL/TK binaries. You could compile Tcl/Tk for windows yourself, + but you can get a stripped down version from our website which is + suitable to include in the final binary package. If you want to supply + tcl/tk yourself, read the instructions about how the tcl/tk tar file + used in the build is constructed under `$ERL_TOP/lib/gs/tcl`. The easy + way is to download <http://www.erlang.org/download/tcltk85_win32_bin.tar.gz> + and unpack it standing in the `$ERL_TOP` directory. This will create the + file `win32.tar.gz` in `$ERL_TOP/lib/gs/tcl/binaries`. + + One last alternative is to create a file named `SKIP` in the + `$ERL_TOP/lib/gs/` after configure is run, but that will give you an + erlang system without gs (which might be okay as you probably will use + wx anyway). + + Note that there is no special 64bit version of TCL/TK needed, you + can use the 32bit program even for a 64bit build. + +The Shell Environment +--------------------- + +So, if you have followed the instructions above, when you start a bash +shell, you should have an INCLUDE environment with a Windows style +path, a LIB environment variable also in Windows style, and finally a +PATH that let's you reach cl, makensis, javac etc from the +command prompt (use `which cl` etc to verify from bash). + +You should also have an `ERL_TOP` environment variable that is *Cygwin +style*, and points to a directory containing, among other files, the +script `otp_build`. + +A final massage of the environment is needed, and that is done by +the script `$ERL_TOP/otp_build`. Start bash and do the following, note +the "back-ticks" (\`), can be quite hard to get on some keyboards, but +pressing the back-tick key followed by the space bar might do it... + + $ cd $ERL_TOP + $ eval `./otp_build env_win32` + +If you're unable to produce back-ticks on your keyboard, you can use +the ksh variant: + + $ cd $ERL_TOP + $ eval $(./otp_build env_win32) + +If you are building a 64 bit version, you supply `otp_build` with an architecture parameter: + + $ cd $ERL_TOP + $ eval `./otp_build env_win32 x64` + + +This should do the final touch to the environment and building should +be easy after this. You could run `./otp_build env_win32` without +`eval` just to see what it does, and to see that the environment it +sets seems OK. The path is cleaned of spaces if possible (using DOS +style short names instead), the variables `OVERRIDE_TARGET`, `CC`, `CXX`, +`AR` and `RANLIB` are set to their respective wrappers and the directories +`$ERL_TOP/erts/etc/win32/cygwin_tools/vc` and +`$ERL_TOP/erts/etc/win32/cygwin_tool` are added first in the PATH. + +Try now a `type erlc`. That should result in the erlc wrapper script +(which does not have the .sh extension, for reasons best kept +untold...). It should reside in `$ERL_TOP/erts/etc/win32/cygwin_tools` +or `$ERL_TOP/erts/etc/win32/msys_tools`. You could also try `which +cc.sh`, which `ar.sh` etc. + +Now you're ready to build... + + +Building and Installing +----------------------- + +Now it's assumed that you have executed `` eval `./otp_build env_win32` `` or +`` eval `./otp_build env_win32 x64` `` for this particular shell... + +Building is easiest using the `otp_build` script. That script takes care +of running configure, bootstrapping etc on Windows in a simple +way. The `otp_build` script is the utility we use ourselves to build on +different platforms and it therefore contains code for all sorts of +platforms. The principle is, however, that for non-Unix platforms, one +uses `./otp_build env_<target>` to set up environment and then the +script knows how to build on the platform "by itself". You've already +run `./otp_build env_win32` in the step above, so now it's mostly like +we build on any platform. OK, here are then steps; Assuming you will +want to build a full installation executable with NSIS, you can omit +`<installation directory>` and the release will be copied to +`$ERL_TOP/release/win32`: and there is where the packed self installing +executable will reside too. + + $ ./otp_build autoconf # Ignore the warning blob about versions of autoconf + $ ./otp_build configure <optional configure options> + $ ./otp_build boot -a + $ ./otp_build release -a <installation directory> + $ ./otp_build installer_win32 <installation directory> # optional + +Now you will have a file called `otp_win32_R12B.exe` in the +`<installation directory>`, i.e. `$ERL_TOP/release/win32`. + +Lets get into more detail: + +1. `$ ./otp_build autoconf` - This step rebuilds the configure scripts + to work correctly in the cygwin environment. In an ideal world, this + would not be needed, but alas, we have encountered several + incompatibilities between our distributed configure scripts (generated + on a Linux platform) and the cygwin environment over the + years. Running autoconf on cygwin ensures that the configure scripts + are generated in a cygwin-compatible way and that they will work well + in the next step. + +2. `$ ./otp_build configure` - This runs the newly generated configure + scripts with options making configure behave nicely. The target machine + type is plainly `win32`, so a lot of the configure-scripts recognize + this awkward target name and behave accordingly. The CC variable also + makes the compiler be `cc.sh`, which wraps MSVC++, so all configure + tests regarding the C compiler gets to run the right compiler. A lot of + the tests are not needed on Windows, but I thought it best to run the + whole configure anyway. The only configure option you might want to + supply is `--with-ssl`, which might be needed if you have built your + own OpenSSL distribution. The Shining Lights distribution should be + found automatically by `configure`, if that fails, add a + `--with-ssl=<dir>` that specifies the root directory of your OpenSSL + installation. + +3. `$ ./otp_build boot -a` - This uses the bootstrap directory (shipped + with the source, `$ERL_TOP/bootstrap`) to build a complete OTP + system. It first builds an emulator and sets up a minimal OTP system + under `$ERL_TOP/bootstrap`, then starts to compile the different OTP + compilers to make the `$ERL_TOP/bootstrap` system potent enough to be + able to compile all Erlang code in OTP. Then, all Erlang and C code + under `$ERL_TOP/lib` is built using the bootstrap system, giving a + complete OTP system (although not installed). When this is done, one + can run Erlang from within the source tree, just type `$ERL_TOP/bin/erl` + and you should have a prompt. If you omit the -a flag, you'll get a + smaller system, that might be useful during development. Now + exit from Erlang and start making a release of the thing: + +4. `$ ./otp_build release -a` - Builds a commercial release tree from the + source tree, default is to put it in `$ERL_TOP/release/win32`, you can + give any directory as parameter (Cygwin style), but it doesn't really + matter if you're going to build a self extracting installer too. You + could of course build release to the final directory and then run + `./Install.exe` standing in the directory where the release was put, + that will create a fully functional OTP installation. But let's make + the nifty installer: + +5. `$ ./otp_build installer_win32` - Create the self extracting installer + executable. The executable `otp_win32_%OTP-REL%.exe` will be placed + in the top directory of the release created in the previous step. If + no release directory is specified, the release is expected to have + been built to `$ERL_TOP/release/win32`, which also will be the place + where the installer executable will be placed. If you specified some + other directory for the release (i.e. `./otp_build release -a + /tmp/erl_release`), you're expected to give the same parameter here, + (i.e. `./otp_build installer_win32 /tmp/erl_release`). You need to have + a full NSIS installation and `makensis.exe` in your path for this to + work of course. Once you have created the installer, you can run it to + install Erlang/OTP in the regular way, just run the executable and + follow the steps in the installation wizard. To get all default settings + in the installation without any questions asked, you run the executable + with the parameter `/S` (capital S) like in: + + $ cd $ERL_TOP + $ release/win32/otp_win32_%OTP-REL% /S + ... + + or + + $ cd $ERL_TOP + $ release/win32/otp_win64_%OTP-REL% /S + ... + + + and after a while Erlang/OTP-%OTP-REL% will have been installed in + `C:\Program Files\erl%ERTS-VSN%\`, with shortcuts in the menu etc. + + The necessary setup of an Erlang installation is actually done by the + program `Install.exe`, which resides in the release top. That program + creates `.ini`-files and copies the correct boot scripts. If one has + the correct directory tree (like after a `./otp_build release -a`), only + the running of `Install.exe` is necessary to get a fully functional + OTP. What the self extracting installer adds is (of course) the + possibility to distribute the binary easily, together with adding + shortcuts to the Windows start menu. There is also some adding of + entries in the registry, to associate `.erl` and `.beam` files with + Erlang and get nifty icons, but that's not something you'll really need + to run Erlang. The registry is also used to store uninstall information, + but if one has not used the self extracting installer, one cannot + (need not) do any uninstall, one just scratches the release directory + and everything is gone. Erlang/OTP does not *need* to put anything + in the Windows registry at all, and does not if you don't use the self + extracting installer. In other words the installer is pure cosmetics. + +> *NOTE*: Beginning with R9C, the Windows installer does *not* add Erlang +> to the system wide path. If one wants to have Erlang in the path, one +> has to add it by hand. + +Development +----------- + +Once the system is built, you might want to change it. Having a test +release in some nice directory might be useful, but you also can run +Erlang from within the source tree. The target `local_setup`, makes +the program `$ERL_TOP/bin/erl.exe` usable and it also uses all the OTP +libraries in the source tree. + +If you hack the emulator, you can then build the emulator executable +by standing in `$ERL_TOP/erts/emulator` and do a simple + + $ make opt + +Note that you need to have run ``(cd $ERL_TOP && eval `./otp_build env_win32`)`` +in the particular shell before building anything on Windows. After +doing a make opt you can test your result by running `$ERL_TOP/bin/erl`. +If you want to copy the result to a release directory (say +`/tmp/erl_release`), you do this (still in `$ERL_TOP/erts/emulator`) + + $ make TESTROOT=/tmp/erl_release release + +That will copy the emulator executables. + +To make a debug build of the emulator, you need to recompile both +`beam.dll` (the actual runtime system) and `erlexec.dll`. Do like this + + $ cd $ERL_TOP + $ rm bin/win32/erlexec.dll + $ cd erts/emulator + $ make debug + $ cd ../etc + $ make debug + +and sometimes + + $ cd $ERL_TOP + $ make local_setup + +So now when you run `$ERL_TOP/erl.exe`, you should have a debug compiled +emulator, which you will see if you do a: + + 1> erlang:system_info(system_version). + +in the erlang shell. If the returned string contains `[debug]`, you +got a debug compiled emulator. + +To hack the erlang libraries, you simply do a `make opt` in the +specific "applications" directory, like: + + $ cd $ERL_TOP/lib/stdlib + $ make opt + +or even in the source directory... + + $ cd $ERL_TOP/lib/stdlib/src + $ make opt + +Note that you're expected o have a fresh Erlang in your path when +doing this, preferably the plain %OTP-REL% you have built in the previous +steps. You could also add `$ERL_TOP/bootstrap/bin` to your `PATH` before +rebuilding specific libraries, that would give you a good enough +Erlang system to compile any OTP erlang code. Setting up the path +correctly is a little bit tricky, you still need to have +`$ERL_TOP/erts/etc/win32/cygwin_tools/vc` and +`$ERL_TOP/erts/etc/win32/cygwin_tools` *before* the actual emulator +in the path. A typical setting of the path for using the bootstrap +compiler would be: + + $ export PATH=$ERL_TOP/erts/etc/win32/cygwin_tools/vc\ + :$ERL_TOP/erts/etc/win32/cygwin_tools:$ERL_TOP/bootstrap/bin:$PATH + +That should make it possible to rebuild any library without hassle... + +If you want to copy a library (an application) newly built, to a +release area, you do like with the emulator: + + $ cd $ERL_TOP/lib/stdlib + $ make TESTROOT=/tmp/erlang_release release + +Remember that: + +* Windows specific C-code goes in the `$ERL_TOP/erts/emulator/sys/win32`, + `$ERL_TOP/erts/emulator/drivers/win32` or `$ERL_TOP/erts/etc/win32`. + +* Windows specific erlang code should be used conditionally and the + host OS tested in *runtime*, the exactly same beam files should be + distributed for every platform! So write code like: + + case os:type() of + {win32,_} -> + do_windows_specific(); + Other -> + do_fallback_or_exit() + end, + +That's basically all you need to get going. + +Using GIT +--------- + +You might want to check out versions of the source code from GitHUB. That is possible directly in cygwin, but not in Msys. There is a project MsysGIT: + +URL:<http://code.google.com/p/msysgit/> + +that makes a nice Git port. The msys prompt you get from MsysGIT is +however not compatible with the full version from MinGW, so you will +need to check out files using MsysGIT's command prompt and then switch +to a common Msys command prompt for building. Also all test suites +cannot be built as MsysGIT/Msys does not handle symbolic links. To +build test suites on Windows, you will need Cygwin for now. Hopefully +all symbolic links will disappear from our repository soon and this +issue will disappear. + +Final Words +----------- +My hope is that the possibility to build the whole system on Windows +will open up for free development on this platform too. There are many +things one might want to do better in the Windows version, like the +window-style command prompt as well as pure Cygwin porting. Although i +realize it's a much larger step to start building on Windows (with all +the software you need) than for instance on Linux, I sincerely hope +that some of you will make the effort and start submitting Windows +friendly patches. + +The first build system for Erlang using Cygwin on Windows was created +by Per Bergkvist. I haven't used his build system, but it's rumored to +be good. The idea to do this came from his work, so credit is well +deserved. + +Of course this would have been completely impossible without the +excellent Cygwin. The guys at Cygnus solutions and +Redhat deserve a huge THANKS! as well as all the other people in the +free software community who have helped in creating the magnificent +software that constitutes Cygwin. + +Also the people developing the alternative command prompt Msys and +the MinGW compiler are worth huge THANKS! The 64bit port would have +been impossible without the 64bit MinGW compiler. + +Good luck and Happy Hacking, +Patrik, OTP + +Copyright and License +--------------------- + +%CopyrightBegin% + +Copyright Ericsson AB 2003-2012. All Rights Reserved. + +The contents of this file are subject to the Erlang Public License, +Version 1.1, (the "License"); you may not use this file except in +compliance with the License. You should have received a copy of the +Erlang Public License along with this software. If not, it can be +retrieved online at http://www.erlang.org/. + +Software distributed under the License is distributed on an "AS IS" +basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +the License for the specific language governing rights and limitations +under the License. + +%CopyrightEnd% + +Modifying This Document +----------------------- + +Before modifying this document you need to have a look at the +[$ERL_TOP/HOWTO/MARKDOWN.md][] document. + + + + [1]: http://www.erlang.org/faq.html "mailing lists" + [$ERL_TOP/HOWTO/MARKDOWN.md]: MARKDOWN.md + + [?TOC]: true diff --git a/HOWTO/INSTALL.md b/HOWTO/INSTALL.md new file mode 100644 index 0000000000..4f7c317a47 --- /dev/null +++ b/HOWTO/INSTALL.md @@ -0,0 +1,843 @@ +Building and Installing Erlang/OTP +================================== + +Introduction +------------ + +This document describes how to build and install Erlang/OTP-%OTP-REL%. You +are advised to read the whole document before attempting to build and install +Erlang/OTP. You can find more information about Open Source Erlang/OTP at: + + <http://www.erlang.org/> + +The source code for Erlang/OTP can also be found in a Git repository: + + <http://github.com/erlang/otp> + +Erlang/OTP should be possible to build from source on any Unix system, +including Mac OS X. This document describes how to native compile Erlang/OTP +on Unix. For detailed instructions on how to + +* cross compile Erlang/OTP, see the [$ERL_TOP/HOWTO/INSTALL-CROSS.md][] + document. + +* build Erlang/OTP on Windows, see the [$ERL_TOP/HOWTO/INSTALL-WIN32.md][] + document. + + Binary releases for Windows can be found at + <http://www.erlang.org/download.html>. + +Before reading the above mentioned documents you are in any case advised to +read this document first, since it covers building Erlang/OTP in general as +well as other important information. + +Daily Build and Test +-------------------- +At Ericsson we have a "Daily Build and Test" that runs on: + +* Solaris 8, 9 + * Sparc32 + * Sparc64 +* Solaris 10 + * Sparc32 + * Sparc64 + * x86 +* SuSE Linux/GNU 9.4, 10.1 + * x86 +* SuSE Linux/GNU 10.0, 10.1, 11.0 + * x86 + * x86\_64 +* openSuSE 11.4 (Celadon) + * x86\_64 (valgrind) +* Fedora 7 + * PowerPC +* Fedora 14 + * x86\_64 +* Gentoo Linux/GNU 1.12.11.1 + * x86 +* Ubuntu Linux/GNU 7.04, 10.04, 10.10, 11.0 + * x86\_64 +* MontaVista Linux/GNU 4.0.1 + * PowerPC +* FreeBSD 8.2 + * x86 +* OpenBSD 5.0 + * x86\_64 +* Mac OS X 10.5.8 (Leopard), 10.6.0 (Snow Leopard), 10.7.3 (Lion) + * x86 +* Windows XP SP3, 2003, Vista, 7 + * x86 +* Windows 7 + * x86\_64 + +We also have the following "Daily Cross Builds": + +* SuSE Linux/GNU 10.1 x86 -> SuSE Linux/GNU 10.1 x86\_64 +* SuSE Linux/GNU 10.1 x86\_64 -> Linux/GNU TILEPro64 + +and the following "Daily Cross Build Tests": + +* SuSE Linux/GNU 10.1 x86\_64 + +Versions Known NOT to Work +-------------------------- + +* Suse linux 9.1 is shipped with a patched GCC version 3.3.3, having the + rpm named `gcc-3.3.3-41`. That version has a serious optimization bug + that makes it unusable for building the Erlang emulator. Please + upgrade GCC to a newer version before building on Suse 9.1. Suse Linux + Enterprise edition 9 (SLES9) has `gcc-3.3.3-43` and is not affected. + +* `gcc-4.3.0` has a serious optimizer bug. It produces an Erlang emulator + that will crash immediately. The bug is supposed to be fixed in + `gcc-4.3.1`. + +* FreeBSD had a bug which caused `kqueue`/`poll`/`select` to fail to detect + that a `writev()` on a pipe has been made. This bug should have been fixed + in FreeBSD 6.3 and FreeBSD 7.0. NetBSD and DragonFlyBSD probably have or + have had the same bug. More information can be found at: + + * <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sys_pipe.c> + * <http://lists.freebsd.org/pipermail/freebsd-arch/2007-September/006790.html> + +* `getcwd()` on Solaris 9 can cause an emulator crash. If you have + async-threads enabled you can increase the stack size of the + async-threads as a temporary workaround. See the `+a` command-line + argument in the documentation of `erl(1)`. Without async-threads the + emulator is not as vulnerable to this bug, but if you hit it without + async-threads the only workaround available is to enable async-threads + and increase the stack size of the async-threads. Sun has however + released patches that fixes the issue: + + > Problem Description: 6448300 large mnttab can cause stack overrun + > during Solaris 9 getcwd + + More information can be found at: + + * <http://sunsolve.sun.com/search/document.do?assetkey=1-21-112874-40-1&searchclause=6448300> + * <http://sunsolve.sun.com/search/document.do?assetkey=1-21-114432-29-1&searchclause=6448300> + +Required Utilities +------------------ + +These are the tools you will need in order to unpack and build Erlang/OTP. + +### Unpacking ### + +* GNU unzip, or a modern uncompress. +* A TAR program that understands the GNU TAR format for long filenames + (such as GNU TAR). + +### Building ### + +* GNU `make` +* `gcc` -- GNU C compiler +* Perl 5 +* GNU `m4` -- If HiPE (native code) support is enabled. HiPE can be + disabled using `--disable-hipe` +* `ncurses`, `termcap`, or `termlib` -- The development headers and + libraries are needed, often known as `ncurses-devel`. Use + `--without-termcap` to build without any of these libraries. Note that + in this case only the old shell (without any line editing) can be used. +* OpenSSL -- Optional, but needed for building the Erlang/OTP applications + `ssl` and `crypto`. You need the "development package" of OpenSSL, i.e. + including the header files. For building the application `ssl` the OpenSSL + binary command program `openssl` is also needed. At least version 0.9.8 + of OpenSSL is required. Can be downloaded from <http://www.openssl.org>. +* Sun Java jdk-1.5.0 or higher -- Optional but needed for building the + Erlang/OTP application `jinterface` and parts of `ic` and `orber`. Can + be downloaded from <http://java.sun.com>. We have also tested IBM's + JDK 1.5.0. +* X Windows -- Optional, but development headers and libraries are needed + to build the Erlang/OTP application `gs` on Unix/Linux. +* `sed` -- There seem to be some problems with some of the `sed` version on + Solaris. Make sure `/bin/sed` or `/usr/bin/sed` is used on the Solaris + platform. +* `flex` -- Optional, headers and libraries are needed to build the `flex` + scanner for the `megaco` application on Unix/Linux. + +#### Building Documentation #### + +* `xsltproc` -- XSLT processor. A tool for applying XSLT stylesheets + to XML documents. Can be downloaded from + <http://xmlsoft.org/XSLT/xsltproc2.html>. +* `fop` -- Apache FOP print formatter (requires Java). Can be downloaded + from <http://xmlgraphics.apache.org/fop>. + +#### Building in Git #### + +* GNU `autoconf` of at least version 2.59. Note that `autoconf` is not + needed when building an unmodified version of the released source. + +### Installing ### + +* An `install` program that can take multiple file names. + +How to Build and Install Erlang/OTP +----------------------------------- + +The following instructions are for building [the released source tar ball][]. + +The variable `$ERL_TOP` will be mentioned a lot of times. It refers to +the top directory in the source tree. More information about `$ERL_TOP` +can be found in the [make and $ERL_TOP][] section below. If you are +building in git you probably want to take a look at the [Building in Git][] +section below before proceeding. + +### Unpacking ### + +Step 1: Start by unpacking the Erlang/OTP distribution file with your GNU +compatible TAR program. + + $ gunzip -c otp_src_%OTP-REL%.tar.gz | tar xf - + +alternatively: + + $ zcat otp_src_%OTP-REL%.tar.gz | tar xf - + + +Step 2: Now cd into the base directory (`$ERL_TOP`). + + $ cd otp_src_%OTP-REL% + +### Configuring ### + +Step 3: On some platforms Perl may behave strangely if certain locales are +set, so optionally you may need to set the LANG variable: + + # Bourne shell + $ LANG=C; export LANG + +or + + # C-Shell + $ setenv LANG C + +Step 4: Run the following commands to configure the build: + + $ ./configure [ options ] + +By default, Erlang/OTP will be installed in `/usr/local/{bin,lib/erlang}`. +To instead install in `<BaseDir>/{bin,lib/erlang}`, use the +`--prefix=<BaseDir>` option. + +If you upgraded the source with some patch you may need to clean up +from previous builds before the new build. Before doing a `make clean`, +be sure to read the [Pre-built Source Release][] section below. + +### Building ### + +Step 5: Build the Erlang/OTP package. + + $ make + +### Installing ### + +Step 6: Install then Erlang/OTP package + + $ make install + +### A Closer Look at the individual Steps ### + +Let us go through them in some detail. + +#### Configuring #### + +Step 4 runs a configuration script created by the GNU autoconf utility, which +checks for system specific features and then creates a number of makefiles. + +The configure script allows you to customize a number of parameters; +type `./configure --help` or `./configure --help=recursive` for details. +`./configure --help=recursive` will give help for all `configure` scripts in +all applications. + +One of the things you can specify is where Erlang/OTP should be installed. By +default Erlang/OTP will be installed in `/usr/local/{bin,lib/erlang}`. +To keep the same structure but install in a different place, `<Dir>` say, +use the `--prefix` argument like this: `./configure --prefix=<Dir>`. + +Some of the available `configure` options are: + +* `--prefix=PATH` - Specify installation prefix. +* `--{enable,disable}-threads` - Thread support (enabled by default if + possible) +* `--{enable,disable}-smp-support` - SMP support (enabled by default if + possible) +* `--{enable,disable}-kernel-poll` - Kernel poll support (enabled by + default if possible) +* `--{enable,disable}-hipe` - HiPE support (enabled by default on supported + platforms) +* `--enable-darwin-universal` - Build universal binaries on darwin i386. +* `--enable-darwin-64bit` - Build 64-bit binaries on darwin +* `--enable-m64-build` - Build 64-bit binaries using the `-m64` flag to + `(g)cc` +* `--enable-m32-build` - Build 32-bit binaries using the `-m32` flag to + `(g)cc` +* `--{with,without}-termcap` - termcap (without implies that only the old + Erlang shell can be used) +* `--with-javac=JAVAC` - Specify Java compiler to use +* `--{with,without}-javac` - Java compiler (without implies that the + `jinterface` application won't be built) +* `--{enable,disable}-dynamic-ssl-lib` - Dynamic OpenSSL libraries +* `--{enable,disable}-shared-zlib` - Shared zlib library +* `--with-ssl=PATH` - Specify location of OpenSSL include and lib +* `--{with,without}-ssl` - OpenSSL (without implies that the `crypto`, + `ssh`, and `ssl` won't be built) +* `--with-libatomic_ops=PATH` - Use the `libatomic_ops` library for atomic + memory accesses. If `configure` should inform you about no native atomic + implementation available, you typically want to try using the + `libatomic_ops` library. It can be downloaded from + <http://www.hpl.hp.com/research/linux/atomic_ops/>. + +If you or your system has special requirements please read the `Makefile` for +additional configuration information. + +#### Building #### + +Step 5 builds the Erlang/OTP system. On a fast computer, this will take about +5 minutes. After completion of this step, you should have a working +Erlang/OTP system which you can try by typing `bin/erl`. This should start +up Erlang/OTP and give you a prompt: + + $ bin/erl + Erlang %OTP-REL% (erts-%ERTS-VSN%) [source] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] + + Eshell V%ERTS-VSN% (abort with ^G) + 1> _ + +#### Installing #### + +Step 6 is optional. It installs Erlang/OTP at a standardized location (if you +change your mind about where you wish to install you can rerun step 4, +without having to do step 5 again). + +##### Alternative Installation Procedures ##### + +* Staged install using [DESTDIR][]. You can perform the install + phase in a temporary directory and later move the installation into + its correct location by use of the `DESTDIR` variable: + + $ make DESTDIR=<tmp install dir> install + + The installation will be created in a location prefixed by `$DESTDIR`. + It can, however, not be run from there. It needs to be moved into the + correct location before it can be run. If `DESTDIR` have not been set + but `INSTALL_PREFIX` has been set, `DESTDIR` will be set to + `INSTALL_PREFIX`. Note that `INSTALL_PREFIX` in pre R13B04 was buggy + and behaved as `EXTRA_PREFIX` (see below). There are lots of areas of + use for an installation procedure using `DESTDIR`, e.g. when creating + a package, cross compiling, etc. Here is an example where the + installation should be located under `/opt/local`: + + $ ./configure --prefix=/opt/local + $ make + $ make DESTDIR=/tmp/erlang-build install + $ cd /tmp/erlang-build/opt/local + $ # gnu-tar is used in this example + $ tar -zcf /home/me/my-erlang-build.tgz * + $ su - + Password: ***** + $ cd /opt/local + $ tar -zxf /home/me/my-erlang-build.tgz + +* Install using the `release` target. Instead of doing `make install` you + can create the installation in whatever directory you like using the + `release` target and run the `Install` script yourself. `RELEASE_ROOT` + is used for specifying the directory where the installation should be + created. This is what by default ends up under `/usr/local/lib/erlang` + if you do the install using `make install`. All installation paths + provided in the `configure` phase are ignored, as well as `DESTDIR`, + and `INSTALL_PREFIX`. If you want links from a specific `bin` directory + to the installation you have to set those up yourself. An example where + Erlang/OTP should be located at `/home/me/OTP`: + + $ ./configure + $ make + $ make RELEASE_ROOT=/home/me/OTP release + $ cd /home/me/OTP + $ ./Install -minimal /home/me/OTP + $ mkdir -p /home/me/bin + $ cd /home/me/bin + $ ln -s /home/me/OTP/bin/erl erl + $ ln -s /home/me/OTP/bin/erlc erlc + $ ln -s /home/me/OTP/bin/escript escript + ... + + The `Install` script should currently be invoked as follows in the + directory where it resides (the top directory): + + $ ./Install [-cross] [-minimal|-sasl] <ERL_ROOT> + + where: + + * `-minimal` Creates an installation that starts up a minimal amount + of applications, i.e., only `kernel` and `stdlib` are started. The + minimal system is normally enough, and is what `make install` uses. + * `-sasl` Creates an installation that also starts up the `sasl` + application. + * `-cross` For cross compilation. Informs the install script that it + is run on the build machine. + * `<ERL_ROOT>` - The absolute path to the Erlang installation to use + at run time. This is often the same as the current working directory, + but does not have to be. It can follow any other path through the + file system to the same directory. + + If neither `-minimal`, nor `-sasl` is passed as argument you will be + prompted. + +* Test install using `EXTRA_PREFIX`. The content of the `EXTRA_PREFIX` + variable will prefix all installation paths when doing `make install`. + Note that `EXTRA_PREFIX` is similar to `DESTDIR`, but it does *not* have + the same effect as `DESTDIR`. The installation can and have to be run + from the location specified by `EXTRA_PREFIX`. That is, it can be useful + if you want to try the system out, running test suites, etc, before doing + the real install without `EXTRA_PREFIX`. + +### Symbolic Links in --bindir ### + +When doing `make install` and the default installation prefix is used, +relative symbolic links will be created from `/usr/local/bin` to all public +Erlang/OTP executables in `/usr/local/lib/erlang/bin`. The installation phase +will try to create relative symbolic links as long as `--bindir` and the +Erlang bin directory, located under `--libdir`, both have `--exec-prefix` as +prefix. Where `--exec-prefix` defaults to `--prefix`. `--prefix`, +`--exec-prefix`, `--bindir`, and `--libdir` are all arguments that can be +passed to `configure`. One can force relative, or absolute links by passing +`BINDIR_SYMLINKS=relative|absolute` as arguments to `make` during the install +phase. Note that such a request might cause a failure if the request cannot +be satisfied. + +### Pre-built Source Release ### + +The source release is delivered with a lot of platform independent +build results already pre-built. If you want to remove these pre-built +files, invoke `./otp_build remove_prebuilt_files` from the `$ERL_TOP` +directory. After you have done this, you can build exactly the same way +as before, but the build process will take a much longer time. + +> *WARNING*: Doing `make clean` in an arbitrary directory of the source +> tree, may remove files needed for bootstrapping the build. +> +> Doing `./otp_build save_bootstrap` from the `$ERL_TOP` directory before +> doing `make clean` will ensure that it will be possible to build after +> doing `make clean`. `./otp_build save_bootstrap` will be invoked +> automatically when `make` is invoked from `$ERL_TOP` with either the +> `clean` target, or the default target. It is also automatically invoked +> if `./otp_build remove_prebuilt_files` is invoked. + +### Building in Git ### + +When building in a Git working directory you also have to have a GNU `autoconf` +of at least version 2.59 on your system, because you need to generate the +`configure` scripts before you can start building. + +The `configure` scripts are generated by invoking `./otp_build autoconf` in +the `$ERL_TOP` directory. The `configure` scripts also have to be regenerated +when a `configure.in` or `aclocal.m4` file has been modified. Note that when +checking out a branch a `configure.in` or `aclocal.m4` file may change +content, and you may therefore have to regenerate the `configure` scripts +when checking out a branch. Regenerated `configure` scripts imply that you +have to run `configure` and build again. + +> *NOTE*: Running `./otp_build autoconf` is **not** needed when building +> an unmodified version of the released source. + +Other useful information can be found at our github wiki: +<http://wiki.github.com/erlang/otp> + +### make and $ERL\_TOP ### + +All the makefiles in the entire directory tree use the environment +variable `ERL_TOP` to find the absolute path of the installation. The +`configure` script will figure this out and set it in the top level +Makefile (which, when building, it will pass on). However, when +developing it is sometimes convenient to be able to run make in a +subdirectory. To do this you must set the `ERL_TOP` variable +before you run make. + +For example, assume your GNU make program is called `make` and you +want to rebuild the application `STDLIB`, then you could do: + + $ cd lib/stdlib; env ERL_TOP=<Dir> make + +where `<Dir>` would be what you find `ERL_TOP` is set to in the top level +Makefile. + +The Erlang/OTP Documentation +---------------------------- + +### How to Build the Documentation ### + + $ cd $ERL_TOP + +If you have just built Erlang/OTP in the current source tree, you have +already ran `configure` and do not need to do this again; otherwise, run +`configure`. + + $ ./configure [Configure Args] + +When building the documentation you need a full Erlang/OTP-%OTP-REL% system in +the `$PATH`. + + $ export PATH=<Erlang/OTP-%OTP-REL% bin dir>:$PATH # Assuming bash/sh + +Build the documentation. + + $ make docs + +The documentation can be installed either using the `install-docs` target, +or using the `release_docs` target. + +* If you have installed Erlang/OTP using the `install` target, install + the documentation using the `install-docs` target. Install locations + determined by `configure` will be used. `$DESTDIR` can be used the + same way as when doing `make install`. + + $ make install-docs + +* If you have installed Erlang/OTP using the `release` target, install + the documentation using the `release_docs` target. You typically want + to use the same `RELEASE_ROOT` as when invoking `make release`. + + $ make release_docs RELEASE_ROOT=<release dir> + +#### Build Issues #### + +We have sometimes experienced problems with Sun's `java` running out of +memory when running `fop`. Increasing the amount of memory available +as follows has in our case solved the problem. + + $ export FOP_OPTS="-Xmx<Installed amount of RAM in MB>m" + +More information can be found at +<http://xmlgraphics.apache.org/fop/0.95/running.html#memory>. + +### How to Install the Pre-formatted Documentation ### + +Pre-formatted [html documentation][] and [man pages][] can be downloaded at +<http://www.erlang.org/download.html>. + +For some graphical tools to find the on-line help you have to install +the HTML documentation on top of the installed OTP applications, i.e. + + $ cd <ReleaseDir> + $ gunzip -c otp_html_%OTP-REL%.tar.gz | tar xf - + +For `erl -man <page>` to work the Unix manual pages have to be +installed in the same way, i.e. + + $ cd <ReleaseDir> + $ gunzip -c otp_man_%OTP-REL%.tar.gz | tar xf - + +Where `<ReleaseDir>` is + +* `<PrefixDir>/lib/erlang` if you have installed Erlang/OTP using + `make install`. +* `$DESTDIR<PrefixDir>/lib/erlang` if you have installed Erlang/OTP + using `make install DESTDIR=<TmpInstallDir>`. +* `RELEASE_ROOT` if you have installed using + `make release RELEASE_ROOT=<ReleaseDir>`. + +Support for SMP (Symmetric Multi Processing) +-------------------------------------------- + +An emulator with SMP support will be built by default on most platforms +if a usable POSIX thread library or native Windows threads is found. + +You can force building of an SMP emulator, by using +`./configure --enable-smp-support`. However, if configure does not +automatically enable SMP support, the build is very likely to fail. + +Use `./configure --disable-smp-support` if you for some reason do not +want to have the emulator with SMP support built. + +If SMP support is enabled, support for threaded I/O will also be turned on +(also in the emulator without SMP support). + +The `erl` command will automatically start the SMP emulator if the +computer has more than one logical processor. You can force a start +of the emulator with SMP support by passing `-smp enable` as +command line arguments to erl, and you can force a start of the +emulator without SMP support by passing `-smp disable`. + +GS (Graphic System) +------------------- + +GS now Tcl/Tk 8.4. It will be searched for when starting GS. + +Using HiPE +---------- + +HiPE supports the following system configurations: + +* x86: All 32-bit and 64-bit mode processors should work. + + * Linux: Fedora Core is supported. Both 32-bit and 64-bit modes are + supported. + + NPTL glibc is strongly preferred, or a LinuxThreads + glibc configured for "floating stacks". Old non-floating + stacks glibcs have a fundamental problem that makes HiPE + support and threads support mutually exclusive. + + * Solaris: Solaris 10 (32-bit and 64-bit) and 9 (32-bit) are supported. + The build requires a version of the GNU C compiler (gcc) + that has been configured to use the GNU assembler (gas). + Sun's x86 assembler is emphatically **not** supported. + + * FreeBSD: FreeBSD 6.1 and 6.2 in 32-bit and 64-bit modes should work. + + * MacOSX/Darwin: Darwin 9.8.0 in 32-bit mode should work. + +* PowerPC: All 32-bit 6xx/7xx(G3)/74xx(G4) processors should work. 32-bit + mode on 970 (G5) and POWER5 processors should work. + + * Linux (Yellow Dog) and Mac OSX 10.4 are supported. + +* SPARC: All UltraSPARC processors running 32-bit user code should work. + + * Solaris 9 is supported. The build requires a `gcc` that has been + configured to use Sun's assembler and linker. Using the GNU assembler + but Sun's linker has been known to cause problems. + + * Linux (Aurora) is supported. + +* ARM: ARMv5TE (i.e. XScale) processors should work. Both big-endian and + little-endian modes are supported. + + * Linux is supported. + +HiPE is automatically enabled on the following systems: + +* x86 in 32-bit mode: Linux, Solaris, FreeBSD +* x86 in 64-bit mode: Linux, Solaris, FreeBSD +* PowerPC: Linux, MacOSX +* SPARC: Linux +* ARM: Linux + +On other supported systems you need to `./configure --enable-hipe`. + +If you are running on a platform supporting HiPE and if you have not disabled +HiPE, you can compile a module into native code like this from the Erlang +shell: + + 1> c(Module, native). + +or + + 1> c(Module, [native|OtherOptions]). + +Using the erlc program, write like this: + + $ erlc +native Module.erl + +The native code will be placed into the beam file and automatically loaded +when the beam file is loaded. + +To add hipe options, write like this from the Erlang shell: + + 1> c(Module, [native,{hipe,HipeOptions}|MoreOptions]). + +Use `hipe:help_options/0` to print out the available options. + + 1> hipe:help_options(). + +Mac OS X (Darwin) +----------------- + +Make sure that the command `hostname` returns a valid fully qualified host +name (this is configured in `/etc/hostconfig`). + +If you develop linked-in drivers (shared library) you need to link using +`gcc` and the flags `-bundle -flat_namespace -undefined suppress`. You also +include `-fno-common` in `CFLAGS` when compiling. Use `.so` as the library +suffix. + +Use the `--enable-darwin-64bit` configure flag to build a 64-bit +binaries on Mac OS X. + +Building universal binaries on Mac OS X (obsolete information) +-------------------------------------------------------------- + +(This information was written when Mac OS X Leopard was the current +release. It may no longer work.) + +Universal 32bit binaries can be built on an Intel Mac using the +`--enable-darwin-universal` configure option. There still may occur +problems with certain applications using this option, but the base +system should run smoothly. + +When building universal binaries on a PowerPC Mac (at least on Tiger), +you must point out a suitable SDK that contains universal binaries. +For instance, to build universal binaries for Tiger (10.4): + + $ CFLAGS="-isysroot /Developer/SDKs/MacOSX10.4u.sdk" \ + LDFLAGS="-isysroot /Developer/SDKs/MacOSX10.4u.sdk" \ + ./configure --enable-darwin-universal + +Also, if you run Leopard, but want to build for Tiger, you must do by +setting the `MACOSX_DEPLOYMENT_TARGET` environmental variable. + + $ export MACOSX_DEPLOYMENT_TARGET=10.4 + +Experimental support for 64bit x86 darwin binaries can be enabled +using the `--enable-darwin-64bit` configure flag. The 64bit binaries are +best built and run on Leopard, but most of the system also works on +Tiger (Tiger's 64bit libraries are, however, limited; therefore e.g. `odbc`, +`crypto`, `ssl` etc. are not supported in Tiger). 64bit PPC binaries are not +supported and we have no plans to add such support (no machines to +test on). + +Universal binaries and 64bit binaries are mutually exclusive options. + +Building a fast Erlang VM on Mac OS Lion +---------------------------------------- + +Starting with Xcode 4.2, Apple no longer includes a "real" `gcc` +compiler (not based on the LLVM). Building with `llvm-gcc` or `clang` +will work, but the performance of the Erlang run-time system will not +be the best possible. + +Note that if you have `gcc-4.2` installed and included in `PATH` +(from a previous version of Xcode), `configure` will automatically +make sure that `gcc-4.2` will be used to compile `beam_emu.c` +(the source file most in need of `gcc`). + +If you don't have `gcc-4.2.` and want to build a run-time system with +the best possible performance, do like this: + +Install Xcode from the AppStore if it is not already installed. + +If you have Xcode 4.3, or later, you will also need to download +"Command Line Tools" via the Downloads preference pane in Xcode. + +Some tools may still be lacking or out-of-date, we recommend using +[Homebrew](https://github.com/mxcl/homebrew/wiki/installation) or +Macports to update those tools. + +Install MacPorts (<http://www.macports.org/>). Then: + + $ sudo port selfupdate + $ sudo port install gcc45 +universal + +### Building with wxErlang ### + +If you want to build the `wx` application, you will need to get wxMac-2.8.12 +(`wxMac-2.8.12.tar.gz` from +<http://sourceforge.net/projects/wxwindows/files/2.8.12/>) and install it. + +Export the path for MacOSX10.6.sdk: + + $ export SDK=/Developer/SDKs/MacOSX10.6.sdk + +In Xcode 4.3 the path has changed so use the following instead, + + $ export SDK=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.6.sdk + +Then configure and build wxMac: + + $ arch_flags="-arch i386" ./configure CFLAGS="$arch_flags" CXXFLAGS="$arch_flags" CPPFLAGS="$arch_flags" LDFLAGS="$arch_flags" OBJCFLAGS="$arch_flags" OBJCXXFLAGS="$arch_flags" --prefix=/usr/local --with-macosx-sdk="$SDK" --with-macosx-version-min=10.6 --enable-unicode --with-opengl --disable-shared + $ make + $ sudo make install + +To link wx properly you will also need to build and install `wxStyledTextCtrl`: + + $ cd contrib/src/stc + $ make + $ sudo make install + +### Finish up ### + +Build Erlang with the MacPorts GCC as the main compiler (using `clang` +for the Objective-C Cocoa code in the `wx` application): + + $ PATH=/usr/local/bin:$PATH CC=/opt/local/bin/gcc-mp-4.5 CXX=/opt/local/bin/g++-mp-4.5 ./configure --enable-m32-build make + $ sudo make install + + +How to Build a Debug Enabled Erlang RunTime System +-------------------------------------------------- + +After completing all the normal building steps described above a debug +enabled runtime system can be built. To do this you have to change +directory to `$ERL_TOP/erts/emulator`. + +In this directory execute: + + $ make debug FLAVOR=$FLAVOR + +where `$FLAVOR` is either `plain` or `smp`. The flavor options will +produce a beam.debug and beam.smp.debug executable respectively. The +files are installed along side with the normal (opt) versions `beam.smp` +and `beam`. + +To start the debug enabled runtime system execute: + + $ $ERL_TOP/bin/cerl -debug + +The debug enabled runtime system features lock violation checking, +assert checking and various sanity checks to help a developer ensure +correctness. Some of these features can be enabled on a normal beam +using appropriate configure options. + +There are other types of runtime systems that can be built as well +using the similar steps just described. + + $ make $TYPE FLAVOR=$FLAVOR + +where `$TYPE` is `opt`, `gcov`, `gprof`, `debug`, `valgrind`, or `lcnt`. +These different beam types are useful for debugging and profiling +purposes. + +Authors +------- + +Authors are mostly listed in the application's `AUTHORS` files, +that is `$ERL_TOP/lib/*/AUTHORS` and `$ERL_TOP/erts/AUTHORS`, +not in the individual source files. + +Copyright and License +--------------------- + +%CopyrightBegin% + +Copyright Ericsson AB 1998-2012. All Rights Reserved. + +The contents of this file are subject to the Erlang Public License, +Version 1.1, (the "License"); you may not use this file except in +compliance with the License. You should have received a copy of the +Erlang Public License along with this software. If not, it can be +retrieved online at http://www.erlang.org/. + +Software distributed under the License is distributed on an "AS IS" +basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +the License for the specific language governing rights and limitations +under the License. + +%CopyrightEnd% + +More Information +---------------- + +More information can be found at <http://www.erlang.org>. + +Modifying This Document +----------------------- + +Before modifying this document you need to have a look at the +[$ERL_TOP/HOWTO/MARKDOWN.md][] document. + + + + [$ERL_TOP/HOWTO/INSTALL-CROSS.md]: INSTALL-CROSS.md + [$ERL_TOP/HOWTO/INSTALL-WIN32.md]: INSTALL-WIN32.md + [DESTDIR]: http://www.gnu.org/prep/standards/html_node/DESTDIR.html + [Building in Git]: #How-to-Build-and-Install-ErlangOTP_Building-in-Git + [Pre-built Source Release]: #How-to-Build-and-Install-ErlangOTP_Prebuilt-Source-Release + [make and $ERL_TOP]: #How-to-Build-and-Install-ErlangOTP_make-and-ERLTOP + [html documentation]: http://www.erlang.org/download/otp_doc_html_%OTP-REL%.tar.gz + [man pages]: http://www.erlang.org/download/otp_doc_man_%OTP-REL%.tar.gz + [the released source tar ball]: http://www.erlang.org/download/otp_src_%OTP-REL%.tar.gz + [$ERL_TOP/HOWTO/MARKDOWN.md]: MARKDOWN.md + + [?TOC]: true diff --git a/HOWTO/MARKDOWN.md b/HOWTO/MARKDOWN.md new file mode 100644 index 0000000000..42045fb10c --- /dev/null +++ b/HOWTO/MARKDOWN.md @@ -0,0 +1,266 @@ +Erlangish Markdown Text Files +============================= + +Introduction +------------ + +If you are looking for information on how to build and install Erlang/OTP you +want to read the [$ERL_TOP/HOWTO/INSTALL.md][] document instead of this document +(where `$ERL_TOP` is the top source directory in the source tree). + +All files with the `.md` suffix (as well as this file) are ordinary text files +written using a Markdown like notation, and can be read using an ordinary text +editor such as for example `emacs`. + +This document describes how `*.md` files in the Erlang/OTP source tree should +be written. + +> *NOTE*: Before modifying a `*.md` file read all of this document. + +Erlangish Markdown +------------------ + +We do not use Markdown straight out of the box. The Markdown syntax we use is +similar to the original Markdown but with a number of tweaks. The original +Markdown is documented at <http://daringfireball.net/projects/markdown/>. You +should read that documentation as well as this document before modifying any +Erlangish Markdown files in the Erlang/OTP source tree. + +The original Markdown syntax was designed for generating HTML code from an +"easy to read/easy to write" plain text. Instead of generating HTML we generate +XML that fits into our documentation, i.e. we generate Erlangish XML from +Erlangish Markdown. + +The `.md` suffix has been chosen since [github][] will generate HTML pages for +files with the `.md` suffix. Github does however not generate HTML according to +Erlangish Markdown, so some features do not work. The Erlangish Markdown +documents viewed at [our github repository][] will typically suffer from broken +links. The original Markdown script, gitub's Markdown, and our Erlangish +Markdown script will generate somewhat different results if you do not follow +indentation rules outlined in the Markdown documentation. You are encouraged to +try to write using a Markdown syntax that also looks nice on github. However, +it is *much* more important that the document is formatted correct in the +Erlang/OTP documentation. + +### Differences Between Markdown and Erlangish Markdown ### + +#### Missing Features #### + +This functionality is missing compared to Markdown version 1.0.1. Do not +depend on the fact that these features are missing. Some of them might appear +in the future. + +* No inline HTML. Currently no inline XML is allowed either. Inline XML might + be allowed in the future, but there is no real need for it since we use + Erlangish Markdown for "readme"s that have a main purpose of being readable + in plain text. + +* Backslash escapes all characters. + +* No support for "horizontal rules". + +* Links. + * No support for the "title" attribute. + * Automatic links does not support email addresses. + +* Images. + * No support for the "title" attribute. Specified "title" will however + be used as `<icaption>`. + * No support for the "alt" attribute. + +* Lists aren't supported inside block quotes. + +* Link and image definition names *are* case sensitive. + +#### Additional Features #### + +* Automatic anchors at each heading. + +* Optionally automatically generated table of contents. + +* Note blocks. + +* Warning blocks. + +#### Extra requirements #### + +* One and only one level 1 heading is allowed and required. + +* The level 1 heading must be the first heading in the document. + +* A level `X` heading must have a level `X-1` heading as parent heading. + +* Link and image definition names aren't allowed to begin with a + question mark (?) character. Names beginning with a question mark have + been reserved for other use. + +* The encoding of the file containing Erlangish Markdown should be + UTF-8. + +### Generated XML ### + +> *WARNING*: The `emd2exml` script will blindly generate XML code according +> to the Erlangish Markdown in a file. Successfully generated XML does **not** +> imply that the generated XML adheres to used DTDs. `emd2exml` does very +> seldom fail and can easily generate XML that will cause the documentation +> build to fail. You always have to keep in mind that the XML generated +> should fit the chapter DTD of Erlang/OTP. Also note that even though HTML +> generation succeeds the PDF generation might fail, etc. +> +> *Always build the Erlang/OTP documentation after modifying an Erlangish +> Markdown document!* + +A note about how we talk about "tags" below. When we say "generate(s) `<X>` +tags" this also imply that ending `</X>` tags will be generated at appropriate +places. Appropriate attributes to the `X` tag will also be generated. + +* Inline and reference style links will either generate `<seealso>` tags + or `<url>` tags. If a "://" character sequence is found in the URL an + `<url>` tag will be generated; otherwise, a `<seealso>` tag is generated. + +* Automatic links will only generate `<url>` tags. This since a + "://" character sequence have to be present in the URL in order + for the link to be identified. + +* Inline and reference style images will generate a `<image file="..."> + <icaption>...</icaption> </image>` sequence where the "title" will be + placed between `<icaption>` and `</icaption>`. + +* Block quotes generate `<blockquote>` tags. + +* If the first line of a top level block quote begins with a `> *NOTE*:` + character sequence, a `<note>` tag will be generated instead of a + `<blockquote>` tag. The note will span the entire block quote. + +* If the first line of a top level block quote begins with a `> *WARNING*:` + character sequence, a `<warning>` tag will be generated instead of a + `<blockquote>` tag. The warning will span the entire block quote. + +* Paragraphs will generate `<p>` tags. + +* Break line (double trailing white space) will generate `<br/>` tags. + +* An unordered list generates a `<list type="bulleted">` tag and `<item>` + tags for each item. + +* An ordered list generates a `<list type="ordered">` tag and `<item>` tags + for each item. + +* Code blocks will generate `<code type="none">` tags. + +* Code span (backticks) will generate `<c>` tags. + +* Emphasis (single `*` or `_`) will generate `<em>` tags. + +* Strong emphasis (double `*` or `_`) will generate `<b>` tags. + +* The level 1 heading will cause the following to be generated: + + <?xml version="1.0" encoding="utf8" ?> + <!DOCTYPE chapter SYSTEM "chapter.dtd"> + <chapter> + <header> + <copyright> + ... + </copyright> + <legalnotice> + ... + </legalnotice> + + <title>...</title> + ... + <file>...</file> + </header> + + ... + + </chapter> + + The content of copyright section and the legalnotice section will + contain information from a \%CopyrightBegin\%, \%CopyrightEnd\% block + if such exist (see below). + +* A level `X` heading where `1 < X <= 3` will cause the the following + to be generated: + + <marker id="..."/> + <section> + <title>...</title> + ... + </section> + + The marker id is automatically generated as a combination of all parent + headings up to and including level 2 separated by a `_` character. As in + `<marker heading 2>_<marker heading 3>_ ... _<current marker heading>` + where each "marker heading" is constructed as follows. All characters a-z + and A-Z are kept as they are, space and tab characters are replaced by + `-` characters, and all other characters are dropped. + + This way it is relatively easy to make sure that all marker ids of a + document are unique, but there is of course no guarantee that they are. + + The upside of these auto generated markers is that we wont have to clutter + the document with XML or something else while being able to refer into + the document. The downside is that if you change a level 2 heading you + change a lot of marker ids which may break links into a document from + other documents. That is, *be careful* when changing headings in an + existing document. + +* A level `X` heading where `3 < X` will cause the the following + to be generated: + + <marker id="..."/> + <p><b>...</b></p> + ... + + Current DTD:s used don't support deeper levels of sections, and we + therefore simulate a section this way. The marker id is generated as for + a true section (see above). + +* If a section enclosed by \%CopyrightBegin\% and \%CopyrightEnd\% is + encountered, it will be interpreted as an EPL copyright and license, + and will be used in the header section of the document. The + \%CopyrightBegin\% and \%CopyrightEnd\% "tags" will be removed from + the output. + +* All occurrences of \%OTP-REL% will be replaced by current release number + (e.g. R14A). + +* All occurrences of \%ERTS-VSN% will be replaced by current ERTS version + (e.g. 5.8). + +* Adding a `[?TOC]: true` line (optionally indented with three spaces) + anywhere in the document will cause a table of contents to be automatically + generated at the beginning of the generated document. + +* Unicode characters (encoded in UTF-8) are allowed and will be passed + through as is to the output file. + +Copyright and License +--------------------- + +%CopyrightBegin% + +Copyright Ericsson AB 2010. All Rights Reserved. + +The contents of this file are subject to the Erlang Public License, +Version 1.1, (the "License"); you may not use this file except in +compliance with the License. You should have received a copy of the +Erlang Public License along with this software. If not, it can be +retrieved online at http://www.erlang.org/. + +Software distributed under the License is distributed on an "AS IS" +basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +the License for the specific language governing rights and limitations +under the License. + +%CopyrightEnd% + + + + [$ERL_TOP/HOWTO/INSTALL.md]: INSTALL.md + [github]: http://github.com + [our github repository]: http://github.com/erlang/otp + + + [?TOC]: true diff --git a/HOWTO/SYSTEMTAP.md b/HOWTO/SYSTEMTAP.md new file mode 100644 index 0000000000..ce9c0b2f0c --- /dev/null +++ b/HOWTO/SYSTEMTAP.md @@ -0,0 +1,75 @@ +SystemTap and Erlang/OTP +======================== + +Introduction +------------ + +SystemTap is DTrace for Linux. In fact Erlang's SystemTap support +is build using SystemTap's DTrace compatibility's layer. For an +introduction to Erlang DTrace support read [$ERL_TOP/HOWTO/DTRACE.md][]. + +Requisites +---------- + +* Linux Kernel with UTRACE support + + check for UTRACE support in your current kernel: + + # grep CONFIG_UTRACE /boot/config-`uname -r` + CONFIG_UTRACE=y + + Fedora 16 is known to contain UTRACE, for most other Linux distributions + a custom build kernel will be required. + Check Fedora's SystemTap documentation for additional required packages + (e.g. Kernel Debug Symbols) + +* SystemTap > 1.6 + + A the time of writing this, the latest released version of SystemTap is + version 1.6. Erlang's DTrace support requires a MACRO that was introduced + after that release. So either get a newer release or build SystemTap from + git yourself (see: http://sourceware.org/systemtap/getinvolved.html) + +Building Erlang +--------------- + +Configure and build Erlang with SystemTap support: + + # ./configure --with-dynamic-trace=systemtap + whatever args you need + # make + +Testing +------- + +SystemTap, unlike DTrace, needs to know what binary it is tracing and has to +be able to read that binary before it starts tracing. Your probe script +therefor has to reference the correct beam emulator and stap needs to be able +to find that binary. +The examples are written for "beam", but other versions such as "beam.smp" or +"beam.debug.smp" might exist (depending on your configuration). Make sure you +either specify the full the path of the binary in the probe or your "beam" +binary is in the search path. + +All available probes can be listed like this: + + # stap -L 'process("beam").mark("*")' + +or: + + # PATH=/path/to/beam:$PATH stap -L 'process("beam").mark("*")' + + +Probes in the dtrace.so NIF library like this: + + # PATH=/path/to/dtrace/priv/lib:$PATH stap -L 'process("dtrace.so").mark("*")' + +Running SystemTap scripts +------------------------- + +Adjust the process("beam") reference to your beam version and attach the script +to a running "beam" instance: + + # stap /path/to/probe/script/port1.systemtap -x <pid of beam> + + + [$ERL_TOP/HOWTO/DTRACE.md]: DTRACE.md diff --git a/HOWTO/TESTING.md b/HOWTO/TESTING.md new file mode 100644 index 0000000000..34eaa68df8 --- /dev/null +++ b/HOWTO/TESTING.md @@ -0,0 +1,150 @@ +Testing Erlang/OTP +================== + +Before you start testing you need to have the Erlang release which you +are going to test in your path. See [$ERL_TOP/HOWTO/INSTALL.md][] for +instructions on how to build an Erlang release. + +Short version +------------- +Move to the top directory of the Erlang release you want to test, i.e. +cd /ldisk/work/otp + + export ERL_TOP=`pwd` + ./otp_build setup -a + export PATH=`pwd`/bin:$PATH + ./otp_build tests + cd release/tests/test_server + erl -s ts install -s ts run all_tests -s init stop + +Where are the tests +------------------- + +There are a lot of tests which test Erlang/OTP (as of 2012 about 12000) and +they are located in different places throughout the source tree. Below is a list +of the places where you can expect to find tests: + +* $ERL_TOP/lib/AppName/test/ +* $ERL_TOP/erts/test/ +* $ERL_TOP/erts/emulator/test/ +* $ERL_TOP/erts/epmd/test/ + +Writing tests +------------- + +All tests are [common_test][] suites and follow the same pattern as all +[common_test][] suites. However, a couple of corner cases are +handled by a test wrapper called `ts`. `ts` allows the test writer to put +`Makefile.src` and `Makefile.first` files in the [data_dir][] and a special +directory called `all_SUITE_data`. + +`Makefile.first` is run before any other Makefile and is typically used to +generate .hrl files which are needed by other test suites. At the moment only +the erl_interface tests use this feature and it should remain that way. + +`Makefile.src` is configured to a `Makefile` using the `variables` created when +[configuring the tests][]. These `Makefile`s are later run by the test suite +to compile whatever platform specific code the tests need to run. + +Releasing tests +--------------- + +If you cannot use [ct_run][] in the source tree you have to release the tests +into a common test directory. The easiest way to do this is to use `otp_build` +like this: + + export ERL_TOP=`pwd`; ./otp_build tests + +This will release all tests in Erlang/OTP to `$ERL_TOP/release/tests/`. If you +want to change the directory where the tests are released to use the `TESTROOT` +environmental variable. + +In the `$TESTROOT` you should now see *_test folders. These folders contain +everything needed to test Erlang/OTP and are platform independent; if you are +testing Erlang on multiple platforms you just have to release on one and copy +the tests to all other platforms. + +### Releasing cross tests + +For releasing tests in a cross compilation environment see [$ERL_TOP/HOWTO/INSTALL-CROSS.md][]. + +Configuring and Running tests +----------------------------- + +Running tests is done by first navigating to the `$TESTROOT/test_server` folder +created when you released the tests and then start `erl` in that directory. The +emulator flags specified will be used in the test runs. For example, if you want +to test using async threads you have to supply `+A 10` to `erl` when you start it. + +To configure and run the tests `ts` is used. `ts` is a wrapper module to +[common_test][] which takes care of configuration and build issues before +[common_test][] is started. + +`ts` has a lot of special options and functions which can be usefull when +testing Erlang/OTP. For a full listing issue `ts:help()` in the erlang shell. + +### Configuring the test environment + +Before running released tests you have to install them on the target system. +Installing the tests is done by calling `ts:install().` in the Erlang shell +which you intend to test from. `ts:install()` is basically a wrapper to a +configure script and some Erlang code which figures out what your system looks +like and what kind of emulator you are testing with. `ts:install()` can also +take some arguments if necessary, see `ts:help()` for details. + +All variables created by `ts:install()` are found in +`$TESTROOT/test_server/variables`. + +### Running the tests + +To run all test suites go to `$TESTROOT/test_server` fire up an Erlang shell and type: + + ts:run(). + +Note that running all tests will require several hours, so you may want to run +the test cases for a single application + + ts:run(Application, [batch]). + +or even part of the test suite for an application, for example + + ts:run(emulator, bs, [batch]). + +to run all test suite modules starting with `bs` (i.e. all modules that test +the bit syntax). + +To run a specific test case in a module, the full name of the module and test +case must be specified: + + ts:run(emulator, bs_bincomp_SUITE, byte_aligned, [batch]). + +Run `ts:help().` for more information. + +As of R14B02 it is also possibly to start all tests but the erl_interface tests +by invoking Common Test directly from the released applications test directory, +i.e. + + cd $TESTROOT/test_server + $ERL_TOP/bin/ct_run -suite ../compiler_test/andor_SUITE -case t_orelse + +Running [ct_run][] from the command line still requires you to do the +`ts:install()` step above. + +Examining the results +--------------------- + +Open the file `release/tests/test_server/index.html` in a web browser. Or open +`release/tests/test_server/last_test.html` when a test suite is running to +examine the results so far for the currently executing test suite (in R14B02 and +later you want to open the `release/tests/test_server/all_runs.html` file to +get to the currently running test) + + [ct_run]: http://www.erlang.org/doc/man/ct_run.html + [ct hook]: http://www.erlang.org/doc/apps/common_test/ct_hooks_chapter.html + [$ERL_TOP/HOWTO/INSTALL.md]: INSTALL.md + [$ERL_TOP/HOWTO/INSTALL-CROSS.md]: INSTALL-CROSS.md#testing-the-cross-compiled-system + [common_test]: http://www.erlang.org/doc/man/ct.html + [data_dir]: http://www.erlang.org/doc/apps/common_test/write_test_chapter.html#data_priv_dir + [configuring the tests]: #configuring-the-test-environment + + [?TOC]: true |