diff options
30 files changed, 928 insertions, 106 deletions
diff --git a/.gitignore b/.gitignore index 18a54c21ca..07b66c3e2b 100644 --- a/.gitignore +++ b/.gitignore @@ -381,6 +381,7 @@ JAVADOC-GENERATED /system/doc/installation_guide/INSTALL.xml /system/doc/installation_guide/INSTALL-CROSS.xml /system/doc/installation_guide/INSTALL-WIN32.xml +/system/doc/installation_guide/OTP-PATCH-APPLY.xml /system/doc/installation_guide/MARKDOWN.xml # test_server diff --git a/HOWTO/OTP-PATCH-APPLY.md b/HOWTO/OTP-PATCH-APPLY.md new file mode 100644 index 0000000000..2aa31629ef --- /dev/null +++ b/HOWTO/OTP-PATCH-APPLY.md @@ -0,0 +1,144 @@ +Patching OTP Applications +========================= + +Introduction +------------ + +This document describes the process of patching an existing OTP +installation with one or more Erlang/OTP applications of newer versions +than already installed. The tool `otp_patch_apply` is available for this +specific purpose. It resides in the top directory of the Erlang/OTP +source tree. + +The `otp_patch_apply` tool utilizes the [runtime_dependencies][] tag in +the [application resource file][]. This information is used to determine +if the patch can be installed in the given Erlang/OTP installation +directory. + +Read more about the [version handling][] introduced in Erlang/OTP release +17, which also describes how to determine if an installation includes one +or more patched applications. + +If you want to apply patches of multiple OTP applications that resides +in different OTP versions, you have to apply these patches in multiple +steps. It is only possible to apply multiple OTP applications from the +same OTP version at once. + +Prerequisites +------------- + +It's assumed that the reader is familiar with +[building and installing Erlang/OTP][]. To be able to patch an +application, the following must exist: + +* An Erlang/OTP installation. + +* An Erlang/OTP source tree containing the updated applications that + you want to patch into the existing Erlang/OTP installation. + +Using otp\_patch\_apply +----------------------- + +> *WARNING*: Patching applications is a one-way process. +> Create a backup of your OTP installation directory before +> proceeding. + +First of all, build the OTP source tree at `$ERL_TOP` containing +the updated applications. + +> *NOTE*: Before applying a patch you need to do a *full* build +> of OTP in the source directory. + +If you are building in `git` you first need to generate the +`configure` scripts: + + $ ./otp_build autoconf + +Configure and build all applications in OTP: + + $ configure + $ make + +or + + $ ./otp_build configure + $ ./otp_build boot -a + +If you have installed documentation in the OTP installation, also +build the documentation: + + $ make docs + +After the successful build it's time to patch. The source tree directory, +the directory of the installation and the applications to patch are given +as arguments to `otp_patch_apply`. The dependencies of each application +are validated against the applications in the installation and the other +applications given as arguments. If a dependency error is detected, the +script will be aborted. + +The `otp_patch_apply` syntax: + + $ otp_patch_apply -s <Dir> -i <Dir> [-l <Dir>] [-c] [-f] [-h] \ + [-n] [-v] <App1> [... <AppN>] + + -s <Dir> -- OTP source directory that contains build results. + -i <Dir> -- OTP installation directory to patch. + -l <Dir> -- Alternative OTP source library directory path(s) + containing build results of OTP applications. + Multiple paths should be colon separated. + -c -- Cleanup (remove) old versions of applications + patched in the installation. + -f -- Force patch of application(s) even though + dependencies are not fulfilled (should only be + considered in a test environment). + -h -- Print help then exit. + -n -- Do not install documentation. + -v -- Print version then exit. + <AppX> -- Application to patch. + + Environment Variable: + ERL_LIBS -- Alternative OTP source library directory path(s) + containing build results of OTP applications. + Multiple paths should be colon separated. + +> *NOTE*: The complete build environment is required while running +> `otp_patch_apply`. + +> *NOTE*: All source directories identified by `-s` and `-l` should +> contain build results of OTP applications. + +For example, if the user wants to install patched versions of `mnesia` +and `ssl` built in `/home/me/git/otp` into the OTP installation +located in `/opt/erlang/my_otp` type + + $ otp_patch_apply -s /home/me/git/otp -i /opt/erlang/my_otp \ + mnesia ssl + +> *NOTE*: If the list of applications contains core applications, +> i.e `erts`, `kernel`, `stdlib` or `sasl`, the `Install` script in +> the patched Erlang/OTP installation must be rerun. + +The patched applications are appended to the list of installed +applications. Take a look at +`<InstallDir>/releases/OTP-REL/installed_application_versions`. + +Sanity check +------------ + +The application dependencies can be checked using the Erlang shell. +Application dependencies are verified among installed applications by +`otp_patch_apply`, but these are not necessarily those actually loaded. +By calling `system_information:sanity_check()` one can validate +dependencies among applications actually loaded. + + 1> system_information:sanity_check(). + ok + +Please take a look at the reference of [sanity_check()][] for more +information. + +[application resource file]: kernel:app +[runtime_dependencies]: kernel:app#runtime_dependencies +[building and installing Erlang/OTP]: INSTALL.md +[version handling]: ../system_principles/versions +[sanity_check()]: runtime_tools:system_information#sanity_check-0 diff --git a/erts/aclocal.m4 b/erts/aclocal.m4 index d78025b0be..5735cdea5c 100644 --- a/erts/aclocal.m4 +++ b/erts/aclocal.m4 @@ -246,31 +246,31 @@ lbl1: return 1; lbl2: return 2; -],ac_cv_prog_emu_cc=$CC,ac_cv_prog_emu_cc=no) +],ac_cv_prog_emu_cc="$CC",ac_cv_prog_emu_cc=no) -if test $ac_cv_prog_emu_cc = no; then +if test "$ac_cv_prog_emu_cc" = no; then for ac_progname in emu_cc.sh gcc-4.2 gcc; do IFS="${IFS= }"; ac_save_ifs="$IFS"; IFS=":" ac_dummy="$PATH" for ac_dir in $ac_dummy; do test -z "$ac_dir" && ac_dir=. - if test -f $ac_dir/$ac_progname; then - ac_cv_prog_emu_cc=$ac_dir/$ac_progname + if test -f "$ac_dir/$ac_progname"; then + ac_cv_prog_emu_cc="$ac_dir/$ac_progname" break fi done IFS="$ac_save_ifs" - if test $ac_cv_prog_emu_cc != no; then + if test "$ac_cv_prog_emu_cc" != no; then break fi done fi -if test $ac_cv_prog_emu_cc != no; then - save_CC=$CC +if test "$ac_cv_prog_emu_cc" != no; then + save_CC="$CC" save_CFLAGS=$CFLAGS save_CPPFLAGS=$CPPFLAGS - CC=$ac_cv_prog_emu_cc + CC="$ac_cv_prog_emu_cc" CFLAGS="" CPPFLAGS="" AC_TRY_COMPILE([],[ @@ -291,17 +291,17 @@ if test $ac_cv_prog_emu_cc != no; then return 1; lbl2: return 2; - ],ac_cv_prog_emu_cc=$CC,ac_cv_prog_emu_cc=no) + ],ac_cv_prog_emu_cc="$CC",ac_cv_prog_emu_cc=no) CC=$save_CC CFLAGS=$save_CFLAGS CPPFLAGS=$save_CPPFLAGS fi ]) -if test $ac_cv_prog_emu_cc = no; then +if test "$ac_cv_prog_emu_cc" = no; then AC_DEFINE(NO_JUMP_TABLE,[],[Defined if no found C compiler can handle jump tables]) - EMU_CC=$CC + EMU_CC="$CC" else - EMU_CC=$ac_cv_prog_emu_cc + EMU_CC="$ac_cv_prog_emu_cc" fi AC_SUBST(EMU_CC) ]) diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index cba2c07959..0e5909a52d 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -1356,7 +1356,7 @@ true <name name="get" arity="1"/> <fsummary>Return a value from the process dictionary</fsummary> <desc> - <p>Returns the value <c><anno>Val</anno></c>associated with <c><anno>Key</anno></c> in + <p>Returns the value <c><anno>Val</anno></c> associated with <c><anno>Key</anno></c> in the process dictionary, or <c>undefined</c> if <c><anno>Key</anno></c> does not exist.</p> <pre> diff --git a/erts/emulator/beam/beam_load.c b/erts/emulator/beam/beam_load.c index cfc6146b0a..41c1b5d2c2 100644 --- a/erts/emulator/beam/beam_load.c +++ b/erts/emulator/beam/beam_load.c @@ -4971,7 +4971,8 @@ get_tag_and_value(LoaderState* stp, Uint len_code, arity = count/sizeof(Eterm); *result = new_literal(stp, &hp, arity+1); - (void) bytes_to_big(bigbuf, count, neg, hp); + if (is_nil(bytes_to_big(bigbuf, count, neg, hp))) + goto load_error; if (bigbuf != default_buf) { erts_free(ERTS_ALC_T_LOADER_TMP, (void *) bigbuf); diff --git a/erts/emulator/beam/big.c b/erts/emulator/beam/big.c index de7d370938..d1e46e3063 100644 --- a/erts/emulator/beam/big.c +++ b/erts/emulator/beam/big.c @@ -1900,6 +1900,8 @@ Eterm bytes_to_big(byte *xp, dsize_t xsz, int xsgn, Eterm *r) *rwp = d; rwp++; } + if (rsz > BIG_ARITY_MAX) + return NIL; if (xsgn) { *r = make_neg_bignum_header(rsz); } diff --git a/erts/emulator/beam/erl_bits.c b/erts/emulator/beam/erl_bits.c index 73765772c8..53c21c40e1 100644 --- a/erts/emulator/beam/erl_bits.c +++ b/erts/emulator/beam/erl_bits.c @@ -403,7 +403,10 @@ erts_bs_get_integer_2(Process *p, Uint num_bits, unsigned flags, ErlBinMatchBuff words_needed = 1+WSIZE(bytes); hp = HeapOnlyAlloc(p, words_needed); res = bytes_to_big(LSB, bytes, sgn, hp); - if (is_small(res)) { + if (is_nil(res)) { + p->htop = hp; + res = THE_NON_VALUE; + } else if (is_small(res)) { p->htop = hp; } else if ((actual = bignum_header_arity(*hp)+1) < words_needed) { p->htop = hp + actual; diff --git a/erts/emulator/beam/external.c b/erts/emulator/beam/external.c index 9b9b4b2a62..45d1f7514e 100644 --- a/erts/emulator/beam/external.c +++ b/erts/emulator/beam/external.c @@ -3056,6 +3056,8 @@ dec_term(ErtsDistExternal *edep, Eterm** hpp, byte* ep, ErlOffHeap* off_heap, big = make_small(0); } else { big = bytes_to_big(first, n, neg, hp); + if (is_nil(big)) + goto error; if (is_big(big)) { hp += big_arity(big) + 1; } diff --git a/erts/emulator/beam/io.c b/erts/emulator/beam/io.c index 012a7d1a4b..be34c6effc 100644 --- a/erts/emulator/beam/io.c +++ b/erts/emulator/beam/io.c @@ -4087,6 +4087,9 @@ erts_port_control(Process* c_p, size, &resp_bufp, &resp_size); + + control_flags = prt->control_flags; + finalize_imm_drv_call(&try_call_state); if (tmp_alloced) erts_free(ERTS_ALC_T_TMP, bufp); @@ -4094,8 +4097,6 @@ erts_port_control(Process* c_p, return ERTS_PORT_OP_BADARG; } - control_flags = prt->control_flags; - hsz = port_control_result_size(control_flags, resp_bufp, &resp_size, diff --git a/erts/emulator/hipe/hipe_bif0.c b/erts/emulator/hipe/hipe_bif0.c index c9eee2acf2..8af174170d 100644 --- a/erts/emulator/hipe/hipe_bif0.c +++ b/erts/emulator/hipe/hipe_bif0.c @@ -1217,22 +1217,32 @@ static struct { * they create a new stub for the mfa, which forces locking. * XXX: Redesign apply et al to avoid those updates. */ - erts_smp_mtx_t lock; + erts_smp_rwmtx_t lock; } hipe_mfa_info_table; static inline void hipe_mfa_info_table_init_lock(void) { - erts_smp_mtx_init(&hipe_mfa_info_table.lock, "hipe_mfait_lock"); + erts_smp_rwmtx_init(&hipe_mfa_info_table.lock, "hipe_mfait_lock"); } -static inline void hipe_mfa_info_table_lock(void) +static inline void hipe_mfa_info_table_rlock(void) { - erts_smp_mtx_lock(&hipe_mfa_info_table.lock); + erts_smp_rwmtx_rlock(&hipe_mfa_info_table.lock); } -static inline void hipe_mfa_info_table_unlock(void) +static inline void hipe_mfa_info_table_runlock(void) { - erts_smp_mtx_unlock(&hipe_mfa_info_table.lock); + erts_smp_rwmtx_runlock(&hipe_mfa_info_table.lock); +} + +static inline void hipe_mfa_info_table_rwlock(void) +{ + erts_smp_rwmtx_rwlock(&hipe_mfa_info_table.lock); +} + +static inline void hipe_mfa_info_table_rwunlock(void) +{ + erts_smp_rwmtx_rwunlock(&hipe_mfa_info_table.lock); } #define HIPE_MFA_HASH(M,F,A) ((M) * (F) + (A)) @@ -1333,7 +1343,7 @@ void *hipe_mfa_find_na(Eterm m, Eterm f, unsigned int arity) } #endif -static struct hipe_mfa_info *hipe_mfa_info_table_put_locked(Eterm m, Eterm f, unsigned int arity) +static struct hipe_mfa_info *hipe_mfa_info_table_put_rwlocked(Eterm m, Eterm f, unsigned int arity) { unsigned long h; unsigned int i; @@ -1362,8 +1372,8 @@ static void hipe_mfa_set_na(Eterm m, Eterm f, unsigned int arity, void *address, { struct hipe_mfa_info *p; - hipe_mfa_info_table_lock(); - p = hipe_mfa_info_table_put_locked(m, f, arity); + hipe_mfa_info_table_rwlock(); + p = hipe_mfa_info_table_put_rwlocked(m, f, arity); #ifdef DEBUG_LINKER printf("%s: ", __FUNCTION__); print_mfa(m, f, arity); @@ -1372,7 +1382,7 @@ static void hipe_mfa_set_na(Eterm m, Eterm f, unsigned int arity, void *address, p->local_address = address; if (is_exported) p->remote_address = address; - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_rwunlock(); } #if defined(__powerpc__) || defined(__ppc__) || defined(__powerpc64__) || defined(__arm__) @@ -1381,10 +1391,10 @@ void *hipe_mfa_get_trampoline(Eterm m, Eterm f, unsigned int arity) struct hipe_mfa_info *p; void *trampoline; - hipe_mfa_info_table_lock(); - p = hipe_mfa_info_table_put_locked(m, f, arity); - trampoline = p->trampoline; - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_rlock(); + p = hipe_mfa_info_table_get_locked(m, f, arity); + trampoline = p ? p->trampoline : NULL; + hipe_mfa_info_table_runlock(); return trampoline; } @@ -1392,10 +1402,10 @@ void hipe_mfa_set_trampoline(Eterm m, Eterm f, unsigned int arity, void *trampol { struct hipe_mfa_info *p; - hipe_mfa_info_table_lock(); - p = hipe_mfa_info_table_put_locked(m, f, arity); + hipe_mfa_info_table_rwlock(); + p = hipe_mfa_info_table_put_rwlocked(m, f, arity); p->trampoline = trampoline; - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_rwunlock(); } #endif @@ -1426,7 +1436,7 @@ BIF_RETTYPE hipe_bifs_invalidate_funinfo_native_addresses_1(BIF_ALIST_1) struct mfa mfa; struct hipe_mfa_info *p; - hipe_mfa_info_table_lock(); + hipe_mfa_info_table_rwlock(); lst = BIF_ARG_1; while (is_list(lst)) { if (!term_to_mfa(CAR(list_val(lst)), &mfa)) @@ -1455,7 +1465,7 @@ BIF_RETTYPE hipe_bifs_invalidate_funinfo_native_addresses_1(BIF_ALIST_1) } } } - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_rwunlock(); if (is_not_nil(lst)) BIF_ERROR(BIF_P, BADARG); BIF_RET(NIL); @@ -1469,8 +1479,8 @@ void hipe_mfa_save_orig_beam_op(Eterm mod, Eterm fun, unsigned int ari, Eterm *p orig_beam_op = pc[0]; if (orig_beam_op != BeamOpCode(op_hipe_trap_call_closure) && orig_beam_op != BeamOpCode(op_hipe_trap_call)) { - hipe_mfa_info_table_lock(); - p = hipe_mfa_info_table_put_locked(mod, fun, ari); + hipe_mfa_info_table_rwlock(); + p = hipe_mfa_info_table_put_rwlocked(mod, fun, ari); #ifdef DEBUG_LINKER printf("%s: ", __FUNCTION__); print_mfa(mod, fun, ari); @@ -1478,7 +1488,7 @@ void hipe_mfa_save_orig_beam_op(Eterm mod, Eterm fun, unsigned int ari, Eterm *p #endif p->beam_code = pc; p->orig_beam_op = orig_beam_op; - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_rwunlock(); } else { #ifdef DEBUG_LINKER printf("%s: ", __FUNCTION__); @@ -1505,7 +1515,7 @@ static void *hipe_make_stub(Eterm m, Eterm f, unsigned int arity, int is_remote) return StubAddress; } -static void *hipe_get_na_nofail_locked(Eterm m, Eterm f, unsigned int a, int is_remote) +static void *hipe_get_na_try_locked(Eterm m, Eterm f, unsigned int a, int is_remote, struct hipe_mfa_info **pp) { struct hipe_mfa_info *p; void *address; @@ -1523,22 +1533,53 @@ static void *hipe_get_na_nofail_locked(Eterm m, Eterm f, unsigned int a, int is_ address = p->remote_address; if (address) return address; - } else - p = hipe_mfa_info_table_put_locked(m, f, a); + } + /* Caller must take the slow path with the write lock held, but allow + it to avoid some work if it already holds the write lock. */ + if (pp) + *pp = p; + return NULL; +} + +static void *hipe_get_na_slow_rwlocked(Eterm m, Eterm f, unsigned int a, int is_remote, struct hipe_mfa_info *p) +{ + void *address; + + if (!p) + p = hipe_mfa_info_table_put_rwlocked(m, f, a); address = hipe_make_stub(m, f, a, is_remote); /* XXX: how to tell if a BEAM MFA is exported or not? */ p->remote_address = address; return address; } +static void *hipe_get_na_nofail_rwlocked(Eterm m, Eterm f, unsigned int a, int is_remote) +{ + struct hipe_mfa_info *p; + void *address; + + address = hipe_get_na_try_locked(m, f, a, is_remote, &p); + if (address) + return address; + + address = hipe_get_na_slow_rwlocked(m, f, a, is_remote, p); + return address; +} + static void *hipe_get_na_nofail(Eterm m, Eterm f, unsigned int a, int is_remote) { - void *p; + void *address; - hipe_mfa_info_table_lock(); - p = hipe_get_na_nofail_locked(m, f, a, is_remote); - hipe_mfa_info_table_unlock(); - return p; + hipe_mfa_info_table_rlock(); + address = hipe_get_na_try_locked(m, f, a, is_remote, NULL); + hipe_mfa_info_table_runlock(); + if (address) + return address; + + hipe_mfa_info_table_rwlock(); + address = hipe_get_na_slow_rwlocked(m, f, a, is_remote, NULL); + hipe_mfa_info_table_rwunlock(); + return address; } /* used for apply/3 in hipe_mode_switch */ @@ -1617,7 +1658,7 @@ int hipe_find_mfa_from_ra(const void *ra, Eterm *m, Eterm *f, unsigned int *a) /* Note about locking: the table is only updated from the loader, which runs with the rest of the system suspended. */ /* XXX: alas not true; see comment at hipe_mfa_info_table.lock */ - hipe_mfa_info_table_lock(); + hipe_mfa_info_table_rlock(); bucket = hipe_mfa_info_table.bucket; nrbuckets = 1 << hipe_mfa_info_table.log2size; mfa = NULL; @@ -1638,7 +1679,7 @@ int hipe_find_mfa_from_ra(const void *ra, Eterm *m, Eterm *f, unsigned int *a) *f = mfa->f; *a = mfa->a; } - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_runlock(); return mfa ? 1 : 0; } @@ -1715,9 +1756,9 @@ BIF_RETTYPE hipe_bifs_add_ref_2(BIF_ALIST_2) default: goto badarg; } - hipe_mfa_info_table_lock(); - callee_mfa = hipe_mfa_info_table_put_locked(callee.mod, callee.fun, callee.ari); - caller_mfa = hipe_mfa_info_table_put_locked(caller.mod, caller.fun, caller.ari); + hipe_mfa_info_table_rwlock(); + callee_mfa = hipe_mfa_info_table_put_rwlocked(callee.mod, callee.fun, callee.ari); + caller_mfa = hipe_mfa_info_table_put_rwlocked(caller.mod, caller.fun, caller.ari); refers_to = erts_alloc(ERTS_ALC_T_HIPE, sizeof(*refers_to)); refers_to->mfa = callee_mfa; @@ -1731,7 +1772,7 @@ BIF_RETTYPE hipe_bifs_add_ref_2(BIF_ALIST_2) ref->flags = flags; ref->next = callee_mfa->referred_from; callee_mfa->referred_from = ref; - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_rwunlock(); BIF_RET(NIL); @@ -1751,12 +1792,12 @@ BIF_RETTYPE hipe_bifs_mark_referred_from_1(BIF_ALIST_1) /* get_refs_from */ if (!term_to_mfa(BIF_ARG_1, &mfa)) BIF_ERROR(BIF_P, BADARG); - hipe_mfa_info_table_lock(); + hipe_mfa_info_table_rwlock(); p = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, mfa.ari); if (p) for (ref = p->referred_from; ref != NULL; ref = ref->next) ref->flags |= REF_FLAG_PENDING_REDIRECT; - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_rwunlock(); BIF_RET(NIL); } @@ -1770,7 +1811,7 @@ static void hipe_purge_all_refs(void) struct hipe_mfa_info **bucket; unsigned int i, nrbuckets; - hipe_mfa_info_table_lock(); + hipe_mfa_info_table_rwlock(); bucket = hipe_mfa_info_table.bucket; nrbuckets = 1 << hipe_mfa_info_table.log2size; @@ -1792,7 +1833,7 @@ static void hipe_purge_all_refs(void) erts_free(ERTS_ALC_T_HIPE, mfa); } } - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_rwunlock(); } BIF_RETTYPE hipe_bifs_remove_refs_from_1(BIF_ALIST_1) @@ -1809,7 +1850,7 @@ BIF_RETTYPE hipe_bifs_remove_refs_from_1(BIF_ALIST_1) if (!term_to_mfa(BIF_ARG_1, &mfa)) BIF_ERROR(BIF_P, BADARG); - hipe_mfa_info_table_lock(); + hipe_mfa_info_table_rwlock(); caller_mfa = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, mfa.ari); if (caller_mfa) { refers_to = caller_mfa->refers_to; @@ -1840,7 +1881,7 @@ BIF_RETTYPE hipe_bifs_remove_refs_from_1(BIF_ALIST_1) } caller_mfa->refers_to = NULL; } - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_rwunlock(); BIF_RET(am_ok); } @@ -1859,7 +1900,7 @@ BIF_RETTYPE hipe_bifs_redirect_referred_from_1(BIF_ALIST_1) if (!term_to_mfa(BIF_ARG_1, &mfa)) BIF_ERROR(BIF_P, BADARG); - hipe_mfa_info_table_lock(); + hipe_mfa_info_table_rwlock(); p = hipe_mfa_info_table_get_locked(mfa.mod, mfa.fun, mfa.ari); if (p) { prev = &p->referred_from; @@ -1867,7 +1908,7 @@ BIF_RETTYPE hipe_bifs_redirect_referred_from_1(BIF_ALIST_1) while (ref) { if (ref->flags & REF_FLAG_PENDING_REDIRECT) { is_remote = ref->flags & REF_FLAG_IS_REMOTE; - new_address = hipe_get_na_nofail_locked(p->m, p->f, p->a, is_remote); + new_address = hipe_get_na_nofail_rwlocked(p->m, p->f, p->a, is_remote); if (ref->flags & REF_FLAG_IS_LOAD_MFA) res = hipe_patch_insn(ref->address, (Uint)new_address, am_load_mfa); else @@ -1890,7 +1931,7 @@ BIF_RETTYPE hipe_bifs_redirect_referred_from_1(BIF_ALIST_1) } } } - hipe_mfa_info_table_unlock(); + hipe_mfa_info_table_rwunlock(); BIF_RET(NIL); } diff --git a/erts/emulator/test/big_SUITE.erl b/erts/emulator/test/big_SUITE.erl index 413bd3bcae..3193d56e2a 100644 --- a/erts/emulator/test/big_SUITE.erl +++ b/erts/emulator/test/big_SUITE.erl @@ -23,7 +23,7 @@ init_per_group/2,end_per_group/2]). -export([t_div/1, eq_28/1, eq_32/1, eq_big/1, eq_math/1, big_literals/1, borders/1, negative/1, big_float_1/1, big_float_2/1, - shift_limit_1/1, powmod/1, system_limit/1, otp_6692/1]). + shift_limit_1/1, powmod/1, system_limit/1, toobig/1, otp_6692/1]). %% Internal exports. -export([eval/1]). @@ -40,7 +40,7 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> [t_div, eq_28, eq_32, eq_big, eq_math, big_literals, borders, negative, {group, big_float}, shift_limit_1, - powmod, system_limit, otp_6692]. + powmod, system_limit, toobig, otp_6692]. groups() -> [{big_float, [], [big_float_1, big_float_2]}]. @@ -370,6 +370,16 @@ maxbig() -> id(I) -> I. +toobig(Config) when is_list(Config) -> + ?line {'EXIT',{{badmatch,_},_}} = (catch toobig()), + ok. + +toobig() -> + A = erlang:term_to_binary(lists:seq(1000000, 2200000)), + ASize = erlang:bit_size(A), + <<ANr:ASize>> = A, % should fail + ANr band ANr. + otp_6692(suite) -> []; otp_6692(doc) -> diff --git a/lib/common_test/doc/src/event_handler_chapter.xml b/lib/common_test/doc/src/event_handler_chapter.xml index 45f01c12ec..f39f391818 100644 --- a/lib/common_test/doc/src/event_handler_chapter.xml +++ b/lib/common_test/doc/src/event_handler_chapter.xml @@ -59,6 +59,15 @@ Event handlers plugged into this manager will receive the events from all the test nodes as well as information from the CT Master server itself.</p> + + <p>User specific event handlers may be plugged into a Common Test event + manager, either by telling Common Test to install them before the test + run (see below), or by adding the handlers dynamically during the test + run by means of + <c>gen_event:add_handler/3</c> or <c>gen_event:add_sup_handler/3</c>. + In the latter scenario, the reference of the Common Test event manager is + required. To get it, call <c>ct:get_event_mgr_ref/0</c> or (on the CT + Master node) <c>ct_master:get_event_mgr_ref/0</c>.</p> </section> <section> <marker id="usage"></marker> diff --git a/lib/common_test/src/ct.erl b/lib/common_test/src/ct.erl index 85afdc7834..9d8fce2789 100644 --- a/lib/common_test/src/ct.erl +++ b/lib/common_test/src/ct.erl @@ -52,6 +52,7 @@ -module(ct). -include("ct.hrl"). +-include("ct_util.hrl"). %% Command line user interface for running tests -export([install/1, run/1, run/2, run/3, @@ -77,6 +78,7 @@ %% Other interface functions -export([get_status/0, abort_current_testcase/1, + get_event_mgr_ref/0, encrypt_config_file/2, encrypt_config_file/3, decrypt_config_file/2, decrypt_config_file/3]). @@ -1005,6 +1007,18 @@ abort_current_testcase(Reason) -> test_server_ctrl:abort_current_testcase(Reason). %%%----------------------------------------------------------------- +%%% @spec get_event_mgr_ref() -> EvMgrRef +%%% EvMgrRef = atom() +%%% +%%% @doc <p>Call this function in order to get a reference to the +%%% CT event manager. The reference can be used to e.g. add +%%% a user specific event handler while tests are running. +%%% Example: +%%% <c>gen_event:add_handler(ct:get_event_mgr_ref(), my_ev_h, [])</c></p> +get_event_mgr_ref() -> + ?CT_EVMGR_REF. + +%%%----------------------------------------------------------------- %%% @spec encrypt_config_file(SrcFileName, EncryptFileName) -> %%% ok | {error,Reason} %%% SrcFileName = string() diff --git a/lib/common_test/src/ct_master.erl b/lib/common_test/src/ct_master.erl index b42ff73846..2cdb259899 100644 --- a/lib/common_test/src/ct_master.erl +++ b/lib/common_test/src/ct_master.erl @@ -25,6 +25,7 @@ -export([run/1,run/3,run/4]). -export([run_on_node/2,run_on_node/3]). -export([run_test/1,run_test/2]). +-export([get_event_mgr_ref/0]). -export([basic_html/1]). -export([abort/0,abort/1,progress/0]). @@ -292,6 +293,18 @@ progress() -> call(progress). %%%----------------------------------------------------------------- +%%% @spec get_event_mgr_ref() -> MasterEvMgrRef +%%% MasterEvMgrRef = atom() +%%% +%%% @doc <p>Call this function in order to get a reference to the +%%% CT master event manager. The reference can be used to e.g. +%%% add a user specific event handler while tests are running. +%%% Example: +%%% <c>gen_event:add_handler(ct_master:get_event_mgr_ref(), my_ev_h, [])</c></p> +get_event_mgr_ref() -> + ?CT_MEVMGR_REF. + +%%%----------------------------------------------------------------- %%% @spec basic_html(Bool) -> ok %%% Bool = true | false %%% diff --git a/lib/common_test/src/ct_netconfc.erl b/lib/common_test/src/ct_netconfc.erl index bded5a15cb..85fb1ea8d2 100644 --- a/lib/common_test/src/ct_netconfc.erl +++ b/lib/common_test/src/ct_netconfc.erl @@ -759,8 +759,9 @@ action(Client,Action) -> Client :: client(), Action :: simple_xml(), Timeout :: timeout(), - Result :: {ok,[simple_xml()]} | {error,error_reason()}. -%% @doc Execute an action. + Result :: ok | {ok,[simple_xml()]} | {error,error_reason()}. +%% @doc Execute an action. If the return type is void, <c>ok</c> will +%% be returned instead of <c>{ok,[simple_xml()]}</c>. %% %% @end %%---------------------------------------------------------------------- @@ -1570,6 +1571,9 @@ decode_ok(Other) -> decode_data([{Tag,Attrs,Content}]) -> case get_local_name_atom(Tag) of + ok -> + %% when action has return type void + ok; data -> %% Since content of data has nothing from the netconf %% namespace, we remove the parent's xmlns attribute here diff --git a/lib/common_test/test/ct_event_handler_SUITE.erl b/lib/common_test/test/ct_event_handler_SUITE.erl index 30a5e650fe..750ccb8659 100644 --- a/lib/common_test/test/ct_event_handler_SUITE.erl +++ b/lib/common_test/test/ct_event_handler_SUITE.erl @@ -29,6 +29,7 @@ -compile(export_all). -include_lib("common_test/include/ct.hrl"). +-include_lib("common_test/src/ct_util.hrl"). %-include_lib("common_test/include/ct_event.hrl"). @@ -59,7 +60,7 @@ end_per_testcase(TestCase, Config) -> suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - [start_stop, results]. + [start_stop, results, event_mgrs]. groups() -> []. @@ -179,5 +180,10 @@ results(Config) when is_list(Config) -> ok = ct_test_support:verify_events(TestEvents++TestEvents, Events, Config). +event_mgrs(_) -> + ?CT_EVMGR_REF = ct:get_event_mgr_ref(), + ?CT_MEVMGR_REF = ct_master:get_event_mgr_ref(). + + %%%----------------------------------------------------------------- %%% HELP FUNCTIONS diff --git a/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl b/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl index 2bcfeeec0c..6f5db21f57 100644 --- a/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl +++ b/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl @@ -488,8 +488,15 @@ action(Config) -> DataDir = ?config(data_dir,Config), {ok,Client} = open_success(DataDir), Data = [{myactionreturn,[{xmlns,"myns"}],["value"]}], - ?NS:expect_reply(action,{data,Data}), - {ok,Data} = ct_netconfc:action(Client,{myaction,[{xmlns,"myns"}],[]}), + %% test either to receive {data,Data} or {ok,Data}, + %% both need to be handled + {Reply,RetVal} = case element(3, now()) rem 2 of + 0 -> {{data,Data},{ok,Data}}; + 1 -> {{ok,Data},ok} + end, + ct:log("Client will receive {~w,Data}", [element(1,Reply)]), + ?NS:expect_reply(action,Reply), + RetVal = ct_netconfc:action(Client,{myaction,[{xmlns,"myns"}],[]}), ?NS:expect_do_reply('close-session',close,ok), ?ok = ct_netconfc:close_session(Client), ok. diff --git a/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl b/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl index fb0734d48e..f503825c4e 100644 --- a/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl +++ b/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl @@ -540,8 +540,13 @@ make_msg({hello,SessionId,Stuff}) -> SessionIdXml/binary,"</hello>">>); make_msg(ok) -> xml(rpc_reply("<ok/>")); + +make_msg({ok,Data}) -> + xml(rpc_reply(from_simple({ok,Data}))); + make_msg({data,Data}) -> xml(rpc_reply(from_simple({data,Data}))); + make_msg(event) -> xml(<<"<notification xmlns=\"",?NETCONF_NOTIF_NAMESPACE,"\">" "<eventTime>2012-06-14T14:50:54+02:00</eventTime>" diff --git a/lib/eldap/vsn.mk b/lib/eldap/vsn.mk index 432ba2e742..adca41ed63 100644 --- a/lib/eldap/vsn.mk +++ b/lib/eldap/vsn.mk @@ -1 +1 @@ -ELDAP_VSN = 1.1 +ELDAP_VSN = 1.1.1 diff --git a/lib/kernel/test/inet_SUITE.erl b/lib/kernel/test/inet_SUITE.erl index 849013ac79..44a32fc1ec 100644 --- a/lib/kernel/test/inet_SUITE.erl +++ b/lib/kernel/test/inet_SUITE.erl @@ -88,10 +88,30 @@ init_per_group(_GroupName, Config) -> end_per_group(_GroupName, Config) -> Config. +init_per_testcase(lookup_bad_search_option, Config) -> + Db = inet_db, + Key = res_lookup, + %% The bad option can not enter through inet_db:set_lookup/1, + %% but through e.g .inetrc. + Prev = ets:lookup(Db, Key), + ets:delete(Db, Key), + ets:insert(Db, {Key,[lookup_bad_search_option]}), + ?t:format("Misconfigured resolver lookup order", []), + Dog = test_server:timetrap(test_server:seconds(60)), + [{Key,Prev},{watchdog,Dog}|Config]; init_per_testcase(_Func, Config) -> Dog = test_server:timetrap(test_server:seconds(60)), [{watchdog,Dog}|Config]. +end_per_testcase(lookup_bad_search_option, Config) -> + Dog = ?config(watchdog, Config), + test_server:timetrap_cancel(Dog), + Db = inet_db, + Key = res_lookup, + Prev = ?config(Key, Config), + ets:delete(Db, Key), + ets:insert(Db, Prev), + ?t:format("Restored resolver lookup order", []); end_per_testcase(_Func, Config) -> Dog = ?config(watchdog, Config), test_server:timetrap_cancel(Dog). @@ -915,10 +935,8 @@ lookup_bad_search_option(suite) -> lookup_bad_search_option(doc) -> ["Test lookup with erroneously configured lookup option (OTP-12133)"]; lookup_bad_search_option(Config) when is_list(Config) -> - Db = inet_db, - %% The bad option can not enter through inet_db:set_lookup/1, - %% but through e.g .inetrc. - ets:insert(Db, {res_lookup,[lookup_bad_search_option]}), + %% Manipulation of resolver config is done in init_per_testcase + %% and end_per_testcase to ensure cleanup. {ok,Hostname} = inet:gethostname(), {ok,_Hent} = inet:gethostbyname(Hostname), % Will hang loop for this bug ok. diff --git a/lib/public_key/src/public_key.erl b/lib/public_key/src/public_key.erl index a0a87e5351..e8ff965982 100644 --- a/lib/public_key/src/public_key.erl +++ b/lib/public_key/src/public_key.erl @@ -114,7 +114,7 @@ pem_encode(PemEntries) when is_list(PemEntries) -> iolist_to_binary(pubkey_pem:encode(PemEntries)). %%-------------------------------------------------------------------- --spec pem_entry_decode(pem_entry(), [string()]) -> term(). +-spec pem_entry_decode(pem_entry(), string()) -> term(). % %% Description: Decodes a pem entry. pem_decode/1 returns a list of %% pem entries. @@ -146,14 +146,16 @@ pem_entry_decode({Asn1Type, CryptDer, {Cipher, #'PBES2-params'{}}} = PemEntry, pem_entry_decode({Asn1Type, CryptDer, {Cipher, {#'PBEParameter'{},_}}} = PemEntry, Password) when is_atom(Asn1Type) andalso is_binary(CryptDer) andalso - is_list(Cipher) -> + is_list(Cipher) andalso + is_list(Password) -> do_pem_entry_decode(PemEntry, Password); pem_entry_decode({Asn1Type, CryptDer, {Cipher, Salt}} = PemEntry, Password) when is_atom(Asn1Type) andalso is_binary(CryptDer) andalso is_list(Cipher) andalso is_binary(Salt) andalso - ((erlang:byte_size(Salt) == 8) or (erlang:byte_size(Salt) == 16)) -> + ((erlang:byte_size(Salt) == 8) or (erlang:byte_size(Salt) == 16)) andalso + is_list(Password) -> do_pem_entry_decode(PemEntry, Password). @@ -626,8 +628,12 @@ pkix_is_fixed_dh_cert(Cert) when is_binary(Cert) -> % %% Description: Returns the issuer id. %%-------------------------------------------------------------------- -pkix_issuer_id(Cert, Signed)-> - pkix_issuer_id(Cert, Signed, decode). +pkix_issuer_id(#'OTPCertificate'{} = OtpCert, Signed) when (Signed == self) or + (Signed == other) -> + pubkey_cert:issuer_id(OtpCert, Signed); +pkix_issuer_id(Cert, Signed) when is_binary(Cert) -> + OtpCert = pkix_decode_cert(Cert, otp), + pkix_issuer_id(OtpCert, Signed). %%-------------------------------------------------------------------- -spec pkix_crl_issuer(CRL::binary()| #'CertificateList'{}) -> @@ -990,17 +996,3 @@ ec_key({PubKey, PrivateKey}, Params) -> parameters = Params, publicKey = {0, PubKey}}. -pkix_issuer_id(#'OTPCertificate'{} = OtpCert, Signed, decode) when (Signed == self) or - (Signed == other) -> - pubkey_cert:issuer_id(OtpCert, Signed); -pkix_issuer_id(#'OTPCertificate'{} = OtpCert, Signed, encode) when (Signed == self) or - (Signed == other) -> - case pubkey_cert:issuer_id(OtpCert, Signed) of - {ok, {Serial, Issuer}} -> - {ok, {Serial, pubkey_cert_records:transform(Issuer, encode)}}; - Error -> - Error - end; -pkix_issuer_id(Cert, Signed, Decode) when is_binary(Cert) -> - OtpCert = pkix_decode_cert(Cert, otp), - pkix_issuer_id(OtpCert, Signed, Decode). diff --git a/lib/ssh/examples/Makefile b/lib/ssh/examples/Makefile index de019f75b5..9280c42076 100644 --- a/lib/ssh/examples/Makefile +++ b/lib/ssh/examples/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2005-2012. All Rights Reserved. +# Copyright Ericsson AB 2005-2015. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -38,7 +38,8 @@ RELSYSDIR = $(RELEASE_PATH)/lib/ssh-$(VSN) MODULES = \ - ssh_sample_cli + ssh_sample_cli \ + ssh_device.erl ERL_FILES= $(MODULES:=.erl) diff --git a/lib/ssh/examples/ssh_device.erl b/lib/ssh/examples/ssh_device.erl new file mode 100644 index 0000000000..f6be812915 --- /dev/null +++ b/lib/ssh/examples/ssh_device.erl @@ -0,0 +1,62 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2015. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(ssh_device). + +%% api +-export([ssh_device/5]). + +%%% I wrote this because of i think a fully ssh client sample will be easy to start the ssh module better than +%%% go though each function file. +ssh_device(Host, Port, User, Pass, Cmd) -> + ssh:start(), + case ssh:connect(Host, Port, + [{user, User}, {password, Pass}, + {silently_accept_hosts, true}, {quiet_mode, true}]) + of + {ok, Conn} -> + {ok, ChannelId} = ssh_connection:session_channel(Conn, + infinity), + ssh_connection:exec(Conn, ChannelId, Cmd, infinity), + Init_rep = <<>>, + wait_for_response(Conn, Host, Init_rep), + ssh:close(Conn); + {error, nxdomain} -> + {error,nxdomain} + end. + +%%-------------------------------------------------------------------- +%%% Internal application API +%%-------------------------------------------------------------------- +wait_for_response(Conn, Host, Acc) -> + receive + {ssh_cm, Conn, Msg} -> + case Msg of + {closed, _ChannelId} -> + {ok,Acc}; + {data, _, _, A} -> + Acc2 = <<Acc/binary, A/binary>>, + wait_for_response(Conn, Host, Acc2); + _ -> + wait_for_response(Conn, Host, Acc) + end + after + 5000 -> + {error,timeout} + end. diff --git a/lib/ssh/vsn.mk b/lib/ssh/vsn.mk index c8cac3e852..bfebe2c60b 100644 --- a/lib/ssh/vsn.mk +++ b/lib/ssh/vsn.mk @@ -1,5 +1,5 @@ #-*-makefile-*- ; force emacs to enter makefile-mode -SSH_VSN = 3.1 +SSH_VSN = 3.1.1 APP_VSN = "ssh-$(SSH_VSN)" diff --git a/lib/tools/src/tags.erl b/lib/tools/src/tags.erl index e3cc51cdb2..e25db2eb1b 100644 --- a/lib/tools/src/tags.erl +++ b/lib/tools/src/tags.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2015. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -297,15 +297,16 @@ word_char(_) -> false. %% Check the options `outfile' and `outdir'. open_out(Options) -> + Opts = [write, {encoding, unicode}], case lists:keysearch(outfile, 1, Options) of {value, {outfile, File}} -> - file:open(File, [write]); + file:open(File, Opts); _ -> case lists:keysearch(outdir, 1, Options) of {value, {outdir, Dir}} -> - file:open(filename:join(Dir, "TAGS"), [write]); + file:open(filename:join(Dir, "TAGS"), Opts); _ -> - file:open("TAGS", [write]) + file:open("TAGS", Opts) end end. diff --git a/otp_patch_apply b/otp_patch_apply new file mode 100755 index 0000000000..947aa1e6ee --- /dev/null +++ b/otp_patch_apply @@ -0,0 +1,480 @@ +#!/bin/sh +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2014. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# + +version="1.0.1" + +force= +lib_path= +orig_dir= +sdir= +idir="/broken/path/here" +cleanup=no +install_docs=yes + +invalid_src="does not seem to be a valid OTP source tree" +not_built="Source in has not been built" +doc_not_built="Documentation has not been built. Either build the +documentation and re-run 'otp_patch_apply', or re-run 'otp_patch_apply' +with the '-n' switch." + +print_usage() +{ + cat <<EOF +otp_patch_apply -s <Dir> -i <Dir> [-l <Dir>] [-c] [-f] [-h] [-n] [-v] \\ + <App1> [... <AppN>] + + -s <Dir> -- OTP source directory that contain build results. + -i <Dir> -- OTP installation directory to patch. + -l <Dir> -- Alternative OTP source library directory path(s) containing + build results of OTP applications. Multiple paths should be + colon separated. + -c -- Cleanup (remove) old versions of applications patched + in the installation. + -f -- Force patch of application(s) even though dependencies are + not fullfilled. + -h -- Print this help then exit. + -n -- Do not install documentation. + -v -- Print version then exit. + <AppX> -- Application to patch. + +Environment Variable: + ERL_LIBS -- Alternative OTP source library directory path(s) containing + build results of OTP applications. Multiple paths should be + colon separated. + +NOTE: + * Complete build environment is required while running otp_patch_apply. + * Before applying a patch you need to build all of OTP in the source + directory. + * All source directories identified by -s and -l should contain build + results of OTP applications. + +Version: $version + +EOF + +} + +error() +{ + echo "ERROR:" "$@" 1>&2 + exit 1 +} + +usage_error() +{ + echo "ERROR:" "$@" 1>&2 + echo "" 1>&2 + print_usage 1>&2 + exit 1 +} + +usage() +{ + print_usage + exit 0 +} + +alt_lib_path() +{ + app=$1 + save_ifs=$IFS + IFS=: + + cd "$orig_dir" || error "Cannot change directory to $orig_dir" + + for lib in $lib_path; do + # Want absolute path + case "$lib" in + /*) + ;; + *) + cd "$lib" || error "Cannot change directory to $lib" + lib=`pwd` + cd "$orig_dir" || error "Cannot change directory to $orig_dir" + esac + if [ -d "$lib/$app" ]; then + echo "$lib/$app" + IFS=$save_ifs + return 0 + fi + done + + IFS=$save_ifs + + return 1 +} + +prog_in_mod_path() +{ + chk_path="/bin:$PATH" + PROG=$1 + save_ifs=$IFS + IFS=: + if [ "X$TARGET" = "Xwin32" ]; then + for p in $chk_path; do + if [ -f "$p/$PROG.exe" ]; then + IFS=$save_ifs + echo "$p/$PROG.exe" + return 0 + fi + done + else + for p in $chk_path; do + if [ -x "$p/$PROG" ]; then + IFS=$save_ifs + echo "$p/$PROG" + return 0 + fi + done + fi + IFS=$save_ifs + return 1 +} + +find_prog() +{ + prog_in_mod_path "$1" + if [ $? -ne 0 ]; then + echo "$1" + fi + return 0 +} + +# Parse arguments + +while [ $# -gt 0 ]; do + case "$1" in + "-s") + shift + if [ ! $# -gt 0 ]; then + usage_error "Missing OTP source directory" + fi + sdir="$1";; + "-i") + shift + if [ ! $# -gt 0 ]; then + usage_error "Missing OTP install directory" + fi + idir="$1";; + "-l") + shift + if [ ! $# -gt 0 ]; then + usage_error "Missing OTP library directory" + fi + if [ "x$lib_path" = "x" ]; then + lib_path="$1" + else + lib_path="$lib_path:$1" + fi;; + "-f") + force="-force";; + "-c") + cleanup=yes;; + "-h") + usage;; + "-n") + install_docs=no;; + "-v") + echo "otp_patch_apply version $version" + exit 0;; + *) + app="$1" + applications="$applications $app";; + esac + shift +done + +# Check that we got mandatory arguments +test "x$sdir" != "x" || usage_error "Missing OTP source directory" +test "x$idir" != "x" || usage_error "Missing OTP install directory" +test "x$applications" != "x" || usage_error "Missing applications" + +orig_dir=`pwd` + +# Check that the source directory seems sane +cd "$sdir" 2>/dev/null || error "Cannot change directory to $sdir" + +# Want absolute path +case "$sdir" in + /*) ;; + *) sdir=`pwd`;; +esac + +export ERL_TOP="$sdir" +test -f "$sdir/otp_build" || error "$ERL_TOP" $invalid_src +test -f "$sdir/OTP_VERSION" || error "$ERL_TOP" $invalid_src +test -f "$sdir/otp_versions.table" || error "$ERL_TOP" $invalid_src +test -f "$sdir/erts/autoconf/config.guess" || error "$ERL_TOP" $invalid_src +test -f "$sdir/make/verify_runtime_dependencies" || error "$ERL_TOP" $invalid_src +test -x "$sdir/bootstrap/bin/erl" || error $not_built +test -x "$sdir/bootstrap/bin/erlc" || error $not_built +test -x "$sdir/bootstrap/bin/escript" || error $not_built +test -f "$sdir/make/otp_built" || error $not_built + +if [ $install_docs = yes ]; then + test -f "$sdir/make/otp_doc_built" || usage_error $doc_not_built +fi + +otp_rel=`sed 's|\([0-9]*\).*|\1|' < $ERL_TOP/OTP_VERSION` || \ + error "Failed to read $ERL_TOP/OTP_VERSION" + +case "$otp_rel" in + 1[7-9]|[2-9][0-9]) ;; # ok; release 17-99 + *) error "Invalid OTP release: $otp_rel";; +esac + +export PATH="$ERL_TOP/bootstrap/bin:$PATH" +erlc="$ERL_TOP/bootstrap/bin/erlc" +erl="$ERL_TOP/bootstrap/bin/erl" + +erl_otp_rel=`$erl -noshell -noinput -eval "io:format(\"~s~n\", [erlang:system_info(otp_release)]), erlang:halt(0)"` || \ + error "Failed to execute: $erl" + +test "$otp_rel" = "$erl_otp_rel" || error "Inconsistent source: $sdir" + +app_dirs= +for app in $applications; do + case "$app" in + "erts") + dir="$ERL_TOP/erts";; + *) + dir="$ERL_TOP/lib/$app";; + esac + if [ ! -d "$dir" ]; then + dir=`alt_lib_path "$app"` + if [ $? -ne 0 ]; then + error "Application missing in source: $app" + fi + fi + app_dirs="$app_dirs $dir" +done + +cd "$orig_dir" 2>/dev/null || error "Cannot change directory to $orig_dir" + +# Check that the install directory seems sane +cd "$idir" 2>/dev/null || error "Cannot change directory to $idir" + +# Want absolute path +case "$idir" in + /*) ;; + *) idir=`pwd`;; +esac + +test -d "$idir/releases/$otp_rel" || \ + error "No OTP-$otp_rel installation present in $idir" + +cd "$ERL_TOP" 2>/dev/null || error "Cannot change directory to $ERL_TOP" + +# Some tools we use +rm=`find_prog rm` +rmdir=`find_prog rmdir` +cp=`find_prog cp` +mv=`find_prog mv` +mkdir=`find_prog mkdir` + +# Setup build stuff +if [ "x$TARGET" = "x" ]; then + TARGET=`$ERL_TOP/erts/autoconf/config.guess` +fi +BUILDSYS=$TARGET +if [ -z "$MAKE" ]; then + case $TARGET in + win32) + MAKE=make;; + *) + prog_in_mod_path gmake >/dev/null + if [ $? -eq 0 ]; then + MAKE=gmake + else + MAKE=make + fi;; + esac +fi +if [ X`$MAKE is_cross_configured` = Xyes ]; then + TARGET=`$MAKE target_configured` +elif [ "x$OVERRIDE_TARGET" != "x" -a "x$OVERRIDE_TARGET" != "xwin32" ]; then + TARGET=$OVERRIDE_TARGET +fi + +# Check for cleanup +inst_app_vers="$idir/releases/$otp_rel/installed_application_versions" +rm_app_vers= +if [ $cleanup = yes ]; then + $mv "$inst_app_vers" "${inst_app_vers}.save" || \ + error "Failed to save $inst_app_vers" + for app in $applications; do + tmp=`grep "$app-*" "${inst_app_vers}.save"` + rm_app_vers="$rm_app_vers $tmp" + done + $cp "${inst_app_vers}.save" "$inst_app_vers" + for rm_app_ver in $rm_app_vers; do + $cp "$inst_app_vers" "${inst_app_vers}.tmp" + grep -v $rm_app_ver "${inst_app_vers}.tmp" > "$inst_app_vers" + done + $rm -f "${inst_app_vers}.tmp" +fi + +# Verify runtime dependencies +$ERL_TOP/make/verify_runtime_dependencies -release "$otp_rel" \ + -source "$ERL_TOP" -target "$idir" $force $applications || { + test ! -f "${inst_app_vers}.save" || \ + $mv "${inst_app_vers}.save" "$inst_app_vers" + exit 1 +} + +# Update OTP_VERSION in installation +otp_version=`cat "$idir/releases/$otp_rel/OTP_VERSION"` || { + test ! -f "${inst_app_vers}.save" || \ + $mv "${inst_app_vers}.save" "$inst_app_vers" + error "Not able to read $idir/releases/$otp_rel/OTP_VERSION" +} + +{ + echo "$otp_version" | sed "s|^\([^\*]*\)\**|\1\*\*|g" > \ + "$idir/releases/$otp_rel/OTP_VERSION" +} 2>/dev/null || { + test ! -f "${inst_app_vers}.save" || \ + $mv "${inst_app_vers}.save" "$inst_app_vers" + error "Not able to update $idir/releases/$otp_rel/OTP_VERSION" +} + +# Do actual cleanup +if [ "x$rm_app_vers" != "x" ]; then + for app_ver in $rm_app_vers; do + case x"$app_ver" in + x) + ;; + xerts-*) + $rm -rf "$idir/$app_ver" ;; + x*) + $rm -rf "$idir/lib/$app_ver" ;; + esac + done + $rm -f "${inst_app_vers}.save" +fi + +# Install application from built source +for app_dir in $app_dirs; do + (cd "$app_dir" && \ + $MAKE MAKE="$MAKE" TARGET=$TARGET RELEASE_ROOT="$idir" \ + RELEASE_PATH="$idir" TESTROOT="$idir" release) || exit 1 +done + +if [ $install_docs = yes ]; then +# Documentation have been built and should be installed + + for app_dir in $app_dirs; do + (cd "$app_dir" && \ + $MAKE MAKE="$MAKE" RELEASE_ROOT="$idir" RELEASE_PATH="$idir" \ + TESTROOT="$idir" release_docs) || exit 1 + done + + (cd "$sdir/system/doc/top" && $MAKE clean) + + (cd "$sdir/system/doc/top" && \ + $MAKE MAKE="$MAKE" RELEASE_ROOT="$idir" RELEASE_PATH="$idir" \ + TESTROOT="$idir" release_docs) || exit 1 + + echo "" + echo "*" + echo "* NOTE! In order to update pre-formatted man pages you" + echo "* need to run the 'Install' script located in:" + echo "* $idir" + echo "*" +fi + +# If erts, kernel, stdlib or sasl is included, find versions +for app in $applications; do + case "$app" in + erts) + erts_vsn=`grep '^VSN' erts/vsn.mk | sed "s|^VSN.*=[^0-9]*\([0-9].*\)$|\1|g"` + update_rel=true;; + kernel) + kernel_vsn=`sed "s|^KERNEL_VSN[^=]*=[^0-9]*\([0-9].*\)$|\1|g" lib/kernel/vsn.mk` + update_rel=true;; + stdlib) + stdlib_vsn=`sed "s|^STDLIB_VSN[^=]*=[^0-9]*\([0-9].*\)$|\1|g" lib/stdlib/vsn.mk` + update_rel=true;; + sasl) + sasl_vsn=`sed "s|^SASL_VSN[^=]*=[^0-9]*\([0-9].*\)$|\1|g" lib/sasl/vsn.mk` + update_rel=true;; + *) + ;; + esac +done + +# and find the old versions for those not included +if [ "X$update_rel" != "X" ]; then + if [ "X$erts_vsn" = "X" ]; then + erts_vsns=`ls -d "$idir"/erts-* | sed "s|$idir/erts-\([0-9\.].*\)|\1|g"` + erts_vsn=`echo "$erts_vsns" | sort -t '.' -g | tail -n 1` + fi + if [ "X$kernel_vsn" = "X" ]; then + kernel_vsns=`ls -d "$idir"/lib/kernel-* | sed "s|$idir/lib/kernel-\([0-9\.].*\)|\1|g"` + kernel_vsn=`echo "$kernel_vsns" | sort -t '.' -g | tail -n 1` + fi + if [ "X$stdlib_vsn" = "X" ]; then + stdlib_vsns=`ls -d "$idir"/lib/stdlib-* | sed "s|$idir/lib/stdlib-\([0-9\.].*\)|\1|g"` + stdlib_vsn=`echo "$stdlib_vsns" | sort -t '.' -g | tail -n 1` + fi + if [ "X$sasl_vsn" = "X" ]; then + sasl_vsns=`ls -d "$idir"/lib/sasl-* | sed "s|$idir/lib/sasl-\([0-9\.].*\)|\1|g"` + sasl_vsn=`echo "$sasl_vsns" | sort -t '.' -g | tail -n 1` + fi + + # Generate .rel, .script and .boot - to tmp dir + start_clean="{release, {\"Erlang/OTP\",\"$otp_rel\"}, {erts, \"$erts_vsn\"}, [{kernel,\"$kernel_vsn\"}, {stdlib,\"$stdlib_vsn\"}]}." + start_sasl="{release, {\"Erlang/OTP\",\"$otp_rel\"}, {erts, \"$erts_vsn\"}, [{kernel,\"$kernel_vsn\"}, {stdlib,\"$stdlib_vsn\"}, {sasl,\"$sasl_vsn\"}]}." + + tmp_dir="$idir/tmp"; + if [ ! -d "$tmp_dir" ]; then + $mkdir "$tmp_dir" + fi + echo "$start_sasl" > "$tmp_dir/start_sasl.rel" + echo "$start_clean" > "$tmp_dir/start_clean.rel" + echo "$start_clean" > "$tmp_dir/no_dot_erlang.rel" + + $erlc -I"$idir"/lib/*/ebin -o"$tmp_dir" "$tmp_dir/start_sasl.rel" || exit 1 + $erlc -I"$idir"/lib/*/ebin -o"$tmp_dir" +no_warn_sasl "$tmp_dir/start_clean.rel" || exit 1 + $erlc -I"$idir"/lib/*/ebin -o"$tmp_dir" +no_warn_sasl +no_dot_erlang "$tmp_dir/no_dot_erlang.rel" || exit 1 + + # Generate RELEASES file + "$erl" -noinput +B -eval "release_handler:create_RELEASES(\"%ERL_ROOT%\", \"$tmp_dir\", \"$tmp_dir/start_sasl.rel\", []), halt()" || exit 1 + + # If all good so far, move generated files into target area + $mv "$tmp_dir/RELEASES" "$idir/releases/RELEASES.src" + $mv "$tmp_dir"/* "$idir/releases/$otp_rel" + $rmdir "$tmp_dir" + + # Remove old start scripts (forces a new run of Install) + $rm -f "$idir"/releases/RELEASES + $rm -f "$idir"/bin/*.script + $rm -f "$idir"/bin/*.boot + $rm -f "$idir"/bin/erl + + echo "" + echo "*" + echo "* NOTE! In order to get a runnable OTP system again you" + echo "* need to run the 'Install' script located in:" + echo "* $idir" + echo "*" +fi + diff --git a/system/doc/installation_guide/Makefile b/system/doc/installation_guide/Makefile index 83210bd21f..a4ef6c9d7c 100644 --- a/system/doc/installation_guide/Makefile +++ b/system/doc/installation_guide/Makefile @@ -58,7 +58,8 @@ XML_FILES = \ GENERATED_XML_FILES = \ INSTALL.xml \ INSTALL-CROSS.xml \ - INSTALL-WIN32.xml + INSTALL-WIN32.xml \ + OTP-PATCH-APPLY.xml # ---------------------------------------------------- @@ -73,7 +74,8 @@ REDIRECT_HTML_DIR = $(HTMLDIR)/source REDIRECT_HTML_FILES = \ $(REDIRECT_HTML_DIR)/INSTALL.html \ $(REDIRECT_HTML_DIR)/INSTALL-CROSS.html \ - $(REDIRECT_HTML_DIR)/INSTALL-WIN32.html + $(REDIRECT_HTML_DIR)/INSTALL-WIN32.html \ + $(REDIRECT_HTML_DIR)/OTP-PATCH-APPLY.html # ---------------------------------------------------- # FLAGS diff --git a/system/doc/installation_guide/part.xml b/system/doc/installation_guide/part.xml index 02bf98db7c..ff17cecd59 100644 --- a/system/doc/installation_guide/part.xml +++ b/system/doc/installation_guide/part.xml @@ -35,4 +35,5 @@ <xi:include href="INSTALL.xml"/> <xi:include href="INSTALL-CROSS.xml"/> <xi:include href="INSTALL-WIN32.xml"/> -</part>
\ No newline at end of file + <xi:include href="OTP-PATCH-APPLY.xml"/> +</part> diff --git a/system/doc/installation_guide/xmlfiles.mk b/system/doc/installation_guide/xmlfiles.mk index c443334cd7..a18c82bc25 100644 --- a/system/doc/installation_guide/xmlfiles.mk +++ b/system/doc/installation_guide/xmlfiles.mk @@ -20,4 +20,5 @@ INST_GUIDE_CHAPTER_FILES = \ install-binary.xml \ INSTALL.xml \ INSTALL-CROSS.xml \ - INSTALL-WIN32.xml + INSTALL-WIN32.xml \ + OTP-PATCH-APPLY.xml diff --git a/system/doc/system_principles/versions.xml b/system/doc/system_principles/versions.xml index ff042f4a3b..ed6fd1f7fe 100644 --- a/system/doc/system_principles/versions.xml +++ b/system/doc/system_principles/versions.xml @@ -61,8 +61,9 @@ <c>filename:join([<seealso marker="kernel:code#root_dir/0">code:root_dir()</seealso>, "releases", <seealso marker="erts:erlang#system_info_otp_release">erlang:system_info(otp_release)</seealso>, "OTP_VERSION"]).</c></p> <p>If the version read from the <c>OTP_VERSION</c> file in a development system has a <c>**</c> suffix, the system has been - patched using the <c>otp_patch_apply</c> tool available to - licensed customers. In this case, the system consists of application + patched using the + <seealso marker="../installation_guide/OTP-PATCH-APPLY"><c>otp_patch_apply</c></seealso> + tool. In this case, the system consists of application versions from multiple OTP versions. The version preceding the <c>**</c> suffix corresponds to the OTP version of the base system that has been patched. Note that if a development system is updated by |